I am trying to migrate Oozie database from derby to MySql. Everything is running fine except when i try to run command
./oozie-setup.sh prepare-war
I got the following error
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
ERROR: Stop Oozie first
And when I try to run
./oozied.sh stop
I get error PID file found but no matching process was found. Stop aborted.
I am struggling as I am not getting a way to stop oozie so that I can proceed.
After a long search and trial i was able to resolve the issue.
You need to delete the pid file which in my case was located in
oozie-4.3.0/oozie-server/temp/oozie.pid
After then you can run ./oozie-setup.sh prepare-war there will be no error.
Actually issue was that tomcat abroted abnormally and pid file was present there. So removing the file did the job.
Related
I’m executing automatic tests using Katalon in console mode.
Sometimes job executed by Jenkins ends unexpectedly without any error.
Katalon Studio version 5.10.1,
Jenkins version 2.121.3
I've tried to find some Jenkins logs other than console logs, which logged why that job is aborted
Jenkins console logs:
/var/lib/jenkins/.katalon/5.10.1/Katalon_Studio_Linux_64-5.10.1/katalon -noSplash -runMode=console -projectPath=/var/lib/jenkins/workspace/(...) -browserType=Firefox -testSuitePath=Test Suites/MainTestingSuite
Delete folder: Libs
Opening project file: /var/lib/jenkins/workspace/(...).prj
Request sent successfully.
[EL Warning]: 2019-05-10 13:54:15.046–Ignoring attribute [lastRun] on class [TestSuiteEntity] as no Property was generated for it.
**Build step ‘Execute shell’ changed build result to UNSTABLE**
Got SIGTERM, exiting
In exit
Terminating xvnc.
$ vncserver -kill :80
Killing Xvnc4 process ID 29224
Archiving artifacts
‘Reports/MainTestingSuite/’ doesn’t match anything, but ‘’ does. Perhaps that’s what you mean?
No artifacts found that match the file pattern “Reports/MainTestingSuite/”. Configuration error?
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE
Someone already had that problem? How to track that case?
I’m waiting for your answer.
I’ve finally found cause of that problem.
Katalon job failed like that, when you try executed two Katalon instances at the moment. I’ve changed number of executors at jenkins to 1, and problem is not occuring anymore.
Best Regards
This is driving me nuts.
I'm setting up airflow in a cloud environment. I have one server running the scheduler and the webserver and one server as a celery worker, and I'm using airflow 1.8.0.
Running jobs works fine. What refuses to work is logging.
I've set up the correct path in airflow.cfg on both servers:
remote_base_log_folder = s3://my-bucket/airflow_logs/
remote_log_conn_id = s3_logging_conn
I've set up s3_logging_conn in the airflow UI, with the access key and the secret key as described here.
I checked the connection using
s3 = airflow.hooks.S3Hook('s3_logging_conn')
s3.load_string('test','test',bucket_name='my-bucket')
This works on both servers. So the connection is properly set up. Yet all I get whenever I run a task is
*** Log file isn't local.
*** Fetching here: http://*******
*** Failed to fetch log file from worker.
*** Reading remote logs...
Could not read logs from s3://my-bucket/airflow_logs/my-dag/my-task/2018-02-15T21:46:47.577537
I tried manually uploading the log following the expected conventions and the webserver still can't pick it up - so the problem is on both ends. I'm at a loss at what to do, everything I've read so far tells me this should be working. I'm close to just installing the 1.9.0 which I hear changes logging and see if I'm more lucky.
UPDATE: I made a clean install of Airflow 1.9 and followed the specific instructions here.
Webserver won't even start now with the following error:
airflow.exceptions.AirflowConfigException: section/key [core/remote_logging] not found in config
There is an explicit reference to this section in this config template.
So I tried removing it and just loading the S3 handler without checking first and I got the following error message instead:
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/lib64/python3.6/logging/config.py", line 384, in resolve:
self.importer(used)
ModuleNotFoundError: No module named
'airflow.utils.log.logging_mixin.RedirectStdHandler';
'airflow.utils.log.logging_mixin' is not a package
I get the feeling that this shouldn't be this hard.
Any help would be much appreciated, cheers
Solved:
upgraded to 1.9
ran the steps described in this comment
added
[core]
remote_logging = True
to airflow.cfg
ran
pip install --upgrade airflow[log]
Everything's working fine now.
I did all the setup for oozie 4.3.0 on Apache hadoop single node cluster, when tried running any standard example workflow.xml that comes with oozie, it is throwing below error.
WARN ActionStartXCommand:523 - SERVER[data01.teg.io] USER[hadoop] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-161215143751620-oozie-hado-W] ACTION[0000000-161215143751620-oozie-hado-W#mr-node] Error starting action [mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.]
I looked at the parameter "mapreduce.framework.name" and it is set to yarn everywhere in all config files. I checked Sharelib is created properly and can see when queried with shareliblist command, i dont see where exactly the problem is. Tried every solution came up in google and could not solve it even after struggling for 2 days with it.
I can start and stop oozie daemon with out any problem.
Any insights are greatly helpful.
I figured out the solution. Unlike the prior versions of Oozie before 4.x.x, 4.3.0 does not generate hadoop-libs.jar file when we run the build command.
In the beginning, i copied jar files only from my hadoop's
/srv/hadoop-2.7.3/share/hadoop/common to oozie's libext folder. After i copied jar files from all the below paths to oozie's libext folder, i was able to successfully setup the Oozie.
/srv/hadoop-2.7.3/share/hadoop/common/*.jar
/srv/hadoop-2.7.3/share/hadoop/common/lib/*.jar
/srv/hadoop-2.7.3/share/hadoop/hdfs/*.jar
/srv/hadoop-2.7.3/share/hadoop/hdfs/lib/*.jar
/srv/hadoop-2.7.3/share/hadoop/mapreduce/*.jar
/srv/hadoop-2.7.3/share/hadoop/mapreduce/lib/*.jar
/srv/hadoop-2.7.3/share/hadoop/yarn/*.jar
/srv/hadoop-2.7.3/share/hadoop/yarn/lib/*.jar
I am trying to start hiveserver2, by going to the bin folder of my Hive install and typing hiveserver2. However, nothing happens - it just hangs there, and when I check if anything is running on Hive ports (the interface on 10002 for example) there is nothing, nor anything in netstat.
Initially I had errors about SLF4J:
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
But I solved that by moving the /usr/lib/hive/lib/log4j-slf4j-impl-2.4.1.jar in to the /tmp directory.
Now when I run bin/hiveserver2 it hangs for quite a while, with no output, and then just returns to the command line - and the Hive server isn't started. I'm struggling to find any logs either.
There is a chance that the hiveserver2 running without showing any information. Try this answer with attached argument flag showing running log
I've setup a TC configuration ready to run Selenium tests once we get a build agent able to run them.
This is the first TC config I've created but it was running until I got TC to run the Selenium test runner. Now it fails when it tries to cleanup the Selenium-server.jar.
Can you exclude file types from the cleanup or is there another solution that I'm missing here?
TC build errors;
> Problem reported from build script. New build status text is: : {build.status.text}; Swabra cleanup failed
> Error while applying patch: Error while applying patch: Failed to delete: C:\BuildAgent\work\f43641868cf93216\src\django_selenium\selenium-server.jar
You can configure Swabra cleaner to ignore your selenium-server.jar using path rule:
-:src\django_selenium\selenium-server.jar
but it looks like there is a problem in selenium tests setup. The error message you posted may be caused by some process (most probably, selenium server) still running after tests are finished. It has locked the library and thats why swabra can't delete it.