When I run a job with Kitchen.bat, the job takes 100 seconds to start and I find the following line in the logs:
ERROR [KarafLifecycleListener] The Kettle Karaf Lifecycle Listener failed to execute properly after waiting for 100 seconds. Releasing lifecycle hold, but some services may be unavailable.
Is there a way to speed up the job start-up time?
Thanks
Related
I have some Tasks in pentaho, and for some reason some times, some tasks simply stall with message Carte - Installing timer to purge stale objects after 1440 minutes. task. For example, I scheduled one task to run at 05h00 AM and this task usually runs in 10 minutes, but some times it never ends. The task stalls with aforementioned message. However, when I run the task run on the Pentaho Data Integration Canvas, the job works.
The .exe that I use to run is:
cd c:\data-integration
kitchen.bat /rep:repo /job:jobs//job_ft_acidentes /dir: /level:Minimal
Picture of the message
Hoe can I prevent this error?
I am running Azure Devops pipeline and this pipeline calls a SQL procedure.
This procedure takes more than 6 hours to complete (it takes around 10 hours)
After procedure execution, I will take a database backup (it takes around 12 hours)
The problem which I'm facing is pipeline execution throws error after 6 hours:
Not received any response, or we stopped hearing from Agent Azure Pipelines 4. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information
We have proper networks and no issues in procedure.
I have done the below setup still same problem
timeout to 0 & 7600
timeout to 7600
Number of retries if task fail to 5
Please let me know
how to make pipeline to wait until SQL procedure complete
how to catch SQL exception to stop pipeline execution. currently the pipeline status updates with success status even if any errors occurs in SQL
Regards,
Kumar
It seems you are using the Microsoft hosted agent, so the pipeline will be time out after 6 hours. Please refer this Capabilities and limitations for Microsoft-hosted agents:
You can pay for additional capacity per parallel job. Paid parallel jobs remove the monthly time limit and allow you to run each job for up to 360 minutes (6 hours).
To resolve the time out issue, as your pipeline will run more than 6 hours, please consider to use a self-hosted agent. Setting the timeout value to zero means that the job can run forever on self-hosted agents.
I have a process in which I have used a timer event. The timer event is of type
Duration and has wait time of 30 minutes(PT30M). This timer event is expected to end exactly after 30 minutes, but it takes additional 15 or 30 seconds.
This behavior is observed and the delay is exactly 15 or 30 seconds every time even if I change (increase or decrease) the duration of timer event. I would like to know why does it take 15 seconds extra then required to execute.
A timer in a BPMN process is persisted as a job in Camunda. The job executor component repeatedly polls the job table for any jobs that are due. In case there are no due jobs, the polling applies exponential backoff, by default sleeping up to 60 seconds between polling attempts. So if your system has little load, this kind of delay is to be expected. You can use the job executor configuration property maxWait to change the maximum sleeping period.
Relevant documentation:
Job executor in general: https://docs.camunda.org/manual/7.10/user-guide/process-engine/the-job-executor/#job-executor-activation
Job executor configuration properties: https://docs.camunda.org/manual/7.10/reference/deployment-descriptors/tags/job-executor/
I'm running into a weird issue with Pentaho 7.1. I run a job that I created and it runs perfectly and quickly the first time I run it.
The job is an ETL Job consisting of a Start widget, 7 Transformations running in a sequence, and a Success widget.
I'm confused as to why the job runs once, and when I try to run it again it says "Spoon - Starting job..." and then the job just hangs.
If I delete the job and I create a brand new one, I am then able to run the job once and I am stuck again with the job no longer able to run after that. I don't understand why the job keeps hanging after it gets executed once, and it is then 100% broken after a Successful run...
I turned up the logging in Pentaho 7.1 Spoon, and it shows this continuously...
2018/08/14 17:30:24 - Job1 - Triggering heartbeat signal for
ported_jobs at every 10 seconds 2018/08/14 17:30:34 - Job1 -
Triggering heartbeat signal for ported_jobs at every 10 seconds
2018/08/14 17:30:44 - Job1 - Triggering heartbeat signal for
ported_jobs at every 10 seconds 2018/08/14 17:30:54 - Job1 -
Triggering heartbeat signal for ported_jobs at every 10 seconds
I can't seem to put my finger on why this happening.
Any help is appreciated
Probable answer: Check that your transformations are not opening the same database for input and output. A quick check may to run the first transformation directly (without the job) and see if it locks.
Happened because the server db you want to update are slow to respond. Probably high CPU and RAM. I tried to increase the RAM and CPU for the db server, now my job runs okay.
I'm working on a workflow that has both Hive and Java actions. Very often we have been noticing that there is a few minutes delay between Java action start time and the job submission time. We don't see that with Hive jobs, meaning Hive jobs seem to be submitted almost immediately after they are started. The Java jobs do not do much and so they finish successfully in seconds after they are submitted but the time between start and submission seem to be very night ( 4 -5 minutes). We are using fair scheduler and the there are enough mapper/reducer slots available. But still even if it's a resource problem the Hive jobs should also show delay between start and submission but they don't ! Java jobs are very simple jobs and they don't process any files etc and basically used to call a web service and they spawn only single mapper and no reducers where are the Hive jobs creates hundreds of mapper/reducer tasks but still there is not delay between start and submission. We are not able to figure out why oozie is not submitting the Java job immediately. Any ideas?