I would like to run multiple scheduled Task Flows against the same data source but only run one at a time.
Example:
Schedule "Nightly" runs once a day (expected runtime 30 minutes),
Schedule "Hourly" runs once an hour (expected runtime 10 minutes),
Schedule "Minute" runs once a minute (expected runtime 5 seconds).
I would like:
#1 "Nightly" test status of "Hourly" and "Minute":
If they are not running, start "Nightly",
If either are running, loop around until both have stopped.
#2 "Hourly" test status of "Nightly" and "Minute":
If they are not running, start "Hourly",
If "Nightly" is running, exit,
If "Minute" is running, loop around until both have stopped.
#3 "Minute" test status of "Nightly" and "Hourly":
If they are not running, start "Minute",
If either are running, exit.
So far, I am using handshakes with several JSON files in the cloud.
Meaning, if "Minute" is running, the file minute.json contains information telling a caller "Minute" is running.
When "Minute" ends, it updates its file, minute.json, to reflect the operation has stopped.
As you can imagine, this is very slow.
Also, Informatica will always create a JSON file when JSON is the target. The issue here is, if there is any issue, Informatica will create a 0 file size JSON file that will fail any operation calling it.
There has got to be a better way.
You could use the Informatica Platform REST API v2 to monitor and control the execution of your jobs programmatically from an external site.
It's a bit involved to set everything up and write the logic or configure driving tools but this setup should give you full control, including error handling, logging, alerting, etc.
login (there are a number of options like SAML, and Salesforce credentials)
Then you could check the status and outcome of your jobs in the activity log or the activity monitor via API
use job and/or schedule via API to run your jobs.
Related
I want to run cyclical tests in jmeter. I want them to run every day and than run for 10 minutes (every day for 10 minutes). How to do it?
For running test for 10 minutes there are following options:
In the Thread Group tick "Specify Thead lifetime" and put 600 into "Duration" field:
Or use Runtime Controller which allows setting how long its children are allowed to run
With regards to running the test every day, you can go for:
Windows Task Scheduler
Linux/Unix crontab
MacOS launchd
or you can put the JMeter job under the orchestration of a Continuous Integration server, any of them can run specified job on the schedule or basing on various different triggers, tracks job status (successful or failed), some of them provide performance trends
There are many ways to answer your (a bit too broad) question.
Here are some insights that could help:
to launch a JMeter test that last 10 minutes, you have to configure a job in JMeter with such a duration. Then you have to lear how to launch it via command line instead of via the graphical interface (see this answer for example)
to launch your JMeter test every day, you can use a Continuous Integration tool like Jenkins. In this tool, you will be able to create some jobs with a specific schedule (every day in your case) and a specific task (launch my JMeter test via command line)
Can someone of you help me, how to make the following service selected in the image get into wait mode after starting the server.
Please let me know if developer trace is required to be posted for resolving this issue.
that particular process is a BATCH process, a process that runs scheduled background tasks (maintained by transaction SM36/SM37). If the process is busy right after starting the server, that means there were scheduled tasks with status released waiting for execution, and as soon as the server was up, it started those tasks.
If you want to make sure the system doesn't immediately start released background tasks, you'll have to set the status back to scheduled (which, thanks to a bit of weird translation, means they won't be executed because they are not released).
if you want to start the server without having a chance to first change the job status in SM37, you would either have to reset the status on database level (likely not officially supported by SAP) or first start the server without any BATCH processes (which would give you a number of great big warning messages upon login) and change the job status before then restarting the server with the BATCH processes. You can set the number of processes for each type in the profile of your instance (parameter rdisp/wp_no_btc).
I'm trying to execute a process to update my database, but the problem is that I set different RecurringJobs for it at different hours.
Today when I checked hangfire status, since yesterday that I instanced hangfire, I found the job should execute yesterday and the one task for today, both executed 30 minutes ago at the same time, and this has created duplicates in the database.
Can you help me with this?
If your problem is one of concurrency, you can solve it by running hangfire single threaded. Simply configure the number of hangfire worker threads on startup:
var server = new BackgroundJobServer(new BackgroundJobServerOptions
{
WorkerCount = 1
});
This will force hangfire to process queued jobs sequentially.
Alternatively, if you have the Pro version of hangfire you can control order using batch chaining.
I don't know if a worker can be considered as a thread.
Within a hangfire worker, single threaded code will be run by exactly one thread
This doesn't look like a concurrency issue as has been suggested. It's not completely clear what you are trying to do but I'm assuming you want the job to run at 7, 12:45, and 17:30 and had issues because both the 7am and 17:30 job ran at the same time (7am).
Based on the created time it looks like you created these around 14:30. That means the 17:30 job should have ran but didn't until the next morning around 7am. My best guess is this was hosted in IIS and the site app pool was recycled.
This would cause any recurring jobs that were supposed to run to be delayed until the app pool / site was started again (which I assume was around 7am).
Check out these documents on how to ensure your site is always running: http://docs.hangfire.io/en/latest/deployment-to-production/making-aspnet-app-always-running.html
If it's not an IIS issue something must have caused the BackgroundJobServer to stop monitoring the database for jobs until ~7:00am (server shutdown, error, etc).
Is there a plugin or can I somehow configure it, that a job (that is triggered by 3 other jobs) queues until a specified time and only then executes the whole queue?
Our case is this:
we have tests run for 3 branches
each of the 3 build jobs for those branches triggers the same smoke-test-job that runs immediately
each of the 3 build jobs for those branches triggers the same complete-test-job
points 1. and 2. work perfectly fine.
The complete-test-job should queue the tests all day long and just execute them in the evening or at night (starting from a defined time like 6 pm), so that the tests are run at night and during the day the job is silent.
It's no option to trigger the complete-test-job on a specified time with the newest version. we absolutely need the trigger of the upstream build-job (because of promotion plugin and we do not want to run already run versions again).
That seems a rather strange request. Why queue a build if you don't want it now... And if you want a build later, then you shouldn't be triggering it now.
You can use Jenkins Exclusion plugin. Have your test jobs use a certain resource. Make another job whose task is to "hold" the resource during the day. While the resource is in use, the test jobs won't run.
Problem with this: you are going to kill your executors by having queued non-executing jobs, and there won't be free executors for other jobs.
Haven't tried it myself, but this sounds like a solution to your problem.
I have a business critical application which needs to run 24*7. Right now its scheduled using Windows Task Scheduler. The problem with current implementation is whenever the application stops it has to wait for 1 minute to run again.(Since one minute is the minimum time to repeat task in Windows Task Scheduler) So I am building my own task scheduler which will start the process(application) within 5 seconds of terminating the process. How should my task scheduler know if the process has terminated. Do I need to keep polling the process every second to check whether its running or not?
You should write your application as a Windows Service, not a standard application.
Among their other advantages, Windows Services give you the ability to define what happens in the event of a failure (e.g. restart application).
They are also very easy to create in C#.