Recently I found myself several times in situations where I need to let run some operation in some background xterm and I'd need to be notified when my input is requested.
I know how to make it so I'm notified when the command ends, but that doesn't help in the cases where the command is not 100% batch (it puts up a prompt every now and then; a common example would be apt-get) or where the command hangs (because of some network failure, for example).
So I'd like to be notified when there's been no output in the last N minutes. Is there some way to configure xterm to do that for me, or maybe some other tool (screen maybe) that could do it?
xterm doesn't notice if the application is actually waiting for input, or simply doing nothing. An application (or shell) could be modified to do this, but that seems like a lot more work than you expected (i.e., many programs could be modified).
I also don't know of a way how to do it for applications that might be waiting for input, but if you have a batch application that should always output log info within a certain time span then you could run an extra process that does the notification if it doesn't get killed within a timeout. The process gets killed whenever a new line is read. Maybe that will help you or someone else to adapt it to processes that might wait for input:
i=0;{ while true;do echo $i;((i++));sleep $i;done }|while read line;do if [ $pid ];then sudo kill $pid;fi;bash -c 'sleep 5;notify-send boom'& pid=$!;echo $line;done
The part before the pipe sign is a process that outputs slower and slower and if it becomes slower than the threshold, notify-send sends notifications. If you wanted output to happen within 3 minutes, use sleep 3m.
Related
My workflow often includes PBS job submissions to a shared cluster that need to either wait in the scheduling queue, take over 24hrs to run or both. I'd like to run snakemake in the 'background' and get my prompt back while these jobs are running. I know this can be done using tmux, screen, or & but is there is a better way to do this?
I guess submitting a bash wrapper script with the snakemake commands inside is an option but I think I'm lacking some understanding of the workflow.
tmux is the recommended way to execute a Snakemake workflow. It will give you all you need, regardless of whether you are in a cluster or on a compute server.
I have exe which runs for 4-5 days, without a fixed frequency.
I have to manually restart it everytime it finishes running.
Is there a way to automate that?
Maybe using RabbitMQ or other Messaging brokers to queue messages to rerun it on completing execution once
In the Windows command line execute mycommand && mycommand. The && operator will wait until first the command is executed successfully and then execute the second one. In your case the first and second one are identical, so you achieve the double execution you want.
Hi spring batch users,
regarding the documentation http://docs.spring.io/spring-batch/reference/htmlsingle/#d5e1320
"If the process died ("kill -9" or server failure) the job is, of course, not running, but the JobRepository has no way of knowing because no-one told it before the process died."
I try to find and restart the stale job executions by using
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(jobName);
...
jobExecution.setStatus(FAILED);
jobExecution.setEndTime(new Date());
jobRepository.update(jobExecution);
jobOperator.restart(jobExecution.getId());
But this seems to be very inconvenient.
1) I have to do this before other (new) jobs could be started.
2) I have to handle multiple instances of running servers so findRunningJobExecutions will not do the trick.
You can find other questions regarding this topic:
https://jira.spring.io/browse/BATCH-2433?jql=project%20%3D%20BATCH%20AND%20status%20%3D%20Open%20ORDER%20BY%20priority%20DESC
Spring Batch after JVM crash
I would love to see a solution to register a "start up clean jobs listener". This will still not fix the problems originated by the multi server environment because spring batch does not know if the JobExecution marked by STARTED is not running on an other instance.
Thanks for any advice
Alex
Your job cannot and should not recover "automatically" from a kill -9 scenario. A kill -9 is treated very differently than you application throwing a caught Exception. The reason for this is that you've effectively pulled the carpet out from under the application without giving it a chance to reach a synchronization point with the database to commit any necessary information to the ExecutionContext or update the job/step status(es). Therefore, the last status touchpoint with the database will remain and the job will still look STARTED.
"OK, fine" you say, "but if I start another execution, I want it to find that STARTED execution, and pick up where it left off." The problem here is that there is no clean way for the application to distinguish a job that is ACTUALLY RUNNING from one that has failed but couldn't up the database. The framework here correctly errs on the side of caution and prevents you from starting a job that already appears running, and this is a GOOD thing.
Why? Because let's assume your job was actually still running and you restarted by accident. As coded, the framework will start to spin up, see your running execution and fail with the following message A job execution for this job is already running. I can't tell you how many times we've been saved by this because someone accidentally launched a job twice!
If you were to implement the listener you suggest, the 2nd execution would instead be allowed to start and you'd have 2 different JVMs repeating the same work, possibly writing to the same files/tables and causing a huge data mess that could be impossible to clean up.
Trust me, in the event the Linux terminal kills your job or your job dies because the connection to the database has been severed, you WANT human eyes on those execution states before you attempt a restart.
Finally, on the off chance you actually wanted to kill you job, you can leverage several other standard patterns for stopping jobs:
Stop via throw Exception
Stop via JobOperator.stop()
I've recently started learning VB.NET and I'm wondering is there an easy way of killing off all processes a VB.NET application uses, for example I've created a form which pings a given IP address, this application creates a process cmd.exe and sends the ping argument, this in turn creates following processes:
cmd.exe
conhost.exe
ping.exe
If I Kill () the main process it kills off cmd.exe but not conhost.exe nor ping.exe, do I need to manually kill these also? By killing off the main process will it not automatically kill off associated processes? If that makes sense. Another thing I don't understand, I tried using Close () but nothing appears to happen, all processes keep on running. I want to be able for a user to close the form and for all associate processes to be closed/killed.
It is much better to use the System.Net.NetworkInformation.Ping class to perform a ping (as Hans Passant mentioned).
In general, if you use proc = System.Diagnostics.Process.Start(...), you should be able to kill the process and its child processes with proc.kill. However, it is possible for a process to launch other processes that will not be immediately terminated with kill. It would be a bad idea to terminate leftover processes manually, for a number of reasons.
I want to write a program, that should be notified by O.S. whenever any running process on that OS dies.
I don't want to myself poll and compare everytime if a previously existing process has died. I want my program to be alerted by OS whenever a process termination happens.
How do I go about it? Some sample code would be very helpful.
PS: Looking for approaches in Java/C++.
Sounds like you want PsSetCreateProcessNotifyRoutine(). See this article to get started:
http://www.codeproject.com/KB/threads/procmon.aspx
Under Unix, you could use the sigchld signal to get notified of the death of the process. This requires, however, that the process being monitored is a child process of the monitoring process.
Under Windows, you might need to have a valid handle to the process. If you spawn the process yourself using CreateProcess, you get the handle for free, otherwise you must acquire by other means. It might then be possible to wait for the process to terminate by calling WaitForSingleObject on the handle.
Sorry, I don't have any example code for this. I am not even sure, that waiting on the process handle under Windows really awaits termination of the process (as opposed to some other "significant" condition, which causes the process handle to enter "signalled" state or something).
I don't have a code sample ready but one idea – on Linux – might be to find out the ID of the process you'd like to watch when first starting your watcher program (e.g. using $ pgrep) and then using inotify to watch /proc/<PID>/ – which gets deleted when the process dies. In contrast to polling, this doesn't cost any significant CPU resources.
Now, procfs is not completely supported by inotify, so I can't guarantee this approach would actually work but it is certainly worth looking into.