I'm having a problem with xargs and Wget when run as shell scripts in an Applescript app. I want Wget to run 4 parallel processes in the background. The problem: basically, when I try to run the process in the background with
cat urls.txt | xargs -P 4 -n 1 /usr/local/bin/wget -q -E -b 1> NUL 2> NUL
a Wget process is apparently started for each URL passed in from the .txt file. This is too burdensome on the user's memory. When I run it in the foreground, however, with something like:
cat urls.txt | xargs -P 4 -n 1 /usr/local/bin/wget -q -E
I seem to get the four parallel Wget processes I need. Does anybody know how to get this script to run in the background with only 4 processes? I'm a bit of a novice, and I'm afraid I can't figure out why backgrounding the process causes this change.
You might run xargs on the background instead:
cat urls.txt | xargs -P4 -n1 wget -q &
Or if you want to return control to the AppleScript, disown the xargs process:
do shell script "cat urls.txt | xargs -P4 -n1 /usr/local/bin/wget -q & disown $!"
As far as I can tell, I have solved the problem with
cat urls.txt| (xargs -P4 -n1 wget -q -E >/dev/null 2>&1) &
There may well be a better solution, though...
Related
I am trying to implement license agreement as shown in the attached image, I can create a new WindowController and design in the same way, but i want to know if we have any default screens in Xcode to do this kind of stuff, as i noticed that most of the apps have similar kind of design.
That's a disk mount license agreement. You can embed a license agreement into a DMG disk image, to be displayed with this screen when a user mounts the image.
If your application is in example/Example.app, you can create example.dmg, embedding EULA.txt into it as follows:
hdiutil create -srcfolder example/ -volname "Example" "example.dmg"
./eula_into_dmg.sh "example.dmg" "EULA.txt"
# you should also codesign the dmg if you're going to distribute it:
codesign -s "$CODE_SIGN_IDENTITY" -v "$dmg"
This is eula_into_dmg.sh:
#!/bin/bash -u
if [[ $# -ne 2 ]]; then
echo "Usage: $0 <dmg> <license>"
exit 1
fi
prefix=$(base64 -D <<<\
H4sIAJtVUFoCA4VWW2vbSBh9bn7Fh3ahbci6o9FcLPahJJIMBbcNjaH7qjpyYupaiyW3pGX/+55vpJF1\
ianBYmTjo/Ody4zv8zqnl6v3t8uX9CqU8ysKlrfbdfCafl28+DMII6XIaGPIhFaTSawiKVSCW5ORsfxZ\
aGxAvdebS5qlxSY/7mpa5vuHY/5Al28YzWgpSMWAVEpb0hIroVVEZsFAGcDVQkVjtILepennT+ksKY/7\
+mPSoqlM4+dKXpO8dhfQ0hE+U5G4IRvZGBcpxmgfVrNLvJZ3q2RWPVXkuSUmBAVjeb4YBMFNWy2xCjFp\
YhZkIhOO0XYY0NPbleu8Q8OkVhrtKDQzSyHnmJS1NBYza6smaHQoKsDRq3KzqYq6RZMCPwIaKKRAi3SE\
i8DlJGMoIjlGo82h/EZaCOHoSY+WGkk2tmws0E6DW80rC9LRQk24/fXlqS7cvDC0eJu2aK1GQnkXcFGJ\
jkiD3ABjgMaEWhsyRvrv74uLe5dFjh+yyLTbEGIlCG9JfjW6KLeahHD0Gj7mbvXpj/YxyHy2f9htq0f6\
cqzrcl8FpycbEqnSjUCQijNl5vDSyQcbrYrCZ57sAeuiqsNWKkSd9bWSG6VJzLlbMUchZPU5LUYLPUG7\
fjgUxSzdVrlbeOEFA8WooVUCKUA4yRpgyExmuFj4EBszRrs9bPf17C7/XoDlz3ebLmKoi6shwtZniS8s\
swTV+bRM9FQeydGiH9v60ZfJKmjk0oWV5tHSpggu/o14cxOP0erHAoIdvlVUbnCz9WhRE9OYG+j2CW6P\
TJhq1H1xM0araLddF/uquKI1Vl+7SSUC21Mctw2jBT8mZEg8ZjJp4GwIqC4pX6+LjhvI+IGcWlEzpFWs\
W+hklNmEW+WGrcpN/SM/dJ6Cm3Cu0cAPzy3DDmDVMy4QfGQn7kval3W3pfWHZLVstyc50qyDmrrg7Lyi\
f7EVVRSkJxeGucikPFPv4aRNaINz7ZOn9g1bZ6etGxbofOuaxvjWnakZDQs0PcJ835rG+NZNanbzjGGT\
HaHt26/Wp+6YOFMzGhZojHbqW9MY79CZmtGwQGO0U9+axnhuvZo9Exyv5RjN9a3NDttwOvoHNXOyR3wb\
GvybSNqZJ0e/69t2X9X5bgeK/T8SiMSCj1JF1rIpsjFFNAe4MdOe9AvHlfG6xdwrdxYqrOBfxk6q0VY4\
RusVLu/lTbpZXLbc+cDaSznJ4BitVziXu4Fu2W+bNtFtdNytsn9W0+PubnntSvc/iogHYQYKAAA= | gunzip -)
suffix=$(base64 -D <<<\
H4sIANNVUFoCA6u15uJKSSxJVFAvLqnMUVfQMDUwMNBRUHLNS8/JLM5QCPZxVNJUqObiVFECShgrAAkD\
OOEMIiyBhKEJQkJJAQnoaynowcAxCKWgpQ81zUABgzAyRzMXp2nqEBNhphliMw3JXEewuThNAwEtZLeh\
+wqNQDEIbtoxZONAZtVacwEAUriwzWEBAAA= | gunzip -)
temp=$(mktemp)
echo -n "$prefix" > "$temp"
cat "$2" | dd conv=swab | hexdump -v -e '8/2 "%04x " "\n"' | sed -e 's/ *$//' -e 's/^/ $"/' -e 's/$/"/' >> "$temp"
echo -n "$suffix" >> "$temp"
hdiutil unflatten "$1"
Rez -a "$temp" -o "$1"
hdiutil flatten "$1"
rm "$temp"
There is no default UI Element for it, nor is there a well known community option. Best option is to design your own.
I have a read-only remote filesystem that stores logs.
I use ssh -t to run grep queries on these logs. Sometimes, the queries can take too long and cause the ssh to timeout.
Is there some way to stream the stdout back and keep ssh connection alive?
Example command:
ssh -t my-host.com "cd /path/to/my/folder ; find ./ -name '*' -print0 | xargs -0 -n1 -P8 zgrep -B 5 -H 'My search string'" > search_result.txt
Thanks
I have the script below.
OUTPUT_FOLDER=/home/user/output
LOGFILE=/root/log/test.log
HOST_FILE=/home/user/host_file.txt
mkdir -p $OUTPUT_FOLDER
rm -f $OUTPUT_FOLDER/*
pssh -h $HOST_FILE -o $OUTPUT_FOLDER "cat $LOGFILE | tail -n 100 | grep foo"
When I run this script on its own, it works fine and the $OUTPUT_FOLDER contains the output from the servers in the $HOST_FILE. However, when I ran the script as a cronjob, the $OUTPUT_FOLDER is created, but it's always empty. It's as if the pssh command was never executed.
Why is this? How do I resolve this?
This is a rather simple question but I haven't been able to find an answer.
I have a large number of jobs running in a cluster (>20) and I'd like to delete them all and start over.
According to this site I should be able to just do:
qdel -u netid
to get rid of them all, but in my case that returns:
qdel: invalid option -- 'u'
usage: qdel [{ -a | -c | -p | -t | -W delay | -m message}] [<JOBID>[<JOBID>]|'all'|'ALL']...
-a -c, -m, -p, -t, and -W are mutually exclusive
which obviously indicates that the command does not work.
Just to check, I did:
qstat -u <username>
and I do get a list of all my jobs, but:
qdel -u <username>
also fails.
Found the answer buried in an old supercluster.org thread:
qselect -u <username> | xargs qdel
Worked flawlessly.
Building on what Gabriel answered:
qselect -u <username> | xargs qdel
qselect -u <username> -s <state> | xargs qdel
<state> would be R for running jobs only.
qselect will allow you to select job based on other criterias, like ressources asked (-l), destination queue (-q) ...
qdel -u <username>
will only work with SGE
sometimes a simple grep/cut can help too:
qstat | grep $USER | cut -d. -f1 | xargs qdel
This way we can also grep on a particular keyword for the jobs and delete them.
HTH
Try
$ qdel {id1..id2}
So for example:
$ qdel {1148613..1148650}
For UGE:
qstat -u | gawk '{print $1}' | xargs qdel
# Delete all jobs owned by the current user.
#
# Command breakdown:
# ------------------
#
# qselect
# -u selects all jobs that belong to the current user
# -s EHQRTW selects all job states except for Complete
#
# xargs
# --no-run-if-empty Do not run qdel if the result set is empty
# to avoid triggering a usage error.
#
# qdel
# -a delete jobs asynchronously
#
# The backslashes are a trick to avoid matching any shell aliases.
\qselect -u $(whoami) -s EHQRTW | \xargs --no-run-if-empty \qdel -a
Another possibility is to do qdel all. It deletes all jobs from everyone. When you don't have access for other people's job, it deletes only your jobs.
It is not the most beautiful solution, but it is surely the shortest!
qstat | cut -d. -f1 | sed "s; \(.*\) 0;qdel \1;" | bash
sed's power.
Just use the following command:
qdel all
It will cancel all jobs running on cluster.
Hi i have the following process which i cant kill:
I am running cygwin in windows xp 32 bit.
I have tried issuing the following commands:
/bin/kill -f 4760
/bin/kill -9 5000
kill -9 5000
kill 5000
When i write /bin/kill -f 4760 i get the message, 'kill: couldn't open pid 4760'.
When i write /bin/kill -9 5000 i get the message, 'kill: 5000: No such process'.
I simply don't understand why this process cant be killed.
Since it has a WINID shouldnt it be killed by /bin/kill -f 4760?
hope someone can help thx :)
The process is locked from Windows most likely. The error you are getting "couldnt open PID XXX" points to this.
To confirm try killing it with windows taskkill
taskkill /PID 4760
Strangely, the following works in Cygwin:
echo PID1 PID2 PID3 | xargs kill -f
For example:
ps -W | grep WindowsPooPoo | awk '{print $1}' | while read line; do echo $line | xargs kill -f; done;
Different Windows programs will handle the signals that kill sends differently; they've never been designed to deal with them in the same way that Linux/Cygwin programs are.
The only reliable method for killing a Windows program is to use a Windows specific tool, such as Task Manager or Process Explorer.
That said, if you've not already, you may have luck with running your Cygwin terminal in administrator mode (right click on your shortcut and select "Run as administrator").
The method presented by #Donal Tobin is correct:
kill -f <pid>
However, I don't need to log in as administrator.
Create a file called killall.sh with this line
ps -W | grep $1 | awk '{print $1}' | while read line; do echo $line | xargs kill -f; done;
Then give it execute permissions.
chmod 777 killall.sh
In your .bash_profile add this line
alias killall="~/killall.sh" (point it to the correct location)
Then you just have to type "killall [name]"
killall.sh - Kill by process name.
#/bin/bash
ps -W | grep "$1" | awk '{print $1}' | xargs kill -f;
Usage:
$ killall <process name>
For me this command does not work on Windows 10 in Cygwin:
$ kill -f 15916
bash: kill: (15916) - No such process
Instead of it, you can use next commands:
$ powershell kill -f 15916
$ netstat -ano | grep ':8080' | awk '{print $5}' | xargs powershell kill -f
$ netstat -ano | grep ':8080' | awk '{print $5}' | while read pid; do powershell kill -f $pid; done;
$ netstat -ano | grep ':8080' | awk '{sub(/\r/,"",$5) ; print $5}' | while read pid; do taskkill /F /PID $pid; done;
SUCCESS: The process with PID 15916 has been terminated.