I've got:
$ rsync -azv zope#myserver:/smb/Data/*/*/* ~/rsynced_samples/
And I want it to run forever, syncing any new file as soon as it appears on myserver:
(specifying a poll interval, such as 4 seconds would be an ok comprise)
Instead of rsync you can use inotifywait which use kernel specific file changes triggers.
This script (inotify.sh) can you give an idea:
#!/bin/bash
directory=$1
inotifywait -q -m --format '%f' -e modify -e move -e create -e delete ${directory} | while read line
do
echo "doing something with: $line";
# for example:
# cp $line to <somewhere>
You can invoke this script specifying the "monitor" directory, in this way
./inotify.sh ~/Desktop/
The $line variable contains the full file path.
If you want to limit to only newly created files you can use on the flag "-e create"
Use cron to set up a check based on your time interval (say, every minute, perhaps?) . This link should help: http://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/
Note that a cron tab is set up on your machine side, not in your bash script
also useful: http://benr75.com/pages/using_crontab_mac_os_x_unix_linux
and here is a code example:
1) crontab -e // this opens up your current crontab or creates one if it does not exist
2) enter: * * * * * file.sh >> log.txt // this would pipe the output of your file to a log file and run it every minute.
hope that helps
Related
I want to Sync my server data to Google Cloud Storage to copy automatically using shell script. I don't know how to make script. Every time i need to use:
gsutil -m rsync -d -r [Source] gs://[Bucket-name]
If anyone knows the answer please help me!
To automate the sync process use cron job:
Create a script to run with cron $ nano backup.sh
Paste your gsutil command in the script $ gsutil -m rsync -d -r [Source_PATH] gs://bucket-name
Make the script executable $ chmod +x backup.sh
Based on your use case, put the shell script (backup.sh) in one of the below folders: a) /etc/cron.daily b) /etc/cron.hourly c) /etc/cron.monthly d)
/etc/cron.weekly
If you want to run this script for a specific time then go to the terminal and type: $ crontab -e
Then simply call out the script with cron as often as you want, for example, in midnight: 00 00 * * * /path/to/your/backup.sh
In case you are using Windows on your local server, The commands will be the same as above but make sure to use Windows path instead.
My problem is that I have a cluster-server with Torque PBS and want to use it to run a sequence-comparison with the program rapsearch.
The normal RapSearch command is:
./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32
Now I want to run it with 2 nodes on the cluster-server.
I've tried with: echo "./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32" | qsub -l nodes=2 but nothing happened.
Do you have any suggestions? Where I'm wrong? Help please.
Standard output (and error output) files are placed in your home directory by default; take a look. You are looking for a file named STDIN.e[numbers], it will contain the error message.
However, I see that you're using ./rapsearch but are not really being explicit about what directory you're in. Your problem is therefore probably a matter of changing directory into the directory that you submitted from. When your terminal is in the directory of the rapsearch executable, try echo "cd \$PBS_O_WORKDIR && ./rapsearch [arguments]" | qsub [arguments] to submit your job to the cluster.
Other tips:
You could add rapsearch to your path if you use it often. Then you can use it like a regular command anywhere. It's a matter of adding the line export PATH=/full/path/to/rapsearch/bin:$PATH to your .bashrc file.
Create a submission script for use with qsub. Here is a good example.
I have this code which created a backup of my database.
pg_dump -U dbadmin -h 127.0.0.1 123telcom -f dbbackup
Now i want to create a backup every night.
Is there a way u can execute this code with crontab?
0 3 * * * pg_dump -U dbadmin -h 127.0.0.1 123telcom -f dbbackup
I'm new to putty so if anyone could help me a little that would be great.
I suspect that you have fallen foul of cron's PATH set up.
If you look in /etc/crontab, it will define a PATH for itself and you will probably have a different PATH set up for your login.
Create your script with the first 2 lines:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
where the PATH includes whatever is set up in your environment and ensure that the script is executable.
To test what is going on try this script:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
echo $PATH >> /home/yourhome/cron.txt
create an entry in /etc/crontab:
* * * * * root /home/yourhome/yourshell.sh
tell cron about the changes by using sudo crontab -e and then just save it and exit (often Ctrl O and Ctrl X if using nano editor) or I think that you can just kill the cron process and it will re-spawn.
Then check the cron.txt file to see what it is using for PATH.
PS Don't forget to remove this script from the crontab afterwards
Looking for the origin of this error message:
Processing: +([^_]).flv
date: +([^_]).flv: No such file or directory
I started getting this at some point in the last few months (can't say when as I wasn't logging my cron output. I know, I know!).
When I originally wrote this, it worked ok for at least two months. I'm wondering if there was an sh update that broke it?
The script runs via crontab and gets all .flv files in the current directory without an underscore and processes each one. It then checks the modified date for files that have been created in the last 24 hours and runs the yamdi meta tag injector for .flv files.
It seems like it's not recognizing the pattern as a pattern and looking for it as an actual file to me. If I run this script from an ssh shell it works ok, it's only when running via cron that it gives this error.
shopt -s extglob
now=$(date +"%s")
for f in +([^_]).flv; do
echo "Processing: $f"
age=$(date -r "$f" +"%s")
calc=$(((now-age) / 60 / 60))
if(( calc < 24 )); then
echo "$f age=$calc"
yamdi -i "$f" -o "$f".seek
rm "$f"
cp "$f".seek "$f"
touch -d #$age "$f"
fi
done
This is most likely a problem of the wrong shell being used; make sure your script's first line represents the right shell:
#!/bin/bash
for bash, or whatever shell you wrote this for. You might want to check your environment variables that cron may set (that's a very common problem -- one assumes everything is set up correctly, but the environment that cron offers to scripts it executes is different).
I have a cron job script and i used >/dev/null 2>&1 to Stop sending Emails. But each time a file is created in the same name of PHP file with trailing numbers like phpfile.php.1, phpfle.php.2, phpfile.php.3….
Is there any script to stop that?
Add -O /dev/null to your wget command.