I have a NAS on my LAN and I want to have an incremental backup once a day on a removable hard disk and then a full backup once a week. I heard about crontab and rsync, but I don't have any idea on how to do incremental and full backups.
My PC is running Linux, so I can use rsync and crontab. How should I set up rsync and crontab working together to have a daily incremental backup and a weekly full backup?
Supposing that NAS IP Address: 192.168.1.100 and the removable hard disk is plugged in my PC with USB, what scripts do I have to write? I need to backup all the folders in the NAS.
Thank you in advance!
Here is a detailed instruction about how to use rsync and crontab to backup linux files.
http://www.cubebackup.com/blog/automatic-backup-linux-using-rsync-crontab/
But you need to understand the process and made modifications based on your need.
Thanks for all the answers and hints. I've found a way to do what I asked for and I want to share it with all of you.
First: backup.sh. In this file we set up the source directory and the destination folder in which all the files we'll be copied. The code is easy to read.
#!/bin/bash
#Backup script
#Config
src="/Users/user/Backup/from/*"
dest="/Users/user/Backup/to"
log="/Users/user/Backup/backup.log"
user=$USER
date=$(date +"%d-%m-%Y %T")
#Output on log file
echo "--- Start Backup" >> $log
echo "--- Source: $src" >> $log
echo "--- Dest : $dest" >> $log
echo "--- $date by $user" >> $log
echo "---" >> $log
#Command
rsync -t -r -v --progress --stats --delete $src $dest >> $log
#Output on log file
echo "---" >> $log
echo "--- End Backup" >> $log
Second: crontab. With a few command lines we can setting up a crontab job every day (night) at 23.59. Open the terminal (with root privileges $ su and enter root's password. If you are using root for the first time, first set the password for it with $ sudo passwd).
Open the crontab editor with
# crontab -e
Then make the previous script (supposed to be saved in /Users/user/Backup/backup.sh) runnable everyday at 23:59.
59 23 * * * /Users/user/Backup/backup.sh
Related
I'm trying to write a batch script in Windows to take a list of IP addresses and ping them. Once a site doesn't respond, I want Windows to take all of the unresponsive IP addresses, and parse them through a comparison file that has the IP and physical street addresses of these systems. Once the unresponsive sites are parsed through the comparison file, I want the end result to be the matching info from the compare file for only the sites that are unresponsive. I already have script written for Linux that does this same thing, but I wanted a Windows version for some of the customers I work with who aren't Linux savvy.
Here is my script:
#Echo Off
Set "ServerList=C:\Users\<mylogin>\ip.txt"
Set "LogFile=C:\Users\<mylogin>\PingResults.txt"
If Not Exist "%ServerList%" Exit /B
>"%LogFile%" (For /F UseBackQ %%A In ("%ServerList%"
) Do Ping -n 1 %%A|Find "TTL=">Nul&&(Echo Yes [%%A] > Nul)||Echo No [%%A])
findstr /f:%LogFile sites.txt > Down.txt
The script itself seems to execute just fine, but it doesn't put anything in the final output file of Down.txt, I'm positive I have something wrong in my findstr command.
Below is my Linux script that does this exact same thing. Yeah it's clunky but it gets the job done:
#!/bin/bash
# Script to test ssh connectivity using expect script
rm -f results.txt
clear
echo "Please be patient while the script runs..."
while read user ip port pass; do
${PWD}/test_ssh_edge_device.sh $user $ip $port $pass >> results.txt
done < rekor_edge_device_list.txt
#This will boil down the results of the ping script so only the IP address is left
cat results.txt | grep -B1 time > refined.txt
cat refined.txt | grep -Eo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" > refinedip.txt
#This will take the boiled down IP addresses and check them against the compare file, final output will be all system info for only downed systems
list=refinedip.txt
rm -f iplist.txt
exec 3<&0
exec 0<$list
while read line
do
cat compare.txt | grep $line >> iplist.txt
done
exec 0<&3
cat iplist.txt
I have to collect measurement files from different servers, so I used scp command to retrieve them.
But in case the distant server is hanged or no response, I need to close the connection and put a 0 in my measurement file.
Is there any option in scp command allow me to close the connection after 10 seconds for example?
for serv in $SERV_LIST
do
echo "--- Working on server: $serv ---"
trc_file=`ssh user#$serv "$(typeset -f collectSTATS); collectSTATS $serv $DATE $LastRunTime
scp user#$serv:/tmp/result_rechHM2_$serv.tmp /home/voms/HDB2/result_rechHM2_$serv.tmp > /dev/null 2>&1
deleteFile=`ssh voms#$serv "rm /tmp/result_rechHM2_$serv.tmp 2> /dev/null"`
if [ -f /home/voms/HDB2/result_rechHM2_* ]
then
cat /home/voms/HDB2/result_rechHM2_* >> /home/voms/HDB2/TraceRecharge.log
rm -rf /home/voms/HDB2/result_rechHM2_*
fi
done
When ssh or scp command fail with no response, I need to wait only 10 seconds.
we just use the ssh -o ConnectTimeout=5.
it resolve my problem
I have this code which created a backup of my database.
pg_dump -U dbadmin -h 127.0.0.1 123telcom -f dbbackup
Now i want to create a backup every night.
Is there a way u can execute this code with crontab?
0 3 * * * pg_dump -U dbadmin -h 127.0.0.1 123telcom -f dbbackup
I'm new to putty so if anyone could help me a little that would be great.
I suspect that you have fallen foul of cron's PATH set up.
If you look in /etc/crontab, it will define a PATH for itself and you will probably have a different PATH set up for your login.
Create your script with the first 2 lines:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
where the PATH includes whatever is set up in your environment and ensure that the script is executable.
To test what is going on try this script:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
echo $PATH >> /home/yourhome/cron.txt
create an entry in /etc/crontab:
* * * * * root /home/yourhome/yourshell.sh
tell cron about the changes by using sudo crontab -e and then just save it and exit (often Ctrl O and Ctrl X if using nano editor) or I think that you can just kill the cron process and it will re-spawn.
Then check the cron.txt file to see what it is using for PATH.
PS Don't forget to remove this script from the crontab afterwards
Example of the contents I require a .cmd to contain
mkdir Output
sqlcmd -S serverName -d dbName -E -i "FILE LOCATION HERE" -o Output\Message.log
sqlcmd -S serverName -d dbName -E -i "FILE LOCATION HERE" >> Output\Messages.log
.
.
.
pause
Specifics: I have a SQL repo and need to generate a code that will take all changes from last revision to this revisionand output the above example.
As far as Mercurial is concerned,
hg status -I re:.*\.sql$ -am --rev 3:7
will give the relation of all files with .sql extension added or modified after changeset 3 and up to changeset 7. You can then massage the output into the desired script with your favourite text processing tools.
N.B.: Blindly running all modified SQL scripts, specially if you intend to use this procedure more than once, seems rather dangerous, unless you are sure all the operations are idempotent.
I've got:
$ rsync -azv zope#myserver:/smb/Data/*/*/* ~/rsynced_samples/
And I want it to run forever, syncing any new file as soon as it appears on myserver:
(specifying a poll interval, such as 4 seconds would be an ok comprise)
Instead of rsync you can use inotifywait which use kernel specific file changes triggers.
This script (inotify.sh) can you give an idea:
#!/bin/bash
directory=$1
inotifywait -q -m --format '%f' -e modify -e move -e create -e delete ${directory} | while read line
do
echo "doing something with: $line";
# for example:
# cp $line to <somewhere>
You can invoke this script specifying the "monitor" directory, in this way
./inotify.sh ~/Desktop/
The $line variable contains the full file path.
If you want to limit to only newly created files you can use on the flag "-e create"
Use cron to set up a check based on your time interval (say, every minute, perhaps?) . This link should help: http://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/
Note that a cron tab is set up on your machine side, not in your bash script
also useful: http://benr75.com/pages/using_crontab_mac_os_x_unix_linux
and here is a code example:
1) crontab -e // this opens up your current crontab or creates one if it does not exist
2) enter: * * * * * file.sh >> log.txt // this would pipe the output of your file to a log file and run it every minute.
hope that helps