For a little while now, I’ve been working on a new backup script, as switching from Windows to a full Linux environment rendered my last script unusable.
So, here is the full script, it works well, but I’m making tweaks to it.
One issue I Have, is certain users don’t exist on certain machines, so my development branch doesn’t quite work. I’ve managed to loop through an array contains user names, and another looping through IP addresses, however I’ve not worked out away yet to filter those arrays to remove IP and user names that don’t exist on the target.
But basically, this script connects to machines using RSYNC, over SSH and copies the files to the backup location, and optionally uploads them to Amazon S3.
So, without further ado, here is a wall of code..
#!/bin/bash#Main Script file for Back ups.#See Bitbucket Repo for further information https://bitbucket.org/thompsonmichael/backup-sys#Michael Thompson 2018# mikethompson@gmx.co.uk (GPG Key-ID: 062C03D9)#Version 0.0.1#VARIABLESBCK_DEST=/mnt/Logical_DataEXCLUDE_FILE=/home/michael/Script/rsync_excludeS3_BUCKET=s3://RSYNC_CMD_STD="azh --progress"RSYNC_CMD_CLEAN=" --delete-after --delete-excluded"S3_CMD="-rHv --skip-existing --acl-private --continue-put --storage-class=STANDARD_IA --no-delete-removed --exclude-from=s3_exclude"S3_EXTRA=$2LOG_FILE="/home/michael/Script/log_file.log"REM_HOST="192.168.0.2"BLUE="\e[1;34m"RED="\e[1;31m"NORMAL_COL="\e[0m"if ! [ -z "$2" ];thenif ! [ "$2" = "-clean" ];thenecho "Running Custom S3 Command"S3_CMD=$S3_CMD" "$S3_EXTRAfifiecho Backing up systemsecho ______________echo -e ${BLUE} S3 Bucket Configured: $RED $S3_BUCKET${NORMAL_COL}echo -e ${BLUE}S3 Command is: $RED $S3_CMD${NORMAL_COL}echo -e ${BLUE}Exclude File Path: $RED $EXCLUDE_FILE${NORMAL_COL}echo -e ${BLUE}Running on: $RED $HOSTNAME${NORMAL_COL}echo -e ${BLUE}Destination is: $RED $BCK_DEST${NORMAL_COL}if [ -z "$1" ];thenecho -e ${BLUE}Command line passed: Empty ${NORMAL_COL}elseecho -e ${BLUE}Command line passed: $1 ${NORMAL_COL}fiechoecho -----------------------------------------------------------------------echo#error function. pass as func error,code,messagefunction_error () {echo -e ${RED}"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"echo -e "CRITICAL ERROR!"echo -e "Error occured in: " $1 "Error returned was: " $2if [ -z "$3" ];thenecho -e "Unknown Error, cannot advise. Check FAQ"fiecho -e "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"${NORMAL_COL}return 1exit}#ENSURE DRIVE IS MOUNTEDif mountpoint -q $BCK_DESTthenecho -e ${BLUE}"Backup Location is mounted " $BCK_DEST ${NORMAL_COL}elsefunction_error "Backup Location not mounted" $? "Mount location and restart"exitfiif [ -z "$1" ];then#--------------------------------------------------------------------------------------#Copy To Local Storageecho Command Returned: $?echo -e ${BLUE}Backing Up Dads${NORMAL_COL}wget -q --tries=10 --timeout=20 --spider http://google.comif [[ $? -eq 0 ]]; thenecho -e "Internet Connected"elsefunction_error "Internet Connection Down" "INT_DOWN $?" "Check Internet Connection"exitfiping -c1 ${REM_HOST} -q 2>&1 >/dev/nullRET=$?if [ ${RET} -eq 0 ]; thenecho -e ${BLUE}"Host Is Alive"${NORMAL_COL}echo -e ${BLUE}Ping Command Returned: $? ${NORMAL_COL}echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( ${REM_HOST} )"${NORMAL_COL}rsync -$RSYNC_CMD_STD -e ssh --exclude-from $EXCLUDE_FILE william@${REM_HOST}:/home/william $BCK_DEST/Backup/Dadsecho -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}if ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"firsync -$RSYNC_CMD_STD -e ssh --exclude-from $EXCLUDE_FILE $USER@${REM_HOST}:/home/$USER $BCK_DEST/Backup/$USERif ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"fiecho -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}elseecho -e ${RED}"Host ${REM_HOST} failed ping monitoring on `date`"${NORMAL_COL}echo -e ${RED}"Ping Command Returned (Ping):" $? ${NORMAL_COL}echo -e ${RED}"${REM_HOST} is Dead"${NORMAL_COL}fiecho -e ${BLUE}Backing up $HOSTNAME${NORMAL_COL}echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( $HOSTNAME )"${NORMAL_COL}rsync -$RSYNC_CMD_STD --exclude-from $EXCLUDE_FILE /home/michael $BCK_DEST/Backup/Michael-Debianif ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"fiecho -e ${BLUE}"Command Returned (RSync):" $?${NORMAL_COL}fiif [ "$1" = "-clean" ];thenping -c1 ${REM_HOST} -q 2>&1 >/dev/nullRET=$?if [ ${RET} -eq 0 ]; thenecho -e ${BLUE}"Host Is Alive"${NORMAL_COL}echo -e ${BLUE}"Ping Command Returned (Ping):" $? ${NORMAL_COL}echo -e ${BLUE}"Host is Alive" ${NORMAL_COL}echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( ${REM_HOST} )" ${NORMAL_COL}rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN -e ssh --exclude-from $EXCLUDE_FILE william@${REM_HOST}:/home/william $BCK_DEST/Backup/Dadsif ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"fiecho -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN -e ssh --exclude-from $EXCLUDE_FILE $USER@${REM_HOST}:/home/$USER $BCK_DEST/Backup/$USERif ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"fiecho "Command Returned (RSync):" $?elseecho -e ${RED}"Host ${REM_HOST} failed ping monitoring on `date`"${NORMAL_COL}echo -e ${RED}"Ping Command Returned:" $? ${NORMAL_COL}echo -e ${RED}"${REM_HOST} is Dead"${NORMAL_COL}fiecho -e ${BLUE}Backing up $HOSTNAME ${NORMAL_COL}rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN --exclude-from $EXCLUDE_FILE /home/michael $BCK_DEST/Backup/Michael-Debianif ! [ "$?" = "0" ];thenfunction_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"fiecho -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}#-----------------------------------------------------------------------------------------------------------------------------------# -s3clean has been added as a command line option, and must be passed a second command to -clean# it will cause a S3 clean event to be processed.# -clean on its own will pass only a standard archive clean. S3 is not routinly cleaned, unless explicity passed with -s3clean.if [ "$2" = "-s3clean" ];then#Call Clean_S3source s3_cmd.scfi#-----------------------------------------------------------------------------------------------------------------------------------fiif [ "$1" = "-s3" ];thenecho S3 destination is: $S3_BUCKETecho Amazon Upload Proceding...echo Uploading $BCK_DEST/Backup/Dads/william/Pictures/s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Pictures/ $S3_BUCKET/Dads/Pictures/if ! [ "$?" = "0" ];thenfunction_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"fiecho "Command Returned (S3CMD):" $?echo Uploading $BCK_DEST/Backup/Dads/william/Documents/s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Documents/ $S3_BUCKET/Dads/Documents/if ! [ "$?" = "0" ];thenfunction_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"fiecho "Command Returned (S3CMD):" $?echo Uploading $BCK_DEST/Backup/Dads/william/Videos/s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Videos/ $S3_BUCKET/Dads/Videos/if ! [ "$?" = "0" ];thenfunction_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"fiecho "Command Returned (S3CMD):" $?echo Uploading $BCK_DEST/Backup/Michael-Debian/s3cmd sync $S3_CMD $BCK_DEST/Backup/Michael-Debian/ $S3_BUCKET/Michael-Debian/if ! [ "$?" = "0" ];thenfunction_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"fiecho "Command Returned (S3CMD):" $?exitfi#EOF
Mikes Blog 