Update to the Backup

A quick update to the backup script. Instead of having a set list of folders stored in the backup directories to be uploaded to S3, the script now uploads everything in the backup location that’s not specifically excluded by the exclude file. This cuts a good few lines out of the script.

A quick note on the exclude file, you must exclude both the directory, and it’s contents as S3 has no concept of folders.

So, to exclude the folder /back/folder and it’s contents your exclude file must contain /back/folder and /back/folder/*


	if [ "$1" = "-s3" ]; 
		then 
			#This code is still subject to Testing. It shouldnt require a re upload of data.
#However, this uploads the WHOLE backup to the cloud. It does not filter out folders at this time.
#Use the S3 Exclude file for this purpose.
			shopt -s dotglob
			shopt -s nullglob
			array=($BCK_DEST/Backup/*)
			for dir in "${array[@]}"
				do
					echo $(basename "${dir%.*}")
					echo Uploading $dir
					FOL=$(basename "${dir%.*}")
					s3cmd sync $S3_CMD $dir $S3_BUCKET/$FOL/
					echo .......
				if ! [ "$?" = "0" ];
					then
					function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"
				fi
				done

	fi

See.Sense Ace

So I’ve had the New Ace lights for awhile now, and have been using them on both day and night rides.

They are designed for commuters, their power output is low, however it’s noticeable, not quite as retina searing bright as the Icon+, but they get the job done.

Ace Rear

But, there are issues. First off was battery life. Initially, it sucked. They would run out of juice just sitting there on the desk, meaning a charge every day. A firmware update went a long way to curing this, unless you are an Android user. The companion app took months to arrive from see.sense, they blamed the exit of their developer, and farmed it out to an external company. A release was made, but it lacked almost all the features, and was pretty much only able to turn the lights on an off. So if you wanted to update the firmware, you needed an iOS device, or at the very least a mate with an iOS device..

Now, in the latest Kickstarter news letter, they’ve stated an in-house android developer will soon be starting, so hopefully we shall have a passable android app soon(ish).

Other than the poor development and battery life issues, they really are quite a good set of lights for commuting with.

The mounting options for the lights are first class, you get a multitude of mounts as standard: including a seat-post mount, aero seat-post mount, and bag clip, which I use to mount to the back of my jacket as I’ve two rears, so one on the seat post, and one on my back. There have been reports of lights falling out of the cradle, but I assume this is due to not being clipped in correctly. My lights seem to be in solid when clipped in correctly. See.Sense, to my horror lobbed a poor innocent bike off a roof, and the lights stayed on, that was the icon2, however it uses the same mounting system.

The ACE lights feature the same intelligence of the previous lights, with a little more processing power. They can track changes in movement, g-force, and ambient lighting; tailoring the light output to the conditions and surroundings: for example, on a quiet road the light lets out just a gentle pulse, but if a car’s headlights are detected then the light lets out a sudden burst to increase awareness.

The reactivity both noticeably improves road presence, compared to a regular mid-power flasher. My only continued niggle with the reactive element of the See.Sense lights is that the sudden bursts of light from the front light can be a little distracting. This is because the light is designed to have good side visibility, which unfortunately includes the upper side too – upwards towards your eyes, should you have it mounted horizontal on the bars. To be fair, my favourite daytime lights also have this issue, the Exposure Trace. A sliver of electrical tape however remedies this.

The Smartphone integration allows you to tailor the output of the light to further improve battery life. It also allows you to use the movement sensor as a theft alert. Both useful features, if they ever get the android app to release.

Backups.. (Again!)

For a little while now, I’ve been working on a new backup script, as switching from Windows to a full Linux environment rendered my last script unusable.

So, here is the full script, it works well, but I’m making tweaks to it.

One issue I Have, is certain users don’t exist on certain machines, so my development branch doesn’t quite work. I’ve managed to loop through an array contains user names, and another looping through IP addresses, however I’ve not worked out away yet to filter those arrays to remove IP and user names that don’t exist on the target.

But basically, this script connects to machines using RSYNC, over SSH and copies the files to the backup location, and optionally uploads them to Amazon S3.

So, without further ado, here is a wall of code..

#!/bin/bash

#Main Script file for Back ups.

#See Bitbucket Repo for further information https://bitbucket.org/thompsonmichael/backup-sys

#Michael Thompson 2018

# mikethompson@gmx.co.uk (GPG Key-ID: 062C03D9)

#Version 0.0.1

#VARIABLES

BCK_DEST=/mnt/Logical_Data

EXCLUDE_FILE=/home/michael/Script/rsync_exclude

S3_BUCKET=s3://

RSYNC_CMD_STD="azh --progress"

RSYNC_CMD_CLEAN=" --delete-after --delete-excluded"

S3_CMD="-rHv --skip-existing --acl-private --continue-put --storage-class=STANDARD_IA --no-delete-removed --exclude-from=s3_exclude"

S3_EXTRA=$2

LOG_FILE="/home/michael/Script/log_file.log"

REM_HOST="192.168.0.2"

BLUE="\e[1;34m"

RED="\e[1;31m"

NORMAL_COL="\e[0m"

if ! [ -z "$2" ];

then

if ! [ "$2" = "-clean" ];

then

echo "Running Custom S3 Command"

S3_CMD=$S3_CMD" "$S3_EXTRA

fi

fi

echo Backing up systems

echo ______________

echo -e ${BLUE} S3 Bucket Configured: $RED $S3_BUCKET${NORMAL_COL}

echo -e ${BLUE}S3 Command is: $RED $S3_CMD${NORMAL_COL}

echo -e ${BLUE}Exclude File Path: $RED $EXCLUDE_FILE${NORMAL_COL}

echo -e ${BLUE}Running on: $RED $HOSTNAME${NORMAL_COL}

echo -e ${BLUE}Destination is: $RED $BCK_DEST${NORMAL_COL}

if [ -z "$1" ];

then

echo -e ${BLUE}Command line passed: Empty ${NORMAL_COL}

else

echo -e ${BLUE}Command line passed: $1 ${NORMAL_COL}

fi

echo

echo -----------------------------------------------------------------------

echo

#error function. pass as func error,code,message

function_error () {

echo -e ${RED}"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

echo -e "CRITICAL ERROR!"

echo -e "Error occured in: " $1 "Error returned was: " $2

if [ -z "$3" ];

then

echo -e "Unknown Error, cannot advise. Check FAQ"

fi

echo -e "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"${NORMAL_COL}

return 1

exit

}

#ENSURE DRIVE IS MOUNTED

if mountpoint -q $BCK_DEST

then

echo -e ${BLUE}"Backup Location is mounted " $BCK_DEST ${NORMAL_COL}

else

function_error "Backup Location not mounted" $? "Mount location and restart"

exit

fi

if [ -z "$1" ];

then

#--------------------------------------------------------------------------------------

#Copy To Local Storage

echo Command Returned: $?

echo -e ${BLUE}Backing Up Dads${NORMAL_COL}

wget -q --tries=10 --timeout=20 --spider http://google.com

if [[ $? -eq 0 ]]; then

echo -e "Internet Connected"

else

function_error "Internet Connection Down" "INT_DOWN $?" "Check Internet Connection"

exit

fi

ping -c1 ${REM_HOST} -q 2>&1 >/dev/null

RET=$?

if [ ${RET} -eq 0 ]; then

echo -e ${BLUE}"Host Is Alive"${NORMAL_COL}

echo -e ${BLUE}Ping Command Returned: $? ${NORMAL_COL}

echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( ${REM_HOST} )"${NORMAL_COL}

rsync -$RSYNC_CMD_STD -e ssh --exclude-from $EXCLUDE_FILE william@${REM_HOST}:/home/william $BCK_DEST/Backup/Dads

echo -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

rsync -$RSYNC_CMD_STD -e ssh --exclude-from $EXCLUDE_FILE $USER@${REM_HOST}:/home/$USER $BCK_DEST/Backup/$USER

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

echo -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}

else

echo -e ${RED}"Host ${REM_HOST} failed ping monitoring on `date`"${NORMAL_COL}

echo -e ${RED}"Ping Command Returned (Ping):" $? ${NORMAL_COL}

echo -e ${RED}"${REM_HOST} is Dead"${NORMAL_COL}

fi

echo -e ${BLUE}Backing up $HOSTNAME${NORMAL_COL}

echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( $HOSTNAME )"${NORMAL_COL}

rsync -$RSYNC_CMD_STD --exclude-from $EXCLUDE_FILE /home/michael $BCK_DEST/Backup/Michael-Debian

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

echo -e ${BLUE}"Command Returned (RSync):" $?${NORMAL_COL}

fi

if [ "$1" = "-clean" ];

then

ping -c1 ${REM_HOST} -q 2>&1 >/dev/null

RET=$?

if [ ${RET} -eq 0 ]; then

echo -e ${BLUE}"Host Is Alive"${NORMAL_COL}

echo -e ${BLUE}"Ping Command Returned (Ping):" $? ${NORMAL_COL}

echo -e ${BLUE}"Host is Alive" ${NORMAL_COL}

echo -e ${BLUE}rsync command is: "-$RSYNC_CMD_STD ( ${REM_HOST} )" ${NORMAL_COL}

rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN -e ssh --exclude-from $EXCLUDE_FILE william@${REM_HOST}:/home/william $BCK_DEST/Backup/Dads

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

echo -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}

rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN -e ssh --exclude-from $EXCLUDE_FILE $USER@${REM_HOST}:/home/$USER $BCK_DEST/Backup/$USER

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

echo "Command Returned (RSync):" $?

else

echo -e ${RED}"Host ${REM_HOST} failed ping monitoring on `date`"${NORMAL_COL}

echo -e ${RED}"Ping Command Returned:" $? ${NORMAL_COL}

echo -e ${RED}"${REM_HOST} is Dead"${NORMAL_COL}

fi

echo -e ${BLUE}Backing up $HOSTNAME ${NORMAL_COL}

rsync -$RSYNC_CMD_STD$RSYNC_CMD_CLEAN --exclude-from $EXCLUDE_FILE /home/michael $BCK_DEST/Backup/Michael-Debian

if ! [ "$?" = "0" ];

then

function_error "RSYNC" $? "Check RSYNC Command Line, and RSYNC Dirs"

fi

echo -e ${BLUE}"Command Returned (RSync):" $? ${NORMAL_COL}

#-----------------------------------------------------------------------------------------------------------------------------------

# -s3clean has been added as a command line option, and must be passed a second command to -clean

# it will cause a S3 clean event to be processed.

# -clean on its own will pass only a standard archive clean. S3 is not routinly cleaned, unless explicity passed with -s3clean.

if [ "$2" = "-s3clean" ];

then

#Call Clean_S3

source s3_cmd.sc

fi

#-----------------------------------------------------------------------------------------------------------------------------------

fi

if [ "$1" = "-s3" ];

then

echo S3 destination is: $S3_BUCKET

echo Amazon Upload Proceding...

echo Uploading $BCK_DEST/Backup/Dads/william/Pictures/

s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Pictures/ $S3_BUCKET/Dads/Pictures/

if ! [ "$?" = "0" ];

then

function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"

fi

echo "Command Returned (S3CMD):" $?

echo Uploading $BCK_DEST/Backup/Dads/william/Documents/

s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Documents/ $S3_BUCKET/Dads/Documents/

if ! [ "$?" = "0" ];

then

function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"

fi

echo "Command Returned (S3CMD):" $?

echo Uploading $BCK_DEST/Backup/Dads/william/Videos/

s3cmd sync $S3_CMD $BCK_DEST/Backup/Dads/william/Videos/ $S3_BUCKET/Dads/Videos/

if ! [ "$?" = "0" ];

then

function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"

fi

echo "Command Returned (S3CMD):" $?

echo Uploading $BCK_DEST/Backup/Michael-Debian/

s3cmd sync $S3_CMD $BCK_DEST/Backup/Michael-Debian/ $S3_BUCKET/Michael-Debian/

if ! [ "$?" = "0" ];

then

function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"

fi

echo "Command Returned (S3CMD):" $?

exit

fi

#EOF

Power of the edits..

While I always like to process images as little as possible, if at all, some times it’s a requirement of every shot.

Here, for instance this shot taken from my Spark. It lacks depth and colour. Visually, it just didn’t stand out.

However, a little time spent in the Google Photos editor tweaking exposure, clarity, saturation and blacks improve things.

Shadows have a bit more depth, the colours are less muted but still hold the autumn setting.

You don’t take a photograph, you make it

Ansel Adams

So, don’t be afraid to play with the sliders, just don’t overdo it, and keep it realistic and you’ll do just fine.

Wind.

The spark has continued to impress with its ability in windy conditions. I’ve flown it in some questionable wind speeds, bit being careful not to take it too far. So far, there’s only been one occasion where I’ve noticed it not being able to cope and that was on blubber houses crag. And by god, it was seriously windy.

Up til now, I have had no way of actually knowing the wind speed conditions, apart from looking on apps and the weather reports, so I’ve invested little more than a tenner for an anemometer from Amazon to measure the wind speed.

This will let me know exactly what the wind speed is, and honestly, for £12, it’s a good buy.

Range..

So, in Europe the power of a radio transmitter used in drones is limited to 18dbm, well below the ability of the controller’s 25dbm.

This equates to a massive cut in the range of the spark, about 500 meters is the limit before the video signal gives up and the drone returns. But fear not, there is something you can do to get the full range.

It basically involves fooling the DJI Go app to think your in a country that uses the FCC rules instead of the CE rules. To do it, follow these steps.

  • Ensure the drone and RC are both off
  • Turn aeroplane mode on your phone or tablet to On, and make sure to fully quit the DJI Go app
  • Install a fake GPS app and enable it.
  • Set your location somewhere in America or Canada.
  • Turn on the RC and drone and allow them to.connect.
  • Connect to the aircraft’s WiFi signal
  • Once the app starts up, it’ll ask you if you want to change the WiFi settings. Select ok.
  • Wait for the drones/RC’s network to come back online, and power everything off
  • Quit the app and the fake GPS provider, restart the drone/RC and app
  • When asked again to alter the WiFi settings, hit cancel

And that’s it. Each time you launch the app.from here, it’ll ask you to change the settings, always cancel the dialog.

To confirm your in FCC mode, goto the wireless page in the app, and look if there is a channel 12/13, if not your in FCC mode. Range in this mode will get you in excess of 1000 meters, as opposed to the 500 in the CE mode.

Happy flights!

A tad breezy

A bit windy for drone work, but the little spark put up a good effort in battling the wind, even if it did kill the battery in short order.

One thing I seem to be having issues with, is the controls. I have it set to the default at the moment, which is mode 2. This means the left stick controls elevation and turning movement, the right stick controls yaw and forward/rear movement. My mind always seems to try and reverse this in my hands and find I use the wrong stick for the command. So maybe I’ll stick with it and try and retrain my mind, or just change it.

The only other thing is working out which way the drone is going to move. Based on the drones orientation, moving the stick may not yield the results you expect. So knowing the orientation of the drone is important, but working it out can be a bit of a headache.