Category: Technical

Another backup post

So I had a thought of redoing my backup scripts, as my existing script file was becoming huge, complicated and difficult to follow.

It was a good way for me to learn Bash scripting however. The idea was to vastly simplify the whole process. I began by thinking the best way, and decided to use each machine to independently backup to a network drive, rather than having a single machine doing the grunt work running a script.

So each computer on the network had the drive mounted, and the script file placed into the main users crontab. I could of used the root crontab to copy the whole /home/ directory of course, but each machine only has one real user so I opted for that.

The file that runs from crontab is very simple

source /mnt/dlink_nfs/backup-script/var-dec
rsync -va --delete-after --delete-excluded --exclude-from=$FOLDER_NFS/backup-script/exclude.lst /home/$USER $FOLDER_NFS/backup-test/$DIRNAME

And that’s it. Just uses rsync to copy the contents to the network drive . The referenced source file is just shared variable declarations.

Now, that’s not quite enough for me to be happy with a backup system, so I use a raspberry pi, to run a second set of scripts from its crontab. Those files are responsible for uploading to Amazon S3, and also copying to a secondary NAS.


#!/bin/bash

source /mnt/dlink_nfs/backup-script/var-dec
echo "Script Started: $(date)" >> uploads3.log

if pidof -x "$script_name" -o $$ >/dev/null;then
   echo "An another instance of this script is already running"
echo "Script Already running, exiting" >>  uploads3.log
echo "-----------------------"
   exit 1
fi

if [[ $1 == 'clean' ]]
	then
		echo "clean command passed" >> uploads3.log
		rsync -vruO --delete-after $FOLDER_NFS/backup-test /mnt/samba/
		echo "Clean compleated $(date)"	
		exit 1
else
	if mountpoint -q /mnt/samba
		then
echo "Samba share mounted, started RSYNC" >> uploads3.log
		rsync -vruO $FOLDER_NFS/backup-test /mnt/samba/
	fi

	cd $FOLDER_NFS/backup-test/
echo "Starting S3 uploads" >> /home/pi/uploads3.log
	shopt -s dotglob
	shopt -s nullglob
	array=(*/)
	echo runing s3

	for dir in "${array[@]}"
	 do 
		echo "Currently Running S3 on $dir" >> /home/pi/uploads3.log
		dir=${dir%/}
	        timeout 30m s3cmd $s3_cmd $dir $s3_bucket
		echo "Compleated uploading $dir" >> /home/pi/uploads3.log

	 done
echo "Finished Script: $(date)" >> /home/pi/uploads3.log
echo "--------------------" >> /home/pi/uploads3.log
fi

And that file basically, ensures the script isn’t already running, copies the backup to another NAS, then iterates through each directory uploading to S3. I use timeout to limit each upload to 30mins to prevent overruns. Once the initial upload has completed, this limit can be removed.

You can view the most up to date git repository at my github site https://github.com/mikethompson/new-backup

New Pedals

So, I’ve had a little upgrade from my Shimano R550 pedals, I’ve gone and installed a pair of Look Blades.

So, initial thoughts on them, is that they are easier to engage into than the Shimano, and oddly more comfortable. But unclipping is another matter. It’s not difficult to unclip, it’s just the range of motion is greater to actually unclip and it seems to throw the foot off, almost in disgust that your unclipping. Any hesitation, or having the cleat near seems to suck the cleat back into the mechanism.

With the Shimano, a kick to the side, and your foot was just on the pedal, ready to put down, but still be able drive forward if needed. Handy for those traffic slow down moments. The blades, a throw from the ankle sees your foot thrown, a more gentle twist, unclip then reclips the cleat.

But in truth, they are a great pedal, and it’ll be time before I’m used to their quirks, but they are on my bike to stay. I might however, fit the tighter blade, as the 12Nm that comes fitted as standard just doesn’t feel tight enough.

I’ve now been riding them for a few months, and occasionally get more clip in fails than I did with the shimano pedals. Wether this is down to lack of skill on my part or the pedal is however up for debate.

Headset service

Headsets are one of the things that people seem to have problems with, so during this maintenance of my headset, I’ll work through the steps needed to break down, clean and rebuild a headset.

The headset on this bike uses two sealed bearings in an integrated, other bikes use caged bearings.

So, the first thing todo is remove the front wheel. Once that’s out the way, remove the brake, and secure it to the frame of the bike. Doing this will let you completely remove the full fork, and make it easier to clean the steerer tube, and fork crown.

Now comes the interesting bits.

Start by removing the top cap, there’s a lot of parts to a headset, so make sure you have somewhere to put them all down. Try to lay out the parts in the order you remove them.

There maybe a small spacer under the top cap.

Slacken off the stem pinch bolts, and gently remove the stem and handlebar assembly, and secure it on the bike frame.

Remove all the spacers from under the stem.

But while your doing that, ensure your supporting the fork, as there is a possibility it might drop out the frame.

Now, if the forks won’t move down, support the fork underneath, and slap the top of the exposed steerer tube, and it should move down. Now push it back up, and the weather cover, and the spacer should be able to be removed. the whole fork should now slide out.

Check the fork to see if the bearings came out with the fork, or if the stayed in the frame.

Now, remove the bearings, from both the top, and bottom, taking note of which way they came out, and which one was top and bottom.

Now, get a rag, and clean up.

  • Clean the top bearing seat
  • Clean the bottom bearing seat.
  • If sealed bearings, wipe off the old grease and dirt, but don’t use degreaser. Spin each bearing in your fingers feeling for any play or grittiness. Replace if any problems
  • Clean off the fork steerer tube, and crown race.
  • Leave all parts to dry out.

When your ready, it’s time to reassemble.

Start off with a liberal amount of grease to each bearing race, top and bottom, and the crown race. Use a good quality grease, the grease both lubricates the races, and protects the bearings from dirt and water. The headset gets absolutely soaked by the front wheel, so a good waterproof grease is the best option.

Now, slide the first bearing onto the steerer tube in the right orientation, and seat it into the grease. I like to take the grease that presses out and smear it over the bearing. Now, taking the fork, slide it back into the frame, making sure the bearing seats correctly.

And, this is where having everything to hand really helps. As your holding the fork in place, slide in the top bearing, and the bearing spacer. I like to smear a little grease on the top of the bearing, just to give it a little more protection. Add the weather cap, spacers and put the stem back on the steerer tube. Rethread the top cap, along with any spacers needed, don’t tighten it, just enough to hold the whole system to get her.

Now take a moment, and clean up any grease that’s squeezed out. Reattach the brake, Put the front wheel back in and finally center up the brakes.

It’s time to tension the whole assembly. Take the bike out of the work stand, nip down the top cap just a touch. Apply the front brake and rock the bike forward and back. What your feeling for, is play in the headset. So any knocking you feel, tighten the top cap by a very small amount. Keep on repeating until you can’t feel any play, or feel any knocking. Check that the whole system can be rotated, with no binding.

If a video, is easier to follow for this, I’ll let the excellent GCN explain it in this video.

Once your happy with the tension, center your stem, and tighten your stem bolts, double check your bars are still straight, and go ride your newly greased bike!

I will update this with pictures the next time I do a full headset service.

Raspberry Pi

I’ve gained another raspberry pi for the home network. I’ve been thinking what I can do with it.

I tried running a squid proxy, and to be honest, it’s just not worth the effort. Most websites now use HTTPS, and a result of which will not be cached or scanned by the proxy. And the overall speedup of the network was non existent or very close to minimal benefit.

So, I settled on running a OpenVPN server, so I can access the whole network when away from the house. This makes things more secure as I no longer have to have the CCTV system open to the internet, and I can use it to route around other networks I don’t trust. Which is pretty much all networks not under my control. I’ve also used the box in the backup scheme. It now takes the archive from its primary location, and copies it to a NAS, and once a month runs a scrip to compress it to a tar.gz and copy it to another NAS on the system. This gives me 3 copies of the data, plus my copies on Amazon S3. So all that data should be pretty safe.

The NAS boxes are connected only when data is being copied, and disconnected when done to avoid any nasties corrupting the backups between updates. I’ll update the scripts I use and publish or update my previous post on the back up later.

Now, I just need to get each of them mounted via NFS to my main desktop so I can work on the scripts without having to SSH to them.

Too Google or not to Google

For the longest time I’ve been an android guy. Recently however, that changed.

I was in the market for a new handset, and it was between the iPhone 11, and the Pixel 4. I watched the keynote from google, and there was nothing for the pixel 4 that really grabbed me, I was underwhelmed. There was nothing on the handset that really grabbed me as a must upgrade feature.

Couple that with Google’s need to grab all the data they possibly can, as evidenced by my piHoles constant DNS blocks from the android handsets, I decided to go down the Apple hole. Which I will be honest about, is not a decision I regret.

Apples stance on privacy is the polar opposite of google. That’s mostly the fact that stood out to me, and weaning my data away from google wasn’t that hard.

iPhones and Apple hardware does indeed have its issues, and a slight learning curve, but I’m an Apple convert.

TwitterBot

I’d thought about writing a script for a Twitter bot. So the only thing I could think of is to push PiHole stats from my PiHole server.

In order to create a Twitter bot, you have to register as a developer but that’s easy enough. Next up is to find a interface that’s scripted in bash, as I didn’t fancy trying python, and I found the excellent Twurl Package.

Here’s a link to my repo where I’ve stored the code: https://github.com/mikethompson/PiHoleStats

I’ll update this, add a step by step and the repo when I have more time.

Pedal Maintenance and new wheels

Respect your pedals. They suffer a lot of abuse but just keep on going..

My pedal maintenance is simple. Every few rides, I lube the contact points with a dry lube, this avoids the annoying pedal squeak when the cleat rubs. Drop a drip into the spring mechanism while your here.

Every once in a while, I’ll crack open the axle, clean it off and fill the chamber with fresh grease. Reinsert the axle and snug it down driving the grease up through the bearings.

I’d used the muc off biogrease for this first time, and the pedals went slack really quick. Did them about 500 miles ago, this time using Park Tools grease. Spinning the pedals today, they are smooth, with a little resistance from the grease. Perfect.

Needless to say, that tube of biogrease has been binned never to be seen again.

I’d bought a new set of wheels, that come with cup and cone bearings, and what excuse do I need for another tube of grease to try. My existing greases would of worked fine, but wanted something that was quite thin, and tacky. The crystal grease is a little too thick, the park tools grease has a tendency to stain light coloured areas. The exus blue would of been my choice, however I’ve sorta stopped using it in favour of the crystal. So I’d ordered a tube of Rock ‘n’ Roll Super Web. The stuff Is amazing in bearings. It’s quite a thin grease but hellishly sticky and coats everything. It’s ideal in the hubs and loose bearings. Being a nice bright white colour, it’s easy too see where it’s applied too.

I’ve also had to replace the rear brake caliper. I’d disconnected the cable to space the pads a little further out, and when I removed the cable, the right hand arm had a massive amount of play. So I’ve ordered a new R8000 ultegra to replace it. I’ll also redo the front caliper later in the month.

Rock ‘n’ Roll

Sometime ago I wrote about trying Rock ‘n’ Roll Gold lube. I did try it, and its an odd lube to use. Its advertised as a cleaner as much as a lubricant. When using it, I found it didn’t last as long as my favourite dry Lube, Squirt.

You literally douse the chain at the cassette with this stuff, you don’t skimp with it, really soak the chain. Once you get over the strong petroleum smell, you run the chain backwards, while wiping. The solvents in the lube penetrate the links in the chain, causing any muck to raise up and get wiped off.

Keep doing this until you have a clean chain. Then reapply to get the lubricant in. Its a long winded job. The instructions do say it will remove other lubes, but I would still suggest a full clean and degrease before application. No doubt it will remove lubes from the same manufacturer, however more stubborn lubes will need a full clean.

The formulation goes deep down into the chain and traps any dirt. Then, with the energy of the chain freewheeling backwards, the dirt and grit floats to the surface so you can wipe it all off, leaving a new and clean application of lube inside the chain, where it’s needed. The lube down inside the chain creates a protective membrane to seal out dirt and moisture from the moving parts of the chain. As well as holding in place the best lube on the planet, for longer chain life, super smooth shifting and pedaling.

http://www.rocklube.com/about-us.html

But, once I was happy the chain was at least lubricated I went out on a ride. To me at least, the chain was noisy, and the lube didn’t last that long. Perhaps with more applications and time it’ll prove its self a bit better.

You never know, I might give it another chance when I clean off the drive chain next time and see what happens.

Update to the Backup

A quick update to the backup script. Instead of having a set list of folders stored in the backup directories to be uploaded to S3, the script now uploads everything in the backup location that’s not specifically excluded by the exclude file. This cuts a good few lines out of the script.

A quick note on the exclude file, you must exclude both the directory, and it’s contents as S3 has no concept of folders.

So, to exclude the folder /back/folder and it’s contents your exclude file must contain /back/folder and /back/folder/*


	if [ "$1" = "-s3" ]; 
		then 
			#This code is still subject to Testing. It shouldnt require a re upload of data.
#However, this uploads the WHOLE backup to the cloud. It does not filter out folders at this time.
#Use the S3 Exclude file for this purpose.
			shopt -s dotglob
			shopt -s nullglob
			array=($BCK_DEST/Backup/*)
			for dir in "${array[@]}"
				do
					echo $(basename "${dir%.*}")
					echo Uploading $dir
					FOL=$(basename "${dir%.*}")
					s3cmd sync $S3_CMD $dir $S3_BUCKET/$FOL/
					echo .......
				if ! [ "$?" = "0" ];
					then
					function_error "S3CMD" $? "Check S3CMD Command Line, and Dirs"
				fi
				done

	fi

See.Sense Ace

So I’ve had the New Ace lights for awhile now, and have been using them on both day and night rides.

They are designed for commuters, their power output is low, however it’s noticeable, not quite as retina searing bright as the Icon+, but they get the job done.

Ace Rear

But, there are issues. First off was battery life. Initially, it sucked. They would run out of juice just sitting there on the desk, meaning a charge every day. A firmware update went a long way to curing this, unless you are an Android user. The companion app took months to arrive from see.sense, they blamed the exit of their developer, and farmed it out to an external company. A release was made, but it lacked almost all the features, and was pretty much only able to turn the lights on an off. So if you wanted to update the firmware, you needed an iOS device, or at the very least a mate with an iOS device..

Now, in the latest Kickstarter news letter, they’ve stated an in-house android developer will soon be starting, so hopefully we shall have a passable android app soon(ish).

Other than the poor development and battery life issues, they really are quite a good set of lights for commuting with.

The mounting options for the lights are first class, you get a multitude of mounts as standard: including a seat-post mount, aero seat-post mount, and bag clip, which I use to mount to the back of my jacket as I’ve two rears, so one on the seat post, and one on my back. There have been reports of lights falling out of the cradle, but I assume this is due to not being clipped in correctly. My lights seem to be in solid when clipped in correctly. See.Sense, to my horror lobbed a poor innocent bike off a roof, and the lights stayed on, that was the icon2, however it uses the same mounting system.

The ACE lights feature the same intelligence of the previous lights, with a little more processing power. They can track changes in movement, g-force, and ambient lighting; tailoring the light output to the conditions and surroundings: for example, on a quiet road the light lets out just a gentle pulse, but if a car’s headlights are detected then the light lets out a sudden burst to increase awareness.

The reactivity both noticeably improves road presence, compared to a regular mid-power flasher. My only continued niggle with the reactive element of the See.Sense lights is that the sudden bursts of light from the front light can be a little distracting. This is because the light is designed to have good side visibility, which unfortunately includes the upper side too – upwards towards your eyes, should you have it mounted horizontal on the bars. To be fair, my favourite daytime lights also have this issue, the Exposure Trace. A sliver of electrical tape however remedies this.

The Smartphone integration allows you to tailor the output of the light to further improve battery life. It also allows you to use the movement sensor as a theft alert. Both useful features, if they ever get the android app to release.