My Digital Story

Just to make live easier…

Backup from Volume_1 to Volume_1 using crontab & rsync

In my previous post, I did mention that I had my DNS-323 Maxtor 500GB HDD fail to work and got it sent for repair. In just two week, I received the HDD. The repaired HDD now has Seagate cover which “Factory Repair” mark shown. I don’t know whether it is new replacement of really factory repaired.

From my research, I discover that DNS-323 is not very efficient to handle RAID1 and there are a lot of complaints where the HDD is degraded and even fail completely. Another weakness by using RAID1 is that both of the HDD will keep on spining since both need to be sync or resync.

So, this time, I configure the two HDDs as Standard means my NAS now will have two independent disks (Volume_1 and Volume_2) where I can access from Windows XP.

I will use Volume_1 (/mnt/HD_a2) as the primary HDD and by using crontab and rsync I will sync/backup Volume_1 to Volume 2 (/mnt/HD_b2) every night.

After that, then oly I will install fun_plug/ffp and then move it to USB stick so that both the HDDs would not spin unnessarily when there is no demand.

Copy Volume_1 to Volume_2

In this strategy, my entire data in the primary Volume_1 will be copied to Volume_2 every night. To accomplish it, I will utilise two command provided by most Linux distro, crontab and rysnc.

crontab

The cron daemon is a long running process that executes commands at specific dates and times. To schedule one-time only tasks with cron, use at or batch. For commands that need to be executed repeatedly (e.g. hourly, daily or weekly), use crontab, which has the following options:

crontab filename Install filename as your crontab file.
crontab -e Edit your crontab file.
crontab -l Show your crontab file.
crontab -r Remove your crontab file.

The following diagram shows you what time options are available for crontab.

# +—————- minute (0 – 59)
# |  +————- hour (0 – 23)
# |  |  +———- day of month (1 – 31)
# |  |  |  +——- month (1 – 12)
# |  |  |  |  +—- day of week (0 – 6) (Sunday=0 or 7)
# |  |  |  |  |
*  *  *  *  *  command to be executed

rysnc

Rsync is a wonderful little utility that’s amazingly easy to set up on your machines. Rather than have a scripted FTP session, or some other form of file transfer script — rsync copies only the diffs of files that have actually changed, compressed and through ssh if you want to for security. That’s a mouthful — but what it means is:

  • Diffs – Only actual changed pieces of files are transferred, rather than the whole file. This makes updates faster, especially over slower links like modems. FTP would transfer the entire file, even if only one byte changed.
  • Compression – The tiny pieces of diffs are then compressed on the fly, further saving you file transfer time and reducing the load on the network.
  • Secure Shell – The security concious of you out there would like this, and you should all be using it. The stream from rsync is passed through the ssh protocol to encrypt your session instead of rsh, which is also an option (and required if you don’t use ssh – enable it in your /etc/inet.d and restart your inet daemon if you disabled it for security).

You may want to learn more about its feature in HERE.

The script

All you need to do is to create a script file with your prefered name e.g. crontab.sh and dump it in /ffp/start. The content of the file:

————————————————————————————————————————————
#!/bin/sh

CRONTEXT=/mnt/HD_a2/crontab.txt

#start with existing crontab
/bin/crontab -l > $CRONTEXT

#add backup entire Volume_1 (/mnt/HD_a2) to Volume_2 (/mnt/HD_b2) at 4.00am
/bin/echo “00 04 * * * /ffp/bin/rsync -av –delete /mnt/HD_a2/* /mnt/HD_b2/Vol_1” >> $CRONTEXT

#install new crontab
/bin/crontab $CRONTEXT

#clean up
/bin/rm $CRONTEXT

————————————————————————————————————————————

The script above will copy the entire content in Volume_1 to Vol_1 folder in your Volume_2 HDD. The first time when it runs the time taken could be pretty long and the subsequent one will be faster due to the incremental feature of rysnc.

Make sure to make it executable by:

chmod a+x crontab.sh

After that reboot your NAS. You may verify your newly added crontab by:

crontab -l

If you see
00 04 * * * /ffp/bin/rsync -av –delete /mnt/HD_a2/* /mnt/HD_b2/Vol_1

then everything should be working fine.

16/05/2009 - Posted by | Uncategorized | , ,

1 Comment »

  1. If your hard disk decide to die .. try using SpinRite. It should solve your problem. Then make a mirror copy of the hard disk .. then send the dying hard disk back to the manufacturer.

    Comment by KayO | 03/10/2009


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: