I know there are lots of different backup tools for Linux and that many people recommend Mondo Rescue but I have never been able to make it work properly (my last attempt resulted in 8 DVD images totaling 31 GB – The source of the backup was 140GB), no doubt my own fault.
So I decided to write down how I made backup and bare-metal restore using rdiff-backup work for my own future reference; and what better place to write it down than in my own blog :)
I am using an external (USB) disk for backups.
First, the backup
Instead of having to run everything by hand each time, I quickly created a script for my backups:
#!/bin/bash
# Backup script - allan@nowhere.dk
# This version is for local backups
#
# Requirements: rdiff-backup, sfdisk
if [ $# -ne 1 ]
then
echo "Usage: $0 <dest dir>"
echo "The directory backup will be created inside <dest dir>"
exit 0
fi
# White-space separated list of directories or files to exclude. Adjust to suit local needs."
EXCLUDE="/tmp /media /proc /sys /dev"
# Time to keep files in backups - See rdiff-backup manual for more info
KEEP="5D"
# Where to put the backup
BU_DIR="$1/backup"
# Temporary file for exclude list
EXCLUDE_FILE=`mktemp /tmp/backup.XXXXXX`
# Test if backup dir exists
if [ ! -d $BU_DIR ]
then
# Create the directory - Note, not using -p for security reasons
mkdir $BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to create $BU_DIR - Please check permissions"
exit 1
fi
fi
# Create a list of all packages currently installed - Used on Debian/Ubuntu
dpkg --get-selections > /etc/dpkg.lst
# Create a copy of the partition table, assumes one disk only
sfdisk -d /dev/sda > /etc/sda.sfdisk
# Generate an exclude file for rdiff-backup
for file in $EXCLUDE
do
echo "$file/**" >> $EXCLUDE_FILE
done
# Run the backup; this will backup ALL files/directories (except does excluded) and
# does not care about filesystems.
rdiff-backup --exclude-device-files --exclude-fifos --exclude-sockets -v5 \
--exclude-globbing-filelist $EXCLUDE_FILE / $BU_DIR
# Remove increments older than $KEEP days
rdiff-backup --remove-older-than $KEEP $BU_DIR
# Cleanup
rm $EXCLUDE_FILE
# Backup script - allan@nowhere.dk
# This version is for local backups
#
# Requirements: rdiff-backup, sfdisk
if [ $# -ne 1 ]
then
echo "Usage: $0 <dest dir>"
echo "The directory backup will be created inside <dest dir>"
exit 0
fi
# White-space separated list of directories or files to exclude. Adjust to suit local needs."
EXCLUDE="/tmp /media /proc /sys /dev"
# Time to keep files in backups - See rdiff-backup manual for more info
KEEP="5D"
# Where to put the backup
BU_DIR="$1/backup"
# Temporary file for exclude list
EXCLUDE_FILE=`mktemp /tmp/backup.XXXXXX`
# Test if backup dir exists
if [ ! -d $BU_DIR ]
then
# Create the directory - Note, not using -p for security reasons
mkdir $BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to create $BU_DIR - Please check permissions"
exit 1
fi
fi
# Create a list of all packages currently installed - Used on Debian/Ubuntu
dpkg --get-selections > /etc/dpkg.lst
# Create a copy of the partition table, assumes one disk only
sfdisk -d /dev/sda > /etc/sda.sfdisk
# Generate an exclude file for rdiff-backup
for file in $EXCLUDE
do
echo "$file/**" >> $EXCLUDE_FILE
done
# Run the backup; this will backup ALL files/directories (except does excluded) and
# does not care about filesystems.
rdiff-backup --exclude-device-files --exclude-fifos --exclude-sockets -v5 \
--exclude-globbing-filelist $EXCLUDE_FILE / $BU_DIR
# Remove increments older than $KEEP days
rdiff-backup --remove-older-than $KEEP $BU_DIR
# Cleanup
rm $EXCLUDE_FILE
If you want to create the backup on another server, first setup key-based SSH login to the remote server.
Here is a short guide if you do not already have key based authentication running
- Create a new ssh key on the source server (for root):
$ sudo ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
16:b6:a3:a9:f2:f4:d3:37:3d:b1:fe:df:8a:00:03:a9 root@bender
The key's randomart image is:
+--[ DSA 1024]----+
| |
| . |
| o o |
| . o o |
| E S |
| + + . |
| . o. .. o |
| .. o. . o.+. .|
| oo .. . oooooo|
+-----------------+Note: Fingerprint and randomart will differ.
- Copy the contents of /root/.ssh/id_dsa.pub on the source machine to ~user/.ssh/authorized_keys on the destination machine. While it is possible to connect as root I recommend using a non-privileged user.
- Test the connection:
# ssh user@remote
And the backup script:
#!/bin/bash
# Backup script - allan@nowhere.dk
# This version is for remote backups.
#
# Requirements: rdiff-backup, sfdisk, ssh
if [ $# -ne 3 ]
then
echo "Usage: $0 <remote host> <remote user> <dest dir>"
echo "The directory backup will be created inside <dest dir>"
exit 0
fi
# White-space separated list of directories or files to exclude. Adjust to suit local needs."
EXCLUDE="/tmp /media /proc /sys /dev"
# Time to keep files in backups - See rdiff-backup manual for more info
KEEP="5D"
# Where to put the backup
RHOST="$1"
RUSER="$2"
SSH="ssh -l $RUSER"
BUDIR="$3/backup"
# Temporary file for exclude list
EXCLUDE_FILE=`mktemp /tmp/backup.XXXXXX`
# Test if remote host is connectable
rdiff-backup --test-server $RUSER@$RHOST::$BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to connect to $RHOST"
exit 1
fi
# Test if backup dir exists
$SSH test -d $BU_DIR
if [ $? -ne 0 ]
then
# Create the directory - Note, not using -p for security reasons
$SSH mkdir $BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to create $BU_DIR - Please check permissions"
exit 1
fi
fi
# Create a list of all packages currently installed - Used on Debian/Ubuntu
dpkg --get-selections > /etc/dpkg.lst
# Create a copy of the partition table, assumes one disk only
sfdisk -d /dev/sda > /etc/sda.sfdisk
# Generate an exclude file for rdiff-backup
for file in $EXCLUDE
do
echo "$file/**" >> $EXCLUDE_FILE
done
# Run the backup; this will backup ALL files/directories (except does excluded) and
# does not care about filesystems.
rdiff-backup --exclude-device-files --exclude-fifos --exclude-sockets -v5 \
--exclude-globbing-filelist $EXCLUDE_FILE / $RUSER@$RHOST::$BU_DIR
# Remove increments older than $KEEP days
rdiff-backup --remove-older-than $KEEP $RUSER@$RHOST::$BU_DIR
# Cleanup
rm $EXCLUDE_FILE
# Backup script - allan@nowhere.dk
# This version is for remote backups.
#
# Requirements: rdiff-backup, sfdisk, ssh
if [ $# -ne 3 ]
then
echo "Usage: $0 <remote host> <remote user> <dest dir>"
echo "The directory backup will be created inside <dest dir>"
exit 0
fi
# White-space separated list of directories or files to exclude. Adjust to suit local needs."
EXCLUDE="/tmp /media /proc /sys /dev"
# Time to keep files in backups - See rdiff-backup manual for more info
KEEP="5D"
# Where to put the backup
RHOST="$1"
RUSER="$2"
SSH="ssh -l $RUSER"
BUDIR="$3/backup"
# Temporary file for exclude list
EXCLUDE_FILE=`mktemp /tmp/backup.XXXXXX`
# Test if remote host is connectable
rdiff-backup --test-server $RUSER@$RHOST::$BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to connect to $RHOST"
exit 1
fi
# Test if backup dir exists
$SSH test -d $BU_DIR
if [ $? -ne 0 ]
then
# Create the directory - Note, not using -p for security reasons
$SSH mkdir $BU_DIR
if [ $? -ne 0 ]
then
echo "Unable to create $BU_DIR - Please check permissions"
exit 1
fi
fi
# Create a list of all packages currently installed - Used on Debian/Ubuntu
dpkg --get-selections > /etc/dpkg.lst
# Create a copy of the partition table, assumes one disk only
sfdisk -d /dev/sda > /etc/sda.sfdisk
# Generate an exclude file for rdiff-backup
for file in $EXCLUDE
do
echo "$file/**" >> $EXCLUDE_FILE
done
# Run the backup; this will backup ALL files/directories (except does excluded) and
# does not care about filesystems.
rdiff-backup --exclude-device-files --exclude-fifos --exclude-sockets -v5 \
--exclude-globbing-filelist $EXCLUDE_FILE / $RUSER@$RHOST::$BU_DIR
# Remove increments older than $KEEP days
rdiff-backup --remove-older-than $KEEP $RUSER@$RHOST::$BU_DIR
# Cleanup
rm $EXCLUDE_FILE
Both scripts need to run as root on the source machine.
Restore
First of all: Test your restore!. Nothing feels worse than a useless backup. Use a virtual machine to test your backup.
- Download a boot CD (and boot it). Some steps of the restore requires Internet access so keep you wireless details handy (if applicable)
- rdiff-backup is not included on the live CD, so we need to “install” it. Go to the System -> Administration and select Synaptics Package Manager, select Settings -> Repository and enable the universe repository
When you click Close, you will be informed that you need to click the Reload button (please do) - Close Synaptics and open a terminal (Applications -> Accessories -> Terminal) and install rdiff-backup by running
sudo apt-get install rdiff-backup
- Create a new partition table using GParted (found in System -> Administration), the size of each partition can be found in /media/usbdisk/backup/etc/sda.sfdisk – the filesystem type of each partition is found in /media/usbdisk/backup/etc/fstab
- Mount filesystems for the restore; in my case I took the filesystems listed in /media/usbdisk/backup/etc/fstab:
/dev/sda1 / ext3 relatime,errors=remount-ro 0 1
/dev/sda6 /home ext3 relatime,user_xattr 0 2This translates to:
$ sudo mount /dev/sda1 /mnt
$ sudo mkdir /mnt/home
$ sudo mount /dev/sda6 /mnt/home - We are now ready to restore:
$ sudo rdiff-backup --force -r now -v5 /media/usbdisk/backup /mnt
The force flag is needed because some of the destination directories are present at target.
- Next step is to reinstall grub into the MBR and fix fstab.
- First set up and enter a chroot:
$ mount -t proc none /mnt/proc
$ mount -o bind /dev /mnt/dev
$ mount -o bind /sys /mnt/sys
$ sudo chroot /mnt /bin/bashAll work done from this point on is done with the actual data from your old system.
- Now find the UUID of your partitions:
# blkid
/dev/sda1: UUID="ff98e4b0-2383-42f7-9812-cd05af1498ea" TYPE="ext4"
/dev/sda5: UUID="a1d38cf3-2a8a-4ca4-b1ee-5f7fcb43d51b" TYPE="swap"
/dev/sda6: UUID="c64dcaa8-2bad-4ecd-b984-ed6b2718b985" SEC_TYPE="ext2" TYPE="ext3" - Open /boot/grub/menu.lst, locate kopt= and groot= and then change the UUID value with the one corresponding to the value of your root filesystem above
- Open /etc/fstab and change the UUID values to match those above
- Install grub:
# grub-install /dev/sda
- Update the grub configuration
# update-grub
- Optional: Open /etc/udev/rules.d/70-persistent-net.rules and remove the rule for eth0 if you are moving to new hardware
- Exit the chroot
# exit
- First set up and enter a chroot:
does this sort out file permissions nicely?
rdiff-backup restores all file permissions.
I just came across this article and decided to try it, I’m now to the “Getting things running” section and realized that you don’t mention Windows 7 at all in this article (Not to surprising considering the date). So, I was wondering what modifications I need to make to your directions to make this work in Ubuntu 10.04 (x86_x64) with Windows 7.
There are two reasons I wish to use 7:
My copy is supposed to be a Single User (Student) copy and it happens to be my primary Operating System on this computer. I know Microsoft doesn’t usually do anything if you install these to ‘multiple machines’.
1.) Reason one is the legal reason, although I’d technically be installing it twice I’m still only using Win7 on one machine (and only one at a time).
2.) Since Win 7 is my Primary OS it will be much easier to keep my settings in sync if I use Win 7 in my Virtual Machine.
Speaking synchronizing settings: I was curious if I could somehow set this up to a folder and use Symbolic Links to keep certain data synced (that’s what I do with WINE but, Wine doesn’t always support the newest versions like I keep on Windows and is still extremely limited).
Lastly (before I forget): I was wondering if I could somehow use this sort of virtualization setup to work with Compressed NTFS files better (at current Mail and Torrents won’t work compressed).
My apologies, my this post is intended for another thread (One of my browser plugins loads the next page automatically and when I hit tab thinking it would go to the next box it must have taken me to the next page…)
Wow, Allan – great job!!!
I am ‘new’ to rdiff-backup. I’m not really happy with the online docs and wiki sites (for rdiff-backup) but with a little patience I know it will do ‘everything’ I want and desperately need.
Sure there are a lot of good backup apps like amanda, backuppc; but rdiff-backup is really the most powerful and ‘simple’ tool for the job.
I’m not the best script programmer – so your ‘script’ is a great start for me; and I can easily modify it to suit my own needs.
The only thing really missing from your ‘setup’ – is a bootable CD distro with the required rdiff-backup packages on-board (loaded). It’s only a want to have – not a must have; since bare-metal restores are not going to happen every day; but and it’s a big BUT – I’m a sysadmin for a LARGE server farm and ‘IF’ I had a really bad SAN failure and had to rebuild EVERYTHING; then having a bootable CD recovery disk would make things faster, less stressful etc.
So, I going to try to build a custom usb/CD and possibly a PXE-boot loader image (just in-case).
I’ll try to post/send it to you when I have something ready.
BFN,
Thanks for your blog Linux rules… to infinity and beyond!