Add exclusion file to backup/restore

Forum: DSL Ideas and Suggestions
Topic: Add exclusion file to backup/restore
started by: hawki

Posted by hawki on May 15 2005,12:53

I would like to see the following changes made to the backup/restore script (filetool.sh). Now the script backs up files and directories listed in filetool.lst and only excludes Cache. I would like to add the ability to exclude files or directories within the listed directories. To do that I added an exclusion file (xfiletool.lst) and added Cache to it. This would allow you to add the whole home directory of dsl and any other users to filetool.lst and exclude files inside them that are large and not changing by adding them to xfiletool.lst. One such file is XUL.mfasl under the .mozilla directory. If it is not there it gets recreated at runtime. Backing up that file adds upwards of .5MB to the backup file. It sometimes gets much larger. If you have added users and are backing up their home directories as well this can be huge. There are probably other files like this that others are aware of.

******************************************************************************
The following is the filetool.sh script. I commented out the original lines and
added my change directly below.
******************************************************************************

#!/bin/bash
# Original script by Robert Shingledecker for John Andrews DSL
# © Robert Shingledecker 2003,2004,2005
# A simple script to save/restore configs, directories, etc defined by the user
# in the file filetool.lst
. /etc/init.d/dsl-functions

HOME="/home/dsl"

if [ -z $1 ] ; then
echo "Usage: filetool.sh {backup|restore}"
echo -n "Press enter to coninue:" ; read ans
exit 1
fi

DEVICE="$3"

if [ -z $DEVICE ]; then

#echo "Get the device name from /opt/.backup_device
if [ -e /opt/.backup_device ]; then
if [ -n /opt/.backup_device ]; then
DEVICE=`cat /opt/.backup_device`
fi
fi
fi

get_mountpoint $DEVICE

if [ -z "$MOUNTPOINT" ]; then
echo "Invalid device $DEVICE"
echo -n "Press enter to coninue:" ; read ans
exit 1
fi

if [ $MOUNTED == "no" ]; then
sudo mount $MOUNTPOINT
if [ "$?" != 0 ]; then
echo "Unable to mount device $DEVICE"
echo -n "Press enter to coninue:" ; read ans
exit 1
fi
fi

echo `basename "$DEVICE"` > /opt/.backup_device

if [ $1 == "backup" ] ; then
if [ -z $2 ]; then

# tar -C / -T $HOME/filetool.lst --exclude=Cache -czvf $MOUNTPOINT/backup.tar.gz
tar -C / -T $HOME/filetool.lst --exclude-from=$HOME/xfiletool.lst -czvf $MOUNTPOINT/backup.tar.gz
echo -n "Press enter to coninue:" ; read ans
else
echo -n "${BLUE}Backing up files to $MOUNTPOINT ${NORMAL}"

# tar -C / -T $HOME/filetool.lst --exclude=Cache -czf $MOUNTPOINT/backup.tar.gz 2>/dev/null &
# rotdash $!

tar -C / -T $HOME/filetool.lst --exclude-from=$HOME/xfiletool.lst -czf $MOUNTPOINT/backup.tar.gz 2>/dev/null &
rotdash $!

if [ -f /etc/sysconfig/des ]; then
KEY=$(cat /etc/sysconfig/des)
des -E -3 -k "$KEY" "$MOUNTPOINT/backup.tar.gz" "$MOUNTPOINT/backup.des"
rm -f "$MOUNTPOINT/backup.tar.gz"
fi
echo "${BLUE}Done.${NORMAL}"
fi
if [ $MOUNTED == "no" ]; then
sudo umount $MOUNTPOINT
fi
exit 0
fi

if [ $1 == "restore" ] ; then
if [ -f /etc/sysconfig/des ]; then
TARGETFILE="backup.des"
else
TARGETFILE="backup.tar.gz"
fi
if [ ! -f $MOUNTPOINT/$TARGETFILE ] ; then
if [ $MOUNTED == "no" ]; then
sudo umount $MOUNTPOINT
fi
fi
if [ -z $2 ]; then
tar -C / -zxvf $MOUNTPOINT/backup.tar.gz
echo -n "Press enter to coninue:" ; read ans
else
echo -n "${BLUE}Restoring backup files from $MOUNTPOINT ${NORMAL}"
if [ -f /etc/sysconfig/des ]; then
KEY=$(cat /etc/sysconfig/des)
des -D -3 -k "$KEY" "$MOUNTPOINT/backup.des" "$MOUNTPOINT/backup.tar.gz"
fi
tar -C / -zxf $MOUNTPOINT/backup.tar.gz 2>/dev/null &
rotdash $!
if [ -f /etc/sysconfig/des ]; then
rm -f "$MOUNTPOINT/backup.tar.gz"
fi
echo "${BLUE}Done.${NORMAL}"
fi
if [ $MOUNTED == "no" ]; then
sudo umount $MOUNTPOINT
fi
exit 0
fi
echo "I don't understand the command line parameter: $1"
echo "Usage: filetool.sh {backup|restore}"
echo -n "Press enter to coninue:" ; read ans
exit 1

****************************************************************
The following is the contents of my filetool.lst file.
I backup my added user hawki home directory also.
***************************************************************

opt/ppp
opt/bootlocal.sh
opt/powerdown.sh
opt/.mydsl_dir
home/dsl
home/hawki
etc/group
etc/passwd
etc/shadow
etc/sudoers

***************************************************************
The following is the contents of my xfiletool.lst file.
This excludes Cache and XUL.mfasl from the backup file for
both users.
***************************************************************

Cache
XUL.mfasl

Posted by mikshaw on May 15 2005,14:43

Very cool. So the only difference is adding --exclude-from?

Posted by roberts on May 15 2005,17:14

hawki, Very Nice! Thanks for sharing.

mikshaw, yes the excude and exclude-from is the only change.

A simple sed command will produce the new script.

sed 's~=Cache~-from=$HOME/xfiletool.lst~' /usr/sbin/filetool.lst > filetool.lst

Then to try out the "new" filetool.lst from /home/dsl

./filetool.lst backup

Posted by hawki on May 15 2005,23:17

Hi
Thanks for the quick response. Yes the --exclude-from= is the change. In fact it looks like -X does the same thing.
It makes a big difference on backup size in some cases.

Thanks again

Posted by pr0f3550r on Feb. 20 2006,19:41

I wonder if adding the -u (--update) argument to tar in filetool.sh would make the backup faster and wear the usb stick less