Like any system admins, we need to ensure that we have good backups. Our systems do not have tape backup devices, and it's easy to rebuild the OS, so we're just backing up our data and config files across the network via rsync tunneled through SSH.

We're also backing up locally to tarballs for easier access, and to have multiple versions of backups.

Create directories to store our backup files, and make sure they're not readable by anyone:

mkdir -p /var/backups
mkdir -p /var/backups/bud
mkdir -p /var/backups/budlight
mkdir -p /var/backups/etc
mkdir -p /var/backups/home
mkdir -p /var/backups/usr_local
mkdir -p /var/backups/email
mkdir -p /var/backups/web
mkdir -p /var/backups/mysql
chmod -R 600 /var/backups

For the procedure used for backups of michelob and dark using rsync, see rsync-aix.

Create the backup script in /etc/cron.daily/backup-etc:

#!/bin/sh
 
SRCDIR=/etc
BACKUPDIR=/var/backups/etc
DATE=`date +'%Y%m%d'`
 
tar cfz $BACKUPDIR/etc-$DATE.tgz -P $SRCDIR

Make the script executable:

chmod 755 /etc/cron.daily/backup-etc

Run the script to test that it works, and saves a file with the correct name in /var/backups/etc/. Run 'tar tfz' on the resulting backup file to make sure it contains the expected files.

Note that we only do weekly backups of home, because it's a lot bigger than the other directories that we back up.

Create the backup script in /etc/cron.weekly/backup-home:

#!/bin/sh
 
SRCDIR=/home
BACKUPDIR=/var/backups/home
DATE=`date +'%Y%m%d'`
 
tar cfz $BACKUPDIR/home-$DATE.tgz -P $SRCDIR

Make the script executable:

chmod 755 /etc/cron.weekly/backup-home

Run the script to test that it works, and saves a file with the correct name in /var/backups/home/. Run 'tar tfz' on the resulting backup file to make sure it contains the expected files.

Create the backup script in /etc/cron.daily/backup-usr_local:

#!/bin/sh
 
SRCDIR=/usr/local
BACKUPDIR=/var/backups/usr_local
DATE=`date +'%Y%m%d'`
 
tar cfz $BACKUPDIR/usr_local-$DATE.tgz -P $SRCDIR

Make the script executable:

chmod 755 /etc/cron.daily/backup-usr_local

Run the script to test that it works, and saves a file with the correct name in /var/backups/usr_local/. Run 'tar tfz' on the resulting backup file to make sure it contains the expected files.

NOTE: We currently store web sites in /home/web, so we do not need to back up anything separately at this point.

We do have a backup procedure for the wiki content. The /etc/cron.daily/backup-dokuwiki script copies a backup of the wiki data files to /var/backups/web/.

#!/bin/sh
 
DATE=`date +'%Y%m%d'`
BACKUPDIR=/var/backups/web
EXCLUDE='data/cache/*'
INCLUDE='data/ conf/'
WIKIDIR=/srv/www/wiki.sluug.org/public
 
tar cfz $BACKUPDIR/wikidata-$DATE.tgz -C $WIKIDIR --exclude="$EXCLUDE" $INCLUDE

This tars all other vhosts in /srv/www

#!/bin/sh
 
SRCDIR=/srv/www
BACKUPDIR=/var/backups/web
EXCLUDE='wiki.sluug.org/* wordpress*'
DATE=`date +'%Y%m%d'`
 
tar cfz $BACKUPDIR/www-$DATE.tgz -P $SRCDIR --exclude="$EXCLUDE"

NOTE: We currently store users' email folders in their home directories, so we do not need to back up anything separately at this point.

TODO: We need to delete tarballs that get old, so we don't keep too many around. Would be nice to keep: 1 backup per day for 1 week (or 2 weeks); 1 backup per week for a year (or 1-6 months); 1 backup per month forever.

Decided not to use mysqlhotcopy, as it does not handle InnoDB databases, which some of our apps may require. So we're using mysqldump. The –add-locks makes loading the resulting dump faster. The –allow-keywords allows fields to have the same name as SQL keywords. The –create-options includes MySQL-specific options on CREATE TABLE statements.

#!/bin/sh
BACKUPDIR=/var/backups/mysql
DATE=`date +'%Y%m%d'`
mysqldump --all-databases --add-locks --allow-keywords --create-options > $BACKUPDIR/mysql-$DATE.sql
gzip $BACKUPDIR/mysql-$DATE.sql

TODO: Add this to cron.

Copy the files in /var/backups off the server to another server.

Delete older backup files. There should be some schedule for this, so that we keep 1 backup per month or something.

Exclude some items in backup of home directories.

Back up email folders separately, if/when they're no longer held in home directories.

Take a look at mysqlsnapshot to see if it might work better for MySQL backups. Note that it has not been updated for MySQL 4.x or 5.x, but it's a Perl script, so it might still work.