Brain on Fire Tim McCormack, distilled.

Duplicity is a backup program that only backs up the files (and parts of files) that have been modified since the last backup. Built on FLOSS (rsync, GnuPG, tar, and rdiff), it allows efficient, locally encrypted, remote backups.

Amazon S3 is a web service that provides cheap, distributed, redundant, web-accessible storage. S3 currently charges only $0.15 per GB-month storage and $0.10 per GB upload. The API is based on HTTP requests such as GET, POST, PUT, and DELETE.

The following is a description of how I made use of these to back up my laptop, which runs Ubuntu Feisty Fawn.

Installation

These packages were sufficiently up-to-date in the Ubuntu repositories, so they can be installed immediately:

• python-dev: python development files
• rsync1: rsync remote-delta algorithm
• librsync-dev: rsync development files

By the time you read this, the following packages and versions may or may not have made their way into the repositories, but as of now, they have to be installed from source. Builds character.

Grab the latest .tar.gz file from each of these sources, extract it (tar -xzf whatever.tar.gz), navigate into the folder, and run python setup.py install as root or using sudo.

• Duplicity 0.4.3-rc7 or later. Currently 0.4.2 is the latest stable (but doesn't have S3 capability), and 0.4.3-rc12 is the latest unstable.
• Boto is a python backend which can talk to S3. Duplicity used to use BitBucket for this. Both are maintained by the same author.

Sign up for S3

Sign up for an Amazon Web Services account, then add the Simple Storage Service. This will involve giving them your credit card number, to be charged monthly.

Make sure to generate and write down your Secret Access Key, along with your Access ID Key. You'll need those for any application that interacts with S3.

Encryption keys

You should encrypt your files so that they are safe from prying eyes in transit and in storage. Signing them protects the files from alteration in storage or transit.

Decide on a GPG key to use for encryption and signing. (The one I use is 3BBF4E12, my main key.) Make sure your encryption/signing key is in your GPG keyring. (You can use separate keys for encryption and signing, but I haven't in this case.)

Optional: Visual S3 interface

JetS3t's Cockpit is a decent tool for viewing and managing your S3 buckets and objects. There is an applet version, but I use the standalone application instead.

Unzip the download to a permanent location, and find the bin/cockpit.sh file. Allow it to be executable (chmod u+x cockpit.sh). Edit it to set the environmental variables properly:

JETS3T_HOME=/home/myusername/programs/JetS3t
JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun/jre

When you run cockpit.sh, you will be presented with a somewhat complicated login screen. The simplest is the Direct Login tab. Local Folder login stores authentication info locally in a password-protected file.

Learn and configure

After scanning over man duplicity and playing with commands like duplicity /home/myusername/junk file:///home/myusername/junkBackup, create a shell script to run duplicity backups automatically. Here's what mine looks like:

export AWS_ACCESS_KEY_ID=myAccessKeyID
export AWS_SECRET_ACCESS_KEY=mySecretAccessKey

duplicity --encrypt-key=myEncryptionKeyFingerprint --sign-key=mySigningKeyFingerprint --exclude=/sys --exclude=/dev --exclude=/proc --exclude=/tmp --exclude=/mnt --exclude=/media --include=/mnt/media / s3+http://myUniqueBucketName

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=

You'll generally want to exclude /proc and /tmp from backups, since they contain constantly changing volatile runtime data that gets wiped every time you shutdown. /dev is full of file representations of your hardware, and /sys... I don't know, it screws up my backups, and has something to do with driver-daemon communication. I exclude /mnt and /media because I don't want to back up my external drives, and I re-include /mnt/media because that's my photos, music, and video partition. Duplicity supports all the rsync options, including fancy wildcards/globbing and the same-filesystem directive.

Notes on duplicity

• The bucket name must be unique among all S3 buckets owned by all S3 users. It does not need to exist before use.
• Duplicity supports a number of protocols: file, ftp, scp, and s3+http are the most relevant here.
• Switch the file path and the URL, and you're doing a restore.
• When using cryptographic signing, duplicity will ask you to type in your key's passphrase twice at the command line. Somewhat annoying if yours is 30-odd characters.