Branching Out

How we handle backups

  • posted Sat Dec 05 07:57:50 PST 2009
  • by krobertson

One subject I wanted to touch on a little bit was how backups are handled within Trunks.

On our main feature list, we highlight that we perform two levels of backups: one on every commit, and another nightly backup. Both backups are to separate, offsite services.

First to touch on is the per-commit backups. These are treated as mirrors of the repositories, where they reflect the current state and do not maintain a history of changes. The main goal of these backups are for backups that are as-fresh-as-possible. You make a commit, and instantly that new data you uploaded is now stored in a second location. There is no waiting for a scheduled backup to run. We are going to convert this one to maintain history soon.

Our per-commit backups are handled by BQ Backup, a rsync-based backup solution. BQ Backup isn't necessarily a big corporate entity, but a fairly well known and established backup provider. They're mostly known among web hosts, but have been around for a couple years and are stable both hardware wise and as a business.

Next are our nightly backups. These backups are historical, where we push changes out every night, and we maintain a history of those changes for 30 days. This allows us to go back in time in the event we need to recover some older data, or something similar.

Nightly backups are hosted with Tarsnap, whose slogan is "online backups for the truly paranoid." We found that very appealing. Since our nightly backups is where we currently have our historical backups, we wanted it to be very solid, secure, and stable. Tarsnap is provides encrypted storage and transmission of data on top of Amazon S3 All data is encrypted before it leaves the system and stored encrypted for the upmost security, and it is stored on Amazon S3, where it is easily scalable and synchronized across Amazon's data centers. What more could you want?

With the combination of the two methods, I am confident our backups are both flexible and secure.

blog comments powered by Disqus