#tarsnap

tarsnap security podcast

@patio11:

Tarsnap is the world’s best secure online backup service.  It’s run by Colin Percival, Security Officer Emeritus at FreeBSD, a truly gifted cryptographer and programmer.  I use it extensively in my company, recommend it to clients doing Serious Business (TM) all the time, and love seeing it successful.

It’s because I am such a fan of Tarsnap and Colin that it frustrates me to death.  Colin is not a great engineer who is bad at business and thus compromising the financial rewards he could get from running his software company.  No, Colin is in fact a great engineer who is so bad at business that it actively is compromising his engineering objectives.

My neck hurts from violently agreeing with the entire article.

But there is more to this story.  I know of engineers that don't take care of their health/fitness, dress well, or manage their social lives, and it actively hurts their engineering objectives.

Exceptional explanation by Colin Percival about the outage of his backup service Tarsnap.

The service went offline due to a power outage in Amazon's EC2 US-East region. In the end of the article Colin explains the cause of the blackout:

After writing so much about Tarsnap, I'd like to take a moment to provide some wider context to this outage. The power outage which knocked Tarsnap offline was big -- judging by Amazon's statement of "7% of instances" and the number of IP addresses in the EC2 US-East region, somewhere around 50,000 instances went offline -- and as I'm sure everybody reading this is aware, also affected such "big names" as Netflix, Pinterest, Instagram, and Heroku. Most people stop at this point, but there's more to the story than that.

Tarsnap is a service offering secure online backups. Colin Percival details the costs Tarsnap would have for using Amazon DynamoDB:

For each TB of data stored, this gives me 30,000,000 blocks requiring 60,000,000 key-value pairs; these occupy 2.31 GB, but for DynamoDB pricing purposes, they count as 8.31 GB, or $8.31 per month. That’s about 2.7% of Tarsnap’s gross revenues (30 cents per GB per month); significant, but manageable. However, each of those 30,000,000 blocks need to go through log cleaning every 14 days, a process which requires a read (to check that the block hasn’t been marked as deleted) and a write (to update the map to point at the new location in S3). That’s an average rate of 25 reads and 25 writes per second, so I’d need to reserve 50 reads and 50 writes per second of DynamoDB capacity. The reads cost $0.01 per hour while the writes cost $0.05 per hour, for a total cost of $0.06 per hour — or $44 per month. That’s 14.6% of Tarsnap’s gross revenues; together with the storage cost, DynamoDB would eat up 17.3% of Tarsnap’s revenue — slightly over $0.05 from every $0.30/GB I take in.

To put it differently getting an 83.7% profit margin sounds like a good deal, but without knowing the costs of the other components (S3, EC2, data transfer) it’s difficult to conclude if this solution would remain profitable at a good margin. Anyway, an interesting aspect of this solution is that the costs of some major components of the platform (S3, DynamoDB) would scale lineary with the revenue.