Rackspace Cloud Files vs. Amazon S3 vs. Other

Portal Home > Knowledgebase > Articles Database > Rackspace Cloud Files vs. Amazon S3 vs. Other

Posted by satoo, 07-02-2011, 05:36 PM
We need to store about 7 million small thumbnail images, ~14kb each. An unknown percentage of them will change daily, but I'm guessing about 5%. Rackspace Cloud Files only charges for total storage and total bandwidth and does not charge you for the request. Amazon S3 charges $0.01 per 1000 PUT/COPY/POST/LIST requests and $0.01 per 10,000 GET requests. Seems to me for what we're doing, Rackspace Cloud Files would be a much better deal. Is there any benefit to using S3 at all? And any other companies we should check out who give you the request for free, but only charge for storage/bandwidth?
Posted by tchen, 07-02-2011, 07:11 PM
Have you considered using a replicated file store like CouchDB or Cassandra? If you're paranoid and want another layer like S3/CloudFiles, you can continue to use it in a diff/digest backup mode to avoid the request overhead.
Posted by satoo, 07-03-2011, 04:26 PM
Well the other issue is how do I upload 7 million thumbnails...that could take days if I have to make an HTTP request for each one even in parallel, and a % of this content changes per day. I see neither S3 or Cloud Files support uploading an archive and extracting. Unless anyone knows a cloud storage solution that allows zipped uploads, I will probably have to use a database of some sort like CouchDB. I'd create the database from the files and then import that.
Posted by tchen, 07-03-2011, 05:16 PM
If you were using CouchDB, you can easily create a query to get the thumbnails since last backup and create your change log digest package that way. I don't think you need to store it in raw accessible form on S3/CloudFiles. Archiving and recovery are a bit more tricky though, since once in a blue moon, you'd have that massive 7 mil thumbnail update in order to flush your change logs. Just one of the things to be aware of.

Add to Favourites Print this Article