The Ny Times open archive and Amazon Mechanical Turk

I am sure everyone has heard the news that the Ny Times has opened up their archive. However, while following a link to Custer’s Last Stand only part of the article is reproduced. To view the full article you have to read the scanned PDF version (yuck). I bet they would increase their traffic ten fold if they were able to translate all there scanned articles. Sounds like a great job for Amazon Mechanical Turk.

Amazon - This is bullshit!

 Update 12-20-2006:

Amazon kept there promise and delivers the SanDisk players Next Day DHL. It was still frustrating but Amazon did the right thing. Thanks!

 I ordered 2 of these SanDisk 1gb players for my kids for x-mas. When I ordred the product it said “in stock” sold by Amazon. “To received it by December 22nd choose free super saver shipping”.

Well guess what! It wasn’t really in-stock nor could you actually receive it by the 22nd. I called customer service to cancel my order. I was told that I couldn’t cancel the order; because it’s in progress. The only problem is that it’s been “in progress” since the 15th. That’s total bullshit!

This issue has really rubed me the wrong way. Can you tell?

Lesson learned never buy Christmas Gifts from Amazon 10 days before Christmas.

Try explaing that to a 5 and 9 year old “well children, you didn’t get your christmas present on time becuase Santa used Amazon for fufillment and they screwed up.”

The funny thing about this is that it still says on my order status page that it will be deliverd by the 22nd oyee! Amazon you have some work to do.

Amazon EC2 very cool

“Amazon EC2 allows you to set up and configure everything about your instances from your operating system up to your applications.”“provides the equivalent of a system with a 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth”

You basically create your own Amazon Virtual Machine Image and upload it configured the way you want.

I was given a beta account this moring and I planed on doing some testing this weekend. John Udell already got and instance up and running very cool. It currently only supports Linux VM see my comments on Dave’s weblog.

Update: TechCrunch Review

More S3 thoughts

If you have been watching the discussion at any one of the familiar spots about Amazon’s new S3 web service you will see a lot of talk about comparing the new S3 service with standard hosting compaines.

How S3 differs

The core of S3 is the Object database, your not just storing bits, your storing bits and meta data. It’s like exposing the underlying layer of a filesystem with a platfom agnostic API. That’s a lot differnt then putting a file up on to your web host’s hardrive.

The killer app(s)

So what is S3’s killer app? I duno for sure. But I think a lot could be done in 2 areas.

- Backup

1. Platform portability is a no-brainer.
2. You can have the user expose certain files to the public(like photos or movies) or the user can could choose whether or not to share the file via BitTorrent.
3. The cost for a standard full 250 gb drive would be $37.50 a month there would be an initial transfer fee of $50.00 all subsequent backups would be differential so you cost for the transfer would only be based off what has changed or what has been added.
4. The benefits are outstanding. You get reliable distributed storage for your entire drive for only $37.50 a month. And that’s if you actually use all 250gb of your drive. I would bet the price will probably get better.

- Bloging apps & casting

1. There are a lot of different casting applications out there today Podcasting, Photocasting etc.. Most bloging systems don’t offer much storage for media beyond images. For instance this WordPress blog only allows me to upload images. If I want to do a podcast, photocast or just upload code for people to view then I have to find other server to store it.
2. Apps like WordPress, Blogspot, TypePad and Live Journal could create a plug-ins that would allow users to use there A3 accounts. A user could then store and link to what ever files they wanted to.

I bet Dave at will be the first to integrate this into his outliner app.

Amazon S3 Test Service

I was able to run a test against Amazon’s S3 Web Service using their example code. I am totally digging this! The possibilities are endless.

So here is the basic run down as far as I can tell,

1. Register with Amazon and get your key(s).
2. Download test code, I used the Curl example code.
3. Create a bucket my first one was faulkingham so my bucket url is http://s3.amazonaws.com/faulkingham/.
4. Create an object, and assign permissions if you want it public.
5. My first object(file) was index.html my second was the aws.gif the object is different from the key but they don’t have to be. Your object could be index.html and your key could be index. So your url would be /bucket/index instead of /bucket/index.html
6. Take a look at my sample files that I have made public.
7. Next step is to try the BitTorrent.

Amazon S3 has BitTorrent support

The Tech Crunch folks published a nice review of Amazon S3 grid storage. But they totaly missed what’s so cool about it. If you read beyond the standard features about price and storage you will find out why it’s really the killer app. Take a look at the developer documentation and have a look at the last section entitled “Using BitTorrent with S3“. That’s right Amazon S3 supports BitTorrent, it will create and seed your object(file).

Here is the snippet from the docs,

There is no extra charge for use of BitTorrent with S3. Data transfer via the BitTorrent protocol is metered at the same rate as client/server delivery. To be precise, whenever a downloading BitTorrent client requests a “piece” of an object from the S3 “seeder,” charges accrue just as if an anonymous request for that piece had been made using the REST or SOAP protocol. These charges will appear on your S3 bill and usage reports in the same way. The difference is that if a lot of clients are requesting the same object simultaneously via BitTorrent, then the amount of data S3 must serve to satisfy those clients will be lower than with client/server delivery. This is because the BitTorrent clients are simultaneously uploading and downloading amongst themselves. The data transfer savings achieved from use of BitTorrent can vary widely depending on how popular your object is. Less popular objects require heavier use of the “seeder” to serve clients, and thus the difference between BitTorrent distribution costs and client/server distribution costs may be small for such objects. In particular, if only one client is ever downloading a particular object at a time, the cost of BitTorrent delivery will be the same as direct download.

Wow! And to create that torrent? No problem,

Retrieving a .torrent file for any publicly available object is easy. Simply add a “?torrent” query string parameter at the end of the REST GET request for the object. No authentication is required. Once you have a BitTorrent client installed, downloading an object using BitTorrent download may be as easy as opening this URL in your web browser.