Monthly Archives: March 2007

Introducing Torrent Utility, an Apollo application

Recently, I’ve been looking into the technology and implementation behind BitTorrent, a peer-to-peer technology that has risen to extreme popularity in the last few years. Adobe’s new cross-platform runtime codenamed Apollo just came out, and I needed a project to get my feet wet. I decided to built a simple application that allows you to read the contents of a .torrent file or even create a new torrents from files directly on your hard drive to upload to your favorite tracker. It’s a fun little project that has already come in handy for a couple people.

Screenshot of Torrent Utility

Let’s start by taking a look at some of the implementation details related to the BitTorrent specification. Particularly of note is the class Bencoder. BitTorrent uses a special encoding system that you might find similar to JSON. It’s a very light-weight way to encode data, and it only a supports a few types: numbers, strings, dictionaries (objects), and lists (arrays). Consider the following Bencoded object for an idea of how it looks:

d4:name4:Josh9:highscorei12100ee

In ActionScript, you would declare the same object like this:

{name: "Josh", highscore: 12100}

The Bencoder class handles encoding and decoding ActionScript objects with this format.

When reading .torrent files, the readTorrentData function is where the magic happens. An interesting thing about this file format is that it isn’t all plain-text, as you’d expect it to be with the Bencode specification I described above. Torrent files have a section of binary data that is hashed with the SHA-1 algorithm, and this is stored in binary. If you try to read this section with ByteArray.readUTFBytes(), the function may stop early and you won’t get the correct data. So, this function reads the first string-based part of the file, then the binary section which is encoded like a string with the length at the beginning, then the final string-based portion.

private function readTorrentData(torrentStream:FileStream):void
{
	//get the first part of the file
	var fileData:String = this.readDataBeforeHashes(torrentStream);

	//read the hash pieces
	var piecesLength:int = this.readPiecesLength(torrentStream);
	var hashes:ByteArray = new ByteArray();
	torrentStream.readBytes(hashes, 0, int(piecesLength));

	//add the readable hash length to the file
	fileData += (piecesLength * 2) + ":";

	//add the readable hex hashes to the file
	fileData += ByteUtil.byteArrayToHexString(hashes);

	//add the end of the file
	fileData += torrentStream.readUTFBytes(torrentStream.bytesAvailable);

	torrentStream.close();

	this.parseTorrentData(fileData);
}

You’ll notice a call to the byteArrayToHexString function in my ByteUtil class. This function converts the raw bytes into human-readable hex strings. The class also includes a corresponding function that will convert a hex string back to ByteArray.

Grab the AIR installation file for Torrent Utility, and give it a try. If you’d like a closer look, download the MIT-licensed source code as well. Please note that you will need the latest build of Adobe’s corelib, version 0.90, to build the source. The version includes the enhancements to the SHA1 class that I wrote specifically when I needed them for this application.

You will, of course, need the alpha release of the Apollo runtime, which you can download on Adobe Labs. Please remember, Apollo is pre-release developer software that will contain bugs, may erase your hard drive, and it might even rape your dog and call your grandma offensive names. In other words, use with caution and don’t expect it to be polished.

Live from Apollo Camp

So here I am at Adobe‘s office at 601 Townsend in SF. The room is packed. Everyone has a beer, and Kevin Lynch is getting ready to speak. Twitter conversations about Apollo Camp are live and running on the big screens. Renaun and Doug are right next to me. I’m loaded up with more Adobe schwag than I can carry. I’m glad I brought my backpack. Let me tell you, this is the place to be in the Flash and Flex world tonight.

Earlier today, I was showing off the cool instant messenger client that I built with Apollo to fellow early arrivers. Don’t worry, source code and a build you can try out yourself will be available for download not long after the Apollo bits are live on Adobe Labs. I’m excited to share toys from my Apollo playground soon.

Mike Chambers says “one minute”… and we’re starting! There’s a ton of stuff here: Q&A rooms, free coffee and drinks, chill-out rooms. I see big cameras near my seat. I guess they’re recording all the sessions so everyone can see the festivities and learn about Apollo. Everyone from the Apollo, Flex framework, and Flex Builder teams are here, and we gave them a big round of applause. We’ll have a chance to chat with anyone.

Kevin Lynch is beginning the keynote to introduce Apollo, and he’s starting with the required marketing-speak. Adobe software has touched almost every product we see everyday. Flash is all over the place (PCs, mobile, video game consoles, etc). The usual stuff. Now we’re getting to the motivations for Apollo. Kevin has a graphic on-screen that shows how web applications are moving out to the desktop after 2006. Mobile MXML in the move to 2007-2008… What’s that? 🙂 He makes fun of Microsoft in a tiny corner of the visualization.

As we’ve heard before, Apollo runs SWF, HTML, and PDF. The browser is the open source WebKit. It has two basic modes. An Apollo application can run on a SWF with a web browser or PDF inside, or HTML with SWF and PDF. Features include file system access, offline mode, drag-and-drop across applications, clipboard access, apps without windows in the background, multiple windows, and custom window chrome. Let’s have a look at some examples!

Demos

Kevin is showing an Amazon Item Watcher application. It can tell you when something changes about an item for sale on Amazon changes. Something like price or availability. They actually built a Greasemonkey script to integrate with a browser. Next he shows a feed reader. It uses Flex’s runtime CSS loading for themes. Very slick appearance. I guess the UI was actually built with HTML. Interesting.

Now Kevin is showing BuzzWord by Virtual Ubiquity. It’s a full-featured word processor written in Flex for Apollo. Multiple pages, images floating in the document with text flowing around them in real time as he types. Very cool toolbars slide in and out as he needs new features. It looks like they’re context-sensitive. The table editing interface looks great and useful.

Last, he’s showing a prototype application that supports PDF. He says that we won’t have PDF support in Alpha 1, but it’ll definitely be available for 1.0. Adobe Reader must be installed for Apollo apps to be able to use it. It won’t be part of the Apollo download. I guess it will prompt the user to install Reader if they don’t have it installed. I don’t know what I think about that. It’s probably a good choice since Reader is a big download. Hopefully, they’ll make the workflow easy for users that need to install it.

Some Q&A

Will QuickTime be supported? No, plugins won’t be available at this time. Of course, Flash Video will be present.

What is different between Java Web Start and Apollo? Java doesn’t have a consistent runtime. Backwards compatibility is extremely important for Apollo, just like it is for Flash Player. Apollo apps will be installed directly to the start menu, dock, etc so that the user can find their apps again.

What will Apollo be called? Adobe loves the name, but they might have trouble getting the rights. Kevin says an alternate name is in the works, and “it’s really cool”.

How will Apollo be distributed? They’re learning from their experiences with Flash Player distribution. The Flash Player is able to install Apollo, and the process has been available for a few years already! Partnerships will be important too, but the biggest way to get Apollo to the people will be with cool apps that people want to use. Kevin is telling a funny story about how they paid millions to Netscape to get Flash Player distributed with the browser. A couple weeks later, Microsoft called up Macromedia, and they asked to bundle it for free. Haha.

Extensibility? They’re working to add more and more over time. This will be driven by feedback from developers. Native code is a “problem” for security and cross-platform support. They learned from Shockwave, and they’re wary because most people will only support Windows. They’re working to figure it out, and they’re not against native code support.

Can Apollo apps be compiled to a native executable? No, the big goal is cross-platform support so that the same bits will run on all supported operating systems.

Database support? They’re looking into it, but only file-system writing is available right now.

Now he’s done, and Mike is going to get us all started with Apollo. I’ll try to share more later. I can’t be typing all night! It’s time to have some fun.

Yahoo! Pipes is Flash-Friendly

A couple weeks ago, Yahoo! Pipes went live to much fanfare from the tech community. For those unfamiliar, Pipes offers a way to mashup, manipulate, and aggregate feeds from all over the net. The slogan in the title says, “rewire the web”, and that’s exactly what you can do with its cool visual interface. As of Saturday, Flash and Flex developers just gained a very powerful tool. Pipes now has a cross-domain policy file.

Now, a million news and blog feeds are at your fingertips. You can use Pipes with RSS or JSON output. Speaking of JSON, you know that there’s a parser already written for ActionScript 3 in the corelib open source project, right? That’s more than enough reason to try it out if you haven’t yet.

Don’t forget, if you make a cool Flash app with Pipes or any of the other Yahoo! APIs that are accessible with Flash, share it with everyone on the Yahoo! Flash Gallery. Now, go have fun.

Extra info on the new Yahoo! AS3 libraries

In case you didn’t know, my team at Yahoo! released a beta version of our Yahoo! ActionScript 3 API library on the Yahoo! Flash Developer Center this week. The announcement officially came as part of our presentations at the 360 Flex conference in San Jose. Wonderful conference, by the way. If you missed it, shame on you! Truly one of the greatest concentrations of Flex developers ever, and I can’t wait until Ted, Tom, and John organize the next one. The place was overflowing with knowledge and excitement.

As you’re exploring the new Flash-friendly APIs, be on the look out for bugs, and please consider what you’d like to see in future versions. If you’re having trouble getting a particular API working, or you’re unsure about what something in the documentation means, I encourage you to contact us. The best way to send us questions and feedback is to send messages to the ydn-flash group on Yahoo! Groups. You’ll have access to not only real Yahoo! engineers like me, but also a knowledgeable community. Remember, we’re working hard to release new code for you guys, so other experienced developers that use our libraries can help you too.

If you want to get started with our new library, be sure to look at the how-to examples that our team developed. We demoed each of them at 360 Flex, and explained much of the code that we wrote with some extra insights. I’ll do a quick post once our slides become available, in case anyone is interested.

Now get out there and build some cool Flex mashups with our cool new Yahoo! libraries!

Jump into RoR with ‘Rails Solutions’

Every time I read a tutorial or book about Ruby on Rails, I get excited. A single book about this framework can contain enough information to build a light, but surprisingly complete, application. In fact, it wouldn’t take too much effort to polish up the code and actually deploy it. I recently picked up a copy of Rails Solutions: Ruby on Rails Made Easy by Justin Williams, and like other Rails-focused material, it makes me want to have some fun and build something cool. Justin shows the reader how to build a web application similar to craigslist in RoR, and he nearly brings it to completion.

The first three chapters get you started pretty quickly. Chapter 1 introduces both Ruby and Rails with a bit of history, a summary of OOP and MVC, and some answers to a couple questions about databases, security, and popularity. The second chapter shows you how to install RoR and it’s required dependencies on both Mac and Windows. The author gives a surprising number of different options without being confusing to new users. In fact, his suggestions to use Instant Rails and Locomotive couldn’t have been better. Finally, the introduction ends with a look at Ruby the language as it’s used with Rails. It’s short and sweet, and anyone with a little programming and OOP knowledge should be able to pick it up without any trouble at all.

Chapters 4 and 5 build the foundation of working with Rails. Getting Started with Rails helps you start up the server and get working with the database. You will build your first controller and corresponding view, and the author explains some of the common scripts you’ll be using. He also shows how you can add common items to your view, like stylesheets. The next chapter, More Advanced Rails, adds to your knowledge with information on data validation, some work with migrations for database alterations, and creating associations between models. The author continues with routing, with an emphasis on clean and human-readable URLs, and finishes up this section with a look at debugging in Rails.

The next several chapters cover various interesting topics that most applications will need. Chapter 6 looks at data formatting with numbers and dates. Chapters 7 and 8 show just how easy Rails makes it to add a dash of AJAX to your app. Out of the box Rails has support for the popular AJAX libraries Prototype and script.aculo.us. The end of Chapter 8 includes a tutorial on using Action Mailer in Rails, but it’s long enough to seem out of place with some of the more obviously-AJAX material earlier in the chapter. Chapter 9 continues work with Action Mailer by adding support for file attachments, and it covers file uploads as well. These last couple of chapters feel clumsy. Good material, but I think it could be organized a bit better.

Chapter 10 shows the reader how to build a user authentication system. This includes sessions, password hashes, and checking to see if a user can perform certain actions within the application. The author points to a commonly-used user library called login_engine in a later chapter. Personally, I would recommend using a tried and tested system like that if you’re building a real-world application, but I’m assuming this chapter exists purely as an exercise in learning Rails.

Later chapters give good information, but they’re a little fluffy, if you know what I mean. Of note, Chapter 12 finally presents one of Rails’ most interesting features, scaffolding. The author explains that he doesn’t mention it in earlier chapters because he feels that the reader may become too dependent on the generated code that it builds. I think this is a novel approach for a Rails book, and I appreciate the author’s choice. The section is a little short, and I recommend reading up on scaffolding a bit more online if you’d like to learn more.

Chapter 13, Deploying with Capistrano, offers the reader a lot of very specific information about deployment. From the way the author presented it, this utility may not be widely used (I could be wrong), and it requires that you use a source control system like Subversion. The chapter came off as overly technical compared to the rest of the book, and far to specific. I would have axed it, but I’m sure many readers will find it useful when they actually use Capistrano.

This is the second Rails book that I’ve reviewed recently, and I promise that I’ll move back into Flash territory with my next review. As a Flex developer, I find many parallels between the Rails framework and Flex. If you’d like to learn more about using Flex with RoR, consider checking out WebORB, a popular implementation of AMF, the Ruby on Rails RIA SDK by Adobe, and Flex on Rails, a blog focused on using these two technologies together. This book, Rails Solutions: Ruby on Rails Made Easy by Justin Williams, and Ruby on Rails: Up and Running by Bruce A. Tate and Curt Hibbs should be good companions as well.

Catch me and Yahoo! at 360 Flex

Since the details weren’t 100% finalized until this week, I held off on mentioning that I’d be on stage at next week’s 360 Flex conference. Myself, along with the rest of the Yahoo! Flash Platform team, will be unveiling some new ActionScript 3 libraries that allow Flash and Flex developers access to our public APIs. I’ll save the details for our sessions on Monday night and Tuesday morning, but we’ve got some cool examples and mashups to show. These two sessions, by the way, will each have different content so be sure to attend both.

In addition to our API sessions, you should also attend one of the cool company get togethers on Monday night. After you grab a bite from the barbecue, head over to meetings put on by Yahoo!, Adobe, and eBay. You’ll get to hear about some cool projects being built inside each of these companies, and they’ll tell you a bit about current Flash and Flex developer openings. Get your foot in the door and have a chat with representatives from all three of these cool companies!

I mentioned it before, but now’s another good opportunity. Coming this June, you’ll have another chance to meet with a bunch of rabid Flex lovers. Flex Maniacs will be in Washington D.C. on June 25th-26th. I’ll be speaking there too. Rather than focusing on Yahoo! work, I’ll be doing a session about my cool open source TreeMap component for Flex. Specifically, I hope to cover some interesting advanced techniques with optimization, data descriptors, and other fun stuff like that.

Keep you travel calendar open this year because Flex is taking over in a big way!