Explore Digg’s Keywords with WordWeb

The deadline for Digg’s API contest is less than a week away. Always ready for a coding challenge, I immediately knew I’d be building something when I first heard about the contest at the Digg Millionth User Party last month. My entry analyzes the keywords from the top stories on Digg, and builds a graph that shows the relationships between those words and the stories in which they appear. For example, the keyword “apple” is often related to “mac” and “ubuntu” is often related to “linux”. Keywords that appear most often will be larger and more prominent than words that appear only once or twice. Clicking on a keyword will re-focus the graph and the user can explore deeper relationships. Try it out!

If you can’t tell, the application was built with Flex. Under the hood, I’ve used a modified version of Mark Shepard’s SpringGraph component. Modifications of note include some customization options for the SpringGraph’s appearance, such as line thickness, and I built in some events for mouse interactions with nodes. I’ll have to contact Mark to see if he’d like to use my changes. They certainly make the component more useful.

The keyword analysis engine is based on the one I built to chart top keywords in my MXNA Dashboard application. Basically, it accepts a list of strings, breaks them apart into individual words, and tracks the number of appearances for each word. You can also pass in a list of words to exclude. In this case, I chose to exclude many common words like “the”, “his”, “were” and others like that. I wish I remember where exactly I found it, but I discovered a great list of nearly 1000 that someone had compiled and offered for free use within this sort of program. With a few changes to stop excluding important words like “president” and “congress” (obviously, people talk about politics a lot), the list became an excellent filter.

Of course, source code is available! You’ll notice that it uses Cairngorm. I’m not necessarily following 100% best practices for this framework (no delegate classes, for instance), but I still like to use it for MVC purposes. This application is mostly quick-and-dirty, so commenting is pretty much non-existent. I don’t expect to touch the code again, so it didn’t seem too important. I figure someone will probably find it useful regardless. Enjoy!