Note: I no longer work for Esria. Some links here may be broken because I no longer maintain the community site there. Being an open source project, though, I still have the original source code, and I’ve made a copy available for download from my own blog for those interested.
Over the last week, I’ve been working on a new Flex component, a heatmap! For those unfamiliar, a heatmap is a form of data visualization that uses color on a two-dimensional plane to compare values. Most recently, in the Flex community, heatmaps have been made famous by Universal Mind’s Spatial Key application. After seeing that go live, I knew people would be interested in an open source heatmap component for Flex, so I put one together and dropped it on Esria’s open source and community site (broken link removed).
Overall, I found the process of building this component very straightforward. There are many similarities to my treemap component, but the heatmap is a bit simpler in many regards. One thing I particularly enjoyed about building the heatmap was that it required some pretty heavy optimization. Since it could potentially display massive data sets, manipulating the data provider and extracting the required numbers needs to be fast. I spent a lot of time diving into Flex Builder’s profiler. By the way, Jun Heider’s profiler session at 360Flex has some great info that helped me understand the profiler’s numbers better. I highly recommend checking it out.
In particular, I made a few good optimizations in some functions that could potentially run hundreds of thousands to even millions of times when preparing the data (basically, sorting a huge list of items). First, I made sure to access member variables directly. Usually, I’ll just use the public getter if it’s available, but it actually became a bottleneck when Array.sort() needed make such a huge number of function calls. Next, since a sort function potentially has to access the same value over and over again, I set up Dictionary-based caches to retrieve an item’s position and weight without having to recalculate these values every time. Personally, I’ve always been a little wary of the Dictionary type, but lookups proved to be very fast. Finally, though it didn’t necessarily have the same impact as the other optimizations, I drew the heatmap using a Bitmap rather than a Graphics object.
In addition to raw code execution speed, I included a couple other features to aid in performance. First, a sampleSize property lets you use every nth item in the data provider instead of every item. This won’t provide a complete picture of the data, obviously, but it will certain speed up the analysis by using a subset. Additionally, you can specify a weightField or a weightFunction. Normally, every item has a weight value of 1 used to determine the color of a region. Differing weight values can allow for more complex data, certainly, but it can also be used to combine multiple points together to form a smaller data set. For example, if you are using a region size of 30 pixels, you can combine all the items in the same region together since the heatmap won’t need more per-pixel position data. If your raw data has an item for every pixel in that 30×30 region, you’ll reduce the number of items in that region from 900 to just 1, but still have the same overall weight for that region. That can be a huge performance increase when you consider that this sort of reduction might be happening in every region on the heatmap. Moreover, by doing this on the backend, you’ll send fewer bytes to the client. Reduced bandwidth usage is always helpful.
The source code for the Flex HeatMap component is available under an MIT license. Be sure to check out the HeatMap API documentation (broken link removed) and look at the examples to get started.
Pingback: Chuck Freedman » Archives » Esria spreading Flex components wide open
the alternative heatmap is this: Link
thousands of dot didn’t see any performance problems, and it’s gradient circle dot look much better.
lzw7512, that’s an interesting alternative. Thanks for sharing. I didn’t know about it. While the gradient circle looks a little smoother, especially in large concentrations, it’s a bit less accurate as every item has a radius (and the default is very large at 25 pixels). Still, though, a well made component, and I enjoyed taking a look at its source code and getting some ideas. I think I’ll play around with it to test the performance. 🙂
A followup to my last comment:
It looks like Spacial Key uses a radius for items too. Like I said, mine can be more accurate in some ways (actually, less so in others, now that I think about it. there are tradeoffs), but I can definitely understand the desire for a smoother view of the data. I may play around with a way to swap out the drawing algorithm to use either method when I work on the next version.
Performancewise, Michael VanDaniker’s heatmap is a little slower than mine. My testing is based on 10000 points and seeing how quickly I can resize the browser. As I mentioned in the blog post, I spent a lot of time in the profiler optimizing data parsing code. When I try throwing a variation of Michael’s drawing method in as a plugin to my heatmap, I think it will end up being faster than the way I do it right now. Can’t wait to see the results!
Great component. I am still playing with it.
Have you consider in using it for maps like Google Maps for Flash? I think it can almost be used directly, check out Link
The idea there is to use coordinates as the data. One thing would be interesting is to have a property to indicate the component to draw the pixels using the mercator projection so that it overlays correctly with small zoom levels.
Do you accept contributions?
jatorre, you could certainly overlay this on a map component like Google’s or Yahoo!’s. Check out the
horizontalFunctionandverticalFunctionproperties. You can use those to specify functions that convert from map coordinates to a position on the heatmap.Nice to see another heatmap out there. I haven’t done a side-by-side comparison of our techniques myself, but congratulations of being faster. I notice that you didn’t use the BitmapData threshold method, and I bet you’ll get a pretty significant speed up if you do. I certainly did. I’ll have to adopt your aggregation technique and see if it improves anything.
Hey Josh,
I was interested in checking out your map component but I could not download any of the files at the site, are the files still available?
Thanks!
-omar
It appears that the Esria community site is no longer serving the ZIP file. I will get it up here for download on my blog soon. Sorry for the inconvenience!
Hi Josh,
Do you have an example of your heatmap used as an overlay on Google Maps? It would be really useful.
Thanks!
-Chris
No, I don’t have an example like that, Chris. If I find some time, I’ll look into it.
Hey Josh,
I just thought that I’d ask. (Didn’t mean to sound ungrateful).
I think that your work is top-notch!!!! Thank you.
-Chris
Hi,
How i can use this component with flash?
How i can store the value for clicks in database and show the heatmap later for the same page?
Thanks,
kanu
Kanu, the component cannot be used with Flash. It is for Flex only. To store the values in a database, you’ll probably need to use a server-side language like PHP as a mediator between Flash Player and the database.
Pingback: Mastering Flex Charts | der hess
Hi Josh,
Thank you so much for this Heat Map component. I’m at a very basic level with Flex and with programming in general. I want to use your component for a project where I have data in an Excel (.csv) format, it’s a large Excel plot/matrix where each cell has a temperature number. I want to import this matrix into Flex somehow, or parse the data in Flex, so that this matrix turns into a heat map an interactive heat map, and each spot on roll over shows the temperature number.
If you have any suggestions or guide me in the right direction as to where I can start and how I can use your component to do that, I’ll really appreciate it. What methods of the heatmap class should I use? also, would you have any suggestions how I can import the Excel/.csv data into Flex and use that as data provider for the heat map?
I’ll really appreciate your help.
Thanks!
Pingback: 7.5th Floor » Blog Archive » Fast Prototyping the Long Here with the Big Now