Combine buttonMode = true and a mouse click to leak memory

Here’s a troublesome Flash Player bug I discovered when I couldn’t understand why my SWF wouldn’t unload properly from a parent SWF. Everything seemed to unload fine unless I clicked on any buttons in my game’s menus. How strange. I loaded my SWF up in the Flex Builder profiler and watched what stuck around. Sure enough, in addition to the main document class and anything referenced there, some button instances that should be gone seemed to be sticking around. The reason they were stuck in memory? I set buttonMode to true.

Sounds weird, right? At first, following my own recent advice on how to cleanup an unloading SWF, I started commenting out certain mouse events used by the buttons. My thinking was that maybe these event listeners were causing the stage to keep a reference to the button. Unfortunately, that didn’t seem to help. In frustration, I started commenting out large portions of the button class. With almost no code left to run, I got the SWF to unload properly. No instances of any class in my SWF were left this time, so it was clear that this button class was the culprit. I began slowly adding bits of functionality back in. I eventually discovered two functions that would cause the button to stay in memory, and both had one thing in common: they modified buttonMode. I re-enabled all my code in that class, except for the lines that set buttonMode, and the SWF began unloading perfectly.

That’s a pretty complex case that involves multiple SWFs, and it certainly wasn’t easy to debug. For my bug report to Adobe, I built a very simple test case that you can look at below. I simply creates two Sprite instances that are clickable, one with buttonMode set to true, and one without:

package
{
	import flash.display.Sprite;
	import flash.events.MouseEvent;
	import flash.system.System;

	public class ButtonModeBug extends Sprite
	{
		public function ButtonModeBug()
		{
			var button1:Sprite = new Sprite();
			button1.graphics.beginFill(0x009900);
			button1.graphics.drawRect(0, 0, 120, 20);
			button1.graphics.endFill();
			button1.x = 10;
			button1.y = 50;
			button1.addEventListener(MouseEvent.CLICK, regularSpriteClickHandler);
			this.addChild(button1);

			var button2:Sprite = new Sprite();
			button2.graphics.beginFill(0x990000);
			button2.graphics.drawRect(0, 0, 120, 20);
			button2.graphics.endFill();
			button2.x = 150;
			button2.y = 50;
			button2.buttonMode = true;
			button2.addEventListener(MouseEvent.CLICK, buttonModeSpriteClickHandler);
			this.addChild(button2);
		}

		private function regularSpriteClickHandler(event:MouseEvent):void
		{
			var button1:Sprite = Sprite(event.currentTarget);
			button1.removeEventListener(MouseEvent.CLICK, regularSpriteClickHandler);
			this.removeChild(button1);
			button1 = null;
			System.gc();
		}

		private function buttonModeSpriteClickHandler(event:MouseEvent):void
		{
			var button2:Sprite = Sprite(event.currentTarget);
			button2.removeEventListener(MouseEvent.CLICK, buttonModeSpriteClickHandler);
			this.removeChild(button2);
			button2 = null;
			System.gc();
		}
	}
}

Notice that when you click either button, it is removed from the display list, the event handler is removed, the variable is set to null, and then System.gc() is called to force the garbage collector to run.

To see the bug in action, you should run that class in the profiler with live memory data displayed (you may need to unfilter the flash.* classes too). When you click button1 (the green one), you can see the number of Sprite instances decrease by one. When you click button2 (the red one), the number of Sprite instances stays the same.

As a control case, comment out the line that says button2.buttonMode = true;, and run again. Both buttons will be removed from memory, and the number of Sprite instances will decrease to zero.

The description of this bug can be summarized like this:

If you set buttonMode to true on a Sprite, and that Sprite gets clicked by the user, it will not be removed from memory after all references have been removed in ActionScript.

It appears, although I cannot be certain, that Flash Player’s internal code isn’t properly releasing its own reference to the Sprite. Maybe related to the hand cursor? Whatever the case may be, I recommend being wary of buttonMode (and the related useHandCursor) until this gets fixed, especially if you have a SWFs that are loaded and unloaded inside other SWFs. I say that because your buttons will hold many more objects in memory when the SWF can’t unload properly. If you’ve encountered the same bug, please vote for it in my report, “buttonMode = true + mouse click keeps Sprite in memory” filed in the public Adobe bug system.