OpenFL devlog: NativeWindow, Lime’s Haxe and C++ bridge, and improved cleanup

Recently, as a member of OpenFL’s leadership team, I decided that I wanted to do more to keep the community up-to-date with what new features I’m working on for the next versions of OpenFL and Lime. In today’s post, I’m going to talk a bit about the upcoming implementation of the openfl.display.NativeWindow class, which is based on a core API introduced way back in Adobe AIR 1.0. The purpose of the NativeWindow class is to open a new window (with its own stage and display list) on desktop operating systems, like Windows, macOS, and Linux.

If you’re not familiar, OpenFL is an implementation of the APIs available in Adobe Flash Player (and Adobe AIR) using the Haxe programming language. Projects built with Haxe can be cross-compiled to JavaScript, C++, and many other targets — making it available on the web, desktop apps, mobile apps, and more. No browser plugins required, and fully native C++ apps.

Implementing NativeWindow

Lime, which provides OpenFL’s integration with the native operating system, has the ability to open multiple windows through its use of SDL. SDL is a C++ library that provides a cross-platform interface for things like graphics, sound, gamepads/joysticks, and more. It’s commonly used in games, but isn’t necessarily restricted to that category of program.

Lime’s Window class was actually already exposed in OpenFL, including creating a new stage for each window. However, one of my goals as a contributor is to ensure that developers adopting OpenFL can easily migrate from Flash or AIR. So I needed to wrap Lime’s API with a new class that offers the familiar interface of AIR’s NativeWindow. In the process, I discovered that I had to expose a few new APIs that Lime didn’t provide yet. And, I was able to find some ways to optimize and improve cleanup of a window and its stage (and the entire application… more on this below).

For the most part, implementing NativeWindow was pretty straightforward because SDL provides many similar APIs to what is available in AIR, and many of these were already exposed through Lime. However, not all of these SDL APIs were fully exposed yet, and there were important differences in behavior between Lime’s Window and AIR’s NativeWindow where Lime needed some tweaks to support both behaviors. For instance, by default, Lime’s Window immediately opened on creation, but AIR keeps a window hidden until the visible property is set to true, or the activate() method is called.

Lime had actually exposed a way to hide a window by default, but there wasn’t yet a property/method to reveal it later. So I needed to dive into Lime’s C++ code and expose a new window visible property to Haxe that could be changed at runtime. Similarly, AIR allows developers to optionally set minimum and maximum dimensions of a NativeWindow, and Lime hadn’t yet exposed SDL’s functions for that feature yet.

Bridging Haxe and C++ in Lime

Adding new C++ code to Lime that can be called from Haxe involves changes in several files.

  1. The .hx class that provides the public API. In the case of the new visible property, that’s Window.hx.
  2. Lime APIs are usually powered by a separate “backend” class for each target. In this case, Window.hx calls into NativeWindow.hx (to be clear, that’s lime._internal.backend.NativeWindow, which is different from openfl.display.NativeWindow).
  3. C++ backend classes call methods defined in the NativeCFFI.hx class. NativeCFFI tells Haxe how to find the C++ APIs exposed by the compiled .ndll binary (or .hdll binary for HashLink). It requires separate declarations for Neko, HashLink, and hxcpp, so it’s kind of a confusing file to work with because there’s a lot of duplication.
  4. The ExternalInterface.cpp file, which exposes the public C++ API for the .ndll and .hdll binaries. This file usually doesn’t contain the concrete implementation, but instead, is more of the glue that talks to other C++ classes that do the actual work.
  5. The .h and .cpp file where the actual C++ concrete implementation exists. In this case, that’s SDLWindow.h and SDLWindow.cpp.

Adding new Haxe APIs to Lime that are implemented in C++ isn’t actually very hard, but it is (admittedly) a complicated process because there are so many places that can be affected by a single API change.

Improved disposal/cleanup

As I was implementing OpenFL’s NativeWindow class, I took some time to consider memory management. Many OpenFL apps have one window only, including mobile and JS/web apps (yes, Lime and OpenFL still have a “window” concept when building for web browsers). With that in mind, there’s often only one Stage and display list too. However, with NativeWindow, each additional window creates a new Stage. When closing a window, it’s important to clean up that stage as well, to make sure that there aren’t any memory leaks, or worse, memory leaks plus extra code running without end — even if it isn’t needed anymore. (The first thing I checked was that Event.ENTER_FRAME wasn’t still firing out of control, but thankfully, that was already working properly.)

Ideally, when an OpenFL app exits, it should close all of its windows and remove any references to itself from static variables. Doing this manually isn’t necessary on desktop and mobile because the operating system will clean everything up automatically when the app exits. However, when targeting JS/web, if you were to manually “exit” an OpenFL application (such as by calling Lime’s System.exit() method), or if you were to manually call close() on the “window”, that doesn’t necessarily mean the browser has navigated somewhere else. The app could be embedded in a page with other content, and it may be important to be able to load a new instance of the same OpenFL app again, or even replace it with a separate OpenFL app (or something entirely different), inside the same parent HTML DOM element at a later time.

The easiest improvement was setting the static properties lime.app.Application.current and openfl.desktop.NativeApplication.nativeApplication both to null on exit. It wouldn’t be possible for the application instance to be garbage collected without this change. Similarly, a number of event listeners defined by the application are added to static dispatchers that expose notifications from C++. These should be removed when the application exits too, so that they don’t prevent garbage collection.

Gamepad.onConnect.remove(__onGamepadConnect);
Joystick.onConnect.remove(__onJoystickConnect);
Touch.onCancel.remove(onTouchCancel);
Touch.onStart.remove(onTouchStart);
Touch.onMove.remove(onTouchMove);
Touch.onEnd.remove(onTouchEnd);

On the JS/HTML side, “closing” the “window” should also clean up the HTML DOM. Lime creates a <canvas> element that either renders to WebGL or falls back to software rendering. It also adds some event listeners to the canvas for mouse and touch. When the window is closed, or the application exits, the canvas should be removed from the DOM, and any listeners added to DOM elements should be removed too. Again, we need things to be garbage collected, and everything returned as close to the original state as possible.

var element = parent.element;
if (element != null)
{
	if (canvas != null)
	{
		if (element != cast canvas)
		{
			element.removeChild(canvas);
		}
		canvas = null;
	}
}

After all of these changes, Lime and OpenFL feel like much better HTML/JavaScript citizens, and it opens up some new possibilities for more complex websites where OpenFL is one small part of a larger system.

Where can I find the code?

If you want to try out the new NativeWindow feature, or the improved application/window/stage disposal, you’ll need to check out both Lime’s 8.1.0-Dev branch and OpenFL’s 9.3.0-Dev branch on Github, or download both the lime-haxelib artifact from a successful Github Actions Lime 8.1.0-Dev nightly build and the openfl-haxelib artifact from a successful Github Actions OpenFL 9.3.0-Dev nightly build. Of course, this code is not yet ready for release to Haxelib, so use at your own risk in production. There may still be some bugs.

OpenFL Devlog: Weak event listeners

Update (January 2024): Unfortunately, weak event listeners have been disabled for the C++ target, due to the way that Haxe internally implements passing methods by reference. Haxe creates a wrapper object around the method, so the wrapper object would be stored weakly, instead of the original method. Weak event listeners remain available for the JavaScript target.

About a year ago, I joined the OpenFL leadership team, in recognition of my contributions to the project. Since then, I’ve been working hard to improve things by fixing bugs and filling in missing features from the Flash API. If you’re not familiar, OpenFL is an implementation of the APIs available in Adobe Flash Player (and Adobe AIR) using the Haxe programming language. Projects built with Haxe can be cross-compiled to JavaScript, C++, and many other targets — making it available on the web, desktop apps, mobile apps, and more.

Over the last year, I’ve become more and more familiar with the OpenFL codebase — including Lime, OpenFL’s lower-level sibling that bridges the high-level Haxe code in OpenFL with low-level C++ code that is needed to integrate with the underlying operating system. It had been nearly 20 years since I last seriously used C++… and it’s been surprisingly enjoyable picking it up again. Sort of. I can do without the manual memory management (give me a garbage collector any day), but it’s exciting that I am no longer limited to working with what is already exposed at a high-level in OpenFL. If I’m feeling like getting my hands dirty, I can expose new native APIs, and hardware capabilities, that will empower other developers using OpenFL.

This is the first post of (I hope) many, where I give a technical summary of the recent features that I’ve implemented in OpenFL and Lime. Ideally, if I have time, I’d like to include some interesting low-level tidbits about each feature that you won’t find in the documentation, and would otherwise need to read OpenFL’s code to learn. In most cases, I plan to write about features that are actively in development, available only on Github at the time of writing, and not yet released to Haxelib. So be warned. The exact implementation details are subject to change, and if you want to try something out, you’ll either need to learn how to download nightly builds of OpenFL and Lime from Github Actions, or you’ll need to clone the repo and build it yourself. Without further ado, let’s get started on devlog #1.

Background on strong and weak event listeners

When you add an event listener by calling addEventListener(), like this…

dispatcher.addEventListener(type, listenerFunction);

…you create a reference from the dispatcher object to the listener function. Most references work this way: If object A is not yet eligible to be garbage collected, and it holds a reference to object B, then object B is also not yet eligible to be garbage collected. Later, if the reference from object A to object B is removed, then object B may be garbage collected (as long as there are no other references to it elsewhere). In terms of event listeners, removing the reference from the dispatcher to the listener function is done by calling removeEventListener(), like this:

dispatcher.removeEventListener(type, listenerFunction);

That type of reference is called a strong reference. There’s another type of reference, which is less common, called a weak reference. If object A is not yet eligible to be garbage collected, but it holds a weak reference to object B (and there are no other strong references to object B), then object B may be garbage collected.

An event listener may be added as a weak reference using the 5th parameter of addEventListener(), as seen in the code below:

dispatcher.addEventListener(type, listenerFunction, useCapture, priority, useWeakReference);

Weak listeners help reduce memory leak bugs by allowing objects to be garbage collected when you forget to call removeEventListener(), and the dispatcher shouldn’t be garbage collected yet. This is most common when adding event listeners to OpenFL’s stage (which is where all display objects get added if you want to render them). Stage listeners are added frequently for mouse and touch events, so ensuring that those listeners are removed is critical to writing correct OpenFL code. If the stage holds a strong reference to an object, that object probably isn’t getting garbage collected any time soon. That’s a memory leak, and if it happens enough times, it really starts to add up. When you add your listener as a weak reference, though, you can feel safe knowing that the object will eventually get garbage collected, even if you forgot the removeEventListener() call.

Implementation of weak listeners

Up until somewhat recently, it wasn’t possible to implement weak event listeners in OpenFL for most available targets, so OpenFL ignored the useWeakReference parameter, and added all references strongly. Event listeners in JS couldn’t be weak, and there simply wasn’t an API to weakly reference an arbitrary object in JS at all. For a long time, this was one place where OpenFL simply couldn’t match the capabilities of Flash.

However, between 2020 and 2021, web browser vendors started shipping the new WeakRef type for JavaScript. It provides a simple API for creating a weak reference to an object, instead of the normal strong reference. Here’s an example in JavaScript:

let ref = new WeakRef(targetObject);
let maybeTarget = ref.deref();
if (maybeTarget) {
    // object hasn't been garbage collected
} else {
    // object no longer exists
}

According to CanIUse, WeakRef has reached 92% availability. It’s ready for prime-time. However, when using WeakRef in OpenFL’s EventDispatcher implementation, I made sure to check if it’s available, and fall back to strong references, if needed.

Similarly, Haxe provides a cpp.vm.WeakRef for C++ desktop/mobile native targets that works very similarly. If you call its get() method, and the result is null, that means the object has been garbage collected.

var ref = new cpp.vm.WeakRef(targetObject);
var maybeTarget = ref.get();
if (maybeTarget != null) {
    // object hasn't been garbage collected
} else {
    // object no longer exists
}

In OpenFL’s EventDispatcher class, each call to addEventListener() results in the creation of a Listener object, which stores data about each listener that is added, such as whether the listener should be called for the capture phase, and what the listener’s priority is, which affects the order in which listeners are called. Basically, it’s a data structure that looks kind of like this:

private class Listener {
    public var callback:(Dynamic)->Void;
    public var priority:Int;
    public var useCapture:Bool;
}

In order to store a listener function as a weak reference, that callback member variable won’t work because setting it creates a strong reference. Weak listeners need to be stored differently.

private class Listener {
    public var callback:(Dynamic)->Void;
    public var priority:Int;
    public var useCapture:Bool;

    public var useWeakReference:Bool;
    #if (js && html5)
    public var weakRefCallback:WeakRef;
    #elseif cpp
    public var weakRefCallback:cpp.vm.WeakRef<(Dynamic)->Void>;
    #end
}

Similar to the useCapture field, I added a useWeakReference boolean field. Then, I added a separate weakRefCallback field to store the listener using the appropriate weak reference type for the target (JS vs C++).

Now, when dispatchEvent() is called, the dispatcher can loop through the Listener objects and quickly check whether a particular listener was added weakly or strongly. If the listener was added weakly, it can check whether the listener was garbage collected or not.

if (listener.useWeakReference) {
    var weakCallback = listener.weakRefCallback.deref();
    if (weakCallback != null) {
        weakCallback(event);
    }
} else {
    listener.callback(event);
}

Now, you might wonder, what happens to the Listener objects once the function held by a WeakRef has been garbage collected? Obviously, they will stick around in memory, which could slow down dispatchEvent() if you added a ton of weak listeners and never removed them. Avoiding memory leaks by trading for slower performance isn’t good, so EventDispatcher needs a way to clean things up from time to time so that it stays fast.

In the code above that calls the listener functions, I skipped some of the details. The Listener and WeakRef are actually cleaned up right away when EventDispatcher detects that the function has been garbage collected. The real code looks much closer to this:

var weakCallback = listener.weakRefCallback.get();
if (weakCallback != null) {
    iterator.remove(listener);
} else {
    weakCallback(event);
}

Maybe in the future, it might make sense to detect garbage collected weak listeners in other methods of EventDispatcher (not only in the dispatchEvent() method), but I feel like this is a good start that adds very little overhead.

Where can I find the code?

If you want to try this feature out, you’ll need to check out OpenFL’s 9.3.0-Dev branch on Github, or download the openfl-haxelib artifact from a successful Github Actions 9.30-Dev nightly build. Of course, this code is not yet ready for release to Haxelib, so use at your own risk in production. There may still be some bugs.

And that’s the end of OpenFL Devlog #1. Stay tuned for more posts about my work on OpenFL internals and some new features that I’ll be implementing in the near future.

Adobe Flex 2 running on OpenFL in JavaScript instead of SWF?

Around 15 years ago, Adobe introduced Flex 2.0, ActionScript 3.0, and Flash Player 9. I jump started my career as an expert in building Flex components and apps. Eventually, my experiences with Adobe’s framework led to my creation of Feathers UI — which today runs on OpenFL with the Haxe programming language. OpenFL aims to be an implementation of the Flash Player API — but it can target many platforms, from native mobile and desktop to web browsers with JavaScript (no plugins!). Over the years, I’ve also been a contributor to the Apache Royale compiler — which generates JavaScript from ActionScript 3.0 code. You are probably already seeing where this is going…

Edit: I updated to Flex 3 now

Is there some way to get a Flex application to run on OpenFL instead of Adobe Flash Player? OpenFL can target JavaScript, and Apache Royale can compile ActionScript 3.0 to JavaScript. Can both technologies work together? See for yourself: Demo: Adobe Flex 2 running on OpenFL.

Before you fall too far down the rabbit hole, let me warn you that this is all held together with bandaids, bubble gum, and duct tape.

Yes, it’s using the real ActionScript 3.0 source code included with the Flex 2 SDK, compiled to JavaScript with Apache Royale. It’s not a mockup or another GUI framework made to look like Flex. I had to make a few minor modifications to Flex classes due to slight differences between Flash Player and OpenFL, but most of the code is exactly the same as what Adobe/Macromedia wrote back in the day.

For an example of what I needed to tweak, the parent property of the DisplayObject in OpenFL is not a getter and setter (like it is in Flash Player). It’s a simple variable field instead. So when Flex tries to override the parent property to hide internal implementation details, that doesn’t work very well. I ended up modifying the code to use the internal/private _parent variable in several places to ensure that layouts worked the correct way. A more robust solution would be necessary for a real-world app (maybe a new public uiParent or flexParent property?), but this got things working quickly for my demo. Similarly, I had to tweak Flex’s overrides of some display list APIs, like addChild(), to handle some minor differences where OpenFL needed some extra hints to use only its internal APIs and not overrides. I also had to comment out some ApplicationDomain stuff that really only applies to SWF files loading child SWF files in Flash Player.

If you’re on a high DPI screen on desktop, you may notice that everything is rendering at the proper size, but it’s not blurry! This required adding some new code to Flex to smartly scale the SystemManager root display object. Basically, I just set scaleX/scaleY, and then any calculations in the framework using stage.stageWidth or stage.stageHeight needed to be tweaked to divide the width/height by by its respective scale property. I can’t stand looking at blurry Canvas/WebGL, so I decided to take some extra time to make things look good.

I did only enough work to get the Application, Button, and Alert components working, and that’s it. I’m sure that every Flex component would require at least some superficial tweaks. But I’m also impressed at how much code “just worked” and that I needed to focus on specific areas only. It really shows how well OpenFL emulates the Flash display list and vector drawing.

I had to manually bootstrap some things when the Flex app starts up, similar to the way that the Flex SDK compiler generates code for you automatically (you can see some of this generated code when you use the -keep-generated-actionscript compiler option when using the original Flex SDK compiler). For instance, I had to manually populate Flex’s StyleManager. The Royale compiler compiles CSS to native browser CSS for HTML, so that was never going to work for Flex running on OpenFL in HTML Canvas/WebGL anyway. Flex CSS is a simple subset of browser CSS, so I mainly just did a few Find/Replace commands in my editor to generate ActionScript from parts of defaults.css I copied from the Flex 2 SDK.

MXML only kind of worked. I could create a root component, set some properties, and listen to events, but adding children failed. The Royale compiler actually supports MXML with all of the bells and whistles, but the way that this gets converted to JavaScript isn’t really in the right data format for Flex to understand. With a few tweaks to the Royale compiler, and by adding some some additional parsing code to Flex, it could easily be done, though. For this demo, that wasn’t necessary.

I chose the very old Adobe Flex 2 released in 2006 instead of the newest Apache Flex 4.16 because I figured I would be most likely to find success creating this simple demo with an early version of the framework that hadn’t grown very complex yet. After completing this demo, I still believe that this was the right choice, but I also feel confident that Flex 3 or Flex 4 could also be feasible now that I have experience getting Flex 2 working.

Unfortunately, I’m not sure that I can legally share my code for this demo. Version 2 of Flex was still a commercial product (even if source code was included for customers to read). It was a later version of Flex that Adobe released as an open source project. I don’t think the open source license extends to previous versions. While I doubt Adobe legal cares about a discontinued product that eventually became open source, better safe than sorry. If you really want to see more, you should try to get in touch with me privately.

EDIT: I updated the project to use Adobe Flex 3, which was the first version that Adobe released as an open source project with the Mozilla Public License (MPL). Download the complete source code for this proof-of-concept from Github:

Github repository: openfl-adobe-flex-poc

Anyway, I just wanted to share a bit of nostalgia with you today. Back in the mid 2000s, Adobe Flex was a really great way to build rich applications for the web — an impressive precursor to today’s SPAs/PWAs (single page apps or progressive web apps) built with frameworks like React, Vue, or Angular. I owe much of my early career growth to Flex. Even if I can’t use Flex in my software development anymore, I still try to remember some of its spirit as I work on Feathers UI for OpenFL. That being said, it was ridiculously fun to use the real thing for the first time in a very long time!