I thought I’d share another update about the new features that I’ve been working on recently for the next versions of OpenFL and Lime. In today’s post, I’m going to talk a bit about expanding OpenFL’s HashLink target to support not just HashLink/JIT, as it has for a while, but also add support for HashLink/C on Windows, macOS, and Linux.

If you’re not familiar, OpenFL is an implementation of the APIs available in Adobe Flash Player (and Adobe AIR) using the Haxe programming language. Projects built with Haxe can be cross-compiled to JavaScript, C++, and many other targets — making it available on the web, desktop apps, mobile apps, game consoles, and more. No browser plugins required, and fully native C++ performance.

HL/JIT vs HL/C

HashLink is a virtual machine for the Haxe programming language. As I understand it, HashLink is considered similar to Adobe AIR in many ways. It has a bytecode format with JIT and garbage collection, which is compiled from a high-level language with a mix of object-oriented and functional programming paradigms. Its APIs are designed to be cross-platform — including window management, graphics, and audio. And it can be extended by loading native code libraries and exposing them to higher-level code.

One interesting thing about HashLink is that it can compile to two different formats. As mentioned above, HashLink supports its own bytecode format, which makes compilation fast and runtime performance relatively decent for day-to-day development. However, it can also cross-compile to low-level C code, which you can compile into fully native machine code with your favorite C compiler on a variety of operating systems (there are HashLink games running on desktop, mobile phones/tablets, and even game consoles). The HL/C workflow may take a bit longer for each individual build, but runtime performance is even better than with HL/JIT. Best of all, your Haxe code will behave exactly the same either way, whether compiled to bytecode or to C.

As of Lime 8.1 and OpenFL 9.3, the HashLink target supports only HL/JIT. As I mentioned, the HL/JIT target performs relatively well and offers a nice development workflow. It’s perfectly fine to release apps using HL/JIT, if it meets your needs, so that made HL/JIT good enough for previous OpenFL contributors to implement as a starting point. After a bit of research, I figured it wouldn’t be too difficult to support HL/C as well, so I started working on it for the upcoming Lime 8.2 release.

Step 1: Haxe compilation for HL/C

From the Haxe side, switching between HL/JIT and HL/C is a minor change. The example below compiles to bytecode for HL/JIT:

haxe -hl bin/MyApp.hl -main MyApp

All you need to do is change the output file extension from .hl to .c when calling the Haxe compiler:

haxe -hl bin/MyApp.c -main MyApp

I added a new lime build hlc command to the Lime tools (or lime build hl -hlc, if you prefer), and that makes Lime generate HL/C code instead of HL/JIT bytecode.

Step 2: C compilation for HL/C

However, an extra step is required after that. Once you have generated some C code, it needs to be passed to a C compiler. Since macOS is my current primary platform, I decided to start there and see how easily I could compile HL/C with the gcc compiler included with Apple’s Xcode.

The Haxe and HashLink documentation both provide pretty good starting points for how to compile the C code. I ended up with something similar to the following command (some file system paths have been simplified a bit, to keep the full command from being overly long):

gcc -O3 -o bin/MyApp -std=c11 -I obj obj/MyApp.c lib/libhl.dylib lib/fmt.hdll lib/mysql.hdll lib/sqlite.hdll lib/ssl.hdll lib/ui.hdll lib/uv.hdll lib/lime.hdll

Basically, you run gcc with a high level of optimization, using the 2011 version of the C language, and including all of the C code and headers generated by Haxe. Additionally, the executable should load libhl.dylib (the core HashLink dynamic library), and a number of .hdll files (other dynamic libraries for HashLink).

On macOS, executables are usually wrapped in .app bundles that macOS treats as a single file, even though it’s more like a directory containing many files. Lime already generated an .app bundle for HashLink/JIT, and that required a bit of finessing to work properly with HashLink/C. The executable generated by gcc was having some trouble finding libhl.dylib and the .hdll files when they were all bundled together inside the .app bundle. I ended up using install_name_tool to force the executable to find the dynamic libraries in the same directory as itself using the @executable_name directory value in the paths. The command looks something like this:

install_name_tool -change lime.hdll @executable_path/lime.hdll bin/MyApp.app/Contents/MacOS/MyApp

Figuring out exactly which install_name_tool sub-command to use (and determining even if that was the correct approach) took a bit of trial and error. To add to the complexity, I was seeing slightly different behavior between the version of HashLink that Lime bundles, and a nightly build of HashLink that I downloaded from the source. Even though Lime bundles HashLink, we also want to support new updates or custom builds, whenever possible. In the end, I got both the bundled and custom versions working, and I was ready to jump to other operating systems.

I thought Linux would be super easy after macOS, since it would also use gcc and both operating systems are Unix-ish. However, I quickly ran into similar issues where the executable couldn’t find the dynamic libraries. After a few hours of banging my head on the desk, I decided to take a break and try Windows instead, and I’d revisit Linux later. Sometimes, when I get stuck, allowing my brain do some background processing for a few hours (or days) can yield surprising successes.

Compiling HL/C on Windows had its own issues, of course. I initially settled on requiring gcc on Windows too, even if it’s not the default C compiler for most Windows developers. I tried to support Visual Studio’s C compiler, but it was failing due to a Haxe compiler issue that I needed to report (more on that in a second).

Anyway, it’s relatively easy to get gcc for Windows thanks to the MinGW-w64 project. I ended up installing it quickly in a terminal with the Chocolatey package manager.

choco install mingw

You could probably also get it with the Scoop package manager, or download the MinGW executables/installer directly from the source.

The gcc command for Windows was very similar. As far as differences go, I mainly needed to set the “windows” subsystem instead of the default “console” subsystem so that an extra terminal didn’t open when double-clicking the .exe file, and I needed to add dbghelp.dll as a dependency, for some reason (if you understand why, I’d be curious for an explanation or workaround). Thankfully, I didn’t need any extra commands similar to install_name_tool on Windows to get it to find the .hdll files.

gcc -O3 -o bin/MyApp -std=c11 -Wl,-subsystem,windows -I obj obj/MyApp.c C:/Windows/System32/dbghelp.dll lib/libhl.dll lib/fmt.hdll lib/mysql.hdll lib/sqlite.hdll lib/ssl.hdll lib/ui.hdll lib/uv.hdll lib/lime.hdll

I was actually able to get a cl.exe (Visual Studio’s C compiler) command working locally, if I did a find/replace for _restrict (which is generated from OpenFL’s TextField.restrict property) on the output C code. It seems that cl.exe treats some things differently than gcc. I submitted a bug report to the Haxe compiler (which has been partially addressed, which I’ll talk about in a second), and I declared MinGW support on Windows good enough to start with.

After I got Windows with MinGW working, I went back to Linux. One morning, I recalled reading about something called rpath, which is used to find dynamic libraries on both macOS and Linux. However, since I hadn’t used it on macOS, I didn’t think much of it when I saw it mentioned with Linux too. This time around, I decided to use rpath as my starting point for web searches, and eventually, I found some random example that mentioned an $ORIGIN value that could be passed to the linker options that worked similarly to @executable_name on macOS.

I also had to specify the libraries a bit differently using -L for the parent directory and listing the .so and .hdll files by name instead of full paths.

gcc -O3 -o bin/MyApp -std=c11 -Wl,-rpath,$ORIGIN -I obj obj/MyApp.c -L lib libhl.dylib fmt.hdll mysql.hdll sqlite.hdll ssl.hdll ui.hdll uv.hdll lime.hdll

I tried doing the same thing with rpath and -L on macOS, to make the commands more consistent, but apparently, gcc (or something else in the operating system) works differently with dynamic libraries, so the macOS command(s) had to remain different from Linux. Not a big deal, but I just wish they were more similar.

At this point, I had a few extra moments to spare, and I realized that I could probably allow HL/C code to be compiled with clang instead of gcc too. It turns out that I was right, and the same command line options worked without changes. Only the command name changed from gcc to clang. At least on macOS and Linux. I ran into some strange issues with certain Windows APIs not being recognized, for some reason, and I honestly don’t know enough about clang (or C development in general) to know how to fix that. Perhaps someone else can contribute that, and we’ll have a full range of compilers supported on all platforms.

There was one other detail that came up. On Apple Silicon CPUs for macOS, gcc and clang default to compiling for Apple Silicon, but currently, Lime’s and HashLink’s .hdll libraries are strictly for Intel CPUs. So I needed to precede the gcc command with arch command to ensure that it compiles for Intel to be compatible with the libraries.

arch -x86_64 gcc [options]

Lime and HashLink for Intel both work well on Apple Silicon with Rosetta, by the way. However, I don’t doubt that we’ll revisit full Apple Silicon support in the future.

After I submitted the bug report about certain HL/C code failing to compile with Visual Studio, the Haxe compiler team fixed the issue right away (at least, partially) and released Haxe 4.3.3 soon after. Most vanilla OpenFL apps should compile with this specific Haxe version, but there are some edge cases affecting Feathers UI that will require another small update to Haxe (or some renamed variables in Feathers UI).

With the other C compilers working on all platforms, I went back to trying to get cl.exe from Visual Studio to compile. I decided that the first step would be to get the cl.exe command working in Visual Studio’s special developer console, which automatically sets up various environment variables for you. I knew I’d eventually want to be able to build OpenFL apps from any random Windows terminal window, but I could look into that later. One step at a time.

I ended up with a command that looks something like this:

cl.exe /Ox /Fe:bin/MyApp.exe -I obj obj/MyApp.c -L lib libhl.lib fmt.lib mysql.lib sqlite.lib ssl.lib ui.lib uv.lib lime.lib /link /subsystem:windows

When an executable is built on Windows with Visual Studio, and it uses a .dll file (or .hdll, in the case of HashLink libraries), the .dll file is associated with a seperate .lib file. gcc and clang seem to be able to link directly to a HashLink .hdll file, but Visual Studio’s cl.exe wants the .lib file instead. HashLink is distributed with .hdll and .lib files for its core library. Lime wasn’t bundling those .lib files in 8.1 and older, but I made a minor tweak to the Lime build process to copy those next to the .hdll files that we were already bundling (the .lib files were already built, but they were simply not copied because HL/JIT doesn’t need them).

With that working, the next step was to figure out how to set up Visual Studio’s build environment in a random command prompt instead of requiring OpenFL users to open the Visual Studio developer console.

First, I had to figure out where Visual Studio was installed. I learned that Visual Studio comes with an executable called vswhere.exe, that is guaranteed to be available at the following location if Visual Studio is installed, regardless of which version of Visual Studio we’re dealing with:

C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe

Running vswhere.exe with the following options should print the location of Visual Studio:

vswhere.exe -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath

Visual Studio includes a batch file, called vcvarsall.bat, which can be used to set the appropriate environment variables. Using the installation path returned by vswhere.exe, you just need to append VC\Auxiliary\Build\vcvarsall.bat to the end. It takes a few options, but the only required one is the archtecture. Lime uses the x86_64 architecture (sometimes abbreviated to x64) by default on Windows, so I can pass x64 for the architecture option.

vcvarsall.bat x64

Finally, I needed to combine everything together into a single command that Lime’s tools could run. This one was a little trickier than I expected, and it took me some trial and error, but I ended up launching a separate cmd.exe instance to make things work. The command looks something like this (cl.exe options omitted for brevity):

cmd.exe /s /c vcvarsall.bat x64 && cl.exe [options]

Whew! And that was the last C compiler that I wanted to get working! Now, users won’t need to manually run all of the commands above. They’ll just need to run this one, and Lime’s tooling will do all of the heavy lifting:

lime build hlc

Much better!

Where can I find the code?

If you want to try out the new hlc target for OpenFL, you’ll need to check out Lime’s 8.2.0-Dev branch on Github, or you can download both the lime-haxelib artifact from a successful Github Actions Lime 8.2.0-Dev nightly build. If you check out Lime’s source code from Github, you’ll need to manually rebuild the .hdll binary files and Lime’s command line tools, so you may find it easier to go with the pre-built artifacts. Of course, nightly builds are not necessarily ready for release to Haxelib, so use at your own risk in production. You may encounter some bugs, but we’d love any feedback you can give. Thanks, and happy coding!