Originally Android apps were only developed in Java, targeting the Dalvik Java Virtual Machine (JVM) and its associated environment. Compared to platforms like iOS with Objective-C, which is just C with Smalltalk uncomfortably crammed into it, an obvious problem here is that any JVM will significantly cripple performance, both due to a lack of direct hardware access and the garbage-collector that makes real-time applications such as games effectively impossible. There is also the issue that there is a lot more existing code written in languages like C and C++, with not a lot of enthusiasm among companies for porting existing codebases to Java, or the mostly Android-specific Kotlin.
The solution here was the Native Development Kit (NDK), which was introduced in 2009 and provides a sandboxed environment that native binaries can run in. The limitations here are mostly due to many standard APIs from a GNU/Linux or BSD environment not being present in Android/Linux, along with the use of the minimalistic Bionic C library and APIs that require a detour via the JVM rather than having it available via the NDK.
Despite these issues, using the NDK can still save a lot of time and allows for the sharing of mostly the same codebase between Android, desktop Linux, BSD and Windows.
NDK Versioning
When implying that use of the NDK can be worth it, I did not mean to suggest that it’s a smooth or painless experience. In fact, the overall experience is generally somewhat frustrating and you’ll run into countless Android-specific issues that cannot be debugged easily or at all with standard development tools like GDB, Valgrind, etc. Compared to something like Linux development, or the pre-Swift world of iOS development where C and C++ are directly supported, it’s quite the departure.
Installing the NDK fortunately doesn’t require that you have the SDK installed, with a dedicated download page. You can also download the command-line tools in order to get the SDK manager. Whether using the CLI tool or the full-fat SDK manager in the IDE, you get to choose from a whole range of NDK versions, which raises the question of why there’s not just a single NDK version.
The answer here is that although generally you can just pick the latest (stable) version and be fine, each update also updates the included toolchain and Android sysroot, which creates the possibility of issues with an existing codebase. You may have to experiment until you find a version that works for your particular codebase if you end up having build issues, so be sure to mark the version that last worked well. Fortunately you can have multiple NDK versions installed side by side without too much fuss.
Simply set the NDK_HOME variable in your respective OS or environment to the NDK folder of your choice and you should be set.
Doing Some Porting
Since Android features a JVM, it’s possible to create the typical native modules for a JVM application using a Java Native Interface (JNI) wrapper to do a small part natively, it’s more interesting to do things the other way around. This is also typically what happens when you take an existing desktop application and port it, with my NymphCast Server (NCS) project as a good example. This is an SDL- and FFmpeg-based application that’s fairly typical for a desktop application.
Unlike the GUI and Qt-based NymphCast Player which was briefly covered in a previous article, NCS doesn’t feature a GUI as such, but uses SDL2 to create a hardware-accelerated window in which content is rendered, which can be an OpenGL-based UI, video playback or a screensaver. This makes SDL2 the first dependency that we have to tackle as we set up the new project.
Of course, first we need to create the Android project folder with its specific layout and files. This is something that has been made increasingly more convoluted by Google, with most recently your options reduced to either use the Android Studio IDE or to assemble it by hand, with the latter option not much fun. Using an IDE for this probably saves you a lot of headaches, even if it requires breaking the ‘no IDE’ rule. Definitely blame Google for this one.
Next is tackling the SDL2 dependency, with the SDL developers fortunately providing direct support for Android. Simply get the current release ZIP file, tarball or whatever your preferred flavor is of SDL2 and put the extracted files into a new folder called SDL2inside the project’s JNI folder, creating the full path of app/jni/SDL2. Inside this folder we should now at least have the SDL2 include and src folders, along with the Android.mk file in the root. This latter file is key to actually building SDL2 during the build process, as we’ll see in a moment.
We first need to take care of the Java connection in SDL2, as the Java files we find in the extracted SDL2 release under android-project/app/src/main/java/org/libsdl\app are the glue between the Android JVM world and the native environment. Copy these files into the newly created folder at src/server/android/app/src/main/java/org/libsdl/app.
Before we call the SDL2 dependency done, there’s one last step: creating a custom Java class derived from SDLActivity, which implements the getLibraries() function. This returns an array of strings with the names of the shared libraries that should be loaded, which for NCS are SDL2 and nymphcastserver, which will load their respective .so files.
Prior to moving on, let’s address the elephant in the room of why we cannot simply use shared libraries from Linux or a project like Termux. There’s no super-complicated reason for this, as it’s mostly about Android’s native environment not supporting versioned shared libraries. This means that a file like widget.so.1.2 will not be found while widget.so without encoded versioning would be, thus severely limiting which libraries we can use in a drop-in fashion.
While there has been talk of an NDK package manager over the years, Google doesn’t seem interested in this, and community efforts seem tepid at most outside of Termux, so this is the reality we have to live with.
Sysroot Things
It’d take at least a couple of articles to fully cover the whole experience of setting up the NCS Android port, but a Cliff’s Notes version can be found in the ‘build steps’ notes which I wrote down primarily for myself and the volunteers on the project as a reference. Especially of note is how many of the dependencies are handled, with static libraries and headers generally added to the sysroot of the target NDK so that they can be used across projects.
For example, NCS relies on the PoCo (portable component) libraries – for which I had to create the Poco-build project to build it for modern Android – with the resulting static libraries being copied into the sysroot. This sysroot and its location for libraries is found for example on Windows under:
${NDK_HOME}\toolchains\llvm\prebuilt\windows-x86_64\usr\lib\
The folder layout of the NDK is incredibly labyrinthine, but if you start under the toolchains/llvm/prebuilt folder it should be fairly evident where to place things. Headers are copied as is typical once in the usr/include folder.
As can be seen in the NCS build notes, we get some static libraries from the Termux project, via its packages server. This includes FreeImage, NGHTTP2 and the header-only RapidJSON, which were the only unversioned dependencies that I could find for NCS from this source. The other dependencies are compiled into a library by placing the source with Makefile in their own folders under app/jni.
Finally, the reason for picking only static libraries for copying into the sysroot is mostly about convenience, as this way the library is merged into the final shared library that gets spit out by the build system and we don’t need to additionally include these .so files in the app/src/main/jniLibs/ for copying into the APK.
Building A Build System
Although Google has been pushing CMake on Android NDK developers, ndk-build is the more versatile and powerful choice, with projects like SDL offering the requisite Android.mk file. To trigger the build of our project from the Gradle wrapper, we need to specify the external native build in app/build.gradle as follows:
externalNativeBuild {
ndkBuild {
path 'jni/Android.mk'
}
}
This references a Makefile that just checks all subfolders for a Makefile to run, thus triggering the build of each Android.mk file of the dependencies, as well as of NCS itself. Since I didn’t want to copy the entire NCS source code into this folder, the Android.mk file is simply an adapted version of the regular NCS Makefile with only the elements that ndk-build needs included.
We can now build a debug APK from the CLI with ./gradlew assembleDebug or equivalent command, before waddling off to have a snack and a relaxing walk to hopefully return to a completed build:
Further Steps
Although the above is a pretty rough overview of the entire NDK porting process, it should hopefully provide a few useful pointers if you are considering either porting an existing C or C++ codebase to Android, or to write one from scratch. There are a lot more gotchas that are not covered in this article, but feel free to sound off in the comment section on what else might be useful to cover.
Another topic that’s not covered yet here is that of debugging and profiling. Although you can set up a debugging session – which I prefer to do via an IDE out of sheer convenience – when it comes to profiling and testing for memory and multi-threading issues, you will run into a bit of a brick wall. Although Valgrind kinda-sorta worked on Android in the distant past, you’re mostly stuck using the LLVM-based Address Sanitizer (ASan) or the newer HWASan to get you sorta what the Memcheck tool in Valgrind provides.
Unlike the Valgrind tools which require zero code modification, you need to specially compile your code with ASan support, add a special wrapper to the APK and a couple of further modifications to the project. Although I have done this for the NCS project, it was a nightmare, and didn’t really net me very useful results. It’s therefore really recommended to avoid ASan and just debug the code on Linux with Valgrind.
Currently NCS is nearly as stable as on desktop OSes, meaning that instead of it being basically bombproof it will occasionally flunk out, with an AAudio-related error on some test devices for so far completely opaque reasons. This, too, is is illustrative of the utter joy that it is to port applications to Android. As long as you can temper your expectations and have some guides to follow it’s not too terrible, but the NDK really rubs in how much Android is not ‘just another Linux distro’.