Compile and use ABI-dependent executable binaries in Android with Android Studio 2.2 and CMake

Question

I'm testing out the new Android Studio C/C++ building via CMake through stable gradle (http://tools.android.com/tech-docs/external-c-builds).

In my app, an already rooted device needs to use an ABI-dependent binary that I compile inside Android Studio.

When I try to compile a standard library with

add_library(mylib SHARED mylib.c)

it gets automatically compiled and copied inside the lib/[ABI] folder of the APK (e.g. /lib/armeabi/mylib.so) but if I compile an executable binary with:

add_executable(mybinary mybinary.cpp)

binaries are corectly generated inside the build folder:

app/build/intermediates/cmake/debug/lib/armeabi/mybinary
app/build/intermediates/cmake/debug/lib/x86_64/mybinary 
...

but they do not seem to be copied anywhere inside the apk.

Which is the correct way to handle this need? Is a gradle-task the way to go?

build.gradle:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 24
    buildToolsVersion "24.0.1"
    defaultConfig {
        applicationId "com.my.app"
        minSdkVersion 10
        targetSdkVersion 24
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags ""
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild{
        cmake{
            path "CMakeLists.txt"
        }
    }

    defaultConfig {
        externalNativeBuild {
            cmake {
                targets "
                arguments "-DANDROID_TOOLCHAIN=clang", "-DANDROID_PLATFORM=android-21"
                cFlags "-DTEST_C_FLAG1", "-DTEST_C_FLAG2"
                cppFlags "-DTEST_CPP_FLAG2", "-DTEST_CPP_FLAG2"
                abiFilters 'x86', 'x86_64', 'armeabi', 'armeabi-v7a'
            }
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:24.1.1'
    compile 'com.android.support:design:24.1.1'
    compile 'com.android.support:recyclerview-v7:24.1.1'
    compile 'eu.chainfire:libsuperuser:1.0.0.201607041850'
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.4.1)

set(CMAKE_VERBOSE_MAKEFILE on)

add_executable(mybinary ${CMAKE_CURRENT_SOURCE_DIR}/mybinary.cpp)
target_link_libraries( mybinary libcustom)
target_include_directories (mybinary PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

mybinary.cpp

#include <stdlib.h>
#include <string>
#include <iostream>

using namespace std;

int main(int argc, char *argv[]) {

    string hello = "Hello from C++";
    cout << "Message from native code: " << hello << "\n";

    return EXIT_SUCCESS;
}

How the app should interact with mybinary:

import eu.chainfire.libsuperuser.Shell;
...
Shell.SU.run("/path/to/mybinary");

Solution 1

Ok, I've found a solution that seems to by quite comfortable but probably there are more proper ways out there;

CMakeLists.txt is by default placed inside myAppProject/app so I've added this line to CMakeLists.txt:

set(EXECUTABLE_OUTPUT_PATH      "${CMAKE_CURRENT_SOURCE_DIR}/src/main/assets/${ANDROID_ABI}")

complete app/CMakeLists.txt:

cmake_minimum_required(VERSION 3.4.1)

set(CMAKE_VERBOSE_MAKEFILE on)

# set binary output folder to Android assets folder
set(EXECUTABLE_OUTPUT_PATH      "${CMAKE_CURRENT_SOURCE_DIR}/src/main/assets/${ANDROID_ABI}")

add_subdirectory (src/main/cpp/mylib)
add_subdirectory (src/main/cpp/mybinary)

complete app/src/main/cpp/mybinary/CMakeLists.txt:

add_executable(mybinary ${CMAKE_CURRENT_SOURCE_DIR}/mybinary.cpp)
# mybinary, in this example, has mylib as dependency
target_link_libraries( mybinary mylib)
target_include_directories (mybinary PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

complete app/src/main/cpp/mylib/CMakeLists.txt:

add_library( # Sets the name of the library.
             mylib

             # Sets the library as a shared library.
             SHARED

             # Provides a relative path to your source file(s).
             # Associated headers in the same location as their source
             # file are automatically included.
             ${CMAKE_CURRENT_SOURCE_DIR}/mylib.cpp )

target_include_directories (mylib PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

Doing so, any executable binary is compiled directly into assets folder, inside a subfolder whose name is the target ABI, eg:

assets/armeabi/mybinary
assets/x86_64/mybinary
... 

In order to use the proper binary inside the App, the correct binary should be selected:

String abi;
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
    abi = Build.SUPPORTED_ABIS[0];
} else {
    //noinspection deprecation
    abi = Build.CPU_ABI;
}
String folder;
if (abi.contains("armeabi-v7a")) {
    folder = "armeabi-v7a";
} else if (abi.contains("x86_64")) {
    folder = "x86_64";
} else if (abi.contains("x86")) {
    folder = "x86";
} else if (abi.contains("armeabi")) {
    folder = "armeabi";
}
...
AssetManager assetManager = getAssets();
InputStream in = assetManager.open(folder+"/" + "mybinary");

Then, the binary should be copied away from assets folder with the correct execute permissions:

OutputStream out = context.openFileOutput("mybinary", MODE_PRIVATE);
long size = 0;
int nRead;
while ((nRead = in.read(buff)) != -1) {
    out.write(buff, 0, nRead);
    size += nRead;
}
out.flush();
Log.d(TAG, "Copy success: " +  " + size + " bytes");
File execFile = new File(context.getFilesDir()+"/mybinary");
execFile.setExecutable(true);

That's all!

UPDATE: gradle.build file:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 25
    buildToolsVersion "25"
    defaultConfig {
        applicationId "com.myapp.example"
        minSdkVersion 10
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags ""
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }

    defaultConfig {
        externalNativeBuild {
            cmake {
                targets "mylib", "mybinary"
                arguments "-DANDROID_TOOLCHAIN=clang"
                cFlags "-DTEST_C_FLAG1", "-DTEST_C_FLAG2"
                cppFlags "-DTEST_CPP_FLAG2", "-DTEST_CPP_FLAG2"
                abiFilters 'armeabi', 'armeabi-v7a', 'x86', 'x86_64'
            }
        }
    }
}

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:25.0.0'
    compile 'com.android.support:design:25.0.0'
    compile 'com.android.support:recyclerview-v7:25.0.0'
    compile 'com.android.support:cardview-v7:25.0.0'
    compile 'eu.chainfire:libsuperuser:1.0.0.201607041850'
}

Solution 2

1. Make the executable files output to where the Android Gradle plugin expects libraries:

   set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")

2. Fool the plugin into thinking your executable is a shared object:

   add_executable(i_am_an_executable.so main.c)

3. Check the APK:

   $ 7z l build/outputs/apk/app-debug.apk lib/                                                                                                                                    [2:08:56]
      Date      Time    Attr         Size   Compressed  Name
   ------------------- ----- ------------ ------------  ------------------------
                       .....         9684         4889  lib/armeabi/i_am_an_executable.so
                       .....         6048         1710  lib/arm64-v8a/i_am_an_executable.so
                       .....         9688         4692  lib/armeabi-v7a/i_am_an_executable.so
                       .....         5484         1715  lib/x86/i_am_an_executable.so
                       .....         6160         1694  lib/x86_64/i_am_an_executable.so
   

4. Access and run your executable; it is located in context.getApplicationInfo().nativeLibraryDir.

The downside to this is that you cannot set android:extractNativeLibs to false — I don't know of any way to access lib/ in an APK from within the app.

Comments

[Gerry]

that is great. why bother it this way? with shared lib, mechanism is there already just use loadlibrary() to load it and use.

[Brett Thomas]

im not getting any built files in my assets folder, just the created empty directories

[ADD]

There was a Gradle update some time ago that gave me a similar issue, if I recall correctly; I've updated the answer with the app's build.gradle file, pay particular attention to "defaultConfig" section.

[Brett Thomas]

it looks like it's not building my executable at all, I was able to get the library working. That's your entire cmakelists file?

[Brett Thomas]

Is there a reason you have one section that's empty, and then another below the cmakelists.txt part that has the actual info in it?

[ADD]

The original gradle.build (as the whole project) is a lot more complex, so I cut the unnecessary parts. Anyway I've updated the answer again to better reflect my actual CMakeLists configuration. Check out the 3 CMakeLists.txt files. However, have you explicitly set "targets" in the gradle build file?

[Brett Thomas]

yeah, many thanks for this post, helped me tremendously. man these backwards compatibility changes in android studio and lack of updated documentation suck. Thanks a bundle!

[Irvin H.]

This answer, plus the added comments are perfect! Totally helped me out :-)

[kangear]

I posted a example code: github.com/kangear/HelloJniExe/commit/…

[RoundSparrow hilltx]

@kangear - the github sample for me fails to load the actual lib .so file into the APK in Android Studio 2.3.3 - on the emulator it crashes with UnsatisfiedLinkError

[j__m]

Use getResources().getAssets().openNonAssetFd() to open your embedded binary so you can extract it yourself. Don't force Android to extract your libraries.

[Jeremy Lakeman]

Instead of building the binaries into your source folder, you could compile / copy them into mergeDebugAssets.outputDir / mergeReleaseAssets.outputDir. And you should probably add an explicit task dependency between the native compile task and the asset task.

[user924]

I also don't understand why even using any executable binaries in android projects, always use SHARED libraries!

[Ribo]

Isn't 'add_executable' to create command line programs rather than shared libraries? Why is it named 'i_am_an_executable.so'

[Robyer]

For me the EXECUTABLE_OUTPUT_PATH didn't work. With current Gradle we must use CMAKE_RUNTIME_OUTPUT_DIRECTORY instead. So use set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/../assets/${ANDROID_ABI}")