unicode support in android ndk

Question

I have a large C/C++ library that I need to use as part of an Android NDK project. This library needs to be able to intelligently process UTF8 strings (for example, conversion to lowercase/uppercase).

The library has conditional compilation to punt to an OS API to do the conversion, but there don't seem to be any Android APIs for UTF8. (mbstowcs, etc)

This thread says to use JNI methods to do it (!), which is a rather heavy-weight solution.

I was thinking about building ICU, but as it uses GNU Autotools I'm not sure I can make it work with the NDK toolchain. :/

Has anyone else confronted this problem and done something other than use JNI?

EDIT: My attempts to get ICU to compile fail at the configure step:

checking wchar.h usability... no
checking wchar.h presence... yes
configure: WARNING: wchar.h: present but cannot be compiled
configure: WARNING: wchar.h:     check for missing prerequisite headers?
configure: WARNING: wchar.h: see the Autoconf documentation
configure: WARNING: wchar.h:     section "Present But Cannot Be Compiled"
configure: WARNING: wchar.h: proceeding with the preprocessor's result
configure: WARNING: wchar.h: in the future, the compiler will take precedence
checking for wchar.h... yes
checking for library containing wcscpy... none required
checking size of wchar_t... 0
configure: error: There is wchar.h but the size of wchar_t is 0

Solution

We are using ICU in the NDK. Follow the steps in the ICU cross building instructions and you'll be OK. Basically you'll have a ICU native directory (for example Windows or Linux), a ICU Cygwin (if using such) and yet another for ICU Android (ARM). Sounds crazy, but it works!

Here are the steps for building under Cygwin. I am using the 'CrystaX' NDK r4, but it should build with the out of the box NDK as well. ICU version 4.4, but has worked with previous versions as well.

1. Patches:

   • source/configure (http://pastebin.com/kDEbYdMe)
   • source/Makefile.in (http://pastebin.com/785Lnsb5)
   • source/config/ (copy mh-linux to mh-unknown)
   • source/tools/pkgdata/pkgdata.cpp (http://pastebin.com/ZiRexjkp)

2. Build your host version of ICU (e.g. Windows) as per normal. (We'll call this $HOST_ICU)

3. Build Cygwin ICU:

   • Create a icu-cygwin directory (We'll call this $ICU_CROSS_BUILD)
   • From icu-cygwin dir, run '$HOST_ICU/source/runConfigureICU Cygwin'
   • make

4. Build NDK version of ICU:

   • Create a icu-android directory
   • From icu-android dir: '$HOST_ICU/source/configure' with appropriate command line options. --with-cross-build=$ICU_CROSS_BUILD and --host=arm-eabi is required.
   • make

I use something like this for passed in (to step #4) CPPFLAGS/CXXFLAGS/CFLAGS:

-I$NDK_ROOT/build/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib

And for LDFLAGS:

-lc -Wl,-rpath-link=$NDK_ROOT/build/platforms/android-8/arch-arm/usr/lib/ -L$NDK_ROOT/build/platforms/android-8/arch-arm/usr/lib/

And additional configure params:

--enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no --enable-tools=no --host=arm-eabi --with-data-packaging=archive

I haven't done this manually for a while, it's currently all in a custom Python based build script. If you run into any other troubles, I can probably tell you what the issue is.

Good luck!

Comments

[Steven R. Loomis]

Added ICU tag. Completely missed this question.

[tofutim]

I'm running into this same problem again with 49.1.2 compiling android on OSX. :(

[tofutim]

Running into exactly this issue.

[paleozogt]

@tofutim no, but i haven't been able to revisit it, either.

[tofutim]

Did you ever get this to work?

[paleozogt]

I edited the question to show what happens when i try to build ICU with the NDK.

[Steven R. Loomis]

interesting patches, you should think about filing a ticket with them to contribute them

[NuSkooler]

I certainly will. My real hope is that the Android devs decide to just expose a full pre-installed ICU since it's used on the devices already.

[NuSkooler]

No updates yet, but it still works even with newer versions of the NDK or ICU. I have been holding off on submitting anything to ICU yet as they have been changing their build system quite a bit to make stuff like this easier. In the near future I plan on upgrading some projects to the latest ICU and will try to implement something a bit cleaner and submit it.

[tofutim]

@NuSkooler Are the patches still necessary?

[NuSkooler]

The patches are still required in 4.6 as far as I remember. In 4.8+ is where things start to change and may not require them or need something else all together. My next update will probably be at 4.8 or 4.9 but I'm not there yet so I can't say for sure.

[tofutim]

At 4.8.1.1 right now. Looks like I need to make at least some patches so that I can have "enable-tools=no".

[tofutim]

checking for arm-eabi-gcc... no

[tofutim]

NuSkooler, any tips on getting the right compiler for Android?

[tofutim]

In the $NDK_ROOT toolchains directory, I see ar-linux-androideabi-4.4.3 - should I be using that?

[Nathan]

I tried to do this.. i downloaded icu4c-49_1_2-src.tgz gzipped tar archive for Unix and other platforms. I compiled for Linux (HOST) as per this answer make make install worked. Since I was not on windows I skipped the Cygwin Portion and I went to build NDK version of ICU but i did not get anything for host So i did ../configure --host=arm-linux After which I still fail to understand where to find the ICU.so for Android?

[abergmeier]

Seems like --enable-tools=no is no longer available with 51.1 :(

[SZH]

@abergmeier were you ever able to get it to compile with 51.1 or 51.2? I've been trying with no success: stackoverflow.com/q/18235428/560722

[abergmeier]

Sadly no - the build system is pretty messy. Might try again in the future.

[Steven R. Loomis]

@tofutim just committed this patch, finally. It will be in ICU 52. bugs.icu-project.org/trac/ticket/9211