Android 7 Native Crash: libc.so tgkill

Question

I'm seeing this native crash with the following stack trace.

This happens in Android 7.0 & 7.1 only. Nothing new has been added to the app, which has been in production for a few years, but with more devices being updated to Nougat this crash happens frequently now and is becoming a nuisance.

Any advice would be appreciated.

native: pc 000000000007a6c4  /system/lib64/libc.so (tgkill+8)
  native: pc 0000000000077920  /system/lib64/libc.so (pthread_kill+64)
  native: pc 000000000002538c  /system/lib64/libc.so (raise+24)
  native: pc 000000000001d24c  /system/lib64/libc.so (abort+52)
  native: pc 000000000001225c  /system/lib64/libcutils.so (__android_log_assert+224)
  native: pc 00000000000610e0  /system/lib64/libhwui.so
  native: pc 000000000003908c  /system/lib64/libhwui.so
  native: pc 000000000003609c  /system/lib64/libhwui.so
  native: pc 000000000003b4fc  /system/lib64/libhwui.so
  native: pc 000000000003c520  /system/lib64/libhwui.so
  native: pc 000000000003e694  /system/lib64/libhwui.so (_ZN7android10uirenderer12renderthread12RenderThread10threadLoopEv+152)
  native: pc 00000000000127f0  /system/lib64/libutils.so (_ZN7android6Thread11_threadLoopEPv+336)
  native: pc 00000000000a50b0  /system/lib64/libandroid_runtime.so (_ZN7android14AndroidRuntime15javaThreadShellEPv+116)
  native: pc 00000000000770f4  /system/lib64/libc.so (_ZL15__pthread_startPv+204)
  native: pc 000000000001e7d0  /system/lib64/libc.so (__start_thread+16)

Here's a list of devices that are affected:

UPDATE 7/18:

Still unable to get to the root of this, so I decided to purchase a device which had most occurrences and was reasonably priced, which turned out to be Samsung Galaxy J3 2017 version with Android 7.0. Unfortunately I am still unable to reproduce the crash.

I've also made some memory usage improvements to the app in production, but the crash is still happening.

From all the comments and my own research it seems to be related to dynamically linked NDKs, but I'm not using any and its hard to find out if any of the dependencies do.

I would like to share my dependencies, it would be great if other folks facing the same issue could call out if they are using one of the same dependencies - perhaps we can spot the culprit this way.

// App Compat
    compile 'com.android.support:support-v4:23.0.1'
    compile 'com.android.support:appcompat-v7:23.0.1'
    compile 'com.android.support:cardview-v7:23.0.1'
    compile 'com.android.support:recyclerview-v7:23.0.1'

    // Play Services
    compile 'com.google.android.gms:play-services-location:8.3.0'
    compile 'com.google.android.gms:play-services-maps:8.3.0'
    compile 'com.google.android.gms:play-services-analytics:8.3.0'
    compile 'com.google.android.gms:play-services-appindexing:8.3.0'
    compile 'com.google.android.gms:play-services-ads:8.3.0'

    // Misc Libraries
    compile 'fr.avianey.com.viewpagerindicator:library:2.4.1@aar'
    compile files('app/libs/htmlcleaner-2.7.jar')
    compile files('app/libs/protobuf-java-2.6.0.jar')
    compile files('app/libs/nineoldandroids-2.4.0.jar')

    // Fabric
    compile('com.twitter.sdk.android:twitter:1.13.0@aar') { transitive = true; }
    compile('com.crashlytics.sdk.android:crashlytics:2.5.5@aar') { transitive = true; }

For folks facing the same crash, please respond in comments if you are using any of these dependencies / versions. Maybe we can single out the problem dependency.

Solution 1

Looking at the dump you provided gives some clues:

_ZN7android10uirenderer12renderthread12RenderThread10threadLoopEv

This indicates that the error occurred in the UI thread.

libhwui.so x 6

This indicates that this is happening in the middle of some graphics/ui related code.

libcutils.so - __android_log_assert

This is an assert handler, so most likely some kind of assert was violated in libwhui.

abort:

This is the application telling the O/S to shut down "abnormally".

raise + pthread_kill + tgkill: This is the O/S (Android) shutting down the app.

You can see some documentation for debugging these kinds of crashes here.

Anyway, I am afraid it is really difficult to speculate beyond this coarse and imprecise interpretation of the data you presented.

Maybe if you caught the bug while it was attached to the Android log viewer, you would have more application specific data (or even an error message which the assert function usually puts out).

My tip is to use something like ACRA to track down all the details pertaining to the error, or get hold of an affected device and actually reproduce it while attached to a debugger.

Good luck!

EDIT 2017-06-16: I just want to add some extra info courtesy comment by Fco P. Apparently Google has decided to make some changes to what native libraries are allowed to run in latest versions of Android (7.x). More details are in this link.

Solution 2

This is reported here: https://issuetracker.google.com/issues/37123764

To reproduce: Get an affected mode, enable developer mode, and set background activities to 0. Also enable "show background crashes".

Then open the app, and close it again: You will see the crash.

Solution 3

Not in comments (insufficient rep).

Of the dependencies you have listed we use:

compile 'com.android.support:cardview-v7:25.3.1'
compile 'com.android.support:appcompat-v7:25.3.1'
compile 'com.android.support:support-v4:25.3.1'

compile 'com.google.android.gms:play-services-maps:10.2.1'
compile 'com.google.android.gms:play-services-location:10.2.1'

different versions than yours. I have strong suspection that play-services-maps contains the bug.

Perhaps you use map fragment in viewpager as we do and many people in issue already mentioned by Koji Matsubara (https://issuetracker.google.com/issues/37123764)

Solution 4

I don't know, maybe this problem like ours, maybe different, because I see in dependencies have including carview. Share here hope useful for someone in future

I also faced issue on Android 7.0 and 7.1 bellow

03-04 23:44:51.489 2173-2173/? A/DEBUG: Abort message: 'Error: Ambient Vertex Buffer overflow!!! used 420, total 284'
03-04 23:44:51.489 2173-2173/? A/DEBUG:     eax 00000000  ebx 0000083b  ecx 00000857  edx 00000006
03-04 23:44:51.489 2173-2173/? A/DEBUG:     esi d19ff978  edi d19ff920
03-04 23:44:51.489 2173-2173/? A/DEBUG:     xcs 00000023  xds 0000002b  xes 0000002b  xfs 0000006b  xss 0000002b
03-04 23:44:51.489 2173-2173/? A/DEBUG:     eip f00a6bb9  ebp d19fee68  esp d19fee0c  flags 00000292
03-04 23:44:51.555 2173-2173/? A/DEBUG: backtrace:
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #00 pc 00000bb9  [vdso:f00a6000] (__kernel_vsyscall+9)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #01 pc 0007a2ec  /system/lib/libc.so (tgkill+28)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #02 pc 00075b35  /system/lib/libc.so (pthread_kill+85)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #03 pc 0002784a  /system/lib/libc.so (raise+42)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #04 pc 0001ee26  /system/lib/libc.so (abort+86)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #05 pc 0000fa65  /system/lib/libcutils.so (__android_log_assert+245)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #06 pc 00084356  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #07 pc 0003a5ba  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #08 pc 00083d04  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #09 pc 0008c5df  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #10 pc 0008e6d8  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #11 pc 0008e5d2  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #12 pc 000350fe  /system/lib/libhwui.so
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #13 pc 0001201f  /system/lib/libutils.so (_ZN7android6Thread11_threadLoopEPv+207)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #14 pc 0006e53b  /system/lib/libandroid_runtime.so (_ZN7android14AndroidRuntime15javaThreadShellEPv+111)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #15 pc 00011873  /system/lib/libutils.so (_ZN13thread_data_t10trampolineEPKS_+259)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #16 pc 00075292  /system/lib/libc.so (_ZL15__pthread_startPv+210)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #17 pc 0002028e  /system/lib/libc.so (__start_thread+30)
03-04 23:44:51.555 2173-2173/? A/DEBUG:     #18 pc 0001e066  /system/lib/libc.so (__bionic_clone+70)

After research and search on gooogle, I replaced cardview by Framelayout then this issue was resolved

Solution 5

I had the same issue in google play console for the same devices as you.

In my case the issue was in TextureView with animation in separate thread with lock and unlock canvas.

I changed TextureView animation to the invalidate-onDraw animation for 7 and 7.1 android and that helped.

View more solutions

Comments

[WindsurferOak]

Most of the crashes I see are coming from Samsung devices, particularly, the Galaxy S6. Any luck figuring this one out?

[AlexVPerl]

There are other devices effected like Nexus & Experia, however mostly its Samsung as they have most of the market share. Adding a list of devices to the question. Still trying to figure this one out, can't repro at all...

[Koji Matsubara]

Perhaps I think your native crash is the same following issue. issuetracker.google.com/issues/37123764 My application has similar bug, but I don't find any solution... I think android 7, 7.1 bug.

[AlexVPerl]

@KojiMatsubara, thanks. I've seen that link before, their issue deals with Camera API. We're not using it anywhere in our code so it has to be something else.

[PiotrWpl]

Do you have any native libraries in your project? We also have this crash in our app. We are using only two libraries with native modules: Realm and Fresco. Do you have one of them? Maybe this is trail.

[AlexVPerl]

We're not using any native libraries directly. Is there any way to find out if any of our dependencies use native libs internally?

[WindsurferOak]

Is anyone using maps and location?

[AlexVPerl]

Yes, both maps and location are being used.

[PiotrWpl]

AlexVPerl please try to use Android Studio Apk Analyzer [developer.android.com/studio/build/apk-(analyzer.html]https‌​://…. If You use any native libraries they are in "libs" folder. Please let me know with one You have there. @ThierryLathuille I tried to keep with this rule: But I can't write a good answer without more information! _ Answers don't have to be exhaustive or infallible, they just need to try to answer the question (...) "I'm not sure what the cause of your problem is, but if it's X, you can solve it by doing_

[Orgmir]

I'm also seeing this, the exact same stack trace and the exact same list of affected devices! The latest version was published 15 of May, but I have two lines in the crashes page with the same "tgkill" name.

[AlexVPerl]

@Orgmir Thanks for sharing, could you please clarify what do you mean by 'two lines in the crashes page? Do you mean you see any of your code in stacktrace or is it all native code? Also just curious, what version of Play-Services & Support-v4 library are you using?

[AlexVPerl]

@PiotrWpl I tried Apk Analyzer but I don't see any info on dependencies - where should I look? Also, my libs folder only has jar files, from what I saw native libs have a different extension, right?

[PN10]

@AlexVPerl to me its seems like memory stomping bugs in your native code...Try using valgrind tools on android to detect native memory bugs...

[Fco P.]

Please, list your dependencies. And remember, nougat changed the way apps can access NDK code: developer.android.com/about/versions/nougat/…

[Cord Rehn]

I'm also experiencing this exact same issue, exact same stack trace, exact same devices affected, using zero native libraries, and using location and maps services. Perhaps it's tied to that? Anyone have a fix?

[FinHead]

Ditto, seeing the exact same tgkill crashes reported in Google Play. I'm running through the app on a S8+, I can't replicate the crash and I don't see see any of the logcat warnings related to the Nougat linked native libraries: developer.android.com/about/versions/nougat/….

[Thuy Trinh]

We ran into the exact same issue. We've tried upgrading to latest Play services and... pray!!!

[AlexVPerl]

@ThuyTrinh did either help ? :)

[hvaughan3]

We also reference the 4 com.android.support libraries (although we only use do not use cardview or recycleview) and the play-services-maps and play-services-location libraries.

[AlexVPerl]

@hvaughan3 could you please share versions of support and map libraries you use? Thanks.

[hvaughan3]

Actually I am not directly referencing the location library. For maps, I am using version 29.0.0.2. For support libraries, I am using version 23.3.0.

[FinHead]

Over the last 2 months, we have over 30k of these tgkill crashes effecting 14k+ users. I have spent the last couple of weeks slowly removing any 3rd party libraries we're using and releasing staged rollouts to see if I can track down what is causing these crashes. Everything is removed except for Retrofit, Okhttp, Jackson, Picasso, Firebase, Google Play Services, MultiDex, and Apache Legacy. Based on this thread, we're discussing releasing to 1% of our users, with our maps removed. Currently running: 'com.google.android.gms:play-services-maps:11.0.1'

[AlexVPerl]

@FinHead thank you for sharing this. We are also using Apache Legacy, Picasso, Retrofit & OkHttp. But we're also using google maps, which is not an option for us to remove. Please share your findings with us once you have rolled out a "map-less" version, your findings will be very valuable.

[FinHead]

We released a staged rollout with only removing 'com.google.android.gms:play-services-maps:11.0.1'. After watching it all weekend, there were no instances of the tgkill crash. Yes, this issue is caused by maps as mentioned by @Deo and linked to the issue tracker below.

[AlexVPerl]

@FinHead thank you so much for following up on this and sharing your results. Finally we have a tangible proof tying this crash to a library. This is very troubling as 11.0.1 is a pretty recent version, June 2017. Do you by any chance use it inside a ViewPager? Also do you plan to upgrade to 11.0.4 to see if it resolves this issue ?

[Thuy Trinh]

@AlexVPerl We upgraded Play services to v11.0.4 and Support Library v26. Sadly, it is still happening right now. We don't have any usage of putting SupportMapFragment or MapView inside a ViewPager at all. We've already also checked we drive the lifecycle of MapView correctly developers.google.com/android/reference/com/google/android/g‌​ms/….

[FinHead]

@AlexVPerl We have 2 maps in our phone code base, and 1 in our tablet code base. We display a GoogleMap in a MapFragment in a ViewPager on one of our main screens and I'm assuming that's what's causing the issue, but our team has not been able to replicate the crash. I have a normal release going out today/tomorrow with v11.0.2. I'm hoping to then start doing more staged rollouts where I confirm the correct map, and then trying to figure out how to fix it. We have a new redesign coming out in a couple of months and we don't want this same issue appearing if Google doesn't fix this.

[Thuy Trinh]

@FinHead Do you have any cases of using SupportMapFragment as a child fragment? We do have some but we're not sure whether it can be the cause or not. Also, if an Activity which contains SupportMapFragment or MapView leaks, will it be the cause too?

[Thuy Trinh]

@FinHead Yeah we're also facing losing users because of this issue. If Google doesn't fix this and we can't find any workaround, we'll consider Mapbox mapbox.com/android-docs/map-sdk/overview.

[Hong]

Exactly the same crash only from Galaxy devices running Android 7.0.

[Ramy Sabry]

I started to see same crash after update Crashlytics(fabric) with com.crashlytics.sdk.android:crashlytics-ndk:1.1.6@aar

[immutable]

occurring in android 6.0 too and mostly samsung galaxy series..

[Kazuki]

For those who use MapFragment, in issuetracker.google.com/issues/35826109 it's suggested that disabling setSharedElementsUseOverlay may fix the crash. Although I'm not sure it's same issue as one described here (problem in my link above seems 100% reproducible, while problem in this SO is not). @FinHead were you able to figure out the root cause or fix the issue by any chance?

[FinHead]

We had a new app design that was we were working on so I removed the display of the map in our tab to get rid of the crash. I'm no longer with that app team and don't have the ability to test the setSharedElementsUseOverlay fix. Good Luck!

[Pnemonic]

I recently added Firebase crashlytics. The tgkill appears in Play Console logs, but not in Firebase logs.

[Alexey Andronov]

It looks suspicious to me that AndroidNativeBuffer from Android 4.4 differs from Android 8.1. They make use of one of reserved pointers, so maybe if you didn't zero the object properly that's the cause of crash. That's just a suggestion which I can't test for now but would be glad if someone checks it

[Ali Zaidi]

was anyone ever able to solve this issue while using Google maps? I am still facing this and it only occurs on Samsung with Android 7.0, we are using SupportMapFragment as a child fragment

[hvaughan3]

Do you know of a work around to prevent this based on that bug report? I am not seeing any solution or workaround or anything.

[3c71]

I have the same issue that affects all my apps, however I only use support libraries: annotations, v4, appcompat and design.

[ASalameh]

raise + pthread_kill + tgkill: This is the O/S (Android) shutting down the app. Is this occur when user kill the app or automatically from OS ?

[Lennart Rolland]

That is the O/S shutting it down a malfunctioning process as far as I know. If the app terminated "peacefully" it would not be a "kill" operation.

[Ran94]

Hi @songoku1610, how did you find out the issue is caused by Cardview.

[songoku1610]

I tried to replaced cardview by Framelayout then this issues was solved, this issues only happen on Android 7.x

[songoku1610]

An other things, The question above have been edited, remove text independencies on grade, so my answer because I see in dependencies have including carview seem did not matched with question

[Shishir Shetty]

My application uses TextureView. Could you please elaborate more on invalidate-onDraw animation.

[Sergei Belozerov]

@ShishirShetty I don't use TextureView anymore I override View, describe all animations in onDraw method and call postInvalidateOnAnimation() method every 16 milliseconds (~60fps)