Android Package manager has died with TransactionTooLargeException

Question

My app reads the list of all installed APK files, and then loop through the list to read the APK info, however it throws a TransactionTooLargeException exception.

From what I have read here http://developer.android.com/reference/android/os/TransactionTooLargeException.html, google recommends to break large transactions into smaller transactions. However it seems this happens in the middle when looping through the APK list. If I catch the exception and continue it, the rest all works fine. Is there a way to reduce the memory usage while calling the getPackageInfo? Does that call hold some thing even after it already returned.

Here is the trace when it happened:

at android.app.ApplicationPackageManager.getPackageInfo(ApplicationPackageManager.java:89)
at com.myapp.appreader.getAppDetails(Appreader.java:207)
at com.myapp.appreader.collectData(Appreader.java:99)
at com.myapp.appreader.AppDataCollectionTask.run(AppDataCollectionTask.java:26)
at com.myapp.appreader.service.AppDataTaskExecutor$AppDataAsyncTask.executeTask(AppDataTaskExecutor.java:439) 
at com.myapp.appreader.service.AppDataTaskExecutor$AppDataAsyncTask.doInBackground(AppDataTaskExecutor.java:327)
at com.myapp.appreader.service.AppDataTaskExecutor$AppDataAsyncTask.doInBackground(AppDataTaskExecutor.java:1)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:234)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
at java.lang.Thread.run(Thread.java:856)\nCaused by: android.os.TransactionTooLargeExceptionat android.os.BinderProxy.transact(Native Method)
at android.content.pm.IPackageManager$Stub$Proxy.getPackageInfo(IPackageManager.java:1538)
at android.app.ApplicationPackageManager.getPackageInfo(ApplicationPackageManager.java:84)

Solution

There is a 1MB limit on a Binder transaction, which means most IPC invocations have to be modest in size.

If you hit a TransactionTooLargeException or similar Binder failures when retrieving data from PackageManager (e.g., getPackageInfo()), try splitting your request over multiple calls, with fewer flags (e.g., GET_META_DATA) per call. Hopefully this will reduce the size of any individual transaction to be under the 1MB limit.

Also, if you are using calls on PackageManager that return multiple results (e.g., getInstalledPackages(), try asking for no flags on that call, then retrieving the values for each package individually, to avoid getting a lot of data on a lot of entries at once.

And, of course, only use flags that you need, particularly if your call might contain some. GET_META_DATA is a classic example of that: many apps use it (e.g., for Play Services), but if you do not need to know the metadata information, don't request it.

Comments

[android developer]

Why can't the original code of Android handle such a request? Shouldn't it be the one that know about the limitations, divide the query and handle it correctly? How could I know how many apps there are, so that I could decide when to divide the query into multiple queries? Can you please show an example of code for having the same functionality as this code does: packageManager.getInstalledPackages(0)

[CommonsWare]

@androiddeveloper: "Shouldn't it be the one that know about the limitations, divide the query and handle it correctly?" -- you are welcome to file an issue to try to get them to improve this. "How could I know how many apps there are" -- call getInstalledPackages() with no flags, aiming for the minimum metadata per entry, and hope for the best. I am not aware of other approaches.

[android developer]

But this is what I did. no flags means 0, no? I still got this exception from at least one user. About reporting Google, I was sure I've reported about it. Odd. Anyway, searched again, and I've found this: code.google.com/p/android/issues/detail?id=93717 . Does it really say that they've fixed it on Android 5.1 ?

[CommonsWare]

@androiddeveloper: "no flags means 0, no?" -- it should be. "I still got this exception from at least one user" -- :: shrug :: Fail as gracefully as you can, I guess. "Does it really say that they've fixed it on Android 5.1 ? " -- that would be my interpretation of comment #11 on the issue.

[android developer]

So there is nothing I can do about it... I wonder how many apps does it take to cause this. Surely the user doesn't have 1M apps installed (that's the max, if each app-info took 1B)... is it possible perhaps that I'd do the query myself? checking the folders of APKs paths?

[CommonsWare]

@androiddeveloper: I would expect PackageInfo and subsidiary objects to be in the hundreds of bytes per entry. It's also possible that your crash is coming from some bot, running on a device or emulator that really does have thousands of apps installed.

[android developer]

The crash reports came from SGS4, Note3 and Ace3. Do you know perhaps of a way to get all apps (even the package names or APKs) without using this API ? Maybe there are known paths that hold all APK files? What I've found from my installed apps are : "/system/app/" , "/data/app/" , "/system/priv-app/" , "/mnt/asec/" . Are they accessible without root?

[CommonsWare]

@androiddeveloper: "Do you know perhaps of a way to get all apps (even the package names or APKs) without using this API ?" -- personally, no.

[android developer]

I've now tested the paths, and sadly (and surprisingly), the main one which is "/data/app" is also the only one that isn't accessible without root.

[android developer]

It seems that the file in the Android source code, which is responsible to actually get the installed apps information is "PackageManagerService" and that it reads from XML file "packages.xml" which is in new File(Environment.getDataDirectory(),"system") . Sadly, trying to read from it failed with exception: "java.io.FileNotFoundException: /data/system/packages.xml: open failed: EACCES (Permission denied)" . I guess I've reached a dead end, unless I use root...

[android developer]

Can you guess the number of apps that could trigger this issue?

[CommonsWare]

@androiddeveloper: Off the cuff? Thousands. Maybe high hundreds. And and this point, please ask a fresh SO question, rather than continuing to chain comments here.

[android developer]

Sorry and thank you. It's just that I've found a lot of questions regarding it, but nobody knows how to deal with it. Nobody has an answer...

[android developer]

I think I've found a solution that works without root: call the console command "pm list packages" , which lists all installed apps' package names. Then, I can get each app's info, one after another. I hope it works. Thank you for trying to help.

[android developer]

OK, published the solution, if you wish to check it out.