Proper way to clear Realm table/database?

Question

I have a realm object with ~30 fields, after adding and removing several objects it seems that realm takes up quite a bit amount of space. The size of the allocated space seems to grow somewhat exponentially:

10*(add 100 + remove all) = 4 mb Data

15*(add 100 + remove all) = 33 mb Data

20*(add 100 + remove all) = 91 mb Data

25*(add 100 + remove all) = 179 mb Data

The file itself in data\data\app_folder\files\default.realm is 200 mb at this point.

Now this serious issue might be because i am not doing something properly. Before every insertion i do

Realm realm = Realm.getInstance(context);

realm.beginTransaction();
realm.where(RealmSubmission.class).findAll().clear();
// if i use realm.allObjects(RealmSubmission.class).clear(); the leak is even bigger, i get to 170mb Data with 20*(add 100 + remove all) even though both calls do the same by looking at their semantics.
realm.commitTransaction();

Adding items into realm looks like this:

    for (Submission submission : submissionList){
        realm.beginTransaction();

        RealmSubmission realmSubmission = realm.createObject(RealmSubmission.class);
        RealmObjectUtils.copySubmission(realmSubmission, submission);

        realm.commitTransaction();
    }

Any ideas?

Solution 1

I tried replicating with a small model class (one String, one int) with no success.

Do you use Links and/or LinkLists in your model? Can I take a look at it?

One reason might be in the case you have for example a Person class that has a RealmListdogs field. When you delete all the elements of the Person type the Dogs are right now retained in the database.

EDIT: After you provided the data I tried with a bit of dummy data:

Realm.deleteRealmFile(this);
Realm realm = Realm.getInstance(this);
File realmFile = new File(this.getFilesDir(), "default.realm");

long tic = System.currentTimeMillis();
for (int i = 0; i < 25; i++) {
    for (int j = 0; j < 100; j++) {
        realm.beginTransaction();
        TestObject testObject = realm.createObject(TestObject.class);
        testObject.setApprovedBy("Approver_" + j);
        testObject.setAuthor("Author_" + j);
        testObject.setBannedBy("Banner_" + j);
        testObject.setClicked(j % 2 == 0);
        testObject.setCommentCount(j);
        testObject.setCreated(System.currentTimeMillis());
        testObject.setCreatedUTC(j*7);
        testObject.setEdited(j % 3 == 0);
        realm.commitTransaction();
    }
    realm.beginTransaction();
    realm.where(TestObject.class).findAll().clear();
    realm.commitTransaction();
    Log.i(TAG, "Size: " + realmFile.length());
}
long toc = System.currentTimeMillis();
Log.i(TAG, "Time: " + (toc - tic));

But I still cannot reproduce:

10-08 14:39:01.579  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 24576
10-08 14:39:01.999  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 24576
10-08 14:39:02.409  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 24576
10-08 14:39:02.809  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 24576
10-08 14:39:03.209  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 24576
10-08 14:39:03.649  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:04.049  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:04.449  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:04.839  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:05.329  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:05.709  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:06.259  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:06.689  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:07.109  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:07.589  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:08.019  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:09.129  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:09.729  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:10.169  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:10.669  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:11.049  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:11.449  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:11.849  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:12.269  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Size: 49152
10-08 14:39:12.269  29233-29233/myapp.realm.io.sizeleak I/REALMTEST﹕ Time: 11265

The size doubling is expected because of fragmentation, but I still see nothing that can suggest your experience.

The timing is high because of the large number of transactions. Batching them together would increase performance considerably:

10-08 14:45:25.009  31593-31593/myapp.realm.io.sizeleak I/REALMTEST﹕ Time: 408

Solution 2

I've made a simple method to delete the realm database file when a migration exception occur (for dev). It also return a new realm instance to prevent any issue.

public Realm buildDatabase(){
    RealmConfiguration realmConfiguration = new RealmConfiguration.Builder(this).build();

    try {
        return Realm.getInstance(realmConfiguration);
    } catch (RealmMigrationNeededException e){
        try {
            Realm.deleteRealm(realmConfiguration);
            //Realm file has been deleted.
            return Realm.getInstance(realmConfiguration);
        } catch (Exception ex){
            throw ex;
            //No Realm file to remove.
        }
    }
}

Solution 3

Delete all objects from Realm database:

realm.executeTransaction(new Realm.Transaction() {
    @Override
    public void execute(Realm realm) {
        realm.deleteAll();
    }
});

Solution 4

You should delete the Realm file. Until first time you use Realm no file is created. That's what you want:

try {
  Realm.deleteRealmFile(context);
  //Realm file has been deleted.
} catch (Exception ex){
  ex.printStackTrace();
  //No Realm file to remove.
}

Be sure no realm instance is been used.

View more solutions

Comments

[chrystolin]

Its a representation of a reddit submission: pastebin.com/b8mPTKam and the copysubmission handles the case if any field is null, because i once had a similar exception, that i cannot insert null pastebin.com/ur6f7jXW

[chrystolin]

might have been mistaken about the inserting null then. :)

[chrystolin]

Batching transactions together did help, 1,5 mb after 20*100, 2,9 after 40*100, 3,7 mb after 60*100, 4,1mb after 80*100 transaction. Stays 4,1mb after 100*100. I have no idea why doing lots of small transactions end up with such a big leak opposed to one big transaction, but it works now, and i am happy with it, its also better practice to do this.

[Emanuelez]

Also you don't need to check primitives for null in the utility method. One more question: is this all happening on one or several threads?

[chrystolin]

Yeah i wanted to do that, i have no idea where did i get this "don't insert null" :) right now i have one big transaction where i create 100 new realmobjects and commiting them at once at the end, all this is happening in a single asynctasks doInBackground. the previous version was also in this one asynctask

[Emanuelez]

I see. Careful with AsyncTask, since some of its processing methods run on a thread and its post-processing methods run on the calling thread. As always you are not allowed to pass Realm related objects among threads.

[chrystolin]

Thanks, will keep this in mind. I remember why i did this null check, jreddit api returns the fields in wrapped classes (Long, Double, Boolean) and i couldn't just set the variables of the Realm object, because one can't set primitive to null, so i would need to check it anyway, what i could do is not to set anything at the respective realmobjects field if the original ones respective field is null

[localhost]

This answer is outdated.

[Draško]

So, localhost, what is your answer then?

[Frank]

Realm.getDefaultInstance().deleteAll();