mongo3.0 restore fails with error Failed: restore error: insertion error: EOF

Question

I'm trying to restore a database with 100 collections (800GB) using mongorestore which was taken as a backup from mongo 2.4.4. I used the following command:

mongorestore -u <username> -p <password> -d bp_prod079 
             /data/backup/2015-11-28/bp_prod079 --batchSize=100

I'm using batchSize =1000, 500, 100 to try to make it work. I'm getting an insertion error after 10-15 collections. It almost runs for almost an hour and half perfectly when I get this error.

Failed: restore error: bp_prod079.ANEpisodeBreakdown: error restoring from /data/backup/2015-11-28/bp_prod079/ANEpisodeBreakdown.bson: insertion error: EOF

Solution 1

For me with a minor batchSize works. First I had --batchSize=50 and changed it to 10 --batchSize=10 and perfect.

Solution 2

On MongoDB servers using the WiredTiger storage engine (default on MongoDB v3.2.x), this error can also be thrown when running out of memory, causing both mongorestore and mongod to crash, as MongoDB developer @mpobrien explains here.

The solution is to add more RAM to the server, or tune the storage.wiredTiger.engineConfig.cacheSizeGB setting as described here: https://groups.google.com/d/msg/mongodb-user/Ms2MXCajreg/OsW6ixcSCgAJ