After Mongodump, calling MongoRestore hangs

Question

We are trying to do a simple MongoDump on a relatively small DB.

our steps are simple:

1. export

2. drop exisiting DB from target machine

3. import on target machine

The MongoDump executes perfectly.

mongodump --out=/root/mongo-prod

The same goes for the DB drop:

mongo db_name --eval "db.dropDatabase()"

On the other hand, After calling mongoRestore

mongorestore --stopOnError --drop --db db_name /root/mongo-prod-{{ build }}/db_name/

The import process starts, and hangs on 3 specific collections with the following error repeating:

    no collection options to restore
restoring db_name.Collection4 from file /root/mongo-prod-31/db_name/Collection4.bson
        file /root/mongo-prod-31/db_name/Collection4.bson is 56625 bytes
using 1 insertion workers
[########################]                 db_name.Collection1  106.7 KB/106.7 KB  (100.0%)
[########################]                db_name.Collection2    63.5 KB/63.5 KB  (100.0%)
[######..................]                db_name.Collection3     6.7 MB/25.9 MB   (25.8%)
[########################]  db_name.Collection4    55.3 KB/55.3 KB  (100.0%)

[########################]                 db_name.Collection1  106.7 KB/106.7 KB  (100.0%)
[########################]                db_name.Collection2    63.5 KB/63.5 KB  (100.0%)
[######..................]                db_name.Collection3     6.7 MB/25.9 MB   (25.8%)
[########################]  db_name.Collection4    55.3 KB/55.3 KB  (100.0%)

******* This loops infinitely *******

p.s adding --repair to the mongodump command, creates a different error on mongorestore:

  Failed: restore error: db_name.Collection1: error restoring from /root/mongo-prod-33/db_name/Collection1.bson: insertion error: E11000 duplicate key error index: db_name.Collection1.$_id_ dup key: { : ObjectId('5651802de4b0293285f7f508') }

Solution 1

I just spent an hour with this:

Read from archive

--archive=/backup...

Ignore the archive argument and wait for stdin

--archive /backup

Seems arguments work with either arg value or arg=value, just not archive...

Solution 2

Obviously, the problem was related to MongoDB write concerns. From the MongoDB documentation (version 3.2):

Write concern describes the level of acknowledgement requested from MongoDB for write operations to a standalone mongod or to replica sets or to sharded clusters.

In a replica set configuration, this option defines how many replicas write operation needs to be written to, before it is considered successful.

The mongorestore utility has default write concern set to majority. Such value require acknowledgment that write operations have propagated to the majority of voting nodes including the primary. If you have >= 1/2 of your replica set (voting!) members step down, mongorestore will hang forever, waiting for acknowledgment by the majority of members. From the docs:

If you do not specify the wtimeout option and the level of write concern is unachievable, the write operation will block indefinitely.

If you still need to perform database restoring, just specify --writeConcern '{w:0}' in you mongorestore command. This will disable acknowledgment of the write operations for current mongo connection and you will be able to restore your database.

Here is a link to documentation: https://docs.mongodb.com/v3.2/reference/write-concern

P.S. The same is acceptable for the latest MongoDB version 3.4.

Solution 3

I was facing the same issue on replica set, I was trying to restore on Primary while secondary (in replica set) was down. I started secondary and then it started restoring successfully.

Solution 4

I ran into a similar issue. My database had a huge collection - few hundred gigabytes big. Mongorestore completed on the primary, indexes got built so I tried to run mongorestore for another db. This instance of mongorestore got stuck at 1%. When I looked at logs of the secondary, it was still building indexes. I had to wait for indexes to be rebuilt on secondary before running mongorestore for other database

Solution 5

Had the same issue on mongo version 3.2.8.

Updated to mongo 3.4.9 and everything worked.

View more solutions

Comments

[burzan]

I had a similar problem (somehow only on my mac, not on linux) and that solved it, thanks!!!

[QkiZ]

RTFM. I also had a similar problem. But in docs syntax is very clear for that: --archive <=file|null>. Use = sign to read from file.

[SMSk]

Omg thanks. That is very not user friendly at all!