Read a file from a mongo shell

Question

I have a DB called wordlists and a collection called rockyou.

I have a file called rockyou with the contents:

123456
12345
123456789
password
iloveyou
princess
1234567
rockyou
12345678
abc123

What I would like to do is load all of the contents of the file rockyou into the collection rockyou in my wordlists DB. I have tried searching google but the resources seem slim for mongo. I could really use some help with this and also maybe some good resources for this new to me DB application.

Thanks!

**EDIT* Here are the command I have used and some of what I have tried.

use wordlists
db.createCollection("rockyou")
db.rockyou.insert(copyFile(~/wordlists/rockyou))

I have tried some other methods to insert but now realize there were even further off than this one...

Solution

If you really want to use only mongoshell, you can use cat() command and do the following (txt is not necessary, it is just how my file was named):

use wordlists
var file = cat('path/to/yourFile.txt');  // read the file
var words = file.split('\n'); // create an array of words
for (var i = 0, l = words.length; i < l; i++){ // for every word insert it in the collection
    db.rockyou.insert({'word': words[i]}); 
}

This was tested on Mongo 3.0.1 and produced something like:

{ "_id" : ObjectId("551491ee909f1a779b467cca"), "word" : "123456" }
{ "_id" : ObjectId("551491ee909f1a779b467ccb"), "word" : "12345" }
...
{ "_id" : ObjectId("551491ee909f1a779b467cd3"), "word" : "abc123" }

But I would introduce an application logic here (for example with python):

import pymongo
connection = pymongo.Connection()
collection = connection.wordlists.rockyou

with open('path/to/yourFile.txt') as f:
    for word in f.readlines():
        collection.insert({'word': word.rstrip()})

Comments

[Dylan]

"if the resources are so abundant please give me a link on how to insert a plain, non JSON format text file and I will give you all of my reputations points." <-- This was not a link but I am still happy to give them to you. Besides creating a bounty in two days how do I give them to you now?

[Salvador Dali]

@Dylan I added a link. The bounty is the only option I am aware of, but to tell the truth it is not compulsory. I just wrote this for fun. Your question would become way better if you would specify that you need only to use mongoshell (if you can use anything, the problem is straight forward, but with mongoshell it is harder to find the solution).

[Dylan]

lol I will edit my question to include that info. This method wont work on my large file so I will look into those other options, I am not stuck using only the shell. Guess I will look into one of those application abstractions you talked about, like pymongo. Thanks and I'll create the bounty in 2 days and let you know so you can answer and I can award it to you ;) I said I would!

[Dylan]

I know javascript, so would you suggest Node.js then?

[Salvador Dali]

@Dylan you can use any programming language (node /php/C#, whatever). I included python, because I had pycharm opened and because python is installed on all linux machines by default.

[Dylan]

Ok great indo, I am in the MongoDB Driver page so I will take it all into consideration.... Thanks again and I'm serious about the bounty. Talk to you in a couple days

[fanbondi]

@SalvadorDali what if your queries are stored in a file. I tried to read a file of queries and in the loop, I am doing queries[i] but it is not working. A sample query is db.coll.find(). I am measuring the time taken and it takes 0 sec and this takes 14 secs in normal mongo shell.

[shijin]

cat command worked, saved my day. I can now use cat command to load json files to mongo shell scripts.