How to insert date into mongo from JSON file

Question

I have a .json file that contains an object with a date. How can I make sure this date field is inserted as a "date" datatype in mongo?

I need this to be done via node.js.

{
    "name": "Jeff Johnson",
    "email": "[email protected]",
    "phone": "5555555555",
    "date_added": "2014-01-22T14:56:59.301Z"
}

Solution 1

The core of this is basically that your JSON you are using does not "preserve" the "data type" so that a "date" is determined and it is in fact just represented as a "string". The other side of course is that MongoDB itself is in fact "schemaless", so there is no inherent information available to the "database" to say that this field should be a date.

With that in mind, there are a few approaches which vary from your "current" implementation to "future" considerations.

However you are parsing the data ( and there are many parser implementations out there ), then it is up to "your" application logic to "know" that this field is in fact a "date", and to then handle that translation itself. So down to a document level:

var obj = JSON.parse(singleJSONDoc);
obj.date_added = new Date(obj.date_added);

That would be the basics of the code you would implement per each valid JSON document in the input, on a very basic assumption that the input can be read one line at a time. The same approach varies for a "stream parser" where you are then fed the basic object to modify, or even some libraries might support "hooks" to allow you to customize the parsed object.

As another approach, the logic can be built in to your applications own concept of "Schema". This is how most ODM implementations work ( one example being mongoose ) where you define the schema and the "types" of data there. The translations are then made for you based on your implementation.

For example: ( mongoose style ):

var personSchema = new Schema({
    "name": { "type": String },
    "email": { "type": String },
    "phone": { "type": String },
    "date_added": { "type": Date, "default": Date.now }
});

var Person = mongoose.model( "Person", personSchema );

var raw = JSON.parse(singleJSONDoc);
var person = new Person(raw);

person.save(function(err,doc) {
   // doc inserted here
})

So the "logic" is just moved from one place in your code to a "compartment" that manages the Schema and "types". Conversions are done for you based on the "type" implemented. Mongoose does Date out of the box for example. Other solutions exist, but this is the basic premise.

Finally, another way to approach this is in the JSON representation itself. MongoDB introduced some time ago a concept known as "extended json syntax", this has notablty been "borrowed" from in the EJSON project.

The general idea here is to "enable" the JSON parser, whether serializing or de-serializing to handle the "type" conversions for you. So basically while JavaScript (and other languages) have a concept of "types", JSON itself is just a "string" format. So there is some special handling here to "preserve" that type information in the "serialized" form so that it can be "de-serialized" back with the "types" preserved.

A format would look like this:

{
    "name": "Jeff Johnson",
    "email": "[email protected]",
    "phone": "5555555555",
    "date_added": { "$date": "2014-01-22T14:56:59.301Z" }
}

This is generally supported by parsers implementing the EJSON spec and in other tools like mongoimport and mongoexport, as well as some of the native driver implementations ( Java, C#, to name two ).

If you can work your input data into this format, then you can just use the parser and nothing else is required to handle the conversion. The good thing here is that you can use the same parser to "export" information, so this is useful when passing JSON data to a client, where now the "client" can also be "aware" and handle the type conversion properly.

So the short answer is, nothing is going to get done without you doing some work. MongoDB does not hold the schema, that is up to your application. And smart "type" conversions are not part of basic JSON. Use other libraries and code to handle the conversions.

Solution 2

Here is an example.

{ "$date": "2018-11-10T22:26:12.111Z" }

Such that

{
  "createdAt": { "$date": "2018-11-10T22:26:12.111Z" }
}

Comments

[Manuel Jordan]

How observation: is mandatory use "$date", so "" are mandatory. It is imported later how a ISODate("same pattern used") type