Mongoose and multiple database in single node.js project

Question

I'm doing a Node.js project that contains sub projects. One sub project will have one Mongodb database and Mongoose will be use for wrapping and querying db. But the problem is

• Mongoose doesn't allow to use multiple databases in single mongoose instance as the models are build on one connection.

• To use multiple mongoose instances, Node.js doesn't allow multiple module instances as it has caching system in require(). I know disable module caching in Node.js but I think it is not the good solution as it is only need for mongoose.

  I've tried to use createConnection() and openSet() in mongoose, but it was not the solution.

  I've tried to deep copy the mongoose instance (http://blog.imaginea.com/deep-copy-in-javascript/) to pass new mongoose instances to the sub project, but it throwing RangeError: Maximum call stack size exceeded.

I want to know is there anyways to use multiple database with mongoose or any workaround for this problem? Because I think mongoose is quite easy and fast. Or any other modules as recommendations?

Solution 1

One thing you can do is, you might have subfolders for each projects. So, install mongoose in that subfolders and require() mongoose from own folders in each sub applications. Not from the project root or from global. So one sub project, one mongoose installation and one mongoose instance.

-app_root/
--foo_app/
---db_access.js
---foo_db_connect.js
---node_modules/
----mongoose/
--bar_app/
---db_access.js
---bar_db_connect.js
---node_modules/
----mongoose/

In foo_db_connect.js

var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo_db');
module.exports = exports = mongoose;

In bar_db_connect.js

var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/bar_db');
module.exports = exports = mongoose;

In db_access.js files

var mongoose = require("./foo_db_connect.js"); // bar_db_connect.js for bar app

Now, you can access multiple databases with mongoose.

Solution 2

According to the fine manual, createConnection() can be used to connect to multiple databases.

However, you need to create separate models for each connection/database:

var conn      = mongoose.createConnection('mongodb://localhost/testA');
var conn2     = mongoose.createConnection('mongodb://localhost/testB');

// stored in 'testA' database
var ModelA    = conn.model('Model', new mongoose.Schema({
  title : { type : String, default : 'model in testA database' }
}));

// stored in 'testB' database
var ModelB    = conn2.model('Model', new mongoose.Schema({
  title : { type : String, default : 'model in testB database' }
}));

I'm pretty sure that you can share the schema between them, but you have to check to make sure.

Solution 3

Pretty late but this might help someone. The current answers assumes you are using the same file for your connections and models.

In real life, there is a high chance that you are splitting your models into different files. You can use something like this in your main file:

mongoose.connect('mongodb://localhost/default');

const db = mongoose.connection;

db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
  console.log('connected');
});

which is just how it is described in the docs. And then in your model files, do something like the following:

import mongoose, { Schema } from 'mongoose';

const userInfoSchema = new Schema({
  createdAt: {
    type: Date,
    required: true,
    default: new Date(),
  },
  // ...other fields
});

const myDB = mongoose.connection.useDb('myDB');

const UserInfo = myDB.model('userInfo', userInfoSchema);

export default UserInfo;

Where myDB is your database name.

Solution 4

As an alternative approach, Mongoose does export a constructor for a new instance on the default instance. So something like this is possible.

var Mongoose = require('mongoose').Mongoose;

var instance1 = new Mongoose();
instance1.connect('foo');

var instance2 = new Mongoose();
instance2.connect('bar');

This is very useful when working with separate data sources, and also when you want to have a separate database context for each user or request. You will need to be careful, as it is possible to create a LOT of connections when doing this. Make sure to call disconnect() when instances are not needed, and also to limit the pool size created by each instance.

Solution 5

Mongoose and multiple database in single node.js project

use useDb to solve this issue

example

//product databse 
const myDB = mongoose.connection.useDb('product');
module.exports = myDB.model("Snack", snackSchema);
//user databse
const myDB = mongoose.connection.useDb('user');
module.exports = myDB.model("User", userSchema);

View more solutions

Comments

[Simon Holmes]

Yes, named connections and a shared schema are the way to go I think. Each connection will need a unique model as per Robert's example.

[aaronheckmann]

Also checkout useDb() available in 3.8 to share the underlying connection pool: github.com/LearnBoost/mongoose/wiki/…

[Anooj Krishnan G]

Suppose I have auto-generated database (Say n number of database). Not one or two. Is there any way to connect to these without creating separate model for each database ?

[robertklep]

@AnoojKrishnanG I don't think that's possible, no. You need to create the model against each database separately. However, as I already stated in my answer, you may be able to share the schema's amongst the connections, which may save some coding time.

[Anooj Krishnan G]

Is this is the only way to connect multiple database at same time ?

[robertklep]

@AnoojKrishnanG you can also use useDb(), as @aaronheckmann already mentioned. That way, you can open multiple database connections through the same connection pool.

[xpepermint]

This means that every project will have its own connection. You will not be able to manage 100k connections. I think it would be better to use useDb command which uses the same connection pool.

[GvSharma]

is there anyway to create two mongoose connections with single model?

[robertklep]

@gvsharma to the same database? Mongoose already maintains a pool of (concurrent) connections (5 by default, I believe) to the database.

[GvSharma]

No it is not the same database..i have two databases. I need to get the data from old database and insert that to new database. Here old and new databases will have some common fields.

[robertklep]

@gvsharma in that case, AFAIK you still need to create two separate models, one for each database. The schema for each model can probably be the same.

[Lion789]

xpepermint are you able to show an example for useDb -- I am having this issue currently stackoverflow.com/questions/37583198/…

[Eshwar Prasad Yaddanapudi]

This looks like a huge burden on the project. don't you think so?

[aquaman]

@AnoojKrishnanG I am facing a similar problem of auto-generating databases for each of my clients. Did you find any solution regarding this?

[pravin]

is this another way to writing 'Above Answer' ?

[grant]

You can share the schema across the different models, and therefore DBs. var newSchema = new mongoose.Schema({ ... }), var model2 = conn1.model('newModel', newSchema), var model2 = conn2.model('newModel', newSchema)

[Iain Collins]

Having a few different connection instances (e.g. for a User DB, a Session DB, and for application data) per application is absolutely fine. It's not 'a huge burden' or going to cause scaling problems and is a common use case.

[Martín Valdés de León]

This is not the above answer, it's better. The above answer installs multiple copies of Mongoose, unnecessarily.

[Lead Developer]

I've tried like above. there is no error. but when I try to modelName.find({}).then(data => {}), data is empty array.

[Lead Developer]

oh, that's my fault. this method works fine. Thank you so much.

[Johnathan Enslin]

Thank you - I was able to use 3 different databases within a single application using: const mongoose = require('mongoose'); const Schema = mongoose.Schema; const mySchema = new Schema ({}); const mydbvar = mongoose.connection.useDb('mydb') module.exports = mydbvar.model('myCollection', MySchema);

[Newclique]

Definitely the best and most real-world example. Connect to the default db (just like if you were using something like SQL Server) and then take advantage of useDb to target your DML at the appropriate database. (Very helpful for keeping your users in one db and your data in another.) No need to start making multiple connections when ultimately you are sending requests to the same server. Now, if you were connecting to two different servers, that's a different kettle of fish.

[shahidfoy]

how would i make queries using this method?

[joniba]

As @Wade said, as far as I understand this solution only works when all of the databases are on the same server. It's not clear if this answers the OP's question and IMO is a bit misleading.

[Biruel Rick]

You are the best my friend! thanks so much! it works for me! thanks!

[Abdallah Al Barmawi]

await instance1.connection.collection('foo').insert({ foo: 'bar', }) await instance2.connection.collection('foo').insert({ foo: 'zoo', })

[Polv]

This is just what I needed for MongoDB Atlas migration from test, and also to avoid having multiple connections. However, I also .db at the end (const v1 = mongoose.connection.useDb('test').db) as the old db doesn't need to be mongoose managed.

[tzn]

In fact better working in my case since I have completely different credentials for each connection, let alone models and databases.

[Faizi]

Note that mongoose.createConnection is asynchronous. Use asyc/await or .then().catch() format

[Curious Flower]

via createConnection on a hobby dyno of Heroku with MongoDB atlas

[robertklep]

@ArnavSingh and how many databases were you able to connect to?

[Curious Flower]

I am working on an open-source alternative to Firebase, it provides firebase like client-side abstraction for MongoDB. So far it is working well with 10 databases. I am wondering if there any kind of limitation on how many connections a server can make and what is that limit? github.com/itsarnavsingh/upon.one

[robertklep]

@ArnavSingh I doubt this is going to scale well, at least with Mongoose. There are probably limits, but those will be relatively high (say 8000 or 16000 connections).

[Curious Flower]

Thank you so much for the info, can u please give me a link to the documentation reference?

[Hacker]

But as this does not have await, i cant stop my server if connection to mongo did not happen.

[Muhammad Usman]

how can I create a modal using this way ?

[Jovanni G]

This should be the best solution. Utilize one connection to access multiple database

[Miquel]

Seems great when db's are in the same server... Is there a similar approach when db's are in differents hosts?