Spring-data-mongodb connect to multiple databases in one Mongo instance

Question

I am using the latest spring-data-mongodb (1.1.0.M2) and the latest Mongo Driver (2.9.0-RC1). I have a situation where I have multiple clients connecting to my application and I want to give each one their own "schema/database" in the same Mongo server. This is not a very difficult task to achieve if I was using the driver directly:

Mongo mongo = new Mongo( new DBAddress( "localhost", 127017 ) );

DB client1DB = mongo.getDB( "client1" );
DBCollection client1TTestCollection = client1DB.getCollection( "test" );
long client1TestCollectionCount = client1TTestCollection.count();

DB client2DB = mongo.getDB( "client2" );
DBCollection client2TTestCollection = client2DB.getCollection( "test" );
long client2TestCollectionCount = client2TTestCollection.count();

See, easy. But spring-data-mongodb does not allow an easy way to use multiple databases. The preferred way of setting up a connection to Mongo is to extend the AbstractMongoConfiguration class:

You will see that you override the following method:

getDatabaseName()

So it forces you to use one database name. The repository interfaces that you then build use that database name inside the MongoTemplate that is passed into the SimpleMongoRepository class.

Where on earth would I stick multiple database names? I have to make multiple database names, multiple MongoTempates (one per database name), and multiple other config classes. And that still doesn't get my repository interfaces to use the correct template. If anyone has tried such a thing let me know. If I figure it out I will post the answer here.

Thanks.

Solution 1

So after much research and experimentation, I have concluded that this is not yet possibly with the current spring-data-mongodb project. I tried baja's method above and ran into a specific hurdle. The MongoTemplate runs its ensureIndexes() method from within its constructor. This method calls out the the database to make sure annotated indexes exist in the database. The constructor for MongoTemplate gets called when Spring starts up so I never even have a chance to set a ThreadLocal variable. I have to have a default already set when Spring starts, then change it when a request comes in. This is not allowable because I don't want nor do I have a default database.

All was not lost though. Our original plan was to have each client running on its own application server, pointed at its own MongoDB database on the MongoDB server. Then we can provide a -Dprovider= system variable and each server runs pointing only to one database.

We were instructed to have a multi-tenant application, hence the attempt at the ThreadLocal variable. But since it did not work, we were able to run the application the way we had originally designed.

I believe there is a way though to make this all work, it just takes more than is described in the other posts. You have to make your own RepositoryFactoryBean. Here is the example from the Spring Data MongoDB Reference Docs. You would still have to implement your own MongoTemplate and delay or remove the ensureIndexes() call. But you would have to rewrite a few classes to make sure your MongoTemplate is called instead of Spring's. In other words, a lot of work. Work that I would like to see happen or even do, I just did not have the time.

Thanks for the responses.

Solution 2

Here is a link to an article I think is what you are looking for http://michaelbarnesjr.wordpress.com/2012/01/19/spring-data-mongo/

The key is to provide multiple templates

configure a template for each database.

<bean id="vehicleTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
    <constructor-arg ref="mongoConnection"/>
    <constructor-arg name="databaseName" value="vehicledatabase"/>
</bean>

configure a template for each database.

<bean id="imageTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
        <constructor-arg ref="mongoConnection"/>
        <constructor-arg name="databaseName" value="imagedatabase"/>
</bean>

<bean id="vehicleTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
    <constructor-arg ref="mongoConnection"/>
    <constructor-arg name="databaseName" value="vehicledatabase"/>
</bean>

Now, you need to tell Spring where your repositories are so it can inject them. They must all be in the same directory. I tried to have them in different sub-directories, and it did not work correctly. So they are all in the repository directory.

<mongo:repositories base-package="my.package.repository">
    <mongo:repository id="imageRepository" mongo-template-ref="imageTemplate"/>
    <mongo:repository id="carRepository" mongo-template-ref="vehicleTemplate"/>
    <mongo:repository id="truckRepository" mongo-template-ref="vehicleTemplate"/>
</mongo:repositories>

Each repository is an Interface and is written as follows (yes, you can leave them blank):

@Repository
public interface ImageRepository extends MongoRepository<Image, String> {

}

@Repository
public interface TruckRepository extends MongoRepository<Truck, String> {

}

The name of the private variable imageRepository is the collection! Image.java will be saved to the image collection within the imagedb database.

Here is how you can find, insert, and delete records:

@Service
public class ImageService {

    @Autowired
    private ImageRepository imageRepository;
}

By Autowiring you match the variable name to the name (id) in your configuration.

Solution 3

You may want to sub-class SimpleMongoDbFactory and strategize how the default DB as returned by getDb is returned. One option is to use thread-local variables to decide on the Db to use, instead of using multiple MongoTemplates.

Something like this:

public class ThreadLocalDbNameMongoDbFactory extends SimpleMongoDbFactory {
    private static final ThreadLocal<String> dbName = new ThreadLocal<String>();
    private final String defaultName; // init in c'tor before calling super

    // omitted constructor for clarity

    public static void setDefaultNameForCurrentThread(String tlName) {
        dbName.set(tlName);
    }
    public static void clearDefaultNameForCurrentThread() {
        dbName.remove();
    }

    public DB getDb() {
        String tlName = dbName.get();
        return super.getDb(tlName != null ? tlName : defaultName);
    }
}

Then, override mongoDBFactory() in your @Configuration class that extends from AbstractMongoConfiguration like so:

@Bean
@Override
public MongoDbFactory mongoDbFactory() throws Exception {
  if (getUserCredentials() == null) {
      return new ThreadLocalDbNameMongoDbFactory(mongo(), getDatabaseName());
  } else {
      return new ThreadLocalDbNameMongoDbFactory(mongo(), getDatabaseName(), getUserCredentials());
  }
}

In your client code (maybe a ServletFilter or some such) you will need to call: ThreadLocalDBNameMongoRepository.setDefaultNameForCurrentThread() before doing any Mongo work and subsequently reset it with: ThreadLocalDBNameMongoRepository.clearDefaultNameForCurrentThread() after you are done.

Solution 4

The spot to look at is the MongoDbFactory interface. The basic implementation of that takes a Mongo instance and works with that throughout all the application lifetime. To achieve a per-thread (and thus per-request) database usage you'll probably have to implement something along the lines of AbstractRoutingDataSource. The idea is pretty much that you have a template method that will have to lookup the tenant per invocation (ThreadLocal bound I guess) and then select a Mongo instance from a set of predefined ones or some custom logic to come up with a fresh one for a new tenant etc.

Keep in mind that MongoDbFactory usually get's used through the getDb() method. However, there are features in MongoDB that need us to provide a getDb(String name). DBRefs (sth. like a foreign key in the relational world) can point to documents an entirely different database. So if you're doing the delegation either avoid using that feature (I think the DBRefs pointing to another DB are the only places calling getDb(name)) or explicitly handle it.

From a configuration point of view you could either simply override mongoDbFactory() entirely or simply not extend the base class at all and come up with your own Java based configuration.

Solution 5

I used different DB using java Config, this is how i did it:

@Bean 
public MongoDbFactory mongoRestDbFactory() throws Exception { 
    MongoClientURI uri=new MongoClientURI(environment.getProperty("mongo.uri")); 
    return new SimpleMongoDbFactory(uri);
}

@Override
public String getDatabaseName() {
    return "rest";
}

@Override
public @Bean(name = "secondaryMongoTemplate") MongoTemplate mongoTemplate() throws Exception{ //hay que cambiar el nombre de los templates para que el contendor de beans sepa la diferencia  
    return new MongoTemplate(mongoRestDbFactory());    
}

And the other was like this:

@Bean 
public MongoDbFactory restDbFactory() throws Exception {
    MongoClientURI uri = new MongoClientURI(environment.getProperty("mongo.urirestaurants")); 
    return new SimpleMongoDbFactory(uri);
}

@Override
public String getDatabaseName() {
    return "rest";
}

@Override
public @Bean(name = "primaryMongoTemplate") MongoTemplate mongoTemplate() throws Exception{ 
    return new MongoTemplate(restDbFactory());    
}

So when i need to change my database i only select which Config to use

View more solutions

Comments

[Sankar]

@sbzomm I'm having the same scenario, Did you find the solution?

[Aditya]

Try this approach - blog.marcosbarbero.com/…. Looks fairly clean and extensible.

[sbzoom]

The SimpleMongoRepository does not have a getDb() method. So you cannot override it or call super.getDb(). That method is buried in the MongoTemplate. The SimpleMongoRepository has a reference to MongoOptions not MongoTemplate, so you can't get to getDB() there either. Maybe a ThreadLocalMongoTemplate? I will keep researching. This is a good path though - thanks.

[sbzoom]

I am torn between using ThreadLocal or not. But probably not. I sometimes want ClientA to read some records from ClientB's database. I would make a second query and pass the name of ClientB's database. What I really need is a MongoRepository interface (and implementation) that adds a "databaseName" to each query. count() -> count(databaseName). Or maybe instead of @Autowired instances of my repositories, I would instantiate them with a MongoTemplate (or MongoDbFactory). None of these really sound all that ideal.

[sbzoom]

Or maybe a getDB/setDB method on the MongoRepository (and SimpleMongoRepository). Then I could do: myRepository.setDB('name'); myRepository.findOne(id); Or, even nicer, myRepository.setDB('name').findOne(id); I'll see what I can work out.

[sbzoom]

The SimpleMongoRepository only has MongoOptions and not MongoTemplate or MongoDbFactory. So there seems to be no easy way to get the DB in the Repository, it is all abstracted.

[baja]

You are right - I made a mistake in pasting the incorrect class name. But the essence is the same, as Oliver describes in his comment.

[sbzoom]

Also, I do not want multiple Mongo instances. I only want one, with multiple databases in it. So I want multiple MongoTemplates.

[sbzoom]

unfortunately that is not what I am looking for. I saw such an implementation and it does work nicely. Just not for my purposes. This setup is if you have certain collections in certain databases. I want all the collections in all of the databases. Each client gets the same schema, just in different locations.

[beku8]

Thanks for this example. I got it working really easy. Is there any way to implement collection per tenant approach. If you have any idea, please share with me on this thread. I would appreciate a lot!

[beku8]

I got it working really easy. Is there any way to implement collection per tenant approach. If you have any idea, please share with me on this thread. I would appreciate a lot!

[milan]

@sbzoom what about having multiple spring contexts, one for each database? it doesnt do exactly what you want (single Mongo, multiple Dbs) but would work with some extra plumbing on top of it.

[Srisudhir T]

Is there any solution with the latest version, i am facing the same issue, the ensureIndexes is killing me :(

[sbzoom]

I looked at the source code of the MongoTemplate and did not see ensureIndexes() anymore - so it may work. The person who would know is @Oliver Gierke, who also posted an answer to this question - he is one of the main developers.

[Srisudhir T]

Finally figured out the issue, i was using Servlet 3.0 initialization and had not set the application context in the mongocontext while cerateing the factory, after setting it up, now everything is smooth

[Zarathustra]

Also note that mongo:repository is not present anymore since 1.1. The mongo-template-ref attribute is now on mongo:repositories level.

[Zarathustra]

I created a github project which addresses the same issue, it is able to create the indicies in each database. github.com/Loki-Afro/multi-tenant-spring-mongodb

[Titi Wangsa bin Damhore]

as of spring data mongodb 1.6.x, mongo:repository is no longer the child of mongo:repositories

[vashishth]

@john how i can reference monog-template using java annotation spring configuration.

[s1moner3d]

How do you change the Config to use?

[DaveStance]

Does anyone have an example of how this implementation might work using Java configurations and annotations? I cannot seem to achieve the same behaviour.

[JaskeyLam]

@TitiWangsabinDamhore how to fix it??

[JaskeyLam]

what about your repositories if I use CrudRepository? how to inject different mongoTemplate to different repo