Azure Cosmos DB: Clone collection to another database

Question

Currently I am trying to clone a cosmos db collection from one database to another database within the cosmos db. The API of the cosmos db is set to Mongo API.

I already tried to use Azure Data factory, but it looks like that there is no support for the Mongo API so far.

Has anyone an idea how to do this respective to efficiency, automation and performance?

Any ideas are appreciated.

Solution 1

You can use data Migration tool suggested by Microsoft to do the same.

There is no way to take a backup and import cosmosdb.

EDIT:

With the new Cosmic Clone tool, you can take a clone/backup with data/stored procedures/triggers/udf, etc. Read my blog on the same.

Solution 2

I already tried to use Azure Data factory, but it looks like that there is no support for the Mongo API so far.

Actually, Cosmos DB Mongo API and SQL API are all belong to Azure Cosmos DB service.So , you still can create cosmos db linked service and dataset in the azure data factory for your database.

Then you could create copy activity to import data from one collection to another collection.

If you want to make it as an automation task, I suggest using following 2 ways to run the copy activity.

1.Azure Time Trigger Function.

2.Web job which is run in the background of Azure Web App.

Hope it helps you.Any concern, please feel free to let me know.

Comments

[Fabian]

I forgot to mention that I need this on a daily basis as an automation task.The software don't seem to be able to be used in an automation environment.

[Sajeetharan]

in that case you need to write a function or web job to do that using api

[Fabian]

that is what I already does, but it seems, that the migration process corrupts my collection. The job finished without error, but from the Azure Portal, as well as with Robo 3T no access to the collection is possible anymore ...

[Fabian]

I guess so ... Hope that something is coming up, which can be used out of the box :)

[Fabian]

Hi Sajeetharan, I looked up your idea with the Migration tool. This will not work in this setup, as the documentation says: "The Data Migration tool does not currently support Azure Cosmos DB MongoDB API either as a source or as a target."

[Udhaya]

use 'skip incompatible rows and log' errors..on fault tolerance.

[Sajeetharan]

@Fabian now you can do backup with cosmosdb check the updated answer

[Sajeetharan]

sajeetharan.com/2019/04/30/…

[Venky]

@Sajeetharan does Cosmic Clone work for cloning gremlin collection ?

[Benjamin Brandt]

Note: Latest commit of Cosmic_Clone adds this to the README: **Disclaimer**: Please note this is not an official tool from the Azure Cosmos DB team, but a utility developed by an independent developer within Microsoft IT and offered on Github as a sample. Any ideas if a production-ready tool is in the works to replace it?