Is there a way to batch read firebase documents

Question

I am making a mobile app using flutter with firebase as my backend.

I have a collection of user document that stores user information. one of the fields is an array of references (reference documents in another collection) which I want to use in an operation like batch that in that would then allow be to read all the documents.

I know batch only allows writes to the database, My second option would be Transaction, which requires writes after reads which I am trying to avoid.

Is there a way to read multiple documents in one operation without having to use Transaction?

Solution 1

Firestore doesn't offer a formal batch read API. As Frank mentions in his comment, there is a way to use IN to fetch multiple documents from a single collection using their IDs. However, all of the documents must be in the same collection, and you can't exceed 10 documents per query. You might as well just get() for each document individually, as the IN query has limitations, and isn't guaranteed to execute any faster than the individual gets. Neither solution is guaranteed to be "consistent", so any one of the documents fetched could be "more fresh" than the others at any given moment in time.

Solution 2

Firestore has a REST API that allows you to do batch GETs with document paths that may be what you need.

See https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents/batchGet

Solution 3

If you know the document IDs and the collection paths of the documents needed to be fetched, you could always use the getAll() method which is exposed in the firebase Admin SDK (at least for Node.js environments).

Then, for example, you could write an HTTPS Callable Function that would accept a list of absolute document paths and perform a "batch get" operation on them using the getAll() method.

e.g.

// Import firebase functionality
const functions = require('firebase-functions');
const admin = require('firebase-admin');

// Configure firebase app
admin.initializeApp(functions.config().firebase);

// HTTPS callable function
exports.getDocs = functions.https.onCall((data, context) => {

    const docPathList = data.list; // e.g. ["users/Jkd94kdmdks", "users/8nkdjsld", etc...]
    const firestore = admin.firestore();

    var docList = [];

    for (var i = 0; i <= docPathList.length - 1; i++) {

        const docPath = docPathList[i];
        const doc = firestore.doc(docPath);
        docList.push(doc);
    }

    // Get all
    return firestore.getAll(...docList)
        .then(results => {
            return { data : results.map(doc => doc.data()) };
        })
        .catch(err => {
            return { error : err };
        })
});

Not sure what the limit (if any) is for the number of documents you can fetch using getAll(), but I do know my application is able to fetch at least 50 documents per call successfully using this method.

Comments

[William]

Thanks for the answer Doug, Although I am trying to get multiple documents from different collect so IN wont necessarily solve my problem. Your answer help me understand the capabilities and restriction of firebase.

[Yacine Benali]

@doug-stevenson "and you can't exceed 10 documents per query", the doc only mentiones that "Use the IN operator to combine up to 10 equality (==) clauses on the same field " from my understanding, the limitation is that you can query up to 10 equality clauses on one field and get back as many documents as there are. not only 10 documents, am I wrong?

[Noam]

@Yachine true, but in this case he's performing an IN query on the id attribute. So it's the same thing here.

[Albert Renshaw]

There is a benefit to using IN over using serial gets, if you are sorting by a field and have a limit on the number of docs you want to be returned. With the IN query those may all come from a single field value as it may have many more documents and it fills up the 'limit', whereas when doing it via concurrent gets you would have to apply the limit to each get (resulting in up to 10x as many total reads) and then sort on your end, and then truncate if desired.