Firestore collection group query on documentId

Question

In my scenario a user can "like" the profile of another user. As a result I have a subcollection called "likedBy" for each user, where I create a document for a specific user, e.g.:

users(col) -> User A(doc) -> likedBy(col) -> User B (doc), User C (doc)

So in the rare scenario of a user being deleted, I want to delete all the likes issues by that user.

I am just not sure if this is even possible (a workaround could be to just save the userId again in said document and query for that).

What I am basically looking for is something like this:

db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();

The problem is, that I can not create an index for the documentId in the Firestore console.

Solution 1

There is no way you can use the following query:

db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();

And this is because collection group queries work only on document properties and not on document ids. According to the official documentation regarding collection group queries:

db.collectionGroup('landmarks').where('type', '==', 'museum');

You query the landmarks subcollection where the type property holds the value of museum.

A workaround could be to store the id of the user in an array and use array-contains but remember, for each collection group query you use, you need an index and unfortunately you cannot create such an index programmatically. Even if you can create an index in the Firebase console, it won't help you since you get those ids dynamically. So is not an option to create an index for each user separately because you'll reach the maximim number of indexes very quickly.

Maximum number of composite indexes for a database: 200

To solve this kind of problems, you should consider adding an array under each user object and use a query like this:

usersRef.where("usersWhoLikedMe", "array-contains", "someUserId")

Where usersWhoLikedMe is a property of type array.

Solution 2

UPDATE 2020-01-23:

For updates on this, see a conversation with Sam Stern on the group board:https://groups.google.com/d/msgid/google-cloud-firestore-discuss/e1b47358-b106-43a0-91fb-83c97d6244de%40googlegroups.com

Much of the discussion comes from the apparent fact that there is a SINGLE "master index" of ALL records in a database based on the FULLY QUALIFIED path to the document (hence only needing to be unique "within a collection").

To "accelerate" document references, the JS SDK actually "pre-pends" the collection path onto whatever info is passed in .doc to "conveniently" use that master index

i.e. all of these are exactly equivalent

db.doc('collection/docId/collection/docId/collection/docId")
db.collection('collection").doc("docId/collection/docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.collection("collection/docId/collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId/collection/docId").collection("collection").doc("docId")
db.collection("collection/docId/collection/docId/collection").doc("docId")

--- they ALL create the same index reference 'collection/docId/collection/docId/collection/docId" to find the document in the "master index".

(in my not-at-all-humble-opinion) FieldPath.documentId() was implemented (incorrectly) to "conveniently" match this behavior thus requiring the fully-qualified path, not the docId, when it should have been implemented like any other query, and required creating and maintaining a NEW INDEX to accommodate the query.

The code for this behavior was written BEFORE collectionGroups were implemented - and never documented the hack used didn't match the METHOD NAME used.

The work around is to require the Coder to copy the docId as a field in each document, and write your queries on that.  I already wrote my own layer between Firestore.js and my application to abstract the behavior, and will probably simply implement this as a basic feature of the library.

But this is clearly a MISTAKE, and so far everybody keeps trying to tell me it makes sense, and that they'll change the documentation to match the existing behavior (but not the method name).

As I wrote previously, I keep getting handed a ratty bunch of daisies, and being told "See? these are roses!! The documentation calls them roses!! Roses by any other name smell as sweet, and all that!!"

No Update Expected Unless They Get Embarrassed Enough

UPDATE 2020-01-10: I have built a demo app showing the exact bug, and have sent it to Firebase support as requested. For some dang reason, the support critter considers it a "feature request", in spite of it clearly a bug. When a URL is called in the form "/ShowInfo/showID", the App signs in to Firebase Auth anonymously; then calls a query on the collectionGroup (3 levels deep) using FieldPath.documentId() "==" showID

It makes the query 3 ways:

1) Once with only the showID- which fails with the familiar "Invalid query. When querying a collection group by FieldPath.documentId(), the value provided must result in a valid document path, but 'pqIPV5I7UWne9QjQMm72'(the actual showID) is not because it has an odd number of segments (1)."

2) Once with a "Relative Path" (/Shows/showID), which doesn't have the error, but returns no document.

3) Finally with the "Full Path" (/Artists/ArtistID/Tour/tourID/Shows/showID). This doesn't have an error, and does return a document - but if I have the full path, why do I need the query on the collectionGroup? And I don't have the full path - the showID (above) comes in as part of a URL (a link to the show data, obviously) - I hand-faked it for the test.

Waiting for response.

UPDATE 2019-12-02: Firebase support reached out to ask if I still wanted this solved. Duh.

UPDATE 2019-09-27: Firebase Support has acknowledged this is a bug. No word on when it will be fixed. documentId() should, indeed, be able to be used directly against only the document Id.

documentID can be used as part of a query but, as @greg-ennis notes above, it needs an even number of segments. As it turns out, if you truly need a collectionGroup (I do), then the "Id" to be compared needs to redundantly add the collectionGroup ID/name as a segment:

db.collectionGroup('likedBy')
.where(firebase.firestore.FieldPath.documentId(), '==', "likedBy" + "/" + "User A")
.get();

I've used it and it (*sorta) works. (admittedly, it took me 2 hours to figure it out, and the above question helped focus the search)

On the other hand, this specific case is not where you want to use collectionGroup. Remember what collection group is - a way to refer to a a set of separate collections as if they were one. In this case the "collection" that holds "User A"s likes exists as a collection under "User A"s doc. Simply delete that single collection before deleting "User A"s doc. No need to bring everybody else's likes into it.

Sorta: the field path compared apparently has to be the complete path to the document. If you know the documentId, but for "reasons" you do not know the complete path, including which document the sub-collection is a part of (kinda why you were using the collectionGroup approach in the first place), this now seems a tadly dead-end. Continuing working on it.

Verified and Bug Report filed: FieldPath.documentID() does not compare against the documentId; it compares against the fully segmented document path, exactly as you would have to provide to .doc(path):

To wit: to find a document at "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want"

using the collectionGroup "Travesty"

db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "document_id_I_want")

...will fail, but...

db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()

...will succeed. Which makes this useless, since if we had all that info, we would just use:

db.doc("TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()

Comments

[Thomas]

Thanks for the answer. So the only restriction with the array approach is to not exceed the 1 MB limit (which will be hard if I just save the IDs) and it is limited to around 40.000 likes (since this is the maximum amount of indexes for a document), am I correct? Also, when I want to retreive the name of the User, I wil lalso need to download the enitre list of likes.

[Alex Mamo]

Yes, you're correct. It's not about that you'll have to download the entire list of likes is about the read operation that you'll perform. Everything in Cloud Firestore is about the number of read and writes.

[Greg Ennis]

@AlexMamo The docs you linked to use an example of query by field, but that doesnt mean that collection group query by document ID is not possible. When I try I get an error message: "When querying a collection group by document ID, the value provided must result in a valid document path, but 'testid' is not because it has an odd number of segments." This error message strongly suggests that this is supported, but I'm not sure what to pass for document ID, since it wants an even number of segments. Any ideas?

[LeadDreamer]

yeah, such a hack can be made to work - it doesn't change that FieldPath.documentID is supposed to work. Why waste the extra storage?

[stevokk]

Love the detail and effort you put into the investigation and discussion with Google "support". If you have a reference to the bug report or a better solution that copying the document ID into each record please let me know! Kudos.

[akauppi]

Also stumbled on this problem. Would you have the issue link that @stevokk asked for?

[LeadDreamer]

Don't think I have the bug report, as such - but I do have the case #: "Case 00013719: FieldPath.documentId() is returning the wrong value"

[Isuru Bandara]

@AlexMamo, Just imaging I have saved a collection named Recipe in firestore For example [Collection(Recipes)-->document(Food_Recipe)-->subCollection‌​(Chiken Recipe),subCollection(Rice Recipe),...--->document(). If I have created subcollection like that can i use CollectionGroup queries to get a specific field value across the all different subcollection in the Food_Recipe Collection?

[Alex Mamo]

@IsuruBandara Yes, you can, as long as the name in all those subcollections is the same.

[Isuru Bandara]

@AlexMamo, I am trying to create an activity for search a food across the Recipe Collection. As you said Collection group does not work if the subcollection names are not the same?

[Alex Mamo]

@IsuruBandara That's right, the names should be the same.

[Isuru Bandara]

@AlexMamo, Is it possible to archive my goal only using firebase instead of using third party apps like Agolia ? Like I want to find a recipe using Recipe Collection :)

[Alex Mamo]

@IsuruBandara I'm afraid this question cannot be simply answered in a comment. So please post a new question using its own MCVE, so I and other Firebase developers can help you.

[Isuru Bandara]

@AlexMamo, Yes, sure. I did :) stackoverflow.com/questions/63988536/…

[Alex Mamo]

@IsuruBandara As I see, you just got an answer from a Firebase expert ;)

[delpo]

This is a pretty good summary. It's sad they don't work on this, this is essential to common queries. In ANY DATABASE if you have the ID, you are be able to fetch. I'm hoping to finish this project and never touch firebase again in my life.

[Denzo]

Solid post! Very much agree!

[LeadDreamer]

I SHOULD ADD: I still use Firebase/Firestore extensively, and am quite pleased with it. I am completing a wrapper package (firebase-wrapper) to generalize many features, in particular to use the same firebase and business logic libraries on both web and node (cloud functions).