How to paginate Firestore with Android?

Question

I read Firestore documentation and all articles on internet(stackoverflow) about Firestore pagination but no luck. I tried to implement the exact code in docs, but nothing happens. I have a basic database with items(over 1250 or more) and I want to get them progressively. By scrolling to load 15 items (to the last item in the database).

If using docs code:

// Construct query for first 25 cities, ordered by population
Query first = db.collection("cities")
    .orderBy("population")
    .limit(25);

first.get()
    .addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
    @Override
    public void onSuccess(QuerySnapshot documentSnapshots) {
        // ...

        // Get the last visible document
        DocumentSnapshot lastVisible = documentSnapshots.getDocuments()
            .get(documentSnapshots.size() -1);

        // Construct a new query starting at this document,
        // get the next 25 cities.
        Query next = db.collection("cities")
            .orderBy("population")
            .startAfter(lastVisible)
            .limit(25);

        // Use the query for pagination
        // ...
    }
});

How to do? Documentation has not too many details.

PS: I need with recycler view (not list view) when user scrolls. Thanks

Solution 1

As it is mentioned in the official documentation, the key for solving this problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You'll be able to use the last document in a batch as the start of a cursor for the next batch.

To solve this pagination problem, please see my answer from this post, in which I have explained step by step, how you can load data from a Cloud Firestore database in smaller chunks and display it in a ListView on button click.

Solution:

To get the data from your Firestore database and display it in smaller chunks in a RecyclerView, please follow the steps below.

Let's take the above example in which I have used products. You can use products, cities or whatever you want. The principles are the same. Assuming that you want to load more products when user scrolls, I'll use RecyclerView.OnScrollListener.

Let's define first the RecyclerView, set the layout manager to LinearLayoutManager and create a list. We also instantiate the adapter using the empty list and set the adapter to our RecyclerView:

RecyclerView recyclerView = findViewById(R.id.recycler_view);
recyclerView.setLayoutManager(new LinearLayoutManager(this));
List<ProductModel> list = new ArrayList<>();
ProductAdapter productAdapter = new ProductAdapter(list);
recyclerView.setAdapter(productAdapter);

Let's assume we have a database structure that looks like this:

Firestore-root
   |
   --- products (collection)
         |
         --- productId (document)
                |
                --- productName: "Product Name"

And a model class that looks like this:

public class ProductModel {
    private String productName;

    public ProductModel() {}

    public ProductModel(String productName) {this.productName = productName;}

    public String getProductName() {return productName;}
}

This how the adapter class should look like:

private class ProductAdapter extends RecyclerView.Adapter<ProductViewHolder> {
    private List<ProductModel> list;

    ProductAdapter(List<ProductModel> list) {
        this.list = list;
    }

    @NonNull
    @Override
    public ProductViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
        View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.item_product, parent, false);
        return new ProductViewHolder(view);
    }

    @Override
    public void onBindViewHolder(@NonNull ProductViewHolder productViewHolder, int position) {
        String productName = list.get(position).getProductName();
        productViewHolder.setProductName(productName);
    }

    @Override
    public int getItemCount() {
        return list.size();
    }
}

The item_product layout contains only one view, a TextView.

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:id="@+id/text_view"
    android:textSize="25sp"/>

And this is how the holder class should look like:

private class ProductViewHolder extends RecyclerView.ViewHolder {
    private View view;

    ProductViewHolder(View itemView) {
        super(itemView);
        view = itemView;
    }

    void setProductName(String productName) {
        TextView textView = view.findViewById(R.id.text_view);
        textView.setText(productName);
    }
}

Now, let's define a limit as a global variable and set it to 15.

private int limit = 15;

Let's define now the query using this limit:

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference productsRef = rootRef.collection("products");
Query query = productsRef.orderBy("productName", Query.Direction.ASCENDING).limit(limit);

Here is the code that also does the magic in your case:

query.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
    @Override
    public void onComplete(@NonNull Task<QuerySnapshot> task) {
        if (task.isSuccessful()) {
            for (DocumentSnapshot document : task.getResult()) {
                ProductModel productModel = document.toObject(ProductModel.class);
                list.add(productModel);
            }
            productAdapter.notifyDataSetChanged();
            lastVisible = task.getResult().getDocuments().get(task.getResult().size() - 1);

            RecyclerView.OnScrollListener onScrollListener = new RecyclerView.OnScrollListener() {
                @Override
                public void onScrollStateChanged(RecyclerView recyclerView, int newState) {
                    super.onScrollStateChanged(recyclerView, newState);
                    if (newState == AbsListView.OnScrollListener.SCROLL_STATE_TOUCH_SCROLL) {
                        isScrolling = true;
                    }
                }

                @Override
                public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
                    super.onScrolled(recyclerView, dx, dy);

                    LinearLayoutManager linearLayoutManager = ((LinearLayoutManager) recyclerView.getLayoutManager());
                    int firstVisibleItemPosition = linearLayoutManager.findFirstVisibleItemPosition();
                    int visibleItemCount = linearLayoutManager.getChildCount();
                    int totalItemCount = linearLayoutManager.getItemCount();

                    if (isScrolling && (firstVisibleItemPosition + visibleItemCount == totalItemCount) && !isLastItemReached) {
                        isScrolling = false;
                        Query nextQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).startAfter(lastVisible).limit(limit);
                        nextQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                            @Override
                            public void onComplete(@NonNull Task<QuerySnapshot> t) {
                                if (t.isSuccessful()) {
                                    for (DocumentSnapshot d : t.getResult()) {
                                        ProductModel productModel = d.toObject(ProductModel.class);
                                        list.add(productModel);
                                    }
                                    productAdapter.notifyDataSetChanged();
                                    lastVisible = t.getResult().getDocuments().get(t.getResult().size() - 1);

                                    if (t.getResult().size() < limit) {
                                        isLastItemReached = true;
                                    }
                                }
                            }
                        });
                    }
                }
            };
            recyclerView.addOnScrollListener(onScrollListener);
        }
    }
});

In which lastVisible is a DocumentSnapshot object which represents the last visible item from the query. In this case, every 15'th one and it is declared as a global variable:

private DocumentSnapshot lastVisible;

And isScrolling and isLastItemReached are also global variables and are declared as:

private boolean isScrolling = false;
private boolean isLastItemReached = false;

---

If you want to get data in realtime, then instead of using a get() call you need to use addSnapshotListener() as explained in the official documentation regarding listening to multiple documents in a collection. More information you can find the following article:

• How to create a clean Firestore pagination with real-time updates?

Solution 2

FirebaseUI-Android also recently came out with a Firestore Paginator.

I have used it in my code, and it works great - just keep in mind that it operates using .get() instead of .addSnapshotListener(), so the recycler is not in realtime.

See the docs here:

https://github.com/firebase/FirebaseUI-Android/tree/master/firestore#using-the-firestorepagingadapter

Solution 3

You can also use FirestorePagingAdapter provided by Firebase-UI-Firestore

You need to install this dependency

 implementation 'com.firebaseui:firebase-ui-firestore:latest_version_here'

Solution

Step 1: Create a global Firestore paging adapter variable and pass the Model class and ViewHolder, and also the Model variable.

private FirestorePagingAdapter<Model, ModelViewHolder> adapter;
private Model model;

Step 2: Create a firebase query

Query query = db.collection("cities")
    .orderBy("population");

Step 3: Let's build the pagedlist config. Here you will pass how much data to be queried in each page;

PagedList.Config config = new PagedList.Config.Builder()
                .setEnablePlaceholders(false)
                .setPrefetchDistance(10)
                .setPageSize(15)
                .build();

Step 4: After setting the config, let's now build the Firestore paging options where you will pass the query and config.

 FirestorePagingOptions<Model> options = new FirestorePagingOptions.Builder<Model>()
                .setLifecycleOwner(this)
                .setQuery(query, config, snapshot -> {
                    model = snapshot.toObject(Model.class);
                    return model;
                })
                .build();

Step: 5 Now let's pass the data to the Recylerview

 adapter = new FirestorePagingAdapter<Model, ModelViewHolder>(options) {
            @Override
            protected void onBindViewHolder(@NonNull ModelViewHolder holder, int position, @NonNull Model model) {
                holder.bindTO(model);
            }

            @NonNull
            @Override
            public ModelViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
                View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.layout_model, parent, false);
                return new ModelViewHolder(view);
            }

            @Override
            protected void onError(@NonNull Exception e) {
                super.onError(e);
                     //logic here
            }

            @Override
            protected void onLoadingStateChanged(@NonNull LoadingState state) {
                switch (state) {
                    case LOADING_INITIAL:
                        break;

                    case LOADING_MORE:
                        break;

                    case LOADED:
                        notifyDataSetChanged();
                        break;

                    case ERROR:
                        Toast.makeText(requireActivity(), "Error", Toast.LENGTH_SHORT).show();
                         //logic here
                        break;

                    case FINISHED:
                        //logic here
                        break;
                }
            }
        };
        productRecycler.setAdapter(adapter);
        adapter.notifyDataSetChanged();
    }

Happy Coding!

Comments

[Alex Mamo]

I think you might be interested in this article, How to paginate Firestore using Paging 3 on Android?.

[Johans Bormman]

I see your post but is with list view and i want with recycler view and scroll. Can you help me with that? I edited my post.

[Alex Mamo]

As I said, the records are displayed in a ListView but that was just an example. You can achieve the same thing using a RecyclerView. Have you even try it?

[Johans Bormman]

I have tried many examples, docs and i'm stuck for days. Help me please. You have many good posts.

[Alex Mamo]

Ok, I'll adapt you the code from there but instead of using a ListView, I'll use a RecyclerView.

[Otieno Rowland]

For some reason, I think you should use the new Android Paging Library: developer.android.com/topic/libraries/architecture/paging as it does not care whether you are using the RecyclerView/ListView. I would give it a try, as it seems to be the future as far as pagination in Android is concerned. @JohansBormman

[Johans Bormman]

@RowlandMtetezi I'm trying now to implemt this code. Do you have an example for that?

[Alex Mamo]

@RowlandMtetezi You're right but I saw in the link you provided, the documentation is only for Kotlin. The OP asked for Android. Thanks!

[Otieno Rowland]

The example I have is Paging with a network sample: github.com/googlesamples/android-architecture-components/tre‌​e/… , I have yet to see ane example working with firebase or firestore. However, I think you can easily adapt/borrow the concept.

[Otieno Rowland]

@AlexMamo does not Kotlin write Android apps as early as 2016? Kotlin is first class, just like Java.

[Alex Mamo]

@RowlandMtetezi Ya, you're right. I haven't seen the Kotlin tag in OP's question, so I wrote the code in Android. Thanks!

[Johans Bormman]

@RowlandMtetezi Thanks for the informations. I don't know kotlin but as soon as I learn, I give it a try. I'll stay for the moment with Java.

[Johans Bormman]

What to say Alex, it took me 3 hours but I finally made it. Thank you by the way!

[gautam]

I am trying to add it from many hours but isscrolling is always false even after scrolling in real device and onScrollStateChanged is not calling at all , please help

[gautam]

Found the problem it is with nested scrolling, anyone know how to fix it?

[Zar E Ahmer]

@AlexMamo plz have a look at stackoverflow.com/questions/52021119/…

[AdamHurwitz]

The FirebaseUI-Android component is quick and easy to get up and running. If you need to do more advanced queries or filtering that Firebase does not allow you can create your own DataSource with a PagedListAdapter as I outlined in my answer here. stackoverflow.com/questions/51859652/…

[Yogesh Rathi]

I converted same code in kotlin, and run then first query works properly second time app crashed with java.lang.RuntimeException: No properties to serialize found on class com.google.firebase.firestore.A this error

[Alex Mamo]

@YogeshRathi Please post another fresh question regarding this issue, so me and other Firebase developers can help you.

[Yogesh Rathi]

@AlexMamo please look into this issue stackoverflow.com/q/56964430/5096868

[Siddhesh Dighe]

I tried using this, and its probably the most quick way of doing it. But Pagination with FirebaseUI doesn't seem like the right way if you have +1000 plus documents. It loads all the documents at once with a .get() query and then paginates it locally. Any idea how can we make this more dynamic? fetching only 10-20 records at once?

[Jeff Padgett]

@SiddheshDighe Why do you think it loads all the documents at once? I think it only loads one page at a time, and you set the size of the page. See: github.com/firebase/FirebaseUI-Android/blob/master/firestore‌​/…

[Qutbuddin Bohra]

how to get post id means document id i put limit 3, so task give me only 3 documents id, but after 4th document i got error, which is i have only 3 documents , please help me about this, your post help me too much, thank you

[Alex Mamo]

@QutbuddinBohra Without seeing your code I cannot be much of a help. So please post another fresh question, so I and other Firebase developers can help you.

[Qutbuddin Bohra]

I posted new questions regarding how to get document id after next query run. link stackoverflow.com/questions/60247307/… please help me about this i also tried firebase pagging adapter ui but its doesnt show me live data.

[Qutbuddin Bohra]

@AlexMamo on your Real time data load work, after notifyDataSetChanged() on adapter its refresh whole item which under LIMIT. Help about how to not refresh things under its item, only refresh that item which is needed.

[Alex Mamo]

@QutbuddinBohra If you are interested in a Firestore real-time pagination algorithm you can take a look this example, which is a repo for this article.

[Kunchok Tashi]

Still looking fore firestore pagination, check this link techtibet.com/blog/android/…

[Alex Mamo]

@kontashi35 I'm sorry but I cannot see any pagination at all in that post.

[Kunchok Tashi]

@AlexMamo Thanks for that, file was missing.. added now

[Alex Mamo]

@kontashi35 I cannot see the reason why you added that article as it's exactly like my answer :|

[Kunchok Tashi]

@AlexMamo they are people who prefer code over explanation ,demo over theory and also the way we explain. I just thought it will be usedful to some one(even a person) .I will remove it if you found it not good.

[luke cross]

He need load the all documents or not? Because if he loads all documents it's not work fine...

[parohy]

For some reason, I am always getting empty result when using snapshot in startAfter. Is there a catch? I have a simple order as explained above. If I use the field value for the cursor it works, but when I put there a DocumentSnapshot it succeeds but the result documents are empty

[Alex Mamo]

@parohy Without seeing your code, I cannot be much of a help. So please post a new question using its own MCVE, so I and other Firebase developers can help you.

[AndroidRocket]

@AlexMamo could please take a look at this stackoverflow.com/questions/63295921/… .

[thinclient]

This approach is not good. While you get updates in real time, realise that when paginating the query you create a new query when fetching the next set of documents. You therefore reassign the old query handle to the new query. Removing the old event listeners is not possible when the lifecycle of the component(activity, fragment...) is done. We know that listeners should be removed when not needed to avoid unnecessary costs: bandwidth, battery and sever reads.

[Alex Mamo]

@thinclient This approach is not for getting updates in real-time. As you can see, it uses get(), which gets the data only once.

[thinclient]

@AlexMamo sorry I missed the get() part. I'm so used to attaching an event listener. I did read another answer of yours where there is a link to a medium article where you are creating a view model and attaching event listeners and actually getting real time updates. I will comment there.

[JustGotStared]

Im using FirestorePagingAdapter from FirebaseUI , and wrote a query which include whereEqualTo() but it doesnt seem to be working . The return data is not sorted even if I used whereEqualTo() is it receiving the complete dataset from firestore