How to sync flutter app's offline data with online database efficiently

Question

lets say I have a todo app that stores the data in sqflite database(locally on the phone) when the app goes online I want the data to be synced with my online database say mongodb or firestore.I don't want to do complete overwrites or creating the new table everytime,I am looking for some efficient solution that only updates the changes to the database.

Solution 1

Firebase Cloud Firestore makes all of this for you. If You can't use this, other solution is much more complicated as syncing information can be very tricky.

Imagine the user has 2 devices, both saving the same data, which version should the server keep, the newest created data, or the data the server received first?

SQLite is good for structured data, but you could save your data as a JSON string using System Preferences and just push that JSON to the server, it's a lot simpler if can do it this way.

Again, this really not a simple answer to give and varies from project to project.

Solution 2

I think this will help to construct offline and online sync：

flutter_offline

http

sqlite

Solution 3

As already pointed out, this is an architectural question and there's probably no simple answer. Here are two main thoughts :

Scenario A : You are just updating the transaction data and sequence doesn't matter e.g. mobile attendance app. The user's location along with date time stamp is recorded on the server. And server does everything else (validation, processing, reporting).

Scenario B : Sequence does matter. If you just keep the offline connectivity then older data may overwrite newer data in the following like scenario : user has two devices. the device having new data comes online first followed by the device with old data

I'd dealt with both of the scenarios and for scenario B, I did the following :

1. Created a 'queue' table.
2. All the entries first goes to the queue (online or offline)
3. The inserts in the queue entry triggers Firebase Cloud function. The function does the validations and processing (example below).
4. The function makes the entry in the actual table.

Step 3 is particularly useful when working with apps such as Todo. It is because you might want to log the event (say in the log file) but not to udpate the todo item status if older data comes later.

Comments

[Aritra Dattagupta]

How about saving the data in local storage with date/timestamp and server stores the data by sorting according to timestamp? So even if the user has multiple devices the data would always be in correct sequence.

[Sukhi]

There are more than one way of using a local storage and keep the data in sync with the server. It depends on the project requirement and the data setup. The important question is - do you want to take a pain and write all the code, test it or use a proven functionality from Google (or like) !

[Aritra Dattagupta]

Many projects wouldn't want to use Firebase/AppSync because of the related cost. I am using Hasura GraphQL and custom node server on Digitalocean which is way cheaper than AWS or Firebase with equal amount of performance.

[Didarul Alam Rahat]

is flutter_offline store data as a local DB?

[tabebqena]

It detects connectivity and change widgets accordingly.