The issue:

For operations involving data collection in areas without mobile coverage, the key challenge was ensuring that field agents could collect and store critical data offline and later synchronise it seamlessly when back online. Initially, the workflow required constant connectivity, which wasn’t feasible in remote locations, leading to disruptions in data gathering and risk of data loss.

How we solved it:

We implemented a solution that allowed field teams to operate fully offline, storing data locally on their devices. The system was designed to seamlessly transition to online mode, automatically syncing data with the central database once a connection was re-established. By ensuring a streamlined and secure workflow for offline data collection, this solution minimised the risk of data loss due to damaged or lost devices and improved the overall efficiency of the data collection process.

The tech used:

Our solution for offline data access focused on ensuring reliability, scalability, and seamless integration with existing infrastructure, making it robust for field operations where connectivity is inconsistent.

• Local storage with SQLite: Field officers’ devices store data locally using SQLite, a lightweight database ideal for mobile or offline-first applications. This ensures fast, reliable access to large datasets without the need for constant server communication.
• Efficient synchronisation: We implemented a differential sync process, syncing only changed data (delta-sync). The system automatically detects changes, ensuring minimal data transfer and reducing the load on both device and server once back online.
• Conflict resolution & data merging: The server holds the “point of truth,” and uses record-based synchronisation to avoid conflicts. Globally unique identifiers (GUIDs) track records, while transaction and sync IDs ensure traceability and transparency. This minimises conflicts by clearly identifying which version of a record is the latest, allowing the system to merge changes confidently.
• Data integrity & audibility: Audit tables and logging mechanisms were implemented to track every sync session and ensure accountability. This allows partial rollbacks, meaning only erroneous data can be reversed without affecting the entire dataset, increasing overall system resilience.
• Security measures: Data is encrypted both at rest and in transit, ensuring compliance with industry security standards. Offline data is encrypted locally, preventing unauthorised access in the event of device loss or theft.

By leveraging a mix of local storage, efficient sync mechanisms, and strong data management practices, we ensured that field officers could operate efficiently in any environment, with a system that scales and adapts to real-world challenges in the field.