A mobile first NoSQL database

I happened to be just browsing through Paxos made simple and trying to undersand the principles that have become the bedrock for clustering and horizontally scalable applications, including databases.

I then put on my mobile developer hat and couldn’t help but wonder what is the most common mobile app building pattern

  • An interface to collect data
  • A server side to store that data
  • A client side cache to store frequently read data

At the surface this is how it looks. However, if we dig a bit deeper into the data interchange and storage some huances emerge

A write operation goes like this:

  • At the client, data is converted from native objects (Swift/Java/Kotlin) to JSON
  • JSON is transmitted to a web service over a network
  • Web service deserializes JSON to native langage objects
  • Web application/service in turn converts the objects to the datbase native
  • format (Could be JSON depending on the database)

A read operation goes like this

  • If data is in client side cache (SQLite etc)
    • Client reads, converts to native object serves the data
  • Else
    • Client sends a HTTP Get
    • Web service retrieves the data from the database/cache. Converts from DB
    • format (tables, joins whatever) to language objects
    • Converts the language objects to JSON
    • Serves the JSON over internet
    • Client converts JSON back into native objects, stores them in cache, and
    • and serves them in the application.

This is still a 10,000 ft view but I already see so many inefficiencies namely:

  • Too many resources spent in data format conversion (native objects -> JSON ->
  • server language -> DB format and vice versa
  • The networking calls related to storing data need to be handled by the
  • application. Hence a plethora of boiler plate code for POST/GET, both asnyc
  • and synchronous variants.
  • Not to mention the GET/POST latencies
  • Non standard client side caching - SQLite is pretty much the only option.
  • Need for a backend just to store data. Again, not very convenient.

So, here is my thought. What if there is a mobile first native database that mobile apps write to directly on the client. Something like couchbase, but this mobile database is ‘one unit’ of a cluster deployed in the cloud.

Data written into the app is replicated in the cloud without any intervention from the developer. This is where Paxos and other clustering algorithms come into play. Seamless data replication across the cloud. The mobile app doesn’t even know the cloud exists but the data is available for the developer. So for example - if the client has to store the profile data

  • Store the profile JSON {id: blah, name: blah} at the client database.
  • DB replicates this object to the cloud.
  • Object is available on the cloud as JSON for querying.
  • Clients always read and write locally - meaning all writes go to their phone
  • first
  • In addition, data is always available at the cloud for querying and analytics
  • or machine learning or whatever that needs to be done at a server level.
  • It will also be possible to transform and write this data back into the
  • database at the server/cloud.
  • Client reads/writes are always local. In clustering terms, the master is
  • always on the local node.
  • A true multi-tenant system for every mobile developer, a separate datbase for
  • each app, a database that grows and shrinks as apps and data grows.

Clustering

  • Yes, each mobile device will be a node on the cluster. So we are talking about millions of logical nodes per application.
  • Depending on the class of applications, data from other nodes should be
  • replicated on the client node. For ex: Social applications.
  • It is a good idea to not support this class of applications to start with.
  • This is just like Parse (which got shut down after FB aquisition) but with
  • added convenience.

Most important

  • Will I install this database separately or will this be included in every
  • application like cocoapods ?

Target market:

Solo mobile app developers, idea stage startups.

Pricing

Free - Upto 1 GB

Written on June 17, 2020