Scaling Limitations of Real-Time Databases

Scaling Limitations of Real-Time Databases

created time
Apr 16, 2022 04:27 AM
last update
Apr 16, 2022 05:06 AM

What & Why

A good survey on scaling limitations of real-time databases by Wolfram Wingerath: https://medium.baqend.com/real-time-databases-explained-why-meteor-rethinkdb-parse-and-firebase-dont-scale-822ff87d2f87 (detailed HOWs), excerpts & summary tables are from the post:
Real-time databases make it easy to implement reactive applications, because they keep your critical information up-to-date. But how do they work and how do they scale? In this article, we dissect the real-time query features of Meteor, RethinkDB, Parse and Firebase to uncover scaling limitations inherent to their respective designs. We then go on to discuss and categorize related real-time systems and share our lessons learned in providing real-time queries without any bottlenecks in  Baqend.
notion image
Wolfram went into details of how different databases implement real-time features. Change feeds are events that apps can subscribe to process changes to data in real-time, instead of polling at intervals. The entry is dated (2017, some of the databases may have changed for the better) but most of the considerations he presented are relevant today.
I am using https://orientdb.org (multi-modal database acquired by SAP) for Free Endpoint Defense & Response backend. Beyond using it both as noSQL collection (a bunch of system-event-log entries) & for graph analysis (how are these events are related), almost all real-time databases support Live Queries or what is known as Change Feeds (e.g. response to anomalies). I was motivated to read his post because I want to know what are the options & other considerations when it comes to real-time processing.