Yugabyte PM here. Yugabyte SQL query layer is a fork of PostgreSQL 11.2's query layer. This query layer runs on DocDB, a distributed document store whose sharding, replication and ACID transactions architecture is inspired by Google Spanner.
Given that the query layer reuses the native PostgreSQL code, it is wire compatible with the PostgreSQL binary protocol and already supports an extensive set of PostgreSQL language features (including partial indexes, triggers and stored procedures). Most tools from the PostgreSQL ecosystem work without any modification. However, PostgreSQL is an extensive language so claiming 100% coverage is not accurate . For example, GIN and GIST indexes are not yet supported since more work is needed to integrate those language constructs to the underlying distributed storage layer. The more accurate answer would be that YugabyteDB has the best PostgreSQL compatibility in the realm of distributed databases and continues to increase coverage with new releases.
Yugabyte PM here. The goal is to demonstrate that even SQL databases can now be scaled to 1M inserts per second, a feat that was previously reserved for NoSQL databases.
This is not the case. There are in fact many SQL capable databases which showed such results, Clustrix and MySQL Cluster (NDB) among them. Here is link to benchmark (to actually rather old version) https://www.mysql.com/why-mysql/benchmarks/mysql-cluster/
Postgres was first written in Lisp and had to be re-written in C because of slow performance [1]. YugaByte DB reuses the same C-based Postgres query layer but runs it on a Google Spanner-inspired distributed document store written in C++. The goal of this new “rewrite” is to add horizontal write scaling, native failover/repair and geo-distribution. As long as users love the language they interact with, the underlying software will continue to see such re-writes in order to meet the needs of the current times.
YugaByte product manager here -- have documented the answer to your question [1] As you can see, there are many similarities but there are also a few important differences such as depth of PostgreSQL compatibility (YB reuses PostgreSQL query layer while CRDB is a re-implementation) and latency/throughput observed for modern OLTP workloads.
True, but note that the comparison only focuses on SQL (as it related to PostgreSQL) features and not any DB-specific features.
The YugaByte DB specific pieces are not included as well - for example, support for YCQL (Cassandra-compatible) and YEDIS (Redis-compatible) APIs to the DB.
Thanks for sharing these. Hadn’t heard of the first 2, so look fwd to reading them. We did review the Amazon Dynamo architecture (which is used in Dynomite) in depth but found it to be lacking for supporting even single-row linearizability. multi-row ACID with serializable isolation builds on top that property. These are must-haves in a storage architecture to power a distributed SQL API.
assume you are referring to https://en.wikipedia.org/wiki/NonStop_SQL -- was not aware of it so thanks for bringing it to attention. first impression is that it was a technology way ahead of its times. in late 80s (and some would argue even recently), majority of apps can be run easily on monolithic/single-node SQL databases. they are not generating data at a volume that requires multiple nodes (in this case parallel computers) and user perception can indeed tolerate high latencies, manual failover and repair. we are now in a different era when it comes to data volume and user perception.
happy that you liked the post. as you could infer, this simplicity is by no means a magic bullet. there are always trade-offs depending on the database type you use as baseline for comparison. we highlight them here: https://docs.yugabyte.com/latest/introduction/#what-are-the-...
regarding the tone of the post, we are always open to feedback on how we can do better. let us know through any means possible incl. github and slack.
YugaByte DB product manager here. we have compared the Spanner and Calvin architectures in depth previously (https://blog.yugabyte.com/google-spanner-vs-calvin-global-co...). one key difference comes from the fact that Calvin’s deterministic locking protocol requires advance knowledge of all transactions’ read/write sets before transaction execution can begin -- this leads to repeated transaction restarts if the secondary index is on a column whose value is changing frequently as well as no support for client-initiated sessions transactions. in other words, Calvin is better suited for a transactional NoSQL database as opposed to a true distributed SQL database.
Under contention, a Calvin-based system will behave similarly to others which use optimistic locking schemes for Serializable isolation such as Postgres, or YB itself. There are advantages to the Calvin approach as well. For example, under Calvin, the system doesn't have to write out speculative intents to all data replicas in order to detect contention: The only writes to data storage are successful ones. The original paper only describes this briefly, but you can read about how FaunaDB has implemented it in more detail: https://fauna.com/blog/consistency-without-clocks-faunadb-tr...
It's also not a stretch to see how the protocol described in that post can be extended to support session transactions: Rather than executing a transaction per request, the transaction context is maintained for the life of the session and then dispatched to Calvin on commit. (This is in fact how we are implementing it in our SQL API feature.)
I would instead say that one of the more significant differences between Calvin and Spanner is the latter's much stricter requirements it places on its hardware (i.e. clock accuracy) in order to maintain its correctness guarantees; a weakness its variants also share.
Only if things were that simple :) Calvin avoids the need to track clock accuracy by making every transaction go through a single consensus leader which inherently becomes a single point of bottleneck for performance and availability. Spanner and its derivatives including YugaByte DB chose not to introduce such a bottleneck in their architecture with the use of per-shard consensus — this means caring about clock skews for multi-shard transactions and not allowing such transactions to proceed on nodes that exhibit skew more than the max configured. The big question is which is more acceptable: lower performance & availability on the normal path OR handling offending nodes with high clock skew on the exceptions path?
Sorry I can't let this go unchallenged. Again, you are inventing an architectural deficiency where the is none. The log in Calvin is partitioned and does not require all transactions to go through a single physical consensus leader. There is no single node in a Calvin cluster which must handle the entire global stream of transactions. The Calvin paper itself extensively covers how this works in detail: http://cs.yale.edu/homes/thomson/publications/calvin-sigmod1...
YugaByte DB product manager here. Yes, CosmosDB and its underlying architecture were indeed not yet publicly available when we started the YugaByte DB project early 2016. However, classifying CosmosDB as a distributed SQL database is a bit of stretch given no support for SQL on the write path (the write path uses a custom document API). This means no support for multi-row/multi-shard ACID transactions as well as features such as client-initiated session transactions.
YugaByte product manager here. The YCQL API which passes Jepsen has its roots in Cassandra Query Language but does not use Cassandra as its backend store. It’s backend store is DocDB, which is a Google Spanner-inspired distributed document store.
Given that the query layer reuses the native PostgreSQL code, it is wire compatible with the PostgreSQL binary protocol and already supports an extensive set of PostgreSQL language features (including partial indexes, triggers and stored procedures). Most tools from the PostgreSQL ecosystem work without any modification. However, PostgreSQL is an extensive language so claiming 100% coverage is not accurate . For example, GIN and GIST indexes are not yet supported since more work is needed to integrate those language constructs to the underlying distributed storage layer. The more accurate answer would be that YugabyteDB has the best PostgreSQL compatibility in the realm of distributed databases and continues to increase coverage with new releases.