A blockchain centric key-value store

Terab is a key-value store, but a rather specific one. The Terab API comes with a blockchain centric perspective on the UTXO: queries made to the UXTO are performed passing a block as context. However, the semantic isn’t that the query is restricted to a specific block - which would be largely useless; the semantic is that the query is performed against the specific version of the UTXO associated to the chaintip defined by the block.

Thus, Terab exposes a versioned vision of the UTXO dataset. This approach differs significantly from the block-do & block-undo mechanism originally adopted by Satoshi’s client. As an approach block-do & block-undo scales poorly: it concentrates all the I/O workload to process a block at the very point of time when a block is propagated through the network, that is, once every 10 mins on average. Worse, if a chain reorganization happens then the I/O workload is twice as heavy as processing an incoming block.

The approach adopted by Terab emphasizes a gradual construction of the next block as transactions are successfully propagated. Indeed, there is no point in waiting 10 mins to start committing to persistent storage, the UTXO entries are almost garanteed_ to be produced and consumed by the next block. With Terab, the next UTXO version is incrementally constructed as transactions are observed. When a block is successfully found, a diff 1 can be applied to add and remove entries in order to finalize the next UTXO version.

Compared to block-do & block-undo, the versioned key-value store approach reduces the I/O pressure on the underlying data storage layer by a factor of 1000, assuming it is desirable to keep the commit block operation under 0.5 seconds 2.

In addition, chain reorgs happen much more gracefully, as the overhead for extending a particular tip of the chain or another is constant, it does not depend on the size of the block. Under the hood in Terab, all blocks having the same blockheight equally compete for the same underlying I/O resources.

  1. The recent advance in block propagation techniques such as Graphene are part of a broad class of algorithms know as set reconciliation algorithms. Terab has been explicitly designed to make sure that the benefits of these techniques permeate the UTXO as well. [return]
  2. Block publishers (e.g. mining pools) have a financial incentive to perform commit block as fast as possible, as it decreases the risk of having their block orphaned by a competing chain. [return]