Libbitcoin Database
The libbitcoin-database library is a dependency of libbitcoin-blockchain, libbitcoin-node and libbitcoin-server.
Contents
Example
#include <string> #include <bitcoin/database.hpp> // Create a new mainnet blockchain store. int main(int argc, char* argv[]) { std::string prefix("blockchain"); const auto genesis = bc::chain::block::genesis_mainnet(); if (argc > 1) prefix = argv[1]; if (!bc::database::data_base::initialize(prefix, genesis) return -1; return 0; } Design
The database is built directly on the operating system’s memory-mapped file system. Logical queries are performed using a set of hash tables. The number of hash buckets is optimized to minimize hash collisions, though collisions are accommodated. These changes resulted in a substantial performance increase, near constant time, for queries against the blockchain. Insert performance was not materially affected. The database uses sequence locking to avoid blocking the writer. This is ideal for a high performance blockchain server as reads are significantly more frequent than writes and yet writes must proceed without delay.
Database
The following files constitute the blockchain database non-volatile storage for a node. As of height 440,000 these consume approximately 102 GB of disk space.
- block_index
- block_table
- transaction_table
The following files constitute the non-volatile storage for the server index.
- history_table
- history_rows
- stealth_rows
- spend_table
The following files are used for locking.
- flush_lock
- exclusive_lock
Considerations
- There is no mechanical hard drive optimization. The implementation is intended for solid state drives (SSD).
- There is a likelihood of index corruption during hard shutdown. In version 2 there is no means of detecting corruption aside from catastrophic fault. However given that the entire blockchain is a cache this is not considered significant. Repair can be accomplished by re-synchronizing the blockchain. Version 3 provides robust detection of corruption and the option to flush after each write, making corruption very unlikely (and still fully detectable) if enabled.
- Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.
- The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing a version 2 server can become continuously unresponsive to requests. This is not a problem for version 3.
History
- The original implementation used LevelDB.
- In late 2014 by Amir Taaki (genjix) replaced LevelDB with memory-mapped hash table storage and indexing.
- In early 2016 Eric Voskuil separated the database from libbitcoin-blockchain into an independent library and used isolated locking to make the concurrent read-while-write safe and to enable concurrent writes.
Twitter
Telegram
