implement protocol 2.0; change db schema #101
Open
Commits on Oct 20, 2021
-
-
-
-
history db: sort key insertion order
Supposedly it makes a difference (see e.g. [0]), and depending on how batching works it makes sense it would, but during a few full syncs of testnet I've done, it was within measurement error. Still, existing code was already doing this. [0]: https://stackoverflow.com/q/54941342
-
-
with the pending db changes, an upgrade is ~as fast as a resync from genesis
-
session: blockchain.transaction.get_merkle: make "height" optional
now that we have our own txindex
-
history db: store a (prev_txnum, prev_idx)->spender_txnum map
This will allow looking up which tx spent an outpoint.
-
-
-
db: change TXOUTIDX_LEN from 4 to 3, to save db storage size
In Bitcoin consensus, a txout index is stored as a uint32_t. However, in practice, an output in a tx uses at least 10 bytes (for an OP_TRUE output), so - to exhaust a 2 byte namespace, a tx would need to have a size of at least 2 ** 16 * 10 = 655 KB, - to exhaust a 3 byte namespace, a tx would need to have a size of at least 2 ** 24 * 10 = 167 MB.
-
session: implement "blockchain.outpoint.subscribe" RPC
notifications not implemented yet
-
"blockchain.outpoint.subscribe" RPC: distinguish heights "-1" and "0"
Similar to scripthash statuses, the height of an unconfirmed tx is: - `-1` if it has any unconfirmed parents, - `0` otherwise.
-
session: wait for protocol ver negotiation to finish before other reqs
We already enforce that server.version must be the first received message in a session, but we also need to ensure that the server finishes processing that message and sets up the correct protocol version before starting to process further messages.
-
(bugfix) db: change tx_num endianness (LE->BE) to match db comparator
History.get_txnums and History.backup depend on ordering of tx_nums, so we want the lexicographical order (used by leveldb comparator) to match the numerical order.
-
protocol change: define order of mempool txs in status of scripthash
Note that this is a soft fork: the server can apply it even for past protocol versions. Previously, with the order being undefined, if an address had multiple mempool transactions touching it, switching between different servers could result in a change in address status simply as a result of these servers ordering mempool txs differently. This would result in the client re-requesting the whole history of the address. ----- D/i | interface.[electrum.blockstream.info:60002] | <-- ('blockchain.scripthash.subscribe', ['660b44502503064f9d5feee48726287c0973e25bc531b4b8a072f57f143d5cd0']) {} (id: 12) D/i | interface.[electrum.blockstream.info:60002] | --> 9da27f9df91e3f860212f65b736fa20a539ba6e3d509f6370367ee7f10a4d5b0 (id: 12) D/i | interface.[electrum.blockstream.info:60002] | <-- ('blockchain.scripthash.get_history', ['660b44502503064f9d5feee48726287c0973e25bc531b4b8a072f57f143d5cd0']) {} (id: 13) D/i | interface.[electrum.blockstream.info:60002] | --> [ {'fee': 200, 'height': 0, 'tx_hash': '3ee6d6e26291ce360127fe039b816470fce6eeea19b5c9d10829a1e4efc2d0c7'}, {'fee': 239, 'height': 0, 'tx_hash': '9e050f09b676b9b0ee26aa02ccee623fae585a85d6a5e24ecedd6f8d6d2d3b1d'}, {'fee': 178, 'height': 0, 'tx_hash': 'fb80adbf8274190418cb3fb0385d82fe9d47a844d9913684fa5fb3d48094b35a'}, {'fee': 200, 'height': 0, 'tx_hash': '713933c50b7c43f606dad5749ea46e3bc6622657e9b13ace9d639697da266e8b'} ] (id: 13) D/i | interface.[testnet.hsmiths.com:53012] | <-- ('blockchain.scripthash.subscribe', ['660b44502503064f9d5feee48726287c0973e25bc531b4b8a072f57f143d5cd0']) {} (id: 12) D/i | interface.[testnet.hsmiths.com:53012] | --> f7ef7237d2d62a3280acae05616200b96ad9dd85fd0473c29152a4a41e05686c (id: 12) D/i | interface.[testnet.hsmiths.com:53012] | <-- ('blockchain.scripthash.get_history', ['660b44502503064f9d5feee48726287c0973e25bc531b4b8a072f57f143d5cd0']) {} (id: 13) D/i | interface.[testnet.hsmiths.com:53012] | --> [ {'tx_hash': '9e050f09b676b9b0ee26aa02ccee623fae585a85d6a5e24ecedd6f8d6d2d3b1d', 'height': 0, 'fee': 239}, {'tx_hash': 'fb80adbf8274190418cb3fb0385d82fe9d47a844d9913684fa5fb3d48094b35a', 'height': 0, 'fee': 178}, {'tx_hash': '3ee6d6e26291ce360127fe039b816470fce6eeea19b5c9d10829a1e4efc2d0c7', 'height': 0, 'fee': 200}, {'tx_hash': '713933c50b7c43f606dad5749ea46e3bc6622657e9b13ace9d639697da266e8b', 'height': 0, 'fee': 200} ] (id: 13) -
DB storage: implement iterator.seek(key)
in a way that works consistently between LevelDB and RocksDB.
-
"scripthash.get_history": impl updates for proto 1.5; no client_* yet
Handling of client_statushash and client_height is not yet implemented.
-
session: rework statushash calculation.
If a client requested the status of a very busy address (with cold cache: no precalc yet), we might have gotten a timeout before we stored any intermediate status calculated. Then, if the client reconnected and requested the same, we would get stuck in this loop and never make progress. With this change, we try to store intermediate hashes sooner, so that even if there is a timeout, if the client reconnects there is less work to be done.
-
-
-
-
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.