Reposting as text, formatted HTML got moderated :/.
---
Weve spent a good amount of time on node, following a rewrite of validation
and script processing, and on the heels of the previous network redesign.
Im very happy with the results. We have a very stable node and excellent
block validation performance. We have been lucky to have the full time
support of Bitprim in operational testing of the node on both high and low
end platforms. You may recall that we previously achieved full chain sync of
about 15 minutes (all blocks checkpointed). We have since validated the full
mainnet chain countless times. Despite the fact that our networking is still
not optimal in terms of catch-up sync (blocks-first), we are able to sync
0-295,000 and validate 295,001-top (Core parameters) in about 7 hrs. I
expect that number to come down significantly in v4 owing to more advanced
networking.
Below is a sample of the new console reporting format for block acceptance,
on a high end server (SSD, 12 Cores, 128GB RAM). You can see that we are
processing at a rate of about 3-4 blocks per second at the top of the
current chain. But given network limitations this is not the full story. As
you can see our median time process a block is about 62.5 µs per input (1 µs
= one millionth of a second), or full block validation times (i.e. excluding
the empty blocks) averaging under .3 seconds. We have not tested on more
than 12 cores. The number of cores used (and validation thread priority) is
configurable, so we can easily see improvement in speed from 1-12 cores
presently. I expect those numbers would continue to improve with additional
cores up to some point of diminishing returns.
A libbitcoin node store is 3 files for a total of ~102GB. The files are
portable across machines. In other words 128GB RAM is more than sufficient
to hold the entire chain, reducing paging costs. However even an 8GB RAM
machine with 4 cores maintains healthy performance. We have not tested at
intermediate RAM levels (e.g. 64 or 32 GB) but these sizes are expected to
be very performant.
I should point out that libbitcoin does not use a UTXO database nor does it
cache UTXOs, so there is no problem with scale as the blockchain grows.
Per-input performance is linear across the full range of the chain
regardless of hardware (assuming a drive of sufficient space for the chain).
Outputs are read, and updated as spent, directly via transaction records.
The cost of all store reads, including outputs, activation state, block and
transaction duplicate checks, difficulty, and timestamps, is incurred all at
once and captured in the pµs value below (averaging 15 µs per input). The
cost of storing the new block, including spent output information, is
reflected in the dµs value below (averaging 5 µs per input). Reading and
writing independently from validation and other processing is not only huge
boost for testability but also dramatically simplifies code and exposes
important optimizations. For example, all per-input operations with the
exception of block check are parallelized either by tx or input.
Our script validation uses a fully-generalized script processor:
https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin
/impl/machine/interpreter.ipp
This makes Bitcoin script easy to understand/ and test, and safer to
maintain. Despite this high level of generalization compared to other
implementations, we are achieving a script validation cost of 41 µs per
input. This is 13.5% faster than we achieve using libconsensus (which can be
swapped in via a configuration option). Testing has confirmed that this is
entirely the consequence of the need to serialize each transaction and
previous output for that librarys interface, and because each must then be
deserialized internal to that library. As such libconsensus is not viable
for high performance implementations, such as mining, but the cost is small
enough to not adversely impact other scenarios.
e
Block Totals (milliseconds)
----------------------------------
txs: count of transactions
ins: count of inputs
wms: wait on previous block
vms: validate (*) time
Per Input (microseconds)
----------------------------------
vµs: validate (*)
rµs: deserialization (read from wire)
cµs: check block (context-free validation)
pµs: populate (all database reads)
aµs: accept block (contextual validation)
sµs: connect block (script verify)
dµs: deposit (all database writes)
(*) validate is the time from start of block-check to the end of
block-connect. This is the time that it takes to fully validate a block
object. This is important in that it is the time, after receipt and
deserialization (time also shown) of a (whole) block from a peer, that a
miner must wait until a notification of validity is available.
09:28:37.985756 INFO [blockchain] Block [441133] 2412 txs 4214 ins 188 wms
307 vms 73 vµs 1 rµs 5 cµs 19 pµs 2 aµs 48 sµs 5 dµs
09:28:38.295939 INFO [blockchain] Block [441134] 2691 txs 4863 ins 291 wms
286 vms 59 vµs 2 rµs 5 cµs 21 pµs 1 aµs 31 sµs 5 dµs
09:28:38.751393 INFO [blockchain] Block [441135] 2136 txs 4484 ins 270 wms
434 vms 97 vµs 1 rµs 4 cµs 26 pµs 2 aµs 65 sµs 5 dµs
09:28:38.966397 INFO [blockchain] Block [441136] 1918 txs 3528 ins 427 wms
198 vms 56 vµs 1 rµs 4 cµs 15 pµs 1 aµs 36 sµs 5 dµs
09:28:39.170047 INFO [blockchain] Block [441137] 1915 txs 3258 ins 185 wms
186 vms 57 vµs 2 rµs 5 cµs 13 pµs 2 aµs 37 sµs 5 dµs
09:28:39.474600 INFO [blockchain] Block [441138] 2367 txs 4197 ins 172 wms
284 vms 68 vµs 1 rµs 5 cµs 17 pµs 2 aµs 45 sµs 5 dµs
09:28:39.798699 INFO [blockchain] Block [441139] 2386 txs 4434 ins 270 wms
302 vms 68 vµs 1 rµs 4 cµs 18 pµs 1 aµs 45 sµs 5 dµs
09:28:40.161076 INFO [blockchain] Block [441140] 2150 txs 4648 ins 293 wms
342 vms 73 vµs 1 rµs 4 cµs 15 pµs 1 aµs 53 sµs 4 dµs
09:28:40.493881 INFO [blockchain] Block [441141] 1768 txs 4754 ins 325 wms
313 vms 66 vµs 2 rµs 4 cµs 19 pµs 1 aµs 42 sµs 4 dµs
09:28:40.688693 INFO [blockchain] Block [441142] 1860 txs 3726 ins 301 wms
178 vms 48 vµs 2 rµs 4 cµs 12 pµs 1 aµs 30 sµs 5 dµs
09:28:40.977133 INFO [blockchain] Block [441143] 2737 txs 4598 ins 161 wms
265 vms 58 vµs 1 rµs 5 cµs 18 pµs 1 aµs 34 sµs 5 dµs
09:28:41.295907 INFO [blockchain] Block [441144] 1530 txs 4592 ins 251 wms
301 vms 66 vµs 1 rµs 4 cµs 16 pµs 1 aµs 45 sµs 4 dµs
09:28:41.539977 INFO [blockchain] Block [441145] 1625 txs 3637 ins 282 wms
228 vms 63 vµs 1 rµs 4 cµs 10 pµs 1 aµs 47 sµs 4 dµs
09:28:41.823650 INFO [blockchain] Block [441146] 2820 txs 4539 ins 210 wms
261 vms 57 vµs 2 rµs 5 cµs 17 pµs 1 aµs 34 sµs 5 dµs
09:28:42.163806 INFO [blockchain] Block [441147] 1395 txs 4515 ins 245 wms
321 vms 71 vµs 1 rµs 3 cµs 20 pµs 2 aµs 46 sµs 4 dµs
09:28:42.476815 INFO [blockchain] Block [441148] 999 txs 5157 ins 312 wms
295 vms 57 vµs 1 rµs 3 cµs 15 pµs 1 aµs 39 sµs 3 dµs
09:28:42.813068 INFO [blockchain] Block [441149] 1026 txs 5413 ins 284 wms
313 vms 58 vµs 1 rµs 2 cµs 18 pµs 1 aµs 37 sµs 4 dµs
09:28:42.813827 INFO [blockchain] Block [441150] 9 txs 8 ins 323 wms
1 vms 67 vµs 4 rµs 17 cµs 10 pµs 9 aµs 30 sµs 15 dµs
09:28:43.101809 INFO [blockchain] Block [441151] 2669 txs 4608 ins 0 wms
253 vms 55 vµs 1 rµs 4 cµs 16 pµs 1 aµs 34 sµs 5 dµs
09:28:43.389835 INFO [blockchain] Block [441152] 2870 txs 4474 ins 238 wms
264 vms 59 vµs 2 rµs 5 cµs 18 pµs 1 aµs 35 sµs 5 dµs
09:28:43.696813 INFO [blockchain] Block [441153] 2451 txs 4484 ins 249 wms
285 vms 64 vµs 1 rµs 5 cµs 16 pµs 1 aµs 41 sµs 5 dµs
09:28:44.005768 INFO [blockchain] Block [441154] 2217 txs 4515 ins 274 wms
288 vms 64 vµs 1 rµs 4 cµs 19 pµs 2 aµs 39 sµs 5 dµs
09:28:44.488287 INFO [blockchain] Block [441155] 1407 txs 4516 ins 279 wms
462 vms 102 vµs 1 rµs 3 cµs 37 pµs 2 aµs 60 sµs 5 dµs
09:28:45.096894 INFO [blockchain] Block [441156] 1291 txs 3971 ins 456 wms
591 vms 149 vµs 1 rµs 4 cµs 34 pµs 1 aµs 109 sµs 4 dµs
09:28:45.373956 INFO [blockchain] Block [441157] 2556 txs 4412 ins 582 wms
255 vms 58 vµs 1 rµs 4 cµs 13 pµs 1 aµs 39 sµs 5 dµs
09:28:45.751849 INFO [blockchain] Block [441158] 2122 txs 4869 ins 245 wms
356 vms 73 vµs 1 rµs 4 cµs 18 pµs 1 aµs 50 sµs 4 dµs
09:28:45.752609 INFO [blockchain] Block [441159] 10 txs 9 ins 359 wms
0 vms 54 vµs 2 rµs 12 cµs 6 pµs 7 aµs 30 sµs 17 dµs
09:28:46.167439 INFO [blockchain] Block [441160] 2260 txs 4798 ins 0 wms
380 vms 79 vµs 1 rµs 3 cµs 24 pµs 1 aµs 51 sµs 5 dµs
09:28:46.534458 INFO [blockchain] Block [441161] 2037 txs 4720 ins 373 wms
347 vms 74 vµs 1 rµs 4 cµs 16 pµs 1 aµs 53 sµs 4 dµs
09:28:46.786271 INFO [blockchain] Block [441162] 1478 txs 4044 ins 328 wms
236 vms 58 vµs 2 rµs 4 cµs 14 pµs 2 aµs 39 sµs 4 dµs
09:28:47.062390 INFO [blockchain] Block [441163] 2620 txs 4455 ins 217 wms
253 vms 57 vµs 2 rµs 5 cµs 13 pµs 1 aµs 37 sµs 5 dµs
09:28:47.063237 INFO [blockchain] Block [441164] 10 txs 9 ins 255 wms
1 vms 69 vµs 3 rµs 15 cµs 7 pµs 7 aµs 40 sµs 14 dµs
09:28:47.381445 INFO [blockchain] Block [441165] 2490 txs 4611 ins 0 wms
285 vms 62 vµs 1 rµs 4 cµs 16 pµs 1 aµs 41 sµs 5 dµs
09:28:47.697020 INFO [blockchain] Block [441166] 2266 txs 4225 ins 271 wms
297 vms 70 vµs 2 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
09:28:47.904200 INFO [blockchain] Block [441167] 1689 txs 3652 ins 283 wms
191 vms 52 vµs 2 rµs 4 cµs 9 pµs 1 aµs 38 sµs 4 dµs
09:28:48.215998 INFO [blockchain] Block [441168] 2301 txs 4441 ins 174 wms
293 vms 66 vµs 2 rµs 4 cµs 12 pµs 1 aµs 48 sµs 4 dµs
09:28:48.490337 INFO [blockchain] Block [441169] 2823 txs 4950 ins 273 wms
250 vms 50 vµs 2 rµs 5 cµs 15 pµs 1 aµs 30 sµs 5 dµs
09:28:48.771430 INFO [blockchain] Block [441170] 2728 txs 4938 ins 232 wms
258 vms 52 vµs 2 rµs 4 cµs 16 pµs 1 aµs 30 sµs 5 dµs
09:28:49.252750 INFO [blockchain] Block [441171] 1967 txs 4328 ins 242 wms
464 vms 107 vµs 2 rµs 4 cµs 14 pµs 2 aµs 87 sµs 4 dµs
09:28:49.564182 INFO [blockchain] Block [441172] 2401 txs 4575 ins 451 wms
291 vms 64 vµs 1 rµs 4 cµs 13 pµs 2 aµs 45 sµs 5 dµs
09:28:50.094058 INFO [blockchain] Block [441173] 1482 txs 4119 ins 281 wms
514 vms 125 vµs 1 rµs 4 cµs 10 pµs 2 aµs 108 sµs 4 dµs
09:28:50.446385 INFO [blockchain] Block [441174] 2216 txs 4791 ins 502 wms
331 vms 69 vµs 1 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
09:28:50.860666 INFO [blockchain] Block [441175] 1172 txs 5169 ins 322 wms
396 vms 77 vµs 1 rµs 3 cµs 12 pµs 1 aµs 61 sµs 4 dµs
09:28:51.197153 INFO [blockchain] Block [441176] 2022 txs 4642 ins 388 wms
317 vms 68 vµs 1 rµs 4 cµs 13 pµs 2 aµs 50 sµs 4 dµs
09:28:51.512118 INFO [blockchain] Block [441177] 1852 txs 4358 ins 307 wms
296 vms 68 vµs 1 rµs 4 cµs 14 pµs 2 aµs 48 sµs 4 dµs
09:28:51.859025 INFO [blockchain] Block [441178] 2319 txs 5030 ins 285 wms
326 vms 65 vµs 1 rµs 4 cµs 13 pµs 1 aµs 47 sµs 4 dµs
09:28:52.185014 INFO [blockchain] Block [441179] 2612 txs 4344 ins 314 wms
305 vms 70 vµs 1 rµs 5 cµs 14 pµs 2 aµs 49 sµs 5 dµs
09:28:52.475703 INFO [blockchain] Block [441180] 2262 txs 4030 ins 287 wms
272 vms 68 vµs 2 rµs 5 cµs 15 pµs 2 aµs 46 sµs 5 dµs
09:28:52.798759 INFO [blockchain] Block [441181] 2885 txs 5092 ins 253 wms
299 vms 59 vµs 2 rµs 4 cµs 17 pµs 1 aµs 36 sµs 5 dµs
09:28:53.078336 INFO [blockchain] Block [441182] 2436 txs 4915 ins 282 wms
257 vms 52 vµs 2 rµs 4 cµs 14 pµs 1 aµs 32 sµs 5 dµs
09:28:53.398266 INFO [blockchain] Block [441183] 2843 txs 4446 ins 238 wms
298 vms 67 vµs 2 rµs 5 cµs 20 pµs 2 aµs 41 sµs 5 dµs
09:28:53.707133 INFO [blockchain] Block [441184] 2324 txs 4476 ins 278 wms
288 vms 64 vµs 2 rµs 4 cµs 16 pµs 2 aµs 42 sµs 5 dµs
09:28:53.918838 INFO [blockchain] Block [441185] 1512 txs 3339 ins 276 wms
196 vms 59 vµs 1 rµs 4 cµs 9 pµs 1 aµs 44 sµs 5 dµs
09:28:54.237412 INFO [blockchain] Block [441186] 2067 txs 4655 ins 180 wms
299 vms 64 vµs 2 rµs 4 cµs 14 pµs 2 aµs 45 sµs 4 dµs
09:28:54.525741 INFO [blockchain] Block [441187] 1403 txs 5186 ins 282 wms
268 vms 52 vµs 1 rµs 3 cµs 12 pµs 1 aµs 35 sµs 4 dµs
09:28:54.768170 INFO [blockchain] Block [441188] 770 txs 4352 ins 259 wms
227 vms 52 vµs 1 rµs 2 cµs 10 pµs 1 aµs 39 sµs 3 dµs
09:28:55.050891 INFO [blockchain] Block [441189] 2428 txs 4573 ins 220 wms
262 vms 57 vµs 1 rµs 4 cµs 14 pµs 1 aµs 38 sµs 5 dµs
09:28:55.129727 INFO [blockchain] Block [441190] 477 txs 1278 ins 261 wms
73 vms 57 vµs 1 rµs 4 cµs 4 pµs 2 aµs 47 sµs 4 dµs
09:28:55.287538 INFO [blockchain] Block [441191] 1355 txs 2932 ins 66 wms
146 vms 50 vµs 1 rµs 4 cµs 8 pµs 1 aµs 37 sµs 4 dµs
09:28:55.622908 INFO [blockchain] Block [441192] 1139 txs 5365 ins 136 wms
305 vms 57 vµs 1 rµs 3 cµs 12 pµs 1 aµs 41 sµs 6 dµs
09:28:55.717459 INFO [blockchain] Block [441193] 904 txs 1852 ins 314 wms
86 vms 46 vµs 2 rµs 4 cµs 7 pµs 1 aµs 34 sµs 5 dµs
09:28:55.986453 INFO [blockchain] Block [441194] 250 txs 4923 ins 74 wms
255 vms 52 vµs 1 rµs 2 cµs 11 pµs 1 aµs 38 sµs 3 dµs
09:28:55.987299 INFO [blockchain] Block [441195] 7 txs 10 ins 261 wms
1 vms 62 vµs 3 rµs 11 cµs 8 pµs 7 aµs 37 sµs 12 dµs
09:28:56.262784 INFO [blockchain] Block [441196] 2502 txs 4894 ins 0 wms
243 vms 50 vµs 1 rµs 4 cµs 14 pµs 1 aµs 31 sµs 4 dµs
09:28:56.539972 INFO [blockchain] Block [441197] 1963 txs 5490 ins 228 wms
257 vms 47 vµs 1 rµs 3 cµs 13 pµs 1 aµs 29 sµs 4 dµs
09:28:56.846339 INFO [blockchain] Block [441198] 2574 txs 4683 ins 246 wms
281 vms 60 vµs 1 rµs 4 cµs 16 pµs 1 aµs 38 sµs 5 dµs
<end>
::
[Libbitcoin] libbitcoin node update
