Reposting as text, formatted HTML got moderated :/.

---

We’ve spent a good amount of time on node, following a rewrite of validation
and script processing, and on the heels of the previous network redesign.
I’m very happy with the results. We have a very stable node and excellent
block validation performance. We have been lucky to have the full time
support of Bitprim in operational testing of the node on both high and low
end platforms. You may recall that we previously achieved full chain sync of
about 15 minutes (all blocks checkpointed). We have since validated the full
mainnet chain countless times. Despite the fact that our networking is still
not optimal in terms of catch-up sync (blocks-first), we are able to sync
0-295,000 and validate 295,001-top (Core parameters) in about 7 hrs. I
expect that number to come down significantly in v4 owing to more advanced
networking.

Below is a sample of the new console reporting format for block acceptance,
on a high end server (SSD, 12 Cores, 128GB RAM). You can see that we are
processing at a rate of about 3-4 blocks per second at the top of the
current chain. But given network limitations this is not the full story. As
you can see our median time process a block is about 62.5 µs per input (1 µs
= one millionth of a second), or full block validation times (i.e. excluding
the empty blocks) averaging under .3 seconds. We have not tested on more
than 12 cores. The number of cores used (and validation thread priority) is
configurable, so we can easily see improvement in speed from 1-12 cores
presently. I expect those numbers would continue to improve with additional
cores up to some point of diminishing returns.

A libbitcoin node store is 3 files for a total of ~102GB. The files are
portable across machines. In other words 128GB RAM is more than sufficient
to hold the entire chain, reducing paging costs. However even an 8GB RAM
machine with 4 cores maintains healthy performance. We have not tested at
intermediate RAM levels (e.g. 64 or 32 GB) but these sizes are expected to
be very performant.

I should point out that libbitcoin does not use a UTXO database nor does it
cache UTXOs, so there is no problem with scale as the blockchain grows.
Per-input performance is linear across the full range of the chain
regardless of hardware (assuming a drive of sufficient space for the chain).
Outputs are read, and updated as spent, directly via transaction records.
The cost of all store reads, including outputs, activation state, block and
transaction duplicate checks, difficulty, and timestamps, is incurred all at
once and captured in the pµs value below (averaging 15 µs per input). The
cost of storing the new block, including spent output information, is
reflected in the dµs value below (averaging 5 µs per input). Reading and
writing independently from validation and other processing is not only huge
boost for testability but also dramatically simplifies code and exposes
important optimizations. For example, all per-input operations with the
exception of block check are parallelized either by tx or input.

Our script validation uses a fully-generalized script processor:

https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin
/impl/machine/interpreter.ipp

This makes Bitcoin script easy to understand/ and test, and safer to
maintain. Despite this high level of generalization compared to other
implementations, we are achieving a script validation cost of 41 µs per
input. This is 13.5% faster than we achieve using libconsensus (which can be
swapped in via a configuration option). Testing has confirmed that this is
entirely the consequence of the need to serialize each transaction and
previous output for that library’s interface, and because each must then be
deserialized internal to that library. As such libconsensus is not viable
for high performance implementations, such as mining, but the cost is small
enough to not adversely impact other scenarios.

e


Block Totals (milliseconds)
----------------------------------
txs: count of transactions
ins: count of inputs
wms: wait on previous block
vms: validate (*) time

Per Input (microseconds)
----------------------------------
vµs: validate (*)
rµs: deserialization (read from wire)
cµs: check block (context-free validation)
pµs: populate (all database reads)
aµs: accept block (contextual validation)
sµs: connect block (script verify)
dµs: deposit (all database writes)

(*) validate is the time from start of block-check to the end of
block-connect. This is the time that it takes to fully validate a block
object. This is important in that it is the time, after receipt and
deserialization (time also shown) of a (whole) block from a peer, that a
miner must wait until a notification of validity is available.

09:28:37.985756 INFO [blockchain] Block [441133] 2412 txs 4214 ins  188 wms
307 vms   73 vµs    1 rµs    5 cµs   19 pµs    2 aµs   48 sµs    5 dµs
09:28:38.295939 INFO [blockchain] Block [441134] 2691 txs 4863 ins  291 wms
286 vms   59 vµs    2 rµs    5 cµs   21 pµs    1 aµs   31 sµs    5 dµs
09:28:38.751393 INFO [blockchain] Block [441135] 2136 txs 4484 ins  270 wms
434 vms   97 vµs    1 rµs    4 cµs   26 pµs    2 aµs   65 sµs    5 dµs
09:28:38.966397 INFO [blockchain] Block [441136] 1918 txs 3528 ins  427 wms
198 vms   56 vµs    1 rµs    4 cµs   15 pµs    1 aµs   36 sµs    5 dµs
09:28:39.170047 INFO [blockchain] Block [441137] 1915 txs 3258 ins  185 wms
186 vms   57 vµs    2 rµs    5 cµs   13 pµs    2 aµs   37 sµs    5 dµs
09:28:39.474600 INFO [blockchain] Block [441138] 2367 txs 4197 ins  172 wms
284 vms   68 vµs    1 rµs    5 cµs   17 pµs    2 aµs   45 sµs    5 dµs
09:28:39.798699 INFO [blockchain] Block [441139] 2386 txs 4434 ins  270 wms
302 vms   68 vµs    1 rµs    4 cµs   18 pµs    1 aµs   45 sµs    5 dµs
09:28:40.161076 INFO [blockchain] Block [441140] 2150 txs 4648 ins  293 wms
342 vms   73 vµs    1 rµs    4 cµs   15 pµs    1 aµs   53 sµs    4 dµs
09:28:40.493881 INFO [blockchain] Block [441141] 1768 txs 4754 ins  325 wms
313 vms   66 vµs    2 rµs    4 cµs   19 pµs    1 aµs   42 sµs    4 dµs
09:28:40.688693 INFO [blockchain] Block [441142] 1860 txs 3726 ins  301 wms
178 vms   48 vµs    2 rµs    4 cµs   12 pµs    1 aµs   30 sµs    5 dµs
09:28:40.977133 INFO [blockchain] Block [441143] 2737 txs 4598 ins  161 wms
265 vms   58 vµs    1 rµs    5 cµs   18 pµs    1 aµs   34 sµs    5 dµs
09:28:41.295907 INFO [blockchain] Block [441144] 1530 txs 4592 ins  251 wms
301 vms   66 vµs    1 rµs    4 cµs   16 pµs    1 aµs   45 sµs    4 dµs
09:28:41.539977 INFO [blockchain] Block [441145] 1625 txs 3637 ins  282 wms
228 vms   63 vµs    1 rµs    4 cµs   10 pµs    1 aµs   47 sµs    4 dµs
09:28:41.823650 INFO [blockchain] Block [441146] 2820 txs 4539 ins  210 wms
261 vms   57 vµs    2 rµs    5 cµs   17 pµs    1 aµs   34 sµs    5 dµs
09:28:42.163806 INFO [blockchain] Block [441147] 1395 txs 4515 ins  245 wms
321 vms   71 vµs    1 rµs    3 cµs   20 pµs    2 aµs   46 sµs    4 dµs
09:28:42.476815 INFO [blockchain] Block [441148]  999 txs 5157 ins  312 wms
295 vms   57 vµs    1 rµs    3 cµs   15 pµs    1 aµs   39 sµs    3 dµs
09:28:42.813068 INFO [blockchain] Block [441149] 1026 txs 5413 ins  284 wms
313 vms   58 vµs    1 rµs    2 cµs   18 pµs    1 aµs   37 sµs    4 dµs
09:28:42.813827 INFO [blockchain] Block [441150]    9 txs    8 ins  323 wms
1 vms   67 vµs    4 rµs   17 cµs   10 pµs    9 aµs   30 sµs   15 dµs
09:28:43.101809 INFO [blockchain] Block [441151] 2669 txs 4608 ins    0 wms
253 vms   55 vµs    1 rµs    4 cµs   16 pµs    1 aµs   34 sµs    5 dµs
09:28:43.389835 INFO [blockchain] Block [441152] 2870 txs 4474 ins  238 wms
264 vms   59 vµs    2 rµs    5 cµs   18 pµs    1 aµs   35 sµs    5 dµs
09:28:43.696813 INFO [blockchain] Block [441153] 2451 txs 4484 ins  249 wms
285 vms   64 vµs    1 rµs    5 cµs   16 pµs    1 aµs   41 sµs    5 dµs
09:28:44.005768 INFO [blockchain] Block [441154] 2217 txs 4515 ins  274 wms
288 vms   64 vµs    1 rµs    4 cµs   19 pµs    2 aµs   39 sµs    5 dµs
09:28:44.488287 INFO [blockchain] Block [441155] 1407 txs 4516 ins  279 wms
462 vms  102 vµs    1 rµs    3 cµs   37 pµs    2 aµs   60 sµs    5 dµs
09:28:45.096894 INFO [blockchain] Block [441156] 1291 txs 3971 ins  456 wms
591 vms  149 vµs    1 rµs    4 cµs   34 pµs    1 aµs  109 sµs    4 dµs
09:28:45.373956 INFO [blockchain] Block [441157] 2556 txs 4412 ins  582 wms
255 vms   58 vµs    1 rµs    4 cµs   13 pµs    1 aµs   39 sµs    5 dµs
09:28:45.751849 INFO [blockchain] Block [441158] 2122 txs 4869 ins  245 wms
356 vms   73 vµs    1 rµs    4 cµs   18 pµs    1 aµs   50 sµs    4 dµs
09:28:45.752609 INFO [blockchain] Block [441159]   10 txs    9 ins  359 wms
0 vms   54 vµs    2 rµs   12 cµs    6 pµs    7 aµs   30 sµs   17 dµs
09:28:46.167439 INFO [blockchain] Block [441160] 2260 txs 4798 ins    0 wms
380 vms   79 vµs    1 rµs    3 cµs   24 pµs    1 aµs   51 sµs    5 dµs
09:28:46.534458 INFO [blockchain] Block [441161] 2037 txs 4720 ins  373 wms
347 vms   74 vµs    1 rµs    4 cµs   16 pµs    1 aµs   53 sµs    4 dµs
09:28:46.786271 INFO [blockchain] Block [441162] 1478 txs 4044 ins  328 wms
236 vms   58 vµs    2 rµs    4 cµs   14 pµs    2 aµs   39 sµs    4 dµs
09:28:47.062390 INFO [blockchain] Block [441163] 2620 txs 4455 ins  217 wms
253 vms   57 vµs    2 rµs    5 cµs   13 pµs    1 aµs   37 sµs    5 dµs
09:28:47.063237 INFO [blockchain] Block [441164]   10 txs    9 ins  255 wms
1 vms   69 vµs    3 rµs   15 cµs    7 pµs    7 aµs   40 sµs   14 dµs
09:28:47.381445 INFO [blockchain] Block [441165] 2490 txs 4611 ins    0 wms
285 vms   62 vµs    1 rµs    4 cµs   16 pµs    1 aµs   41 sµs    5 dµs
09:28:47.697020 INFO [blockchain] Block [441166] 2266 txs 4225 ins  271 wms
297 vms   70 vµs    2 rµs    4 cµs   15 pµs    1 aµs   49 sµs    4 dµs
09:28:47.904200 INFO [blockchain] Block [441167] 1689 txs 3652 ins  283 wms
191 vms   52 vµs    2 rµs    4 cµs    9 pµs    1 aµs   38 sµs    4 dµs
09:28:48.215998 INFO [blockchain] Block [441168] 2301 txs 4441 ins  174 wms
293 vms   66 vµs    2 rµs    4 cµs   12 pµs    1 aµs   48 sµs    4 dµs
09:28:48.490337 INFO [blockchain] Block [441169] 2823 txs 4950 ins  273 wms
250 vms   50 vµs    2 rµs    5 cµs   15 pµs    1 aµs   30 sµs    5 dµs
09:28:48.771430 INFO [blockchain] Block [441170] 2728 txs 4938 ins  232 wms
258 vms   52 vµs    2 rµs    4 cµs   16 pµs    1 aµs   30 sµs    5 dµs
09:28:49.252750 INFO [blockchain] Block [441171] 1967 txs 4328 ins  242 wms
464 vms  107 vµs    2 rµs    4 cµs   14 pµs    2 aµs   87 sµs    4 dµs
09:28:49.564182 INFO [blockchain] Block [441172] 2401 txs 4575 ins  451 wms
291 vms   64 vµs    1 rµs    4 cµs   13 pµs    2 aµs   45 sµs    5 dµs
09:28:50.094058 INFO [blockchain] Block [441173] 1482 txs 4119 ins  281 wms
514 vms  125 vµs    1 rµs    4 cµs   10 pµs    2 aµs  108 sµs    4 dµs
09:28:50.446385 INFO [blockchain] Block [441174] 2216 txs 4791 ins  502 wms
331 vms   69 vµs    1 rµs    4 cµs   15 pµs    1 aµs   49 sµs    4 dµs
09:28:50.860666 INFO [blockchain] Block [441175] 1172 txs 5169 ins  322 wms
396 vms   77 vµs    1 rµs    3 cµs   12 pµs    1 aµs   61 sµs    4 dµs
09:28:51.197153 INFO [blockchain] Block [441176] 2022 txs 4642 ins  388 wms
317 vms   68 vµs    1 rµs    4 cµs   13 pµs    2 aµs   50 sµs    4 dµs
09:28:51.512118 INFO [blockchain] Block [441177] 1852 txs 4358 ins  307 wms
296 vms   68 vµs    1 rµs    4 cµs   14 pµs    2 aµs   48 sµs    4 dµs
09:28:51.859025 INFO [blockchain] Block [441178] 2319 txs 5030 ins  285 wms
326 vms   65 vµs    1 rµs    4 cµs   13 pµs    1 aµs   47 sµs    4 dµs
09:28:52.185014 INFO [blockchain] Block [441179] 2612 txs 4344 ins  314 wms
305 vms   70 vµs    1 rµs    5 cµs   14 pµs    2 aµs   49 sµs    5 dµs
09:28:52.475703 INFO [blockchain] Block [441180] 2262 txs 4030 ins  287 wms
272 vms   68 vµs    2 rµs    5 cµs   15 pµs    2 aµs   46 sµs    5 dµs
09:28:52.798759 INFO [blockchain] Block [441181] 2885 txs 5092 ins  253 wms
299 vms   59 vµs    2 rµs    4 cµs   17 pµs    1 aµs   36 sµs    5 dµs
09:28:53.078336 INFO [blockchain] Block [441182] 2436 txs 4915 ins  282 wms
257 vms   52 vµs    2 rµs    4 cµs   14 pµs    1 aµs   32 sµs    5 dµs
09:28:53.398266 INFO [blockchain] Block [441183] 2843 txs 4446 ins  238 wms
298 vms   67 vµs    2 rµs    5 cµs   20 pµs    2 aµs   41 sµs    5 dµs
09:28:53.707133 INFO [blockchain] Block [441184] 2324 txs 4476 ins  278 wms
288 vms   64 vµs    2 rµs    4 cµs   16 pµs    2 aµs   42 sµs    5 dµs
09:28:53.918838 INFO [blockchain] Block [441185] 1512 txs 3339 ins  276 wms
196 vms   59 vµs    1 rµs    4 cµs    9 pµs    1 aµs   44 sµs    5 dµs
09:28:54.237412 INFO [blockchain] Block [441186] 2067 txs 4655 ins  180 wms
299 vms   64 vµs    2 rµs    4 cµs   14 pµs    2 aµs   45 sµs    4 dµs
09:28:54.525741 INFO [blockchain] Block [441187] 1403 txs 5186 ins  282 wms
268 vms   52 vµs    1 rµs    3 cµs   12 pµs    1 aµs   35 sµs    4 dµs
09:28:54.768170 INFO [blockchain] Block [441188]  770 txs 4352 ins  259 wms
227 vms   52 vµs    1 rµs    2 cµs   10 pµs    1 aµs   39 sµs    3 dµs
09:28:55.050891 INFO [blockchain] Block [441189] 2428 txs 4573 ins  220 wms
262 vms   57 vµs    1 rµs    4 cµs   14 pµs    1 aµs   38 sµs    5 dµs
09:28:55.129727 INFO [blockchain] Block [441190]  477 txs 1278 ins  261 wms
73 vms   57 vµs    1 rµs    4 cµs    4 pµs    2 aµs   47 sµs    4 dµs
09:28:55.287538 INFO [blockchain] Block [441191] 1355 txs 2932 ins   66 wms
146 vms   50 vµs    1 rµs    4 cµs    8 pµs    1 aµs   37 sµs    4 dµs
09:28:55.622908 INFO [blockchain] Block [441192] 1139 txs 5365 ins  136 wms
305 vms   57 vµs    1 rµs    3 cµs   12 pµs    1 aµs   41 sµs    6 dµs
09:28:55.717459 INFO [blockchain] Block [441193]  904 txs 1852 ins  314 wms
86 vms   46 vµs    2 rµs    4 cµs    7 pµs    1 aµs   34 sµs    5 dµs
09:28:55.986453 INFO [blockchain] Block [441194]  250 txs 4923 ins   74 wms
255 vms   52 vµs    1 rµs    2 cµs   11 pµs    1 aµs   38 sµs    3 dµs
09:28:55.987299 INFO [blockchain] Block [441195]    7 txs   10 ins  261 wms
1 vms   62 vµs    3 rµs   11 cµs    8 pµs    7 aµs   37 sµs   12 dµs
09:28:56.262784 INFO [blockchain] Block [441196] 2502 txs 4894 ins    0 wms
243 vms   50 vµs    1 rµs    4 cµs   14 pµs    1 aµs   31 sµs    4 dµs
09:28:56.539972 INFO [blockchain] Block [441197] 1963 txs 5490 ins  228 wms
257 vms   47 vµs    1 rµs    3 cµs   13 pµs    1 aµs   29 sµs    4 dµs
09:28:56.846339 INFO [blockchain] Block [441198] 2574 txs 4683 ins  246 wms
281 vms   60 vµs    1 rµs    4 cµs   16 pµs    1 aµs   38 sµs    5 dµs

<end>