We didn't make the end of year objective, so I've reset to end of
January. There was some unexpected time spent tracking down a couple of
additional networking issues. The root problem was isolated and resolved
and we have been running smoothly and at record speed for days now.
I am now running fully-synced mainnet and testnet chains on my dev
machines (Linux and Windows) which are 8GB 4 core 10 year old machines,
with no problems at all. I've validated both chains in full, with
mainnet taking less than two days. Several others are reporting good
results as well.
Our network stack is not fully-optimized for v3, though validation
performance is outstanding. I mentioned below that we did not have a
utxo cache, which was hurting us on smaller machines (we have no utxo
store). I implemented a size-configurable cache last month and that is
working really well. It dramatically cut validation time on my smaller
dev machines.
We've recently completed the replacement of the orphan pool with a block
pool. This is a significant improvement in several areas. It will allow
the node to withstand a reorg of any depth, up to the depth configured.
The default is 50 blocks. There is no material performance hit for
having a large block pool, since the pool typically remains empty. The
old orphan pool by contrast was guaranteed to always be full after some
small amount of time. We only store valid blocks (branches) that only
lack sufficient work to force a reorg. This means that we only ever
validate one block as blocks arrive and we never hold orphans. If we see
an orphan we ask the peer to send the intervening blocks and then we
discard the orphan.
Currently there are two major work items remaining for v3. First is the
transaction pool. Like the orphan pool replacement we are replacing the
tx pool as well. Once this is complete the currently-disabled server
APIs that depend on the tx pool will be reconnected. As with the block
pool, the new tx pool never holds orphan transactions. Such transactions
cannot be validated and therefore create a DoS vector. This also means
that all pool-validated transactions will preclude the need to
revalidate as blocks containing the txs are received. I expect this work
to be done within a week.
Following this work we need to complete the update to the initial block
download (checkpointed) sync, which was disabled a few months ago. That
should take 4-5 days. After that it's just release work and testing. So
I'm optimistic about schedule, especially given that the node/server are
very stable at this point after heavy testing.
e
On 12/09/2016 05:29 PM, Eric Voskuil wrote:
> We’ve spent a good amount of time on node, following a rewrite of validation
> and script processing, and on the heels of the previous network redesign.
> I’m very happy with the results. We have a very stable node and excellent
> block validation performance. We have been lucky to have the full time
> support of Bitprim in operational testing of the node on both high and low
> end platforms. You may recall that we previously achieved full chain sync of
> about 15 minutes (all blocks checkpointed). We have since validated the full
> mainnet chain countless times. Despite the fact that our networking is still
> not optimal in terms of catch-up sync (blocks-first), we are able to sync
> 0-295,000 and validate 295,001-top (Core parameters) in about 7 hrs. I
> expect that number to come down significantly in v4 owing to more advanced
> networking.
>
> Below is a sample of the new console reporting format for block acceptance,
> on a high end server (SSD, 12 Cores, 128GB RAM). You can see that we are
> processing at a rate of about 3-4 blocks per second at the top of the
> current chain. But given network limitations this is not the full story. As
> you can see our median time process a block is about 62.5 µs per input (1 µs
> = one millionth of a second), or full block validation times (i.e. excluding
> the empty blocks) averaging under .3 seconds. We have not tested on more
> than 12 cores. The number of cores used (and validation thread priority) is
> configurable, so we can easily see improvement in speed from 1-12 cores
> presently. I expect those numbers would continue to improve with additional
> cores up to some point of diminishing returns.
>
> A libbitcoin node store is 3 files for a total of ~102GB. The files are
> portable across machines. In other words 128GB RAM is more than sufficient
> to hold the entire chain, reducing paging costs. However even an 8GB RAM
> machine with 4 cores maintains healthy performance. We have not tested at
> intermediate RAM levels (e.g. 64 or 32 GB) but these sizes are expected to
> be very performant.
>
> I should point out that libbitcoin does not use a UTXO database nor does it
> cache UTXOs, so there is no problem with scale as the blockchain grows.
> Per-input performance is linear across the full range of the chain
> regardless of hardware (assuming a drive of sufficient space for the chain).
> Outputs are read, and updated as spent, directly via transaction records.
> The cost of all store reads, including outputs, activation state, block and
> transaction duplicate checks, difficulty, and timestamps, is incurred all at
> once and captured in the pµs value below (averaging 15 µs per input). The
> cost of storing the new block, including spent output information, is
> reflected in the dµs value below (averaging 5 µs per input). Reading and
> writing independently from validation and other processing is not only huge
> boost for testability but also dramatically simplifies code and exposes
> important optimizations. For example, all per-input operations with the
> exception of block check are parallelized either by tx or input.
>
> Our script validation uses a fully-generalized script processor:
>
> https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin
> /impl/machine/interpreter.ipp
>
> This makes Bitcoin script easy to understand/ and test, and safer to
> maintain. Despite this high level of generalization compared to other
> implementations, we are achieving a script validation cost of 41 µs per
> input. This is 13.5% faster than we achieve using libconsensus (which can be
> swapped in via a configuration option). Testing has confirmed that this is
> entirely the consequence of the need to serialize each transaction and
> previous output for that library’s interface, and because each must then be
> deserialized internal to that library. As such libconsensus is not viable
> for high performance implementations, such as mining, but the cost is small
> enough to not adversely impact other scenarios.
>
> e
>
>
> Block Totals (milliseconds)
> ----------------------------------
> txs: count of transactions
> ins: count of inputs
> wms: wait on previous block
> vms: validate (*) time
>
> Per Input (microseconds)
> ----------------------------------
> vµs: validate (*)
> rµs: deserialization (read from wire)
> cµs: check block (context-free validation)
> pµs: populate (all database reads)
> aµs: accept block (contextual validation)
> sµs: connect block (script verify)
> dµs: deposit (all database writes)
>
> (*) validate is the time from start of block-check to the end of
> block-connect. This is the time that it takes to fully validate a block
> object. This is important in that it is the time, after receipt and
> deserialization (time also shown) of a (whole) block from a peer, that a
> miner must wait until a notification of validity is available.
>
> 09:28:37.985756 INFO [blockchain] Block [441133] 2412 txs 4214 ins 188 wms
> 307 vms 73 vµs 1 rµs 5 cµs 19 pµs 2 aµs 48 sµs 5 dµs
> 09:28:38.295939 INFO [blockchain] Block [441134] 2691 txs 4863 ins 291 wms
> 286 vms 59 vµs 2 rµs 5 cµs 21 pµs 1 aµs 31 sµs 5 dµs
> 09:28:38.751393 INFO [blockchain] Block [441135] 2136 txs 4484 ins 270 wms
> 434 vms 97 vµs 1 rµs 4 cµs 26 pµs 2 aµs 65 sµs 5 dµs
> 09:28:38.966397 INFO [blockchain] Block [441136] 1918 txs 3528 ins 427 wms
> 198 vms 56 vµs 1 rµs 4 cµs 15 pµs 1 aµs 36 sµs 5 dµs
> 09:28:39.170047 INFO [blockchain] Block [441137] 1915 txs 3258 ins 185 wms
> 186 vms 57 vµs 2 rµs 5 cµs 13 pµs 2 aµs 37 sµs 5 dµs
> 09:28:39.474600 INFO [blockchain] Block [441138] 2367 txs 4197 ins 172 wms
> 284 vms 68 vµs 1 rµs 5 cµs 17 pµs 2 aµs 45 sµs 5 dµs
> 09:28:39.798699 INFO [blockchain] Block [441139] 2386 txs 4434 ins 270 wms
> 302 vms 68 vµs 1 rµs 4 cµs 18 pµs 1 aµs 45 sµs 5 dµs
> 09:28:40.161076 INFO [blockchain] Block [441140] 2150 txs 4648 ins 293 wms
> 342 vms 73 vµs 1 rµs 4 cµs 15 pµs 1 aµs 53 sµs 4 dµs
> 09:28:40.493881 INFO [blockchain] Block [441141] 1768 txs 4754 ins 325 wms
> 313 vms 66 vµs 2 rµs 4 cµs 19 pµs 1 aµs 42 sµs 4 dµs
> 09:28:40.688693 INFO [blockchain] Block [441142] 1860 txs 3726 ins 301 wms
> 178 vms 48 vµs 2 rµs 4 cµs 12 pµs 1 aµs 30 sµs 5 dµs
> 09:28:40.977133 INFO [blockchain] Block [441143] 2737 txs 4598 ins 161 wms
> 265 vms 58 vµs 1 rµs 5 cµs 18 pµs 1 aµs 34 sµs 5 dµs
> 09:28:41.295907 INFO [blockchain] Block [441144] 1530 txs 4592 ins 251 wms
> 301 vms 66 vµs 1 rµs 4 cµs 16 pµs 1 aµs 45 sµs 4 dµs
> 09:28:41.539977 INFO [blockchain] Block [441145] 1625 txs 3637 ins 282 wms
> 228 vms 63 vµs 1 rµs 4 cµs 10 pµs 1 aµs 47 sµs 4 dµs
> 09:28:41.823650 INFO [blockchain] Block [441146] 2820 txs 4539 ins 210 wms
> 261 vms 57 vµs 2 rµs 5 cµs 17 pµs 1 aµs 34 sµs 5 dµs
> 09:28:42.163806 INFO [blockchain] Block [441147] 1395 txs 4515 ins 245 wms
> 321 vms 71 vµs 1 rµs 3 cµs 20 pµs 2 aµs 46 sµs 4 dµs
> 09:28:42.476815 INFO [blockchain] Block [441148] 999 txs 5157 ins 312 wms
> 295 vms 57 vµs 1 rµs 3 cµs 15 pµs 1 aµs 39 sµs 3 dµs
> 09:28:42.813068 INFO [blockchain] Block [441149] 1026 txs 5413 ins 284 wms
> 313 vms 58 vµs 1 rµs 2 cµs 18 pµs 1 aµs 37 sµs 4 dµs
> 09:28:42.813827 INFO [blockchain] Block [441150] 9 txs 8 ins 323 wms
> 1 vms 67 vµs 4 rµs 17 cµs 10 pµs 9 aµs 30 sµs 15 dµs
> 09:28:43.101809 INFO [blockchain] Block [441151] 2669 txs 4608 ins 0 wms
> 253 vms 55 vµs 1 rµs 4 cµs 16 pµs 1 aµs 34 sµs 5 dµs
> 09:28:43.389835 INFO [blockchain] Block [441152] 2870 txs 4474 ins 238 wms
> 264 vms 59 vµs 2 rµs 5 cµs 18 pµs 1 aµs 35 sµs 5 dµs
> 09:28:43.696813 INFO [blockchain] Block [441153] 2451 txs 4484 ins 249 wms
> 285 vms 64 vµs 1 rµs 5 cµs 16 pµs 1 aµs 41 sµs 5 dµs
> 09:28:44.005768 INFO [blockchain] Block [441154] 2217 txs 4515 ins 274 wms
> 288 vms 64 vµs 1 rµs 4 cµs 19 pµs 2 aµs 39 sµs 5 dµs
> 09:28:44.488287 INFO [blockchain] Block [441155] 1407 txs 4516 ins 279 wms
> 462 vms 102 vµs 1 rµs 3 cµs 37 pµs 2 aµs 60 sµs 5 dµs
> 09:28:45.096894 INFO [blockchain] Block [441156] 1291 txs 3971 ins 456 wms
> 591 vms 149 vµs 1 rµs 4 cµs 34 pµs 1 aµs 109 sµs 4 dµs
> 09:28:45.373956 INFO [blockchain] Block [441157] 2556 txs 4412 ins 582 wms
> 255 vms 58 vµs 1 rµs 4 cµs 13 pµs 1 aµs 39 sµs 5 dµs
> 09:28:45.751849 INFO [blockchain] Block [441158] 2122 txs 4869 ins 245 wms
> 356 vms 73 vµs 1 rµs 4 cµs 18 pµs 1 aµs 50 sµs 4 dµs
> 09:28:45.752609 INFO [blockchain] Block [441159] 10 txs 9 ins 359 wms
> 0 vms 54 vµs 2 rµs 12 cµs 6 pµs 7 aµs 30 sµs 17 dµs
> 09:28:46.167439 INFO [blockchain] Block [441160] 2260 txs 4798 ins 0 wms
> 380 vms 79 vµs 1 rµs 3 cµs 24 pµs 1 aµs 51 sµs 5 dµs
> 09:28:46.534458 INFO [blockchain] Block [441161] 2037 txs 4720 ins 373 wms
> 347 vms 74 vµs 1 rµs 4 cµs 16 pµs 1 aµs 53 sµs 4 dµs
> 09:28:46.786271 INFO [blockchain] Block [441162] 1478 txs 4044 ins 328 wms
> 236 vms 58 vµs 2 rµs 4 cµs 14 pµs 2 aµs 39 sµs 4 dµs
> 09:28:47.062390 INFO [blockchain] Block [441163] 2620 txs 4455 ins 217 wms
> 253 vms 57 vµs 2 rµs 5 cµs 13 pµs 1 aµs 37 sµs 5 dµs
> 09:28:47.063237 INFO [blockchain] Block [441164] 10 txs 9 ins 255 wms
> 1 vms 69 vµs 3 rµs 15 cµs 7 pµs 7 aµs 40 sµs 14 dµs
> 09:28:47.381445 INFO [blockchain] Block [441165] 2490 txs 4611 ins 0 wms
> 285 vms 62 vµs 1 rµs 4 cµs 16 pµs 1 aµs 41 sµs 5 dµs
> 09:28:47.697020 INFO [blockchain] Block [441166] 2266 txs 4225 ins 271 wms
> 297 vms 70 vµs 2 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
> 09:28:47.904200 INFO [blockchain] Block [441167] 1689 txs 3652 ins 283 wms
> 191 vms 52 vµs 2 rµs 4 cµs 9 pµs 1 aµs 38 sµs 4 dµs
> 09:28:48.215998 INFO [blockchain] Block [441168] 2301 txs 4441 ins 174 wms
> 293 vms 66 vµs 2 rµs 4 cµs 12 pµs 1 aµs 48 sµs 4 dµs
> 09:28:48.490337 INFO [blockchain] Block [441169] 2823 txs 4950 ins 273 wms
> 250 vms 50 vµs 2 rµs 5 cµs 15 pµs 1 aµs 30 sµs 5 dµs
> 09:28:48.771430 INFO [blockchain] Block [441170] 2728 txs 4938 ins 232 wms
> 258 vms 52 vµs 2 rµs 4 cµs 16 pµs 1 aµs 30 sµs 5 dµs
> 09:28:49.252750 INFO [blockchain] Block [441171] 1967 txs 4328 ins 242 wms
> 464 vms 107 vµs 2 rµs 4 cµs 14 pµs 2 aµs 87 sµs 4 dµs
> 09:28:49.564182 INFO [blockchain] Block [441172] 2401 txs 4575 ins 451 wms
> 291 vms 64 vµs 1 rµs 4 cµs 13 pµs 2 aµs 45 sµs 5 dµs
> 09:28:50.094058 INFO [blockchain] Block [441173] 1482 txs 4119 ins 281 wms
> 514 vms 125 vµs 1 rµs 4 cµs 10 pµs 2 aµs 108 sµs 4 dµs
> 09:28:50.446385 INFO [blockchain] Block [441174] 2216 txs 4791 ins 502 wms
> 331 vms 69 vµs 1 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
> 09:28:50.860666 INFO [blockchain] Block [441175] 1172 txs 5169 ins 322 wms
> 396 vms 77 vµs 1 rµs 3 cµs 12 pµs 1 aµs 61 sµs 4 dµs
> 09:28:51.197153 INFO [blockchain] Block [441176] 2022 txs 4642 ins 388 wms
> 317 vms 68 vµs 1 rµs 4 cµs 13 pµs 2 aµs 50 sµs 4 dµs
> 09:28:51.512118 INFO [blockchain] Block [441177] 1852 txs 4358 ins 307 wms
> 296 vms 68 vµs 1 rµs 4 cµs 14 pµs 2 aµs 48 sµs 4 dµs
> 09:28:51.859025 INFO [blockchain] Block [441178] 2319 txs 5030 ins 285 wms
> 326 vms 65 vµs 1 rµs 4 cµs 13 pµs 1 aµs 47 sµs 4 dµs
> 09:28:52.185014 INFO [blockchain] Block [441179] 2612 txs 4344 ins 314 wms
> 305 vms 70 vµs 1 rµs 5 cµs 14 pµs 2 aµs 49 sµs 5 dµs
> 09:28:52.475703 INFO [blockchain] Block [441180] 2262 txs 4030 ins 287 wms
> 272 vms 68 vµs 2 rµs 5 cµs 15 pµs 2 aµs 46 sµs 5 dµs
> 09:28:52.798759 INFO [blockchain] Block [441181] 2885 txs 5092 ins 253 wms
> 299 vms 59 vµs 2 rµs 4 cµs 17 pµs 1 aµs 36 sµs 5 dµs
> 09:28:53.078336 INFO [blockchain] Block [441182] 2436 txs 4915 ins 282 wms
> 257 vms 52 vµs 2 rµs 4 cµs 14 pµs 1 aµs 32 sµs 5 dµs
> 09:28:53.398266 INFO [blockchain] Block [441183] 2843 txs 4446 ins 238 wms
> 298 vms 67 vµs 2 rµs 5 cµs 20 pµs 2 aµs 41 sµs 5 dµs
> 09:28:53.707133 INFO [blockchain] Block [441184] 2324 txs 4476 ins 278 wms
> 288 vms 64 vµs 2 rµs 4 cµs 16 pµs 2 aµs 42 sµs 5 dµs
> 09:28:53.918838 INFO [blockchain] Block [441185] 1512 txs 3339 ins 276 wms
> 196 vms 59 vµs 1 rµs 4 cµs 9 pµs 1 aµs 44 sµs 5 dµs
> 09:28:54.237412 INFO [blockchain] Block [441186] 2067 txs 4655 ins 180 wms
> 299 vms 64 vµs 2 rµs 4 cµs 14 pµs 2 aµs 45 sµs 4 dµs
> 09:28:54.525741 INFO [blockchain] Block [441187] 1403 txs 5186 ins 282 wms
> 268 vms 52 vµs 1 rµs 3 cµs 12 pµs 1 aµs 35 sµs 4 dµs
> 09:28:54.768170 INFO [blockchain] Block [441188] 770 txs 4352 ins 259 wms
> 227 vms 52 vµs 1 rµs 2 cµs 10 pµs 1 aµs 39 sµs 3 dµs
> 09:28:55.050891 INFO [blockchain] Block [441189] 2428 txs 4573 ins 220 wms
> 262 vms 57 vµs 1 rµs 4 cµs 14 pµs 1 aµs 38 sµs 5 dµs
> 09:28:55.129727 INFO [blockchain] Block [441190] 477 txs 1278 ins 261 wms
> 73 vms 57 vµs 1 rµs 4 cµs 4 pµs 2 aµs 47 sµs 4 dµs
> 09:28:55.287538 INFO [blockchain] Block [441191] 1355 txs 2932 ins 66 wms
> 146 vms 50 vµs 1 rµs 4 cµs 8 pµs 1 aµs 37 sµs 4 dµs
> 09:28:55.622908 INFO [blockchain] Block [441192] 1139 txs 5365 ins 136 wms
> 305 vms 57 vµs 1 rµs 3 cµs 12 pµs 1 aµs 41 sµs 6 dµs
> 09:28:55.717459 INFO [blockchain] Block [441193] 904 txs 1852 ins 314 wms
> 86 vms 46 vµs 2 rµs 4 cµs 7 pµs 1 aµs 34 sµs 5 dµs
> 09:28:55.986453 INFO [blockchain] Block [441194] 250 txs 4923 ins 74 wms
> 255 vms 52 vµs 1 rµs 2 cµs 11 pµs 1 aµs 38 sµs 3 dµs
> 09:28:55.987299 INFO [blockchain] Block [441195] 7 txs 10 ins 261 wms
> 1 vms 62 vµs 3 rµs 11 cµs 8 pµs 7 aµs 37 sµs 12 dµs
> 09:28:56.262784 INFO [blockchain] Block [441196] 2502 txs 4894 ins 0 wms
> 243 vms 50 vµs 1 rµs 4 cµs 14 pµs 1 aµs 31 sµs 4 dµs
> 09:28:56.539972 INFO [blockchain] Block [441197] 1963 txs 5490 ins 228 wms
> 257 vms 47 vµs 1 rµs 3 cµs 13 pµs 1 aµs 29 sµs 4 dµs
> 09:28:56.846339 INFO [blockchain] Block [441198] 2574 txs 4683 ins 246 wms
> 281 vms 60 vµs 1 rµs 4 cµs 16 pµs 1 aµs 38 sµs 5 dµs
>
> <end>
>
>
::
[Libbitcoin] libbitcoin node/server…
