:: [Libbitcoin] libbitcoin node/server…
Top Pagina
Delete this message
Reply to this message
Auteur: Eric Voskuil
Datum:  
Aan: libbitcoin
Oude Onderwerpen: [Libbitcoin] libbitcoin node update
Onderwerp: [Libbitcoin] libbitcoin node/server update
We didn't make the end of year objective, so I've reset to end of
January. There was some unexpected time spent tracking down a couple of
additional networking issues. The root problem was isolated and resolved
and we have been running smoothly and at record speed for days now.

I am now running fully-synced mainnet and testnet chains on my dev
machines (Linux and Windows) which are 8GB 4 core 10 year old machines,
with no problems at all. I've validated both chains in full, with
mainnet taking less than two days. Several others are reporting good
results as well.

Our network stack is not fully-optimized for v3, though validation
performance is outstanding. I mentioned below that we did not have a
utxo cache, which was hurting us on smaller machines (we have no utxo
store). I implemented a size-configurable cache last month and that is
working really well. It dramatically cut validation time on my smaller
dev machines.

We've recently completed the replacement of the orphan pool with a block
pool. This is a significant improvement in several areas. It will allow
the node to withstand a reorg of any depth, up to the depth configured.
The default is 50 blocks. There is no material performance hit for
having a large block pool, since the pool typically remains empty. The
old orphan pool by contrast was guaranteed to always be full after some
small amount of time. We only store valid blocks (branches) that only
lack sufficient work to force a reorg. This means that we only ever
validate one block as blocks arrive and we never hold orphans. If we see
an orphan we ask the peer to send the intervening blocks and then we
discard the orphan.

Currently there are two major work items remaining for v3. First is the
transaction pool. Like the orphan pool replacement we are replacing the
tx pool as well. Once this is complete the currently-disabled server
APIs that depend on the tx pool will be reconnected. As with the block
pool, the new tx pool never holds orphan transactions. Such transactions
cannot be validated and therefore create a DoS vector. This also means
that all pool-validated transactions will preclude the need to
revalidate as blocks containing the txs are received. I expect this work
to be done within a week.

Following this work we need to complete the update to the initial block
download (checkpointed) sync, which was disabled a few months ago. That
should take 4-5 days. After that it's just release work and testing. So
I'm optimistic about schedule, especially given that the node/server are
very stable at this point after heavy testing.

e

On 12/09/2016 05:29 PM, Eric Voskuil wrote:
> We’ve spent a good amount of time on node, following a rewrite of validation
> and script processing, and on the heels of the previous network redesign.
> I’m very happy with the results. We have a very stable node and excellent
> block validation performance. We have been lucky to have the full time
> support of Bitprim in operational testing of the node on both high and low
> end platforms. You may recall that we previously achieved full chain sync of
> about 15 minutes (all blocks checkpointed). We have since validated the full
> mainnet chain countless times. Despite the fact that our networking is still
> not optimal in terms of catch-up sync (blocks-first), we are able to sync
> 0-295,000 and validate 295,001-top (Core parameters) in about 7 hrs. I
> expect that number to come down significantly in v4 owing to more advanced
> networking.
>
> Below is a sample of the new console reporting format for block acceptance,
> on a high end server (SSD, 12 Cores, 128GB RAM). You can see that we are
> processing at a rate of about 3-4 blocks per second at the top of the
> current chain. But given network limitations this is not the full story. As
> you can see our median time process a block is about 62.5 µs per input (1 µs
> = one millionth of a second), or full block validation times (i.e. excluding
> the empty blocks) averaging under .3 seconds. We have not tested on more
> than 12 cores. The number of cores used (and validation thread priority) is
> configurable, so we can easily see improvement in speed from 1-12 cores
> presently. I expect those numbers would continue to improve with additional
> cores up to some point of diminishing returns.
>
> A libbitcoin node store is 3 files for a total of ~102GB. The files are
> portable across machines. In other words 128GB RAM is more than sufficient
> to hold the entire chain, reducing paging costs. However even an 8GB RAM
> machine with 4 cores maintains healthy performance. We have not tested at
> intermediate RAM levels (e.g. 64 or 32 GB) but these sizes are expected to
> be very performant.
>
> I should point out that libbitcoin does not use a UTXO database nor does it
> cache UTXOs, so there is no problem with scale as the blockchain grows.
> Per-input performance is linear across the full range of the chain
> regardless of hardware (assuming a drive of sufficient space for the chain).
> Outputs are read, and updated as spent, directly via transaction records.
> The cost of all store reads, including outputs, activation state, block and
> transaction duplicate checks, difficulty, and timestamps, is incurred all at
> once and captured in the pµs value below (averaging 15 µs per input). The
> cost of storing the new block, including spent output information, is
> reflected in the dµs value below (averaging 5 µs per input). Reading and
> writing independently from validation and other processing is not only huge
> boost for testability but also dramatically simplifies code and exposes
> important optimizations. For example, all per-input operations with the
> exception of block check are parallelized either by tx or input.
>
> Our script validation uses a fully-generalized script processor:
>
> https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin
> /impl/machine/interpreter.ipp
>
> This makes Bitcoin script easy to understand/ and test, and safer to
> maintain. Despite this high level of generalization compared to other
> implementations, we are achieving a script validation cost of 41 µs per
> input. This is 13.5% faster than we achieve using libconsensus (which can be
> swapped in via a configuration option). Testing has confirmed that this is
> entirely the consequence of the need to serialize each transaction and
> previous output for that library’s interface, and because each must then be
> deserialized internal to that library. As such libconsensus is not viable
> for high performance implementations, such as mining, but the cost is small
> enough to not adversely impact other scenarios.
>
> e
>
>
> Block Totals (milliseconds)
> ----------------------------------
> txs: count of transactions
> ins: count of inputs
> wms: wait on previous block
> vms: validate (*) time
>
> Per Input (microseconds)
> ----------------------------------
> vµs: validate (*)
> rµs: deserialization (read from wire)
> cµs: check block (context-free validation)
> pµs: populate (all database reads)
> aµs: accept block (contextual validation)
> sµs: connect block (script verify)
> dµs: deposit (all database writes)
>
> (*) validate is the time from start of block-check to the end of
> block-connect. This is the time that it takes to fully validate a block
> object. This is important in that it is the time, after receipt and
> deserialization (time also shown) of a (whole) block from a peer, that a
> miner must wait until a notification of validity is available.
>
> 09:28:37.985756 INFO [blockchain] Block [441133] 2412 txs 4214 ins  188 wms
> 307 vms   73 vµs    1 rµs    5 cµs   19 pµs    2 aµs   48 sµs    5 dµs
> 09:28:38.295939 INFO [blockchain] Block [441134] 2691 txs 4863 ins  291 wms
> 286 vms   59 vµs    2 rµs    5 cµs   21 pµs    1 aµs   31 sµs    5 dµs
> 09:28:38.751393 INFO [blockchain] Block [441135] 2136 txs 4484 ins  270 wms
> 434 vms   97 vµs    1 rµs    4 cµs   26 pµs    2 aµs   65 sµs    5 dµs
> 09:28:38.966397 INFO [blockchain] Block [441136] 1918 txs 3528 ins  427 wms
> 198 vms   56 vµs    1 rµs    4 cµs   15 pµs    1 aµs   36 sµs    5 dµs
> 09:28:39.170047 INFO [blockchain] Block [441137] 1915 txs 3258 ins  185 wms
> 186 vms   57 vµs    2 rµs    5 cµs   13 pµs    2 aµs   37 sµs    5 dµs
> 09:28:39.474600 INFO [blockchain] Block [441138] 2367 txs 4197 ins  172 wms
> 284 vms   68 vµs    1 rµs    5 cµs   17 pµs    2 aµs   45 sµs    5 dµs
> 09:28:39.798699 INFO [blockchain] Block [441139] 2386 txs 4434 ins  270 wms
> 302 vms   68 vµs    1 rµs    4 cµs   18 pµs    1 aµs   45 sµs    5 dµs
> 09:28:40.161076 INFO [blockchain] Block [441140] 2150 txs 4648 ins  293 wms
> 342 vms   73 vµs    1 rµs    4 cµs   15 pµs    1 aµs   53 sµs    4 dµs
> 09:28:40.493881 INFO [blockchain] Block [441141] 1768 txs 4754 ins  325 wms
> 313 vms   66 vµs    2 rµs    4 cµs   19 pµs    1 aµs   42 sµs    4 dµs
> 09:28:40.688693 INFO [blockchain] Block [441142] 1860 txs 3726 ins  301 wms
> 178 vms   48 vµs    2 rµs    4 cµs   12 pµs    1 aµs   30 sµs    5 dµs
> 09:28:40.977133 INFO [blockchain] Block [441143] 2737 txs 4598 ins  161 wms
> 265 vms   58 vµs    1 rµs    5 cµs   18 pµs    1 aµs   34 sµs    5 dµs
> 09:28:41.295907 INFO [blockchain] Block [441144] 1530 txs 4592 ins  251 wms
> 301 vms   66 vµs    1 rµs    4 cµs   16 pµs    1 aµs   45 sµs    4 dµs
> 09:28:41.539977 INFO [blockchain] Block [441145] 1625 txs 3637 ins  282 wms
> 228 vms   63 vµs    1 rµs    4 cµs   10 pµs    1 aµs   47 sµs    4 dµs
> 09:28:41.823650 INFO [blockchain] Block [441146] 2820 txs 4539 ins  210 wms
> 261 vms   57 vµs    2 rµs    5 cµs   17 pµs    1 aµs   34 sµs    5 dµs
> 09:28:42.163806 INFO [blockchain] Block [441147] 1395 txs 4515 ins  245 wms
> 321 vms   71 vµs    1 rµs    3 cµs   20 pµs    2 aµs   46 sµs    4 dµs
> 09:28:42.476815 INFO [blockchain] Block [441148]  999 txs 5157 ins  312 wms
> 295 vms   57 vµs    1 rµs    3 cµs   15 pµs    1 aµs   39 sµs    3 dµs
> 09:28:42.813068 INFO [blockchain] Block [441149] 1026 txs 5413 ins  284 wms
> 313 vms   58 vµs    1 rµs    2 cµs   18 pµs    1 aµs   37 sµs    4 dµs
> 09:28:42.813827 INFO [blockchain] Block [441150]    9 txs    8 ins  323 wms
> 1 vms   67 vµs    4 rµs   17 cµs   10 pµs    9 aµs   30 sµs   15 dµs
> 09:28:43.101809 INFO [blockchain] Block [441151] 2669 txs 4608 ins    0 wms
> 253 vms   55 vµs    1 rµs    4 cµs   16 pµs    1 aµs   34 sµs    5 dµs
> 09:28:43.389835 INFO [blockchain] Block [441152] 2870 txs 4474 ins  238 wms
> 264 vms   59 vµs    2 rµs    5 cµs   18 pµs    1 aµs   35 sµs    5 dµs
> 09:28:43.696813 INFO [blockchain] Block [441153] 2451 txs 4484 ins  249 wms
> 285 vms   64 vµs    1 rµs    5 cµs   16 pµs    1 aµs   41 sµs    5 dµs
> 09:28:44.005768 INFO [blockchain] Block [441154] 2217 txs 4515 ins  274 wms
> 288 vms   64 vµs    1 rµs    4 cµs   19 pµs    2 aµs   39 sµs    5 dµs
> 09:28:44.488287 INFO [blockchain] Block [441155] 1407 txs 4516 ins  279 wms
> 462 vms  102 vµs    1 rµs    3 cµs   37 pµs    2 aµs   60 sµs    5 dµs
> 09:28:45.096894 INFO [blockchain] Block [441156] 1291 txs 3971 ins  456 wms
> 591 vms  149 vµs    1 rµs    4 cµs   34 pµs    1 aµs  109 sµs    4 dµs
> 09:28:45.373956 INFO [blockchain] Block [441157] 2556 txs 4412 ins  582 wms
> 255 vms   58 vµs    1 rµs    4 cµs   13 pµs    1 aµs   39 sµs    5 dµs
> 09:28:45.751849 INFO [blockchain] Block [441158] 2122 txs 4869 ins  245 wms
> 356 vms   73 vµs    1 rµs    4 cµs   18 pµs    1 aµs   50 sµs    4 dµs
> 09:28:45.752609 INFO [blockchain] Block [441159]   10 txs    9 ins  359 wms
> 0 vms   54 vµs    2 rµs   12 cµs    6 pµs    7 aµs   30 sµs   17 dµs
> 09:28:46.167439 INFO [blockchain] Block [441160] 2260 txs 4798 ins    0 wms
> 380 vms   79 vµs    1 rµs    3 cµs   24 pµs    1 aµs   51 sµs    5 dµs
> 09:28:46.534458 INFO [blockchain] Block [441161] 2037 txs 4720 ins  373 wms
> 347 vms   74 vµs    1 rµs    4 cµs   16 pµs    1 aµs   53 sµs    4 dµs
> 09:28:46.786271 INFO [blockchain] Block [441162] 1478 txs 4044 ins  328 wms
> 236 vms   58 vµs    2 rµs    4 cµs   14 pµs    2 aµs   39 sµs    4 dµs
> 09:28:47.062390 INFO [blockchain] Block [441163] 2620 txs 4455 ins  217 wms
> 253 vms   57 vµs    2 rµs    5 cµs   13 pµs    1 aµs   37 sµs    5 dµs
> 09:28:47.063237 INFO [blockchain] Block [441164]   10 txs    9 ins  255 wms
> 1 vms   69 vµs    3 rµs   15 cµs    7 pµs    7 aµs   40 sµs   14 dµs
> 09:28:47.381445 INFO [blockchain] Block [441165] 2490 txs 4611 ins    0 wms
> 285 vms   62 vµs    1 rµs    4 cµs   16 pµs    1 aµs   41 sµs    5 dµs
> 09:28:47.697020 INFO [blockchain] Block [441166] 2266 txs 4225 ins  271 wms
> 297 vms   70 vµs    2 rµs    4 cµs   15 pµs    1 aµs   49 sµs    4 dµs
> 09:28:47.904200 INFO [blockchain] Block [441167] 1689 txs 3652 ins  283 wms
> 191 vms   52 vµs    2 rµs    4 cµs    9 pµs    1 aµs   38 sµs    4 dµs
> 09:28:48.215998 INFO [blockchain] Block [441168] 2301 txs 4441 ins  174 wms
> 293 vms   66 vµs    2 rµs    4 cµs   12 pµs    1 aµs   48 sµs    4 dµs
> 09:28:48.490337 INFO [blockchain] Block [441169] 2823 txs 4950 ins  273 wms
> 250 vms   50 vµs    2 rµs    5 cµs   15 pµs    1 aµs   30 sµs    5 dµs
> 09:28:48.771430 INFO [blockchain] Block [441170] 2728 txs 4938 ins  232 wms
> 258 vms   52 vµs    2 rµs    4 cµs   16 pµs    1 aµs   30 sµs    5 dµs
> 09:28:49.252750 INFO [blockchain] Block [441171] 1967 txs 4328 ins  242 wms
> 464 vms  107 vµs    2 rµs    4 cµs   14 pµs    2 aµs   87 sµs    4 dµs
> 09:28:49.564182 INFO [blockchain] Block [441172] 2401 txs 4575 ins  451 wms
> 291 vms   64 vµs    1 rµs    4 cµs   13 pµs    2 aµs   45 sµs    5 dµs
> 09:28:50.094058 INFO [blockchain] Block [441173] 1482 txs 4119 ins  281 wms
> 514 vms  125 vµs    1 rµs    4 cµs   10 pµs    2 aµs  108 sµs    4 dµs
> 09:28:50.446385 INFO [blockchain] Block [441174] 2216 txs 4791 ins  502 wms
> 331 vms   69 vµs    1 rµs    4 cµs   15 pµs    1 aµs   49 sµs    4 dµs
> 09:28:50.860666 INFO [blockchain] Block [441175] 1172 txs 5169 ins  322 wms
> 396 vms   77 vµs    1 rµs    3 cµs   12 pµs    1 aµs   61 sµs    4 dµs
> 09:28:51.197153 INFO [blockchain] Block [441176] 2022 txs 4642 ins  388 wms
> 317 vms   68 vµs    1 rµs    4 cµs   13 pµs    2 aµs   50 sµs    4 dµs
> 09:28:51.512118 INFO [blockchain] Block [441177] 1852 txs 4358 ins  307 wms
> 296 vms   68 vµs    1 rµs    4 cµs   14 pµs    2 aµs   48 sµs    4 dµs
> 09:28:51.859025 INFO [blockchain] Block [441178] 2319 txs 5030 ins  285 wms
> 326 vms   65 vµs    1 rµs    4 cµs   13 pµs    1 aµs   47 sµs    4 dµs
> 09:28:52.185014 INFO [blockchain] Block [441179] 2612 txs 4344 ins  314 wms
> 305 vms   70 vµs    1 rµs    5 cµs   14 pµs    2 aµs   49 sµs    5 dµs
> 09:28:52.475703 INFO [blockchain] Block [441180] 2262 txs 4030 ins  287 wms
> 272 vms   68 vµs    2 rµs    5 cµs   15 pµs    2 aµs   46 sµs    5 dµs
> 09:28:52.798759 INFO [blockchain] Block [441181] 2885 txs 5092 ins  253 wms
> 299 vms   59 vµs    2 rµs    4 cµs   17 pµs    1 aµs   36 sµs    5 dµs
> 09:28:53.078336 INFO [blockchain] Block [441182] 2436 txs 4915 ins  282 wms
> 257 vms   52 vµs    2 rµs    4 cµs   14 pµs    1 aµs   32 sµs    5 dµs
> 09:28:53.398266 INFO [blockchain] Block [441183] 2843 txs 4446 ins  238 wms
> 298 vms   67 vµs    2 rµs    5 cµs   20 pµs    2 aµs   41 sµs    5 dµs
> 09:28:53.707133 INFO [blockchain] Block [441184] 2324 txs 4476 ins  278 wms
> 288 vms   64 vµs    2 rµs    4 cµs   16 pµs    2 aµs   42 sµs    5 dµs
> 09:28:53.918838 INFO [blockchain] Block [441185] 1512 txs 3339 ins  276 wms
> 196 vms   59 vµs    1 rµs    4 cµs    9 pµs    1 aµs   44 sµs    5 dµs
> 09:28:54.237412 INFO [blockchain] Block [441186] 2067 txs 4655 ins  180 wms
> 299 vms   64 vµs    2 rµs    4 cµs   14 pµs    2 aµs   45 sµs    4 dµs
> 09:28:54.525741 INFO [blockchain] Block [441187] 1403 txs 5186 ins  282 wms
> 268 vms   52 vµs    1 rµs    3 cµs   12 pµs    1 aµs   35 sµs    4 dµs
> 09:28:54.768170 INFO [blockchain] Block [441188]  770 txs 4352 ins  259 wms
> 227 vms   52 vµs    1 rµs    2 cµs   10 pµs    1 aµs   39 sµs    3 dµs
> 09:28:55.050891 INFO [blockchain] Block [441189] 2428 txs 4573 ins  220 wms
> 262 vms   57 vµs    1 rµs    4 cµs   14 pµs    1 aµs   38 sµs    5 dµs
> 09:28:55.129727 INFO [blockchain] Block [441190]  477 txs 1278 ins  261 wms
> 73 vms   57 vµs    1 rµs    4 cµs    4 pµs    2 aµs   47 sµs    4 dµs
> 09:28:55.287538 INFO [blockchain] Block [441191] 1355 txs 2932 ins   66 wms
> 146 vms   50 vµs    1 rµs    4 cµs    8 pµs    1 aµs   37 sµs    4 dµs
> 09:28:55.622908 INFO [blockchain] Block [441192] 1139 txs 5365 ins  136 wms
> 305 vms   57 vµs    1 rµs    3 cµs   12 pµs    1 aµs   41 sµs    6 dµs
> 09:28:55.717459 INFO [blockchain] Block [441193]  904 txs 1852 ins  314 wms
> 86 vms   46 vµs    2 rµs    4 cµs    7 pµs    1 aµs   34 sµs    5 dµs
> 09:28:55.986453 INFO [blockchain] Block [441194]  250 txs 4923 ins   74 wms
> 255 vms   52 vµs    1 rµs    2 cµs   11 pµs    1 aµs   38 sµs    3 dµs
> 09:28:55.987299 INFO [blockchain] Block [441195]    7 txs   10 ins  261 wms
> 1 vms   62 vµs    3 rµs   11 cµs    8 pµs    7 aµs   37 sµs   12 dµs
> 09:28:56.262784 INFO [blockchain] Block [441196] 2502 txs 4894 ins    0 wms
> 243 vms   50 vµs    1 rµs    4 cµs   14 pµs    1 aµs   31 sµs    4 dµs
> 09:28:56.539972 INFO [blockchain] Block [441197] 1963 txs 5490 ins  228 wms
> 257 vms   47 vµs    1 rµs    3 cµs   13 pµs    1 aµs   29 sµs    4 dµs
> 09:28:56.846339 INFO [blockchain] Block [441198] 2574 txs 4683 ins  246 wms
> 281 vms   60 vµs    1 rµs    4 cµs   16 pµs    1 aµs   38 sµs    5 dµs

>
> <end>
>
>