Yes I'm very excited for this release. I'm currently working on porting
the C wrapper to the latest API.
Neill Miller:
> Excellent news -- looking forward to v3 all around!
>
> -Neill.
>
> On Fri, Dec 09, 2016 at 05:29:21PM -0800, Eric Voskuil wrote:
>> Reposting as text, formatted HTML got moderated :/.
>>
>> ---
>>
>> Weve spent a good amount of time on node, following a rewrite of validation
>> and script processing, and on the heels of the previous network redesign.
>> Im very happy with the results. We have a very stable node and excellent
>> block validation performance. We have been lucky to have the full time
>> support of Bitprim in operational testing of the node on both high and low
>> end platforms. You may recall that we previously achieved full chain sync of
>> about 15 minutes (all blocks checkpointed). We have since validated the full
>> mainnet chain countless times. Despite the fact that our networking is still
>> not optimal in terms of catch-up sync (blocks-first), we are able to sync
>> 0-295,000 and validate 295,001-top (Core parameters) in about 7 hrs. I
>> expect that number to come down significantly in v4 owing to more advanced
>> networking.
>>
>> Below is a sample of the new console reporting format for block acceptance,
>> on a high end server (SSD, 12 Cores, 128GB RAM). You can see that we are
>> processing at a rate of about 3-4 blocks per second at the top of the
>> current chain. But given network limitations this is not the full story. As
>> you can see our median time process a block is about 62.5 µs per input (1 µs
>> = one millionth of a second), or full block validation times (i.e. excluding
>> the empty blocks) averaging under .3 seconds. We have not tested on more
>> than 12 cores. The number of cores used (and validation thread priority) is
>> configurable, so we can easily see improvement in speed from 1-12 cores
>> presently. I expect those numbers would continue to improve with additional
>> cores up to some point of diminishing returns.
>>
>> A libbitcoin node store is 3 files for a total of ~102GB. The files are
>> portable across machines. In other words 128GB RAM is more than sufficient
>> to hold the entire chain, reducing paging costs. However even an 8GB RAM
>> machine with 4 cores maintains healthy performance. We have not tested at
>> intermediate RAM levels (e.g. 64 or 32 GB) but these sizes are expected to
>> be very performant.
>>
>> I should point out that libbitcoin does not use a UTXO database nor does it
>> cache UTXOs, so there is no problem with scale as the blockchain grows.
>> Per-input performance is linear across the full range of the chain
>> regardless of hardware (assuming a drive of sufficient space for the chain).
>> Outputs are read, and updated as spent, directly via transaction records.
>> The cost of all store reads, including outputs, activation state, block and
>> transaction duplicate checks, difficulty, and timestamps, is incurred all at
>> once and captured in the pµs value below (averaging 15 µs per input). The
>> cost of storing the new block, including spent output information, is
>> reflected in the dµs value below (averaging 5 µs per input). Reading and
>> writing independently from validation and other processing is not only huge
>> boost for testability but also dramatically simplifies code and exposes
>> important optimizations. For example, all per-input operations with the
>> exception of block check are parallelized either by tx or input.
>>
>> Our script validation uses a fully-generalized script processor:
>>
>> https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin
>> /impl/machine/interpreter.ipp
>>
>> This makes Bitcoin script easy to understand/ and test, and safer to
>> maintain. Despite this high level of generalization compared to other
>> implementations, we are achieving a script validation cost of 41 µs per
>> input. This is 13.5% faster than we achieve using libconsensus (which can be
>> swapped in via a configuration option). Testing has confirmed that this is
>> entirely the consequence of the need to serialize each transaction and
>> previous output for that librarys interface, and because each must then be
>> deserialized internal to that library. As such libconsensus is not viable
>> for high performance implementations, such as mining, but the cost is small
>> enough to not adversely impact other scenarios.
>>
>> e
>>
>>
>> Block Totals (milliseconds)
>> ----------------------------------
>> txs: count of transactions
>> ins: count of inputs
>> wms: wait on previous block
>> vms: validate (*) time
>>
>> Per Input (microseconds)
>> ----------------------------------
>> vµs: validate (*)
>> rµs: deserialization (read from wire)
>> cµs: check block (context-free validation)
>> pµs: populate (all database reads)
>> aµs: accept block (contextual validation)
>> sµs: connect block (script verify)
>> dµs: deposit (all database writes)
>>
>> (*) validate is the time from start of block-check to the end of
>> block-connect. This is the time that it takes to fully validate a block
>> object. This is important in that it is the time, after receipt and
>> deserialization (time also shown) of a (whole) block from a peer, that a
>> miner must wait until a notification of validity is available.
>>
>> 09:28:37.985756 INFO [blockchain] Block [441133] 2412 txs 4214 ins 188 wms
>> 307 vms 73 vµs 1 rµs 5 cµs 19 pµs 2 aµs 48 sµs 5 dµs
>> 09:28:38.295939 INFO [blockchain] Block [441134] 2691 txs 4863 ins 291 wms
>> 286 vms 59 vµs 2 rµs 5 cµs 21 pµs 1 aµs 31 sµs 5 dµs
>> 09:28:38.751393 INFO [blockchain] Block [441135] 2136 txs 4484 ins 270 wms
>> 434 vms 97 vµs 1 rµs 4 cµs 26 pµs 2 aµs 65 sµs 5 dµs
>> 09:28:38.966397 INFO [blockchain] Block [441136] 1918 txs 3528 ins 427 wms
>> 198 vms 56 vµs 1 rµs 4 cµs 15 pµs 1 aµs 36 sµs 5 dµs
>> 09:28:39.170047 INFO [blockchain] Block [441137] 1915 txs 3258 ins 185 wms
>> 186 vms 57 vµs 2 rµs 5 cµs 13 pµs 2 aµs 37 sµs 5 dµs
>> 09:28:39.474600 INFO [blockchain] Block [441138] 2367 txs 4197 ins 172 wms
>> 284 vms 68 vµs 1 rµs 5 cµs 17 pµs 2 aµs 45 sµs 5 dµs
>> 09:28:39.798699 INFO [blockchain] Block [441139] 2386 txs 4434 ins 270 wms
>> 302 vms 68 vµs 1 rµs 4 cµs 18 pµs 1 aµs 45 sµs 5 dµs
>> 09:28:40.161076 INFO [blockchain] Block [441140] 2150 txs 4648 ins 293 wms
>> 342 vms 73 vµs 1 rµs 4 cµs 15 pµs 1 aµs 53 sµs 4 dµs
>> 09:28:40.493881 INFO [blockchain] Block [441141] 1768 txs 4754 ins 325 wms
>> 313 vms 66 vµs 2 rµs 4 cµs 19 pµs 1 aµs 42 sµs 4 dµs
>> 09:28:40.688693 INFO [blockchain] Block [441142] 1860 txs 3726 ins 301 wms
>> 178 vms 48 vµs 2 rµs 4 cµs 12 pµs 1 aµs 30 sµs 5 dµs
>> 09:28:40.977133 INFO [blockchain] Block [441143] 2737 txs 4598 ins 161 wms
>> 265 vms 58 vµs 1 rµs 5 cµs 18 pµs 1 aµs 34 sµs 5 dµs
>> 09:28:41.295907 INFO [blockchain] Block [441144] 1530 txs 4592 ins 251 wms
>> 301 vms 66 vµs 1 rµs 4 cµs 16 pµs 1 aµs 45 sµs 4 dµs
>> 09:28:41.539977 INFO [blockchain] Block [441145] 1625 txs 3637 ins 282 wms
>> 228 vms 63 vµs 1 rµs 4 cµs 10 pµs 1 aµs 47 sµs 4 dµs
>> 09:28:41.823650 INFO [blockchain] Block [441146] 2820 txs 4539 ins 210 wms
>> 261 vms 57 vµs 2 rµs 5 cµs 17 pµs 1 aµs 34 sµs 5 dµs
>> 09:28:42.163806 INFO [blockchain] Block [441147] 1395 txs 4515 ins 245 wms
>> 321 vms 71 vµs 1 rµs 3 cµs 20 pµs 2 aµs 46 sµs 4 dµs
>> 09:28:42.476815 INFO [blockchain] Block [441148] 999 txs 5157 ins 312 wms
>> 295 vms 57 vµs 1 rµs 3 cµs 15 pµs 1 aµs 39 sµs 3 dµs
>> 09:28:42.813068 INFO [blockchain] Block [441149] 1026 txs 5413 ins 284 wms
>> 313 vms 58 vµs 1 rµs 2 cµs 18 pµs 1 aµs 37 sµs 4 dµs
>> 09:28:42.813827 INFO [blockchain] Block [441150] 9 txs 8 ins 323 wms
>> 1 vms 67 vµs 4 rµs 17 cµs 10 pµs 9 aµs 30 sµs 15 dµs
>> 09:28:43.101809 INFO [blockchain] Block [441151] 2669 txs 4608 ins 0 wms
>> 253 vms 55 vµs 1 rµs 4 cµs 16 pµs 1 aµs 34 sµs 5 dµs
>> 09:28:43.389835 INFO [blockchain] Block [441152] 2870 txs 4474 ins 238 wms
>> 264 vms 59 vµs 2 rµs 5 cµs 18 pµs 1 aµs 35 sµs 5 dµs
>> 09:28:43.696813 INFO [blockchain] Block [441153] 2451 txs 4484 ins 249 wms
>> 285 vms 64 vµs 1 rµs 5 cµs 16 pµs 1 aµs 41 sµs 5 dµs
>> 09:28:44.005768 INFO [blockchain] Block [441154] 2217 txs 4515 ins 274 wms
>> 288 vms 64 vµs 1 rµs 4 cµs 19 pµs 2 aµs 39 sµs 5 dµs
>> 09:28:44.488287 INFO [blockchain] Block [441155] 1407 txs 4516 ins 279 wms
>> 462 vms 102 vµs 1 rµs 3 cµs 37 pµs 2 aµs 60 sµs 5 dµs
>> 09:28:45.096894 INFO [blockchain] Block [441156] 1291 txs 3971 ins 456 wms
>> 591 vms 149 vµs 1 rµs 4 cµs 34 pµs 1 aµs 109 sµs 4 dµs
>> 09:28:45.373956 INFO [blockchain] Block [441157] 2556 txs 4412 ins 582 wms
>> 255 vms 58 vµs 1 rµs 4 cµs 13 pµs 1 aµs 39 sµs 5 dµs
>> 09:28:45.751849 INFO [blockchain] Block [441158] 2122 txs 4869 ins 245 wms
>> 356 vms 73 vµs 1 rµs 4 cµs 18 pµs 1 aµs 50 sµs 4 dµs
>> 09:28:45.752609 INFO [blockchain] Block [441159] 10 txs 9 ins 359 wms
>> 0 vms 54 vµs 2 rµs 12 cµs 6 pµs 7 aµs 30 sµs 17 dµs
>> 09:28:46.167439 INFO [blockchain] Block [441160] 2260 txs 4798 ins 0 wms
>> 380 vms 79 vµs 1 rµs 3 cµs 24 pµs 1 aµs 51 sµs 5 dµs
>> 09:28:46.534458 INFO [blockchain] Block [441161] 2037 txs 4720 ins 373 wms
>> 347 vms 74 vµs 1 rµs 4 cµs 16 pµs 1 aµs 53 sµs 4 dµs
>> 09:28:46.786271 INFO [blockchain] Block [441162] 1478 txs 4044 ins 328 wms
>> 236 vms 58 vµs 2 rµs 4 cµs 14 pµs 2 aµs 39 sµs 4 dµs
>> 09:28:47.062390 INFO [blockchain] Block [441163] 2620 txs 4455 ins 217 wms
>> 253 vms 57 vµs 2 rµs 5 cµs 13 pµs 1 aµs 37 sµs 5 dµs
>> 09:28:47.063237 INFO [blockchain] Block [441164] 10 txs 9 ins 255 wms
>> 1 vms 69 vµs 3 rµs 15 cµs 7 pµs 7 aµs 40 sµs 14 dµs
>> 09:28:47.381445 INFO [blockchain] Block [441165] 2490 txs 4611 ins 0 wms
>> 285 vms 62 vµs 1 rµs 4 cµs 16 pµs 1 aµs 41 sµs 5 dµs
>> 09:28:47.697020 INFO [blockchain] Block [441166] 2266 txs 4225 ins 271 wms
>> 297 vms 70 vµs 2 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
>> 09:28:47.904200 INFO [blockchain] Block [441167] 1689 txs 3652 ins 283 wms
>> 191 vms 52 vµs 2 rµs 4 cµs 9 pµs 1 aµs 38 sµs 4 dµs
>> 09:28:48.215998 INFO [blockchain] Block [441168] 2301 txs 4441 ins 174 wms
>> 293 vms 66 vµs 2 rµs 4 cµs 12 pµs 1 aµs 48 sµs 4 dµs
>> 09:28:48.490337 INFO [blockchain] Block [441169] 2823 txs 4950 ins 273 wms
>> 250 vms 50 vµs 2 rµs 5 cµs 15 pµs 1 aµs 30 sµs 5 dµs
>> 09:28:48.771430 INFO [blockchain] Block [441170] 2728 txs 4938 ins 232 wms
>> 258 vms 52 vµs 2 rµs 4 cµs 16 pµs 1 aµs 30 sµs 5 dµs
>> 09:28:49.252750 INFO [blockchain] Block [441171] 1967 txs 4328 ins 242 wms
>> 464 vms 107 vµs 2 rµs 4 cµs 14 pµs 2 aµs 87 sµs 4 dµs
>> 09:28:49.564182 INFO [blockchain] Block [441172] 2401 txs 4575 ins 451 wms
>> 291 vms 64 vµs 1 rµs 4 cµs 13 pµs 2 aµs 45 sµs 5 dµs
>> 09:28:50.094058 INFO [blockchain] Block [441173] 1482 txs 4119 ins 281 wms
>> 514 vms 125 vµs 1 rµs 4 cµs 10 pµs 2 aµs 108 sµs 4 dµs
>> 09:28:50.446385 INFO [blockchain] Block [441174] 2216 txs 4791 ins 502 wms
>> 331 vms 69 vµs 1 rµs 4 cµs 15 pµs 1 aµs 49 sµs 4 dµs
>> 09:28:50.860666 INFO [blockchain] Block [441175] 1172 txs 5169 ins 322 wms
>> 396 vms 77 vµs 1 rµs 3 cµs 12 pµs 1 aµs 61 sµs 4 dµs
>> 09:28:51.197153 INFO [blockchain] Block [441176] 2022 txs 4642 ins 388 wms
>> 317 vms 68 vµs 1 rµs 4 cµs 13 pµs 2 aµs 50 sµs 4 dµs
>> 09:28:51.512118 INFO [blockchain] Block [441177] 1852 txs 4358 ins 307 wms
>> 296 vms 68 vµs 1 rµs 4 cµs 14 pµs 2 aµs 48 sµs 4 dµs
>> 09:28:51.859025 INFO [blockchain] Block [441178] 2319 txs 5030 ins 285 wms
>> 326 vms 65 vµs 1 rµs 4 cµs 13 pµs 1 aµs 47 sµs 4 dµs
>> 09:28:52.185014 INFO [blockchain] Block [441179] 2612 txs 4344 ins 314 wms
>> 305 vms 70 vµs 1 rµs 5 cµs 14 pµs 2 aµs 49 sµs 5 dµs
>> 09:28:52.475703 INFO [blockchain] Block [441180] 2262 txs 4030 ins 287 wms
>> 272 vms 68 vµs 2 rµs 5 cµs 15 pµs 2 aµs 46 sµs 5 dµs
>> 09:28:52.798759 INFO [blockchain] Block [441181] 2885 txs 5092 ins 253 wms
>> 299 vms 59 vµs 2 rµs 4 cµs 17 pµs 1 aµs 36 sµs 5 dµs
>> 09:28:53.078336 INFO [blockchain] Block [441182] 2436 txs 4915 ins 282 wms
>> 257 vms 52 vµs 2 rµs 4 cµs 14 pµs 1 aµs 32 sµs 5 dµs
>> 09:28:53.398266 INFO [blockchain] Block [441183] 2843 txs 4446 ins 238 wms
>> 298 vms 67 vµs 2 rµs 5 cµs 20 pµs 2 aµs 41 sµs 5 dµs
>> 09:28:53.707133 INFO [blockchain] Block [441184] 2324 txs 4476 ins 278 wms
>> 288 vms 64 vµs 2 rµs 4 cµs 16 pµs 2 aµs 42 sµs 5 dµs
>> 09:28:53.918838 INFO [blockchain] Block [441185] 1512 txs 3339 ins 276 wms
>> 196 vms 59 vµs 1 rµs 4 cµs 9 pµs 1 aµs 44 sµs 5 dµs
>> 09:28:54.237412 INFO [blockchain] Block [441186] 2067 txs 4655 ins 180 wms
>> 299 vms 64 vµs 2 rµs 4 cµs 14 pµs 2 aµs 45 sµs 4 dµs
>> 09:28:54.525741 INFO [blockchain] Block [441187] 1403 txs 5186 ins 282 wms
>> 268 vms 52 vµs 1 rµs 3 cµs 12 pµs 1 aµs 35 sµs 4 dµs
>> 09:28:54.768170 INFO [blockchain] Block [441188] 770 txs 4352 ins 259 wms
>> 227 vms 52 vµs 1 rµs 2 cµs 10 pµs 1 aµs 39 sµs 3 dµs
>> 09:28:55.050891 INFO [blockchain] Block [441189] 2428 txs 4573 ins 220 wms
>> 262 vms 57 vµs 1 rµs 4 cµs 14 pµs 1 aµs 38 sµs 5 dµs
>> 09:28:55.129727 INFO [blockchain] Block [441190] 477 txs 1278 ins 261 wms
>> 73 vms 57 vµs 1 rµs 4 cµs 4 pµs 2 aµs 47 sµs 4 dµs
>> 09:28:55.287538 INFO [blockchain] Block [441191] 1355 txs 2932 ins 66 wms
>> 146 vms 50 vµs 1 rµs 4 cµs 8 pµs 1 aµs 37 sµs 4 dµs
>> 09:28:55.622908 INFO [blockchain] Block [441192] 1139 txs 5365 ins 136 wms
>> 305 vms 57 vµs 1 rµs 3 cµs 12 pµs 1 aµs 41 sµs 6 dµs
>> 09:28:55.717459 INFO [blockchain] Block [441193] 904 txs 1852 ins 314 wms
>> 86 vms 46 vµs 2 rµs 4 cµs 7 pµs 1 aµs 34 sµs 5 dµs
>> 09:28:55.986453 INFO [blockchain] Block [441194] 250 txs 4923 ins 74 wms
>> 255 vms 52 vµs 1 rµs 2 cµs 11 pµs 1 aµs 38 sµs 3 dµs
>> 09:28:55.987299 INFO [blockchain] Block [441195] 7 txs 10 ins 261 wms
>> 1 vms 62 vµs 3 rµs 11 cµs 8 pµs 7 aµs 37 sµs 12 dµs
>> 09:28:56.262784 INFO [blockchain] Block [441196] 2502 txs 4894 ins 0 wms
>> 243 vms 50 vµs 1 rµs 4 cµs 14 pµs 1 aµs 31 sµs 4 dµs
>> 09:28:56.539972 INFO [blockchain] Block [441197] 1963 txs 5490 ins 228 wms
>> 257 vms 47 vµs 1 rµs 3 cµs 13 pµs 1 aµs 29 sµs 4 dµs
>> 09:28:56.846339 INFO [blockchain] Block [441198] 2574 txs 4683 ins 246 wms
>> 281 vms 60 vµs 1 rµs 4 cµs 16 pµs 1 aµs 38 sµs 5 dµs
>>
>> <end>
>>
>>
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>
::
Re: [Libbitcoin] libbitcoin node up…
