Thanks for your work and this updates Eric!

Very interesting and insightful and makes it very convenient to follow
libbitcoin projects progress :)

Looking forward to the v3 release.

Best regards
Lucas

2016-11-03 23:28 GMT+00:00 Eric Voskuil <eric@???>:

> In my last major update (9/15, below) I said that we had taken a detour
> into validation, in order to get the performance up to a level
> consistent with other aspects of the code. This code has been complete
> for block validation for a couple of weeks, although we still have some
> work to do in reconnecting the pools.
>
> Validation is now and very easy to follow. For example, see the
> "validation" sections of these files:
>
> https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/header.cpp
>
> https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/block.cpp
>
> https://github.com/libbitcoin/libbitcoin/blob/master/src/
> chain/transaction.cpp
>
> Blockchain enhances this simple single-threaded type system validation
> with a small bit of code to parallelize the invocation of input
> validation. This is very fast, and well factored for test.
>
> By populating the necessary metadata and the chain state, validation can
> be executed completely independent of the store. This relies on
> validation metadata on header, block, transaction and input, as well as
> a new chain_state class in libbitcoin:
>
> https://github.com/evoskuil/libbitcoin/blob/master/src/
> chain/chain_state.cpp
>
> This is a big boost for testability and
> simplicity/readability/reliability of the implementation.
>
> In the past week I've taken the time to rewrite our script
> implementation I've covered the details in the extensive comments of
> this pull request:
>
> https://github.com/libbitcoin/libbitcoin/pull/577
>
> This is the first time we've reviewed the script implementation for
> consensus-criticality. You will notice that three consensus bugs have
> been resolved. These would have been avoided by the use of
> libbitcoin-consensus (libconsensus).
>
> We've got a little more work to do in this area, specifically emitting
> precise error codes for script validation failures. Then it's back to
> finish up networking. In the mean time we've disabled initial block
> download (and catch-up sync cannot surpass a multiblock reorg due to
> pool disconnection).
>
> We've also exposed database hash table bucket counts:
>
> https://github.com/libbitcoin/libbitcoin-node/blob/master/
> data/bn.cfg#L80-L87
>
> and database file growth rate:
>
> https://github.com/libbitcoin/libbitcoin-node/blob/master/
> data/bn.cfg#L78-L79
>
> to config. This not only significantly speeds up database unit testing
> but also allows users to tune hash tables for expected use. The current
> values are (sort-of) tuned for mainnet. They are certainly not optimal
> for testnet. But it would be great if someone would optimize these for
> the current chains. We'll probably also configure initial file sizes so
> that users can make optimize for expected use. Note that bucket size
> changes will invalidate an existing hash table.
>
> Another important change has been the addition of corruption detection.
> I wrote about this recently. It is now impossible to corrupt the store
> via hard shutdown and not know it. There is a trade-off in terms of
> precision of that knowledge and performance. The default is in favor of
> performance, but for wallet-like scenarios this can be changed with no
> noticeable impact. This reduces the chance of a hard shutdown flagging a
> database corruption to almost zero, but materially slows block acceptance.
>
> https://github.com/libbitcoin/libbitcoin-node/blob/master/
> data/bn.cfg#L98-L99
>
> At the same time pmienk has integrated boost_log. We now have logs that
> can be rotated, and we have configurable internal rotation (in server
> and node):
>
> https://github.com/libbitcoin/libbitcoin-node/blob/master/
> data/bn.cfg#L3-L15
>
> He's now working on statsd integration via a new log sink, which we
> expect to have complete for v3 as well.
>
> We have also enabled SOCKS5 client support in libbitcoin-client, and in
> bx configuration:
>
> [server]
> socks_proxy = 127.0.0.1:1080
>
> and we have received word that we will have dedicated hidden services
> for both mainnet and testnet, as well as public Server and P2P endpoints
> on libbitcoin.net!
>
> e
>
> On 09/15/2016 04:12 AM, Eric Voskuil wrote:
> > Just wanted to give a quick update on our v3 progress. I didn't make my
> > target date before heading off to Mongolia for a couple of weeks. After
> > getting back I started back up on validating the new network protocols
> > work. I resolved a couple of issues in the new implementation and it was
> > going great. At that point I decided to fully validate the mainnet and
> > testnet chains, as I like to do before a release.
> >
> > As many of you know, the blockchain library is slated for a major rework
> > in v4, so we've been deferring optimizations in that code. For example,
> > we don't even parallelize script validation. But v3 has been looking so
> > good in other areas this started too look even worse by comparison. So
> > we decided to take a week or two to redesign the block and transaction
> > validation code, allowing us to get some significant optimizations
> > implemented. This is coming along nicely and I hope to have some
> > improvements to report soon.
> >
> > Once we complete this work we will need to finish protocol validation.
> > The incoming block protocol is done, working perfectly. We've written
> > the other major protocols, including outgoing block and in/out
> > transaction, just need to verify them as well. The minor protocols have
> > been implemented and working well for some time. I don't spend a lot of
> > time documenting, because there has just been so far to go in just the
> > code. But believe me, it's getting really good.
> >
> > For example the network redesign has allowed me to isolate protocols by
> > version and implement robust version negotiation based on config. So you
> > can specify protocol version min/max in config and the node will
> > perfectly implement that range. So it can simulate older and eventually
> > newer node behavior. It's also really easy to see protocol behavior
> > through source inspection, and will get even better before it ships.
> > Right now I'm thinking about 4 more weeks until we can complete the
> > blockchain work and get back to validating the remaining network
> > protocols. Sorry about the scope creep, I just couldn't help myself :).
> >
> > e
> >
>
>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>
>


--
Best regards

Lucas