Thanks for the feedback.

Yesterday I merged hash_number into unint256_t, removed a bunch of dead code resulting from the borrowing of the latter from satoshi, and redesigned the implementation IAW modern c++ best practices and our style. This area concerned me, as it was big and opaque, and untested, and consensus critical.

I was able to significantly reduce the code size while retaining the ability to visually correlate methods to early satoshi implementation (the BN-based implemention). The test plan is now pretty reasonable. In order to generalize to a pure big bit class and to allow simple construction via "compact numbers" (scientific notation used in work proof), I created a small class to wrap compact numbers. Today I'm going to test this and merge it back to master.

I also renamed chain::script_number to simply chain::number and moved it into the /chain/script directory. This was not really a general math type as its behavior is very specific to bitcoin script. But this also completes the move of all script-specific utilities into the same location.

I've already inlined the interpreter and will soon inline the remaining utilities (operation, number, program). This will give the compiler to option to remove the call stack overhead during script processing, without losing the testability and readability benefits of well-factored code.

Finally, recent changes now make it possible to run Bitcoin script as one would expect to run any program...

// my program (false)
operation::list ops
{
    { opcode::positive_2 },
    { { 2 } },
    { opcode::equal },
    { opcode::not_ }
};

script foo(ops);
program bar(foo, [environment]);
auto result = bar.evaluate();

If (result == error::success)
{
    // script completed
    If (!program.stack_false())
    {
        // number 2 is not data 2
    }
}

... where [environment] contains constant optional parameters, specifically transaction, input_index, fork_flags. These are only necessary for certain opcodes and if not present the ops simply evaluate against defaults.

I can see a libbitcoin-interpreter and/or extensions to bx in the future :).

e

> On Nov 7, 2016, at 1:39 PM, Police Terror <PoliceTerror@???> wrote:
>
> Thanks for all this top quality work. We're looking forward to v3
> release to use in our projects. All the changes are excellent.
>
> Eric Voskuil:
>> In my last major update (9/15, below) I said that we had taken a detour
>> into validation, in order to get the performance up to a level
>> consistent with other aspects of the code. This code has been complete
>> for block validation for a couple of weeks, although we still have some
>> work to do in reconnecting the pools.
>>
>> Validation is now and very easy to follow. For example, see the
>> "validation" sections of these files:
>>
>> https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/header.cpp
>>
>> https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/block.cpp
>>
>> https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/transaction.cpp
>>
>> Blockchain enhances this simple single-threaded type system validation
>> with a small bit of code to parallelize the invocation of input
>> validation. This is very fast, and well factored for test.
>>
>> By populating the necessary metadata and the chain state, validation can
>> be executed completely independent of the store. This relies on
>> validation metadata on header, block, transaction and input, as well as
>> a new chain_state class in libbitcoin:
>>
>> https://github.com/evoskuil/libbitcoin/blob/master/src/chain/chain_state.cpp
>>
>> This is a big boost for testability and
>> simplicity/readability/reliability of the implementation.
>>
>> In the past week I've taken the time to rewrite our script
>> implementation I've covered the details in the extensive comments of
>> this pull request:
>>
>> https://github.com/libbitcoin/libbitcoin/pull/577
>>
>> This is the first time we've reviewed the script implementation for
>> consensus-criticality. You will notice that three consensus bugs have
>> been resolved. These would have been avoided by the use of
>> libbitcoin-consensus (libconsensus).
>>
>> We've got a little more work to do in this area, specifically emitting
>> precise error codes for script validation failures. Then it's back to
>> finish up networking. In the mean time we've disabled initial block
>> download (and catch-up sync cannot surpass a multiblock reorg due to
>> pool disconnection).
>>
>> We've also exposed database hash table bucket counts:
>>
>> https://github.com/libbitcoin/libbitcoin-node/blob/master/data/bn.cfg#L80-L87
>>
>> and database file growth rate:
>>
>> https://github.com/libbitcoin/libbitcoin-node/blob/master/data/bn.cfg#L78-L79
>>
>> to config. This not only significantly speeds up database unit testing
>> but also allows users to tune hash tables for expected use. The current
>> values are (sort-of) tuned for mainnet. They are certainly not optimal
>> for testnet. But it would be great if someone would optimize these for
>> the current chains. We'll probably also configure initial file sizes so
>> that users can make optimize for expected use. Note that bucket size
>> changes will invalidate an existing hash table.
>>
>> Another important change has been the addition of corruption detection.
>> I wrote about this recently. It is now impossible to corrupt the store
>> via hard shutdown and not know it. There is a trade-off in terms of
>> precision of that knowledge and performance. The default is in favor of
>> performance, but for wallet-like scenarios this can be changed with no
>> noticeable impact. This reduces the chance of a hard shutdown flagging a
>> database corruption to almost zero, but materially slows block acceptance.
>>
>> https://github.com/libbitcoin/libbitcoin-node/blob/master/data/bn.cfg#L98-L99
>>
>> At the same time pmienk has integrated boost_log. We now have logs that
>> can be rotated, and we have configurable internal rotation (in server
>> and node):
>>
>> https://github.com/libbitcoin/libbitcoin-node/blob/master/data/bn.cfg#L3-L15
>>
>> He's now working on statsd integration via a new log sink, which we
>> expect to have complete for v3 as well.
>>
>> We have also enabled SOCKS5 client support in libbitcoin-client, and in
>> bx configuration:
>>
>> [server]
>> socks_proxy = 127.0.0.1:1080
>>
>> and we have received word that we will have dedicated hidden services
>> for both mainnet and testnet, as well as public Server and P2P endpoints
>> on libbitcoin.net!
>>
>> e
>>
>>> On 09/15/2016 04:12 AM, Eric Voskuil wrote:
>>> Just wanted to give a quick update on our v3 progress. I didn't make my
>>> target date before heading off to Mongolia for a couple of weeks. After
>>> getting back I started back up on validating the new network protocols
>>> work. I resolved a couple of issues in the new implementation and it was
>>> going great. At that point I decided to fully validate the mainnet and
>>> testnet chains, as I like to do before a release.
>>>
>>> As many of you know, the blockchain library is slated for a major rework
>>> in v4, so we've been deferring optimizations in that code. For example,
>>> we don't even parallelize script validation. But v3 has been looking so
>>> good in other areas this started too look even worse by comparison. So
>>> we decided to take a week or two to redesign the block and transaction
>>> validation code, allowing us to get some significant optimizations
>>> implemented. This is coming along nicely and I hope to have some
>>> improvements to report soon.
>>>
>>> Once we complete this work we will need to finish protocol validation.
>>> The incoming block protocol is done, working perfectly. We've written
>>> the other major protocols, including outgoing block and in/out
>>> transaction, just need to verify them as well. The minor protocols have
>>> been implemented and working well for some time. I don't spend a lot of
>>> time documenting, because there has just been so far to go in just the
>>> code. But believe me, it's getting really good.
>>>
>>> For example the network redesign has allowed me to isolate protocols by
>>> version and implement robust version negotiation based on config. So you
>>> can specify protocol version min/max in config and the node will
>>> perfectly implement that range. So it can simulate older and eventually
>>> newer node behavior. It's also really easy to see protocol behavior
>>> through source inspection, and will get even better before it ships.
>>> Right now I'm thinking about 4 more weeks until we can complete the
>>> blockchain work and get back to validating the remaining network
>>> protocols. Sorry about the scope creep, I just couldn't help myself :).
>>>
>>> e
>>>
>>
>>
>>
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin