Bitcoin is not identical though.
The consensus code in Satoshi client has been changed massively. Take a
look yourself.
This seems more like an argument always of our code is best code.
In fact the libbitcoin code is more faithful to the original Satoshi
version.

On 02/03/2015 12:13 AM, Jorge Timón wrote:
> On Sat, Jan 31, 2015 at 12:05 AM, Eric Voskuil <eric@???> wrote:
>>> Well, a "standard" for consensus-related code as you propose may be a
>>> good idea as well, I don't know.
>>> But bitcoin has unique requirements when it comes to consensus-related
>>> code acting identically and that's what libconsensus is about.
>>
>> I'm not aware of any unique requirements. Many systems rely on
>> protocols/standards that if not properly implemented can lead to loss of
>> value. The purpose of protocols is to define these requirements so that
>> there can be multiple implementations.
>
> What defines the chain you listen to (you can think of this as a
> communication channel) is the genesis block, the consensus rules and
> only after that the proof of work.
> If your consensus checks are different from anyone else's you can end
> up listening to a different chain and then...well...people can cheat
> you.
> I believe this property of the system is quite unique.
> There can be multiple implementations of the protocol (which includes
> many non-consensus things like the p2p messages) and that's healthy.
> But all implementations must be functionally identical when it comes
> to consensus checks, period.
>
>>>>> I disagree that libconsensus is against decentralization. Just like a
>>>>> wide use of libsecp256k1 is not against decentralization.
>>>>
>>>> I'm much less concerned about one of multiple implementations of
>>>> well-defined standards rooted in math than definition of an ultimately
>>>> subjective standard based on whatever gets implemented by single
>>>> organization.
>>>
>>> Consensus::VerifyScript() is rooted in math as well, it's just an
>>> interpreter of a language with crypto features.
>>> Ideally once completed libconsensus should be as said touched rarely
>>> and with many organizations involved in those changes, not just one.
>>> Ideally libconsensus would be the only part of bitcoin core that
>>> everybody cares about.
>>>
>>>>> Consensus failures don't increase decentralization in any way that I'm aware of.
>>>>
>>>> One implementation across the entire ecosystem certainly increases
>>>> centralization.
>>>
>>> I believe there will be still be reimpletations for software where
>>> having a C lib as a dependency is problematic, for example, javascript
>>> where the target platform may be an html5 interpreter.
>>> It is always very risky though since they have to be "bug by bug
>>> identical" to prevent consensus failures.
>>
>> In which case the need for conformance testing remains and the reasoning
>> for a common implementation doesn't hold up.
>
> Again, tests can't cover all cases. There's simply too many possible cases.
> It's not that a common implementation for consensus code is *needed*
> but I believe it can be useful for many implementations and in the
> case of libbitcoin I still cannot see any good reason not to depend on
> it in the long term.
>
>>>> If multiple implementations strengthen Bitcoin it's in large part
>>>> because there are multiple implementations of consensus.
>>>
>>> I completely disagree. It is because there are multiple
>>> implementations of everything else (which is most of the code IMO).
>>> For example, bitcoinj having its own interpreter doesn't "strengthen
>>> Bitcoin" in any way that I'm aware of, I believe it only introduces
>>> high risks that come from subtle differences between C and java.
>>
>> It's very possible that higher level language implementations will catch
>> failures in what is considered the "reference" implementation.
>
> I still don't think that's the reason why having multiple
> implementations of the protocol is healthy.
> And I don't think you can extend that to consensus code.
> In fact, I think a clean libconsensus could be a great formal
> specification of the consensus rules.
>
>> OpenSSL has been the reference implementation for the curve, but it
>> turns out that there are problems that are only being discovered due to
>> alternative implementations.
>
> I don't think secp256k1 is not a high priority for OpenSSL and
> certainly bitcoin has special properties when it comes to signatures.
> Checking signatures differently can produce consensus failures!
> Those are some of the reasons for bitcoind to move to libsecp256k1.
> I still don't understand why you think libbitcoin using libconsensus
> is a bad idea but using libsecp256k1 instead of reimplementing your
> own library.
> The main difference in my mind is that libsecp256k1 may actually get
> some non-bitcoin related uses, but that's about it.
>
>>>>> Besides, once finished libconsensus is expected to be very stable,
>>>>> only changing when consensus rules change (ie softforks or hardforks).
>>>>
>>>> Who decides when the rules are to change?
>>>
>>> Certainly not libconsensus maintainers.
>>> Miners decide in the case of softforks, everyone needs to agree in the
>>> case of hardforks.
>>
>> It is not the case that everyone needs to agree. Bitcoin is not that
>> fragile. If it was it would not stand a chance of survival. Miners can
>> forking the code and doing what they want is the example of disagreement.
>>
>> But with one implementation that choice is much harder. At some point
>> down the road, when the bulk of the world's transactions rely on the
>> blockchain, there is likely to be significant controversy of such choices.
>
>
> Miners can only collude to make softforks. They cannot force people to
> accept as valid things that were previously not valid.
> They can mine a chain with other rules, sure, but everybody will ignore them.
> Nobody can push hardforks unilaterally, period.
> IMO this is not a fragility but a strength.
> I think you're talking about miners having their own policies or maybe
> about some other parts of the protocol different from the consensus
> code (which excludes p2p messages).
>
>> A good example is what is happening presently with the Swift network and
>> the associated saber-rattling. Who gets to make these choices? The
>> people who can control the network operators. Such operators are
>> relatively few and exist in regulated meatspace. They do what they must
>> to continue to do business, including cutting others off the network.
>>
>> Bitcoin can end up being no different. The majority of hashing power may
>> easily fall prey to the same attacks. The resilience of Bitcoin against
>> such attacks is in the broad distribution of mining.
>>
>> https://blockchain.info/pools
>
> I agree with this, though I'm not sure what this has to do with
> consensus rules...
>
>> For an alternative consensus to arise in this likely future scenario,
>> the "unknown" miners need to be able to form their own consensus. This
>> requires credible alternative implementations and maintainers with
>> experience to support them and help drive towards agreement.
>>
>> As we see presently, for an implementation to be credible it needs to be
>> in-use. That can take a long time to develop, but ultimately it brings
>> more people to the table to guide such choices, and may even help to
>> forestall significant conflict down the road.
>
> Remember, no majority of miners can force users to accept hardfork
> changes to the consensus rules.
> Again I think you're talking about "choices" beyond consensus rules.
> For example, I think libbitcoin has more p2p messages for prefix-based
> utxo queries or something similar, and it probably has different
> policies than bitcoind's standard policy.
> That doesn't have to change.
>
>>> Please try to come up with an example of something that could go wrong
>>> with different bitcoin implementations sharing libconsensus code,
>>
>> It has already happened that a bug has been pushed out broadly,
>> requiring a temporary shut-down of the entire network. Imagine a
>> scenario where there are many disparate code bases. The impact of a
>> failure caused by any one of them is minimized.
>
> That happened because the consensus reference implementation was
> coupled with bitcoind's bdb store.
> libconsensus needs to be completely storage agnostic so that nothing
> like this can happen again.
>
>>> it is very hard for me to argue about "decentralization" and "strength"
>>> in this abstract and vague way.
>>> Please explain to me in which ways libconsensus implies risks that
>>> libsecp256k1 doesn't, for the shake of your examples.
>>
>> The curve library implies the same technical risk, but is independent of
>> the political risk.
>
> There's no political risk on libbitcoin using libconsensus.
> libconsensus cannot be forced to upgrade so no new checks can be
> pushed onto libbitcoin.
> If, for example, libbitcoin wanted to implement a softfork change that
> the official distribution of libconsensus doesn't implement,
> libbitcoin can implement the additional checks outside of libconsensus
> (or just fork libconsensus).
> Seriously, I don't think there's any "political risk" in libconsensus.
>
>>>>> I would at least suggest to use libconsensus for testing
>>>>> your script interpreter implementation and your signatures checks.
>>>>
>>>> Consider for a moment what this implies. In order to run any such
>>>> testing we need test vectors to run through both implementations. There
>>>> is no other way to perform such testing, and notably there is no need
>>>> for shared code.
>>>
>>> Well, even running random bitstrings in both interpreters would be
>>> useful, as any difference in results would be a symptom of an
>>> extremely dangerous bug.
>>
>> Fuzzing is simply a means for generating test vectors.
>
> Yes, and it is still not enough for having 100% certainty that
> libbitcoin consensus code is functionally identical to other nodes'
> consensus code.
> All I was saying here is than even libbitcoin depending on
> libconsensus only for tests would be something good.
>
>> I'm not arguing for a libconsensus replacement, just pointing out that
>> comparative testing is not an integration scenario.
>
> Sure, and I was just saying that since there's no test vectors that
> can cover all cases, running random test against libconsensus would
> increase the testing surface.
>
>> My "in this case" was a reference to the conformance testing (vs.
>> integration) scenario. In the integration scenario OpenSSL remains a
>> current technical issue, but ultimately it's not the central issue.
>
> I see.
>
>> My first recommendation is to extract libconsensus into an independent
>> git repository, as I have done with libbitcoin-consensus. As a rule, if
>> one wants to publish an API all consumers of that API should be treated
>> equally, including the "native" implementations. Otherwise the
>> implementer fails to fully appreciate situations faced by users.
>
> I agree. Hopefully will get there but a previous step is isolating all
> the consensus code within bitcoind (decoupling it from other
> functionality).
>
>> I think that you will very quickly recognize one of the core technical
>> problems with the current approach when you do this. You will be faced
>> with a choice to (1) dramatically increase the surface area of the API,
>> or (2) reimplement a large portion of the API in the calling code.
>
> I'm not sure I understand this point but as said libconsensus
> interface is still to be defined (and other bitcoin implementations
> different from bitcoind will get involved in defining it).
>
>>> I think starting with VerifyScript it's very good because its
>>> interface was easier than it will be for fully checking transactions
>>> or blocks, for example (but please help fix verifyscript's interface
>>> if it's flawed).
>>> I believe that libbitcoin eventually depending on libconsensus will be
>>> very good for libconsensus because it needs more software depending on
>>> it than the reference implementation to make sense.
>>
>> This is true for libconsensus, but it certainly has value even without
>> external uptake. I think that factoring it out fully and cleanly would
>> make it much more attractive to other implementations.
>
> I agree that isolating the consensus code within bitcoind itself. My
> point is that building libconsensus separately and exposing an API
> makes no sense if only bitcoind will use it.
> Currently libbitcoin only exposes the VerifyScript() function. Do you
> think its interface is bitcoind-specific in any way?
> How would you improve it so that it's more "libbitcoin-friendly".
>
>> I think that the independent repo approach I gave above would be a good
>> starting point for identifying an interface.
>
> I think it's not so urgent. I mean, we still need at the very least a
> function to verify transactions and another to verify blocks. Provably
> each of them divided in several functions to give more flexibility to
> the different implementations to fight DoS attacks. Or for example,
> maybe one implementation wants to do some policy checks for
> transactions before verifying the scripts of its inputs (which, with
> signatures and all, it's computationally more epensive).
>
>> I am very much against the idea of conditionally compiling libbitcoin
>> source for various dependencies. We had several such conditions in the
>> build previously and they have all been removed.
>>
>> The proper way to switch dependencies is to define a common interface
>> and switch at link time. This could be implemented using an internal
>> stub for the interface, as secp256k1 did originally for OpenSSL and GMP
>> WRT big number. But that was only necessary because both are independent
>> and mature libs with heavily utilized public interfaces. In consensus
>> case we have the opportunity to define the interface. The problem may be
>> however the choice between a large interface and quite redundant code.
>
> I meant, for example a compile option designed only for testing that
> devs may use but not usd in the official distributions, like using
> ./configure --disable-wallet --without-gui for building bitcoind, for
> example.
> Something like --with-libconsensus-tests. Maybe this doesn't make
> sense in the context of libbitcoin, I don't know.
> If you mean for using libconsensus or libbitcoin's equivalent code,
> sure, you need an interface and it's still being defined (we only have
> one function).
> Anyway, it is of course up to you guys to decide how to integrate
> libconsensus if you decide to do so.
> I'm just trying to convince you to decide to use it at some point.
>
>>>>> On Sat, Jan 24, 2015 at 12:05 AM, Eric Voskuil <eric@???> wrote:
>>>>>> Hi Jorje,
>>>>>>
>>>>>> There are several potential problems with libbitcoin taking a dependency
>>>>>> on a big chunk of the Satoshi implementation. However the most
>>>>>> straightforward to address is the OpenSSL dependency:
>>>>>>
>>>>>> https://github.com/bitcoin/bitcoin/blob/master/libbitcoinconsensus.pc.in#L11
>>>>>>
>>>>>> After spending months getting OpenSSL out of libbitcoin I couldn't
>>>>>> imagine bringing it back.
>>>>>>
>>>>>> We do use bitcoin/libsep256k1, which will eventually help in consensus
>>>>>> on crypto. And that compact C lib has recently removed all dependencies
>>>>>> (GMP is now optional and OpenSSL is only used for regression testing).
>>>>>>
>>>>>> I think it makes a lot more sense to verify consensus in the same way
>>>>>> libsecp256k1 is verified, through a comprehensive test matrix. We used
>>>>>> to perform testing against OpenSSL directly, now we have a large matrix
>>>>>> of test vectors that we generated from OpenSSL:
>>>>>>
>>>>>> https://github.com/libbitcoin/libbitcoin/blob/master/test/script_number.hpp
>>>>>>
>>>>>> Community-published and verified test vectors are reusable regardless of
>>>>>> language or implementation. As an example, our implementation for script
>>>>>> is here:
>>>>>>
>>>>>> https://github.com/libbitcoin/libbitcoin/blob/master/test/script.hpp#L33
>>>>>>
>>>>>> In fact we tend to use vectors from public sources for just this reason,
>>>>>> for example:
>>>>>>
>>>>>> https://github.com/libbitcoin/libbitcoin/blob/master/test/ec_keys.cpp#L55
>>>>>>
>>>>>> https://github.com/libbitcoin/libbitcoin-explorer/blob/master/test/commands/hd-private.cpp
>>>>>>
>>>>>> The best way to ensure consensus is for everyone to use the same
>>>>>> implementation. However this is ultimately at odds with the higher
>>>>>> objective of decentralization.
>>>>>>
>>>>>> e
>>>>>>
>>>>>> On 01/23/2015 12:50 PM, Jorge Timón wrote:
>>>>>>> If libbitcoin doesn't produce non-standard signatures it cannot create
>>>>>>> a fork as William says.
>>>>>>> The risk is that libbitcoin nodes get fork because they think that an
>>>>>>> invalid signature is valid.
>>>>>>> By using libconsensus (which is a dynamic library that 0.10 produces)
>>>>>>> you can avoid any danger with regard to these changes completely.
>>>>>>> Currently libconsensus only exposes a C function to verify a script in
>>>>>>> a transaction:
>>>>>>>
>>>>>>> https://github.com/bitcoin/bitcoin/blob/master/src/script/bitcoinconsensus.h#L55
>>>>>>>
>>>>>>> C was chosen so that it's easier to call from other languages like python.
>>>>>>> The rest of the consensus checks will have to keep being done by libbitcoin.
>>>>>>> Hopefully by 0.11 libconsensus will be able to check a full
>>>>>>> transaction, although the interface for the relevant utxo subset is
>>>>>>> not clear (I'm working on those code movements with so if you have
>>>>>>> suggestions on what parameters Consensus::CheckTxInputs() should take
>>>>>>> I would love to hear them).
>>>>>>> But not being able to fork due to signature or script checks is
>>>>>>> already a huge deal IMO.
>>>>>>> It is a small dependency (and will be smaller when libsecp256k1 is
>>>>>>> more mature and bitcoin core drops SSL for signature checking) and it
>>>>>>> is specially easy to use by C or C++ alternative implementations like
>>>>>>> libbitcoin.
>>>>>>>
>>>>>>> I can't find the where you check scripts in libbitcoin, only where you
>>>>>>> check signatures:
>>>>>>> https://github.com/libbitcoin/libbitcoin/search?utf8=%E2%9C%93&q=check_signature
>>>>>>> But it seems it's only used on the tests so maybe your VerifyScript()
>>>>>>> is in some other project that consumes libbitcoin?
>>>>>>>
>>>>>>> Anyway, I know it's not the only solution, but I highly recommend
>>>>>>> libconsensus as a dependency for any bitcoin alternative
>>>>>>> implementation that can afford to use it (like it's clearly the case
>>>>>>> for any implementation that already uses C or C++).
>>>>>>>
>>>>>>> I will be glad to help with this if you want to do it but I don't know
>>>>>>> libbitcoin's code.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 23, 2015 at 3:04 AM, William Swanson <swansontec@???> wrote:
>>>>>>>> Libbitcoin will never produce non-standard signatures, so there is no
>>>>>>>> chance of us *creating* a fork.
>>>>>>>>
>>>>>>>> Besides this, there aren't any mining implementations running on top
>>>>>>>> of libbitcoin, so our consensus rules can't possibly lead to bad
>>>>>>>> blocks. The flip side is that we are at risk of being locked out if we
>>>>>>>> somehow reject a block that the rest of the network thinks is valid.
>>>>>>>>
>>>>>>>> I don't know if libsecp256k1 accepts non-standard signatures or not.
>>>>>>>> If libsecp256k1 rejects non-standard signatures, then we are already
>>>>>>>> in danger. If somebody mines a block with one of these signatures,
>>>>>>>> libbitcoin will reject the block and be unable to continue, even
>>>>>>>> though the rest of the network is fine. Hopefully this isn't the case.
>>>>>>>>
>>>>>>>> If libsecp256k1 is willing to accept non-strict signatures, then we
>>>>>>>> are fine in any case. If the network accepts this rule and becomes
>>>>>>>> more strict than us, we certainly won't disagree with any mined
>>>>>>>> blocks. This would need to be tightened up if we were to ever try
>>>>>>>> mining on top of libbitcoin, though.
>>>>>>>>
>>>>>>>> -William
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 22, 2015 at 2:39 PM, Noel Maersk <veox@???> wrote:
>>>>>>>>> As you know, 0.10 is around the corner, and I was idly wondering about
>>>>>>>>> libbitcoin's compatibility with that. Chances are a noticeable portion
>>>>>>>>> of the network will be switching to 0.10.
>>>>>>>>>
>>>>>>>>> gmaxwell has also asked on #darkwallet@freenode:
>>>>>>>>>
>>>>>>>>>> < gmaxwell> genjix: Care to review
>>>>>>>>>> http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg06744.html
>>>>>>>>>> < gmaxwell> ?
>>>>>>>>>
>>>>>>>>> I didn't review it myself yet, hence asking.
>>>>>>>>>
>>>>>>>>> The proposal is "to make non-DER signatures illegal (they've been
>>>>>>>>> non-standard since v0.8.0)".
>>>>>>>>>
>>>>>>>>> We're not using OpenSSL in the new libbitcoin-server. However, what
>>>>>>>>> about Obelisk, older libbitcoin versions, and code that relies on that?
>>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Libbitcoin mailing list
>>>> Libbitcoin@???
>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>
>>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>