I agree that this conversation is not being productive anymore. I'm
doing my best to understand you but I just happen to disagree with
many of your arguments.
I doubt it makes you feel better but it's being tedious and
frustrating for me as well.
I don't know about other people's intentions, but I know that my only
intention when recommending libbitcoin to depend on libconsensus is to
prevent its direct and indirect users from accidentally being forked
off the network due to a consensus failure.
If you don't believe that reimplementing consensus checks is an
unnecessary risk, then I'm lacking the strongest argument for using
libconsensus.
I'm telling you that the plan is to expose only c functions and no
types, classes or utility code, but you're convinced that more things
will be exposed and it will end up being functionally equivalent to
libbitcoin. You're convinced that you know the future of libconsensus
better than I do, even before its first release. I guess I can't do
anything to convince you on this either.
You also insist that libconsensus is a huge dependency, here it is:
crypto/hmac_sha512.cpp \
crypto/ripemd160.cpp \
crypto/sha1.cpp \
crypto/sha256.cpp \
crypto/sha512.cpp \
eccryptoverify.cpp \
ecwrapper.cpp \
hash.cpp \
primitives/transaction.cpp \
pubkey.cpp \
script/bitcoinconsensus.cpp \
script/interpreter.cpp \
script/script.cpp \
uint256.cpp \
utilstrencodings.cpp

libbitcoinconsensus.la is currently 939 bytes. But you consider that
if it lives in bitcoin core's repository it's in great danger to
become as fat as bitcoin core itself. And somehow unilaterally pushed
to its user like if it were proprietary software.
Again, I don't think I have any more arguments to convince you that
this will not be the case. By the way, the windows analogy it's
extremely bad because windows is not free software.

You even resist to test your own interpreter against the one in
libconsensus. I know the test surface is huge, even bigger than
libsecp256k1's.
I haven't heard anyone saying that random tests a comparing it with
openSSL was a bad idea for libsecp256k1 (see
https://www.reddit.com/r/Bitcoin/comments/2rrxq7/on_why_010s_release_notes_say_we_have_reason_to),
but somehow random tests comparing libbitcoin to libconsensus don't
make sense, because apparently one drop in the ocean (multiplied by an
unspecified number of times, from an unspecified number of computers)
is worse than nothing.

My presumtion is NOT "that "the network" must always be defined by
whatever is implemented in bitcoind". That's precisely one reason to
separate libconsensus from the rest of bitcoin core (being in the same
repository doesn't make them less separated).
As I repeated several times the bitcoin protocol is much bigger than
the consensus checks.

So although my hope was that libbitcoin used libconsensus and it seems
that won't happen, I still wish you the best luck to this project.
I specially hope that nobody loses any money or gets forked off the
network while using libbitcoin, and that we can one day look at the
past and say that I was just overestimating the consensus failure
risks of reimplementing consensus checks.
In other words, I hope I'm wrong and, again, good luck.


On Sat, Feb 14, 2015 at 9:30 AM, Eric Voskuil <eric@???> wrote:
> On 02/13/2015 03:29 PM, Jorge Timón wrote:
>> On Fri, Feb 13, 2015 at 11:09 PM, Eric Voskuil <eric@???> wrote:
>>> On 02/13/2015 01:07 PM, Jorge Timón wrote:
>>>> On Tue, Feb 3, 2015 at 4:32 AM, Eric Voskuil <eric@???> wrote:
>>>>>>> For an alternative consensus to arise in this likely future scenario,
>>>>>>> the "unknown" miners need to be able to form their own consensus. This
>>>>>>> requires credible alternative implementations and maintainers with
>>>>>>> experience to support them and help drive towards agreement.
>>>>>>>
>>>>>>> As we see presently, for an implementation to be credible it needs to be
>>>>>>> in-use. That can take a long time to develop, but ultimately it brings
>>>>>>> more people to the table to guide such choices, and may even help to
>>>>>>> forestall significant conflict down the road.
>>>>>>
>>>>>> Remember, no majority of miners can force users to accept hardfork
>>>>>> changes to the consensus rules.
>>>>>
>>>>> That statement doesn't make sense. If no majority can do it, then it
>>>>> can't be done. You seem to believe that a hardfork requires universal
>>>>> acceptance. This is not the case.
>>>>
>>>> No majority of miners can force a hardfork. Majority of USERS.
>>>> Universal acceptance by users instead of just a majority?
>>>> I don't know, honestly. I think it should be something
>>>> uncontroversial that can be universally accepted (like getting rid of
>>>> obscure bdb-specific consensus rules, that was the last hardfork).
>>>> My point is that users decide the rules they're checking and miners
>>>> can't change that.
>>>> If 95% of the miners change the subsidy schedule to produce more than
>>>> 21 M but users reject this change, they will just listen to the
>>>> blocks produced by the honest 5%.
>>>
>>> This argument makes my point.
>>
>> Well, I'm not sure what your point here was anymore, but my previous
>> statement did make sense.
>
> My point, in this instance, was that a change to consensus does not
> require universal acceptance.
>
>>>> Why this cannot happen with libsecp256k1 ?
>>>> I don't think you're answering to the question. In which way it is
>>>> more risky to depend on libconsensus than to depend on libsecp256k1?
>>>
>>> I did not say it could not happen with the crypto lib. I said it would
>>> not matter.
>>
>> I agree, you would fork it, just like you would do with libconsensus.
>
> And then we maintain the Satoshi code base?
>
> The curve library is a small and independent C library, with no other
> dependencies, based on a standard not defined by Bitcoin, producing no
> redundancies with the libbitcoin code base.
>
>>> See previous comments about user base.
>>
>> I don't think you have answered to this question anywhere.
>> If you did, I'm sorry, but I missed it.
>> Can you repeat it again? Please be as specific as possible (general
>> concerns about a specific foundation I don't belong to won't help me
>> understand).
>
> I have made this point a couple of times. A check on power requires
> alternatives that are maintained and widely deployed *before* an issue
> arises. See: http://www.libbitcoin.org
>
> By way of analogy, there were always alternatives to Windows. Yet
> without those alternatives widely used and supported their existence
> mattered little.
>
>>>> Retesting without changing the code makes sense if some of your tests
>>>> are random.
>>>
>>> I didn't say it didn't make sense, I said the coverage would be comparable.
>>
>> No, the coverage is bigger adding random tests, this can be very
>> useful as demonstrated by libsecp256k1 random testing.
>>
>>>>> Furthermore, given that we're talking about randomly changing random
>>>>> tests over a practically infinite space, is practically zero percent of
>>>>> the surface area.
>>>>
>>>> What? The more random tests you run the more surface of that
>>>> practically infinite space you cover.
>>>> This seems obvious to me.
>>>
>>> Each time you change the code you start over. Coverage on old code says
>>> nothing about coverage on new code.
>>
>> I didn't make the assumption that the code is changed, you did.
>> You could also not change the tests and leave the random tests running
>> forever (or until one test fails).
>>
>>>> It turned out that openSSL had a very specific bug that had been
>>>> there for years.
>>>
>>> Because of the practically infinite surface area it's possible that a
>>> near infinity of bugs remain.
>>
>> Yes, it is also possible that the random tests never show you anything
>> interesting.
>> In this case, it is possible that random scripts are never executed
>> differently in libconsensus and libbitcoin. But if they do once, you
>> have the certainty that libbitcoin has a bug (given that libconsensus
>> is the specification of the consensus rules).
>
> Do you realize how large the space is that we are talking about?
>
>>>> Ok, I finally see what you're saying. Separating libconsensus would
>>>> require code duplication since, for example, the script type will be
>>>> required to produce scripts to sign them.
>>>> That may actually a good reason to never separate libconsensus from
>>>> bitcoin core, or do so only as a sub-repository or something.
>>>
>>> Any argument that it should not be isolated from core is an argument for
>>> not incorporating it into another lib. Why would one expect other libs
>>> to accept deficiencies that core would not?
>>
>> You're saying that Bitcoin core reusing libconsensus code directly
>> instead of via its c API is an argument against using libconsensus in
>> other bitcoin implementations.
>> I disagree and I don't understand why you reach that conlusion.
>
> If you don't understand you probably shouldn't disagree.
>
> The argument you are making against ever factoring it out of core is
> that it would cause significant redundancy across the two repos. One
> argument against incorporating it into libbitcoin is that it would
> produce *the same* redundancy problem. You seem to think others should
> be expected to take on that burden, but rule it out for core.
>
>> libconsensus only supports a small part of everything you need to
>> implement the bitcoin protocol, but that's not a deficiency, that's a
>> conscious design decision.
>> Reimplementing the p2p messages, for example, doesn't have consensus
>> failure fork risks.
>>
>>>> So I think your complain is that libbitcoin will have some code
>>>> duplication from libconsensus.
>>>
>>> Not just some, a very significant portion.
>>
>> Whatever, you're still able to avoid reimplementing the parts that are
>> more risky to reimplement (because you an get forked out of the
>> network if you do so).
>
> Your presumption is that "the network" must always be defined by
> whatever is implemented in bitcoind.
>
> The attempt to get other "independent" implementations to accept a
> dependency on libconsensus, while not even contemplating the isolation
> of libconsensus from core *ever happening* strongly implies that
> maintaining this central role is an underlying objective.
>
>> You already have that code implemented so I don't see what the problem is.
>> You seem to be complaining about the fact that lbconsensus won't allow
>> you to delete more code from libbitcoin.
>>
>>>> libconsensus will only have a c interface with simple functions.
>>>> So, no, a common type lib is not what libconsensus is about.
>>>> I don't think there's any way around this.
>>>
>>> There doesn't really need to be a way around it. Alternate
>>> implementations are a good thing.
>>
>> Alternative implementations of the consensus checks are an unnecessary risk.
>> Innovation in alternative implementations happens in higher layers.
>
> See previous and above discussions on why this not a valid assumption.
>
>>> I would really prefer to agree, as I think we are all working toward a
>>> better future. But as I see it there would eventually be little reason
>>> to maintain libbitcoin if we took this dependency.
>>
>> I strongly disagree. libconsensus [libbitcoin] implements the p2p protocol,
>> storage, concurrency, creation and signing of transactions, innovative
>> features like coinjoin, etc.
>
> There's no coinjoin or storage in libbitcoin, and we have contemplated
> factoring out network.
>
> You might want to look at libbitcoin organization and roadmap.
>
> https://wiki.unsystem.net/en/index.php/Libbitcoin
> https://wiki.unsystem.net/en/index.php/Toolchain/Roadmap
>
>> libconsensus will never implement any of that: it's meant to be a
>> stateless check-only tiny lib
>
> You have to stop thinking of this as a "tiny lib". It's a *massive*
> repo. All of the code in the repo is potentially connected now, or at
> some point in the future, to the build. It's not properly factored to be
> used as a library.
>
> I quote from the first link above, regarding libbitcoin isolation of libs:
>
> "This factoring is designed in part to isolate dependencies so that they
> don't get unnecessarily dragged into deployments. This is really the big
> issue. Without strong isolation we will have to be very watchful for
> dependency creep, even between the libbitcoin libs themselves."
>
>> with no features beyond the consensus checks.
>
> Given the significant redundancy issues this is practically guaranteed
> to change. It would most certainly evolve into the form similar to that
> of libbitcoin-consensus today. The exposure of the lower level calls
> would allow the lib to be properly factored for reuse. At that point a
> rational person would have to ask why there are two of all of these
> functions and types. The only logical conclusion would be to drop one of
> the libraries. That means they either become one and the same, or they
> become independent. The idea behind libbitcoin is that they *not* become
> one and the same.
>
>>>>> Developing for end-users is different than developing for developers.
>>>>
>>>> Agreed. I'm not sure if you're implying that current libconsensus
>>>> design is for end-users...It is definitely intended to be used by developers.
>>>
>>> The Satoshi code was built for users, not developers. The consensus
>>> interface, as a developer API, suffers from that underlying approach.
>>
>> Bitcoin core is for users, not developers. libconsensus is for
>> developers not users.
>> It is a dynamic library (.dll .so) with a C API, it doesn't even have
>> command line tools and of course it doesn't have a GUI.
>>
>> libconsensus != Bitcoin core (satoshi code)
>> libconsensus != libbitcoin
>
> It's getting a bit tedious to carry on this discussion when it doesn't
> appear at times that you are reading what I've written. I did not say it
> was written *for users*, I said that it "suffers from that underlying
> approach". In other words, it's a hack against an end-user application.
>
> e
>