On 02/14/2015 01:51 AM, Jorge Timón wrote:
> I agree that this conversation is not being productive anymore. I'm
> doing my best to understand you but I just happen to disagree with
> many of your arguments.
> I doubt it makes you feel better but it's being tedious and
> frustrating for me as well.
> I don't know about other people's intentions, but I know that my only
> intention when recommending libbitcoin to depend on libconsensus is to
> prevent its direct and indirect users from accidentally being forked
> off the network due to a consensus failure.

If you want to achieve that goal, I would again recommend that a
standard suite of test vectors be published that other implementations
can easily consume. Everyone runs the tests and compares results - just
like deterministic build verification.

Not doing this just because of a desire to *also* test by perpetual
fuzzing is I think a mistake.

> If you don't believe that reimplementing consensus checks is an
> unnecessary risk, then I'm lacking the strongest argument for using
> libconsensus.
> I'm telling you that the plan is to expose only c functions and no
> types, classes or utility code, but you're convinced that more things
> will be exposed

Plans do change.

> and it will end up being functionally equivalent to libbitcoin.
> You're convinced that you know the future of libconsensus
> better than I do, even before its first release. I guess I can't do
> anything to convince you on this either.
> You also insist that libconsensus is a huge dependency,

No, I said, "it's a massive repo," where the code is, "not properly
factored for use as a library."

> here it is:
> crypto/hmac_sha512.cpp \
> crypto/ripemd160.cpp \
> crypto/sha1.cpp \
> crypto/sha256.cpp \
> crypto/sha512.cpp \
> eccryptoverify.cpp \
> ecwrapper.cpp \
> hash.cpp \
> primitives/transaction.cpp \
> pubkey.cpp \
> script/bitcoinconsensus.cpp \
> script/interpreter.cpp \
> script/script.cpp \
> uint256.cpp \
> utilstrencodings.cpp
>
> libbitcoinconsensus.la is currently 939 bytes.

libbitcoin-consensus.so (dynamic) is 1.9 MB and libbitcoin-consensus.a
(static) is 4.5 MB, you are referring to the Libtool alias file.

> But you consider that
> if it lives in bitcoin core's repository it's in great danger to
> become as fat as bitcoin core itself. And somehow unilaterally pushed
> to its user like if it were proprietary software.
> Again, I don't think I have any more arguments to convince you that
> this will not be the case. By the way, the windows analogy it's
> extremely bad because windows is not free software.
>
> You even resist to test your own interpreter against the one in
> libconsensus.

No, I do not. I object to *integration*.

> I know the test surface is huge, even bigger than
> libsecp256k1's.
> I haven't heard anyone saying that random tests a comparing it with
> openSSL was a bad idea for libsecp256k1 (see
> https://www.reddit.com/r/Bitcoin/comments/2rrxq7/on_why_010s_release_notes_say_we_have_reason_to),
> but somehow random tests comparing libbitcoin to libconsensus don't
> make sense, because apparently one drop in the ocean (multiplied by an
> unspecified number of times, from an unspecified number of computers)
> is worse than nothing.
>
> My presumtion is NOT "that "the network" must always be defined by
> whatever is implemented in bitcoind". That's precisely one reason to
> separate libconsensus from the rest of bitcoin core (being in the same
> repository doesn't make them less separated).

Of course it does.

> As I repeated several times the bitcoin protocol is much bigger than
> the consensus checks.

And as I've repeated in response, that isn't the central issue.

> So although my hope was that libbitcoin used libconsensus and it seems
> that won't happen, I still wish you the best luck to this project.
> I specially hope that nobody loses any money or gets forked off the
> network while using libbitcoin, and that we can one day look at the
> past and say that I was just overestimating the consensus failure
> risks of reimplementing consensus checks.
> In other words, I hope I'm wrong and, again, good luck.

Thanks, I do wish you well also, there are no hard feelings here.

I do understand all of your arguments, I just don't think integration
makes sense for libbitcoin, or for bitcoin in general.

> On Sat, Feb 14, 2015 at 9:30 AM, Eric Voskuil <eric@???> wrote:
>> On 02/13/2015 03:29 PM, Jorge Timón wrote:
>>> On Fri, Feb 13, 2015 at 11:09 PM, Eric Voskuil <eric@???> wrote:
>>>> On 02/13/2015 01:07 PM, Jorge Timón wrote:
>>>>> On Tue, Feb 3, 2015 at 4:32 AM, Eric Voskuil <eric@???> wrote:
>>>>>>>> For an alternative consensus to arise in this likely future scenario,
>>>>>>>> the "unknown" miners need to be able to form their own consensus. This
>>>>>>>> requires credible alternative implementations and maintainers with
>>>>>>>> experience to support them and help drive towards agreement.
>>>>>>>>
>>>>>>>> As we see presently, for an implementation to be credible it needs to be
>>>>>>>> in-use. That can take a long time to develop, but ultimately it brings
>>>>>>>> more people to the table to guide such choices, and may even help to
>>>>>>>> forestall significant conflict down the road.
>>>>>>>
>>>>>>> Remember, no majority of miners can force users to accept hardfork
>>>>>>> changes to the consensus rules.
>>>>>>
>>>>>> That statement doesn't make sense. If no majority can do it, then it
>>>>>> can't be done. You seem to believe that a hardfork requires universal
>>>>>> acceptance. This is not the case.
>>>>>
>>>>> No majority of miners can force a hardfork. Majority of USERS.
>>>>> Universal acceptance by users instead of just a majority?
>>>>> I don't know, honestly. I think it should be something
>>>>> uncontroversial that can be universally accepted (like getting rid of
>>>>> obscure bdb-specific consensus rules, that was the last hardfork).
>>>>> My point is that users decide the rules they're checking and miners
>>>>> can't change that.
>>>>> If 95% of the miners change the subsidy schedule to produce more than
>>>>> 21 M but users reject this change, they will just listen to the
>>>>> blocks produced by the honest 5%.
>>>>
>>>> This argument makes my point.
>>>
>>> Well, I'm not sure what your point here was anymore, but my previous
>>> statement did make sense.
>>
>> My point, in this instance, was that a change to consensus does not
>> require universal acceptance.
>>
>>>>> Why this cannot happen with libsecp256k1 ?
>>>>> I don't think you're answering to the question. In which way it is
>>>>> more risky to depend on libconsensus than to depend on libsecp256k1?
>>>>
>>>> I did not say it could not happen with the crypto lib. I said it would
>>>> not matter.
>>>
>>> I agree, you would fork it, just like you would do with libconsensus.
>>
>> And then we maintain the Satoshi code base?
>>
>> The curve library is a small and independent C library, with no other
>> dependencies, based on a standard not defined by Bitcoin, producing no
>> redundancies with the libbitcoin code base.
>>
>>>> See previous comments about user base.
>>>
>>> I don't think you have answered to this question anywhere.
>>> If you did, I'm sorry, but I missed it.
>>> Can you repeat it again? Please be as specific as possible (general
>>> concerns about a specific foundation I don't belong to won't help me
>>> understand).
>>
>> I have made this point a couple of times. A check on power requires
>> alternatives that are maintained and widely deployed *before* an issue
>> arises. See: http://www.libbitcoin.org
>>
>> By way of analogy, there were always alternatives to Windows. Yet
>> without those alternatives widely used and supported their existence
>> mattered little.
>>
>>>>> Retesting without changing the code makes sense if some of your tests
>>>>> are random.
>>>>
>>>> I didn't say it didn't make sense, I said the coverage would be comparable.
>>>
>>> No, the coverage is bigger adding random tests, this can be very
>>> useful as demonstrated by libsecp256k1 random testing.
>>>
>>>>>> Furthermore, given that we're talking about randomly changing random
>>>>>> tests over a practically infinite space, is practically zero percent of
>>>>>> the surface area.
>>>>>
>>>>> What? The more random tests you run the more surface of that
>>>>> practically infinite space you cover.
>>>>> This seems obvious to me.
>>>>
>>>> Each time you change the code you start over. Coverage on old code says
>>>> nothing about coverage on new code.
>>>
>>> I didn't make the assumption that the code is changed, you did.
>>> You could also not change the tests and leave the random tests running
>>> forever (or until one test fails).
>>>
>>>>> It turned out that openSSL had a very specific bug that had been
>>>>> there for years.
>>>>
>>>> Because of the practically infinite surface area it's possible that a
>>>> near infinity of bugs remain.
>>>
>>> Yes, it is also possible that the random tests never show you anything
>>> interesting.
>>> In this case, it is possible that random scripts are never executed
>>> differently in libconsensus and libbitcoin. But if they do once, you
>>> have the certainty that libbitcoin has a bug (given that libconsensus
>>> is the specification of the consensus rules).
>>
>> Do you realize how large the space is that we are talking about?
>>
>>>>> Ok, I finally see what you're saying. Separating libconsensus would
>>>>> require code duplication since, for example, the script type will be
>>>>> required to produce scripts to sign them.
>>>>> That may actually a good reason to never separate libconsensus from
>>>>> bitcoin core, or do so only as a sub-repository or something.
>>>>
>>>> Any argument that it should not be isolated from core is an argument for
>>>> not incorporating it into another lib. Why would one expect other libs
>>>> to accept deficiencies that core would not?
>>>
>>> You're saying that Bitcoin core reusing libconsensus code directly
>>> instead of via its c API is an argument against using libconsensus in
>>> other bitcoin implementations.
>>> I disagree and I don't understand why you reach that conlusion.
>>
>> If you don't understand you probably shouldn't disagree.
>>
>> The argument you are making against ever factoring it out of core is
>> that it would cause significant redundancy across the two repos. One
>> argument against incorporating it into libbitcoin is that it would
>> produce *the same* redundancy problem. You seem to think others should
>> be expected to take on that burden, but rule it out for core.
>>
>>> libconsensus only supports a small part of everything you need to
>>> implement the bitcoin protocol, but that's not a deficiency, that's a
>>> conscious design decision.
>>> Reimplementing the p2p messages, for example, doesn't have consensus
>>> failure fork risks.
>>>
>>>>> So I think your complain is that libbitcoin will have some code
>>>>> duplication from libconsensus.
>>>>
>>>> Not just some, a very significant portion.
>>>
>>> Whatever, you're still able to avoid reimplementing the parts that are
>>> more risky to reimplement (because you an get forked out of the
>>> network if you do so).
>>
>> Your presumption is that "the network" must always be defined by
>> whatever is implemented in bitcoind.
>>
>> The attempt to get other "independent" implementations to accept a
>> dependency on libconsensus, while not even contemplating the isolation
>> of libconsensus from core *ever happening* strongly implies that
>> maintaining this central role is an underlying objective.
>>
>>> You already have that code implemented so I don't see what the problem is.
>>> You seem to be complaining about the fact that lbconsensus won't allow
>>> you to delete more code from libbitcoin.
>>>
>>>>> libconsensus will only have a c interface with simple functions.
>>>>> So, no, a common type lib is not what libconsensus is about.
>>>>> I don't think there's any way around this.
>>>>
>>>> There doesn't really need to be a way around it. Alternate
>>>> implementations are a good thing.
>>>
>>> Alternative implementations of the consensus checks are an unnecessary risk.
>>> Innovation in alternative implementations happens in higher layers.
>>
>> See previous and above discussions on why this not a valid assumption.
>>
>>>> I would really prefer to agree, as I think we are all working toward a
>>>> better future. But as I see it there would eventually be little reason
>>>> to maintain libbitcoin if we took this dependency.
>>>
>>> I strongly disagree. libconsensus [libbitcoin] implements the p2p protocol,
>>> storage, concurrency, creation and signing of transactions, innovative
>>> features like coinjoin, etc.
>>
>> There's no coinjoin or storage in libbitcoin, and we have contemplated
>> factoring out network.
>>
>> You might want to look at libbitcoin organization and roadmap.
>>
>> https://wiki.unsystem.net/en/index.php/Libbitcoin
>> https://wiki.unsystem.net/en/index.php/Toolchain/Roadmap
>>
>>> libconsensus will never implement any of that: it's meant to be a
>>> stateless check-only tiny lib
>>
>> You have to stop thinking of this as a "tiny lib". It's a *massive*
>> repo. All of the code in the repo is potentially connected now, or at
>> some point in the future, to the build. It's not properly factored to be
>> used as a library.
>>
>> I quote from the first link above, regarding libbitcoin isolation of libs:
>>
>> "This factoring is designed in part to isolate dependencies so that they
>> don't get unnecessarily dragged into deployments. This is really the big
>> issue. Without strong isolation we will have to be very watchful for
>> dependency creep, even between the libbitcoin libs themselves."
>>
>>> with no features beyond the consensus checks.
>>
>> Given the significant redundancy issues this is practically guaranteed
>> to change. It would most certainly evolve into the form similar to that
>> of libbitcoin-consensus today. The exposure of the lower level calls
>> would allow the lib to be properly factored for reuse. At that point a
>> rational person would have to ask why there are two of all of these
>> functions and types. The only logical conclusion would be to drop one of
>> the libraries. That means they either become one and the same, or they
>> become independent. The idea behind libbitcoin is that they *not* become
>> one and the same.
>>
>>>>>> Developing for end-users is different than developing for developers.
>>>>>
>>>>> Agreed. I'm not sure if you're implying that current libconsensus
>>>>> design is for end-users...It is definitely intended to be used by developers.
>>>>
>>>> The Satoshi code was built for users, not developers. The consensus
>>>> interface, as a developer API, suffers from that underlying approach.
>>>
>>> Bitcoin core is for users, not developers. libconsensus is for
>>> developers not users.
>>> It is a dynamic library (.dll .so) with a C API, it doesn't even have
>>> command line tools and of course it doesn't have a GUI.
>>>
>>> libconsensus != Bitcoin core (satoshi code)
>>> libconsensus != libbitcoin
>>
>> It's getting a bit tedious to carry on this discussion when it doesn't
>> appear at times that you are reading what I've written. I did not say it
>> was written *for users*, I said that it "suffers from that underlying
>> approach". In other words, it's a hack against an end-user application.
>>
>> e
>>