:: Re: [Libbitcoin] Single repo for al…
Top Page
Delete this message
Reply to this message
Author: Eric Voskuil
Date:  
To: Amir Taaki, libbitcoin
Subject: Re: [Libbitcoin] Single repo for all?
It's not difficult from my point of view, in maintaining the Windows
builds, but there are certainly drawbacks as well as advantages.

Given the path that we've laid out, it's not just a question of merging
libwallet into libbitcoin. This is the current plan:

libbitcoin (utils)
libbitcoin_node (full node)
libbitcoin_server (full node with ZMQ API)
libbitcoin_client (client to ZMQ API)
libbitcoin_wallet (offline key management)

Amir has also expressed a desire to factor out libbitcoin_blockchain
from libbitcoin_node and has started that work. I've never been
convinced that it is better to maintain this as an independent repo,
since the blockchain functionality is necessary to the node and AFICT is
not useful independent of a node. I could see merging this with
libbitcoin_node.

When it comes down to it a repo is a completely arbitrary boundary. What
matters to the organization above is that the 5 libraries above can be
deployed independently. Considering interlib dependencies there are five
deployment scenarios:

libbitcoin
libbictoin_client->libbitcoin
libbictoin_wallet->libbitcoin
libbitcoin_node->libbitcoin
libbitcoin_server->libbitcoin_node->libbitcoin

This factoring is designed in part to isolate dependencies so that they
don't get unnecessarily dragged into deployments. This is really the big
issue. My main concern would be that without strong isolation we will
have to be very watchful for dependency creep, even between the
libbitcoin libs themselves.

Another benefit to the library/repo alignment we have planned is that it
makes the toolkit very approachable to the newcommer. When one looks at
bitcoin (core) there is one repo, it looks like a monolith (and it is).
Even so new work has recently been taken on outside of that repo
(secp256k1). Imagine trying to do that within the bitcoin repo, and
imagine forking bitcoin to work with secp256k1.

The path we are on presently was Amir's suggestion modeled on boost's
organization. Each library exists in its own repo with its own master
header, tests, examples, docs and make files. Similarly, defines are
prefixed with the repo/lib name in order to prevent conflicts in use of
the headers.

This issue that William and I were discussing was the need for
independent defines:

>> There is a reason for the export macros to be uniquely-named per
>> project. Each project needs to define a symbol to differentiate
>> between dll export and static. Each dependent project needs to
>> define a symbol to differentiate between dll import and static. So
>> there are three contexts since import and export static are the
>> same. If two projects utilize the same symbols for declaring exports
>> it becomes hard for dependent projects to treat them independently.
>> While it may be common for a project to link all dependent projects
>> in the same manner it's certainly not a requirement and therefore
>> would be bad practice to couple these symbols across projects.


>Ah, that makes perfect sense. Since these are, in fact, separate
>libraries, using the same symbol across library boundaries could lead
>to problems if the libraries are compiled with different flags. The
>only way to make the defines match the compilation mode on a
>project-by-project basis is to use a separate define per project.


>Under that constraint, I suppose "BCW" really is the shortest
>reasonable abbreviation of "libbitcoin-wallet".


> Either that, or we unite the different projects...


Merging the repos wouldn't really change this situation. If we
consequently merged the defines we would then lose the ability to treat
the libraries independently. This is actually an example of the subtle
entanglements that we would have to be very watchful for.

Storing the source in the same repo but keeping it seperated as
independent libraries will likely make it harder to maintain this
independence over time. The main benefit of consolidation is that we can
share build, versioning, and repo sync. This will come at the cost of a
higher level of conditionality in build and versioning, more complex
merges, and loss of per lib github permissions and issues.

I think it's a very good question to consider, but my inclination is not
to take on the disruption unless there is a compelling reason, and IMO
the drawbacks are at least as strong or stronger than the potential
benefits.

e

On 05/24/2014 06:18 PM, Amir Taaki wrote:
>> Either that, or we unite the different projects in a single source
>> repo with a single build system, and just have separate --enable-foo
>> options to turn each individual library on and off. In Visual Studio
>> land, that would be done as a single solution file with a project file
>> per lib. A single "make" invocation would produce
>> libbitcoin.{so,a,pc}, libbitcoin-wallet.{so,a,pc},
>> libbitcoin-server.{so,a,pc}, and so forth.
>>
>> Actually, this would be kinda nice. It's already hard enough to keep
>> the various repos in sync, and we haven't even divided them down the
>> granularity we are hoping for.
>>
>> -William
>
> William does have a point.
>
> * Simpler setup for users.
> * (for now) a single package to distribute in Linux repos.
> * Easier to track updates.
> * Share same define and compile symbols easily.
> * Ability to provide a single unified <bitcoin.hpp>
>
> It could be possible with proper organisation to use a single repo and
> keep everything properly split and modularised.
>
> We also reduce the overhead for developers and users alike.
>
> Eric what do you reckon if we merge libwallet back into libbitcoin?
>
> We can also use conditional compilation (through macros like NO_WALLET or
> NO_BLOCKCHAIN) to prevent including certain headers, which could be
> enabled by certain ./configure switches (--no-wallet or --no-blockchain).
>
>
>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>