Author: Eric Voskuil Date: To: libbitcoin Subject: [Libbitcoin] libbitcoin version3 release
The libbitcoin version3 release is pending.
The version3 release branches and installers have been created and
verified. There are no open 3.0 issues and at this point we are only
awaiting the deployment of community libbitcoin v3 servers and update of
documentation before tagging the release. Neill is currently syncing the
servers and I'll be working on docs for the next couple of days. The
master branch has been versioned as 4.0.0.
Anyone running v2 servers is strongly advised to upgrade as soon as
possible. The client-server (zeromq) API is mostly compatible with v2.
I'll be documenting the API and differences with v2, but if you
encounter any problems please post a GitHub issue, post to this list, or
contact us on Freenode.
I won't use the space here to duplicate the release notes that I have
yet to write. But I can provide just a bit of perspective.
The libbitcoin utilities, SX command line tools and Obelisk server were
the first release of libbitcoin as developed by Amir and the community,
starting in early 2011. SX was released in late 2011 and Obelisk was
released in January 2014. This was the version1 library.
The upgrade from SX to BX (bitcoin explorer) was released in December
2014. During that same period the store was replaced with a
memory-mapped file hash table, which resulted in the release of BS
(bitcoin server) in early 2015, which was generalized to include BN
(bitcoin node) in December 2015. This was the version2 library.
This week we are releasing the full version3 library:
along with three revised command line tools: BX, BN, BS. The command
line tools are thin wrappers around the development libraries, adding
command line and config settings readers.
There are significant improvements in many areas: test coverage,
automation, reliability, interface consistency, standards conformance,
memory usage, validation speed, code readability, modularity,
diagnostics, scalability, and manageability.
There are however still a few areas that I consider deficient with
respect to production node operation:
(1) The node does not have sufficient DOS protection. As with other
features, we have taken a different approach to this than other
implementations. There is no plan to score behavior or ban endpoints.
Banning by IP address is trivially circumvented and therefore not
interesting. A peer that conforms to the protocol is a "good" peer. A
peer that violates the protocol is immediately dropped. Beyond this it
is only a question of how much of the node's resources the owners wants
to allow a peer to "consume" in a given amount of time. In this sense it
is very similar to our approach to policy. There is only one tx pool
policy, the price per byte. Similarly there is one network policy,
maximum average outgoing bytes per minute. A peer that exceeds the
configured limit is dropped. However the node does not yet reject a
class of invalid messages (those which are otherwise valid but are not
anticipated by the session's configured protocols). And there is not yet
a consumption measure.
(2) Parallel initial block download is currently disabled. This is
intended to allow a previously validated chain to be checkpointed so
that it may be re-downloaded from the P2P network quickly in the future.
We have been able to complete a full mainnet sync in 16 minutes. However
this feature was disabled as the result of some unrelated changes and
remains so in the interest of shipping this week. As a result the
operational initial block download is a simple blocks-first
implementation, intended only for operation in a fully-synced node. This
requires a manual configuration transition in order to avoid costly sync
delays or sub-par fully-synced performance. It is our intent to
reconnect PIBD, which will restore high speed sync. But we also need to
optimize operational sync so that it efficiently selects peers and
avoids redundant block download - transitioning smoothly to/from
fully-synced configuration. So basically it's not consumer friendly
because it requires manual config to be optimal.
(3) The node does not fully implement a couple of aspects of the P2P
protocol. Specifically an incoming "mempool" message is dropped due to
lack of a tx pool index. The tx pool index is under development for
support of this message and mining block template production. Also
"compact blocks" messages are not supported. The supported (and
configurable) network protocol range is 31402 through 70013. Compact
blocks is the most recent operational protocol level, at 70014. This
will be fully supported in a future release. Most of the work for it has
already been done. All other aspects of this protocol range are
supported, though we do not intend to support the NODE_BLOOM flag (which
is not required for protocol conformance), which excludes that related
set of protocol messages. These could be implemented using our
extensible sessions and protocols, using the serializable filtered block
message types, which do exist, and existing blockchain queries. We just
do not support the idea of what is a client-server protocol in the P2P
protocol and have chosen to not spend time on it.
Block validation is extremely fast and scales very well. On a high end
machine we see full block validation times of 250-300ms (from
deserialization to notification). This is with no UTXO cache, no
signature cache, no tx memory pool, and without compact blocks. We have
determined that we can reduce this to ~100ms with compact blocks.
Optional opaque (in-store) caching has been added to optimize
performance on lower RAM machines, as the memory mapped store can
otherwise produce material paging costs.
Also, the node can support reorganizations of any depth and large reorgs
do not materially impact validation performance. Neither restarting the
node nor reorganization nor soft fork transition loses unconfirmed
transactions. Disk space for mainnet is currently 113GB in three files.
The server can index all addresses in the block chain, stealth and from
standard outputs, as well as spends. This can double the disk space
requirement, however queries remain constant time and are blazing fast.
This has been a monumental effort with the support of a community that
may not be recognized given that the contributions do not always result
in a pull request. This includes those who have maintained this list,
answered questions on issues, IRC and here, developed the libbitcoin.org
site, operated community servers, provided financial support to
developers and testers, tested and documented altcoin integration,
provided detailed feedback on server management in deployed operations,
provided big-endian platform validation, provided extensive Linux and
OSX operational node and server testing, developed c and python
interfaces, added significant features to bx with library support in
libbitcoin, developed AppVeyor integration, spent significant amounts of
time on futures work developing privacy interfaces, coding patches and
unit tests, and developing Debian packaging.
::
[Libbitcoin] libbitcoin version3 re…
