Author: Eric Voskuil Date: To: libbitcoin Subject: [Libbitcoin] networking update/roadmap
I'm very happy with progress in the P2P networking stack.
libbitcoin-node has been a significant area of weakness in libbitcoin
for some time. This unfortunately prevents many people from experiencing
the server.
I've revised all of the libbitcoin::network namespace in the 'version3'
branch. This includes a number of refinements and fixes. Among these I
recently resolved a long standing issue in re-subscription which
resulted in dropped messages. This led to gaps in the chain during sync,
resulting in excessive reorganizing. This led to increasing backlog,
which exacerbated the subscription problem. This was probably the
primary cause of the excessive backlogging and memory exhaustion during
sync.
For the past few weeks I've been working on libbitcoin-node networking,
which extends libbitcoin::network with block and transaction protocols.
In doing so I've added several new capabilities, including header first
sync and rapid connection establishment (using connection pooling). This
work is in the 'sync' feature branches.
I have some more to do in headers first sync, but I'm very excited about
it so far. We can presently download nearly 400,000 headers in 15
seconds and the first 50k blocks in 6 minutes. I'm in the processes of
parallelizing block download. While I don't expect to be able to process
all 400k blocks (most of which are quite a bit larger) in 6 minutes
total against 8 peers, I am hopeful I can get it down to 1-2 hours.
I will also be implementing channel throttling for active protection of
external and self-inflicted DoS due to message backlogging.
WRT longer term optimizations, I'm supportive of both IBLT and segmented
witness.