On 05/29/2015 01:47 PM, mlmikael wrote:
> On 2015-05-25 06:52, Eric Voskuil wrote:
>> So while Neill is working on Bitcoin protocol review I'm now working on
>> instrumenting the network stack so that we can get a view into
>> concurrency issues.
>
> Eric,
>
> Just a low-prio wishlist thing, it would be nice if at least the most
> fundamental of the LB state instrumentation/introspection routines you
> implement, would be exported and have well-defined interfaces, so that a
> program using LB could get some nice stat of thread use & activity and
> heap activity, even if just a listing like "Thread 1, blockchain, req
> 123", "Thread 2, network IO, select()", "Orphaned blocks 123 occupying
> 234MB", "Total blocks cached on the heap 345 occupying 456MB".

Sure, makes sense.

I've found that the orphan pool and transaction pool during sync are
generally empty, although the orphan pool can spike. The larger memory
variations are from internal asynchronous message queuing.

What we see is the network operations continue even on the stalled
strand (accumulating memory), and operations continue that are not
ordered by the stalled strand, like heartbeat. And eventually the
stalled strand will clear, freeing up its accumulated blocks and
transactions. I don't think anything is leaking, but because the node
continues to ask for and obtain blocks and transactions, it can generate
quite a backlog if the strand is held up.

e