On 2015-06-28 21:07, Eric Voskuil wrote:
> There were no locking issues found in libbitcoin-node (or server). The
> stall resulted from the failure to poll. Please see the detailed
> description of the stall issue in the link below.

Ah sorry I meant stall.

> The proactor pattern is used by libbitcoin::network (as well as node
> and
> server). This pattern inherently allows an unbounded increase in memory
> consumption in the case where incoming messages are not processed as
> fast as they are received. This is a very efficient pattern in one
> sense
> as it avoids most locking and minimizes thread consumption. On the
> other
> hand it achieves this through queueing all actions.
>
> It is currently possible for libbitcoin to fail during sync due to
> excess memory consumption. I consider this a bug. The fix I provided
> prevents the stall but in some cases it may exacerbate the memory
> consumption issue. This could be considered a self-inflicted DoS. The
> networking stack needs a more robust poller (so we don't self-induce a
> failure) as well as DOS protection (so we can shed messages and/or
> connections that are likely to induce a failure).

Ah right, this was discussed almost two years ago.

What's the very-worst-case memory consumption scenario right now, like
Big (as in 10:s of GB:s)?

Is there any straightforward way to limit this - many times there's an
entry point, or work propagation point, that could be throttled. Perhaps
making a job scheduler that keeps the jobs list on the disk.