:: Re: [Libbitcoin] libbitcoin-server …
Top Page
Delete this message
Reply to this message
Author: Eric Voskuil
Date:  
To: mlmikael
CC: libbitcoin@lists.dyne.org
Subject: Re: [Libbitcoin] libbitcoin-server update
Worst case is as much memory as the OS will give the process. Without code changes the easiest mitigation is the restart the process if memory exceeds your limit.

e


> On Jun 28, 2015, at 9:41 PM, mlmikael <mlmikael@???> wrote:
>
>> On 2015-06-28 21:07, Eric Voskuil wrote:
>> There were no locking issues found in libbitcoin-node (or server). The
>> stall resulted from the failure to poll. Please see the detailed
>> description of the stall issue in the link below.
>
> Ah sorry I meant stall.
>
>> The proactor pattern is used by libbitcoin::network (as well as node and
>> server). This pattern inherently allows an unbounded increase in memory
>> consumption in the case where incoming messages are not processed as
>> fast as they are received. This is a very efficient pattern in one sense
>> as it avoids most locking and minimizes thread consumption. On the other
>> hand it achieves this through queueing all actions.
>> It is currently possible for libbitcoin to fail during sync due to
>> excess memory consumption. I consider this a bug. The fix I provided
>> prevents the stall but in some cases it may exacerbate the memory
>> consumption issue. This could be considered a self-inflicted DoS. The
>> networking stack needs a more robust poller (so we don't self-induce a
>> failure) as well as DOS protection (so we can shed messages and/or
>> connections that are likely to induce a failure).
>
> Ah right, this was discussed almost two years ago.
>
> What's the very-worst-case memory consumption scenario right now, like Big (as in 10:s of GB:s)?
>
> Is there any straightforward way to limit this - many times there's an entry point, or work propagation point, that could be throttled. Perhaps making a job scheduler that keeps the jobs list on the disk.
>