On 2015-07-03 16:02, Eric Voskuil wrote:

>>> The proactor pattern is used by libbitcoin::network (as well as node
>>> and
>>> server). This pattern inherently allows an unbounded increase in
>>> memory
>>> consumption in the case where incoming messages are not processed as
>>> fast as they are received. This is a very efficient pattern in one
>>> sense
>>> as it avoids most locking and minimizes thread consumption. On the
>>> other
>>> hand it achieves this through queueing all actions.
>>> It is currently possible for libbitcoin to fail during sync due to
>>> excess memory consumption. I consider this a bug. The fix I provided
>>> prevents the stall but in some cases it may exacerbate the memory
>>> consumption issue. This could be considered a self-inflicted DoS. The
>>> networking stack needs a more robust poller (so we don't self-induce
>>> a
>>> failure) as well as DOS protection (so we can shed messages and/or
>>> connections that are likely to induce a failure).

>> Ah right, this was discussed almost two years ago.
>>
>> What's the very-worst-case memory consumption scenario right now, like
>> Big (as in 10:s of GB:s)?

> Worst case is as much memory as the OS will give the process. Without
> code changes the easiest mitigation is the restart the process if
> memory exceeds your limit.


Say you give it 2GB max memory and have a script that automatically
restarts after crash, will the error repair itself over time?


Any plans to fix it?


While this architecture glitch is there, are there any particular do:s
and don't:s terms of system setup and node API use, for it to work as
well as possible?


(Btw some Linuxes lack max memory size so by default they'd crash and
burn.)

Thanks!