Hi Eric,

Thanks a lot for your response -

So startup peaks ~1GB independent of blockchain size, however some of
that could be offloaded to swap.

The resident RAM consumption after that is made up of libBitcoin's
internal mempool and that one is ~~100MB,

so a sanctions ~1.2GB resident RAM for the v2 node should be safe,
though really 100-200MB is the more correct figure after startup?


Am all with you that everything about LibBitcoin and BitcoinD is
different altogether.

Some OS environments have max OS process resident RAM size limits and
virtual memory limits (ulimit) and take down the process or make malloc
return NULL, leading to c++ heap overflow exception or worse, so it's
good to have an idea here.

Thanks!
Mlmikael


On 2015-05-19 11:36, Eric Voskuil wrote:
> Hi,
>
> There is not specific number for minimum required required RAM. If you
> are low enough on RAM then the process will page virtual memory.
> Depending on what that is being used for this can really slow things
> down or have a minimal effect. If you run out of swap space you should
> see an exception bring the process down (which is intended behavior).
> I'd like to give better info, but that's what we have right now.
>
> There is no relationship between the implementation of libbitcoin and
> bitcoind. Libbitcoin uses memory-mapped I/O, optimized for fast query
> response but not conservative use of memory. It's an intentional
> trade-off. The various data files are mapped into memory and can claim
> as much as they do on disk. However this will most certainly be virtual
> memory, so it has nothing to do with the amount of hardware RAM
> available.
>
> I've uploaded some resource snapshots one of my built test platforms:
>
> https://github.com/libbitcoin/libbitcoin-server/issues/76
>
> This is after doing some work today to manage thread priorities, which
> I
> am in the process of merging. That helps quite a bit with CPU hogging.
> As you can see CPU and network are typically pretty reasonable. This
> makes it possible to sync and still use the machine.
>
> Private bytes sort of represents the memory (excluding memory mapped
> I/O
> for the data files and binaries) that the process is claiming. As you
> can see this spikes to around 1GB when starting up. It can also climb
> over time as the mempool is filling up but drops down significantly
> once
> blocks are identified to consume those transactions.
>
> Also notice that this amount tends to plateau. This is probably the
> result of hitting maximums for the mempool, not some memory limit
> imposed in code. This machine has 8GB RAM, so this 1GB+ amount is not
> actually a significant issue. I've seen this number stay in the
> 50KB-100KB range for some time.
>
> I see no unexpected behavior in the logs, although the lack of
> header-first sync certainly consumes some time and memory (due to the
> receipt of blocks and transactions before they can be connected to the
> chain). I believe the blocking issues in the 330k range is resolved. I
> had misparameterized the libbitcoin-consensus library. The OpenSSL
> issue
> is worked around in the consensus lib, so we should not have to worry
> about that until the next OpenSSL update.
>
> We are working now to see if there is any actual issue remaining.
> Working off and an I've synced 270,000 blocks today and it's moving
> along as I would expect.
>
> e
>
> On 05/18/2015 10:34 AM, mlmikael wrote:
>> Hi!
>>
>> How much (resident) RAM should be sanctioned for a fully syncing v2
>> node?
>>
>> Is the RAM use here proportionate to the amount of parallellism in the
>> syncing work (if so a configurable upper limit possible)?
>>
>> BitcoinD required ~700MB last I checked.
>>
>> Thanks!
>> Mlmikael
>>
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin