Aha Amir, if basically on an SSD the IO issues should *never* happen
with very very very good margins, then I agree with you, this is a good
solution, in particular if this workaround you suggested (quoted below;
which as a side-effefct provides a non-graceful shutdown) does prevent a
full safeguard against the crash and burn scenario i.e. libbitcoin is
robust. That's what it does right?

Is it committed so l.b. is now safe?

Thanks

On 2014-01-09 00:29, Amir Taaki wrote:

> As a programmer writing systems code targetting servers, I have to make
> tradeoffs to better serve configurations/uses/scenarios over others.
> I'm of the view that for old style HDDs are obsolete on servers,
> especially for databases, network apps, ... anything doing disk IO.
>
> Seek times of SSDs are basically zero. To jump around a file is no
> longer costly (think of HDDs like VCRs and SSDs like DVDs), so many
> things change with how you optimise your software. An example, a large
> part of LevelDBs design/activity is about locating data close together
> to minimise head movement. The semantics of SSDs on the OS layer are
> different too.
>
> This is why libbitcoin doesn't work so well on HDDs. If we can find a
> way to make it perform better without significantly comprimising
> elegance, and without comprimising on SSD performance then it should be
> fixed. But I don't think it's a good idea to loose too much sleep over it.
>
> Especially since consumer SSDs are wildly popular and dropping in price
> fast. With the advent of new affordable high-end DRAM SSDs entering the
> market, I expect to see a further drop in the price of SSDs.
>
> http://www.techpowerup.com/188507/crossbar-unveils-resistive-ram-non-volatile-memory-technology.html [1]libbitcoin is a design for tomorrow; solving yesterday's problems today
> is for slow-moving leviathans.
>
> On 08/01/14 15:51, mlmikael wrote:
> Aha. So what's the way to a "clean" (as in, robust/non-leaky) feedback mechanism for making earlier parts degrade gracefully and stop adding more work, when the buffer is full/close to full? Again thank you for the libbitcoin, it's basically essential to the Bitcoin world and may be the primary implementation if you look at it really. On 2014-01-08 11:42, Amir Taaki wrote: Not really, and it doesn't seem like a good solution (more of a workaround) now I'm thinking. The problem (if this is it), is that the block writer is IO bound, and blocks are being queued such that memory is exhausted before writes can complete. I'm not sure if it is the problem, but certainly should be something along these lines if it's only caused by HDD. Only 500 blocks are downloaded at a time and attempted to be stored. _______________________________________________ Libbitcoin mailing list Libbitcoin@??? https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin [2]

_______________________________________________
Libbitcoin mailing list
Libbitcoin@???
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin [2]

On 2014-01-07 22:08, Amir Taaki wrote:

> So if the buffer queue is too big, then don't queue writes?
>
> Would this be a good scheme:
>
> * Have a value in the service poller called:
>
> size_t blocks_queued_count_ = 0;
>
> * Change the block storage code when a new block is received to:
>
> if (blocks_queued_count_ > queue_capacity_)
> {
> log_debug(LOG_POLLER) << "Throttling writer.";
> return;
> }
> chain_.store(block, ...)
> ++blocks_queued_count_;
>
> * Change the callback in the store to decrement the counter:
>
> --blocks_queued_count_;
>
> How does that sound?



Links:
------
[1]
http://www.techpowerup.com/188507/crossbar-unveils-resistive-ram-non-volatile-memory-technology.html
[2] https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin