I am pretty close to being able to merge the single_file SX branch. It's a total rewrite so it may not be worth your time to fix the SX dev branch. Once obelisk and libbitcoin dev branches are updated I can merge my SX branch into dev and reconcile open issues. That would of course lead to a period of breakage on the dev branch until the merge is complete. I can wait to do this until after the updated obelisk client/server merge from s_tec.

Adding paging to fetch_history makes sense.

Are you replacing fetch_stealth(prefix, handler, height) with scan(prefix, handler, from_height, to_height)? If so it also makes sense to me. Prefix searches (scans/subscriptions) should always return the full transaction, whether or not the tx appears to include a stealth payment. Currently fetch_stealth returns the tx hash, so a subsequent call is required to chase down the tx. This seems like a material privacy issue with fetch_stealth(). Also searching for transactions using a prefix is useful for both stealth and non-stealth payments, meaning that the return of a "stealth row" from a prefix search is problematic. The paging is also good.

In general we should isolate the idea of a "prefix scan" with a "stealth payment". Any transaction may or may not include stealth payments, and there may or may not be stealth metadata associated with a stealth payment in a transaction (implying out of band / off blockchain communication of the ephemeral key). Identifying a stealth payment within a transaction should be strictly a client role subsequent to a prefix search (or other means of obtaining the transaction). If a server can readily identify and index all (or even a slight superset of) stealth payments then it's clear that stealth is not gaining meaningful cover from non-stealth transactions, which I would consider another material privacy issue.

e

-----Original Message-----
From: Libbitcoin [mailto:libbitcoin-bounces@lists.dyne.org] On Behalf Of Amir Taaki
Sent: Tuesday, August 12, 2014 8:37 AM
To: libbitcoin@???
Subject: [Libbitcoin] Proposed blockchain changes

Beginning now I will begin integrating blockchain database changes
inside libbitcoin_blockchain, afterwards porting obelisk and sx on the
develop branches.

New methods I propose:

    fetch_block_data(...) -> block_header + [tx_positions]
    fetch_transaction(position_type position, ...)
    scan(prefix, handler, from_height, to_height)

    fetch_history will have stop and start indexes.
    (* or maybe 2 methods)

Also there will be a distinction between required and optional databases.
Optional databases:
* history database
* history scan database
* stealth database
You will have the option to activate these after certain block heights
making initial sync much faster.