Oh Eric, I forgot to mention, the scan() method is *not* for stealth...
now Obelisk will support privacy enhanced history for address- scan by
prefix.

When you restore your wallet, you should use the much faster
fetch_history(). But for updating the wallet, use the scan() method-
it's also faster because you avoid pulling the entire history. Also we
can increase privacy and increase performance (less calls to scan()) by
'mining' addresses with the same prefix, therefore using single calls to
scan() at a time. However I'm not going to push that on DW team- at
least not right away. Designer also has his own list of requirements.

On 13/08/14 02:46, Amir Taaki wrote:
> Yep ok, I'll this list updated with news of my progress. Right now I'm
> going to create the main database abstractions to encapsulate the lower
> level database objects (transaction_database, history_database,
> spend_database, block_database, ...). Then I'll integrate these into the
> blockchain.
>
> It's better with stealth stuff to have the tx_hash because you'll
> discard most of the results, and then you can choose to query a certain
> subset of it. tx data is ~200 bytes so even with 10 blocks containing
> say >10 txs, that's already 20 kb which from many hosts is a lot of data
> (this is minimum case).
>
> Anyway I'm willing to implement whatever as long as it has decent
> consideration given to scalability issues. We do have to make some
> tradeoffs. Also the initial design is never going to be perfect.
>
> More is possible, but I want to focus on the absolute bare basics for
> now to get them 100% solid and our the door on a production level. That
> will involve significant Q&A work between now and the end of the year.
> I'm also working with a designer for DW to craft the tools to create a
> beautiful and great experience for the users rather than a tool that
> "just works" but is horrible to use.
>
> On 12/08/14 22:25, Eric Voskuil wrote:
>> I am pretty close to being able to merge the single_file SX branch. It's a total rewrite so it may not be worth your time to fix the SX dev branch. Once obelisk and libbitcoin dev branches are updated I can merge my SX branch into dev and reconcile open issues. That would of course lead to a period of breakage on the dev branch until the merge is complete. I can wait to do this until after the updated obelisk client/server merge from s_tec.
>>
>> Adding paging to fetch_history makes sense.
>>
>> Are you replacing fetch_stealth(prefix, handler, height) with scan(prefix, handler, from_height, to_height)? If so it also makes sense to me. Prefix searches (scans/subscriptions) should always return the full transaction, whether or not the tx appears to include a stealth payment. Currently fetch_stealth returns the tx hash, so a subsequent call is required to chase down the tx. This seems like a material privacy issue with fetch_stealth(). Also searching for transactions using a prefix is useful for both stealth and non-stealth payments, meaning that the return of a "stealth row" from a prefix search is problematic. The paging is also good.
>>
>> In general we should isolate the idea of a "prefix scan" with a "stealth payment". Any transaction may or may not include stealth payments, and there may or may not be stealth metadata associated with a stealth payment in a transaction (implying out of band / off blockchain communication of the ephemeral key). Identifying a stealth payment within a transaction should be strictly a client role subsequent to a prefix search (or other means of obtaining the transaction). If a server can readily identify and index all (or even a slight superset of) stealth payments then it's clear that stealth is not gaining meaningful cover from non-stealth transactions, which I would consider another material privacy issue.
>>
>> e
>>
>> -----Original Message-----
>> From: Libbitcoin [mailto:libbitcoin-bounces@lists.dyne.org] On Behalf Of Amir Taaki
>> Sent: Tuesday, August 12, 2014 8:37 AM
>> To: libbitcoin@???
>> Subject: [Libbitcoin] Proposed blockchain changes
>>
>> Beginning now I will begin integrating blockchain database changes
>> inside libbitcoin_blockchain, afterwards porting obelisk and sx on the
>> develop branches.
>>
>> New methods I propose:
>>

>>     fetch_block_data(...) -> block_header + [tx_positions]
>>     fetch_transaction(position_type position, ...)
>>     scan(prefix, handler, from_height, to_height)

>>

>>     fetch_history will have stop and start indexes.
>>     (* or maybe 2 methods)

>>
>> Also there will be a distinction between required and optional databases.
>> Optional databases:
>> * history database
>> * history scan database
>> * stealth database
>> You will have the option to activate these after certain block heights
>> making initial sync much faster.
>>
>>
>>
>
>
>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>