I agree with you totally, I got very excited when Eric was showing me
through the design. It's definitely very well thought out.

The only thing is that you have address prefix search paginated by block
heights.

The current database scheme has 2 databases:

* history_scan_database
Scanning from a certain block height (to get updates) by prefix
is fast.
Fetching the entire history for an address from 0 is very slow.
Paginating on block height is constant speed.

* history_database
The more classic scheme of fetching a history for a fixed address.
Doesn't not support prefix.
Has a different pagination scheme which is constant speed.
Using block heights to paginate would mean linear speed indexing.

And it has its uses (i.e restoring new wallet). I wouldn't expect you to
use the scan database in all scenarios.

http://blog.coinkite.com/post/97397052686/public-obelisk-server-for-the-community

http://www.reddit.com/r/Bitcoin/comments/2go7qm/devs_be_sure_to_test_your_bitcoin_apidata/

https://wiki.unsystem.net/en/index.php/Libbitcoin/Blockchain/htdb_slab_Performance

we've got some good promotion based on the speed of fetching address
history recently. So it's still something I feel we should support like
if you use the server on the backend as a private API with
encryption/signing support. Some people don't need the extra security,
privacy and don't want to do the SPV processing. For Darkwallet that's
great, and we want to move to this new protocol.

Since the fetch_history has its own semantics (query based on full
address only- not prefix) and returns results different to what's
expected, I thought it'd be apt to separate into its own call. But let
me know what's the best option here.

On 09/26/2014 06:39 AM, William Swanson wrote:
> I started writing this as part of the other thread, but it got
> super-long, so I decided to post it separately...
>
> The raw blockchain has two data-structures which can exist
> independently: the transaction and the block. Everything else is
> contained within these two data structures, and never appears
> separately (inputs, outputs, signatures, scripts, etc.). Transactions
> can be free-floating in the mempool, for instance, but there is no
> such thing as a loose input script; it's always part of a transaction.
>
> Our protocol mirrors this fact by providing two query functions: "get
> transactions" and "get blocks". Aside from details about result
> encoding (full transactions vs hashes vs utxo's), this is all we could
> ever need.
>
> On the push side, if we just provide "push transaction" and "push
> block," we would actually have enough power to run the full network
> with an alternative to the satoshi protocol.
>
> Of course, these four messages (get/push transaction and get/push
> block) don't include housekeeping things like "get peer IP list". If
> we include those, we get a nice three-layer protocol:
>
> // Read-only blockchain access:
> interface read_blockchain
> {
> get_transactions(...);
> get_blocks(...);
> }
>
> // Write-only blockchain access:
> interface write_blockchain
> {
> push_transaction(...);
> push_block(...);
> }
>

> // Full-node p2p interface:
> interface blockchain_node
>   : public write_blockchain,
>     public read_blockchain
> {
>   get_version(...);
>   get_peer_list(...);
>   validate_transaction(...);
>   ...
> }

>
> Notice that I have put validate_transaction with the housekeeping
> stuff, since it's mainly a debugging thing.
>
> This protocol is truly universal. It doesn't care whether the
> connection uses JSON over websockets, protobuff over zeromq, or even
> if there is a connection at all.
>
> This last option is really interesting. Imagine what would happen if
> we create a nice C++ interface that mirrors this protocol, and start
> using it internally. Suddenly, all the different parts of our system
> (libbitcoin-blockchain, libbitcoin-server, libbitcoin-client, etc.)
> all share a common language for talking about the blockchain.
>
> The libbitcoin-blockchain library would start out by implementing this
> C++ interface as a way of querying its blockchain (at least the
> read_blockchain part). The results wouldn't include the mempool, of
> course, but that's not a problem. The libbitcoin-server node would
> take the results from libbitcoin-blockchain, augment them with results
> from its mempool, and expose them again under the exact same C++
> read_blockchain interface. Then, libbitcoin-protocol would consume the
> interface, marshal it over zeromq, and reconstitute it on the other
> side. A hypothetical libbitcoin-spv library would consume the
> interface again, validate it, cache it, and present it to the client.
>
> Under this model, any trusting wallet can be turned into a
> non-trusting wallet by slipping an SPV module into the data pipeline.
> It's just like clicking in Legos - the common bump-and-socket
> interface makes everything interchangeable. For example, if we create
> a three-layer sandwich with libbitcoin-protocol, libbitcoin-spv, and
> another libbitcoin-protocol, we suddenly have a lightweight cache that
> we could put in front of a full-node server for load-balancing. The
> possibilities are endless.
>
> I think a design like this would represent a really powerful evolution
> of the libbitcoin architecture. It will be interesting to see how much
> work it takes, or even if it's really possible.
>
> -William
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>