I started writing this as part of the other thread, but it got
super-long, so I decided to post it separately...

The raw blockchain has two data-structures which can exist
independently: the transaction and the block. Everything else is
contained within these two data structures, and never appears
separately (inputs, outputs, signatures, scripts, etc.). Transactions
can be free-floating in the mempool, for instance, but there is no
such thing as a loose input script; it's always part of a transaction.

Our protocol mirrors this fact by providing two query functions: "get
transactions" and "get blocks". Aside from details about result
encoding (full transactions vs hashes vs utxo's), this is all we could
ever need.

On the push side, if we just provide "push transaction" and "push
block," we would actually have enough power to run the full network
with an alternative to the satoshi protocol.

Of course, these four messages (get/push transaction and get/push
block) don't include housekeeping things like "get peer IP list". If
we include those, we get a nice three-layer protocol:

// Read-only blockchain access:
interface read_blockchain
{
get_transactions(...);
get_blocks(...);
}

// Write-only blockchain access:
interface write_blockchain
{
push_transaction(...);
push_block(...);
}

// Full-node p2p interface:
interface blockchain_node
  : public write_blockchain,
    public read_blockchain
{
  get_version(...);
  get_peer_list(...);
  validate_transaction(...);
  ...
}

Notice that I have put validate_transaction with the housekeeping
stuff, since it's mainly a debugging thing.

This protocol is truly universal. It doesn't care whether the
connection uses JSON over websockets, protobuff over zeromq, or even
if there is a connection at all.

This last option is really interesting. Imagine what would happen if
we create a nice C++ interface that mirrors this protocol, and start
using it internally. Suddenly, all the different parts of our system
(libbitcoin-blockchain, libbitcoin-server, libbitcoin-client, etc.)
all share a common language for talking about the blockchain.

The libbitcoin-blockchain library would start out by implementing this
C++ interface as a way of querying its blockchain (at least the
read_blockchain part). The results wouldn't include the mempool, of
course, but that's not a problem. The libbitcoin-server node would
take the results from libbitcoin-blockchain, augment them with results
from its mempool, and expose them again under the exact same C++
read_blockchain interface. Then, libbitcoin-protocol would consume the
interface, marshal it over zeromq, and reconstitute it on the other
side. A hypothetical libbitcoin-spv library would consume the
interface again, validate it, cache it, and present it to the client.

Under this model, any trusting wallet can be turned into a
non-trusting wallet by slipping an SPV module into the data pipeline.
It's just like clicking in Legos - the common bump-and-socket
interface makes everything interchangeable. For example, if we create
a three-layer sandwich with libbitcoin-protocol, libbitcoin-spv, and
another libbitcoin-protocol, we suddenly have a lightweight cache that
we could put in front of a full-node server for load-balancing. The
possibilities are endless.

I think a design like this would represent a really powerful evolution
of the libbitcoin architecture. It will be interesting to see how much
work it takes, or even if it's really possible.

-William