Do you want the new blockchain for fetch_history() to get outputs or
fetch_transaction() on special txs right? Is there a way to only migrate
the data you're interested in rather than the whole blockchain?
Maybe you can even sign it before to show it's valid.
LevelDB is not a good database and the next release of libbitcoin will
have a newer faster slimmer database system for the blockchain.

On 26/03/14 13:08, Grazcoin wrote:
> This version seems to work fine offline on ubuntu 13.10, so I have an
> offline obelisk that I can play with :)
> Thanks you Amir!
>
> Now the question is how to update the setup while staying offline. For
> security reasons, I would like to update it using a CD/DVD. I thought I
> would tar the updated files from the blockchain directory and overwrite
> that directory on my offline setup, e.g:
> cd worker
> tar cz `find blockchain -type f -newermt "Mar 23"` -f
> ~/blockchain-update-20140326.tgz
>
> But I get way too many files (probably also old files get a newer
> timestamp), so even a DVD is not enough for 3 days.
> Any idea of how to do it in a simple and reasonably optimized way?
>
> Grazcoin
>
>
> On 2014-03-24 22:58, Grazcoin wrote:
>> No problems till now after this change.
>> Thanks!
>>
>> On 2014-03-24 21:27, Amir Taaki wrote:
>>> https://github.com/spesmilo/libbitcoin/commit/d73e6f03771c6e5e4ced166151d99f352da0b626
>>>
>>> new connections are limited to 10 per second now. if this is too low then
>>> we should lower the time interval rather than the connection limit.
>>>
>>>> I was too fast on the "Fix verified" trigger.
>>>> The assert comes in the same virtual box setup a little later, e.g.
>>>> after 10 times of running:
>>>> sx fetch-transaction
>>>> fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33
>>>> (it is a tx from block 1000)
>>>> This is the case for both the commit that I claimed to be "Fix
>>>> verified", and for the HEAD.
>>>> Grazcoin
>>>>
>>>> On 2014-03-23 17:42, Grazcoin wrote:
>>>>> Fix verified on the virtual box setup :)
>>>>> It required a fresh build (running again install-sx.sh did not fix the
>>>>> problem).
>>>>>
>>>>> Thanks!
>>>>> Grazcoin
>>>>>
>>>>> On 2014-03-22 15:26, Amir Taaki wrote:
>>>>>> Fixed:
>>>>> https://github.com/spesmilo/libbitcoin/commit/9104dfa1661fcc2e61a576de446ad7e0771e74e8
>>>>>
>>>>>> On 21/03/14 11:24, Grazcoin wrote:
>>>>>>> I reproduced the bug on the virtual box image.
>>>>>>>
>>>>>>> 1. I followed the image installation instructions that I gave
>>>>> previously
>>>>>>> on this thread.
>>>>>>> 2. I ran install-sx.sh script of master git HEAD.
>>>>>>> 3. created ~/worker/blockchain and ran sx initchain
>>>>>>> 4. ran obworker (it took ~2 minutes for obworker to start connecting
>>>>> though)
>>>>>>> 5. I waited until block 3136, the stopped obworker.
>>>>>>> 6. ifconfig eth0 down to disable networking (don't try the service
>>>>>>> networking stop, since that kills the vm)
>>>>>>> 7. .sx.cfg change to service = "tcp://127.0.0.1:9091"
>>>>>>> 8. started again obworker (same output like reported in bug)
>>>>>>> 9. sx fetch-last-height returns 3136 (a little too slow for my
>>>>> taste).
>>>>>>> 10. repeat 6 times, and them *boom* - assertion socket.self() like in
>>>>>>> original bug description.
>>>>>>>
>>>>>>> On 2014-03-20 20:09, Thomas Hartman wrote:
>>>>>>>> Sounds like a legitimate problem. Created issue in bugtracker,
>>>>> linked
>>>>>>>> back to this discussion.
>>>>>>>>
>>>>>>>> https://github.com/spesmilo/sx/issues/50
>>>>>>>>
>>>>>>>> On Thu, Mar 20, 2014 at 10:07 AM, Grazcoin <grazcoin@???>
>>>>> wrote:
>>>>>>>>> You can use virtualbox for testing ubuntu using console.
>>>>>>>>> I verified the procedure myself (at least until running the sx
>>>>> install
>>>>>>>>> script).
>>>>>>>>>
>>>>>>>>> Instructions:
>>>>>>>>> 1. download (less than 1GB)
>>>>>>>>>
>>>>> http://virtualboximages.com/Ubuntu+13.10+amd64+VirtualBox+VDI+Virtual+Appliance
>>>>>>>>> 2. unrar e
>>>>>>>>>
>>>>> VirtualBox_-_Ubuntu-13.10-amd64-Desktop_VDI-[VirtualBoxImages.com].rar
>>>>>>>>> 3. start virtualbox (I assume it is installed).
>>>>>>>>> 4. New (name obelisk), Next.
>>>>>>>>> 5. RAM - I gave 4GB, Next
>>>>>>>>> 6. Use an existing virtual hard drive file - choose
>>>>>>>>> ubuntu-13.10-desktop-amd64.vdi
>>>>>>>>> 7. Start machine
>>>>>>>>> 8. sudo su (password is adminuser)
>>>>>>>>> 9. apt-get update
>>>>>>>>> 10. apt-get install git
>>>>>>>>> 11. git clone https://github.com/spesmilo/sx.git
>>>>>>>>> 12. sh sx/install-sx.sh
>>>>>>>>>
>>>>>>>>> to stop and start networking
>>>>>>>>> service networking stop
>>>>>>>>> service networking start
>>>>>>>>>
>>>>>>>>> I hope it helps :)
>>>>>>>>> Grazcoin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 2014-03-19 22:17, Amir Taaki wrote:
>>>>>>>>>> hey I'll take a look soon. I just accidentally shut off my server
>>>>> today
>>>>>>>>>> when connected via ssh and tries to shut down the networking. doh
>>>>>>>>>>
>>>>>>>>>> On 19/03/14 21:36, Thomas Hartman wrote:
>>>>>>>>>>> my guess would be that libbitcoin might be the component with
>>>>>>>>>>> lingering libs. so you might need to "nuke" that more thoroughly.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 19, 2014 at 1:35 PM, Thomas Hartman
>>>>>>>>>>> <thomas@???> wrote:
>>>>>>>>>>>> shot in the dark, but
>>>>>>>>>>>>
>>>>>>>>>>>> $ make clean
>>>>>>>>>>>>
>>>>>>>>>>>> and then run install-sx again?
>>>>>>>>>>>>
>>>>>>>>>>>> Sounds like some library component still might have old code.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 19, 2014 at 11:12 AM, Grazcoin <grazcoin@???>
>>>>> wrote:
>>>>>>>>>>>>> I was using the install-sx.sh script cloned today which updates
>>>>> anyway
>>>>>>>>>>>>> all sources, so I am in HEAD on all components (libbitcoin,
>>>>> libwallet,
>>>>>>>>>>>>> obelisk, sx, czmqpp, czmq, etc).
>>>>>>>>>>>>> If you give me some hints or debug instructions, I could try to
>>>>> fix it
>>>>>>>>>>>>> myself.
>>>>>>>>>>>>> I am working with ubuntu 13.10
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 2014-03-19 18:25, Amir Taaki wrote:
>>>>>>>>>>>>>> czmqpp-gitUpdate Obelisk and libczmq++
>>>>>>>>>>>>>> I fixed this a few days ago but should work now.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 19/03/14 17:22, Grazcoin wrote:
>>>>>>>>>>>>>>> I decided to give the latest obelisk a try on ubuntu with
>>>>> offline usage.
>>>>>>>>>>>>>>> It works already partially.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Starting from a normal setup with network - everything works
>>>>> fine.
>>>>>>>>>>>>>>> Turning off networking.
>>>>>>>>>>>>>>> # service networking stop
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Start worker
>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ obworker
>>>>>>>>>>>>>>> Using config file: /etc/obelisk/worker.cfg
>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>> DEBUG [worker]: Listening: tcp://*:9091
>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>> DEBUG [worker]: Heartbeat: tcp://*:9092
>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service: Bad
>>>>> stream
>>>>>>>>>>>>>>> Node started.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then send a request
>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>> 291345
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> great! it works!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then a second request
>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>> oh ...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I get:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> obworker: worker/worker.cpp:29: void
>>>>>>>>>>>>>>> obelisk::send_worker::queue_send(const
>>>>> obelisk::outgoing_message&):
>>>>>>>>>>>>>>> Assertion `socket.self()' failed.
>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I use the master HEAD.
>>>>>>>>>>>>>>> This behavior is repeatable.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Quick fix?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 2014-01-31 20:35, Grazcoin wrote:
>>>>>>>>>>>>>>>> I tried it on the latest ubuntu...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 2014-01-31 20:34, Thomas Hartman wrote:
>>>>>>>>>>>>>>>>> I don't know the answer to your question, but what distro
>>>>> are you running?
>>>>>>>>>>>>>>>>> It ran out of the box for me on debian 7.1 but not on
>>>>> latest ubuntu.
>>>>>>>>>>>>>>>>> On Jan 31, 2014, at 6:05 AM, Grazcoin wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I was trying to test the functioning of obelisk offline:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. started with a normal working online updated setup.
>>>>>>>>>>>>>>>>>> 2. stopped the obworker and obbalancer.
>>>>>>>>>>>>>>>>>> 3. updated /usr/local/etc/obelisk/worker.cfg with
>>>>> outgoing-connections = 0 and listener_enabled = false
>>>>>>>>>>>>>>>>>> 4. tried to start obworker, and got a crash. No extra info
>>>>> on the error.log
>>>>>>>>>>>>>>>>>> What is the correct config to make this work?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> git is in sync.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [jan-31 14:57:29] dev@dev-laptop:~/worker1$ obworker
>>>>>>>>>>>>>>>>>> Using config file: /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>>>>> DEBUG [worker]: Connecting: tcp://localhost:9092
>>>>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service: Bad
>>>>> stream
>>>>>>>>>>>>>>>>>> ERROR: Unable to start session: Bad stream
>>>>>>>>>>>>>>>>>> terminate called without an active exception
>>>>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>>>> [jan-31 14:57:39] dev@dev-laptop:~/worker1$ cat
>>>>> /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>> # Debug messages output file.
>>>>>>>>>>>>>>>>>> output-file = "debug.log"
>>>>>>>>>>>>>>>>>> # Error messages output file.
>>>>>>>>>>>>>>>>>> error-file = "error.log"
>>>>>>>>>>>>>>>>>> # Path to blockchain directory.
>>>>>>>>>>>>>>>>>> blockchain-path = "./blockchain/"
>>>>>>>>>>>>>>>>>> # List of hosts (loaded on startup, saved on exit).
>>>>>>>>>>>>>>>>>> # If non-existant or empty then Obelisk/libbitcoin will
>>>>>>>>>>>>>>>>>> # fall back to trying seed nodes instead.
>>>>>>>>>>>>>>>>>> hosts-file = "hosts"
>>>>>>>>>>>>>>>>>> # Next 2 values ignored if publisher is disabled
>>>>>>>>>>>>>>>>>> publisher_enabled = false
>>>>>>>>>>>>>>>>>> block-publish = "tcp://*:9093"
>>>>>>>>>>>>>>>>>> tx-publish = "tcp://*:9094"
>>>>>>>>>>>>>>>>>> # Connection to the balancer.
>>>>>>>>>>>>>>>>>> # Which port to connect to the load balancer on.
>>>>>>>>>>>>>>>>>> service = "tcp://localhost:9092"
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> # Advanced features:
>>>>>>>>>>>>>>>>>> # Uncomment to give this worker a named UUID. Must be
>>>>> unique.
>>>>>>>>>>>>>>>>>> #name = "ada"
>>>>>>>>>>>>>>>>>> # Number of outgoing network connections to p2p network.
>>>>>>>>>>>>>>>>>> outgoing-connections = 0
>>>>>>>>>>>>>>>>>> # Listening for incoming connections.
>>>>>>>>>>>>>>>>>> listener_enabled = false
>>>>>>>>>>>>>>>>>> # Add manual outgoing nodes below.
>>>>>>>>>>>>>>>>>> #nodes = (
>>>>>>>>>>>>>>>>>> # ("localhost", 8333)
>>>>>>>>>>>>>>>>>> #)
>>>>>>>>>>>>>>>>>> # Show Obelisk requests in the log file for debugging or
>>>>> security.
>>>>>>>>>>>>>>>>>> # Slows down Obelisk speed. Disabled by default.
>>>>>>>>>>>>>>>>>> #log_requests = true
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>>
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>> Libbitcoin@???
>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>> _______________________________________________
>>>>>>>>> Libbitcoin mailing list
>>>>>>>>> Libbitcoin@???
>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>> _______________________________________________
>>>>>>> Libbitcoin mailing list
>>>>>>> Libbitcoin@???
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>
>>>>>
>>>>>> _______________________________________________
>>>>>> Libbitcoin mailing list
>>>>>> Libbitcoin@???
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>> _______________________________________________
>>>>> Libbitcoin mailing list
>>>>> Libbitcoin@???
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>> _______________________________________________
>>>> Libbitcoin mailing list
>>>> Libbitcoin@???
>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>
>>> _______________________________________________
>>> Libbitcoin mailing list
>>> Libbitcoin@???
>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>