:: Re: [Libbitcoin] trying offline obe…
Top Pagina
Delete this message
Reply to this message
Auteur: Thomas Hartman
Datum:  
Aan: Amir Taaki
CC: libbitcoin
Onderwerp: Re: [Libbitcoin] trying offline obelisk
What are the replacement candidates for the leveldb component?

Is the story pretty much told here?

https://wiki.unsystem.net/index.php/Libbitcoin/Blockchain

On Wed, Mar 26, 2014 at 9:05 AM, Amir Taaki <genjix@???> wrote:
> Do you want the new blockchain for fetch_history() to get outputs or
> fetch_transaction() on special txs right? Is there a way to only migrate
> the data you're interested in rather than the whole blockchain?
> Maybe you can even sign it before to show it's valid.
> LevelDB is not a good database and the next release of libbitcoin will
> have a newer faster slimmer database system for the blockchain.
>
> On 26/03/14 13:08, Grazcoin wrote:
>> This version seems to work fine offline on ubuntu 13.10, so I have an
>> offline obelisk that I can play with :)
>> Thanks you Amir!
>>
>> Now the question is how to update the setup while staying offline. For
>> security reasons, I would like to update it using a CD/DVD. I thought I
>> would tar the updated files from the blockchain directory and overwrite
>> that directory on my offline setup, e.g:
>> cd worker
>> tar cz `find blockchain -type f -newermt "Mar 23"` -f
>> ~/blockchain-update-20140326.tgz
>>
>> But I get way too many files (probably also old files get a newer
>> timestamp), so even a DVD is not enough for 3 days.
>> Any idea of how to do it in a simple and reasonably optimized way?
>>
>> Grazcoin
>>
>>
>> On 2014-03-24 22:58, Grazcoin wrote:
>>> No problems till now after this change.
>>> Thanks!
>>>
>>> On 2014-03-24 21:27, Amir Taaki wrote:
>>>> https://github.com/spesmilo/libbitcoin/commit/d73e6f03771c6e5e4ced166151d99f352da0b626
>>>>
>>>> new connections are limited to 10 per second now. if this is too low then
>>>> we should lower the time interval rather than the connection limit.
>>>>
>>>>> I was too fast on the "Fix verified" trigger.
>>>>> The assert comes in the same virtual box setup a little later, e.g.
>>>>> after 10 times of running:
>>>>> sx fetch-transaction
>>>>> fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33
>>>>> (it is a tx from block 1000)
>>>>> This is the case for both the commit that I claimed to be "Fix
>>>>> verified", and for the HEAD.
>>>>> Grazcoin
>>>>>
>>>>> On 2014-03-23 17:42, Grazcoin wrote:
>>>>>> Fix verified on the virtual box setup :)
>>>>>> It required a fresh build (running again install-sx.sh did not fix the
>>>>>> problem).
>>>>>>
>>>>>> Thanks!
>>>>>> Grazcoin
>>>>>>
>>>>>> On 2014-03-22 15:26, Amir Taaki wrote:
>>>>>>> Fixed:
>>>>>> https://github.com/spesmilo/libbitcoin/commit/9104dfa1661fcc2e61a576de446ad7e0771e74e8
>>>>>>
>>>>>>> On 21/03/14 11:24, Grazcoin wrote:
>>>>>>>> I reproduced the bug on the virtual box image.
>>>>>>>>
>>>>>>>> 1. I followed the image installation instructions that I gave
>>>>>> previously
>>>>>>>> on this thread.
>>>>>>>> 2. I ran install-sx.sh script of master git HEAD.
>>>>>>>> 3. created ~/worker/blockchain and ran sx initchain
>>>>>>>> 4. ran obworker (it took ~2 minutes for obworker to start connecting
>>>>>> though)
>>>>>>>> 5. I waited until block 3136, the stopped obworker.
>>>>>>>> 6. ifconfig eth0 down to disable networking (don't try the service
>>>>>>>> networking stop, since that kills the vm)
>>>>>>>> 7. .sx.cfg change to service = "tcp://127.0.0.1:9091"
>>>>>>>> 8. started again obworker (same output like reported in bug)
>>>>>>>> 9. sx fetch-last-height returns 3136 (a little too slow for my
>>>>>> taste).
>>>>>>>> 10. repeat 6 times, and them *boom* - assertion socket.self() like in
>>>>>>>> original bug description.
>>>>>>>>
>>>>>>>> On 2014-03-20 20:09, Thomas Hartman wrote:
>>>>>>>>> Sounds like a legitimate problem. Created issue in bugtracker,
>>>>>> linked
>>>>>>>>> back to this discussion.
>>>>>>>>>
>>>>>>>>> https://github.com/spesmilo/sx/issues/50
>>>>>>>>>
>>>>>>>>> On Thu, Mar 20, 2014 at 10:07 AM, Grazcoin <grazcoin@???>
>>>>>> wrote:
>>>>>>>>>> You can use virtualbox for testing ubuntu using console.
>>>>>>>>>> I verified the procedure myself (at least until running the sx
>>>>>> install
>>>>>>>>>> script).
>>>>>>>>>>
>>>>>>>>>> Instructions:
>>>>>>>>>> 1. download (less than 1GB)
>>>>>>>>>>
>>>>>> http://virtualboximages.com/Ubuntu+13.10+amd64+VirtualBox+VDI+Virtual+Appliance
>>>>>>>>>> 2. unrar e
>>>>>>>>>>
>>>>>> VirtualBox_-_Ubuntu-13.10-amd64-Desktop_VDI-[VirtualBoxImages.com].rar
>>>>>>>>>> 3. start virtualbox (I assume it is installed).
>>>>>>>>>> 4. New (name obelisk), Next.
>>>>>>>>>> 5. RAM - I gave 4GB, Next
>>>>>>>>>> 6. Use an existing virtual hard drive file - choose
>>>>>>>>>> ubuntu-13.10-desktop-amd64.vdi
>>>>>>>>>> 7. Start machine
>>>>>>>>>> 8. sudo su (password is adminuser)
>>>>>>>>>> 9. apt-get update
>>>>>>>>>> 10. apt-get install git
>>>>>>>>>> 11. git clone https://github.com/spesmilo/sx.git
>>>>>>>>>> 12. sh sx/install-sx.sh
>>>>>>>>>>
>>>>>>>>>> to stop and start networking
>>>>>>>>>> service networking stop
>>>>>>>>>> service networking start
>>>>>>>>>>
>>>>>>>>>> I hope it helps :)
>>>>>>>>>> Grazcoin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2014-03-19 22:17, Amir Taaki wrote:
>>>>>>>>>>> hey I'll take a look soon. I just accidentally shut off my server
>>>>>> today
>>>>>>>>>>> when connected via ssh and tries to shut down the networking. doh
>>>>>>>>>>>
>>>>>>>>>>> On 19/03/14 21:36, Thomas Hartman wrote:
>>>>>>>>>>>> my guess would be that libbitcoin might be the component with
>>>>>>>>>>>> lingering libs. so you might need to "nuke" that more thoroughly.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 19, 2014 at 1:35 PM, Thomas Hartman
>>>>>>>>>>>> <thomas@???> wrote:
>>>>>>>>>>>>> shot in the dark, but
>>>>>>>>>>>>>
>>>>>>>>>>>>> $ make clean
>>>>>>>>>>>>>
>>>>>>>>>>>>> and then run install-sx again?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sounds like some library component still might have old code.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 19, 2014 at 11:12 AM, Grazcoin <grazcoin@???>
>>>>>> wrote:
>>>>>>>>>>>>>> I was using the install-sx.sh script cloned today which updates
>>>>>> anyway
>>>>>>>>>>>>>> all sources, so I am in HEAD on all components (libbitcoin,
>>>>>> libwallet,
>>>>>>>>>>>>>> obelisk, sx, czmqpp, czmq, etc).
>>>>>>>>>>>>>> If you give me some hints or debug instructions, I could try to
>>>>>> fix it
>>>>>>>>>>>>>> myself.
>>>>>>>>>>>>>> I am working with ubuntu 13.10
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2014-03-19 18:25, Amir Taaki wrote:
>>>>>>>>>>>>>>> czmqpp-gitUpdate Obelisk and libczmq++
>>>>>>>>>>>>>>> I fixed this a few days ago but should work now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 19/03/14 17:22, Grazcoin wrote:
>>>>>>>>>>>>>>>> I decided to give the latest obelisk a try on ubuntu with
>>>>>> offline usage.
>>>>>>>>>>>>>>>> It works already partially.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Starting from a normal setup with network - everything works
>>>>>> fine.
>>>>>>>>>>>>>>>> Turning off networking.
>>>>>>>>>>>>>>>> # service networking stop
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Start worker
>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ obworker
>>>>>>>>>>>>>>>> Using config file: /etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>>> DEBUG [worker]: Listening: tcp://*:9091
>>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>>> DEBUG [worker]: Heartbeat: tcp://*:9092
>>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service: Bad
>>>>>> stream
>>>>>>>>>>>>>>>> Node started.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Then send a request
>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>>> 291345
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> great! it works!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Then a second request
>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>>> oh ...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I get:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> obworker: worker/worker.cpp:29: void
>>>>>>>>>>>>>>>> obelisk::send_worker::queue_send(const
>>>>>> obelisk::outgoing_message&):
>>>>>>>>>>>>>>>> Assertion `socket.self()' failed.
>>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I use the master HEAD.
>>>>>>>>>>>>>>>> This behavior is repeatable.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Quick fix?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 2014-01-31 20:35, Grazcoin wrote:
>>>>>>>>>>>>>>>>> I tried it on the latest ubuntu...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 2014-01-31 20:34, Thomas Hartman wrote:
>>>>>>>>>>>>>>>>>> I don't know the answer to your question, but what distro
>>>>>> are you running?
>>>>>>>>>>>>>>>>>> It ran out of the box for me on debian 7.1 but not on
>>>>>> latest ubuntu.
>>>>>>>>>>>>>>>>>> On Jan 31, 2014, at 6:05 AM, Grazcoin wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I was trying to test the functioning of obelisk offline:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1. started with a normal working online updated setup.
>>>>>>>>>>>>>>>>>>> 2. stopped the obworker and obbalancer.
>>>>>>>>>>>>>>>>>>> 3. updated /usr/local/etc/obelisk/worker.cfg with
>>>>>> outgoing-connections = 0 and listener_enabled = false
>>>>>>>>>>>>>>>>>>> 4. tried to start obworker, and got a crash. No extra info
>>>>>> on the error.log
>>>>>>>>>>>>>>>>>>> What is the correct config to make this work?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> git is in sync.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [jan-31 14:57:29] dev@dev-laptop:~/worker1$ obworker
>>>>>>>>>>>>>>>>>>> Using config file: /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>>>>>> DEBUG [worker]: Connecting: tcp://localhost:9092
>>>>>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service: Bad
>>>>>> stream
>>>>>>>>>>>>>>>>>>> ERROR: Unable to start session: Bad stream
>>>>>>>>>>>>>>>>>>> terminate called without an active exception
>>>>>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>>>>> [jan-31 14:57:39] dev@dev-laptop:~/worker1$ cat
>>>>>> /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>>> # Debug messages output file.
>>>>>>>>>>>>>>>>>>> output-file = "debug.log"
>>>>>>>>>>>>>>>>>>> # Error messages output file.
>>>>>>>>>>>>>>>>>>> error-file = "error.log"
>>>>>>>>>>>>>>>>>>> # Path to blockchain directory.
>>>>>>>>>>>>>>>>>>> blockchain-path = "./blockchain/"
>>>>>>>>>>>>>>>>>>> # List of hosts (loaded on startup, saved on exit).
>>>>>>>>>>>>>>>>>>> # If non-existant or empty then Obelisk/libbitcoin will
>>>>>>>>>>>>>>>>>>> # fall back to trying seed nodes instead.
>>>>>>>>>>>>>>>>>>> hosts-file = "hosts"
>>>>>>>>>>>>>>>>>>> # Next 2 values ignored if publisher is disabled
>>>>>>>>>>>>>>>>>>> publisher_enabled = false
>>>>>>>>>>>>>>>>>>> block-publish = "tcp://*:9093"
>>>>>>>>>>>>>>>>>>> tx-publish = "tcp://*:9094"
>>>>>>>>>>>>>>>>>>> # Connection to the balancer.
>>>>>>>>>>>>>>>>>>> # Which port to connect to the load balancer on.
>>>>>>>>>>>>>>>>>>> service = "tcp://localhost:9092"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> # Advanced features:
>>>>>>>>>>>>>>>>>>> # Uncomment to give this worker a named UUID. Must be
>>>>>> unique.
>>>>>>>>>>>>>>>>>>> #name = "ada"
>>>>>>>>>>>>>>>>>>> # Number of outgoing network connections to p2p network.
>>>>>>>>>>>>>>>>>>> outgoing-connections = 0
>>>>>>>>>>>>>>>>>>> # Listening for incoming connections.
>>>>>>>>>>>>>>>>>>> listener_enabled = false
>>>>>>>>>>>>>>>>>>> # Add manual outgoing nodes below.
>>>>>>>>>>>>>>>>>>> #nodes = (
>>>>>>>>>>>>>>>>>>> # ("localhost", 8333)
>>>>>>>>>>>>>>>>>>> #)
>>>>>>>>>>>>>>>>>>> # Show Obelisk requests in the log file for debugging or
>>>>>> security.
>>>>>>>>>>>>>>>>>>> # Slows down Obelisk speed. Disabled by default.
>>>>>>>>>>>>>>>>>>> #log_requests = true
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>>>
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>> Libbitcoin@???
>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>> _______________________________________________
>>>>>>>> Libbitcoin mailing list
>>>>>>>> Libbitcoin@???
>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>>>>
>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Libbitcoin mailing list
>>>>>>> Libbitcoin@???
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>> _______________________________________________
>>>>>> Libbitcoin mailing list
>>>>>> Libbitcoin@???
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>> _______________________________________________
>>>>> Libbitcoin mailing list
>>>>> Libbitcoin@???
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>>>>
>>>> _______________________________________________
>>>> Libbitcoin mailing list
>>>> Libbitcoin@???
>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>> _______________________________________________
>>> Libbitcoin mailing list
>>> Libbitcoin@???
>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>>
>
>
> _______________________________________________
> Libbitcoin mailing list
> Libbitcoin@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin
>