Autore: Amir Taaki Data: To: Thomas Hartman CC: libbitcoin Oggetto: Re: [Libbitcoin] trying offline obelisk
yep but there's more but thats the base.
> What are the replacement candidates for the leveldb component?
>
> Is the story pretty much told here?
>
> https://wiki.unsystem.net/index.php/Libbitcoin/Blockchain >
> On Wed, Mar 26, 2014 at 9:05 AM, Amir Taaki <genjix@???> wrote:
>> Do you want the new blockchain for fetch_history() to get outputs or
>> fetch_transaction() on special txs right? Is there a way to only migrate
>> the data you're interested in rather than the whole blockchain?
>> Maybe you can even sign it before to show it's valid.
>> LevelDB is not a good database and the next release of libbitcoin will
>> have a newer faster slimmer database system for the blockchain.
>>
>> On 26/03/14 13:08, Grazcoin wrote:
>>> This version seems to work fine offline on ubuntu 13.10, so I have an
>>> offline obelisk that I can play with :)
>>> Thanks you Amir!
>>>
>>> Now the question is how to update the setup while staying offline. For
>>> security reasons, I would like to update it using a CD/DVD. I thought I
>>> would tar the updated files from the blockchain directory and overwrite
>>> that directory on my offline setup, e.g:
>>> cd worker
>>> tar cz `find blockchain -type f -newermt "Mar 23"` -f
>>> ~/blockchain-update-20140326.tgz
>>>
>>> But I get way too many files (probably also old files get a newer
>>> timestamp), so even a DVD is not enough for 3 days.
>>> Any idea of how to do it in a simple and reasonably optimized way?
>>>
>>> Grazcoin
>>>
>>>
>>> On 2014-03-24 22:58, Grazcoin wrote:
>>>> No problems till now after this change.
>>>> Thanks!
>>>>
>>>> On 2014-03-24 21:27, Amir Taaki wrote:
>>>>> https://github.com/spesmilo/libbitcoin/commit/d73e6f03771c6e5e4ced166151d99f352da0b626 >>>>>
>>>>> new connections are limited to 10 per second now. if this is too low
>>>>> then
>>>>> we should lower the time interval rather than the connection limit.
>>>>>
>>>>>> I was too fast on the "Fix verified" trigger.
>>>>>> The assert comes in the same virtual box setup a little later, e.g.
>>>>>> after 10 times of running:
>>>>>> sx fetch-transaction
>>>>>> fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33
>>>>>> (it is a tx from block 1000)
>>>>>> This is the case for both the commit that I claimed to be "Fix
>>>>>> verified", and for the HEAD.
>>>>>> Grazcoin
>>>>>>
>>>>>> On 2014-03-23 17:42, Grazcoin wrote:
>>>>>>> Fix verified on the virtual box setup :)
>>>>>>> It required a fresh build (running again install-sx.sh did not fix
>>>>>>> the
>>>>>>> problem).
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Grazcoin
>>>>>>>
>>>>>>> On 2014-03-22 15:26, Amir Taaki wrote:
>>>>>>>> Fixed:
>>>>>>> https://github.com/spesmilo/libbitcoin/commit/9104dfa1661fcc2e61a576de446ad7e0771e74e8 >>>>>>>
>>>>>>>> On 21/03/14 11:24, Grazcoin wrote:
>>>>>>>>> I reproduced the bug on the virtual box image.
>>>>>>>>>
>>>>>>>>> 1. I followed the image installation instructions that I gave
>>>>>>> previously
>>>>>>>>> on this thread.
>>>>>>>>> 2. I ran install-sx.sh script of master git HEAD.
>>>>>>>>> 3. created ~/worker/blockchain and ran sx initchain
>>>>>>>>> 4. ran obworker (it took ~2 minutes for obworker to start
>>>>>>>>> connecting
>>>>>>> though)
>>>>>>>>> 5. I waited until block 3136, the stopped obworker.
>>>>>>>>> 6. ifconfig eth0 down to disable networking (don't try the
>>>>>>>>> service
>>>>>>>>> networking stop, since that kills the vm)
>>>>>>>>> 7. .sx.cfg change to service = "tcp://127.0.0.1:9091"
>>>>>>>>> 8. started again obworker (same output like reported in bug)
>>>>>>>>> 9. sx fetch-last-height returns 3136 (a little too slow for my
>>>>>>> taste).
>>>>>>>>> 10. repeat 6 times, and them *boom* - assertion socket.self()
>>>>>>>>> like in
>>>>>>>>> original bug description.
>>>>>>>>>
>>>>>>>>> On 2014-03-20 20:09, Thomas Hartman wrote:
>>>>>>>>>> Sounds like a legitimate problem. Created issue in bugtracker,
>>>>>>> linked
>>>>>>>>>> back to this discussion.
>>>>>>>>>>
>>>>>>>>>> https://github.com/spesmilo/sx/issues/50 >>>>>>>>>>
>>>>>>>>>> On Thu, Mar 20, 2014 at 10:07 AM, Grazcoin <grazcoin@???>
>>>>>>> wrote:
>>>>>>>>>>> You can use virtualbox for testing ubuntu using console.
>>>>>>>>>>> I verified the procedure myself (at least until running the sx
>>>>>>> install
>>>>>>>>>>> script).
>>>>>>>>>>>
>>>>>>>>>>> Instructions:
>>>>>>>>>>> 1. download (less than 1GB)
>>>>>>>>>>>
>>>>>>> http://virtualboximages.com/Ubuntu+13.10+amd64+VirtualBox+VDI+Virtual+Appliance >>>>>>>>>>> 2. unrar e
>>>>>>>>>>>
>>>>>>> VirtualBox_-_Ubuntu-13.10-amd64-Desktop_VDI-[VirtualBoxImages.com].rar
>>>>>>>>>>> 3. start virtualbox (I assume it is installed).
>>>>>>>>>>> 4. New (name obelisk), Next.
>>>>>>>>>>> 5. RAM - I gave 4GB, Next
>>>>>>>>>>> 6. Use an existing virtual hard drive file - choose
>>>>>>>>>>> ubuntu-13.10-desktop-amd64.vdi
>>>>>>>>>>> 7. Start machine
>>>>>>>>>>> 8. sudo su (password is adminuser)
>>>>>>>>>>> 9. apt-get update
>>>>>>>>>>> 10. apt-get install git
>>>>>>>>>>> 11. git clone https://github.com/spesmilo/sx.git >>>>>>>>>>> 12. sh sx/install-sx.sh
>>>>>>>>>>>
>>>>>>>>>>> to stop and start networking
>>>>>>>>>>> service networking stop
>>>>>>>>>>> service networking start
>>>>>>>>>>>
>>>>>>>>>>> I hope it helps :)
>>>>>>>>>>> Grazcoin
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 2014-03-19 22:17, Amir Taaki wrote:
>>>>>>>>>>>> hey I'll take a look soon. I just accidentally shut off my
>>>>>>>>>>>> server
>>>>>>> today
>>>>>>>>>>>> when connected via ssh and tries to shut down the networking.
>>>>>>>>>>>> doh
>>>>>>>>>>>>
>>>>>>>>>>>> On 19/03/14 21:36, Thomas Hartman wrote:
>>>>>>>>>>>>> my guess would be that libbitcoin might be the component with
>>>>>>>>>>>>> lingering libs. so you might need to "nuke" that more
>>>>>>>>>>>>> thoroughly.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 19, 2014 at 1:35 PM, Thomas Hartman
>>>>>>>>>>>>> <thomas@???> wrote:
>>>>>>>>>>>>>> shot in the dark, but
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $ make clean
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and then run install-sx again?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sounds like some library component still might have old
>>>>>>>>>>>>>> code.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 19, 2014 at 11:12 AM, Grazcoin
>>>>>>>>>>>>>> <grazcoin@???>
>>>>>>> wrote:
>>>>>>>>>>>>>>> I was using the install-sx.sh script cloned today which
>>>>>>>>>>>>>>> updates
>>>>>>> anyway
>>>>>>>>>>>>>>> all sources, so I am in HEAD on all components (libbitcoin,
>>>>>>> libwallet,
>>>>>>>>>>>>>>> obelisk, sx, czmqpp, czmq, etc).
>>>>>>>>>>>>>>> If you give me some hints or debug instructions, I could
>>>>>>>>>>>>>>> try to
>>>>>>> fix it
>>>>>>>>>>>>>>> myself.
>>>>>>>>>>>>>>> I am working with ubuntu 13.10
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 2014-03-19 18:25, Amir Taaki wrote:
>>>>>>>>>>>>>>>> czmqpp-gitUpdate Obelisk and libczmq++
>>>>>>>>>>>>>>>> I fixed this a few days ago but should work now.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 19/03/14 17:22, Grazcoin wrote:
>>>>>>>>>>>>>>>>> I decided to give the latest obelisk a try on ubuntu with
>>>>>>> offline usage.
>>>>>>>>>>>>>>>>> It works already partially.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Starting from a normal setup with network - everything
>>>>>>>>>>>>>>>>> works
>>>>>>> fine.
>>>>>>>>>>>>>>>>> Turning off networking.
>>>>>>>>>>>>>>>>> # service networking stop
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Start worker
>>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ obworker
>>>>>>>>>>>>>>>>> Using config file: /etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>>>> DEBUG [worker]: Listening: tcp://*:9091
>>>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>>>> DEBUG [worker]: Heartbeat: tcp://*:9092
>>>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service: Bad
>>>>>>> stream
>>>>>>>>>>>>>>>>> Node started.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Then send a request
>>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>>>> 291345
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> great! it works!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Then a second request
>>>>>>>>>>>>>>>>> ubuntu@bitcoin:~/worker1$ sx fetch-last-height
>>>>>>>>>>>>>>>>> oh ...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I get:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> obworker: worker/worker.cpp:29: void
>>>>>>>>>>>>>>>>> obelisk::send_worker::queue_send(const
>>>>>>> obelisk::outgoing_message&):
>>>>>>>>>>>>>>>>> Assertion `socket.self()' failed.
>>>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I use the master HEAD.
>>>>>>>>>>>>>>>>> This behavior is repeatable.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Quick fix?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 2014-01-31 20:35, Grazcoin wrote:
>>>>>>>>>>>>>>>>>> I tried it on the latest ubuntu...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 2014-01-31 20:34, Thomas Hartman wrote:
>>>>>>>>>>>>>>>>>>> I don't know the answer to your question, but what
>>>>>>>>>>>>>>>>>>> distro
>>>>>>> are you running?
>>>>>>>>>>>>>>>>>>> It ran out of the box for me on debian 7.1 but not on
>>>>>>> latest ubuntu.
>>>>>>>>>>>>>>>>>>> On Jan 31, 2014, at 6:05 AM, Grazcoin wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I was trying to test the functioning of obelisk
>>>>>>>>>>>>>>>>>>>> offline:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1. started with a normal working online updated setup.
>>>>>>>>>>>>>>>>>>>> 2. stopped the obworker and obbalancer.
>>>>>>>>>>>>>>>>>>>> 3. updated /usr/local/etc/obelisk/worker.cfg with
>>>>>>> outgoing-connections = 0 and listener_enabled = false
>>>>>>>>>>>>>>>>>>>> 4. tried to start obworker, and got a crash. No extra
>>>>>>>>>>>>>>>>>>>> info
>>>>>>> on the error.log
>>>>>>>>>>>>>>>>>>>> What is the correct config to make this work?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> git is in sync.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [jan-31 14:57:29] dev@dev-laptop:~/worker1$ obworker
>>>>>>>>>>>>>>>>>>>> Using config file: /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>>>> Press CTRL-C to shut down.
>>>>>>>>>>>>>>>>>>>> DEBUG [worker]: Connecting: tcp://localhost:9092
>>>>>>>>>>>>>>>>>>>> INFO [worker]: worker ready
>>>>>>>>>>>>>>>>>>>> ERROR [protocol]: Failed to start handshake service:
>>>>>>>>>>>>>>>>>>>> Bad
>>>>>>> stream
>>>>>>>>>>>>>>>>>>>> ERROR: Unable to start session: Bad stream
>>>>>>>>>>>>>>>>>>>> terminate called without an active exception
>>>>>>>>>>>>>>>>>>>> Aborted (core dumped)
>>>>>>>>>>>>>>>>>>>> [jan-31 14:57:39] dev@dev-laptop:~/worker1$ cat
>>>>>>> /usr/local/etc/obelisk/worker.cfg
>>>>>>>>>>>>>>>>>>>> # Debug messages output file.
>>>>>>>>>>>>>>>>>>>> output-file = "debug.log"
>>>>>>>>>>>>>>>>>>>> # Error messages output file.
>>>>>>>>>>>>>>>>>>>> error-file = "error.log"
>>>>>>>>>>>>>>>>>>>> # Path to blockchain directory.
>>>>>>>>>>>>>>>>>>>> blockchain-path = "./blockchain/"
>>>>>>>>>>>>>>>>>>>> # List of hosts (loaded on startup, saved on exit).
>>>>>>>>>>>>>>>>>>>> # If non-existant or empty then Obelisk/libbitcoin
>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>> # fall back to trying seed nodes instead.
>>>>>>>>>>>>>>>>>>>> hosts-file = "hosts"
>>>>>>>>>>>>>>>>>>>> # Next 2 values ignored if publisher is disabled
>>>>>>>>>>>>>>>>>>>> publisher_enabled = false
>>>>>>>>>>>>>>>>>>>> block-publish = "tcp://*:9093"
>>>>>>>>>>>>>>>>>>>> tx-publish = "tcp://*:9094"
>>>>>>>>>>>>>>>>>>>> # Connection to the balancer.
>>>>>>>>>>>>>>>>>>>> # Which port to connect to the load balancer on.
>>>>>>>>>>>>>>>>>>>> service = "tcp://localhost:9092"
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> # Advanced features:
>>>>>>>>>>>>>>>>>>>> # Uncomment to give this worker a named UUID. Must be
>>>>>>> unique.
>>>>>>>>>>>>>>>>>>>> #name = "ada"
>>>>>>>>>>>>>>>>>>>> # Number of outgoing network connections to p2p
>>>>>>>>>>>>>>>>>>>> network.
>>>>>>>>>>>>>>>>>>>> outgoing-connections = 0
>>>>>>>>>>>>>>>>>>>> # Listening for incoming connections.
>>>>>>>>>>>>>>>>>>>> listener_enabled = false
>>>>>>>>>>>>>>>>>>>> # Add manual outgoing nodes below.
>>>>>>>>>>>>>>>>>>>> #nodes = (
>>>>>>>>>>>>>>>>>>>> # ("localhost", 8333)
>>>>>>>>>>>>>>>>>>>> #)
>>>>>>>>>>>>>>>>>>>> # Show Obelisk requests in the log file for debugging
>>>>>>>>>>>>>>>>>>>> or
>>>>>>> security.
>>>>>>>>>>>>>>>>>>>> # Slows down Obelisk speed. Disabled by default.
>>>>>>>>>>>>>>>>>>>> #log_requests = true
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>>>>
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>>
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>>
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>>>
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Libbitcoin mailing list
>>>>>>>>>>> Libbitcoin@???
>>>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>> _______________________________________________
>>>>>>>>> Libbitcoin mailing list
>>>>>>>>> Libbitcoin@???
>>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>>>>
>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Libbitcoin mailing list
>>>>>>>> Libbitcoin@???
>>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>> _______________________________________________
>>>>>>> Libbitcoin mailing list
>>>>>>> Libbitcoin@???
>>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>> _______________________________________________
>>>>>> Libbitcoin mailing list
>>>>>> Libbitcoin@???
>>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>>>>
>>>>> _______________________________________________
>>>>> Libbitcoin mailing list
>>>>> Libbitcoin@???
>>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>> _______________________________________________
>>>> Libbitcoin mailing list
>>>> Libbitcoin@???
>>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>
>>> _______________________________________________
>>> Libbitcoin mailing list
>>> Libbitcoin@???
>>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>>
>>
>>
>> _______________________________________________
>> Libbitcoin mailing list
>> Libbitcoin@???
>> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin >>
>
::
Re: [Libbitcoin] trying offline obe…
