Hi Tito, thanks for following up on this.

The main problem is that I'm trying to emulate the upgrade of a number
of interconnected boxes as a proof-of-concept. So I can't change
anything on the originals.

I've used the trick of copying the entire mysql directory a number of
times, and it works well in some situations. But it can't be used in
the current one. Basically, I'm trying to migrate a mysql 5.6 server to
the azure mysql server offering: the lowest version of mysql available
on the latter is 5.7, and it won't allow the import/upgrade of a 5.6
database. It also doesn't allow replication from mysql 5.6 - even
though mysql say that replication from 5.6 -> 5.7 is OK. To complicate
things further, the mysql 5.6 server has to be up 24/7. 

At the moment, the 5.6 server has a slave server, also running 5.6, fed
using mysql replication in real-time. The option I'm looking at is to
upgrade the slave to mysql 5.7, check that replication still works,
dump out the 5.7 database on the slave (using mysqldump with the "dump-
slave=2" option which includes the binary log co-ordinates as a comment
in the dump) and then import it into the microsoft mysql box, setting
it up as a downstream slave. Given the potential pitfalls of this, I
want to try simulating it using local kit and hopefully hit and solve
at least some of the problems, before trying it for real. (Upgrading
mysql 5.6 to 5.7 is a nightmare: there are a number of 5.6 config
variables which if 5.7 sees them, it doesn't just ignore them - it
refuses to start! It just seems as though the 5.7 team had never heard
of the concept of backwards compatibility.)

I've been using mysqldump and importing dumps from it for years, and
never had a problem with it. When I first started having the problems
on the virtualbox VMs, I did try using the hex-blob option as an
experiment, but it didn't make a difference - the import still failed.
There just shouldn't be any binary zeros in the mysqldump output. The
import of the dump from the original server, on the ubuntu 14 physical
box which I've just set up, succeeded without a hitch. It's just got to
be something in virtualbox which is the problem, and it would be good
to know what (I use virtualbox for a lot of things - some quite
critical!).

On Tue, 2022-10-18 at 14:14 +0200, tito via Dng wrote:
> On Tue, 18 Oct 2022 11:52:42 +0100
> Peter Duffy <peter@???> wrote:
>
> > Sorry, this isn't specific to devuan (although it's involved
> > indirectly) - I'm really just looking for thoughts and comments on
> > a
> > problem which has been driving me insane for the last few weeks.
> >
> > I'm trying to replicate a setup of several fairly old linux/mysql
> > systems, as part of a project to upgrade them to devuan. I was
> > trying
> > to use virtualbox VMs to do the replication (virtualbox 6.1.32).
> > First
> > VM was ubuntu 14 and percona mysql 5.6.37; second VM was devuan
> > chimaera and percona mysql 5.6.37. Each VM had 32g RAM, 1 CPU core
> > and
> > 500g disk. I had a copy of the database from one of the original
> > systems (gzipped output from mysqldump) and the first step was to
> > load
> > it into the first VM. That was when the problems started.
>
> Hi,
> try this ugly and dirty hack, copy to the vm the full mysql
> data folder from the original server through a shared folder,
> rather than doing the mysqldump dance.
> Also be aware that mysqldump has a lot of command
> line options that could be incompatible between different
> versions. You need to restore the dump with the same
> version of mysql/mysqldump that created it to be sure
> it will work, later you can upgrade mysql, hoping that
> the upgrade scripts will take care of your db.
> Also take a look at the hex-blob option of mysqldump.
>
> Hope this helps
>
> Ciao,
> Tito
>  
>
> > Loading the dump just consisted of "gunzip -c <file> | mysql
> > <credentials and options>" - I've done it successfully on many
> > systems
> > before now. This time, the mysql client kept choking, with
> > complaints
> > that the input contained binary zeroes. That's not supposed to
> > happen -
> > mysqldump is supposed to output a file consisting of valid SQL
> > statements and comments, designed to be poured straight back into
> > mysql
> > (I've never had a problem with it before now). I tried gunzipping
> > the
> > file on the VM, and on a separate physical box with the same
> > version of
> > gunzip, then comparing the md5sums of the files: original - md5sum
> > matches; unzipped file - md5sums didn't match. I then wrote a
> > little
> > prog to scan for binary zeros, and ran it on both files: not there
> > in
> > the one done on the physical box; there in the one done on the VM.
> >
> > After tearing my hair over this for over a week and getting
> > nowhere, I
> > decided to ditch the idea of using virtualbox VMs, dug out some old
> > retired boxes, set them up to match the VMs, and tried the load on
> > the
> > first one again. This time, no problems (apart from the fact that
> > the
> > load takes forever).
> >
> > At the moment, what seems to be inescapable is that gunzipping
> > files in
> > linux on virtualbox VMs can in some cases cause file corruption.
> > Given
> > that the identical procedure works on a physical box, it seems to
> > exclude gzip itself, the shell (bash), the file system, and the
> > kernel.
> > The only thing which seems to be left is virtualbox itself. 
> >
> > Has anyone else ever experienced anything else even remotely
> > similar?
> > I've not been able to find anything relevant on google. Also - has
> > anyone any preferences for virtualisation under linux: what seems
> > to be
> > the most reliable from the range of available tools to do it?
> >
> > Thoughts would be most welcome!
> >
> >
> > _______________________________________________
> > Dng mailing list
> > Dng@???
> > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
>
> _______________________________________________
> Dng mailing list
> Dng@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng