On 2022-10-13 00:03:32, Boian Bonev wrote:
> Hi,
>
> On Wed, 2022-10-12 at 02:11 +1000, onefang wrote:
> > When xx.deb.devuan.org was first proposed on IRC, I was just getting
> > ready to go to bed, but stayed up an extra hour to help discuss
> > that.  So
> > I probably wasn't fully awake at the time.  lol
> >
> > Now I have just woken up, but at least I've had brekky.
> >
> > Also life has been getting in the way and I'm way behind on things I
> > need
> > to do in general.
> >
> > NOTE that all of this only applies to HTTP.  HTTPS can't be supported
> > by
> > DNS-RR coz the mirrors don't have the cert for deb.devuan.org, nor
> > should
> > they.  I don't know if FTP or rsync can support this sort of thing.
> > apt-panopticon doesn't test FTP nor rsync yet.
>
> Obviously both deb.devuan.org and CC.deb.devuan.org can not support
> https. Sharing the cert and the coordination involved with that is not
> worth the effort. xx.deb.devuan.org also does not support https. It
> points to an IPv4/IPv6 pair and there is a web server that redirects
> all requests to deb.devuan.org, regardless the hostname.
>
> In this way if somecode.deb.devuan.org becomes a CNAME of
> xx.deb.devuan.org, then all the requests will be redirected by
> xx.deb.devuan.org to deb.devuan.org. As a result the members of
> deb.devuan.org should not add support for all the country codes.

xx.deb.devuan.org seems a little over complicated, but I'll leave the
actual DNS shenanigans in the hands of those that are already doing it.

> > On 2022-10-11 09:35:03, B Stack wrote:
> > > ### bb|hcb
> > >
> > > #### Mirrors DNS round robin
> > > - `xx.deb.devuan.org` is now a redirector to `deb.devuan.org`. The
> > > point in adding that is pointing all country codes without a mirror
> > > serving them to `xx.deb.devuan.org` instead of
> > > `pkgmaster.devuan.org`.
> >
> > Does this mean that all mirrors on the existing DNS-RR now have to
> > support xx.deb.devuan.org AS WELL AS the deb.devuan.org they support
> > now?
> > Or is it setup so that they can stick with their existing
> > deb.devuan.org
> > support?
>
> See above, nothing to be changed.

OK, that I and the other mirror admins can live with. B-)

> > Currently when a user does apt requests from deb.devuan.org, the DNS
> > system gives them some random IP from our DNS-RR, and apt then sends
> > that
> > IP a request asking for the deb.devuan.org domain, and that mirrors
> > server has to be configured to respond to deb.devuan.org requests
> > with
> > the usual Devuan package mirror responses.
> >
> > If this new xx.deb.devuan.org thing results in requests for
> > deb.devuan.org being sent to the mirrors, then the mirrors don't have
> > to
> > change anything.  Which is my preference.  Much simpler from my /
> > mirror
> > admins point of view.
> >
> > Naturally any mirror that volunteers to support a CC.deb.devuan.org
> > should configure their mirror accordingly.  Though having them
> > configured
> > to support a wildcard .deb.devuan.org covers all the things.
>
> Preferably, but maybe not easily doable on all the mirrors. We even do
> not know the details.
>
> > > - Is it time to make the `CC.deb.devuan.org` official and announce
> > > it?
> >
> > I suspect yes, though see below.
> >
> > > As far as I am aware the only pending thing for this is to make
> > > `apt-panopticon` to automatically change the mirror list.
> >
> > Ah apt-panopticon is not yet actually testing CC.deb.devuan.org, but
> > it
> > should.  Might be best to have those tests in place before we
> > announce
> > "CC.deb.devuan.org is official", but on the other hand announcing
> > "CC.deb.devuan.org is coming back once we iron out the kinks, feel
> > free
> > to help us test it" probably should be announced now.
>
> Yes, it will be good to have all mirrors that announce to support a
> list of country codes, if they really do. Doing all the tests does not
> seem needed, because that was already done on the mirror, the only
> thing is to verify if the announced CCs are properly served.

I did go out of my way to keep the tests bandwidth usage down to a
minimum, like picking the smallest changed package for testing. Though I
do some tests on all the IPs and host names and such.

> > Back in the day there was this TODO item to have apt-panopticon
> > driving
> > DNS-RR.  https://sledjhamr.org/mantisbt/view.php?id=142 where you can
> > see
> > that I have been really slack about doing that.  lol
>
> That full automation would be nice. :)

It's coming. Sooner or later. This is why I wrote this lengthy email,
to sort out the details.

> > To summarize, three separate instances of apt-panopticon could vote
> > on
> > who should be in the DNS-RR for the next ten minutes (until they all
> > run
> > their tests again).  Then they somehow cryptographically let the DNS-
> > RR
> > know about the results of that vote so that it can update
> > deb.devuan.org
> > to point to the current good ones.
> >
> > Alas we only have two running every ten minutes, since rrq wanted his
> > borta instance to only run once an hour.  So sledjhamr and veritas
> > run
> > every ten minutes.  Each run on all but borta takes about a minute.
> >
> > https://borta.devuan.dev/apt-panopticon/results/Report-web.html
> > https://sledjhamr.org/apt-panopticon/results/Report-web.html
> > http://veritas.devuan.org/apt-panopticon/results/Report-web.html
> >
> > Borta is in Australia, sledjhamr is in Netherlands, and I think
> > veritas
> > is in France.  Would be nice to have one in the Americas somewhere as
> > well.  All three apt-panopticon instances tend to be in general
> > agreement
> > most of the time.
>
> Isn't that complicating it too much? Just using the results from
> veritas should be fine. But if you decide to implement the voting and
> stuff, I can help to run another instance.

Yes, too complicated which is why I've been thinking about "just
apt-panopticon on pkgmaster which already contrals the DNS-RR".

> > Currently there is a master file on pkgmaster with the details of
> > every
> > package mirror.  A cron job runs every hour to update the DNS-RR
> > based on
> > that, and also updates https://pkgmaster.devuan.org/mirror_list.txt 
> > I
> > keep an eye on the apt-panopticons and manually edit that master
> > file.
> > It's very rare that I need to make changes, our package mirrors tend
> > to
> > be good most of the time.  Note that this cron job is also updating
> > the
> > CC.deb.devuan.org DNS-RR.  In the apt-panopticon main results table
> > you
> > can see a red question mark instead of a green tick in the "DNS round
> > robin" column for any DNS-RR mirror that I have deemed "not
> > acceptable
> > for now".
> >
> > Also more code would need to be added to figure out "all country
> > codes
> > without a mirror" so we can point them at xx.deb.devuan.org.  The
> > existing cron job that parses the master file has enough info, or I
> > could
> > add it to apt-panopticon's DNS-RR deliberations.
>
> This is already implemented by wildcards, *.rr.deb.devuan.org points to
> xx.deb.devuan.org. All country codes with mirrors behind them will
> override that while all the rest will point to xx, which in turn will
> redirect to deb.devuan.org.
>
> > I have been thinking for a while that a better option than the
> > complex
> > cryptograph voting system would be to simply run apt-panopticon on
> > pkgmaster.  Though obviously that would increase the bandwidth load a
> > bit.  rrq should be able to give us some details on what the load on
> > borta and veritas is, I can't separate out the sledjhamr load from
> > all
> > the other things it does.  Which means the pkgmaster apt-panopticon
> > can
> > directly change that master file, and the cron job can update DNS-RR
> > as
> > usual.  On the other hand, if we manage to move some of the bandwidth
> > load off pkgmaster with these shenanigans, then it could probably
> > afford
> > this extra apt-panopticon load.
> >
> > BTW the sledjhamr instance uses less bandwidth than the others, coz
> > it's
> > also one of the mirrors that it checks, so doesn't use any bandwidth
> > checking itself.  Which is also why the speed of sledjhamr as
> > measured on
> > the sledjhamr apt-panopticon is incredibly fast.  Local SSD beats
> > Internet across the globe hands down.  B-)
> >
> > So we could have pkgmaster run apt-panopticon from the cron job that
> > updates DNS-RR, and that apt-panopticon updates the master file, so
> > that
> > later in the cron job the master file is used to change the DNS-RR. 
> > Then
> > add CC's into that mix when I get a round tuit.  Not sure if we
> > should
> > have the cron job run every ten minutes as originally planned, or
> > once an
> > hour like it does now, or split the difference.
> >
> > I had already designed apt-panopticon to have output modules, and it
> > already has a few.  Adding one more isn't hard.
> >
> > It should be noted that the first thing apt-panopticon does is to
> > check
> > pkgmaster for changes, which wont use any extra bandwidth running on
> > pkgmaster.  Apt-panopticon is built to keep it's bandwidth needs low,
> > mostly by picking out the smallest test packages to download from the
> > mirrors, and only doing some downloads when things change.
> >
> > apt-panopticon is written in Lua / LuaJIT, which is not currently
> > installed on pkgmaster.  May need some rrd packages as well.
>
> I can't say which way is better. Most probably it would be best you to
> decide on some way and if it needs changing then it can be considered
> again?

I'm thinking I'll go ahead with the pkgmaster apt-panopticon, coz no
messing about with writing voting and crypto code.

Though there is the issue of apt-panopticon keeping historical data for
the graphs and such. I do have to log onto borta and veritas every now
and then to remove the oldest histories, and pkgmaster doesn't have much
free disk space. So I'll add an option to only keep X days of historical
data.

> If you need help with installing stuff, let me know :)

I am an experienced professional Linux sysadmin, and I have sudo on
pkgmaster, so no need for help. B-)

> With best regards,
> b.

--
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.