Hey,

thank you again for the email, I commented some on IRC a few days
ago and updated the list as necessary in some points.

On dv., set. 06 2019, onefang wrote:

> I've been developing my mirror checker script for some time now,
> and I
> spent most of the last day testing it and fixing up things /
> adding
> better checks. This is the second week this weekly update has
> given
> lots of errors. I'll make some comparisons with these results.
>
> For the few mirrors I don't mention here, my script agrees with
> this
> email.
>
> My script doesn't yet support Updated and Integrity tests. It
> also
> doesn't yet produce this email, nor the similar web page. All
> of these
> are TODO items.

TBH, I'd be less interested in an email that says "this is what
works and what doesn't *right now, this time on Friday*" than on
"this week, these were the downtimes / times out of sync of the
mirrors".

There could be some issues on the current checker, but also it can
be a matter of timing and that's why some mirrors appear as
"faulty" on this picture that is sent every week.


> On Fri, 6 Sep 2019 16:37:01 +0100 Devuan Mirrors said :
>

>> mirror.koddos.net/devuan/devuan....
>>       http: [OK] https: [OK] DNS-RR: [OK] Updated: [SKIP] 
>>       Integrity:
>> [OK]

>
> The official mirror list says "only usable via deb.devuan.org",
> so my
> mirror checker doesn't bother to check it directly. It does
> check it
> via the deb.devuan.org DNS-RR. The HTTP tests and DNS-RR tests
> passed.
> My script doesn't do HTTPS checks on mirrors accessed via the
> deb.devuan.org DNS-RR, as the mirrors should in theory NOT have
> the
> deb.devuan.org certificate. During testing I noticed that some
> of them
> "work" anyway.
>
> In not so sure what to do about this, flag them as WARNING since
> they
> should not do that? Check further to see if there is some sort
> of
> valid cert that somehow passes my checks?

Probably something fishy in your checker script, I'd adventure to
suggest it only checks that a TLS connection is offered and the
right content is served, and not the validity / trust chain of the
cert.

This mirror however does not serve the right content (even with
the wrong cert) over HTTPS, maybe some other server does, it
wouldn't be crazy.

But this particular one is indeed misconfigured over IPv6, IIRC I
mentioned that exclusively checking based on a hostname is
dangerous because these situations are awfully common (or
redirections to other hosts, ...):

# host mirror.koddos.net
mirror.koddos.net has address 31.220.0.151
mirror.koddos.net has IPv6 address 2001:590:3803::31:151

# openssl s_client -servername deb.devuan.org -connect
31.220.0.151:443
Shows: subject=OU = Domain Control Validated, CN = *.koddos.net

$ curl -Ik -H 'Host: deb.devuan.org'
https://31.220.0.151/mirror_list.txt
HTTP/1.1 404 Not Found

# openssl s_client -servername deb.devuan.org -connect
[2001:590:3803::31:151]:443
Times out, as does:
# curl -6 http://mirror.koddos.net/devuan/devuan

>> devuan.ipacct.com/devuan....
>>       http: [OK] https: [OK] DNS-RR: [OK] Updated: [SKIP] 
>>       Integrity:
>> [OK]

>
> The official mirror list says this mirror doesn't support HTTPS,
> so I
> don't test that. It's also not on the DNS-RR.


For the class of issues "HTTPS supported but not listed":
ATM the list is maintained manually, so if changes are not
notified, they are not reflected. The list was updated with this
class of issues unless I missed some.

For the class of issues "RR is supported but it hasn't been
added": maybe we should fix that :-). I'd feel much more confident
adding things if we had that email with a % of proper responses /
uptime over the week.

>> ==== faulty mirrors: ====
>>
>> pkgmaster.devuan.org (https)
>
> All tests passed, including HTTPS.

Given this is the reference server, it was possibly a network
hiccup. As mentioned, the email is based on taking a picture of
current state and therefore these things can happen.

>> mirror.4isp.it (DNS-RR)
>
> The official mirror list says "temporarily-offline", so my
> mirror
> checker doesn't bother to check it. I should probably add an
> option to
> check it anyway, so the admin of this mirror knows what works
> and what
> doesn't.

It was removed of the RR because it was being problematic; we
never got an answer so I just changed the state to "offline".


>> dist-mirror.fem.tu-ilmenau.de/devuan (http)
>> dist-mirror.fem.tu-ilmenau.de/devuan (https)
>
> This mirror has some issues due to it automatically redirecting
> HTTP to
> HTTPS. I spent most of the day trying to sort this out, not
> sure
> exactly where the problem was. Likely it shouldn't be doing
> that, since
> not everyone has apt-transport-https installed. If you don't
> have that
> installed and you use this mirror, things will probably break.
> It
> should work fine if you actually use HTTPS with
> apt-transport-https.
> I'd give this one a WARNING instead of an ERROR, but it's
> debatable
> that maybe this should be flagged as an ERROR.

apt-transport-https comes by default on buster/beowulf, so I'd
expect this to be more common over time (also given the history of
apt issues that have proven that there is some value in HTTPS for
these things as well).

Redirection to HTTPS is only a *problem* if the used hostname is
deb.devuan.org, otherwise the user is free to pick a different
mirror that does plain HTTP.

>> devuan.bio.lmu.de (DNS-RR)
>
> The official mirror list says "Active: not yet", so my mirror
> checker doesn't bother to check it.

That should be 'Active: yes'. It is part of the RR. Fixed.


> I'll keep working on my script. It may replace the script that
> generates this email, and the similar web page one day.

Thank you for this, looking forward to seeing how it evolves, I
hope these changes / comments were useful.
--
Evilham