:: Re: [devuan-mirrors] [DNG] Artifici…
Top Page
Delete this message
Reply to this message
Author: onefang
Date:  
To: devuan-mirrors
Subject: Re: [devuan-mirrors] [DNG] Artificial Idiocy attacking mirrors.
Look for lots of 404 errors resulting from AI bots trying to download
lots of things that are not Devuan from deb.devuan.org. An example from
Apache is -

216.73.216.33 - - [01/Feb/2026:13:01:01 +0100] "GET /pub/mirrors/opensuse/update/leap/15.3/sle/s390x/nftables-0.9.8-150300.3.3.1.s390x.rpm HTTP/1.1" 404 496 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@???)"

ClaudeBot trying to get an RPM package from my non existant OpenSUSE
mirror, in my non existant /pub/ directory. I have seen it and others
trying to get all manner of packages from all manner of non Devuan
package repos. And trying the same files up to 7000 times per day.

216.73.216.33 and others are now banned at my firewall. Seems to have
helped, so far.

On 2026-02-03 07:07:37, ariel sabiguero yawelak wrote:
> What should we look for? I don't see differences in the traffic patterns or
> number of hits. Anything that I can search for?
>
> We host many other mirrors, but "deb.devuan.org" only offers devuan content,
> as only:
>
>     <Directory "/zp0/mirrors/devuan/devuan">
>
> is associated to that vhost, and other mirrors are siblings
> of /zp0/mirrors/devuan and are served under espejito.fder.edu.uy/XXXXX
>
>
> regards
>
>
> ariel
>
> El 3/2/26 a las 1:10, Derek escribió:
> > I don't see any activity that looks like this on my mirror. I don't have anything non-Devuan on my mirror either. All I see is regular internet background noise.
> >
> > Regards,
> > Derek
> >
> >
> > On Monday, February 2nd, 2026 at 8:01 PM, onefang <onefang_devuan@???> wrote:
> >
> > >
> > > I have two questions for all of us package mirror admins.
> > >
> > > Have you also been seeing these AI bots scanning your deb.devuan.org
> > > Devuan mirror for lots and lots of non Devuan packages?
> > >
> > > Does your deb.devuan.org mirror have non Devuan things available at that
> > > address? Please don't do that, I think it's causing these attacks.
> > > Devuan mirrors should all be showing the exact same thing at
> > > deb.devuan.org, that's what they are for.
> > >
> > > On 2026-01-31 14:25:01, onefang wrote:
> > >
> > > > On 2025-11-27 15:02:35, onefang wrote:
> > > >
> > > > > ClaudeBot is currently doing something really stupid. It somehow has
> > > > > come to the conclusion that my Devuan mirror mirrors EVERYTHING!
> > > > >
> > > > > So it's probing my mirror looking for things like rpms, jars, npn, cpan
> > > > > packages, etc, and specifically looking in directories named after other
> > > > > operating systems or package collections. Even stuff I have never heard
> > > > > of. I certainly don't mirror any of it, only Devuan.
> > > > >
> > > > > And it's doing it stupidly. It obviously knows what to expect, since it
> > > > > keeps asking me for the actual packages and metadata files that would be
> > > > > there if I was mirroring that particular thing. For example failing to
> > > > > download several rpm files from a directory that is named after some rpm
> > > > > based distro. After the first failure, it should figure out I'm not
> > > > > mirroring that, give up. On top of that, these are mirrors it is
> > > > > probing, the word "mirror" is even in some of the paths, they should all
> > > > > be identical. If it already knows what to expect, it doesn't need to
> > > > > download it yet again.
> > > > >
> > > > > This may be going on with other mirrors, though some of them do mirror
> > > > > other things as well. All it's doing is wasting bandwidth and space for
> > > > > log files. Slowing everything down. And clogging up the AI with lots of
> > > > > duplicates of binary files.
> > > > I have a theory.
> > > >
> > > > I'm only seeing this nonsense for deb.devuan.org, not sledjhamr.org.
> > > >
> > > > AI bots are scanning deb.devuan.org, and the country code versions, no
> > > > idea how they handle a DNS-RR. One or more of our package mirrors is
> > > > also hosting mirrors of Fedora, SUSE, and the dozen other mirrors my
> > > > server doesn't host that the AI bots are scanning for. These other
> > > > Devuan mirrors have those non Devuan things available from deb.devuan.org
> > > > I guess, which is how the bots figured all that non Devuan stuff is
> > > > at deb.devuan.org
> > > >
> > > > So the AIs just think that all of us Devuan packge mirrors have all of
> > > > the things that the other package mirrors have. And end up scanning my
> > > > server for non existent directories thousands of times per day, looking
> > > > for identical files. That's how the bots know what to scan for, they
> > > > found it at deb.devuan.org before.
> > > >
> > > > I wonder if other Devuan mirrors are seeing this? We should block off
> > > > Fedora / SUSE / whatever mirrors and things from deb.devuan.org. Might
> > > > still take ages for the AI bots to forget to look for these things.