:: Re: [devuan-mirrors] [DNG] Artifici…
Top Page
Delete this message
Reply to this message
Author: ariel sabiguero yawelak
Date:  
To: devuan-mirrors
Subject: Re: [devuan-mirrors] [DNG] Artificial Idiocy attacking mirrors.
Getting traffic right now:

    74.7.243.197 - - [03/Feb/2026:07:27:59 -0300] "GET
    /merged/dists/testing/contrib/debian-installer/binary-amd64/by-hash/SHA256/59a3bf90cab125fef4afb3197d3b56c76526fb7b2bb0983d73aec538d2b0e5f6
    HTTP/1.1" 200 391
    35.233.94.99 - - [03/Feb/2026:07:27:59 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-armel/by-hash/SHA256/
    HTTP/1.1" 200 25139
    74.7.227.48 - - [03/Feb/2026:07:27:59 -0300] "GET
    /merged/dists/testing-updates/non-free/binary-i386/by-hash/SHA256/6f727602a45ef2f4108cfc495ec03d6108685a372daaabe7f1fa5306c287cadd
    HTTP/1.1" 200 29
    35.233.94.99 - - [03/Feb/2026:07:27:59 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-armhf/ HTTP/1.1"
    200 1072
    35.233.94.99 - - [03/Feb/2026:07:28:00 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-armhf/by-hash/
    HTTP/1.1" 200 867
    74.7.227.48 - - [03/Feb/2026:07:28:00 -0300] "GET
    /merged/dists/testing-updates/non-free/binary-i386/by-hash/SHA256/cc6ffc1f78d18c3ab1e65a444f255f8be8c8c57aad553a2d452d84da592802b8
    HTTP/1.1" 200 29
    74.7.243.197 - - [03/Feb/2026:07:27:59 -0300] "GET
    /merged/dists/testing/contrib/debian-installer/binary-amd64/by-hash/SHA256/4c486672026369b59f2ab075e96fa30608a5d96fa065e5dfacfa6efdded3f96f
    HTTP/1.1" 200 391
    35.233.94.99 - - [03/Feb/2026:07:28:00 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-armhf/by-hash/SHA256/
    HTTP/1.1" 200 26411
    74.7.227.48 - - [03/Feb/2026:07:28:00 -0300] "GET
    /merged/dists/testing-updates/non-free/binary-i386/by-hash/SHA256/fe8d8d9de30464df58743aab7b2a6df0a1c05aa69676cb764eaf34f77e1c7545
    HTTP/1.1" 200 29
    35.233.94.99 - - [03/Feb/2026:07:28:00 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-i386/ HTTP/1.1"
    200 1070
    35.233.94.99 - - [03/Feb/2026:07:28:01 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-i386/by-hash/
    HTTP/1.1" 200 861
    74.7.243.197 - - [03/Feb/2026:07:28:00 -0300] "GET
    /merged/dists/testing/contrib/debian-installer/binary-amd64/by-hash/SHA256/3ee01ec5971930412f77d342c09e7cdb8fe6abd09049d1e0b7e90fdef3acbd73
    HTTP/1.1" 200 391
    35.233.94.99 - - [03/Feb/2026:07:28:01 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-i386/by-hash/SHA256/
    HTTP/1.1" 200 26408
    74.7.227.48 - - [03/Feb/2026:07:28:01 -0300] "GET
    /merged/dists/testing-updates/non-free/binary-i386/by-hash/SHA256/19fec0cc35be2ce2dea8b6b2b91a195005e7e73de5746d8c1e8e1919a0010ad0
    HTTP/1.1" 200 29
    74.7.243.197 - - [03/Feb/2026:07:28:01 -0300] "GET
    /devuan/dists/experimental/main/debian-installer/binary-amd64/by-hash/SHA256/25231f17952cac512b323279ba12d7e1949fc1c869e6e7a20348533ca9e5275c
    HTTP/1.1" 200 521
    35.233.94.99 - - [03/Feb/2026:07:28:01 -0300] "GET
    /devuan/dists/stable/main/debian-installer/binary-ppc64el/ HTTP/1.1"
    200 1076



but if i grep -i  "claudebot" I don't get a single entry.

If you check what the bot is searching for, it seems a server exposing
the root of the www server directory. In our case, if there was some
misconfiguration, it should start with /zp0/mirrors/..... and not /pub
(maybe it is the pub inside /srv/www folder).

Seems that our end is working, I went for
http://uy.deb.devuan.org/tralala and it is

    espejito:/var/log/apache2 # cat deb.devuan.org_access_log | grep tralala
    190.133.48.56 - - [03/Feb/2026:07:34:45 -0300] "GET /tralala
    HTTP/1.1" 404 1079



correctly failing


regards


ariel

El 3/2/26 a las 7:21, onefang escribió:
> Look for lots of 404 errors resulting from AI bots trying to download
> lots of things that are not Devuan from deb.devuan.org. An example from
> Apache is -
>
> 216.73.216.33 - - [01/Feb/2026:13:01:01 +0100] "GET /pub/mirrors/opensuse/update/leap/15.3/sle/s390x/nftables-0.9.8-150300.3.3.1.s390x.rpm HTTP/1.1" 404 496 "-""Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible;
> ClaudeBot/1.0; +claudebot@???)"
>
> ClaudeBot trying to get an RPM package from my non existant OpenSUSE
> mirror, in my non existant /pub/ directory. I have seen it and others
> trying to get all manner of packages from all manner of non Devuan
> package repos. And trying the same files up to 7000 times per day.
>
> 216.73.216.33 and others are now banned at my firewall. Seems to have
> helped, so far.
>
> On 2026-02-03 07:07:37, ariel sabiguero yawelak wrote:
>> What should we look for? I don't see differences in the traffic patterns or
>> number of hits. Anything that I can search for?
>>
>> We host many other mirrors, but "deb.devuan.org" only offers devuan content,
>> as only:
>>
>>     <Directory "/zp0/mirrors/devuan/devuan">
>>
>> is associated to that vhost, and other mirrors are siblings
>> of /zp0/mirrors/devuan and are served under espejito.fder.edu.uy/XXXXX
>>
>>
>> regards
>>
>>
>> ariel
>>
>> El 3/2/26 a las 1:10, Derek escribió:
>>> I don't see any activity that looks like this on my mirror. I don't have anything non-Devuan on my mirror either. All I see is regular internet background noise.
>>>
>>> Regards,
>>> Derek
>>>
>>>
>>> On Monday, February 2nd, 2026 at 8:01 PM, onefang<onefang_devuan@???> wrote:
>>>
>>>> I have two questions for all of us package mirror admins.
>>>>
>>>> Have you also been seeing these AI bots scanning your deb.devuan.org
>>>> Devuan mirror for lots and lots of non Devuan packages?
>>>>
>>>> Does your deb.devuan.org mirror have non Devuan things available at that
>>>> address? Please don't do that, I think it's causing these attacks.
>>>> Devuan mirrors should all be showing the exact same thing at
>>>> deb.devuan.org, that's what they are for.
>>>>
>>>> On 2026-01-31 14:25:01, onefang wrote:
>>>>
>>>>> On 2025-11-27 15:02:35, onefang wrote:
>>>>>
>>>>>> ClaudeBot is currently doing something really stupid. It somehow has
>>>>>> come to the conclusion that my Devuan mirror mirrors EVERYTHING!
>>>>>>
>>>>>> So it's probing my mirror looking for things like rpms, jars, npn, cpan
>>>>>> packages, etc, and specifically looking in directories named after other
>>>>>> operating systems or package collections. Even stuff I have never heard
>>>>>> of. I certainly don't mirror any of it, only Devuan.
>>>>>>
>>>>>> And it's doing it stupidly. It obviously knows what to expect, since it
>>>>>> keeps asking me for the actual packages and metadata files that would be
>>>>>> there if I was mirroring that particular thing. For example failing to
>>>>>> download several rpm files from a directory that is named after some rpm
>>>>>> based distro. After the first failure, it should figure out I'm not
>>>>>> mirroring that, give up. On top of that, these are mirrors it is
>>>>>> probing, the word "mirror" is even in some of the paths, they should all
>>>>>> be identical. If it already knows what to expect, it doesn't need to
>>>>>> download it yet again.
>>>>>>
>>>>>> This may be going on with other mirrors, though some of them do mirror
>>>>>> other things as well. All it's doing is wasting bandwidth and space for
>>>>>> log files. Slowing everything down. And clogging up the AI with lots of
>>>>>> duplicates of binary files.
>>>>> I have a theory.
>>>>>
>>>>> I'm only seeing this nonsense for deb.devuan.org, not sledjhamr.org.
>>>>>
>>>>> AI bots are scanning deb.devuan.org, and the country code versions, no
>>>>> idea how they handle a DNS-RR. One or more of our package mirrors is
>>>>> also hosting mirrors of Fedora, SUSE, and the dozen other mirrors my
>>>>> server doesn't host that the AI bots are scanning for. These other
>>>>> Devuan mirrors have those non Devuan things available from deb.devuan.org
>>>>> I guess, which is how the bots figured all that non Devuan stuff is
>>>>> at deb.devuan.org
>>>>>
>>>>> So the AIs just think that all of us Devuan packge mirrors have all of
>>>>> the things that the other package mirrors have. And end up scanning my
>>>>> server for non existent directories thousands of times per day, looking
>>>>> for identical files. That's how the bots know what to scan for, they
>>>>> found it at deb.devuan.org before.
>>>>>
>>>>> I wonder if other Devuan mirrors are seeing this? We should block off
>>>>> Fedora / SUSE / whatever mirrors and things from deb.devuan.org. Might
>>>>> still take ages for the AI bots to forget to look for these things.
> _______________________________________________
> devuan-mirrors mailing list
> devuan-mirrors@???
> Manage your subscription:https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/devuan-mirrors
> Archive:https://lists.dyne.org/lurker/list/devuan-mirrors.en.html