On Thu, 27 Nov 2025 12:05:28 +0000
g4sra <g4sra@???> wrote:
> > ClaudeBot is currently doing something really stupid. It somehow has
> > come to the conclusion that my Devuan mirror mirrors EVERYTHING!
>
> Welcome to the AI future.
> anthropic.com have been having bad press since last year and don't
> appear to care how much upset they cause.
>
> Using Agent Headers or robots.txt is pointless as the aggressor cannot
> be trusted to be honest. I am currently looking for some form of
> proactive\reactive dynamic firewall blocking for this type of 'attack'.
> I am also investigating if I can tarpit (without causing myself
> significant overhead) to actually slow down probes onto my network by
> causing heightened resource usage at the aggressor's end.
>
> If you know of anything that may help please give me a head-up.

The easiest thing, and I have seen other OSS admins do this before AI
came along, is to make some of these non-existant paths trigger a
blacklist for that IP. Then no matter what it requests the AI just gets a
response like, "You've been blacklisted. If you think you've been blocked
in error please reach out to ..."

As for why 404 is not respected, some hosts report this when the file
actually exists. So you can't trust 404 on some hosts. The web is broken
and AI is just better exposing it.

I hope it helps,
David