http://deb.devuan.org/index.html now exists, please serve that file and
not some bespoke custom version, or an autogenerated index, when you
serve deb.devuan.org and it's CC friends.

Thanks.

On 2026-02-10 04:10:49, onefang wrote:
> What people do with their own domains is their own business, but
> deb.devuan.org is the Devuan teams business, it's our domain. The Devuan
> team decides what goes on http://deb.devuan.org/ The same applies to the
> CC.deb.devuan.org domains.
>
> So all Devuan package mirrors should be serving the exact same thing when
> responding to http://deb.devuan.org/ and the country code versions.
>
> We have https://pkginfo.devuan.org/ for searching for packages, so we
> don't really need the search bots scanning us. The AI bots found non
> Devuan things on some of our mirrors at http://deb.devuan.org/, hence why
> they keep scanning http://deb.devuan.org/ for non Devuan package repos.
>
> So we decided to just stop the bots from scanning http://deb.devuan.org/
> which is why there is now a http://deb.devuan.org/robots.txt that tells
> all the bots that bother to obey it not to scan http://deb.devuan.org/
>
> For those bots that ignore robots.txt there will soon be a
> http://deb.devuan.org/index.html file. This will include links to the
> package repo stuff and no other links. "http://deb.devuan.org/" should
> serve this index.html.
>
> In this way the behaving bots just wont scan anything, and the
> misbehaving bots will only find Devuan package repo, nothing else, when
> they scan http://deb.devuan.org/
>
> Hopefully within a week the bots will have forgotten the non Devuan stuff
> they have been scanning http://deb.devuan.org/ for, and they wont be
> bogging down our mirrors and filling our logs with 404s. I'm certainly
> tired of the same non existent files being scanned for 7000 times a day.
>
> This should also help with the bots scanning Devuan package mirrors for
> non existant packages, that are not at deb.devuan.org, but that can still
> leak a bit.
>
> Cross our fingers, hope this helps.