> LWN recently had a writeup on this problem: https://lwn.net/Articles/1008897/
> Many sites started using https://anubis.techaro.lol/ to repent the bots.

Thanks for the link Tomasz.
I was aware of Anubis but had not realised it was open source.

Their document page made me giggle, must have written that before last week and not updated it.

"In most cases, you should not need this and can probably get by using Cloudflare".