ai.robots.txt for haproxy
=========================

Bots are reported to now make up half of the web's traffic. There are several
ways of blocking these.

While there are more advanced tools like Anubis[1] or iocaine[2], for smaller
sites often the simpler approach of blocking access by user-agent is currently
still enough.

Here are the different ways how to enable this blocklist with your website.


Blocking bots from within the haproxy configuration
---------------------------------------------------

If you have write access to the apache2 configuration, the block-ai-bot
configuration snipped can be included in the frontend section:

---snip---

acl ai_robot hdr_sub(user-agent) -i -f /etc/haproxy/block-ai-bots.txt
http-request deny if ai_robot

---snap---

and reload haproxy:

  $ sudo service haproxy reload


[1] https://github.com/TecharoHQ/anubis
    https://bugs.debian.org/1102132

[2] https://iocaine.madhouse-project.org
    https://bugs.debian.org/1115902

 -- Daniel Baumann <daniel@debian.org>  Sun, 08 Mar 2026 18:57:46 +0100