ai.robots.txt for nginx
=======================

Bots are reported to now make up half of the web's traffic. There are several
ways of blocking these.

While there are more advanced tools like Anubis[1] or iocaine[2], for smaller
sites often the simpler approach of blocking access by user-agent is currently
still enough.

Here are the different ways how to enable this blocklist with your website.


1. Blocking bots from within the nginx configuration
----------------------------------------------------

If you have write access to the nginx configuration, the block-ai-bot
configuration snipped can be included either in a server or location block:

---snip---

include snippets/block-ai-bots.conf;

---snap---

and reload nginx:

  $ sudo service nginx reload


2. Blocking bots from within the websites document root
-------------------------------------------------------

Alternatively you can symlink the robots.txt file into your document root, e.g.:

  $ ln -s /usr/share/nginx-block-ai-bots/robots.txt /var/www/html/robots.txt

Please make sure in your nginx configuration that symlinks are followed
(by default enabled, i.e. 'disable_symlinks' is not set to 'off').

Note that this only works for clients that actually respect robots.txt,
which makes this almost useless these days.


[1] https://github.com/TecharoHQ/anubis
    https://bugs.debian.org/1102132

[2] https://iocaine.madhouse-project.org
    https://bugs.debian.org/1115902

 -- Daniel Baumann <daniel@debian.org>  Sun, 08 Mar 2026 18:57:46 +0100