ai.robots.txt for apache2
=========================

Bots are reported to now make up half of the web's traffic. There are several
ways of blocking these.

While there are more advanced tools like Anubis[1] or iocaine[2], for smaller
sites often the simpler approach of blocking access by user-agent is currently
still enough.

Here are the different ways how to enable this blocklist with your website.


1. Blocking bots from within the apache2 configuration
------------------------------------------------------

Preferably if you have write access to the apache2 configuration,
the block-ai-bot configuration snipped can be included in individual VirtualHost
directives:

---snip---
<VirtualHost www.example.org>
  Include /etc/apache2/conf-available/block-ai-bots.conf
  [...]

</VirtualHost>
---snap---

or be enabled globally:

  $ sudo a2enconf block-ai-bots
  $ sudo service apache2 reload


2. Blocking bots from within the websites document root
-------------------------------------------------------

Alternatively you can symlink the htaccess file into your document root, e.g.:

  $ ln -s /usr/share/apache2-block-ai-bots/htaccess /var/www/html/.htaccess
  $ ln -s /usr/share/apache2-block-ai-bots/robots.txt /var/www/html/robots.txt

Please make sure that htaccess usage is enabled in your apache2 configuration
(AllowOverride setting) and that symlinks are followed
(Options +SymLinksIfOwnerMatch or +FollowSymLinks, depending on your setup).


[1] https://github.com/TecharoHQ/anubis
    https://bugs.debian.org/1102132

[2] https://iocaine.madhouse-project.org
    https://bugs.debian.org/1115902

 -- Daniel Baumann <daniel@debian.org>  Sun, 08 Mar 2026 18:57:46 +0100