aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--README.md7
1 files changed, 7 insertions, 0 deletions
diff --git a/README.md b/README.md
index 5e4b156..d6a5acd 100644
--- a/README.md
+++ b/README.md
@@ -159,6 +159,11 @@ carousel:
height: 655
```
+## Robots.txt
+
+The contents of `static/robots.txt` are from [this site][robotstxt-ai]
+and used to prevent the entire site from being indexed by LLM crawlers.
+
## Deploy Site
To clone site repo:
@@ -248,3 +253,5 @@ The [Bulma][] [SASS][] is:
"Progressive enhancement picture and figure shortcodes for Hugo."
[pngquant]: https://pngquant.org/
"Lossy compression of PNG images."
+[robotstxt-ai]: https://robotstxt.com/ai
+ "AI / LLM User-Agents: Blocking Guide"