diff options
-rw-r--r-- | README.md | 7 |
1 files changed, 7 insertions, 0 deletions
@@ -159,6 +159,11 @@ carousel: height: 655 ``` +## Robots.txt + +The contents of `static/robots.txt` are from [this site][robotstxt-ai] +and used to prevent the entire site from being indexed by LLM crawlers. + ## Deploy Site To clone site repo: @@ -248,3 +253,5 @@ The [Bulma][] [SASS][] is: "Progressive enhancement picture and figure shortcodes for Hugo." [pngquant]: https://pngquant.org/ "Lossy compression of PNG images." +[robotstxt-ai]: https://robotstxt.com/ai + "AI / LLM User-Agents: Blocking Guide" |