Audit checks · no_robots_txt
No robots.txt
No robots.txt: why it keeps you out of AI answers and how to fix it. Audit check no_robots_txt (notice).
↓ Markdown for agents- Check id
no_robots_txt- Severity
- notice
- Tool
site_audit
Without robots.txt, bots fall back to defaults and you lose the easiest place to list your sitemap and state your AI policy.
How to fix it
Publish /robots.txt, even if it allows everything, and list your sitemap in it.
How LLMentions checks it
site_audit crawls your site the way AI bots do (raw HTML, no JavaScript), reads robots.txt for every known AI and search bot, and renders a few pages in a real browser for comparison. This check reports as no_robots_txt with severity notice.
terminal
llmentions audit example.comThe result lists every issue with the pages affected, most severe first, and what changed since your last audit. See the site audit guide.