LLMentions
AI crawlers · Amazonbot

Amazonbot: Amazon's AI training crawler

Amazonbot is Amazon's AI training crawler. What it does, whether to allow it, robots.txt rules and how to verify real Amazonbot traffic.

↓ Markdown for agents
Vendor
Amazon
Type
AI training crawler
Runs JavaScript
No
IP ranges published
No

Amazonbot is Amazon's crawler for collecting training data for Amazon's AI and search features. It reads public pages so future models know about them. It does not fetch pages to answer a specific question.

Should you allow Amazonbot?

Your choice. Blocking Amazonbot keeps your content out of future training data. It does not remove you from AI search answers, which use separate bots. Being in training data can still help models know your brand when they answer without searching.

robots.txt

To allow Amazonbot everywhere:

text
User-agent: Amazonbot
Allow: /

To block it:

text
User-agent: Amazonbot
Disallow: /

Bots follow the most specific User-agent group that matches them. If you have a User-agent: Amazonbot group, the rules under User-agent: * no longer apply to it. Matching on the token is case-insensitive.

Identifying Amazonbot

Amazonbot identifies itself with Amazonbot in its User-Agent header. It does not run JavaScript: it reads the raw HTML only. Content your site adds with JavaScript is invisible to it.

Amazon doesn't publish a machine-readable IP range list for Amazonbot that we verify against, so treat the user agent alone as unverified. bots_report marks these hits as unverifiable rather than genuine.

Check your site

terminal
llmentions audit example.com            # is Amazonbot allowed by robots.txt, and what does it see?
llmentions bots /var/log/nginx/access.log   # did Amazonbot really visit, and is it genuine?

Or identify any user agent over REST (free): GET https://api.llmentions.com/v1/bots/lookup?ua=Amazonbot.