ClaudeBot, Claude-User and Claude-SearchBot: Anthropic's three agents
Last reviewed
Short answer
Anthropic runs three documented agents. ClaudeBot collects web content that may contribute to model training. Claude-User visits sites on behalf of a person using Claude. Claude-SearchBot indexes content to improve Claude's search results. All three share one published IP range file at claude.com/crawling/bots.json.
The three tokens
Quoted from Anthropic's support documentation, read 23 August 2026. Anthropic publishes the robots.txt tokens and a shared IP range file, and does not publish full header strings the way OpenAI and Perplexity do, so match on the token.
| Token | Stated purpose |
|---|---|
| ClaudeBot | Collects web content that could potentially contribute to training generative models |
| Claude-User | Accesses websites when a person asks Claude a question |
| Claude-SearchBot | Analyses content to improve the relevance and accuracy of search responses |
The older anthropic-ai token
You will find `anthropic-ai` in blocklists copied around since 2023. Anthropic's current documentation lists the three tokens above, so those are what a rule should target today.
Leaving a stale `anthropic-ai` group in your file is harmless, since a group nothing matches simply does nothing. It is worth removing anyway, because the next person to read the file will assume it is doing something.
Which to allow
The same split as everywhere else on this site. ClaudeBot is the training crawler and blocking it is a values decision with little revenue attached. Claude-SearchBot and Claude-User are how Claude finds and reads your pages, and blocking those makes your store unreadable to someone actively asking about it.
Claude cites its sources with links when it browses, which puts it in the same category as Perplexity: a surface that can actually send a click rather than only absorbing the answer.
How you actually change this on Shopify
Shopify generates robots.txt for you and there is no field in the admin for it. To change a rule you add a `robots.txt.liquid` template to your theme, which Shopify then renders instead of the default while still keeping its own future updates.
Shopify's help page is blunt about the risk: editing this file is an unsupported customization, Shopify Support will not help with it, and it says in as many words that incorrect use of the feature can result in loss of all traffic. Copy the default rules first, add your block, and read the rendered file at /robots.txt afterwards.
- Online Store, then Themes, then Edit code
- Add a new template called robots.txt.liquid
- Keep the default rules, then add your own user-agent group
- Load yourstore.com/robots.txt and read what actually rendered
Frequently asked questions
What is ClaudeBot's robots.txt user agent?
The token is ClaudeBot. Anthropic also documents Claude-User for user-initiated visits and Claude-SearchBot for search indexing, each taking its own robots.txt group.
Is anthropic-ai still the right token to block?
Anthropic's current documentation lists ClaudeBot, Claude-User and Claude-SearchBot. Rules written against anthropic-ai are matching a token the current docs do not list, so a block written that way is unlikely to be doing what its author intended.
Where are Anthropic's crawler IP ranges published?
All three agents share one file at https://claude.com/crawling/bots.json. Verify a request against it rather than trusting the user-agent header, which anything can send.
Does blocking ClaudeBot stop Claude from reading my store?
No. ClaudeBot is the training crawler. Claude-SearchBot handles search indexing and Claude-User handles live fetches, so blocking ClaudeBot alone leaves Claude able to find and read your pages.
Sources
Keep reading
Score your own catalog
Entitled grades every product against the rules on this page, explains each failure, and fixes them with AI you approve. The score is free forever.