Cloudflare blocks AI crawlers by default on September 15. Is your store affected?
Last reviewed
Short answer
Only if you run your own Cloudflare account. From 15 September 2026, new domains onboarding to Cloudflare get AI training and agent bots blocked by default on pages that display ads, and customers who ever enabled the legacy Block AI bots setting will start blocking multi-purpose crawlers like Googlebot. A Shopify store with no Cloudflare account of its own is not affected.
What changes on September 15
Cloudflare announced the change on 1 July 2026. It sorts crawler behaviour into three configurable categories: Search, which collects and indexes content and sends referral traffic, Agent, which acts in real time on a person's behalf, and Training, which takes content to train a model. From 15 September, all new domains onboarding to Cloudflare get Training and Agent blocked by default on pages that display ads, while Search remains allowed.
There is a second change that reaches back into existing accounts, and it is the sharper one. From the same date, multi-purpose crawlers such as Googlebot, Applebot and BingBot become subject to the most restrictive rule that applies to them. Cloudflare's own post says these will be blocked by customers who have selected to block Training, whether through the new options or through the legacy Block AI bots service. If you clicked Block AI bots on a zone at some point and forgot about it, that setting is about to grow teeth.
Whether your store is affected
A Shopify store on the default setup is not. Cloudflare zone settings only act on traffic that is proxied through your own Cloudflare zone, and Shopify's help documentation tells merchants not to proxy: it states that Cloudflare proxy setups, including O2O, are not supported and could break at any time. A correctly configured custom domain points at Shopify with DNS-only records, and DNS-only traffic never touches your Cloudflare rules. If you have never made a Cloudflare account, this change does not touch your store.
The merchants who are affected run their own Cloudflare zone in front of something: an external landing page, a blog on separate infrastructure, a headless or Hydrogen storefront on their own hosting, or the unsupported proxied setup in front of the store itself. On those zones, the September defaults and the multi-purpose crawler rule are yours to manage.
- Not affected: default Shopify setup, custom domain with DNS-only records, no Cloudflare account
- Affected: your own Cloudflare zone proxying landing pages, blogs or headless storefronts
- Affected: the unsupported orange-cloud proxy in front of a Shopify store
- Most exposed: any zone where Block AI bots was ever enabled, because Googlebot joins the blast radius on September 15
Which bots land in which category
Cloudflare publishes its own per-bot categories, read from its bot reference documentation on 29 August 2026. The mapping is what decides your AI visibility, because the user-fetch agents fall on the blocked-by-default side for new domains.
The practical translation: the bots that fetch your page because a shopper just asked an assistant about your product are Agent-class, and Agent is a default block on ad-bearing pages for new domains. Blocking those bots means the assistant answers about your store without reading it.
| Bot | Cloudflare category | September 15 default for new domains |
|---|---|---|
| OAI-SearchBot | AI Search, treated as Search | Allowed |
| PerplexityBot | AI Search, treated as Search | Allowed |
| Claude-SearchBot | AI Search, treated as Search | Allowed |
| ChatGPT-User | AI Assistant, treated as Agent | Blocked on pages that display ads |
| Perplexity-User | AI Assistant, treated as Agent | Blocked on pages that display ads |
| Claude-User | AI Assistant, treated as Agent | Blocked on pages that display ads |
| GPTBot | AI Crawler, treated as Training | Blocked on pages that display ads |
| ClaudeBot | AI Crawler, treated as Training | Blocked on pages that display ads |
How to check, in ten minutes
Log in to the Cloudflare dashboard, pick the zone, and open AI Crawl Control, which is available on all plans. The Crawlers tab lists each AI crawler with its category, request counts and an Action column. That table is the ground truth for what your zone is doing to each bot today.
Then open Security settings on the same zone and look for the legacy Block AI bots toggle. Cloudflare says customers can opt out of the new defaults any time before 15 September, and that the setting for multi-purpose crawlers lives in Security settings. Five minutes per zone, once.
- Cloudflare dashboard, select the zone, open AI Crawl Control, then the Crawlers tab
- Read the Action column for OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-User
- Security settings: check whether Block AI bots was ever enabled
- Review WAF custom rules, which act on bots independently of AI Crawl Control
- Repeat for every zone you own, including the forgotten ones
How to fix what you find
Per-crawler allows are one click: the Crawlers tab's Action column takes Allow or Block per bot, and Cloudflare's docs name the use case as allowing crawlers that provide value through citations and referrals. Allow the search and user-fetch agents, and treat the training pair as the values call it is.
Two things the dashboard will not do for you. AI Crawl Control implements blocks as a WAF custom rule, so an older custom rule of your own can keep blocking a bot you just allowed, and it is worth reading your rule list once. And verification is changing under the hood: Cloudflare now says a verified bot is allowable within its category rather than allowed by default, so the category settings, not the verified label, are what decide.
Frequently asked questions
My store is on Shopify with a normal custom domain. Do I need to do anything?
No. Cloudflare's rules only apply to traffic proxied through your own Cloudflare zone. Shopify instructs merchants to use DNS-only records and states proxy setups are unsupported, so on a correct setup there is nothing between your store and these bots for Cloudflare to block.
Does this change affect existing Cloudflare zones or only new ones?
Cloudflare's published wording applies the new category defaults to new domains onboarding from 15 September 2026. The change that does reach existing zones: anyone blocking Training, including via the legacy Block AI bots setting, will start blocking multi-purpose crawlers like Googlebot from that date.
Could this block Google and hurt my rankings?
On your own zones, yes, that is the sharpest risk. Cloudflare states multi-purpose crawlers such as Googlebot, Applebot and BingBot will be blocked for customers who block Training, including through the legacy Block AI bots service. Check Security settings on every zone before September 15.
What does 'pages that display ads' mean for a store?
Cloudflare scopes the new default blocks to ad-bearing pages. Most store pages do not display third-party ads, but content sites and blogs monetised with ads do. Cloudflare's per-crawler controls apply either way, so check the Crawlers tab rather than reasoning about it.
Sources
Keep reading
Score your own catalog
Entitled grades every product against the rules on this page, explains each failure, and fixes them with AI you approve. The score is free forever.