Are you blocking the AI bots you want? A robots.txt guide for stores
GPTBot, OAI-SearchBot, PerplexityBot, Claude-SearchBot — which ones matter, which ones train models, and the three-line fix most stores need.
September 26, 2026 · 3 min read · robots.txt · AI bots · technical SEO
The most common problem we see in AI visibility checks isn't a bad product page. It's a store that blocks the very bots that would recommend it — usually by accident, often from a well-meaning "block AI" rule copied from a blog post.
Two kinds of AI bots
This is the distinction that changes everything:
Training bots collect data to train future models. Blocking them is a legitimate choice — it affects whether your content ends up in a model's training set, not whether the assistant can find you today.
GPTBot(OpenAI training)ClaudeBot(Anthropic training)Google-Extended(Google's AI training opt-out)CCBot,Bytespider,Meta-ExternalAgent
Search / answer bots fetch pages at question time to answer a user. Block these and the assistant literally cannot see your store when someone asks.
OAI-SearchBotandChatGPT-User(ChatGPT search and browsing)PerplexityBotandPerplexity-UserClaude-SearchBotandClaude-UserGooglebot(also powers AI Overviews and AI Mode) andBingbot(ChatGPT search leans on Bing's index)
A rule like User-agent: GPTBot / Disallow: / does not block ChatGPT search. A rule blocking OAI-SearchBot does. Many stores have the second one without knowing it.
The three-line fix
If you want to be recommendable, make sure the answer bots are explicitly allowed. You can still block training bots if that's your policy:
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
# Optional: opt out of training only
User-agent: GPTBot
Disallow: /
The firewall problem nobody mentions
Your robots.txt can be perfect and AI bots can still be blocked — by Cloudflare or another WAF showing a "checking your browser" challenge to anything that isn't a normal browser. From the bot's side, your store is an empty page. The NIMSEO check tests this by fetching your home page as each bot and comparing what comes back.
On Salla, Zid and Shopify
Shopify lets you edit robots.txt.liquid from the theme code editor. On Salla and Zid, robots.txt is platform-managed: check what it contains (visit yourstore.com/robots.txt), and if an answer bot is blocked, raise it with platform support — it's a platform-level setting, not something to hack around.
Run the free check. If G1 or G2 is red, this article is your fix.