# Let AI answer engines in, *keep training bots out*

Tick the crawlers you want to allow or block, set your Content-Signal, and copy the result. Pick Shopify to get a robots.txt.liquid template that keeps Shopify's default rules for cart, checkout and account pages.

Some links may earn us a commission at no cost to you. Drafted with AI, checked by a human. [How we earn](https://storevetted.com/disclosure/).

- Your policy, line by line
- Shopify robots.txt.liquid option
- Runs in your browser

## Your file

Generated robots.txt

[Check your site after publishing](https://storevetted.com/ai-agent-readiness-checker/)

## How the file is built

The plain version starts with a User-agent: \* group carrying your Content-Signal line and any paths you listed. Allowed AI bots get their own group with Allow: /; your paths are repeated there, because a bot with its own group ignores the \* group. Blocked bots get Disallow: /.

The Shopify version keeps the `robots.default_groups` loop from Shopify's documentation, adds your Content-Signal and paths to the \* group, and appends groups only for the bots you block. Shopify notes that adding robots.txt.liquid replaces its generated file, so save a copy of your current /robots.txt first and compare.

Bot names come from each operator's own documentation: OpenAI lists GPTBot, OAI-SearchBot and ChatGPT-User; Anthropic lists ClaudeBot, Claude-SearchBot and Claude-User. robots.txt is a request, not a lock, so use firewall rules for anything that must be enforced.

From the guides

## Related guides

[All guides](https://storevetted.com/guides/)

- [Fall '26 Editions for BFCM](https://storevetted.com/guides/shopify-editions-fall-2026-bfcm/)
- [Migration: lost tracking links](https://storevetted.com/guides/shopify-migration-lost-tracking-links/)
- [Refund policy](https://storevetted.com/guides/shopify-refund-policy-example/)

Keep going

## Related free tools

[All 81 tools](https://storevetted.com/tools/)

[AI agent readiness checker](https://storevetted.com/ai-agent-readiness-checker/): Score how AI crawlers and shopping agents read your store.

[Store launch checklist](https://storevetted.com/store-launch-checklist/): Everything to set up before the first order. Checklist

[Refund policy generator](https://storevetted.com/refund-policy-generator/): A plain returns page agents can quote.

[Shopify vs WooCommerce](https://storevetted.com/shopify-vs-woocommerce/): Both platforms side by side, every line sourced.

## FAQ

Which AI bots should an online store allow?

The ones that put you in answers: OAI-SearchBot for ChatGPT search, Claude-SearchBot, PerplexityBot, plus the user-fetch agents (ChatGPT-User, Claude-User, Perplexity-User) that open a page when a shopper asks. Blocking them can remove the store from those answers.

Does blocking GPTBot or ClaudeBot hurt my visibility?

OpenAI and Anthropic both document separate crawlers for training and for search, so blocking the training crawler is meant to keep content out of model training without removing the site from their search results.

What is Content-Signal?

A robots.txt line that states your preferences in three parts: search, ai-input (using content in AI answers) and ai-train (training models). It sits inside the User-agent: \* group. It states a preference; it does not technically block anyone, which is why the generator also writes Disallow groups.

How do I edit robots.txt on Shopify?

In the theme code editor, add a robots.txt.liquid template. Adding it replaces Shopify's generated file, so the template here keeps Shopify's default rules loop and adds your lines to it. Save a copy of your current /robots.txt before you publish.

Is robots.txt enforced?

No. Reputable crawlers follow it; others may not. Use your firewall or bot-protection settings for anything that must be enforced.

---

Canonical: https://storevetted.com/ai-crawler-robots-txt-generator/  
Publisher: StoreVetted (https://storevetted.com/). Some links may earn a commission at no cost to you: https://storevetted.com/disclosure/  
Site index for agents: https://storevetted.com/llms.txt
