Is my site visible to AI?
ChatGPT, Claude, Perplexity, Gemini… Enter your address and you’ll see which AI crawlers can read your site, and which lines to change.
Verdict for
- Answer engines allowed
- Training opted out
- llms.txt
Crawler by crawler
| Crawler | Role | Access to your site | Where the rule comes from |
|---|
“Search”: indexes your pages to cite and link to them in answers. “Training”: collects text to train models. “On demand”: reads a page when a user asks a question — some vendors state that these crawlers may not follow robots.txt.
robots.txt & sitemap
llms.txt
What to do
Sources: each crawler’s official documentation
- Google — Googlebot
- Google — Google-Extended
- OpenAI — OAI-SearchBot, GPTBot, ChatGPT-User
- Anthropic — Claude-SearchBot, ClaudeBot, Claude-User
- Perplexity — PerplexityBot, Perplexity-User
- Apple — Applebot, Applebot-Extended
- Meta — Meta-WebIndexer, Meta-ExternalAgent, Meta-ExternalFetcher
- Mistral AI — MistralAI-Index, MistralAI-Training, MistralAI-User
- Amazon — Amzn-SearchBot, Amazonbot, Amzn-User
- Common Crawl — CCBot
- How the rules are read: RFC 9309 (Robots Exclusion Protocol) · llms.txt: llmstxt.org
- List checked on November 19, 2026. Only crawlers whose vendor publishes an official page are included.
Opening the door isn’t enough. Getting cited is what counts.
Leave your details: I’ll get back to you with an initial take on your visibility in AI answers (GEO audit). Your test result is attached automatically. Free, no commitment.
Got it, !
I’ll get back to you shortly with an initial take on your situation. Your result has been sent to me.
Three kinds of crawlers, three different decisions
AI vendors no longer rely on a single crawler. Most publish several, each with a documented role, and that’s what lets you decide with precision:
- Search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, MistralAI-Index, Meta-WebIndexer…) index your pages to cite and link to them in answers. Block them and you disappear from those answers. For Google, AI Overviews and AI Mode rely on Googlebot, according to Google.
- Training crawlers (GPTBot, ClaudeBot, MistralAI-Training, Meta-ExternalAgent, Applebot-Extended, Google-Extended…) collect text to train future models. Allowing them or not is an editorial choice: it doesn’t change your presence in answers that cite sources.
- “On-demand” crawlers (ChatGPT-User, Claude-User, Perplexity-User…) read a specific page when a user asks for it. Several vendors warn that they may not follow robots.txt.
The most common trap isn’t a deliberate block: it’s a robots.txt copied from a template, a “*” rule that’s too broad, or bot protection that makes the file return an error. The first two take five minutes to fix; the third makes the site invisible without anyone noticing. To go further, read the complete GEO guide or see the AI search optimization service.
Frequently asked questions
What exactly does this tool check?
Will an “allowed” crawler necessarily cite me?
Can you be cited by AI without feeding its training?
Why can some crawlers “ignore robots.txt”?
Is an llms.txt file essential?
My robots.txt can’t be found: is that a problem?
Getting cited by AI
Get expert help
Understanding GEO
Your site is open to AI. Is it getting cited?
Describe your needs in a few lines: I reply within 24 hours with an initial analysis and concrete recommendations.