How to use AI Crawler Check
Task-oriented guides for checking whether AI systems and search engines can read your site, understanding what the results mean, and fixing what is wrong. No account needed for any of it.
In a hurry?
Quick start: your first crawler check
From a bare domain to a fix list, in about a minute.
4 min readUnderstanding your score and grade
What the number means, and what it deliberately does not mean.
5 min readFixing a crawler that is blocked by mistake
The rules that most often block a crawler unintentionally, and how to undo them.
6 min readTools reference
What each tool does, and when to reach for it.
5 min readUsing AI Crawler Check for client work
A repeatable audit workflow, and how to present the findings.
5 min readThe two ideas worth knowing first
Access is not comprehension
A crawler can reach your page and still fail to understand it well enough to quote you. That is why the report covers both: crawler access across 196 bots, and 58 checks on whether the content can actually be used.
Training is not retrieval
Blocking a crawler that trains a model is an editorial choice. Blocking one that fetches your page to cite it removes you from AI answers. The two look almost identical in robots.txt, which is why they get confused.
What the database covers
196 crawlers, sorted into 8 categories and 6 tiers by how much their access actually matters.