Quick start: your first crawler check

From a bare domain to a fix list, in about a minute.

4 min read

AI Crawler Check answers one question: can AI systems and search engines actually read your website? You do not need an account, a credit card, or anything installed. If you only have a minute, this is the whole workflow.

  1. Enter your domain

    Type your site into the box on the home page and press Check. You can paste a bare domain like example.com or a full URL. There is nothing to install and no account to create.

  2. Wait for the scan

    The scan fetches your robots.txt, resolves it against all 196 crawlers in the database, then looks for llms.txt and runs the GEO checks. It usually takes a few seconds.

  3. Read your score first

    The score out of 100 and its letter grade are a summary, not the answer. Treat them as a way to see whether anything urgent is wrong, then move to the detail below.

  4. Check the Tier 1 row

    Tier 1 is the group that decides whether AI assistants can quote you at all. If anything there is blocked and you did not intend it, fix that before anything else on the page.

  5. Work the recommendations

    Each recommendation comes with the exact lines to paste. They are ordered by impact, so working top-down is the fastest route to a better result.

  6. Re-run after deploying

    The check reads your live site, so it only reflects a change once that change is deployed. Re-run it to confirm the fix landed rather than assuming.

What the scan actually reads

It fetches your live robots.txt, your homepage HTML, and (if present) llms.txt and llms-full.txt. It does not crawl your whole site, so it is fast and it does not add meaningful load to your server.

Reading the results

The report is organised so the most consequential information is nearest the top.

What each section of the report tells you
Section What it answers
Score and gradeRoughly how discoverable you are right now.
Bot access tableWhich of the 196 crawlers you allow, block, or partly block.
AI filesWhether you publish llms.txt, and whether it is valid.
GEO report58 checks on whether your content can be understood and cited.
RecommendationsWhat to change, with copy-paste code, ranked by impact.

Blocked is not always bad

Blocking a training crawler is a legitimate editorial choice and costs you nothing in search rankings. What you almost never want is to block a retrieval crawler, because those are the ones that fetch your page in order to cite it in an answer. The report labels which is which so you are not guessing.

Where to go next

Common questions

Do I need an account to use AI Crawler Check?
No. Every tool is free and requires no sign-up, no email and no credit card. You can run a check immediately.
How long does a check take?
Usually a few seconds. It fetches your robots.txt, your homepage and your AI files rather than crawling your entire site, which is why it stays fast.
Does running a check affect my site or my SEO?
No. It makes a small number of ordinary HTTP requests, the same as any visitor. It changes nothing on your site and has no effect on your rankings.
Can I check a site I do not own?
Yes. It only reads publicly available files, so you can check competitors, clients or prospects. Nothing private is accessed.
Why did my score not change after I edited robots.txt?
The scan reads your live site, so the edit has to be deployed first. If you changed a local file or a staging environment, the production robots.txt is still the old one. Confirm by opening yourdomain.com/robots.txt in a browser.

Ready to check your site?

196 crawlers, 58 GEO checks, no sign-up.

Run a free check