Tools reference
What each tool does, and when to reach for it.
5 min read
Everything here is free and needs no account. This page is about picking the right one for the job in front of you.
Crawler check
The main scan. Resolves your robots.txt against all 196 crawlers across 8 categories, detects your AI files, runs the 58 GEO checks and returns a scored report with fixes. Start here unless you already know exactly what you need.
robots.txt generator
Builds a correct file from per-bot toggles and presets. Use it when you want to change policy. It repeats shared rules into every group, which is the step people miss when writing the file by hand.
robots.txt validator
Checks a file you already have and warns when a rule would block something SEO-critical. Use it before deploying a hand-edited robots.txt, and after inheriting a site whose file you did not write.
llms.txt generator
Produces an llms.txt describing what your site is about. robots.txt grants permission; llms.txt supplies context. They solve different problems and you can publish both.
Schema generator
Generates JSON-LD structured data. Structured data is one of the more direct ways to make a page machine-readable, which is why it carries real weight in the GEO checks.
Batch checker
Checks up to 20 URLs in one run and exports CSV. Built for agencies and anyone auditing a portfolio, where running scans one at a time is the bottleneck.
GEO audit
A deeper pass focused on whether your content can be understood and cited, scored across 5 areas. Use it once access is already correct and you want to improve how you are represented.
Bot directory
297 reference pages: every crawler in the database, 8 category overviews and 91 operator profiles. Use it when you have a user agent from your logs and need to know who it belongs to and whether to allow it.
Which tool for which job
| Your situation | Use this |
|---|---|
| "Can AI read my site at all?" | Crawler check |
| "I want to change what I allow" | Generator |
| "Is my existing file safe?" | Validator |
| "I have many sites to audit" | Batch checker |
| "What is this user agent in my logs?" | Bot directory |
| "Access is fine, citations are not" | GEO audit |