Free AI agent check
AI agent readiness check can agents use this page?
AI agents such as Grok Bot and ChatGPT Agent open your website in a cloud browser to book, buy and request quotes for their users. They ignore robots.txt. What stops them is a firewall challenge, a CAPTCHA, a login wall or a page that needs heavy JavaScript. Enter a page to see which of those it puts in their way.
- 8checks per page
- 2requests per run
- 0 to 100readiness score
Agent readiness
0/100
We request the page from our own servers, not from any agent's network, and read the raw HTML without running JavaScript. A site can treat a real agent differently. Methodology.
Why agents fail on websites
xAI's Grok Bot documentation says each Bot works on "a persistent cloud computer with a browser", reaching sites through shared static IP addresses. The same documentation adds that some services block Bots because they "flag datacenter IP addresses", and that CAPTCHAs, passwords and two-factor codes are steps the Bot hands back to its user.
So an agent fails on the same things as a person browsing from a cloud server, except that it has no patience and no way to ask for help: a firewall that challenges data-centre traffic, a CAPTCHA on a form, a login before any useful content, or a page that only appears after heavy scripts. For the full background read how to make your website work for AI agents and can you block Grok Bot?
The eight checks, and how to pass each one
| Check | What we look for | How to pass |
|---|---|---|
| Reachable from a cloud browser | A normal answer to a Linux Chrome request from a data centre, with no redirect loop | Do not refuse data-centre IPs on public pages; fix loops |
| No firewall challenge | Challenge or block pages from Cloudflare, Akamai, Imperva, DataDome, PerimeterX, AWS WAF, Sucuri or DDoS-Guard | Lower the challenge level on read-only pages, or exempt the agents you want |
| No CAPTCHA on the page | reCAPTCHA, hCaptcha, Turnstile, DataDome, Arkose or Friendly Captcha in the HTML | Use an invisible or risk-based mode on search, contact and quote forms |
| Content visible without logging in | HTTP 401, or a page that is mostly a password form | Show prices, features and contact details before the login |
| Content in the HTML | Enough text without running scripts | Render key information on the server |
| Labelled form fields | A label, aria-label or placeholder on 90% or more of fields | Give every field a visible label |
| Real buttons and links | Clickable div or span elements and links to "#" | Use button and a elements with text |
| Same response for automated browsers | The cloud-browser request gets the same status as a desktop browser | Do not serve automated browsers a different page |
Common blockers, by where they come from
| Source | What it does to agents | What to change |
|---|---|---|
| CDN bot protection (for example Cloudflare Bot Fight Mode or a WAF managed challenge) | Serves a challenge page to data-centre traffic, which is where agents run | Skip the challenge on public, read-only pages; keep it on login and checkout |
| "Under attack" or high-security modes | Challenges every visitor, people and agents alike | Use it only during an attack, then switch it off |
| Form plugins with a visible CAPTCHA | Stops the agent at the form and hands the step back to its user | Use an invisible or score-based mode, or a honeypot field |
| Single-page app frameworks without server rendering | The HTML is an empty shell until scripts run | Server-render or pre-render public pages |
| Membership and paywall plugins | Hide content behind a login | Show a public summary, prices and contact details |
| Cookie or newsletter pop-ups that cover the page | Block clicks until dismissed | Make the pop-up dismissible with a labelled button, or delay it |
Related: how AI engines choose their sources
Each change keeps the protection where abuse is costly and removes it where an agent is doing what a customer would.
How the score is calculated
Each check is a pass (1), a warning (0.5) or a fail (0); checks that do not apply, such as forms on a page without forms, are left out. The score is the average, out of 100. A plain average would let one serious problem hide behind seven easy passes, so the worst problem caps the score:
| Worst problem found | Score at most | Verdict |
|---|---|---|
| Page refused, looped or challenged | 30 | Agents are likely stopped before the page loads |
| Any other fail (JavaScript shell, fake controls) | 60 | Agents will struggle on this page |
| CAPTCHA or login wall on the page | 75 | Agents may need their user to step in |
| Any other warning | 85 | Mostly ready, with friction |
| Nothing | 100 | Ready for AI agents |
Three results, read and fixed
- A score of 30: "Agents are likely stopped before the page loads"
- The No firewall challenge check failed: a bot-protection page was served instead of the contact page. Fix: lower the challenge level on public, read-only pages, or allow the verified agents you want through, then run the check again.
- A score of 60: "Agents will struggle on this page"
- Content in the HTML failed: the product page is an empty shell until scripts run. Agents do run JavaScript, but slowly and unreliably. Fix: render the name, price, stock and specifications on the server.
- A score of 75: "Agents may need their user to step in"
- The quote form loads reCAPTCHA. Everything else passes. Fix: switch to an invisible or risk-based mode for the quote form, and keep the visible challenge for login and payment.
What this check cannot see
- It requests the page from our servers, not from any agent's network, so a site that recognises a real agent by IP or signature can treat it differently.
- It reads raw HTML and does not run JavaScript. A CAPTCHA added later by a script, or one that appears only when a form is submitted, will not show.
- It checks one page per run. A booking flow that spans four pages needs four runs.
- It cannot sign in. A page behind your own login is reported as a login wall, which may be exactly what you intend.
Which pages to test first
- Search and category pages, where agents compare options for their user.
- Product or service pages, where prices, stock and specifications must be in the HTML.
- Booking, quote and contact pages, where a CAPTCHA or unlabelled form ends the task.
- Your homepage, because a firewall challenge there usually applies everywhere.
Most of this work is also good accessibility practice, which is not a coincidence: pages that work for screen readers tend to work for agents.
After you change something
- Run the check again on the same URL; the address bar keeps the URL, so you can bookmark it.
- Check the neighbouring step of the flow too. Fixing the contact page is little use if the confirmation page still challenges.
- Ask a colleague on a different network to try the flow in a private window, because some bot protection treats new visitors differently.
- If the score is unchanged, look at the Fix line of the first failing check: the cap means one serious problem outweighs every pass.
Agents compared with crawlers
| Feature | Crawler | Agent |
|---|---|---|
| Acts for | An operator building an index or dataset | A person, right now |
| Reads robots.txt | Usually yes | Usually no |
| Runs JavaScript | Often not | Yes, in a real browser |
| Stopped by | robots.txt, firewall | Challenges, CAPTCHAs, logins, broken UI |
| Check with | The AI crawler diagnostic and the AI search visibility check | This readiness check |
Agent readiness questions
What does the AI Agent Readiness Check test?
It loads one page the way a cloud-based AI agent would, from a server in a data centre with a Linux browser user agent, and checks eight things that stop agents: refusals, redirect loops, firewall challenges, CAPTCHA widgets, login walls, content hidden behind JavaScript, unlabelled form fields and fake buttons.
Is this the same as checking AI crawlers?
No. Crawlers such as GrokBot or GPTBot read robots.txt and fetch pages automatically. Agents such as Grok Bot and ChatGPT Agent use a real browser for a person and ignore robots.txt. For crawlers, use the AI Crawler Diagnostic.
Does it use the real Grok Bot or ChatGPT Agent?
No. We cannot send traffic from those products. We request your page from our own servers with an ordinary Linux Chrome user agent. A site that recognises the real agents by IP or signature can treat them differently, so treat the result as a strong signal, not proof.
Can it see a CAPTCHA that only appears when I submit a form?
Only if the CAPTCHA script is in the page HTML. We do not run JavaScript or submit forms, so a challenge added later by script will not appear in the result.
Should I remove every CAPTCHA?
No. Keep them on login, payment and anywhere abuse is costly. Remove them from search, browsing and simple contact or quote forms, where they stop AI agents doing legitimate research for customers.
Related: writing content AI engines can cite