Can You Block Grok Bot? Why robots.txt Does Not Apply to AI Agents

Since xAI launched Grok Bot on 11 August 2026, a lot of site owners have asked the same question: can I block it? The short answer is yes, but not the way most people try first. robots.txt will not do it. A firewall can, with a catch. And for most businesses, blocking it is the wrong goal anyway.

This guide explains why, using xAI's own documentation, and gives you the options in order from lightest to heaviest. If you want to see what your site does to agents today, run the AI Agent Readiness Check first, or the AI crawler checker for the crawler side.

First, are you trying to block the agent or the crawler?

Grok reaches websites in two different ways, and they need different controls. Mixing them up is the most common reason a block "does not work".

If what you see in your logs is a request that names GrokBot or xAI-SearchBot, you are dealing with the crawler, and our Grok crawler user agent guide has the rules. The rest of this article is about the agent. For the full comparison, see Grok Bot vs GrokBot.

Why robots.txt does not block Grok Bot

robots.txt is a set of instructions for automated crawlers, standardised in RFC 9309. Each group starts with a User-agent line naming the crawler it applies to. A crawler that honours the file reads it before fetching, finds its group and follows the rules.

Grok Bot is not a crawler in that sense. The Grok Bot overview says each Bot works on "a persistent cloud computer with a browser, filesystem, and terminal" and "takes on multi-step work across apps and websites". It opens the page its user asked about, in a browser, the way the user would. There is nothing for robots.txt to match:

• xAI documents no robots.txt token for Grok Bot.
• The agent sends an ordinary browser user agent, not an xAI name.
• A rule like User-agent: Grok Bot is not even valid, because tokens cannot contain spaces.

This is not unique to xAI. OpenAI's ChatGPT Agent works the same way, and is identified by signed requests rather than a robots.txt token. Browsing agents act for people, and robots.txt has never applied to people.

What actually stops Grok Bot

xAI is unusually open about where its agent gets stuck. Three things stop it, all from its own documentation.

1. Human verification

The Grok Bot FAQ says a site "may still block automation, require a new login, present a CAPTCHA, or require human confirmation", and that the Bot should hand those steps back. The approvals page tells users to enter passwords, two-factor codes and CAPTCHA answers themselves. So a CAPTCHA does not keep a determined person out, but it does stop the agent until its user steps in.

2. Logins

A Bot can only see what its user can see. The computer and apps page explains that sessions persist on the Bot's computer once the user has signed in, so a Bot can use a site its user has an account on. Without an account, it hits the same login wall as anyone else.

3. Firewalls that refuse datacenter traffic

The security FAQ answers "Why do some websites block Bots?" directly: "Some services flag datacenter IP addresses." Grok Bot runs on cloud computers, so a firewall rule that challenges or refuses cloud-provider traffic will stop it, along with every other cloud-based agent.

Option 1: Leave it alone (recommended for most sites)

Before reaching for a block, consider who is behind the visit. A Grok Bot request is a person who has asked an AI to do something on your site: compare your prices, request a quote, check your opening hours, book a table. Blocking it turns away that person.

For shops, service businesses, SaaS products and B2B suppliers, the better move is usually the opposite: make sure the agent can finish the job. Our guide to making your website work for AI agents covers how.

Option 2: Protect only the sensitive actions

Most sites do not need to block agents everywhere, only at the points where automation causes harm: account creation, ticket purchases, reviews, high-value orders, password resets. Put human checks there and nowhere else.

• Use challenges on actions, not on pages. A CAPTCHA on the final "place order" step stops automated purchasing. The same CAPTCHA on the product page just stops research.
• Use risk-based modes. Invisible or managed challenges that only appear for suspicious sessions let most agents through while catching abuse.
• Rely on accounts. If an action needs a logged-in user, the agent can only do it with that user's own account, which is exactly the accountability you want.

This approach costs nothing in lost customers and keeps your real protections where they matter.

Option 3: Block automated browsers with bot management

If your business genuinely needs to keep automation out, for example a ticketing or limited-release site, use the bot-management features of your CDN. Cloudflare, Akamai, DataDome and HUMAN all classify automated browsers by behaviour and fingerprint, not just by IP.

This catches Grok Bot as one of many cloud agents. It is the most robust option because it does not depend on knowing any vendor's IP list, which xAI does not publish. Expect some false positives: real people on VPNs and corporate networks sometimes look automated too.

Note that Cloudflare's AI crawler reference does not list any xAI crawler or agent, so there is no one-click "Grok" toggle. Use the general automated-traffic controls.

Option 4: Block Grok Bot by IP address

This is the only way to target Grok Bot specifically. The Grok Bot security page explains how its network works:

• Hosted computers "reach the internet through shared static egress IP addresses by default".
• The ranges are "shared across Grok Bot customers", and "dedicated per-customer IPs are not available".
• "Current ranges are available from your account team." They are not on a public page.

So to block by IP you first need the ranges, either from xAI directly or from someone who uses Grok Bot through a business account. Once you have them, a firewall rule is simple:

(ip.src in {198.51.100.0/24 203.0.113.0/24})

deny 198.51.100.0/24;
deny 203.0.113.0/24;

The addresses above are documentation examples, not xAI's. Do not copy IP lists from third-party sites: an out-of-date or wrong list blocks innocent traffic and misses the real thing.

The catch with IP blocking

Because the ranges are shared, blocking them blocks every Grok Bot user, including your own customers and partners. And they are not the only way out: the Grok Bot security page says members "can route traffic through their desktop to use its network and IP address", and Enterprise teams can install their own networking client. Traffic sent those ways will not come from the shared ranges.

In other words, an IP block stops the default path, not every path. It is a reasonable measure for a site that must keep automation out, alongside bot management, not a guarantee on its own.

What a Grok Bot user sees when you block it

It helps to picture the other side. When a Bot is blocked it does not quietly fail. xAI designs it to report back to the person who gave it the task, and what that person sees depends on which control the Bot met.

The Grok Bot FAQ confirms the design: when a site blocks automation or asks for a human step, "the Bot should hand those steps back". The approvals page adds that users should never paste a password or one-time code into chat, and should take over the computer to type it themselves.

That has a practical consequence. A CAPTCHA does not lose you the customer if they are motivated enough to solve it. A firewall refusal usually does, because there is nothing for them to solve.

Why there is no public Grok Bot block list

Site owners are used to bots that publish their addresses. Googlebot supports reverse DNS checks, and OpenAI and Perplexity publish JSON files of their crawler IPs. It is natural to look for the same from xAI. Here is why you will not find it.

• The ranges are a customer-facing detail. xAI documents them so that businesses using Grok Bot can allowlist their own services, which is why they are "available from your account team" rather than published.
• They identify the product, not one company. Traffic from all customers shares the ranges, so a list identifies "someone using Grok Bot", much as an IP list for a browser vendor would.
• They do not cover every route. Desktop routing and Enterprise networking clients send traffic from other addresses, so even a complete list would be partial.

If a website offers "the Grok Bot IP list", ask where it came from. Unless it links to an xAI source, treat it as unverified. Blocking the wrong ranges is worse than blocking none, because it turns away real visitors who happen to share a cloud provider.

What xAI tells its own customers to do

xAI's documentation is written for the people running Bots, but several passages tell you, as a site owner, what to expect from them.

• Approval before consequential actions. The Grok Bot security page tells teams to state "what a Bot may change and where it must stop". Well-configured Bots pause before sending, purchasing or publishing.
• Respect for site terms. xAI's use-case examples tell users to connect services "as permitted by their terms". Clear, visible terms on your side give you a basis to object to misuse.
• Caution with web content. xAI warns that "content a Bot reads from the outside world, like web pages" can try to steer it, and marks outside content as untrusted. Your pages are treated as data, not instructions.

None of this is enforcement, and you should not rely on another company's guidance to protect your site. It does mean a well-run Bot is more likely to stop and ask than to push through, which is one more reason targeted human checks work well.

Blocking the Grok crawler instead

Many people who search for "block Grok Bot" actually want to stop Grok reading their content in the background, which is the crawler's job, not the agent's. If that is you, use robots.txt with both reported names, and back it with a firewall rule:

User-agent: GrokBot
User-agent: xAI-SearchBot
Disallow: /

(lower(http.user_agent) contains "grokbot") or (lower(http.user_agent) contains "xai-searchbot")

The robots.txt generator has a one-click "Block Grok only" preset that writes the robots.txt part for you. To see whether those names are already visiting, paste your access log into the Grok log checker. It runs in your browser and nothing is uploaded.

Remember that neither name is documented by xAI, so a robots.txt rule is a request, and that blocking the crawler does not affect Grok Bot at all.

How to check whether a block is working

1. For the agent: run the AI Agent Readiness Check on the pages you protected. It loads them from a cloud server with a Linux browser user agent, which is the closest public test to how Grok Bot arrives, and shows any challenge, refusal or login wall it meets.
2. For the crawler: run the AI crawler diagnostic or check your logs with the Grok log checker.
3. For IP rules: check your firewall's event log. Matches on the ranges you added confirm the rule is firing.

None of these tests can send traffic from Grok Bot's own network, so they show what a cloud browser meets, not what xAI's computers meet. That is the honest limit of any public test today.

Three examples

A concert ticketing site

Ticket drops attract resale bots, and any agent product can be used for them. The site already runs strict bot management at the edge, which catches automated browsers whatever their vendor. It adds an IP rule for the Grok Bot ranges, which it obtained through a partner with an xAI business account, and keeps a CAPTCHA on checkout. It accepts that a few genuine fans using Grok Bot will be turned away, because the cost of a bulk-buying bot is far higher.

A furniture retailer

The retailer had a CAPTCHA on its product search to stop scraping. Customers asking Grok Bot to "find a walnut sideboard under 900 dollars" were getting nowhere, and the retailer's analytics showed abandoned searches from data-centre IPs. It moved the challenge to the account sign-up page, added rate limiting on search instead, and kept the product pages open. Scraping dropped because of the rate limit, and agents could browse again.

A law firm with a client portal

The firm worried that agents would reach confidential documents. They could not: the portal already requires each client's own login and two-factor code, and xAI's documentation says the Bot hands those steps back to its user. A client who uses Grok Bot can only reach their own files, exactly as they could in their own browser. The firm changed nothing on the portal and added a note to its terms about automated access.

In each case the right control was the one already aimed at the real risk. None of the three needed robots.txt, and only one needed an IP block.

A decision table

Common questions from site owners

Will blocking Grok Bot hurt my SEO?

No. Grok Bot plays no part in Google or Bing rankings. What you lose is the customers who use it.

Does Grok Bot respect my terms of service?

xAI's use-case documentation tells users to connect professional networks and other services "as permitted by their terms", which puts the responsibility on the user. If your terms forbid automated access, say so clearly, and enforce it with the technical options above.

Is Grok Bot traffic counted in my analytics?

Usually, yes. Because it runs a real browser, Grok Bot executes your analytics script like any visitor, unless the Bot's computer blocks it. Expect agent sessions to appear as ordinary visits from data-centre locations, often short and direct. If your analytics tool lets you filter by network or ASN, that is the easiest way to estimate how much of your traffic is agents.

Should I tell customers whether agents are welcome?

It is worth saying something. A short line in your terms, or on a help page, that says which tasks agents may perform and which need a human removes doubt for users and gives you a clear position if something goes wrong. Keep it plain: for example, "AI assistants may search and request quotes on behalf of customers; account creation and payment must be completed by the account holder."

Can I let some Bots in and keep others out?

Not by IP, because the ranges are shared by all customers. You can, however, require accounts for the actions that matter, which lets you decide per user rather than per agent.

Your checklist

• □Decide whether your goal is the agent (Grok Bot) or the crawler (GrokBot, xAI-SearchBot). They need different controls.
• □Remove any User-agent: Grok Bot or similar lines from robots.txt. They do nothing.
• □List the actions where automation causes real harm, and put human checks only there.
• □If you must block agents site-wide, use your CDN's bot management first.
• □If you need to target Grok Bot specifically, ask xAI or a customer for the current egress ranges and add an IP rule. Do not use lists from third parties.
• □Re-test with the AI Agent Readiness Check after any change.

For the other side of the decision, read how to make your website work for AI agents. For agents from other vendors, see the AI and LLM bot directory.