GrokBot (xAI)
Operated by xAI
Quick Facts
- User-Agent:
- GrokBot
- Category:
- AI & LLM Bots
- Operator:
- xAI
- Safety:
- Safe
- Blocking Impact:
- Low - No SEO ranking impact
- SEO Impact Score:
- 2/10
- Official docs:
- None published by xAI
- Named by:
- Trakkr AI Crawler Observatory
Reported, not operator documented
xAI has not published a crawler page that names GrokBot. It is listed here because a credible public source does: Trakkr AI Crawler Observatory. xAI Grok crawler token, listed by public registries. xAI publishes no crawler documentation.
What that means for you: the robots.txt rule on this page is the best available spelling, and a well-behaved crawler using this string will honour it, but there is no vendor statement of purpose, crawl rate or robots.txt compliance to rely on. Treat a matching request in your logs as a detection clue, not proof of who sent it, and combine the rule with a firewall rule if the traffic matters to you. If the operator publishes documentation, this profile moves to Official docs.
What is GrokBot?
GrokBot is a user-agent name that public crawler registries link to xAI's Grok assistant. xAI has not published a page that documents it, so its exact purpose is not confirmed.
Registries describe GrokBot as fetching pages for Grok answers. xAI documents a web search tool that lets Grok "search the web in real-time and browse web pages", but that page does not name the user agent the tool sends. Two other names cause confusion:
• xAI-SearchBot is a second Reported xAI name, seen as Mozilla/5.0 (compatible; xAI-SearchBot/1.0; +https://x.ai).
• Grok Bot (two words) is a different thing: an AI agent product xAI launched on 11 August 2026. It browses websites in a cloud browser on behalf of a user, so robots.txt does not govern it.
What happens if you block GrokBot?
User-agent: GrokBot / Disallow: / without any SEO penalty. The trade-off is visibility inside Grok answers, if xAI does use this name.Is GrokBot safe?
Yes, GrokBot is safe to allow: there are no credible reports of it crawling abusively, and it is rated Safe in this directory. The caveat is proof, not behaviour. xAI has not published a crawler page, user-agent string or IP ranges for it, so anyone can send the name and robots.txt rules for it are best effort. If you want Grok to be able to cite you, allow it; if you do not, block it in robots.txt and at the server. For monitoring, see how do I monitor AI bot traffic to my website?
GrokBot user agent: is there an official xAI crawler string?
No. xAI has not published an official crawler user agent. The strings in circulation come from public registries: GrokBot, and Mozilla/5.0 (compatible; xAI-SearchBot/1.0; +https://x.ai) for xAI-SearchBot. Name both in robots.txt. The Grok crawler is not the same as Grok Bot, xAI's browsing agent, which robots.txt does not control.
User-agent: GrokBot
User-agent: xAI-SearchBot
Allow: /GrokBot vs xAI-SearchBot vs the Grok Bot agent
| Name | What it is | Evidence tier | Does robots.txt apply? |
|---|---|---|---|
| GrokBot | Reported crawler name linked to Grok answers | Reported | Yes, if the client honours it |
| xAI-SearchBot | Reported search crawler, xAI-SearchBot/1.0 | Reported | Yes, if the client honours it |
| Grok Bot (agent) | AI teammate that browses websites in a cloud browser for a user | Documented by xAI | No, it acts like a person |
| Twitterbot | X link previews, not Grok | Official | Yes |
| OAI-SearchBot | OpenAI's documented equivalent, for comparison | Official | Yes |
Because neither xAI crawler name is documented, blocking them by name is a preference that compliant clients will follow, not a guarantee. For enforcement, see spoofed AI crawler user agents.
What is Grok Bot, and can it visit my site?
Grok Bot is an AI agent product that xAI launched on 11 August 2026 (xAI launch post, 11 August 2026). According to the xAI Grok Bot documentation, each Bot works on "a persistent cloud computer with a browser, filesystem, and terminal" and "takes on multi-step work across apps and websites". So yes, it can visit your site, but as a browser acting for a person, not as a crawler.
- robots.txt does not govern it. Like ChatGPT Agent, it opens pages the way a person would, so a
User-agent: GrokBotrule does not stop it. - It comes from shared static IPs. The xAI Grok Bot security page says hosted computers use "shared static egress IP addresses", shared across all Grok Bot customers. xAI gives the ranges to customers through its account team; they are not published.
- It stops at human checks. xAI tells users to complete passwords, two-factor codes and CAPTCHAs themselves, and notes that some services block it because they "flag datacenter IP addresses".
Practical meaning: if you want agents like Grok Bot to complete bookings, sign-ups or purchases on your site, avoid putting a CAPTCHA or firewall challenge on those flows. If you want to stop them, use a firewall or bot-management rule, not robots.txt. The full comparison is in Grok Bot vs GrokBot; test a page with the AI Agent Readiness Check, or read can you block Grok Bot?.
GrokBot use cases
| If you are... | Recommended setting and why |
|---|---|
| You want to appear in Grok answers | Allow GrokBot and xAI-SearchBot, and keep public pages server-rendered. |
| You block AI but want X link previews | Block the xAI tokens; X link previews use Twitterbot, a separate token. |
| You only allow documented bots | Block both; neither has an operator specification yet. |
User-agent: GrokBot
User-agent: xAI-SearchBot
Disallow: /
User-agent: Twitterbot
Allow: /Confirm with a free AI crawler checker scan and read the Grok crawler user agent guide and the xAI operator page. Seeing these names in your logs? Paste them into the Grok log checker. Want Grok to cite you? Read how to get cited by Grok.
Test xAI crawler access
- AI crawler checker: tests 257 bots against your robots.txt in one scan.
- AI crawler diagnostic: fetches a page as each AI bot and names the layer that blocks it.
- robots.txt validator: shows allowed, blocked and partial status per bot.
- robots.txt generator: builds a file with presets for search, agents and training.
- batch URL checker: compares up to 20 sites side by side, with CSV export.
- llms.txt generator: writes the file AI assistants read for a site summary.
- schema markup generator: produces JSON-LD for eight common types.
- GEO audit: scores a page for generative engine optimization.
How to block GrokBot with robots.txt
<code>User-agent: GrokBot</code> - Matching is case-insensitive. Robots.txt is fetched from the root of each subdomain separately.
Is GrokBot safe to allow?
What does GrokBot do?
As reported by public registries; xAI has not confirmed any of this.
- Reported to fetch pages that Grok uses in its answers
- May supply web results for Grok search and DeepSearch
- Purpose, crawl rate and robots.txt behaviour are not documented by xAI
- Not the same as Grok Bot, xAI's browsing agent, which robots.txt does not control
- Not used for X (Twitter) link previews, which use Twitterbot
Frequently Asked Questions
Is GrokBot safe?
What is the official GrokBot user agent?
GrokBot is the name public registries report, so use it in robots.txt as the best available match. A second reported xAI name, xAI-SearchBot, should get its own rule.Is GrokBot the same as Grok Bot?
How do I block GrokBot in robots.txt?
User-agent: GrokBot User-agent: xAI-SearchBot Disallow: /
A client that honours robots.txt will stop. Because xAI has not documented either name, add a firewall or CDN rule if you need enforcement.
Will blocking GrokBot hurt my SEO?
Does GrokBot respect robots.txt?
How do I see if GrokBot visits my site?
grep -iE "GrokBot|xAI-SearchBot" /var/log/nginx/access.log. A match shows that something used the name. xAI publishes no IP ranges for it, so you cannot confirm the sender.