Exa AI Search
Operated by Exa
Quick Facts
- User-Agent:
- ExaBot
- Category:
- AI & LLM Bots
- Operator:
- Exa
- Safety:
- Safe
- Blocking Impact:
- Low - No SEO ranking impact
- SEO Impact Score:
- 2/10
- Official docs:
- None published by Exa
Reported, not operator documented
Exa has not published a crawler page that names ExaBot. It is listed here on our own evidence. Verified by AI Crawler Check: 133 of the 5,583 valid robots.txt files in the Tranco top 10,000 carry a rule for ExaBot, so site owners treat it as a real crawler name (how we measure this). That proves the spelling is in use, not who sends it. Reported earlier Exa token; Exa now documents ExaSearchBot.
What that means for you: the ExaBot rule on this page is the best available spelling, but Exa has made no statement of purpose, crawl rate or robots.txt compliance. Treat a ExaBot request in your logs as a clue, not proof of who sent it, and add a firewall rule if that traffic matters to you. If the operator publishes documentation, this profile moves to Official docs.
What is Exa AI Search?
Exa AI Search is an AI search-index crawler operated by Exa. It collects and indexes web content so it can be surfaced as citations or answers inside Exa's AI search experience. Allowing Exa AI Search can increase your visibility in AI-powered search results, which is why most sites choose to permit it.
Exa AI Search is an AI search-index crawler operated by Exa. It collects and indexes web content so it can be surfaced as citations or answers inside Exa's AI search experience. Allowing Exa AI Search can increase your visibility in AI-powered search results, which is why most sites choose to permit it.
Exa AI Search uses the user-agent token ExaBot. You can control it via robots.txt, meta tags (noai), or the emerging llms.txt standard. Robots.txt is voluntary; for hard enforcement, combine it with server-level IP blocking.
What happens if you block Exa AI Search?
User-agent / Disallow: / without any SEO penalty.ExaBot vs ExaSearchBot: which token Exa actually documents
| Question | ExaBot | ExaSearchBot |
|---|---|---|
| Documented by Exa? | No (Reported) | Yes (Official) |
| Request signing | Unknown | Web Bot Auth, RFC 9421 signatures |
| If robots.txt has no group for it | Unknown | Follows the rules you set for major search engines, otherwise * |
| Removing an indexed page | Not documented | Serve noindex; Exa removes it |
| Index scale | Not stated | 1.4 trillion URLs, 100 billion pages (Exa) |
That fallback matters: if you have a Googlebot group and no Exa group, Exa applies your Googlebot rules. To treat Exa differently, give ExaSearchBot its own group.
ExaBot use cases and a safe robots.txt
| If you are... | Recommended setting and why |
|---|---|
| A publisher who wants AI-app citations | Allow ExaSearchBot. Exa powers search inside many AI agents and developer tools, so its index is a retrieval source, not a training crawl. |
| A site that blocks all AI | Add an ExaSearchBot group with Disallow: /. A Googlebot-only allow list would otherwise let Exa in through the search-engine fallback. |
| Seeing "ExaBot" in logs | Check the signature or reverse DNS before trusting it; an unsigned request claiming to be Exa may be spoofed. |
User-agent: ExaSearchBot
Allow: /
Disallow: /account/Confirm the group with a crawler check and compare Exa with other AI search indexes in the Exa operator page.
Check Exa access on your site
- AI crawler checker: tests 265 bots against your robots.txt in one scan.
- AI crawler diagnostic: fetches a page as each AI bot and names the layer that blocks it.
- robots.txt validator: shows allowed, blocked and partial status per bot.
- robots.txt generator: builds a file with presets for search, agents and training.
- batch URL checker: compares up to 20 sites side by side, with CSV export.
- llms.txt generator: writes the file AI assistants read for a site summary.
- schema markup generator: produces JSON-LD for eight common types.
- GEO audit: scores a page for generative engine optimization.
Want to see whether Exa AI Search actually visits you? Read how do I monitor AI bot traffic to my website?
How to block Exa AI Search with robots.txt
<code>User-agent: ExaBot</code> - Matching is case-insensitive. Robots.txt is fetched from the root of each subdomain separately.
Is Exa AI Search safe to allow?
What does Exa AI Search do?
Understanding Exa AI Search's purpose helps you decide whether to allow or block it.
- Indexes web pages for AI-powered search answers
- Surfaces your content as citations inside AI search results
- Builds a knowledge index used to answer user questions
- Allowing it can increase visibility in AI search
- robots.txt behaviour not documented by the operator
Exa AI Search in context
Exa AI Search is a retrieval crawler that feeds AI search answers operated by Exa. In plain terms, it builds a search or answer index. Our database files it under AI and LLM crawlers and ranks it Tier 2, important crawlers from established AI and search operators. 74 of the 104 AI and LLM crawlers in our database share this tier.
Its safety rating is safe and its blocking impact is low. So the decision about Exa AI Search rests on what you want from Exa, not on its behaviour. How we rate safety and impact.
How site owners treat Exa AI Search
In our own scan of the Tranco top 10,000 (5,583 valid robots.txt files), 133 files (2.4%) name ExaBot in a User-agent line. That shows site owners treat the token as a real crawler worth a rule. It does not, on its own, prove who operates it.
Does it follow robots.txt?
Exa AI Search's operator does not clearly document how it treats robots.txt. Write the Exa AI Search rule anyway, then confirm in your logs that its requests stop.
Exa crawlers and close alternatives
Exa AI Search is one of 2 Exa crawlers in our database. A User-agent: ExaBot group controls Exa AI Search only; the other Exa tokens below need their own lines.
ExaBot: this crawler (Tier 2)ExaSearchBot: Exa Search, builds a search or answer index (Tier 2)
To give Exa AI Search the same answer as its siblings, stack User-agent: ExaBot with their lines above one set of rules; to treat Exa AI Search differently, give it its own group.
Other AI and LLM crawlers that do the same job (builds a search or answer index) include AddSearchBot from AddSearch, AIWebIndex from Lyrenth, atlassian-bot from Atlassian, AzureAI-SearchBot from Microsoft, Cloudflare-AI-Search from Cloudflare, Cloudflare-AutoRAG from Cloudflare. Deciding about Exa AI Search is usually a decision about the whole group, so check how your file treats these too.
We first recorded ExaBot on June 18, 2026. We could not find a page where Exa names it, so treat the token as reported rather than documented.
Which rule Exa AI Search actually follows
Exa AI Search reads exactly one group in robots.txt, the most specific one that names ExaBot, and never combines rules across groups. This short file shows the trap:
User-agent: *
Disallow: /private/
User-agent: ExaBot
Disallow: /drafts/- Find the group. Exa AI Search looks for a group naming
ExaBot. One exists, so it ignoresUser-agent: *completely. - Apply only that group. It is refused
/drafts/and nothing else. - The surprise.
/private/is open to Exa AI Search, because that rule lives in the*group it no longer reads. To keep it out of both, repeatDisallow: /private/inside its own group.
Test yourself: does a rule under User-agent: * apply to Exa AI Search if the file also has a User-agent: ExaBot group?
No. Once a group names it, Exa AI Search reads only that group. Put every rule you want it to follow inside its own group.
Test yourself: will blocking Exa AI Search remove your pages from AI answers?
Often, yes. Exa AI Search is a retrieval crawler that feeds AI search answers, so pages it cannot fetch can drop out of the answers or results it supports.
Test yourself: is User-agent: exabot the same as User-agent: ExaBot?
Yes. RFC 9309 makes user-agent matching case-insensitive. Paths are a different story: Disallow: /Blog/ does not block /blog/.
Should you allow Exa AI Search? Decide by situation
| If you are... | Recommended setting |
|---|---|
| You publish content and want AI answers to cite it | Allow it. Exa AI Search fetches pages for answers, so blocking it can remove you from them. |
| You license content and want to stop AI training | Blocking it does not stop training, because Exa AI Search is a retrieval crawler that feeds AI search answers. Block the operator's training crawler instead. |
| You run an online shop | Allow it on product and category pages, which are the pages shoppers ask assistants about. |
| Your server is under load from bots | Rate-limit before blocking. It is rated safe in our database. |
Check Exa AI Search on your own site
- Open
/robots.txton your domain and search forExaBot. If it is not named, it followsUser-agent: *. - List the
Disallowlines in the group Exa AI Search follows and test your key URLs against them; the longest match wins, andAllowwins a tie. - Look beyond the file. A firewall or bot manager can refuse Exa AI Search even when robots.txt allows it, and that never shows in the file.
- Run our crawler access checker to see how Exa AI Search and every other crawler we track are treated in one pass, including firewall refusals.
Common mistakes with Exa AI Search
The errors we see most often with Exa AI Search in the files and firewalls we test:
- Blocking Exa AI Search while trying to stop AI training. Retrieval crawlers fetch pages so answers can cite them; blocking it can remove you from those answers without affecting training.
- Letting a firewall or bot-fight mode refuse Exa AI Search while robots.txt allows it. The file looks right, but the crawler still gets a 403 or a challenge page.
- Forgetting that once a
User-agent: ExaBotgroup exists, Exa AI Search ignores every rule underUser-agent: *.
All three hide inside a file that looks correct, so check Exa AI Search from outside after every change.
Spotting Exa AI Search in your logs
robots.txt says what Exa AI Search may do; your access logs show what it actually did.
Count requests
On a server with standard Nginx or Apache logs, this counts matching requests per day (adjust the log path for your host):
grep -iE "ExaBot" /var/log/nginx/access.log \
| awk '{print $4}' | cut -d: -f1 | sort | uniq -cAnd this shows which paths it requested most, which is the quickest way to see whether your Disallow lines are being respected:
grep -iE "ExaBot" /var/log/nginx/access.log \
| awk '{print $7}' | sort | uniq -c | sort -rn | head -20Check the requests are genuine
Anyone can send the Exa AI Search user-agent, so confirm the source before acting on a match:
ExaBot: we hold no published IP list, so use reverse DNS where the operator documents a hostname and treat an unverifiable Exa AI Search request with caution.
Frequently Asked Questions
What user-agent string does Exa AI Search use?
ExaBot. Evidence that the spelling is in real use comes from our own robots.txt study: 133 of the 5,583 valid robots.txt files in the Tranco top 10,000 name it (fetched October 20, 2026), not from Exa, which has not published crawler documentation naming it. Use it in robots.txt as the best available match, and treat a request carrying it as a clue rather than proof of who sent it.Is Exa AI Search safe?
Will blocking Exa AI Search hurt my SEO?
User-agent / Disallow: / without any SEO penalty.How do I block Exa AI Search in robots.txt?
/robots.txt file:User-agent: ExaBot Disallow: /
This instructs Exa AI Search not to crawl any path on your site. To block only specific sections, replace / with the path (e.g.,
Disallow: /blog/).Does Exa AI Search respect robots.txt?
ExaBot will stop a client that chooses to honour it; for enforcement, add a firewall or CDN rule as well.How do I verify if Exa AI Search is crawling my site?
ExaBot (for example grep -i "ExaBot" /var/log/nginx/access.log). Because the operator publishes no IP ranges or reverse-DNS pattern, a match shows that something used the name, not that Exa sent it.