Exa AI Search
SafeUser-agent: ExaBot
Category: AI & LLM Bots
SEO impact if blocked: 2/10
Exa
2 crawlers operated by Exa
https://exa.ai/Who Exa is, for site owners
Exa operates 2 crawlers in our directory, filed under AI & LLM Bots. Of those 2, 2 rated safe for crawl behaviour.
Blocking Exa carries an SEO impact score of 2/10 across its crawlers. Every crawler here carries the same blocking impact: Low - No SEO ranking impact.
Our recommendation for Exa, set by its least reassuring crawler (Exa AI Search, rated safe): Generally safe to allow; provides legitimate crawling value.
Exa at a glance
| Operator | Exa |
|---|---|
| Crawlers in directory | 2 |
| Categories | AI & LLM Bots |
| SEO impact if blocked | 2/10 |
| Crawl behaviour | 2 rated safe |
| Website | exa.ai |
| Official crawler docs | Exa crawler documentation |
Crawlers operated by Exa
What Exa's crawlers are for
Exa runs 2 crawlers in our database. Grouped by the job each one does:
- a retrieval crawler that feeds AI search answers:
ExaBot,ExaSearchBot
The highest-priority token is Tier 2. That puts Exa among the operators whose crawlers have a direct effect on whether AI answers or search results can include your pages.
How site owners treat Exa
How often site owners write rules for Exa's tokens, from our own robots.txt study of the Tranco top 10,000:
| Token | Files naming it | Files blocking it |
|---|---|---|
ExaBot | 133 (2.4%) | Not measured |
A Exa token can be blocked without being named, through a User-agent: * group with Disallow: /, so its blocking share can be higher than its naming share.
All counts come from 5,583 valid files fetched October 20, 2026.
Writing one policy for Exa
A robots.txt file cannot say "Exa"; it can only name Exa's tokens, one group at a time. So "block Exa" really means naming each of its 2 tokens, and you can give them different answers.
# Allow Exa search and assistants, refuse training
User-agent: ExaBot
Disallow: /
User-agent: ExaBot
User-agent: ExaSearchBot
Allow: /- Decide per job, not per company. Training, search and user-triggered fetching have different trade-offs, even from the same operator.
- Name every token you mean. A token you leave out follows
User-agent: *. - Repeat shared rules. A named group ignores the
*group, so copy anyDisallowlines you still want it to obey. - Verify from outside. Run the GPTBot and ClaudeBot access check after any change to see the result for each Exa token, including firewall refusals.
Test yourself: does User-agent: Exa block all of Exa's crawlers?
No. robots.txt matches crawler tokens, not company names. Name each token: ExaBot, ExaSearchBot.
Test yourself: if you block one Exa token, are the others blocked too?
No. Each token is read separately. Blocking one leaves the rest following whatever group applies to them.
Each Exa crawler, explained
A one-paragraph brief for every Exa token in our database, covering what it does, how it treats robots.txt and how often large sites write rules for it:
- Exa AI Search (
ExaBot, Tier 2): a retrieval crawler that feeds AI search answers that builds a search or answer index. Rated safe. Exa AI Search's operator does not clearly document how it treats robots.txt. Write the Exa AI Search rule anyway, then confirm in your logs that its requests stop. 2.4% of top-site robots.txt files name it. - Exa Search (
ExaSearchBot, Tier 2): a retrieval crawler that feeds AI search answers that builds a search or answer index. Rated safe. Exa Search's operator does not clearly document how it treats robots.txt. Write the Exa Search rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
Common mistakes with Exa's crawlers
The errors we see most often with Exa's crawlers in the files and firewalls we test:
- Blocking Exa's crawlers while trying to stop AI training. Retrieval crawlers fetch pages so answers can cite them; blocking it can remove you from those answers without affecting training.
- Letting a firewall or bot-fight mode refuse Exa's crawlers while robots.txt allows it. The file looks right, but the crawler still gets a 403 or a challenge page.
- Forgetting that once a
User-agent: ExaBotgroup exists, Exa's crawlers ignores every rule underUser-agent: *.
All three hide inside a file that looks correct, so check Exa's crawlers from outside after every change.
Spotting Exa in your logs
robots.txt says what Exa AI Search and Exa Search may do; your access logs show what they actually did.
Count requests
On a server with standard Nginx or Apache logs, this counts matching requests per day (adjust the log path for your host):
grep -iE "ExaBot|ExaSearchBot" /var/log/nginx/access.log \
| awk '{print $4}' | cut -d: -f1 | sort | uniq -cAnd this shows which paths it requested most, which is the quickest way to see whether your Disallow lines are being respected:
grep -iE "ExaBot|ExaSearchBot" /var/log/nginx/access.log \
| awk '{print $7}' | sort | uniq -c | sort -rn | head -20Check the requests are genuine
Anyone can send these user-agents, so confirm the source before acting on a match:
ExaBot: we hold no published IP list, so use reverse DNS where Exa documents a hostname and treat an unverifiable Exa AI Search request with caution. Check it separately from Exa Search.ExaSearchBot: we hold no published IP list, so use reverse DNS where Exa documents a hostname and treat an unverifiable Exa Search request with caution. Check it separately from Exa AI Search.
What our own data shows about Exa
Everything below comes from the AI Crawler Check bot database and our two most recent robots.txt scans of the Tranco top 10,000, not from Exa's marketing. It covers 2 Exa tokens in turn.
ExaBot in our records
We first recorded ExaBot on June 18, 2026. We could not find a page where Exa names it, so treat the token as reported rather than documented. ExaBot is outside our tracked token set, but 133 of 5,583 top-site files already name it, which is enough to show owners treat it as a real crawler.
Among the 104 AI and LLM crawlers in our database, ExaBot sits 40th by priority (Tier 2), level with AddSearchBot, AI2Bot, AI2Bot-DeepResearchEval, aiHitBot. 81 of the 104 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for ExaBot is usually a policy for these too.
A decision record for ExaBot
Teams that review crawlers once a quarter keep one short entry per token, so anyone can see why a rule exists and when it was last checked. Filled in from our data, the ExaBot entry reads:
- Operator and job: Exa; it builds a search or answer index.
- Priority and behaviour: Tier 2, rated safe, blocking impact low.
- Suggested starting point: Allow, and watch it in your logs.
- Enforcement: ExaBot's operator does not clearly document how it treats robots.txt. Write the ExaBot rule anyway, then confirm in your logs that its requests stop.
User-agent: ExaBot
Allow: /Paste the line into the validator with one of your own URLs before it goes live. Revisit the ExaBot entry whenever Exa announces a change, because a Tier 2 crawler affects visibility directly.
ExaSearchBot in our records
We first recorded ExaSearchBot on September 22, 2026. Exa documents it on exa.ai, and we recheck that page before each release. No file in our top-10,000 scans names ExaSearchBot, so any rule you write for it puts you ahead of almost every large site.
Among the 104 AI and LLM crawlers in our database, ExaSearchBot sits 41th by priority (Tier 2), level with AddSearchBot, AI2Bot, AI2Bot-DeepResearchEval, aiHitBot. 81 of the 104 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for ExaSearchBot is usually a policy for these too.
A decision record for ExaSearchBot
Teams that review crawlers once a quarter keep one short entry per token, so anyone can see why a rule exists and when it was last checked. Filled in from our data, the ExaSearchBot entry reads:
- Operator and job: Exa; it builds a search or answer index.
- Priority and behaviour: Tier 2, rated safe, blocking impact low.
- Suggested starting point: Allow, and watch it in your logs.
- Enforcement: ExaSearchBot's operator does not clearly document how it treats robots.txt. Write the ExaSearchBot rule anyway, then confirm in your logs that its requests stop.
User-agent: ExaSearchBot
Allow: /Paste the line into the validator with one of your own URLs before it goes live. Revisit the ExaSearchBot entry whenever Exa announces a change, because a Tier 2 crawler affects visibility directly.
How top sites treat Exa's crawlers
From our robots.txt study of the Tranco top 10,000 (5,583 valid files, fetched October 20, 2026):
None of the tokens for Exa's crawlers is tracked by name in our study. For scale, 20.1% of top-site files block at least one AI training crawler and 4.3% refuse every unnamed crawler with User-agent: * and Disallow: /, which would catch Exa's crawlers too.
More questions about Exa
- Will blocking Exa affect my Google rankings?
- Only if one of its tokens is a Google search crawler. None of Exa's tokens are Googlebot, so classic Google rankings are unaffected.
- How quickly does a robots.txt change take effect for Exa?
- Crawlers cache robots.txt, commonly for up to a day. Expect a change to apply within about 24 hours, and confirm it in your logs.
- Can I allow Exa on some sections and block it on others?
- Yes. Inside the group for
ExaBot, addDisallowlines for the paths to keep private andAllowlines for exceptions. The longest matching rule wins. - Does blocking Exa delete content it already collected?
- No. robots.txt only controls future visits. For content already collected, check whether Exa offers a removal or opt-out process.
- Is a firewall block better than robots.txt for Exa?
- A firewall enforces the decision, while robots.txt asks politely. Many operators recommend robots.txt because blocking their IPs can stop them reading your rules at all. Use robots.txt first, and add a firewall rule only if requests continue.
A five-minute Exa review
- Confirm the token. Compare the user-agent in your logs with the tokens listed on this page. A close spelling is not the same crawler.
- Decide the outcome per crawler. Use the briefs above to choose allow, limit or block for each token.
- Write the group. Add or edit one
User-agentgroup per decision and repeat any shared rules inside it. - Publish and wait. Allow about a day for crawlers to fetch the new file.
- Confirm the result. Check your logs, then run the blocked AI crawler check to see the outcome from outside, including any firewall layer.
How to block or allow Exa in robots.txt
These rules cover all 2 crawlers Exa operates. Add them to the robots.txt file at the root of your domain.
Block Exa completely
User-agent: ExaBot
User-agent: ExaSearchBot
Disallow: /
Allow Exa explicitly
User-agent: ExaBot
User-agent: ExaSearchBot
Allow: /
These Exa rules ask; they do not enforce. If a Exa token ignores them, block it at the server or CDN. Paths are case-sensitive, and each rule matches the token, not the full user-agent string.
Frequently asked questions about Exa
What crawlers does Exa run?
Exa operates 2 crawlers in our directory: Exa AI Search (user-agent ExaBot), Exa Search (user-agent ExaSearchBot).
Should I block Exa?
Generally safe to allow; provides legitimate crawling value. That guidance follows this operator's least reassuring crawler, Exa AI Search, rated safe, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 2/10, where 10 means blocking costs you the most visibility.
How do I block Exa in robots.txt?
Add the following to robots.txt at your domain root: User-agent: ExaBot then User-agent: ExaSearchBot then Disallow: /. One Disallow line covers all 2 user-agent tokens.
Does blocking Exa hurt my search rankings?
Every crawler here carries the same blocking impact: Low - No SEO ranking impact. Google's own ranking crawler is Googlebot, which this operator does not run, so blocking Exa does not by itself remove you from Google's index. What it can change is whether AI assistants and analytics tools can read your pages.