Mistral Search Index
SafeUser-agent: MistralAI-Index
Category: AI & LLM Bots
SEO impact if blocked: 2/10
Mistral AI
3 crawlers operated by Mistral AI
https://mistral.ai/Who Mistral AI is, for site owners
Mistral AI operates 3 crawlers in our directory, filed under AI & LLM Bots. Of those 3, 3 rated safe for crawl behaviour.
Blocking Mistral AI carries an SEO impact score of 2/10 across its crawlers. Every crawler here carries the same blocking impact: Low - No SEO ranking impact.
Our recommendation for Mistral AI, set by its least reassuring crawler (Mistral Search Index, rated safe): Generally safe to allow; provides legitimate crawling value.
Mistral AI at a glance
| Operator | Mistral AI |
|---|---|
| Crawlers in directory | 3 |
| Categories | AI & LLM Bots |
| SEO impact if blocked | 2/10 |
| Crawl behaviour | 3 rated safe |
| Website | mistral.ai |
| Official crawler docs | Mistral AI crawler documentation |
Crawlers operated by Mistral AI
What Mistral AI's crawlers are for
Mistral AI runs 3 crawlers in our database. Grouped by the job each one does:
- a retrieval crawler that feeds AI search answers:
MistralAI-Index - a training crawler:
MistralAI-Training - a user-triggered agent:
MistralAI-User
The highest-priority token is Tier 2. That puts Mistral AI among the operators whose crawlers have a direct effect on whether AI answers or search results can include your pages.
How site owners treat Mistral AI
None of Mistral AI's tokens were among the names tracked in our robots.txt study of the Tranco top 10,000, which usually means few large sites write rules for them yet. Your own logs are the best evidence of how often they visit.
Writing one policy for Mistral AI
A robots.txt file cannot say "Mistral AI"; it can only name Mistral AI's tokens, one group at a time. So "block Mistral AI" really means naming each of its 3 tokens, and you can give them different answers.
# Allow Mistral AI search and assistants, refuse training
User-agent: MistralAI-Training
Disallow: /
User-agent: MistralAI-Index
User-agent: MistralAI-User
Allow: /- Decide per job, not per company. Training, search and user-triggered fetching have different trade-offs, even from the same operator.
- Name every token you mean. A token you leave out follows
User-agent: *. - Repeat shared rules. A named group ignores the
*group, so copy anyDisallowlines you still want it to obey. - Verify from outside. Run the AI bot crawlability test after any change to see the result for each Mistral AI token, including firewall refusals.
Test yourself: does User-agent: Mistral AI block all of Mistral AI's crawlers?
No. robots.txt matches crawler tokens, not company names. Name each token: MistralAI-Index, MistralAI-Training, MistralAI-User.
Test yourself: if you block one Mistral AI token, are the others blocked too?
No. Each token is read separately. Blocking one leaves the rest following whatever group applies to them.
Each Mistral AI crawler, explained
A one-paragraph brief for every Mistral AI token in our database, covering what it does, how it treats robots.txt and how often large sites write rules for it:
- Mistral Search Index (
MistralAI-Index, Tier 2): a retrieval crawler that feeds AI search answers that builds a search or answer index. Rated safe. Mistral Search Index's operator says it follows robots.txt, so a correctly written Mistral Search Index group is enough to control it. Not tracked in our robots.txt study. - Mistral Training (
MistralAI-Training, Tier 2): a training crawler that collects pages to train AI models. Rated safe. Mistral Training's operator says it follows robots.txt, so a correctly written Mistral Training group is enough to control it. Not tracked in our robots.txt study. - MistralAI-User/1.0 (
MistralAI-User, Tier 2): a user-triggered agent that fetches a page when a person asks an assistant or app to open it. Rated safe. MistralAI-User/1.0's operator says it follows robots.txt, so a correctly written MistralAI-User/1.0 group is enough to control it. Not tracked in our robots.txt study.
Common mistakes with Mistral AI's crawlers
The errors we see most often with Mistral AI's crawlers in the files and firewalls we test:
- Blocking Mistral AI's crawlers while trying to stop AI training. Retrieval crawlers fetch pages so answers can cite them; blocking it can remove you from those answers without affecting training.
- Letting a firewall or bot-fight mode refuse Mistral AI's crawlers while robots.txt allows it. The file looks right, but the crawler still gets a 403 or a challenge page.
- Forgetting that once a
User-agent: MistralAI-Indexgroup exists, Mistral AI's crawlers ignores every rule underUser-agent: *.
All three hide inside a file that looks correct, so check Mistral AI's crawlers from outside after every change.
Spotting Mistral AI in your logs
robots.txt says what Mistral Search Index and Mistral Training and MistralAI-User/1.0 may do; your access logs show what they actually did.
Count requests
On a server with standard Nginx or Apache logs, this counts matching requests per day (adjust the log path for your host):
grep -iE "MistralAI-Index|MistralAI-Training|MistralAI-User" /var/log/nginx/access.log \
| awk '{print $4}' | cut -d: -f1 | sort | uniq -cAnd this shows which paths it requested most, which is the quickest way to see whether your Disallow lines are being respected:
grep -iE "MistralAI-Index|MistralAI-Training|MistralAI-User" /var/log/nginx/access.log \
| awk '{print $7}' | sort | uniq -c | sort -rn | head -20Check the requests are genuine
Anyone can send these user-agents, so confirm the source before acting on a match:
MistralAI-Index: we hold no published IP list, so use reverse DNS where Mistral AI documents a hostname and treat an unverifiable Mistral Search Index request with caution. Check it separately from Mistral Training and MistralAI-User/1.0.MistralAI-Training: we hold no published IP list, so use reverse DNS where Mistral AI documents a hostname and treat an unverifiable Mistral Training request with caution. Check it separately from Mistral Search Index and MistralAI-User/1.0.MistralAI-User: we hold no published IP list, so use reverse DNS where Mistral AI documents a hostname and treat an unverifiable MistralAI-User/1.0 request with caution. Check it separately from Mistral Search Index and Mistral Training.
What our own data shows about Mistral AI
Everything below comes from the AI Crawler Check bot database and our two most recent robots.txt scans of the Tranco top 10,000, not from Mistral AI's marketing. It covers 3 Mistral AI tokens in turn.
MistralAI-Index in our records
We first recorded MistralAI-Index on May 31, 2026. Mistral AI documents it on docs.mistral.ai, and we recheck that page before each release. No file in our top-10,000 scans names MistralAI-Index, so any rule you write for it puts you ahead of almost every large site.
Among the 104 AI and LLM crawlers in our database, MistralAI-Index sits 65th by priority (Tier 2), level with AddSearchBot, AI2Bot, AI2Bot-DeepResearchEval, aiHitBot. 81 of the 104 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for MistralAI-Index is usually a policy for these too.
A decision record for MistralAI-Index
Teams that review crawlers once a quarter keep one short entry per token, so anyone can see why a rule exists and when it was last checked. Filled in from our data, the MistralAI-Index entry reads:
- Operator and job: Mistral AI; it builds a search or answer index.
- Priority and behaviour: Tier 2, rated safe, blocking impact low.
- Suggested starting point: Allow, and watch it in your logs.
- Enforcement: MistralAI-Index's operator says it follows robots.txt, so a correctly written MistralAI-Index group is enough to control it.
User-agent: MistralAI-Index
Allow: /Paste the line into the validator with one of your own URLs before it goes live. Revisit the MistralAI-Index entry whenever Mistral AI announces a change, because a Tier 2 crawler affects visibility directly.
MistralAI-Training in our records
We first recorded MistralAI-Training on September 22, 2026. Mistral AI documents it on docs.mistral.ai, and we recheck that page before each release. No file in our top-10,000 scans names MistralAI-Training, so any rule you write for it puts you ahead of almost every large site.
Among the 104 AI and LLM crawlers in our database, MistralAI-Training sits 66th by priority (Tier 2), level with AddSearchBot, AI2Bot, AI2Bot-DeepResearchEval, aiHitBot. 81 of the 104 share its safe rating. Other operators with a crawler that collects pages to train AI models: ClaudeBot (Anthropic, Tier 1), Google-Extended (Google, Tier 1), GPTBot (OpenAI, Tier 1), Meta-ExternalAgent (Meta, Tier 1), AI2Bot (Allen Institute for AI, Tier 2). A policy for MistralAI-Training is usually a policy for these too.
MistralAI-User in our records
We first recorded MistralAI-User on March 17, 2026. Mistral AI documents it on docs.mistral.ai, and we recheck that page before each release. No file in our top-10,000 scans names MistralAI-User, so any rule you write for it puts you ahead of almost every large site.
Among the 104 AI and LLM crawlers in our database, MistralAI-User sits 67th by priority (Tier 2), level with AddSearchBot, AI2Bot, AI2Bot-DeepResearchEval, aiHitBot. 81 of the 104 share its safe rating. Other operators with a crawler that fetches a page when a person asks an assistant or app to open it: ChatGPT Agent (OpenAI, Tier 1), ChatGPT Operator (OpenAI, Tier 1), ChatGPT-User (OpenAI, Tier 1), ChatGPT-User/2.0 (OpenAI, Tier 1), Claude-SearchBot (Anthropic, Tier 1). A policy for MistralAI-User is usually a policy for these too.
How top sites treat Mistral AI's crawlers
From our robots.txt study of the Tranco top 10,000 (5,583 valid files, fetched October 20, 2026):
None of the tokens for Mistral AI's crawlers is tracked by name in our study. For scale, 20.1% of top-site files block at least one AI training crawler and 4.3% refuse every unnamed crawler with User-agent: * and Disallow: /, which would catch Mistral AI's crawlers too.
More questions about Mistral AI
- Will blocking Mistral AI affect my Google rankings?
- Only if one of its tokens is a Google search crawler. None of Mistral AI's tokens are Googlebot, so classic Google rankings are unaffected.
- How quickly does a robots.txt change take effect for Mistral AI?
- Crawlers cache robots.txt, commonly for up to a day. Expect a change to apply within about 24 hours, and confirm it in your logs.
- Can I allow Mistral AI on some sections and block it on others?
- Yes. Inside the group for
MistralAI-Index, addDisallowlines for the paths to keep private andAllowlines for exceptions. The longest matching rule wins. - Does blocking Mistral AI delete content it already collected?
- No. robots.txt only controls future visits. For content already collected, check whether Mistral AI offers a removal or opt-out process.
- Is a firewall block better than robots.txt for Mistral AI?
- A firewall enforces the decision, while robots.txt asks politely. Many operators recommend robots.txt because blocking their IPs can stop them reading your rules at all. Use robots.txt first, and add a firewall rule only if requests continue.
A five-minute Mistral AI review
- Confirm the token. Compare the user-agent in your logs with the tokens listed on this page. A close spelling is not the same crawler.
- Decide the outcome per crawler. Use the briefs above to choose allow, limit or block for each token.
- Write the group. Add or edit one
User-agentgroup per decision and repeat any shared rules inside it. - Publish and wait. Allow about a day for crawlers to fetch the new file.
- Confirm the result. Check your logs, then run the AI crawler access test to see the outcome from outside, including any firewall layer.
How to block or allow Mistral AI in robots.txt
These rules cover all 3 crawlers Mistral AI operates. Add them to the robots.txt file at the root of your domain.
Block Mistral AI completely
User-agent: MistralAI-Index
User-agent: MistralAI-Training
User-agent: MistralAI-User
Disallow: /
Allow Mistral AI explicitly
User-agent: MistralAI-Index
User-agent: MistralAI-Training
User-agent: MistralAI-User
Allow: /
These Mistral AI rules ask; they do not enforce. If a Mistral AI token ignores them, block it at the server or CDN. Paths are case-sensitive, and each rule matches the token, not the full user-agent string.
Frequently asked questions about Mistral AI
What crawlers does Mistral AI run?
Mistral AI operates 3 crawlers in our directory: Mistral Search Index (user-agent MistralAI-Index), Mistral Training (user-agent MistralAI-Training), MistralAI-User/1.0 (user-agent MistralAI-User).
Should I block Mistral AI?
Generally safe to allow; provides legitimate crawling value. That guidance follows this operator's least reassuring crawler, Mistral Search Index, rated safe, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 2/10, where 10 means blocking costs you the most visibility.
How do I block Mistral AI in robots.txt?
Add the following to robots.txt at your domain root: User-agent: MistralAI-Index then User-agent: MistralAI-Training then User-agent: MistralAI-User then Disallow: /. One Disallow line covers all 3 user-agent tokens.
Does blocking Mistral AI hurt my search rankings?
Every crawler here carries the same blocking impact: Low - No SEO ranking impact. Google's own ranking crawler is Googlebot, which this operator does not run, so blocking Mistral AI does not by itself remove you from Google's index. What it can change is whether AI assistants and analytics tools can read your pages.