Baidu

10 crawlers operated by Baidu

https://www.baidu.com/

Who Baidu is, for site owners

Baidu operates 10 crawlers in our directory, filed under Search Engines and AI & LLM Bots. Of those 10, 8 rated safe, 2 rated caution for crawl behaviour.

Blocking Baidu carries an SEO impact score of 2/10 to 10/10 across its crawlers. 5 of them share the highest blocking impact here: Critical - Blocking removes you from search results.

Our recommendation for Baidu, set by its least reassuring crawler (ERNIE (Baidu), rated caution): Generally safe to allow; provides legitimate crawling value.

Baidu at a glance

OperatorBaidu
Crawlers in directory10
CategoriesSearch Engines, AI & LLM Bots
SEO impact if blocked2/10 to 10/10
Crawl behaviour8 rated safe, 2 rated caution
Websitewww.baidu.com
Official crawler docsBaidu crawler documentation

Crawlers operated by Baidu

What Baidu's crawlers are for

Baidu runs 10 crawlers in our database. Grouped by the job each one does:

  • a search engine crawler: baidu, Baiduspider-favo, Baiduspider-image, Baiduspider-news, Baiduspider-video, Baiduspider
  • a utility crawler: Baiduspider-ads, Baiduspider-cpro
  • a training crawler: ERNIEBot
  • a user-triggered agent: YiyanBot

The highest-priority token is Tier 2. That puts Baidu among the operators whose crawlers have a direct effect on whether AI answers or search results can include your pages.

How site owners treat Baidu

How often site owners write rules for Baidu's tokens, from our own robots.txt study of the Tranco top 10,000:

Baidu tokens named and blocked in top-site robots.txt files
TokenFiles naming itFiles blocking it
ERNIEBot38 (0.7%)Not measured
YiyanBot20 (0.4%)Not measured

A Baidu token can be blocked without being named, through a User-agent: * group with Disallow: /, so its blocking share can be higher than its naming share.

All counts come from 5,583 valid files fetched October 20, 2026.

Writing one policy for Baidu

A robots.txt file cannot say "Baidu"; it can only name Baidu's tokens, one group at a time. So "block Baidu" really means naming each of its 10 tokens, and you can give them different answers.

# Allow Baidu search and assistants, refuse training
User-agent: ERNIEBot
Disallow: /

User-agent: baidu
User-agent: Baiduspider-favo
User-agent: Baiduspider-ads
User-agent: Baiduspider-image
User-agent: Baiduspider-news
User-agent: Baiduspider-cpro
User-agent: Baiduspider-video
User-agent: Baiduspider
User-agent: YiyanBot
Allow: /
  1. Decide per job, not per company. Training, search and user-triggered fetching have different trade-offs, even from the same operator.
  2. Name every token you mean. A token you leave out follows User-agent: *.
  3. Repeat shared rules. A named group ignores the * group, so copy any Disallow lines you still want it to obey.
  4. Verify from outside. Run the website crawlability test after any change to see the result for each Baidu token, including firewall refusals.
Test yourself: does User-agent: Baidu block all of Baidu's crawlers?

No. robots.txt matches crawler tokens, not company names. Name each token: baidu, Baiduspider-favo, Baiduspider-ads, Baiduspider-image, Baiduspider-news, Baiduspider-cpro, Baiduspider-video, Baiduspider, ERNIEBot, YiyanBot.

Test yourself: if you block one Baidu token, are the others blocked too?

No. Each token is read separately. Blocking one leaves the rest following whatever group applies to them.

Each Baidu crawler, explained

A one-paragraph brief for every Baidu token in our database, covering what it does, how it treats robots.txt and how often large sites write rules for it:

  • baidu (baidu, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. baidu's operator does not clearly document how it treats robots.txt. Write the baidu rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu Bookmark (Baiduspider-favo, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. Baidu Bookmark's operator does not clearly document how it treats robots.txt. Write the Baidu Bookmark rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu Business Search (Baiduspider-ads, Tier 3): a utility crawler that checks pages for an SEO, uptime or analytics service. Rated safe. Baidu Business Search's operator does not clearly document how it treats robots.txt. Write the Baidu Business Search rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu Image Search (Baiduspider-image, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. Baidu Image Search's operator does not clearly document how it treats robots.txt. Write the Baidu Image Search rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu News Search (Baiduspider-news, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. Baidu News Search's operator does not clearly document how it treats robots.txt. Write the Baidu News Search rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu Union (Baiduspider-cpro, Tier 3): a utility crawler that checks pages for an SEO, uptime or analytics service. Rated safe. Baidu Union's operator does not clearly document how it treats robots.txt. Write the Baidu Union rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baidu Video Search (Baiduspider-video, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. Baidu Video Search's operator does not clearly document how it treats robots.txt. Write the Baidu Video Search rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • Baiduspider (Baiduspider, Tier 3): a search engine crawler that builds a search or answer index. Rated safe. Baiduspider's operator does not clearly document how it treats robots.txt. Write the Baiduspider rule anyway, then confirm in your logs that its requests stop. Not tracked in our robots.txt study.
  • ERNIE (Baidu) (ERNIEBot, Tier 2): a training crawler that collects pages to train AI models. Rated caution. ERNIE (Baidu)'s operator does not clearly document how it treats robots.txt. Write the ERNIE (Baidu) rule anyway, then confirm in your logs that its requests stop. 0.7% of top-site robots.txt files name it.
  • Yiyan Fetcher (Baidu) (YiyanBot, Tier 2): a user-triggered agent that fetches a page when a person asks an assistant or app to open it. Rated caution. Yiyan Fetcher (Baidu)'s operator does not clearly document how it treats robots.txt. Write the Yiyan Fetcher (Baidu) rule anyway, then confirm in your logs that its requests stop. 0.4% of top-site robots.txt files name it.

Common mistakes with Baidu's crawlers

The errors we see most often with Baidu's crawlers in the files and firewalls we test:

  1. Blocking Baidu's crawlers to save bandwidth and then wondering why pages drop out of that search engine. Use crawl-rate settings or caching first.
  2. Writing a User-agent: baidu group to add one rule for Baidu's crawlers and accidentally dropping every rule it used to follow under User-agent: *.
  3. Disallowing CSS and JavaScript folders, which can stop Baidu's crawlers rendering your pages properly.

All three hide inside a file that looks correct, so check Baidu's crawlers from outside after every change.

Spotting Baidu in your logs

robots.txt says what baidu and Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Union and Baidu Video Search and Baiduspider may do; your access logs show what they actually did.

Count requests

On a server with standard Nginx or Apache logs, this counts matching requests per day (adjust the log path for your host):

grep -iE "baidu|Baiduspider-favo|Baiduspider-ads|Baiduspider-image|Baiduspider-news|Baiduspider-cpro|Baiduspider-video|Baiduspider" /var/log/nginx/access.log \
  | awk '{print $4}' | cut -d: -f1 | sort | uniq -c

And this shows which paths it requested most, which is the quickest way to see whether your Disallow lines are being respected:

grep -iE "baidu|Baiduspider-favo|Baiduspider-ads|Baiduspider-image|Baiduspider-news|Baiduspider-cpro|Baiduspider-video|Baiduspider" /var/log/nginx/access.log \
  | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

Check the requests are genuine

Anyone can send these user-agents, so confirm the source before acting on a match:

  • baidu: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable baidu request with caution. Check it separately from Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Union and Baidu Video Search and Baiduspider.
  • Baiduspider-favo: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu Bookmark request with caution. Check it separately from baidu and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Union and Baidu Video Search and Baiduspider.
  • Baiduspider-ads: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu Business Search request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Image Search and Baidu News Search and Baidu Union and Baidu Video Search and Baiduspider.
  • Baiduspider-image: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu Image Search request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Business Search and Baidu News Search and Baidu Union and Baidu Video Search and Baiduspider.
  • Baiduspider-news: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu News Search request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu Union and Baidu Video Search and Baiduspider.
  • Baiduspider-cpro: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu Union request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Video Search and Baiduspider.
  • Baiduspider-video: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baidu Video Search request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Union and Baiduspider.
  • Baiduspider: we hold no published IP list, so use reverse DNS where Baidu documents a hostname and treat an unverifiable Baiduspider request with caution. Check it separately from baidu and Baidu Bookmark and Baidu Business Search and Baidu Image Search and Baidu News Search and Baidu Union and Baidu Video Search.

Related: how AI engines choose their sources

What our own data shows about Baidu

Everything below comes from the AI Crawler Check bot database and our two most recent robots.txt scans of the Tranco top 10,000, not from Baidu's marketing. It covers 6 Baidu tokens in turn.

baidu in our records

We first recorded baidu on March 17, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names baidu, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, baidu sits fifth by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, Baiduspider. 62 of the 63 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for baidu is usually a policy for these too.

A decision record for baidu

Teams that review crawlers once a quarter keep one short entry per token, so anyone can see why a rule exists and when it was last checked. Filled in from our data, the baidu entry reads:

  • Operator and job: Baidu; it builds a search or answer index.
  • Priority and behaviour: Tier 3, rated safe, blocking impact critical.
  • Suggested starting point: Allow by default; revisit if it costs you bandwidth.
  • Enforcement: baidu's operator does not clearly document how it treats robots.txt. Write the baidu rule anyway, then confirm in your logs that its requests stop.
User-agent: baidu
Allow: /

Paste the line into the validator with one of your own URLs before it goes live. Revisit the baidu entry once a quarter, or when Baidu publishes a new token next to it.

Baiduspider-favo in our records

We first recorded Baiduspider-favo on September 22, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names Baiduspider-favo, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, Baiduspider-favo sits ninth by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, baidu. 62 of the 63 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for Baiduspider-favo is usually a policy for these too.

Baiduspider-ads in our records

We first recorded Baiduspider-ads on September 22, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names Baiduspider-ads, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, Baiduspider-ads sits seventh by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, baidu. 62 of the 63 share its safe rating. Other operators with a crawler that checks pages for an SEO, uptime or analytics service: OAI-AdsBot (OpenAI, Tier 2), AdIdxBot (Microsoft, Tier 3), AdsBot-Google (Google, Tier 3), AdsBot-Google-Mobile (Google, Tier 3), Google-InspectionTool (Google, Tier 3). A policy for Baiduspider-ads is usually a policy for these too.

Baiduspider-image in our records

We first recorded Baiduspider-image on September 22, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names Baiduspider-image, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, Baiduspider-image sits tenth by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, baidu. 62 of the 63 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for Baiduspider-image is usually a policy for these too.

Baiduspider-news in our records

We first recorded Baiduspider-news on September 22, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names Baiduspider-news, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, Baiduspider-news sits 11th by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, baidu. 62 of the 63 share its safe rating. Other operators with a crawler that builds a search or answer index: AddSearchBot (AddSearch, Tier 2), AIWebIndex (Lyrenth, Tier 2), atlassian-bot (Atlassian, Tier 2), AzureAI-SearchBot (Microsoft, Tier 2), Cloudflare-AI-Search (Cloudflare, Tier 2). A policy for Baiduspider-news is usually a policy for these too.

Baiduspider-cpro in our records

We first recorded Baiduspider-cpro on September 22, 2026. Baidu documents it on help.baidu.com, and we recheck that page before each release. No file in our top-10,000 scans names Baiduspider-cpro, so any rule you write for it puts you ahead of almost every large site.

Among the 63 search engines in our database, Baiduspider-cpro sits eighth by priority (Tier 3), level with AdIdxBot, Amzn-SearchBot, Applebot, baidu. 62 of the 63 share its safe rating. Other operators with a crawler that checks pages for an SEO, uptime or analytics service: OAI-AdsBot (OpenAI, Tier 2), AdIdxBot (Microsoft, Tier 3), AdsBot-Google (Google, Tier 3), AdsBot-Google-Mobile (Google, Tier 3), Google-InspectionTool (Google, Tier 3). A policy for Baiduspider-cpro is usually a policy for these too.

How top sites treat Baidu's crawlers

From our robots.txt study of the Tranco top 10,000 (5,583 valid files, fetched October 20, 2026):

None of the tokens for Baidu's crawlers is tracked by name in our study. For scale, 20.1% of top-site files block at least one AI training crawler and 4.3% refuse every unnamed crawler with User-agent: * and Disallow: /, which would catch Baidu's crawlers too.

More questions about Baidu

Will blocking Baidu affect my Google rankings?
Only if one of its tokens is a Google search crawler. None of Baidu's tokens are Googlebot, so classic Google rankings are unaffected.
How quickly does a robots.txt change take effect for Baidu?
Crawlers cache robots.txt, commonly for up to a day. Expect a change to apply within about 24 hours, and confirm it in your logs.
Can I allow Baidu on some sections and block it on others?
Yes. Inside the group for baidu, add Disallow lines for the paths to keep private and Allow lines for exceptions. The longest matching rule wins.
Does blocking Baidu delete content it already collected?
No. robots.txt only controls future visits. For content already collected, check whether Baidu offers a removal or opt-out process.
Is a firewall block better than robots.txt for Baidu?
A firewall enforces the decision, while robots.txt asks politely. Many operators recommend robots.txt because blocking their IPs can stop them reading your rules at all. Use robots.txt first, and add a firewall rule only if requests continue.

A five-minute Baidu review

  1. Confirm the token. Compare the user-agent in your logs with the tokens listed on this page. A close spelling is not the same crawler.
  2. Decide the outcome per crawler. Use the briefs above to choose allow, limit or block for each token.
  3. Write the group. Add or edit one User-agent group per decision and repeat any shared rules inside it.
  4. Publish and wait. Allow about a day for crawlers to fetch the new file.
  5. Confirm the result. Check your logs, then run the AI crawlability test to see the outcome from outside, including any firewall layer.

How to block or allow Baidu in robots.txt

These rules cover all 10 crawlers Baidu operates. Add them to the robots.txt file at the root of your domain.

Block Baidu completely
User-agent: baidu User-agent: Baiduspider-favo User-agent: Baiduspider-ads User-agent: Baiduspider-image User-agent: Baiduspider-news User-agent: Baiduspider-cpro User-agent: Baiduspider-video User-agent: Baiduspider User-agent: ERNIEBot User-agent: YiyanBot Disallow: /
Allow Baidu explicitly
User-agent: baidu User-agent: Baiduspider-favo User-agent: Baiduspider-ads User-agent: Baiduspider-image User-agent: Baiduspider-news User-agent: Baiduspider-cpro User-agent: Baiduspider-video User-agent: Baiduspider User-agent: ERNIEBot User-agent: YiyanBot Allow: /

These Baidu rules ask; they do not enforce. If a Baidu token ignores them, block it at the server or CDN. Paths are case-sensitive, and each rule matches the token, not the full user-agent string.

Frequently asked questions about Baidu

What crawlers does Baidu run?
Baidu operates 10 crawlers in our directory: baidu (user-agent baidu), Baidu Bookmark (user-agent Baiduspider-favo), Baidu Business Search (user-agent Baiduspider-ads), Baidu Image Search (user-agent Baiduspider-image), Baidu News Search (user-agent Baiduspider-news), Baidu Union (user-agent Baiduspider-cpro), Baidu Video Search (user-agent Baiduspider-video), Baiduspider (user-agent Baiduspider), ERNIE (Baidu) (user-agent ERNIEBot), Yiyan Fetcher (Baidu) (user-agent YiyanBot).
Should I block Baidu?
Generally safe to allow; provides legitimate crawling value. That guidance follows this operator's least reassuring crawler, ERNIE (Baidu), rated caution, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 2/10 to 10/10, where 10 means blocking costs you the most visibility.
How do I block Baidu in robots.txt?
Add the following to robots.txt at your domain root: User-agent: baidu then User-agent: Baiduspider-favo then User-agent: Baiduspider-ads then User-agent: Baiduspider-image then User-agent: Baiduspider-news then User-agent: Baiduspider-cpro then User-agent: Baiduspider-video then User-agent: Baiduspider then User-agent: ERNIEBot then User-agent: YiyanBot then Disallow: /. One Disallow line covers all 10 user-agent tokens.
Does blocking Baidu hurt my search rankings?
5 of these share the highest blocking impact here, rated: Critical - Blocking removes you from search results. Those are search-indexing crawlers, so blocking them can remove your pages from the corresponding search results. Block them only if you intend to leave that index. The remaining 5 crawlers in the list above do not affect search indexing and can be blocked separately.

Other operators in the same categories

Allen Institute for AI
2 bots
OpenAI
6 bots
Anthropic
6 bots
DuckDuckGo
2 bots
Google
32 bots
Mistral AI
3 bots
Huawei
2 bots
Perplexity AI
2 bots
WhatsApp us