Caution Reported AI & LLM Bots

ChatGLM Spider

Operated by Zhipu AI

Quick Facts

User-Agent:
ChatGLM-Spider
Category:
AI & LLM Bots
Operator:
Zhipu AI
Safety:
Use Caution
Blocking Impact:
Low - No SEO ranking impact
SEO Impact Score:
2/10
Official docs:
None published by Zhipu AI
Evidence:
Verified by AI Crawler Check: in 109 top-10k robots.txt files

Reported, not operator documented

Zhipu AI has not published a crawler page that names ChatGLM-Spider. It is listed here on our own evidence. Verified by AI Crawler Check: 109 of the 5,583 valid robots.txt files in the Tranco top 10,000 carry a rule for ChatGLM-Spider, so site owners treat it as a real crawler name (how we measure this). That proves the spelling is in use, not who sends it. Reported Zhipu AI crawler. No vendor page.

What that means for you: the ChatGLM-Spider rule on this page is the best available spelling, but Zhipu AI has made no statement of purpose, crawl rate or robots.txt compliance. Treat a ChatGLM-Spider request in your logs as a clue, not proof of who sent it, and add a firewall rule if that traffic matters to you. If the operator publishes documentation, this profile moves to Official docs.

What is ChatGLM Spider?

ChatGLM Spider is an AI data-collection crawler operated by Zhipu AI. It gathers public web content to build or refine training datasets for large language models. Like other AI training crawlers, ChatGLM Spider does not influence your search-engine rankings, so you can block it via robots.txt without any SEO penalty if you wish to opt out of AI training.

ChatGLM Spider is an AI data-collection crawler operated by Zhipu AI. It gathers public web content to build or refine training datasets for large language models. Like other AI training crawlers, ChatGLM Spider does not influence your search-engine rankings, so you can block it via robots.txt without any SEO penalty if you wish to opt out of AI training.
ChatGLM Spider uses the user-agent token ChatGLM-Spider. You can control it via robots.txt, meta tags (noai), or the emerging llms.txt standard. Robots.txt is voluntary; for hard enforcement, combine it with server-level IP blocking.

What happens if you block ChatGLM Spider?

✅ No SEO Impact - Blocking ChatGLM Spider does not affect your rankings in Google, Bing, or any other search engine. ChatGLM Spider is an AI crawler, not a traditional search indexer. You can freely block it via User-agent / Disallow: / without any SEO penalty.
Generally safe to allow; provides legitimate crawling value.

ChatGLM Spider and etaospider

ChatGLM Spider is the Reported crawler for Zhipu AI. Searches for etaospider land here, but etaospider is not a token any operator documents, and this directory does not list it as a separate bot, because no credible public source names its operator. If it appears in your logs, block it by user agent at the server and verify the IP before assuming who runs it.

Want to see whether ChatGLM Spider actually visits you? Read how do I monitor AI bot traffic to my website?

How to block ChatGLM Spider with robots.txt

<code>User-agent: ChatGLM-Spider</code> - Matching is case-insensitive. Robots.txt is fetched from the root of each subdomain separately.

Block completely (robots.txt)
User-agent: ChatGLM-Spider Disallow: /
Allow all (robots.txt)
User-agent: ChatGLM-Spider Allow: /
Block private only (robots.txt)
User-agent: ChatGLM-Spider Disallow: /private/ Disallow: /api/ Disallow: /admin/ Allow: /
Nginx server block
# Nginx: Hard-block ChatGLM Spider if ($http_user_agent ~* "ChatGLM-Spider") { return 403 "Bot blocked"; }
Apache .htaccess
# Apache: Hard-block ChatGLM Spider SetEnvIfNoCase User-Agent "ChatGLM-Spider" bad_bot Order Allow,Deny Allow from all Deny from env=bad_bot
Meta robots tag
<meta name="robots" content="noindex, nofollow">
X-Robots-Tag header
X-Robots-Tag: noindex, nofollow

Is ChatGLM Spider safe to allow?

ChatGLM Spider is rated Caution in this directory. It is a Reported token: Zhipu AI has not published documentation for it, so its purpose, crawl rate and robots.txt behaviour are not confirmed by the operator, and anyone can send the same user-agent string. Allow or block it on its stated purpose, and back the robots.txt rule with a firewall rule if the traffic matters to you.
Zhipu AI publishes no IP ranges or reverse-DNS pattern for ChatGLM Spider, so the user-agent string cannot be verified. Rate-limit unusual volume instead of trusting the name.

What does ChatGLM Spider do?

Understanding ChatGLM Spider's purpose helps you decide whether to allow or block it.

  • Collects public web content to train large language models
  • Builds and refreshes AI training datasets
  • Harvests text, code, and structured data for model improvement
  • Does NOT affect search-engine rankings
  • Can be blocked to opt out of AI training without SEO loss

ChatGLM Spider in context

ChatGLM Spider is a training crawler operated by Zhipu AI. In plain terms, it collects pages to train AI models. Our database files it under AI and LLM crawlers and ranks it Tier 2, important crawlers from established AI and search operators. 74 of the 104 AI and LLM crawlers in our database share this tier.

Its safety rating is caution and its blocking impact is low. So the decision about ChatGLM Spider rests on what you want from Zhipu AI, not on its behaviour. How we rate safety and impact.

How site owners treat ChatGLM Spider

In our own scan of the Tranco top 10,000 (5,583 valid robots.txt files), 109 files (2%) name ChatGLM-Spider in a User-agent line. That shows site owners treat the token as a real crawler worth a rule. It does not, on its own, prove who operates it.

Does it follow robots.txt?

ChatGLM Spider's operator does not clearly document how it treats robots.txt. Write the ChatGLM Spider rule anyway, then confirm in your logs that its requests stop.

Zhipu AI crawlers and close alternatives

Zhipu AI runs only this one crawler in our database, so a single User-agent: ChatGLM-Spider group controls everything Zhipu AI fetches through it.

Other AI and LLM crawlers that do the same job (collects pages to train AI models) include ClaudeBot from Anthropic, Google-Extended from Google, GPTBot from OpenAI, Meta-ExternalAgent from Meta, AI2Bot from Allen Institute for AI, aiHitBot from aiHit. Deciding about ChatGLM Spider is usually a decision about the whole group, so check how your file treats these too.

We first recorded ChatGLM-Spider on June 18, 2026. We could not find a page where Zhipu AI names it, so treat the token as reported rather than documented.

Which rule ChatGLM Spider actually follows

ChatGLM Spider reads exactly one group in robots.txt, the most specific one that names ChatGLM-Spider, and never combines rules across groups. This short file shows the trap:

User-agent: *
Disallow: /private/

User-agent: ChatGLM-Spider
Disallow: /drafts/
  1. Find the group. ChatGLM Spider looks for a group naming ChatGLM-Spider. One exists, so it ignores User-agent: * completely.
  2. Apply only that group. It is refused /drafts/ and nothing else.
  3. The surprise. /private/ is open to ChatGLM Spider, because that rule lives in the * group it no longer reads. To keep it out of both, repeat Disallow: /private/ inside its own group.
Test yourself: does a rule under User-agent: * apply to ChatGLM Spider if the file also has a User-agent: ChatGLM-Spider group?

No. Once a group names it, ChatGLM Spider reads only that group. Put every rule you want it to follow inside its own group.

Test yourself: will blocking ChatGLM Spider remove your pages from AI answers?

No. ChatGLM Spider collects training data. Blocking it changes what future models learn, not which pages AI answers cite today.

Test yourself: is User-agent: chatglm-spider the same as User-agent: ChatGLM-Spider?

Yes. RFC 9309 makes user-agent matching case-insensitive. Paths are a different story: Disallow: /Blog/ does not block /blog/.

Should you allow ChatGLM Spider? Decide by situation

Recommended setting for ChatGLM Spider by type of site
If you are...Recommended setting
You publish content and want AI answers to cite itYour call. ChatGLM Spider is a training crawler, so blocking it does not remove you from AI answers; retrieval crawlers do that job.
You license content and want to stop AI trainingBlock it by name with User-agent: ChatGLM-Spider and Disallow: /.
You run an online shopAllow it unless your logs show it is costly.
Your server is under load from botsRate-limit before blocking. It is rated caution in our database.

Check ChatGLM Spider on your own site

  1. Open /robots.txt on your domain and search for ChatGLM-Spider. If it is not named, it follows User-agent: *.
  2. List the Disallow lines in the group ChatGLM Spider follows and test your key URLs against them; the longest match wins, and Allow wins a tie.
  3. Look beyond the file. A firewall or bot manager can refuse ChatGLM Spider even when robots.txt allows it, and that never shows in the file.
  4. Run our AI search visibility check to see how ChatGLM Spider and every other crawler we track are treated in one pass, including firewall refusals.

Common mistakes with ChatGLM Spider

The errors we see most often with ChatGLM Spider in the files and firewalls we test:

  1. Assuming that blocking ChatGLM Spider removes you from AI answers. It is a training crawler; AI answers are fed by retrieval and user-triggered crawlers with different tokens.
  2. Blocking ChatGLM Spider with a broad User-agent: * rule that also catches search engines. Name ChatGLM-Spider in its own group instead.
  3. Expecting a ChatGLM Spider block to remove content already collected. robots.txt only affects future visits.

All three hide inside a file that looks correct, so check ChatGLM Spider from outside after every change.

Spotting ChatGLM Spider in your logs

robots.txt says what ChatGLM Spider may do; your access logs show what it actually did.

Count requests

On a server with standard Nginx or Apache logs, this counts matching requests per day (adjust the log path for your host):

grep -iE "ChatGLM-Spider" /var/log/nginx/access.log \
  | awk '{print $4}' | cut -d: -f1 | sort | uniq -c

And this shows which paths it requested most, which is the quickest way to see whether your Disallow lines are being respected:

grep -iE "ChatGLM-Spider" /var/log/nginx/access.log \
  | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

Check the requests are genuine

Anyone can send the ChatGLM Spider user-agent, so confirm the source before acting on a match:

  • ChatGLM-Spider: we hold no published IP list, so use reverse DNS where the operator documents a hostname and treat an unverifiable ChatGLM Spider request with caution.

Frequently Asked Questions

What user-agent string does ChatGLM Spider use?
ChatGLM Spider is reported to identify itself with ChatGLM-Spider. Evidence that the spelling is in real use comes from our own robots.txt study: 109 of the 5,583 valid robots.txt files in the Tranco top 10,000 name it (fetched October 20, 2026), not from Zhipu AI, which has not published crawler documentation naming it. Use it in robots.txt as the best available match, and treat a request carrying it as a clue rather than proof of who sent it.
Is ChatGLM Spider safe?
ChatGLM Spider is rated Caution in this directory. It is a Reported token: Zhipu AI has not published documentation for it, so its purpose, crawl rate and robots.txt behaviour are not confirmed by the operator, and anyone can send the same user-agent string. Allow or block it on its stated purpose, and back the robots.txt rule with a firewall rule if the traffic matters to you.
Will blocking ChatGLM Spider hurt my SEO?
✅ No SEO Impact - Blocking ChatGLM Spider does not affect your rankings in Google, Bing, or any other search engine. ChatGLM Spider is an AI crawler, not a traditional search indexer. You can freely block it via User-agent / Disallow: / without any SEO penalty.
How do I block ChatGLM Spider in robots.txt?
Add the following lines to your /robots.txt file:
User-agent: ChatGLM-Spider
Disallow: /

This instructs ChatGLM Spider not to crawl any path on your site. To block only specific sections, replace / with the path (e.g., Disallow: /blog/).
Does ChatGLM Spider respect robots.txt?
Not documented. Zhipu AI has published no statement that ChatGLM Spider reads robots.txt. A rule for ChatGLM-Spider will stop a client that chooses to honour it; for enforcement, add a firewall or CDN rule as well.
How do I verify if ChatGLM Spider is crawling my site?
Search your access logs for ChatGLM-Spider (for example grep -i "ChatGLM-Spider" /var/log/nginx/access.log). Because the operator publishes no IP ranges or reverse-DNS pattern, a match shows that something used the name, not that Zhipu AI sent it.

Related Bots

WhatsApp us