AI Crawler Check
Free Bot Analysis Tool
Check
Home / Directory / AI & LLM Bots

AI & LLM Bots Directory

AI training crawlers collecting content for LLM datasets. Blocking has zero SEO impact.

34 total
24 safe
8 caution
2 aggressive

AI2Bot

Safe
User-agent: AI2Bot
Operator: Allen Institute for AI

Ai2Bot-Dolma

Caution
User-agent: Ai2Bot-Dolma
Operator: Allen Institute for AI

Big Sur AI

Safe
User-agent: Big Sur AI
Operator: Big Sur AI

ChatGPT Operator

Safe
User-agent: ChatGPT Operator
Operator: OpenAI

ChatGPT-User

Safe
User-agent: ChatGPT-User
Operator: OpenAI

ChatGPT-User/2.0

Aggressive
User-agent: ChatGPT-User
Operator: OpenAI

Claude-Code

Safe
User-agent: Claude-Code
Operator: Anthropic

Claude-SearchBot

Safe
User-agent: Claude-SearchBot
Operator: Anthropic

Claude-User

Safe
User-agent: Claude-User
Operator: Anthropic

Claude-Web

Safe
User-agent: Claude-Web
Operator: Anthropic

ClaudeBot

Safe
User-agent: ClaudeBot
Operator: Anthropic

DeepSeekBot

Aggressive
User-agent: DeepSeekBot
Operator: DeepSeek

DigitalOceanGenAICrawler

Caution
User-agent: DigitalOceanGenAICrawler
Operator: DigitalOcean

DuckAssistBot

Safe
User-agent: DuckAssistBot
Operator: DuckDuckGo

GPTBot

Safe
User-agent: GPTBot
Operator: OpenAI

Google-Extended

Safe
User-agent: Google-Extended
Operator: Google

Grok

Safe
User-agent: Grok
Operator: Unknown

GrokBot

Safe
User-agent: GrokBot
Operator: Unknown

LINER Bot

Safe
User-agent: LINER Bot
Operator: LINER

MistralAI-User

Caution
User-agent: MistralAI-User
Operator: Mistral AI

MistralAI-User/1.0

Caution
User-agent: MistralAI-User
Operator: Mistral AI

MyCentralAIScraperBot

Safe
User-agent: MyCentralAIScraperBot
Operator: Unknown

OAI-SearchBot

Safe
User-agent: OAI-SearchBot
Operator: OpenAI

PanguBot

Safe
User-agent: PanguBot
Operator: Huawei

Perplexity-User

Caution
User-agent: Perplexity-User
Operator: Perplexity AI

Perplexity-User/1.0

Caution
User-agent: Perplexity-User
Operator: Perplexity AI

PerplexityBot

Safe
User-agent: PerplexityBot
Operator: Perplexity AI

SBIntuitionsBot

Safe
User-agent: SBIntuitionsBot
Operator: Unknown

YouBot

Safe
User-agent: YouBot
Operator: You

anthropic-ai

Safe
User-agent: anthropic-ai
Operator: Anthropic

cohere-ai

Safe
User-agent: cohere-ai
Operator: Cohere

cohere-training-data-crawler

Caution
User-agent: cohere-training-data-crawler
Operator: Cohere

img2dataset

Safe
User-agent: img2dataset
Operator: Open Source Community

quillbot.com

Caution
User-agent: quillbot.com
Operator: QuillBot

FAQ

How many AI & LLM Bots are tracked?
We currently track 34 AI & LLM Bots with user-agent strings, safety ratings, and blocking rules.
Which AI & LLM Bots should I never block?
Never block bots with 'critical' impact rating — there are 0 in this category. Blocking them removes your site from their search results.
Are all AI & LLM Bots safe?
Out of 34 bots in this category: 24 safe, 8 caution, 2 aggressive.
How do I block all AI & LLM Bots at once?
Generate a combined robots.txt block rule for all AI & LLM Bots using our bulk robots.txt generator, or manually list each User-agent + Disallow: / directive.