Search Engines Directory

Search engine crawlers that index your content for organic search visibility. Critical to allow.

62 tokens
61 safe
1 caution
0 aggressive

Common providers

Pick a provider to see only its crawlers. Each provider also has a full profile under Bot Operators.

All 62 Search Engines

FAQ

How many Search Engines are tracked?
We currently track 62 Search Engines with user-agent strings, safety ratings, and blocking rules.
Which Search Engines should I never block?
Never block bots with 'critical' impact rating - there are 24 in this category. Blocking them removes your site from their search results.
Are all Search Engines safe?
Out of 62 bots in this category: 61 safe, 1 caution, 0 aggressive.
How do I block all Search Engines at once?
Generate a combined robots.txt block rule for all Search Engines using our bulk robots.txt generator, or manually list each User-agent + Disallow: / directive.