Ahrefs Site Audit
SafeUser-agent: AhrefsSiteAudit
Operator: Ahrefs
Added:
SEO Tools Directory
Commercial SEO platform crawlers (Ahrefs, Semrush etc.). Blocking hides your data from competitors.
19 tokens
16 safe
3 caution
0 aggressive
Common providers
Pick a provider to see only its crawlers. Each provider also has a full profile under Bot Operators.
All 19 SEO Tools
AhrefsBot
SafeUser-agent:
AhrefsBotOperator: Ahrefs
Added:
AwarioBot
SafeUser-agent:
AwarioBotOperator: Awario
Added:
AwarioRssBot
SafeUser-agent:
AwarioRssBotOperator: Awario
Added:
AwarioSmartBot
SafeUser-agent:
AwarioSmartBotOperator: Awario
Added:
DataForSeoBot
SafeUser-agent:
DataForSeoBotOperator: DataForSEO
Added:
DotBot
CautionUser-agent:
DotBotOperator: Moz
Added:
Lighthouse Audit
SafeUser-agent:
Chrome-LighthouseOperator: Google
Added:
MJ12bot
CautionUser-agent:
MJ12botOperator: Majestic
Added:
Moz Campaign Crawler
SafeUser-agent:
RogerbotOperator: Moz
Added:
NeticleBot
SafeUser-agent:
NeticleBotOperator: Neticle
Added:
Netvibes
SafeUser-agent:
NetvibesOperator: Dassault Systemes (Netvibes)
Added:
Screaming Frog SEO Spider
SafeUser-agent:
Screaming Frog SEO SpiderOperator: Screaming Frog
Added:
Semrush Writing Assistant
SafeUser-agent:
SemrushBot-SWAOperator: Semrush
Added:
SemrushBot
SafeUser-agent:
SemrushBotOperator: Semrush
Added:
SemrushBot-OCOB
CautionUser-agent:
SemrushBot-OCOBOperator: Semrush
Added:
Sidetrade Indexer Bot
SafeUser-agent:
Sidetrade Indexer BotOperator: Sidetrade
Added:
Yandex Metrica
SafeUser-agent:
YandexMetrikaOperator: Yandex
Added:
Yandex Tracker
SafeUser-agent:
YandexTrackerOperator: Yandex
Added:
No bots match those filters. Try a shorter search or browse every category. Missing a crawler? Tell us and we will research it.
FAQ
How many SEO Tools are tracked?
We currently track 19 SEO Tools with user-agent strings, safety ratings, and blocking rules.
Which SEO Tools should I never block?
Never block bots with 'critical' impact rating - there are 0 in this category. Blocking them removes your site from their search results.
Are all SEO Tools safe?
Out of 19 bots in this category: 16 safe, 3 caution, 0 aggressive.
How do I block all SEO Tools at once?
Generate a combined robots.txt block rule for all SEO Tools using our bulk robots.txt generator, or manually list each User-agent + Disallow: / directive.