Semrush

3 crawlers operated by Semrush

https://www.semrush.com/

Who Semrush is, for site owners

Semrush operates 3 crawlers in our directory, filed under SEO Tools. Of those 3, 2 rated safe, 1 rated caution for crawl behaviour.

Blocking Semrush carries an SEO impact score of 0/10 to 2/10 across its crawlers. 2 of them share the highest blocking impact here: Varies - Evaluate before blocking.

Our recommendation for Semrush, set by its least reassuring crawler (SemrushBot-OCOB, rated caution): Consider blocking based on your content strategy.

Semrush at a glance

Operator	Semrush
Crawlers in directory	3
Categories	SEO Tools
SEO impact if blocked	0/10 to 2/10
Crawl behaviour	2 rated safe, 1 rated caution
Website	www.semrush.com
Official crawler docs	Semrush crawler documentation

Crawlers operated by Semrush

How to block or allow Semrush in robots.txt

These rules cover all 3 crawlers Semrush operates. Add them to the robots.txt file at the root of your domain.

Block Semrush completely

User-agent: SemrushBot-SWA
User-agent: SemrushBot
User-agent: SemrushBot-OCOB
Disallow: /

Allow Semrush explicitly

User-agent: SemrushBot-SWA
User-agent: SemrushBot
User-agent: SemrushBot-OCOB
Allow: /

robots.txt is a request, not an enforcement mechanism. A crawler that ignores it needs a block at the server or CDN layer. Rules are case-sensitive on the path and matched against the user-agent token, not the full user-agent string.

Frequently asked questions about Semrush

What crawlers does Semrush run?

Semrush operates 3 crawlers in our directory: Semrush Writing Assistant (user-agent SemrushBot-SWA), SemrushBot (user-agent SemrushBot), SemrushBot-OCOB (user-agent SemrushBot-OCOB).

Should I block Semrush?

Consider blocking based on your content strategy. That guidance follows this operator's least reassuring crawler, SemrushBot-OCOB, rated caution, because allowing an operator allows every crawler it runs. The SEO impact score for blocking is 0/10 to 2/10, where 10 means blocking costs you the most visibility.

How do I block Semrush in robots.txt?

Add the following to robots.txt at your domain root: User-agent: SemrushBot-SWA then User-agent: SemrushBot then User-agent: SemrushBot-OCOB then Disallow: /. One Disallow line covers all 3 user-agent tokens.

Does blocking Semrush hurt my search rankings?

2 of these share the highest blocking impact here, rated: Varies - Evaluate before blocking. Google's own ranking crawler is Googlebot, which this operator does not run, so blocking Semrush does not by itself remove you from Google's index. What it can change is whether AI assistants and analytics tools can read your pages.