Operated by Microsoft
Azure AI Search is an AI search-index crawler operated by Microsoft. It collects and indexes web content so it can be surfaced as citations or answers inside Microsoft's AI search experience. Allowing Azure AI Search can increase your visibility in AI-powered search results, which is why most sites choose to permit it.
Azure AI Search is an AI search-index crawler operated by Microsoft. It collects and indexes web content so it can be surfaced as citations or answers inside Microsoft's AI search experience. Allowing Azure AI Search can increase your visibility in AI-powered search results, which is why most sites choose to permit it.
Azure AI Search uses the user-agent token AzureAI-SearchBot. You can control it via robots.txt, meta tags (noai), or the emerging llms.txt standard. Robots.txt is voluntary; for hard enforcement, combine it with server-level IP blocking.
User-agent / Disallow: / without any SEO penalty.<code>User-agent: AzureAI-SearchBot</code> — Matching is case-insensitive. Robots.txt is fetched from the root of each subdomain separately.
Understanding Azure AI Search's purpose helps you decide whether to allow or block it.
AzureAI-SearchBot. Use this exact string in robots.txt, Nginx, Apache, or Cloudflare firewall rules to target this bot. Matching in robots.txt is case-insensitive. Verify a request genuinely comes from Azure AI Search by performing a reverse-DNS lookup on the source IP.User-agent / Disallow: / without any SEO penalty./robots.txt file:
User-agent: AzureAI-SearchBot Disallow: /This instructs Azure AI Search not to crawl any path on your site. To block only specific sections, replace / with the path (e.g.,
Disallow: /blog/)./robots.txt before crawling, following RFC 9309. For hard enforcement, combine robots.txt with server-level IP or user-agent blocking.AzureAI-SearchBot (case-insensitive: grep -i "AzureAI-SearchBot" /var/log/nginx/access.log). Filter by user-agent in your log analytics tool (GoAccess, AWStats, etc.).Check instantly with our free AI Bot Checker
Check Your Website