Why Am I Not Cited in AI Answers? 9 Common Reasons
A practical, visual guide to diagnose why AI engines ignore your content. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
AI Crawler Check Blog
148 guides on AI crawlers, robots.txt and AI search visibility, written and maintained by Horatos.ai.
Latest articles
Cohere AI Crawler (cohere-ai): What Website Owners Need to Know (2026)
Cohere says it does not currently crawl the web to train its foundation models, yet cohere-ai still appears in logs and blocklists. Here is what that means and how to manage the token.
Google-Extended vs Googlebot: What Each Controls in 2026 (and the New AI Overviews Switch)
Googlebot crawls for Search, including AI Overviews. Google-Extended controls Gemini training and grounding. How the two differ, and the new Search Console switch.
Generative Engine Optimization (GEO): A Beginner's Guide
A practical, visual guide to learn GEO from scratch and where to start. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
How to Block AI Crawlers with Robots.txt (2026 Complete Guide)
A step-by-step guide to blocking (or allowing) AI crawlers like GPTBot, ClaudeBot, and Google-Extended using robots.txt. Includes code examples, best practices, and tools.
GEO Ranking Factors: What Gets You Cited in 2026
A practical, visual guide to know the signals that drive AI citations. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
How AI Engines Choose Which Sources to Cite
A practical, visual guide to understand the source-selection logic of LLMs. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
Robots.txt Best Practices for AI SEO in 2026 (With Data From 10,000 Sites)
How to write robots.txt for AI search and SEO: separate search from training bots, templates for five site types, and real mistakes from the top 10,000 sites.
Content Citability: How to Write So AI Quotes You
A practical, visual guide to structure content to be easy for AI to quote. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
What is GPTBot? OpenAI's Web Crawler Explained (2026)
Everything you need to know about GPTBot, OpenAI's web crawler for ChatGPT training. User-agent string, blocking rules, impact on SEO, and how it compares to other AI crawlers.
Answer-First Content: The Structure AI Engines Love
A practical, visual guide to lead with answers to win AI citations. Includes step-by-step instructions, common mistakes to avoid, and free tools to verify your results.
AI Visibility Score: Why Your Website Needs One in 2026
The AI Visibility Score measures how discoverable your website is to AI systems. Learn how it works, why it matters, and how to improve yours from 0 to 100.
Page 7 of 13, showing 73 to 84 of 148 articles
Stay Updated on AI SEO
Check your website's AI visibility and discover optimization opportunities
Free AI Crawler Check