Applebot-Extended

Applebot-Extended in context

Applebot-Extended is an opt-out token rather than a crawler operated by Apple. In plain terms, it collects pages to train AI models. Our database files it under search engines and ranks it Tier 2, important crawlers from established AI and search operators. 1 of the 63 search engines in our database share this tier.

Its safety rating is caution and its blocking impact is critical. So the decision about Applebot-Extended rests on what you want from Apple, not on its behaviour. How we rate safety and impact.

How site owners treat Applebot-Extended

In our own study of robots.txt files across the Tranco top 10,000 sites (fetched October 20, 2026, 5,583 valid files), 600 files (10.7%) name Applebot-Extended in a User-agent line, and 694 (12.4%) block it from the homepage. 478 of those blocks come from a group written for this crawler by name, and 216 come from a broad User-agent: * rule that catches it by accident. Larger sites are stricter: 17.6% of the top 1,000 block it, against 12.7% of the middle band and 11.3% of the tail.

Does it follow robots.txt?

Applebot-Extended is a control token, not a crawler, so it never appears in your logs. You set Applebot-Extended in robots.txt and the operator reads that setting through its other crawlers.

Apple crawlers and close alternatives

Applebot-Extended is one of 2 Apple crawlers in our database. A User-agent: Applebot-Extended group controls Applebot-Extended only; the other Apple tokens below need their own lines.

• Applebot-Extended: this crawler (Tier 2)
• Applebot: Apple Search, builds a search or answer index (Tier 3)

To give Applebot-Extended the same answer as its siblings, stack User-agent: Applebot-Extended with their lines above one set of rules; to treat Applebot-Extended differently, give it its own group.

No other crawler in our search engines category has exactly the same purpose, which makes Applebot-Extended an unusual case: decide about it on its own merits.

We first recorded Applebot-Extended on March 17, 2026. Apple documents it on support.apple.com, and we recheck that page before each release.

Which rule Applebot-Extended actually follows

Applebot-Extended reads exactly one group in robots.txt, the most specific one that names Applebot-Extended, and never combines rules across groups. This short file shows the trap:

User-agent: *
Disallow: /private/

User-agent: Applebot-Extended
Disallow: /drafts/

1. Find the group. Applebot-Extended looks for a group naming Applebot-Extended. One exists, so it ignores User-agent: * completely.
2. Apply only that group. It is refused /drafts/ and nothing else.
3. The surprise. /private/ is open to Applebot-Extended, because that rule lives in the * group it no longer reads. To keep it out of both, repeat Disallow: /private/ inside its own group.

Should you allow Applebot-Extended? Decide by situation

Check Applebot-Extended on your own site

1. Open /robots.txt on your domain and search for Applebot-Extended. If it is not named, it follows User-agent: *.
2. List the Disallow lines in the group Applebot-Extended follows and test your key URLs against them; the longest match wins, and Allow wins a tie.
3. Look beyond the file. A firewall or bot manager can refuse Applebot-Extended even when robots.txt allows it, and that never shows in the file.
4. Run our robots.txt AI crawler test to see how Applebot-Extended and every other crawler we track are treated in one pass, including firewall refusals.