What is Google-Extended?

How does Google-Extended work?

Google-Extended lets publishers express preferences for the AI uses described in Google’s crawler documentation.

Google uses content collected by existing crawlers, so there is no separate Google-Extended HTTP user-agent string to find in access logs.

The token belongs in robots.txt, but its function concerns downstream uses rather than a distinct stream of crawling requests. Its documented scope should be checked before policy changes.

• Use the product token exactly as documented by Google.
• Distinguish the robots.txt setting from Googlebot crawl permissions.
• Do not invent a separate Google-Extended IP verification workflow.

Why does Google-Extended matter for B2B SaaS?

A SaaS publisher may want to set AI-use preferences while keeping documentation discoverable through Google Search. Google states that Google-Extended does not affect Search inclusion or serve as a Search ranking signal.

It also does not govern eligibility for Search features such as AI Overviews. Confusing these controls can lead to unnecessary blocks on Googlebot that undermine the publisher’s actual discovery goals.

• Write separate policies for Search access and the covered Gemini uses.
• Check current documentation because product scope can evolve.
• Keep sensitive customer material protected by authentication regardless of policy.

What is an example of Google-Extended?

A publisher could disallow Google-Extended while continuing to allow Googlebot to crawl public product guides. That expresses a preference about the covered AI uses without intentionally withdrawing those pages from Google Search.

The rule does not erase prior content collection or act as a general prohibition on every AI product. The implementation should be evaluated against the exact use cases Google describes.

• Keep Googlebot rules aligned with the desired Search policy.
• Document why the Google-Extended preference was selected.
• Avoid describing the setting as a universal AI opt-out.

How do you improve or measure Google-Extended?

Review the live robots.txt configuration and record the intended policy outcome. Server logs cannot prove that a separate Google-Extended crawler obeyed the rule because that separate crawler does not exist.

Google Search indexing reports measure a different system and should not be treated as confirmation of downstream Gemini use.

Reassess the setting when Google revises the product-token documentation or the business changes its content policy.

• Verify that broad robots.txt groups do not accidentally block Googlebot.
• Preserve a dated record of policy decisions and configuration changes.
• Do not infer model-training inclusion from ordinary crawl activity.

Frequently asked questions about Google-Extended