AI crawler
Google-Extended
Also: Google Extended, Gemini training signal, Google AI crawler control
Google introduced Google-Extended in late 2023 as the controllable signal for AI training and retrieval distinct from organic search indexing. The distinction matters for healthcare and legal practices that want to be cited by Gemini but have not previously thought about AI training opt in versus opt out.
A correctly configured kailxlabs.co style robots.txt explicitly Allows Google-Extended along with GPTBot, ClaudeBot, PerplexityBot, and the standard Googlebot. Most clinic and law firm sites built before 2024 inherit a default robots.txt that does not name Google-Extended, which by default opts the site out of contributing to Gemini training quality.