“Google-Extended, the new standalone product token, is impractical in communicating your request to Google via robots.txt to exclude your site’s content from being used in projects such as Bard, Vertex AI, and other AI initiatives. It’s important to note that Google Extended does not apply to AI answers and snapshots provided within the Search Generative Experience (SGE).
According to a statement from Google, Google-Extended is not designed to affect SGE. As SGE is considered a Search experiment, website administrators should continue to rely on the Googlebot user agent specified in their robots.txt file and the NOINDEX meta tag to regulate the appearance of their content in search results, including within experimental features like Search Generative Experience. A spokesperson from Google clarified this approach.”
Please note that this revision aims to rephrase the provided information for clarity and readability while retaining the core message.
“Why isn’t Google-Extended effective for Search Generative Experience (SGE)? Google has clarified that SGE is an integral part of the Google Search experience; it functions as a core search feature and, therefore, should adhere to standard search directives. Google emphasized that AI is seamlessly integrated into Search rather than being an add-on. Robots.txt is the control mechanism for web publishers to manage access to how their websites are crawled.
In the context of SGE, it’s important to note that AI-generated answers are sometimes displayed, complete with website cards, even for websites that have explicitly directed Google not to use their content for AI purposes. An example is Glenn Gabe’s screenshot of VentureBeat.com, which had Google-Extended disallowed in its robots.txt file but still appeared in an AI-generated answer within SGE.”
This revision aims to convey the information with added clarity and structure while maintaining the core details of the original content.
“Why is this important? If you believe using Google-Extended would safeguard your content from being utilized for AI-powered answers in the Search Generative Experience (SGE), it’s crucial to understand that this assumption is incorrect. Google-Extended is designed for Bard, Vertex AI, and other non-search-specific products.
If your objective is to prevent your content from appearing in SGE-generated answers, you would need to take the more drastic step of completely blocking Googlebot access to your site. However, for many website administrators, this is an action they are reluctant to take.”
This revision aims to maintain the original message while making it more clear and concise for the reader.
If you need help finding it, explore our monthly SEO packages and let our experts handle it.
