Gary Ilyes cautioned that blocking GoogleOther “could impact various aspects throughout the Google ecosystem.”

Google’s Gary Illyes responded to a question regarding the non-search features supported by the GoogleOther crawler and warned about the potential repercussions of blocking GoogleOther.

What Is GoogleOther?

GoogleOther is a generic crawler created by Google for various purposes beyond those of specialized bots for Search, Ads, Video, Images, News, Desktop, and Mobile. Internal teams at Google can use it for research and development related to different products. According to the official description:

“GoogleOther is the generic crawler that various product teams may use to fetch publicly accessible content from sites.”

Interestingly, there are three kinds of GoogleOther crawlers:

Three Kinds Of GoogleOther Crawlers

1. GoogleOther
   • Generic crawler for public URLs
2. GoogleOther-Image
   • Optimized to crawl public image URLs
3. GoogleOther-Video
   • Optimized to crawl public video URLs

All three GoogleOther crawlers can be used for research and development purposes, which is one of the publicly acknowledged uses for all versions of GoogleOther.

What Non-Search Features Does GoogleOther Support?

Google does not specify the exact non-search features supported by GoogleOther, likely because it doesn’t support any particular feature. GoogleOther exists for research and development crawling, which could help a new product or improve an existing one. Its purpose is highly open and generic.

Here’s the question Gary Illyes addressed:

“What non-search features does GoogleOther crawling support?”

Gary Illyes responded:

“This is a very topical question, and I think it is excellent. I don’t have much else to share besides what’s in the public.

GoogleOther is a generic crawler that various product teams may use to fetch publicly accessible content from sites. It may also be used for one-off crawls for internal research and development.

Historically, Googlebot was used for this, but that made things murky and less transparent, so we launched GoogleOther. Now, you have better control over what your site is crawled for.

That said, GoogleOther is not tied to a single product, so opting out of GoogleOther crawling might affect a wide range of things across the Google universe; alas, not Search—search is only Googlebot.”

It Might Affect A Wide Range Of Things

Gary Ilyes is clear that blocking GoogleOther wouldn’t affect Google Search since Googlebot is the crawler used for indexing content. Therefore, if a site owner wishes to block any of the three versions of GoogleOther, it should not negatively impact search rankings.

However, Gary also cautioned about the consequences of blocking GoogleOther, stating that it could affect other products and services across Google. He did not specify which other products could be impacted or elaborate on the pros or cons of blocking GoogleOther.

Pros and Cons of Blocking GoogleOther

Deciding whether to block GoogleOther is not straightforward and requires careful consideration of several factors.

Pros

1. Inclusion in Research for Future Google Products: Allowing GoogleOther to crawl your site might make it part of research for new Google products related to search, such as maps, shopping, images, or new search features. This could be beneficial, as it leads to early testing of new features that could increase your site’s earnings.
2. Minimal Impact on Server Resources: Blocking GoogleOther to save on server resources may not be a valid concern, as this crawler seems to need to visit sites more frequently to cause a noticeable impact.
3. No Impact on AI Training Concerns: If your primary concern is blocking Google from using your site’s content for AI purposes, blocking GoogleOther is irrelevant. GoogleOther is not used for Google Gemini apps, Vertex AI, or AI training. The specific bot for AI-related tasks is Google-Extended.

Cons

1. Potential Spam-Related Testing: Allowing GoogleOther to crawl your site could be unhelpful if it is being used to test spam-fighting techniques and your site has something to hide.
2. Market Research or Internal Training: Site owners might want their site separate from market research or internal machine learning model training unrelated to public-facing products like Gemini and Vertex.
3. Uncertain Use of Site Data: Permitting GoogleOther to crawl your site for unspecified purposes is akin to giving Google a blank check to use your site’s data in various ways, aside from training public-facing LLMs or purposes related to specific bots like GoogleBot.

Ultimately, the decision to block GoogleOther should be based on carefully evaluating these pros and cons, considering the potential benefits and drawbacks of your site’s needs and goals.

Takeaway

Should you block GoogleOther? It’s a coin toss. While there may be potential benefits, there isn’t enough information to make a fully informed decision. If you still find it difficult and confusing, check out our monthly SEO packages and let the experts help you.