AI Crawler Directory

GoogleOther Google's ai training crawler

GoogleOther is Google's generic crawler for internal research and development uses that aren't Google Search — offloading non-Search crawling from Googlebot so publishers can control it separately. Google is deliberately vague about what it feeds, which is precisely why many publishers choose to disallow it.

AI training crawlerReviewed August 2026
Quick facts
Operator
Google
Feeds
Google research and development crawling (non-Search)
Type
AI training crawler
robots.txt token
GoogleOther
Respects robots.txt
Yes

What is GoogleOther?

GoogleOther shares Googlebot's infrastructure and crawl politeness but serves unspecified internal Google purposes; variants exist for image and video fetching.

Blocking GoogleOther has no effect on Google Search: that separation is its stated purpose.

How to identify GoogleOther

GoogleOther identifies itself with the following user-agent string:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther)

Never trust the user-agent string alone. Scrapers routinely impersonate well-known crawlers to inherit their access. GoogleOther crawls from Google's published crawler IP ranges and can be verified the same way as Googlebot — reverse DNS plus the published JSON ranges.

Published IP ranges: https://developers.google.com/search/apis/ipranges/googlebot.json

Note that most AI crawlers and fetchers, GoogleOther included, do not execute JavaScript — so this traffic is invisible to GA4 and every script-based analytics tool. Server logs, CDN analytics, or a dedicated bot-analytics layer are the only places you will see it. For the full picture of measuring AI-driven visits, see our guide on how to track AI traffic.

Controlling GoogleOther with robots.txt

To refuse GoogleOther access to your entire site, add this to your robots.txt:

User-agent: GoogleOther
Disallow: /

To restrict it from specific sections only (for example, premium content) while leaving the rest open:

User-agent: GoogleOther
Disallow: /premium/
Disallow: /members/

Should you block or monetize GoogleOther?

The case for blocking: Unspecified R&D use of your content, with zero return traffic, is an easy no for most content businesses — and blocking is free of search consequences.

The case for allowing or monetizing: There is no direct monetization path for an unspecified internal crawler; the value is in the precedent that non-Search access is not free by default.

Bottom line
Disallow it unless you have a reason not to. Like Google-Extended, it costs nothing in search visibility, and consenting to unspecified uses by default is poor hygiene for a content business.
Where Oasy fits

See exactly what GoogleOther does on your site — then decide what that access is worth.

Oasy detects and fingerprints 50+ AI crawlers with per-URL analytics, blocks the ones you exclude at the edge, and turns the rest into revenue — licensed RAG access and sponsored placement inside AI answers, settled weekly. Analytics scripts can't see this traffic; your server logs can, and so can we.

Join the waitlist

Frequently asked questions

Does blocking GoogleOther affect Google Search or AI Overviews?+

No. GoogleOther exists specifically to separate non-Search crawling from Googlebot, so blocking it leaves Search and its AI features untouched.

What does Google actually use GoogleOther for?+

Google describes it only as internal research and development crawling. The vagueness is the point of contention — you are being asked to consent to unspecified uses.

Stay informed on life in Europe as an expat. The Local delivers daily news, guides and essential info across 9 European countries. Sign up for the free newsletter at thelocal.com/free-newsletter