What It Takes to Be Cited by AI — Checking the Official Documentation from OpenAI, Anthropic, and Google

As ChatGPT and Google’s AI Overviews increasingly become the entry point for gathering information, we see more and more terms referring to “optimization for being cited by AI” — AIO (AI Optimization), AEO (Answer Engine Optimization), GEO (Generative Engine Optimization), and so on. Note that the abbreviation AIO is also sometimes used to refer to Google Search’s AI feature “AI Overviews,” and the terminology is not yet standardized. Much of the information in circulation, however, consists of speculation and know-how from third-party SEO vendors, and what the companies that provide AI services themselves officially state is surprisingly poorly consolidated. In this article, drawing solely on the documentation and help centers that OpenAI (ChatGPT), Anthropic (Claude), and Google (AI Overviews, AI Mode, Gemini) publish themselves, we organize what is officially written as “what it takes to be cited by AI” (all quotations were verified on the original pages on July 12, 2026).

Starting with the conclusion — what is officially written is “allow the crawler” and “conventional SEO”

Cross-referencing the official documents from the three companies yields the following table.

Distilled down, the officially written “what it takes” comes down to the following three points.

1. Do not obstruct the necessary search/retrieval access. This is the minimum condition clearly common to all three companies. Control over training and model use is split into tokens separate from the search ones (GPTBot, ClaudeBot, Google-Extended).
2. On Google Search, conventional SEO and distinctive content continue to be the foundation. The technical foundation — crawlable and indexed — is common to all three, but it is mainly Google that goes as far as concretely recommending content with a unique perspective or firsthand experience.
3. No special technique that guarantees citation or top placement exists. OpenAI states outright that “there is no way to guarantee top placement,” and Google explicitly states that it “ignores” “special files for AI” such as llms.txt.

Below, we look at each company’s documents in turn.

OpenAI (ChatGPT) — allowing OAI-SearchBot is the first condition; there is no guarantee of top placement

There are four crawlers, controllable independently via robots.txt

In its official crawler document “Overview of OpenAI Crawlers,” OpenAI publishes four bots with different purposes.

What matters is that each setting is independent of the others.

Each setting is independent of the others – for example, a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models.

(Each setting is independent of the others. For example, a webmaster can allow OAI-SearchBot so as to appear in search results while disallowing GPTBot as a way of indicating that crawled content should not be used to train OpenAI's generative AI foundation models.)

Written in robots.txt, this looks as follows.

# Example: refuse training, allow display in ChatGPT search
User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

According to the same document, it takes about 24 hours for a change to robots.txt to be reflected in the search-side systems. The behavior when OAI-SearchBot is opted out is also spelled out.

Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.

(Sites that have opted out of OAI-SearchBot will no longer be shown in ChatGPT's search answers, though they can still appear as navigational links.)

Note that ChatGPT-User is access that goes to look at a page on the spot in response to a user’s question; it is explained that “because it is a user-initiated action, robots.txt rules may not apply” and that it “is not used to determine whether a site is eligible for inclusion in search.” The official guidance is to use OAI-SearchBot if you want to control inclusion in search.

The official answer to “how do I get listed in the search results?”

The help center article “Publishers and Developers - FAQ” contains the very question “How do I get my site to appear in ChatGPT’s search results?” The opening of the answer is as follows.

Any public website can appear in ChatGPT search. To help ensure your content can be discovered, surfaced, and clearly cited and linked, follow these guidelines: For your site content to be included in summaries and snippets in ChatGPT, make sure you aren’t blocking OAI-SearchBot.

(Any public website can appear in ChatGPT search. To help ensure your content can be discovered, surfaced, and clearly cited and linked, follow these guidelines: for your site's content to be included in ChatGPT's summaries and snippets, make sure you are not blocking OAI-SearchBot.)

The help center article “ChatGPT Search” goes further and states outright that “there is no guarantee.”

Ranking in ChatGPT Search is based on a number of factors designed to help users find reliable, relevant information. There is no way to guarantee top placement. However, to be included it is important to allow OAI-Searchbot to crawl your site, and ensure your site host and/or content delivery network allows traffic from our published IP addresses.

(Ranking in ChatGPT Search is based on a number of factors designed to help users find reliable, relevant information. There is no way to guarantee top placement. However, to be included, it is important to allow OAI-SearchBot to crawl your site and to ensure that your hosting and/or CDN allows traffic from our published IP addresses.)

The point that, in addition to “allowing the crawler,” the condition also includes “your hosting or CDN not rejecting access from OpenAI’s published IP addresses (published at openai.com/searchbot.json)” is easy to overlook. Depending on your WAF or bot-mitigation settings, you may be blocking AI crawlers unintentionally.

Beyond this, the following three pieces of official information can be read from the two articles.

• Even if you disallow via robots.txt, links may still appear. If a URL is obtained through the crawling of other pages or via a third-party search provider, the link and page title alone may be displayed; the guidance is that if you want to avoid even that, you should use a noindex meta tag (with the caveat that the page must be crawlable so the meta tag can be read).
• Referral traffic can be measured via utm_source=chatgpt.com. It is explicitly stated that this UTM parameter is automatically appended to referral links from ChatGPT.
• ChatGPT search also works with external search providers. It is explained that when searching, it “may partner with other search providers,” and Microsoft (Bing) and Shopify are named as referral sources. This is a description showing that ChatGPT’s search relies not only on its own crawler but also on existing search infrastructure, and that a site’s indexing status on conventional search engines is indirectly involved.

Anthropic (Claude) — what is published is only the crawler-control document

Three bots and the impact of blocking them

The document Anthropic publishes for site operators is the help center article “Does Anthropic crawl data from the web, and how can site owners block the crawler?.” Here, too, three bots with different roles are defined.

The two relevant to “what it takes to be cited” are the latter two. The impact of blocking them is spelled out.

Disabling Claude-User on your site prevents our system from retrieving your content in response to a user query, which may reduce your site’s visibility for user-directed web search.

(Disabling Claude-User on your site prevents our system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search.)

Disabling Claude-SearchBot on your site prevents our system from indexing your content for search optimization, which may reduce your site’s visibility and accuracy in user search results.

(Disabling Claude-SearchBot on your site prevents our system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results.)

What Anthropic officially indicates goes only as far as this: allowing these bots is the prerequisite for visibility in search and retrieval. The same document also notes that the bots respect the standard directives of robots.txt and the Crawl-delay extension, that they do not attempt to bypass CAPTCHAs and the like, and that Anthropic publishes a list of the IP addresses it crawls from.

No “optimization guide for being cited” can be found

To the extent verified, the only document Anthropic publishes for site operators is this single crawler-control document, and no official guide explaining “how to write in order to be more easily cited” could be found (as of July 12, 2026). Many articles that advertise “optimization for being cited by Claude” can be found on the web, but they are all by third parties and are not official recommendations from Anthropic — a point that warrants caution.

Google (AI Overviews, AI Mode) — “Optimizing for generative AI search is still SEO”

The official view: additional requirements are “none”

Google has the most extensive documentation of the three. First, Google Search Central’s “AI features and your website” states the conclusion at the outset.

The best practices for SEO remain relevant for AI features in Google Search (such as AI Overviews and AI Mode). There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.

(The best practices for SEO remain relevant for AI features in Google Search, such as AI Overviews and AI Mode. There are no additional requirements to appear in AI Overviews or AI Mode, nor are any other special optimizations necessary.)

The technical requirements are also identical to those of conventional search.

To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, fulfilling the Search technical requirements. There are no additional technical requirements.

(To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, fulfilling the Search technical requirements. There are no additional technical requirements.)

The means of restricting display are also the same as before: nosnippet, data-nosnippet, max-snippet, and noindex are described as applying to the AI features as well.

The official optimization guide updated in July 2026 — a direct answer to AEO/GEO

Google also publishes a dedicated guide, “Optimizing your website for generative AI features on Google Search” (the English version was last updated on July 10, 2026). It is a valuable document in that it contains a direct answer, from the provider itself, to the terms AEO and GEO.

From Google Search’s perspective, optimizing for generative AI search is optimizing for the search experience, and thus still SEO.

(From Google Search's perspective, optimizing for generative AI search is optimizing for the search experience, and thus still SEO.)

This guide explains the mechanism by which AI features retrieve pages from the search index and use them to support answers (retrieval-augmented generation, or RAG, also called grounding), as well as “query fan-out,” in which multiple related searches are run simultaneously from a single question, and then lists its recommendations. The key points are as follows.

• Create “non-commodity” content with a unique perspective. Rather than a rehash of generalities anyone could write, unique content based on firsthand experience or expertise is said to be most effective for one’s presence in generative AI search.
• Maintain technical clarity. Being indexable and crawlable, following JavaScript SEO best practices, and improving page experience — conventional technical SEO forms the foundation directly.
• Do not mass-produce pages for every variation of an anticipated question. Mass generation whose primary purpose is to manipulate ranking or generative AI answers is explicitly stated to violate the spam policies (scaled content abuse).

On the other hand, a distinctive feature of this guide is a section enumerating “what you don’t need to do.” Regarding llms.txt files, it states the following.

You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.

(You don't need to create new machine-readable files, AI text files, markup, or Markdown to appear in Google Search, including its generative AI capabilities, because Google Search itself does not use them.)

In the same section, chopping content into fragments (chunking) so AI can understand it, rewriting into an AI-oriented style, collecting unnatural “mentions” across the web, and excessive focus on structured data are also deemed “unnecessary.” Items frequently seen in third-party AIO/GEO tactics are, in effect, being rejected by name from the provider’s side.

Regarding measurement, it had long been advised that displays and clicks via AI features are included in Search Console’s Search performance report, but this optimization guide advises using a dedicated “Generative AI performance report.” It also states that, in addition to the search technical requirements, “your site must be included in (enabled for) Search’s generative AI features in Search Console.” Note that, at the time of verification, these descriptions exist only in the English version, and the measurement section is not yet reflected on the Japanese-language page.

Google-Extended — Gemini training and grounding are a separate token

Google, too, has a mechanism that separates training and model use from search, but unlike OpenAI and Anthropic, it is not an independent crawler. According to the official crawler-list document, Google-Extended does not have its own User-Agent (the crawling itself is done by the existing Google crawlers); it is a robots.txt product token that controls whether the fetched content may be used for training Gemini models and for grounding (providing the contents of the search index to the model) in the Gemini app and Vertex AI. And it is stated explicitly as follows.

Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.

(Google-Extended does not affect a site's inclusion in Google Search, nor is it used as a ranking signal in Google Search.)

In other words, the search AI features (AI Overviews, AI Mode) are strictly the domain of Googlebot and the search index, and blocking Google-Extended does not affect display in search — that is the separation.

The structure common to the three companies

Control over training/model use is separated from control over search/retrieval

Laying the three companies’ documents side by side, the same structure comes into view.

*Google-Extended is not an independent crawler but a robots.txt product token; what it controls is training and grounding in the Gemini app and Vertex AI. Google Search (including AI Overviews) is the domain of Googlebot.

Therefore, with OpenAI and Anthropic, one can configure a setup that refuses the training bot while allowing the search/user-retrieval bots — “I don’t want it used for training, but I do want to be cited in AI answers.” With Google, refusing Google-Extended does not affect inclusion in Google Search’s AI Overviews or AI Mode, but it may affect grounding in the Gemini app and Vertex AI. Conversely, sites that in the past blocked AI-related bots wholesale are worth checking to make sure the search bots were not caught up in it. The treatment of access initiated by user action differs by company: OpenAI states that “robots.txt may not apply,” whereas Anthropic states that all of its bots respect robots.txt.

An aside — “AI agent support” that lies beyond citation

This moves a step beyond search and citation, but there is a new point of discussion that appeared in common across two companies’ documents. OpenAI’s FAQ says the following about how the agent in its own browser, “ChatGPT Atlas,” understands pages.

ChatGPT Atlas uses ARIA tags—the same labels and roles that support screen readers—to interpret page structure and interactive elements.

(ChatGPT Atlas uses ARIA tags — the same labels and roles that support screen readers — to interpret page structure and interactive elements.)

Google’s optimization guide, too, explains that browser agents “interpret the accessibility tree” in order to carry out tasks. Semantic markup for screen readers and support for WAI-ARIA double directly as support for AI agents — a description showing that accessibility work is taking on a new, practical meaning.

Summary

• The first condition the three companies officially require is not blocking the search crawlers (OAI-SearchBot, Claude-SearchBot, Googlebot) via robots.txt. In addition to this, OpenAI lists that your hosting or CDN must not be rejecting access from the published IP addresses.
• Control over training/model use (GPTBot, ClaudeBot, Google-Extended) can be done with tokens separate from the search ones. OpenAI and Anthropic allow “refuse training, be listed in search.” With Google, refusing Google-Extended does not affect the AI features of search, but it may affect grounding in the Gemini app and Vertex AI.
• No technique that guarantees citation or top placement exists in any company’s official documents. OpenAI states outright that “there is no way to guarantee top placement,” and Google likewise states that “indexing and serving are not guaranteed.”
• No company requires special AI-oriented files such as llms.txt. Google goes so far as to state explicitly that “search ignores them,” and deems chunking, AI-oriented rewriting, and collecting unnatural mentions unnecessary as well.
• In Google’s official view, optimizing for generative AI search is within the scope of conventional SEO (“still SEO”). Distinctive, non-commodity content and technical soundness continue to be the foundation.
• What Anthropic publishes is only the crawler-control document, and no official optimization guide could be found (as of July 12, 2026).

When you go looking for “a special measure to be cited by AI” and arrive at the official documents, what is written there is disappointingly down-to-earth — allow the crawler, be indexed, and write content that has unique value. On the basics of conventional SEO, we have also organized the official material in “How Much Does Page Speed Affect SEO?”.

References

• Overview of OpenAI Crawlers — OpenAI (verified July 12, 2026)
• Publishers and Developers - FAQ — OpenAI Help Center (verified July 12, 2026)
• ChatGPT Search — OpenAI Help Center (verified July 12, 2026)
• Does Anthropic crawl data from the web, and how can site owners block the crawler? — Claude Help Center (verified July 12, 2026)
• AI features and your website — Google Search Central (verified July 12, 2026)
• Optimizing your website for generative AI features on Google Search — Google Search Central (verified July 12, 2026. The English version was last updated July 10, 2026. The measurement section existed only in the English version at the time of verification)
• List of Google’s common crawlers — Google for Developers (verified July 12, 2026)
• “How Much Does Page Speed Affect SEO?” — Kobayashi Shoji Blog