- Whether llms.txt helps AI search visibility is unresolved. Google says its Search ignores the file. Other claims that some AI engines read and weight it are reported by third parties and are not confirmed by the companies concerned.
- No validated, reproducible case study shows that publishing the file raises citations. Practitioner server-log audits report that AI crawlers largely do not fetch it.
- The file is cheap and carries no known downside, so publish one and then stop. Its honest ceiling is a slightly cleaner starting point for a crawler that already chooses to fetch it.
- The real risk is mistaking the file for the work. A brand that ships llms.txt and considers its AI visibility handled has done the one task fully within its control and none of the harder ones above it.
- In Singapore, the file is sold inside a uniform GEO checklist. Separating each item by evidence quality is the more useful service.
What is llms.txt, and what does it claim to do?
llms.txt is a proposed convention, not a formal standard. It was put forward in September 2024 by Answer.AI as a way to give language models concise context about a site without making them crawl and interpret every page. The file is written in Markdown. It typically opens with the site’s name and a one-line description, then lists key pages with a short note on each.
Its claim is simple. If AI systems increasingly decide what gets surfaced, a brand should be able to tell them which pages matter. The comparison to robots.txt and sitemap.xml is where the trouble starts. Those files have formal roles and documented support from the engines that use them. llms.txt, as Contentful’s June 2026 review puts it, "doesn’t currently have the same level of formal standardization, platform adoption, or public documentation from major AI providers."
What llms.txt is not is a ranking mechanism, an instruction that engines must obey, or a substitute for the content it points to. A file can describe a house. It cannot make the house worth visiting.
Google says plainly that it does not use the file, and its reason is structural
Google’s guide to generative AI features on Google Search sets out the position in its "what you don’t need to do" section. The operative sentence reads: "You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them."
The reason is architectural. Google explains that its AI features are rooted in its core ranking and quality systems, using retrieval-augmented generation to pull pages from the Search index and then summarise them. A page must already be indexed and eligible to appear with a snippet. A separate text file sits outside that pipeline.
There is a nuance worth stating precisely. Google clarified its wording in June 2026. Search Engine Journal’s report of that update quotes Google’s guidance as saying it is "completely fine" to maintain such files for other services that use them, and that doing so "won’t harm (nor help)" visibility in Google Search. So Google’s position is narrow and clear. It speaks for Google Search. It does not speak for any other engine.
Google does not say llms.txt is useless. It says its own Search ignores the file, and that is a statement about one engine.
Other engines are reported to read it, and "reported" is doing all the work in that sentence
The claim that Anthropic’s and Perplexity’s products fetch and weight llms.txt circulates widely, mostly in GEO research and vendor content. We could not locate a primary statement from either company that confirms it.
What the primary documents do show is more modest. Perplexity’s crawler documentation describes PerplexityBot and Perplexity-User and tells site owners to manage access through robots.txt. It makes no statement in the passage we opened that llms.txt is read or weighted. Perplexity does publish an llms.txt for its own documentation, and the page itself invites readers to fetch the index. That shows the file is being published. It does not show an answer engine consuming other brands' files to decide who gets cited.
Publishing a file and consuming one are different acts, and the whole claim rests on the second. Until a company states in its own documentation that its systems read and use the file, the honest label is "claimed, not independently verified".
Zero validated case studies is not a rounding error, it is the finding
Contentful’s review, published on 3 June 2026, concludes that "there is no widely validated evidence that llms.txt reliably improves AEO visibility, citation frequency, referral traffic, or inclusion in generated answers." It recommends treating the file as experimental, low cost and low confidence.
The same review quotes Olivya Pastis, Senior SEO/GEO Analyst at Seer Interactive, on the practitioner view: "Google has explicitly stated they don’t use the file for their AI experiences, and server log audits back that up. Actual LLM crawlers largely aren’t fetching it." Seer Interactive’s position is that content quality, structured data and entity clarity have demonstrated impact on generative search visibility, and that the file is not a priority for most clients.
The reason a case study is hard to build is also instructive. AI answers are non-deterministic, attribution is patchy, and visibility moves for many reasons at once. A brand that launches the file in May and sees more citations in June cannot say the file caused it, because content updates, new brand mentions and platform changes all landed in the same window. That is not a flaw in any one study. It is why the confident claims on both sides outrun the evidence.
Exhibit 1: The llms.txt Reality Check, what is verified against what is claimed
The table separates each claim made for llms.txt from what can be verified at the primary source. It is deliberately even-handed. Where a claim cannot be confirmed, the entry says so rather than resolving it either way.
| Claim made for llms.txt | What is verified, as of 3 October 2026 | Who is making the claim |
|---|---|---|
| Improves inclusion in Google’s AI Overviews or generative search | No. Google states that Google Search itself does not use such files and that no special file or markup is needed to appear in its generative AI features. | Google, in its Search Central guidance |
| Improves citation frequency across AI engines generally | Not demonstrated. No validated, reproducible multi-domain case study has been published. Practitioner server-log audits report inconsistent crawler fetching. | Practitioner audits, such as Seer Interactive as quoted by Contentful |
| Is read and weighted by Anthropic’s and Perplexity’s products | Claimed, not independently verified. No primary statement from either company confirming this was found. Perplexity’s crawler documentation points site owners to robots.txt. | Third-party GEO research |
| Works as a machine-readable summary a crawler can fetch cheaply if it chooses to | Yes, verifiably. This is what the file mechanically is: a plain-text page a crawler may request. Whether any crawler requests or acts on it is the unresolved part. | The file’s own specification |
| Can replace schema markup, a definition-lead opening or a maintained entity description | No. Treating it as a substitute is the real risk. | This article’s argument |
Exhibit 1. Compiled from Google Search Central, Contentful (June 2026) and Perplexity’s crawler documentation.
Publish one anyway, for the one thing it verifiably does, and then stop
The cost-benefit is short. A competent team writes and publishes the file in under an hour, and no source reviewed here reports any penalty or harm from a clean, accurate one. The ceiling on the benefit, given Exhibit 1, is that a crawler which already chooses to fetch the file gets a slightly cleaner starting point. That is the whole case for publishing one, and the whole case against treating it as strategy.
Think of a building permit application. Filing it correctly satisfies a requirement, but it does not decide whether the building is approved. The decision rests on what is being built.
The failure mode worth avoiding is not publishing the file. It is mistaking the file for the work. Influence over what an AI engine says about a brand rises as the brand’s control falls: its own pages matter, but the engines weight what others say about it more heavily. The file sits on the one rung that is entirely within a brand’s control. A brand that stops there has done the easy rung and skipped the others.
We publish an llms.txt file for the reason given in the fourth row of Exhibit 1, and for no larger claim than that. For the broader argument, see our analysis of why AI does not cite your website. For the practical steps that do have support, see how to get your brand mentioned by ChatGPT.
In Singapore the file is sold as one item in a uniform checklist, and the items are not equal
The on-page GEO checklist offered across Singapore’s branding and digital marketing sector, typically schema markup, an llms.txt file, FAQ blocks and a definition-lead opening, is usually presented as a single low-controversy bundle. The evidence does not treat the items equally. Google’s own guidance says no special schema is required for its generative AI features, though it advises continuing to use structured data as part of overall SEO. A definition-lead opening and a clear entity description are ordinary clarity, which Google’s guidance supports through its call for unique, non-commodity content.
The useful service for a Singapore founder or in-house brand lead is to separate the checklist by evidence quality. Some items are good practice with a clear rationale. Others are cheap hedges with no demonstrated effect. A few are marketed with more confidence than the sources allow. No regional data point on llms.txt effect has been published to our knowledge, and none is offered here.
Sizing the effort follows from that. Put an afternoon into the file if a vendor has asked for it, and put the programme budget into the work that has evidence behind it. For context on where the measurable returns sit, see our review of whether generative engine optimisation works and our overview of how AI is changing branding.
What to watch before the position changes
The picture is live, and it will change. Contentful’s review sets out what would shift it: major AI platforms documenting support for the file, a formal standard emerging, server logs showing consistent bot behaviour tied to the file, or independent case studies showing reproducible impact across multiple domains and query types. Any one of those would justify revisiting this article. None had been reported at the time of writing.
Until then, the calibrated position is neither that llms.txt works nor that it is a myth. It is that the question is open, the file is inexpensive, and the decision that matters is where the real effort goes.
Related Vantage insights
- For the broader argument that on-page tactics are necessary but insufficient, read why AI does not cite your website.
- For the general how-to, see how to get your brand mentioned by ChatGPT.
- For the wider evidence review, see does generative engine optimisation work.
- For the strategic context, see how AI is changing branding.
- For the foundation that makes any description accurate, see our brand positioning framework.
About Vantage Branding
Vantage is a Singapore brand consultancy specialising in brand research, strategy, and identity design for ambitious organisations across Southeast Asia, with particular depth in healthcare, finance, government, and cultural-institution branding. Vantage builds fewer, stronger brands, pairing research rigour with senior craft across strategy, identity, experience and activation. Enterprise Singapore PMC-certified and EDG-eligible. For a conversation about how we could help your brand, get in touch.