Adding an llms.txt file to your website and expecting ChatGPT, Perplexity or Claude to suddenly recommend your brand more often is like leaving a sticky note on your fridge and expecting your colleagues at the office to read it. The file is not where those models are looking, and it does not work the way most people think it does.
This is one of the most popular misconceptions in the AI visibility space right now — and it is spreading fast. Let's walk through what the evidence actually says.
What llms.txt was actually designed for
The llms.txt standard was proposed by Jeremy Howard (of fast.ai) in late 2024. The idea was practical: websites have complex HTML, navigation bars, cookie banners, and boilerplate that waste tokens when an AI agent needs to quickly read your documentation or extract structured content. An llms.txt file offers a clean, Markdown-formatted map of your site's most important pages, making it easier for a tool-using AI agent to navigate your content efficiently during a task — for example, a developer asking Claude to "read the Stripe docs and write integration code."
That is the use case: tool-assisted documentation reading at inference time, not brand recommendation. The standard was never designed to influence which brands a language model recommends in response to "what's the best CRM?" or "which project management tool should I use?"
The myth spreading through the industry
Despite that original intent, a large number of marketers and SEOs have concluded that implementing llms.txt is a meaningful lever for improving brand visibility in LLM responses. The logic goes roughly: "AI crawlers read my llms.txt → the LLM learns about my brand → the LLM recommends me more." Each step in that chain has a problem.
What the evidence says
Google's John Mueller: "No AI system currently uses llms.txt"
In June 2025, Google's Search Advocate John Mueller posted on Bluesky: "FWIW no AI system currently uses llms.txt." [1] He later doubled down by comparing llms.txt to the old keywords meta tag — an HTML tag that search engines abandoned years ago precisely because site owners tried to use it to game rankings rather than inform content. Google's stance is unambiguous: the consumer-facing LLMs and AI chatbots that marketers care about do not use this file. [2]
Controlled experiments show no measurable effect
Search Engine Land tracked 10 live websites before and after implementing llms.txt to measure the effect on AI traffic and citations. The results: eight sites saw no measurable change, two saw traffic increases of 12.5% and 25% — but the researchers found llms.txt was not the cause. One site actually declined 19.7%. The file upload "didn't seem to have any influence on brand recommendations." [3]
OtterlyAI ran a separate GEO study testing the same hypothesis. Their conclusion matched: implementing llms.txt produced no observable improvement in AI visibility or citation rates. [4]
SE Ranking analyzed more than 300,000 domains and found no statistical correlation between having an llms.txt file and being cited by AI systems. Both statistical analysis and machine learning showed no effect. [5]
No major AI platform has confirmed using it
Despite well over 800,000 websites now serving an llms.txt file, not a single major AI platform — OpenAI, Anthropic, Google, Perplexity, Microsoft — has officially confirmed that this file influences how their systems generate responses or recommend brands. [6]
Why the confusion exists: crawling ≠ recommending
One reason the myth persists is that AI crawlers do sometimes request llms.txt. Server logs occasionally show GPTBot or other bots fetching the file. But there is a crucial distinction between a bot fetching a file and that file having any effect on what the model says when a user asks "what tool should I use for X?"
LLM responses are shaped by two very different mechanisms:
Training data — the massive snapshot of text the model was trained on, baked into its weights. This is fixed until the next training run. Your
llms.txtfile, even if crawled, is not a preferential brand-recommendation signal; at best it may eventually end up in some future training corpus as one of billions of documents, indistinguishable from any other page on your site.Retrieval (RAG) — some AI search products like Perplexity or ChatGPT with Browse use real-time retrieval to ground answers in fresh web content. Here, what matters is whether your page ranks well enough to be retrieved for a given query — exactly like traditional search. An
llms.txtfile provides no ranking advantage in this retrieval layer either.
The llms.txt file sits outside both of these mechanisms as a meaningful influence on brand recommendations.
What actually moves the needle
If llms.txt doesn't drive brand recommendations, what does? The research is fairly consistent:
Allowing AI crawlers. Making sure GPTBot, ClaudeBot and PerplexityBot are not blocked in your
robots.txtis the actual technical lever worth checking — and it has nothing to do withllms.txt.Publishing authoritative, AI-readable content. Long-form pages (4,000–10,000 words), clear structure, and a Flesch Reading Ease of 50–65 consistently correlate with higher citation rates across ChatGPT, Perplexity and Google AI Overviews.
Being cited in third-party content. Reddit threads, review platforms, journalism, and industry roundups are heavily represented in LLM training data and in real-time retrieval. Getting mentioned in those contexts is far more impactful than any file you put on your own server.
Should you still have an llms.txt?
Having an llms.txt is not harmful, and the original use case — helping AI agents navigate your documentation during tool-assisted tasks — is legitimate. If developers are likely to ask AI tools to work with your API or docs, a well-structured llms.txt is a sensible addition. You can generate one from your sitemap using our llms.txt generator.
Just don't expect it to move your brand visibility metrics. The time saved from not obsessing over llms.txt is better spent on content, community presence, and getting your brand mentioned in the places that LLMs actually learn from.
Sources
[1] seroundtable.com — Google Says No AI System Currently Uses LLMs.txt
[2] searchenginejournal.com — Google Says LLMs.Txt Comparable To Keywords Meta Tag
[3] searchengineland.com — Does llms.txt matter? We tracked 10 sites to find out
[4] otterly.ai — llms.txt and AI Visibility: Results from OtterlyAI's GEO Study
[5] seranking.com — LLMs.txt: Why Brands Rely On It and Why It Doesn't Work
[6] amicited.com — The Truth About LLMs.txt: Overhyped or Essential?