If someone told you that you have to add an llms.txt file or fall behind in AI search, take a breath. You do not have to decide today, and you do not have to trust the hype. Let's look at what the evidence actually shows about llms.txt effectiveness, so you can make a calm, informed call instead of a fearful one.
Here is the honest headline, and we will spend the rest of this piece backing it up: for most marketing and content sites, there is no current evidence that llms.txt improves your visibility in AI answers. Not from the engines. Not from the studies. Not from the traffic logs. This piece is not about how to build the file. It is about whether the file does anything worth your time.
You are not behind for asking the question. You are ahead for asking it before you spend a single hour on it.
The Short Answer: Does llms.txt Work?
Does llms.txt work the way its fans promise? Based on every study and engine statement we have as of mid-2026, no, not for AI search visibility on a typical marketing site.
llms.txt is a real, documented proposal. It was published on September 3, 2024 by Jeremy Howard, co-founder of Answer.AI and fast.ai. The idea is genuine and the format is well specified. What it is not, and this is the part the hype skips, is a standard, a crawler rule, or a recognized signal that any major AI engine has agreed to honor.
So the question is not "is llms.txt real?" It is real. The real question is whether it works as an llms.txt ranking signal, moving how often ChatGPT, Perplexity, Gemini, or Google's AI answers cite or mention your brand. On that specific question, the evidence points one direction, and it is not the hopeful one.
Let's walk through why, step by step. You can decide for yourself at the end.
What llms.txt Actually Is (and What It Is Not)
Before we judge the evidence, let's get the object right, because a lot of confusion comes from calling llms.txt something it is not.
llms.txt is a Markdown file placed at the root of a domain, like yoursite.com/llms.txt. It summarizes your most important pages in a clean, machine-readable index: a title, a short summary, and sections of links with brief descriptions. There is also an optional companion file that bundles the full text of every linked page into one document.
That is the whole thing. A curated map of your site, written for machines that choose to read it.
Now the part that matters for your decision. llms.txt is not a W3C, IETF, or RFC standard. No governing body oversees it. It is not a crawler control mechanism, so it does not block or allow bots the way robots.txt does. And nothing compels any AI company to fetch it or honor it. It is opt-in on both sides: you offer it, and the AI system decides whether to care.
That last point is the crux. robots.txt works because every compliant crawler respects it under threat of being shut out. llms.txt has no such enforcement and no such adoption. Calling it "the new robots.txt" is where a lot of people go wrong.
What the AI Engines Have Actually Said
If llms.txt were a real llms.txt ranking signal, the companies behind the AI engines would tell you. So what have they actually said? Mostly, they have told you the opposite.
Start with Google, because Google was the most direct. Google's official guide on optimizing your website for generative AI features states plainly that llms.txt is not necessary for inclusion in AI Overviews or AI Mode. The guide lists what publishers should do (quality content, technical SEO fundamentals, structured data, crawlability) and singles out llms.txt by name as something you do not need to add. That is about as close to an official "no" as the industry gets.
Google's John Mueller went further in public. He described llms.txt as "not done for search" and called it a "temporary crutch, perhaps to save some tokens" for AI coding tools reading developer docs. When someone pointed out that Google had shipped an llms.txt check in its Lighthouse tool, Mueller held his ground: site owners who check their server logs will see very little AI agent traffic hitting the file. He even compared llms.txt to the old keywords meta tag, the tag search engines abandoned years ago.
That Lighthouse check is worth understanding, because it gets misread as an endorsement. The audit only confirms the file exists and parses correctly. It returns "not applicable" when the file is missing, passes when it is well formed, and fails only when it exists but is broken. Presence in a testing tool is not a promise that anyone uses the file.
What about the AI labs themselves? OpenAI, Anthropic, and Perplexity all host llms.txt on their own documentation sites. That sounds like a signal until you look closer. None of them has confirmed that their consumer product, ChatGPT or Claude or Perplexity, fetches and uses third-party llms.txt files when answering a user's question. They publish the file for their own API docs, so AI coding assistants can read them. That is a very different thing from grounding search answers in your llms.txt.
So the engine scorecard reads like this: one explicit "you do not need this" from Google, and silence from everyone else on the claim that matters. Silence is not proof. But when a signal is real, the platforms usually say so.
What the Studies Actually Measured
This is the part that should settle your nerves, because you do not have to take anyone's word for it. Several independent teams went and measured what happens to llms.txt files in the wild. The picture they paint is remarkably consistent.
The traffic almost nobody sees
Ahrefs ran the largest look, studying 137,210 domains that received real traffic in May 2026. Their headline finding is hard to argue with: 97% of llms.txt files received zero requests that month. No bots, no humans, nothing. Of the small slice that got any traffic at all, 96% of it came from bots rather than people.
And here is the detail that reframes everything. The single biggest fetcher of llms.txt files was Claude Code, an AI coding assistant that pulls documentation, sending more requests than every named AI search and retrieval bot combined. Slackbot, the thing that generates link previews in Slack, requested llms.txt more often than PerplexityBot did. Googlebot did not request the file as part of its crawl pipeline at all. And crawlers never went looking for llms.txt on sites that did not have one, meaning they do not hunt for it.
Read that again. The tools touching these files are developer tools and link-preview bots, not the consumer AI search products you actually care about being visible in.
No correlation with citations
SE Ranking scanned around 300,000 domains in November 2025 and found llms.txt adoption at about 10%. More important than adoption, they checked for a link between having the file and getting cited by LLMs. They found no statistical correlation.
They went one step further. They trained a machine learning model to predict how often a page gets cited in AI answers, then tested whether the llms.txt feature helped the prediction. The model performed better with llms.txt removed. In plain terms, the file added noise, not signal. (Worth noting: this is an observational study, so it shows a lack of correlation, not a proven cause. But a lack of correlation is exactly what you would expect from a file nobody reads.)
A field test on a live site
OtterlyAI ran a 90-day experiment: publish a valid llms.txt, then watch the AI bot traffic. Over three months, the file got 84 requests out of more than 62,100 total AI bot visits to the site. That is roughly one tenth of one percent of all AI bot activity. A normal content page on the same site pulled around 265 AI bot visits in the same window. The file was, in practical terms, invisible.
Real adoption is tiny
Back in May 2025, Chris Green crawled the top one million domains looking for a valid llms.txt. He found 15. Most sites returning a page at the llms.txt address were actually serving a default CMS page, not a real file. llms.txt adoption has grown since (Ahrefs found 28% among trafficked domains a year later), but a lot of that growth is plugins switching the file on by default, not teams choosing it on purpose.
Put the four studies together and the story is simple. Most llms.txt files are never fetched. When they are, the traffic comes from coding assistants and preview bots, not answer engines. And having one shows no measurable link to getting cited. That is what the llms.txt effectiveness evidence looks like when you actually count.
The Risks the Hype Skips
Here is where a low-cost file can quietly cost you something. "It cannot hurt" is the most common reason people add llms.txt. It can, a little, and you deserve to know how.
The sharpest critique comes from SEO consultant Jono Alderson. The moment you publish an llms.txt, you create a second canonical version of your site's content sitting alongside your real pages. AI agents are built to treat that file as authoritative input. If it goes stale, gets inaccurate, or quietly misrepresents what you offer, a model that reads it absorbs those errors as truth. You now have two versions of the truth to keep in sync, and only one of them is the one your team actually updates.
There is a security angle too. A curated text file is a clean delivery vehicle for hidden instructions. Indirect prompt injection through external content is listed as the top risk in OWASP's Top 10 for LLM applications. A file you control, that models are designed to trust, is exactly the surface that risk describes.
Then there is the default-on problem. Plugins for WordPress, Wix, and various docs platforms now add llms.txt automatically. Many teams end up with one they never chose and never maintain. The result is an ecosystem filling with thousands of auto-generated, never-updated files. That makes the whole signal noisier for any engine that might one day want to use it, and it hands content teams a false sense that they have "done the AI visibility thing" when they have not touched the things that matter.
None of this is catastrophic. But a file that does not move citations, and does carry maintenance load and accuracy risk, is not the free win it gets sold as.
Where llms.txt Genuinely Earns Its Place
Let's be fair, because the file is not useless. It just has a narrow, real home, and it is probably not your home.
llms.txt has found genuine product-market fit in one place: developer documentation and API reference sites read by AI coding assistants. Tools like Claude Code, Cursor, Continue, Cody, and Windsurf routinely fetch llms.txt to ground their code generation in current library and API docs. That is why Mintlify, Stripe, OpenAI, Anthropic, Cloudflare, and similar developer-focused companies publish it. For that audience, the file measurably cuts token usage and improves the quality of generated code. It does a real job.
Notice what that job is not. None of those companies claims llms.txt boosts their visibility in ChatGPT, Gemini, or Perplexity answers. They publish it so coding agents can read their docs, full stop.
So if your site is developer documentation, an SDK guide, a framework tutorial, or an API reference, llms.txt is a reasonable, low-cost add, because your consumers (coding agents) are proven to read it.
If your site is marketing, e-commerce, B2B SaaS landing pages, news, or content marketing, there is no current evidence the file influences AI Overview inclusion, Perplexity citations, ChatGPT mentions, or Gemini answers. Different audience, different file, different outcome.
So Is llms.txt Worth It for Your Site?
Let's turn all of this into a decision you can actually make. Is llms.txt worth it? It depends on one question: who reads your site's machine-readable index, coding agents or answer engines?
Publish llms.txt if any of these are true:
- Your site is developer docs, an API reference, an SDK guide, or a framework tutorial. Coding agents are the proven consumer.
- Your CMS generates and maintains the file for you, with no manual upkeep on your side.
- You can genuinely keep it accurate, current, and free of dead links.
Skip llms.txt if any of these are true:
- Your site is marketing, e-commerce, SaaS landing pages, news, or content marketing. The evidence does not support a visibility lift here.
- You cannot keep it in sync with your real content. A stale llms.txt is worse than none, because models are built to trust it.
- You are reaching for it instead of the fundamentals that actually drive AI visibility.
And whatever you decide, do these two things. Audit what your llms.txt says before it goes live, since models will treat it as ground truth. And treat the file as a developer-ergonomics tool, not an SEO lever, so you set your expectations where the evidence sets them.
If you want to keep an eye on the future, watch four things: a formal adoption announcement from an AI lab, a standards body picking it up, agentic browsers changing how they behave, or Google's AI optimization guidance shifting. Any of those would change this math. As of mid-2026, none has.
What Actually Moves Your AI Visibility
Here is the reassuring part. The time you were about to spend agonizing over llms.txt is better spent on things that are proven to matter, and you probably already know how to start on most of them.
AI answers, including Google's AI Overviews, run on the same core retrieval and ranking systems as regular search, layered with retrieval-augmented generation. What feeds those systems is not a summary file. It is crawlable HTML, clear structure, strong entity signals, schema markup where it fits, sensible internal linking, and content that is genuinely worth citing. Those are the levers with evidence behind them.
This is exactly the trap the fundamentals protect you from: chasing a file that feels productive while the real signals go untended. The honest move is to measure where you actually show up in AI answers today, find the specific questions where a competitor gets cited and you do not, and fix those pages. That is the work that pays off.
If you want that measurement handled for you, this is where a platform like DeepSmith helps. It tracks how AI engines answer the questions that matter in your space, shows you the exact prompts and pages where you are invisible or losing, and produces the on-brand content to close those gaps, all from the same data. You get to spend your attention on the signals that move citations, not on a file the studies say nobody reads.
You do not need a bigger checklist. You need to point your effort at what the evidence rewards. Want to see where you actually stand in AI search right now? Start a free DeepSmith trial and get real data before you spend another hour guessing.
