Does llms.txt Work in 2026? What the AI Citation Data Shows

No, llms.txt does not improve AI citations in 2026. Ahrefs analyzed 137,210 domains and found that 97% of published llms.txt files received zero requests in May 2026. Google's own generative AI search guide lists llms.txt among tactics site owners can ignore entirely. AI retrieval bots such as OAI-SearchBot and PerplexityBot made up only 1.1% of the few requests these files did get. So for most businesses chasing mentions in AI Overviews, ChatGPT, or Perplexity, llms.txt is not the lever that moves the needle.

llms.txt itself is a plain Markdown file placed at a website's root. It lists a site's key pages so AI systems can reference them without crawling everything. Proposed in 2024 as a lightweight alternative to full-site crawling, no major AI platform has committed to reading it inside production search or citation systems.

What llms.txt Is (and Isn't)

llms.txt is an index file, not a ranking signal or crawler directive. It sits at a domain's root in Markdown format and points AI systems toward important pages on the site.

Think of it as a map, not a set of directions. It can show an AI system what you consider important, but it can't force the system to crawl, trust, or cite those pages.

Unlike robots.txt, it doesn't control crawler access. And unlike a ranking signal, it doesn't directly improve your visibility. If you're trying to separate llms.txt hype from the AI-visibility work that actually matters, it's worth understanding where GEO and AEO diverge, since the two are often confused.

Does llms.txt Improve AI Citations in 2026?

In June 2026, Ahrefs published a study analyzing 137,000+ domains to see how llms.txt files are actually being used. The results were pretty clear: 97% of published llms.txt files received zero requests of any kind in May 2026. Among the small share that were fetched, AI retrieval bots accounted for just 1.1% of that traffic.

Named AI tools overall made up 19.5% of requests, but most of that came from coding agents rather than AI search crawlers.

We see the same pattern across PilotDeck client accounts. Sites that gained AI citations did it through fresh, well-structured content, not through an llms.txt file.

"Ahrefs' own numbers show that SEO audit tools generate more llms.txt traffic than any AI bot category combined. That is the file's real audience today, not AI search." — JC, Co-founder, PilotDeck

Why Do AI Crawlers Skip llms.txt Files?

AI crawlers like GPTBot, ClaudeBot, and PerplexityBot are built to read HTML pages directly, so a separate summary file gives them nothing they can independently verify. Retrieval-augmented generation systems pull from indexed, crawled content, not from a webmaster's self-description.

The same research found zero AI bots requesting llms.txt on domains where the file did not exist. So AI systems aren't exactly scouring the web looking for your missing llms.txt file. Publishing one doesn't appear to be what puts a site on their radar.

Before publishing anything, it also helps to check which AI crawlers you currently allow, since a stale llms.txt raises security questions faster than SEO ones.

Is llms.txt Necessary for AI Visibility?

No. Google's guide to optimizing for generative AI search lists llms.txt under tactics site owners can ignore, according to its generative AI optimization guide (2026). The guide states plainly that Google Search does not use machine-readable files, AI text files, or Markdown copies as a visibility input.

What still matters is the same technical and content foundation that has always driven organic rankings: crawlable pages, clear structure, and content people find genuinely useful. Google's guidance covers its own generative AI features specifically, though independent crawl data shows the same low engagement pattern across ChatGPT, Claude, and Perplexity too.

Should You Still Create an llms.txt File?

Yes, but treat it as nice to have not something to bet your AI visibility on. The file costs little to build and neither helps nor hurts rankings or citations either way, based on every data point gathered so far.

The clearest use case is coding agents. Claude Code out-fetched every AI search and retrieval bot in the Ahrefs dataset, so teams with developer-facing products may see more return than a marketing-only site would. Checking AI visibility tracking tools first will tell you whether the effort is worth your time at all.

How Should You Implement llms.txt?

Keep it short, link only to pages you maintain, and treat it like code, not a marketing asset. A bloated or stale file works against the one use case that still holds up.

1. List 3 to 8 of your most important pages with a one-line description each, not your entire sitemap.
2. Use plain Markdown links and skip vague anchors like "click here" or "read more."
3. Host the file at your domain root, such as yoursite.com/llms.txt, not in a subfolder.
4. Version-control the file and restrict who can edit it, since agents are built to trust its content.
5. Review it every quarter, and update it immediately after removing or merging pages.

At PilotDeck, we fold this kind of check into monthly site audits, so it never becomes a project of its own.

What This Means for Your Site

llms.txt is cheap to publish and easy to overestimate its effectiveness. The 2026 data is consistent across every major study: AI search and citation systems are not reading it, and Google has said as much directly.

Spend the effort instead on content structure, freshness, and third-party mentions, since those are the levers with measured ties to AI citations. Ship llms.txt if it takes half an hour. Do not build a strategy around it.

JC

Co-founder, PilotDeck

Jignesh brings years of experience in software development and technology, helping build scalable web applications and data-driven solutions. He explores AI, software engineering, and emerging technologies to turn complex ideas into practical solutions.