Does llms.txt work? It does for AI coding assistants that read developer documentation. There is no public evidence that it helps you get cited in ChatGPT, Perplexity or Google's AI Overviews. No major AI company has said it reads the file in its answer systems, and large crawl studies show bots almost never request it.
So treat it like this. It's a cheap, low-risk file that might help a few tools. It won't move your AI visibility much. Below you'll find who reads it, the study numbers with their sources, a test you can run in your own logs, and a grid to decide if your site needs one.
What llms.txt is (and what it is not)
llms.txt is a plain Markdown file at the root of your site (yoursite.com/llms.txt). It lists your most useful pages, each with a short description, so a language model can find the right content without crawling everything. Jeremy Howard proposed it on 3 September 2024, and the spec has been updated since.
The format is simple:
- One H1 with the name of the site or project (the only required part)
- A short summary in a blockquote
- H2 sections with lists of links, each followed by a note
- An "Optional" section that a model can skip when it's short on context
Here's what it is not:
| File | What it does | Who has committed to reading it |
|---|---|---|
| robots.txt | Tells crawlers what they may or may not fetch | Major search engines and AI crawlers, including OpenAI's bots |
| sitemap.xml | Lists the URLs you want indexed | Google and Bing, through Search Console and Webmaster Tools |
| llms.txt | Suggests which pages matter, in Markdown | No major AI provider publicly |
llms.txt is not an access control file. It blocks nothing and allows nothing. It's also not an official standard: Geekflare notes that it's a community convention, not a W3C or IETF standard.
If you want to try one, start with our free llms.txt generator, then read the file once before you upload it.
Do ChatGPT, Claude, Gemini or Perplexity read llms.txt?
Going by public statements, they don't. Here's what each one has said, with the source.
| Who | What is publicly known | Source |
|---|---|---|
| OpenAI (ChatGPT) | No official support for llms.txt. Follows robots.txt. | Ahrefs, 2026 |
| Google (Search, AI Overviews, Gemini) | Has said it doesn't use llms.txt for Search or its AI features. John Mueller compared it to the old keywords meta tag. | Core dna, Ahrefs |
| Anthropic (Claude) | Publishes its own llms.txt but hasn't said its crawlers read other sites' files. | Ahrefs |
| Meta | No public guidance that points to llms.txt. | Ahrefs |
| Coding assistants (Cursor, Windsurf, Claude Code, GitHub Copilot) | Look for it when you point them at a documentation site. | Core dna |
Mueller's comparison is the useful one. The keywords meta tag was easy to fill in, and search engines stopped trusting it. A file that any site can write to describe itself has the same weakness. That helps explain why answer engines haven't adopted it.
What the studies found about llms.txt
Three kinds of numbers keep coming up: how many sites publish the file, how often bots request it, and whether having it goes with more AI citations.
Adoption depends on the sample:
- Ahrefs looked at 137,000 domains and found that 28% publish an llms.txt.
- SE Ranking found about 10% across roughly 300,000 domains (as reported by Core dna). Geekflare gives its breakdown: 10.13% overall, 10.54% for mid-traffic sites and 8.27% for high-traffic sites.
Requests are close to zero:
- Ahrefs looked at about 38,000 domains with a valid file. 97% got no request for it at all in May 2026 (Ahrefs).
- An OtterlyAI test found that only about 0.1% of more than 60,000 AI bot visits touched llms.txt over 90 days (Core dna).
No link with citations: the SE Ranking and Trakkr data found no link between having the file and being cited more often (Core dna).
Put together: plenty of sites publish the file, bots barely ask for it, and having it doesn't predict citations.
Does llms.txt help you get cited in AI answers?
Not in any way anyone has measured. ChatGPT, Perplexity and AI Overviews find pages through their crawlers and search indexes. The file is not part of that path.
Where it does work today is developer documentation. When a developer tells Cursor or Claude Code to "read the docs at example.com", the assistant can open llms.txt to find the right pages quickly. That's why Stripe, Vercel, Cloudflare and Anthropic publish one (Core dna). Their readers are developers who use those tools every day.
Test it on your own site in 10 minutes
Don't take any study's word for it. Your server logs will show whether AI bots ask for your file. This works on any site where you can read the access logs: WordPress on most hosts, a VPS, a SaaS behind Nginx or Apache, or a CDN like Cloudflare.
- Download your access log. In cPanel it's under "Raw Access" or "Metrics". On a server it's usually
/var/log/nginx/access.logor/var/log/apache2/access.log. - Count the requests for the file:
grep "llms.txt" access.log | wc -l - See who asked for it:
grep "llms.txt" access.log | grep -iE "GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|PerplexityBot|Perplexity-User|bingbot" - Compare with your real pages. Run the same bot filter on a popular blog post's URL. If GPTBot fetched that post 40 times and llms.txt twice, you know where the bots spend their time.
- Check the status code. A 404 on
/llms.txtmeans the file isn't where you think it is. A 403 on your blog posts for bot user agents is a much bigger problem than a missing llms.txt.
Hosted Shopify stores don't give you raw access logs. If your domain goes through Cloudflare, its analytics can show requests by path and by bot. If not, you can't measure it, which is one more reason not to make it a priority.
Is llms.txt worth it for your site?
Use this grid. It's based on who actually reads the file today.
| Your site | Add llms.txt? | Why |
|---|---|---|
| Developer docs, API, open source project | Yes, and keep it current | Coding assistants read it when users point them at your docs |
| SaaS with public documentation | Yes, for the docs section | Same reason, aimed at the developers who integrate your product |
| SaaS marketing site with no docs | Optional, 15 minutes at most | No measured effect on AI answers |
| Shopify store | Optional, low priority | No proven effect, and no logs to check it |
| WordPress blog or content site | Optional, low priority | Your posts and robots.txt matter far more |
| Local service business | Skip it for now | Your time goes further on service pages and reviews |
On a "Yes" row? Try our free llms.txt generator, then read the file once before you upload it.
Can llms.txt hurt your SEO? No case of that has been reported. It's a text file that Google says it doesn't use. The only real cost is your time, plus the risk of a stale file that points to deleted pages.
Quick setup by platform
- WordPress: upload
llms.txtto the root folder (the one withwp-config.php) by SFTP or your host's file manager. Some SEO plugins now offer an llms.txt option, so check your plugin's settings before installing anything new. - Shopify: you can't drop a file at the domain root from the admin. A common workaround is to upload the file under Content > Files, then create a URL redirect from
/llms.txtto that file's address. Apps from the Shopify App Store can also do it. - Next.js, Astro, Hugo or another static site: put
llms.txtin thepublic(orstatic) folder, deploy, then open yoursite.com/llms.txt to check.
Whatever the platform, open the URL in your browser after uploading. If you see your Markdown as plain text, it's live.
What to fix before llms.txt
llms.txt comes last. Bots first have to be allowed in. Then they have to find pages that answer the question clearly, with sources. We cover those steps in how to rank in ChatGPT and on our LLM SEO page. To check a page before anything else, run it through our free SEO checker.
How AI SEO Crew handles this
The crew writes researched articles that answer real Google searches. The editor checks every fact and link and adds internal links before anything goes out. After every article, the Publisher sends your sitemap to Google and pings IndexNow. llms.txt stays a free extra you can make yourself with the generator.
Want the crew to write the pages AI tools and Google actually read? Free keyword plan and your first article. No card needed.
Frequently asked questions
Does Google use llms.txt for AI Overviews?
No. Google has said it doesn't use llms.txt for Search or its AI features, and John Mueller compared the file to the old keywords meta tag. AI Overviews draw on pages Google has already crawled and indexed.
Do I need an llms.txt file to show up in ChatGPT?
No. ChatGPT finds pages through its crawlers and search, not through llms.txt, and OpenAI hasn't said it reads the file. Make sure robots.txt lets OpenAI's bots in and that your pages answer questions clearly.
What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers what they're allowed to fetch, and major crawlers follow it. llms.txt only suggests which pages matter, in Markdown, and blocks nothing. If you only have time for one, check robots.txt.
How do I know if AI bots are reading my llms.txt?
Search your server access log for "llms.txt", then filter by AI user agents such as GPTBot, ClaudeBot or PerplexityBot. If you get no lines, the bots aren't asking for it. On Shopify without Cloudflare, you can't see this data.