llms.txt Guide: How to Write One AI Engines Can Actually Use
Published 2026-09-07 · Updated 2026-09-07 · 6 min read · AI 工作術 (FreeCo Co., Ltd.)
llms.txt is a Markdown file that tells AI models what your site is about. Full structure, a worked example, writing rules, and an honest look at adoption.
If robots.txt tells crawlers where they're allowed to go, llms.txt tells AI models what your website is actually about. It's a plain Markdown file at your site root — yourdomain.com/llms.txt — that introduces your site to language models in the most compact form possible: who you are, what you offer, and where the important pages live. We run our own AI tool platform and we ship an llms.txt on it; this article lays out exactly how to write one.
Why bother? Because of how AI engines read websites. They fetch pages, strip the HTML, and try to reconstruct meaning from the leftover text — with navigation labels, footer links, and cookie-banner copy all mixed in. That process is expensive and lossy. The logic of llms.txt is blunt: instead of making the AI guess what your site is from a pile of scraped fragments, hand it a manual you wrote yourself.
Fair warning up front: this is an emerging convention, not a ratified standard, and no engine guarantees it reads the file. We'll get to that honestly below. But the cost is one afternoon, and the downside is essentially zero.
The Standard Structure, Top to Bottom
The format was proposed by Jeremy Howard of Answer.AI in 2024, and it's deliberately simple — everything is plain Markdown, in this order:
- One H1 heading — your site or organization name. This is the only element the spec actually requires.
- One blockquote — a one-to-two-sentence summary of what the site is. This is the first self-introduction any AI reads, so it deserves more revision passes than anything else in the file.
- A few freeform paragraphs — supporting context: who you serve, what you specialize in, where you're located, how to reach you. Short, plain sentences.
- Several H2 link sections — each section holds a list of links, and each link gets a one-line description of what's on that page. Typical sections: Products, Services, Docs, Articles, Contact.
- An "Optional" section — a specially named H2 for secondary links, which explicitly signals to the AI: "if you're short on context budget, skip these."
That's the entire format. No XML, no schema registration, no build step. If you can write a README, you can write an llms.txt.
A Worked Example: How We Structure Ours
Here's the shape of a real file, using ours as the template:
- H1: the platform name.
- Blockquote: "An AI tools platform offering one-click video clipping, ad copy generation, and a free AI knowledge base for working professionals." One sentence, and a model instantly knows the category, the offerings, and the audience.
- Description paragraphs: two or three short ones — what makes the products different, who they're built for, and that the knowledge base is free.
- Link sections: a Products section listing the core tools (like our video clipping tool and ad generator), each with a one-liner on what it does and who it's for; an Articles section with roughly a dozen of our most representative guides — not the whole archive; a Contact section with exactly one link.
- Optional section: legal pages, changelogs, anything an AI can safely skip.
Total length: under 100 lines. That's a feature, not a limitation.
Three Writing Rules That Matter More Than the Format
Getting the structure right is trivial. These three rules are where files actually succeed or fail:
- The one-line description matters more than the link. The AI uses that sentence to decide whether the page is worth fetching at all. "Pricing" tells it nothing. "Plans and what each tier includes, with a free tier available" tells it exactly when to follow the link.
- Shorter is better. llms.txt is an index, not a content dump. Dumping 500 URLs into it defeats the purpose — you're supposed to be doing the curation so the model doesn't have to. Pick the pages that best represent you and cut the rest.
- Write in the user's language, not yours. Imagine how someone would actually phrase the question to ChatGPT — "tool that turns long videos into short clips automatically" — and make sure your description sentences naturally contain those phrasings. Internal jargon and product codenames are invisible to someone who's never heard of you, and the AI is answering on behalf of exactly that person.
There's a hidden benefit here too. We've watched teams struggle to write this file, and it's almost never because Markdown is hard — it's because they've never actually compressed "who we are and what we sell" into three clear sentences. If writing your llms.txt is painful, that pain is diagnostic. Fix the positioning and the file writes itself.
Does AI Actually Read It? An Honest Answer
Sometimes, and it's improving, but nobody guarantees it. Adoption among AI engines is uneven and changing quarter to quarter. We're not going to pretend this file is a magic ranking lever.
Here's why we ship one anyway. The cost is a single Markdown file and one afternoon. The potential upside is that the fastest-growing discovery channel on the internet understands your business correctly. And even in the worst case — no engine ever reads it — the exercise forces you to produce the clearest possible statement of what you do, which improves every other page you write. Asymmetric bets like this don't need certainty.
Two practical warnings that do have teeth:
- llms.txt is useless if crawlers are blocked. The manual doesn't matter if the reader can't get in the door. Check that robots.txt allows GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot — we cover the full context in our GEO guide.
- A stale llms.txt is worse than none. A file pointing to dead pages or discontinued products actively teaches AI engines wrong facts about you. Put it on your release checklist: ship a redesign, update the file, same day.
How llms.txt Fits Your Broader AI Visibility Work
Don't treat this file as a standalone trick. It's one layer in a stack: crawler access at the bottom (robots.txt), the site manual above that (llms.txt), machine-quotable content on your actual pages (plain entity statements, structured FAQs), and real-world credibility on top — the stuff Google calls E-E-A-T, which transfers directly to how AI engines weigh sources; see our E-E-A-T explainer for that layer.
Ultimately, llms.txt is a small file with a demanding prerequisite: understanding your own business well enough to explain it in three sentences. Write those sentences. The machines are the excuse; the clarity is the prize.
FAQ
Q: What is llms.txt?
llms.txt is a plain Markdown file placed at your website's root that introduces your site to AI language models — who you are, what you offer, and which pages matter. Proposed by Jeremy Howard of Answer.AI in 2024, it works like a curated manual so AI engines don't have to reconstruct your business from scraped HTML.
Q: Where does the llms.txt file go?
At your site root, right next to robots.txt — for example, yourdomain.com/llms.txt. It's a static text file, so publishing it usually takes minutes: write the Markdown, upload it, confirm it loads in a browser.
Q: Do AI engines actually read llms.txt?
Adoption is real but uneven — it's an emerging convention, not a guaranteed standard, and support varies by engine and changes over time. We publish one anyway because it costs an afternoon, carries no downside, and the clarity exercise improves your positioning even if no bot ever fetches it.
Q: What's the difference between llms.txt and robots.txt?
robots.txt controls access — which crawlers may visit which paths. llms.txt provides understanding — what the site is about and where the important content lives. They're complementary: a great llms.txt is worthless if robots.txt blocks the AI crawlers from entering at all.
Q: How long should an llms.txt file be?
Short — we keep ours under 100 lines. It's an index, not an archive: a name, a two-sentence summary, a few descriptive paragraphs, and curated link sections with one-line descriptions. If you're pasting in your entire sitemap, you've missed the point.