What Is llms.txt? How to Write One and Where to Put It: A Complete Guide to the Concise Site Map for AI
When an AI wants to cite your site, it first has to find the 'few pages that actually matter' buried under navigation bars, ads, and JavaScript. llms.txt is a concise map you hand the AI on purpose: a single Markdown file that tells it 'who I am and where my most important content lives.' It isn't a mandatory standard, but getting it right spares the AI a wasted trip and makes it far easier for it to understand and cite you.
What Is llms.txt? A One-Sentence Definition
llms.txt is a plain-text file placed at your site's root (the URL ends in /llms.txt) and written in Markdown. Its purpose is to give large language models (LLMs) a 'concise site map' that guides them straight to your most important, most citation-worthy content.
It complements traditional sitemap.xml but is not the same thing: sitemap.xml is a list of 'all' your URLs for search engine crawlers, while llms.txt is a 'curated tour' for AI, written in language humans can read too, picking out a handful of key pages and briefly describing what each one is.
What Problem Does It Solve? Why AI Needs This File
When an AI generates an answer, its context window limits it from reading your entire site. Real web pages are also packed with navigation bars, footers, pop-ups, ads, and front-end JavaScript, which is a lot of noise and a low signal-to-noise ratio for a machine.
llms.txt effectively does a first-pass filtering for the AI: it lays out, ready-made, 'what this site is about and which few pages are worth reading,' lowering the chance the AI misreads or skips your key content and raising the odds it cites you correctly.
Format Conventions: One H1, an Intro, Then Links Grouped Into Sections
llms.txt has no complex syntax; it's simply a Markdown file with a fixed structure. Write it in the order below and it'll read well for both AI and humans:
- 1. Start with one H1 (# Site / Brand Name): the whole file contains only this single H1, which serves as the title. This is the only hard format requirement.
- 2. Follow it immediately with an intro (use a > blockquote or a plain paragraph): two or three sentences making clear who you are and what you offer, so the AI grasps the site's positioning at a glance.
- 3. Group with H2s (## Section Name): for example '## Core Products,' '## Documentation,' '## Blog,' '## About Us.'
- 4. Under each section, list pages as a Markdown link list: in the format - [Page Title](full URL): a one-sentence description of what the page is. The description matters—the AI relies on it to decide whether to read further.
- 5. (Optional) An '## Optional' section: for secondary links that can be skipped. When the AI is short on context, it'll skip this section first.
What's the Difference Between llms.txt and llms-full.txt?
The two often appear together but play completely different roles—don't mix them up.
Put simply: llms.txt is the 'table of contents and index,' while llms-full.txt is 'the book laid open in full text.' For a brand site without much content, just providing llms.txt is enough; only documentation-heavy technical sites or knowledge bases really need to also provide llms-full.txt.
- llms.txt: a concise index. It contains only structure, an intro, and 'links to each page'—it's short by itself and guides the AI to fetch the pages it needs on its own.
- llms-full.txt: the complete full text. It writes the main content 'flattened directly' into a single file (typically a documentation site consolidating all its docs into one big Markdown file), letting the AI read all the body text at once without fetching page by page.
- Trade-off: llms-full.txt is convenient for the AI to consume in one go, but the file can be very large, easily exceeding the context window and harder to keep in sync; llms.txt is lightweight and easy to maintain, but the AI has to make extra requests to fetch the pages it links to.
Curious how your site scores in AI's eyes?
Free scan — get your 0–100 AI-readability score and copy-paste fixes instantly.
Free GEO check →An Important Premise: It's Not a Mandatory Standard, but an Emerging Convention
llms.txt is a 'proposed convention' put forward by the community and gradually being adopted—it's not an official spec like those from the W3C, and no AI vendor guarantees it will read or act on it. It's more pragmatic to treat it as a 'bonus' rather than a 'must-have.'
So keep the right mindset: don't bet your entire GEO strategy on llms.txt. What truly decides whether the AI cites you is still the fundamentals—'crawlers can get in, content is readable server-side, structured data is complete, and authority signals are clear.' llms.txt is icing on the cake: doing it does no harm and costs little, but it can't replace the foundational engineering.
Recommended Approaches for Content Sites and Brand Sites
The priorities differ by site type. Below are practical recommendations for two common scenarios that you can apply directly.
- Brand / corporate sites: put the brand name in the H1, and use the intro to state in one sentence 'what you sell, who you serve, and where.' Recommended sections: core product or service pages, About Us, contact information, FAQ, and representative case studies. Keep links to 10–20 or fewer, picking only the ones that most deserve to be cited.
- Content sites / blogs / knowledge bases: put the site name in the H1, and use the intro to make the subject area clear. Group sections by 'topic category' rather than 'date' (for example '## GEO Basics,' '## Technical Setup'), list a representative pillar article under each category, and point out in the description what question the article answers—this helps the AI match the right question to the right article.
- Common to both: always use full URLs for links (including https://); write descriptions around 'user intent' rather than marketing speak (writing 'explains how to allow AI crawlers in robots.txt' is more useful than 'the ultimate guide'); and remember to keep things in sync when URLs or key pages change, since dead links lower your credibility.
Where to Put It and How to Verify It's Working
The file must go at the 'domain root' so the URL is exactly https://yourdomain/llms.txt—on the same level as robots.txt and sitemap.xml. A subdirectory (for example /docs/llms.txt) is not the conventional location, and the AI won't proactively look there.
Verifying is easy: open https://yourdomain/llms.txt directly in a browser and confirm you can see plain text, the response status is 200, the Content-Type is ideally text/plain or text/markdown, and it's readable without logging in. While you're at it, run a free GEO health check once to measure your starting point alongside structured data and crawler accessibility—it gives you a fuller picture than looking at llms.txt alone.
FAQ
Q. Is llms.txt mandatory? What happens if I don't write one?
Not necessarily. llms.txt is an emerging convention, not a mandatory standard, and no AI vendor guarantees it will read it—so you won't be penalized for not having one. Its role is a 'bonus': having it helps the AI find your key content faster and lowers the chance of misreading; but what decides whether you get cited is still the fundamentals—crawlers can get in, content is readable server-side, and structured data is complete.
Q. How is llms.txt different from sitemap.xml? Should I have both?
sitemap.xml is a machine-readable list of 'all' your URLs for search engines; llms.txt is a 'curated tour' for AI written in Markdown, picking only a few important pages and adding plain-language descriptions. The two serve different purposes and don't replace each other, so it's best to have both. sitemap.xml helps you get fully indexed, while llms.txt helps the AI grab the content most worth citing.
Q. Should I use llms.txt or llms-full.txt?
llms.txt is a concise index (just structure and links), while llms-full.txt is the complete full text (the body flattened directly into the file). For a typical brand site or a site without much content, providing llms.txt is enough; only documentation-heavy technical sites, product docs, or knowledge bases really need to also provide llms-full.txt so the AI can read it all at once. For most small and medium businesses, getting llms.txt right first is enough.
Q. Where on the site should llms.txt go?
Put it at the domain root so the URL is exactly https://yourdomain/llms.txt, on the same level as robots.txt and sitemap.xml. Don't put it in a subdirectory (for example /docs/llms.txt)—that's not the conventional location and the AI won't go looking for it there. Once it's in place, just open that URL in a browser to confirm the response is 200, it's plain text, and it's readable without logging in.
Q. Are there any hard rules for the llms.txt format?
The only relatively hard rule is that 'the file must start with one H1' (# Site Name), with that single H1 as the title for the whole file. The rest is convention: follow the H1 with an intro stating the site's positioning, then use H2s to group links into sections, with each link in the format '- [Title](URL): a one-sentence description.' You can optionally add an Optional section for secondary links that the AI can skip when space is tight.
Q. How can I confirm the AI actually read my llms.txt?
Right now no tool can guarantee or prove that a specific AI 'read it and acted on it,' because whether each vendor uses it isn't public. What you can do in practice is make sure it's accessible: open the URL in a browser or with curl and check that the response status is 200, the content is plain text, the Content-Type is text/plain or text/markdown, and no login is required. Get the accessibility and content right, and leave the rest to time and your overall GEO foundation.
Put what you learned to the test on your site in 10 seconds
Free scan — get your 0–100 AI-readability score and copy-paste fixes instantly.
Free GEO check →