Quick answer: llms.txt is a plain-text file you place at the root of your website (for example, yoursite.co.uk/llms.txt) that gives AI models like ChatGPT, Claude, Gemini and Perplexity a clean, curated summary of what your site is about and which pages matter most. It works like a sitemap, but written for AI assistants instead of search engine crawlers.
Key takeaways
- What it is: a Markdown file at /llms.txt that summarises your site for large language models.
- Who proposed it: developer Jeremy Howard, in September 2024 (documented at llmstxt.org).
- Why it matters: it supports Answer Engine Optimisation (AEO), helping AI tools describe your business accurately.
- How it fits in: it complements your SEO; it does not replace your sitemap, robots.txt or content.
- The catch: it is still a proposed standard, not officially adopted, but it is quick, free and low-risk to add.
AI tools like ChatGPT, Gemini, Claude and Perplexity are now a real route to your website. People ask them questions, and those tools pull answers from across the web. The trouble is, your website was built for humans and search engine crawlers, not for AI models trying to make sense of it quickly.
That is where llms.txt comes in. It is a small file that tells AI models what your site is about, what matters most, and how you would like to be understood. This guide explains what it is, why it is worth adding, and exactly how to set one up. It pairs naturally with the work covered in our guide to AI-powered marketing for Chester businesses.
What is llms.txt?
llms.txt is a plain text file you place at the root of your website (for example, yoursite.co.uk/llms.txt). It gives large language models a clean, structured summary of your site and points them to your most important pages.
The idea was proposed by developer Jeremy Howard in September 2024 and is documented at llmstxt.org. It works a little like a sitemap, but instead of listing every URL for a search engine, it gives AI models a curated, easy-to-read map of who you are and what you offer.
The file is written in Markdown, which is a simple text format that both people and machines read easily. A standard llms.txt includes a heading with your brand or project name, a short summary in a quote block, and then sections of links to your key pages with brief descriptions.
There is also a longer variant called llms-full.txt, which contains the actual content of your important pages in one file rather than just links to them. Some sites publish both.
Here is how llms.txt compares to the two files you may already know:
| File | Who it is for | What it does |
|---|---|---|
| llms.txt | AI models and answer engines | Offers a clean, curated summary of your key pages |
| robots.txt | Search engine crawlers | Controls which pages crawlers are allowed to access |
| sitemap.xml | Search engine crawlers | Lists every URL you want indexed |
Why llms.txt is useful
AI models have a built-in limitation: they can only process a certain amount of text at once. A typical web page is full of navigation menus, adverts, cookie banners, pop-ups and JavaScript, and stripping all of that away to find the useful content is slow and error-prone. llms.txt hands the model a clean version of what matters, so it does not have to guess.
Here is why that is worth your time.
It improves how AI describes your business. The file lets you state plainly who you are, what you do, the areas you serve and how you would like to be referenced. That reduces the chance of an AI tool getting your details wrong.
It supports Answer Engine Optimisation (AEO). As more people get answers directly from AI rather than clicking through a list of blue links, being clearly understood by those models becomes its own form of visibility. llms.txt is one of the levers you can pull.
It complements your existing SEO. It does not replace good content marketing, a clean sitemap or solid technical SEO. It sits alongside them, helping AI tools find and interpret the work you have already done.
It is low cost and low risk. The file is small, quick to produce and easy to update. Even while the standard is still settling, the effort to add one is minimal.
A fair note on where things stand: llms.txt is a proposed standard rather than an official one, and the major AI companies have not all formally committed to reading it. Adoption is growing steadily, though, and given how little work it takes, many sites are adding one now to be ready rather than waiting.
What goes inside an llms.txt file
The format is deliberately simple. A basic file looks like this:
# Spectra Media
> Spectra is a Chester-based full-circle marketing agency. One in-house team
> runs strategy, branding, web, SEO, content, social, advertising, video,
> photography and print for businesses across Chester and the North West.
## Core pages
- [Home](https://www.spectra.media/): What we do and how we work.
- [Services](https://www.spectra.media/services/): Our full-service marketing offering.
- [Web design](https://www.spectra.media/services/web-design/): Fast, conversion-led websites.
- [About](https://www.spectra.media/about/): Who we are and how we work.
## Resources
- [Insights](https://www.spectra.media/insights/): Guides on SEO, AI search and marketing.
- [Contact](https://www.spectra.media/contact/): Get in touch for a quote.The only part that is strictly required is the H1 heading with your name at the top. Everything else is recommended structure that makes the file genuinely useful. Keep the summary quote concise and factual, group your links into clear sections, and write a short description after each link so the model knows why the page matters.
How to add llms.txt to your website
You do not need to be a developer to do this. Here is the process from start to finish.
Step 1: Write the file
Open a plain text editor such as Notepad on Windows, TextEdit on Mac (set to plain text), or a code editor like VS Code. Write your content following the structure above. Save it with the exact filename llms.txt.
Step 2: Decide what to include
List the pages that best represent your business and that you would want an AI tool to rely on. Think homepage, core service pages, your about page, key resources and your contact page. Add a one-line description for each. Resist the urge to include everything; a focused file is more useful than an exhaustive one.
Step 3: Upload it to your root directory
The file needs to live at the top level of your site so it is reachable at yoursite.co.uk/llms.txt. There are a few ways to do this:
By FTP or your hosting file manager, upload llms.txt into your root folder (often called public_html).
On WordPress, you can use a file manager plugin to upload it, add it via your theme’s functions.php, or use an SEO plugin that supports it. Yoast SEO, for example, has an AI discovery file option under Settings, Site features, that generates and serves an llms.txt for you.
If you only want to guide AI through one part of your site, such as documentation, you can place a file inside that specific folder instead of the root.
Step 4: Check it works
Open a new browser tab and go to yoursite.co.uk/llms.txt. You should see your file displayed as plain text. If you do, it is live.
Step 5: Keep it current
Treat it like any other part of your site. When you launch a new service, rebrand, or publish important new content, update the file so it stays accurate.
Frequently asked questions
Is llms.txt the same as robots.txt?
No. robots.txt tells search engine crawlers which pages they may or may not access. llms.txt does the opposite in spirit: it actively offers AI models a clean summary and points them towards your best content. They serve different jobs and can both exist on the same site.
Will llms.txt help my Google rankings?
Not directly. It is aimed at AI models rather than traditional search ranking. Its value is in AI search and answer engines. Your normal SEO work still does the heavy lifting for Google results.
Do I need both llms.txt and llms-full.txt?
No, llms.txt alone is a sensible starting point. llms-full.txt is an optional, longer file containing your actual page content. Larger sites or documentation-heavy sites tend to benefit most from it.
Is llms.txt officially supported by AI companies?
It is a proposed standard rather than a formally adopted one, and support varies between providers. Adoption is growing, and because the file is quick to create and carries little downside, many businesses are adding one now.
The bottom line
llms.txt is a small, quick win for a web that is increasingly shaped by AI. It will not replace your SEO, but it gives AI tools a clear, accurate picture of your business at very little cost. As more people search through AI assistants, helping those tools understand you correctly is becoming part of being found at all.
If you would like a hand setting one up, or you want your wider site geared up for AI search, that is exactly the kind of thing our SEO team helps local businesses with. Get in touch and we will sort it.
