What Is llms.txt?
Your website has a robots.txt that tells search engines what to crawl. It has a sitemap.xml that lists every page. But when ChatGPT or Perplexity tries to understand your business, neither of those files helps much.
That's where llms.txt comes in — a simple file that gives AI models a clean, human-readable overview of your website. It's the newest standard in web visibility, and most businesses don't have one yet.
This article is part of our AI Visibility Glossary — plain-language definitions of every term you need to understand how AI models find, evaluate, and recommend businesses online.
What Is llms.txt?
llms.txt is a plain-text file you place at the root of your website (at /llms.txt) that provides a concise, structured summary of your site's content — written specifically for large language models (LLMs).
It was proposed by Jeremy Howard, co-founder of Answer.AI, in September 2024. The idea is simple: give AI models a cheat sheet about your website so they don't have to crawl and interpret messy HTML with navigation menus, ads, and JavaScript.
Think of it this way: robots.txt tells bots what they can access. llms.txt tells AI models what they should read.
Why Does llms.txt Matter?
Large language models have a fundamental limitation: their context windows are too small to process an entire website. When ChatGPT or Perplexity wants to learn about your business, it can't just load every page you have — it needs a curated summary.
Without llms.txt, AI models have to:
- Crawl individual pages and parse messy HTML
- Guess which pages contain the most important information
- Deal with navigation, pop-ups, and JavaScript-rendered content
With llms.txt, you hand them a clean Markdown file that says: "Here's who we are, here's what matters, and here are the detailed pages you should read."
How Does llms.txt Work?
The file uses Markdown — a format that both humans and AI models can read easily. It follows a specific structure:
# Your Business Name
> A short description of what your business does and who you serve.
## Products
- [MenuBoost](https://hd-webdesign.si/menu-boost/) : AI-powered menu descriptions for restaurants
- [BoostSuite](https://hd-webdesign.si/boost-suite/) : Marketing toolkit for agencies
## Resources
- [About Us](https://hd-webdesign.si/about/) : Company story and team
- [Pricing](https://hd-webdesign.si/pricing/) : Plans and pricing detailsThe structure is simple:
- H1 heading — Your business or project name (required)
- Blockquote — A short summary of what you do
- H2 sections — Lists of important pages grouped by topic
- Links — Each entry is a Markdown link with an optional description
That's it. No complex schema, no special syntax — just clean Markdown that AI models already understand perfectly.
llms.txt vs. robots.txt vs. sitemap.xml
These three files serve different purposes, and a well-structured website has all three:
| File | Purpose | Audience | Format |
|---|---|---|---|
robots.txt |
Tells bots which pages they can or cannot crawl | Search engine crawlers | Plain text |
sitemap.xml |
Lists all pages on your site for indexing | Search engines | XML |
llms.txt |
Provides a curated summary for AI understanding | AI models (ChatGPT, Claude, Gemini) | Markdown |
The key difference: robots.txt and sitemap.xml are designed for traditional search engines. llms.txt is designed for AI systems that generate answers rather than show lists of links.
They coexist peacefully. llms.txt doesn't replace either of the other files — it adds a new layer specifically for AI visibility.
Who Should Create an llms.txt File?
Every business that wants to be visible to AI search should have one. But it's especially valuable for:
- SaaS companies — Help AI understand your product features, pricing, and use cases
- E-commerce sites — Let AI describe your products accurately when users ask for recommendations
- Service businesses — Make sure AI knows what you offer, where you operate, and how to contact you
- Agencies and consultancies — Help AI explain your expertise to potential clients
- Documentation-heavy sites — Give AI a roadmap to your technical content
The adoption is growing fast. Projects using nbdev now create Markdown versions of all pages by default, and the FastHTML project was one of the first to implement the full llms.txt specification.
How to Create Your llms.txt File
Step 1: Identify Your Key Pages
List the 10-20 pages that best represent your business. Think about what an AI model would need to know to recommend you accurately:
- Homepage (your elevator pitch)
- Product/service pages
- Pricing page
- About page
- Key blog posts or resources
Step 2: Write the Markdown
Create a file called llms.txt following the format above. Keep descriptions short — one sentence per link is enough. Group related pages under H2 headings.
Step 3: Place It at Your Root
Upload the file to https://yourdomain.com/llms.txt. It must be accessible at the root path, just like robots.txt.
Step 4 (Optional): Create Full-Content Versions
The llms.txt spec also suggests providing clean Markdown versions of key pages at URLs with .md appended. For example, your pricing page at /pricing.html could also be available at /pricing.html.md. This gives AI models access to full page content in a clean format.
Visit
https://yourdomain.com/llms.txt in your browser. If you see a 404, you don't have one yet. Run a free AI Visibility Audit with BoostSuite to see how AI models currently see your website.
Real-World Example: HD WebDesign
Here's what a practical llms.txt looks like for a web design business:
# HD WebDesign — AI-Powered Web Design & Marketing
> Web design, AI chatbots, and marketing tools for businesses
> in Slovenia, Croatia, Italy, and the DACH region. Products:
> MenuBoost (restaurant menus), BoostSuite (agency toolkit).
## Services
- [Web Design](https://hd-webdesign.si/web-design/) : Custom websites from €890
- [AI Chatbots](https://hd-webdesign.si/boost-suite/) : AI-powered customer support
## Products
- [MenuBoost](https://hd-webdesign.si/menu-boost/) : AI menu descriptions, 10 languages, €19/mo
- [BoostSuite](https://hd-webdesign.si/boost-suite/) : SEO audit, GEO check, ad copy, listing optimizer
## Resources
- [Blog](https://hd-webdesign.si/blog/) : Articles on AI, GEO, and web design
- [About](https://hd-webdesign.si/about/) : Team and company informationThis gives an AI model everything it needs to accurately describe and recommend the business — in under 200 words.
What Happens If You Don't Have One?
AI models will still try to learn about your website — they'll crawl your pages, parse your HTML, and guess what's important. But the results are unpredictable:
- AI might miss your most important pages
- It might misinterpret your products or services
- It might pull outdated information from third-party sources instead of your own site
- Your competitors who do have llms.txt will be described more accurately
As AI search grows — ChatGPT alone now has 900+ million weekly users — the businesses that make it easy for AI to understand them will have a measurable advantage.
FAQ
Is llms.txt an official standard?
It's an open proposal by Jeremy Howard (Answer.AI), published in September 2024. While not an official W3C standard, it's gaining rapid adoption. The GitHub repository has over 2,500 stars, and major projects like FastHTML and nbdev already implement it. Think of it like how robots.txt started — as a practical convention that became universal.
Will llms.txt replace my sitemap?
No. llms.txt complements your sitemap. The sitemap lists all your pages for search engine indexing. llms.txt provides a curated summary for AI understanding. A well-optimized website has all three: robots.txt, sitemap.xml, and llms.txt.
Do I need technical skills to create one?
Not really. If you can write a list in plain text, you can create an llms.txt file. It's just Markdown — a heading, a description, and a list of links. No coding required. If your website already has a robots.txt, you can create an llms.txt in the same way.
How often should I update my llms.txt?
Update it whenever you add new products, services, or important pages. For most businesses, once a quarter is enough. The file should always reflect your current offerings and most valuable content.
Do AI models actually read llms.txt files?
Yes. AI systems that browse the web — like ChatGPT with browsing, Perplexity, and Claude — can access and parse llms.txt files. The Markdown format is already native to how LLMs process text, making it one of the most efficient ways to communicate with AI models.
This article is part of the AI Visibility Glossary — plain-language definitions of every term you need to understand how AI models find, evaluate, and recommend businesses online.
Related articles: