How to Write an llms.txt File (Step-by-Step With Free Generator)

Published 1 September 2026 · Updated 3 September 2026 · 8 min read

Your website ranks well. Your robots.txt is clean. But AI systems still don’t seem to know what you do. In many cases, the missing piece is a single plain-text file that takes less than an hour to write: llms.txt.

This guide walks through exactly what to put in yours, step by step, with real examples and the five mistakes that make otherwise well-built files useless to AI systems.

Before You Write: The 4 Decisions

Answer these four questions before opening a text editor:

1. What is your site for, in one sentence? Not a tagline — a factual description. “Aura AI is a free AI-visibility scanner that scores any domain across 18 deterministic checks in five categories.” That goes in the description block.
2. What are your 5–10 most important pages? Include the pages an AI system should cite if someone asks about your topic. Not your homepage by itself — the pages with actual content: guides, how-tos, pricing, FAQ, key landing pages.
3. Do you have topic clusters? If you have blog posts, documentation, or resources organized by theme, grouping them by topic in llms.txt helps AI systems understand the structure of your content library.
4. Are any pages off-limits? If there is content you do not want AI systems to read or cite (private customer data, staging pages, internal tools), block it in robots.txt — not in llms.txt. llms.txt is a guide to what you want AI to know, so only include pages you are happy to have cited.

The Recommended Structure

The llmstxt.org spec requires only the H1; everything else is recommended structure, and there is no minimum number of entries:

# Site Name                          ← Required: H1 is your brand name

> One sentence: what the site does   ← Recommended: blockquote description
> and who it serves.

## Key Pages                          ← Recommended: your key pages (no minimum)
- [Page Title](/path): Description
- [Page Title](/path): Description

## Content by Topic                   ← Optional: for sites with many pages
### Topic Name
- [Article Title](/blog/slug): Description — approx. N tokens

Follow the Markdown conventions the spec describes: the description goes in a blockquote (lines starting with >), and pages use Markdown link syntax [Title](/path) followed by a colon and a description. Plain URLs or bare titles are not parsed correctly by all systems.

Step-by-Step: Writing Each Section

Section 1: Site Name

The H1 is simply your brand or product name. Keep it short. Do not add a tagline here — the description line does that job.

# Aura AI Visibility

Section 2: Description (Most Important)

This is the single most important line in your llms.txt. AI systems read it first to understand what your site does. A good description is:

• Specific — says what you actually do, not what benefit you deliver
• Factual — no superlatives, no vague claims
• Complete — includes the what, who for, and how

Section 3: Key Pages

List the pages you most want AI systems to know about. Each entry needs a link and a one-line description. Think: which pages would a journalist, a researcher, or a prospective customer want to find?

## Key Pages
- [AI Visibility Scanner](/): Free tool to score any domain across 18 AI-readiness checks
- [Pricing](/#pricing): Monitor ($199/mo) and Agency ($1,199/mo) tiers with feature comparison
- [What is AI visibility?](/blog/geo-optimization-guide): Complete guide to GEO and why it matters
- [18 AI Visibility Checks](/blog/18-ai-visibility-checks): Every check explained with how to pass it
- [What is llms.txt?](/blog/what-is-llms-txt): What the file does and a template to publish your own
- [Free llms.txt Generator](/tools/llms-txt): Build your llms.txt file from your live sitemap

Keep descriptions factual and specific. “Complete guide to X” is fine if it is actually complete. “Best guide to X” is not — it is an unverifiable claim that some AI systems flag as promotional.

Section 4: Content by Topic (Optional)

If your site has more than 15 pages of substantive content, organizing by topic helps AI systems navigate to the right area. Some publishers append approximate token counts to long pages so AI systems can decide whether to fetch the full page or rely on the description — an informal convention, not part of the spec.

## Content by Topic

### AI Visibility Fundamentals
- [GEO Optimization Guide](/blog/geo-optimization-guide): What GEO is and the five pillars — approx. 3,200 tokens
- [AEO vs SEO vs GEO](/blog/aeo-vs-seo-vs-geo): How the three disciplines differ — approx. 2,800 tokens

### Technical Checks
- [AI Crawler Check](/blog/ai-crawler-check-robots-txt): How to verify AI crawlers can access your site — approx. 2,100 tokens
- [Schema Markup for AI](/blog/schema-markup-ai-search): Which schema types matter for AI citation — approx. 3,600 tokens

The 5 Most Common llms.txt Mistakes

1. Vague description line. “We help businesses grow” tells AI nothing about what your site actually contains. If an AI system cannot determine your topic from the description, it may misclassify or ignore your content. Write a description a journalist could quote.
2. Linking to marketing pages instead of content pages. Your homepage and a generic “About” page do not help AI systems cite you. Link to the pages that contain verifiable facts, how-to guidance, pricing details, and substantive explanations.
3. Skipping the description field on linked pages. The one-line description after the colon is how AI systems understand what a page contains without fetching it. Blank descriptions — - [Page Title](/path): with nothing after the colon — are worse than omitting the page entirely.
4. Blocking llms.txt in robots.txt. If your robots.txt disallows all user agents from /llms.txt, crawlers that honour robots.txt will not read the file. Check it yourself: read your robots.txt and fetch the file with curl -A GPTBot https://yourdomain.com/llms.txt. The Aura scan fetches /llms.txt directly and does not evaluate robots rules for that path, so it will not catch this for you.
5. Publishing it and never updating it. llms.txt is not a set-and-forget file. If you add a major section to your site, change your product focus, or remove pages you previously linked, your llms.txt needs an update. Quarterly reviews are sufficient for most sites; monthly for fast-moving content.

Publishing Your llms.txt

Once written, publish the file at the root of your domain. Serve it as plain text withContent-Type: text/plain. For most platforms:

• Static sites / Vercel / Netlify: Place llms.txt in your public/ folder. It will be served at /llms.txt automatically.
• WordPress: Upload llms.txt to your server root via FTP or file manager, or use a plugin that manages root-level text files.
• Next.js: Place it in /public/llms.txt or generate it dynamically via a route handler at app/llms.txt/route.ts.
• Shopify / hosted platforms: Some platforms restrict root-level file access. Check platform documentation for custom file hosting options.

After publishing, verify it is accessible by opening yourdomain.com/llms.txt directly in your browser. You should see plain text, not a 404 or HTML page. Then run a free scan to confirm Aura’s llms.txt check passes.

Use the Free Generator

If you want to skip the manual writing, the free llms.txt generator builds a draft file for your domain. Enter your domain; it reads your live homepage and sitemap and drafts the file for you to edit, then copy or download it ready to publish. It follows the spec’s format and Markdown syntax — review the draft before you publish.

Generate your llms.txt — free, no account required

Frequently Asked Questions

What should I put in my llms.txt?
Only the H1 site name is required by the spec. Add a one-sentence blockquote description and a list of your most important pages with brief descriptions; optionally, organize pages by topic if you have many.

How long should my llms.txt be?
Under 500 lines for most sites. It is a navigation aid, not a content dump. Use llms-full.txt if you want AI systems to read your complete page text.

Do I need to update my llms.txt?
Yes. Update it when you add major pages, change your product focus, or remove linked content. Review it quarterly at minimum.

Will llms.txt guarantee AI mentions my site?
No. It improves discoverability — AI systems understand your site faster and more accurately. Whether they cite you depends on content quality, topic relevance, and the user’s query. Run the full free AI-visibility scan to see all 18 factors that affect your score.

Where do I publish llms.txt?
At the root of your domain: yourdomain.com/llms.txt. Serve as plain text. Confirm it is not blocked in your robots.txt.

Run your free AI-visibility scan — see all 18 checks