llms.txt is a navigation file for systems that know how to read it. It may help an agent find documentation, but it is not a universal way to appear in search results or AI answers. First distinguish the file’s consumer, access to the page, and actual visibility.
What is llms.txt
The llms.txt proposal was published by Jeremy Howard on 3 September 2024; version v2 was updated on 10 August 2026. It is a Markdown description of a project with links to useful pages. The file can apply to a site root or a specific path, such as documentation.
The file does not control crawl permissions or confirm indexing. Before preparing one, identify which agent will fetch it and what reader task it is meant to support.
Example of a minimal llms.txt:
# Acme Corp
> Acme Corp — SaaS-платформа для управления проектами.
## Документация
- [Быстрый старт](https://acme.com/docs/quickstart): установка и первый проект
- [API Reference](https://acme.com/docs/api): полное описание REST API
- [Тарифы](https://acme.com/pricing): план и лимиты
## Дополнительно
- [Блог](https://acme.com/blog): статьи о продукте
- [Статус сервиса](https://acme.com/status): аптайм и инцидентыThere is also an expanded version — llms-full.txt — which contains the full text of key pages in a single file instead of links, so a model can load the content without additional requests.
How it differs from robots.txt and sitemap.xml
The three files serve different purposes, and confusing them is the most common mistake:
| File | For whom | What it does |
|---|---|---|
| robots.txt | Search and AI crawlers | Allows or disallows crawling specific paths |
| sitemap.xml | Search engine crawlers | Tells a search engine about selected URLs for discovery |
| llms.txt | LLM agents, AI assistants | Provides a human-readable summary: what matters and where to look first |
robots.txt says nothing about a site's content — only what may or may not be crawled. sitemap.xml provides selected URLs, but does not guarantee their indexing. llms.txt is the only one of the three that tries to explain the meaning of a site, rather than simply provide a list of URLs. You can check whether an existing robots.txt is configured correctly with the robots.txt analyzer — this does not replace llms.txt, but it is an essential foundation that llms.txt builds on.
Where use is confirmed and where it should not be promised
OpenAI's official crawler documentation itself suggests llms.txt as a way to navigate its documentation. This confirms that the documentation publisher has the file; it does not confirm that the ChatGPT search crawler uses any third-party llms.txt or that the file affects citations.
For Google Search, there is a direct answer: llms.txt is ignored and does not help with search visibility or rankings, including AI features. See Google's official guide. So do not treat it as a foundation of Google SEO.
For another agent, check its documentation and observed requests. A specific tool's support for the format does not mean that all systems support it. The full technical checking procedure is described in the AI site audit.
How to create llms.txt
If you decide to try it, it is not difficult:
- Describe the project in one paragraph. Be specific about what it is and who it is for; avoid marketing language.
- List 5–15 key pages needed to understand the product: documentation, API reference, pricing, and key guides.
- Group pages into sections (
## Документация,## Дополнительно) — this makes the file easier to read, including for people who happen to open it directly. - Do not duplicate the entire sitemap. If all 500 pages on the site are included in llms.txt, its purpose is lost — the model will not be able to identify priorities, and the file will become another sitemap.xml, only less well structured. Check the sitemap to see which sections your site has, then deliberately select the important ones.
- Update it when the site structure changes, but there is no need to automate updates for every new blog post — this file is about navigation by meaning, not a change log.
Is it worth implementing right now
Implementation makes sense if you know who will consume the file: for example, an agent that reads documentation in your use case. Prepare a short list, check that its links are accessible and its content is up to date, then monitor usage. Do not publish private information in it.
If your goal is visibility in Google Search, first fix confirmed issues with access, content, and internal links. llms.txt is not a Google search signal. For other systems, evaluate the experiment separately from search traffic; publishing the file alone is not a promotion result.
Checklist
- Check that the basic robots.txt and sitemap.xml structure is in order — this is a higher priority
- If you have technical documentation, create llms.txt with 5–15 key links
- Do not copy the entire sitemap into it
- Add a short, non-promotional description of the project at the beginning of the file
- Review the file quarterly or after major structural changes to the site, but no more often
- Do not expect an immediate effect on traffic or citations — the file’s presence does not prove that the system you selected uses it
Next step
Check whether the required content is being delivered with a technical site audit, and review crawler roles and actual access using the practical AI guide. If a specific agent needs llms.txt, maintain it as navigation with verifiable links. Evaluate the result based on use in that scenario, while accounting for the experiment's limitations.