Give AI agents a map of your site they'll actually use
Build a spec-compliant llms.txt and AGENTS.md in the browser, then score them against the official format before you ship, so agents stop guessing what your site and repo contain.
llms.txt is a proposed standard: a single markdown file at the root of your site that gives AI agents a curated map of what matters. It starts with an H1 name, a blockquote summary, and H2 sections of annotated links, so a model can orient itself without crawling every page or burning context on navigation and boilerplate.
AGENTS.md serves the working side of the same problem. Where llms.txt tells an agent what your site contains, AGENTS.md tells an agent how to work in your repository: the setup and build commands, the conventions, and the rules it must not break. Coding agents read it before they touch your code.
Both files exist because agents now arrive at your content and your codebase without a human in the loop. If the signal is not machine-readable and easy to find, the agent guesses. This tool generates both from a short form and validates pasted files against the format, so the signal you publish is the signal an agent can actually use.
How it works
- On the Generate tab, enter your site name, a one-line summary, and any sections of links. The tool writes both llms.txt and AGENTS.md live, in the exact byte format the API produces.
- On the Validate tab, paste an existing llms.txt or AGENTS.md and pick which format to check. Scoring runs instantly against structure, link, and quality rules drawn from the spec.
- Copy the output, commit llms.txt to your web root and AGENTS.md to your repository root, and re-validate whenever either file changes.
What does a valid llms.txt look like?
Here is a complete, spec-valid llms.txt. The rule catalogue below is exactly what the validator checks, so you can see what each part earns. Filter the rules by file to separate the ones that apply to llms.txt from the ones that apply to AGENTS.md.
A canonical llms.txt
# Harbor Docs
> Documentation for Harbor, an open-source job queue for background tasks.
Harbor runs durable background jobs with retries, scheduling, and a web dashboard. These pages cover setup, the API, and day-to-day operations.
## Docs
- [Quickstart](https://harbor.example.com/docs/quickstart): install Harbor and run your first job in five minutes.
- [Configuration](https://harbor.example.com/docs/configuration): environment variables, queues, and worker settings.
- [Job API](https://harbor.example.com/docs/api): enqueue, schedule, and inspect jobs from your code.
## Guides
- [Deploying to production](https://harbor.example.com/guides/production): process supervision, scaling workers, and health checks.
- [Handling failures](https://harbor.example.com/guides/failures): retries, dead-letter queues, and alerting.
## Optional
- [Changelog](https://harbor.example.com/changelog): release notes for each version.
Structure
File starts with a single H1 title. high llms #
If it fails: File does not begin with an H1 heading.
How to fix: Start the file with a single '# Name' line. The H1 is the only required section in the llms.txt spec.
A blockquote summary is present. medium llms #
If it fails: No blockquote summary found.
How to fix: Add a '> ' blockquote directly under the H1 with a one-line summary an agent needs before reading the rest.
H2 file-list sections are present. high llms #
If it fails: No H2 sections found.
How to fix: Group curated links under '## ' section headings such as Docs, Guides, or API.
An Optional section marks secondary links. low llms #
If it fails: No Optional section defined.
How to fix: Move links that can be skipped under a '## Optional' section so agents drop them first under tight context limits.
Sections use markdown list items. low llms #
If it fails: No markdown list items found.
How to fix: List each resource on its own '- ' bullet inside its section.
File starts with an H1 heading. high agents #
If it fails: File does not begin with an H1 heading.
How to fix: Start AGENTS.md with a single '# AGENTS.md' or project H1 so agents can identify the file.
Section headings are present. medium agents #
If it fails: No section headings found.
How to fix: Organise guidance under '## ' headings such as Project, Setup, Build, or Conventions.
Guidance uses scannable list items. low agents #
If it fails: No list items found.
How to fix: Break conventions into '- ' bullets so agents can parse each rule independently.
Links
Markdown links are present. high llms #
If it fails: No markdown links found.
How to fix: List resources as '- [Title](url): note'. Bare URLs are not part of the format.
Absolute https URLs are present. medium llms #
If it fails: No absolute https URLs found.
How to fix: Use absolute https URLs so agents can resolve links without a base path.
Links include descriptive notes. low llms #
If it fails: Links have no descriptive notes.
How to fix: Add a ': short note' after each link so an agent can pick the right resource without fetching every URL.
Quality
The H1 title looks specific. low llms #
If it fails: The H1 title looks like a placeholder.
How to fix: Replace the placeholder H1 with your real project or site name.
File has enough substance. high llms #
If it fails: File is too short to be useful.
How to fix: A useful llms.txt needs a title, a summary, and at least one section of curated links (100+ characters).
File is a concise index. low llms #
If it fails: File is very large for an index.
How to fix: Keep llms.txt as a curated index. Push full content into linked pages or an llms-full.txt rather than inlining it here.
No raw HTML block tags. medium llms #
If it fails: Raw HTML tags are present.
How to fix: llms.txt is markdown. Remove HTML tags like div, span, script, or table.
Setup or build commands are shown in a code fence. high agents #
If it fails: No fenced commands found.
How to fix: Show the exact setup, build, and test commands in a fenced code block so an agent can run them verbatim.
Guidance is written as direct instructions. medium agents #
If it fails: No imperative guidance detected.
How to fix: Write rules as direct instructions an agent can follow: 'Run just build', 'Never edit generated files'.
File has enough guidance to be useful. high agents #
If it fails: File is too short to guide an agent.
How to fix: Describe the project, setup commands, and at least a few conventions (100+ characters).
File is a reasonable length. low agents #
If it fails: File is very large for agent guidance.
How to fix: Keep AGENTS.md focused. Link out to deeper docs instead of inlining everything.
No raw HTML block tags. low agents #
If it fails: Raw HTML tags are present.
How to fix: AGENTS.md is markdown. Remove HTML tags like div, span, script, or table.
For AI agents
This tool is fully usable by agents. Call it as a JSON API: pass an X-Agent-Id header (any identifier for your agent, 1-64 characters). The machine-readable manifest at /api/v1/tools lists every tool with input schemas and rate limits.
curl -s https://systemprompt.io/api/v1/tools/llms-txt/run \
-H "Content-Type: application/json" -H "X-Agent-Id: my-agent" \
-d '{"mode": "validate", "target": "llms", "content": "# My site\n\n> Summary..."}'
Frequently asked questions
What is llms.txt?
How is llms.txt different from robots.txt?
What is the difference between llms.txt and llms-full.txt?
Do AI crawlers actually read llms.txt?
What is AGENTS.md?
Where do I put the files?
Does this validate against the official spec?
Your files are valid. Now decide what agents can do once they arrive.
Publishing llms.txt and AGENTS.md gets agents to your content. Once they start acting on your infrastructure, discovery becomes governance. This guide compares how the tools handle agents in production.