Beyond /llms.txt: The Full Machine-Readable Stack for AI Agents
The llms.txt standard arrived at exactly the right moment.
As AI coding agents became mainstream — Cursor, Claude Code, Copilot, Windsurf — it became obvious that the existing web was not built for machine traversal. HTML was designed for browsers. Search engines built their own crawling infrastructure. But AI agents were attempting to operate software using documentation that was built entirely for human reading.
llms.txt was the first broadly adopted answer to this problem.
Drop a structured markdown file at yoursite.dev/llms.txt. Give it your product's purpose, key links, and machine-readable navigation hints. Now an AI agent that arrives at your documentation has a reliable entry point.
It was a good start.
But in 2026, with autonomous agents running complex multi-step workflows, llms.txt alone is not enough.
llms.txt is an excellent foundation — but it is a discovery artifact, not an operational surface. Building a complete machine-readable stack means layering five distinct artifacts, each solving a different part of the AI operability problem.
The Five Layers of the Machine-Readable Stack
A software platform that is genuinely optimized for AI agent operation maintains five distinct machine-readable artifacts. Each one serves a different function in the agent's workflow.
Layer 1: /llms.txt — The Discovery Index
Purpose: Tell AI agents what your product is and where to start.
llms.txt is the sitemap for AI agents. It should contain:
- A concise product description (1–3 sentences)
- The canonical homepage URL
- Links to the most important documentation sections
- Links to the machine-readable API specification
- A plain-language summary of what the product does for a developer
# Glintbase
> AI documentation quality testing and Agent Readiness benchmarking for software platforms.
Glintbase measures how well your documentation, APIs, and SDK surfaces
enable AI coding agents to operate your software independently.
## Key Resources
- [ARS 2026 Report](/research): State of Agent Readiness benchmark across 75 platforms
- [Documentation](/docs): CLI setup, MCP server configuration, API reference
- [Blog](/blog): Research articles on Agent Readiness and documentation intelligence
- [OpenAPI Spec](/openapi.json): Full REST API specification
- [MCP Config](/mcp.json): Model Context Protocol server configuration
## What Glintbase Does
Glintbase runs automated AI agent journey simulations against your documentation
surface and produces an Agent Readiness Score (ARS) — a composite metric measuring
how efficiently AI agents can discover, understand, and operate your software.
Layer 2: /llms-full.txt — The Full Content Mirror
Purpose: Provide AI agents with complete documentation content in a single, parseable plain-text file.
HTML documentation requires agents to parse markup, follow links, and reassemble fragmented content. llms-full.txt eliminates that overhead by providing the entire documentation corpus in one clean, flat file.
The format should be:
- Plain UTF-8 text
- Section headers in markdown format (
## Section Name) - Code blocks fenced with triple backticks and language identifiers
- No navigation chrome, no sidebars, no ads — content only
For platforms with large documentation bodies, this file commonly runs 50,000–300,000 tokens. Agents can ingest it in a single context window load and build a complete semantic model of the product without link traversal.
In our ARS 2026 benchmark, platforms with a valid llms-full.txt showed 2.3× lower average tokens per task compared to platforms relying solely on HTML documentation traversal.
Layer 3: /openapi.json (or openapi.yaml) — The Operational Contract
Purpose: Give AI agents a machine-readable definition of every API endpoint, parameter, and response schema.
OpenAPI specifications are the gold standard for machine-readable API description. A well-structured OpenAPI 3.1 spec enables AI agents to:
- enumerate all available endpoints without traversing documentation
- understand required vs. optional parameters
- validate request structures before sending
- understand response schemas and error codes
The critical details that maximize agent utility:
paths:
/v1/scan:
post:
operationId: createScan
summary: Initiate a documentation quality scan
description: |
Submits a documentation URL for AI agent journey simulation.
Returns a scan ID for polling. The scan typically completes
within 90–180 seconds depending on documentation size.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [url]
properties:
url:
type: string
format: uri
example: "https://docs.stripe.com"
description: "Root URL of the documentation to scan"
The key requirement: description fields on every parameter must be thorough. Agents rely heavily on these descriptions to understand what values are valid and what the parameter controls.
Layer 4: /mcp.json — The Model Context Protocol Configuration
Purpose: Enable AI agents with MCP support (Claude, Cursor, Windsurf) to connect directly to your product as a tool.
The Model Context Protocol (MCP) is the emerging standard for giving AI agents direct, structured access to software capabilities — without requiring the agent to parse documentation and construct raw API calls.
An mcp.json at your documentation root tells MCP-compatible agents exactly how to connect:
{
"mcpServers": {
"glintbase": {
"command": "npx",
"args": ["-y", "@glintbase/mcp-server@latest"],
"description": "Glintbase documentation scanner and Agent Readiness Score API",
"tools": [
{
"name": "scan_documentation",
"description": "Scan a software platform's documentation and return an Agent Readiness Score"
},
{
"name": "get_scan_results",
"description": "Retrieve the results of a completed documentation scan by scan ID"
}
]
}
}
}
When a developer's AI coding agent can connect to your product via MCP, they never need to read your documentation at all. The agent talks directly to your product through a structured tool interface. This is the highest level of AI operability a platform can achieve.
Layer 5: Semantic Navigation Metadata — The Structural Signal
Purpose: Embed machine-readable structure directly in your HTML documentation so crawlers and agents can understand page hierarchy without parsing visual layout.
This layer is the most often neglected — and it makes a significant difference.
Concretely, it means:
- OpenGraph meta tags on every documentation page
- Schema.org
TechArticleJSON-LD on guides and tutorials - Breadcrumb JSON-LD on all pages
- Consistent
data-page-typeattributes (reference,guide,tutorial,changelog) - Canonical URLs on every page
These signals help AI agents understand the role of each page in the documentation ecosystem — not just its content. A page tagged as changelog should be treated differently than a page tagged as reference. An agent that understands this distinction makes smarter retrieval decisions.
The Stack in Practice: Glintbase's Own Implementation
Glintbase maintains all five layers.
At glintbase.dev:
/llms.txt— 3,690 bytes, updated with each major documentation change/llms-full.txt— 112,000+ bytes, full content mirror of all documentation/docs.md— plain-text documentation mirror/blog.md— plain-text blog content mirror/mcp.json— MCP server configuration for Claude, Cursor, and Windsurf
The result: in our own internal ARS scoring, Glintbase scores 91 out of 100 — the highest score in our benchmark cohort. Our AI-assisted integration tasks complete in an average of 14,200 tokens. The median platform in our benchmark consumes 72,700 tokens for the same tasks.
How to Prioritize Your Implementation
If you are building out your machine-readable stack from scratch, here is the priority order based on agent resilience impact:
| Priority | Artifact | Impact | Effort |
|---|---|---|---|
| 1 | /llms.txt | High — eliminates discovery overhead | Low — 30 minutes |
| 2 | OpenAPI spec | Very High — enables structured API access | Medium — varies by API complexity |
| 3 | /llms-full.txt | High — eliminates HTML traversal | Medium — requires content aggregation |
| 4 | /mcp.json | Very High — enables direct tool access | High — requires MCP server implementation |
| 5 | Semantic metadata | Medium — improves crawl quality | Low — JSON-LD addition |
Start with llms.txt. It takes 30 minutes and immediately improves how every AI agent experiences your documentation. Then work down the list.
The llms.txt specification is maintained at llmstxt.org. For MCP server implementation guidance, the official reference is at modelcontextprotocol.io. For OpenAPI 3.1, the specification is maintained by the OpenAPI Initiative at spec.openapis.org.
The Standard Is Forming Now
The machine-readable documentation stack is not hypothetical.
Platforms that have already published llms.txt include Anthropic, Vercel, Cloudflare, Supabase, and hundreds of developer tool companies. The MCP ecosystem is growing rapidly, with native support now embedded in Claude, Cursor, Windsurf, and an expanding list of AI development environments.
Teams that implement this stack in 2026 will have a structural advantage over teams that implement it in 2027 — because AI-assisted development workflows lock in habits early. The platforms that AI agents learn to navigate quickly become the default recommendations.
This is the documentation layer of the competitive moat.
Build it now.
See how your platform scores on machine-readable context availability in the ARS 2026 methodology → glintbase.dev/research