LLMs.txt: The New Standard for AI-Friendly Websites

llms.txt is a proposed standard that helps AI language models understand your website. It is to AI crawlers what robots.txt is to search engine crawlers — but instead of saying what can be accessed, it says what is important.

What is llms.txt?

llms.txt is a plain text file placed at the root of your website (at /llms.txt). It provides a curated summary of your site for AI models. The format was proposed at llmstxt.org (updated to v2 in August 2026, adding per-page .md variants and link-discovery mechanisms).

An honest look at adoption

The honest data first: an Ahrefs study of 137K sites (2026) found about 28% publish an llms.txt — but 97% of those files received zero crawler requests. Major AI crawlers today largely ignore the file. Vendors including Anthropic and OpenAI publish llms.txt for their own developer documentation, and Chrome Lighthouse checks for it, but no engine currently depends on it for visibility.

How to create an llms.txt file

Creating an llms.txt file is straightforward. It is a plain text file with a simple markdown-like format. Here is the structure:

• •Title and description of your site
• •Key pages with URLs and one-line descriptions
• •Optional: links to documentation, API docs, or important content

Place the file at /llms.txt on your domain. GEO Probe checks for its presence — weighted deliberately low in our methodology, because the measured impact today is small.

Should you adopt llms.txt now?

Yes — as low-cost hygiene, not as a visibility lever. The cost is near zero (one text file), and the standard is evolving in the right direction. But spend your real effort on the things with measured impact: Bing index coverage, edge-level AI crawler access, and citable content (statistics and citations). See our research posts on each.