Generate
LLMs.txt file
Create an llms.txt file for your website to enhance LLMs' understanding of your site.
What is LLMs.txt?
LLMs.txt is a proposed markdown file placed at the root of a website to help Large Language Models better access and understand website content.
It addresses the challenge that LLMs can't efficiently process entire complex websites through their limited context windows.
LLMs.txt provides concise, LLM-friendly content in markdown format about your website and its architecture.
How to create the LLMs.txt file?
Creating an LLMs.txt file is simple - just make a plain text file named "LLMs.txt" and save it in your website's root directory.
For the content of the file, you can use the generator 👆 by entering the URL of your sitemap xml file and it'll generate the llms.txt file in a few seconds to a few minutes, depending on the amount of pages of your site.
Otherwise, you can create the content manually, following llmstxt guidelines.
How does LLMs.txt differ from Sitemap.xml?
LLMs.txt is a proposed file format that helps AI crawlers & agents understand your website and its structure.
Sitemap.xml is an existing standard file that helps search engines understand your website.
While both files provide instructions to automated systems, LLMs.txt focuses specifically on LLMs, whereas Sitemap.xml guides search engines.
How does LLMs.txt differ from Robots.txt?
LLMs.txt is a proposed standard that helps LLMs better access and understand your website's content.
Robots.txt is an established web protocol that instructs search engine and AI crawlers which pages to access or avoid on a website, and has been widely used since the 1990s to manage search engine indexing behavior.
While both provide instructions to automated systems, LLMs.txt focuses specifically on helping LLMs consume content, whereas Robots.txt serves as permission file for all bots and crawlers.
How to use LLMs.txt?
An LLMs.txt file is not something you use yourself—it's a proposed markdown file placed at the root of your website specifically to help Large Language Models process your content more effectively.
How LLMs use LLMs.txt?
When a language-model bot/agent/crawler reaches a site, it can grab the small Markdown file /llms.txt.
That file gives a one-line summary and a short, hand-picked list of key links, so the bot can skip ads and code, pull only the chosen pages into its tight context window, and use that clean text to answer users faster and with fewer mistakes.
Do you have examples of LLMs.txt files?
You can find many examples from established brands, including LLM websites themselves on the LLMS Hub.