The article discusses the debate over adding an llms.txt file to websites, prompted by John Mueller's claim that no AI system currently uses it. Despite some tools like Yoast offering support for creating llms.txt files to guide AI in understanding website content, the consensus is that it remains largely unused. Mueller notes that standard sitemaps generated by CMSs should suffice for AI crawling needs. Overall, the llms.txt's future effectiveness is uncertain, highlighted by Mueller’s comment on its lack of current support from AI systems.
FWIW no AI system currently uses llms.txt.
Generate an llms.txt file automatically with a click. Offer to guide LLMs like ChatGPT to your most important content.
It's super-obvious if you look at your server logs. The consumer LLMs... will fetch your pages - for training and grounding, but none of them fetch the llms.txt file.
Having multiple sitemap files won't cause problems, especially if they're generated through the same CMS.
Collection
[
|
...
]