XML Sitemap

An XML sitemap is a machine-readable list of the pages on a site that you want search engines to crawl, with optional hints about when each was last updated. It lives at a known address — usually yoursite.com/sitemap.xml — and is submitted once in Search Console. Its job is discovery: it lets a crawler find new products, deep pages and poorly linked pages without having to stumble on them by following links, and it signals which pages you consider canonical.

Every major store platform generates and updates the sitemap automatically, so the file itself is rarely the problem. What goes wrong is around it: it was never submitted to Search Console; it includes hidden, draft or out-of-stock products you'd rather not index; a domain change left it pointing at the old address; or its "discovered" count in Search Console is far below the product count, which reveals products the platform isn't listing. For a large catalogue the sitemap is the difference between a new product being found in days and in weeks, and resubmitting it after a big catalogue change is a cheap way to prompt a recrawl. The proposed llms.txt file is a distant cousin: a curated guide for AI readers rather than an exhaustive list for crawlers.