GEO-300 · Lesson 6 of 12 · Module 2: Machine-Readable by Design · 3:06

Discovery files and feeds

Discovery files tell bots what you have and what changed. A sitemap lists canonical URLs with an honest lastmod (50,000 URLs and 50 MB per file; index files above that), announced with a Sitemap line in robots.txt. IndexNow pushes changed URLs to Bing, Yandex, Naver, Seznam, Yep and Amazon, but not to Google. Feeds (RSS, Atom) and HTTP validators (ETag, Last-Modified) tell fetchers what is fresh. llms.txt is a proposal: a markdown map of your key pages; as of 2026 no major assistant documents reading it when it answers, and Google says its AI features need no AI text files, so publish it cheaply and judge it by the fetches in your own logs.

Frame from lesson 6 of Sophyx Certified Technical GEO Specialist – Professional, "Discovery files and feeds"

What you will learn

  • Publish a sitemap with honest lastmod
  • Notify engines of changes with IndexNow or feeds
  • Judge llms.txt and other proposals by evidence

The one thing to remember

Tell bots what changed, and tell the truth.

Common mistake: Every URL in the sitemap has lastmod = today

Bots learn the date means nothing and stop trusting the sitemap.

The fix: Set lastmod from the content change date, and leave it alone otherwise.

Key terms

lastmod
The date a page last changed in meaning; bots learn to ignore a lastmod that is always "today".
IndexNow
A protocol to push changed URLs to participating search engines with a key file; Google does not take part.
llms.txt
A proposed markdown file at the site root that lists key pages for AI systems. Support is not confirmed.

Part of Technical GEO Specialist (GEO-300)

For developers and technical SEOs: build a site AI can fetch, read and act on, and keep it that way.

12 lessons, practice quizzes, a final exam and the Sophyx Certified Technical GEO Specialist – Professional certificate. Regular price USD 129. Your first certification is free. Free with any paid Sophyx plan.

Start the course free
Book a Demo