Why do meta-frameworks generate a sitemap, a robots file or a preview image as routes rather than checked-in static files?

## What these companions are Alongside the pages a site serves, there is a small set of machine-readable files that describe the site to software rather than to people: a listing of the URLs worth visiting, a file stating what automated clients may fetch, and per-URL preview images that link-unfurling consumers render into a card. Nobody reads them directly. They are noticed only when they are wrong. Their content is not independent of the application. It is a **projection of the same two inputs the pages come from**: the route table (which URLs exist) and the content source (which records those URLs are about). That is the whole argument for generating them. ## Why generation beats a checked-in file A file committed to the repository is a snapshot of what the site contained when someone last remembered to update it. Every new record, every removed route, every renamed path makes it more wrong, and nothing fails when it does — no test breaks, no page 500s. The failure is silent and external. Declaring the file as a route inverts that. The handler reads the route table and the same data access the pages use, and produces the body on demand. It cannot describe URLs that do not exist and it cannot miss ones that do, because it is computed from the definition rather than transcribed from it. The mechanics a framework provides are modest but necessary: - a **fixed, well-known path** the route answers at, because these files are fetched by convention, not by a link; - control of the **content type** of the response, since the consumer parses by type, not by extension; - the ability to **read data** exactly as a page route does; - placement **outside the client bundle** — these handlers are server or build artifacts and ship no JavaScript to the browser. ## They are routes, so they get the same choices Because it is a route, a generated companion inherits the ordinary rendering decision: | produced | good when | watch out for | |---|---|---| | once at build, stored as a file | content changes on the release cadence, or the host serves only files | the file ages exactly like any other stored artifact | | per request | content changes constantly and the output is small | an uncached full-catalogue query on a public path | | per request, then cached | the common middle ground | choosing a lifetime that matches how fast the listing really changes | ## Cost and limits This is where interviews go, because the naive implementation works in development and is a standing cost in production: - A listing built by querying an entire catalogue is not cheap, and its path is public and unauthenticated. Uncached, it is a query anybody can trigger repeatedly. - Listing formats impose a **per-file entry limit**; past it the output has to be split and fronted by an index. A single file that quietly grows past the limit may be rejected in its entirety. - A **per-URL preview image** does real rendering work — laying out text and imagery into a raster — orders of magnitude more expensive than emitting a few lines of text. Cache the result, keep the inputs bounded by the route parameters so the set of possible images is finite, and key the URL to the content so consumers refetch only when it actually changed. - Generated companions should be excluded from whatever authentication a site applies to its pages, or the consumers they exist for cannot read them at all. ## The environment question A preview or staging deployment is a separate host serving the same content. A permissive access-policy file copied there invites automated clients to fetch and index a duplicate of the site — a classic way to discover, months later, that a staging host is publicly listed. A generated route can read the deployment environment and return a restrictive body anywhere that is not production; a file checked into the repository is the same bytes everywhere by construction. That single capability is often the strongest argument for the generated form. ## What a good answer covers State the principle first — these files are derived, so derive them — then show you have paid the bill: caching or prebuilding the expensive ones, splitting output that outgrows the format's limit, setting the content type deliberately, and letting the environment decide what a non-production host advertises. Finish with the invariant worth defending: the machine-readable surface of a site should never be hand-maintained, because nothing fails when it silently diverges from the routes the product actually serves.