snappy-prerender 2.6.0-rc.2 → 2.6.0-rc.3

package/README.md

@@ -3,11 +3,14 @@
 **Prerender your single-page app without rewriting it.** Point it at your build output, and every route is rendered in a real browser and written to disk as static HTML — content, titles, meta tags, canonical links and Open Graph tags included. Search engines and social crawlers get the full page; your users get the same app they had before.
 
 [![npm](https://img.shields.io/npm/v/snappy-prerender.svg)](https://www.npmjs.com/package/snappy-prerender)
+[![npm downloads](https://img.shields.io/npm/dm/snappy-prerender.svg)](https://www.npmjs.com/package/snappy-prerender)
 [![node](https://img.shields.io/node/v/snappy-prerender.svg)](https://www.npmjs.com/package/snappy-prerender)
 [![license](https://img.shields.io/npm/l/snappy-prerender.svg)](https://github.com/sayakmukherjee080/snappy-prerender/blob/main/LICENSE)
 
+Package page: [npmjs.com/package/snappy-prerender](https://www.npmjs.com/package/snappy-prerender). While 2.6.0 is a release candidate it is published under the `next` tag, so install with `npm install --save-dev snappy-prerender@next`; the plain command below works once 2.6.0 is stable.
+
 - **No SSR entry, no framework migration, no app changes.** If it builds to a `dist` directory, it can be prerendered.
-- **React 18 and React 19, Vite 6/7/8, or any static build.**
+- **React 16.8, 17, 18 and 19, Vite 6/7/8, or any static build.**
 - **Inspired by [react-snap](https://github.com/stereobooster/react-snap)** and rebuilt from scratch for current toolchains: a browser-based prerenderer with a hydration report attached, not another SSR framework.
 
 ## Who this is for
@@ -60,6 +63,7 @@ npx snappy-prerender dist --include /,/pricing,/blog/* --exclude /blog/drafts/*
 npx snappy-prerender dist --block-third-party --allowed-hosts cms.example.com
 npx snappy-prerender dist --metadata-site-url https://example.com --metadata-title-template '%s | Example'
 npx snappy-prerender dist --inline-css critical --minify-html --preload-manifest
+npx snappy-prerender dist --sitemap --verify-changed-only
 npx snappy-prerender dist --csp strict --inline-css critical
 ```
 
@@ -175,6 +179,10 @@ After writing the output, every route is loaded again with the real client bundl
 
 **Page errors.** Errors the browser logs while verifying are reported too: uncaught exceptions as warnings, console errors at debug level. Errors thrown during rendering itself are counted in the summary and reported per route, and `failOnPageError: true` turns them into a build failure; by default an app that throws still ships what it managed to render, because the static output is what crawlers read.
 
+**React 16 and 17.** Hydration and boot modes are judged the same way, and their dev mismatch warnings ("Text content did not match", "Did not expect server HTML to contain") are recognised. Their production builds report nothing for a mismatch at all — no warning, no minified code — so on 16 and 17 the boot-mode report (`hydrated` versus `re-rendered`) is the signal that something drifted.
+
+**Rebuilds.** On a large site most routes are unchanged between builds, and reloading them proves nothing new. `verifyChangedOnly: true` skips the verification pass for routes whose HTML file was left untouched, which roughly halves rebuild time; the report notes how many routes were skipped, and the first run after a change still verifies everything that moved.
+
 ## Ready contract
 
 Most apps work with no configuration because the network-quiet heuristic is enough. Apps that fetch data after load can opt into a stronger signal:
@@ -239,6 +247,16 @@ snappy({ preconnectThirdParty: true, preloadImages: true, preloadManifest: true
 
 react-snap called this `http2PushManifest`. Browsers removed HTTP/2 push (Chrome in v106), so the manifest is now a header list rather than a push list.
 
+### Sitemap
+
+```js
+snappy({ sitemap: true, metadata: { siteUrl: 'https://example.com' } });
+```
+
+`snappy` already knows every route it rendered, so it can write the sitemap for you. `sitemap.xml` is generated from the output files — a directory route becomes `/about/`, a `flatOutput` route becomes `/about.html` — with the base path applied and the not-found route left out. Each entry carries a `<lastmod>` of the run date.
+
+When the output directory already has a `robots.txt`, a `Sitemap:` line is appended to it if it does not point anywhere yet; an existing directive is left alone. Without a `robots.txt`, the run logs the exact line to add instead of creating a file you did not ask for.
+
 ### Tag surgery
 
 ```js
@@ -252,6 +270,24 @@ snappy({
 
 The persisted metadata element is exempt from `removeScriptTags`; everything else is removed as asked.
 
+### Removing volatile markup
+
+Mark any element with `data-prerender-remove` and it is dropped from the output:
+
+```jsx
+<div data-prerender-remove>{ads.render()}</div>
+```
+
+Use it for regions that only make sense after the client takes over — ads, chat widgets, anything injected after mount. The rule of thumb: remove markup that React will *not* render on its first client pass. Removing something React does expect causes the very hydration mismatch it was meant to avoid, which is exactly what the verifier would then report.
+
+Some libraries stamp values that change on every page load — antd's `rc-menu` writes `data-menu-id="rc-menu-uuid-…"` — which makes every rebuild rewrite every file and leaves `verifyChangedOnly` nothing to skip. List those attribute names (or RegExps) and they are dropped from the output:
+
+```js
+snappy({ removeAttributes: ['data-menu-id', 'aria-controls'] });
+```
+
+As with removed markup, only strip attributes React does not render as props; if you guess wrong, the hydration verifier tells you which route.
+
 ### Strict CSP
 
 One dial decides how the injected pieces are delivered. `csp: 'off'` (default) leaves them inline, which is cheapest to load. `csp: 'strict'` makes the whole output work under `script-src 'self'` with no `unsafe-inline`:
@@ -370,6 +406,23 @@ export default {
 };
 ```
 
+The config can also be an async function that returns the options. That is how routes get discovered from a CMS, a sitemap or a directory listing, without a separate prebuild script:
+
+```js
+// snappy.config.js
+export default async function () {
+  const response = await fetch('https://cms.example.com/api/posts?fields=slug');
+  const posts = await response.json();
+  return {
+    sourceDir: 'dist',
+    include: ['/', '/blog', ...posts.map((post) => `/blog/${post.slug}`)],
+    metadata: { siteUrl: 'https://example.com', siteName: 'Example' },
+  };
+}
+```
+
+The function is awaited once per run, and whatever it returns must be a plain options object. If it throws, the CLI reports the reason and stops before rendering anything.
+
 ## Options
 
 | Option | Default | Description |
@@ -413,9 +466,11 @@ export default {
 | `preloadImages` | `false` | Add preload hints for same-origin images |
 | `preloadManifest` | `false` | Write `preload-manifest.json` with Link header hints |
 | `ignoreForPreload` | `['service-worker.js']` | File names excluded from the manifest |
+| `sitemap` | `false` | Write `sitemap.xml` from the rendered routes and point an existing `robots.txt` at it; needs `metadata.siteUrl` |
 | `cacheAjaxRequests` | `false` | Expose captured JSON responses as `window.snapStore` |
 | `maxCachedBytes` | `5242880` | Largest JSON response cached per route, measured on the serialised body |
 | `removeBlobs` | `true` | Drop stylesheet links pointing at dead `blob:` URLs |
+| `removeAttributes` | `[]` | Attribute names or RegExps to drop from the output, for libraries that stamp per-load ids |
 | `removeStyleTags` | `false` | Strip every style tag from the output |
 | `removeScriptTags` | `false` | Strip every script tag from the output |
 | `asyncScriptTags` | `false` | Mark external scripts async |
@@ -425,6 +480,7 @@ export default {
 | `saveAs` | `html` | `html`, `png` or `jpeg` |
 | `metadata` | `null` | Defaults for the head component: `siteUrl`, `siteName`, `titleTemplate`, `defaultImage`, `trailingSlash`, `base` |
 | `verify` | `true` | Run the hydration verification pass |
+| `verifyChangedOnly` | `false` | Skip verification for routes whose HTML file did not change since the last run |
 | `failOnHydrationError` | `false` | Fail the run on hydration errors; off by default, where they are reported instead |
 | `failOnRerender` | `false` | Fail the run when a route re-renders instead of hydrating |
 | `failOnPageError` | `false` | Fail the run when a page throws while rendering; off by default, where page errors are reported |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "snappy-prerender",
-  "version": "2.6.0-rc.2",
+  "version": "2.6.0-rc.3",
   "description": "Drop-in prerendering for existing single-page apps. Renders your built app in a real browser, writes static HTML per route, reports React hydration mismatches, and optimises the output with critical CSS, link hints and cached AJAX state.",
   "type": "module",
   "license": "MIT",
@@ -68,11 +68,13 @@
     "vite": "7.3.6"
   },
   "workspaces": [
+    "tests/fixtures/react16-app",
+    "tests/fixtures/react17-app",
     "tests/fixtures/react18-app"
   ],
   "peerDependencies": {
     "beasties": ">=0.4.1",
-    "react": ">=18.0.0",
+    "react": ">=16.8.0",
     "vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
   },
   "peerDependenciesMeta": {

package/src/cli.js

@@ -37,6 +37,7 @@ Options:
       --browser-download-hash <sha256>  Pin the fallback download to a SHA-256 digest
       --storage-state <file>  Playwright storage state for authenticated routes
       --no-verify             Skip the hydration verification pass
+      --verify-changed-only   Skip verification for routes whose HTML did not change
       --fail-on-hydration-error  Fail the build when hydration errors are found
       --fail-on-rerender      Fail when a route re-renders instead of hydrating
       --fail-on-page-error    Fail when a page throws while it is being rendered
@@ -65,6 +66,7 @@ Options:
       --no-preconnect-third-party  Disable preconnect hints
       --preload-images        Add preload hints for images on the page
       --preload-manifest      Write preload-manifest.json with Link header hints
+      --sitemap               Write sitemap.xml from the rendered routes (needs metadata.siteUrl)
       --ignore-for-preload <names>  File names excluded from the manifest, comma separated
       --cache-ajax-requests   Expose captured JSON responses as window.snapStore
       --remove-scripts        Strip every script tag from the output
@@ -73,6 +75,7 @@ Options:
       --external-scripts      Write injected state scripts as same-origin files, for a strict CSP
       --csp <mode>            off | strict: strict makes the output work under script-src 'self'
       --no-remove-blobs       Keep blob stylesheet links instead of dropping them
+      --remove-attributes <names>  Drop these attributes from the output, comma separated
       --ignore-https-errors   Ignore TLS errors while rendering
       --browser-args <args>   Extra browser launch arguments, comma separated
       --url <url>             App URL to connect to when using --serve-cmd
@@ -108,6 +111,7 @@ async function main() {
       'browser-download-hash': { type: 'string' },
       'storage-state': { type: 'string' },
       'no-verify': { type: 'boolean' },
+      'verify-changed-only': { type: 'boolean' },
       'fail-on-hydration-error': { type: 'boolean' },
       'fail-on-rerender': { type: 'boolean' },
       'fail-on-page-error': { type: 'boolean' },
@@ -136,6 +140,7 @@ async function main() {
       'no-preconnect-third-party': { type: 'boolean' },
       'preload-images': { type: 'boolean' },
       'preload-manifest': { type: 'boolean' },
+      sitemap: { type: 'boolean' },
       'ignore-for-preload': { type: 'string', multiple: true },
       'cache-ajax-requests': { type: 'boolean' },
       'remove-scripts': { type: 'boolean' },
@@ -143,6 +148,7 @@ async function main() {
       'async-scripts': { type: 'boolean' },
       'external-scripts': { type: 'boolean' },
       'no-remove-blobs': { type: 'boolean' },
+      'remove-attributes': { type: 'string', multiple: true },
       'ignore-https-errors': { type: 'boolean' },
       'browser-args': { type: 'string', multiple: true },
       url: { type: 'string' },
@@ -207,6 +213,7 @@ function cliOptions({ values, positionals }) {
   }
   if (values['storage-state'] !== undefined) options.storageState = values['storage-state'];
   if (values['no-verify']) options.verify = false;
+  if (values['verify-changed-only']) options.verifyChangedOnly = true;
   if (values['fail-on-hydration-error']) options.failOnHydrationError = true;
   if (values['fail-on-rerender']) options.failOnRerender = true;
   if (values['fail-on-page-error']) options.failOnPageError = true;
@@ -236,6 +243,7 @@ function cliOptions({ values, positionals }) {
   if (values['no-preconnect-third-party']) options.preconnectThirdParty = false;
   if (values['preload-images']) options.preloadImages = true;
   if (values['preload-manifest']) options.preloadManifest = true;
+  if (values.sitemap) options.sitemap = true;
   if (values['ignore-for-preload'] !== undefined) {
     options.ignoreForPreload = splitList(values['ignore-for-preload']);
   }
@@ -245,6 +253,9 @@ function cliOptions({ values, positionals }) {
   if (values['async-scripts']) options.asyncScriptTags = true;
   if (values['external-scripts']) options.externalScripts = true;
   if (values['no-remove-blobs']) options.removeBlobs = false;
+  if (values['remove-attributes'] !== undefined) {
+    options.removeAttributes = splitList(values['remove-attributes']);
+  }
   if (values['ignore-https-errors']) options.ignoreHTTPSErrors = true;
   if (values['browser-args'] !== undefined) options.browserArgs = splitList(values['browser-args']);
   if (values.url !== undefined) options.url = values.url;
@@ -435,7 +446,9 @@ function metadataOptions(values) {
 
 /**
  * Loads a JS config file. An explicitly requested file must exist; the implicit
- * snappy.config.js is optional. The default export must be a plain options object.
+ * snappy.config.js is optional. The default export is either a plain options object or an
+ * async function that returns one, which is how routes are computed from a CMS, a sitemap or
+ * a file listing without a separate prebuild script.
  */
 async function loadConfigFile(explicitPath) {
   if (explicitPath === undefined) {
@@ -445,9 +458,17 @@ async function loadConfigFile(explicitPath) {
     throw new Error(`Config file not found: ${explicitPath}`);
   }
   const module = await import(pathToFileURL(explicitPath).href);
-  const loaded = module.default ?? {};
+  const exported = module.default ?? {};
+  let loaded;
+  try {
+    loaded = typeof exported === 'function' ? await exported() : exported;
+  } catch (error) {
+    throw new Error(`Config file failed to load: ${error.message}`);
+  }
   if (typeof loaded !== 'object' || loaded === null || Array.isArray(loaded)) {
-    throw new Error(`Config file must export a plain options object: ${explicitPath}`);
+    throw new Error(
+      `Config file must export a plain options object or a function returning one: ${explicitPath}`,
+    );
   }
   return loaded;
 }

package/src/core/config.js

@@ -27,7 +27,14 @@ const NULLABLE_KEYS = [
   'url',
   'serveCmd',
 ];
-const ARRAY_KEYS = ['include', 'exclude', 'allowedHosts', 'browserArgs', 'ignoreForPreload'];
+const ARRAY_KEYS = [
+  'include',
+  'exclude',
+  'allowedHosts',
+  'browserArgs',
+  'ignoreForPreload',
+  'removeAttributes',
+];
 
 export const DEFAULTS = Object.freeze({
   sourceDir: 'dist',
@@ -68,6 +75,7 @@ export const DEFAULTS = Object.freeze({
   minifyHtml: false,
   minifyCss: false,
   removeBlobs: true,
+  removeAttributes: [],
   removeStyleTags: false,
   removeScriptTags: false,
   asyncScriptTags: false,
@@ -75,12 +83,14 @@ export const DEFAULTS = Object.freeze({
   preconnectThirdParty: true,
   preloadImages: false,
   preloadManifest: false,
+  sitemap: false,
   ignoreForPreload: ['service-worker.js'],
   cacheAjaxRequests: false,
   maxCachedBytes: 5 * 1024 * 1024,
   flatOutput: false,
   notFoundRoute: '/404',
   verify: true,
+  verifyChangedOnly: false,
   failOnHydrationError: false,
   failOnRerender: false,
   failOnPageError: false,
@@ -189,6 +199,7 @@ export function validateConfig(config) {
   stringList(config.allowedHosts, 'allowedHosts');
   stringList(config.browserArgs, 'browserArgs');
   stringList(config.ignoreForPreload, 'ignoreForPreload');
+  patternList(config.removeAttributes, 'removeAttributes');
   for (const key of ['readyFlag', 'readySelector', 'waitFor', 'notFoundRoute']) nullableString(key);
   for (const key of ['userAgent', 'destination']) nullableString(key);
   for (const key of BOOLEAN_KEYS) booleanOption(key);
@@ -271,6 +282,9 @@ export function validateConfig(config) {
     throw new TypeError(`saveAs must be one of: ${SAVE_AS_CHOICES.join(', ')}`);
   }
   validateMetadata(config);
+  if (config.sitemap && !config.metadata?.siteUrl) {
+    throw new TypeError('sitemap needs metadata.siteUrl so the sitemap can list absolute URLs');
+  }
 }
 
 // The metadata keys the head component reads, so a typo is reported instead of ignored.

package/src/core/normalize.js

@@ -16,6 +16,7 @@ export function normaliseHtml(html, options = {}) {
   const document = parse(html);
   const stats = {
     removedElements: 0,
+    removedAttributes: 0,
     dedupedStyles: 0,
     hints: 0,
     textSeparators: 0,
@@ -41,6 +42,9 @@ function cleanChildren(parent, stats, seenStyles, inHead, options) {
   for (const child of children) {
     const childInHead = inHead || child.tagName === 'head';
     if (shouldRemove(child, stats, options)) continue;
+    if (child.attrs && options.removeAttributes?.length > 0) {
+      stripAttributes(child, options.removeAttributes, stats);
+    }
     if (childInHead && child.tagName === 'style' && dedupeStyle(child, stats, seenStyles)) continue;
     if (childInHead && child.tagName === 'title') stats.titleElements += 1;
     if (childInHead) collectMetadataOrigins(child, stats);
@@ -67,6 +71,22 @@ function createComment() {
   return { nodeName: '#comment', data: ' ', parentNode: null };
 }
 
+// Drops attributes whose names the caller marked as volatile, such as the per-load ids a UI
+// library writes into the DOM, so repeated runs produce identical output.
+function stripAttributes(node, patterns, stats) {
+  const kept = node.attrs.filter(
+    (attribute) => !patterns.some((pattern) => matchesAttribute(attribute.name, pattern)),
+  );
+  if (kept.length === node.attrs.length) return;
+  stats.removedAttributes += node.attrs.length - kept.length;
+  node.attrs = kept;
+}
+
+// Reports whether an attribute name matches a string exactly or a RegExp pattern.
+function matchesAttribute(name, pattern) {
+  return pattern instanceof RegExp ? pattern.test(name) : name === pattern;
+}
+
 // Reports whether a node is removed for any of the configured reasons. The metadata script is
 // exempt: the head layer reads it on the client, so stripping it would lose the site defaults.
 function shouldRemove(node, stats, options) {

package/src/core/run.js

@@ -18,6 +18,7 @@ import { renderRoute } from './render.js';
 import { crawlRoots, filterRoutes, normaliseRoute, toPublicPath, withinDepth } from './routes.js';
 import { createPool } from './scheduler.js';
 import { startCommandServer, startStaticServer } from './server.js';
+import { buildSitemap, withSitemapDirective } from './sitemap.js';
 import { verifyRoutes } from './verify.js';
 
 /**
@@ -57,6 +58,7 @@ export async function prerender(userOptions = {}, { log: injectedLog } = {}) {
     duplicateTitles: [],
     verification: null,
     preloadManifest: null,
+    sitemap: null,
     truncated: null,
     ok: false,
     durationMs: 0,
@@ -94,6 +96,10 @@ export async function prerender(userOptions = {}, { log: injectedLog } = {}) {
     logNotFoundNotice(report, config, log);
     logDuplicateTitles(report, log);
 
+    if (config.sitemap) {
+      await writeSitemap({ outputDir, config, log, report });
+    }
+
     if (report.truncated) {
       log.error(
         `maxRoutes (${report.truncated.limit}) reached after ${report.truncated.routes} route(s); output is incomplete`,
@@ -115,13 +121,25 @@ export async function prerender(userOptions = {}, { log: injectedLog } = {}) {
         'hydration verification is skipped with serveCmd: the command server decides which document it serves, so the written files cannot be checked',
       );
     }
-    if (canVerify && !config.serveCmd && verifiable.length > 0) {
+    // With verifyChangedOnly, a route whose file was left untouched is not reloaded: the HTML
+    // and the bundle are the same as the last time it was verified.
+    const toVerify = config.verifyChangedOnly
+      ? verifiable.filter(
+          (route) => report.files.find((file) => file.route === route)?.status !== 'unchanged',
+        )
+      : verifiable;
+    const skipped = verifiable.length - toVerify.length;
+    if (canVerify && !config.serveCmd && skipped > 0) {
+      log.info(`Verification skipped for ${skipped} unchanged route(s), as verifyChangedOnly asks`);
+    }
+    if (canVerify && !config.serveCmd && toVerify.length > 0) {
       report.verification = await verifyRoutes({
         browser,
         origin: server.origin,
-        routes: verifiable.sort(),
+        routes: toVerify.sort(),
         config,
       });
+      report.verification.skipped = skipped;
       logVerification(report.verification, config, log);
     }
   } finally {
@@ -360,6 +378,7 @@ async function writeRouteOutput({ route, result, outputDir, config, log, state,
     removeScriptTags: config.removeScriptTags,
     asyncScriptTags: config.asyncScriptTags,
     removeBlobs: config.removeBlobs,
+    removeAttributes: config.removeAttributes,
     preconnectOrigins: config.preconnectThirdParty
       ? [...result.collector.thirdPartyOrigins].sort()
       : [],
@@ -369,12 +388,13 @@ async function writeRouteOutput({ route, result, outputDir, config, log, state,
   });
   if (
     stats.removedElements > 0 ||
+    stats.removedAttributes > 0 ||
     stats.dedupedStyles > 0 ||
     stats.hints > 0 ||
     stats.textSeparators > 0
   ) {
     log.debug(
-      `  cleaned ${route}: ${stats.removedElements} element(s) removed, ${stats.dedupedStyles} duplicate style(s), ${stats.hints} hint(s), ${stats.textSeparators} text separator(s)`,
+      `  cleaned ${route}: ${stats.removedElements} element(s) removed, ${stats.removedAttributes} attribute(s) removed, ${stats.dedupedStyles} duplicate style(s), ${stats.hints} hint(s), ${stats.textSeparators} text separator(s)`,
     );
   }
   if ((result.inlineCssSkipped ?? []).length > 0) {
@@ -452,6 +472,63 @@ async function writePreloadManifest({ manifestEntries, outputDir, config, log })
   return { file, routes: manifest.length };
 }
 
+/**
+ * Writes sitemap.xml from the routes the run produced, and points an existing robots.txt at
+ * it. Every loc must be absolute, which is why the option needs metadata.siteUrl.
+ */
+async function writeSitemap({ outputDir, config, log, report }) {
+  if (config.saveAs !== 'html') {
+    log.warn(
+      `sitemap is skipped when saveAs writes ${config.saveAs} files, as there are no pages to list`,
+    );
+    return;
+  }
+  const sitemap = buildSitemap({ files: report.files, config });
+  if (!sitemap) {
+    log.warn('sitemap is skipped: no pages were written');
+    return;
+  }
+
+  const siteUrl = config.metadata.siteUrl.replace(/\/$/, '');
+  const sitemapUrl = `${siteUrl}${toPublicPath('/sitemap.xml', config.base)}`;
+  const robots = await fs.readFile(path.join(outputDir, 'robots.txt'), 'utf8').catch(() => null);
+  const patched = robots === null ? null : withSitemapDirective(robots, sitemapUrl);
+
+  if (config.dryRun) {
+    report.files.push({
+      route: null,
+      file: 'sitemap.xml',
+      status: 'dry-run',
+      bytes: Buffer.byteLength(sitemap.contents),
+    });
+    report.sitemap = { file: 'sitemap.xml', routes: sitemap.routes, robotsUpdated: false };
+    log.info(`Dry run: would write sitemap.xml listing ${sitemap.routes} route(s)`);
+    return;
+  }
+
+  const written = await writeFileIfChanged({
+    dir: outputDir,
+    file: 'sitemap.xml',
+    contents: sitemap.contents,
+  });
+  report.files.push({ route: null, ...written });
+  report.sitemap = {
+    file: 'sitemap.xml',
+    routes: sitemap.routes,
+    robotsUpdated: patched !== null,
+  };
+  log.info(`Wrote sitemap.xml listing ${sitemap.routes} route(s)`);
+
+  if (robots === null) {
+    log.info(`Add "Sitemap: ${sitemapUrl}" to a robots.txt so crawlers find it`);
+    return;
+  }
+  if (patched !== null) {
+    await writeFileIfChanged({ dir: outputDir, file: 'robots.txt', contents: patched });
+    log.info('Added the sitemap location to robots.txt');
+  }
+}
+
 // Notes when a not-found route is configured but was never rendered, so no 404.html
 // was written and a static host would serve its own default page instead.
 function logNotFoundNotice(report, config, log) {

package/src/core/sitemap.js

@@ -0,0 +1,62 @@
+import { normaliseRoute, toPublicPath } from './routes.js';
+
+const XML_ESCAPES = { '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&apos;' };
+
+/**
+ * Builds the sitemap for the routes a run wrote. Locs are derived from the output file
+ * names, so a directory route becomes /about/ and a flat one becomes /about.html, then
+ * prefixed with the public base path. Returns null when there is no public site URL or
+ * nothing to list, so callers can skip the write entirely.
+ */
+export function buildSitemap({ files, config, generatedAt = new Date() }) {
+  const siteUrl = config.metadata?.siteUrl?.replace(/\/$/, '');
+  if (!siteUrl) return null;
+  const notFound = config.notFoundRoute ? normaliseRoute(config.notFoundRoute) : null;
+
+  const locs = new Set();
+  for (const file of files) {
+    if (!file.file.endsWith('.html')) continue;
+    if (file.route && notFound && normaliseRoute(file.route) === notFound) continue;
+    locs.add(`${siteUrl}${toPublicPath(publicPath(file.file), config.base)}`);
+  }
+  if (locs.size === 0) return null;
+
+  const lastmod = generatedAt.toISOString().slice(0, 10);
+  const urls = [...locs]
+    .sort()
+    .map(
+      (loc) =>
+        `  <url>\n    <loc>${escapeXml(loc)}</loc>\n    <lastmod>${lastmod}</lastmod>\n  </url>`,
+    )
+    .join('\n');
+  return {
+    contents: `<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${urls}\n</urlset>\n`,
+    routes: locs.size,
+  };
+}
+
+/**
+ * Maps an output file to the path a host serves it at: index.html is the root,
+ * about/index.html is /about/, and anything else keeps its name.
+ */
+export function publicPath(file) {
+  const normalised = String(file).replaceAll('\\', '/');
+  if (normalised === 'index.html') return '/';
+  if (normalised.endsWith('/index.html')) return `/${normalised.slice(0, -'index.html'.length)}`;
+  return `/${normalised}`;
+}
+
+/**
+ * Appends the sitemap location to a robots.txt that exists but does not mention it yet,
+ * leaving a file that already points somewhere alone.
+ */
+export function withSitemapDirective(robots, sitemapUrl) {
+  if (/^\s*sitemap:/im.test(robots)) return null;
+  const separator = robots.endsWith('\n') ? '' : '\n';
+  return `${robots}${separator}Sitemap: ${sitemapUrl}\n`;
+}
+
+// Escapes the characters that would break a sitemap element.
+function escapeXml(value) {
+  return value.replace(/[&<>"']/g, (char) => XML_ESCAPES[char]);
+}

package/src/core/verify.js

@@ -7,6 +7,10 @@ const HYDRATION_PATTERNS = [
   /hydrat/i,
   /did not match/i,
   /server-rendered HTML/i,
+  // React 16 and 17 phrase their mismatch warnings differently and never use the
+  // minified codes, which only exist from React 18 on.
+  /did not expect server HTML/i,
+  /expected server HTML to contain/i,
   /react\.dev\/errors\/(418|419|421|422|423|424|425)/i,
   /Minified React error #(418|419|421|422|423|424|425)\b/i,
 ];