@warlock.js/sitemap 5.15.0 → 5.16.0

package/CHANGELOG.md

@@ -4,6 +4,24 @@ All notable changes to `@warlock.js/sitemap` are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
 
+## 5.16.0 - 2026-09-18
+
+### Upgrading
+
+- The 5.15 connector API is gone. In a Warlock app, remove `sitemapConnector()` from `warlock.config.ts` and `src/config/sitemap.ts`, then configure `web.sitemap` in `src/config/web.ts` (`warlock add sitemap` writes this section). Outside Warlock, build the sitemap with `new Sitemap({ baseUrl })`.
+
+### Added
+
+- `SitemapIndex`: streams entries into size-capped shards plus a master index, and optionally writes `.xml.gz` files. It publishes the whole output directory atomically and marks it with `.sitemap-set.json`.
+- `Sitemap.publishTo(outDir, fileName?)`, and `saveTo()` now writes atomically (temp file, then rename, with retries on Windows `EPERM`/`EBUSY`).
+- `UnownedOutputDirectoryError`: publishing refuses to replace a non-empty directory that lacks the ownership marker.
+
+### Changed
+
+- **BREAKING:** the package no longer depends on any framework. The connector API is removed: `sitemapConnector()`, `collectSitemapEntries()`, the `SitemapConfig` config module, `MissingPublicUrlError`, `NoPageRegistryError` and `RoutablePage`. Use the `Sitemap` builder class instead (`new Sitemap({ baseUrl })`, `add` / `addMany` / `declareRoute`, `toXML` / `saveTo`). Warlock apps configure `web.sitemap` in `@warlock.js/web` instead.
+- `baseUrl` is validated in the constructor (`InvalidBaseUrlError`). Entries are keyed by path, and `duplicates()` reports every collision.
+- Dropped the `@warlock.js/core` and `@warlock.js/web` dependencies.
+
 ## 5.15.0 - 2026-09-18
 
 ### Added

package/README.md

@@ -1,14 +1,10 @@
 # @warlock.js/sitemap
 
-A `sitemap.xml` generator: exclusion rules (`noindex`, `sitemap: false`),
-per-entry `changefreq`/`priority` defaults, and a loud diagnostic when a
-dynamic route can't be enumerated. `@warlock.js/core` and `@warlock.js/web`
-are **optional peers** — everything except `sitemapConnector()` imports
-nothing from either, and `sitemapConnector()` itself only reaches them
-through a lazy `import()` at boot. Three ways to use it, below.
+A sitemap builder that knows nothing about any framework. Entries in, a valid
+[sitemaps.org](https://www.sitemaps.org/protocol.html) document out.
 
-Generation only in this release. No remote sitemap parser — emitting XML
-needs no dependency; parsing one does, and nothing needs it yet.
+**Zero runtime dependencies.** Generation only — emitting XML needs no
+dependency; parsing one does, and nothing here needs it yet.
 
 ## Install
 
@@ -16,170 +12,216 @@ needs no dependency; parsing one does, and nothing needs it yet.
 npm install @warlock.js/sitemap
 ```
 
-## Mode 1 — Standalone, any Node app
-
-No Warlock at all. Build the `RoutablePage[]` array yourself — from a route
-table, a database, wherever your app already knows its own URLs — and call
-`collectSitemapEntries` + `buildSitemapXml` directly. Nothing on this path
-resolves `@warlock.js/core` or `@warlock.js/web`.
+## Use it
 
 ```ts
-import express from "express";
-import { buildSitemapXml, collectSitemapEntries, type RoutablePage } from "@warlock.js/sitemap";
-
-const app = express();
-
-app.get("/sitemap.xml", async (_req, res) => {
-  const pages: RoutablePage[] = [
-    { routeName: "home", routePath: "/" },
-    { routeName: "about", routePath: "/about" },
-    {
-      routeName: "post-details",
-      routePath: "/posts/:id",
-      sitemap: async () => (await db.posts.find()).map((post) => ({ path: `/posts/${post.slug}` })),
-    },
-  ];
-
-  const { entries } = await collectSitemapEntries(pages, {
-    defaults: { changefreq: "weekly", priority: 0.5 },
+import { Sitemap } from "@warlock.js/sitemap";
+
+const sitemap = new Sitemap({
+  baseUrl: "https://example.com",
+  changefreq: "weekly",
+  priority: 0.5,
+});
+
+sitemap.add({ path: "/" });
+
+for (const post of await Post.all()) {
+  sitemap.add({
+    name: "post-details",
+    route: "/posts/:id",
+    path: `/posts/${post.slug}`,
+    lastmod: post.updatedAt,
+    priority: 0.8,
   });
+}
 
-  const xml = buildSitemapXml(entries, "https://example.com");
+const xml = sitemap.toXML();
 
-  res.type("application/xml").send(xml);
-});
+await sitemap.saveTo("public/sitemap.xml");
 ```
 
-## Mode 2 — Warlock, API-only (no `@warlock.js/web`)
+That is the whole common case. It is the same three lines from an Express
+handler, a cron script, a Warlock app, or a `node build-sitemap.mjs` you wrote
+in five minutes.
 
-An API-only Warlock app has no page registry — `listRoutablePages()` would
-have nothing to list. Pass `entries` and `sitemapConnector()` uses it instead;
-`@warlock.js/web` is not required on this path.
+## The API
 
-```ts
-// warlock.config.ts
-import { sitemapConnector } from "@warlock.js/sitemap";
-
-export default defineConfig({
-  connectors: [
-    sitemapConnector({
-      entries: async () => {
-        const products = await db.products.find();
-
-        return products.map((product) => ({ path: `/products/${product.slug}` }));
-      },
-    }),
-  ],
-});
-```
+### `new Sitemap(options)`
 
-If neither `entries` nor `@warlock.js/web` is available, the connector
-refuses to boot rather than serving an empty `<urlset>` — see
-[`NoPageRegistryError`](#no-page-registry-no-entries) below.
+| Option | Meaning |
+| --- | --- |
+| `baseUrl` | **Required.** Absolute origin, e.g. `https://example.com` or `https://example.com/docs`. |
+| `changefreq` | Applied to any entry that does not set its own. |
+| `priority` | Same. |
+| `lastmod` | Same. |
 
-## Mode 3 — Warlock web
+`baseUrl` is validated **in the constructor**. A `Sitemap` that cannot produce
+a valid URL should not exist, and you should learn about a typo at the line
+that wrote it rather than at the first request. Anything that is not an
+absolute `http(s)` URL throws `InvalidBaseUrlError`.
 
-```bash
-warlock add sitemap
-```
+### `add(entry)` / `addMany(entries)`
+
+Only `path` is required, and it must be a **concrete path** — `/posts/123`, not
+`/posts/:id`. A pattern is not a URL.
+
+| Field | Meaning |
+| --- | --- |
+| `path` | **Required.** Resolved against `baseUrl`. An absolute URL is used as given. |
+| `name` | Optional label for the route, for your own diagnostics. |
+| `route` | Optional pattern this URL came from, e.g. `/posts/:id`. |
+| `lastmod` | `Date` (serialised as W3C datetime) or a string (passed through untouched). |
+| `changefreq` | One of the seven protocol values. |
+| `priority` | `0.0`–`1.0`. Anything else throws. |
+| `alternates` | Language versions of this page — see below. |
+
+Both return `this`, so they chain.
+
+**Adding the same path twice keeps the later entry, silently.** Entries are
+stored keyed by path, because a duplicate `<loc>` makes the document invalid
+and two loops legitimately covering an overlapping set is the normal cause. It
+is silent, not hidden — see `duplicates()`.
 
-writes `src/config/sitemap.ts`:
+### `entries()` / `size`
+
+What will actually be emitted, with defaults already folded in. `entries()`
+hands back a copy.
+
+### `routes()` — the diagnostic
 
 ```ts
-export const sitemapConfig: SitemapConfig = {
-  enabled: true,
-  path: "/sitemap.xml",
-  defaults: { changefreq: "weekly", priority: 0.5 },
-};
+sitemap.routes();
+// [ { route: "/posts/:id", count: 400 }, { route: "/products/:slug", count: 0 } ]
 ```
 
-`sitemapConnector()` reads the page registry from `@warlock.js/web`'s
-`listRoutablePages()` on every request — not once at boot — so it reflects
-the app's current shape under `warlock dev` too.
-
-Only a dynamic route needs a page-level `sitemap` export; a static route is
-included automatically at its own path.
+**The interesting row is the zero.** It means a route you expected to
+contribute URLs contributed none, and a whole section of your site is missing
+from a document that otherwise looks perfect. Use `declareRoute()` to name a
+pattern you expect to produce URLs:
 
 ```ts
-// any *.page.tsx
-export const sitemap: SitemapEntries = async () => [
-  { path: "/posts/hello-world", lastmod: "2026-09-17", priority: 0.8 },
-];
+sitemap.declareRoute("/products/:slug");
+
+const empty = sitemap.routes().filter((route) => route.count === 0);
 
-// or, to keep a page out of the sitemap deliberately
-export const sitemap = false;
+if (empty.length > 0) {
+  throw new Error(`empty sitemap routes: ${empty.map((route) => route.route).join(", ")}`);
+}
 ```
 
-`changefreq` and `priority` are per-entry and optional — they fall back to
-the config's `defaults`. They are not part of `PageMetadata`; they mean
-nothing outside a sitemap.
+**This package reports; it never prints.** Whether a zero is a warning or a
+build failure is your decision, not ours.
 
-**A dynamic route with no `sitemap` export is silently omitted from the
-generated XML if nobody is watching.** In development this package reports it
-instead: a dynamic route cannot be enumerated without application data, and
-the framework's whole job here is to make sure you find out, rather than
-shipping a sitemap that looks complete while it quietly drops every product
-page on the site.
+### `duplicates()`
 
-If a project has both a page registry **and** `entries`, they are combined —
-`entries` are added to the page-derived entries, not a replacement for them.
-Where the same `path` appears in both, the `entries` version wins.
+```ts
+sitemap.duplicates();
+// [ { path: "/posts/1", count: 2, routes: ["/posts/:id", "/:slug"] } ]
+```
 
-## Config reference (`src/config/sitemap.ts`)
+Every path that was added more than once, and the `route` of each add — so a
+collision between two sources is nameable. Never throws.
 
-| key | meaning |
-| --- | --- |
-| `enabled` | no-op when not `true` — the connector registers no route |
-| `path` | defaults to `/sitemap.xml` |
-| `defaults.changefreq` / `defaults.priority` | applied to any entry that omits them, page-derived or app-supplied |
+### `toXML()`
 
-The public origin the sitemap is served from is **not** configured here — it
-lives in `app.publicUrl` (or the `PUBLIC_APP_URL` environment variable), one
-level up in `@warlock.js/core`, because canonical links, OG tags and absolute
-mail URLs need the same value. If the sitemap is `enabled` and no origin is
-configured, the app refuses to boot rather than guess — a sitemap served with
-the wrong host is worse than one that never started.
+Pure and repeatable: calling it twice returns the same string, and it mutates
+nothing. It is **synchronous and stays synchronous** — there is no I/O in it.
 
-## What goes in the sitemap
+`Sitemap` structurally satisfies `@warlock.js/core`'s `XMLable` contract
+(`toXML(): string`), so a Warlock controller can `return response.xml(sitemap)`
+directly — this package still depends on nothing from core to make that true.
+That path is for a single `Sitemap` under the sitemaps.org 50,000-URL / 50MB
+ceiling. `SitemapIndex` has no `toXML()` and is never passed to `response.xml()`;
+see [Very large sites](#very-large-sites) below for how its output is served instead.
 
-| case | behaviour |
-| --- | --- |
-| static route | included |
-| not-found route | excluded |
-| error page | excluded — it isn't a routable page at all |
-| page whose `metadata.robots` says `noindex` | excluded |
-| page exporting `sitemap: false` | excluded |
-| dynamic route (`[id]`, `[...slug]`) **with** a `sitemap` export | the entries that export returns |
-| dynamic route **without** a `sitemap` export | **omitted, and named in a dev-mode diagnostic** |
-| `sitemapConnector({ entries })` result | added to the above, `entries` wins on a `path` collision |
-
-## No page registry, no `entries`
-
-If `@warlock.js/web` is not installed and no `entries` option is supplied,
-`sitemapConnector()` throws `NoPageRegistryError` at `boot()` — before the
-route is even registered — rather than serving an empty sitemap that looks
-correct. Fix it either way:
+### `saveTo(filePath)`
 
-```
-Sitemap is enabled but has no source of entries: `@warlock.js/web` is not installed,
-so there is no page registry to read, and no `entries` option was supplied either.
-Fix this by installing `@warlock.js/web`, or by passing
-`sitemapConnector({ entries: async () => [...] })` with your own supplier.
-```
+Writes the document, creating parent directories. Works on a clean checkout
+with no `dist/`.
 
-## Full documentation
+### `publishTo(outDir, fileName = "sitemap.xml")`
 
-The complete guide lives at
-**[warlock.js.org](https://warlock.js.org/v/latest/sitemap/)**.
+Publishes the document as the whole content of `outDir`, the same way
+`SitemapIndex.saveTo(outDir)` publishes a set: swapped in atomically and marked
+as owned (see below). Use it when the sitemap may later grow into an index: an
+index can then be published into the same directory, and publishing a single
+file again clears out old shards. Returns the published file's path.
 
-## Tests
+## Language alternates
 
-This package uses Vitest:
+`alternates` emits `<xhtml:link rel="alternate" hreflang="…">` inside each
+`<url>` — the thing search engines actually consume to learn that two URLs are
+one page in two languages.
 
-```bash
-yarn test
+```ts
+const alternates = [
+  { hreflang: "en", path: "/en/about" },
+  { hreflang: "ar", path: "/ar/about-us" },
+  { hreflang: "x-default", path: "/en/about" },
+];
+
+sitemap.add({ path: "/en/about", alternates });
+sitemap.add({ path: "/ar/about-us", alternates });
 ```
 
-## License
+Two rules worth knowing:
+
+1. **Every language version is also its own `<url>`, carrying the complete
+   alternate set including itself.** Listing alternates on only one of them is
+   the usual way this gets shipped broken.
+2. **This package does not know what a locale is.** `hreflang` is any string —
+   `en`, `en-GB`, `x-default` — and it never derives `/{locale}/…` for you. You
+   supply each path explicitly, which is the only thing that works when slugs
+   diverge between languages.
+
+The `xhtml` namespace is declared only when something actually uses it.
+
+## Very large sites
+
+The protocol caps one file at **50,000 URLs or 50MB uncompressed**. `Sitemap`
+retains every entry — that is what makes `entries()` and a repeatable
+`toXML()` possible — so it is the right tool up to that ceiling and the wrong
+one above it. `SitemapIndex` is the streaming writer for sites past it: it
+emits shards plus a flat master index and retains nothing but the current
+shard's buffer.
+
+`alternates` works the same way across a shard boundary — each `<xhtml:link>`
+resolves to an absolute URL, so it is invisible to where one shard ends and
+the next begins. Each shard declares `xmlns:xhtml` only when its own entries
+carry alternates, and an alternate's bytes count toward that shard's byte
+ceiling exactly like the rest of its `<url>` block.
+
+`maxBytesPerFile` is enforced at shard boundaries, not within an entry: a
+single entry (with its alternates) larger than the ceiling is written alone
+in its own shard rather than split, since an entry can't be split.
+
+### `saveTo(outDir)` owns the directory it is given
+
+`SitemapIndex.saveTo(outDir)` publishes atomically: it writes the full set
+into a sibling temp directory, then swaps it into `outDir` in one `rename` —
+a crawler never sees a partial set, and a failed run leaves the previous set
+untouched.
+
+That swap replaces the ENTIRE contents of `outDir`, so `outDir` must be a
+directory **dedicated to this sitemap set** — never an app's `public/` or
+any other directory something else writes to. Every successful publish marks
+`outDir` with `.sitemap-set.json`; a later `saveTo()` call only swaps a
+directory that either does not exist yet, is empty, or already carries that
+marker. If `outDir` exists, is non-empty, and has no marker,
+`saveTo()` throws `UnownedOutputDirectoryError` and leaves the directory
+completely untouched. Point `saveTo()` at a directory this package alone
+writes to. `Sitemap.publishTo(outDir)` follows the same rule.
+
+## Errors
+
+| Error | When |
+| --- | --- |
+| `InvalidBaseUrlError` | `baseUrl` is missing, relative, or not `http(s)`. Thrown from the constructor. |
+| `InvalidSitemapEntryError` | An entry the protocol cannot represent: no path, a priority outside `0.0`–`1.0`, an unknown `changefreq`, an invalid `Date`, an alternate with no `hreflang`. |
+| `UnownedOutputDirectoryError` | `SitemapIndex.saveTo(outDir)` or `Sitemap.publishTo(outDir)`: `outDir` is a non-empty directory with no `.sitemap-set.json` marker from a previous publish, so it is refused rather than swapped or deleted. |
+
+## Also exported
 
-MIT
+`buildSitemapXml(entries, baseUrl)`, `escapeXml(value)` and
+`joinOrigin(origin, path)` — the pieces `Sitemap` is built from, for when you
+want the serialiser without the builder.