mikser-io 9.70.0 → 9.73.0

package/docs/configuration.md

@@ -64,6 +64,46 @@ These options are part of `runtime.options` and apply to the engine itself.
 | `server.requestTimeout` | — | number | node default (`300000`) | Milliseconds a single request may take, set on the underlying `http.Server`. Node's 5-minute default is effectively an **upload size limit expressed in seconds** — a large file over a slow link is indistinguishable from a stalled request, so it is cut off and the caller sees a truncated write rather than a readable error. Raised to two hours automatically when any route registers as `streaming` (an upload surface such as `mikser-io-drive`), because Node's default mismeasures exactly those; set this explicitly to override. `0` disables the cap: reasonable on a trusted-network build server, bad facing the internet, where it removes the only bound on how long a client can hold a connection open doing nothing. `headersTimeout` is clamped to stay at or below it. The server is also exposed as `runtime.options.httpServer`. |
 | `url` | `-u, --url <url>` | string | — | Public URL where this mikser is reachable (e.g. `https://blog.me.com`). Validated, trailing slash stripped, stamped on `runtime.options.url`. Read by webhook-capable plugins for push-vs-poll gating (`url.startsWith('https://')`); used by anything that surfaces absolute URLs externally — MCP preview URLs returned to agents, forms share links, email tracking pixels. Plugins that just need internal URLs keep using `runtime.options.port`. |
 
+### `siteRoots`
+
+Which subtrees of the output folder are deployed as their own domain root.
+
+```js
+export default {
+    // out/bg, out/en and out/mk each deploy to their own domain
+    siteRoots: ['bg', 'en', 'mk'],
+}
+```
+
+Read by the url helpers and by the broken-reference check.
+
+**It changes the output.** `asset`, `href` and `resource` build a path from the
+page to the target, and without this they measure from the output folder. When
+`out/bg` is what gets deployed, that is one directory too far — every url
+carries an extra `..` for the site segment. A browser floors a climb above the
+origin root rather than failing, so those urls load and nothing reports them.
+Declaring the roots makes the helpers measure from the site instead, and the
+urls become what they should have been.
+
+The check reads it for the same reason, and resolves urls the way a browser
+does. Where the site root is decides whether `../../x.svg` on a given page is
+correct, merely over-deep, or broken.
+
+Default is the output folder itself, which is right for the ordinary case of
+one site per build — and with nothing declared every url is byte-identical to
+what it was. Only a build that emits several sites moves, and it moves from
+working-by-flooring to correct.
+
+Declaring it also asserts something: **a site root is a deployable unit.** It
+is served alone, so anything its pages reference has to exist beneath it —
+share a common assets folder into each root rather than beside them. A url to
+a target in a *different* root is left as it was, because on a per-domain
+deploy no relative path reaches another origin; the check will report it, which
+is the honest answer.
+
+Nothing can infer any of this: it is a fact about where the bytes get deployed,
+not about the bytes.
+
 ## Engine Substrate
 
 The catalog, inverse-ref graph, render snapshot manifest, and per-cycle

package/docs/diagnostics.md

@@ -231,7 +231,7 @@ the whole question:
 ```json
 { "destination": "/index.html", "reason": "query-matched",
   "matched": { "filter": { "id": { "$regex": "^/documents/devices/" } },
-               "by": "/documents/devices/hera.md" } }
+               "by": "/documents/devices/model-a.md" } }
 ```
 
 A `matched.filter` of `null` is a different statement: the page's predicate
@@ -873,10 +873,30 @@ surfaces that turn silence into a statement:
   `resource()` *build* a URL from a naming convention rather than looking
   an entity up, so they cannot fail: a preset that never ran, or a
   template naming an extension the preset no longer emits, yields a
-  well-formed URL to nothing. Every such call is recorded on the render
-  track and checked against the output folder at finalize; what is missing
-  is warned under `asset-missing`, naming the path and the pages that
-  linked it.
+  well-formed URL to nothing. Two checks run at finalize, from different
+  evidence:
+  - **The emitted output**, read back and resolved the way a browser
+    would — `src`, `href`, `poster`, `srcset` and CSS `url()` across
+    html and css. Anything resolving to no file warns under
+    `reference-broken`, naming the url and the pages carrying it. This
+    one sees paths written by hand, not just helper output.
+  - **The render track**, which records every `asset()` / `resource()`
+    call and tests the destination it built. This catches a url that
+    never reaches an html file at all — one emitted into a feed or a
+    sitemap — and warns under `asset-missing`. Where both can see the
+    same file, the output scan reports it and this one stays quiet.
+- **A link that works only by accident** — a url with one `..` too many
+  still loads, because a browser discards a climb above the origin root
+  rather than failing. Reported under `reference-over-deep`, separately
+  from the outright failures, and grouped by how far each climbed:
+  - **One url, or several climbing different distances** — each is a
+    latent 404, working today and broken as soon as the same markup
+    renders one level deeper.
+  - **Every url climbing the same distance** — not N problems but one
+    base that is off by a constant, reported once and flagged
+    `structural` in `--json`. The urls work at every depth. Usually it
+    means `siteRoots` is undeclared for a build that emits several
+    sites; see [configuration](./configuration.md#siteroots).
 
 ## See also
 

package/docs/rendering.md

@@ -392,7 +392,15 @@ naming the path and the pages that linked it, with a summary under
 deployed into the output as a link resolves normally.
 
 It is a warning, not an error: a build can legitimately link a file that
-some later step supplies. What it removes is the silence.
+some later step supplies, and a missing asset must not stop a dev server.
+What it removes is the silence.
+
+A second check reads the **emitted output** rather than the render track —
+every `src` / `href` / `poster` / `srcset` / CSS `url()` in the html and
+css that shipped, resolved the way a browser resolves it. It covers paths
+written by hand, which no helper ever saw, and it is the only one that can
+see a url which loads solely because the browser floored a `..` run at the
+site root. See [diagnostics](./diagnostics.md#when-mikser-is-silent).
 
 ---
 

package/package.json

@@ -1,6 +1,17 @@
 {
   "name": "mikser-io",
-  "version": "9.70.0",
+  "version": "9.73.0",
+  "files": [
+    "app.js",
+    "index.js",
+    "src/",
+    "testing/",
+    "docs/",
+    "favicon.ico",
+    "favicon.svg",
+    "mikser-mark.svg",
+    "mikser-lockup-stacked.svg"
+  ],
   "description": "A mixer for content: entities in, configurable render pipelines, outputs of any kind. Static sites are the canonical recipe, not the definition — the same engine renders PDFs, emails and whatever a renderer plugin produces. Files are the source of truth, every lifecycle phase is observable, and the build graph is queryable by an agent.",
   "main": "index.js",
   "exports": {

package/src/engine.js

@@ -12,6 +12,7 @@ import { globby } from 'globby'
 import { OPERATION, TASKS } from './constants.js'
 import { changeExtension, formatErrorContext, projectMeta, lookupKeys } from './utils.js'
 import { reportRendered, reportSkipped, reportError, renderErrorCount, emitReport, finishCycle, reportAssetUse, assetUse } from './report.js'
+import { checkReferences } from './references.js'
 import { toolSchemas, invokeTool, toolResultText, toolResultFailed } from './tools.js'
 import { registerBuiltinTools } from './builtin-tools.js'
 import { useDatabase } from './database/index.js'
@@ -89,6 +90,87 @@ function workerSafeOptions(opts) {
     return result
 }
 
+// Warn for anything the EMITTED output points at that is not there.
+//
+// Complements the helper-call check below rather than repeating it: this reads
+// what shipped, so it also sees paths written by hand, and it resolves them the
+// way a browser does, which is the only way to see a url that works solely
+// because a `..` run was floored at the site root.
+//
+// A floored url is not broken today. It is the same markup one level deeper
+// away from being broken, and it means the emitted depth does not match the
+// page — so it is reported separately rather than folded in with the failures.
+//
+// Warn, never fail: a missing asset must not stop a dev server. Both lists
+// carry stable codes into `--json` so a deploy script can decide for itself.
+// Returns the set of broken targets so the helper-call check can skip them.
+async function reportBrokenReferences(logger) {
+    const outputFolder = runtime.options.outputFolder
+    if (!outputFolder || !existsSync(outputFolder)) return new Set()
+
+    const siteRoots = runtime.config?.siteRoots ?? []
+    const { broken, overDeep, checked } = await checkReferences(outputFolder, { siteRoots })
+    if (!checked) return new Set()
+
+    const SHOWN = 10
+    const named = (files) =>
+        files.slice(0, 3).join(', ') + (files.length > 3 ? ` and ${files.length - 3} more` : '')
+
+    for (const { url, target, files } of broken.slice(0, SHOWN)) {
+        logger.warn({ code: 'reference-broken', url, target, files },
+            'Resolves to nothing: %s (from %s) — %s', url, named(files), target)
+    }
+    if (broken.length) {
+        logger.warn({ code: 'reference-broken-summary', broken: broken.length, checked },
+            '%d of %d reference(s) in the output resolve to nothing%s. A URL helper builds the '
+            + 'path rather than looking it up, so these are links to files nothing produced.',
+            broken.length, checked, broken.length > SHOWN ? `, ${SHOWN} shown` : '')
+    }
+
+    // Grouped by how FAR each climbed, because a site whose every over-deep url
+    // climbs the same distance does not have N problems — it has one base that
+    // is off by a constant. Printing it N times is precisely how a real signal
+    // gets filtered out, which is the failure this check exists to prevent.
+    const byClimb = new Map()
+    for (const entry of overDeep) {
+        if (!byClimb.has(entry.floored)) byClimb.set(entry.floored, [])
+        byClimb.get(entry.floored).push(entry)
+    }
+    // One distance, many urls: structural. The helper's base is wrong, the urls
+    // are not — they load at every depth, because the climb is always floored.
+    // That wants a different reaction than a hand-written `../..` that happens
+    // to be right on one page and is a 404 waiting on the next.
+    const structural = byClimb.size === 1 && overDeep.length > 1
+
+    for (const [climb, entries] of [...byClimb].sort(([a], [b]) => a - b)) {
+        const examples = entries.slice(0, 3).map(e => e.url)
+        logger.warn(
+            {
+                code: 'reference-over-deep', climbs: climb, count: entries.length,
+                structural, urls: examples,
+                files: [...new Set(entries.flatMap(e => e.files))].slice(0, 3),
+            },
+            structural
+                ? '%d references climb %d level(s) above the site root — every one of them, by the '
+                  + 'same amount. They load: a browser discards the extra `..`. What is wrong is the '
+                  + 'base they were built from, not the links. Examples: %s'
+                : '%d reference(s) climb %d level(s) above the site root and load only because a '
+                  + 'browser discards the extra `..`. Each breaks if the same markup renders one '
+                  + 'level deeper. Examples: %s',
+            entries.length, climb, examples.join(', '))
+    }
+
+    if (overDeep.length) {
+        logger.warn({ code: 'reference-over-deep-summary', overDeep: overDeep.length, checked, structural },
+            '%d of %d reference(s) resolve above the site root.%s',
+            overDeep.length, checked,
+            siteRoots.length ? '' : ' No siteRoots are declared, so this resolved against the output '
+                + 'root — declare siteRoots if a subtree is deployed as its own domain.')
+    }
+
+    return new Set(broken.map(b => b.target))
+}
+
 // Warn for anything a render linked to that is not in the output.
 //
 // Deliberately phrased as what was OBSERVED. Only entities that rendered this
@@ -96,7 +178,7 @@ function workerSafeOptions(opts) {
 // and says nothing about the rest — the same reasoning the assets plugin
 // already applies to its preset warning, and for the same reason: a warning
 // that overclaims gets filtered, and the filtered-out line is the real one.
-async function reportMissingAssets(logger) {
+async function reportMissingAssets(logger, alreadyReported = new Set()) {
     const used = assetUse()
     if (!used.length) return
     const outputFolder = runtime.options.outputFolder
@@ -105,6 +187,10 @@ async function reportMissingAssets(logger) {
     const missing = []
     for (const [destination, ids] of used) {
         const file = path.join(outputFolder, destination.replace(/^\//, ''))
+        // The output scan resolves the same file the way a browser does and
+        // names the pages that link it, which is strictly more useful. Where
+        // both would fire, one warning is enough.
+        if (alreadyReported.has(destination.replace(/^\//, ''))) continue
         if (!existsSync(file)) missing.push([destination, ids])
     }
     if (!missing.length) return
@@ -412,6 +498,15 @@ export async function setup(options) {
     // runtime.config) and before any plugin's onLoaded.
     onLoad(async () => {
         const logger = useLogger()
+
+        // Onto OPTIONS, not left on config: the url helpers read it, and they
+        // run in render workers, which receive the worker-safe options and
+        // never see runtime.config.
+        runtime.options.siteRoots = runtime.config?.siteRoots ?? []
+        if (runtime.options.siteRoots.length) {
+            logger.info('Site roots: %s', runtime.options.siteRoots.join(', '))
+        }
+
         const cli = runtime.options.url
         const cfg = runtime.config?.url
         const raw = cli ?? cfg
@@ -1138,7 +1233,8 @@ export async function setup(options) {
         // Checked at the end of the cycle because that is the first moment the
         // answer is stable: derivatives are produced during the cycle, so
         // asking any earlier would report files that were about to appear.
-        await reportMissingAssets(useLogger())
+        const brokenTargets = await reportBrokenReferences(useLogger())
+        await reportMissingAssets(useLogger(), brokenTargets)
 
         // After the cycle, and only under --json. stdout has been kept clear
         // for exactly this (the logger writes to stderr under --json), so the

package/src/plugins/render/asset.js

@@ -1,6 +1,6 @@
 import path from 'node:path'
 
-import { changeExtension } from '../../utils.js'
+import { changeExtension, siteRelativeUrl } from '../../utils.js'
 
 // `{{asset 'web' '/media/hero.jpg'}}` — the deployed URL of a preset
 // derivative, relative to the page asking for it.
@@ -71,8 +71,10 @@ export function load({ runtime, entity, state, options, logger, track }) {
         // itself — it takes a path, not an entity, so there is nothing to look
         // up and the URL is well-formed whether or not anything produced it.
         track?.asset?.(destination)
-        const from = path.dirname(entity.destination || '/')
-        return { url: path.relative(from, destination) }
+        // Resolved within the site this page belongs to, not against the
+        // output root — see siteRelativeUrl. With no siteRoots declared the two
+        // are the same folder and the url is byte-identical.
+        return { url: siteRelativeUrl(entity.destination, destination, options?.siteRoots) }
     }
 }
 

package/src/plugins/render/file.js

@@ -72,7 +72,7 @@ function warnIfUntrackable(options, resolved, logger) {
     //
     // The first version of this hardcoded five content folders, and a project
     // registering its own collections through sources() has more than five. On
-    // lmed that meant 63 warnings per build, one for every stylesheet and
+    // one real site that meant 63 warnings per build, one for every stylesheet and
     // script, all of them tracked correctly and every one of them saying the
     // opposite. Which is worse than not warning: 63 spurious lines a build
     // teaches you to filter the channel, and the filtered-out line is the real

package/src/plugins/render/href.js

@@ -1,4 +1,5 @@
 import path from 'node:path'
+import { siteRelativeUrl } from '../../utils.js'
 
 export function load({ entity, runtime, options }) {
     const { clear } = options
@@ -19,16 +20,14 @@ export function load({ entity, runtime, options }) {
 
         let found = runtime.hrefLang(href)
         if (!found) {
-            const from = path.dirname(entity.destination || '/')
-            return { url: path.relative(from, href) }
+            return { url: siteRelativeUrl(entity.destination, href, options?.siteRoots) }
         } else {
             if (!found.id) {
                 found = found[lang]
             }
             if (found?.destination) {
                 const destination = clear ? found.destination.replace('index.html', '') : found.destination
-                const from = path.dirname(entity.destination || '/')
-                found.url = path.relative(from, destination)
+                found.url = siteRelativeUrl(entity.destination, destination, options?.siteRoots)
             }
             return found
         }

package/src/plugins/render/resource.js

@@ -1,10 +1,14 @@
 import path from 'node:path'
+import { matchesLibrary, siteRelativeUrl } from '../../utils.js'
 
 export function load({ runtime, entity, state, options, track }) {
     runtime.resource = (url) => {
         const { resourceLib } = state.resources
         for (let library in resourceLib) {
-            if (url.match(library)) {
+            // Same matcher the resources plugin uses to decide what to
+            // DOWNLOAD. When these disagreed, this built urls for files the
+            // plugin never fetched.
+            if (matchesLibrary(url, library)) {
                 const { origin } = new URL(url)
                 const name = url.replace(origin, `${resourceLib[library]}`)
                 const relative = url.replace(origin, `${state.resources.resourcesFolder}/${resourceLib[library]}`)
@@ -13,8 +17,7 @@ export function load({ runtime, entity, state, options, track }) {
                 // than resolving one, so a library that was never copied
                 // yields a link to nothing on a green build.
                 track?.asset?.(destination)
-                const from = path.dirname(entity.destination || '/')
-                return { url: path.relative(from, destination), name }
+                return { url: siteRelativeUrl(entity.destination, destination, options?.siteRoots), name }
             }
         }
     }

package/src/plugins/resources.js

@@ -10,6 +10,7 @@ import * as stream from 'stream'
 import { promisify } from 'util'
 import isUrl from 'is-url'
 import map from 'p-map'
+import { matchesLibrary } from '../utils.js'
 
 export function resources(options = {}) {
     return ({
@@ -24,7 +25,6 @@ export function resources(options = {}) {
         checksum,
         trackProgress,
         updateProgress,
-        matchEntity,
         constants: { OPERATION },
     }) => {
     const collection = 'resources'
@@ -52,6 +52,11 @@ export function resources(options = {}) {
 
         for (let library in (options.libraries || [])) {
             let resource = options.libraries[library]
+                // The key is a REGULAR EXPRESSION source, which is what the
+            // escapeStringRegexp call says: you only escape a string you are
+            // about to compile. The render helper has always read it that way
+            // (`url.match(library)`), so a library declared by `url` is a
+            // prefix pattern matching anything under it.
             runtime.state.resources.resourceLib[resource.match || escapeStringRegexp(resource.url)] = library
         }
     })
@@ -66,7 +71,15 @@ export function resources(options = {}) {
                 _.eachDeep(entity.meta, resource => {
                     if (typeof resource == 'string') {
                         for (let library in resourceLib) {
-                            if (matchEntity(resource, library)) {
+                            // Regex, matching the render helper. This used
+                            // matchEntity, which is a GLOB demanding a full
+                            // match — so a key derived from `url` (a bare
+                            // prefix, no trailing wildcard) matched nothing and
+                            // NO url-declared library was ever downloaded. The
+                            // helper still built urls for them, so pages linked
+                            // files nothing fetched and the build stayed green:
+                            // one string read with two incompatible matchers.
+                            if (matchesLibrary(resource, library)) {
                                 resourceMap[entity.id].push({ library, resource, entity })
                             }
                         }

package/src/references.js

@@ -0,0 +1,177 @@
+// What the build actually shipped, checked against what it actually wrote.
+//
+// The URL helpers BUILD paths from a naming convention rather than resolving
+// an entity, so they cannot fail — `asset` composes
+// `<assetsFolder>/<preset>/<path>` with whatever extension it was handed and
+// never asks whether that file exists. A wrong preset name, a wrong extension,
+// or a source whose derivative silently failed to render all produce a
+// well-formed url pointing at nothing, and every existing surface stays green:
+// nothing threw, --verify compares snapshots against what was rendered rather
+// than against what those renders point at, and mikser_refs_broken tracks
+// document-to-document refs, not urls.
+//
+// This reads the emitted bytes instead. Everything it needs is on disk at the
+// end of a cycle and nothing has to be inferred.
+//
+// It is deliberately NOT the same check as `asset-missing` in engine.js. That
+// one records helper CALLS on the render track and tests the output-root
+// absolute destination each one built; it knows the referencing entity, and it
+// sees urls that never reach an html file at all (a sitemap, a feed). This one
+// sees everything that shipped, including paths written by hand, and resolves
+// them the way a browser would — which is the only way to catch a url that
+// resolves solely because the browser floored a `..` run at the site root.
+
+import path from 'node:path'
+import { existsSync } from 'node:fs'
+import { readFile } from 'node:fs/promises'
+import { globby } from 'globby'
+import { siteRootFor } from './utils.js'
+
+// Documents that can carry a reference. Anything else in the output is either
+// an asset itself or something whose internal structure this has no business
+// guessing at.
+const SCANNED = ['**/*.html', '**/*.htm', '**/*.css']
+
+// Attributes whose value is a single url.
+const ATTR = /(?:src|href|poster|data-bg)\s*=\s*["']([^"']*)["']/gi
+// srcset / imagesrcset: a comma-separated list of `url [descriptor]`.
+const SRCSET = /(?:img|image)?srcset\s*=\s*["']([^"']*)["']/gi
+// css url(), in a stylesheet and in an inline style attribute alike.
+const CSS_URL = /url\(\s*(['"]?)([^'")]*)\1\s*\)/gi
+
+// A url this check has nothing to say about: another origin, an inline
+// payload, a fragment or an in-page action. `//host/path` is protocol-relative
+// and therefore external too.
+function isExternal(url) {
+    if (!url) return true
+    const u = url.trim()
+    if (!u) return true
+    if (u.startsWith('#') || u.startsWith('//')) return true
+    // Template syntax that reached the output unrendered — `{{link}}`,
+    // `${x}`, `<%= y %>`. It is not a path, so "resolves to nothing" says
+    // nothing useful about it; the real problem is that it did not render,
+    // which is a different question than this one is asking. Documentation
+    // pages showing escaped template syntax are the common source, and they
+    // are not broken at all.
+    if (/\{\{|\}\}|\$\{|<%/.test(u)) return true
+    // A scheme — http:, data:, mailto:, tel:, javascript:.
+    if (/^[a-z][a-z0-9+.-]*:/i.test(u)) return true
+    // The same thing percent-encoded, which is how an external url arrives
+    // when it was built as a query parameter — `https%3A%2F%2F...` in a maps
+    // link. It has no scheme until it is decoded, so the test above misses it
+    // and the whole encoded string gets resolved as a path segment.
+    try {
+        if (/^[a-z][a-z0-9+.-]*:/i.test(decodeURIComponent(u))) return true
+    } catch { /* malformed escape — treat as a path and let it resolve */ }
+    return false
+}
+
+// Quotes inside an attribute value arrive encoded, and a CSS custom property
+// in an inline style is the common way that happens:
+//
+//     style="--icon-src:url(&quot;../media/raw/icons/x.svg&quot;)"
+//
+// Without decoding, the captured url is the entity text itself, which resolves
+// nowhere and reports as broken — a false positive that would have buried the
+// real ones. Only the quote and ampersand forms are decoded; turning &lt; back
+// into a bracket could invent markup that was deliberately escaped.
+function decodeEntities(source) {
+    return source
+        .replace(/&quot;|&#34;/g, '"')
+        .replace(/&apos;|&#39;/g, "'")
+        .replace(/&amp;/g, '&')
+}
+
+// Everything a page points at, as raw url strings.
+export { siteRootFor }
+
+export function extractReferences(rawSource) {
+    const source = decodeEntities(rawSource)
+    const found = new Set()
+    for (const [, url] of source.matchAll(ATTR)) found.add(url)
+    for (const [, , url] of source.matchAll(CSS_URL)) found.add(url)
+    for (const [, list] of source.matchAll(SRCSET)) {
+        for (const candidate of list.split(',')) {
+            const url = candidate.trim().split(/\s+/)[0]
+            if (url) found.add(url)
+        }
+    }
+    return [...found].filter(u => !isExternal(u))
+}
+
+// Resolve the way a browser does, which is the whole point.
+//
+// A browser walks the page's directory segments, pops one per `..`, and
+// DISCARDS a `..` that would climb above the origin root — it does not error
+// and it does not escape. So a url with one `..` too many still loads, and the
+// page looks correct while carrying a path that breaks the moment the same
+// markup is used one level deeper. That flooring is what `overDeep` records.
+//
+// `pageDir` and the result are both relative to `root`.
+export function resolveUrl(pageDir, url, { root = '' } = {}) {
+    const clean = url.split('#')[0].split('?')[0]
+    const absolute = clean.startsWith('/')
+    const segments = clean.split('/').filter(s => s !== '' && s !== '.')
+
+    const parts = absolute ? [] : pageDir.split('/').filter(Boolean)
+    // How FAR above the root it climbed, not merely that it did. When every
+    // over-deep url on a site climbs the same distance, that is one base
+    // mismatch reported once — not N findings, which is how a real signal gets
+    // filtered.
+    let floored = 0
+    for (const segment of segments) {
+        if (segment !== '..') { parts.push(segment); continue }
+        if (parts.length) parts.pop()
+        else floored++            // a climb above the root, discarded
+    }
+    return { target: path.join(root, ...parts), overDeep: floored > 0, floored }
+}
+
+// Everything the output points at that is not there.
+//
+// Returns { broken, overDeep, checked }, where each entry is
+// { url, target, files } — the target with the pages that named it, because
+// "this is missing" is only actionable next to "and these link it".
+export async function checkReferences(outputFolder, { siteRoots = [] } = {}) {
+    const files = await globby(SCANNED, {
+        cwd: outputFolder,
+        followSymbolicLinks: false,
+        suppressErrors: true,
+    })
+
+    const broken = new Map()
+    const overDeepRefs = new Map()
+    let checked = 0
+    // Existence is the expensive part and the same target repeats across a
+    // site — one lookup each.
+    const exists = new Map()
+
+    for (const file of files) {
+        let source
+        try { source = await readFile(path.join(outputFolder, file), 'utf8') }
+        catch { continue }
+
+        const root = siteRootFor(file, siteRoots)
+        // The page's directory, relative to its own site root.
+        const pageDir = path.dirname(file).slice(root.length).replace(/^\/+/, '')
+
+        for (const url of extractReferences(source)) {
+            const { target, overDeep, floored } = resolveUrl(pageDir, url, { root })
+            checked++
+
+            if (!exists.has(target)) {
+                exists.set(target, existsSync(path.join(outputFolder, target)))
+            }
+            // Broken outranks over-deep: a url that resolves nowhere is the
+            // failure, and adding that it is also one level too deep is noise.
+            const bucket = !exists.get(target) ? broken : (overDeep ? overDeepRefs : null)
+            if (!bucket) continue
+
+            const key = `${target} ${url}`
+            if (!bucket.has(key)) bucket.set(key, { url, target, floored, files: [] })
+            bucket.get(key).files.push(file)
+        }
+    }
+
+    return { broken: [...broken.values()], overDeep: [...overDeepRefs.values()], checked }
+}

package/src/utils.js

@@ -1316,3 +1316,77 @@ export function junkFilter() {
         return registered.match.some(test => test(name))
     }
 }
+
+// Does a value fall under a resources library?
+//
+// The library key is a REGULAR EXPRESSION source — `resources()` derives it
+// with escapeStringRegexp(url), and you only escape a string you are about to
+// compile. It has two consumers: the plugin's discovery walk, which decides
+// what to download, and the `resource` render helper, which builds the url.
+// They read the same string with two different matchers — discovery used a
+// GLOB, which demands a full match, so a key derived from `url` (a bare prefix
+// with no trailing wildcard) matched nothing. Nothing was ever downloaded for
+// a url-declared library, while the helper happily built links to the files
+// that were not fetched. Green build, missing images.
+//
+// One function, so the two cannot drift again.
+const libraryPatterns = new Map()
+export function matchesLibrary(value, pattern) {
+    if (typeof value !== 'string' || !pattern) return false
+    if (!libraryPatterns.has(pattern)) {
+        let re
+        try { re = new RegExp(pattern) }
+        // A hand-written `match` that is not valid regex would otherwise throw
+        // mid-walk and take the build down.
+        catch { re = { test: () => false } }
+        libraryPatterns.set(pattern, re)
+    }
+    return libraryPatterns.get(pattern).test(value)
+}
+
+// Which declared site root a path belongs to.
+//
+// A build can emit one subtree per language and deploy each as its own domain
+// root, which puts the site root at out/<lang>/ rather than at out/. Nothing
+// can derive that — it is a fact about where the bytes get deployed, not about
+// the bytes — so it is declared as `siteRoots` and the default is the output
+// root itself. Accepts a path with or without a leading slash, because an
+// entity destination has one and an output-relative file path does not.
+export function siteRootFor(file, roots = []) {
+    const relative = String(file ?? '').replace(/^\/+/, '')
+    let best = ''
+    for (const root of roots) {
+        if (!root) continue
+        if (relative.startsWith(`${root}/`) && root.length > best.length) best = root
+    }
+    return best
+}
+
+// A page-relative url from one output destination to another, addressed within
+// the site the page belongs to.
+//
+// Both are output-root absolute (`/bg/aparati/index.html`, `/derived/x.webp`),
+// which is the only shape the engine has. With one site per build that is also
+// the deployed root and this is a plain path.relative. With several, it is not:
+// out/bg IS the domain root, so a url computed against out/ carries one extra
+// `..` for the language segment. The browser floors that rather than failing,
+// which is why it worked and why nothing said so.
+//
+// Three cases, and the middle one is the reason this is not a one-liner:
+//
+//   target outside every root   a shared asset. It has to be reachable from
+//                               inside this page's site, so it is addressed
+//                               there — this is the case that was wrong.
+//   target in the same root     already correct; a plain relative path.
+//   target in a DIFFERENT root  a cross-site link. On a per-domain deploy the
+//                               other site is another origin and no relative
+//                               path reaches it. Left as it was, so the
+//                               reference check reports it broken instead of
+//                               this silently inventing a path that is not.
+export function siteRelativeUrl(pageDestination, target, siteRoots = []) {
+    const from = path.dirname(pageDestination || '/')
+    const pageRoot = siteRootFor(pageDestination, siteRoots)
+    if (!pageRoot) return path.relative(from, target)
+    if (siteRootFor(target, siteRoots)) return path.relative(from, target)
+    return path.relative(from, path.join('/', pageRoot, target))
+}