mikser-io 9.22.0 → 9.24.0

package/docs/diagnostics.md

@@ -102,21 +102,60 @@ under this flag, so stdout parses whole.
 npx mikser --json | jq '.summary'
 ```
 
-Four buckets, and the distinction between them is the point:
+Five buckets, and the distinction between them is the point:
 
 | Bucket | Meaning |
 | --- | --- |
-| `rendered` | the render ran, with a `reason` per entity |
+| `rendered` | the render ran and the output moved, with a `reason` per entity |
 | `skipped` | the manifest decided not to render, with a `reason` |
 | `unchanged` | the render ran and produced bytes identical to what was already on disk |
+| `errors` | the render ran and **threw** — with `id`, `destination`, `error`, `layout` |
 | `gated` | a count — the source was unchanged, so no render was ever scheduled |
 
+A failed render appears in `errors` and **not** in `rendered`: that bucket
+means the output moved, and a throw writes nothing. The previous good bytes
+stay on disk, which is what makes a failed render survivable — and also
+what makes it invisible without this bucket.
+
+**A one-shot build with render errors exits `1`.** That is the signal a CI
+gate needs, because `mikser && mikser --verify` would otherwise pass a
+build in which nothing rendered: `--verify` compares the output against the
+manifest, both of which still describe the last good render. Watch mode
+keeps running — a failed render there is a state to fix on the next cycle,
+not a reason to tear down the watcher.
+
 `reason` is a stable vocabulary you can assert on: `unchanged`,
 `never-rendered`, `inputs-changed`, `ref-changed`, `query-matched`,
 `cache-disabled`, `postprocessor`, `force`, `no-manifest`.
 
-`inputs-changed` carries a `changed` array naming **which** input moved,
-so you do not have to go to the database to find out:
+Each reason carries the detail behind it, so the answer does not require a
+database query. The key is per-reason — `changed`, `matched`,
+`dependency` — because each means something specific:
+
+| reason | detail | says |
+| --- | --- | --- |
+| `inputs-changed` | `changed: ["meta.title"]` | which of the entity's own inputs moved |
+| `query-matched` | `matched: { filter, by }` | which query fired, and which mutated entity tripped it |
+| `ref-changed` | `dependency: { kind, target, key, cause }` | which dependency, and whether it `changed`, was `deleted`, or was `unhashed` (nothing resolved when the edge was recorded) |
+
+`query-matched` is the one worth reading. It fires on pages you were not
+thinking about — a listing whose filter happens to cover the document you
+just edited — and on a page with a dozen query edges, *which* one fired is
+the whole question:
+
+```json
+{ "destination": "/index.html", "reason": "query-matched",
+  "matched": { "filter": { "id": { "$regex": "^/documents/devices/" } },
+               "by": "/documents/devices/hera.md" } }
+```
+
+A `matched.filter` of `null` is a different statement: the page's predicate
+could not be serialized (`findEntities()` with no argument, or a function
+filter), so it re-renders on **any** mutation. That is not a query that
+matched — it is a page with no filter, and narrowing it is the fix the
+catalog already warns about at record time.
+
+`inputs-changed` carries a `changed` array naming **which** input moved:
 
 ```json
 { "id": "/files/hero.jpg",     "reason": "inputs-changed", "changed": ["checksum"] }
@@ -132,9 +171,14 @@ reads as `meta.weight (added)` / `(removed)`, which is the answer when a
 document gains or loses front-matter.
 
 The array is absent on a first render — there is no prior snapshot to
-compare against — and on `ref-changed`, because that is a dependency
-moving rather than the entity's own inputs. Conflating the two would make
-the attribution misleading.
+compare against. It never appears alongside `matched` or `dependency`
+either: a consumer switches on `reason` and reads one field, so a stray
+key from another branch would make that switch wrong.
+
+The same detail appears at `--debug` for a watch run, one line per render.
+It is deliberately not in the build's normal output — the counts are the
+summary and `--json` is the record — but when you are watching one page
+misbehave, the trigger is the point.
 
 `unchanged` is the interesting one. It means invalidation was coarser
 than it needed to be — the render was scheduled, ran, and produced

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mikser-io",
-  "version": "9.22.0",
+  "version": "9.24.0",
   "description": "<p align=\"center\">   <img src=\"mikser-lockup-stacked.svg\" alt=\"mikser\" width=\"198\" /> </p>",
   "main": "index.js",
   "exports": {

package/src/engine.js

@@ -10,7 +10,7 @@ import { useJournal, updateEntry } from './journal.js'
 import { globby } from 'globby'
 import { OPERATION, TASKS } from './constants.js'
 import { changeExtension, formatErrorContext, projectMeta, lookupKeys } from './utils.js'
-import { reportRendered, reportSkipped, emitReport } from './report.js'
+import { reportRendered, reportSkipped, reportError, renderErrorCount, emitReport } from './report.js'
 import render from './render.js'
 import postprocess, { loadPlugin as loadPostPlugin } from './postprocess.js'
 import map from 'p-map'
@@ -374,7 +374,24 @@ export async function setup(options) {
                     logger.debug('Manifest skip: %s → %s', entity.name || entity.id, entity.destination)
                     return
                 }
-                reportRendered(entity, decision.reason, decision.changed)
+                // Reported on the way OUT, not here: `rendered` means the
+                // output moved, and a render that throws writes nothing. The
+                // decision is carried down to the success path so the reason
+                // and its detail still travel with it.
+                //
+                // Same detail at debug, for tailing a watch run. One line per
+                // render is too much for a build's normal output — the counts
+                // are the summary and --json is the record — but when you are
+                // watching one page misbehave, the trigger is the whole point.
+                if (logger.isLevelEnabled?.('debug') ?? true) {
+                    logger.debug('Render %s: %s%s', entity.id, decision.reason,
+                        decision.changed?.length ? ` (${decision.changed.join(', ')})`
+                        : decision.matched ? ` (${JSON.stringify(decision.matched.filter)}`
+                            + `${decision.matched.by ? ` ← ${decision.matched.by}` : ''})`
+                        : decision.dependency ? ` (${decision.dependency.kind} `
+                            + `${decision.dependency.target} ${decision.dependency.cause})`
+                        : '')
+                }
                 // Project reference-marker keys (`$author`, `$hero`, …)
                 // into their normalized form (`author`, `hero`) before
                 // the entity crosses into the renderer — applies whether
@@ -509,11 +526,22 @@ export async function setup(options) {
                         await updateEntry({ id, output: entry.output, deps: edges })
                     }
 
+                    reportRendered(entity, decision.reason, decision)
                     logger.debug('Rendered: [%s] %s → %s', options.renderer, entity.name || entity.id, entity.destination)
                 } catch (err) {
                     if (!signal.aborted) {
                         await updateEntry({ id, output: { success: false } })
-                        logger.error('Render error: %s%s %s', entity.id, formatErrorContext(entity, err, runtime.options), err.message)
+                        const context = formatErrorContext(entity, err, runtime.options)
+                        logger.error('Render error: %s%s %s', entity.id, context, err.message)
+                        // The machine-readable half of that same line. Without
+                        // it a build that fails every page reports rendered:N,
+                        // warnings:0 and exits 0 — three clean signals and only
+                        // the human log knowing otherwise.
+                        reportError(entity, err, {
+                            renderer: options.renderer ?? null,
+                            layout: entity.layout?.id ?? null,
+                            context: context.trim() || null,
+                        })
                     }
                     logger.debug('Render canceled')
                 }
@@ -524,8 +552,12 @@ export async function setup(options) {
             concurrency: runtime.options.threads,
             signal
         })
-        renderJobs.size && logger.info('Rendered: %d', renderJobs.size - skipped)
+        // Jobs minus skips minus THROWS. Counting a failed render as
+        // rendered is the same overstatement the report used to make.
+        const failed = renderErrorCount()
+        renderJobs.size && logger.info('Rendered: %d', renderJobs.size - skipped - failed)
         skipped && logger.info('Manifest skipped: %d', skipped)
+        failed && logger.error('Render errors: %d', failed)
     })
 
     onBeforePostprocess(async (signal) => {
@@ -752,11 +784,31 @@ export async function setup(options) {
                 }
             }
         }
-        logger.notice('Mikser completed')
+        // A cycle with failed renders is not a completed build, and the word
+        // people read is this one.
+        const failed = renderErrorCount()
+        if (failed) logger.error('Mikser completed with %d render error%s', failed, failed === 1 ? '' : 's')
+        else logger.notice('Mikser completed')
+
         // After the cycle, and only under --json. stdout has been kept clear
         // for exactly this (the logger writes to stderr under --json), so the
         // document is the only thing on it and can be piped to jq.
         emitReport()
+
+        // Non-zero for a one-shot build, so `mikser && mikser --verify` cannot
+        // pass with every page in the site stale. `exitCode` rather than
+        // process.exit so the report above is flushed and shutdown runs.
+        //
+        // Watch mode keeps going: a failed render there is a state to fix in
+        // the next cycle, not a reason to tear down the watcher. That is also
+        // what makes the failure self-concealing in watch — the errors scroll
+        // past between two green builds — so the exit code is precisely the
+        // signal CI needs and the one interactive use must not have.
+        //
+        // 1, not 2: --verify already uses 2 for output drift and --explain 3
+        // for not-found. "The build ran and some renders threw" is its own
+        // thing.
+        if (failed && !runtime.options.watch) process.exitCode = 1
     })
 
     onCancelled(async () => {

package/src/manifest.js

@@ -389,11 +389,36 @@ export function createManifest(db) {
             const sourceLang = entity?.meta?.lang ?? null
             for (const entry of snapshot.refClosure) {
                 if (entry.kind === 'query') {
-                    if (!entry.filter) return { skip: false, reason: 'query-matched' }
+                    // A null filter is the sentinel for a predicate that
+                    // could not be serialized — findEntities() with no
+                    // argument, or a function filter. It re-renders on ANY
+                    // mutation, which is a materially different situation
+                    // from "this specific query matched" and is reported as
+                    // such: an aggregate page that always re-renders is a
+                    // filter worth narrowing, which is what catalog.js
+                    // already warns about at record time.
+                    if (!entry.filter) {
+                        return {
+                            skip: false,
+                            reason: 'query-matched',
+                            matched: { filter: null, by: null },
+                        }
+                    }
                     if (!mutatedEntities?.size) continue
                     const matcher = sift(entry.filter)
                     for (const mutated of mutatedEntities.values()) {
-                        if (matcher(mutated)) return { skip: false, reason: 'query-matched' }
+                        // Both the filter and the entity that tripped it are
+                        // live here. Reporting only "a query matched" throws
+                        // away the two facts the reader needs — on a page with
+                        // eighteen query edges, which one fired and what set
+                        // it off is the entire question.
+                        if (matcher(mutated)) {
+                            return {
+                                skip: false,
+                                reason: 'query-matched',
+                                matched: { filter: entry.filter, by: mutated.id ?? null },
+                            }
+                        }
                     }
                     continue
                 }
@@ -433,11 +458,26 @@ export function createManifest(db) {
                             continue
                         }
                     }
-                    if (!entry.hash) return { skip: false, reason: 'ref-changed' }
+                    // Which dependency, and why. Three distinct causes reach
+                    // the same reason, and the difference is what the reader
+                    // is after: a layout whose bytes moved, a $-ref whose
+                    // target was deleted, and an edge recorded with no hash
+                    // are three different things to go and look at.
+                    //
+                    //   unhashed  nothing resolved when the edge was recorded,
+                    //             so any mutation of that name re-renders
+                    //   deleted   the target is gone from the catalog
+                    //   changed   the target's own input hash moved
+                    const dependency = (cause) => ({
+                        skip: false,
+                        reason: 'ref-changed',
+                        dependency: { kind: entry.kind, target: entry.target, key, cause },
+                    })
+                    if (!entry.hash) return dependency('unhashed')
                     const currentHash = currentHashes?.get(key)
                     if (currentHash === undefined) continue
-                    if (currentHash === null) return { skip: false, reason: 'ref-changed' }
-                    if (currentHash !== entry.hash) return { skip: false, reason: 'ref-changed' }
+                    if (currentHash === null) return dependency('deleted')
+                    if (currentHash !== entry.hash) return dependency('changed')
                 }
             }
             return { skip: true, reason: 'unchanged' }

package/src/report.js

@@ -13,7 +13,7 @@ import runtime from './runtime.js'
 
 function store() {
     runtime.state ??= {}
-    runtime.state.report ??= { rendered: [], skipped: [], unchanged: [], warnings: [], gated: 0 }
+    runtime.state.report ??= { rendered: [], skipped: [], unchanged: [], errors: [], warnings: [], gated: 0 }
     return runtime.state.report
 }
 
@@ -28,15 +28,25 @@ export function reportGated(count = 1) {
     store().gated += count
 }
 
-export function reportRendered(entity, reason, changed) {
+// `decision` is the skipDecision that led here. Its detail travels with the
+// reason, because a reason on its own answers a question nobody asked: WHICH
+// input moved, WHICH query matched and what tripped it, WHICH dependency
+// changed and how. All of it is in hand where the decision is made, and a
+// consumer that has to go to the database for it is one the report failed.
+//
+// Detail keys are per-reason rather than one generic field — `changed`,
+// `matched`, `dependency` each mean something specific, and a single
+// polymorphic key would push the type switch onto every consumer.
+export function reportRendered(entity, reason, decision = {}) {
     if (!runtime.options?.json) return
     store().rendered.push({
         id: entity?.id,
         destination: entity?.destination ?? null,
         reason,
-        // Which input moved, when the reason is inputs-changed. Omitted
-        // rather than empty so a consumer can test for its presence.
-        ...(changed?.length ? { changed } : {}),
+        // Omitted rather than empty, so presence is meaningful.
+        ...(decision.changed?.length ? { changed: decision.changed } : {}),
+        ...(decision.matched ? { matched: decision.matched } : {}),
+        ...(decision.dependency ? { dependency: decision.dependency } : {}),
     })
 }
 
@@ -67,15 +77,53 @@ export function reportWarning(code, fields = {}) {
     store().warnings.push({ code, ...fields })
 }
 
+// A render that RAN and THREW. Recorded unconditionally — not gated on
+// --json like the other buckets — because the exit code depends on the
+// count, and a build that fails 12 renders must not exit 0 just because
+// nobody asked for a report.
+//
+// Failed entities do NOT appear in `rendered`. That bucket means the output
+// moved, and a throw writes nothing: the previous good bytes stay on disk,
+// which is what makes the failure survivable and also what makes it
+// invisible. `rendered: 12` beside zero written files is the misleading
+// half, and summing buckets should not require knowing that.
+export function reportError(entity, err, context = {}) {
+    const store = errorStore()
+    store.push({
+        id: entity?.id ?? null,
+        destination: entity?.destination ?? null,
+        error: err?.message ?? String(err),
+        ...context,
+    })
+}
+
+// Errors are counted even without --json, so they need a store that does
+// not depend on the report being requested.
+function errorStore() {
+    runtime.state ??= {}
+    runtime.state.renderErrors ??= []
+    return runtime.state.renderErrors
+}
+
+// How many renders threw this cycle. Read by the engine to decide the
+// process exit code.
+export function renderErrorCount() {
+    return errorStore().length
+}
+
 export function buildReport() {
     const report = store()
     return {
         rendered: report.rendered,
         skipped: report.skipped,
         unchanged: report.unchanged,
+        // Renders that ran and threw. A build with a non-empty `errors` is a
+        // failed build, whatever the other counts say.
+        errors: errorStore(),
         warnings: report.warnings,
         summary: {
             rendered: report.rendered.length,
+            errors: errorStore().length,
             // Of those renders, how many produced bytes identical to
             // what was already on disk — see reportUnchanged.
             unchanged: report.unchanged.length,