mikser-io 9.21.1 → 9.23.0

package/docs/diagnostics.md

@@ -59,20 +59,39 @@ snapshot: if you have edited a file and not yet built, the verdict is
 `source differs from the catalog` — the edit has not been imported yet, so
 there is nothing to attribute. Build, then ask.
 
-Each `refClosure` edge shows the name that was asked for and the entity
-it bound to, and flags the ones that bound to nothing:
+Each `refClosure` edge shows what was asked for and what answers to it,
+and flags the ones nothing answers to:
 
 ```
-refClosure  4 edges
+refClosure  6 edges
   ref        /hero.txt → /files/hero.txt  24dc5b87
   ref        /does-not-exist  [UNRESOLVED — nothing answers to this name]
   layout     /layouts/page.hbs  f274678c
   lookup     /contacts  [UNRESOLVED — nothing answers to this name]
+  query      {"meta.href":"/system/navigation"}  → /documents/navigation.yml
+  query      {"meta.href":"/cosmetics/celestetic"}  [MATCHES NOTHING]
 ```
 
-An `[UNRESOLVED]` edge is usually the answer on its own. Add `--json` for
-the same report as a machine-readable object. Exits `3` when the entity
-cannot be found.
+A dangling edge is usually the answer on its own, and `query` edges are
+where most of it hides on a site whose references go through a sidecar's
+`findEntity` — there, `ref` and `lookup` may be zero and `query`
+everything.
+
+Query counts are computed when you ask, not when the edge was recorded.
+That is deliberate: an edge stores the filter WITHOUT its results, so an
+entity appearing tomorrow still invalidates the page that links to it
+today. An edge that recorded its bindings would record nothing for a query
+matching nothing, and creating the target later would re-render nothing.
+
+In `--json`, every query edge carries `matched` explicitly — an integer,
+or `null` for a predicate that could not be serialized (`findEntities()`
+with no argument, or a function filter), which invalidates on any mutation
+by design. `matched: 1` also carries `sample` with the id. The field is
+always present, so a consumer never has to read absence as meaning
+anything.
+
+Add `--json` for the whole report as a machine-readable object. Exits `3`
+when the entity cannot be found.
 
 ### `--json`
 
@@ -96,8 +115,34 @@ Four buckets, and the distinction between them is the point:
 `never-rendered`, `inputs-changed`, `ref-changed`, `query-matched`,
 `cache-disabled`, `postprocessor`, `force`, `no-manifest`.
 
-`inputs-changed` carries a `changed` array naming **which** input moved,
-so you do not have to go to the database to find out:
+Each reason carries the detail behind it, so the answer does not require a
+database query. The key is per-reason — `changed`, `matched`,
+`dependency` — because each means something specific:
+
+| reason | detail | says |
+| --- | --- | --- |
+| `inputs-changed` | `changed: ["meta.title"]` | which of the entity's own inputs moved |
+| `query-matched` | `matched: { filter, by }` | which query fired, and which mutated entity tripped it |
+| `ref-changed` | `dependency: { kind, target, key, cause }` | which dependency, and whether it `changed`, was `deleted`, or was `unhashed` (nothing resolved when the edge was recorded) |
+
+`query-matched` is the one worth reading. It fires on pages you were not
+thinking about — a listing whose filter happens to cover the document you
+just edited — and on a page with a dozen query edges, *which* one fired is
+the whole question:
+
+```json
+{ "destination": "/index.html", "reason": "query-matched",
+  "matched": { "filter": { "id": { "$regex": "^/documents/devices/" } },
+               "by": "/documents/devices/hera.md" } }
+```
+
+A `matched.filter` of `null` is a different statement: the page's predicate
+could not be serialized (`findEntities()` with no argument, or a function
+filter), so it re-renders on **any** mutation. That is not a query that
+matched — it is a page with no filter, and narrowing it is the fix the
+catalog already warns about at record time.
+
+`inputs-changed` carries a `changed` array naming **which** input moved:
 
 ```json
 { "id": "/files/hero.jpg",     "reason": "inputs-changed", "changed": ["checksum"] }
@@ -113,9 +158,14 @@ reads as `meta.weight (added)` / `(removed)`, which is the answer when a
 document gains or loses front-matter.
 
 The array is absent on a first render — there is no prior snapshot to
-compare against — and on `ref-changed`, because that is a dependency
-moving rather than the entity's own inputs. Conflating the two would make
-the attribution misleading.
+compare against. It never appears alongside `matched` or `dependency`
+either: a consumer switches on `reason` and reads one field, so a stray
+key from another branch would make that switch wrong.
+
+The same detail appears at `--debug` for a watch run, one line per render.
+It is deliberately not in the build's normal output — the counts are the
+summary and `--json` is the record — but when you are watching one page
+misbehave, the trigger is the point.
 
 `unchanged` is the interesting one. It means invalidation was coarser
 than it needed to be — the render was scheduled, ran, and produced

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mikser-io",
-  "version": "9.21.1",
+  "version": "9.23.0",
   "description": "<p align=\"center\">   <img src=\"mikser-lockup-stacked.svg\" alt=\"mikser\" width=\"198\" /> </p>",
   "main": "index.js",
   "exports": {

package/src/engine.js

@@ -374,7 +374,20 @@ export async function setup(options) {
                     logger.debug('Manifest skip: %s → %s', entity.name || entity.id, entity.destination)
                     return
                 }
-                reportRendered(entity, decision.reason, decision.changed)
+                reportRendered(entity, decision.reason, decision)
+                // Same detail at debug, for tailing a watch run. One line per
+                // render is too much for a build's normal output — the counts
+                // are the summary and --json is the record — but when you are
+                // watching one page misbehave, the trigger is the whole point.
+                if (logger.isLevelEnabled?.('debug') ?? true) {
+                    logger.debug('Render %s: %s%s', entity.id, decision.reason,
+                        decision.changed?.length ? ` (${decision.changed.join(', ')})`
+                        : decision.matched ? ` (${JSON.stringify(decision.matched.filter)}`
+                            + `${decision.matched.by ? ` ← ${decision.matched.by}` : ''})`
+                        : decision.dependency ? ` (${decision.dependency.kind} `
+                            + `${decision.dependency.target} ${decision.dependency.cause})`
+                        : '')
+                }
                 // Project reference-marker keys (`$author`, `$hero`, …)
                 // into their normalized form (`author`, `hero`) before
                 // the entity crosses into the renderer — applies whether

package/src/explain.js

@@ -9,7 +9,8 @@
 //
 // Follows --verify's shape: report and exit, no build phases run.
 import { inputHashOf, inputPartsOf, diffInputParts, lookupKeys, checksum as fileChecksum } from './utils.js'
-import { findEntity } from './catalog.js'
+import { filterKey } from './track.js'
+import { findEntity, findEntities } from './catalog.js'
 import runtime from './runtime.js'
 
 const shortHash = (h) => (h ? String(h).slice(0, 8) : null)
@@ -28,6 +29,62 @@ async function resolve(reference) {
     return like ?? null
 }
 
+// How many entities each recorded query filter matches RIGHT NOW.
+//
+// A query edge is a predicate — "any entity matching this" — and the
+// recording layer deliberately stores the filter without its results, so
+// that an entity appearing later still invalidates the render. That is the
+// property that makes aggregate pages work and it must not change.
+//
+// Explaining is a different situation: read-only, reporting instead of
+// building, with the catalog already open. So the count is computed here,
+// at read time, where it costs nothing and answers "is this reference
+// dangling?" for the edge kind that carries most of a real site's
+// references.
+//
+// Run through findEntities rather than hand-rolled SQL: stored filters
+// include $regex and anything else sift accepts, and one code path is the
+// only way the count means the same thing the render meant.
+//
+// A null filter is the sentinel for a predicate that could not be
+// serialized (a function filter, or findEntities() with no argument). It
+// invalidates on any mutation by design, and there is nothing to evaluate,
+// so it reports null rather than a misleading zero.
+async function countQueryMatches(snapshots) {
+    const counts = new Map()
+    for (const snap of snapshots) {
+        for (const entry of snap.refClosure ?? []) {
+            if (entry.kind !== 'query') continue
+            const key = filterKey(entry.filter ?? null)
+            if (counts.has(key)) continue
+            if (entry.filter == null) {
+                counts.set(key, null)
+                continue
+            }
+            try {
+                // recordQuery no-ops outside a render's queryContext, so
+                // explaining a page cannot record edges onto it.
+                const matches = await findEntities(entry.filter)
+                counts.set(key, { count: matches.length, sample: matches[0]?.id ?? null })
+            } catch {
+                // A stored filter the catalog can no longer evaluate is worth
+                // saying so about, not worth failing the whole report for.
+                counts.set(key, 'unevaluable')
+            }
+        }
+    }
+    return counts
+}
+
+// Project a counted filter into the report's shape.
+function queryCount(counts, filter) {
+    const hit = counts.get(filterKey(filter ?? null))
+    if (hit === null) return { matched: null }
+    if (hit === 'unevaluable') return { matched: null, unevaluable: true }
+    if (!hit) return { matched: null }
+    return { matched: hit.count, ...(hit.count === 1 && hit.sample ? { sample: hit.sample } : {}) }
+}
+
 // The verdict line names what moved when it can. That line is the one
 // people read, so "the input hash differs" there is the answer stopping one
 // step short of useful.
@@ -60,6 +117,7 @@ export async function explain(reference) {
     const snapshots = runtime.manifest?.snapshotsFor(entity.id) ?? []
     const currentHash = inputHashOf(entity)
     const currentParts = inputPartsOf(entity)
+    const queryMatches = await countQueryMatches(snapshots)
 
     // The catalog is as of the LAST BUILD. If the file has been edited since,
     // nothing here knows it yet — the hashes would all agree and the verdict
@@ -141,7 +199,17 @@ export async function explain(reference) {
             parent: snap.parent ?? null,
             refClosure: (snap.refClosure ?? []).map(entry =>
                 entry.kind === 'query'
-                    ? { kind: 'query', filter: entry.filter }
+                    ? {
+                        kind: 'query',
+                        filter: entry.filter,
+                        // An explicit integer, because a MISSING key is
+                        // ambiguous to a consumer: an audit script reading
+                        // absence as "unresolved" reports every query edge
+                        // as dangling. `null` means the filter could not be
+                        // evaluated (unserializable predicate); a number is
+                        // a number.
+                        ...queryCount(queryMatches, entry.filter),
+                    }
                     : {
                         kind: entry.kind,
                         target: entry.target,
@@ -222,7 +290,16 @@ export function formatExplain(report) {
         row('refClosure', `${closure.length} edge${closure.length === 1 ? '' : 's'}`)
         for (const e of closure) {
             if (e.kind === 'query') {
-                out.push(`  query      ${JSON.stringify(e.filter)}`)
+                // The count is the point: a query matching nothing is the
+                // dangling-reference case for the edge kind that carries most
+                // of a real site's references, and it printed identically to
+                // one that matched.
+                const state = e.unevaluable ? '  [FILTER CANNOT BE EVALUATED]'
+                    : e.matched === null ? '  (untracked predicate — any mutation invalidates)'
+                    : e.matched === 0 ? '  [MATCHES NOTHING]'
+                    : e.matched === 1 ? `  → ${e.sample ?? '1 entity'}`
+                    : `  → ${e.matched} entities`
+                out.push(`  query      ${JSON.stringify(e.filter)}${state}`)
                 continue
             }
             // Show the name asked for and the entity it bound to, since

package/src/manifest.js

@@ -389,11 +389,36 @@ export function createManifest(db) {
             const sourceLang = entity?.meta?.lang ?? null
             for (const entry of snapshot.refClosure) {
                 if (entry.kind === 'query') {
-                    if (!entry.filter) return { skip: false, reason: 'query-matched' }
+                    // A null filter is the sentinel for a predicate that
+                    // could not be serialized — findEntities() with no
+                    // argument, or a function filter. It re-renders on ANY
+                    // mutation, which is a materially different situation
+                    // from "this specific query matched" and is reported as
+                    // such: an aggregate page that always re-renders is a
+                    // filter worth narrowing, which is what catalog.js
+                    // already warns about at record time.
+                    if (!entry.filter) {
+                        return {
+                            skip: false,
+                            reason: 'query-matched',
+                            matched: { filter: null, by: null },
+                        }
+                    }
                     if (!mutatedEntities?.size) continue
                     const matcher = sift(entry.filter)
                     for (const mutated of mutatedEntities.values()) {
-                        if (matcher(mutated)) return { skip: false, reason: 'query-matched' }
+                        // Both the filter and the entity that tripped it are
+                        // live here. Reporting only "a query matched" throws
+                        // away the two facts the reader needs — on a page with
+                        // eighteen query edges, which one fired and what set
+                        // it off is the entire question.
+                        if (matcher(mutated)) {
+                            return {
+                                skip: false,
+                                reason: 'query-matched',
+                                matched: { filter: entry.filter, by: mutated.id ?? null },
+                            }
+                        }
                     }
                     continue
                 }
@@ -433,11 +458,26 @@ export function createManifest(db) {
                             continue
                         }
                     }
-                    if (!entry.hash) return { skip: false, reason: 'ref-changed' }
+                    // Which dependency, and why. Three distinct causes reach
+                    // the same reason, and the difference is what the reader
+                    // is after: a layout whose bytes moved, a $-ref whose
+                    // target was deleted, and an edge recorded with no hash
+                    // are three different things to go and look at.
+                    //
+                    //   unhashed  nothing resolved when the edge was recorded,
+                    //             so any mutation of that name re-renders
+                    //   deleted   the target is gone from the catalog
+                    //   changed   the target's own input hash moved
+                    const dependency = (cause) => ({
+                        skip: false,
+                        reason: 'ref-changed',
+                        dependency: { kind: entry.kind, target: entry.target, key, cause },
+                    })
+                    if (!entry.hash) return dependency('unhashed')
                     const currentHash = currentHashes?.get(key)
                     if (currentHash === undefined) continue
-                    if (currentHash === null) return { skip: false, reason: 'ref-changed' }
-                    if (currentHash !== entry.hash) return { skip: false, reason: 'ref-changed' }
+                    if (currentHash === null) return dependency('deleted')
+                    if (currentHash !== entry.hash) return dependency('changed')
                 }
             }
             return { skip: true, reason: 'unchanged' }

package/src/report.js

@@ -28,15 +28,25 @@ export function reportGated(count = 1) {
     store().gated += count
 }
 
-export function reportRendered(entity, reason, changed) {
+// `decision` is the skipDecision that led here. Its detail travels with the
+// reason, because a reason on its own answers a question nobody asked: WHICH
+// input moved, WHICH query matched and what tripped it, WHICH dependency
+// changed and how. All of it is in hand where the decision is made, and a
+// consumer that has to go to the database for it is one the report failed.
+//
+// Detail keys are per-reason rather than one generic field — `changed`,
+// `matched`, `dependency` each mean something specific, and a single
+// polymorphic key would push the type switch onto every consumer.
+export function reportRendered(entity, reason, decision = {}) {
     if (!runtime.options?.json) return
     store().rendered.push({
         id: entity?.id,
         destination: entity?.destination ?? null,
         reason,
-        // Which input moved, when the reason is inputs-changed. Omitted
-        // rather than empty so a consumer can test for its presence.
-        ...(changed?.length ? { changed } : {}),
+        // Omitted rather than empty, so presence is meaningful.
+        ...(decision.changed?.length ? { changed: decision.changed } : {}),
+        ...(decision.matched ? { matched: decision.matched } : {}),
+        ...(decision.dependency ? { dependency: decision.dependency } : {}),
     })
 }