mikser-io 11.2.2 → 11.3.0

package/docs/diagnostics.md

@@ -771,6 +771,29 @@ layout it went through. Entities with no layout — a copied asset, a
 `files()` passthrough — stay out, or the answer would be "everything,
 always", which is the same non-answer as "nothing" with the sign flipped.
 
+**`recordNoOutput(id)` — the one write on this facade a plugin should
+make.** Everything above answers questions; this one tells the manifest
+something only a dispatcher can know: that it looked at an entity and found
+nothing to render it with. The manifest then drops the snapshots that entity
+still holds, and unlinks the files they claim, in its own finalize
+transaction — so the cleanup gets the same guard as every other: a
+destination another entity still claims is never taken away.
+
+It exists because the two states that matter are indistinguishable from
+inside the manifest. An entity whose `layout:` was removed and an asset whose
+preset threw both arrive with no successful render and no destination on their
+catalog row, and pruning on that resemblance deletes the good derivative a
+failed preset exists to keep. So the call is only correct for "I dispatched
+this and matched nothing" — never for a failure, and never for a declaration
+that could not be resolved, since mikser keeps the last good output through an
+error rather than taking the page down. `mikser-io-layouts` makes it; any
+other dispatcher can.
+
+Without it, an entity that stops producing output leaves its page on disk with
+a snapshot still vouching for it — and because the file still matches the hash
+its own render recorded, `--audit-output` reads OK while the site serves
+something the source no longer asks for.
+
 ### `runtime.provenance`
 
 Where a value was **written** — source file, field path, line and column.

package/docs/getting-started.md

@@ -187,8 +187,9 @@ const documents = useCollection(runtime, 'documents')
 //    that's what mikser's normal lifecycle does (anything persisted is
 //    queryable via findEntities). For on-demand renders where the
 //    bytes are the work product and you don't want the metadata row to
-//    accumulate, pass { catalog: false } to opt out. The rendered
-//    output file is kept on disk either way.
+//    accumulate, pass { catalog: false } and the catalog ends the call
+//    exactly as it began it. The rendered output file is kept on disk
+//    either way.
 const { output, entity } = await render({
   id: '/documents/en/report.md',
   type: 'document',
@@ -217,9 +218,24 @@ parallelism within the cycle is governed by `runtime.options.threads`.
 `render` options:
 
 - `timeout` — per-call timeout in ms (default 30_000).
-- `catalog` (default `true`) — keep the entity in the catalog after the
-  render. Pass `catalog: false` to prune the row, useful for on-demand
-  renders where the metadata would just accumulate.
+- `catalog` (default `true`) — let this render's changes reach the catalog.
+  Pass `catalog: false` and the catalog ends the call exactly as it began
+  it: the row that was there goes back as it was, and a row this render
+  created is removed. Useful for on-demand renders where the metadata would
+  just accumulate, and for previews, where the entity you hand in is usually
+  an altered copy (a surface forcing its own layout onto it) that has no
+  business becoming the entity's production state.
+
+  The render still travels the lifecycle, so the copy does reach the row
+  while the render needs it — the pipeline reads it back to resolve the
+  layout. It just does not outlive the call, whether the render succeeds or
+  throws. Nothing is journalled to achieve this, so the file a
+  `save: true, catalog: false` render produced stays on disk.
+
+  `save: false` implies it: a render that keeps nothing on disk has no
+  business moving a row either. Neither combination records a manifest
+  snapshot — a snapshot is a claim about a catalog entity's output, and
+  after a neutral render there is no such entity to speak for.
 - `save` (default `true`) — write the rendered output to disk at
   `<outputFolder>/<entity.destination>`. Pass `save: false` to skip the
   final disk write; the bytes still come back in `output.result` for you

package/docs/plugins.md

@@ -702,7 +702,7 @@ Writes content to a file in a collection folder. The file change is picked up by
 ```
 
 `options` is optional. Strict opt-outs via the literal `false`:
-- `options.catalog: false` — prune the catalog row after render
+- `options.catalog: false` — leave the catalog exactly as the call found it (the row goes back as it was; a row this render created is removed). The output file is kept.
 - `options.save: false` — skip the final disk write (bytes still in the response)
 
 **`GET /<endpoint>/entities/subscribe`**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mikser-io",
-  "version": "11.2.2",
+  "version": "11.3.0",
   "files": [
     "app.js",
     "index.js",

package/src/catalog.js

@@ -191,6 +191,58 @@ function entityToRow(entity) {
     }
 }
 
+// Catalog-neutral renders in flight: entity id -> the row as it stood before
+// the render, or null where there was no row.
+//
+// Module-local rather than a field on runtime, deliberately. useRenderer takes
+// its runtime by injection while this module imports the singleton, so a field
+// there is written on one object and read on another the moment anyone injects
+// anything else — which every unit test does. One owner, and the writer has to
+// come through markNeutralRender.
+const neutralRenders = new Map()
+
+// Ids whose neutral render has finished — successfully or not — and whose
+// entry is therefore the next finalize drain's to clear.
+//
+// Without this the map is a leak of exactly the kind this feature exists to
+// prevent: gpoint-api renders with a fresh uuid per request, so an entry that
+// is only ever removed when a matching write shows up accumulates one row per
+// render for the life of the process. A render that THROWS journals nothing at
+// all, so that is not a hypothetical.
+const settledNeutralRenders = new Set()
+
+/**
+ * Declare that a render of `id` must leave the catalog as it found it, and
+ * hand over the row to put back — or null where there was none.
+ *
+ * Called by useRenderer BEFORE dispatch: the row is written during the cycle
+ * because the pipeline reads it back to resolve the layout, so what makes the
+ * render neutral is the restore afterwards, not a suppressed write.
+ *
+ * @param {string} id
+ * @param {object|null} prior
+ */
+export function markNeutralRender(id, prior) {
+    if (!id) return
+    neutralRenders.set(id, prior ?? null)
+    settledNeutralRenders.delete(id)
+}
+
+/**
+ * Report that the render of `id` has finished, however it finished. The next
+ * finalize drain restores anything it still owes and drops the entry.
+ *
+ * Separate from markNeutralRender because the two happen either side of the
+ * cycle: a render registered while a cycle is running is served by the NEXT
+ * one, so entries cannot simply be cleared at each cycle's end — an unsettled
+ * one is still waiting for its turn.
+ *
+ * @param {string} id
+ */
+export function settleNeutralRender(id) {
+    if (id && neutralRenders.has(id)) settledNeutralRenders.add(id)
+}
+
 // Apply per-cycle journal mutations inside one transaction (per the
 // migration plan's per-phase transaction granularity). Maintains
 // `mikser_refs` alongside `mikser_entities` so refs and entities
@@ -198,14 +250,26 @@ function entityToRow(entity) {
 //
 // better-sqlite3's transaction wrapper is sync-only, so we drain the
 // journal first and then sync-apply in one call.
-async function applyJournalMutations() {
+// `phase` is which drain this is — 'persist' (before render) or 'finalize'
+// (after it). It decides what happens to a catalog-neutral render's entry:
+// the persist drain applies it, because the pipeline reads the row back to
+// resolve the layout and the render fails outright without it; the finalize
+// drain puts the prior row back instead, because by then the render is done
+// and the altered copy has no business outliving it.
+async function applyJournalMutations(phase) {
     const logger = useLogger()
     const refsIndex = useRefsIndex()
     const mutations = []
     for await (const { operation, entity } of useJournal('Catalog')) {
         mutations.push({ operation, entity })
     }
-    if (!mutations.length) return
+    // Ids whose neutral render finished in this cycle. Collected during the
+    // pass and settled inside the same transaction, so no reader ever sees
+    // the altered row.
+    const toRestore = phase === 'finalize' ? new Map() : null
+    // A settled entry is cleared even in a cycle that journalled nothing —
+    // which is precisely the shape a failed render leaves behind.
+    if (!mutations.length && !settledNeutralRenders.size) return
     db.transaction(() => {
         // Two passes, deliberately. indexEntity resolves each $-ref
         // against mikser_entities to record what it bound to, so it has
@@ -217,6 +281,14 @@ async function applyJournalMutations() {
             switch (operation) {
                 case OPERATION.CREATE:
                 case OPERATION.UPDATE:
+                    if (toRestore && neutralRenders.has(entity.id)) {
+                        // A catalog-neutral render's own write, arriving after
+                        // the render. Restoring rather than applying is the
+                        // whole of `catalog: false`.
+                        logger.trace('Database restore after neutral render: %s', entity.id)
+                        toRestore.set(entity.id, neutralRenders.get(entity.id))
+                        break
+                    }
                     logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
                     stmtUpsert.run(entityToRow(entity))
                     toIndex.push(entity)
@@ -241,9 +313,57 @@ async function applyJournalMutations() {
         // delete-then-insert per source internally, so this stays
         // idempotent across UPDATE.
         for (const entity of toIndex) refsIndex?.indexEntity(entity)
+
+        // Last, so it wins over anything else this batch wrote for the id.
+        for (const [id, prior] of toRestore ?? []) {
+            restoreRow(id, prior, refsIndex)
+            neutralRenders.delete(id)
+            settledNeutralRenders.delete(id)
+        }
+        // Whatever is left over from a render that has finished without
+        // journalling anything for us to undo. Nothing to restore — useRenderer
+        // already put the row back the moment the render settled — so this only
+        // releases the entry.
+        if (toRestore) {
+            for (const id of settledNeutralRenders) neutralRenders.delete(id)
+            settledNeutralRenders.clear()
+        }
     })
 }
 
+// Put a row back exactly as it stood, or remove it where there was none.
+//
+// Sync, and assumes it is already inside a transaction — both callers are.
+// Writes nothing to the journal on purpose: a journal entry would dispatch
+// another render, and a journaled DELETE drags the manifest's file cleanup
+// with it, which would unlink the output a `save: true, catalog: false`
+// render was asked to produce.
+function restoreRow(id, prior, refsIndex = useRefsIndex()) {
+    if (prior) {
+        stmtUpsert.run(entityToRow(prior))
+        refsIndex?.indexEntity(prior)
+    } else {
+        stmtDelete.run(id)
+    }
+    cacheEvict(id)
+}
+
+/**
+ * Restore an entity to the row it had before a catalog-neutral render, or
+ * remove it if it had none. Called by useRenderer as soon as the render
+ * resolves, so a caller awaiting `render()` sees the catalog as it was; the
+ * finalize drain then does the same again for the write that lands after.
+ *
+ * @param {string} id
+ * @param {object|null} prior - the row as it stood, or null if there was none
+ */
+export function restoreEntity(id, prior) {
+    if (!id || !db?.isOpen) return
+    const refsIndex = useRefsIndex()
+    // mikser's db.transaction RUNS the function; it does not return one.
+    db.transaction(() => restoreRow(id, prior, refsIndex))
+}
+
 onLoaded(async () => {
     db = useDatabase()
     if (!db) {
@@ -329,7 +449,7 @@ onLoaded(async () => {
 })
 
 onPersist(async () => {
-    await applyJournalMutations()
+    await applyJournalMutations('persist')
 })
 
 onFinalize(async () => {
@@ -353,7 +473,7 @@ onFinalize(async () => {
     // render reads the catalog), and journal consumers are named and
     // independent, so a second pass only ever picks up what the first could
     // not have seen.
-    await applyJournalMutations()
+    await applyJournalMutations('finalize')
 
     // Checkpoint the WAL so the main file size stays representative
     // and external tools (mikser --audit-output on a separate run, debug

package/src/manifest/cycle.js

@@ -57,20 +57,51 @@ onFinalize(async () => {
         deletedIds.push(entity.id)
     }
 
+    // Every destination claimed by a render task this cycle — whether it
+    // succeeded, was skipped as current, or failed.
+    //
+    // This is deliberately WIDER than renderedEntries, and the width is the
+    // safety. The staleness test below asks "is this row's destination one its
+    // entity still claims", and all three outcomes answer yes:
+    //
+    //   skipped   the manifest said the output was already current, which is
+    //             an assertion that it BELONGS. One entity can match several
+    //             layouts, and a dependency change invalidates them
+    //             independently — so a cycle where layout A re-renders and
+    //             layout B is skipped is ordinary, and reading B's silence as
+    //             "no longer produced" would unlink a live page.
+    //   failed    a failed render writes no snapshot on purpose, so the last
+    //             good bytes survive. Treating the gap as abandonment
+    //             destroys exactly what that rule protects.
+    const claimedByRenderTasks = new Map()
     for await (const { output, entity, deps } of useJournal('Output', [OPERATION.RENDER])) {
-        // A render that wrote NOTHING has no snapshot to record. A snapshot is
-        // the manifest's claim that a file exists at a destination — it is what
-        // --audit-output verifies and what invalidation compares against — so a
-        // `save: false` render (a preview: bytes back to the caller, nothing on
-        // disk) recording one makes the manifest assert a file nobody wrote.
+        if (entity?.id && entity.destination) {
+            if (!claimedByRenderTasks.has(entity.id)) claimedByRenderTasks.set(entity.id, new Set())
+            claimedByRenderTasks.get(entity.id).add(entity.destination)
+        }
+        // A catalog-neutral render leaves no snapshot. A snapshot is the
+        // manifest's claim about a CATALOG ENTITY's output — it is what
+        // --audit-output verifies and what invalidation compares against — and
+        // both halves of `neutral` end the call with no such entity to speak
+        // for.
         //
-        // Observed on a live site: an MCP app rendered on demand left
+        // For `save: false` the claim is false outright: nothing was written.
+        // Observed on a live site — an MCP app rendered on demand left
         // /internal/customer-registration.html claimed and absent, and the
-        // audit went red for a page that was never supposed to exist. The
-        // entity is simply left unrecorded, which is also the truthful state
-        // for invalidation — nothing was produced, so the next real build has
-        // nothing to reuse.
-        if (entity?.options?.save === false) continue
+        // audit went red for a page that was never supposed to exist.
+        //
+        // For `save: true, catalog: false` the file is real but the entity is
+        // deliberately not mikser's to track, and recording it is what makes
+        // the two accounts disagree: the manifest holds a snapshot whose
+        // entity no longer exists. It also breaks the caller directly —
+        // invalidation compares against the snapshot, finds the destination
+        // already current, and SKIPS the render, so the next call for the same
+        // id gets no output at all.
+        //
+        // Leaving it unrecorded is the truthful state for invalidation either
+        // way: mikser is not tracking this output, so there is nothing to
+        // reuse and nothing to verify.
+        if (entity?.options?.neutral) continue
         if (output?.success && !output.skipped) {
             renderedEntries.push({ entity, deps, metaReads: output.metaReads,
                                    consumedReads: output.consumedReads })
@@ -119,13 +150,25 @@ onFinalize(async () => {
 
     // 2b. Pagination shrunk — drop children whose destination wasn't
     // re-emitted this cycle.
-    const childrenToDelete = []   // [{id, destination, reason}]
+    const snapshotsToDelete = []   // [{id, destination}] — rows to drop by PK
+    // Ids that DEPART with their snapshot: a pagination child that no longer
+    // exists. Feeds `goingAway`, which asks "is this destination still claimed
+    // by something that survives" — an id-level question, and the right one
+    // for an entity that is gone.
+    const departingIds = new Set()
+    // Rows dropped for a destination their own entity no longer produces, as
+    // `id \t destination`. Deliberately NOT an id: an entity whose
+    // destination merely MOVED is still very much here, and calling it
+    // departed would let another entity's cleanup unlink a destination this
+    // one still claims.
+    const droppedClaims = new Set()
     for (const [parentId, keep] of newDestinationsByParent) {
         const rows = m._stmtSelectByParent.all(parentId)
         for (const row of rows) {
             if (keep.has(row.destination)) continue
             filesToUnlink.push({ destination: row.destination, reason: 'Pagination shrunk' })
-            childrenToDelete.push({ id: row.id, destination: row.destination })
+            snapshotsToDelete.push({ id: row.id, destination: row.destination })
+            departingIds.add(row.id)
         }
     }
 
@@ -134,14 +177,88 @@ onFinalize(async () => {
         const rows = m._stmtSelectByParent.all(parentId)
         for (const row of rows) {
             filesToUnlink.push({ destination: row.destination, reason: 'Pagination dropped' })
-            childrenToDelete.push({ id: row.id, destination: row.destination })
+            snapshotsToDelete.push({ id: row.id, destination: row.destination })
+            departingIds.add(row.id)
+        }
+    }
+
+    // 2c'. Destination moved — a SURVIVING entity no longer produces a
+    // destination it used to.
+    //
+    // Change a layout's `destination:` template, or flip `cleanUrls`, and the
+    // entity keeps its id and renders to a new path. Nothing before this
+    // noticed: the snapshot table is keyed by (id, destination), so recording
+    // the new render INSERTS a second row rather than replacing the first, and
+    // the old row goes on claiming the old file forever.
+    //
+    // Worse than it sounds, because the state is self-consistent and therefore
+    // silent: the stale file is still on disk, still matches the hash its own
+    // render recorded, so --audit-output reports OK — 0 missing, 0 orphaned —
+    // while the site serves a page the project no longer produces. Verified on
+    // a two-build fixture: one entity, two snapshots, two files, green audit.
+    //
+    // The comparison is against the SET of destinations the id claimed this
+    // cycle, never a single value. One entity can legitimately claim several —
+    // one per matched layout — and taking the last one to render as "the"
+    // destination would delete the others' output on every build. See
+    // claimedByRenderTasks above for why a skipped or failed task counts as a
+    // claim.
+    //
+    // Two ways in, and the difference is who observed what.
+    //
+    // An entity with render tasks is compared against what they claimed —
+    // that is the moved-destination case above.
+    //
+    // An entity that produces nothing at all has no tasks to compare against,
+    // so it cannot be recognised from here: an asset whose preset threw looks
+    // exactly the same, and pruning on that resemblance deletes the good
+    // derivative a failed preset exists to keep. So the dispatcher says it
+    // outright, through manifest.recordNoOutput, and an empty claim set means
+    // every destination the entity used to hold is stale. Reported by
+    // mikser-io-layouts when an entity it dispatched matched no layout; the
+    // call is general, and any other dispatcher can make it.
+    const claims = new Map(claimedByRenderTasks)
+    for (const id of m._noOutputIds) {
+        // A task contradicts the report — something did render, so believe
+        // what happened over what was predicted.
+        if (!claims.has(id)) claims.set(id, new Set())
+    }
+    m._noOutputIds.clear()
+
+    for (const [id, claimed] of claims) {
+        if (deleted.has(id)) continue
+        // An entity that renders nothing takes its paginated children with
+        // it. There are no tasks left to own them, and 2b/2c reach children
+        // only when a render told them this cycle's page count — so for this
+        // entity they never fire, and the children would be left claiming
+        // files nothing produces. Where tasks DID run, children stay with
+        // 2b/2c, which know about shrinking in a way this pass does not.
+        const rows = claimed.size
+            ? m._stmtDestinationsById.all(id)
+            : m._stmtSelectByIdOrParent.all(id, id)
+        for (const row of rows) {
+            if (claimed.has(row.destination)) continue
+            if (claimed.size && row.parent) continue
+            filesToUnlink.push({
+                destination: row.destination,
+                reason: claimed.size ? 'Destination moved' : 'Entity renders nothing',
+            })
+            snapshotsToDelete.push({ id: row.id, destination: row.destination })
+            if (row.parent) {
+                // A child is genuinely gone, not moved — its id departs.
+                departingIds.add(row.id)
+            } else {
+                droppedClaims.add(`${row.id}\t${row.destination}`)
+            }
         }
     }
 
-    // Everything whose snapshot this pass removes: deleted entities, their
-    // paginated children, and children dropped by a pagination shrink.
+    // Ids whose snapshots this pass removes ENTIRELY: deleted entities and
+    // their paginated children. Not the moved claims — those belong to
+    // entities that are still here, and `droppedClaims` carries them at
+    // (id, destination) granularity instead.
     const goingAway = new Set(deleted)
-    for (const { id } of childrenToDelete) goingAway.add(id)
+    for (const id of departingIds) goingAway.add(id)
     for (const parentId of deleted) {
         for (const row of m._stmtSelectByParent.all(parentId)) goingAway.add(row.id)
     }
@@ -194,7 +311,9 @@ onFinalize(async () => {
         // snapshot as a claimant would keep every shrunk page on disk
         // forever.
         const stillClaimed = m._stmtSelectByDestination.all(destination)
-            .filter(row => row.id !== undefined && !goingAway.has(row.id))
+            .filter(row => row.id !== undefined
+                && !goingAway.has(row.id)
+                && !droppedClaims.has(`${row.id}\t${destination}`))
         if (stillClaimed.length) {
             // Keep the file — deleting a live page's output is worse than any
             // staleness — but do NOT let the state go quiet. The bytes on
@@ -251,7 +370,7 @@ onFinalize(async () => {
             m._stmtDeleteByIdOrParent.run(id, id)
         }
         // 3b. Pagination children cleanup.
-        for (const { id, destination } of childrenToDelete) {
+        for (const { id, destination } of snapshotsToDelete) {
             m._stmtDeleteByPK.run(id, destination)
         }
         // 3c. Record successful renders.

package/src/manifest/index.js

@@ -84,6 +84,7 @@ export function createManifest(db) {
         stmtSelectByIdOrParent,
         stmtDeleteByIdOrParent,
         stmtSelectByParent,
+        stmtDestinationsById,
         stmtSelectAll,
         stmtCount,
         stmtEntityInputHashes,
@@ -92,6 +93,10 @@ export function createManifest(db) {
         stmtSnapshotsWithLayout,
         edgeCandidates,
     } = prepareStatements(db)
+    // Ids a dispatcher reported as producing nothing this cycle. Owned by
+    // this instance rather than the module, so a test that builds its own
+    // manifest gets its own set. Drained and cleared by onFinalize.
+    const noOutputIds = new Set()
     const manifest = {
         // Look up a previously-recorded entry by entity (or by an
         // object with `{id, destination}`). Returns the snapshot, or
@@ -548,6 +553,29 @@ export function createManifest(db) {
         },
 
         // Drop all snapshots owned by entity id (direct outputs and any
+        // Report that a dispatcher looked at this entity and found nothing
+        // to render it with — no layout matched, no preset claimed it.
+        //
+        // The distinction this exists to draw is between "produced no output
+        // this cycle" and "no longer produces output at all". They are
+        // indistinguishable from inside the manifest: an asset whose preset
+        // threw and an entity whose `layout:` was removed both arrive with no
+        // successful render and no destination on their catalog row, and
+        // guessing from that signal deletes the good derivative a failed
+        // preset is explicitly meant to keep. Only the dispatcher knows which
+        // it is, so only the dispatcher can say.
+        //
+        // Recorded, not applied. The removal joins onFinalize's single
+        // transaction and its unlink path, so it gets the same
+        // still-claimed-by-a-survivor guard as every other cleanup — a
+        // destination two entities write is not this one's to take away.
+        //
+        // Per cycle: onFinalize clears the set after draining it. Saying it
+        // twice for one entity is harmless.
+        recordNoOutput(id) {
+            if (id) noOutputIds.add(id)
+        },
+
         // paginated children whose `parent` is set to this id). Returns
         // the destinations that were removed so callers can unlink the
         // corresponding files when desired. Two queries (SELECT for
@@ -794,6 +822,8 @@ export function createManifest(db) {
         _stmtSelectByIdOrParent: stmtSelectByIdOrParent,
         _stmtDeleteByIdOrParent: stmtDeleteByIdOrParent,
         _stmtSelectByParent:     stmtSelectByParent,
+        _noOutputIds:            noOutputIds,
+        _stmtDestinationsById:   stmtDestinationsById,
         _stmtSelectByDestination: stmtSelectByDestination,
         _stmtDeleteByDestination: stmtDeleteByDestination,
         _stmtDeleteByPK:         stmtDeleteByPK,

package/src/manifest/statements.js

@@ -82,6 +82,13 @@ export function prepareStatements(db) {
     const stmtSelectByParent = db.prepare(`
         SELECT id, destination FROM mikser_snapshots WHERE parent = ?
     `)
+    // Every destination one entity claims. An entity legitimately claims
+    // several — one per matched layout, and one per page when paginated — so
+    // "the destinations this id produced" is a SET, never a single value, and
+    // that is the whole reason this query exists separately from stmtLookup.
+    const stmtDestinationsById = db.prepare(`
+        SELECT id, destination, parent FROM mikser_snapshots WHERE id = ?
+    `)
     const stmtSelectAll = db.prepare(`
         SELECT id, destination, inputHash, inputParts, outputHash, refClosure, metaReads, consumedReads, renderedAt, parent
         FROM mikser_snapshots
@@ -179,6 +186,7 @@ export function prepareStatements(db) {
         stmtSelectByIdOrParent,
         stmtDeleteByIdOrParent,
         stmtSelectByParent,
+        stmtDestinationsById,
         stmtSelectAll,
         stmtCount,
         stmtEntityInputHashes,

package/src/plugins/api.js

@@ -1036,7 +1036,7 @@ export function api(options = {}) {
                     // straight to render(entity, options). Defaults match
                     // mikser's lifecycle (save and keep the catalog row);
                     // strict opt-outs via the literal `false`:
-                    //   options.catalog: false → prune the catalog row
+                    //   options.catalog: false → leave the catalog as found
                     //   options.save:    false → skip the final disk write
                     //                            (bytes still in the response)
                     const { options = {}, ...entityShape } = req.body

package/src/render.js

@@ -506,13 +506,16 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
      * Two control flags mirror mikser's default-keep-everything behavior;
      * both opt-out via strict `=== false`:
      *
-     * - `catalog: true` (default) — keep the entity in the catalog after
-     *   the render. Pass `catalog: false` to prune the catalog row;
-     *   useful for on-demand renders where the metadata row would just
-     *   accumulate. Requires `save: false` — the prune goes through the
-     *   journal, and a DELETE takes the manifest's file cleanup with it,
-     *   so it is only safe for a render that wrote nothing. With
-     *   `save: true` the row is kept and a warning is logged.
+     * - `catalog: true` (default) — pass `catalog: false` and the catalog
+     *   ends the call exactly as it began it: the row that was there is put
+     *   back as it was, and a row this render created is removed. The render
+     *   still travels the lifecycle, so the caller's copy — usually altered,
+     *   since an on-demand surface forces its own layout onto it — does reach
+     *   the row while the render needs it; it just does not outlive the call.
+     *   Combines with either `save`, which is the point: gpoint-api renders
+     *   its emails with `save: true, catalog: false` — the file is wanted, the
+     *   row is not. `save: false` implies it, since a render that keeps
+     *   nothing on disk has no business moving a row either.
      * - `save: true` (default) — write the rendered output to disk at
      *   `<outputFolder>/<entity.destination>`. Pass `save: false` to
      *   skip the final disk write; the bytes still come back via
@@ -539,31 +542,40 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
      * @param {object} entity                - any entity-shaped object
      * @param {object} [opts]
      * @param {number}  [opts.timeout]       - override the default timeout
-     * @param {boolean} [opts.catalog=true]  - keep the catalog row after render
+     * @param {boolean} [opts.catalog=true]  - let this render's changes reach
+     *                                          the catalog; false restores it
      * @param {boolean} [opts.save=true]     - write the rendered output to disk
      * @returns {Promise<{output, entity}>}
      */
     async function render(entity, { timeout = defaultTimeout, catalog = true, save = true } = {}) {
-        // Was this row already in the catalog BEFORE the render? `catalog:
-        // false` means "do not leave a row behind", and that is only the
-        // render's to decide for a row the render created. Asked here, before
-        // the render puts one there.
+        // What the catalog held before this render touched it.
         //
-        // Without it, `catalog: false` deleted whatever it was handed: a
-        // preview of an EXISTING entity pruned the real row, so the entity
-        // vanished from the site while its file sat on disk — no change set,
-        // no cycle, and the removal logged only at debug. It cost a day to
-        // find, in a form that rendered once and then answered "Entity not
-        // found".
-        // Imported HERE, not at the top: catalog.js reaches back into this
-        // module, and a static import makes that cycle load-bearing at
+        // Read BEFORE dispatch, because the render itself overwrites it: the
+        // entity has to travel the lifecycle to render at all, and the
+        // pipeline reads the row back to resolve the layout, so suppressing
+        // that write outright fails the render with "requested layout X but
+        // produced no output". The row is written, used, and then put back.
+        //
+        // Imported here rather than at the top: catalog.js reaches back into
+        // this module, and a static import makes that cycle load-bearing at
         // module-evaluation time — it surfaced as "Cannot access 'schemas'
         // before initialization" in three unrelated test files.
-        const preexisting = catalog === false && entity?.id
-            ? Boolean(await (await import('./catalog.js')).findById(entity.id))
-            : false
+        const neutral = save === false || catalog === false
+        // Restoring is about a ROW, so it needs an id; the flag itself does
+        // not, and the manifest reads the flag to decide whether to record a
+        // snapshot. Keeping the two apart means an id-less entity — which has
+        // no row to put back — still gets the rest of neutrality.
+        let priorRow = null
+        if (neutral && entity?.id) {
+            const { findById, markNeutralRender } = await import('./catalog.js')
+            priorRow = findById(entity.id)
+            // Handed over before dispatch, so it is in place however early the
+            // cycle starts. The catalog's finalize drain is where the row goes
+            // back for good — see applyJournalMutations.
+            markNeutralRender(entity.id, priorRow)
+        }
 
-        const result = await new Promise((resolve, reject) => {
+        const dispatched = new Promise((resolve, reject) => {
             const correlationId = randomUUID()
             // Engine-set fields live under entity.options. The caller's
             // render(entity, { save: false }) becomes
@@ -580,6 +592,10 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
                     ...entity.options,
                     correlationId,
                     ...(save === false ? { save: false } : {}),
+                    // Marks every journal entry this render produces, however
+                    // many phases later they arrive, as belonging to a
+                    // catalog-neutral call.
+                    ...(neutral ? { neutral: true } : {}),
                 },
             }
             pending.push({
@@ -593,32 +609,28 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
             if (!cycleRunning) setImmediate(runBatch)
         })
 
-        if (catalog === false) {
-            // Prune the row through the journal, so the DELETE lands in
-            // sqlite at onPersist alongside the CREATE that put it there.
-            // Strict equality — null / "false" / 0 keep the row.
-            //
-            // Only when `save` is also false. A DELETE carries the
-            // manifest's file cleanup with it, which unlinks the render's
-            // output; that is correct for an entity that produced no file
-            // and wrong for one that did. `catalog: false, save: true`
-            // therefore keeps its row, and says so rather than dropping
-            // the output on the floor.
-            if (preexisting) {
-                // Someone else's row. It was here before this render and is
-                // not this render's to remove.
-                useLogger()?.debug(
-                    'render: catalog:false ignored for %s — the entity was already in the catalog',
-                    result.entity.id,
-                )
-            } else if (save === false) {
-                await runtime.delete(result.entity)
-            } else {
-                useLogger()?.warn(
-                    'render: catalog:false ignored for %s — it needs save:false, ' +
-                    'because pruning the row also unlinks the rendered output',
-                    result.entity.id,
-                )
+        let result
+        try {
+            result = await dispatched
+        } finally {
+            // In a finally, because a render that THROWS has still altered the
+            // row on its way through — an unrenderable entity leaves the
+            // catalog holding the caller's copy of it, which is the same
+            // damage as a successful render leaving its layout behind.
+            if (neutral && entity?.id) {
+                // Put the row back for whoever is awaiting us. The finalize
+                // drain does it again, and has to: it runs after the render and
+                // would otherwise re-apply the altered copy over this one. Both
+                // restore the same prior row, so their order does not matter.
+                //
+                // Deliberately not journaled. A journal entry would dispatch
+                // another render, and a journaled DELETE drags the manifest's
+                // file cleanup along with it — which would unlink the very file
+                // a `save: true, catalog: false` render was asked to produce.
+                // That coupling is why this used to refuse `save: true`.
+                const { restoreEntity, settleNeutralRender } = await import('./catalog.js')
+                restoreEntity(entity.id, priorRow)
+                settleNeutralRender(entity.id)
             }
         }