mikser-io 11.2.2 → 11.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,281 @@
1
+ # ADR-0008 — MCP-UI: spec-compatible shell + `tools/call` delivery + optional webhook
2
+
3
+ **Status:** Superseded
4
+ **Date:** 2026
5
+ **Supersedes:** —
6
+ **Superseded by:** MCP Apps (SEP-1865), implemented in `mikser-io-mcp-app`
7
+
8
+ ## Why this is kept
9
+
10
+ Archived here because the index row above it linked to this file in the
11
+ `mikser-io-mcp` repository, and the surface moved on to `mikser-io-mcp-app`,
12
+ taking the file with it — leaving core's own decision log with a dead link and
13
+ a hole at 0008.
14
+
15
+ Most of it was right, which is the interesting part. What survived into
16
+ `mikser-io-mcp-app` is the shape decided here: a static shell resource
17
+ declared as the tool's `_meta.ui.resourceUri`, per-call fragments delivered as
18
+ data rather than markup, clicks arriving as an ordinary `tools/call` against
19
+ an app-only tool, and the declared action list as the authorization boundary.
20
+
21
+ Two things did not.
22
+
23
+ **The webhook handler.** `mcpUi.handler.url` forwarded the action to an
24
+ external endpoint, and the ADR argued this respects ADR-0001 because the
25
+ application logic stays outside mikser. The logic did; the URL did not. It
26
+ was an MCP concern sitting in the engine, read by nothing but the MCP
27
+ surface — a field core carried on behalf of one plugin. It is gone, and what
28
+ it reached for is a layout sidecar instead: `<layout>.js` beside the template,
29
+ exporting `call` / `read` / `list`, running in the project rather than at the
30
+ end of an HTTP hop.
31
+
32
+ **The package.** A host for applications wants a route whose tool list *is*
33
+ the surface, and `mikser_app_action` is app-callable by specification — it has
34
+ no business appearing on the agent's endpoint. So the surface left
35
+ `mikser-io-mcp` for `mikser-io-mcp-app`, and `mcpUi` became `mcpApp`.
36
+
37
+ One earlier draft of this ADR, on the since-deleted `feat/mcp-ui-handler`
38
+ branch of this repository, is worth one line because the error is easy to
39
+ repeat: it had the iframe POST its action to mikser, reasoning the fetch is
40
+ same-origin because the iframe is served from mikser's origin. Under MCP Apps
41
+ the iframe's CSP is `default-src 'none'; connect-src 'none'` — there is no
42
+ network inside it at all, same-origin included. `postMessage` to the host is
43
+ the only way out. The version below had already corrected that to
44
+ `tools/call`.
45
+
46
+ ## Context
47
+
48
+ ADR-0007 (`MCP-UI: layouts as the agent's UI surface`) introduced the idea that mikser layouts can serve as the agent's UI surface inside an MCP host. The `mikser_preview_ui` tool renders an `mcpUi`-decorated layout against an entity and returns HTML for the host to surface as a sandboxed iframe. The user interacts with the iframe — clicks Approve, fills a form, picks a status — and the click needs to get back to mikser as a structured tool result.
49
+
50
+ Three facts about the surrounding ecosystem shape this decision:
51
+
52
+ 1. **The MCP Apps spec ([2026-01-26](https://github.com/modelcontextprotocol/ext-apps/blob/main/specification/2026-01-26/apps.mdx)) defines how iframe-to-server delivery works.** The iframe runs as an MCP client speaking JSON-RPC over `window.parent.postMessage` to the host. The host's "AppBridge" translates `tools/call` frames into real MCP tool calls on the existing transport. Every conformant host (Goose, ChatGPT/Apps SDK, mcp-ui's reference host, basic-host, VS Code Insiders) implements this same pattern.
53
+
54
+ 2. **The spec mandates the iframe is cross-origin from the host with a restrictive default CSP.** "The Host and the Sandbox MUST have different origins." Default CSP when `ui.csp` is omitted: `default-src 'none'; connect-src 'none'`. So a `fetch` to mikser from inside the iframe — even on the same machine — is blocked by the browser. The only outbound channel is `postMessage`.
55
+
56
+ 3. **The spec mandates that UI is delivered as a static *resource*, not inline tool-result HTML.** Tools that want UI declare `_meta.ui.resourceUri` on their tool definition, pointing at a `ui://` resource with `mimeType: 'text/html;profile=mcp-app'`. Spec-conformant hosts fetch that resource once via `resources/read`, load it in a sandboxed iframe, then push the tool's per-call result to the iframe via `ui/notifications/tool-result`. Empirically, hosts like basic-host display tools that lack `_meta.ui.resourceUri` as plain text rather than rendering an iframe — even if `content[0]` declares `mimeType: 'text/html'`.
57
+
58
+ These three facts together rule out two tempting alternatives:
59
+
60
+ - **An in-process HTTP endpoint that the iframe POSTs to**, with a server-minted "callId" as the capability URL. Prototyped on `feat/mcp-ui-handler`, ruled out — incompatible with `connect-src 'none'`, invisible to the host's consent/audit surface, inverts every other MCP App implementation in the ecosystem.
61
+ - **Returning the rendered HTML inline as `content[0].text` and trusting hosts to render it.** Initially shipped in 8.0.x; surfaced as the symptom "basic-host received the tool result but shows it as text, not an iframe." Spec-conformant hosts need the resource URI on the tool definition; they ignore content with `mimeType: 'text/html'` when no resource URI is set.
62
+
63
+ Separately, productized workflows want to intercept the action server-side without forcing the agent to learn application-specific schemas — a CRM, support system, or admin tool wants to receive the click, do its work, and return a domain-specific result. Pure relay (mikser returns the click data, agent decides what `approve` means) is right for AI-native workflows; webhook delegation (mikser forwards the click to an external URL) is right for productized ones. We want both, without turning mikser into a workflow engine.
64
+
65
+ ## Decision
66
+
67
+ ### Part A — Rendering: static shell resource + structured tool result
68
+
69
+ **A1. Mikser ships a single static UI resource: `ui://mikser/preview-ui-shell`.**
70
+
71
+ ```js
72
+ mcp.registerResource(
73
+ 'mikser-preview-ui-shell',
74
+ 'ui://mikser/preview-ui-shell',
75
+ { mimeType: 'text/html;profile=mcp-app', ... },
76
+ async (uri) => ({
77
+ contents: [{ uri: uri.href, mimeType: 'text/html;profile=mcp-app', text: SHELL_HTML }],
78
+ }),
79
+ )
80
+ ```
81
+
82
+ The shell is ~120 lines of self-contained HTML+JS implementing the MCP Apps protocol:
83
+
84
+ - Sends `ui/initialize` to the host on load (2-second timeout, fails open if no host replies).
85
+ - Listens for `ui/notifications/tool-result`. Reads `structuredContent.html` from the result and injects it into a `#mikser-ui-root` div. Re-executes any `<script>` tags the layout brought (innerHTML doesn't execute embedded scripts by default).
86
+ - Exposes `window.sendAction(action, payload?)` — the API layouts use to deliver clicks back as `tools/call` against `mikser_ui_action`. The shell tracks `entityId` and `layoutId` from the tool result, so layouts don't have to.
87
+ - Renders an in-iframe debug panel showing every protocol event with timestamps — so authors can see exactly where the round-trip fails on hosts that don't bridge.
88
+
89
+ The shell is static. It does not change between tool calls, between entities, or between mikser versions. It is a fixed bundle of protocol plumbing.
90
+
91
+ **A2. `mikser_preview_ui` declares `_meta.ui.resourceUri` pointing at the shell.**
92
+
93
+ ```js
94
+ mcp.registerTool('mikser_preview_ui', {
95
+ description: '...',
96
+ inputSchema: { entityId, mode },
97
+ _meta: {
98
+ ui: { resourceUri: 'ui://mikser/preview-ui-shell' },
99
+ },
100
+ }, handler)
101
+ ```
102
+
103
+ This is the spec-mandated signal that this tool renders UI. Hosts that implement MCP Apps fetch the resource once via `resources/read` and use it as the iframe template for every call to this tool. Hosts that don't implement MCP Apps display `content[0].text` as plain text (the fallback path; better than nothing).
104
+
105
+ **A3. The tool result returns content + `structuredContent`.**
106
+
107
+ ```jsonc
108
+ {
109
+ "content": [
110
+ { "type": "text", "text": "<rendered fragment>", "mimeType": "text/html" }
111
+ ],
112
+ "structuredContent": {
113
+ "entityId": "/blog/launch.md",
114
+ "layoutId": "/layouts/mcp-ui/post-approval.hbs",
115
+ "mode": "approval",
116
+ "html": "<rendered fragment>",
117
+ "mcpUi": { "actions": [...], "sandbox": [...], "actionTool": "mikser_ui_action" }
118
+ },
119
+ "_meta": { "mcpUi": { ... } }
120
+ }
121
+ ```
122
+
123
+ `content[0].text` is the fallback for non-UI hosts. `structuredContent` is what the host passes to the iframe via `ui/notifications/tool-result`. The shell reads `structuredContent.html` and injects it; it reads `entityId` and `layoutId` to scope subsequent `sendAction` calls.
124
+
125
+ **A4. Layouts are body fragments, not full HTML documents.**
126
+
127
+ The shell wraps `<!DOCTYPE>` / `<html>` / `<head>` / `<body>` around the injected content. Layouts produce a fragment containing inline `<style>`, body content, and optional inline `<script>`. The script can call `sendAction(action, payload?)` directly — it's exposed on `window` by the shell. No protocol code, no `ui/initialize`, no RPC helper, no postMessage shape.
128
+
129
+ Compare a layout authored before this ADR (~85 lines) with one authored after (~40 lines): everything below `<style>` stays; everything above `<style>` and the entire 50-line protocol `<script>` block disappears.
130
+
131
+ ### Part B — Action delivery: `tools/call` against `mikser_ui_action` (visibility=['app'])
132
+
133
+ **B1. Mikser registers a separate, app-callable tool: `mikser_ui_action`.**
134
+
135
+ ```js
136
+ mcp.registerTool('mikser_ui_action', {
137
+ description: '...',
138
+ inputSchema: { entityId, layoutId, action, payload },
139
+ _meta: { ui: { visibility: ['app'] } },
140
+ }, handler)
141
+ ```
142
+
143
+ `visibility: ['app']` makes the tool invisible to the agent — it never appears in the agent's tool surface — but callable from inside iframes via the host's AppBridge. The agent sees the result as a normal tool turn in its conversation.
144
+
145
+ **B2. The action allow-list is the auth boundary.**
146
+
147
+ `mikser_ui_action`'s handler looks up the layout by `layoutId`, reads its `mcpUi.actions` list, and rejects any action not in that list with an error result. This is the single place where layout-declared "you can do these things" meets iframe-supplied "I want to do this thing." Unknown actions never reach pure relay, never reach `handler.url`.
148
+
149
+ **B3. There is no callId, no signature, no token on this channel.**
150
+
151
+ The host's MCP transport is already authenticated. The visibility flag already gates which tools the iframe can invoke. The action allow-list already scopes what the iframe can ask for. Layered defenses; no per-call crypto.
152
+
153
+ **B4. There is no in-process HTTP endpoint for action delivery.**
154
+
155
+ `/api/mcp-ui/action/...` does not exist. Adding one would create a second delivery path with a different auth model, double the test surface, and offer no benefit on conformant hosts (CSP blocks the fetch) or non-conformant ones (the iframe wouldn't render anyway). One channel, one auth model.
156
+
157
+ ### Part C — Optional webhook handler
158
+
159
+ **C1. Layouts may declare a `handler` block in their `mcpUi` frontmatter.**
160
+
161
+ ```yaml
162
+ ---
163
+ match: "@/articles/*"
164
+ mcpUi:
165
+ mode: approval
166
+ actions: [approve, reject, request-changes]
167
+ sandbox: [allow-scripts]
168
+ handler:
169
+ url: https://app.example.com/mikser-actions
170
+ secret: ${MIKSER_HANDLER_SECRET} # optional, enables HMAC signing
171
+ timeout: 5000 # optional, ms; default 5000
172
+ ---
173
+ ```
174
+
175
+ When `handler.url` is set, `mikser_ui_action`'s handler forwards the action data to that URL instead of returning the pure-relay payload. The handler's JSON response body becomes the tool result.
176
+
177
+ **C2. The forwarded request is a standard webhook.**
178
+
179
+ ```http
180
+ POST https://app.example.com/mikser-actions
181
+ Content-Type: application/json
182
+ X-Mikser-Signature: sha256=...
183
+ X-Mikser-Request-Id: <opaque uuid for idempotency>
184
+ X-Mikser-Layout-Id: /layouts/mcp-ui/post-approval.hbs
185
+ X-Mikser-Mode: approval
186
+
187
+ {
188
+ "entityId": "/documents/blog/launch.md",
189
+ "layoutId": "/layouts/mcp-ui/post-approval.hbs",
190
+ "action": "approve",
191
+ "payload": {},
192
+ "mode": "approval",
193
+ "timestamp": "2026-06-07T15:00:00Z"
194
+ }
195
+ ```
196
+
197
+ `X-Mikser-Signature` is HMAC-SHA256 of the request body using `handler.secret`. Receivers verify before processing. If `secret` is unset, no signature is sent — fine for development; not recommended in production.
198
+
199
+ **C3. The handler's JSON response is the tool result.**
200
+
201
+ ```json
202
+ {
203
+ "ok": true,
204
+ "summary": "Committed to main; deployment queued (build #4821).",
205
+ "url": "https://app.example.com/deploys/4821",
206
+ "_meta": { "buildId": 4821 }
207
+ }
208
+ ```
209
+
210
+ Mikser passes this through to the agent unchanged. The agent composes its next message from it. No domain knowledge in mikser.
211
+
212
+ **C4. Handler failures fall back to pure relay.**
213
+
214
+ Network error, timeout, non-2xx response, non-JSON response: mikser logs a warning and resolves the tool call with the default `{ entityId, action, payload }` plus a `handlerError` field carrying the failure reason. The user's click is never lost.
215
+
216
+ ```json
217
+ {
218
+ "entityId": "/documents/blog/launch.md",
219
+ "action": "approve",
220
+ "payload": {},
221
+ "handlerError": "Handler timeout (5000ms) — https://app.example.com/mikser-actions"
222
+ }
223
+ ```
224
+
225
+ **C5. The handler block is the entire extension surface.**
226
+
227
+ Mikser does not learn about action semantics — what `approve` means, what `request-changes` should do, where the result goes. Adding that knowledge to mikser would violate ADR-0001 (`Mikser is the content layer of the application, not the app`). The webhook contract IS the extension point; if you want behaviour, write a service.
228
+
229
+ ## Consequences
230
+
231
+ **Easier:**
232
+
233
+ - Spec compliance is unconditional. Spec-conformant hosts (Goose, ChatGPT, mcp-ui's reference host, basic-host, VS Code Insiders) render mikser layouts as iframes correctly. No host-specific shims.
234
+ - Layouts shrink dramatically — no more per-layout 50-line protocol boilerplate. Author content + inline styles + click handlers; the shell handles everything else. Comparing the blog example's `post-approval.hbs` v8.0.x vs v8.1.0: ~85 lines → ~40 lines, roughly half. The protocol bug surface collapses to one place.
235
+ - The action allow-list + visibility flag are the entire auth model. No bespoke crypto primitive to debug.
236
+ - One static resource served forever; one tool result shape that's the same on every call. Cacheable, predictable, testable.
237
+
238
+ **Harder:**
239
+
240
+ - Layouts authored before 8.1.0 — full HTML documents with embedded `ui/initialize` and RPC plumbing — need rewriting. The new shape is mechanically simpler but it is *not* drop-in compatible with 8.0.x layouts. Mikser's blog example layouts ship pre-rewritten as canonical references.
241
+ - Hosts that haven't implemented MCP Apps (`Claude Desktop` per the open `upstream-host` bug [anthropics/claude-ai-mcp#165](https://github.com/anthropics/claude-ai-mcp/issues/165)) show the iframe as raw text. There is no fallback — and that's deliberate. Both alternatives explored above introduce more problems than they solve.
242
+ - The shell's debug panel is on by default. Production use will want to either gate it behind a query param or remove it entirely. Tracked as a follow-up.
243
+
244
+ ## Examples
245
+
246
+ See `documentation/mcp.md` "Layout frontmatter and MCP-UI" — every worked example was rewritten when this ADR landed. Each one collapses to roughly 30-50 lines including inline styles.
247
+
248
+ ## Alternatives considered
249
+
250
+ **Direct HTTP from iframe to mikser (capability URL pattern).** Prototyped on `feat/mcp-ui-handler`. Random callId, single-use, action allow-list, loopback bind — textbook capability URL, genuinely secure against CSRF/replay/unknown-actions. Ruled out because:
251
+
252
+ 1. Default MCP Apps CSP is `connect-src 'none'` — the browser blocks the fetch on conformant hosts.
253
+ 2. The iframe is cross-origin from the host by spec — there is no "same-origin" with mikser to leverage.
254
+ 3. The action bypasses the host's audit/consent surface.
255
+ 4. It inverts the direction of every other MCP App implementation, making mikser layouts non-portable.
256
+
257
+ **Returning HTML inline in `content[0].text` with `mimeType: 'text/html'`.** Shipped initially in 8.0.x. Surfaced as the symptom "basic-host received the tool result, displayed it as plain text, never rendered an iframe." Conformant hosts read `_meta.ui.resourceUri` off the tool definition (static) to decide what iframe template to load — they do not infer it from response content. The fix is the resource pattern in Part A.
258
+
259
+ **Per-layout tools (`mikser_preview_approval`, `mikser_preview_edit`, etc.) each with their own static `_meta.ui.resourceUri`.** Cleaner mapping but tool count grows with layout count, and the agent has to learn which tool handles which mode. The shell-as-template approach keeps the agent's tool surface stable (one `mikser_preview_ui` for all UIs) while still satisfying the spec's static-URI requirement.
260
+
261
+ **Dual-channel (postMessage primary, HTTP fallback).** Rejected as "no legacy" — two delivery paths means two auth models, two test surfaces, two failure modes to debug, and no host where both are needed. Pick one channel; pick the one the spec specifies.
262
+
263
+ **Built-in action vocabulary (mikser knows what `approve` / `reject` mean).** Rejected. This is the application layer; mikser is the content layer (ADR-0001). The agent owns semantics by default; `handler.url` is the escape hatch for productized cases.
264
+
265
+ **Server-side handler scripts (layouts ship a `handler:` callback in JS).** Rejected. Same reasoning — mikser would become a workflow engine. External webhooks compose; in-mikser handlers would couple action behaviour to mikser deployment.
266
+
267
+ **Per-action handlers (`handler` is a map: `{ approve: url1, reject: url2 }`).** Rejected as YAGNI. Single URL with the action in the payload is enough — the receiver multiplexes.
268
+
269
+ ## Watch for drift
270
+
271
+ These are the failure modes this decision is protecting against. If you see them, push back.
272
+
273
+ - **The shell grows application-specific knowledge.** Someone proposes "the shell could pre-format the date before injecting" or "the shell could add a global retry banner." Refuse. The shell is protocol + injection + sendAction relay. Application concerns live in layouts.
274
+ - **A second action-delivery channel sneaks in.** Someone notices `mikser_ui_action` doesn't work on a non-conformant host and proposes adding an HTTP endpoint as a "fallback." Don't. The 8.1.0 design is one channel by deliberate choice.
275
+ - **`_meta.ui.resourceUri` drift on the tool definition.** Someone removes it or changes it to point at something dynamic. Spec-conformant hosts will stop rendering iframes — they only read this field at tool-list time, not per-call.
276
+ - **Layouts start re-implementing the protocol.** Someone writes a layout with its own `ui/initialize` handshake "for control." That layout will fight the shell. The shell exposes `sendAction`; that's the entire contract a layout uses.
277
+ - **The visibility flag drifts on `mikser_ui_action`.** If `_meta.ui.visibility` is removed or changed to `['model', 'app']`, the tool leaks into the agent's surface — it'll appear as an action the agent can take "out of context" without any iframe ever rendering. Strict `['app']`.
278
+ - **Action vocabulary creeps into core.** Someone proposes a built-in `approve` semantic so simple layouts don't need a handler. Refuse; that's application-layer logic.
279
+ - **Per-action handler URLs.** Someone proposes `handler: { approve: '...', reject: '...' }`. Refuse. One URL, the action goes in the payload, the receiver routes.
280
+ - **Retry / backoff on the handler.** Mikser doesn't retry. If the handler is down, the user re-clicks.
281
+ - **Persistent pending state.** There is no pending state — `mikser_preview_ui` returns synchronously and the shell handles per-render state in the iframe. Don't add a "pending action" table.
@@ -33,7 +33,7 @@ Decisions don't expire. They get **superseded** when we learn enough to change t
33
33
  | [0005](./0005-engine-infrastructure-runs-before-plugin-hooks.md) | Engine infrastructure (journal, catalog) is ready before any plugin hook runs; `runtime.update` is upsert; `useSource` codifies the folder-of-files pattern | Accepted |
34
34
  | [0006](./0006-when-to-add-to-core.md) | The five-test check for adding capability to the engine vs. shipping it as a plugin | Accepted |
35
35
  | [0007](./0007-references-declaration-and-expansion.md) | Entity references: `$`-prefixed declaration (canonical on disk, normalized for render/SDK) and `expand` resolution (inline, GET-cacheable, engine-level `runtime.refs` drives invalidation + live-expand) | Accepted |
36
- | [0008](https://github.com/almero-digital-marketing/mikser-io-mcp/blob/main/documentation/decisions/0008-mcp-ui-action-delivery.md) | MCP-UI rendering + action delivery: static shell resource at `ui://mikser/preview-ui-shell` (declared as `_meta.ui.resourceUri` on `mikser_preview_ui`) hosts the [MCP Apps](https://github.com/modelcontextprotocol/ext-apps) protocol; per-call rendered fragments delivered via `structuredContent` and `ui/notifications/tool-result`; the shell exposes `window.sendAction()` so layouts are content-only HTML. Clicks ride `tools/call` against `mikser_ui_action` (`_meta.ui.visibility = ['app']`). Action allow-list is the auth boundary. Optional `mcpUi.handler.url` forwards the action to an external webhook (HMAC-signed if `handler.secret` set); handler failures fall back to pure relay. No HTTP delivery surface. Lives in [`mikser-io-mcp`](https://github.com/almero-digital-marketing/mikser-io-mcp). | Accepted |
36
+ | [0008](./0008-mcp-ui-action-delivery.md) | MCP-UI rendering + action delivery: static shell resource declared as `_meta.ui.resourceUri`, per-call fragments via `structuredContent`, clicks riding `tools/call` against an app-only tool, declared action list as the authorization boundary. The shape survived; the `mcpUi.handler.url` webhook did not an MCP concern in the engine, replaced by a layout sidecar. The surface moved to [`mikser-io-mcp-app`](https://github.com/almero-digital-marketing/mikser-io-mcp-app) and `mcpUi` became `mcpApp`. Archived here because the file left with it. | Superseded |
37
37
  | [0009](./0009-database-engine-substrate.md) | Sqlite is the engine's persistence substrate. Single `runtime/mikser.sqlite` file holds catalog/refs/manifest/journal as `mikser_entities` / `mikser_refs` / `mikser_snapshots` / `mikser_journal` tables. Plugins register schemas via `registerSchema(name, sql)` + `useDatabase()`. Sift→SQL translator with indexed pushdown, LRU cache for `findById`, worker-side read-only sqlite for sync template helpers, chunked journal walks + `iterateEntities` streaming, `useJournal` auto-persist (mutate the yielded entity; no explicit `updateEntry` needed), `--resume` after interrupted cycles. Replaces `Map<id, entity>` + NDJSON across every engine subsystem. | Accepted |
38
38
  | [0010](./0010-plugin-bundles-and-inline-options.md) | Plugin bundles + factory-call form + inline options. Plugins are imported by name and called as factories; `plugins: []` carries factory returns, never strings. Lifecycle plugins are `(options) => (core) => void`; renderers return `{name, options, load?, render?}`; postprocessors return `{name, options, output?, setup?, postprocess, teardown?}`. Per-plugin config moved off `runtime.config.<plugin>` — it arrives as the factory arg and is passed as `config` to `load`/`render`/`setup`/`postprocess`. | Accepted |
39
39
  | [0011](./0011-served-entities-expose-deployed-urls.md) | File and resource entities expose deployed URLs. References to served files (image/video/PDF) are `$`-keyed **served paths** (`/img/X.jpg`, `/media/clip.mp4` — the path content authors, = the entity's `meta.url`), resolving through a new `refFilter` `{ 'meta.url': … }` clause backed by an indexed `meta_url` column (schema 9.0.1 → 9.0.2). No collection-prefixed ids leak into content. (Id-refs were tried and rejected — gpoint references content by served path and its `/media/**` `resources()` library means the entity only exists because content references `/media/…`.) The `files`/`resources` plugins stamp `meta.url`, the `assets` plugin stamps `meta.presets` — so expanding a ref yields the served entity's URL set instead of a string to reconstruct. Base-relative in the live catalog (host-agnostic; consumer holds `base`), absolute in static renders (baked from `runtime.options.url`, so logic-less consumers — email, RSS, foreign apps — read a whole URL); `lookupUrl` render helper resolves a ref to `meta.url` or a named preset. SDK collapses `assetUrl(source, preset, {ext})` into one `url(ref)` join + a dev-mode SPA-fallback detector. *("Served entity" — file/resource/preset — avoids colliding with the `assets` plugin's own "asset reference" term.)* | Accepted (proven against gpoint) |
@@ -771,6 +771,29 @@ layout it went through. Entities with no layout — a copied asset, a
771
771
  `files()` passthrough — stay out, or the answer would be "everything,
772
772
  always", which is the same non-answer as "nothing" with the sign flipped.
773
773
 
774
+ **`recordNoOutput(id)` — the one write on this facade a plugin should
775
+ make.** Everything above answers questions; this one tells the manifest
776
+ something only a dispatcher can know: that it looked at an entity and found
777
+ nothing to render it with. The manifest then drops the snapshots that entity
778
+ still holds, and unlinks the files they claim, in its own finalize
779
+ transaction — so the cleanup gets the same guard as every other: a
780
+ destination another entity still claims is never taken away.
781
+
782
+ It exists because the two states that matter are indistinguishable from
783
+ inside the manifest. An entity whose `layout:` was removed and an asset whose
784
+ preset threw both arrive with no successful render and no destination on their
785
+ catalog row, and pruning on that resemblance deletes the good derivative a
786
+ failed preset exists to keep. So the call is only correct for "I dispatched
787
+ this and matched nothing" — never for a failure, and never for a declaration
788
+ that could not be resolved, since mikser keeps the last good output through an
789
+ error rather than taking the page down. `mikser-io-layouts` makes it; any
790
+ other dispatcher can.
791
+
792
+ Without it, an entity that stops producing output leaves its page on disk with
793
+ a snapshot still vouching for it — and because the file still matches the hash
794
+ its own render recorded, `--audit-output` reads OK while the site serves
795
+ something the source no longer asks for.
796
+
774
797
  ### `runtime.provenance`
775
798
 
776
799
  Where a value was **written** — source file, field path, line and column.
@@ -187,8 +187,9 @@ const documents = useCollection(runtime, 'documents')
187
187
  // that's what mikser's normal lifecycle does (anything persisted is
188
188
  // queryable via findEntities). For on-demand renders where the
189
189
  // bytes are the work product and you don't want the metadata row to
190
- // accumulate, pass { catalog: false } to opt out. The rendered
191
- // output file is kept on disk either way.
190
+ // accumulate, pass { catalog: false } and the catalog ends the call
191
+ // exactly as it began it. The rendered output file is kept on disk
192
+ // either way.
192
193
  const { output, entity } = await render({
193
194
  id: '/documents/en/report.md',
194
195
  type: 'document',
@@ -217,9 +218,24 @@ parallelism within the cycle is governed by `runtime.options.threads`.
217
218
  `render` options:
218
219
 
219
220
  - `timeout` — per-call timeout in ms (default 30_000).
220
- - `catalog` (default `true`) — keep the entity in the catalog after the
221
- render. Pass `catalog: false` to prune the row, useful for on-demand
222
- renders where the metadata would just accumulate.
221
+ - `catalog` (default `true`) — let this render's changes reach the catalog.
222
+ Pass `catalog: false` and the catalog ends the call exactly as it began
223
+ it: the row that was there goes back as it was, and a row this render
224
+ created is removed. Useful for on-demand renders where the metadata would
225
+ just accumulate, and for previews, where the entity you hand in is usually
226
+ an altered copy (a surface forcing its own layout onto it) that has no
227
+ business becoming the entity's production state.
228
+
229
+ The render still travels the lifecycle, so the copy does reach the row
230
+ while the render needs it — the pipeline reads it back to resolve the
231
+ layout. It just does not outlive the call, whether the render succeeds or
232
+ throws. Nothing is journalled to achieve this, so the file a
233
+ `save: true, catalog: false` render produced stays on disk.
234
+
235
+ `save: false` implies it: a render that keeps nothing on disk has no
236
+ business moving a row either. Neither combination records a manifest
237
+ snapshot — a snapshot is a claim about a catalog entity's output, and
238
+ after a neutral render there is no such entity to speak for.
223
239
  - `save` (default `true`) — write the rendered output to disk at
224
240
  `<outputFolder>/<entity.destination>`. Pass `save: false` to skip the
225
241
  final disk write; the bytes still come back in `output.result` for you
package/docs/plugins.md CHANGED
@@ -702,7 +702,7 @@ Writes content to a file in a collection folder. The file change is picked up by
702
702
  ```
703
703
 
704
704
  `options` is optional. Strict opt-outs via the literal `false`:
705
- - `options.catalog: false` — prune the catalog row after render
705
+ - `options.catalog: false` — leave the catalog exactly as the call found it (the row goes back as it was; a row this render created is removed). The output file is kept.
706
706
  - `options.save: false` — skip the final disk write (bytes still in the response)
707
707
 
708
708
  **`GET /<endpoint>/entities/subscribe`**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mikser-io",
3
- "version": "11.2.2",
3
+ "version": "11.3.1",
4
4
  "files": [
5
5
  "app.js",
6
6
  "index.js",
package/src/catalog.js CHANGED
@@ -191,6 +191,58 @@ function entityToRow(entity) {
191
191
  }
192
192
  }
193
193
 
194
+ // Catalog-neutral renders in flight: entity id -> the row as it stood before
195
+ // the render, or null where there was no row.
196
+ //
197
+ // Module-local rather than a field on runtime, deliberately. useRenderer takes
198
+ // its runtime by injection while this module imports the singleton, so a field
199
+ // there is written on one object and read on another the moment anyone injects
200
+ // anything else — which every unit test does. One owner, and the writer has to
201
+ // come through markNeutralRender.
202
+ const neutralRenders = new Map()
203
+
204
+ // Ids whose neutral render has finished — successfully or not — and whose
205
+ // entry is therefore the next finalize drain's to clear.
206
+ //
207
+ // Without this the map is a leak of exactly the kind this feature exists to
208
+ // prevent: gpoint-api renders with a fresh uuid per request, so an entry that
209
+ // is only ever removed when a matching write shows up accumulates one row per
210
+ // render for the life of the process. A render that THROWS journals nothing at
211
+ // all, so that is not a hypothetical.
212
+ const settledNeutralRenders = new Set()
213
+
214
+ /**
215
+ * Declare that a render of `id` must leave the catalog as it found it, and
216
+ * hand over the row to put back — or null where there was none.
217
+ *
218
+ * Called by useRenderer BEFORE dispatch: the row is written during the cycle
219
+ * because the pipeline reads it back to resolve the layout, so what makes the
220
+ * render neutral is the restore afterwards, not a suppressed write.
221
+ *
222
+ * @param {string} id
223
+ * @param {object|null} prior
224
+ */
225
+ export function markNeutralRender(id, prior) {
226
+ if (!id) return
227
+ neutralRenders.set(id, prior ?? null)
228
+ settledNeutralRenders.delete(id)
229
+ }
230
+
231
+ /**
232
+ * Report that the render of `id` has finished, however it finished. The next
233
+ * finalize drain restores anything it still owes and drops the entry.
234
+ *
235
+ * Separate from markNeutralRender because the two happen either side of the
236
+ * cycle: a render registered while a cycle is running is served by the NEXT
237
+ * one, so entries cannot simply be cleared at each cycle's end — an unsettled
238
+ * one is still waiting for its turn.
239
+ *
240
+ * @param {string} id
241
+ */
242
+ export function settleNeutralRender(id) {
243
+ if (id && neutralRenders.has(id)) settledNeutralRenders.add(id)
244
+ }
245
+
194
246
  // Apply per-cycle journal mutations inside one transaction (per the
195
247
  // migration plan's per-phase transaction granularity). Maintains
196
248
  // `mikser_refs` alongside `mikser_entities` so refs and entities
@@ -198,14 +250,26 @@ function entityToRow(entity) {
198
250
  //
199
251
  // better-sqlite3's transaction wrapper is sync-only, so we drain the
200
252
  // journal first and then sync-apply in one call.
201
- async function applyJournalMutations() {
253
+ // `phase` is which drain this is — 'persist' (before render) or 'finalize'
254
+ // (after it). It decides what happens to a catalog-neutral render's entry:
255
+ // the persist drain applies it, because the pipeline reads the row back to
256
+ // resolve the layout and the render fails outright without it; the finalize
257
+ // drain puts the prior row back instead, because by then the render is done
258
+ // and the altered copy has no business outliving it.
259
+ async function applyJournalMutations(phase) {
202
260
  const logger = useLogger()
203
261
  const refsIndex = useRefsIndex()
204
262
  const mutations = []
205
263
  for await (const { operation, entity } of useJournal('Catalog')) {
206
264
  mutations.push({ operation, entity })
207
265
  }
208
- if (!mutations.length) return
266
+ // Ids whose neutral render finished in this cycle. Collected during the
267
+ // pass and settled inside the same transaction, so no reader ever sees
268
+ // the altered row.
269
+ const toRestore = phase === 'finalize' ? new Map() : null
270
+ // A settled entry is cleared even in a cycle that journalled nothing —
271
+ // which is precisely the shape a failed render leaves behind.
272
+ if (!mutations.length && !settledNeutralRenders.size) return
209
273
  db.transaction(() => {
210
274
  // Two passes, deliberately. indexEntity resolves each $-ref
211
275
  // against mikser_entities to record what it bound to, so it has
@@ -217,6 +281,14 @@ async function applyJournalMutations() {
217
281
  switch (operation) {
218
282
  case OPERATION.CREATE:
219
283
  case OPERATION.UPDATE:
284
+ if (toRestore && neutralRenders.has(entity.id)) {
285
+ // A catalog-neutral render's own write, arriving after
286
+ // the render. Restoring rather than applying is the
287
+ // whole of `catalog: false`.
288
+ logger.trace('Database restore after neutral render: %s', entity.id)
289
+ toRestore.set(entity.id, neutralRenders.get(entity.id))
290
+ break
291
+ }
220
292
  logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
221
293
  stmtUpsert.run(entityToRow(entity))
222
294
  toIndex.push(entity)
@@ -241,9 +313,57 @@ async function applyJournalMutations() {
241
313
  // delete-then-insert per source internally, so this stays
242
314
  // idempotent across UPDATE.
243
315
  for (const entity of toIndex) refsIndex?.indexEntity(entity)
316
+
317
+ // Last, so it wins over anything else this batch wrote for the id.
318
+ for (const [id, prior] of toRestore ?? []) {
319
+ restoreRow(id, prior, refsIndex)
320
+ neutralRenders.delete(id)
321
+ settledNeutralRenders.delete(id)
322
+ }
323
+ // Whatever is left over from a render that has finished without
324
+ // journalling anything for us to undo. Nothing to restore — useRenderer
325
+ // already put the row back the moment the render settled — so this only
326
+ // releases the entry.
327
+ if (toRestore) {
328
+ for (const id of settledNeutralRenders) neutralRenders.delete(id)
329
+ settledNeutralRenders.clear()
330
+ }
244
331
  })
245
332
  }
246
333
 
334
+ // Put a row back exactly as it stood, or remove it where there was none.
335
+ //
336
+ // Sync, and assumes it is already inside a transaction — both callers are.
337
+ // Writes nothing to the journal on purpose: a journal entry would dispatch
338
+ // another render, and a journaled DELETE drags the manifest's file cleanup
339
+ // with it, which would unlink the output a `save: true, catalog: false`
340
+ // render was asked to produce.
341
+ function restoreRow(id, prior, refsIndex = useRefsIndex()) {
342
+ if (prior) {
343
+ stmtUpsert.run(entityToRow(prior))
344
+ refsIndex?.indexEntity(prior)
345
+ } else {
346
+ stmtDelete.run(id)
347
+ }
348
+ cacheEvict(id)
349
+ }
350
+
351
+ /**
352
+ * Restore an entity to the row it had before a catalog-neutral render, or
353
+ * remove it if it had none. Called by useRenderer as soon as the render
354
+ * resolves, so a caller awaiting `render()` sees the catalog as it was; the
355
+ * finalize drain then does the same again for the write that lands after.
356
+ *
357
+ * @param {string} id
358
+ * @param {object|null} prior - the row as it stood, or null if there was none
359
+ */
360
+ export function restoreEntity(id, prior) {
361
+ if (!id || !db?.isOpen) return
362
+ const refsIndex = useRefsIndex()
363
+ // mikser's db.transaction RUNS the function; it does not return one.
364
+ db.transaction(() => restoreRow(id, prior, refsIndex))
365
+ }
366
+
247
367
  onLoaded(async () => {
248
368
  db = useDatabase()
249
369
  if (!db) {
@@ -329,7 +449,7 @@ onLoaded(async () => {
329
449
  })
330
450
 
331
451
  onPersist(async () => {
332
- await applyJournalMutations()
452
+ await applyJournalMutations('persist')
333
453
  })
334
454
 
335
455
  onFinalize(async () => {
@@ -353,7 +473,7 @@ onFinalize(async () => {
353
473
  // render reads the catalog), and journal consumers are named and
354
474
  // independent, so a second pass only ever picks up what the first could
355
475
  // not have seen.
356
- await applyJournalMutations()
476
+ await applyJournalMutations('finalize')
357
477
 
358
478
  // Checkpoint the WAL so the main file size stays representative
359
479
  // and external tools (mikser --audit-output on a separate run, debug
@@ -57,20 +57,51 @@ onFinalize(async () => {
57
57
  deletedIds.push(entity.id)
58
58
  }
59
59
 
60
+ // Every destination claimed by a render task this cycle — whether it
61
+ // succeeded, was skipped as current, or failed.
62
+ //
63
+ // This is deliberately WIDER than renderedEntries, and the width is the
64
+ // safety. The staleness test below asks "is this row's destination one its
65
+ // entity still claims", and all three outcomes answer yes:
66
+ //
67
+ // skipped the manifest said the output was already current, which is
68
+ // an assertion that it BELONGS. One entity can match several
69
+ // layouts, and a dependency change invalidates them
70
+ // independently — so a cycle where layout A re-renders and
71
+ // layout B is skipped is ordinary, and reading B's silence as
72
+ // "no longer produced" would unlink a live page.
73
+ // failed a failed render writes no snapshot on purpose, so the last
74
+ // good bytes survive. Treating the gap as abandonment
75
+ // destroys exactly what that rule protects.
76
+ const claimedByRenderTasks = new Map()
60
77
  for await (const { output, entity, deps } of useJournal('Output', [OPERATION.RENDER])) {
61
- // A render that wrote NOTHING has no snapshot to record. A snapshot is
62
- // the manifest's claim that a file exists at a destination — it is what
63
- // --audit-output verifies and what invalidation compares against — so a
64
- // `save: false` render (a preview: bytes back to the caller, nothing on
65
- // disk) recording one makes the manifest assert a file nobody wrote.
78
+ if (entity?.id && entity.destination) {
79
+ if (!claimedByRenderTasks.has(entity.id)) claimedByRenderTasks.set(entity.id, new Set())
80
+ claimedByRenderTasks.get(entity.id).add(entity.destination)
81
+ }
82
+ // A catalog-neutral render leaves no snapshot. A snapshot is the
83
+ // manifest's claim about a CATALOG ENTITY's output — it is what
84
+ // --audit-output verifies and what invalidation compares against — and
85
+ // both halves of `neutral` end the call with no such entity to speak
86
+ // for.
66
87
  //
67
- // Observed on a live site: an MCP app rendered on demand left
88
+ // For `save: false` the claim is false outright: nothing was written.
89
+ // Observed on a live site — an MCP app rendered on demand left
68
90
  // /internal/customer-registration.html claimed and absent, and the
69
- // audit went red for a page that was never supposed to exist. The
70
- // entity is simply left unrecorded, which is also the truthful state
71
- // for invalidation nothing was produced, so the next real build has
72
- // nothing to reuse.
73
- if (entity?.options?.save === false) continue
91
+ // audit went red for a page that was never supposed to exist.
92
+ //
93
+ // For `save: true, catalog: false` the file is real but the entity is
94
+ // deliberately not mikser's to track, and recording it is what makes
95
+ // the two accounts disagree: the manifest holds a snapshot whose
96
+ // entity no longer exists. It also breaks the caller directly —
97
+ // invalidation compares against the snapshot, finds the destination
98
+ // already current, and SKIPS the render, so the next call for the same
99
+ // id gets no output at all.
100
+ //
101
+ // Leaving it unrecorded is the truthful state for invalidation either
102
+ // way: mikser is not tracking this output, so there is nothing to
103
+ // reuse and nothing to verify.
104
+ if (entity?.options?.neutral) continue
74
105
  if (output?.success && !output.skipped) {
75
106
  renderedEntries.push({ entity, deps, metaReads: output.metaReads,
76
107
  consumedReads: output.consumedReads })
@@ -119,13 +150,25 @@ onFinalize(async () => {
119
150
 
120
151
  // 2b. Pagination shrunk — drop children whose destination wasn't
121
152
  // re-emitted this cycle.
122
- const childrenToDelete = [] // [{id, destination, reason}]
153
+ const snapshotsToDelete = [] // [{id, destination}] — rows to drop by PK
154
+ // Ids that DEPART with their snapshot: a pagination child that no longer
155
+ // exists. Feeds `goingAway`, which asks "is this destination still claimed
156
+ // by something that survives" — an id-level question, and the right one
157
+ // for an entity that is gone.
158
+ const departingIds = new Set()
159
+ // Rows dropped for a destination their own entity no longer produces, as
160
+ // `id \t destination`. Deliberately NOT an id: an entity whose
161
+ // destination merely MOVED is still very much here, and calling it
162
+ // departed would let another entity's cleanup unlink a destination this
163
+ // one still claims.
164
+ const droppedClaims = new Set()
123
165
  for (const [parentId, keep] of newDestinationsByParent) {
124
166
  const rows = m._stmtSelectByParent.all(parentId)
125
167
  for (const row of rows) {
126
168
  if (keep.has(row.destination)) continue
127
169
  filesToUnlink.push({ destination: row.destination, reason: 'Pagination shrunk' })
128
- childrenToDelete.push({ id: row.id, destination: row.destination })
170
+ snapshotsToDelete.push({ id: row.id, destination: row.destination })
171
+ departingIds.add(row.id)
129
172
  }
130
173
  }
131
174
 
@@ -134,14 +177,88 @@ onFinalize(async () => {
134
177
  const rows = m._stmtSelectByParent.all(parentId)
135
178
  for (const row of rows) {
136
179
  filesToUnlink.push({ destination: row.destination, reason: 'Pagination dropped' })
137
- childrenToDelete.push({ id: row.id, destination: row.destination })
180
+ snapshotsToDelete.push({ id: row.id, destination: row.destination })
181
+ departingIds.add(row.id)
182
+ }
183
+ }
184
+
185
+ // 2c'. Destination moved — a SURVIVING entity no longer produces a
186
+ // destination it used to.
187
+ //
188
+ // Change a layout's `destination:` template, or flip `cleanUrls`, and the
189
+ // entity keeps its id and renders to a new path. Nothing before this
190
+ // noticed: the snapshot table is keyed by (id, destination), so recording
191
+ // the new render INSERTS a second row rather than replacing the first, and
192
+ // the old row goes on claiming the old file forever.
193
+ //
194
+ // Worse than it sounds, because the state is self-consistent and therefore
195
+ // silent: the stale file is still on disk, still matches the hash its own
196
+ // render recorded, so --audit-output reports OK — 0 missing, 0 orphaned —
197
+ // while the site serves a page the project no longer produces. Verified on
198
+ // a two-build fixture: one entity, two snapshots, two files, green audit.
199
+ //
200
+ // The comparison is against the SET of destinations the id claimed this
201
+ // cycle, never a single value. One entity can legitimately claim several —
202
+ // one per matched layout — and taking the last one to render as "the"
203
+ // destination would delete the others' output on every build. See
204
+ // claimedByRenderTasks above for why a skipped or failed task counts as a
205
+ // claim.
206
+ //
207
+ // Two ways in, and the difference is who observed what.
208
+ //
209
+ // An entity with render tasks is compared against what they claimed —
210
+ // that is the moved-destination case above.
211
+ //
212
+ // An entity that produces nothing at all has no tasks to compare against,
213
+ // so it cannot be recognised from here: an asset whose preset threw looks
214
+ // exactly the same, and pruning on that resemblance deletes the good
215
+ // derivative a failed preset exists to keep. So the dispatcher says it
216
+ // outright, through manifest.recordNoOutput, and an empty claim set means
217
+ // every destination the entity used to hold is stale. Reported by
218
+ // mikser-io-layouts when an entity it dispatched matched no layout; the
219
+ // call is general, and any other dispatcher can make it.
220
+ const claims = new Map(claimedByRenderTasks)
221
+ for (const id of m._noOutputIds) {
222
+ // A task contradicts the report — something did render, so believe
223
+ // what happened over what was predicted.
224
+ if (!claims.has(id)) claims.set(id, new Set())
225
+ }
226
+ m._noOutputIds.clear()
227
+
228
+ for (const [id, claimed] of claims) {
229
+ if (deleted.has(id)) continue
230
+ // An entity that renders nothing takes its paginated children with
231
+ // it. There are no tasks left to own them, and 2b/2c reach children
232
+ // only when a render told them this cycle's page count — so for this
233
+ // entity they never fire, and the children would be left claiming
234
+ // files nothing produces. Where tasks DID run, children stay with
235
+ // 2b/2c, which know about shrinking in a way this pass does not.
236
+ const rows = claimed.size
237
+ ? m._stmtDestinationsById.all(id)
238
+ : m._stmtSelectByIdOrParent.all(id, id)
239
+ for (const row of rows) {
240
+ if (claimed.has(row.destination)) continue
241
+ if (claimed.size && row.parent) continue
242
+ filesToUnlink.push({
243
+ destination: row.destination,
244
+ reason: claimed.size ? 'Destination moved' : 'Entity renders nothing',
245
+ })
246
+ snapshotsToDelete.push({ id: row.id, destination: row.destination })
247
+ if (row.parent) {
248
+ // A child is genuinely gone, not moved — its id departs.
249
+ departingIds.add(row.id)
250
+ } else {
251
+ droppedClaims.add(`${row.id}\t${row.destination}`)
252
+ }
138
253
  }
139
254
  }
140
255
 
141
- // Everything whose snapshot this pass removes: deleted entities, their
142
- // paginated children, and children dropped by a pagination shrink.
256
+ // Ids whose snapshots this pass removes ENTIRELY: deleted entities and
257
+ // their paginated children. Not the moved claims those belong to
258
+ // entities that are still here, and `droppedClaims` carries them at
259
+ // (id, destination) granularity instead.
143
260
  const goingAway = new Set(deleted)
144
- for (const { id } of childrenToDelete) goingAway.add(id)
261
+ for (const id of departingIds) goingAway.add(id)
145
262
  for (const parentId of deleted) {
146
263
  for (const row of m._stmtSelectByParent.all(parentId)) goingAway.add(row.id)
147
264
  }
@@ -194,7 +311,9 @@ onFinalize(async () => {
194
311
  // snapshot as a claimant would keep every shrunk page on disk
195
312
  // forever.
196
313
  const stillClaimed = m._stmtSelectByDestination.all(destination)
197
- .filter(row => row.id !== undefined && !goingAway.has(row.id))
314
+ .filter(row => row.id !== undefined
315
+ && !goingAway.has(row.id)
316
+ && !droppedClaims.has(`${row.id}\t${destination}`))
198
317
  if (stillClaimed.length) {
199
318
  // Keep the file — deleting a live page's output is worse than any
200
319
  // staleness — but do NOT let the state go quiet. The bytes on
@@ -251,7 +370,7 @@ onFinalize(async () => {
251
370
  m._stmtDeleteByIdOrParent.run(id, id)
252
371
  }
253
372
  // 3b. Pagination children cleanup.
254
- for (const { id, destination } of childrenToDelete) {
373
+ for (const { id, destination } of snapshotsToDelete) {
255
374
  m._stmtDeleteByPK.run(id, destination)
256
375
  }
257
376
  // 3c. Record successful renders.
@@ -84,6 +84,7 @@ export function createManifest(db) {
84
84
  stmtSelectByIdOrParent,
85
85
  stmtDeleteByIdOrParent,
86
86
  stmtSelectByParent,
87
+ stmtDestinationsById,
87
88
  stmtSelectAll,
88
89
  stmtCount,
89
90
  stmtEntityInputHashes,
@@ -92,6 +93,10 @@ export function createManifest(db) {
92
93
  stmtSnapshotsWithLayout,
93
94
  edgeCandidates,
94
95
  } = prepareStatements(db)
96
+ // Ids a dispatcher reported as producing nothing this cycle. Owned by
97
+ // this instance rather than the module, so a test that builds its own
98
+ // manifest gets its own set. Drained and cleared by onFinalize.
99
+ const noOutputIds = new Set()
95
100
  const manifest = {
96
101
  // Look up a previously-recorded entry by entity (or by an
97
102
  // object with `{id, destination}`). Returns the snapshot, or
@@ -548,6 +553,29 @@ export function createManifest(db) {
548
553
  },
549
554
 
550
555
  // Drop all snapshots owned by entity id (direct outputs and any
556
+ // Report that a dispatcher looked at this entity and found nothing
557
+ // to render it with — no layout matched, no preset claimed it.
558
+ //
559
+ // The distinction this exists to draw is between "produced no output
560
+ // this cycle" and "no longer produces output at all". They are
561
+ // indistinguishable from inside the manifest: an asset whose preset
562
+ // threw and an entity whose `layout:` was removed both arrive with no
563
+ // successful render and no destination on their catalog row, and
564
+ // guessing from that signal deletes the good derivative a failed
565
+ // preset is explicitly meant to keep. Only the dispatcher knows which
566
+ // it is, so only the dispatcher can say.
567
+ //
568
+ // Recorded, not applied. The removal joins onFinalize's single
569
+ // transaction and its unlink path, so it gets the same
570
+ // still-claimed-by-a-survivor guard as every other cleanup — a
571
+ // destination two entities write is not this one's to take away.
572
+ //
573
+ // Per cycle: onFinalize clears the set after draining it. Saying it
574
+ // twice for one entity is harmless.
575
+ recordNoOutput(id) {
576
+ if (id) noOutputIds.add(id)
577
+ },
578
+
551
579
  // paginated children whose `parent` is set to this id). Returns
552
580
  // the destinations that were removed so callers can unlink the
553
581
  // corresponding files when desired. Two queries (SELECT for
@@ -794,6 +822,8 @@ export function createManifest(db) {
794
822
  _stmtSelectByIdOrParent: stmtSelectByIdOrParent,
795
823
  _stmtDeleteByIdOrParent: stmtDeleteByIdOrParent,
796
824
  _stmtSelectByParent: stmtSelectByParent,
825
+ _noOutputIds: noOutputIds,
826
+ _stmtDestinationsById: stmtDestinationsById,
797
827
  _stmtSelectByDestination: stmtSelectByDestination,
798
828
  _stmtDeleteByDestination: stmtDeleteByDestination,
799
829
  _stmtDeleteByPK: stmtDeleteByPK,
@@ -82,6 +82,13 @@ export function prepareStatements(db) {
82
82
  const stmtSelectByParent = db.prepare(`
83
83
  SELECT id, destination FROM mikser_snapshots WHERE parent = ?
84
84
  `)
85
+ // Every destination one entity claims. An entity legitimately claims
86
+ // several — one per matched layout, and one per page when paginated — so
87
+ // "the destinations this id produced" is a SET, never a single value, and
88
+ // that is the whole reason this query exists separately from stmtLookup.
89
+ const stmtDestinationsById = db.prepare(`
90
+ SELECT id, destination, parent FROM mikser_snapshots WHERE id = ?
91
+ `)
85
92
  const stmtSelectAll = db.prepare(`
86
93
  SELECT id, destination, inputHash, inputParts, outputHash, refClosure, metaReads, consumedReads, renderedAt, parent
87
94
  FROM mikser_snapshots
@@ -179,6 +186,7 @@ export function prepareStatements(db) {
179
186
  stmtSelectByIdOrParent,
180
187
  stmtDeleteByIdOrParent,
181
188
  stmtSelectByParent,
189
+ stmtDestinationsById,
182
190
  stmtSelectAll,
183
191
  stmtCount,
184
192
  stmtEntityInputHashes,
@@ -1036,7 +1036,7 @@ export function api(options = {}) {
1036
1036
  // straight to render(entity, options). Defaults match
1037
1037
  // mikser's lifecycle (save and keep the catalog row);
1038
1038
  // strict opt-outs via the literal `false`:
1039
- // options.catalog: false → prune the catalog row
1039
+ // options.catalog: false → leave the catalog as found
1040
1040
  // options.save: false → skip the final disk write
1041
1041
  // (bytes still in the response)
1042
1042
  const { options = {}, ...entityShape } = req.body
package/src/render.js CHANGED
@@ -506,13 +506,16 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
506
506
  * Two control flags mirror mikser's default-keep-everything behavior;
507
507
  * both opt-out via strict `=== false`:
508
508
  *
509
- * - `catalog: true` (default) — keep the entity in the catalog after
510
- * the render. Pass `catalog: false` to prune the catalog row;
511
- * useful for on-demand renders where the metadata row would just
512
- * accumulate. Requires `save: false` the prune goes through the
513
- * journal, and a DELETE takes the manifest's file cleanup with it,
514
- * so it is only safe for a render that wrote nothing. With
515
- * `save: true` the row is kept and a warning is logged.
509
+ * - `catalog: true` (default) — pass `catalog: false` and the catalog
510
+ * ends the call exactly as it began it: the row that was there is put
511
+ * back as it was, and a row this render created is removed. The render
512
+ * still travels the lifecycle, so the caller's copy usually altered,
513
+ * since an on-demand surface forces its own layout onto it — does reach
514
+ * the row while the render needs it; it just does not outlive the call.
515
+ * Combines with either `save`, which is the point: gpoint-api renders
516
+ * its emails with `save: true, catalog: false` — the file is wanted, the
517
+ * row is not. `save: false` implies it, since a render that keeps
518
+ * nothing on disk has no business moving a row either.
516
519
  * - `save: true` (default) — write the rendered output to disk at
517
520
  * `<outputFolder>/<entity.destination>`. Pass `save: false` to
518
521
  * skip the final disk write; the bytes still come back via
@@ -539,31 +542,40 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
539
542
  * @param {object} entity - any entity-shaped object
540
543
  * @param {object} [opts]
541
544
  * @param {number} [opts.timeout] - override the default timeout
542
- * @param {boolean} [opts.catalog=true] - keep the catalog row after render
545
+ * @param {boolean} [opts.catalog=true] - let this render's changes reach
546
+ * the catalog; false restores it
543
547
  * @param {boolean} [opts.save=true] - write the rendered output to disk
544
548
  * @returns {Promise<{output, entity}>}
545
549
  */
546
550
  async function render(entity, { timeout = defaultTimeout, catalog = true, save = true } = {}) {
547
- // Was this row already in the catalog BEFORE the render? `catalog:
548
- // false` means "do not leave a row behind", and that is only the
549
- // render's to decide for a row the render created. Asked here, before
550
- // the render puts one there.
551
+ // What the catalog held before this render touched it.
551
552
  //
552
- // Without it, `catalog: false` deleted whatever it was handed: a
553
- // preview of an EXISTING entity pruned the real row, so the entity
554
- // vanished from the site while its file sat on disk — no change set,
555
- // no cycle, and the removal logged only at debug. It cost a day to
556
- // find, in a form that rendered once and then answered "Entity not
557
- // found".
558
- // Imported HERE, not at the top: catalog.js reaches back into this
559
- // module, and a static import makes that cycle load-bearing at
553
+ // Read BEFORE dispatch, because the render itself overwrites it: the
554
+ // entity has to travel the lifecycle to render at all, and the
555
+ // pipeline reads the row back to resolve the layout, so suppressing
556
+ // that write outright fails the render with "requested layout X but
557
+ // produced no output". The row is written, used, and then put back.
558
+ //
559
+ // Imported here rather than at the top: catalog.js reaches back into
560
+ // this module, and a static import makes that cycle load-bearing at
560
561
  // module-evaluation time — it surfaced as "Cannot access 'schemas'
561
562
  // before initialization" in three unrelated test files.
562
- const preexisting = catalog === false && entity?.id
563
- ? Boolean(await (await import('./catalog.js')).findById(entity.id))
564
- : false
563
+ const neutral = save === false || catalog === false
564
+ // Restoring is about a ROW, so it needs an id; the flag itself does
565
+ // not, and the manifest reads the flag to decide whether to record a
566
+ // snapshot. Keeping the two apart means an id-less entity — which has
567
+ // no row to put back — still gets the rest of neutrality.
568
+ let priorRow = null
569
+ if (neutral && entity?.id) {
570
+ const { findById, markNeutralRender } = await import('./catalog.js')
571
+ priorRow = findById(entity.id)
572
+ // Handed over before dispatch, so it is in place however early the
573
+ // cycle starts. The catalog's finalize drain is where the row goes
574
+ // back for good — see applyJournalMutations.
575
+ markNeutralRender(entity.id, priorRow)
576
+ }
565
577
 
566
- const result = await new Promise((resolve, reject) => {
578
+ const dispatched = new Promise((resolve, reject) => {
567
579
  const correlationId = randomUUID()
568
580
  // Engine-set fields live under entity.options. The caller's
569
581
  // render(entity, { save: false }) becomes
@@ -580,6 +592,10 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
580
592
  ...entity.options,
581
593
  correlationId,
582
594
  ...(save === false ? { save: false } : {}),
595
+ // Marks every journal entry this render produces, however
596
+ // many phases later they arrive, as belonging to a
597
+ // catalog-neutral call.
598
+ ...(neutral ? { neutral: true } : {}),
583
599
  },
584
600
  }
585
601
  pending.push({
@@ -593,32 +609,28 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
593
609
  if (!cycleRunning) setImmediate(runBatch)
594
610
  })
595
611
 
596
- if (catalog === false) {
597
- // Prune the row through the journal, so the DELETE lands in
598
- // sqlite at onPersist alongside the CREATE that put it there.
599
- // Strict equality — null / "false" / 0 keep the row.
600
- //
601
- // Only when `save` is also false. A DELETE carries the
602
- // manifest's file cleanup with it, which unlinks the render's
603
- // output; that is correct for an entity that produced no file
604
- // and wrong for one that did. `catalog: false, save: true`
605
- // therefore keeps its row, and says so rather than dropping
606
- // the output on the floor.
607
- if (preexisting) {
608
- // Someone else's row. It was here before this render and is
609
- // not this render's to remove.
610
- useLogger()?.debug(
611
- 'render: catalog:false ignored for %s the entity was already in the catalog',
612
- result.entity.id,
613
- )
614
- } else if (save === false) {
615
- await runtime.delete(result.entity)
616
- } else {
617
- useLogger()?.warn(
618
- 'render: catalog:false ignored for %s — it needs save:false, ' +
619
- 'because pruning the row also unlinks the rendered output',
620
- result.entity.id,
621
- )
612
+ let result
613
+ try {
614
+ result = await dispatched
615
+ } finally {
616
+ // In a finally, because a render that THROWS has still altered the
617
+ // row on its way through an unrenderable entity leaves the
618
+ // catalog holding the caller's copy of it, which is the same
619
+ // damage as a successful render leaving its layout behind.
620
+ if (neutral && entity?.id) {
621
+ // Put the row back for whoever is awaiting us. The finalize
622
+ // drain does it again, and has to: it runs after the render and
623
+ // would otherwise re-apply the altered copy over this one. Both
624
+ // restore the same prior row, so their order does not matter.
625
+ //
626
+ // Deliberately not journaled. A journal entry would dispatch
627
+ // another render, and a journaled DELETE drags the manifest's
628
+ // file cleanup along with it — which would unlink the very file
629
+ // a `save: true, catalog: false` render was asked to produce.
630
+ // That coupling is why this used to refuse `save: true`.
631
+ const { restoreEntity, settleNeutralRender } = await import('./catalog.js')
632
+ restoreEntity(entity.id, priorRow)
633
+ settleNeutralRender(entity.id)
622
634
  }
623
635
  }
624
636