mikser-io 11.2.2 → 11.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/decisions/0008-mcp-ui-action-delivery.md +281 -0
- package/docs/decisions/README.md +1 -1
- package/docs/diagnostics.md +23 -0
- package/docs/getting-started.md +21 -5
- package/docs/plugins.md +1 -1
- package/package.json +1 -1
- package/src/catalog.js +124 -4
- package/src/manifest/cycle.js +138 -19
- package/src/manifest/index.js +30 -0
- package/src/manifest/statements.js +8 -0
- package/src/plugins/api.js +1 -1
- package/src/render.js +62 -50
|
@@ -0,0 +1,281 @@
|
|
|
1
|
+
# ADR-0008 — MCP-UI: spec-compatible shell + `tools/call` delivery + optional webhook
|
|
2
|
+
|
|
3
|
+
**Status:** Superseded
|
|
4
|
+
**Date:** 2026
|
|
5
|
+
**Supersedes:** —
|
|
6
|
+
**Superseded by:** MCP Apps (SEP-1865), implemented in `mikser-io-mcp-app`
|
|
7
|
+
|
|
8
|
+
## Why this is kept
|
|
9
|
+
|
|
10
|
+
Archived here because the index row above it linked to this file in the
|
|
11
|
+
`mikser-io-mcp` repository, and the surface moved on to `mikser-io-mcp-app`,
|
|
12
|
+
taking the file with it — leaving core's own decision log with a dead link and
|
|
13
|
+
a hole at 0008.
|
|
14
|
+
|
|
15
|
+
Most of it was right, which is the interesting part. What survived into
|
|
16
|
+
`mikser-io-mcp-app` is the shape decided here: a static shell resource
|
|
17
|
+
declared as the tool's `_meta.ui.resourceUri`, per-call fragments delivered as
|
|
18
|
+
data rather than markup, clicks arriving as an ordinary `tools/call` against
|
|
19
|
+
an app-only tool, and the declared action list as the authorization boundary.
|
|
20
|
+
|
|
21
|
+
Two things did not.
|
|
22
|
+
|
|
23
|
+
**The webhook handler.** `mcpUi.handler.url` forwarded the action to an
|
|
24
|
+
external endpoint, and the ADR argued this respects ADR-0001 because the
|
|
25
|
+
application logic stays outside mikser. The logic did; the URL did not. It
|
|
26
|
+
was an MCP concern sitting in the engine, read by nothing but the MCP
|
|
27
|
+
surface — a field core carried on behalf of one plugin. It is gone, and what
|
|
28
|
+
it reached for is a layout sidecar instead: `<layout>.js` beside the template,
|
|
29
|
+
exporting `call` / `read` / `list`, running in the project rather than at the
|
|
30
|
+
end of an HTTP hop.
|
|
31
|
+
|
|
32
|
+
**The package.** A host for applications wants a route whose tool list *is*
|
|
33
|
+
the surface, and `mikser_app_action` is app-callable by specification — it has
|
|
34
|
+
no business appearing on the agent's endpoint. So the surface left
|
|
35
|
+
`mikser-io-mcp` for `mikser-io-mcp-app`, and `mcpUi` became `mcpApp`.
|
|
36
|
+
|
|
37
|
+
One earlier draft of this ADR, on the since-deleted `feat/mcp-ui-handler`
|
|
38
|
+
branch of this repository, is worth one line because the error is easy to
|
|
39
|
+
repeat: it had the iframe POST its action to mikser, reasoning the fetch is
|
|
40
|
+
same-origin because the iframe is served from mikser's origin. Under MCP Apps
|
|
41
|
+
the iframe's CSP is `default-src 'none'; connect-src 'none'` — there is no
|
|
42
|
+
network inside it at all, same-origin included. `postMessage` to the host is
|
|
43
|
+
the only way out. The version below had already corrected that to
|
|
44
|
+
`tools/call`.
|
|
45
|
+
|
|
46
|
+
## Context
|
|
47
|
+
|
|
48
|
+
ADR-0007 (`MCP-UI: layouts as the agent's UI surface`) introduced the idea that mikser layouts can serve as the agent's UI surface inside an MCP host. The `mikser_preview_ui` tool renders an `mcpUi`-decorated layout against an entity and returns HTML for the host to surface as a sandboxed iframe. The user interacts with the iframe — clicks Approve, fills a form, picks a status — and the click needs to get back to mikser as a structured tool result.
|
|
49
|
+
|
|
50
|
+
Three facts about the surrounding ecosystem shape this decision:
|
|
51
|
+
|
|
52
|
+
1. **The MCP Apps spec ([2026-01-26](https://github.com/modelcontextprotocol/ext-apps/blob/main/specification/2026-01-26/apps.mdx)) defines how iframe-to-server delivery works.** The iframe runs as an MCP client speaking JSON-RPC over `window.parent.postMessage` to the host. The host's "AppBridge" translates `tools/call` frames into real MCP tool calls on the existing transport. Every conformant host (Goose, ChatGPT/Apps SDK, mcp-ui's reference host, basic-host, VS Code Insiders) implements this same pattern.
|
|
53
|
+
|
|
54
|
+
2. **The spec mandates the iframe is cross-origin from the host with a restrictive default CSP.** "The Host and the Sandbox MUST have different origins." Default CSP when `ui.csp` is omitted: `default-src 'none'; connect-src 'none'`. So a `fetch` to mikser from inside the iframe — even on the same machine — is blocked by the browser. The only outbound channel is `postMessage`.
|
|
55
|
+
|
|
56
|
+
3. **The spec mandates that UI is delivered as a static *resource*, not inline tool-result HTML.** Tools that want UI declare `_meta.ui.resourceUri` on their tool definition, pointing at a `ui://` resource with `mimeType: 'text/html;profile=mcp-app'`. Spec-conformant hosts fetch that resource once via `resources/read`, load it in a sandboxed iframe, then push the tool's per-call result to the iframe via `ui/notifications/tool-result`. Empirically, hosts like basic-host display tools that lack `_meta.ui.resourceUri` as plain text rather than rendering an iframe — even if `content[0]` declares `mimeType: 'text/html'`.
|
|
57
|
+
|
|
58
|
+
These three facts together rule out two tempting alternatives:
|
|
59
|
+
|
|
60
|
+
- **An in-process HTTP endpoint that the iframe POSTs to**, with a server-minted "callId" as the capability URL. Prototyped on `feat/mcp-ui-handler`, ruled out — incompatible with `connect-src 'none'`, invisible to the host's consent/audit surface, inverts every other MCP App implementation in the ecosystem.
|
|
61
|
+
- **Returning the rendered HTML inline as `content[0].text` and trusting hosts to render it.** Initially shipped in 8.0.x; surfaced as the symptom "basic-host received the tool result but shows it as text, not an iframe." Spec-conformant hosts need the resource URI on the tool definition; they ignore content with `mimeType: 'text/html'` when no resource URI is set.
|
|
62
|
+
|
|
63
|
+
Separately, productized workflows want to intercept the action server-side without forcing the agent to learn application-specific schemas — a CRM, support system, or admin tool wants to receive the click, do its work, and return a domain-specific result. Pure relay (mikser returns the click data, agent decides what `approve` means) is right for AI-native workflows; webhook delegation (mikser forwards the click to an external URL) is right for productized ones. We want both, without turning mikser into a workflow engine.
|
|
64
|
+
|
|
65
|
+
## Decision
|
|
66
|
+
|
|
67
|
+
### Part A — Rendering: static shell resource + structured tool result
|
|
68
|
+
|
|
69
|
+
**A1. Mikser ships a single static UI resource: `ui://mikser/preview-ui-shell`.**
|
|
70
|
+
|
|
71
|
+
```js
|
|
72
|
+
mcp.registerResource(
|
|
73
|
+
'mikser-preview-ui-shell',
|
|
74
|
+
'ui://mikser/preview-ui-shell',
|
|
75
|
+
{ mimeType: 'text/html;profile=mcp-app', ... },
|
|
76
|
+
async (uri) => ({
|
|
77
|
+
contents: [{ uri: uri.href, mimeType: 'text/html;profile=mcp-app', text: SHELL_HTML }],
|
|
78
|
+
}),
|
|
79
|
+
)
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
The shell is ~120 lines of self-contained HTML+JS implementing the MCP Apps protocol:
|
|
83
|
+
|
|
84
|
+
- Sends `ui/initialize` to the host on load (2-second timeout, fails open if no host replies).
|
|
85
|
+
- Listens for `ui/notifications/tool-result`. Reads `structuredContent.html` from the result and injects it into a `#mikser-ui-root` div. Re-executes any `<script>` tags the layout brought (innerHTML doesn't execute embedded scripts by default).
|
|
86
|
+
- Exposes `window.sendAction(action, payload?)` — the API layouts use to deliver clicks back as `tools/call` against `mikser_ui_action`. The shell tracks `entityId` and `layoutId` from the tool result, so layouts don't have to.
|
|
87
|
+
- Renders an in-iframe debug panel showing every protocol event with timestamps — so authors can see exactly where the round-trip fails on hosts that don't bridge.
|
|
88
|
+
|
|
89
|
+
The shell is static. It does not change between tool calls, between entities, or between mikser versions. It is a fixed bundle of protocol plumbing.
|
|
90
|
+
|
|
91
|
+
**A2. `mikser_preview_ui` declares `_meta.ui.resourceUri` pointing at the shell.**
|
|
92
|
+
|
|
93
|
+
```js
|
|
94
|
+
mcp.registerTool('mikser_preview_ui', {
|
|
95
|
+
description: '...',
|
|
96
|
+
inputSchema: { entityId, mode },
|
|
97
|
+
_meta: {
|
|
98
|
+
ui: { resourceUri: 'ui://mikser/preview-ui-shell' },
|
|
99
|
+
},
|
|
100
|
+
}, handler)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
This is the spec-mandated signal that this tool renders UI. Hosts that implement MCP Apps fetch the resource once via `resources/read` and use it as the iframe template for every call to this tool. Hosts that don't implement MCP Apps display `content[0].text` as plain text (the fallback path; better than nothing).
|
|
104
|
+
|
|
105
|
+
**A3. The tool result returns content + `structuredContent`.**
|
|
106
|
+
|
|
107
|
+
```jsonc
|
|
108
|
+
{
|
|
109
|
+
"content": [
|
|
110
|
+
{ "type": "text", "text": "<rendered fragment>", "mimeType": "text/html" }
|
|
111
|
+
],
|
|
112
|
+
"structuredContent": {
|
|
113
|
+
"entityId": "/blog/launch.md",
|
|
114
|
+
"layoutId": "/layouts/mcp-ui/post-approval.hbs",
|
|
115
|
+
"mode": "approval",
|
|
116
|
+
"html": "<rendered fragment>",
|
|
117
|
+
"mcpUi": { "actions": [...], "sandbox": [...], "actionTool": "mikser_ui_action" }
|
|
118
|
+
},
|
|
119
|
+
"_meta": { "mcpUi": { ... } }
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`content[0].text` is the fallback for non-UI hosts. `structuredContent` is what the host passes to the iframe via `ui/notifications/tool-result`. The shell reads `structuredContent.html` and injects it; it reads `entityId` and `layoutId` to scope subsequent `sendAction` calls.
|
|
124
|
+
|
|
125
|
+
**A4. Layouts are body fragments, not full HTML documents.**
|
|
126
|
+
|
|
127
|
+
The shell wraps `<!DOCTYPE>` / `<html>` / `<head>` / `<body>` around the injected content. Layouts produce a fragment containing inline `<style>`, body content, and optional inline `<script>`. The script can call `sendAction(action, payload?)` directly — it's exposed on `window` by the shell. No protocol code, no `ui/initialize`, no RPC helper, no postMessage shape.
|
|
128
|
+
|
|
129
|
+
Compare a layout authored before this ADR (~85 lines) with one authored after (~40 lines): everything below `<style>` stays; everything above `<style>` and the entire 50-line protocol `<script>` block disappears.
|
|
130
|
+
|
|
131
|
+
### Part B — Action delivery: `tools/call` against `mikser_ui_action` (visibility=['app'])
|
|
132
|
+
|
|
133
|
+
**B1. Mikser registers a separate, app-callable tool: `mikser_ui_action`.**
|
|
134
|
+
|
|
135
|
+
```js
|
|
136
|
+
mcp.registerTool('mikser_ui_action', {
|
|
137
|
+
description: '...',
|
|
138
|
+
inputSchema: { entityId, layoutId, action, payload },
|
|
139
|
+
_meta: { ui: { visibility: ['app'] } },
|
|
140
|
+
}, handler)
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
`visibility: ['app']` makes the tool invisible to the agent — it never appears in the agent's tool surface — but callable from inside iframes via the host's AppBridge. The agent sees the result as a normal tool turn in its conversation.
|
|
144
|
+
|
|
145
|
+
**B2. The action allow-list is the auth boundary.**
|
|
146
|
+
|
|
147
|
+
`mikser_ui_action`'s handler looks up the layout by `layoutId`, reads its `mcpUi.actions` list, and rejects any action not in that list with an error result. This is the single place where layout-declared "you can do these things" meets iframe-supplied "I want to do this thing." Unknown actions never reach pure relay, never reach `handler.url`.
|
|
148
|
+
|
|
149
|
+
**B3. There is no callId, no signature, no token on this channel.**
|
|
150
|
+
|
|
151
|
+
The host's MCP transport is already authenticated. The visibility flag already gates which tools the iframe can invoke. The action allow-list already scopes what the iframe can ask for. Layered defenses; no per-call crypto.
|
|
152
|
+
|
|
153
|
+
**B4. There is no in-process HTTP endpoint for action delivery.**
|
|
154
|
+
|
|
155
|
+
`/api/mcp-ui/action/...` does not exist. Adding one would create a second delivery path with a different auth model, double the test surface, and offer no benefit on conformant hosts (CSP blocks the fetch) or non-conformant ones (the iframe wouldn't render anyway). One channel, one auth model.
|
|
156
|
+
|
|
157
|
+
### Part C — Optional webhook handler
|
|
158
|
+
|
|
159
|
+
**C1. Layouts may declare a `handler` block in their `mcpUi` frontmatter.**
|
|
160
|
+
|
|
161
|
+
```yaml
|
|
162
|
+
---
|
|
163
|
+
match: "@/articles/*"
|
|
164
|
+
mcpUi:
|
|
165
|
+
mode: approval
|
|
166
|
+
actions: [approve, reject, request-changes]
|
|
167
|
+
sandbox: [allow-scripts]
|
|
168
|
+
handler:
|
|
169
|
+
url: https://app.example.com/mikser-actions
|
|
170
|
+
secret: ${MIKSER_HANDLER_SECRET} # optional, enables HMAC signing
|
|
171
|
+
timeout: 5000 # optional, ms; default 5000
|
|
172
|
+
---
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
When `handler.url` is set, `mikser_ui_action`'s handler forwards the action data to that URL instead of returning the pure-relay payload. The handler's JSON response body becomes the tool result.
|
|
176
|
+
|
|
177
|
+
**C2. The forwarded request is a standard webhook.**
|
|
178
|
+
|
|
179
|
+
```http
|
|
180
|
+
POST https://app.example.com/mikser-actions
|
|
181
|
+
Content-Type: application/json
|
|
182
|
+
X-Mikser-Signature: sha256=...
|
|
183
|
+
X-Mikser-Request-Id: <opaque uuid for idempotency>
|
|
184
|
+
X-Mikser-Layout-Id: /layouts/mcp-ui/post-approval.hbs
|
|
185
|
+
X-Mikser-Mode: approval
|
|
186
|
+
|
|
187
|
+
{
|
|
188
|
+
"entityId": "/documents/blog/launch.md",
|
|
189
|
+
"layoutId": "/layouts/mcp-ui/post-approval.hbs",
|
|
190
|
+
"action": "approve",
|
|
191
|
+
"payload": {},
|
|
192
|
+
"mode": "approval",
|
|
193
|
+
"timestamp": "2026-06-07T15:00:00Z"
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
`X-Mikser-Signature` is HMAC-SHA256 of the request body using `handler.secret`. Receivers verify before processing. If `secret` is unset, no signature is sent — fine for development; not recommended in production.
|
|
198
|
+
|
|
199
|
+
**C3. The handler's JSON response is the tool result.**
|
|
200
|
+
|
|
201
|
+
```json
|
|
202
|
+
{
|
|
203
|
+
"ok": true,
|
|
204
|
+
"summary": "Committed to main; deployment queued (build #4821).",
|
|
205
|
+
"url": "https://app.example.com/deploys/4821",
|
|
206
|
+
"_meta": { "buildId": 4821 }
|
|
207
|
+
}
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Mikser passes this through to the agent unchanged. The agent composes its next message from it. No domain knowledge in mikser.
|
|
211
|
+
|
|
212
|
+
**C4. Handler failures fall back to pure relay.**
|
|
213
|
+
|
|
214
|
+
Network error, timeout, non-2xx response, non-JSON response: mikser logs a warning and resolves the tool call with the default `{ entityId, action, payload }` plus a `handlerError` field carrying the failure reason. The user's click is never lost.
|
|
215
|
+
|
|
216
|
+
```json
|
|
217
|
+
{
|
|
218
|
+
"entityId": "/documents/blog/launch.md",
|
|
219
|
+
"action": "approve",
|
|
220
|
+
"payload": {},
|
|
221
|
+
"handlerError": "Handler timeout (5000ms) — https://app.example.com/mikser-actions"
|
|
222
|
+
}
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
**C5. The handler block is the entire extension surface.**
|
|
226
|
+
|
|
227
|
+
Mikser does not learn about action semantics — what `approve` means, what `request-changes` should do, where the result goes. Adding that knowledge to mikser would violate ADR-0001 (`Mikser is the content layer of the application, not the app`). The webhook contract IS the extension point; if you want behaviour, write a service.
|
|
228
|
+
|
|
229
|
+
## Consequences
|
|
230
|
+
|
|
231
|
+
**Easier:**
|
|
232
|
+
|
|
233
|
+
- Spec compliance is unconditional. Spec-conformant hosts (Goose, ChatGPT, mcp-ui's reference host, basic-host, VS Code Insiders) render mikser layouts as iframes correctly. No host-specific shims.
|
|
234
|
+
- Layouts shrink dramatically — no more per-layout 50-line protocol boilerplate. Author content + inline styles + click handlers; the shell handles everything else. Comparing the blog example's `post-approval.hbs` v8.0.x vs v8.1.0: ~85 lines → ~40 lines, roughly half. The protocol bug surface collapses to one place.
|
|
235
|
+
- The action allow-list + visibility flag are the entire auth model. No bespoke crypto primitive to debug.
|
|
236
|
+
- One static resource served forever; one tool result shape that's the same on every call. Cacheable, predictable, testable.
|
|
237
|
+
|
|
238
|
+
**Harder:**
|
|
239
|
+
|
|
240
|
+
- Layouts authored before 8.1.0 — full HTML documents with embedded `ui/initialize` and RPC plumbing — need rewriting. The new shape is mechanically simpler but it is *not* drop-in compatible with 8.0.x layouts. Mikser's blog example layouts ship pre-rewritten as canonical references.
|
|
241
|
+
- Hosts that haven't implemented MCP Apps (`Claude Desktop` per the open `upstream-host` bug [anthropics/claude-ai-mcp#165](https://github.com/anthropics/claude-ai-mcp/issues/165)) show the iframe as raw text. There is no fallback — and that's deliberate. Both alternatives explored above introduce more problems than they solve.
|
|
242
|
+
- The shell's debug panel is on by default. Production use will want to either gate it behind a query param or remove it entirely. Tracked as a follow-up.
|
|
243
|
+
|
|
244
|
+
## Examples
|
|
245
|
+
|
|
246
|
+
See `documentation/mcp.md` "Layout frontmatter and MCP-UI" — every worked example was rewritten when this ADR landed. Each one collapses to roughly 30-50 lines including inline styles.
|
|
247
|
+
|
|
248
|
+
## Alternatives considered
|
|
249
|
+
|
|
250
|
+
**Direct HTTP from iframe to mikser (capability URL pattern).** Prototyped on `feat/mcp-ui-handler`. Random callId, single-use, action allow-list, loopback bind — textbook capability URL, genuinely secure against CSRF/replay/unknown-actions. Ruled out because:
|
|
251
|
+
|
|
252
|
+
1. Default MCP Apps CSP is `connect-src 'none'` — the browser blocks the fetch on conformant hosts.
|
|
253
|
+
2. The iframe is cross-origin from the host by spec — there is no "same-origin" with mikser to leverage.
|
|
254
|
+
3. The action bypasses the host's audit/consent surface.
|
|
255
|
+
4. It inverts the direction of every other MCP App implementation, making mikser layouts non-portable.
|
|
256
|
+
|
|
257
|
+
**Returning HTML inline in `content[0].text` with `mimeType: 'text/html'`.** Shipped initially in 8.0.x. Surfaced as the symptom "basic-host received the tool result, displayed it as plain text, never rendered an iframe." Conformant hosts read `_meta.ui.resourceUri` off the tool definition (static) to decide what iframe template to load — they do not infer it from response content. The fix is the resource pattern in Part A.
|
|
258
|
+
|
|
259
|
+
**Per-layout tools (`mikser_preview_approval`, `mikser_preview_edit`, etc.) each with their own static `_meta.ui.resourceUri`.** Cleaner mapping but tool count grows with layout count, and the agent has to learn which tool handles which mode. The shell-as-template approach keeps the agent's tool surface stable (one `mikser_preview_ui` for all UIs) while still satisfying the spec's static-URI requirement.
|
|
260
|
+
|
|
261
|
+
**Dual-channel (postMessage primary, HTTP fallback).** Rejected as "no legacy" — two delivery paths means two auth models, two test surfaces, two failure modes to debug, and no host where both are needed. Pick one channel; pick the one the spec specifies.
|
|
262
|
+
|
|
263
|
+
**Built-in action vocabulary (mikser knows what `approve` / `reject` mean).** Rejected. This is the application layer; mikser is the content layer (ADR-0001). The agent owns semantics by default; `handler.url` is the escape hatch for productized cases.
|
|
264
|
+
|
|
265
|
+
**Server-side handler scripts (layouts ship a `handler:` callback in JS).** Rejected. Same reasoning — mikser would become a workflow engine. External webhooks compose; in-mikser handlers would couple action behaviour to mikser deployment.
|
|
266
|
+
|
|
267
|
+
**Per-action handlers (`handler` is a map: `{ approve: url1, reject: url2 }`).** Rejected as YAGNI. Single URL with the action in the payload is enough — the receiver multiplexes.
|
|
268
|
+
|
|
269
|
+
## Watch for drift
|
|
270
|
+
|
|
271
|
+
These are the failure modes this decision is protecting against. If you see them, push back.
|
|
272
|
+
|
|
273
|
+
- **The shell grows application-specific knowledge.** Someone proposes "the shell could pre-format the date before injecting" or "the shell could add a global retry banner." Refuse. The shell is protocol + injection + sendAction relay. Application concerns live in layouts.
|
|
274
|
+
- **A second action-delivery channel sneaks in.** Someone notices `mikser_ui_action` doesn't work on a non-conformant host and proposes adding an HTTP endpoint as a "fallback." Don't. The 8.1.0 design is one channel by deliberate choice.
|
|
275
|
+
- **`_meta.ui.resourceUri` drift on the tool definition.** Someone removes it or changes it to point at something dynamic. Spec-conformant hosts will stop rendering iframes — they only read this field at tool-list time, not per-call.
|
|
276
|
+
- **Layouts start re-implementing the protocol.** Someone writes a layout with its own `ui/initialize` handshake "for control." That layout will fight the shell. The shell exposes `sendAction`; that's the entire contract a layout uses.
|
|
277
|
+
- **The visibility flag drifts on `mikser_ui_action`.** If `_meta.ui.visibility` is removed or changed to `['model', 'app']`, the tool leaks into the agent's surface — it'll appear as an action the agent can take "out of context" without any iframe ever rendering. Strict `['app']`.
|
|
278
|
+
- **Action vocabulary creeps into core.** Someone proposes a built-in `approve` semantic so simple layouts don't need a handler. Refuse; that's application-layer logic.
|
|
279
|
+
- **Per-action handler URLs.** Someone proposes `handler: { approve: '...', reject: '...' }`. Refuse. One URL, the action goes in the payload, the receiver routes.
|
|
280
|
+
- **Retry / backoff on the handler.** Mikser doesn't retry. If the handler is down, the user re-clicks.
|
|
281
|
+
- **Persistent pending state.** There is no pending state — `mikser_preview_ui` returns synchronously and the shell handles per-render state in the iframe. Don't add a "pending action" table.
|
package/docs/decisions/README.md
CHANGED
|
@@ -33,7 +33,7 @@ Decisions don't expire. They get **superseded** when we learn enough to change t
|
|
|
33
33
|
| [0005](./0005-engine-infrastructure-runs-before-plugin-hooks.md) | Engine infrastructure (journal, catalog) is ready before any plugin hook runs; `runtime.update` is upsert; `useSource` codifies the folder-of-files pattern | Accepted |
|
|
34
34
|
| [0006](./0006-when-to-add-to-core.md) | The five-test check for adding capability to the engine vs. shipping it as a plugin | Accepted |
|
|
35
35
|
| [0007](./0007-references-declaration-and-expansion.md) | Entity references: `$`-prefixed declaration (canonical on disk, normalized for render/SDK) and `expand` resolution (inline, GET-cacheable, engine-level `runtime.refs` drives invalidation + live-expand) | Accepted |
|
|
36
|
-
| [0008](
|
|
36
|
+
| [0008](./0008-mcp-ui-action-delivery.md) | MCP-UI rendering + action delivery: static shell resource declared as `_meta.ui.resourceUri`, per-call fragments via `structuredContent`, clicks riding `tools/call` against an app-only tool, declared action list as the authorization boundary. The shape survived; the `mcpUi.handler.url` webhook did not — an MCP concern in the engine, replaced by a layout sidecar. The surface moved to [`mikser-io-mcp-app`](https://github.com/almero-digital-marketing/mikser-io-mcp-app) and `mcpUi` became `mcpApp`. Archived here because the file left with it. | Superseded |
|
|
37
37
|
| [0009](./0009-database-engine-substrate.md) | Sqlite is the engine's persistence substrate. Single `runtime/mikser.sqlite` file holds catalog/refs/manifest/journal as `mikser_entities` / `mikser_refs` / `mikser_snapshots` / `mikser_journal` tables. Plugins register schemas via `registerSchema(name, sql)` + `useDatabase()`. Sift→SQL translator with indexed pushdown, LRU cache for `findById`, worker-side read-only sqlite for sync template helpers, chunked journal walks + `iterateEntities` streaming, `useJournal` auto-persist (mutate the yielded entity; no explicit `updateEntry` needed), `--resume` after interrupted cycles. Replaces `Map<id, entity>` + NDJSON across every engine subsystem. | Accepted |
|
|
38
38
|
| [0010](./0010-plugin-bundles-and-inline-options.md) | Plugin bundles + factory-call form + inline options. Plugins are imported by name and called as factories; `plugins: []` carries factory returns, never strings. Lifecycle plugins are `(options) => (core) => void`; renderers return `{name, options, load?, render?}`; postprocessors return `{name, options, output?, setup?, postprocess, teardown?}`. Per-plugin config moved off `runtime.config.<plugin>` — it arrives as the factory arg and is passed as `config` to `load`/`render`/`setup`/`postprocess`. | Accepted |
|
|
39
39
|
| [0011](./0011-served-entities-expose-deployed-urls.md) | File and resource entities expose deployed URLs. References to served files (image/video/PDF) are `$`-keyed **served paths** (`/img/X.jpg`, `/media/clip.mp4` — the path content authors, = the entity's `meta.url`), resolving through a new `refFilter` `{ 'meta.url': … }` clause backed by an indexed `meta_url` column (schema 9.0.1 → 9.0.2). No collection-prefixed ids leak into content. (Id-refs were tried and rejected — gpoint references content by served path and its `/media/**` `resources()` library means the entity only exists because content references `/media/…`.) The `files`/`resources` plugins stamp `meta.url`, the `assets` plugin stamps `meta.presets` — so expanding a ref yields the served entity's URL set instead of a string to reconstruct. Base-relative in the live catalog (host-agnostic; consumer holds `base`), absolute in static renders (baked from `runtime.options.url`, so logic-less consumers — email, RSS, foreign apps — read a whole URL); `lookupUrl` render helper resolves a ref to `meta.url` or a named preset. SDK collapses `assetUrl(source, preset, {ext})` into one `url(ref)` join + a dev-mode SPA-fallback detector. *("Served entity" — file/resource/preset — avoids colliding with the `assets` plugin's own "asset reference" term.)* | Accepted (proven against gpoint) |
|
package/docs/diagnostics.md
CHANGED
|
@@ -771,6 +771,29 @@ layout it went through. Entities with no layout — a copied asset, a
|
|
|
771
771
|
`files()` passthrough — stay out, or the answer would be "everything,
|
|
772
772
|
always", which is the same non-answer as "nothing" with the sign flipped.
|
|
773
773
|
|
|
774
|
+
**`recordNoOutput(id)` — the one write on this facade a plugin should
|
|
775
|
+
make.** Everything above answers questions; this one tells the manifest
|
|
776
|
+
something only a dispatcher can know: that it looked at an entity and found
|
|
777
|
+
nothing to render it with. The manifest then drops the snapshots that entity
|
|
778
|
+
still holds, and unlinks the files they claim, in its own finalize
|
|
779
|
+
transaction — so the cleanup gets the same guard as every other: a
|
|
780
|
+
destination another entity still claims is never taken away.
|
|
781
|
+
|
|
782
|
+
It exists because the two states that matter are indistinguishable from
|
|
783
|
+
inside the manifest. An entity whose `layout:` was removed and an asset whose
|
|
784
|
+
preset threw both arrive with no successful render and no destination on their
|
|
785
|
+
catalog row, and pruning on that resemblance deletes the good derivative a
|
|
786
|
+
failed preset exists to keep. So the call is only correct for "I dispatched
|
|
787
|
+
this and matched nothing" — never for a failure, and never for a declaration
|
|
788
|
+
that could not be resolved, since mikser keeps the last good output through an
|
|
789
|
+
error rather than taking the page down. `mikser-io-layouts` makes it; any
|
|
790
|
+
other dispatcher can.
|
|
791
|
+
|
|
792
|
+
Without it, an entity that stops producing output leaves its page on disk with
|
|
793
|
+
a snapshot still vouching for it — and because the file still matches the hash
|
|
794
|
+
its own render recorded, `--audit-output` reads OK while the site serves
|
|
795
|
+
something the source no longer asks for.
|
|
796
|
+
|
|
774
797
|
### `runtime.provenance`
|
|
775
798
|
|
|
776
799
|
Where a value was **written** — source file, field path, line and column.
|
package/docs/getting-started.md
CHANGED
|
@@ -187,8 +187,9 @@ const documents = useCollection(runtime, 'documents')
|
|
|
187
187
|
// that's what mikser's normal lifecycle does (anything persisted is
|
|
188
188
|
// queryable via findEntities). For on-demand renders where the
|
|
189
189
|
// bytes are the work product and you don't want the metadata row to
|
|
190
|
-
// accumulate, pass { catalog: false }
|
|
191
|
-
// output file is kept on disk
|
|
190
|
+
// accumulate, pass { catalog: false } and the catalog ends the call
|
|
191
|
+
// exactly as it began it. The rendered output file is kept on disk
|
|
192
|
+
// either way.
|
|
192
193
|
const { output, entity } = await render({
|
|
193
194
|
id: '/documents/en/report.md',
|
|
194
195
|
type: 'document',
|
|
@@ -217,9 +218,24 @@ parallelism within the cycle is governed by `runtime.options.threads`.
|
|
|
217
218
|
`render` options:
|
|
218
219
|
|
|
219
220
|
- `timeout` — per-call timeout in ms (default 30_000).
|
|
220
|
-
- `catalog` (default `true`) —
|
|
221
|
-
|
|
222
|
-
|
|
221
|
+
- `catalog` (default `true`) — let this render's changes reach the catalog.
|
|
222
|
+
Pass `catalog: false` and the catalog ends the call exactly as it began
|
|
223
|
+
it: the row that was there goes back as it was, and a row this render
|
|
224
|
+
created is removed. Useful for on-demand renders where the metadata would
|
|
225
|
+
just accumulate, and for previews, where the entity you hand in is usually
|
|
226
|
+
an altered copy (a surface forcing its own layout onto it) that has no
|
|
227
|
+
business becoming the entity's production state.
|
|
228
|
+
|
|
229
|
+
The render still travels the lifecycle, so the copy does reach the row
|
|
230
|
+
while the render needs it — the pipeline reads it back to resolve the
|
|
231
|
+
layout. It just does not outlive the call, whether the render succeeds or
|
|
232
|
+
throws. Nothing is journalled to achieve this, so the file a
|
|
233
|
+
`save: true, catalog: false` render produced stays on disk.
|
|
234
|
+
|
|
235
|
+
`save: false` implies it: a render that keeps nothing on disk has no
|
|
236
|
+
business moving a row either. Neither combination records a manifest
|
|
237
|
+
snapshot — a snapshot is a claim about a catalog entity's output, and
|
|
238
|
+
after a neutral render there is no such entity to speak for.
|
|
223
239
|
- `save` (default `true`) — write the rendered output to disk at
|
|
224
240
|
`<outputFolder>/<entity.destination>`. Pass `save: false` to skip the
|
|
225
241
|
final disk write; the bytes still come back in `output.result` for you
|
package/docs/plugins.md
CHANGED
|
@@ -702,7 +702,7 @@ Writes content to a file in a collection folder. The file change is picked up by
|
|
|
702
702
|
```
|
|
703
703
|
|
|
704
704
|
`options` is optional. Strict opt-outs via the literal `false`:
|
|
705
|
-
- `options.catalog: false` —
|
|
705
|
+
- `options.catalog: false` — leave the catalog exactly as the call found it (the row goes back as it was; a row this render created is removed). The output file is kept.
|
|
706
706
|
- `options.save: false` — skip the final disk write (bytes still in the response)
|
|
707
707
|
|
|
708
708
|
**`GET /<endpoint>/entities/subscribe`**
|
package/package.json
CHANGED
package/src/catalog.js
CHANGED
|
@@ -191,6 +191,58 @@ function entityToRow(entity) {
|
|
|
191
191
|
}
|
|
192
192
|
}
|
|
193
193
|
|
|
194
|
+
// Catalog-neutral renders in flight: entity id -> the row as it stood before
|
|
195
|
+
// the render, or null where there was no row.
|
|
196
|
+
//
|
|
197
|
+
// Module-local rather than a field on runtime, deliberately. useRenderer takes
|
|
198
|
+
// its runtime by injection while this module imports the singleton, so a field
|
|
199
|
+
// there is written on one object and read on another the moment anyone injects
|
|
200
|
+
// anything else — which every unit test does. One owner, and the writer has to
|
|
201
|
+
// come through markNeutralRender.
|
|
202
|
+
const neutralRenders = new Map()
|
|
203
|
+
|
|
204
|
+
// Ids whose neutral render has finished — successfully or not — and whose
|
|
205
|
+
// entry is therefore the next finalize drain's to clear.
|
|
206
|
+
//
|
|
207
|
+
// Without this the map is a leak of exactly the kind this feature exists to
|
|
208
|
+
// prevent: gpoint-api renders with a fresh uuid per request, so an entry that
|
|
209
|
+
// is only ever removed when a matching write shows up accumulates one row per
|
|
210
|
+
// render for the life of the process. A render that THROWS journals nothing at
|
|
211
|
+
// all, so that is not a hypothetical.
|
|
212
|
+
const settledNeutralRenders = new Set()
|
|
213
|
+
|
|
214
|
+
/**
|
|
215
|
+
* Declare that a render of `id` must leave the catalog as it found it, and
|
|
216
|
+
* hand over the row to put back — or null where there was none.
|
|
217
|
+
*
|
|
218
|
+
* Called by useRenderer BEFORE dispatch: the row is written during the cycle
|
|
219
|
+
* because the pipeline reads it back to resolve the layout, so what makes the
|
|
220
|
+
* render neutral is the restore afterwards, not a suppressed write.
|
|
221
|
+
*
|
|
222
|
+
* @param {string} id
|
|
223
|
+
* @param {object|null} prior
|
|
224
|
+
*/
|
|
225
|
+
export function markNeutralRender(id, prior) {
|
|
226
|
+
if (!id) return
|
|
227
|
+
neutralRenders.set(id, prior ?? null)
|
|
228
|
+
settledNeutralRenders.delete(id)
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
/**
|
|
232
|
+
* Report that the render of `id` has finished, however it finished. The next
|
|
233
|
+
* finalize drain restores anything it still owes and drops the entry.
|
|
234
|
+
*
|
|
235
|
+
* Separate from markNeutralRender because the two happen either side of the
|
|
236
|
+
* cycle: a render registered while a cycle is running is served by the NEXT
|
|
237
|
+
* one, so entries cannot simply be cleared at each cycle's end — an unsettled
|
|
238
|
+
* one is still waiting for its turn.
|
|
239
|
+
*
|
|
240
|
+
* @param {string} id
|
|
241
|
+
*/
|
|
242
|
+
export function settleNeutralRender(id) {
|
|
243
|
+
if (id && neutralRenders.has(id)) settledNeutralRenders.add(id)
|
|
244
|
+
}
|
|
245
|
+
|
|
194
246
|
// Apply per-cycle journal mutations inside one transaction (per the
|
|
195
247
|
// migration plan's per-phase transaction granularity). Maintains
|
|
196
248
|
// `mikser_refs` alongside `mikser_entities` so refs and entities
|
|
@@ -198,14 +250,26 @@ function entityToRow(entity) {
|
|
|
198
250
|
//
|
|
199
251
|
// better-sqlite3's transaction wrapper is sync-only, so we drain the
|
|
200
252
|
// journal first and then sync-apply in one call.
|
|
201
|
-
|
|
253
|
+
// `phase` is which drain this is — 'persist' (before render) or 'finalize'
|
|
254
|
+
// (after it). It decides what happens to a catalog-neutral render's entry:
|
|
255
|
+
// the persist drain applies it, because the pipeline reads the row back to
|
|
256
|
+
// resolve the layout and the render fails outright without it; the finalize
|
|
257
|
+
// drain puts the prior row back instead, because by then the render is done
|
|
258
|
+
// and the altered copy has no business outliving it.
|
|
259
|
+
async function applyJournalMutations(phase) {
|
|
202
260
|
const logger = useLogger()
|
|
203
261
|
const refsIndex = useRefsIndex()
|
|
204
262
|
const mutations = []
|
|
205
263
|
for await (const { operation, entity } of useJournal('Catalog')) {
|
|
206
264
|
mutations.push({ operation, entity })
|
|
207
265
|
}
|
|
208
|
-
|
|
266
|
+
// Ids whose neutral render finished in this cycle. Collected during the
|
|
267
|
+
// pass and settled inside the same transaction, so no reader ever sees
|
|
268
|
+
// the altered row.
|
|
269
|
+
const toRestore = phase === 'finalize' ? new Map() : null
|
|
270
|
+
// A settled entry is cleared even in a cycle that journalled nothing —
|
|
271
|
+
// which is precisely the shape a failed render leaves behind.
|
|
272
|
+
if (!mutations.length && !settledNeutralRenders.size) return
|
|
209
273
|
db.transaction(() => {
|
|
210
274
|
// Two passes, deliberately. indexEntity resolves each $-ref
|
|
211
275
|
// against mikser_entities to record what it bound to, so it has
|
|
@@ -217,6 +281,14 @@ async function applyJournalMutations() {
|
|
|
217
281
|
switch (operation) {
|
|
218
282
|
case OPERATION.CREATE:
|
|
219
283
|
case OPERATION.UPDATE:
|
|
284
|
+
if (toRestore && neutralRenders.has(entity.id)) {
|
|
285
|
+
// A catalog-neutral render's own write, arriving after
|
|
286
|
+
// the render. Restoring rather than applying is the
|
|
287
|
+
// whole of `catalog: false`.
|
|
288
|
+
logger.trace('Database restore after neutral render: %s', entity.id)
|
|
289
|
+
toRestore.set(entity.id, neutralRenders.get(entity.id))
|
|
290
|
+
break
|
|
291
|
+
}
|
|
220
292
|
logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
|
|
221
293
|
stmtUpsert.run(entityToRow(entity))
|
|
222
294
|
toIndex.push(entity)
|
|
@@ -241,9 +313,57 @@ async function applyJournalMutations() {
|
|
|
241
313
|
// delete-then-insert per source internally, so this stays
|
|
242
314
|
// idempotent across UPDATE.
|
|
243
315
|
for (const entity of toIndex) refsIndex?.indexEntity(entity)
|
|
316
|
+
|
|
317
|
+
// Last, so it wins over anything else this batch wrote for the id.
|
|
318
|
+
for (const [id, prior] of toRestore ?? []) {
|
|
319
|
+
restoreRow(id, prior, refsIndex)
|
|
320
|
+
neutralRenders.delete(id)
|
|
321
|
+
settledNeutralRenders.delete(id)
|
|
322
|
+
}
|
|
323
|
+
// Whatever is left over from a render that has finished without
|
|
324
|
+
// journalling anything for us to undo. Nothing to restore — useRenderer
|
|
325
|
+
// already put the row back the moment the render settled — so this only
|
|
326
|
+
// releases the entry.
|
|
327
|
+
if (toRestore) {
|
|
328
|
+
for (const id of settledNeutralRenders) neutralRenders.delete(id)
|
|
329
|
+
settledNeutralRenders.clear()
|
|
330
|
+
}
|
|
244
331
|
})
|
|
245
332
|
}
|
|
246
333
|
|
|
334
|
+
// Put a row back exactly as it stood, or remove it where there was none.
|
|
335
|
+
//
|
|
336
|
+
// Sync, and assumes it is already inside a transaction — both callers are.
|
|
337
|
+
// Writes nothing to the journal on purpose: a journal entry would dispatch
|
|
338
|
+
// another render, and a journaled DELETE drags the manifest's file cleanup
|
|
339
|
+
// with it, which would unlink the output a `save: true, catalog: false`
|
|
340
|
+
// render was asked to produce.
|
|
341
|
+
function restoreRow(id, prior, refsIndex = useRefsIndex()) {
|
|
342
|
+
if (prior) {
|
|
343
|
+
stmtUpsert.run(entityToRow(prior))
|
|
344
|
+
refsIndex?.indexEntity(prior)
|
|
345
|
+
} else {
|
|
346
|
+
stmtDelete.run(id)
|
|
347
|
+
}
|
|
348
|
+
cacheEvict(id)
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
/**
|
|
352
|
+
* Restore an entity to the row it had before a catalog-neutral render, or
|
|
353
|
+
* remove it if it had none. Called by useRenderer as soon as the render
|
|
354
|
+
* resolves, so a caller awaiting `render()` sees the catalog as it was; the
|
|
355
|
+
* finalize drain then does the same again for the write that lands after.
|
|
356
|
+
*
|
|
357
|
+
* @param {string} id
|
|
358
|
+
* @param {object|null} prior - the row as it stood, or null if there was none
|
|
359
|
+
*/
|
|
360
|
+
export function restoreEntity(id, prior) {
|
|
361
|
+
if (!id || !db?.isOpen) return
|
|
362
|
+
const refsIndex = useRefsIndex()
|
|
363
|
+
// mikser's db.transaction RUNS the function; it does not return one.
|
|
364
|
+
db.transaction(() => restoreRow(id, prior, refsIndex))
|
|
365
|
+
}
|
|
366
|
+
|
|
247
367
|
onLoaded(async () => {
|
|
248
368
|
db = useDatabase()
|
|
249
369
|
if (!db) {
|
|
@@ -329,7 +449,7 @@ onLoaded(async () => {
|
|
|
329
449
|
})
|
|
330
450
|
|
|
331
451
|
onPersist(async () => {
|
|
332
|
-
await applyJournalMutations()
|
|
452
|
+
await applyJournalMutations('persist')
|
|
333
453
|
})
|
|
334
454
|
|
|
335
455
|
onFinalize(async () => {
|
|
@@ -353,7 +473,7 @@ onFinalize(async () => {
|
|
|
353
473
|
// render reads the catalog), and journal consumers are named and
|
|
354
474
|
// independent, so a second pass only ever picks up what the first could
|
|
355
475
|
// not have seen.
|
|
356
|
-
await applyJournalMutations()
|
|
476
|
+
await applyJournalMutations('finalize')
|
|
357
477
|
|
|
358
478
|
// Checkpoint the WAL so the main file size stays representative
|
|
359
479
|
// and external tools (mikser --audit-output on a separate run, debug
|
package/src/manifest/cycle.js
CHANGED
|
@@ -57,20 +57,51 @@ onFinalize(async () => {
|
|
|
57
57
|
deletedIds.push(entity.id)
|
|
58
58
|
}
|
|
59
59
|
|
|
60
|
+
// Every destination claimed by a render task this cycle — whether it
|
|
61
|
+
// succeeded, was skipped as current, or failed.
|
|
62
|
+
//
|
|
63
|
+
// This is deliberately WIDER than renderedEntries, and the width is the
|
|
64
|
+
// safety. The staleness test below asks "is this row's destination one its
|
|
65
|
+
// entity still claims", and all three outcomes answer yes:
|
|
66
|
+
//
|
|
67
|
+
// skipped the manifest said the output was already current, which is
|
|
68
|
+
// an assertion that it BELONGS. One entity can match several
|
|
69
|
+
// layouts, and a dependency change invalidates them
|
|
70
|
+
// independently — so a cycle where layout A re-renders and
|
|
71
|
+
// layout B is skipped is ordinary, and reading B's silence as
|
|
72
|
+
// "no longer produced" would unlink a live page.
|
|
73
|
+
// failed a failed render writes no snapshot on purpose, so the last
|
|
74
|
+
// good bytes survive. Treating the gap as abandonment
|
|
75
|
+
// destroys exactly what that rule protects.
|
|
76
|
+
const claimedByRenderTasks = new Map()
|
|
60
77
|
for await (const { output, entity, deps } of useJournal('Output', [OPERATION.RENDER])) {
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
//
|
|
78
|
+
if (entity?.id && entity.destination) {
|
|
79
|
+
if (!claimedByRenderTasks.has(entity.id)) claimedByRenderTasks.set(entity.id, new Set())
|
|
80
|
+
claimedByRenderTasks.get(entity.id).add(entity.destination)
|
|
81
|
+
}
|
|
82
|
+
// A catalog-neutral render leaves no snapshot. A snapshot is the
|
|
83
|
+
// manifest's claim about a CATALOG ENTITY's output — it is what
|
|
84
|
+
// --audit-output verifies and what invalidation compares against — and
|
|
85
|
+
// both halves of `neutral` end the call with no such entity to speak
|
|
86
|
+
// for.
|
|
66
87
|
//
|
|
67
|
-
//
|
|
88
|
+
// For `save: false` the claim is false outright: nothing was written.
|
|
89
|
+
// Observed on a live site — an MCP app rendered on demand left
|
|
68
90
|
// /internal/customer-registration.html claimed and absent, and the
|
|
69
|
-
// audit went red for a page that was never supposed to exist.
|
|
70
|
-
//
|
|
71
|
-
//
|
|
72
|
-
//
|
|
73
|
-
|
|
91
|
+
// audit went red for a page that was never supposed to exist.
|
|
92
|
+
//
|
|
93
|
+
// For `save: true, catalog: false` the file is real but the entity is
|
|
94
|
+
// deliberately not mikser's to track, and recording it is what makes
|
|
95
|
+
// the two accounts disagree: the manifest holds a snapshot whose
|
|
96
|
+
// entity no longer exists. It also breaks the caller directly —
|
|
97
|
+
// invalidation compares against the snapshot, finds the destination
|
|
98
|
+
// already current, and SKIPS the render, so the next call for the same
|
|
99
|
+
// id gets no output at all.
|
|
100
|
+
//
|
|
101
|
+
// Leaving it unrecorded is the truthful state for invalidation either
|
|
102
|
+
// way: mikser is not tracking this output, so there is nothing to
|
|
103
|
+
// reuse and nothing to verify.
|
|
104
|
+
if (entity?.options?.neutral) continue
|
|
74
105
|
if (output?.success && !output.skipped) {
|
|
75
106
|
renderedEntries.push({ entity, deps, metaReads: output.metaReads,
|
|
76
107
|
consumedReads: output.consumedReads })
|
|
@@ -119,13 +150,25 @@ onFinalize(async () => {
|
|
|
119
150
|
|
|
120
151
|
// 2b. Pagination shrunk — drop children whose destination wasn't
|
|
121
152
|
// re-emitted this cycle.
|
|
122
|
-
const
|
|
153
|
+
const snapshotsToDelete = [] // [{id, destination}] — rows to drop by PK
|
|
154
|
+
// Ids that DEPART with their snapshot: a pagination child that no longer
|
|
155
|
+
// exists. Feeds `goingAway`, which asks "is this destination still claimed
|
|
156
|
+
// by something that survives" — an id-level question, and the right one
|
|
157
|
+
// for an entity that is gone.
|
|
158
|
+
const departingIds = new Set()
|
|
159
|
+
// Rows dropped for a destination their own entity no longer produces, as
|
|
160
|
+
// `id \t destination`. Deliberately NOT an id: an entity whose
|
|
161
|
+
// destination merely MOVED is still very much here, and calling it
|
|
162
|
+
// departed would let another entity's cleanup unlink a destination this
|
|
163
|
+
// one still claims.
|
|
164
|
+
const droppedClaims = new Set()
|
|
123
165
|
for (const [parentId, keep] of newDestinationsByParent) {
|
|
124
166
|
const rows = m._stmtSelectByParent.all(parentId)
|
|
125
167
|
for (const row of rows) {
|
|
126
168
|
if (keep.has(row.destination)) continue
|
|
127
169
|
filesToUnlink.push({ destination: row.destination, reason: 'Pagination shrunk' })
|
|
128
|
-
|
|
170
|
+
snapshotsToDelete.push({ id: row.id, destination: row.destination })
|
|
171
|
+
departingIds.add(row.id)
|
|
129
172
|
}
|
|
130
173
|
}
|
|
131
174
|
|
|
@@ -134,14 +177,88 @@ onFinalize(async () => {
|
|
|
134
177
|
const rows = m._stmtSelectByParent.all(parentId)
|
|
135
178
|
for (const row of rows) {
|
|
136
179
|
filesToUnlink.push({ destination: row.destination, reason: 'Pagination dropped' })
|
|
137
|
-
|
|
180
|
+
snapshotsToDelete.push({ id: row.id, destination: row.destination })
|
|
181
|
+
departingIds.add(row.id)
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
// 2c'. Destination moved — a SURVIVING entity no longer produces a
|
|
186
|
+
// destination it used to.
|
|
187
|
+
//
|
|
188
|
+
// Change a layout's `destination:` template, or flip `cleanUrls`, and the
|
|
189
|
+
// entity keeps its id and renders to a new path. Nothing before this
|
|
190
|
+
// noticed: the snapshot table is keyed by (id, destination), so recording
|
|
191
|
+
// the new render INSERTS a second row rather than replacing the first, and
|
|
192
|
+
// the old row goes on claiming the old file forever.
|
|
193
|
+
//
|
|
194
|
+
// Worse than it sounds, because the state is self-consistent and therefore
|
|
195
|
+
// silent: the stale file is still on disk, still matches the hash its own
|
|
196
|
+
// render recorded, so --audit-output reports OK — 0 missing, 0 orphaned —
|
|
197
|
+
// while the site serves a page the project no longer produces. Verified on
|
|
198
|
+
// a two-build fixture: one entity, two snapshots, two files, green audit.
|
|
199
|
+
//
|
|
200
|
+
// The comparison is against the SET of destinations the id claimed this
|
|
201
|
+
// cycle, never a single value. One entity can legitimately claim several —
|
|
202
|
+
// one per matched layout — and taking the last one to render as "the"
|
|
203
|
+
// destination would delete the others' output on every build. See
|
|
204
|
+
// claimedByRenderTasks above for why a skipped or failed task counts as a
|
|
205
|
+
// claim.
|
|
206
|
+
//
|
|
207
|
+
// Two ways in, and the difference is who observed what.
|
|
208
|
+
//
|
|
209
|
+
// An entity with render tasks is compared against what they claimed —
|
|
210
|
+
// that is the moved-destination case above.
|
|
211
|
+
//
|
|
212
|
+
// An entity that produces nothing at all has no tasks to compare against,
|
|
213
|
+
// so it cannot be recognised from here: an asset whose preset threw looks
|
|
214
|
+
// exactly the same, and pruning on that resemblance deletes the good
|
|
215
|
+
// derivative a failed preset exists to keep. So the dispatcher says it
|
|
216
|
+
// outright, through manifest.recordNoOutput, and an empty claim set means
|
|
217
|
+
// every destination the entity used to hold is stale. Reported by
|
|
218
|
+
// mikser-io-layouts when an entity it dispatched matched no layout; the
|
|
219
|
+
// call is general, and any other dispatcher can make it.
|
|
220
|
+
const claims = new Map(claimedByRenderTasks)
|
|
221
|
+
for (const id of m._noOutputIds) {
|
|
222
|
+
// A task contradicts the report — something did render, so believe
|
|
223
|
+
// what happened over what was predicted.
|
|
224
|
+
if (!claims.has(id)) claims.set(id, new Set())
|
|
225
|
+
}
|
|
226
|
+
m._noOutputIds.clear()
|
|
227
|
+
|
|
228
|
+
for (const [id, claimed] of claims) {
|
|
229
|
+
if (deleted.has(id)) continue
|
|
230
|
+
// An entity that renders nothing takes its paginated children with
|
|
231
|
+
// it. There are no tasks left to own them, and 2b/2c reach children
|
|
232
|
+
// only when a render told them this cycle's page count — so for this
|
|
233
|
+
// entity they never fire, and the children would be left claiming
|
|
234
|
+
// files nothing produces. Where tasks DID run, children stay with
|
|
235
|
+
// 2b/2c, which know about shrinking in a way this pass does not.
|
|
236
|
+
const rows = claimed.size
|
|
237
|
+
? m._stmtDestinationsById.all(id)
|
|
238
|
+
: m._stmtSelectByIdOrParent.all(id, id)
|
|
239
|
+
for (const row of rows) {
|
|
240
|
+
if (claimed.has(row.destination)) continue
|
|
241
|
+
if (claimed.size && row.parent) continue
|
|
242
|
+
filesToUnlink.push({
|
|
243
|
+
destination: row.destination,
|
|
244
|
+
reason: claimed.size ? 'Destination moved' : 'Entity renders nothing',
|
|
245
|
+
})
|
|
246
|
+
snapshotsToDelete.push({ id: row.id, destination: row.destination })
|
|
247
|
+
if (row.parent) {
|
|
248
|
+
// A child is genuinely gone, not moved — its id departs.
|
|
249
|
+
departingIds.add(row.id)
|
|
250
|
+
} else {
|
|
251
|
+
droppedClaims.add(`${row.id}\t${row.destination}`)
|
|
252
|
+
}
|
|
138
253
|
}
|
|
139
254
|
}
|
|
140
255
|
|
|
141
|
-
//
|
|
142
|
-
// paginated children
|
|
256
|
+
// Ids whose snapshots this pass removes ENTIRELY: deleted entities and
|
|
257
|
+
// their paginated children. Not the moved claims — those belong to
|
|
258
|
+
// entities that are still here, and `droppedClaims` carries them at
|
|
259
|
+
// (id, destination) granularity instead.
|
|
143
260
|
const goingAway = new Set(deleted)
|
|
144
|
-
for (const
|
|
261
|
+
for (const id of departingIds) goingAway.add(id)
|
|
145
262
|
for (const parentId of deleted) {
|
|
146
263
|
for (const row of m._stmtSelectByParent.all(parentId)) goingAway.add(row.id)
|
|
147
264
|
}
|
|
@@ -194,7 +311,9 @@ onFinalize(async () => {
|
|
|
194
311
|
// snapshot as a claimant would keep every shrunk page on disk
|
|
195
312
|
// forever.
|
|
196
313
|
const stillClaimed = m._stmtSelectByDestination.all(destination)
|
|
197
|
-
.filter(row => row.id !== undefined
|
|
314
|
+
.filter(row => row.id !== undefined
|
|
315
|
+
&& !goingAway.has(row.id)
|
|
316
|
+
&& !droppedClaims.has(`${row.id}\t${destination}`))
|
|
198
317
|
if (stillClaimed.length) {
|
|
199
318
|
// Keep the file — deleting a live page's output is worse than any
|
|
200
319
|
// staleness — but do NOT let the state go quiet. The bytes on
|
|
@@ -251,7 +370,7 @@ onFinalize(async () => {
|
|
|
251
370
|
m._stmtDeleteByIdOrParent.run(id, id)
|
|
252
371
|
}
|
|
253
372
|
// 3b. Pagination children cleanup.
|
|
254
|
-
for (const { id, destination } of
|
|
373
|
+
for (const { id, destination } of snapshotsToDelete) {
|
|
255
374
|
m._stmtDeleteByPK.run(id, destination)
|
|
256
375
|
}
|
|
257
376
|
// 3c. Record successful renders.
|
package/src/manifest/index.js
CHANGED
|
@@ -84,6 +84,7 @@ export function createManifest(db) {
|
|
|
84
84
|
stmtSelectByIdOrParent,
|
|
85
85
|
stmtDeleteByIdOrParent,
|
|
86
86
|
stmtSelectByParent,
|
|
87
|
+
stmtDestinationsById,
|
|
87
88
|
stmtSelectAll,
|
|
88
89
|
stmtCount,
|
|
89
90
|
stmtEntityInputHashes,
|
|
@@ -92,6 +93,10 @@ export function createManifest(db) {
|
|
|
92
93
|
stmtSnapshotsWithLayout,
|
|
93
94
|
edgeCandidates,
|
|
94
95
|
} = prepareStatements(db)
|
|
96
|
+
// Ids a dispatcher reported as producing nothing this cycle. Owned by
|
|
97
|
+
// this instance rather than the module, so a test that builds its own
|
|
98
|
+
// manifest gets its own set. Drained and cleared by onFinalize.
|
|
99
|
+
const noOutputIds = new Set()
|
|
95
100
|
const manifest = {
|
|
96
101
|
// Look up a previously-recorded entry by entity (or by an
|
|
97
102
|
// object with `{id, destination}`). Returns the snapshot, or
|
|
@@ -548,6 +553,29 @@ export function createManifest(db) {
|
|
|
548
553
|
},
|
|
549
554
|
|
|
550
555
|
// Drop all snapshots owned by entity id (direct outputs and any
|
|
556
|
+
// Report that a dispatcher looked at this entity and found nothing
|
|
557
|
+
// to render it with — no layout matched, no preset claimed it.
|
|
558
|
+
//
|
|
559
|
+
// The distinction this exists to draw is between "produced no output
|
|
560
|
+
// this cycle" and "no longer produces output at all". They are
|
|
561
|
+
// indistinguishable from inside the manifest: an asset whose preset
|
|
562
|
+
// threw and an entity whose `layout:` was removed both arrive with no
|
|
563
|
+
// successful render and no destination on their catalog row, and
|
|
564
|
+
// guessing from that signal deletes the good derivative a failed
|
|
565
|
+
// preset is explicitly meant to keep. Only the dispatcher knows which
|
|
566
|
+
// it is, so only the dispatcher can say.
|
|
567
|
+
//
|
|
568
|
+
// Recorded, not applied. The removal joins onFinalize's single
|
|
569
|
+
// transaction and its unlink path, so it gets the same
|
|
570
|
+
// still-claimed-by-a-survivor guard as every other cleanup — a
|
|
571
|
+
// destination two entities write is not this one's to take away.
|
|
572
|
+
//
|
|
573
|
+
// Per cycle: onFinalize clears the set after draining it. Saying it
|
|
574
|
+
// twice for one entity is harmless.
|
|
575
|
+
recordNoOutput(id) {
|
|
576
|
+
if (id) noOutputIds.add(id)
|
|
577
|
+
},
|
|
578
|
+
|
|
551
579
|
// paginated children whose `parent` is set to this id). Returns
|
|
552
580
|
// the destinations that were removed so callers can unlink the
|
|
553
581
|
// corresponding files when desired. Two queries (SELECT for
|
|
@@ -794,6 +822,8 @@ export function createManifest(db) {
|
|
|
794
822
|
_stmtSelectByIdOrParent: stmtSelectByIdOrParent,
|
|
795
823
|
_stmtDeleteByIdOrParent: stmtDeleteByIdOrParent,
|
|
796
824
|
_stmtSelectByParent: stmtSelectByParent,
|
|
825
|
+
_noOutputIds: noOutputIds,
|
|
826
|
+
_stmtDestinationsById: stmtDestinationsById,
|
|
797
827
|
_stmtSelectByDestination: stmtSelectByDestination,
|
|
798
828
|
_stmtDeleteByDestination: stmtDeleteByDestination,
|
|
799
829
|
_stmtDeleteByPK: stmtDeleteByPK,
|
|
@@ -82,6 +82,13 @@ export function prepareStatements(db) {
|
|
|
82
82
|
const stmtSelectByParent = db.prepare(`
|
|
83
83
|
SELECT id, destination FROM mikser_snapshots WHERE parent = ?
|
|
84
84
|
`)
|
|
85
|
+
// Every destination one entity claims. An entity legitimately claims
|
|
86
|
+
// several — one per matched layout, and one per page when paginated — so
|
|
87
|
+
// "the destinations this id produced" is a SET, never a single value, and
|
|
88
|
+
// that is the whole reason this query exists separately from stmtLookup.
|
|
89
|
+
const stmtDestinationsById = db.prepare(`
|
|
90
|
+
SELECT id, destination, parent FROM mikser_snapshots WHERE id = ?
|
|
91
|
+
`)
|
|
85
92
|
const stmtSelectAll = db.prepare(`
|
|
86
93
|
SELECT id, destination, inputHash, inputParts, outputHash, refClosure, metaReads, consumedReads, renderedAt, parent
|
|
87
94
|
FROM mikser_snapshots
|
|
@@ -179,6 +186,7 @@ export function prepareStatements(db) {
|
|
|
179
186
|
stmtSelectByIdOrParent,
|
|
180
187
|
stmtDeleteByIdOrParent,
|
|
181
188
|
stmtSelectByParent,
|
|
189
|
+
stmtDestinationsById,
|
|
182
190
|
stmtSelectAll,
|
|
183
191
|
stmtCount,
|
|
184
192
|
stmtEntityInputHashes,
|
package/src/plugins/api.js
CHANGED
|
@@ -1036,7 +1036,7 @@ export function api(options = {}) {
|
|
|
1036
1036
|
// straight to render(entity, options). Defaults match
|
|
1037
1037
|
// mikser's lifecycle (save and keep the catalog row);
|
|
1038
1038
|
// strict opt-outs via the literal `false`:
|
|
1039
|
-
// options.catalog: false →
|
|
1039
|
+
// options.catalog: false → leave the catalog as found
|
|
1040
1040
|
// options.save: false → skip the final disk write
|
|
1041
1041
|
// (bytes still in the response)
|
|
1042
1042
|
const { options = {}, ...entityShape } = req.body
|
package/src/render.js
CHANGED
|
@@ -506,13 +506,16 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
|
|
|
506
506
|
* Two control flags mirror mikser's default-keep-everything behavior;
|
|
507
507
|
* both opt-out via strict `=== false`:
|
|
508
508
|
*
|
|
509
|
-
* - `catalog: true` (default) —
|
|
510
|
-
* the
|
|
511
|
-
*
|
|
512
|
-
*
|
|
513
|
-
*
|
|
514
|
-
*
|
|
515
|
-
* `save
|
|
509
|
+
* - `catalog: true` (default) — pass `catalog: false` and the catalog
|
|
510
|
+
* ends the call exactly as it began it: the row that was there is put
|
|
511
|
+
* back as it was, and a row this render created is removed. The render
|
|
512
|
+
* still travels the lifecycle, so the caller's copy — usually altered,
|
|
513
|
+
* since an on-demand surface forces its own layout onto it — does reach
|
|
514
|
+
* the row while the render needs it; it just does not outlive the call.
|
|
515
|
+
* Combines with either `save`, which is the point: gpoint-api renders
|
|
516
|
+
* its emails with `save: true, catalog: false` — the file is wanted, the
|
|
517
|
+
* row is not. `save: false` implies it, since a render that keeps
|
|
518
|
+
* nothing on disk has no business moving a row either.
|
|
516
519
|
* - `save: true` (default) — write the rendered output to disk at
|
|
517
520
|
* `<outputFolder>/<entity.destination>`. Pass `save: false` to
|
|
518
521
|
* skip the final disk write; the bytes still come back via
|
|
@@ -539,31 +542,40 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
|
|
|
539
542
|
* @param {object} entity - any entity-shaped object
|
|
540
543
|
* @param {object} [opts]
|
|
541
544
|
* @param {number} [opts.timeout] - override the default timeout
|
|
542
|
-
* @param {boolean} [opts.catalog=true] -
|
|
545
|
+
* @param {boolean} [opts.catalog=true] - let this render's changes reach
|
|
546
|
+
* the catalog; false restores it
|
|
543
547
|
* @param {boolean} [opts.save=true] - write the rendered output to disk
|
|
544
548
|
* @returns {Promise<{output, entity}>}
|
|
545
549
|
*/
|
|
546
550
|
async function render(entity, { timeout = defaultTimeout, catalog = true, save = true } = {}) {
|
|
547
|
-
//
|
|
548
|
-
// false` means "do not leave a row behind", and that is only the
|
|
549
|
-
// render's to decide for a row the render created. Asked here, before
|
|
550
|
-
// the render puts one there.
|
|
551
|
+
// What the catalog held before this render touched it.
|
|
551
552
|
//
|
|
552
|
-
//
|
|
553
|
-
//
|
|
554
|
-
//
|
|
555
|
-
//
|
|
556
|
-
//
|
|
557
|
-
//
|
|
558
|
-
// Imported
|
|
559
|
-
// module, and a static import makes that cycle load-bearing at
|
|
553
|
+
// Read BEFORE dispatch, because the render itself overwrites it: the
|
|
554
|
+
// entity has to travel the lifecycle to render at all, and the
|
|
555
|
+
// pipeline reads the row back to resolve the layout, so suppressing
|
|
556
|
+
// that write outright fails the render with "requested layout X but
|
|
557
|
+
// produced no output". The row is written, used, and then put back.
|
|
558
|
+
//
|
|
559
|
+
// Imported here rather than at the top: catalog.js reaches back into
|
|
560
|
+
// this module, and a static import makes that cycle load-bearing at
|
|
560
561
|
// module-evaluation time — it surfaced as "Cannot access 'schemas'
|
|
561
562
|
// before initialization" in three unrelated test files.
|
|
562
|
-
const
|
|
563
|
-
|
|
564
|
-
|
|
563
|
+
const neutral = save === false || catalog === false
|
|
564
|
+
// Restoring is about a ROW, so it needs an id; the flag itself does
|
|
565
|
+
// not, and the manifest reads the flag to decide whether to record a
|
|
566
|
+
// snapshot. Keeping the two apart means an id-less entity — which has
|
|
567
|
+
// no row to put back — still gets the rest of neutrality.
|
|
568
|
+
let priorRow = null
|
|
569
|
+
if (neutral && entity?.id) {
|
|
570
|
+
const { findById, markNeutralRender } = await import('./catalog.js')
|
|
571
|
+
priorRow = findById(entity.id)
|
|
572
|
+
// Handed over before dispatch, so it is in place however early the
|
|
573
|
+
// cycle starts. The catalog's finalize drain is where the row goes
|
|
574
|
+
// back for good — see applyJournalMutations.
|
|
575
|
+
markNeutralRender(entity.id, priorRow)
|
|
576
|
+
}
|
|
565
577
|
|
|
566
|
-
const
|
|
578
|
+
const dispatched = new Promise((resolve, reject) => {
|
|
567
579
|
const correlationId = randomUUID()
|
|
568
580
|
// Engine-set fields live under entity.options. The caller's
|
|
569
581
|
// render(entity, { save: false }) becomes
|
|
@@ -580,6 +592,10 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
|
|
|
580
592
|
...entity.options,
|
|
581
593
|
correlationId,
|
|
582
594
|
...(save === false ? { save: false } : {}),
|
|
595
|
+
// Marks every journal entry this render produces, however
|
|
596
|
+
// many phases later they arrive, as belonging to a
|
|
597
|
+
// catalog-neutral call.
|
|
598
|
+
...(neutral ? { neutral: true } : {}),
|
|
583
599
|
},
|
|
584
600
|
}
|
|
585
601
|
pending.push({
|
|
@@ -593,32 +609,28 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
|
|
|
593
609
|
if (!cycleRunning) setImmediate(runBatch)
|
|
594
610
|
})
|
|
595
611
|
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
//
|
|
601
|
-
//
|
|
602
|
-
//
|
|
603
|
-
//
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
//
|
|
609
|
-
//
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
await
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
'render: catalog:false ignored for %s — it needs save:false, ' +
|
|
619
|
-
'because pruning the row also unlinks the rendered output',
|
|
620
|
-
result.entity.id,
|
|
621
|
-
)
|
|
612
|
+
let result
|
|
613
|
+
try {
|
|
614
|
+
result = await dispatched
|
|
615
|
+
} finally {
|
|
616
|
+
// In a finally, because a render that THROWS has still altered the
|
|
617
|
+
// row on its way through — an unrenderable entity leaves the
|
|
618
|
+
// catalog holding the caller's copy of it, which is the same
|
|
619
|
+
// damage as a successful render leaving its layout behind.
|
|
620
|
+
if (neutral && entity?.id) {
|
|
621
|
+
// Put the row back for whoever is awaiting us. The finalize
|
|
622
|
+
// drain does it again, and has to: it runs after the render and
|
|
623
|
+
// would otherwise re-apply the altered copy over this one. Both
|
|
624
|
+
// restore the same prior row, so their order does not matter.
|
|
625
|
+
//
|
|
626
|
+
// Deliberately not journaled. A journal entry would dispatch
|
|
627
|
+
// another render, and a journaled DELETE drags the manifest's
|
|
628
|
+
// file cleanup along with it — which would unlink the very file
|
|
629
|
+
// a `save: true, catalog: false` render was asked to produce.
|
|
630
|
+
// That coupling is why this used to refuse `save: true`.
|
|
631
|
+
const { restoreEntity, settleNeutralRender } = await import('./catalog.js')
|
|
632
|
+
restoreEntity(entity.id, priorRow)
|
|
633
|
+
settleNeutralRender(entity.id)
|
|
622
634
|
}
|
|
623
635
|
}
|
|
624
636
|
|