mikser-io 6.27.0 → 7.6.0

package/README.md

@@ -6,7 +6,7 @@
 
 **Mikser is the content layer of your application.** Business logic, user accounts, transactions live in their own services; mikser handles the parts that *are* content — pages, docs, the published catalog, multi-format outputs. [Vue](https://github.com/almero-digital-marketing/mikser-io-sdk-vue), [React](https://github.com/almero-digital-marketing/mikser-io-sdk-react), and [Svelte](https://github.com/almero-digital-marketing/mikser-io-sdk-svelte) SDKs are the seam between them — same surface across all three (`useDocument`, `useDocuments`, multilingual `useHref`, live SSE updates), each in its framework's idiomatic shape.
 
-Built for Node.js around a strict lifecycle, a composable plugin system, and direct control over every output. Every document, asset, and template flows through the same deterministic pipeline. Plugins hook in at any phase; nothing runs outside the cycle. It scales from a single markdown blog to a multi-language, multi-format publishing platform with image / video / AI pipelines, live SSE-driven editors, semantic search, and typed frontend contracts — and stays predictable in both directions.
+Built for Node.js around a strict lifecycle and a composable plugin system. Every document, asset, and template flows through the same pipeline; plugins hook in at any phase. The same engine runs a single markdown blog and a multi-language publishing platform with PDF / email / AI-augmented asset pipelines — same lifecycle, more plugins.
 
 It's MIT-licensed, runs on Node 18+, has zero hosted dependencies, and the entire content tree it manages is a folder of `.md` and `.yml` files you can copy, diff, and version-control. **The portability promise is the architecture, not a feature.**
 
@@ -30,25 +30,27 @@ Build mikser into the parts of your application that are content-shaped. Keep th
 
 **Your content stays yours.** Source files live on disk as `.md`, `.yml`, `.html` with YAML front-matter. The build output is plain static files. No database lock-in, no proprietary export format. The whole content tree is copyable, diffable, and version-controllable with git — your site is portable on day one and on year ten.
 
+**Static-first with a built-in live channel.** Most content engines pick one side: static-site generators (Hugo, Eleventy, Jekyll) are fast but rebuild-only; headless CMSes (Sanity, Contentful, Strapi) are live but every page is an API round-trip. Mikser composes both — content publishes as static files by default (fast first paint, no API on the happy path), and the live channel arrives on top, so edits show up in connected clients without a refresh and without losing the static advantage.
+
 **Incremental builds that scale.** Mikser tracks every entity in a journal. When a file changes, only the affected entities re-process — not the whole site graph. On 10k+ documents this dramatically outpaces tools that rebuild more on every change.
 
 **Concurrent rendering.** Renders fan out across a worker pool that keeps every CPU core hot. Multi-format outputs (HTML, PDF, MJML email, etc.) generate in parallel from the same source.
 
-**Image, video, and AI pipelines authored as plugins — not configured.** The `assets` plugin runs user-written preset modules over binary inputs. A preset is a plain Node module: ~10 lines around `sharp` resize an image, ~10 lines around `fluent-ffmpeg` transcode a video, ~30 lines around the Replicate API run an AI upscaler. The pipeline isn't a fixed menu of operations — it's whatever Node can call. Most SSGs cap your asset processing at "resize and convert format." Mikser caps it at "what can Node do." Compose with the `resources` plugin and your DAM, CDN, or company content server becomes an upstream input — uploaded files end up transcoded, watermarked, AI-enhanced, and deployed without manual handoff. See [the assets / resources docs](./documentation/plugins.md#assets) for end-to-end examples.
+**Asset pipelines are whatever Node can do.** Most static frameworks (Astro, Next.js, Hugo) ship image optimization and stop there — video transcoding, AI upscaling, watermarking all need a separate service. Mikser runs user-written modules over binary inputs: ~10 lines around `sharp` resize an image, ~10 around `fluent-ffmpeg` transcode a video, ~30 around the Replicate API upscale with AI. Anything an npm package can do, your pipeline can do — including pulling uploads from a DAM or CDN through the same flow.
 
 **One lifecycle, everything composes.** Plugins hook into 20+ named lifecycle phases. A search-indexing plugin shares the same journal iteration as an email-rendering plugin and a PDF-postprocessing plugin — no glue code, no orchestration layer. The engine doesn't know which plugins are loaded; plugins don't have to know about each other.
 
 **Run anywhere.** The same CLI handles one-shot builds, watch-mode dev loops, and a long-running HTTP server with a shared Express app. `npx mikser` ships a static site; `mikser --watch` is the dev loop; `mikser --server` exposes a live admin/API.
 
-**Survives backend outages.** The `api` plugin's [per-query disk cache](./documentation/caching.md) writes every cacheable list response to `out/` keyed by the request URL. A stock-nginx reverse proxy can fail over to the cached file when mikser is unreachable — same URL, transparent to the client, no Lua or extra modules required. Production frontends keep rendering routes during deploys, brief outages, and the upstream-blip-of-the-week. Opt in per endpoint with `cache: true`; see [Caching and reverse-proxy failover](./documentation/caching.md) for the working config.
+**Outages don't take you down.** Headless CMSes (Contentful, Sanity, Strapi) treat the API as the source of truth — when it blinks, every frontend errors out. Mikser inverts that: reads become static files on disk, the live channel layers on top. A reverse proxy keeps serving the files when mikser blips. Visitors don't notice; live updates pause until mikser returns.
 
-**Library mode.** Mikser is also a library. `useRenderer`, `useCollection`, `findSimilar`, and direct lifecycle hooks let you embed the engine inside an existing Node app instead of running it as a CLI.
+**Library mode.** Mikser is also a library. `useRenderer`, `useCollection`, and direct lifecycle hooks let you embed the engine inside an existing Node app instead of running it as a CLI — plugins like `vector` add their own primitives the same way.
 
 **Open source.** MIT-licensed, on GitHub, no telemetry, no auth wall, no SaaS dependency. What you see is what runs.
 
 ## Built for AI-assisted development
 
-Files-as-source isn't just a portability story — it makes the project unusually friendly to AI coding agents. Most setups lose time configuring an agent's access to the data: tokens, schemas, MCP servers, sandboxed query environments. Mikser sidesteps all of it because the content is text files the agent can already read with the tools it already has.
+Files-as-source isn't just a portability story — it makes the project unusually friendly to AI coding agents. There's a static-time half (the agent reads your tree the way it reads any repo — no DB connection, no schema upload, no sandboxed query layer to learn) and a runtime half (when the agent needs to write or render, mikser ships its own MCP server in core, so it talks to the live engine instead of a parallel REST shim you have to maintain).
 
 **Zero infra friction for discovery.** An agent can `rg "type: product"` across the content tree to find every product doc in a second. No DB connection, no API token, no schema file to parse.
 
@@ -60,10 +62,39 @@ Files-as-source isn't just a portability story — it makes the project unusuall
 
 **Plugin-by-example.** Authoring a new plugin? There are 15+ existing ones in the same shape to pattern-match against. Convention is dense enough that new plugins look like the old ones without coaching.
 
-**One-shot bootstrap via Claude Code.** The [`mikser-io-claude-plugin`](https://github.com/almero-digital-marketing/mikser-io-claude-plugin) wraps the setup story above into a single skill. Register the repo as a Claude Code marketplace and install the plugin (`/plugin marketplace add almero-digital-marketing/mikser-io-claude-plugin` then `/plugin install mikser-io-claude-plugin@mikser-io`), then in any Vue 3, React, or SvelteKit project — or in a blank directory — say "add mikser to this app." It detects the framework (or scaffolds a fresh starter via `create-vite` / `sv create`), wires the matching framework SDK, composes with your existing router rather than replacing it, and optionally lays down a `mikser-content/` sibling folder with Zod schemas and starter documents so the backend works on first run.
+**One-shot bootstrap via Claude Code.** The [`mikser-io-claude-plugin`](https://github.com/almero-digital-marketing/mikser-io-claude-plugin) wraps the whole setup into a single skill. Register the marketplace, install once:
+
+```
+/plugin marketplace add almero-digital-marketing/mikser-io-claude-plugin
+/plugin install mikser-io-claude-plugin@mikser-io
+```
+
+…then in any Vue 3, React, or SvelteKit project — or in a blank directory — say *"add mikser to this app."* It detects the framework (or scaffolds a fresh starter via `create-vite` / `sv create`), wires the matching framework SDK without replacing your router, and optionally drops a `mikser-content/` sibling folder with Zod schemas and starter documents so the backend works on first run.
+
+The runtime half — the agent driving the live engine, not just reading the tree — gets its own section below.
 
 The honest caveat: this advantage is real on **content-shaped work** — adding pages, restructuring collections, generating new layouts, building frontends. It doesn't make mikser better for non-content tasks (concurrency bugs in the worker pool, database tuning elsewhere in your stack); those are plain Node debugging like anywhere else. The visibility advantage also degrades past ~10k documents — at that scale the agent queries via the SDK instead of grepping the tree, which is still good but less "see everything at once."
 
+## Control mikser from your AI agent
+
+Add `--mcp` to your mikser command and any MCP-speaking client — Claude Desktop, Claude Code, ChatGPT, custom agents — connects to the running engine. From inside a chat, your AI can:
+
+- read every entity in the catalog
+- write new content files (markdown, layouts, configuration) — writes land on disk and the next cycle picks them up
+- render any layout for preview without touching the output folder
+- watch every build log as it streams past
+- introspect engine state — current lifecycle phase, effective config, recent log buffer
+
+Plugins extend the tool surface the same way they mount HTTP routes; install the plugin, the agent gets new verbs. No glue code, no per-project agent wiring.
+
+```bash
+mikser --server --mcp           # mounts MCP at /mcp on the same port as --server
+```
+
+What that feels like in practice: *"draft three hero-section variants and show me previews"* — three layouts written, three previews returned inline, one chat turn. *"Why did the build break?"* — the agent reads the rolling log buffer and answers from the same view your terminal sees. Operator, AI, and any observer dashboard share the same engine because mikser is single-tenant by design.
+
+Full tool reference and twelve worked scenarios in [MCP — talking to mikser from AI](./documentation/mcp.md).
+
 ## Plugins on top of the engine
 
 The engine is what stays stable — the lifecycle, the catalog, the file-based content model. Plugins are independent npm packages sitting on the plugin API: some are essential to the SSG workflow, some give external systems HTTP access to the catalog, some are integrations that earn their keep on real projects, and some are probes that test how far the lifecycle stretches without touching the core. Install what a project needs; drop what it doesn't.
@@ -85,6 +116,7 @@ The engine is what stays stable — the lifecycle, the catalog, the file-based c
 |---|---|
 | `data` | JSON snapshots of entities / context / catalog, written to disk for static serving |
 | `api` | REST endpoints with sift-backed queries, per-endpoint tokens, optional render, opt-in [per-query disk cache](./documentation/caching.md) for reverse-proxy failover |
+| `preview` | In-memory render cache + `GET /preview/:filename` route. Companion to the [`mikser_preview`](./documentation/mcp.md) MCP tool — transient render bytes served at a clickable URL, no filesystem footprint |
 
 **Integrations:**
 
@@ -92,7 +124,10 @@ The engine is what stays stable — the lifecycle, the catalog, the file-based c
 |---|---|
 | [`mikser-io-vector`](https://github.com/almero-digital-marketing/mikser-io-vector) | OpenAI embeddings + semantic search (sqlite-vec or pgvector) |
 | [`mikser-io-plugin-schemas`](https://github.com/almero-digital-marketing/mikser-io-plugin-schemas) | Zod-backed entity validation + auto-generated TypeScript declarations for the SDK |
-| `archive`, `mapper`, `live`, `aml` | Specialty integrations |
+| [`mikser-io-archive`](https://github.com/almero-digital-marketing/mikser-io-archive) | Persist matching entities to YAML — audit trail, versioned content history, downstream export |
+| `mapper` | Run config-supplied transforms over matched entities each cycle (in-core, generic transformation layer) |
+| [`mikser-io-live`](https://github.com/almero-digital-marketing/mikser-io-live) | Lightweight dev server with browser auto-refresh — pair with `--watch` for the classic save→reload loop |
+| [`mikser-io-aml`](https://github.com/almero-digital-marketing/mikser-io-aml) | Parse [ArchieML](https://archieml.org/) (the NYT/ProPublica format) into `entity.meta` for non-technical authors |
 
 **Integration probes** — wrap a substantial external project as a plugin to confirm the lifecycle is open enough to host it without core changes. Treat these as feasibility evidence, not as a statement about where mikser is heading:
 
@@ -127,22 +162,14 @@ Each SDK ships TypeScript declarations so client projects get autocomplete on fi
 npm install mikser-io
 ```
 
-```js
-// mikser.config.js
-export default {
-	plugins: ['documents', 'layouts'],
-	layouts: {
-		cleanUrls: true,
-	},
-}
-```
-
 ```bash
 npx mikser              # one-shot build
 npx mikser --watch      # incremental dev loop
 npx mikser --server     # build + serve at :3001
 ```
 
+For a working starter — config with a real plugin set, sample `documents/`, expected output — see [Getting Started](./documentation/getting-started.md). Or skip straight to "add mikser to this app" via the [Claude Code plugin](#built-for-ai-assisted-development) above.
+
 ## Core Concepts
 
 - **Lifecycle** — Processing runs through fixed phases: initialize → load → import → process → persist → render → finalize. Plugins hook into any phase.
@@ -166,12 +193,18 @@ The shape mikser **doesn't** fit cleanly: anything with non-technical content au
 
 ## Engineering discipline
 
-A few things this project takes seriously:
+What you get from how this project is built:
+
+- **Every load-bearing decision has an ADR.** The [`decisions/`](./documentation/decisions/) folder names which choices are structural — files-as-source, journal+catalog split, plugin-as-factory, when something goes in core vs. ships as a plugin — and explains what protects them. When you push against one, there's a written answer waiting instead of folklore.
+- **Engine stays small; capability ships in plugins.** The 15+ plugin ecosystem adds features without core changes, so your upgrade cost stays low. Probes like `decap` (a full third-party CMS mounted in ~150 lines, zero engine changes) are deliberate evidence the extension model holds where it counts.
+- **Builds are deterministic; no async middleware layer.** The journal is the only synchronization primitive — no event bus, no IoC container, no orchestrator running plugins in surprising order. The lifecycle is a list of named phases; "what ran when?" has an answer you can read off the source.
+- **The whole engine is one read.** The [Architecture Overview](./documentation/overview.md) walks the full pipeline top to bottom. Onboarding a new engineer is an afternoon, not a tour through fifteen reference docs.
+
+## Acknowledgments
+
+The earliest version of mikser was inspired by [DocPad](https://github.com/docpad/docpad) (Benjamin Lupton, with Michael Duane Mooring and Rob Loach). DocPad's "freeway, not a box" philosophy — files on disk, any pre-processor or template engine, plugin-by-convention extension — shaped how mikser started.
 
-- **ADRs for load-bearing decisions.** The [`decisions/`](./documentation/decisions/) folder names which choices are structural — files-as-source, journal+catalog split, plugin-as-factory, compose-via-protocols — and explains what protects them. Read those before proposing a feature that pushes against one.
-- **Engine stability, plugin churn.** The 15+ plugins in the ecosystem add capability without core changes. The integration probes (e.g. `decap`, mounting a third-party CMS in ~150 lines) are deliberate evidence that the extension model holds.
-- **Deterministic builds.** The journal is the synchronization primitive. There's no event-passing layer, no IoC container, no plugin orchestrator. The engine knows how to fire phases in order; everything else falls out of that.
-- **The mental model is one document.** The [Architecture Overview](./documentation/overview.md) is one read for the full top-to-bottom picture. The reference docs exist for lookup; the overview exists for comprehension.
+Mikser itself has a previous chapter: the [legacy 7.x line](https://github.com/almero-digital-marketing/mikser) (last release 2022) introduced the real-time SSG model the current engine still carries forward. The redesign dropped MongoDB (the catalog lives in-process now, not in a database), modernized to Node 18+ ESM with a structured 20-phase lifecycle, added the live SSE channel that powers the framework SDKs, and replaced cluster-based rendering with an async worker pool. Same intent — content as files, real-time previews, multi-format output at scale — clearer foundations.
 
 ## Documentation Index
 
@@ -185,6 +218,7 @@ A few things this project takes seriously:
 | [Entities](./documentation/entities.md)               | Users & Developers | Entity model, operations, journal, catalog         |
 | [Rendering](./documentation/rendering.md)             | Users & Developers | Render pipeline, render plugins, render modes      |
 | [Watch Mode](./documentation/watch-mode.md)           | Users              | File watching, scheduled tasks, incremental builds |
+| [MCP](./documentation/mcp.md)                         | Users              | The `--mcp` server — tool surface, `mikser://` resources, twelve worked AI-driven scenarios |
 | [Caching](./documentation/caching.md)                 | Users (production) | The `cache: true` disk cache + working nginx config for reverse-proxy failover |
 | [Architecture](./documentation/architecture.md)       | Developers         | Module-level reference — what's in each file       |
 | [API Reference](./documentation/api-reference.md)     | Developers         | Complete public API reference                      |

package/index.js

@@ -9,4 +9,6 @@ export * from './src/plugins.js'
 export * from './src/manager.js'
 export * from './src/tracking.js'
 export * from './src/engine.js'
-export * from './src/api.js'
+export * from './src/api.js'
+export * from './src/source.js'
+export * from './src/mcp.js'

package/ngrok.ymlt

@@ -0,0 +1,3 @@
+version: 3
+agent:
+  authtoken: 3EcICPaDtSN54h63Y5v8g2zKOJ9_5oMnCdXyPmMmKVXH9ba1c

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "mikser-io",
-	"version": "6.27.0",
+	"version": "7.6.0",
 	"description": "<p align=\"center\">   <img src=\"mikser-lockup-stacked.svg\" alt=\"mikser\" width=\"198\" /> </p>",
 	"main": "index.js",
 	"scripts": {
@@ -20,13 +20,14 @@
 	"author": "",
 	"license": "ISC",
 	"dependencies": {
-		"@budibase/handlebars-helpers": "^0.14.2",
+		"@budibase/handlebars-helpers": "^0.14.3",
+		"@modelcontextprotocol/sdk": "^1.29.0",
 		"await-semaphore": "^0.1.3",
-		"axios": "^1.16.0",
+		"axios": "^1.17.0",
 		"chokidar": "^5.0.0",
 		"cli-progress": "^3.12.0",
-		"commander": "^14.0.3",
-		"dayjs": "^1.11.20",
+		"commander": "^15.0.0",
+		"dayjs": "^1.11.21",
 		"deepdash": "^5.3.9",
 		"escape-string-regexp": "^5.0.0",
 		"execa": "^9.6.1",
@@ -42,18 +43,18 @@
 		"minimatch": "^10.2.5",
 		"node-cron": "^4.2.1",
 		"p-map": "^7.0.4",
-		"p-queue": "^9.2.0",
+		"p-queue": "^9.3.0",
 		"pino": "^10.3.1",
 		"pino-pretty": "^13.1.3",
 		"piscina": "^5.1.4",
 		"sift": "^17.1.3",
 		"sqlite3": "^6.0.1",
 		"truncate-stream": "^1.0.2",
-		"yaml": "^2.8.4"
+		"yaml": "^2.9.0"
 	},
 	"optionalDependencies": {
 		"express": "^5.2.1",
-		"puppeteer": "^24.43.0"
+		"puppeteer": "^25.1.0"
 	},
 	"devDependencies": {
 		"fluent-ffmpeg": "^2.1.3",
@@ -62,8 +63,10 @@
 		"mikser-io-render-eta": "file:../mikser-io-render-eta",
 		"mikser-io-render-liquid": "file:../mikser-io-render-liquid",
 		"mikser-io-render-markdown": "file:../mikser-io-render-markdown",
+		"mikser-io-schemas": "file:../mikser-io-schemas",
 		"mikser-io-vector": "file:../mikser-io-vector",
-		"sharp": "^0.34.5"
+		"sharp": "^0.34.5",
+		"zod": "^4.4.3"
 	},
 	"directories": {
 		"test": "test"

package/src/api.js

@@ -37,10 +37,20 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
         const remaining = new Map(batch.map(b => [b.correlationId, b]))
         const completedHooks = runtime.hooks.completed
         const hook = async (entry) => {
-            const cid = entry.entity?._correlationId
+            const cid = entry.entity?.options?.correlationId
             if (!cid) return
             const item = remaining.get(cid)
             if (!item) return
+            // Only resolve on the FINAL completion. For postprocessor-
+            // equipped entities the engine fires runtime.complete twice:
+            // once after render (intermediate bytes, entity.origin not
+            // set) and once after postprocess (final bytes, entity.origin
+            // set to the intermediate destination). useRenderer's
+            // contract is "return the pipeline's final output", so we
+            // skip the intermediate fire and wait for the final.
+            const hasPostprocessor = entry.entity?.layout?.postprocessor
+            const isFinal = !hasPostprocessor || entry.entity?.origin != null
+            if (!isFinal) return
             remaining.delete(cid)
             clearTimeout(item.timer)
             item.resolve({ output: entry.output, entity: entry.entity })
@@ -99,13 +109,24 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
      *   skip the final disk write; the bytes still come back via
      *   `output.result` for you to pipe wherever you want (HTTP
      *   response, S3, …). For layouts with a postprocessor (e.g.
-     *   `*.html-pdf.*`), the intermediate file is still written so the
-     *   postprocessor can consume it; only the FINAL output is skipped.
+     *   `*.html-pdf.*`), the intermediate is written to a scratch path
+     *   under `runtime.options.previewFolder` (engine-owned, never
+     *   in outputFolder) so the postprocessor can consume it; only
+     *   the FINAL output is skipped from disk. `output.result` is
+     *   always the FINAL pipeline output — PDF bytes for a
+     *   `*.html-pdf.*` layout, MJML-derived HTML for
+     *   `*.html-mjml.*`, etc., not the intermediate.
      *
      * The rendered output's bytes are always returned in `output.result`
      * regardless of either flag — `save` only affects whether they also
      * end up on disk.
      *
+     * Per-entity engine state (correlation id, control flags) lives at
+     * `entity.options.*` — same noun mikser uses for engine config
+     * (`runtime.options`) and plugin params, scoped to one entity's
+     * pass through the lifecycle. Consumers should not set
+     * `entity.options.correlationId` themselves; useRenderer owns it.
+     *
      * @param {object} entity                - any entity-shaped object
      * @param {object} [opts]
      * @param {number}  [opts.timeout]       - override the default timeout
@@ -116,12 +137,25 @@ export function useRenderer(runtime, { defaultTimeout = 30_000 } = {}) {
     async function render(entity, { timeout = defaultTimeout, catalog = true, save = true } = {}) {
         const result = await new Promise((resolve, reject) => {
             const correlationId = randomUUID()
-            const stamped = { ...entity, _correlationId: correlationId }
-            // Only stamp _save when explicitly opting out — keeps the
-            // entity object clean for the common case.
-            if (save === false) stamped._save = false
+            // Engine-set fields live under entity.options. The caller's
+            // render(entity, { save: false }) becomes
+            // entity.options.save = false here — same noun mikser uses
+            // for engine config (runtime.options) and plugin params,
+            // just scoped to one entity's pass through the lifecycle.
+            //
+            // Only set `save` when explicitly opting out — leaves the
+            // entity.options as a clean { correlationId } in the common
+            // case rather than carrying a redundant save:true.
+            const prepared = {
+                ...entity,
+                options: {
+                    ...entity.options,
+                    correlationId,
+                    ...(save === false ? { save: false } : {}),
+                },
+            }
             pending.push({
-                entity: stamped,
+                entity: prepared,
                 correlationId,
                 timeout,
                 resolve,

package/src/catalog.js

@@ -1,6 +1,6 @@
 import runtime from './runtime.js'
 import { useLogger } from './engine.js'
-import { onLoaded, onPersist, onFinalized } from './lifecycle.js'
+import { onInitialized, onPersist, onFinalized } from './lifecycle.js'
 import { useJournal } from './journal.js'
 import { OPERATION } from './constants.js'
 import { Low } from 'lowdb'
@@ -10,7 +10,13 @@ import _ from 'lodash'
 
 let catalog
 
-onLoaded(async () => {
+// Same reasoning as journal.js — initialize the catalog in
+// onInitialized so every plugin hook from onLoad onwards can safely
+// call findEntity / findEntities and write through the journal.
+// catalog.js imports useJournal from journal.js, so journal.js's
+// onInitialized registers first within the phase — its hook runs
+// before this one.
+onInitialized(async () => {
 	const adapter = new JSONFile(path.join(runtime.options.runtimeFolder, `catalog.json`))
 	catalog = new Low(adapter, {
 		entities: [],
@@ -27,10 +33,21 @@ onPersist(async () => {
 				logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
 				catalog.data.entities.push(entity)
 				break
-			case OPERATION.UPDATE:
+			case OPERATION.UPDATE: {
 				logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
-				catalog.chain.get('entities').find({ id: entity.id }).assign(entity).value()
+				// Upsert semantics: if the entity doesn't already exist,
+				// treat UPDATE as CREATE. Plugins that "ensure an entity
+				// is in the catalog" can call runtime.update without a
+				// findEntity-then-branch dance. Previously a no-op when
+				// the id was new, which was a silent footgun.
+				const existing = catalog.chain.get('entities').find({ id: entity.id }).value()
+				if (existing) {
+					catalog.chain.get('entities').find({ id: entity.id }).assign(entity).value()
+				} else {
+					catalog.data.entities.push(entity)
+				}
 				break
+			}
 			case OPERATION.DELETE:
 				logger.trace('Database %s %s: %s', entity.collection, operation, entity.id)
 				catalog.chain.get('entities').remove({ id: entity.id }).value()

package/src/engine.js

@@ -49,6 +49,7 @@ export async function setup(options) {
             .option('-s --server [port]', 'start an Express server on the given port (defaults to 3001)')
             .option('--cors [origin]', 'restrict server CORS to a specific origin (default *)')
             .option('--no-cors', 'disable server CORS headers')
+            .option('--mcp [path]', 'enable MCP server (mounts at <path>, default /mcp)')
 
         Object.assign(runtime.options, options || runtime.engine.commander.parse(process.argv).opts())
         runtime.options.info = true
@@ -62,23 +63,28 @@ export async function setup(options) {
             runtime.options.info = false
         }
         runtime.engine.logger.notice = runtime.engine.logger.info
-    })
-
-    onInitialized(async () => {
-        const logger = useLogger()
 
+        // Resolve folders inside onInitialize so journal.js and
+        // catalog.js (which initialize in onInitialized) see absolute
+        // paths and a guaranteed-existing runtimeFolder. The split is
+        // deliberate: engine's onInitialize does setup that the rest of
+        // the engine infrastructure depends on; onInitialized does
+        // things plugins may need.
         runtime.options.workingFolder = path.resolve(runtime.options.workingFolder)
         process.chdir(runtime.options.workingFolder)
 
         runtime.options.runtimeFolder = path.join(runtime.options.workingFolder, runtime.options.runtimeFolder || 'runtime')
         runtime.options.outputFolder = path.join(runtime.options.workingFolder, runtime.options.outputFolder || 'out')
-
-        logger.info('Working folder: %s', runtime.options.workingFolder)
-        logger.info('Output folder: %s', runtime.options.outputFolder)
+        // Scratch path for intermediate render artifacts when a caller
+        // opted out of disk writes via render({ save: false }) but the
+        // layout has a postprocessor that still needs to read the
+        // intermediate. Lives under runtimeFolder so it's engine-owned
+        // and never appears in outputFolder.
+        runtime.options.previewFolder = path.join(runtime.options.runtimeFolder, 'preview')
 
         if (runtime.options.clear) {
             try {
-                logger.info('Clearing folders')
+                runtime.engine.logger.info('Clearing folders')
                 await rm(runtime.options.outputFolder, { recursive: true })
                 await rm(runtime.options.runtimeFolder, { recursive: true })
             } catch (err) {
@@ -87,6 +93,13 @@ export async function setup(options) {
             }
         }
         await mkdir(runtime.options.runtimeFolder, { recursive: true })
+    })
+
+    onInitialized(async () => {
+        const logger = useLogger()
+
+        logger.info('Working folder: %s', runtime.options.workingFolder)
+        logger.info('Output folder: %s', runtime.options.outputFolder)
 
         // Server bring-up: two paths, controlled by which of these are set.
         //
@@ -146,6 +159,35 @@ export async function setup(options) {
                 logger.info('CORS enabled: %s', origin)
             }
         }
+
+        // MCP substrate — same engine-provides-transport, plugins-
+        // register-tools shape as Express. See
+        // documentation/decisions/0006-when-to-add-to-core.md for
+        // the justification. The substrate object is exposed at
+        // runtime.options.mcp; plugins use it directly via the SDK
+        // (server.registerTool / server.registerResource). The
+        // transport is mounted later, after plugin routes have had
+        // a chance to register (see the onLoad below that handles
+        // static serving + listen).
+        if (runtime.options.mcp) {
+            try {
+                const { createMcpSubstrate, wireLoggerToMcp } = await import('./mcp.js')
+                runtime.options.mcpPath = typeof runtime.options.mcp === 'string'
+                    ? runtime.options.mcp
+                    : '/mcp'
+                runtime.options.mcp = createMcpSubstrate()
+
+                // Fan every engine log call out to MCP clients as
+                // `notifications/message`. Wraps in-place so the
+                // existing logger reference (held by plugins, render
+                // workers, useLogger consumers) gains the side-channel
+                // automatically — no second logger to thread through.
+                wireLoggerToMcp(runtime.engine.logger, runtime.options.mcp)
+                logger.info('MCP substrate ready (mounts at %s when server is up)', runtime.options.mcpPath)
+            } catch (err) {
+                logger.error('Failed to enable MCP: %s', err.message)
+            }
+        }
     })
 
     // Registered here (inside setup) so it runs AFTER plugins.js's onLoad
@@ -165,6 +207,15 @@ export async function setup(options) {
             const logger = useLogger()
             const { default: express } = await import('express')
 
+            // Mount MCP transport (if active) BEFORE the static
+            // catch-all so /mcp isn't swallowed. After this runs the
+            // server is fully reachable over HTTP for AI clients.
+            if (runtime.options.mcp && runtime.options.mcpPath) {
+                const { mountMcpOnExpress } = await import('./mcp.js')
+                await mountMcpOnExpress(runtime.options.app, runtime.options.mcp, runtime.options.mcpPath)
+                logger.info('MCP mounted: %s', runtime.options.mcpPath)
+            }
+
             // Serve the output folder as the catch-all static route.
             // Mounted LAST in the middleware chain so plugin routes
             // (e.g. /api/*) match first; anything that didn't match a
@@ -370,7 +421,22 @@ export async function setup(options) {
                         origin: entity.destination,
                         destination
                     },
-                    options: { postprocessor: options.postprocessor, tasks: options.tasks },
+                    options: {
+                        postprocessor: options.postprocessor,
+                        tasks: options.tasks,
+                        // When the originating render call passed
+                        // `save: false`, the layouts plugin wrote the
+                        // intermediate into runtime.options.previewFolder
+                        // rather than outputFolder. Postprocess plugins
+                        // resolve `entity.origin` against
+                        // `options.outputFolder`, so swap it here so
+                        // they look in the right place. No change for
+                        // normal builds (entity.options.save unset →
+                        // outputFolder).
+                        ...(entity.options?.save === false
+                            ? { outputFolder: runtime.options.previewFolder }
+                            : {}),
+                    },
                     context
                 })
             }

package/src/journal.js

@@ -1,5 +1,5 @@
 import runtime from './runtime.js'
-import { onLoaded, onCancelled, onFinalized } from './lifecycle.js'
+import { onInitialized, onCancelled, onFinalized } from './lifecycle.js'
 import { unlink } from 'fs/promises'
 import knex from 'knex'
 import path from 'path'
@@ -81,7 +81,15 @@ export async function clearJournal(aborted) {
     }
 }
 
-onLoaded(async () => {
+// Initialize the journal in onInitialized rather than onLoaded.
+// The engine resolves runtime.options.runtimeFolder in its own
+// onInitialized hook; ours runs after (engine.js is imported before
+// any plugin loads, so its onInitialized registers first). This way
+// every plugin hook from onLoad onwards — including plugin onLoaded
+// — can safely call runtime.create / runtime.update without
+// depending on the order in which journal.js's module registered
+// relative to theirs.
+onInitialized(async () => {
     const filename = path.join(runtime.options.runtimeFolder, `journal.db`)
     try {
         await unlink(filename)