mikser-io 8.3.9 → 9.0.1

package/10.0-PLAN.md

@@ -37,7 +37,7 @@ When the working signals dominate over a sustained period of real use, v10 desig
 
 ## Why v10 (not 9.x)
 
-The 9.0 plan deliberately deferred mikser-to-mikser composition to v10. The reasoning is in `9.0-PLAN.md` under "Out of scope":
+The 9.0 design deliberately deferred mikser-to-mikser composition to v10. The reasoning:
 
 > Cross-instance concerns (discovery, peer identity, routing, cross-mikser refs, watch propagation, liveness, partition behavior, peer auth) belong to a distributed-systems substrate that iterates at "we hit a production partition bug" speed, not at engine-substrate speed. Folding any of it into `mikser-io` would re-fail ADR-0006 test #5 (release cadence) the moment it ships.
 

package/CLAUDE.md

@@ -10,22 +10,51 @@ paths. No migration markdown. No `task: pool` legacy aliasing for
 `task: inline`. Update READMEs and ADRs in place as source-of-truth
 changes; rewrite, don't supersede. The freedom is the point.
 
-**v9 has to pay out before v10 starts.** Not "ship v9 and start v10."
-**Use v9 in real daily work until the workflow pays out**, then
-v10. The v10 architecture (party-mikser-io) is more fun to design
-than v9 documentation is to write — the gate exists because that
-asymmetry kills projects. Concrete pass/fail signals in
-`10.0-PLAN.md` under "The gate." If a session brings up v10 design
-work, check the gate first; if v9 isn't paying out yet, the
-question to answer is "what about v9 isn't working in daily use?"
-— not "what should v10 look like?"
-
-**Position mikser by what it is, not by speed.** Hugo wins the speed
-race; mikser doesn't compete there. Position: AI-native, lifecycle-
-observable, files-as-source-of-truth, full-cycle introspection. Speed
-claim cap: "fast enough that watch-mode rebuilds feel instant."
-Honest range: 200–800 docs/sec depending on corpus size
-(see `test/perf/`).
+**v9 shipped. The gate now is daily-use payoff, not completion.**
+The engine, every sibling plugin, the agency template — all live.
+The next step isn't "finish v9," it's **use v9 in real daily work
+until the workflow earns the v10 layer**. The v10 architecture
+(party-mikser-io) is more fun to design than v9 dogfooding is to
+sustain — the gate exists because that asymmetry kills projects.
+Concrete pass/fail signals in `10.0-PLAN.md` under "The gate." If
+a session brings up v10 design work, check the gate first; if v9
+isn't paying out yet, the question to answer is "what about v9
+isn't working in daily use?" — not "what should v10 look like?"
+
+**mikser is a mixer.** The name is the architecture. It combines
+inputs (entities — from files, forms, queries, anywhere) through
+configurable rendering pipelines (renderer plugins + task-production
+policies) into outputs of any kind. Static-site generation is the
+canonical recipe — what mikser ships with and what most projects
+use today — but it's a recipe, not the definition. The same
+substrate runs:
+
+- HTML SSG via `layouts` + `renderHbs` (the recipe everything else
+  is currently shaped around)
+- PDFs via `renderHbs` + `post-pdf` (puppeteer-driven)
+- Emails via `renderHbs` → `post-mjml` → `post-email` (chain)
+- 3D renders by composing a hypothetical `renderBlender` against
+  `.blend` entities — same engine, different renderer + recipe
+- Video / audio / archives / data-export — same shape
+
+The engine has no opinion on what a renderer produces. `render.js`
+and `postprocess.js` are abstract worker entries; `manifest.js`
+tracks snapshots of *any* output; refs invalidation re-runs
+*any* render task. SSG is the dominant case in v9 because it's
+where the dogfooding lives, not because the architecture only
+fits there. When proposing changes, hold the broader view: a
+change that helps SSG but blocks the Blender/video/audio/archive
+cases is constraining the engine's actual reach.
+
+**Position mikser by what it is, not by speed.** Hugo wins the
+speed race for SSG specifically; mikser doesn't compete there
+(and "fastest mixer" isn't a useful claim because the bottleneck
+is the renderer, not the engine). Secondary attributes that
+follow from the mixer framing: AI-native, lifecycle-observable,
+files-as-source-of-truth, full-cycle introspection. Speed claim
+cap: "fast enough that watch-mode rebuilds feel instant." Honest
+range: 200–800 docs/sec on the SSG-flavor pipeline depending on
+corpus size (see `test/perf/`).
 
 **Direct critique preferred.** Skip "great question," skip "solid but,"
 skip softening. When something didn't pay off, say so. Match user
@@ -81,7 +110,10 @@ brevity.
   (`minThreads: 0` + `idleTimeout: 30_000`) so INLINE-only workloads
   pay no worker overhead. `workerSafeOptions(runtime.options)`
   strips plugin-surface functions before TASKS.WORKER dispatch so
-  Piscina's structured clone doesn't choke.
+  Piscina's structured clone doesn't choke. Renderer / postprocessor
+  descriptors in `runtime.options.plugins` are projected to their
+  `render-${name}` / `post-${name}` identifier so workers can
+  resolve them via dynamic import.
 - `database/` — `createSqliteDatabase()`, `registerSchema()`,
   `useDatabase()` (the `mikser_meta` table stamps schema_version).
   `sift-to-sql.js` translates sift filters to SQL WHERE clauses
@@ -97,7 +129,16 @@ brevity.
   plugins to push onto), late-binding static mount + listen.
 - `logger.js` — pino + pino-pretty (inline) + gauge progress + custom
   Writable for progress coordination + `pino.multistream` for
-  third-party shipping (`runtime.config.logging.transports`).
+  third-party shipping. Two registration surfaces for transports:
+  - `runtime.config.logging.transports` — declarative; user config.
+  - `addLogTransport({ target, options?, level? })` — public export;
+    plugins call this from their factory (queues until logger build)
+    or any later hook (live-rebuilds the multistream + swaps
+    `runtime.engine.logger`). `useLogger()` reads the swap-target
+    fresh on each call so new transports start receiving records
+    immediately. Both surfaces compose; same `{level, target, options}`
+    shape. Enables Better Stack / Datadog / Loki / Axiom / Sentry as
+    standard sibling plugins without engine changes per vendor.
 - `utils.js` — shared pure helpers: `mimeForEntity`, `isLoopback`,
   `expandEntity`, `projectMeta`, `useCollection`, `useRenderer` (via
   render.js), `isTextEntity`, `readEntityContent`, `extractRefs`,
@@ -111,21 +152,62 @@ brevity.
   the engine's logger. Each worker opens its own read-only sqlite
   handle on first task (`ensureWorkerDb` in render.js) so template
   helpers like `runtime.lookupHref` stay sync. Never touch the journal
-  directly.
-- `config.js` — loads `mikser.config.js` at `onLoad`.
-- `plugins.js` — loads user plugins at `onLoad`. Plugin factories
-  receive the full `core` exports as their first argument.
+  directly. Plugin loading: main-thread INLINE dispatch reads
+  `runtime.renderers` / `runtime.postprocessors` first (populated by
+  plugins.js from descriptor returns); workers see empty registries
+  in their separate-process runtime and fall through to dynamic-import
+  by `mikser-io-${name}` package name. Per-plugin options flow through
+  `descriptor.options` and arrive as the `config` arg to
+  `load`/`render`/`setup`/`postprocess`/`teardown`.
+- `config.js` — loads `mikser.config.js` at `onLoad` into
+  `runtime.config`. v9 holds only engine-level keys (`server`,
+  `logging`, `catalog` if tuned) plus the `plugins` array — all
+  plugin options moved to the factory call site (ADR-0010).
+- `plugins.js` — dispatches v9 plugin entries at `onLoad`. Each
+  entry in `plugins: []` is a factory call return; the dispatcher
+  duck-types on shape:
+  - function → lifecycle plugin; called with `core` so it can
+    register hooks.
+  - `{ name, options, load?, render? }` → renderer descriptor;
+    stored in `runtime.renderers`.
+  - `{ name, options, postprocess, output?, setup?, teardown? }` →
+    postprocessor descriptor; stored in `runtime.postprocessors`.
+  Strings produce a v9 migration error pointing at the new shape.
 - `manager.js` — file watching (chokidar) and cron scheduling.
 - `source.js` — `useSource` codifies the folder-of-files pattern.
 - `constants.js` — `OPERATION` (CREATE/UPDATE/DELETE/RENDER/
   POSTPROCESS), `ACTION` (sync action types), `TASKS` (`INLINE`/
   `SERIAL`/`WORKER` — dispatch modes).
 
+## Canonical sibling plugins
+
+These ship as separate repos but most projects pull them in. They're
+not bundled because the engine is renderer-agnostic — each plugin
+encodes a particular recipe (SSG, MJML, PDF, email) rather than
+substrate.
+
+- **`mikser-io-layouts`** — the canonical SSG-flavor task-production
+  policy. Owns multi-match layout assignment (`meta.layout` /
+  `meta.layouts` dual key + `autoLayouts` peel ladder), per-layout
+  destination Handlebars templates, collision detection, postprocess
+  chain parsing (filename `name.html-mjml-email.hbs` → `postprocessors:
+  ['mjml','email']` + frontmatter `postprocessor`/`postprocessors`),
+  `inspect()` primitive at `runtime.options.layouts.inspect`. The
+  engine has no opinion on layouts; this plugin is one way of producing
+  render tasks. Other domains (3D rendering, video) would have their
+  own task-production plugins.
+- **`mikser-io-render-{hbs,liquid,eta,markdown,metatext}`** — renderer
+  plugins. `renderHbs` is bundled in core; the rest are siblings.
+- **`mikser-io-post-{mjml,pdf,email}`** — postprocessor plugins,
+  composable in chains (see `documentation/rendering.md#postprocess`).
+- **`mikser-io-mcp`** — AI tooling surface (ADR-0006 test #5).
+- **`mikser-io-vector`** — vector index over the catalog.
+- **`mikser-io-forms`** — HTTP form receivers → entities.
+- **`mikser-io-schemas`** — TypeScript type generation from entity meta.
+
 ## Plugin map (`src/plugins/`)
 
 - `documents` — file→entity sync for the documents collection
-- `layouts` — layout matching + sitemap + `inspect()` primitive
-  (exposed at `runtime.options.layouts.inspect`)
 - `files` — file→entity sync for the files collection
 - `assets` — asset references and copy
 - `resources` — resource references
@@ -140,16 +222,83 @@ brevity.
 - `observer` / `mapper` / `validator` / `commands` / `shares` —
   utility plugins
 
+## Built-in content providers (`src/plugins/providers/`)
+
+The substrate-side `readEntityContent(entity)` dispatches by URI
+scheme. Two schemes are handled inline by built-in providers
+(no separate package required, ships with mikser-io):
+
+- **No scheme / plain path / `file://`** — filesystem read. Lives
+  inline in `src/utils.js readEntityContent` because the engine
+  already touches the filesystem everywhere.
+- **`http://` / `https://`** — `src/plugins/providers/http.js`.
+  Conditional GET with ETag + Last-Modified, in-memory response
+  cache, inflight coalescing on the same URL, binary mirror to
+  `runtime/http-cache/<sha-of-url>.<ext>`, operator-supplied
+  headers via `entity.meta.httpHeaders`, configurable timeout via
+  `entity.meta.httpTimeoutMs`. Test affordance:
+  `__resetHttpCacheForTests()` clears the module-level caches
+  between unit tests.
+
+Any other scheme (`gdrive://`, `notion://`, `s3://`, …) routes
+through the dynamic-import-by-name path — `mikser-io-provider-<scheme>`
+must be installed as a sibling package and export a top-level
+`read(entity)` function. See ADR-0010 / the gdrive provider for
+the convention.
+
+## Multi-emitter collections (sweep ownership)
+
+Collections like `documents` / `files` / `assets` aren't owned by a
+single plugin. The file source (via `useSource`) emits into them
+from its scanned folder, but other plugins legitimately emit into
+the same collection: `mikser-io-csv` fans CSV rows into `documents`,
+a gdrive sync would emit there too, an API endpoint can push into
+`files`. The catalog has no opinion on emitter identity — only on
+collection.
+
+The delete sweep MUST respect that. `sweepDeleted(collection,
+scanned, onDelete, ownerPrefix)` requires `ownerPrefix` (typically
+`absFolder` for file sources, `layoutsFolder` for layouts) and only
+considers entities whose `entity.uri` is rooted under that prefix.
+Foreign-emitter entities have a different `uri` shape:
+- CSV row entities: empty `uri` (synthetic — meta is the content)
+- HTTP-fetched CSV parent: `uri = 'https://…'`
+- gdrive-sourced docs: `uri = 'gdrive://…'`
+
+…and the LIKE clause excludes them. Without the scope, every cycle's
+file-source sweep silently wipes every co-collection emitter's
+entities. Throwing instead of silently sweeping the whole collection
+is intentional — this used to be a class-of-bugs landmine. If you're
+writing a new source-shaped plugin, the scoping is mandatory.
+
+Test coverage: `test/unit/source-sweep.test.js`.
+
 ## Naming conventions
 
 - **Engine state** at `runtime.<name>` — `runtime.refs`, `runtime.catalog`.
 - **Plugin surfaces** at `runtime.options.<plugin>` —
   `runtime.options.preview`, `runtime.options.layouts.inspect`.
+- **Reachability:** `runtime.options.url` is the engine's resolved
+  public URL (from `--url` or `config.url`, trailing slash stripped,
+  validated). Plugins that need external reachability — webhook
+  receivers, absolute links in emails, MCP preview URLs returned to
+  agents, forms share links — read this. Standard gating pattern:
+  ```js
+  const canPush = runtime.options.url?.startsWith('https://')
+  if (canPush) registerWebhookAt(`${runtime.options.url}/api/X/webhook`)
+  else         setupPollingFallback()
+  ```
+  `runtime.options.port` is the internal listener port (kept for
+  loopback/dev URL building); `runtime.options.url` is the external
+  origin. When in doubt: external = `url`, internal = `port`.
 - **Engine functions** as module-level exports from `mikser-io`:
   `import { queryEntities, subscribe, useRenderer, useCollection,
   readEntityContent, isTextEntity } from 'mikser-io'`.
 - **Plugin packages**: `mikser-io-<name>` (mikser-io-mcp, mikser-io-vector,
-  mikser-io-schemas, etc.).
+  mikser-io-schemas, etc.). Each exports a v9 named factory in
+  camelCase: `import { vector } from 'mikser-io-vector'`,
+  `import { renderHbs } from 'mikser-io'`. Consumer uses
+  `plugins: [vector({...})]` — never the bare string.
 - **MCP tools**: `mikser_<verb>` or `mikser_<subsystem>_<verb>`:
   `mikser_query_entities`, `mikser_read_entity`, `mikser_update_entity`,
   `mikser_delete_entity`, `mikser_render`, `mikser_refs_inbound`,
@@ -206,25 +355,39 @@ brevity.
   persistence pattern. Journal-on-sqlite (Phase 7) enables `--resume`;
   auto-persist (Phase 9) means plugins mutate the yielded entity and
   the journal writes back without an explicit `updateEntry` call.
+- **0010** — Plugin bundles + factory-call form + inline options.
+  Plugins are imported by name and called as factories;
+  `plugins: []` carries factory returns, never strings.
+  Lifecycle plugins are `(options) => (core) => void`; renderers
+  return `{name, options, load?, render?}`; postprocessors return
+  `{name, options, output?, setup?, postprocess, teardown?}`. Per-
+  plugin config moved entirely off `runtime.config.<plugin>`; it
+  arrives as factory args, gets stashed on the descriptor, and is
+  passed as `config` to `load`/`render`/`setup`/`postprocess`.
 
 ## MCP
 
-Lives in `mikser-io-mcp` plugin (separate repo). Activate by listing
-`'mcp'` **first** in your `mikser.config.js` plugins array:
+Lives in `mikser-io-mcp` plugin (separate repo). Activate by calling
+`mcp()` **first** in your `mikser.config.js` plugins array:
 
 ```js
+import { mcp } from 'mikser-io-mcp'
+
 export default {
-    plugins: ['mcp', /* ...other plugins */],
-    mcp: {
-        path: '/mcp',          // optional; default '/mcp'
-        // endpoints: { ... }  // optional; per-endpoint token + scope
-    },
+    plugins: [
+        mcp({
+            // path: '/mcp',          // default '/mcp'
+            // endpoints: { ... },    // per-endpoint token + scope
+        }),
+        /* ...other plugins */
+    ],
 }
 ```
 
 Must be first because its factory creates `runtime.options.mcp`
-synchronously, and other plugins gate their MCP tool registration on
-`if (runtime.options.mcp)` in their own `onLoaded`.
+synchronously when its closure runs, and other plugins gate their
+MCP tool registration on `if (runtime.options.mcp)` in their own
+`onLoaded`.
 
 There is **no `--mcp` CLI flag**. Activation is plugin-presence only.
 
@@ -276,15 +439,35 @@ There is **no `--mcp` CLI flag**. Activation is plugin-presence only.
 
 - **New engine capability?** Run through ADR-0006's five tests. Bar
   is high. Express is the only earned addition.
-- **New plugin?** Own repo, named `mikser-io-<name>`. Composes
-  against `runtime.options.app` / `runtime.options.mcp` / lifecycle
-  hooks. Never imports another plugin's source.
+- **New plugin?** Own repo, named `mikser-io-<name>`. Exports a
+  named v9 factory (e.g. `export function vector(options = {}) {
+  return (core) => { ... } }`). Composes against
+  `runtime.options.app` / `runtime.options.mcp` / lifecycle hooks.
+  Never imports another plugin's source — and the engine never
+  reads `runtime.config.<plugin>` for plugin options; everything
+  flows through the factory arg (ADR-0010).
 - **New MCP tool?** Add to `mikser-io-mcp/index.js` via
   `mcp.simpleTool(name, description, zodSchema, handler)`. Tool name
   follows `mikser_*` convention.
 - **New lifecycle hook?** Almost certainly no. Existing hooks cover
   all known patterns. If you think you need one, post the use case
   to ADR-0006 review.
+- **New content source (gdrive, notion, s3, github, …)?** Two
+  things ship together:
+  - A regular lifecycle plugin (the sync — emits entities into the
+    catalog, sets `entity.uri = '<scheme>://...'` so the dispatch
+    knows where to route reads).
+  - A named export `read(entity)` from the same package, which
+    must be named `mikser-io-provider-<scheme>` so the engine can
+    find it. `readEntityContent(entity)` parses the scheme out of
+    `entity.uri`, dynamic-imports the package, calls its `read`.
+    No registry, no descriptor — same package-name convention as
+    `mikser-io-render-*` and `mikser-io-post-*`.
+  Built-in: plain local paths (no scheme) and `file://` URIs are
+  read via `fs.readFile` directly — no provider package needed for
+  the canonical local-filesystem case. `entity.content` already
+  populated is a fast-path that skips the dispatch entirely (for
+  small remote docs eager-fetched at sync time).
 
 ## Test suites
 
@@ -301,6 +484,48 @@ There is **no `--mcp` CLI flag**. Activation is plugin-presence only.
   + clean-build timing). `SIZE=realistic` + entity count knobs
   documented in **Perf**.
 
+## Dev workspace
+
+The siblings live side-by-side under `/Users/dick/Projects/mikser/`
+and share an npm workspace declared at the parent's `package.json`:
+
+```json
+{
+  "private": true,
+  "workspaces": ["mikser-io", "mikser-io-*"]
+}
+```
+
+`npm install` at that root hoists everything: each sibling's
+`node_modules/mikser-io` (and any other cross-workspace dep)
+becomes a symlink to the working copy. The whole tree runs
+against ONE module instance of mikser-io.
+
+**This isn't optional ergonomics — it's correctness.** Plugins
+use `AsyncLocalStorage` (currently `queryContext` for sidecar
+findEntities tracking; the schema-version meta could grow more).
+Without workspace deduplication, npm 7+ auto-installs the peer
+dep into each sibling's own `node_modules`. Layouts'
+`import { queryContext } from 'mikser-io'` then resolves to its
+bundled copy — a different AsyncLocalStorage instance than the
+engine uses. Sidecar queries don't get tracked → no `query`
+edges in `mikser_snapshots.refClosure` →
+`manifest.queryAffected` returns empty → aggregate layouts
+(index pages, sitemaps, RSS feeds) never invalidate when new
+matching entities land. Silent broken-incremental.
+
+`test/scenarios/_harness.js` also auto-coalesces (replaces any
+real `mikser-io-layouts/node_modules/mikser-io` directory with a
+symlink to MIKSER_ROOT on every `setupFixture`) as a backstop —
+the scenarios pass with or without the workspace, but the
+workspace is the canonical setup. Same shape applies to any
+future sibling that uses `queryContext` or other module-level
+state from mikser-io.
+
+For production consumers (someone `npm install mikser-io
+mikser-io-layouts` in their own project), npm resolves both from
+the consumer's own project tree — no duplication, no bug.
+
 ## Reference
 
 - `documentation/architecture.md` — module map (audit before relying
@@ -309,5 +534,8 @@ There is **no `--mcp` CLI flag**. Activation is plugin-presence only.
 - `documentation/configuration.md` — config reference
 - `documentation/api-reference.md` — public API
 - `test/perf/` — render-pipeline perf rig
-- Sibling repos: `mikser-io-mcp`, `mikser-io-vector`, `mikser-io-schemas`,
+- Sibling repos: `mikser-io-layouts` (canonical SSG-flavor render-task
+  production policy), `mikser-io-mcp`, `mikser-io-vector`,
+  `mikser-io-schemas`, `mikser-io-forms`, `mikser-io-post-{mjml,pdf,email}`,
+  `mikser-io-render-{eta,liquid,markdown,metatext}`,
   `mikser-io-sdk-{api,react,svelte,vue,vector}`, `mikser-io-example-blog`

package/README.md

@@ -50,6 +50,28 @@ Build mikser into the parts of your application that are content-shaped. Keep th
 
 **Open source.** MIT-licensed, on GitHub, no telemetry, no auth wall, no SaaS dependency. What you see is what runs.
 
+## Mix content from anywhere, ship to anywhere
+
+A real content stack pulls from more than one system. Marketing copy lives in a CMS or a spreadsheet. Prices and stock live in an ERP. Hero images live in a DAM. Editorial pages live in markdown files in the repo. Most teams either pick one tool and contort the rest to fit it, or build sync services that copy everything into one database — and then more sync services when things drift out of sync.
+
+Mikser is built around a different bet: **one queryable substrate that any source can pour into, and any frontend can read from.**
+
+**Any source.** Provider plugins let you treat external systems as content sources. A Google Sheet, an ERP feed, a Drive folder, a Notion database, an HTTP webhook, the local repo's `.md` files — each becomes a stream of entities flowing into mikser's catalog with the same shape. The authoring tool doesn't change. The editorial workflow doesn't change. The team writing product copy in Google Sheets keeps writing product copy in Google Sheets — mikser just notices when they save.
+
+**Cross-source composition through references.** Mikser tracks references between entities the way a graph database does. A product page can declare `$marketing` pointing at a row pulled from a spreadsheet, `$pricing` at a record pulled from your ERP, `$hero` at an asset pulled from your DAM. A single API call from your frontend asks for the product page *with* its referenced data — and gets the page, the marketing copy, the price, and the hero image merged into one response. One round trip, not three or four, and no consumer-side join logic to maintain.
+
+**Live updates are uniform across every source.** Edit a cell in the spreadsheet, change a price in the ERP, replace an asset in the DAM — the pages depending on those entities update within seconds, in every frontend connected to mikser's live channel. You don't write per-source invalidation; mikser already knows which pages reference what.
+
+**Your frontend is whatever you want.** Mikser exposes the catalog over HTTP (and over MCP, for AI agents). You query it from React, Vue, Svelte, SvelteKit, Next.js, an iOS app, a kiosk — anything that speaks HTTP. The framework SDKs (`mikser-io-sdk-react`, `mikser-io-sdk-vue`, `mikser-io-sdk-svelte`) make the calls feel native, but they're optional. Headless CMSes free you from the database lock. Frameworks like Astro free you from one kind of authoring lock. Mikser frees you from both at once: **any source on the input side, any framework on the output side.**
+
+What this looks like in practice:
+
+- Marketing edits product copy in Google Sheets, prices come from your ERP, both flow into the same product page in your storefront — changes appear within seconds, without your e-commerce team writing webhook handlers or cache-invalidation logic.
+- A SaaS pulls customer-facing release notes from `release-notes/*.md` in the repo and feature flags from LaunchDarkly; both render in a dashboard built in the team's own React stack, not a framework somebody else picked.
+- A magazine pulls editorial articles from local markdown, contributor bios from Notion, sponsorship info from Airtable, and renders the same article surface across web, email (MJML), and PDF — one catalog, one set of references, three outputs.
+
+Adding a source is mechanical. See [`mikser-io-csv`](https://github.com/almero-digital-marketing/mikser-io-csv) (any HTTP-served CSV, with live polling), [`mikser-io-provider-gdrive`](https://github.com/almero-digital-marketing/mikser-io-provider-gdrive) (Google Drive), and the built-in `http` provider for the shape. If a system has an API, it can be a mikser source, usually in well under 100 lines.
+
 ## Built for AI-assisted development
 
 Files-as-source isn't just a portability story — it makes the project unusually friendly to AI coding agents. There's a static-time half (the agent reads your tree the way it reads any repo — no DB connection, no schema upload, no sandboxed query layer to learn) and a runtime half (when the agent needs to write or render, mikser ships its own MCP server in core, so it talks to the live engine instead of a parallel REST shim you have to maintain).
@@ -92,9 +114,11 @@ Plugins extend the tool surface the same way they mount HTTP routes; install the
 
 ```js
 // mikser.config.js
+import { mcp } from 'mikser-io-mcp'
+
 export default {
-    plugins: ['mcp', /* … */],
-    // optional: mcp: { path: '/mcp', endpoints: { … } }
+    plugins: [mcp(), /* … */],
+    // mcp({ path: '/mcp', endpoints: { … } }) when options are needed
 }
 ```
 
@@ -148,6 +172,7 @@ The engine is what stays stable — the lifecycle, the catalog, the file-based c
 |---|---|
 | [`mikser-io-vector`](https://github.com/almero-digital-marketing/mikser-io-vector) | OpenAI embeddings + semantic search (sqlite-vec or pgvector) |
 | [`mikser-io-schemas`](https://github.com/almero-digital-marketing/mikser-io-schemas) | Zod-backed entity validation + auto-generated TypeScript declarations for the SDK. Auto-detects `$`-keyed references and warns on broken ones — see [ADR-0007](./documentation/decisions/0007-references-declaration-and-expansion.md) |
+| [`mikser-io-forms`](https://github.com/almero-digital-marketing/mikser-io-forms) | Public form-submission endpoints. POST → captcha + schema validation → write a document file plus uploaded files to disk; the `documents` / `files` plugins pick them up via their normal watch loop. Composes with `mikser-io-schemas` for schema-by-name; built-in captcha providers (Google v2/v3, hCaptcha, Turnstile) plus a custom-verify escape hatch |
 | [`mikser-io-archive`](https://github.com/almero-digital-marketing/mikser-io-archive) | Persist matching entities to YAML — audit trail, versioned content history, downstream export |
 | `mapper` | Run config-supplied transforms over matched entities each cycle (in-core, generic transformation layer) |
 | [`mikser-io-live`](https://github.com/almero-digital-marketing/mikser-io-live) | Lightweight dev server with browser auto-refresh — pair with `--watch` for the classic save→reload loop |

package/index.js

@@ -15,4 +15,35 @@ export * from './src/manager.js'
 export * from './src/logger.js'
 export * from './src/engine.js'
 export * from './src/render.js'
-export * from './src/source.js'
+export * from './src/source.js'
+
+// Built-in plugin factories. Each takes options and returns the
+// (core) => void closure the engine calls at onLoad time. See ADR-0010
+// for the v9 plugin shape.
+export { api }           from './src/plugins/api.js'
+export { assets }        from './src/plugins/assets.js'
+export { commands }      from './src/plugins/commands.js'
+export { data }          from './src/plugins/data.js'
+export { documents }     from './src/plugins/documents.js'
+export { files }         from './src/plugins/files.js'
+export { frontMatter }   from './src/plugins/front-matter.js'
+export { json }          from './src/plugins/json.js'
+// layouts moved to sibling: import from 'mikser-io-layouts'
+export { mapper }        from './src/plugins/mapper.js'
+export { observer }      from './src/plugins/observer.js'
+export { preview }       from './src/plugins/preview.js'
+export { resources }     from './src/plugins/resources.js'
+export { shares }        from './src/plugins/shares.js'
+export { validator }     from './src/plugins/validator.js'
+export { yaml }          from './src/plugins/yaml.js'
+
+// Built-in renderers. v9 factory shape returns the descriptor that the
+// loader stores in `runtime.renderers`; the same module also still
+// exports `load`/`render` at the top level so Piscina worker dispatch
+// can resolve via dynamic import. ADR-0010.
+export { renderAsset }    from './src/plugins/render/asset.js'
+export { renderFile }     from './src/plugins/render/file.js'
+export { renderHbs }      from './src/plugins/render/hbs.js'
+export { renderHref }     from './src/plugins/render/href.js'
+export { renderPreset }   from './src/plugins/render/preset.js'
+export { renderResource } from './src/plugins/render/resource.js'

package/package.json

@@ -1,8 +1,13 @@
 {
 	"name": "mikser-io",
-	"version": "8.3.9",
+	"version": "9.0.1",
 	"description": "<p align=\"center\">   <img src=\"mikser-lockup-stacked.svg\" alt=\"mikser\" width=\"198\" /> </p>",
 	"main": "index.js",
+	"exports": {
+		".": "./index.js",
+		"./package.json": "./package.json",
+		"./*": "./*"
+	},
 	"scripts": {
 		"debug": "node --no-warnings app.js --server --watch --working-folder test/fixture",
 		"test:unit": "node --test --test-reporter=spec 'test/unit/**/*.test.js'",
@@ -56,6 +61,7 @@
 		"express": "^5.2.1"
 	},
 	"devDependencies": {
+		"@ai-sdk/openai": "^3.0.71",
 		"fluent-ffmpeg": "^2.1.3",
 		"mikser-io-decap": "file:../mikser-io-decap",
 		"mikser-io-post-mjml": "file:../mikser-io-post-mjml",

package/src/config.js

@@ -4,34 +4,25 @@ import { onLoad } from './lifecycle.js'
 import path from 'node:path'
 
 onLoad(async () => {
-	const logger = useLogger()
-	const configFile = path.resolve(runtime.options.config)
-	logger.info('Config: %s', configFile)
-	try {
-		const config = await import(configFile)
-		if (typeof config.default == 'function') {
-			runtime.config = await config.default(runtime)
-		} else if (typeof config.default == 'object') {
-			runtime.config = config.default
-		}
-	} catch (err) {
-		if (err.code != 'ERR_MODULE_NOT_FOUND') throw err
-	}
+    const logger = useLogger()
+    const configFile = path.resolve(runtime.options.config)
+    logger.info('Config: %s', configFile)
+    try {
+        const config = await import(configFile)
+        if (typeof config.default == 'function') {
+            runtime.config = await config.default(runtime)
+        } else if (typeof config.default == 'object') {
+            runtime.config = config.default
+        }
+    } catch (err) {
+        if (err.code != 'ERR_MODULE_NOT_FOUND') throw err
+    }
 
-	const plugins = runtime.options.plugins.concat(runtime.config.plugins).filter((plugin) => plugin)
-	for (const plugin of plugins) {
-		if (!runtime.config[plugin]) {
-			try {
-				const pluginConfig = path.join(runtime.options.workingFolder, 'config', `${plugin}.config.js`)
-				const config = await import(pluginConfig)
-				if (typeof config.default == 'function') {
-					runtime.config[plugin] = await config.default(runtime)
-				} else if (typeof config.default == 'object') {
-					runtime.config[plugin] = config.default
-				}
-			} catch (err) {
-				if (err.code != 'ERR_MODULE_NOT_FOUND') throw err
-			}
-		}
-	}
+    // v8 used to walk `runtime.config.plugins` looking for matching
+    // `config/<plugin>.config.js` files to merge into `runtime.config`,
+    // because plugin entries were strings (names). v9 entries are factory
+    // call results (closures or descriptors), and plugin options arrive
+    // as factory args — see ADR-0010 — so per-plugin auxiliary config
+    // files have nothing to bind to. The loader was removed when the
+    // plugins list stopped carrying names.
 })