mikser-io 9.0.1 → 9.0.4

package/CLAUDE.md

@@ -175,6 +175,19 @@ brevity.
   Strings produce a v9 migration error pointing at the new shape.
 - `manager.js` — file watching (chokidar) and cron scheduling.
 - `source.js` — `useSource` codifies the folder-of-files pattern.
+- `routes.js` — HTTP route registry. Plugins mount on
+  `runtime.options.app` directly; the Express router stack has the
+  paths but not the intent (loopback-only? streaming?). So plugins
+  declare each mount via `registerRoute({ path, plugin, reachability,
+  streaming })` as they make it. `reachability` is `public` | `token`
+  | `loopback`; `streaming` flags SSE/WS routes a facade must not
+  buffer. Inventory lives at `runtime.routes`; consumers (a Caddy/nginx
+  facade generator, healthcheck list, `mikser://routes` resource) read
+  it — none baked in. `registerRoute` also folds in the origin/location
+  URL building and the standard `"<label> mounted: <loc>
+  [<reachability>]"` boot log that api/preview/mcp/vector/forms/decap
+  were each copy-pasting. Pure inventory — takes no position on what to
+  do with the routes.
 - `constants.js` — `OPERATION` (CREATE/UPDATE/DELETE/RENDER/
   POSTPROCESS), `ACTION` (sync action types), `TASKS` (`INLINE`/
   `SERIAL`/`WORKER` — dispatch modes).
@@ -199,7 +212,7 @@ substrate.
 - **`mikser-io-render-{hbs,liquid,eta,markdown,metatext}`** — renderer
   plugins. `renderHbs` is bundled in core; the rest are siblings.
 - **`mikser-io-post-{mjml,pdf,email}`** — postprocessor plugins,
-  composable in chains (see `documentation/rendering.md#postprocess`).
+  composable in chains (see `docs/rendering.md#postprocess`).
 - **`mikser-io-mcp`** — AI tooling surface (ADR-0006 test #5).
 - **`mikser-io-vector`** — vector index over the catalog.
 - **`mikser-io-forms`** — HTTP form receivers → entities.
@@ -291,6 +304,18 @@ Test coverage: `test/unit/source-sweep.test.js`.
   `runtime.options.port` is the internal listener port (kept for
   loopback/dev URL building); `runtime.options.url` is the external
   origin. When in doubt: external = `url`, internal = `port`.
+- **Trust proxy default is `'loopback'`** (server.js), not Express's
+  `false`. The dominant deployment puts a same-host reverse proxy in
+  front, where every socket peer is 127.0.0.1; with no trust, mikser
+  reads every proxied request as loopback and the loopback-only gate
+  (api/mcp/forms/decap enforce `isLoopback(req.ip)`) inverts the
+  moment a facade is added. `'loopback'` honors `X-Forwarded-For` only
+  from a loopback peer — safe everywhere (a remote attacker's kernel-
+  set peer is never 127.0.0.1, so they can't forge a loopback
+  `req.ip`). Override via `config.server.trustProxy` — `'uniquelocal'`
+  for a sibling-container proxy, a CIDR for a specific subnet, `false`
+  to opt out. The loopback-enforcement and this default are a pair; a
+  facade that injects `X-Forwarded-For` (Caddy/nginx do) makes it work.
 - **Engine functions** as module-level exports from `mikser-io`:
   `import { queryEntities, subscribe, useRenderer, useCollection,
   readEntityContent, isTextEntity } from 'mikser-io'`.
@@ -364,6 +389,26 @@ Test coverage: `test/unit/source-sweep.test.js`.
   plugin config moved entirely off `runtime.config.<plugin>`; it
   arrives as factory args, gets stashed on the descriptor, and is
   passed as `config` to `load`/`render`/`setup`/`postprocess`.
+- **0011** (implemented; proven against gpoint) — File and resource
+  entities expose deployed URLs. References to served files
+  (image/video/PDF) are `$`-keyed **served paths** (`/img/X.jpg`,
+  `/media/clip.mp4` — the path content authors, = the entity's
+  `meta.url`), resolving via a new `refFilter` `{ 'meta.url': … }`
+  clause (utils.js) backed by an indexed `meta_url` column
+  (catalog.js + the `INDEXED_COLUMNS` map in sift-to-sql.js; schema
+  9.0.1 → 9.0.2). No collection-prefixed ids leak into content.
+  Id-refs were tried and rejected — gpoint references content by
+  served path everywhere, and its `/media/**` `resources()` library
+  means the entity only exists *because* content references
+  `/media/…`. The `files` plugin stamps `meta.url`; the `resources`
+  plugin stamps its library's served location; the `assets` plugin
+  stamps `meta.presets` in `onProcessed` via the pure `presetUrl()`
+  helper. Expanding a ref then yields the served entity's URL set,
+  not a string to reconstruct; `lookupUrl` (render.js) is the sync
+  Handlebars helper resolving a ref to `meta.url` or a named preset.
+  Base-relative in the live catalog, absolute in static renders
+  (from `runtime.options.url`). Umbrella term is *served entity*
+  (file/resource/preset).
 
 ## MCP
 
@@ -528,11 +573,11 @@ the consumer's own project tree — no duplication, no bug.
 
 ## Reference
 
-- `documentation/architecture.md` — module map (audit before relying
+- `docs/architecture.md` — module map (audit before relying
   on specifics; may have drift)
-- `documentation/decisions/` — ADRs
-- `documentation/configuration.md` — config reference
-- `documentation/api-reference.md` — public API
+- `docs/decisions/` — ADRs
+- `docs/configuration.md` — config reference
+- `docs/api-reference.md` — public API
 - `test/perf/` — render-pipeline perf rig
 - Sibling repos: `mikser-io-layouts` (canonical SSG-flavor render-task
   production policy), `mikser-io-mcp`, `mikser-io-vector`,

package/README.md

@@ -2,17 +2,21 @@
   <img src="mikser-lockup-stacked.svg" alt="mikser" width="198" />
 </p>
 
+<p align="center"><strong>The AI-native content engine.</strong></p>
+
 # Mikser
 
-**Mikser is the content layer of your application.** Business logic, user accounts, transactions live in their own services; mikser handles the parts that *are* content — pages, docs, the published catalog, multi-format outputs. [Vue](https://github.com/almero-digital-marketing/mikser-io-sdk-vue), [React](https://github.com/almero-digital-marketing/mikser-io-sdk-react), and [Svelte](https://github.com/almero-digital-marketing/mikser-io-sdk-svelte) SDKs are the seam between them — same surface across all three (`useDocument`, `useDocuments`, multilingual `useHref`, live SSE updates), each in its framework's idiomatic shape.
+**Mikser is the AI-native content engine.** It mixes content from anywhere — markdown files, Google Sheets, your ERP, a CMS, any API — into one live catalog, links it together with references, and ships it to any frontend. Edit a price in your ERP or a cell in a spreadsheet, and every page that uses it updates within seconds — in whatever framework you built the site with.
+
+**Framework-agnostic on both ends.** Headless CMSes (Sanity, Contentful) free you from the database but lock authoring into their UI. Frameworks like Astro let you bring any frontend but lock you into the framework. Mikser frees both: **any source in, any frontend out.** Provider plugins turn external systems into content sources; the catalog is served over plain HTTP, with idiomatic [Vue](https://github.com/almero-digital-marketing/mikser-io-sdk-vue) / [React](https://github.com/almero-digital-marketing/mikser-io-sdk-react) / [Svelte](https://github.com/almero-digital-marketing/mikser-io-sdk-svelte) SDKs on top (`useDocument`, `useDocuments`, multilingual `useHref`, live SSE). It's also AI-native: agents read and write the same catalog over MCP using the same calls a frontend developer uses — there's no separate "AI API" to keep in sync.
 
-**An AI-native file-based content engine with live graph queries.** Content lives as plain text files you can read, search, and version-control like code. Mikser knows how those files reference each other — who an article's author is, which images a landing page uses, who else mentions a given product. When you fetch an article, you can pull its author and that author's organization along with it in one round-trip. When the author's bio changes, every page subscribed to a reference touching them updates live, without polling. AI agents talk to mikser through the same calls a frontend developer uses — there's no separate "AI API" to keep in sync. The whole graph is queryable from both sides at once.
+**References merge sources into one call.** Mikser tracks how entities reference each other — an article's author, the marketing copy and ERP price behind a product page, the images a landing page uses. Fetch the product and pull its marketing copy, price, and hero image along with it in one round-trip. Change any of them and every page subscribed to a reference touching it re-renders live, without polling — and mikser invalidates *exactly* the pages that depend on it, nothing more.
 
-Built for Node.js around a strict lifecycle and a composable plugin system. Every document, asset, and template flows through the same pipeline; plugins hook in at any phase. The same engine runs a single markdown blog and a multi-language publishing platform with PDF / email / AI-augmented asset pipelines — same lifecycle, more plugins.
+**Your content stays yours.** Source files live on disk as `.md` / `.yml` — diffable, version-controllable, portable on day one and year ten. No database lock-in, no proprietary export. And it's the content layer, not your whole backend: business logic, accounts, and transactions stay in their own services; mikser handles the part that's actually content — rendered to HTML, PDF, email, and other formats from the same source.
 
-It's MIT-licensed, runs on Node, has zero hosted dependencies, and the entire content tree it manages is a folder of `.md` and `.yml` files you can copy, diff, and version-control. **The portability promise is the architecture, not a feature.**
+Built for Node.js around a strict lifecycle and a composable plugin system: every document, asset, and template flows through the same pipeline, and plugins hook in at any phase. MIT-licensed, runs on Node, zero hosted dependencies. **The portability promise is the architecture, not a feature.**
 
-> **New to mikser?** Read the [Architecture Overview](./documentation/overview.md) — one document, end-to-end walkthrough of how a file becomes a deployed page across all twenty lifecycle phases. It's the doc most projects need first.
+> **New to mikser?** Read the [Architecture Overview](./docs/overview.md) — one document, end-to-end walkthrough of how a file becomes a deployed page across all twenty lifecycle phases. It's the doc most projects need first.
 
 ## Where it fits
 
@@ -163,7 +167,7 @@ The engine is what stays stable — the lifecycle, the catalog, the file-based c
 | Plugin | What it does |
 |---|---|
 | `data` | JSON snapshots of entities / context / catalog, written to disk for static serving |
-| `api` | REST endpoints with sift-backed queries, per-endpoint tokens, optional render, opt-in [per-query disk cache](./documentation/caching.md) for reverse-proxy failover |
+| `api` | REST endpoints with sift-backed queries, per-endpoint tokens, optional render, opt-in [per-query disk cache](./docs/caching.md) for reverse-proxy failover |
 | `preview` | In-memory render cache + `GET /preview/:filename` route. Companion to the `mikser_preview_render` MCP tool (in [`mikser-io-mcp`](https://github.com/almero-digital-marketing/mikser-io-mcp)) — transient render bytes served at a clickable URL, no filesystem footprint |
 
 **Integrations:**
@@ -171,7 +175,7 @@ The engine is what stays stable — the lifecycle, the catalog, the file-based c
 | Plugin | What it does |
 |---|---|
 | [`mikser-io-vector`](https://github.com/almero-digital-marketing/mikser-io-vector) | OpenAI embeddings + semantic search (sqlite-vec or pgvector) |
-| [`mikser-io-schemas`](https://github.com/almero-digital-marketing/mikser-io-schemas) | Zod-backed entity validation + auto-generated TypeScript declarations for the SDK. Auto-detects `$`-keyed references and warns on broken ones — see [ADR-0007](./documentation/decisions/0007-references-declaration-and-expansion.md) |
+| [`mikser-io-schemas`](https://github.com/almero-digital-marketing/mikser-io-schemas) | Zod-backed entity validation + auto-generated TypeScript declarations for the SDK. Auto-detects `$`-keyed references and warns on broken ones — see [ADR-0007](./docs/decisions/0007-references-declaration-and-expansion.md) |
 | [`mikser-io-forms`](https://github.com/almero-digital-marketing/mikser-io-forms) | Public form-submission endpoints. POST → captcha + schema validation → write a document file plus uploaded files to disk; the `documents` / `files` plugins pick them up via their normal watch loop. Composes with `mikser-io-schemas` for schema-by-name; built-in captcha providers (Google v2/v3, hCaptcha, Turnstile) plus a custom-verify escape hatch |
 | [`mikser-io-archive`](https://github.com/almero-digital-marketing/mikser-io-archive) | Persist matching entities to YAML — audit trail, versioned content history, downstream export |
 | `mapper` | Run config-supplied transforms over matched entities each cycle (in-core, generic transformation layer) |
@@ -217,13 +221,13 @@ npx mikser --watch      # incremental dev loop
 npx mikser --server     # build + serve at :3001
 ```
 
-For a working starter — config with a real plugin set, sample `documents/`, expected output — see [Getting Started](./documentation/getting-started.md). Or skip straight to "add mikser to this app" via the [Claude Code plugin](#built-for-ai-assisted-development) above.
+For a working starter — config with a real plugin set, sample `documents/`, expected output — see [Getting Started](./docs/getting-started.md). Or skip straight to "add mikser to this app" via the [Claude Code plugin](#built-for-ai-assisted-development) above.
 
 ## Core Concepts
 
 - **Lifecycle** — Processing runs through fixed phases: initialize → load → import → process → persist → render → finalize. Plugins hook into any phase.
 - **Entities** — Everything is an entity (document, file, layout, asset). Entities flow through the journal and are tracked in the catalog.
-- **References between entities** — A front-matter key starting with `$` (e.g. `$author: /authors/dick`) points at another entity. The engine knows the whole graph: templates can follow the links, the schemas plugin checks they resolve, and a single query can pull referenced entities along inline instead of one round trip per link. See [ADR-0007](./documentation/decisions/0007-references-declaration-and-expansion.md).
+- **References between entities** — A front-matter key starting with `$` (e.g. `$author: /authors/dick`) points at another entity. The engine knows the whole graph: templates can follow the links, the schemas plugin checks they resolve, and a single query can pull referenced entities along inline instead of one round trip per link. See [ADR-0007](./docs/decisions/0007-references-declaration-and-expansion.md).
 - **Plugins** — Functionality is delivered via plugins. Built-in plugins handle common sources (documents, files, layouts, assets). Custom plugins can be added to any project.
 - **Runtime Singleton** — A plain module-level object holds all global state and coordinates the lifecycle. The ES module cache guarantees every importer gets the same instance.
 - **Watch Mode** — In watch mode, file changes trigger incremental re-processing without restarting.
@@ -239,16 +243,16 @@ The shape mikser fits cleanly:
 - **Mixed-output publishing** — the same source document renders to HTML, PDF (via `post-pdf`), MJML email (via `post-mjml`), and JSON snapshots. One catalog, many output formats, all concurrent.
 - **Headless backends for static frontends** — pair the `api` plugin with `sdk-api` for SSE-driven live frontends; pair the `data` plugin output with any static host for pre-rendered consumption.
 
-The shape mikser **doesn't** fit cleanly: anything with non-technical content authors who can't or won't work with files, anything with non-content business logic at the core, anything needing multi-tenant / per-user auth. Those aren't bugs — they're outside the design envelope. See [`decisions/0001-content-layer-not-the-app.md`](./documentation/decisions/0001-content-layer-not-the-app.md) for the explicit scope decision.
+The shape mikser **doesn't** fit cleanly: anything with non-technical content authors who can't or won't work with files, anything with non-content business logic at the core, anything needing multi-tenant / per-user auth. Those aren't bugs — they're outside the design envelope. See [`decisions/0001-content-layer-not-the-app.md`](./docs/decisions/0001-content-layer-not-the-app.md) for the explicit scope decision.
 
 ## Engineering discipline
 
 What you get from how this project is built:
 
-- **Every load-bearing decision has an ADR.** The [`decisions/`](./documentation/decisions/) folder names which choices are structural — files-as-source, journal+catalog split, plugin-as-factory, when something goes in core vs. ships as a plugin — and explains what protects them. When you push against one, there's a written answer waiting instead of folklore.
+- **Every load-bearing decision has an ADR.** The [`decisions/`](./docs/decisions/) folder names which choices are structural — files-as-source, journal+catalog split, plugin-as-factory, when something goes in core vs. ships as a plugin — and explains what protects them. When you push against one, there's a written answer waiting instead of folklore.
 - **Engine stays small; capability ships in plugins.** The 15+ plugin ecosystem adds features without core changes, so your upgrade cost stays low. Probes like `decap` (a full third-party CMS mounted in ~150 lines, zero engine changes) are deliberate evidence the extension model holds where it counts.
 - **Builds are deterministic; no async middleware layer.** The journal is the only synchronization primitive — no event bus, no IoC container, no orchestrator running plugins in surprising order. The lifecycle is a list of named phases; "what ran when?" has an answer you can read off the source.
-- **The whole engine is one read.** The [Architecture Overview](./documentation/overview.md) walks the full pipeline top to bottom. Onboarding a new engineer is an afternoon, not a tour through fifteen reference docs.
+- **The whole engine is one read.** The [Architecture Overview](./docs/overview.md) walks the full pipeline top to bottom. Onboarding a new engineer is an afternoon, not a tour through fifteen reference docs.
 
 ## Mikser among static site generators
 
@@ -274,19 +278,19 @@ Mikser itself has a previous chapter: the [legacy 7.x line](https://github.com/a
 
 | Document                                              | Audience           | Description                                        |
 | ----------------------------------------------------- | ------------------ | -------------------------------------------------- |
-| [Architecture Overview](./documentation/overview.md)  | Everyone           | **Start here.** End-to-end walkthrough of how a file becomes a deployed page across all lifecycle phases. |
-| [Getting Started](./documentation/getting-started.md) | Users              | Installation, first project, basic usage           |
-| [Configuration](./documentation/configuration.md)     | Users              | All CLI options and config file reference          |
-| [Lifecycle](./documentation/lifecycle.md)             | Users & Developers | Complete lifecycle phases and hook system          |
-| [Plugins](./documentation/plugins.md)                 | Users & Developers | Built-in plugins, writing custom plugins, the assets / resources / AI pipeline |
-| [Entities](./documentation/entities.md)               | Users & Developers | Entity model, operations, journal, catalog         |
-| [Rendering](./documentation/rendering.md)             | Users & Developers | Render pipeline, render plugins, render modes      |
-| [Watch Mode](./documentation/watch-mode.md)           | Users              | File watching, scheduled tasks, incremental builds |
+| [Architecture Overview](./docs/overview.md)  | Everyone           | **Start here.** End-to-end walkthrough of how a file becomes a deployed page across all lifecycle phases. |
+| [Getting Started](./docs/getting-started.md) | Users              | Installation, first project, basic usage           |
+| [Configuration](./docs/configuration.md)     | Users              | All CLI options and config file reference          |
+| [Lifecycle](./docs/lifecycle.md)             | Users & Developers | Complete lifecycle phases and hook system          |
+| [Plugins](./docs/plugins.md)                 | Users & Developers | Built-in plugins, writing custom plugins, the assets / resources / AI pipeline |
+| [Entities](./docs/entities.md)               | Users & Developers | Entity model, operations, journal, catalog         |
+| [Rendering](./docs/rendering.md)             | Users & Developers | Render pipeline, render plugins, render modes      |
+| [Watch Mode](./docs/watch-mode.md)           | Users              | File watching, scheduled tasks, incremental builds |
 | [MCP](https://github.com/almero-digital-marketing/mikser-io-mcp#readme) | Users              | The `mikser-io-mcp` plugin — tool surface, `mikser://` resources, twelve worked AI-driven scenarios |
-| [Caching](./documentation/caching.md)                 | Users (production) | The `cache: true` disk cache + working nginx config for reverse-proxy failover |
-| [Architecture](./documentation/architecture.md)       | Developers         | Module-level reference — what's in each file       |
-| [API Reference](./documentation/api-reference.md)     | Developers         | Complete public API reference                      |
-| [Decisions (ADRs)](./documentation/decisions/)        | Developers         | Load-bearing architectural choices and what protects them |
+| [Caching](./docs/caching.md)                 | Users (production) | The `cache: true` disk cache + working nginx config for reverse-proxy failover |
+| [Architecture](./docs/architecture.md)       | Developers         | Module-level reference — what's in each file       |
+| [API Reference](./docs/api-reference.md)     | Developers         | Complete public API reference                      |
+| [Decisions (ADRs)](./docs/decisions/)        | Developers         | Load-bearing architectural choices and what protects them |
 
 ## License