mikser-io 9.3.1 → 9.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -58,7 +58,10 @@ These options are part of `runtime.options` and apply to the engine itself.
58
58
  | `trace` | `-t, --trace` | boolean | `false` | Enable trace-level logging (very verbose). |
59
59
  | `threads` | — | number | `4` | Worker thread count for the Piscina pools (`renderWorkers`, `postprocessWorkers`). Both pools are lazy (`minThreads: 0` + `idleTimeout: 30_000`) so INLINE-only workloads spin up zero workers. |
60
60
  | `server` | `-s, --server [port]` | number\|boolean | — | When set, the engine creates a shared Express app on `runtime.options.app` and listens on the given port (default `3001`) after all plugins have mounted their routes. Plugins like `api` attach to it instead of starting their own server. The `outputFolder` is also served as a static catch-all route at `/` (plugin routes match first; anything that doesn't match falls through to the rendered output). Requires `express` to be installed. |
61
+ | `junk` | — | array\|false | built-in list | OS and file-manager litter, filtered out of both the scan and the watcher. The dot-prefixed files (`.DS_Store`, `._*`) were already invisible — globby defaults to `dot: false` and the watcher ignores leading dots — but the Windows ones are **not** dotfiles: `Thumbs.db` and `desktop.ini` were measurably scanned *and* watched, and became entities. The list is deliberately conservative (OS/file-manager artifacts and application lock files only, no `*.tmp`, `*.bak` or editor backups), because a filter that silently drops content is worse than the litter it prevents. `false` disables it; an array replaces it. See `isJunkPath` / `JUNK_IGNORE` in `src/utils.js`. Plugins that write metadata next to content add their own patterns with `registerJunk({ ignore, match })` — the engine provides the mechanism and the plugin the knowledge of what its files are called (`mikser-io-webdav` registers `*.nephelemeta`). Plugin registrations survive an array override, since narrowing the OS list is not a request to start importing a library's sidecars. |
62
+ | — (plugin) | — | object | — | `sources({ styles: { folder: 'styles', extensions: ['css'] } })` registers build inputs as catalog entities, one collection per key. A sidecar can then read them with `findEntities()`, whose queries land in the render's `refClosure` — so editing, adding or removing a part re-renders the bundle and nothing else. Reading the same files with `fs` instead works for one build and silently breaks watch, because the engine has no dependency on a file it never saw. Nothing is linked into `outputFolder`: these are inputs, not output. Named `sources` rather than `inputs` because `entity.inputs` already means something adjacent — bytes an output depends on without being entities at all. |
61
63
  | `cors` / `no-cors` | `--cors` / `--no-cors` | boolean | — | Toggle CORS on the engine's shared Express app. See `src/server.js` for the extensible header arrays plugins push onto. |
64
+ | `server.requestTimeout` | — | number | node default (`300000`) | Milliseconds a single request may take, set on the underlying `http.Server`. Node's 5-minute default is effectively an **upload size limit expressed in seconds** — a large file over a slow link is indistinguishable from a stalled request, so it is cut off and the caller sees a truncated write rather than a readable error. Only reachable from the `http.Server`, which the engine owns, so a plugin that mounts an upload surface (`mikser-io-webdav`, `forms` with large attachments) cannot raise it for itself. `0` disables the cap: reasonable on a trusted-network build server, bad facing the internet, where it removes the only bound on how long a client can hold a connection open doing nothing. `headersTimeout` is clamped to stay at or below it. The server is also exposed as `runtime.options.httpServer`. |
62
65
  | `url` | `-u, --url <url>` | string | — | Public URL where this mikser is reachable (e.g. `https://blog.me.com`). Validated, trailing slash stripped, stamped on `runtime.options.url`. Read by webhook-capable plugins for push-vs-poll gating (`url.startsWith('https://')`); used by anything that surfaces absolute URLs externally — MCP preview URLs returned to agents, forms share links, email tracking pixels. Plugins that just need internal URLs keep using `runtime.options.port`. |
63
66
 
64
67
  ## Engine Substrate
@@ -0,0 +1,194 @@
1
+ # 12. Authentication is an engine seam; identity providers are plugins
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ Mikser already authenticates requests. It does so in at least three places, by copy-paste, and the copies have drifted.
10
+
11
+ The rule every copy claims to implement — call it the *uniform rule* — is:
12
+
13
+ - token presented and valid → allow from anywhere
14
+ - token presented and invalid → reject
15
+ - token absent → require loopback, unless `allowRemote`
16
+
17
+ `src/plugins/api.js` and `mikser-io-mcp` implement that. `mikser-io-forms` does not, despite a comment saying it does:
18
+
19
+ | | token set, none presented | loopback denial | wrong token |
20
+ | --- | --- | --- | --- |
21
+ | `api.js` | falls back to loopback | `403` | `401` |
22
+ | `mikser-io-mcp` | falls back to loopback | `403` (JSON-RPC `-32001`) | `401` |
23
+ | `mikser-io-forms` | **rejects — no fallback** | **`401`** | `401` |
24
+
25
+ Neither behaviour is obviously wrong. Forms is arguably the safer reading. The problem is that nobody chose: three authors wrote the same paragraph of comments and three different policies, and no test can catch it because there is nothing shared to test.
26
+
27
+ The engine already exports the *primitive* (`isLoopback`) but not the *policy*. `src/utils.js` says so out loud: "For more nuanced policies (token gate + loopback fallback), plugins usually inline the check using `isLoopback()` directly." That was an accurate description of the status quo and is now the thing to fix.
28
+
29
+ Three further consequences follow from having no seam:
30
+
31
+ - **Token comparison is `presented !== expectedAuth`** — a plain string compare, not constant-time. Correct in every copy, and correct nowhere in particular.
32
+ - **There is no way to add a second credential type.** Supporting OAuth for remote MCP (the MCP spec's `WWW-Authenticate` + RFC 9728 challenge, which nothing in the tree implements today) currently means editing every plugin that mounts a route.
33
+ - **`registerRoute({ reachability })` already carries the vocabulary** — `'loopback' | 'token' | 'public'` — but each plugin recomputes it from its own config by hand, so the facade's view of exposure is only as accurate as the least careful copy.
34
+
35
+ Separately, and deliberately kept separate: outbound credentials (`GOOGLE_APPLICATION_CREDENTIALS`, `GH_TOKEN`, `DIRECTUS_TOKEN`) are each read ad hoc by their provider. That is a different problem with a different answer — see *Decision*, point 5.
36
+
37
+ ## Decision
38
+
39
+ **1. The engine gains an authentication seam. It gains no identity model.**
40
+
41
+ A *verifier* is a provider-agnostic descriptor:
42
+
43
+ ```js
44
+ {
45
+ name: 'bearer',
46
+ async verify(req) {}, // → null (no credential) | false (bad) | { subject, capabilities }
47
+ challenge? (res) {}, // optional: emit WWW-Authenticate for this scheme
48
+ authorizationServers?: [], // optional: OAuth discovery metadata
49
+ resource?, scopesSupported?, // consumed by mikser-io-mcp for RFC 9728
50
+ }
51
+ ```
52
+
53
+ Core ships exactly one verifier — `bearer({ token })`, constant-time via `crypto.timingSafeEqual` — plus:
54
+
55
+ ```js
56
+ resolveAuth(config) // string | fn | verifier | { token } | [ … ] → verifier | null
57
+ anyOf(...verifiers) // accept ANY of several credentials
58
+ authorize(req, verifier, opts) // the uniform rule, transport-agnostic
59
+ requireAuth(verifier, opts) // express middleware over authorize()
60
+ reachabilityOf({ auth, token, allowRemote }) // → 'loopback' | 'token' | 'public'
61
+ hasCapability(principal, capability) // endpoint ceiling ∩ principal grant
62
+ ```
63
+
64
+ **Several credentials on one surface.** A deployment reaches for this almost
65
+ immediately: a handful of long-lived machine tokens (one per caller, so a leak
66
+ scopes its own blast radius) alongside a human's OAuth token on the same routes.
67
+ `anyOf` lives in the engine rather than in userland because the last project that
68
+ hand-rolled it shipped a silent auth *bypass* — a composed verifier the resolver
69
+ didn't recognise degraded to a pass-through `next()` and merely warned. The
70
+ failure mode of getting this wrong is not "denied", it is "wide open".
71
+
72
+ The three-valued merge is the whole subtlety: any verifier accepting wins; all
73
+ reporting "nothing presented" yields `null` so loopback policy may still apply;
74
+ **any** reporting "presented and wrong" yields `false`, which never falls back to
75
+ loopback. A caller who presented a credential has identified itself and must be
76
+ judged on it.
77
+
78
+ `verify()` returning a principal is the primitive; `requireAuth()` is the Express convenience built on it. Both are exported, because MCP must render its denial as a JSON-RPC error body rather than an HTTP JSON body and therefore cannot use a generic middleware.
79
+
80
+ Core takes no position on who a subject is, where it came from, or how it was proven. It knows only: a credential was presented or it wasn't, it verified or it didn't, and what capabilities it carries.
81
+
82
+ **2. The uniform rule becomes one implementation.**
83
+
84
+ The three copies are replaced by `requireAuth()`. `api.js`'s semantics win — loopback fallback, `403` for reachability denial, `401` for a bad credential — because two of three already implement it and because `403` is the honest code when the caller's *origin* is the problem rather than their credential. Forms changes behaviour; that is a deliberate, documented break, not an accident.
85
+
86
+ **3. Identity providers are plugins, on the same contract.**
87
+
88
+ An OAuth verifier — and, if wanted, a self-hosted authorization server — ships as `mikser-io-auth`, exporting a verifier that `resolveAuth()` accepts wherever a token string is accepted today:
89
+
90
+ ```js
91
+ api({ endpoints: { admin: { auth: oauth({ issuer, audience }) } } })
92
+ ```
93
+
94
+ This mirrors what WhiteBox already proved out: a seam in `server/src/auth.js`, a first-party authorization server in `server-plugin-oauth`, and an external provider in `whitebox-pro-auth-auth0` — all three interchangeable at the same seam. The one place mikser deliberately diverges is the user store: WhiteBox keeps users in Postgres because WhiteBox is a running product with members; mikser keeps them in files because mikser is a build tool with operators (see point 4).
95
+
96
+ **4. Capabilities ride the credential, not a user.**
97
+
98
+ `operations: ['list', 'update', 'delete', 'render']` is already a capability set; today it is bound to an *endpoint*. It becomes the verifier's output instead, so a static token is simply an identity whose capability set is fixed by config, and an OAuth token is one whose capability set is minted at login. Route gating (`allow(op)`) is unchanged and stays where it is.
99
+
100
+ **Where identity is stored: Apache-format files in the working folder.**
101
+
102
+ Mikser does not grow a user table. Users and groups live as `htpasswd` and `htgroup` files alongside the content they govern:
103
+
104
+ ```
105
+ <workingFolder>/
106
+ users.htpasswd alice:$2y$10$… (bcrypt, apr1, or sha1)
107
+ groups.htgroup editors: alice bob
108
+ ```
109
+
110
+ This is ADR-0002 applied to identity rather than content, and it buys the same things files buy everywhere else in mikser: reviewable in a pull request, diffable, deployable by copying a directory, editable with `htpasswd(1)` — a tool that predates every framework this project will outlive. It also means the *format* is not ours to design, version, or migrate.
111
+
112
+ Groups map to capabilities in config, so the files stay pure identity and the product decides what a group can do:
113
+
114
+ ```js
115
+ auth({ groups: { editors: ['api:update', 'mcp:use'], viewers: ['api:list'] } })
116
+ ```
117
+
118
+ The obvious objection is that a file cannot be written safely by a running server under concurrent requests. That is a real constraint and it is also the point: this is a store you *provision*, not one the product mutates at runtime. Self-service signup, password reset, and invite flows are explicitly not in scope — a build tool has operators, not members.
119
+
120
+ **Row scope rides the credential too, opaquely.**
121
+
122
+ Capabilities are verbs (`api:delete`); they cannot express *which rows exist for
123
+ you*. A principal may therefore carry a `scope` — a value the **engine never
124
+ inspects**. Only the plugin whose surface issued the credential knows what it
125
+ means; for `api`/`vector`/`data` it is a sift filter, `$and`-ed with the
126
+ endpoint's own `query`. Keeping it opaque is what stops the engine from growing
127
+ an opinion about content.
128
+
129
+ Always `$and`, never `$or`: a credential may narrow what an endpoint exposes,
130
+ never widen it. Groups map to scopes the same way they map to capabilities, and
131
+ membership in several groups unions with `$or` — more groups must mean more
132
+ reach, or adding one would make a user less able.
133
+
134
+ Two consequences fell out of this and are load-bearing:
135
+
136
+ - **A principal-scoped response is never written to the query cache.** The cache
137
+ mirrors the request URL into the output folder, where nginx `try_files` serves
138
+ it without reaching the process. One caller's scoped rows would be handed to
139
+ every later caller of the same URL, unauthenticated. The key is
140
+ endpoint+querystring by design (it must match what nginx sees) so it cannot be
141
+ salted with the principal — the only safe answer is not to write.
142
+ - **A principal scope must be a sift object, never a function.** An object is
143
+ pushed into the WHERE clause; a function is applied post-fetch, the form that
144
+ pinned a production box at 111% CPU on 1,367 entities. A function is refused
145
+ rather than silently accepted.
146
+
147
+ The aggregated cross-plugin permission *catalog* WhiteBox uses stays **out of scope**. If it lands, it must be assembled during engine infrastructure setup, before any plugin hook runs (ADR-0005), or plugin load order becomes observable.
148
+
149
+ **5. Outbound credentials stay a convention, not core.**
150
+
151
+ A provider reading `process.env.GH_TOKEN` creates no god-plugin and forces no coordination — by the ADR-0006 bar it does not earn engine code. What it needs is a documented convention (`MIKSER_<PLUGIN>_<NAME>`, config-over-env precedence, redaction in logs), which is a docs change and a lint, not a subsystem.
152
+
153
+ ## Why this clears ADR-0006
154
+
155
+ 1. **Substrate, not domain.** The engine says *how* a credential is proven; plugins say *what* they gate. Core never learns what a route does.
156
+ 2. **Strengthens ADR-0004.** Today's "shared" policy is shared by duplication — the exact failure that ADR is written against. This is a consolidation of a concern mikser already has, not a new one it is claiming.
157
+ 3. **The plugin alternative is a god-plugin.** An auth plugin would have to know every other plugin's routes to gate them — the "express plugin" failure mode by name.
158
+ 4. **Plugins compose independently.** A plugin calls `resolveAuth(options.auth)` and mounts the result. No ordering, no coordination, no collision. This is the test the *catalog* fails, which is why the catalog is deferred.
159
+ 5. **Cadence.** The seam is ~80 lines whose shape has been stable in WhiteBox across its lifetime. OAuth is not stable — the MCP auth spec, RFC 9728 adoption, and the DCR debate all move monthly. Test 5 is what forces the split: **seam in core, OAuth in a plugin.**
160
+
161
+ ## Consequences
162
+
163
+ **Easier.** One auth policy, one place, one test suite. Adding OAuth becomes a plugin nobody else has to know about. `reachability` is computed by the engine, so the facade's exposure map stops being a per-plugin promise. Constant-time comparison happens once, correctly.
164
+
165
+ **Harder.** Every route-mounting plugin gets a coordinated release: core first, then a mechanical edit in `api`, `mcp`, `forms`, `decap`, `vector`. Forms changes behaviour for anyone who configured a token *and* relied on loopback bypassing it. The engine takes on a security-sensitive surface, where being wrong is worse than being absent — which is the argument for keeping it at 80 lines that do one thing.
166
+
167
+ **Staging.** Core seam ships first and is purely additive; the existing inline checks keep working untouched. Plugins migrate one at a time. `mikser-io-auth` is independent of both and needed only when a remote MCP endpoint has to satisfy a spec-compliant client.
168
+
169
+ ## Examples
170
+
171
+ - `src/auth.js` — the seam. `src/plugins/api.js` — the reference caller: auth, the
172
+ operation ceiling intersected with the principal's grant, and the per-request
173
+ scope combination. Note that every route lists `auth` **before** `allow(op)`;
174
+ the old order ran the capability check against an undefined principal, which
175
+ passes every check — a silent bypass, not a failure.
176
+ - `mikser-io-mcp/index.js` `mountEndpoint()` — why `authorize()` exists separately
177
+ from `requireAuth()`: MCP answers in JSON-RPC, so it shapes its own denials.
178
+ - `mikser-io-forms/index.js` — the drift that motivated this, now migrated. Its
179
+ behaviour changed: a token-gated form endpoint accepts loopback without the
180
+ token, and reachability denial answers 403 rather than 401, matching the others.
181
+ - `mikser-io-auth` — identity: htpasswd/htgroup, bcrypt, an ES256 key file, and
182
+ the Basic and JWT verifiers.
183
+ - `src/utils.js` `isLoopback` / `loopbackOnly` — the primitive that was already shared, and the comment that documented the missing half.
184
+ - WhiteBox `server/src/auth.js` + `server-plugin-oauth` + `whitebox-pro-auth-auth0` — the same split, already load-bearing in production.
185
+
186
+ ## Watch for drift
187
+
188
+ The failure mode is the engine growing an opinion about *identity*. Drift looks like: a users table; a `role` field; a login route in core; a permission catalog added "while we're here"; the seam accepting a provider-specific option so one integration is easier.
189
+
190
+ For the identity files specifically, drift looks like the product writing to them — a signup route, a password-reset endpoint, a "create user" tool. The moment mikser writes an htpasswd file at runtime it has a concurrency problem, a locking problem, and a database it refuses to admit it has.
191
+
192
+ The counter-question is always the same: could `mikser-io-auth` be uninstalled, leaving a working token-and-loopback engine behind? If not, the concern crossed the seam.
193
+
194
+ The second, quieter drift is a plugin inlining its own check again because the seam didn't quite fit. That's a signal the seam is wrong, not a licence — the fix is to change `requireAuth()`, not to route around it.
@@ -37,3 +37,4 @@ Decisions don't expire. They get **superseded** when we learn enough to change t
37
37
  | [0009](./0009-database-engine-substrate.md) | Sqlite is the engine's persistence substrate. Single `runtime/mikser.sqlite` file holds catalog/refs/manifest/journal as `mikser_entities` / `mikser_refs` / `mikser_snapshots` / `mikser_journal` tables. Plugins register schemas via `registerSchema(name, sql)` + `useDatabase()`. Sift→SQL translator with indexed pushdown, LRU cache for `findById`, worker-side read-only sqlite for sync template helpers, chunked journal walks + `iterateEntities` streaming, `useJournal` auto-persist (mutate the yielded entity; no explicit `updateEntry` needed), `--resume` after interrupted cycles. Replaces `Map<id, entity>` + NDJSON across every engine subsystem. | Accepted |
38
38
  | [0010](./0010-plugin-bundles-and-inline-options.md) | Plugin bundles + factory-call form + inline options. Plugins are imported by name and called as factories; `plugins: []` carries factory returns, never strings. Lifecycle plugins are `(options) => (core) => void`; renderers return `{name, options, load?, render?}`; postprocessors return `{name, options, output?, setup?, postprocess, teardown?}`. Per-plugin config moved off `runtime.config.<plugin>` — it arrives as the factory arg and is passed as `config` to `load`/`render`/`setup`/`postprocess`. | Accepted |
39
39
  | [0011](./0011-served-entities-expose-deployed-urls.md) | File and resource entities expose deployed URLs. References to served files (image/video/PDF) are `$`-keyed **served paths** (`/img/X.jpg`, `/media/clip.mp4` — the path content authors, = the entity's `meta.url`), resolving through a new `refFilter` `{ 'meta.url': … }` clause backed by an indexed `meta_url` column (schema 9.0.1 → 9.0.2). No collection-prefixed ids leak into content. (Id-refs were tried and rejected — gpoint references content by served path and its `/media/**` `resources()` library means the entity only exists because content references `/media/…`.) The `files`/`resources` plugins stamp `meta.url`, the `assets` plugin stamps `meta.presets` — so expanding a ref yields the served entity's URL set instead of a string to reconstruct. Base-relative in the live catalog (host-agnostic; consumer holds `base`), absolute in static renders (baked from `runtime.options.url`, so logic-less consumers — email, RSS, foreign apps — read a whole URL); `lookupUrl` render helper resolves a ref to `meta.url` or a named preset. SDK collapses `assetUrl(source, preset, {ext})` into one `url(ref)` join + a dev-mode SPA-fallback detector. *("Served entity" — file/resource/preset — avoids colliding with the `assets` plugin's own "asset reference" term.)* | Accepted (proven against gpoint) |
40
+ | [0012](./0012-auth-seam-in-core-identity-in-plugins.md) | Authentication is an engine seam; identity providers are plugins. Core ships a provider-agnostic verifier contract (`{ name, verify(req), challenge? }`), a constant-time `bearer()`, `resolveAuth()`, `authorize()`/`requireAuth()` and `reachabilityOf()` — replacing the same token+loopback rule hand-copied into `api`, `mcp` and `forms`, where it had already drifted into three behaviours. A configured verifier gates every request (no loopback bypass, WhiteBox posture); a plain `token:` keeps the trusted-local-host model. OAuth, RFC 9728 discovery and identity ship as `mikser-io-auth`; users and groups are Apache-format `htpasswd`/`htgroup` files in the working folder (ADR-0002), never written at runtime. Credentials compose (`anyOf`) and may carry an opaque row `scope` the engine never inspects, `$and`-ed with the endpoint's own query; scoped responses are never cached. No user table, no roles, no permission catalog in core. | Accepted |
package/index.js CHANGED
@@ -1,6 +1,7 @@
1
1
  export { default as runtime } from './src/runtime.js'
2
2
  export * as constants from './src/constants.js'
3
3
  export * from './src/utils.js'
4
+ export * from './src/auth.js'
4
5
  export * from './src/lifecycle.js'
5
6
  export * from './src/database/index.js'
6
7
  export * from './src/journal.js'
@@ -23,6 +24,7 @@ export * from './src/routes.js'
23
24
  // for the v9 plugin shape.
24
25
  export { api } from './src/plugins/api.js'
25
26
  export { assets } from './src/plugins/assets.js'
27
+ export { sources } from './src/plugins/sources.js'
26
28
  export { commands } from './src/plugins/commands.js'
27
29
  export { data } from './src/plugins/data.js'
28
30
  export { documents } from './src/plugins/documents.js'
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mikser-io",
3
- "version": "9.3.1",
3
+ "version": "9.12.0",
4
4
  "description": "<p align=\"center\"> <img src=\"mikser-lockup-stacked.svg\" alt=\"mikser\" width=\"198\" /> </p>",
5
5
  "main": "index.js",
6
6
  "exports": {
@@ -64,8 +64,10 @@
64
64
  "@ai-sdk/openai": "^3.0.71",
65
65
  "fluent-ffmpeg": "^2.1.3",
66
66
  "mikser-io-decap": "file:../mikser-io-decap",
67
+ "mikser-io-layouts": "file:../mikser-io-layouts",
67
68
  "mikser-io-post-mjml": "file:../mikser-io-post-mjml",
68
69
  "mikser-io-post-pdf": "file:../mikser-io-post-pdf",
70
+ "mikser-io-provider-gdrive": "file:../mikser-io-provider-gdrive",
69
71
  "mikser-io-render-eta": "file:../mikser-io-render-eta",
70
72
  "mikser-io-render-liquid": "file:../mikser-io-render-liquid",
71
73
  "mikser-io-render-markdown": "file:../mikser-io-render-markdown",
package/src/auth.js ADDED
@@ -0,0 +1,229 @@
1
+ import crypto from 'node:crypto'
2
+ import { isLoopback } from './utils.js'
3
+
4
+ // Authentication seam (ADR-0012).
5
+ //
6
+ // The engine says HOW a credential is proven. Plugins say WHAT they gate.
7
+ // Nothing here knows about users, sessions, roles, or OAuth — a verifier
8
+ // that does is an external package (mikser-io-auth), plugged in at the
9
+ // same seam a bare token string plugs into.
10
+ //
11
+ // A verifier is a provider-agnostic descriptor:
12
+ //
13
+ // {
14
+ // name, // for logs: 'bearer', 'oauth', …
15
+ // async verify(req), // → null (no credential presented)
16
+ // // false (presented, rejected)
17
+ // // { subject, capabilities?, scope? }
18
+ // challenge?(req, res), // set WWW-Authenticate before a 401
19
+ // authorizationServers?, resource?, scopesSupported?, // RFC 9728
20
+ // }
21
+ //
22
+ // A principal carries three things, and the engine understands only two:
23
+ //
24
+ // subject who this is — a string, for logs and audit
25
+ // capabilities which verbs they hold, or null for "not verb-scoped"
26
+ // scope which ROWS they may see — an OPAQUE value the engine
27
+ // never inspects. The plugin that issued the credential's
28
+ // surface is the only thing that knows what it means (for
29
+ // api/vector/data it's a sift filter ANDed with the
30
+ // endpoint's own `query`). Keeping it opaque here is what
31
+ // stops the engine from growing an opinion about content.
32
+ //
33
+ // `verify()` is the primitive; requireAuth() is the Express convenience
34
+ // built on it. Both are exported because a surface whose denials aren't
35
+ // HTTP-shaped (MCP answers in JSON-RPC) needs the primitive directly.
36
+
37
+ // The static long-lived token verifier — mikser's default and, for a build
38
+ // tool driven by config files, usually the only one anybody needs.
39
+ export function bearer({ token, name = 'bearer', subject = 'token', capabilities = null, scope = null } = {}) {
40
+ if (!token) throw new Error('bearer({ token }) requires a token')
41
+ const expected = Buffer.from(`Bearer ${token}`, 'utf8')
42
+
43
+ return {
44
+ name,
45
+ capabilitiesDeclared: capabilities,
46
+ async verify(req) {
47
+ const header = req.headers?.authorization ?? req.get?.('authorization')
48
+ if (!header) return null
49
+ const presented = Buffer.from(header, 'utf8')
50
+ // timingSafeEqual throws on a length mismatch, so the length
51
+ // check has to come first — it leaks length, which a Bearer
52
+ // header already does structurally.
53
+ const ok = presented.length === expected.length &&
54
+ crypto.timingSafeEqual(presented, expected)
55
+ return ok ? { subject, capabilities, scope } : false
56
+ },
57
+ challenge(req, res) {
58
+ res.set('WWW-Authenticate', 'Bearer')
59
+ },
60
+ }
61
+ }
62
+
63
+ // Normalize whatever a plugin's `auth:` option holds into a verifier, or
64
+ // null for "nothing configured". Accepts:
65
+ //
66
+ // undefined / null → null
67
+ // 'sekrit' → bearer({ token })
68
+ // { token: 'sekrit' } → bearer({ token }) (the ep.token shape)
69
+ // { verify: async fn } → used as-is (mikser-io-auth, custom verifiers)
70
+ // async (req) => {…} → wrapped as a bare verify function
71
+ //
72
+ // Returning null rather than throwing is deliberate: "no auth configured"
73
+ // is a legitimate, common state for a build tool running on localhost.
74
+ // What it MEANS is the caller's decision — see requireAuth's allowRemote.
75
+ export function resolveAuth(config) {
76
+ if (!config) return null
77
+ // An array is "any of these credentials will do" — several machine
78
+ // tokens, or machine tokens alongside a human's OAuth token. See anyOf().
79
+ if (Array.isArray(config)) {
80
+ const verifiers = config.map(resolveAuth).filter(Boolean)
81
+ if (!verifiers.length) return null
82
+ return verifiers.length === 1 ? verifiers[0] : anyOf(...verifiers)
83
+ }
84
+ if (typeof config === 'string') return bearer({ token: config })
85
+ // An object carrying verify() wins BEFORE the bare-function branch: a
86
+ // verifier may itself be a callable (mikser-io-auth's auth() is both the
87
+ // plugin and the verifier), and treating that as a raw verify function
88
+ // would call the plugin with a request.
89
+ if (typeof config.verify === 'function') return config
90
+ if (typeof config === 'function') return { name: 'custom', verify: config }
91
+ if (config.token) return bearer({ token: config.token })
92
+ return null
93
+ }
94
+
95
+ // How exposed is a surface? The vocabulary registerRoute() already uses.
96
+ // Computed here so a facade's view of exposure stops being a per-plugin
97
+ // promise (three plugins, three hand-rolled copies — see ADR-0012).
98
+ export function reachabilityOf({ auth, token, allowRemote } = {}) {
99
+ if (auth || token) return 'token'
100
+ return allowRemote ? 'public' : 'loopback'
101
+ }
102
+
103
+ // The uniform rule, in one place.
104
+ //
105
+ // verifier configured:
106
+ // credential valid → allow, from anywhere
107
+ // credential presented, invalid → 401
108
+ // credential absent → 401 (loopback does NOT bypass a
109
+ // configured verifier — same posture
110
+ // as WhiteBox), unless trustLoopback
111
+ // no verifier:
112
+ // loopback, or allowRemote → allow
113
+ // otherwise → 403
114
+ //
115
+ // `trustLoopback: true` restores the older mikser behaviour where a
116
+ // token-gated endpoint stayed open to localhost. It exists so the api and
117
+ // mcp plugins can keep their documented semantics for a plain `token:`
118
+ // while a real verifier gets the stricter default.
119
+ export async function authorize(req, verifier, { allowRemote = false, trustLoopback = false } = {}) {
120
+ const local = isLoopback(req.ip)
121
+
122
+ if (verifier) {
123
+ const result = await verifier.verify(req)
124
+ if (result) return { ok: true, principal: result }
125
+ if (result === false) {
126
+ return { ok: false, status: 401, reason: 'invalid',
127
+ error: 'Invalid credential' }
128
+ }
129
+ // Nothing presented.
130
+ if (trustLoopback && local) return { ok: true, principal: { subject: 'loopback' } }
131
+ return { ok: false, status: 401, reason: 'missing',
132
+ error: 'Authentication required' }
133
+ }
134
+
135
+ if (allowRemote || local) return { ok: true, principal: { subject: 'anonymous' } }
136
+ return {
137
+ ok: false, status: 403, reason: 'reachability',
138
+ error: 'Endpoint accepts loopback connections only — configure a token or set allowRemote: true to enable remote access',
139
+ }
140
+ }
141
+
142
+ // Express flavour of the same rule.
143
+ export function requireAuth(verifier, options = {}) {
144
+ return async (req, res, next) => {
145
+ let outcome
146
+ try {
147
+ outcome = await authorize(req, verifier, options)
148
+ } catch (err) {
149
+ options.logger?.error?.('auth: verifier threw — %s', err.message)
150
+ return res.status(500).json({ error: 'Authentication failed' })
151
+ }
152
+ if (outcome.ok) {
153
+ req.principal = outcome.principal
154
+ return next()
155
+ }
156
+ if (outcome.status === 401) verifier?.challenge?.(req, res)
157
+ res.status(outcome.status).json({ error: outcome.error })
158
+ }
159
+ }
160
+
161
+ // Accept ANY of several credentials on one surface.
162
+ //
163
+ // This is the shape a real deployment reaches for almost immediately: a
164
+ // handful of long-lived machine tokens (one per caller, so a leaked token
165
+ // scopes the blast radius) plus a human's OAuth token on the same routes.
166
+ // It lives here rather than in userland because the last project that
167
+ // hand-rolled it shipped a silent auth BYPASS — a composed verifier that
168
+ // the resolver didn't recognise degraded to a pass-through `next()` and
169
+ // merely warned. The failure mode of getting this wrong is not "denied",
170
+ // it is "wide open", so it belongs in one tested place.
171
+ //
172
+ // The three-valued merge is the whole subtlety:
173
+ //
174
+ // any verifier accepts → accept (first wins)
175
+ // all say "nothing presented" → null (loopback policy may still apply)
176
+ // any says "presented, bad" → false (NEVER falls back to loopback —
177
+ // a caller who presented a credential has
178
+ // identified itself and must be judged)
179
+ export function anyOf(...verifiers) {
180
+ const list = verifiers.filter(Boolean)
181
+ if (!list.length) return null
182
+ if (list.length === 1) return list[0]
183
+
184
+ // Preserve discovery metadata so an MCP client can still find the
185
+ // authorization server through the composite. First verifier that
186
+ // advertises one wins; a static token has nothing to advertise.
187
+ const discovering = list.find(v => v.authorizationServers?.length)
188
+
189
+ return {
190
+ name: `anyOf(${list.map(v => v.name ?? '?').join(',')})`,
191
+ authorizationServers: discovering?.authorizationServers,
192
+ resource: discovering?.resource,
193
+ scopesSupported: [...new Set(list.flatMap(v => v.scopesSupported ?? []))],
194
+
195
+ async verify(req) {
196
+ let rejected = false
197
+ for (const verifier of list) {
198
+ const result = await verifier.verify(req)
199
+ if (result) return result
200
+ if (result === false) rejected = true
201
+ }
202
+ return rejected ? false : null
203
+ },
204
+
205
+ // Challenge with the verifier that can actually be satisfied
206
+ // interactively — pointing a browser at "Bearer" when the real
207
+ // option is OAuth discovery helps nobody.
208
+ challenge(req, res) {
209
+ const chooser = discovering ?? list.find(v => v.challenge)
210
+ chooser?.challenge?.(req, res)
211
+ },
212
+ }
213
+ }
214
+
215
+ // Does this principal hold a capability?
216
+ //
217
+ // `capabilities: null` means "this credential is not capability-scoped" —
218
+ // a bare static token with nothing declared, which is every endpoint that
219
+ // worked before ADR-0012. Such a credential passes any capability check,
220
+ // because the endpoint's own `operations` list is what bounds it. A
221
+ // credential that DOES declare capabilities is bound by them as well, so
222
+ // a request's reach is the INTERSECTION of the endpoint's ceiling and the
223
+ // principal's grant — never the union.
224
+ export function hasCapability(principal, capability) {
225
+ if (!capability) return true
226
+ const caps = principal?.capabilities
227
+ if (caps == null) return true
228
+ return caps.includes(capability)
229
+ }
package/src/config.js CHANGED
@@ -1,12 +1,34 @@
1
1
  import runtime from './runtime.js'
2
2
  import { useLogger } from './engine.js'
3
3
  import { onLoad } from './lifecycle.js'
4
+ import { checksum } from './utils.js'
4
5
  import path from 'node:path'
5
6
 
6
7
  onLoad(async () => {
7
8
  const logger = useLogger()
8
9
  const configFile = path.resolve(runtime.options.config)
9
10
  logger.info('Config: %s', configFile)
11
+
12
+ // Stamp the config so a change to it can invalidate the derived cache.
13
+ //
14
+ // Without this, editing mikser.config.js invalidated NOTHING: flipping
15
+ // an option that changes every page's destination reported "36 unchanged"
16
+ // and left the previous output in place. The config was genuinely read —
17
+ // --force applied it immediately — it simply took part in no
18
+ // invalidation, so the only symptom was output that did not match the
19
+ // config, with nothing saying so.
20
+ //
21
+ // The file's bytes only. A config that imports other modules will not
22
+ // notice a change in those, which is a real limit worth knowing rather
23
+ // than a reason to hash the whole module graph.
24
+ try {
25
+ runtime.options.configChecksum = await checksum(configFile)
26
+ } catch {
27
+ // No config file is a legitimate state (defaults all the way down);
28
+ // absent stamp means "nothing to compare", not "changed".
29
+ runtime.options.configChecksum = null
30
+ }
31
+
10
32
  try {
11
33
  const config = await import(configFile)
12
34
  if (typeof config.default == 'function') {
@@ -247,10 +247,38 @@ export function createSqliteDatabase({
247
247
  handle = new Database(dbPath)
248
248
  setupConnection()
249
249
 
250
- const recorded = handle.prepare('SELECT value FROM mikser_meta WHERE key = ?')
251
- .get('schema_version')?.value
250
+ const stmtMeta = handle.prepare('SELECT value FROM mikser_meta WHERE key = ?')
251
+ const recorded = stmtMeta.get('schema_version')?.value
252
+
253
+ // A config change invalidates the cache for the same reason a version
254
+ // change does: the derived state was computed under different rules.
255
+ //
256
+ // Before this, editing mikser.config.js invalidated nothing — flipping
257
+ // an option that changes every page's destination reported "36
258
+ // unchanged" and left the previous output in place. The config was
259
+ // read; it simply took part in no invalidation, so the only symptom
260
+ // was output that did not match the config and nothing saying so.
261
+ //
262
+ // Treated exactly like a version mismatch rather than something
263
+ // narrower: a config edit can change how sources are PARSED (a mapper
264
+ // transform, documents() options) as well as how they are rendered,
265
+ // so invalidating only the render manifest would still leave stale
266
+ // entities. Per ADR-0002 the files are the source of truth, so
267
+ // rebuilding is always safe — just slower.
268
+ const recordedConfig = stmtMeta.get('config_checksum')?.value
269
+ const currentConfig = runtime.options.configChecksum ?? null
270
+ const configChanged = Boolean(recordedConfig && currentConfig && recordedConfig !== currentConfig)
271
+
252
272
  let upgradedFromVersion = null
253
- if (recorded && recorded !== version) {
273
+ if (configChanged && !(recorded && recorded !== version)) {
274
+ logger?.warn(
275
+ 'Config changed since the last run. Wiping the cache and rebuilding from sources ' +
276
+ '(files are the source of truth — no source data is affected). Note this tracks the ' +
277
+ 'bytes of %s only: a change in a module it imports is not seen.',
278
+ runtime.options.config,
279
+ )
280
+ }
281
+ if ((recorded && recorded !== version) || configChanged) {
254
282
  // Schema mismatch on upgrade or downgrade. Per ADR-0002 the
255
283
  // files on disk are the source of truth and this database
256
284
  // is a derived cache, so the right behavior is to wipe the
@@ -261,10 +289,12 @@ export function createSqliteDatabase({
261
289
  // expect a cold-start rebuild on this run. No data loss
262
290
  // beyond the cache itself; everything in mikser.sqlite is
263
291
  // recoverable from the working folder.
264
- logger?.warn(
265
- 'Database schema mismatch: stored=%s, current=%s. Wiping the cache and rebuilding from sources (files are the source of truth — no source data is affected).',
266
- recorded, version,
267
- )
292
+ if (recorded && recorded !== version) {
293
+ logger?.warn(
294
+ 'Database schema mismatch: stored=%s, current=%s. Wiping the cache and rebuilding from sources (files are the source of truth — no source data is affected).',
295
+ recorded, version,
296
+ )
297
+ }
268
298
  handle.close()
269
299
  handle = null
270
300
 
@@ -277,12 +307,16 @@ export function createSqliteDatabase({
277
307
  }
278
308
  }
279
309
 
280
- upgradedFromVersion = recorded
310
+ // Non-null marks the provisioning context as firstRun/upgraded,
311
+ // which is the right shape for a config change too: what the
312
+ // provisioners see is an empty-state database either way.
313
+ upgradedFromVersion = recorded ?? 'config'
281
314
  handle = new Database(dbPath)
282
315
  setupConnection()
283
316
  }
284
- handle.prepare('INSERT OR REPLACE INTO mikser_meta (key, value) VALUES (?, ?)')
285
- .run('schema_version', version)
317
+ const stmtStamp = handle.prepare('INSERT OR REPLACE INTO mikser_meta (key, value) VALUES (?, ?)')
318
+ stmtStamp.run('schema_version', version)
319
+ if (currentConfig) stmtStamp.run('config_checksum', currentConfig)
286
320
 
287
321
  // Build provisioning context. firstRun is true when the file
288
322
  // didn't exist before this open OR when the schema mismatch
package/src/manager.js CHANGED
@@ -4,6 +4,7 @@ import cron from 'node-cron'
4
4
  import { onProcess, onFinalized } from './lifecycle.js'
5
5
  import { useLogger } from './engine.js'
6
6
  import { ACTION } from './constants.js'
7
+ import { junkFilter } from './utils.js'
7
8
 
8
9
  const tasks = []
9
10
 
@@ -67,7 +68,13 @@ export async function deletedHook(name, context) {
67
68
  }
68
69
  }
69
70
 
70
- export function watch(name, folder, options = { interval: 1000, binaryInterval: 3000, ignored: /[\/\\]\./, ignoreInitial: true }) {
71
+ // Dot-prefixed anything, plus the OS/file-manager litter that is NOT
72
+ // dot-prefixed — Thumbs.db and desktop.ini were measurably being watched.
73
+ // A function rather than a regex because chokidar 4+ dropped glob support in
74
+ // `ignored` and a function is the one form that has stayed stable.
75
+ const ignoreJunk = (filePath) => /[/\\]\./.test(filePath) || junkFilter()(filePath)
76
+
77
+ export function watch(name, folder, options = { interval: 1000, binaryInterval: 3000, ignored: ignoreJunk, ignoreInitial: true }) {
71
78
  if (runtime.options.watch !== true) return
72
79
 
73
80
  chokidar.watch(folder, options)