mikser-io 9.72.0 → 9.73.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/configuration.md +27 -12
- package/docs/diagnostics.md +11 -7
- package/package.json +12 -1
- package/src/engine.js +46 -11
- package/src/plugins/render/asset.js +5 -3
- package/src/plugins/render/file.js +1 -1
- package/src/plugins/render/href.js +3 -4
- package/src/plugins/render/resource.js +2 -3
- package/src/references.js +15 -25
- package/src/utils.js +47 -0
- package/10.0-PLAN.md +0 -182
- package/CLAUDE.md +0 -787
- package/CLOUD-PLAN.md +0 -147
- package/runtime/mikser.sqlite +0 -0
- package/runtime/mikser.sqlite-shm +0 -0
- package/runtime/mikser.sqlite-wal +0 -0
package/docs/configuration.md
CHANGED
|
@@ -70,24 +70,39 @@ Which subtrees of the output folder are deployed as their own domain root.
|
|
|
70
70
|
|
|
71
71
|
```js
|
|
72
72
|
export default {
|
|
73
|
-
// out/bg
|
|
73
|
+
// out/bg, out/en and out/mk each deploy to their own domain
|
|
74
74
|
siteRoots: ['bg', 'en', 'mk'],
|
|
75
75
|
}
|
|
76
76
|
```
|
|
77
77
|
|
|
78
|
-
Read
|
|
79
|
-
does, and a browser cannot climb above the origin root — it discards the
|
|
80
|
-
extra `..` and loads the file. Where the site root actually is therefore
|
|
81
|
-
decides whether `../../x.svg` on a given page is correct, merely over-deep,
|
|
82
|
-
or broken.
|
|
78
|
+
Read by the url helpers and by the broken-reference check.
|
|
83
79
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
80
|
+
**It changes the output.** `asset`, `href` and `resource` build a path from the
|
|
81
|
+
page to the target, and without this they measure from the output folder. When
|
|
82
|
+
`out/bg` is what gets deployed, that is one directory too far — every url
|
|
83
|
+
carries an extra `..` for the site segment. A browser floors a climb above the
|
|
84
|
+
origin root rather than failing, so those urls load and nothing reports them.
|
|
85
|
+
Declaring the roots makes the helpers measure from the site instead, and the
|
|
86
|
+
urls become what they should have been.
|
|
87
|
+
|
|
88
|
+
The check reads it for the same reason, and resolves urls the way a browser
|
|
89
|
+
does. Where the site root is decides whether `../../x.svg` on a given page is
|
|
90
|
+
correct, merely over-deep, or broken.
|
|
88
91
|
|
|
89
|
-
|
|
90
|
-
|
|
92
|
+
Default is the output folder itself, which is right for the ordinary case of
|
|
93
|
+
one site per build — and with nothing declared every url is byte-identical to
|
|
94
|
+
what it was. Only a build that emits several sites moves, and it moves from
|
|
95
|
+
working-by-flooring to correct.
|
|
96
|
+
|
|
97
|
+
Declaring it also asserts something: **a site root is a deployable unit.** It
|
|
98
|
+
is served alone, so anything its pages reference has to exist beneath it —
|
|
99
|
+
share a common assets folder into each root rather than beside them. A url to
|
|
100
|
+
a target in a *different* root is left as it was, because on a per-domain
|
|
101
|
+
deploy no relative path reaches another origin; the check will report it, which
|
|
102
|
+
is the honest answer.
|
|
103
|
+
|
|
104
|
+
Nothing can infer any of this: it is a fact about where the bytes get deployed,
|
|
105
|
+
not about the bytes.
|
|
91
106
|
|
|
92
107
|
## Engine Substrate
|
|
93
108
|
|
package/docs/diagnostics.md
CHANGED
|
@@ -231,7 +231,7 @@ the whole question:
|
|
|
231
231
|
```json
|
|
232
232
|
{ "destination": "/index.html", "reason": "query-matched",
|
|
233
233
|
"matched": { "filter": { "id": { "$regex": "^/documents/devices/" } },
|
|
234
|
-
"by": "/documents/devices/
|
|
234
|
+
"by": "/documents/devices/model-a.md" } }
|
|
235
235
|
```
|
|
236
236
|
|
|
237
237
|
A `matched.filter` of `null` is a different statement: the page's predicate
|
|
@@ -887,12 +887,16 @@ surfaces that turn silence into a statement:
|
|
|
887
887
|
same file, the output scan reports it and this one stays quiet.
|
|
888
888
|
- **A link that works only by accident** — a url with one `..` too many
|
|
889
889
|
still loads, because a browser discards a climb above the origin root
|
|
890
|
-
rather than failing.
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
890
|
+
rather than failing. Reported under `reference-over-deep`, separately
|
|
891
|
+
from the outright failures, and grouped by how far each climbed:
|
|
892
|
+
- **One url, or several climbing different distances** — each is a
|
|
893
|
+
latent 404, working today and broken as soon as the same markup
|
|
894
|
+
renders one level deeper.
|
|
895
|
+
- **Every url climbing the same distance** — not N problems but one
|
|
896
|
+
base that is off by a constant, reported once and flagged
|
|
897
|
+
`structural` in `--json`. The urls work at every depth. Usually it
|
|
898
|
+
means `siteRoots` is undeclared for a build that emits several
|
|
899
|
+
sites; see [configuration](./configuration.md#siteroots).
|
|
896
900
|
|
|
897
901
|
## See also
|
|
898
902
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "mikser-io",
|
|
3
|
-
"version": "9.
|
|
3
|
+
"version": "9.73.0",
|
|
4
|
+
"files": [
|
|
5
|
+
"app.js",
|
|
6
|
+
"index.js",
|
|
7
|
+
"src/",
|
|
8
|
+
"testing/",
|
|
9
|
+
"docs/",
|
|
10
|
+
"favicon.ico",
|
|
11
|
+
"favicon.svg",
|
|
12
|
+
"mikser-mark.svg",
|
|
13
|
+
"mikser-lockup-stacked.svg"
|
|
14
|
+
],
|
|
4
15
|
"description": "A mixer for content: entities in, configurable render pipelines, outputs of any kind. Static sites are the canonical recipe, not the definition — the same engine renders PDFs, emails and whatever a renderer plugin produces. Files are the source of truth, every lifecycle phase is observable, and the build graph is queryable by an agent.",
|
|
5
16
|
"main": "index.js",
|
|
6
17
|
"exports": {
|
package/src/engine.js
CHANGED
|
@@ -127,19 +127,45 @@ async function reportBrokenReferences(logger) {
|
|
|
127
127
|
broken.length, checked, broken.length > SHOWN ? `, ${SHOWN} shown` : '')
|
|
128
128
|
}
|
|
129
129
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
130
|
+
// Grouped by how FAR each climbed, because a site whose every over-deep url
|
|
131
|
+
// climbs the same distance does not have N problems — it has one base that
|
|
132
|
+
// is off by a constant. Printing it N times is precisely how a real signal
|
|
133
|
+
// gets filtered out, which is the failure this check exists to prevent.
|
|
134
|
+
const byClimb = new Map()
|
|
135
|
+
for (const entry of overDeep) {
|
|
136
|
+
if (!byClimb.has(entry.floored)) byClimb.set(entry.floored, [])
|
|
137
|
+
byClimb.get(entry.floored).push(entry)
|
|
134
138
|
}
|
|
139
|
+
// One distance, many urls: structural. The helper's base is wrong, the urls
|
|
140
|
+
// are not — they load at every depth, because the climb is always floored.
|
|
141
|
+
// That wants a different reaction than a hand-written `../..` that happens
|
|
142
|
+
// to be right on one page and is a 404 waiting on the next.
|
|
143
|
+
const structural = byClimb.size === 1 && overDeep.length > 1
|
|
144
|
+
|
|
145
|
+
for (const [climb, entries] of [...byClimb].sort(([a], [b]) => a - b)) {
|
|
146
|
+
const examples = entries.slice(0, 3).map(e => e.url)
|
|
147
|
+
logger.warn(
|
|
148
|
+
{
|
|
149
|
+
code: 'reference-over-deep', climbs: climb, count: entries.length,
|
|
150
|
+
structural, urls: examples,
|
|
151
|
+
files: [...new Set(entries.flatMap(e => e.files))].slice(0, 3),
|
|
152
|
+
},
|
|
153
|
+
structural
|
|
154
|
+
? '%d references climb %d level(s) above the site root — every one of them, by the '
|
|
155
|
+
+ 'same amount. They load: a browser discards the extra `..`. What is wrong is the '
|
|
156
|
+
+ 'base they were built from, not the links. Examples: %s'
|
|
157
|
+
: '%d reference(s) climb %d level(s) above the site root and load only because a '
|
|
158
|
+
+ 'browser discards the extra `..`. Each breaks if the same markup renders one '
|
|
159
|
+
+ 'level deeper. Examples: %s',
|
|
160
|
+
entries.length, climb, examples.join(', '))
|
|
161
|
+
}
|
|
162
|
+
|
|
135
163
|
if (overDeep.length) {
|
|
136
|
-
logger.warn({ code: 'reference-over-deep-summary', overDeep: overDeep.length, checked },
|
|
137
|
-
'%d of %d reference(s)
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
siteRoots.length ? '' : ' No siteRoots are declared, so this resolved against the '
|
|
142
|
-
+ 'output root — declare runtime.config.siteRoots if a subtree is deployed as its own domain.')
|
|
164
|
+
logger.warn({ code: 'reference-over-deep-summary', overDeep: overDeep.length, checked, structural },
|
|
165
|
+
'%d of %d reference(s) resolve above the site root.%s',
|
|
166
|
+
overDeep.length, checked,
|
|
167
|
+
siteRoots.length ? '' : ' No siteRoots are declared, so this resolved against the output '
|
|
168
|
+
+ 'root — declare siteRoots if a subtree is deployed as its own domain.')
|
|
143
169
|
}
|
|
144
170
|
|
|
145
171
|
return new Set(broken.map(b => b.target))
|
|
@@ -472,6 +498,15 @@ export async function setup(options) {
|
|
|
472
498
|
// runtime.config) and before any plugin's onLoaded.
|
|
473
499
|
onLoad(async () => {
|
|
474
500
|
const logger = useLogger()
|
|
501
|
+
|
|
502
|
+
// Onto OPTIONS, not left on config: the url helpers read it, and they
|
|
503
|
+
// run in render workers, which receive the worker-safe options and
|
|
504
|
+
// never see runtime.config.
|
|
505
|
+
runtime.options.siteRoots = runtime.config?.siteRoots ?? []
|
|
506
|
+
if (runtime.options.siteRoots.length) {
|
|
507
|
+
logger.info('Site roots: %s', runtime.options.siteRoots.join(', '))
|
|
508
|
+
}
|
|
509
|
+
|
|
475
510
|
const cli = runtime.options.url
|
|
476
511
|
const cfg = runtime.config?.url
|
|
477
512
|
const raw = cli ?? cfg
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import path from 'node:path'
|
|
2
2
|
|
|
3
|
-
import { changeExtension } from '../../utils.js'
|
|
3
|
+
import { changeExtension, siteRelativeUrl } from '../../utils.js'
|
|
4
4
|
|
|
5
5
|
// `{{asset 'web' '/media/hero.jpg'}}` — the deployed URL of a preset
|
|
6
6
|
// derivative, relative to the page asking for it.
|
|
@@ -71,8 +71,10 @@ export function load({ runtime, entity, state, options, logger, track }) {
|
|
|
71
71
|
// itself — it takes a path, not an entity, so there is nothing to look
|
|
72
72
|
// up and the URL is well-formed whether or not anything produced it.
|
|
73
73
|
track?.asset?.(destination)
|
|
74
|
-
|
|
75
|
-
|
|
74
|
+
// Resolved within the site this page belongs to, not against the
|
|
75
|
+
// output root — see siteRelativeUrl. With no siteRoots declared the two
|
|
76
|
+
// are the same folder and the url is byte-identical.
|
|
77
|
+
return { url: siteRelativeUrl(entity.destination, destination, options?.siteRoots) }
|
|
76
78
|
}
|
|
77
79
|
}
|
|
78
80
|
|
|
@@ -72,7 +72,7 @@ function warnIfUntrackable(options, resolved, logger) {
|
|
|
72
72
|
//
|
|
73
73
|
// The first version of this hardcoded five content folders, and a project
|
|
74
74
|
// registering its own collections through sources() has more than five. On
|
|
75
|
-
//
|
|
75
|
+
// one real site that meant 63 warnings per build, one for every stylesheet and
|
|
76
76
|
// script, all of them tracked correctly and every one of them saying the
|
|
77
77
|
// opposite. Which is worse than not warning: 63 spurious lines a build
|
|
78
78
|
// teaches you to filter the channel, and the filtered-out line is the real
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import path from 'node:path'
|
|
2
|
+
import { siteRelativeUrl } from '../../utils.js'
|
|
2
3
|
|
|
3
4
|
export function load({ entity, runtime, options }) {
|
|
4
5
|
const { clear } = options
|
|
@@ -19,16 +20,14 @@ export function load({ entity, runtime, options }) {
|
|
|
19
20
|
|
|
20
21
|
let found = runtime.hrefLang(href)
|
|
21
22
|
if (!found) {
|
|
22
|
-
|
|
23
|
-
return { url: path.relative(from, href) }
|
|
23
|
+
return { url: siteRelativeUrl(entity.destination, href, options?.siteRoots) }
|
|
24
24
|
} else {
|
|
25
25
|
if (!found.id) {
|
|
26
26
|
found = found[lang]
|
|
27
27
|
}
|
|
28
28
|
if (found?.destination) {
|
|
29
29
|
const destination = clear ? found.destination.replace('index.html', '') : found.destination
|
|
30
|
-
|
|
31
|
-
found.url = path.relative(from, destination)
|
|
30
|
+
found.url = siteRelativeUrl(entity.destination, destination, options?.siteRoots)
|
|
32
31
|
}
|
|
33
32
|
return found
|
|
34
33
|
}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import path from 'node:path'
|
|
2
|
-
import { matchesLibrary } from '../../utils.js'
|
|
2
|
+
import { matchesLibrary, siteRelativeUrl } from '../../utils.js'
|
|
3
3
|
|
|
4
4
|
export function load({ runtime, entity, state, options, track }) {
|
|
5
5
|
runtime.resource = (url) => {
|
|
@@ -17,8 +17,7 @@ export function load({ runtime, entity, state, options, track }) {
|
|
|
17
17
|
// than resolving one, so a library that was never copied
|
|
18
18
|
// yields a link to nothing on a green build.
|
|
19
19
|
track?.asset?.(destination)
|
|
20
|
-
|
|
21
|
-
return { url: path.relative(from, destination), name }
|
|
20
|
+
return { url: siteRelativeUrl(entity.destination, destination, options?.siteRoots), name }
|
|
22
21
|
}
|
|
23
22
|
}
|
|
24
23
|
}
|
package/src/references.js
CHANGED
|
@@ -25,6 +25,7 @@ import path from 'node:path'
|
|
|
25
25
|
import { existsSync } from 'node:fs'
|
|
26
26
|
import { readFile } from 'node:fs/promises'
|
|
27
27
|
import { globby } from 'globby'
|
|
28
|
+
import { siteRootFor } from './utils.js'
|
|
28
29
|
|
|
29
30
|
// Documents that can carry a reference. Anything else in the output is either
|
|
30
31
|
// an asset itself or something whose internal structure this has no business
|
|
@@ -82,6 +83,8 @@ function decodeEntities(source) {
|
|
|
82
83
|
}
|
|
83
84
|
|
|
84
85
|
// Everything a page points at, as raw url strings.
|
|
86
|
+
export { siteRootFor }
|
|
87
|
+
|
|
85
88
|
export function extractReferences(rawSource) {
|
|
86
89
|
const source = decodeEntities(rawSource)
|
|
87
90
|
const found = new Set()
|
|
@@ -111,30 +114,17 @@ export function resolveUrl(pageDir, url, { root = '' } = {}) {
|
|
|
111
114
|
const segments = clean.split('/').filter(s => s !== '' && s !== '.')
|
|
112
115
|
|
|
113
116
|
const parts = absolute ? [] : pageDir.split('/').filter(Boolean)
|
|
114
|
-
|
|
117
|
+
// How FAR above the root it climbed, not merely that it did. When every
|
|
118
|
+
// over-deep url on a site climbs the same distance, that is one base
|
|
119
|
+
// mismatch reported once — not N findings, which is how a real signal gets
|
|
120
|
+
// filtered.
|
|
121
|
+
let floored = 0
|
|
115
122
|
for (const segment of segments) {
|
|
116
123
|
if (segment !== '..') { parts.push(segment); continue }
|
|
117
124
|
if (parts.length) parts.pop()
|
|
118
|
-
else
|
|
119
|
-
}
|
|
120
|
-
return { target: path.join(root, ...parts), overDeep }
|
|
121
|
-
}
|
|
122
|
-
|
|
123
|
-
// Which declared site root a file belongs to.
|
|
124
|
-
//
|
|
125
|
-
// lmed emits one subtree per language and deploys each as its own domain root
|
|
126
|
-
// (out/bg becomes lmed.bg), so the site root is out/<lang>/ and every url
|
|
127
|
-
// carries one extra `..` for the language segment that the browser then floors.
|
|
128
|
-
// Resolving against the output root instead would miss the over-escape entirely
|
|
129
|
-
// and report working urls as broken. Nothing can derive this — it is deployment
|
|
130
|
-
// intent — so it is declared, and the default is the output root itself.
|
|
131
|
-
export function siteRootFor(relativeFile, roots) {
|
|
132
|
-
let best = ''
|
|
133
|
-
for (const root of roots) {
|
|
134
|
-
if (!root) continue
|
|
135
|
-
if (relativeFile.startsWith(`${root}/`) && root.length > best.length) best = root
|
|
125
|
+
else floored++ // a climb above the root, discarded
|
|
136
126
|
}
|
|
137
|
-
return
|
|
127
|
+
return { target: path.join(root, ...parts), overDeep: floored > 0, floored }
|
|
138
128
|
}
|
|
139
129
|
|
|
140
130
|
// Everything the output points at that is not there.
|
|
@@ -150,7 +140,7 @@ export async function checkReferences(outputFolder, { siteRoots = [] } = {}) {
|
|
|
150
140
|
})
|
|
151
141
|
|
|
152
142
|
const broken = new Map()
|
|
153
|
-
const
|
|
143
|
+
const overDeepRefs = new Map()
|
|
154
144
|
let checked = 0
|
|
155
145
|
// Existence is the expensive part and the same target repeats across a
|
|
156
146
|
// site — one lookup each.
|
|
@@ -166,7 +156,7 @@ export async function checkReferences(outputFolder, { siteRoots = [] } = {}) {
|
|
|
166
156
|
const pageDir = path.dirname(file).slice(root.length).replace(/^\/+/, '')
|
|
167
157
|
|
|
168
158
|
for (const url of extractReferences(source)) {
|
|
169
|
-
const { target, overDeep
|
|
159
|
+
const { target, overDeep, floored } = resolveUrl(pageDir, url, { root })
|
|
170
160
|
checked++
|
|
171
161
|
|
|
172
162
|
if (!exists.has(target)) {
|
|
@@ -174,14 +164,14 @@ export async function checkReferences(outputFolder, { siteRoots = [] } = {}) {
|
|
|
174
164
|
}
|
|
175
165
|
// Broken outranks over-deep: a url that resolves nowhere is the
|
|
176
166
|
// failure, and adding that it is also one level too deep is noise.
|
|
177
|
-
const bucket = !exists.get(target) ? broken : (
|
|
167
|
+
const bucket = !exists.get(target) ? broken : (overDeep ? overDeepRefs : null)
|
|
178
168
|
if (!bucket) continue
|
|
179
169
|
|
|
180
170
|
const key = `${target} ${url}`
|
|
181
|
-
if (!bucket.has(key)) bucket.set(key, { url, target, files: [] })
|
|
171
|
+
if (!bucket.has(key)) bucket.set(key, { url, target, floored, files: [] })
|
|
182
172
|
bucket.get(key).files.push(file)
|
|
183
173
|
}
|
|
184
174
|
}
|
|
185
175
|
|
|
186
|
-
return { broken: [...broken.values()], overDeep: [...
|
|
176
|
+
return { broken: [...broken.values()], overDeep: [...overDeepRefs.values()], checked }
|
|
187
177
|
}
|
package/src/utils.js
CHANGED
|
@@ -1343,3 +1343,50 @@ export function matchesLibrary(value, pattern) {
|
|
|
1343
1343
|
}
|
|
1344
1344
|
return libraryPatterns.get(pattern).test(value)
|
|
1345
1345
|
}
|
|
1346
|
+
|
|
1347
|
+
// Which declared site root a path belongs to.
|
|
1348
|
+
//
|
|
1349
|
+
// A build can emit one subtree per language and deploy each as its own domain
|
|
1350
|
+
// root, which puts the site root at out/<lang>/ rather than at out/. Nothing
|
|
1351
|
+
// can derive that — it is a fact about where the bytes get deployed, not about
|
|
1352
|
+
// the bytes — so it is declared as `siteRoots` and the default is the output
|
|
1353
|
+
// root itself. Accepts a path with or without a leading slash, because an
|
|
1354
|
+
// entity destination has one and an output-relative file path does not.
|
|
1355
|
+
export function siteRootFor(file, roots = []) {
|
|
1356
|
+
const relative = String(file ?? '').replace(/^\/+/, '')
|
|
1357
|
+
let best = ''
|
|
1358
|
+
for (const root of roots) {
|
|
1359
|
+
if (!root) continue
|
|
1360
|
+
if (relative.startsWith(`${root}/`) && root.length > best.length) best = root
|
|
1361
|
+
}
|
|
1362
|
+
return best
|
|
1363
|
+
}
|
|
1364
|
+
|
|
1365
|
+
// A page-relative url from one output destination to another, addressed within
|
|
1366
|
+
// the site the page belongs to.
|
|
1367
|
+
//
|
|
1368
|
+
// Both are output-root absolute (`/bg/aparati/index.html`, `/derived/x.webp`),
|
|
1369
|
+
// which is the only shape the engine has. With one site per build that is also
|
|
1370
|
+
// the deployed root and this is a plain path.relative. With several, it is not:
|
|
1371
|
+
// out/bg IS the domain root, so a url computed against out/ carries one extra
|
|
1372
|
+
// `..` for the language segment. The browser floors that rather than failing,
|
|
1373
|
+
// which is why it worked and why nothing said so.
|
|
1374
|
+
//
|
|
1375
|
+
// Three cases, and the middle one is the reason this is not a one-liner:
|
|
1376
|
+
//
|
|
1377
|
+
// target outside every root a shared asset. It has to be reachable from
|
|
1378
|
+
// inside this page's site, so it is addressed
|
|
1379
|
+
// there — this is the case that was wrong.
|
|
1380
|
+
// target in the same root already correct; a plain relative path.
|
|
1381
|
+
// target in a DIFFERENT root a cross-site link. On a per-domain deploy the
|
|
1382
|
+
// other site is another origin and no relative
|
|
1383
|
+
// path reaches it. Left as it was, so the
|
|
1384
|
+
// reference check reports it broken instead of
|
|
1385
|
+
// this silently inventing a path that is not.
|
|
1386
|
+
export function siteRelativeUrl(pageDestination, target, siteRoots = []) {
|
|
1387
|
+
const from = path.dirname(pageDestination || '/')
|
|
1388
|
+
const pageRoot = siteRootFor(pageDestination, siteRoots)
|
|
1389
|
+
if (!pageRoot) return path.relative(from, target)
|
|
1390
|
+
if (siteRootFor(target, siteRoots)) return path.relative(from, target)
|
|
1391
|
+
return path.relative(from, path.join('/', pageRoot, target))
|
|
1392
|
+
}
|
package/10.0-PLAN.md
DELETED
|
@@ -1,182 +0,0 @@
|
|
|
1
|
-
# Mikser 10.0 — Planning (stub)
|
|
2
|
-
|
|
3
|
-
**The shift in one line:** scale moves from "make one mikser hold more" to "compose more miksers." A new coordination layer ships as `party-mikser-io` — separate package, separate ADR series, separate release cadence. `mikser-io` itself stays single-instance-pure.
|
|
4
|
-
|
|
5
|
-
This is a v10 placeholder — direction-setting, not a finished design. The hard pieces (cross-mikser refs, discovery, partition behavior) get worked out as `party-mikser-io` takes shape. This file gets dropped at release; the surviving decisions become ADRs.
|
|
6
|
-
|
|
7
|
-
## The gate: v9 has to pay out first
|
|
8
|
-
|
|
9
|
-
**No v10 work begins until v9 is incorporated into our own daily work and the workflow pays out.** Not "v9 has shipped." Not "v9 has been tested." **Used, daily, for our own real work, until the format-liberation pitch is something we'd defend with our own time.**
|
|
10
|
-
|
|
11
|
-
This is the strongest gate in the roadmap. It exists because the v10 architecture is more interesting to build than the v9 documentation is to write, and that asymmetry kills projects with strong substrates. The discipline is to refuse v10 design work — even thinking about it as an escape — until v9 has earned the next layer through real use.
|
|
12
|
-
|
|
13
|
-
What "pays out" means operationally — concrete signals to watch:
|
|
14
|
-
|
|
15
|
-
| signal | reading |
|
|
16
|
-
|---|---|
|
|
17
|
-
| New work naturally flows into mikser, not into "I should mikser-ize this later" | ✅ working |
|
|
18
|
-
| You stop wanting to switch back to your previous setup | ✅ working |
|
|
19
|
-
| The format-liberation flows (PDF in → CSV out, CSV in → HTML out) are things you actually run, not just demos | ✅ working |
|
|
20
|
-
| The MCP agent integration makes daily work faster, not just possible | ✅ working |
|
|
21
|
-
| Watch-mode + incremental rebuild holds up under your real editing patterns | ✅ working |
|
|
22
|
-
| You stop writing helper scripts that work around mikser instead of through it | ✅ working |
|
|
23
|
-
| Friends/colleagues see what you have running and ask how to use it themselves | ✅ working — the strongest signal |
|
|
24
|
-
|
|
25
|
-
Failure signals — any of these means v9 hasn't paid out yet, fix before v10:
|
|
26
|
-
|
|
27
|
-
| signal | reading |
|
|
28
|
-
|---|---|
|
|
29
|
-
| You manually run `mikser --clear` more than weekly | ❌ incremental rebuild has rough edges |
|
|
30
|
-
| You keep editing files outside the mikser folder because watch-mode is slow / breaks | ❌ watch-mode regression somewhere |
|
|
31
|
-
| Specific plugins break in ways that make you avoid them | ❌ plugin needs work, not v10 work |
|
|
32
|
-
| You stop using MCP integration because grep is faster | ❌ MCP surface isn't carrying its weight |
|
|
33
|
-
| You compose pipelines outside mikser (one-off scripts) because composing in mikser is awkward | ❌ recipe documentation or substrate ergonomics gap |
|
|
34
|
-
| You don't find yourself reaching for it for new tasks | ❌ the substrate isn't actually changing how you work |
|
|
35
|
-
|
|
36
|
-
When the working signals dominate over a sustained period of real use, v10 design starts. Not before.
|
|
37
|
-
|
|
38
|
-
## Why v10 (not 9.x)
|
|
39
|
-
|
|
40
|
-
The 9.0 design deliberately deferred mikser-to-mikser composition to v10. The reasoning:
|
|
41
|
-
|
|
42
|
-
> Cross-instance concerns (discovery, peer identity, routing, cross-mikser refs, watch propagation, liveness, partition behavior, peer auth) belong to a distributed-systems substrate that iterates at "we hit a production partition bug" speed, not at engine-substrate speed. Folding any of it into `mikser-io` would re-fail ADR-0006 test #5 (release cadence) the moment it ships.
|
|
43
|
-
|
|
44
|
-
v10 is the version where that substrate ships, alongside the small `mikser-io` additions that make it possible.
|
|
45
|
-
|
|
46
|
-
Until then: one folder, one mikser, domain-sized catalog. That's the v9 design center, and it earns the v10 work by staying small.
|
|
47
|
-
|
|
48
|
-
## The split
|
|
49
|
-
|
|
50
|
-
```
|
|
51
|
-
mikser-io → single-instance substrate (unchanged from 9.0)
|
|
52
|
-
party-mikser-io → coordination layer between mikser instances
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
The repo-split check is one line: **does this code reference anything outside this mikser's `runtime`? If yes → wrong repo.** That makes ADR-0006 test #1 (substrate?) concrete for distributed code, which is otherwise ambiguous.
|
|
56
|
-
|
|
57
|
-
| concern | lives in | why |
|
|
58
|
-
|---|---|---|
|
|
59
|
-
| Catalog / refs / manifest / journal | `mikser-io` | every mikser needs this |
|
|
60
|
-
| Source plugins, render dispatch, lifecycle | `mikser-io` | same |
|
|
61
|
-
| Discovery (how A finds B) | `party-mikser-io` | mikser-io never needs to know peers exist |
|
|
62
|
-
| Identity (`mikser.name` as a stable handle) | `party-mikser-io` | mikser-io has no notion of "self" beyond `runtime` |
|
|
63
|
-
| Routing (which mikser owns entity X) | `party-mikser-io` | local catalog queries don't ask this |
|
|
64
|
-
| Cross-mikser refs (`$author` in A → entity in B) | `party-mikser-io` | refs.js stays local-graph-pure |
|
|
65
|
-
| Watch propagation across instances | `party-mikser-io` | chokidar covers one folder; cross-mikser is a different transport |
|
|
66
|
-
| Liveness / health / partition behavior | `party-mikser-io` | distributed-systems posture, not file-engine posture |
|
|
67
|
-
| Auth / trust between peers | `party-mikser-io` | engine auth is per-endpoint; peer auth is its own model |
|
|
68
|
-
|
|
69
|
-
## The metaphor
|
|
70
|
-
|
|
71
|
-
A "party" — voluntary attendance, each guest brings what they own, the host knows who's there but nobody's centrally in charge.
|
|
72
|
-
|
|
73
|
-
More accurate than:
|
|
74
|
-
|
|
75
|
-
- **Cluster** — implies homogeneity (every node same code, same role). A photos mikser and an invoices mikser are different roles.
|
|
76
|
-
- **Mesh** — implies n-to-n by default. Real parties talk to small explicit peer sets.
|
|
77
|
-
- **Federation** — implies central authority or shared schema. Each mikser owns its catalog and exposes whatever shape it wants.
|
|
78
|
-
|
|
79
|
-
Reads correctly at the call site:
|
|
80
|
-
|
|
81
|
-
```js
|
|
82
|
-
import { join } from 'party-mikser-io'
|
|
83
|
-
|
|
84
|
-
await join({
|
|
85
|
-
as: 'invoices',
|
|
86
|
-
peers: [
|
|
87
|
-
{ name: 'documents', url: 'http://localhost:8081' },
|
|
88
|
-
{ name: 'photos', url: 'http://localhost:8082' },
|
|
89
|
-
],
|
|
90
|
-
})
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
## What lands in `mikser-io` to support this
|
|
94
|
-
|
|
95
|
-
These are the only `mikser-io` 10.0 changes — small, surgical, and they don't introduce peer-awareness into the engine. They expose information party-mikser-io needs in order to coordinate.
|
|
96
|
-
|
|
97
|
-
### `mikser-io-api` surface additions
|
|
98
|
-
|
|
99
|
-
Three new endpoints. None of them know about peers; they just expose what a peer would need to read.
|
|
100
|
-
|
|
101
|
-
| endpoint | purpose | rough LOC |
|
|
102
|
-
|---|---|---|
|
|
103
|
-
| `GET /api/identity` | mikser name (from config) + version + capabilities (plugin list + their versions) | ~20 |
|
|
104
|
-
| `GET /api/checkpoint` | current catalog snapshot id (for partition-recovery sync) | ~30 |
|
|
105
|
-
| `GET /api/refs/:id?direction=in|out` | inbound or outbound refs for an entity | ~40 |
|
|
106
|
-
|
|
107
|
-
The `subscribe` endpoint also gains a `?since=<checkpoint>` query param for replay-on-reconnect. ~20 LOC.
|
|
108
|
-
|
|
109
|
-
Total: ~110 LOC of additive endpoints. Nothing existing changes. Each one is independently testable.
|
|
110
|
-
|
|
111
|
-
### `mikser.config.js` gets a `name` field
|
|
112
|
-
|
|
113
|
-
```js
|
|
114
|
-
export default {
|
|
115
|
-
name: 'invoices', // stable identifier; used by /api/identity and party.join({as})
|
|
116
|
-
plugins: [...],
|
|
117
|
-
}
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
Engine reads it, exposes via `runtime.name`. Default falls back to the working folder's basename. That's it — `mikser-io` doesn't act on it; party-mikser-io does.
|
|
121
|
-
|
|
122
|
-
## What `party-mikser-io` ships in 10.0
|
|
123
|
-
|
|
124
|
-
Not designing this in detail yet — the design happens as the package gets built. The shape at this stage is:
|
|
125
|
-
|
|
126
|
-
1. **`join(config)`** — handshake with each declared peer via `/api/identity`. Verifies peer is reachable and reports name/capabilities.
|
|
127
|
-
2. **`runtime.options.party.peers`** — map of peer name → metadata. Available to plugins after join completes.
|
|
128
|
-
3. **`mikser-io-source-party`** — source plugin: mounts a peer's `/api` SSE feed as a local collection. Live-reload via the existing subscribe semantics.
|
|
129
|
-
4. **Cross-mikser ref resolution** — `$author@documents` syntax (entity id + peer name). Resolution proxies through the source-party plugin's mirrored cache. **Design call deferred** to first implementation pass.
|
|
130
|
-
5. **Liveness checks** — periodic `/api/checkpoint` poll per peer. Stale peers logged but don't break the build (their entities go cold; resync on reconnect).
|
|
131
|
-
|
|
132
|
-
## Open questions (working proposals at v10 design time)
|
|
133
|
-
|
|
134
|
-
1. **Cross-mikser ref shape — mirror or proxy?**
|
|
135
|
-
- (a) Mirror: source-party plugin pulls all peer entities into local catalog. Refs resolve locally. Cost: memory grows with peer corpus size.
|
|
136
|
-
- (b) Proxy: refs resolve via HTTP at lookup time. Cost: latency on every render that touches a cross-mikser ref.
|
|
137
|
-
- Working proposal: **mirror with TTL** — pull peer entities lazily on first ref, cache with checkpoint-based invalidation. Worst-case memory is bounded by what's actually referenced.
|
|
138
|
-
|
|
139
|
-
2. **Discovery mechanism.**
|
|
140
|
-
- Config-pinned peer URLs (the snippet above) for v10. Anything more (mDNS, registry, DNS-SD) lives in 10.x or 11.0 after we see what real deployments need. Working proposal: **explicit URLs only for v10.**
|
|
141
|
-
|
|
142
|
-
3. **Authentication between peers.**
|
|
143
|
-
- The api plugin already supports per-endpoint tokens. Reuse that: each peer in `join()` config carries a token. Working proposal: **bearer token per peer, configured in `join()`.**
|
|
144
|
-
|
|
145
|
-
4. **What if two peers claim entities with the same id?**
|
|
146
|
-
- The fully-qualified form is `id@peer-name`. Local entities have no `@`. Working proposal: **namespacing by peer name avoids collision entirely** — cross-mikser refs always carry the peer name.
|
|
147
|
-
|
|
148
|
-
5. **What about cycles?**
|
|
149
|
-
- A subscribes to B, B subscribes to A. Working proposal: **explicitly allowed**. Each side mirrors the other; the checkpoint mechanism prevents replay loops.
|
|
150
|
-
|
|
151
|
-
## Out of scope for 10.0
|
|
152
|
-
|
|
153
|
-
- **Write-across-peers** — `party.create(entity, { in: 'invoices' })` from peer A creating an entity in peer B's catalog. Probably 10.x or 11.0. Single-direction read is enough to validate the architecture.
|
|
154
|
-
- **Distributed consensus / conflict resolution** — each peer owns its catalog, period. Nobody writes to anyone else's.
|
|
155
|
-
- **Replication / HA** — that's a different problem. A party isn't a cluster; if a peer goes down, its entities go cold, not get failed-over.
|
|
156
|
-
- **Plugin marketplace / discovery service** — same answer as 9.0. Not building this.
|
|
157
|
-
|
|
158
|
-
## ADR series
|
|
159
|
-
|
|
160
|
-
`party-mikser-io` gets its own decision log under `party-mikser-io/documentation/decisions/`, starting from 0001. Not extending the `mikser-io` ADR series — keeps the substrate test (#1) clean. `mikser-io` ADRs cover the single-instance engine; `party-mikser-io` ADRs cover the coordination plane.
|
|
161
|
-
|
|
162
|
-
Cross-references between the two ADR series are fine; the boundary stays at "does this code reference anything outside this mikser's `runtime`?"
|
|
163
|
-
|
|
164
|
-
## Concretely, what work this is (rough order)
|
|
165
|
-
|
|
166
|
-
1. **`mikser-io` engine** — add `runtime.name` (read from `mikser.config.js`). One file change.
|
|
167
|
-
2. **`mikser-io-api`** — add `/api/identity`, `/api/checkpoint`, `/api/refs/:id`, and `?since=` on subscribe. ~110 LOC.
|
|
168
|
-
3. **`party-mikser-io`** — new repo. `join()`, peer registry, liveness loop. The boring substrate of the coordination layer. ~400 LOC.
|
|
169
|
-
4. **`mikser-io-source-party`** — new repo (or sub-package). Source plugin that mounts a peer's catalog. The mirror cache + checkpoint invalidation logic. ~300 LOC.
|
|
170
|
-
5. **Cross-mikser ref resolution** — extends `mikser-io-source-party`. Parses `id@peer` syntax in `$`-refs and resolves through the mirror. ~150 LOC.
|
|
171
|
-
6. **Recipe documentation** — README composition examples: PDF mikser + CSV mikser + HTML mikser, piped together. The example that proves the v10 pitch works end-to-end.
|
|
172
|
-
7. **`mikser-io-example-party`** — new example repo. Three miksers in one repo (subfolders), each with its own config, demonstrating the full composition.
|
|
173
|
-
|
|
174
|
-
Items 1-2 ship in `mikser-io` 10.0 itself. Items 3-7 are the v10 ecosystem release.
|
|
175
|
-
|
|
176
|
-
## What this earns us, when it lands
|
|
177
|
-
|
|
178
|
-
The 9.0 pitch was "file-based knowledge substrate with AI superpowers." v10 extends that with one word: **composable**.
|
|
179
|
-
|
|
180
|
-
> A file-based knowledge substrate with AI superpowers, composable across instances.
|
|
181
|
-
|
|
182
|
-
The PDF-in-CSV-out-HTML-elsewhere story works as separate miksers connected through a party — not as one process trying to be everything. That's the architecture the substrate was designed for; v10 just makes it real.
|