@ctxr/skill-llm-wiki 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. package/CHANGELOG.md +134 -0
  2. package/LICENSE +21 -0
  3. package/README.md +484 -0
  4. package/SKILL.md +252 -0
  5. package/guide/basics/concepts.md +74 -0
  6. package/guide/basics/index.md +45 -0
  7. package/guide/basics/schema.md +140 -0
  8. package/guide/cli.md +256 -0
  9. package/guide/correctness/index.md +45 -0
  10. package/guide/correctness/invariants.md +89 -0
  11. package/guide/correctness/safety.md +96 -0
  12. package/guide/history/diff.md +110 -0
  13. package/guide/history/hidden-git.md +130 -0
  14. package/guide/history/index.md +52 -0
  15. package/guide/history/remote-sync.md +113 -0
  16. package/guide/index.md +134 -0
  17. package/guide/isolation/coexistence.md +134 -0
  18. package/guide/isolation/index.md +44 -0
  19. package/guide/isolation/scale.md +251 -0
  20. package/guide/layout/in-place-mode.md +97 -0
  21. package/guide/layout/index.md +53 -0
  22. package/guide/layout/layout-contract.md +131 -0
  23. package/guide/layout/layout-modes.md +115 -0
  24. package/guide/operations/index.md +76 -0
  25. package/guide/operations/ingest/build.md +75 -0
  26. package/guide/operations/ingest/extend.md +61 -0
  27. package/guide/operations/ingest/index.md +54 -0
  28. package/guide/operations/ingest/join.md +65 -0
  29. package/guide/operations/maintain/fix.md +66 -0
  30. package/guide/operations/maintain/index.md +47 -0
  31. package/guide/operations/maintain/rebuild.md +86 -0
  32. package/guide/operations/validate.md +48 -0
  33. package/guide/substrate/index.md +47 -0
  34. package/guide/substrate/operators.md +96 -0
  35. package/guide/substrate/tiered-ai.md +363 -0
  36. package/guide/ux/index.md +44 -0
  37. package/guide/ux/preflight.md +150 -0
  38. package/guide/ux/user-intent.md +135 -0
  39. package/package.json +55 -0
  40. package/scripts/cli.mjs +893 -0
  41. package/scripts/commands/remote.mjs +93 -0
  42. package/scripts/commands/review.mjs +253 -0
  43. package/scripts/commands/sync.mjs +84 -0
  44. package/scripts/lib/chunk.mjs +421 -0
  45. package/scripts/lib/cluster-detect.mjs +516 -0
  46. package/scripts/lib/decision-log.mjs +343 -0
  47. package/scripts/lib/draft.mjs +158 -0
  48. package/scripts/lib/embeddings.mjs +366 -0
  49. package/scripts/lib/frontmatter.mjs +497 -0
  50. package/scripts/lib/git-commands.mjs +155 -0
  51. package/scripts/lib/git.mjs +486 -0
  52. package/scripts/lib/gitignore.mjs +62 -0
  53. package/scripts/lib/history.mjs +331 -0
  54. package/scripts/lib/indices.mjs +510 -0
  55. package/scripts/lib/ingest.mjs +258 -0
  56. package/scripts/lib/intent.mjs +713 -0
  57. package/scripts/lib/interactive.mjs +99 -0
  58. package/scripts/lib/migrate.mjs +126 -0
  59. package/scripts/lib/nest-applier.mjs +260 -0
  60. package/scripts/lib/operators.mjs +1365 -0
  61. package/scripts/lib/orchestrator.mjs +718 -0
  62. package/scripts/lib/paths.mjs +197 -0
  63. package/scripts/lib/preflight.mjs +213 -0
  64. package/scripts/lib/provenance.mjs +672 -0
  65. package/scripts/lib/quality-metric.mjs +269 -0
  66. package/scripts/lib/query-fixture.mjs +71 -0
  67. package/scripts/lib/rollback.mjs +95 -0
  68. package/scripts/lib/shape-check.mjs +172 -0
  69. package/scripts/lib/similarity-cache.mjs +126 -0
  70. package/scripts/lib/similarity.mjs +230 -0
  71. package/scripts/lib/snapshot.mjs +54 -0
  72. package/scripts/lib/source-frontmatter.mjs +85 -0
  73. package/scripts/lib/tier2-protocol.mjs +470 -0
  74. package/scripts/lib/tiered.mjs +453 -0
  75. package/scripts/lib/validate.mjs +362 -0
@@ -0,0 +1,96 @@
1
+ ---
2
+ id: operators
3
+ type: primary
4
+ depth_role: leaf
5
+ focus: the four rewrite operators that shape wiki trees toward a token-minimal normal form
6
+ parents:
7
+ - index.md
8
+ covers:
9
+ - "DECOMPOSE: horizontal split when one entry covers disjoint concerns"
10
+ - "NEST: vertical specialisation when an entry's sections are narrower derivations of its focus"
11
+ - "MERGE: two siblings with compatible covers and activation collapse into one"
12
+ - "LIFT: single-child folder collapses up one level"
13
+ - "DESCEND: gravity toward leaves, push leaf-shaped content out of parent indices"
14
+ - "detection criteria, application procedures, and priority order (DESCEND > LIFT > MERGE > NEST > DECOMPOSE)"
15
+ - "contract-gating: hosted-mode operator applications are rejected when they would violate the layout contract"
16
+ tags:
17
+ - operators
18
+ - rebuild
19
+ - normal-form
20
+ activation:
21
+ keyword_matches:
22
+ - operator
23
+ - rewrite
24
+ - decompose
25
+ - nest
26
+ - merge
27
+ - lift
28
+ - descend
29
+ - restructure
30
+ tag_matches:
31
+ - structural-change
32
+ escalation_from:
33
+ - build
34
+ - rebuild
35
+ source:
36
+ origin: file
37
+ path: operators.md
38
+ hash: "sha256:ef8ac35df32870960806bc1f401168e5993ec3759497bffa3e81dec0552a6ead"
39
+ ---
40
+
41
+ # Rewrite operators
42
+
43
+ Reshape trees toward a token-minimal normal form. Applied in fixed priority order: **DESCEND > LIFT > MERGE > NEST > DECOMPOSE**.
44
+
45
+ ## DECOMPOSE (horizontal split)
46
+
47
+ **Rule:** If a single entry covers N ≥ 2 disjoint concerns, split it into N peer entries under a common parent. The parent holds what they share; each peer holds its specifics.
48
+
49
+ **Detection:** `covers[]` clusters into ≥2 disjoint groups by tag/keyword similarity; OR `activation.file_globs` contain patterns with no common prefix/suffix; OR body has ≥2 H2 sections each meaningful standalone; OR `covers[]` exceeds 12 items.
50
+
51
+ **Application:** partition covers into clusters; create sibling entries with narrower focus; hoist shared items to the parent index's `shared_covers[]`; add `aliases[]` entry pointing to the original id so existing references don't break; delete the original file.
52
+
53
+ ## NEST (vertical specialisation + cluster-based grouping)
54
+
55
+ **Rule:** If an entry's internal structure reveals narrower specialisations of its focus, OR if multiple sibling leaves form a coherent cluster that deserves a subcategory, extract them into leaf files under a new child folder; the original entries become children of a new parent index.
56
+
57
+ NEST fires in two modes:
58
+
59
+ - **Nests-into hint (legacy).** A leaf's frontmatter carries an explicit `nests_into[]` list. Detection is syntactic; application splits the leaf into the hinted children.
60
+ - **Cluster-based (corpus-adaptive).** The cluster detector (`scripts/lib/cluster-detect.mjs`) computes an affinity matrix across the siblings of each parent directory using four signals — Tier 0 TF-IDF cosine, Tier 1 embedding cosine on focus/covers/body sample, tag-Jaccard, and activation-keyword-Jaccard. The matrix is fused with default weights (`0.25 / 0.40 / 0.20 / 0.15`) and clustered into connected components under candidate thresholds `[0.30, 0.38, 0.46]`. The threshold whose partition produces the best shape score wins. The detector is corpus-agnostic — it has NO knowledge of the specific wiki being optimised.
61
+
62
+ **Cluster-based application.** Each accepted cluster is named via a Tier 2 `cluster_name` request (slug + purpose) — or receives a slug directly from a `propose_structure` Tier 2 response. Names are NEVER shortcut from shared tags; if the sub-agent cannot name a cluster, that cluster does not nest. The NEST applier (`scripts/lib/nest-applier.mjs`) then:
63
+
64
+ 1. **Atomic slug resolution.** Before touching the filesystem, `resolveNestSlug(slug, proposal)` checks whether the proposed slug collides with (a) any member leaf's id, (b) any non-member sibling leaf's id in the same parent, or (c) an existing sibling subdirectory name. On collision the slug is auto-suffixed deterministically (`<slug>-group`, then `<slug>-group-2`, `-group-3`, …) until it's non-colliding. The rename is audited in `decisions.yaml` as `decision: slug-renamed`. This pre-empts the DUP-ID class of validation failure that would otherwise rollback the entire NEST after apply.
65
+ 2. Creates `<parent>/<slug>/` (using the resolved slug).
66
+ 3. Moves each cluster member into the new directory and rewrites its `parents[]` to `["index.md"]`.
67
+ 4. Writes a minimal `index.md` stub carrying `id` (= resolved slug), `type: index`, `depth_role: subcategory`, a `focus:` line from the cluster purpose, and — when the members share them — `shared_covers[]` (intersection of member covers) and `tags[]` (intersection of member tags). The stub does NOT carry aggregated `activation_defaults`: routing is semantic, and descent decisions are made against the stub's `focus` + `shared_covers`, not against a literal keyword union.
68
+ 5. Rebuilds all indices so the parent directory's `entries[]` now lists the new subcategory instead of the moved leaves.
69
+
70
+ **Quality-metric gating.** Every cluster NEST application is scored against the `routing_cost` metric before and after. Metric = sum over a fixed query distribution (`scripts/lib/query-fixture.mjs`) of bytes read during simulated routing, normalised by total leaf bytes. If the post-apply metric is worse than the pre-apply metric, the application is rolled back and the next-best proposal is tried. This is the "let data pick the cluster" discipline — we never apply a cluster just because the affinity matrix liked it, only when the resulting tree routes queries more cheaply. The metric trajectory is logged to `decisions.yaml`.
71
+
72
+ **Recursive-nest safety.** Directories freshly created by a NEST in the current convergence run are excluded from subsequent cluster detection in the same run. This prevents noise-driven infinite sub-clustering.
73
+
74
+ **Legacy nests-into path.** The nests-into-hint proposals still emit as detect-only suggestions in the convergence audit trail; they are not auto-applied because the per-leaf hint mechanism predates the cluster detector and is kept for hand-authored hints.
75
+
76
+ ## MERGE / LIFT (redundancy collapse)
77
+
78
+ **MERGE — two siblings collapse into one.** Detection: `focus` similarity above threshold, `covers[]` overlap > 70%, compatible activation, compatible `parents[]`. Application: union the covers, pick the more general focus, take the union of activation and parents, write the merged entry with both original ids in `aliases[]`, delete the sources, rewire references via alias resolution.
79
+
80
+ **LIFT — single-child folder collapses up.** Detection: a non-root folder contains exactly one non-index entry. Application: move the child up one level, update its `parents[]` to point at the grandparent, delete the now-empty folder and its `index.md`, preserve the folder's id on the lifted child as an alias.
81
+
82
+ ## DESCEND (gravity toward leaves)
83
+
84
+ **Rule:** Substantive domain knowledge must live at leaves. Parent indices contain only navigation and shared context. Push leaf-shaped content from parent bodies down into child leaves.
85
+
86
+ **Detection:** parent index body (authored zone) exceeds 2 KB budget; OR contains leaf-content signatures (checklist items, code fences, multi-paragraph exposition, data tables).
87
+
88
+ **Application:** create a new leaf (or append to an existing relevant one) to host the extracted content; move the content; leave a short link reference in the parent's orientation if navigation benefits.
89
+
90
+ ## Priority rationale
91
+
92
+ Information-preserving reductions happen first (DESCEND moves content deeper without losing it; LIFT removes empty structure). Collapses happen next (MERGE reduces byte count). Expansions happen last (NEST and DECOMPOSE add structural surface area). This order prevents operators from creating structure that would immediately be collapsed.
93
+
94
+ ## Contract-gating in hosted mode
95
+
96
+ Every operator application is checked against the layout contract **before** being accepted. Rejected moves include: NEST that would exceed a directory's `max_depth`; LIFT that would remove a contract-required directory; MERGE across dynamic subdirs where the contract treats them as separate (e.g. two different days in a `daily/` tree); DECOMPOSE that would place peers into a non-existing contract directory. Rejected moves are suppressed; remaining operators still run until convergence.
@@ -0,0 +1,363 @@
1
+ ---
2
+ id: tiered-ai
3
+ type: primary
4
+ depth_role: leaf
5
+ focus: tiered AI ladder — TF-IDF → local embeddings → Claude — with quality modes
6
+ parents:
7
+ - index.md
8
+ covers:
9
+ - "Tier 0 is TF-IDF over frontmatter (focus + covers + tags) with fixed thresholds"
10
+ - "Tier 1 is local embeddings via @xenova/transformers (MiniLM, REQUIRED dep)"
11
+ - "Tier 2 is a sub-agent, executed via the CLI exit-7 handshake (never inline)"
12
+ - default quality mode is tiered-fast; claude-first and tier0-only are opt-in
13
+ - "similarity-cache at <wiki>/.llmwiki/similarity-cache/ memoises pairwise results"
14
+ - "decision-log at <wiki>/.llmwiki/decisions.yaml records every non-trivial decision"
15
+ - operator-convergence routes every MERGE similarity check through tiered.decide
16
+ - cluster_name Tier 2 requests name NEST subcategories; never shortcut from tags
17
+ - "exit-7 handshake: CLI writes pending batch to .work/tier2/ and exits 7 so the wiki-runner can spawn sub-agents"
18
+ - Tier 2 model + effort defaults are per-task; user overrides propagate to every sub-agent
19
+ tags:
20
+ - ai-strategy
21
+ - operators
22
+ - similarity
23
+ activation:
24
+ keyword_matches:
25
+ - similarity
26
+ - cluster
27
+ - merge
28
+ - decompose
29
+ - tokens
30
+ - speed
31
+ - cost
32
+ - embeddings
33
+ - tfidf
34
+ - quality mode
35
+ - claude
36
+ - tier
37
+ tag_matches:
38
+ - ai-strategy
39
+ - operators
40
+ escalation_from:
41
+ - build
42
+ - rebuild
43
+ - operator-convergence
44
+ - merge
45
+ source:
46
+ origin: file
47
+ path: tiered-ai.md
48
+ hash: "sha256:e7e8f12bc0486b7462350ffb0ff7e8bed34813c779139b49b33608b0346d9bcb"
49
+ ---
50
+
51
+ # Tiered AI ladder
52
+
53
+ `skill-llm-wiki` routes every similarity decision through a
54
+ three-tier ladder. The **design principle** is crucial:
55
+
56
+ > Claude is used for deep-understanding decisions (structural
57
+ > judgments on semantically ambiguous entries, HUMAN-class Fix
58
+ > decisions, prose-heavy draft-frontmatter, user-intent resolution),
59
+ > **never for routing, never for lightweight pairwise similarity
60
+ > when a local tier is decisive.**
61
+
62
+ Every pairwise check runs Tier 0 first. If Tier 0 is decisive, the
63
+ ladder halts. If it's mid-band, the decision escalates to Tier 1.
64
+ If Tier 1 is also mid-band, the decision escalates to Tier 2.
65
+ Tier 2 is a real Claude sub-agent spawned by the wiki-runner via the
66
+ **exit-7 handshake** described below. Tier 1 is a REQUIRED dependency
67
+ — the optional-install flow was removed in v0.4.0 when the overhaul
68
+ discovered Tier 0 alone was too weak to drive the ladder on terse
69
+ technical frontmatter.
70
+
71
+ ## Tier 0 — TF-IDF + cosine (scripts/lib/similarity.mjs)
72
+
73
+ Pure, deterministic, no dependencies. Runs on frontmatter fields
74
+ only: `focus` (weighted 2×), `covers[]`, `tags[]`, `domains[]`.
75
+ Never touches entry bodies.
76
+
77
+ Thresholds:
78
+
79
+ - `similarity >= 0.85` → **decisive SAME**
80
+ - `similarity <= 0.30` → **decisive DIFFERENT**
81
+ - otherwise → **escalate to Tier 1**
82
+
83
+ Tier 0 is *intended* to resolve the bulk of decisions on
84
+ well-structured corpora — pairs of near-duplicate entries
85
+ should collapse as SAME, obviously unrelated pairs as DIFFERENT
86
+ — leaving only genuinely ambiguous pairs to escalate. The actual
87
+ Tier 0 hit rate on a given wiki depends on how informative the
88
+ frontmatter is; run with `--quality-mode tier0-only` and inspect
89
+ `decisions.yaml` to measure the tier distribution for your corpus.
90
+
91
+ ## Tier 1 — local embeddings (scripts/lib/embeddings.mjs)
92
+
93
+ Backed by `@xenova/transformers` running MiniLM-L6-v2 locally. 384
94
+ dimensions. Cached at `<wiki>/.llmwiki/embedding-cache/<ns>/<sha>.f32`
95
+ (the namespace differs between real-model and mock runs so a
96
+ mock-mode test never pollutes a real-model cache).
97
+
98
+ **Required dependency.** `@xenova/transformers` is listed in the
99
+ skill's `dependencies` (not devDependencies, not optional). A
100
+ `node_modules/` lacking it means the skill is broken — re-run
101
+ `npm install` in the skill directory. There is no install prompt,
102
+ no persistent decline marker, no optional-dependency fallback.
103
+
104
+ The model weights (~23 MB) are downloaded on first use by
105
+ `@xenova/transformers` into its HuggingFace cache directory.
106
+ Preflight warns when `TRANSFORMERS_CACHE` is set but the model
107
+ hasn't been materialised yet, so the operator is aware a first
108
+ call will pay the one-time download latency.
109
+
110
+ Thresholds:
111
+
112
+ - `similarity >= 0.80` → **decisive SAME**
113
+ - `similarity <= 0.45` → **decisive DIFFERENT**
114
+ - otherwise → **escalate to Tier 2**
115
+
116
+ **Mock mode:** set `LLM_WIKI_MOCK_TIER1=1` and the skill substitutes
117
+ a deterministic hash-based vector for the real model. **Tests only.**
118
+ CI uses this so the test suite stays hermetic; production builds
119
+ must never set it, because the mock collapses pairwise distances
120
+ to a 384-dim-hash function and is not a real sentence encoder.
121
+
122
+ ## Tier 2 — sub-agent via exit-7 handshake (scripts/lib/tier2-protocol.mjs)
123
+
124
+ Tier 2 is reserved for decisions that TF-IDF and local embeddings
125
+ both declined to resolve, plus every cluster-naming step emitted
126
+ by the cluster detector (`cluster_name` requests are NEVER
127
+ shortcut from shared tags — a cluster the sub-agent can't name
128
+ isn't a cluster). Because every Tier 2 call is a Claude call, it
129
+ carries a token cost, a latency cost, and — most importantly — a
130
+ **context-window cost** if it runs in the wrong place. The rule
131
+ is simple:
132
+
133
+ > **Every Tier 2 call runs in a dedicated sub-agent, spawned by
134
+ > the wiki-runner via the exit-7 handshake.** The CLI never spawns
135
+ > sub-agents directly — it can't, it's a Node subprocess with no
136
+ > access to Claude Code's `Agent` tool. Instead it writes pending
137
+ > requests to `<wiki>/.work/tier2/pending-<batch>.json` and exits
138
+ > with code **7** (`NEEDS_TIER2`). Exit 7 is not a failure; it is
139
+ > a suspend-and-resume signal.
140
+
141
+ ### The exit-7 handshake, step by step
142
+
143
+ 1. The operator-convergence phase accumulates Tier 2 requests
144
+ (mid-band MERGE checks, cluster-naming requests from NEST
145
+ proposals, rebuild-plan review questions, etc.) on an
146
+ in-memory queue via `tiered.enqueuePending`.
147
+ 2. When the phase finishes, the orchestrator drains the queue
148
+ via `takePendingRequests`, writes the batch to
149
+ `<wiki>/.work/tier2/pending-<batch-id>.json`, and throws
150
+ `NeedsTier2Error`.
151
+ 3. The CLI catches it, prints a summary to stderr, and exits 7.
152
+ The working tree is NOT rolled back; the partial-converge
153
+ commits in the private git stay put.
154
+ 4. The wiki-runner (a Claude Code sub-agent with `Agent` tool
155
+ access) sees exit 7, reads every pending file under
156
+ `<wiki>/.work/tier2/`, and spawns one `Agent` sub-agent per
157
+ request. The sub-agent receives only the request's `prompt`,
158
+ `inputs`, `response_schema`, `model_hint`, and `effort_hint` —
159
+ never the whole wiki.
160
+ 5. The wiki-runner collects the structured JSON responses and
161
+ writes them to `<wiki>/.work/tier2/responses-<batch-id>.json`
162
+ next to the pending file.
163
+ 6. The wiki-runner re-invokes the CLI with the same positional
164
+ args. The orchestrator reads every `responses-*.json` at
165
+ startup, seeds the tiered decision cache, and resumes
166
+ convergence from the last committed iteration.
167
+ 7. If the resumed run emits a new pending batch (sub-clusters
168
+ discovered at the next depth), steps 2–6 repeat. Termination
169
+ is guaranteed by the `nestedParents` exclusion set — a dir
170
+ that was the target of a NEST in the current run is never
171
+ re-clustered.
172
+
173
+ ### Tier 2 request kinds
174
+
175
+ The protocol defines a fixed set of request kinds. Each kind has
176
+ a response schema the sub-agent must match. See
177
+ `scripts/lib/tier2-protocol.mjs::TIER2_DEFAULTS` for the source
178
+ of truth; the table below is the human summary.
179
+
180
+ | Kind | Purpose | Model hint | Effort |
181
+ |-----------------------|----------------------------------------|------------|---------|
182
+ | `merge_decision` | Are these two entries SAME/DIFFERENT? | sonnet | low |
183
+ | `nest_decision` | Should this set nest or stay flat? | sonnet | medium |
184
+ | `cluster_name` | Name a NEST cluster (slug + purpose) | sonnet | low |
185
+ | `draft_frontmatter` | Draft focus/covers for a leaf | sonnet | medium |
186
+ | `rebuild_plan_review` | Review a rebuild plan | opus | high |
187
+ | `human_fix_item` | Decide on a HUMAN-class Fix | sonnet | low |
188
+
189
+ Every request carries a deterministic `request_id` (sha256 of
190
+ the kind + canonical-JSON of the inputs, truncated to 16 hex
191
+ chars). Asking the same question twice within a run produces
192
+ the same id and the wiki-runner only needs to answer it once.
193
+
194
+ ### Test hermeticity
195
+
196
+ Set `LLM_WIKI_TIER2_FIXTURE=<path>` to a JSON file containing
197
+ `{ "<request_id>": { response body } }` (or an array of
198
+ `{request_id, response}` pairs) and the CLI will resolve Tier 2
199
+ requests against the fixture INSTEAD OF exiting 7. Used
200
+ exclusively by tests; must never be set in production.
201
+
202
+ ### Why dedicated sub-agents per decision
203
+
204
+ - **Context isolation.** A 10k-entry wiki with 200 mid-band pairs
205
+ would drown the wiki-runner's context if every Claude call
206
+ landed inline. Per-decision sub-agents let the wiki-runner hold
207
+ only the final decision, not the prompt+response.
208
+ - **Parallelism where safe.** Non-conflicting Tier 2 decisions
209
+ (different entry pairs, different draft-frontmatter jobs, etc.)
210
+ can fan out to parallel sub-agents. The wiki-runner collects
211
+ results and writes them into the decision log in deterministic
212
+ order.
213
+ - **Model choice per task.** Different Tier 2 workloads want
214
+ different models. A draft-frontmatter pass on a short structured
215
+ file needs the cheapest capable model; a rebuild plan review
216
+ needs a strong reasoning model. Sub-agent spawning lets each
217
+ call pick the right tool.
218
+ - **Cost attribution.** Each sub-agent's token usage is attributable
219
+ to a specific decision, visible in the session's agent log, and
220
+ traceable via `decisions.yaml`.
221
+
222
+ ### Per-call sub-agent prompt shape
223
+
224
+ The wiki-runner spawns a Tier 2 sub-agent with a self-contained
225
+ prompt that includes:
226
+
227
+ 1. **The question** — "are these two frontmatters the same
228
+ concept? (MERGE candidate)", "draft a concrete `focus` string
229
+ plus 3–5 `covers[]` bullets for this entry", "review this rebuild
230
+ plan and flag any move that would break the narrowing chain",
231
+ etc.
232
+ 2. **Only the inputs the question needs** — two frontmatter blobs,
233
+ one source file, one plan excerpt. Never the whole wiki, never
234
+ unrelated context.
235
+ 3. **The decision schema** — a strict JSON shape the sub-agent must
236
+ return (`{decision, reason}` for MERGE, `{focus, covers, tags}`
237
+ for draft-frontmatter, etc.) so the wiki-runner can parse the
238
+ response without further chat.
239
+ 4. **Any model / effort override** — if the user specified one, it
240
+ propagates through to every Tier 2 sub-agent the operation
241
+ spawns. No sub-agent silently upgrades or downgrades the model.
242
+
243
+ ### Default model + effort matrix
244
+
245
+ | Tier 2 task | Default model | Default effort | Notes |
246
+ |---|---|---|---|
247
+ | draft-frontmatter (single entry) | Cheapest capable model for short-form writing | minimal | One sub-agent per entry that needs it; parallel safe. |
248
+ | operator-convergence (single pair) | Cost-effective model with strong short-form judgment | minimal | One sub-agent per mid-band pair; parallel safe. |
249
+ | rebuild plan review (whole plan) | Strong reasoning model | medium | Single sub-agent; reads the plan + current tree summary. |
250
+ | HUMAN-class Fix item | Strong reasoning model | medium | One sub-agent per item; each needs to justify its proposal to the user. |
251
+ | Join id-collision resolution | Strong reasoning model | minimal | One sub-agent per collision cluster. |
252
+
253
+ Unless the user specifies otherwise in the main session ("use
254
+ sonnet", "minimal effort everywhere", "use opus 1M"), pick from
255
+ this matrix. User overrides pass through verbatim to every Tier 2
256
+ spawn under the current operation.
257
+
258
+ ### Caching still short-circuits
259
+
260
+ The similarity cache (see below) is consulted **before** the
261
+ sub-agent spawn. A cache hit never triggers a Tier 2 call at all —
262
+ the decision is reused from `.llmwiki/similarity-cache/`. Cache
263
+ misses are the only pairs that reach the Tier 2 sub-agent. This
264
+ means a 10k-entry wiki that has been rebuilt once amortises almost
265
+ all of its Tier 2 cost on subsequent rebuilds.
266
+
267
+ ### What the wiki-runner keeps after a Tier 2 call
268
+
269
+ - The final decision (`same` / `different` / `undecidable`).
270
+ - The tier used, confidence band, similarity score, and one-line
271
+ reason — all written into `decisions.yaml`.
272
+ - **Not** the prompt, not the response body, not the sub-agent's
273
+ chain of thought. Those live only in the sub-agent's transcript
274
+ and are dropped when the sub-agent returns.
275
+
276
+ ## Quality modes
277
+
278
+ Choose via `--quality-mode` or the `LLM_WIKI_QUALITY_MODE` env var.
279
+
280
+ | Mode | Behaviour | Use when |
281
+ |------|-----------|----------|
282
+ | **`tiered-fast`** (default) | Full ladder. Tier 0 → Tier 1 → Tier 2 on mid-band escalations. | General-purpose builds. |
283
+ | `claude-first` | Tier 0 is still consulted for decisive cases. Mid-band Tier 0 skips Tier 1 and goes directly to Tier 2. | When the user values Claude's judgment over speed/cost. |
284
+ | `tier0-only` | Tier 0 only. Mid-band decisions become "undecidable" and the caller must resolve manually. | Air-gapped, hermetic CI, and smoke tests that must not reach out to Claude. |
285
+
286
+ ## Similarity cache
287
+
288
+ Every decision is cached at
289
+ `<wiki>/.llmwiki/similarity-cache/<hashA-hashB>.json`, keyed by the
290
+ sorted pair of content hashes. Subsequent lookups short-circuit the
291
+ entire ladder — the convergence loop can iterate over a pair many
292
+ times without re-paying the TF-IDF + embedding cost.
293
+
294
+ The cache is symmetric: `cacheKey(a, b) === cacheKey(b, a)`.
295
+
296
+ ## Decision log
297
+
298
+ `<wiki>/.llmwiki/decisions.yaml` records every non-trivial decision
299
+ with:
300
+
301
+ - `op_id` — the operation that triggered the check
302
+ - `operator` — MERGE / DECOMPOSE / NEST / DESCEND / LIFT / METRIC_TRAJECTORY
303
+ - `sources[]` — the entry ids involved
304
+ - `tier_used` — 0, 1, or 2
305
+ - `similarity` — the final similarity value (or metric cost for trajectories)
306
+ - `confidence_band` — one of:
307
+ - pairwise ladder: `decisive-same` / `decisive-different` / `mid-band`
308
+ - NEST outcomes: `tier2-proposed` / `math-gated` / `tier2-and-math`
309
+ - `decision` — one of:
310
+ - pairwise ladder: `same` / `different` / `undecidable`
311
+ - NEST outcomes: `applied` / `rejected-by-metric` / `rejected-by-gate` / `rejected-stale` / `slug-renamed` / `pending-tier2`
312
+ - metric trajectory: `measured`
313
+ - `reason` — free-form, populated when the decision carries
314
+ explanatory context
315
+
316
+ The `slug-renamed` entry deserves a note: it is audit-trail only,
317
+ not a failure. It is written when `resolveNestSlug` pre-empts a
318
+ DUP-ID collision by suffixing a proposed slug with `-group` (or
319
+ `-group-N`). The rename is only logged if the subsequent NEST
320
+ actually commits — see `guide/substrate/operators.md` for the
321
+ contract. A reader scanning for `decision: slug-renamed` is looking
322
+ at a landed NEST whose directory name does not exactly match the
323
+ slug the Tier 2 response proposed.
324
+
325
+ Claude-at-session-time reads this log when a user asks "why was
326
+ this merged?" — the audit trail answers the question from recorded
327
+ history rather than re-running the computation.
328
+
329
+ ## Operators that use the ladder
330
+
331
+ - **LIFT** — doesn't use the ladder (structural detection: one leaf
332
+ in a folder)
333
+ - **MERGE** — uses the ladder to decide whether sibling pairs are
334
+ the same
335
+ - **DESCEND** — doesn't use the ladder (structural detection:
336
+ authored zone byte budget + leaf-content signatures)
337
+ - **NEST** — uses the ladder via the cluster detector and a Tier 2
338
+ `propose_structure` / `cluster_name` / `nest_decision` round-trip.
339
+ Applied with quality-metric gating; see
340
+ `guide/substrate/operators.md`.
341
+ - **DECOMPOSE** — detect-only (fires suggestions for the shape-check
342
+ log; application is deferred to a human-supervised pass).
343
+
344
+ The convergence loop applies proposals in the order DESCEND > LIFT >
345
+ MERGE > NEST > DECOMPOSE so reducing moves always precede expanding
346
+ moves (methodology §3.5 tie-break).
347
+
348
+ ## What this does NOT do
349
+
350
+ - Invoke Claude during routing. The router walks frontmatter
351
+ deterministically and never consults similarity scores.
352
+ - Cache across wikis. Each wiki owns its own `similarity-cache/`
353
+ and `embedding-cache/`.
354
+ - Share cache entries across mock / real model boundaries. The
355
+ embedding cache is namespaced by mode: mock-mode vectors live at
356
+ `<wiki>/.llmwiki/embedding-cache/mock/` and real-model vectors at
357
+ `<wiki>/.llmwiki/embedding-cache/model-minilm/`. Switching modes
358
+ is equivalent to a fresh cache — a `LLM_WIKI_MOCK_TIER1=1` run
359
+ cannot pollute a subsequent real-model run and vice versa.
360
+ - Fall back to Tier 0 when a Tier 1 real-model call errors. An
361
+ error in the embedder is a hard fail for the current decision —
362
+ the caller re-runs or the user fixes the environment. We don't
363
+ silently lower quality under load.
@@ -0,0 +1,44 @@
1
+ ---
2
+ id: ux
3
+ type: index
4
+ depth_role: subcategory
5
+ depth: 1
6
+ focus: User-facing intent resolution and preflight failure messaging.
7
+ parents:
8
+ - "../index.md"
9
+ shared_covers: []
10
+ entries:
11
+ - id: preflight
12
+ file: preflight.md
13
+ type: primary
14
+ focus: "user-facing messages for preflight failures (node / git / wiki-fsck)"
15
+ tags:
16
+ - preflight
17
+ - user-messages
18
+ - id: user-intent
19
+ file: user-intent.md
20
+ type: primary
21
+ focus: "ask, don't guess — how to resolve ambiguous user requests before running the skill"
22
+ tags:
23
+ - ux
24
+ - intent
25
+ - prompting
26
+ children: []
27
+ ---
28
+ <!-- BEGIN AUTO-GENERATED NAVIGATION -->
29
+
30
+ # Ux
31
+
32
+ **Focus:** User-facing intent resolution and preflight failure messaging.
33
+
34
+ ## Children
35
+
36
+ | File | Type | Focus |
37
+ |------|------|-------|
38
+ | [preflight.md](preflight.md) | 📄 primary | user-facing messages for preflight failures (node / git / wiki-fsck) |
39
+ | [user-intent.md](user-intent.md) | 📄 primary | ask, don't guess — how to resolve ambiguous user requests before running the skill |
40
+
41
+ <!-- END AUTO-GENERATED NAVIGATION -->
42
+
43
+ <!-- BEGIN AUTHORED ORIENTATION -->
44
+ <!-- END AUTHORED ORIENTATION -->
@@ -0,0 +1,150 @@
1
+ ---
2
+ id: preflight
3
+ type: primary
4
+ depth_role: leaf
5
+ focus: "user-facing messages for preflight failures (node / git / wiki-fsck)"
6
+ parents:
7
+ - index.md
8
+ covers:
9
+ - "Case A message: Node.js is not installed, with install options per platform"
10
+ - "Case B message: Node.js version is too old, with upgrade options per platform"
11
+ - "Case C message: git missing or older than 2.25 (exit 5)"
12
+ - "Case D message: existing wiki's private git is corrupt (exit 6)"
13
+ - "Case E message: required runtime dependencies missing (exit 8)"
14
+ - post-install verification command
15
+ - PATH-staleness hint for existing shell sessions
16
+ tags:
17
+ - preflight
18
+ - user-messages
19
+ activation:
20
+ tag_matches:
21
+ - preflight-failure
22
+ keyword_matches:
23
+ - node missing
24
+ - node too old
25
+ - install node
26
+ - upgrade node
27
+ source:
28
+ origin: file
29
+ path: preflight.md
30
+ hash: "sha256:ddf2d24577bef0beaa1b15b1e9e39a073fc04a6016fbe000faec3e99ac1a2e9a"
31
+ ---
32
+
33
+ # Preflight — user-facing messages
34
+
35
+ Relay one of the messages below **verbatim** to the user when the Node.js preflight fails. Do not paraphrase. Do not try to install or upgrade Node yourself. Do not propose workarounds. After relaying, stop the operation and wait for the user to take the action.
36
+
37
+ ## Case A — Node.js is not installed
38
+
39
+ > **Cannot proceed: Node.js is not installed.**
40
+ >
41
+ > The `skill-llm-wiki` skill requires Node.js ≥ 18.0.0 to run its deterministic CLI (`scripts/cli.mjs`). This machine does not have Node.js installed, so no operation can be performed until you install it. I will not install Node.js for you — please do it yourself so you stay in control of your environment.
42
+ >
43
+ > Installation options (pick one for your platform):
44
+ >
45
+ > - **macOS (Homebrew):** `brew install node`
46
+ > - **macOS / Linux (nvm, recommended for dev machines):** `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/master/install.sh | bash` then `nvm install 20 && nvm use 20`
47
+ > - **Linux (Debian/Ubuntu):** `curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt-get install -y nodejs`
48
+ > - **Linux (RHEL/Fedora):** `curl -fsSL https://rpm.nodesource.com/setup_20.x | sudo bash - && sudo dnf install -y nodejs`
49
+ > - **Windows (winget):** `winget install OpenJS.NodeJS`
50
+ > - **Windows (Chocolatey):** `choco install nodejs-lts`
51
+ > - **Any platform (official installer):** download from <https://nodejs.org/en/download/>
52
+ >
53
+ > After installing, verify in a fresh terminal:
54
+ >
55
+ > ```bash
56
+ > node --version # should print v18.0.0 or newer
57
+ > ```
58
+ >
59
+ > If `node --version` works in a new terminal but not in this session, your shell's `PATH` may be stale — open a fresh terminal or source your shell profile (`source ~/.zshrc` / `source ~/.bashrc`), then ask me to retry the operation.
60
+
61
+ ## Case B — Node.js is installed but too old
62
+
63
+ Substitute `${VERSION}` with the exact version string you received from `node --version` (e.g. `v16.17.0`).
64
+
65
+ > **Cannot proceed: Node.js ${VERSION} is too old.**
66
+ >
67
+ > The `skill-llm-wiki` skill requires Node.js ≥ 18.0.0. Your installed version is `${VERSION}`, which is below the minimum. Please upgrade Node.js before retrying the operation. I will not upgrade it for you.
68
+ >
69
+ > Upgrade options:
70
+ >
71
+ > - **macOS (Homebrew):** `brew upgrade node`
72
+ > - **macOS / Linux (nvm):** `nvm install 20 && nvm use 20`
73
+ > - **Linux (NodeSource, Debian/Ubuntu):** `curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt-get install -y nodejs`
74
+ > - **Linux (NodeSource, RHEL/Fedora):** `curl -fsSL https://rpm.nodesource.com/setup_20.x | sudo bash - && sudo dnf install -y nodejs`
75
+ > - **Windows (winget):** `winget upgrade OpenJS.NodeJS`
76
+ > - **Windows (Chocolatey):** `choco upgrade nodejs-lts`
77
+ > - **Any platform (official installer):** download from <https://nodejs.org/en/download/>
78
+ >
79
+ > After upgrading, verify in a fresh terminal:
80
+ >
81
+ > ```bash
82
+ > node --version # should print v18.0.0 or newer
83
+ > ```
84
+ >
85
+ > Then ask me to retry the operation.
86
+
87
+ ## Case C — git binary missing or too old (exit code 5)
88
+
89
+ Emitted by `preflightGit` in `scripts/lib/preflight.mjs`. The CLI uses the private-git backbone (`<wiki>/.llmwiki/git/`) for every operation; without a modern enough git binary, nothing works. The skill requires git ≥ **2.25**.
90
+
91
+ > **Cannot proceed: `git` is missing or too old.**
92
+ >
93
+ > The `skill-llm-wiki` skill requires `git` ≥ 2.25 on `PATH` to run its private-git substrate. This machine either does not have git installed or has a version too old for the features the skill depends on (`git -c core.hooksPath=/dev/null`, `git rev-parse --verify`, isolated-config env vars). Please install or upgrade git before retrying.
94
+ >
95
+ > Installation / upgrade options:
96
+ >
97
+ > - **macOS (Homebrew):** `brew install git` / `brew upgrade git`
98
+ > - **Linux (Debian/Ubuntu):** `sudo apt-get install git`
99
+ > - **Linux (RHEL/Fedora):** `sudo dnf install git`
100
+ > - **Windows:** download from <https://git-scm.com/download/win>
101
+ >
102
+ > After installing, verify in a fresh terminal:
103
+ >
104
+ > ```bash
105
+ > git --version # should print git version 2.25 or newer
106
+ > ```
107
+ >
108
+ > Then ask me to retry the operation.
109
+
110
+ ## Case D — existing wiki's private git is corrupt (exit code 6)
111
+
112
+ Emitted by `preflightWiki` in `scripts/lib/preflight.mjs` when a target wiki has a `.llmwiki/git/` directory but `git fsck --no-dangling --no-reflogs` fails. This indicates the private repo has been damaged — possibly by a parallel process writing into it, a filesystem crash mid-commit, or manual edits to `.llmwiki/git/`.
113
+
114
+ > **Cannot proceed: the wiki's private git repository is corrupt.**
115
+ >
116
+ > `git fsck` failed inside `${WIKI}/.llmwiki/git/`. The skill will not run any operation against a corrupt repo because the Phase 1 safety contract (losslessness, rollback) depends on `GIT-01` holding. Options:
117
+ >
118
+ > 1. **Inspect the damage yourself.** Run `skill-llm-wiki reflog ${WIKI}` and `skill-llm-wiki log ${WIKI}` to see what's still reachable, then roll back to the last known-good tag with `skill-llm-wiki rollback ${WIKI} --to <op-id>`.
119
+ > 2. **Rebuild from source.** If the original source tree is still available, delete the wiki and re-run `skill-llm-wiki build <source>`. The private repo will be reinitialised from scratch.
120
+ > 3. **Ask me for help.** Paste the `git fsck` output you received and I can help diagnose whether the damage is recoverable.
121
+ >
122
+ > I will not attempt automatic repair — a broken repo is the kind of thing that should be an explicit decision, not a silent fix.
123
+
124
+ ## Case E — skill-llm-wiki dependencies are missing (exit code 8)
125
+
126
+ Emitted by `preflightDependencies` in `scripts/lib/preflight.mjs`. The CLI checks both runtime dependencies on every invocation (excluding `--version` / `--help`) and refuses to proceed if either cannot be resolved from the skill's `node_modules/`.
127
+
128
+ > **Cannot proceed: one or more `skill-llm-wiki` runtime dependencies could not be found.**
129
+ >
130
+ > The skill needs both of the following packages installed in its own `node_modules/`:
131
+ >
132
+ > - `gray-matter` — required for parsing authored frontmatter in source files during ingest.
133
+ > - `@xenova/transformers` — required for the local Tier 1 embeddings model used during operator-convergence.
134
+ >
135
+ > To install them, run this in the skill directory:
136
+ >
137
+ > ```bash
138
+ > cd /path/to/skill-llm-wiki
139
+ > npm install
140
+ > ```
141
+ >
142
+ > If the CLI was started in an interactive terminal, it will prompt `Install now? [Y/n]` and run `npm install --silent` for you on confirmation. If it was started non-interactively (no TTY, or `LLM_WIKI_NO_PROMPT=1`), it will attempt the silent install itself before re-checking.
143
+ >
144
+ > If `npm install` fails, the underlying error is shown above. Common causes:
145
+ >
146
+ > - **No network access.** `@xenova/transformers` is large (~25 MB) and the local model file is downloaded on first embed call (another ~23 MB). Both `npm install` and the first build need network unless the cache is pre-warmed.
147
+ > - **Corrupted `node_modules/`.** Delete `node_modules/` and `package-lock.json` and re-run `npm install`.
148
+ > - **Read-only filesystem.** The skill cannot install into a read-only deployment; the dependencies must be vendored in by whoever produced the deployment.
149
+ >
150
+ > The CLI exits with code **8** (`DEPS_MISSING`) when the install attempt is declined or fails.