liteagents 3.6.0 → 3.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,6 @@
1
1
  ---
2
2
  description: Reorg a docs corpus, split an oversized doc, search it, keep pages current, index them
3
3
  ---
4
-
5
4
  # docs-builder
6
5
 
7
6
  Keep project docs **current, complete and findable**, and split a file when it outgrows
@@ -47,8 +46,13 @@ Every command below is `node $DB …`; everything the script writes (`docs/.docs
47
46
  JSON state, `docs/index.md`, the ledger, the log, the config pointer) lands under the target
48
47
  repo. `REPO=` is optional and only needed when not running from the repo root.
49
48
 
50
- **With an argument** (`reorg`, `cleanup <file>`, or `search <query words...>`) — run that mode
51
- directly, no question asked.
49
+ **With an argument** (`reorg [dir]`, `cleanup <file>`, or `search <query words...>`) — run that
50
+ mode directly, no question asked. `reorg <dir>` re-checks every doc already inside `<dir>` —
51
+ useful to scope a check to one directory outside the default scan. A **bare** `reorg`/`discover`
52
+ (no argument) already re-checks `docs/product/`, `docs/wiki/` and `docs/logs/` on every run —
53
+ only `docs/archive/` stays frozen and out of scope. Scan scope with no argument is otherwise
54
+ narrow by design: root-level `.md` files (top level only, not recursive) plus everything under
55
+ `docs/` (recursive) — nothing else in the repo is ever listed or moved.
52
56
 
53
57
  **Bare `/docs-builder`, no argument — ALWAYS ask, never auto-detect.** Run `due` first and
54
58
  put its one-line verdict in the question text so the choice is informed. Then use
@@ -56,9 +60,9 @@ put its one-line verdict in the question text so the choice is informed. Then us
56
60
 
57
61
  > **Question: What should docs-builder do?**
58
62
  >
59
- > - **First run** — sort every `.md` in `docs/` into product / archive, then split anything
60
- > too big into pages and index them. Use when docs are a pile of loose files, or
61
- > docs-builder has never run here.
63
+ > - **First run** — sort root-level `.md` files and everything under `docs/` into
64
+ > product/wiki/logs/archive, then split anything too big into pages and index them. Use
65
+ > when docs are a pile of loose files, or docs-builder has never run here.
62
66
  > - **Docs drift** — docs moved on since the last run: report what changed, rebuild the
63
67
  > index, re-run lint. Nothing is restructured and nothing is split.
64
68
 
@@ -85,8 +89,9 @@ read-only — no model cost, no interview, nothing moves.
85
89
  bucket). `bucket` itself starts **empty** on every row.
86
90
  2. **The classification interview.** Feed the model the WHOLE plan table (file, h1, snip,
87
91
  lines, suggested+reason) in one call and have it fill `bucket` for every row —
88
- `product`/`logs`/`archive` — with a one-line reason: honour that a SHOUTED self-declared
89
- status is near-conclusive for `archive` and that `suggested` is a prior, not an authority.
92
+ `product`/`wiki`/`logs`/`archive` — with a one-line reason: honour that a SHOUTED
93
+ self-declared status is near-conclusive for `archive` and that `suggested` is a prior, not
94
+ an authority.
90
95
  The model writes its answers straight into `reorg-plan.json`. Then show the user the full
91
96
  resulting table via `AskUserQuestion` (approve all / correct specific rows / abort) — a
92
97
  correction changes the plan file before anything moves.
@@ -98,12 +103,10 @@ read-only — no model cost, no interview, nothing moves.
98
103
  to split** (any, all, none). Only then run `cleanup <file>` (Mode 1) on each chosen file —
99
104
  `cleanup` itself prints the estimated split cost for that one file, then a mechanical
100
105
  shape report, then stops for its own interview (Mode 1, step 1b) before anything else runs.
101
- Before that first commit, add `docs/.docs-builder/` to `.gitignore` if it is not already
102
- ignored: it is machine state, regenerated every run, and the ledger stamp is per-clone by
103
- design — it must never ride into history on a later `git add -A`.
104
- Once the moves are committed, run `node $DB ledger` — nothing in steps 1-3 stamps the
105
- ledger, and without the stamp `due` stays NOT due, the picker's verdict stays uninformed,
106
- and `/remember`'s docs nudge never fires.
106
+ Add `docs/.docs-builder/` to `.gitignore` if it is not already ignored: it is machine state,
107
+ regenerated every run, and the ledger stamp is per-clone by design — it must never ride into
108
+ history on a later `git add -A`. Then follow "Finishing a run" (below `apply-reorg`, Mode 0
109
+ step 3) to commit and stamp the ledger.
107
110
 
108
111
  The two stops are deliberate and different. Step 2 guards *correctness* — the interview and
109
112
  the user's approval, before a single file moves. Step 3's follow-up guards *cost* — splitting
@@ -114,7 +117,7 @@ when they pick "First run". Never split N files in one shot on an unseen list.
114
117
  first, if a ledger stamp exists, then it runs `discover`. If any row's `bucket` is still
115
118
  empty (true on a genuine first run, or when new files appeared since the last classification),
116
119
  `reorg` **stops right there** and prints what to do next — it never silently proceeds past an
117
- unclassified plan. Commit what it changed, then run `node $DB ledger` to move the stamp. Once
120
+ unclassified plan. Otherwise follow "Finishing a run" once it's done. Once
118
121
  the plan is fully classified (an already-sorted corpus's re-run
119
122
  carries its prior classifications forward automatically — see "Discover is idempotent"
120
123
  below), `reorg` continues straight through `apply-reorg` → `lint`, no further stop, so
@@ -127,7 +130,7 @@ already sorted: nothing new to classify, so the interview gate never fires.
127
130
 
128
131
  | Mode | Menu option | Does | Destructive |
129
132
  |---|---|---|---|
130
- | `/docs-builder reorg` (discover, classification interview, confirm, then apply-reorg) | *First run*, steps 1-3 | classify a WHOLE corpus into product/logs/archive | no (moves are `git mv`, plan classified and reviewed first) |
133
+ | `/docs-builder reorg` (discover, classification interview, confirm, then apply-reorg) | *First run*, steps 1-3 | classify a WHOLE corpus into product/wiki/logs/archive | no (moves are `git mv`, plan classified and reviewed first) |
131
134
  | `/docs-builder cleanup <file>` | *First run*, step 3's split question | measure ONE named oversized doc (cost, scan, heading shape) → **stops for the interview** | no (measure-only; original preserved) |
132
135
  | `/docs-builder reorg` (bare `docs-builder.cjs reorg`) | *Docs drift* | due's drift summary (if a ledger stamp exists) + discover → (stops here if anything is still unclassified) → apply-reorg → lint, whole corpus | no |
133
136
  | `/docs-builder search <query words...>` | *(none — explicit-argument mode only, never offered in the bare picker)* | BM25-rank sections of `docs/.docs-builder/outline.json` against the query, read-only | no |
@@ -151,16 +154,31 @@ docs/
151
154
  README.md entry point, referenced from AGENTS.md
152
155
  index.md GENERATED by index-flat/apply-reorg/cleanup-apply. never hand-edited.
153
156
  READER-FACING. The WHOLE-CORPUS map — the only file with a completeness
154
- guarantee. ## Product, ## Logs, ## Archive.
157
+ guarantee. ## Product, ## Logs (grouped by subdir), ## Archive.
155
158
  log.md append-only: ## [DATE] operation | description — written by
156
159
  `archive`, `apply-reorg`, `validate`, and `reorg`; NOT written by
157
160
  read-only commands (`due`, `search`, `discover`).
158
- product/ specs, designs, plansthe default. `apply-reorg` MOVES files here
159
- (`git mv`); content is never rewritten.
160
- logs/ pre-registrations, results, learnings, reports historical, still
161
- relevant. Same MOVE discipline as product/archive.
162
- wiki/ synthesised pages, written by Mode 1 (`cleanup`)'s page writers.
163
- archive/ what got cleaned up: self-declared dead. Originals are BYTE-FROZEN:
161
+ product/ docs ABOUT THE PRODUCT ITSELF specs, PRDs, designs, guides for this
162
+ product. FLAT, no subdirs. `apply-reorg` MOVES files here (`git mv`);
163
+ content is never rewritten. Re-checked on EVERY reorg (files sitting here
164
+ are re-classified along with everything else — agents litter buckets
165
+ over time).
166
+ wiki/ GENERIC knowledge for the whole repo, not product-specific — conventions,
167
+ how-tos, standards, reference. FLAT, no subdirs. Also where Mode 1
168
+ (`cleanup`)'s split page writers put synthesised pages. Re-checked every
169
+ reorg, same as product/.
170
+ logs/ ONE-TIME, specific, timely knowledge tied to one section/scenario/case —
171
+ POCs, experiments, investigations, incident/session write-ups, reports. The
172
+ ONLY bucket that may nest, ONE level: `docs/logs/<group>/*.md` — the group
173
+ is the FIRST path segment under `docs/` (a special subfolder is one
174
+ self-explanatory group, e.g. every POC under `docs/fwd/` stays together as
175
+ `fwd`, however deep a file actually sits inside it), UNLESS that segment is
176
+ itself a bucket name (`product`/`wiki`/`archive` → flat; `logs` → the group
177
+ is the SECOND segment instead). A file with no first segment — loose at the
178
+ repo root, or directly under `docs/` — stays flat. Re-checked every reorg,
179
+ same as product/wiki/.
180
+ archive/ what got cleaned up: self-declared dead. FROZEN — never re-checked, never
181
+ walked by a bare `discover`/`reorg` at all. Originals are BYTE-FROZEN:
164
182
  nothing under here is ever a rewrite target, so a doc lands byte-identical
165
183
  to what it carried in (a clean R100 rename) and stays that way. Links
166
184
  elsewhere POINTING AT it are still repaired. History via `git mv`.
@@ -198,14 +216,19 @@ moved the problem (it then clobbered `outline.json` across concurrent splits ins
198
216
  **the themed index was removed outright, 2026-08-24**. One index, rebuilt on every reorg and
199
217
  after every split, is the whole design.
200
218
 
201
- **Never moved — enforced in code, not just documented** (`PROTECTED_NAMES` / `walkMd`):
219
+ **Never moved — enforced in code, not just documented** (`PROTECTED_NAMES` / `isProtectedName` /
220
+ `walkMd`):
202
221
 
203
- - **Files, at any depth:** `README.md`, `index.md`, `log.md`, `CHANGELOG.md`, `LICENSE.md`,
204
- `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md`, `CLAUDE.md`, `AGENTS.md`, `AGENT.md`.
205
- Bare `LICENSE`/`NOTICE` have no `.md` extension, so the walker never sees them.
222
+ - **Files, at any depth, matched CASE-INSENSITIVELY:** `README.md`, `index.md`, `log.md`,
223
+ `CHANGELOG.md`, `LICENSE.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md`,
224
+ `CLAUDE.md`, `AGENTS.md`, `AGENT.md` so `readme.md`, `Claude.md`, `changelog.md`,
225
+ `agents.md`, etc. are protected too, not just their exact-case forms. Bare `LICENSE`/`NOTICE`
226
+ have no `.md` extension, so the walker never sees them.
206
227
  - **Directories:** every dot-dir (`.git/`, `.github/`, `.claude/`, `.factory/`, `.opencode/`,
207
- `.amp/`, `.docs-builder/`) plus `node_modules/`, and the dirs reorg itself owns
208
- (`product/`, `logs/`, `archive/`, `wiki/`) so a second run is idempotent.
228
+ `.amp/`, `.docs-builder/`) plus `node_modules/`. `archive/` is skipped unconditionally by a
229
+ bare `discover`/`reorg` (frozen, never re-checked); `product/`, `wiki/` and `logs/` are
230
+ entered and re-checked on that same bare run — only an explicit `discover <dir>` naming one
231
+ of the reserved names directly bypasses this at all four.
209
232
 
210
233
  ---
211
234
 
@@ -225,11 +248,31 @@ purpose, not silently dropped.
225
248
  ### 1. Discover (script) — enriches and PROPOSES, never classifies, never moves
226
249
 
227
250
  ```bash
228
- node $DB discover # defaults to docs/
251
+ node $DB discover # root-level .md files (top level only) + everything under docs/
252
+ node $DB discover <dir> # scopes to exactly <dir> instead — e.g. a directory outside docs/
229
253
  ```
230
254
 
231
- Recursively finds every `*.md` under the root (skipping `wiki/`, `logs/`, `archive/`,
232
- `product/`, `.docs-builder/`, and the protected files), and for each one writes a row with:
255
+ **Scan scope with no argument is deliberately narrow:** (a) `.md` files sitting directly at
256
+ the repo root (top level, not recursive) and (b) everything under `docs/` (recursive, entering
257
+ `product/`, `wiki/` and `logs/` — only `docs/archive/` stays frozen and skipped). Every other
258
+ `.md` file anywhere else in the repo is out of scope entirely — never listed, never moved,
259
+ whether or not `docs/` exists. `PROTECTED_NAMES` still applies at the repo root (case-
260
+ insensitively — see Layout above), so `README.md`, `CLAUDE.md`, `CHANGELOG.md`, `AGENTS.md`,
261
+ `AGENT.md`, `LICENSE.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md`, `index.md`
262
+ and `log.md` are never planned or moved. `ROOT=` is NOT read by this script — setting it
263
+ prints a `WARN`; pass the folder as the argument.
264
+
265
+ **Because `product/`, `wiki/` and `logs/` are re-checked every bare run,** a file already
266
+ sitting in one of them is a REAL row too, not folded away — its `suggested` bucket defaults to
267
+ wherever it already sits (a stronger signal, like a filename token or a SHOUTED status word,
268
+ can still override that). It still needs `bucket` filled by the interview like any other row
269
+ (even if the answer is "yes, stays put") — that settles after one classify+apply pass and
270
+ carries forward from then on, same as everything else. An explicit `discover <dir>` still
271
+ scopes to exactly that one directory, unaffected — useful for anything outside the default
272
+ root+docs/ coverage.
273
+
274
+ Recursively finds every in-scope `*.md` (skipping `.docs-builder/` and the protected files, at
275
+ any depth in scope), and for each one writes a row with:
233
276
 
234
277
  - `h1` and a short `snip` (first ~200 chars of body, fence-masked) — reused straight from the
235
278
  same `headings()`/`snippet()`/`fenceMask()` parsers `scan` uses, no second extraction path.
@@ -238,14 +281,22 @@ Recursively finds every `*.md` under the root (skipping `wiki/`, `logs/`, `archi
238
281
  *sorted* — it is no longer a bucket.
239
282
  - `suggested` + `reason` — a mechanical PRIOR, never a verdict:
240
283
 
284
+ Checked in this order — the first rule that matches wins:
285
+
241
286
  | suggested | rule |
242
287
  |---|---|
243
288
  | `archive` | path already under `archive/old/reports/phases`, **or** the doc's own opening declares a SHOUTED status word (`CLOSED`, `DEPRECATED`, `SUPERSEDED`, `WITHDRAWN`, `RETRACTED`, `REFUTED`, `ARCHIVAL`, `ARCHIVED`), **or** the filename matches an archive-shaped prefix (`REPORT`, `STATUS`, `SUMMARY`, `FIX_`, `PHASE_`, `SPRINT_`, `DRAFT`, `WIP`, `OLD`, `TEMP` followed by `-` or `_`) |
244
289
  | `logs` | filename carries an experiment-record token — `PREREG`, `LEARNINGS`, `REPORT`, `RESULTS`, `POSTMORTEM`, `RETRO` (case-sensitive, word-boundary, checked ONLY after the archive rules above, so a `REPORT-old.md` still reads as archive, not logs) |
245
- | `product` | has an H1, no archive/logs signal the default when nothing else applies |
290
+ | *(residency)* | already resident under `docs/product/`, `docs/wiki/` or `docs/logs/` — its own current bucket is its prior (a stronger signal above can still override, e.g. a product-resident doc now shouting `DEPRECATED`) |
291
+ | `archive`/`logs`/`wiki` | a WEAKER, case-insensitive secondary prior read from the doc's own **H1 + first 3 H2s** — e.g. "Postmortem"/"Retrospective"/"Investigation" → `logs`; "deprecated"/"retired"/"superseded" → `archive`; "Conventions"/"How-to"/"Style Guide"/"Glossary" → `wiki`. Deliberately narrow: bare "guide"/"reference" were tried and DROPPED after false-positiving on an ordinary doc plainly titled "Guide". |
292
+ | `product` | has an H1, no archive/logs/wiki signal — the default when nothing else applies |
246
293
  | `product` | no H1, but an **include stub** — its whole non-blank content (≤3 lines) is nothing but include directives (mkdocs `--8<--`, `{% include %}`, `{{ .. }}`, `<!-- include -->`) and/or markdown links | a live pointer, not an unknown doc — real-world miss: uv's `docs/reference/contributing.md` |
247
294
  | `product` | no H1 at all, and not an include stub — no strong signal either way; the interview decides, same as any other row |
248
295
 
296
+ `suggested` never proposes `wiki` from a residency check alone reaching further than its own
297
+ match — the heuristic is only ever a prior; the interview decides the real `bucket`, including
298
+ routing a doc INTO `wiki` from anywhere.
299
+
249
300
  - `bucket` — **empty on any row discover has not classified before** (see carry-forward
250
301
  below; a re-run keeps a bucket the interview already set). This is the field the
251
302
  classification interview (step 2) fills, and the ONLY field `apply-reorg` reads to decide
@@ -304,7 +355,7 @@ was the silent move, not the judgement.
304
355
 
305
356
  1. Read `docs/.docs-builder/reorg-plan.json`. Feed the model the WHOLE table — `file`, `h1`,
306
357
  `snip`, `lines`, `suggested`+`reason` — **in one call**, and have it fill `bucket` for
307
- every row (`product`/`logs`/`archive`) with a one-line reason. `suggested` is a PRIOR the
358
+ every row (`product`/`wiki`/`logs`/`archive`) with a one-line reason. `suggested` is a PRIOR the
308
359
  model is shown, never an authority over it — but a SHOUTED self-declared status
309
360
  (`**Status: CLOSED**`) is near-conclusive for `archive` regardless of what the mechanical
310
361
  prior says.
@@ -344,9 +395,15 @@ message, not a crash — so nothing can move on an unreviewed plan. A plan from
344
395
  version (`bucket: 'oversized'` or `'review'`, both gone from the schema) is refused too, with
345
396
  a pointer to re-run `discover`.
346
397
 
347
- - `product` → verified `git mv` to `docs/product/<basename>`
348
- - `logs` → verified `git mv` to `docs/logs/<basename>`
349
- - `archive` → verified `git mv` to `docs/archive/<basename>`
398
+ - `product` → verified `git mv` to `docs/product/<basename>` (always flat)
399
+ - `wiki` → verified `git mv` to `docs/wiki/<basename>` (always flat)
400
+ - `logs` → verified `git mv` to `docs/logs/<basename>` (loose) **or**
401
+ `docs/logs/<group>/<basename>` — the ONE bucket that may nest, ONE level, where the group is
402
+ the file's FIRST path segment under `docs/` (unless that segment is itself a bucket name —
403
+ see the Layout section above). A deeper path inside the same special subfolder joins the
404
+ SAME group (no ratchet on a re-check either).
405
+ - `archive` → verified `git mv` to `docs/archive/<basename>` (always flat, and frozen — never
406
+ re-checked again once there)
350
407
  - **Oversized files move too — size decides splittable, not sorted.** No bucket is exempt.
351
408
  After the move, every oversized row is printed as a follow-up list at its NEW path, one
352
409
  `cleanup <path> (N lines)` line per file — run `cleanup` (Mode 1, below) on each, by hand,
@@ -365,20 +422,10 @@ a pointer to re-run `discover`.
365
422
  static — it never varies with row count, so a re-run rewrites identical bytes. Idempotent:
366
423
  an existing block is replaced in place, never duplicated; other content is left alone.
367
424
  The target is `CONFIG=` (default `CLAUDE.md`); this package uses `CONFIG=AGENTS.md`.
368
- - **The moves land STAGED in your git index — commit them promptly.** `git mv` stages each
369
- rename immediately (that is what preserves history), so when `apply-reorg` returns the repo
370
- is holding N staged renames. Any other session's `git add -A` or `git commit -a` will absorb
371
- them into an unrelated commit OBSERVED TWICE, in two different repos. `apply-reorg` prints
372
- a closing advisory naming the counts and a copy-pasteable recipe. **Run that recipe
373
- VERBATIM. Do NOT hand-edit it, and do NOT stage by hand instead.** If it looks incomplete
374
- or names a path that errors, that is a BUG in the recipe — stop and report it to the user;
375
- do not silently repair it and move on. OBSERVED, real (privcloud first field run): the
376
- recipe omitted `docs/log.md`, the operator quietly added it by hand, and the bug only
377
- surfaced because they were later asked for near-misses — a silent repair is a lost bug
378
- report. Do NOT scope the commit to `docs` alone either: the renames are staged, but the
379
- inbound-link rewrites are UNSTAGED and reach outside `docs/` (`src/`, `scripts/`, `tests/`,
380
- `README.md`). Both belong in ONE commit, or you ship moved files whose links were never
381
- repaired. The tool never auto-commits, by design.
425
+ - **The moves land STAGED in your git index — nothing is committed for you.** `git mv` stages
426
+ each rename immediately (that is what preserves history), so when `apply-reorg` returns the
427
+ repo is holding N staged renames plus unstaged link rewrites. `apply-reorg` prints a closing
428
+ commit advisory; see "Finishing a run" below for the actual commit flow.
382
429
  - A basename collision (two files, same name, different original folders) is
383
430
  disambiguated (`-2`, `-3`, …); a collision with a **file that already exists at the
384
431
  destination** is skipped, logged, and does not stop the rest of the run.
@@ -389,7 +436,9 @@ a pointer to re-run `discover`.
389
436
  touches — is never removed. Only directories THIS run emptied are candidates; a dir that
390
437
  happened to already be empty before this run started is not this tool's to remove.
391
438
  - **After every move, `apply-reorg` re-scans the whole corpus** — `docs/product/`,
392
- `docs/logs/`, and `docs/archive/` all straight into `outline.json`, the database `search`
439
+ `docs/logs/`, and `docs/archive/` all (`docs/wiki/` is excluded, same as any other
440
+ `PAGES` dir — it holds synthesised pages, not source docs to reorg) — straight into
441
+ `outline.json`, the database `search`
393
442
  reads. Not a hint, not opt-in: it runs every time, even when nothing moved this run (e.g.
394
443
  re-running on a corpus already sorted from a previous pass). Measured bug this closes: on a
395
444
  real 37-doc corpus, `outline.json` used to hold records for only the 12 files a split had
@@ -398,6 +447,47 @@ a pointer to re-run `discover`.
398
447
  content, so a pre-move scan would just be redone), and reuses the same `scan` used
399
448
  everywhere else in this pipeline — no second scanner, no second outline format.
400
449
 
450
+ ### Finishing a run — the commit flow, run by hand every time
451
+
452
+ `apply-reorg`/`archive`/`cleanup-apply` never auto-commit, by design. Each run's staged renames
453
+ and unstaged link rewrites (in `.md` files only, but anywhere — e.g. the root `README.md`)
454
+ belong in ONE commit, including the
455
+ `AGENTS.md` docs-pointer block `apply-reorg` writes — it is in `commit-files.txt` along with
456
+ everything else. Run these steps literally, in order:
457
+
458
+ 1. `git branch --show-current`. If it prints `main` or `master`, do **NOT** commit — tell the
459
+ user the files are ready and to switch to a branch first.
460
+ 2. `cat docs/.docs-builder/commit-files.txt` to see exactly what this run touched, and
461
+ `cat docs/.docs-builder/commit-dirty.txt` to see which of those files already carried the
462
+ operator's own uncommitted edits BEFORE this run — pathspec can't split hunks, so committing
463
+ the file commits that edit too. Then ask with `AskUserQuestion`, header `Commit`: if
464
+ `commit-dirty.txt` lists anything, **"Commit these N files now? Note: these files also
465
+ carry your own uncommitted edits, which will be committed too: …"** (name them); otherwise
466
+ **"Commit these N files now?"** — **Commit** (run the printed recipe) / **Leave uncommitted**
467
+ (say what is pending; nothing this run did gets undone).
468
+ 3. On **Commit**, run the printed recipe line EXACTLY as printed:
469
+ ```
470
+ git add --pathspec-from-file=docs/.docs-builder/commit-add.txt && git commit -m "docs: reorg" --pathspec-from-file=docs/.docs-builder/commit-files.txt
471
+ ```
472
+ (when `REPO` isn't the shell's cwd, both commands are printed as `git -C '<REPO>' …` instead
473
+ — run that form, unmodified). Do not hand-edit it, do not stage by hand instead, and do not
474
+ scope it to `docs/` alone — a `.md` outside `docs/` (e.g. `README.md`) can carry a repaired
475
+ link. If it errors or names a path that doesn't exist, that is a BUG: stop and report it to
476
+ the user; do not silently hand-repair and move on.
477
+ OBSERVED, real (privcloud first field run, pre-dating the pathspec-file recipe): a
478
+ hand-rolled recipe once omitted `docs/log.md` and the operator quietly added it by hand —
479
+ the bug only surfaced later, when asked for near-misses. A silent repair is a lost bug
480
+ report.
481
+ 4. After a successful commit, run `node $DB ledger` to stamp the consolidation — nothing in
482
+ `discover`/`apply-reorg`/`archive`/`cleanup-apply` stamps it for you, and without the stamp
483
+ `due` stays NOT due, the picker's verdict stays uninformed, and `/remember`'s docs nudge
484
+ never fires.
485
+
486
+ Add `docs/.docs-builder/` to `.gitignore` before the first commit if it is not already
487
+ ignored — it is machine state, regenerated every run, and the ledger stamp is per-clone by
488
+ design. Never use `git add -A` / `git add -u` / `git commit -a` for this: either would ALSO
489
+ absorb any unrelated in-flight work in the tree — OBSERVED TWICE, in two different repos.
490
+
401
491
  **After each move it repairs the paths that move just broke** — the whole point of doing this
402
492
  in a script. Both movers (`apply-reorg` and `archive`) go through ONE function, `moveDoc`, so
403
493
  the follow-up list cannot be added to one and missed by the other; that exact miss shipped
@@ -409,8 +499,12 @@ reported as a file that needs re-moving. Two follow-ups:
409
499
  (`records[].file`, and the `<file> :: ` prefix inside every key). This is the same
410
500
  function `archive` calls; `apply-reorg` used to bypass it, which silently invalidated
411
501
  every key of every file it moved. Both now reach it through `moveDoc`.
412
- 2. **Inbound links** — every git-tracked `.md`/`.js`/`.cjs`/`.mjs`/`.json`/`.yml` file that
413
- points at the old path (repo-rooted, e.g. `docs/GUIDE.md`) is rewritten to the new one.
502
+ 2. **Inbound links** — every git-tracked `.md` file, anywhere in the repo, that points at the
503
+ old path (repo-rooted, e.g. `docs/GUIDE.md`) is rewritten to the new one. Only `.md` files
504
+ are ever opened, read, or edited — this is a DOCS tool, not a repo-wide text rewriter.
505
+ FIELD BUG, real (bareloop, 2026-09-10): the previous version also scanned
506
+ `.js`/`.cjs`/`.mjs`/`.json`/`.yml`, and rewrote 6 signed JSON job specs (breaking their
507
+ hashes), a byte-signed `.mjs` close script, and a code comment that tripped a commit gate.
414
508
  In `.md` files specifically, a RELATIVE link is also caught: inside actual markdown link
415
509
  syntax only (`[text](../concepts/x.md)` or a reference-style `[label]: ./tools.md`), never
416
510
  bare prose, the target is resolved against the SCANNING file's own directory, and — if it
@@ -445,13 +539,14 @@ predicate, `isRewriteExempt`, at one call site. `docs/.docs-builder/` is exclude
445
539
 
446
540
  **A known, deliberate trade-off: this is a literal exact-path match over raw file bytes, not
447
541
  fence-aware or context-aware.** It rewrites every exact, word-bounded occurrence of the old path
448
- in every git-tracked `.md`/`.js`/`.cjs`/`.mjs`/`.json`/`.yml` file (except the two exemptions
449
- above) — inside a code fence, inside a sentence describing history ("this used to live at
450
- docs/OLD.md"), anywhere. A prose mention of where a file *used to be* WILL be rewritten to say
451
- where it is now, changing what the sentence says. This is intentional, not an oversight: a dead
452
- link is worse than a reworded sentence, the match is exact rather than inferred (unlike the
453
- dangling-reference *lint*, which infers and was cut outright at 1/27 precision), and every
454
- rewrite is printed per file so it is visible, never silent.
542
+ in every git-tracked `.md` file (except the two exemptions above) — inside a code fence, inside
543
+ a sentence describing history ("this used to live at docs/OLD.md"), anywhere. A prose mention of
544
+ where a file *used to be* WILL be rewritten to say where it is now, changing what the sentence
545
+ says. This is intentional, not an oversight: a dead link is worse than a reworded sentence, the
546
+ match is exact rather than inferred (unlike the dangling-reference *lint*, which infers and was
547
+ cut outright at 1/27 precision), and every rewrite is printed per file so it is visible, never
548
+ silent. Only `.md` files are ever in scope, so this trade-off never touches non-doc files —
549
+ see "Inbound links" above.
455
550
 
456
551
  ---
457
552
 
@@ -742,14 +837,16 @@ node $DB index-flat
742
837
  Writes **one** `docs/index.md` covering the whole corpus, in three sections: `## Product`
743
838
  (one row per file under `docs/product/`, plus any pages under `PAGES` — default `docs/wiki/`
744
839
  — if they exist, plus any doc still sitting in place elsewhere), `## Logs` (one row per file
745
- under `docs/logs/`), and `## Archive` (one row per file under `docs/archive/`). Each row is
746
- an H1 title, a line count, and a link, plus one indented line per H2 heading (in document
747
- order) so an agent can find and slice-read a section without opening the doc — each H2
748
- line carries its own `(Lstart–end)` line range, reusing the SAME `headings()`+`fenceMask()`
749
- boundaries `scan` already writes to `outline.json` (no second parser). Omitted when the doc
750
- has no H2s. `## Archive` rows are H1-only, never H2 lines an archived doc is frozen
751
- history, not a live section to route into. No theme grouping, no `labels.json`, no model
752
- call.
840
+ under `docs/logs/`, **grouped by its one-level subdir** a loose `docs/logs/x.md` row is
841
+ listed ungrouped first, then each subdir gets its own `### <subdir>/` header with its files
842
+ under it, subdirs sorted alphabetically), and `## Archive` (one row per file under
843
+ `docs/archive/`). Each row (product or logs, grouped or not) is an H1 title, a line count, and
844
+ a link, plus one indented line per H2 heading (in document order) so an agent can find and
845
+ slice-read a section without opening the doc each H2 line carries its own `(Lstart–end)`
846
+ line range, reusing the SAME `headings()`+`fenceMask()` boundaries `scan` already writes to
847
+ `outline.json` (no second parser). Omitted when the doc has no H2s. `## Archive` rows are
848
+ H1-only, never H2 lines — an archived doc is frozen history, not a live section to route into.
849
+ No theme grouping, no `labels.json`, no model call.
753
850
  Default destination `docs/index.md` — **the only writer of that default path** in this whole
754
851
  pipeline (nothing else writes an index at all).
755
852
  `search` reads `outline.json`, never `index.md`. Prints the row counts and records a `log.md`
@@ -770,7 +867,8 @@ v3 folds the old `reconcile` and `due` commands into one: "first run" (nothing s
770
867
  state, and two separate commands only made users guess which one to run.
771
868
 
772
869
  ```bash
773
- node $DB reorg
870
+ node $DB reorg # root-level .md files (top level only) + everything under docs/
871
+ node $DB reorg <dir> # re-checks every doc already inside <dir> — passes <dir> to discover
774
872
  ```
775
873
 
776
874
  If a ledger stamp exists (see "Knowing when reorg is due" below), its `due`-style drift
@@ -123,19 +123,49 @@ confirmed in Phase 0.5.
123
123
  summarize, escalate. Do not weigh it yourself.
124
124
  - **All clean** → continue.
125
125
 
126
- ## Phase 2 — Docs sweep
127
- Update what this feature actually changed, wherever those docs live in this
128
- project — match each file's existing format, touch nothing unrelated. Use
129
- `docs/index.md` when the project has one to find what exists.
130
-
131
- - **CHANGELOG.md** — new entry.
132
- - **README.md** — only if user-facing usage changed.
126
+ ## Phase 2 — Docs sweep (required — no skipping, no sampling)
127
+ Update what this branch changed, wherever those docs live in this project —
128
+ match each file's existing format, touch nothing unrelated. Use
129
+ `docs/index.md` when the project has one to find what exists. **All three
130
+ passes run on every release**; the size of the branch or the change never
131
+ cuts one.
132
+
133
+ 1. **List every change.** Read `git log --format='%h %s%n%b'
134
+ origin/main..HEAD` — the bodies, not just the subjects — and the diff.
135
+ Write one line per user-visible change: feature, command, flag, behaviour,
136
+ fix, dependency bump. A subject is a summary; the body is the list, and a
137
+ sweep built from subjects drops whatever only a body mentions.
138
+ 2. **CHANGELOG.md** — a new entry holding every line from pass 1, each under
139
+ the heading the file already uses for its kind (Added / Changed / Fixed /
140
+ Security). A new capability is **Added** even when it shipped in a `fix:`
141
+ commit. Then check it back: every pass-1 line maps to an entry, and the
142
+ semver level in Phase 3 agrees with the headings (an Added entry means at
143
+ least minor).
144
+ 3. **Grep for stale text.** For every string the diff removed or replaced
145
+ that a reader might have copied — a command line, flag, file name, recipe,
146
+ env var, printed message — search the docs for the old form:
147
+ `grep -rnF "<old string>" --include='*.md' .` (past CHANGELOG entries are
148
+ history; leave them). Every other hit is stale **because of this branch** —
149
+ update it. "It was already stale before this branch" is a claim: prove it
150
+ with `git show $(git merge-base origin/main HEAD):<source path>`. If the
151
+ base code already disagreed with the doc, report it as out of scope;
152
+ otherwise it is yours to fix.
153
+
154
+ Then judge each of these against the pass-1 list:
155
+ - **README.md** — if user-facing usage changed.
133
156
  - **PRD** — the feature's entry / status.
134
157
  - **Guide / context docs** — the project's standing context.
135
158
  - **Findings / learnings** — where the project keeps them.
136
159
  - **Any other frequently-updated doc** this change makes stale.
137
160
 
138
- If a doc needs no change, **say so** rather than editing it for its own sake.
161
+ Report **one row per doc**: file · changed / no change · the evidence (the
162
+ grep or diff you ran). A "no change" with no evidence is a skip, and a skip
163
+ fails this phase — it is not a pass. If a doc truly needs no change, say so
164
+ with the evidence rather than editing it for its own sake.
165
+
166
+ **The sweep is the worker's job, start to finish.** The orchestrator checks
167
+ the report; it does not redo or patch the sweep. A gap it finds goes back to
168
+ the worker, and it counts as a failed sweep, not a small follow-up.
139
169
 
140
170
  ## Phase 3 — Cut (local only)
141
171
  1. **Version bump** — pick the semver level from the change (patch / minor /
@@ -19,9 +19,12 @@ These show up in nearly every quickly-built app regardless of stack:
19
19
 
20
20
  1. **Secrets in the repo.** Tokens / API keys / `.env` files committed to
21
21
  tracked files or anywhere in git history. Verify `.env` is gitignored and
22
- only a value-less `.env.example` is tracked; scan history (`git log -p`,
23
- `git grep`) for leaked keys. Secrets must load from env / a secret store at
24
- runtime never hardcoded, never logged.
22
+ only a value-less `.env.example` is tracked. Scan ALL history, every
23
+ branch a range or target never narrows this with:
24
+ `git log --all -p | grep -nE '(sk-[A-Za-z0-9_-]{16,}|AKIA[0-9A-Z]{16}|gh[pousr]_[A-Za-z0-9]{30,}|xox[abprs]-[A-Za-z0-9-]{10,}|AIza[0-9A-Za-z_-]{35}|-----BEGIN [A-Z ]*PRIVATE KEY)'`
25
+ plus any key shape specific to this project's providers. Report each hit
26
+ as real or a known test fixture. Secrets must load from env / a secret
27
+ store at runtime — never hardcoded, never logged.
25
28
  2. **Data-access authorization (tenant isolation).** Every record read or
26
29
  written must be scoped to the requesting principal — via DB-level rules
27
30
  (RLS / row policies) and/or application-layer ownership checks. Flag any