@codewizard-dt/bootstrap 2.15.0 → 2.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/lib/hooks/README.md +192 -184
- package/lib/hooks/absolute-path-guard.js +5 -0
- package/lib/hooks/claude-settings-guard.js +43 -73
- package/lib/hooks/env-content-read-guard.js +13 -3
- package/lib/hooks/env-file-guard.js +42 -0
- package/lib/hooks/interpreter-indirection-guard.js +7 -0
- package/lib/hooks/lib/command-parse.js +8 -0
- package/lib/hooks/lib/serena-languages.js +94 -1
- package/lib/hooks/lib/serena.js +228 -0
- package/lib/hooks/mv-absolute-path-block.js +84 -3
- package/lib/hooks/package-install-consent.js +11 -0
- package/lib/hooks/protected-write-guard.js +7 -0
- package/lib/hooks/serena-bash-grep-block.js +507 -44
- package/lib/hooks/serena-first-glob-guard.js +65 -20
- package/lib/hooks/serena-first-guard.js +98 -2
- package/lib/hooks/serena-first-read-guard.js +92 -2
- package/lib/hooks/serena-pre-delegation.js +123 -0
- package/lib/hooks/serena-write-guard.js +56 -3
- package/lib/scripts/README.md +319 -16
- package/lib/scripts/bootstrap-prefs.js +734 -0
- package/lib/scripts/install-global.sh +182 -25
- package/lib/scripts/install-mcps.sh +102 -14
- package/lib/scripts/lib.sh +595 -20
- package/lib/scripts/merge-gitignore.sh +254 -16
- package/lib/scripts/merge-settings-hooks.js +374 -0
- package/lib/scripts/sync-wiki-scaffold.sh +33 -4
- package/lib/scripts/templates/CLAUDE-wiki.md +1 -1
- package/lib/scripts/templates/bootstrap-prefs-schema.json +175 -0
- package/lib/scripts/templates/settings-hooks.json +151 -0
- package/lib/scripts/update-project.sh +42 -9
- package/lib/skills/README.md +2 -1
- package/lib/skills/bootstrap-config/SKILL.md +154 -0
- package/lib/skills/git-commit/SKILL.md +54 -1
- package/lib/skills/research/SKILL.md +34 -3
- package/lib/skills/simplify/SKILL.md +1 -1
- package/lib/skills/uat-auto/SKILL.md +9 -4
- package/lib/skills/uat-auto-plus/SKILL.md +6 -6
- package/lib/skills/uat-generate/SKILL.md +30 -3
- package/lib/skills/uat-walk/UAT-CORE.md +41 -6
- package/lib/skills/wiki-query/SKILL.md +48 -8
- package/package.json +1 -1
|
@@ -31,7 +31,7 @@ Autonomous variant of `/uat-auto`: run every eligible test, auto-judge from dete
|
|
|
31
31
|
|
|
32
32
|
## Prime Directive: truthful pass
|
|
33
33
|
|
|
34
|
-
**Never mark `[x] Pass` unless the test currently passes against unmodified test logic.** Fixes go into application code, build/config, infrastructure, or test-environment setup — **never** into the UAT file's Expected/Steps/Command, and never by weakening assertions. If a failure can't be fixed without changing what is being tested, record `[FAIL: auto-fix-declined: <reason>]` and leave the code unchanged. Related: **never delete or `[SKIP: ...]` a test to avoid fixing it** — skip is a human verdict. A false pass is the worst outcome — it ships a broken feature *and* erodes trust in every later run.
|
|
34
|
+
**Never mark `[x] Pass` unless the test currently passes against unmodified test logic.** Fixes go into application code, build/config, infrastructure, or test-environment setup — **never** into the UAT file's Expected/Steps/Command, **never** into the unit test file backing a unit-backed case, and never by weakening assertions. Editing the oracle you are about to be judged by is the one move that turns this mode from useful into dangerous. If a failure can't be fixed without changing what is being tested, record `[FAIL: auto-fix-declined: <reason>]` and leave the code unchanged. Related: **never delete or `[SKIP: ...]` a test to avoid fixing it** — skip is a human verdict. A false pass is the worst outcome — it ships a broken feature *and* erodes trust in every later run.
|
|
35
35
|
|
|
36
36
|
---
|
|
37
37
|
|
|
@@ -42,7 +42,7 @@ Run UAT-CORE headless throughout, adding the fix loop:
|
|
|
42
42
|
1. **Step 1** — resolve/parse (auto/plus resolution: never prompt; auto-pick lowest-numbered pending on empty/unresolved).
|
|
43
43
|
2. **Step 2** — prerequisites with **one autonomous repair attempt** each (start dev server in background + `Monitor` for ready; run migrate; run seed; obtain a documented env var), then re-verify. Still failing after one repair → hard gate (`[FAIL: auto-judge: prerequisite not satisfied — <which>]` on every untested test → Closure). Track every background process started — Closure terminates them.
|
|
44
44
|
3. **Step 3 + 3.5** — classify, then stub-detect (leave stubs `- [ ] Pass`; **do not enter the fix loop** for them — implementing a feature from scratch is out of scope; surface in the summary).
|
|
45
|
-
4. **Step 4** — auto-judge in document order: API/CLI
|
|
45
|
+
4. **Step 4** — auto-judge in document order by evidence channel: **live command** (API/CLI) and **unit-backed** (the recorded `Repeatable Unit Test` file) are both machine-checkable → on `[FAIL]`, **enter the fix loop (below)**. Bare Manual cases and every UI case are `[FAIL: auto-judge: ... requires human verification]` and **never** enter the fix loop (no machine-checkable signal that a fix worked). Eligibility = pending + previously-failed; also reset `[FIXING: ...]` → `- [ ] Pass` before running.
|
|
46
46
|
5. **Step 5** — write each verdict/`[FIXING: ...]` transition **immediately** via `Edit` (no buffering).
|
|
47
47
|
6. **Closure** — archive on all-pass; on any fail leave in place and **exit 0**. This mode additionally deletes this task's screenshots, **terminates every background process** it started, and supports multi-task WIP tracking in decision annotation (all per UAT-CORE Closure "Mode differences: uat-auto-plus"). Log tag ` (auto-plus)`.
|
|
48
48
|
|
|
@@ -67,9 +67,9 @@ The defining feature. On an API/CLI `[FAIL]` in Step 4, do not move on — diagn
|
|
|
67
67
|
**Loop (per failing test, attempts 1–3):**
|
|
68
68
|
1. Mark `[FIXING: attempt N/3 — <short reason>]` via `Edit`.
|
|
69
69
|
2. **Diagnose** — read the test, the relevant app code, recent logs. Navigate with Serena (`find_symbol`, `find_referencing_symbols`, `get_symbols_overview`) — not Grep/Glob. For runtime failures, consider the `debug-logs` skill for a ranked hypothesis list.
|
|
70
|
-
3. **Decide the smallest correct fix** that makes the test pass without weakening it. Acceptable targets: application code, build/config (missing env default, misrouted path, wrong port), test fixtures/seed data. **Never** the UAT file's Steps/Expected/Command; **never** test assertions in code.
|
|
70
|
+
3. **Decide the smallest correct fix** that makes the test pass without weakening it. Acceptable targets: application code, build/config (missing env default, misrouted path, wrong port), test fixtures/seed data. **Never** the UAT file's Steps/Expected/Command; **never** test assertions in code — including the unit test file named in a unit-backed case's `Repeatable Unit Test` metadata, which is off-limits for the whole loop.
|
|
71
71
|
4. **Delegate to a sub-agent** (`Agent` tool, `subagent_type: general-purpose`, or `Plan` for design-heavy fixes). Prompt must include: the test ID + full body; the actual-vs-expected evidence from Step 4; the constraint "do not modify the UAT file or test assertions; fix only application code, build, config, or fixtures"; a request for a short report of files changed + rationale.
|
|
72
|
-
5. **Re-run** the test with the same Step 4 procedure and the same Bash call —
|
|
72
|
+
5. **Re-run** the test with the same Step 4 procedure and the same Bash call — same curl, or the same unit-test command over an unmodified test file (verify it is unchanged before trusting the result). Do not relax assertions or narrow the run.
|
|
73
73
|
6. **Evaluate:** Pass → `[x] Pass`, delete this test's fail screenshots, exit loop, next test. Fail & attempt < 3 → next attempt with diagnosis refined by the prior report. Fail & attempt = 3 → `[FAIL: auto-fix-exhausted: <last reason>]`, exit loop.
|
|
74
74
|
|
|
75
75
|
**Fix-decline cases** — mark `[FAIL: auto-fix-declined: <why>]` immediately (declining is correct judgment, not failure) when:
|
|
@@ -88,7 +88,7 @@ The defining feature. On an API/CLI `[FAIL]` in Step 4, do not move on — diagn
|
|
|
88
88
|
━━━ UAT AUTO-PLUS COMPLETE ━━━
|
|
89
89
|
File: wiki/work/uat/5-positions.uat.md · Source: wiki/work/tasks/5-positions.md · Mode: autonomous-fix
|
|
90
90
|
Budget: attempts 7/N · wall 12m22s/30m
|
|
91
|
-
✅ Passed 7 (fixed during run: 4) · ⚠️ Skipped 1 (pre-existing) · ❌ Failed 1 (auto-fix-exhausted 1 · declined 0 · auto-judge-uncertain 0)
|
|
91
|
+
✅ Passed 7 (live command 5 · unit-backed 2 · fixed during run: 4) · ⚠️ Skipped 1 (pre-existing) · ❌ Failed 1 (auto-fix-exhausted 1 · declined 0 · auto-judge-uncertain 0)
|
|
92
92
|
❔ Pending 0 · 🔲 Stub-detected 0 (left untouched — implement first) · Total 9
|
|
93
93
|
Fixes Applied:
|
|
94
94
|
• UAT-API-001: added missing 422 handler in src/api/positions.ts
|
|
@@ -104,7 +104,7 @@ On all-pass, replace `Next action` with `Archived to archive/` and the new paths
|
|
|
104
104
|
|
|
105
105
|
## Rules
|
|
106
106
|
|
|
107
|
-
**Autonomy boundaries** — the agent **may** modify application code, build/config, fixtures, and seed data. It **may not**: modify any UAT file's Steps/Expected/Command; modify test assertions in code (`expect(...)`); disable, comment out, `xfail`, or `[SKIP: ...]` a failing test; weaken type signatures, lower error-throwing branches, or remove validation to force a pass. When in doubt, decline and surface for review.
|
|
107
|
+
**Autonomy boundaries** — the agent **may** modify application code, build/config, fixtures, and seed data. It **may not**: modify any UAT file's Steps/Expected/Command; modify test assertions in code (`expect(...)`), including any unit test file backing a unit-backed case; disable, comment out, `xfail`, or `[SKIP: ...]` a failing test; narrow a test filter so fewer cases run; weaken type signatures, lower error-throwing branches, or remove validation to force a pass. When in doubt, decline and surface for review.
|
|
108
108
|
|
|
109
109
|
**Verdict discipline** — `[x] Pass` only on machine-verified evidence against unmodified logic; `[FAIL: auto-fix-exhausted: ...]` after 3 attempts; `[FAIL: auto-fix-declined: ...]` when out of scope/unsafe; `[FAIL: auto-judge: ...]` when unverifiable (UI/manual); **never** `[SKIP]`.
|
|
110
110
|
|
|
@@ -17,6 +17,8 @@ UAT owns **runtime and end-to-end verification**. `/tackle` only runs static gat
|
|
|
17
17
|
|
|
18
18
|
When a UAT case also contains behavior that can easily be preserved as a deterministic repeatable unit test, create that unit test immediately. UAT should not be the only place a cheap, automatable assertion lives.
|
|
19
19
|
|
|
20
|
+
Promoting a case pays off twice: the assertion becomes repeatable, **and** the case becomes headlessly judgeable. `/uat-auto` treats a recorded unit test as a first-class evidence channel, so a Manual or edge case with one auto-passes on a green run instead of fail-closing to human verification. Every case you promote is one fewer case a human has to walk.
|
|
21
|
+
|
|
20
22
|
---
|
|
21
23
|
|
|
22
24
|
**Target**: $ARGUMENTS
|
|
@@ -85,6 +87,23 @@ Do not proceed to Step 3 until you can answer yes to all:
|
|
|
85
87
|
|
|
86
88
|
#### Step 2.5: Unit-test promotion checkpoint (mandatory)
|
|
87
89
|
|
|
90
|
+
##### Preference gate
|
|
91
|
+
|
|
92
|
+
This gate governs **only whether files are written under `test/`**. It never affects the UAT file itself or how UAT verdicts are judged.
|
|
93
|
+
|
|
94
|
+
Read the preference **once per invocation** (not per case), before Step 2.5 does any other work:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
node ~/.claude/bootstrap-prefs.js --get uatGenerate.promoteTests --project . 2>/dev/null || echo unset
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
| Value | Behaviour |
|
|
101
|
+
|-------|-----------|
|
|
102
|
+
| `true` | Today's behaviour — run the promotion checkpoint and create/update the unit test files. |
|
|
103
|
+
| `false` | Do **not** create or update anything under `test/`. Still run the promotability *analysis* and still record it per case, as `- **Repeatable Unit Test**: Skipped by preference (uatGenerate.promoteTests=false)`. Say once in the completion report (Step 7): "No test/ files were written (uatGenerate.promoteTests=false). Change with /bootstrap-config." |
|
|
104
|
+
| `ask` | Ask the user **once per invocation** (AskUserQuestion), before Step 2.5 does any work, whether to promote tests this run. Honour the answer for this run only — never write it back to the store. |
|
|
105
|
+
| `unset` | Promote (today's behaviour) and note once: "uatGenerate.promoteTests is unanswered — /uat-generate promotes tests by default. Set it with: node ~/.claude/bootstrap-prefs.js --set uatGenerate.promoteTests --value <true\|false\|ask> --global" |
|
|
106
|
+
|
|
88
107
|
For every planned UAT case, decide whether any part of it can be captured as a repeatable unit test with low setup cost.
|
|
89
108
|
|
|
90
109
|
A UAT case is **unit-test promotable** when all are true:
|
|
@@ -104,7 +123,7 @@ Before writing the UAT file, inspect the existing test layout and runner:
|
|
|
104
123
|
- Prefer adding focused assertions to an existing nearby test file when that is the local pattern.
|
|
105
124
|
- Otherwise create the smallest conventional new test file beside the unit under test.
|
|
106
125
|
|
|
107
|
-
If a case is promotable
|
|
126
|
+
If a case is promotable **and the preference gate allows promotion**, you **must** create or update the unit test file. Do not merely list it as a recommendation. If you cannot create it because the test framework is absent or the harness would require new infrastructure, record that as a gap with the concrete blocker. When the gate suppresses promotion, do the analysis and record `Skipped by preference` — write nothing under `test/`.
|
|
108
127
|
|
|
109
128
|
### Step 3: Generate UAT test cases
|
|
110
129
|
|
|
@@ -168,6 +187,7 @@ implements::[[TASK-NNN]]
|
|
|
168
187
|
- **Steps**: [How to trigger this scenario]
|
|
169
188
|
- **Expected Result**: [How the system should handle it]
|
|
170
189
|
- **Repeatable Unit Test**: [Created: `path/to/test-file` | Not applicable: reason | Blocked: reason]
|
|
190
|
+
- **Unit Test Command**: `node --test test/example.test.js`
|
|
171
191
|
- [ ] Pass
|
|
172
192
|
```
|
|
173
193
|
|
|
@@ -190,6 +210,13 @@ implements::[[TASK-NNN]]
|
|
|
190
210
|
- Add `- **Repeatable Unit Test**: Created: \`path/to/test-file\`` when you created or updated a unit test for the behavior.
|
|
191
211
|
- Add `- **Repeatable Unit Test**: Not applicable: <short reason>` when the case cannot reasonably become a unit test.
|
|
192
212
|
- Add `- **Repeatable Unit Test**: Blocked: <short reason>` only when it was promotable in principle but the repo lacks the needed test harness or fixtures.
|
|
213
|
+
- Add `- **Repeatable Unit Test**: Skipped by preference (uatGenerate.promoteTests=false)` when the Step 2.5 preference gate suppressed promotion for this run. Record the promotability analysis as usual; only the file write is skipped.
|
|
214
|
+
|
|
215
|
+
**Unit test command** (required whenever, and only when, the line above says `Created:`):
|
|
216
|
+
- Add `- **Unit Test Command**: \`<cmd>\`` on the next line, holding the **file-scoped** command that runs exactly that test file — e.g. `node --test test/foo.test.js`, `npx vitest run test/foo.test.ts`, `pytest tests/test_foo.py`, `go test ./pkg/foo/`.
|
|
217
|
+
- Verify it by running it once before writing the UAT file. A command that has never been run is a guess, and a wrong one costs a false failure later.
|
|
218
|
+
- This is what makes a Manual or edge-case test **auto-judgeable**: `/uat-auto` and `/uat-auto-plus` run it as their evidence channel instead of fail-closing the case to human verification (UAT-CORE Step 4, "Unit-backed"). Omitting it forces them to derive a command from the repo's conventions, and a case they cannot resolve one for fails closed. It costs one line here to save a human triage pass later.
|
|
219
|
+
- Do not point it at the whole suite (`npm test`) when a file-scoped form exists — a green suite is weak evidence about one case, and slow to re-run inside the auto-plus fix loop.
|
|
193
220
|
|
|
194
221
|
**Auth metadata** (required for every auth-gated test):
|
|
195
222
|
```
|
|
@@ -201,7 +228,7 @@ Auth-Role: user
|
|
|
201
228
|
|
|
202
229
|
### Step 4: Write repeatable unit tests, the UAT file, and cross-reference
|
|
203
230
|
|
|
204
|
-
1.
|
|
231
|
+
1. **If the Step 2.5 preference gate allows promotion**, create or update the actual unit test file for each promotable case from Step 2.5, before writing the UAT file. If the gate suppressed promotion, skip this step entirely — write nothing under `test/` — and record each promotable case as `Skipped by preference` instead.
|
|
205
232
|
- Use existing test framework conventions and nearby examples.
|
|
206
233
|
- Keep the test focused on the behavior from the UAT case.
|
|
207
234
|
- Do not add broad snapshots or duplicate end-to-end checks as unit tests.
|
|
@@ -209,7 +236,7 @@ Auth-Role: user
|
|
|
209
236
|
- If editing code test files, use Serena editing tools.
|
|
210
237
|
- If creating new config/markdown/test files where Serena is unavailable, use the appropriate project-approved edit/write tool; never shell redirection.
|
|
211
238
|
|
|
212
|
-
2. Include the created test path in the corresponding UAT case's `Repeatable Unit Test` metadata.
|
|
239
|
+
2. Include the created test path in the corresponding UAT case's `Repeatable Unit Test` metadata, plus the verified file-scoped `Unit Test Command` — or `Skipped by preference (uatGenerate.promoteTests=false)` when the gate suppressed promotion.
|
|
213
240
|
|
|
214
241
|
3. Write `wiki/work/uat/UAT-NNN-slug.md` using the `Write` tool.
|
|
215
242
|
|
|
@@ -58,6 +58,8 @@ For each unchecked prerequisite: if it is a **runnable check** (server on a port
|
|
|
58
58
|
| **UI** | `UAT-UI-*` prefix, or `Page:` / `Components:` metadata |
|
|
59
59
|
| **Manual** | anything else (edge cases, concurrency, integration logic) |
|
|
60
60
|
|
|
61
|
+
**Unit-backed** is an *attribute*, not a type — it cuts across all three. A test is unit-backed when it records `- **Repeatable Unit Test**: Created: \`<path>\`` (written by `/uat-generate` Step 2.5 when it promoted the case to a repeatable unit test). Note the path, and the `- **Unit Test Command**: \`<cmd>\`` line if present; Step 4 uses them as a machine-checkable evidence channel. The other three values of that field — `Not applicable:`, `Blocked:`, `Skipped by preference` — are **not** unit-backed: no test file exists to run.
|
|
62
|
+
|
|
61
63
|
---
|
|
62
64
|
|
|
63
65
|
## Step 3.5 — Stub detection (headless modes only)
|
|
@@ -71,6 +73,8 @@ A stub test **stays `- [ ] Pass`** — do not execute it, do not record `[FAIL]`
|
|
|
71
73
|
3. **Stub indicators** (via `find_symbol` / `search_for_pattern`): `TODO`/`FIXME`/`HACK` inside a body; `throw new Error('not implemented')`, `raise NotImplementedError`, `notImplemented()`, `unimplemented!()`; a bare `pass` / `pass # TODO|stub`; empty body (`{}`, `return`, `return null|undefined|None`); placeholder comments (`// stub`, `# TODO: implement`, etc.).
|
|
72
74
|
4. **Found** → leave status untouched, record `stub-detected` (with file + indicator) in run tracking, skip execution. **Not found / unlocatable** → proceed to Step 4.
|
|
73
75
|
|
|
76
|
+
Stub detection outranks unit-backing: a stub-detected test stays `- [ ] Pass` and its unit test is not run. If you have already run one and it passed against a body carrying stub indicators, that is a contradiction, not a pass — record `[FAIL: auto-judge: unit test passes but implementation appears stubbed — <file>: <indicator>]`.
|
|
77
|
+
|
|
74
78
|
---
|
|
75
79
|
|
|
76
80
|
## Step 4 — Execute & judge
|
|
@@ -81,26 +85,57 @@ Work through eligible tests in document order; update the file immediately after
|
|
|
81
85
|
|
|
82
86
|
### Auto-judge criteria (uat-auto / uat-auto-plus)
|
|
83
87
|
|
|
84
|
-
|
|
88
|
+
Judge each test on the **strongest evidence channel available to it**:
|
|
89
|
+
|
|
90
|
+
| # | Channel | Applies when | Verdict source |
|
|
91
|
+
|---|---------|--------------|----------------|
|
|
92
|
+
| 1 | **Live command** | API/CLI test with an extractable `**Command**:` | run the command (below) |
|
|
93
|
+
| 2 | **Unit-backed** | channel 1 unavailable **and** the test is unit-backed (Step 3) | run that unit test file (below) |
|
|
94
|
+
| 3 | **Human** | neither — no machine-checkable oracle | fail-closed `[FAIL: auto-judge: ...]` |
|
|
95
|
+
|
|
96
|
+
**Channel 1 outranks channel 2 and is never overridden by it.** A live command that fails is `[FAIL]` even when the test is unit-backed and its unit test passes — a green unit test beside a red live call means the live path is broken, and the narrower oracle does not get a vote. Channel 2 is reached only when channel 1 returns no verdict at all: a Manual/Edge case, or an API/CLI case whose body carries no extractable command or no machine-checkable Expected.
|
|
97
|
+
|
|
98
|
+
**API/CLI** — extract the command from `**Command**:`. No extractable command → fall through to channel 2 if the test is unit-backed, else `[FAIL: auto-judge: no machine-executable command in test body]`. Pass requires **ALL**:
|
|
85
99
|
1. Command exited cleanly (response returned; no connection error).
|
|
86
100
|
2. HTTP status matches the Expected section's explicit status; if none specified, any 2xx is pass-eligible on status alone.
|
|
87
101
|
3. Response body satisfies **every** machine-checkable assertion in Expected (literal substring, JSON key presence, JSON value equality, array length, type-of) — via `jq` or substring match.
|
|
88
102
|
4. If the test references `$UAT_AUTH_TOKEN`, the token must be present in the environment.
|
|
89
103
|
|
|
90
|
-
Any criterion fails → `[FAIL: auto-judge: <criterion, actual vs expected>]
|
|
104
|
+
Any criterion fails → `[FAIL: auto-judge: <criterion, actual vs expected>]` — a run that produced a verdict is final, and no unit test overrides it. Expected section with no machine-checkable assertion → channel 1 yielded no verdict at all, so fall through to channel 2 if the test is unit-backed, else `[FAIL: auto-judge: expected section not machine-verifiable]`.
|
|
91
105
|
|
|
92
|
-
**
|
|
106
|
+
**Unit-backed (channel 2)** — a recorded, passing unit test *is* hard evidence, so a unit-backed test **does not fail-closed to human verification**. Resolve a run command, run it in one Bash call, and judge the result.
|
|
93
107
|
|
|
94
|
-
|
|
108
|
+
Resolve the command:
|
|
109
|
+
- Use the test's `- **Unit Test Command**: \`<cmd>\`` verbatim when present.
|
|
110
|
+
- Otherwise derive a **file-scoped** invocation for the project's runner — e.g. `node --test test/foo.test.js` · `npx vitest run test/foo.test.ts` · `npx jest test/foo.test.js` · `pytest tests/test_foo.py` · `go test ./pkg/foo/` · `cargo test --test foo`. Match the repo's existing convention (`package.json` `scripts.test`, CI config, nearby tests).
|
|
111
|
+
- A whole-suite run (`npm test`) is acceptable **only** when its output names this file's own cases and shows them passing — otherwise a green suite says nothing about this file.
|
|
112
|
+
- Cannot resolve either form → `[FAIL: auto-judge: no runnable unit-test command for <path>]`.
|
|
95
113
|
|
|
96
|
-
|
|
114
|
+
Pass requires **ALL**:
|
|
115
|
+
1. The recorded path resolves to a file that exists (`mcp__serena__find_file`). Missing → `[FAIL: auto-judge: unit test file not found — <path>]`; stale metadata pointing at a moved or deleted test is not evidence.
|
|
116
|
+
2. The runner exited 0.
|
|
117
|
+
3. The run **executed ≥1 test from that file**, and the output states the count. `0 tests` / `no tests ran` / `0 matched` / an empty pass line → `[FAIL: auto-judge: unit-test run matched 0 tests in <path>]`. **This is the load-bearing check** — a filter typo, a renamed file, or a wrong path produces exit 0 with nothing executed, which is indistinguishable from a real pass on exit code alone.
|
|
118
|
+
4. Zero failing, errored, or timed-out tests in the run.
|
|
119
|
+
5. The file's cases covering this UAT case actually ran — all `skip`/`todo`/`xfail`/`.only`-excluded → `[FAIL: auto-judge: unit test coverage skipped, not executed]`.
|
|
120
|
+
6. The unit test file is unmodified by this run (auto-plus: its fix loop may never touch it — see that skill's Autonomy boundaries).
|
|
121
|
+
7. The Expected Result contains no observable the unit test demonstrably cannot assert (live HTTP round-trip, rendering, deploy/filesystem state). Partial coverage → `[FAIL: auto-judge: unit test covers only part of Expected — <what is uncovered>]`. Apply this narrowly: a unit test that asserts the case's stated outcome by construction is complete evidence, not partial.
|
|
97
122
|
|
|
98
|
-
|
|
123
|
+
On pass: `- [x] Pass <!-- YYYY-MM-DD -->`; record `via unit test <path>` in run tracking so the summary can count it.
|
|
124
|
+
|
|
125
|
+
**UI** → always `[FAIL: auto-judge: UI test requires human verification — use /uat-walk]`, **even when unit-backed**. No navigation, screenshot, or browser interaction. A unit test cannot cover what a UI case exists to check (rendering, layout, flow timing) — `/uat-generate` Step 2.5 classifies exactly those as non-promotable, so a `Created:` on a UI case covers a fragment at best.
|
|
126
|
+
|
|
127
|
+
**Manual** → unit-backed goes to channel 2. Otherwise always `[FAIL: auto-judge: manual test requires human verification]`, with no heuristic evaluation — intentional fail-closed: `/uat-generate` produced a bare manual test because it expected a human.
|
|
128
|
+
|
|
129
|
+
`/uat-walk` does not auto-judge — the user issues every verdict. `/uat-auto-plus` enters its fix loop after a channel 1 or channel 2 `[FAIL]` (both have a machine-checkable signal that a fix worked); never after a channel 3 `[FAIL]`.
|
|
130
|
+
|
|
131
|
+
### Bash hygiene — API/CLI and unit-backed tests (all modes)
|
|
99
132
|
|
|
100
133
|
**One program invocation per Bash call: one `curl` (as-is), optionally one `\| jq` stage. Nothing else.** If a generated `**Command**:` block violates this, rewrite it to the clean form before running (canonical style: `lib/skills/uat-generate/SKILL.md` "Curl command standards"). Never emit literal password/token values — only `"$UAT_AUTH_TOKEN"` / `"$UAT_TEST_PASSWORD"` references.
|
|
101
134
|
|
|
102
135
|
Forbidden (each triggers an approval prompt and obscures output): multiple `curl`s in one call · `&&` / `;` / `\|\|` chaining · `echo` banners · `-w` format strings · `-o /tmp/...` then re-read · defensive flags (`--max-time`) · pre-assigned `TOKEN=...` vars · `\` line-continuations · piping to `head`/`tee`/redirection. Capture per test: HTTP status, response body (truncate display to ~50 lines; never write a temp file to read it back), connection errors.
|
|
103
136
|
|
|
137
|
+
Unit-backed runs follow the same rule: **one test-runner invocation per Bash call**, unchained and unredirected, so the runner's own pass/fail/count lines reach you intact. Capture per test: exit code, tests-run count, failing test names.
|
|
138
|
+
|
|
104
139
|
---
|
|
105
140
|
|
|
106
141
|
## Step 5 — Update the file per verdict
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: wiki-query
|
|
3
|
-
description: Answer a question from the wiki — locate pages via the index, synthesize a cited answer, and offer to file valuable answers back as new wiki pages
|
|
3
|
+
description: Answer a question from the wiki — locate pages via the index, synthesize a cited answer, auto-ingest relevant un-ingested raw sources when the wiki has no coverage, and offer to file valuable answers back as new wiki pages
|
|
4
4
|
category: researching
|
|
5
5
|
model: claude-sonnet-5
|
|
6
6
|
argument-hint: <question>
|
|
@@ -11,7 +11,7 @@ user-invocable: true
|
|
|
11
11
|
|
|
12
12
|
# Wiki Query
|
|
13
13
|
|
|
14
|
-
Answer a question using the wiki as the primary source of truth. If the wiki lacks coverage,
|
|
14
|
+
Answer a question using the wiki as the primary source of truth. If the wiki lacks coverage, fall back to ingesting relevant un-ingested `raw/` sources (Step 4), then answer from the enlarged wiki. **Never answer from general knowledge alone when a wiki page exists — cite it.**
|
|
15
15
|
|
|
16
16
|
---
|
|
17
17
|
|
|
@@ -37,13 +37,51 @@ Answer the question in 2–5 paragraphs. For each claim, cite the wiki page: `([
|
|
|
37
37
|
|
|
38
38
|
If you found contradictions between pages, surface them explicitly with a `> **Contradiction:**` callout and note which source is more recent.
|
|
39
39
|
|
|
40
|
-
If the wiki has no coverage at all
|
|
40
|
+
If the wiki has no coverage at all — no page matched, or the pages that matched do not actually contain what was asked — **do not stop here**. Go to Step 4.
|
|
41
|
+
|
|
42
|
+
## Step 4: Fallback — Ingest Relevant Raw Sources
|
|
43
|
+
|
|
44
|
+
Run this step **only** when Step 3 produced no answer. Partial-but-sufficient coverage is not a trigger; neither is an answer you merely wish were more detailed.
|
|
45
|
+
|
|
46
|
+
### 4.1 Find un-ingested sources
|
|
47
|
+
|
|
48
|
+
1. `mcp__serena__list_dir` on `raw/` (recursive) for the full candidate list; skip `.gitkeep`.
|
|
49
|
+
2. `mcp__serena__list_dir` on `wiki/knowledge/sources/` and read the `sources:` frontmatter of each summary page.
|
|
50
|
+
3. A raw file is **un-ingested** when no summary page's `sources:` list resolves to it. The `sources:` back-link is authoritative — a matching filename slug is a hint, not proof.
|
|
51
|
+
4. Treat each `raw/research/<slug>/` and `raw/companies/<slug>/` directory as a **single candidate** represented by its main report file, not as one candidate per supporting file.
|
|
52
|
+
|
|
53
|
+
### 4.2 Rank by relevance — cheaply
|
|
54
|
+
|
|
55
|
+
Do not read candidate files in full at this stage. Triage with:
|
|
56
|
+
|
|
57
|
+
- filename and directory slugs against the question's keywords;
|
|
58
|
+
- `mcp__serena__search_for_pattern` for question keywords restricted to `raw/`;
|
|
59
|
+
- frontmatter and headings only, for the surviving few.
|
|
60
|
+
|
|
61
|
+
Keep only sources that plausibly answer *this* question. If none survive, say so and stop:
|
|
41
62
|
```
|
|
42
|
-
This question isn't covered in the wiki
|
|
43
|
-
To add coverage: /wiki-ingest raw/<
|
|
63
|
+
This question isn't covered in the wiki, and no un-ingested raw source looks relevant.
|
|
64
|
+
To add coverage: add a source to raw/ then run /wiki-ingest raw/<file>
|
|
44
65
|
```
|
|
45
66
|
|
|
46
|
-
|
|
67
|
+
### 4.3 Ingest the relevant sources
|
|
68
|
+
|
|
69
|
+
Announce before writing anything:
|
|
70
|
+
```
|
|
71
|
+
No wiki coverage for this question. Un-ingested raw sources that look relevant:
|
|
72
|
+
1. raw/<file> — <one line on why>
|
|
73
|
+
Ingesting these now, then re-answering.
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
For each selected source, run the `/wiki-ingest` procedure (`lib/skills/wiki-ingest/SKILL.md`) in full against that path. One deviation only: its **Step 2 checkpoint becomes a non-blocking announcement** — print the takeaways and continue instead of waiting for the user. Every other step (summary page, entity pages, concept pages, index, log, hot cache) runs exactly as written there.
|
|
77
|
+
|
|
78
|
+
**Cap: at most 3 sources per invocation.** If more look relevant, ingest the 3 strongest and list the remainder for the user to run manually.
|
|
79
|
+
|
|
80
|
+
### 4.4 Re-answer once
|
|
81
|
+
|
|
82
|
+
Redo Steps 1–3 against the now-updated wiki and cite the newly created pages like any other. **Re-enter Step 4 at most once per invocation** — if the answer is still not there, report the gap plainly and list what was ingested. Never loop.
|
|
83
|
+
|
|
84
|
+
## Step 5: Offer to File the Answer Back
|
|
47
85
|
|
|
48
86
|
If the synthesized answer represents **durable, timeless knowledge** (a pattern, a design decision rationale, an entity summary) that doesn't currently exist as its own wiki page, offer:
|
|
49
87
|
|
|
@@ -59,13 +97,15 @@ If the user says yes:
|
|
|
59
97
|
2. Add a typed link `derived_from::[[source-slug]]` if the answer came from an ingested source.
|
|
60
98
|
3. Add the new page to the **Concepts** section of `wiki/index.md`.
|
|
61
99
|
4. Append to `wiki/log.md`: `## [YYYY-MM-DD] query | <question summary>\nFiled answer as wiki/knowledge/concepts/<slug>.md.`
|
|
62
|
-
5. **Only on this filing path** (never on the read-only answer path): run the **Hot Cache Refresh** procedure — defined in `/wiki-ingest` Step 8 (`lib/skills/wiki-ingest/SKILL.md`) — to regenerate `wiki/hot.md` in full, surfacing the newly filed page under Recent Changes. Do not restate the procedure here; follow it as written.
|
|
100
|
+
5. **Only on this filing path** (never on the read-only answer path): run the **Hot Cache Refresh** procedure — defined in `/wiki-ingest` Step 8 (`lib/skills/wiki-ingest/SKILL.md`) — to regenerate `wiki/hot.md` in full, surfacing the newly filed page under Recent Changes. Do not restate the procedure here; follow it as written. Step 4's ingests refresh the hot cache on their own as part of the `/wiki-ingest` procedure — that is a write path, not the read-only answer path.
|
|
63
101
|
|
|
64
102
|
---
|
|
65
103
|
|
|
66
104
|
## CRITICAL RULES
|
|
67
105
|
|
|
68
|
-
1. **Answer from the wiki, not general knowledge** — if the wiki lacks coverage, say so.
|
|
106
|
+
1. **Answer from the wiki, not general knowledge** — if the wiki still lacks coverage after Step 4, say so.
|
|
69
107
|
2. **The two-domain rule** — filed answers (durable synthesis) go to `wiki/knowledge/`; never file under `wiki/work/`.
|
|
70
108
|
3. All citations use relative markdown links to the wiki page, not raw file paths.
|
|
71
109
|
4. Do not modify any `wiki/work/` files — this skill is read-only for work artifacts.
|
|
110
|
+
5. `raw/` is **immutable** — Step 4 reads and ingests raw sources; it never creates, edits, moves, or deletes anything under `raw/`.
|
|
111
|
+
6. **Step 4 runs at most once per invocation and ingests at most 3 sources.** A query must never trigger an ingest cascade or a re-query loop.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@codewizard-dt/bootstrap",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.18.0",
|
|
4
4
|
"description": "Bootstrap Claude Code projects with an LLM Wiki, reusable skills, hooks, and MCP setup",
|
|
5
5
|
"homepage": "https://github.com/codewizard-dt/bootstrap-wiki#readme",
|
|
6
6
|
"bugs": {
|