wicked-garden 12.36.0 → 12.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +8 -0
- package/package.json +1 -1
- package/skills/domain-coverage/SKILL.md +5 -0
- package/skills/governed-worker/SKILL.md +12 -5
- package/skills/qe/refs/accept.md +18 -12
- package/skills/qe/refs/review.md +16 -10
- package/skills/qe-a11y-test-engineer/SKILL.md +17 -9
- package/skills/qe-ai-feature-test-engineer/SKILL.md +12 -2
- package/skills/qe-security-test-engineer/SKILL.md +13 -3
- package/skills/qe-snapshot-hygiene-auditor/SKILL.md +12 -4
- package/skills/qe-test-code-quality-auditor/SKILL.md +15 -5
- package/skills/wickedizer/refs/patterns.md +1 -1
|
@@ -1 +1 @@
|
|
|
1
|
-
{"name": "wicked-garden", "owner": {"name": "Mike Parcewski"}, "plugins": [{"name": "wicked-garden", "source": "./", "description": "Your coding agent already plans and swarms \u2014 Wicked Garden is the curated toolkit for what it can't: proof-not-claims gates, code links grep can't see, deterministic refactor, cross-session memory, real multi-model second opinions. Don't fight the harness; fill its gaps. The gate needs wicked-vault (a self-contained infra peer, installed directly) and re-derives through garden's built-in engine; estate/bus are opt-in layers (wicked-testing and wicked-brain are retired \u2014 QE ships in-catalog as the qe domain, memory lives in wicked-estate).", "version": "12.
|
|
1
|
+
{"name": "wicked-garden", "owner": {"name": "Mike Parcewski"}, "plugins": [{"name": "wicked-garden", "source": "./", "description": "Your coding agent already plans and swarms \u2014 Wicked Garden is the curated toolkit for what it can't: proof-not-claims gates, code links grep can't see, deterministic refactor, cross-session memory, real multi-model second opinions. Don't fight the harness; fill its gaps. The gate needs wicked-vault (a self-contained infra peer, installed directly) and re-derives through garden's built-in engine; estate/bus are opt-in layers (wicked-testing and wicked-brain are retired \u2014 QE ships in-catalog as the qe domain, memory lives in wicked-estate).", "version": "12.37.0"}], "version": "12.37.0"}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "wicked-garden",
|
|
3
|
-
"version": "12.
|
|
3
|
+
"version": "12.37.0",
|
|
4
4
|
"description": "Your coding agent's harness already plans and swarms; Wicked Garden is the curated toolkit for what it can't do alone. Proof, not claims \u2014 gates re-derive 'done' through the internal loom engine (fail-closed, independently attested via wicked-vault), so a self-asserted pass can't lie its way green. Relationships grep can't see \u2014 the wicked-estate code graph + injected bus/dispatch/capability/archetype edges power blast-radius & lineage. Plus deterministic multi-file refactor (wicked-patch), cross-session memory + knowledge (the mem domain over wicked-estate), real multi-model second opinions (jam:council), portable expertise as on-demand skill-refs, and evidence-gated testing (the in-catalog qe domain). It reads the shape of your work to apply the right rigor \u2014 steering, not a pipeline \u2014 then gets out of the way. The compiler stamps a self-contained evidence gate into any repo that runs with no wicked-garden installed. Don't fight the harness; fill its gaps. The evidence gate requires wicked-vault, a self-contained infra peer wicked-garden installs directly; wicked-bus is an opt-in layer \u2014 install what you'll use, the toolkit works without the rest.",
|
|
5
5
|
"wicked_vault_version": "^0.5.0",
|
|
6
6
|
"wicked_ledger_version": "^0.4.0",
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [12.37.0] — 2026-09-14
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
- **Evaluator verdicts: one grammar — `VERDICT: PASS` or `VERDICT: FAIL`, one plain-text line, the LAST line of the message (FIX-IT-ALL L6-0; the garden half of D-9, mirrored by wicked-crew's qe review text and wicked-core's evaluator-verdict carrier).** The review surfaces said APPROVE / CONDITIONAL / REJECT in one place (`wicked-garden-governed-worker`), `VERDICT={PASS|CONDITIONAL|FAIL} MODE=produced-test` in another (the `qe` router's `review` action), `{PASS|CONDITIONAL|FAIL|SKIP}` in a third (`qe-security-test-engineer`), TWO verdict lines in one message (`qe-test-code-quality-auditor` §8 + §9) and no verdict line at all in an evaluator skill (`domain-coverage`) — a gate cannot read five spellings. wicked-core's evaluator gate (wave 3, core-ts 0.7.27) reads the LAST `^VERDICT[:=]` line of an evaluator unit's output and passes ONLY on the token `PASS`; any other token — or no line — parks the run at the human gate, never a false pass. Every garden text surface now asks for exactly that: `governed-worker` (evaluator method; output contract item 6 "the contract line — the LAST line of your message"; a self-reviewing seat writes no line and the gate treats that as FAIL), `qe/refs/review.md` (vocabulary + the produced-test dispatch — `MODE: produced-test` on its own line), `qe-test-code-quality-auditor` (the §8 audit line becomes `AUDIT: {p0}/{p1}/{p2} REVIEWER: … RUN_ID: …` with no VERDICT token — a smell audit certifies nothing; §9 carries the ONE contract line with `MODE:` / `REVIEWER:` / `RUN_ID:` on their own lines), `qe-security-test-engineer` (`SKIP` → `VERDICT: FAIL` + `reason: not applicable: <why>` — a review with no security surface parks at the gate instead of passing silently), `qe-a11y-test-engineer` / `qe-ai-feature-test-engineer` / `qe-snapshot-hygiene-auditor` (the footer is the grammar with `REVIEWER:` / `RUN_ID:` on their own lines; CONDITIONAL stays a ledger RECORD value, the output line is FAIL with the conditions listed above it), `domain-coverage` (gains the line: PASS iff `coverage == 1.0`, zero unaccounted and no `BLOCK`), `qe/refs/accept.md` (the acceptance reviewer writes PASS|FAIL; `VERDICT_TO_STATUS` is unchanged — PARTIAL / CONDITIONAL / INCONCLUSIVE are legacy record values readers keep mapping) and `wickedizer/refs/patterns.md` (the comparison template's `VERDICT:` heading is prose again). **Behaviour change (D-9, text half):** CONDITIONAL and SKIP are no longer verdicts — unmet conditions and not-applicable reviews FAIL into the human gate; the wicked-ledger `Verdict` enum and the DomainStore `verdict:` records are untouched (record vocabulary). **Lint:** `tests/test_skill_portability.py` gains the garden-only token `verdict-spelling` — every line-leading `VERDICT[:=]` line under `skills/**/*.md` (fenced blocks included) must be `VERDICT: PASS`, `VERDICT: FAIL` or the template `VERDICT: PASS|FAIL`; nothing that is not line-leading is judged. The legacy `VERDICT=… REVIEWER=… RUN_ID=…` footers of seven `qe-*` specialists already parse as PASS/FAIL and are tolerated ONLY in the files listed under `verdict_spelling.legacy_footer_files` in `tests/portability_rules.garden.json`; a listed file with no footer left fails the build, so the qe batches that rewrite them shrink the list in the same change. **Mixed generations:** a skills generation published from garden ≤ 12.36.0 still writes the old words and gates every evaluator unit under core-ts 0.7.27 — publish garden ≥ 12.37.0 first.
|
|
9
|
+
|
|
5
10
|
## [12.36.0] — 2026-09-13
|
|
6
11
|
|
|
7
12
|
### Fixed
|
|
@@ -852,3 +857,6 @@ through `v9.2.18`. Major themes across that span:
|
|
|
852
857
|
condition-resolution mechanism, scope-delta HITL trigger.
|
|
853
858
|
|
|
854
859
|
For each of those releases, see the corresponding tag in git history.
|
|
860
|
+
|
|
861
|
+
[Unreleased]: https://github.com/mikeparcewski/wicked-garden/compare/v12.37.0...HEAD
|
|
862
|
+
[12.37.0]: https://github.com/mikeparcewski/wicked-garden/compare/v12.36.0...v12.37.0
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "wicked-garden",
|
|
3
|
-
"version": "12.
|
|
3
|
+
"version": "12.37.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Curated toolkit for what AI coding agents can't do alone: evidence-gated work, codegraph relationships, deterministic multi-file refactor, multi-model second opinions, and cross-session memory. Installs as a Claude Code plugin.",
|
|
6
6
|
"keywords": [
|
|
@@ -77,3 +77,8 @@ Then one threat per line: `[RISK|BLOCK|NOTE] <location> — <finding>`.
|
|
|
77
77
|
|
|
78
78
|
A `BLOCK` means the domain model must not proceed to build until resolved.
|
|
79
79
|
`coverage == 1.0` with zero unaccounted is the gate bar; any hole is a `BLOCK`.
|
|
80
|
+
|
|
81
|
+
Then the verdict — the LAST line of your output, one plain-text line, never
|
|
82
|
+
quoting another VERDICT line: `VERDICT: PASS` iff `coverage == 1.0`, zero
|
|
83
|
+
unaccounted and no `BLOCK`; otherwise `VERDICT: FAIL` (the threats above it
|
|
84
|
+
are the conditions).
|
|
@@ -93,9 +93,13 @@ Method: read the outcome and design, then the changed files; for each, ask
|
|
|
93
93
|
whether it follows the design, whether tests cover the key paths, and whether
|
|
94
94
|
the evidence (commands + exit codes) supports the claims; re-run what you can
|
|
95
95
|
without mutating; sort findings into Critical (must fix), Concern, Suggestion;
|
|
96
|
-
verdict
|
|
97
|
-
|
|
98
|
-
|
|
96
|
+
verdict: make the LAST line of your output exactly `VERDICT: PASS` or
|
|
97
|
+
`VERDICT: FAIL` — one plain-text line (no `**`, backticks or `#` around it),
|
|
98
|
+
findings above it. FAIL needs at least one Critical or a CONDITIONS list;
|
|
99
|
+
there is no conditional pass (CONDITIONAL, APPROVE, REJECT and SKIP are not
|
|
100
|
+
verdicts); never quote another VERDICT line in your message; no verdict line
|
|
101
|
+
= FAIL (human gate). If you are the seat that created the work, say so and
|
|
102
|
+
refuse the verdict — write no VERDICT line; the gate treats that as FAIL.
|
|
99
103
|
|
|
100
104
|
## Neutral (triage / recon / plan)
|
|
101
105
|
|
|
@@ -117,10 +121,13 @@ Every unit's final message contains, in this order:
|
|
|
117
121
|
1. **What you did** — role, scope, files touched (creator) or read (others).
|
|
118
122
|
2. **Commands run** — each with its exit code, in the order run.
|
|
119
123
|
3. **Counts** — derived N / submitted M / failed K (A4), with K explained.
|
|
120
|
-
4. **
|
|
121
|
-
|
|
124
|
+
4. **Findings or plan** — evaluator: findings by severity with `file:line`;
|
|
125
|
+
neutral: the plan; creator: what changed against the design.
|
|
122
126
|
5. **Open questions as statements** — "X is unknown; I assumed Y" — never a
|
|
123
127
|
request that waits for an answer (A3).
|
|
128
|
+
6. **The contract line (evaluators only)** — exactly `VERDICT: PASS` or
|
|
129
|
+
`VERDICT: FAIL`, plain text, as the LAST line of your message; nothing
|
|
130
|
+
after it, and no other line of yours quotes a VERDICT line.
|
|
124
131
|
|
|
125
132
|
## Provenance
|
|
126
133
|
|
package/skills/qe/refs/accept.md
CHANGED
|
@@ -301,14 +301,17 @@ Skill(
|
|
|
301
301
|
{EVIDENCE_DIR}
|
|
302
302
|
(May contain an optional context.md with pre-vetted cold domain knowledge —
|
|
303
303
|
treat it as evidence. If it contains prior verdicts or historical outcomes,
|
|
304
|
-
flag as CONTEXT_CONTAMINATION and return
|
|
304
|
+
flag as CONTEXT_CONTAMINATION and return `VERDICT: FAIL` with
|
|
305
|
+
`reason: inconclusive — context contaminated`.)
|
|
305
306
|
|
|
306
307
|
## Instructions
|
|
307
308
|
1. Read the scenario file at the path above
|
|
308
309
|
2. Read the test plan file at the path above
|
|
309
310
|
3. Read evidence files from the evidence directory (including context.md if present)
|
|
310
311
|
4. Evaluate each assertion against evidence
|
|
311
|
-
5.
|
|
312
|
+
5. End with one plain-text line `VERDICT: PASS` or `VERDICT: FAIL` as the LAST
|
|
313
|
+
line of your reply (never quote another VERDICT line); missing evidence or
|
|
314
|
+
CONTEXT_CONTAMINATION is FAIL with `reason: inconclusive — <why>`
|
|
312
315
|
6. For any EQUIVALENT_TO_BASELINE assertion, also return the equivalence facet
|
|
313
316
|
{ baseline_ref, baseline_sha, method, diff_count, tolerance, matched } so the
|
|
314
317
|
orchestrator can persist it on the verdict (see `verdict.equivalence`).
|
|
@@ -338,15 +341,18 @@ Two writes and one manifest build, in order:
|
|
|
338
341
|
//
|
|
339
342
|
// PASS → passed
|
|
340
343
|
// FAIL → failed
|
|
341
|
-
// PARTIAL → partial (
|
|
342
|
-
// CONDITIONAL → partial (
|
|
343
|
-
// INCONCLUSIVE → inconclusive (
|
|
344
|
+
// PARTIAL → partial (legacy record value)
|
|
345
|
+
// CONDITIONAL → partial (legacy record value)
|
|
346
|
+
// INCONCLUSIVE → inconclusive (legacy record value)
|
|
344
347
|
//
|
|
345
|
-
//
|
|
346
|
-
//
|
|
347
|
-
//
|
|
348
|
-
//
|
|
349
|
-
//
|
|
348
|
+
// Since garden 12.37.0 a garden evaluator's output line is `VERDICT: PASS` or
|
|
349
|
+
// `VERDICT: FAIL` only — PARTIAL, CONDITIONAL and INCONCLUSIVE are legacy
|
|
350
|
+
// record values a garden evaluator no longer writes on that line (unmet
|
|
351
|
+
// conditions and missing evidence are FAIL with the reason listed above the
|
|
352
|
+
// verdict); readers keep mapping them because older ledgers carry them. The
|
|
353
|
+
// Tier-2 gate agents' DomainStore records may still carry CONDITIONAL as a
|
|
354
|
+
// record value — the `partial` run status, distinct from the `?? 'inconclusive'`
|
|
355
|
+
// fallback (reserved for "couldn't evaluate").
|
|
350
356
|
//
|
|
351
357
|
// `errored` and `skipped` remain reachable only via the run lifecycle
|
|
352
358
|
// (executor crash / stale-run sweep, all-steps-skipped) — never from a verdict.
|
|
@@ -383,7 +389,7 @@ const reviewerEquivalenceFacet = reviewerResponse.equivalence ?? null;
|
|
|
383
389
|
const reviewerEquivalence = reviewerEquivalenceFacet ?? null;
|
|
384
390
|
const verdictRecord = store.create('verdicts', {
|
|
385
391
|
run_id: run.id,
|
|
386
|
-
verdict: reviewerVerdict, // 'PASS' | 'FAIL'
|
|
392
|
+
verdict: reviewerVerdict, // 'PASS' | 'FAIL' from the reviewer's VERDICT line (PARTIAL / CONDITIONAL / INCONCLUSIVE: legacy record values readers still map)
|
|
387
393
|
evidence_path: EVIDENCE_DIR,
|
|
388
394
|
reviewer: 'acceptance-test-reviewer',
|
|
389
395
|
reason: reviewerSummary,
|
|
@@ -426,7 +432,7 @@ emitBusEvent('wicked.test.evidence.captured', {
|
|
|
426
432
|
```markdown
|
|
427
433
|
## Acceptance Test Results: {scenario name}
|
|
428
434
|
|
|
429
|
-
### Verdict: {PASS | FAIL
|
|
435
|
+
### Verdict: {PASS | FAIL}
|
|
430
436
|
|
|
431
437
|
### Acceptance Criteria
|
|
432
438
|
| Criterion | Verdict | Evidence |
|
package/skills/qe/refs/review.md
CHANGED
|
@@ -142,13 +142,16 @@ where the harness is runnable it re-runs the produced files itself.
|
|
|
142
142
|
merges the PR and never pushes — the run's deliver phase (or the human)
|
|
143
143
|
does; a reviewer that ships is a creator grading itself.
|
|
144
144
|
|
|
145
|
-
Verdict vocabulary (`MODE
|
|
145
|
+
Verdict vocabulary (`MODE: produced-test`): **`PASS`** only when 1–7 hold with
|
|
146
146
|
zero findings — every `covered` claim verified at its `path:line`, every
|
|
147
147
|
produced test executed and green (your re-run reproduces the record), no
|
|
148
|
-
`unverified` or `failing` row; **`
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
test,
|
|
148
|
+
`unverified` or `failing` row; **`FAIL`** otherwise — on any
|
|
149
|
+
`[unexecuted-test]`, red produced test, tautological test,
|
|
150
|
+
unsupported-`covered` claim or `[scenario-defect]` oracle, AND when only P1/P2
|
|
151
|
+
rows fail (implementation-shaped test, padded total, silent scope gap, style):
|
|
152
|
+
list those fixes as CONDITIONS above the verdict line; there is no conditional
|
|
153
|
+
pass. The verdict is one plain-text line, `VERDICT: PASS` or `VERDICT: FAIL`,
|
|
154
|
+
the LAST line of the message; never quote another VERDICT line. Cite
|
|
152
155
|
`file:line` for every finding. The auditor's §9 carries the same rule.
|
|
153
156
|
|
|
154
157
|
Dispatch fallback as in § Dispatch block above: without a Skill tool, open the
|
|
@@ -175,7 +178,7 @@ MODE: produced-test — grade per the auditor's §9 produced-test verdict
|
|
|
175
178
|
`unverified`; recount new vs pre-existing tests.
|
|
176
179
|
2. Confirm every produced file has an execution record (file · command ·
|
|
177
180
|
result). None — or a `needs-fixture` e2e never run against its fixture —
|
|
178
|
-
is VERDICT
|
|
181
|
+
is `VERDICT: FAIL` `[unexecuted-test]`.
|
|
179
182
|
3. Re-run the produced files when the harness is available; compare with
|
|
180
183
|
the record.
|
|
181
184
|
4. Mutate-or-reason about at least two tested behaviours; name the failing
|
|
@@ -188,10 +191,13 @@ MODE: produced-test — grade per the auditor's §9 produced-test verdict
|
|
|
188
191
|
a mock, a snapshot of a fixture, or one that would pass with the
|
|
189
192
|
implementation deleted.
|
|
190
193
|
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
194
|
+
Report `MODE: produced-test` on its own line and per-row findings with
|
|
195
|
+
file:line, then end with one plain-text line `VERDICT: PASS` or `VERDICT: FAIL`
|
|
196
|
+
as the LAST line of your message — PASS only with every claim verified at
|
|
197
|
+
path:line, every produced test executed and green, no `unverified`; anything
|
|
198
|
+
less is FAIL with the conditions listed above it; never quote another VERDICT
|
|
199
|
+
line. Do NOT push, open, edit or merge a PR — the run's deliver phase
|
|
200
|
+
delivers."""
|
|
195
201
|
)
|
|
196
202
|
```
|
|
197
203
|
|
|
@@ -39,8 +39,11 @@ This skill is designed to run as an isolated worker; when your harness cannot fo
|
|
|
39
39
|
You test that the UI works for people who don't use a mouse or don't see
|
|
40
40
|
the screen. Accessibility is a gate, not a review — but automation alone
|
|
41
41
|
cannot clear that gate. Axe-core catches roughly 30% of WCAG failures;
|
|
42
|
-
the rest require a human. Your
|
|
43
|
-
with an explicit list of
|
|
42
|
+
the rest require a human. Your ledger record is therefore **CONDITIONAL** by
|
|
43
|
+
default, and your output line is `VERDICT: FAIL` with an explicit list of the
|
|
44
|
+
unchecked manual items as conditions — there is no conditional pass;
|
|
45
|
+
`VERDICT: PASS` only when the manual checklist is complete or the scenario
|
|
46
|
+
waives it.
|
|
44
47
|
|
|
45
48
|
## Runtime
|
|
46
49
|
Script-backed steps use the `wicked-garden` launcher: `wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]`. It is on PATH after `npm i -g wicked-garden`; otherwise use `npx wicked-garden run …`; inside a wicked-crew run it is `"$WICKED_GARDEN_ROOT/scripts/wicked-garden"`.
|
|
@@ -225,10 +228,11 @@ refuse a verdict — a11y is a gate.
|
|
|
225
228
|
|
|
226
229
|
- **WCAG 2.1 AA is the floor, not the ceiling.** If the scenario's
|
|
227
230
|
context.md requires 2.2 AA, use the `wcag22aa` tag as well.
|
|
228
|
-
- **Zero axe violations ≠ compliant.** Always
|
|
229
|
-
`CONDITIONAL` (approve with the manual checklist as the listed fixes)
|
|
230
|
-
|
|
231
|
-
|
|
231
|
+
- **Zero axe violations ≠ compliant.** Always record the ledger verdict as
|
|
232
|
+
`CONDITIONAL` (approve with the manual checklist as the listed fixes) and
|
|
233
|
+
end your output with `VERDICT: FAIL` listing the unchecked manual items as
|
|
234
|
+
conditions, unless the scenario explicitly waives manual review — never
|
|
235
|
+
`N-A`: the a11y item always applies (see the DomainStore write above).
|
|
232
236
|
- **Manual keyboard flow check is mandatory** — record the exact keys
|
|
233
237
|
pressed and the elements focused in `a11y-manual-checklist.md`. If
|
|
234
238
|
Playwright isn't available, leave the checklist items as unchecked
|
|
@@ -258,7 +262,8 @@ every test you produce, whether dispatched or invoked directly:
|
|
|
258
262
|
## 7. Output format
|
|
259
263
|
|
|
260
264
|
Print a compact summary to stdout; the full detail lives in the evidence
|
|
261
|
-
files. The final line MUST be
|
|
265
|
+
files. The final line MUST be exactly `VERDICT: PASS` or `VERDICT: FAIL` —
|
|
266
|
+
one plain-text line, nothing after it, never quoting another VERDICT line.
|
|
262
267
|
|
|
263
268
|
```
|
|
264
269
|
## A11y: {scenario.name}
|
|
@@ -266,9 +271,12 @@ target: {TARGET_URL}
|
|
|
266
271
|
axe: {axeViolationCount} violations ({critical} critical, {serious} serious, {moderate} moderate, {minor} minor)
|
|
267
272
|
pa11y: {pa11yErrorCount} errors, {pa11yWarningCount} warnings
|
|
268
273
|
keyboard: {keyboardPassCount}/{keyboardTotalCount} steps reached target without a trap
|
|
269
|
-
|
|
274
|
+
record: CONDITIONAL (axe+pa11y clean, manual review required)
|
|
275
|
+
conditions: {unchecked manual checklist items}
|
|
270
276
|
|
|
271
|
-
|
|
277
|
+
REVIEWER: wicked-garden-qe-a11y-test-engineer
|
|
278
|
+
RUN_ID: {RUN_ID}
|
|
279
|
+
VERDICT: FAIL
|
|
272
280
|
```
|
|
273
281
|
|
|
274
282
|
## Helper resolution (`{WT_LIB}`)
|
|
@@ -295,6 +295,11 @@ Verdict decision tree:
|
|
|
295
295
|
- Otherwise → **PASS** with a manual-review task (red-team coverage is
|
|
296
296
|
never 100%)
|
|
297
297
|
|
|
298
|
+
Those are ledger record values (`verdict: decision` below). The output line
|
|
299
|
+
(§11) maps them: `PASS` → `VERDICT: PASS`; `CONDITIONAL` → `VERDICT: FAIL`
|
|
300
|
+
with the monitoring / citation fixes listed as conditions — there is no
|
|
301
|
+
conditional pass; `FAIL` → `VERDICT: FAIL`.
|
|
302
|
+
|
|
298
303
|
```js
|
|
299
304
|
store.create("verdicts", {
|
|
300
305
|
run_id: RUN_ID,
|
|
@@ -364,6 +369,9 @@ store.create("tasks", {
|
|
|
364
369
|
|
|
365
370
|
## 11. Output
|
|
366
371
|
|
|
372
|
+
The final line is exactly `VERDICT: PASS` or `VERDICT: FAIL` — one plain-text
|
|
373
|
+
line, nothing after it, never quoting another VERDICT line.
|
|
374
|
+
|
|
367
375
|
```
|
|
368
376
|
## AI: {scenario.name}
|
|
369
377
|
target: {TARGET} model_pin: {id} temp: {t}
|
|
@@ -374,9 +382,11 @@ schema: violations={N}/{total}
|
|
|
374
382
|
citations: bad={pct}%
|
|
375
383
|
successes: direct={N} indirect={N} smuggle={N} multiturn={N} jb_families={names}
|
|
376
384
|
|
|
377
|
-
|
|
385
|
+
record: {PASS|CONDITIONAL|FAIL} reason: {short}
|
|
378
386
|
|
|
379
|
-
|
|
387
|
+
REVIEWER: wicked-garden-qe-ai-feature-test-engineer
|
|
388
|
+
RUN_ID: {RUN_ID}
|
|
389
|
+
VERDICT: PASS|FAIL
|
|
380
390
|
```
|
|
381
391
|
|
|
382
392
|
## Helper resolution (`{WT_LIB}`)
|
|
@@ -378,6 +378,13 @@ Verdict decision tree:
|
|
|
378
378
|
- DAST refused for safety perimeter → **SKIP** with reason
|
|
379
379
|
`insufficient-authorization`
|
|
380
380
|
|
|
381
|
+
Those are ledger record values (`verdict: decision` below). The output line
|
|
382
|
+
(§10) maps them: `PASS` → `VERDICT: PASS`; `CONDITIONAL` → `VERDICT: FAIL`
|
|
383
|
+
with the Medium findings listed as conditions; `SKIP` → `VERDICT: FAIL` with
|
|
384
|
+
`reason: not applicable: <why>`; `FAIL` → `VERDICT: FAIL`. There is no
|
|
385
|
+
conditional pass and no skipped pass on the output line — a review with no
|
|
386
|
+
security surface parks at the human gate rather than passing silently.
|
|
387
|
+
|
|
381
388
|
```js
|
|
382
389
|
store.create("verdicts", {
|
|
383
390
|
run_id: RUN_ID,
|
|
@@ -453,7 +460,8 @@ Do NOT touch `runs` status directly — the orchestrator owns that field.
|
|
|
453
460
|
## 12. Output
|
|
454
461
|
|
|
455
462
|
Print a compact summary to stdout; full detail lives in the evidence
|
|
456
|
-
files.
|
|
463
|
+
files. The final line is exactly `VERDICT: PASS` or `VERDICT: FAIL` — one
|
|
464
|
+
plain-text line, nothing after it, never quoting another VERDICT line.
|
|
457
465
|
|
|
458
466
|
```
|
|
459
467
|
## Security: {scenario.name}
|
|
@@ -465,9 +473,11 @@ authz: idor={pass|FAIL} role-esc={pass|FAIL} jwt-none={pass|FAIL} session-fix=
|
|
|
465
473
|
severity: critical={N} high={N} medium={N} low={N} info={N}
|
|
466
474
|
asvs: exercised={N} not_exercised={N}
|
|
467
475
|
|
|
468
|
-
|
|
476
|
+
reason: {short — "not applicable: <why>" when nothing was in scope}
|
|
469
477
|
|
|
470
|
-
|
|
478
|
+
REVIEWER: wicked-garden-qe-security-test-engineer
|
|
479
|
+
RUN_ID: {RUN_ID}
|
|
480
|
+
VERDICT: PASS|FAIL
|
|
471
481
|
```
|
|
472
482
|
|
|
473
483
|
## Helper resolution (`{WT_LIB}`)
|
|
@@ -233,9 +233,12 @@ findings. Return a PASS with `reason: "no snapshots found"` and
|
|
|
233
233
|
|
|
234
234
|
## 7. Non-negotiable rules
|
|
235
235
|
|
|
236
|
-
- **Default verdict is CONDITIONAL
|
|
237
|
-
|
|
238
|
-
|
|
236
|
+
- **Default ledger verdict is CONDITIONAL; the output line is then
|
|
237
|
+
`VERDICT: FAIL` with the top-N to re-review as conditions.** This is a
|
|
238
|
+
reviewer tool, not a gate — but there is no conditional pass on the output
|
|
239
|
+
line: `VERDICT: PASS` only for a clean tree (or the `ERR_NO_SNAPSHOTS_FOUND`
|
|
240
|
+
noop). The ledger's FAIL is reserved for severe rubber-stamp counts or
|
|
241
|
+
catastrophic dead-file counts.
|
|
239
242
|
- **Rubber-stamp detection requires git history.** Shallow clones with
|
|
240
243
|
depth < `rubber_stamp_window + 5` trigger `ERR_GIT_LOG_FAILED`; do
|
|
241
244
|
not guess.
|
|
@@ -248,6 +251,9 @@ findings. Return a PASS with `reason: "no snapshots found"` and
|
|
|
248
251
|
|
|
249
252
|
## 8. Output
|
|
250
253
|
|
|
254
|
+
The final line is exactly `VERDICT: PASS` or `VERDICT: FAIL` — one plain-text
|
|
255
|
+
line, nothing after it, never quoting another VERDICT line.
|
|
256
|
+
|
|
251
257
|
```
|
|
252
258
|
## Snapshot audit: {scenario.name}
|
|
253
259
|
target_dirs: {target_dirs}
|
|
@@ -267,7 +273,9 @@ top-5 (full list in snapshot-top-n.csv):
|
|
|
267
273
|
score=3 tests/cassettes/legacy/old.yaml (dead) action=delete
|
|
268
274
|
...
|
|
269
275
|
|
|
270
|
-
|
|
276
|
+
REVIEWER: wicked-garden-qe-snapshot-hygiene-auditor
|
|
277
|
+
RUN_ID: {RUN_ID}
|
|
278
|
+
VERDICT: PASS|FAIL
|
|
271
279
|
```
|
|
272
280
|
|
|
273
281
|
## Helper resolution (`{WT_LIB}`)
|
|
@@ -307,9 +307,12 @@ top-5 (full list in test-quality-top-n.csv):
|
|
|
307
307
|
P1 tests/api/retry.test.ts:88 hardcoded-sleep "setTimeout(() => ..., 5000)"
|
|
308
308
|
...
|
|
309
309
|
|
|
310
|
-
|
|
310
|
+
AUDIT: {p0}/{p1}/{p2} REVIEWER: wicked-garden-qe-test-code-quality-auditor RUN_ID: {RUN_ID}
|
|
311
311
|
```
|
|
312
312
|
|
|
313
|
+
The audit certifies nothing, so it writes no VERDICT line — the `AUDIT:` line
|
|
314
|
+
is its report line; the ledger record (§5) carries the CONDITIONAL / FAIL.
|
|
315
|
+
|
|
313
316
|
## 9. Produced-test review (a PLAN + the tests it claims)
|
|
314
317
|
|
|
315
318
|
When the qe `review` action dispatches you in `produced-test` mode (`MODE:
|
|
@@ -364,16 +367,23 @@ PLAN; the author's "all green" is a claim.
|
|
|
364
367
|
mutated-or-reasoned with a nameable failing mutation each (duty 3); e2e
|
|
365
368
|
oracles verified (duty 5); `not covered` rows account for the intent
|
|
366
369
|
(duty 6); and the §2 detectors report P0 = P1 = P2 = 0 on the produced files.
|
|
367
|
-
- **`
|
|
368
|
-
total, silent scope gap, style smells):
|
|
370
|
+
- **`FAIL` with CONDITIONS** — no P0, but any P1/P2 (implementation-shaped
|
|
371
|
+
test, padded total, silent scope gap, style smells): list the fixes as
|
|
372
|
+
conditions above the verdict line. There is no conditional pass — the ledger
|
|
373
|
+
record (§5) may still say `CONDITIONAL`; the output line says `FAIL`.
|
|
369
374
|
- **`FAIL`** — any P0: `[unexecuted-test]`, a red produced test, an
|
|
370
375
|
unsupported `covered` claim, a tautological test, or an impossible oracle
|
|
371
376
|
`[scenario-defect]`.
|
|
372
377
|
|
|
373
|
-
Report
|
|
378
|
+
Report block for this mode — the §8 `AUDIT:` line stays for the audit; a
|
|
379
|
+
message never carries two VERDICT lines and never quotes another one. The
|
|
380
|
+
verdict is one plain-text line, the LAST line of your output:
|
|
374
381
|
|
|
375
382
|
```
|
|
376
|
-
|
|
383
|
+
MODE: produced-test
|
|
384
|
+
REVIEWER: wicked-garden-qe-test-code-quality-auditor
|
|
385
|
+
RUN_ID: {RUN_ID}
|
|
386
|
+
VERDICT: PASS|FAIL
|
|
377
387
|
```
|
|
378
388
|
|
|
379
389
|
## Helper resolution (`{WT_LIB}`)
|