@lemoncode/lemony 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -14
- package/catalog/VERSION +1 -1
- package/catalog/agents/architect.md +3 -1
- package/catalog/agents/implementer.md +43 -5
- package/catalog/agents/orchestrator.md +519 -60
- package/catalog/agents/partition.md +316 -0
- package/catalog/agents/reviewer.md +279 -67
- package/catalog/agents/spec-author.md +12 -3
- package/catalog/agents/triage.md +8 -5
- package/catalog/commands/add-capability.md +4 -4
- package/catalog/commands/define.md +7 -0
- package/catalog/commands/hotfix.md +15 -1
- package/catalog/commands/pause.md +5 -0
- package/catalog/commands/resume.md +37 -9
- package/catalog/commands/triage.md +2 -1
- package/catalog/harness.config.schema.json +40 -0
- package/catalog/hooks/lib/merge-pr.sh +699 -0
- package/catalog/skills/mutation-testing/SKILL.md +78 -21
- package/catalog/skills/prd-to-spec/SKILL.md +48 -2
- package/catalog/skills/raise-discovery/SKILL.md +6 -0
- package/catalog/skills/resolve-discovery/SKILL.md +6 -5
- package/catalog/skills/security-review/SKILL.md +119 -6
- package/catalog/skills/spec-to-issue/SKILL.md +7 -1
- package/catalog/skills/task-closeout/SKILL.md +82 -18
- package/catalog/skills/triage-issue/SKILL.md +65 -4
- package/catalog/templates/claude-code/agents.md.tpl +41 -12
- package/catalog/templates/claude-code/harness.config.yml.tpl +33 -0
- package/dist/cli.mjs +737 -36
- package/package.json +10 -6
|
@@ -1,9 +1,8 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: mutation-testing
|
|
3
|
-
description: Behavioral test-strength analysis of a change — mutate the changed source and check whether the suite actually fails. Surviving mutants mark tests that pass regardless of behavior. Advisory
|
|
3
|
+
description: Behavioral test-strength analysis of a change — mutate the changed source and check whether the suite actually fails. Surviving mutants mark tests that pass regardless of behavior. Advisory and diff-scoped; a declared `test:mutation` script selects the cheap scripted path, its absence the hand-probe path. Distinct from `test-gap-report` (structural — which files lack a spec) and `verify` (which just runs the suite).
|
|
4
4
|
origin: vendor
|
|
5
5
|
vendor_version: '{{vendor_version}}'
|
|
6
|
-
applies-when: [has-mutation-testing]
|
|
7
6
|
phase: post-implementation
|
|
8
7
|
invoked-by: [reviewer]
|
|
9
8
|
---
|
|
@@ -14,7 +13,7 @@ The **behavioral** counterpart to `test-gap-report`'s structural analysis. A str
|
|
|
14
13
|
gap report asks "which changed files lack a dedicated spec?"; mutation testing asks the
|
|
15
14
|
harder question: **do the tests that exist actually fail when the code misbehaves?** It
|
|
16
15
|
mutates the source (flips `>` to `>=`, drops a line, swaps `&&`/`||`) and re-runs the
|
|
17
|
-
|
|
16
|
+
tests. A mutant that **survives** — the tests still pass on the broken code — marks a
|
|
18
17
|
test that asserts nothing load-bearing. Line coverage can't catch this; only mutation
|
|
19
18
|
can.
|
|
20
19
|
|
|
@@ -23,17 +22,31 @@ output is noisy — equivalent mutants are real and unavoidable), and it only mu
|
|
|
23
22
|
code the change touched, because mutating the whole tree per review is prohibitively
|
|
24
23
|
slow.
|
|
25
24
|
|
|
26
|
-
##
|
|
25
|
+
## Two paths, chosen at runtime
|
|
27
26
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
script exists — run it. The harness stays tool-agnostic: the script may drive Stryker or
|
|
31
|
-
any mutation tool. Delegate to it exactly as `verify` delegates to the project's
|
|
32
|
-
declared scripts; never assume a specific tool's CLI.
|
|
27
|
+
Check whether `package.json` declares a non-empty **`test:mutation`** script. That one
|
|
28
|
+
fact picks the path:
|
|
33
29
|
|
|
34
|
-
|
|
30
|
+
- **Declared** → the **scripted path**: one batch run of the project's own script,
|
|
31
|
+
diff-scoped. The harness stays tool-agnostic — the script may drive Stryker or any
|
|
32
|
+
mutation tool. Delegate to it exactly as `verify` delegates to the project's declared
|
|
33
|
+
scripts; never assume a specific tool's CLI.
|
|
34
|
+
- **Absent** → the **hand-probe path**: you mutate the source yourself, one composite
|
|
35
|
+
trip per mutant, and revert every time.
|
|
35
36
|
|
|
36
|
-
|
|
37
|
+
The two paths do **not** apply equally, by design:
|
|
38
|
+
|
|
39
|
+
- The **scripted path runs on every review** where the script exists — it is a cheap
|
|
40
|
+
batch, and its advisory output also catches weak tests in changes that declared no
|
|
41
|
+
risk.
|
|
42
|
+
- The **hand-probe path is never a per-review mandate.** What you owe by hand is the
|
|
43
|
+
Reviewer's evidence-ledger mutant floor (accounting for every changed non-test file
|
|
44
|
+
under `declared-risk` — the Reviewer contract's Evidence-ledger section owns that
|
|
45
|
+
rule; this skill never duplicates it), plus your own judgment when a survivor-shaped
|
|
46
|
+
doubt is worth one probe. Probing beyond the floor is a judgment call, never a
|
|
47
|
+
mandate.
|
|
48
|
+
|
|
49
|
+
## Scripted path: run the project's mutation script, scoped to the diff
|
|
37
50
|
|
|
38
51
|
Run the declared script focused on the **files this change modified** — not the whole
|
|
39
52
|
tree. How to scope depends on the project's tool; read its config and adapt:
|
|
@@ -46,10 +59,47 @@ tree. How to scope depends on the project's tool; read its config and adapt:
|
|
|
46
59
|
|
|
47
60
|
Capture the surviving mutants with their file + line.
|
|
48
61
|
|
|
49
|
-
|
|
62
|
+
## Hand-probe path: mutate, test, revert — one composite trip per mutant
|
|
63
|
+
|
|
64
|
+
Without a script, each probe is yours to run. The mechanics:
|
|
65
|
+
|
|
66
|
+
- **Choose mutants that would expose a weak test.** Target the load-bearing logic the
|
|
67
|
+
change touched: flip a comparison (`>` ↔ `>=`), negate a guard, drop an early return,
|
|
68
|
+
swap `&&`/`||`, delete a write. A good mutant is one a real assertion should kill; a
|
|
69
|
+
cosmetic mutation (rename, whitespace) proves nothing.
|
|
70
|
+
- **How many is judgment, anchored by the floor.** What a review owes, and what counts
|
|
71
|
+
as accounted-for, is the ledger contract's rule, not this skill's — the floor asks
|
|
72
|
+
for accounting, never exhaustion. One or two well-aimed probes per logic-bearing
|
|
73
|
+
file usually answer the question.
|
|
74
|
+
- **The round trip is one composite command with the revert unconditional.** Apply the
|
|
75
|
+
mutation, run the focused test, and restore — `;`-separated or a scripted loop,
|
|
76
|
+
**never `&&` before the revert** (a red test must still revert), and never separate
|
|
77
|
+
edit / test / revert calls. It leaves the tree exactly as you found it:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
sed -i.bak '128s/>=/>/' src/queue.ts; \
|
|
81
|
+
npx vitest run src/queue.spec.ts; \
|
|
82
|
+
mv src/queue.ts.bak src/queue.ts
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
(Line-addressed on purpose — one targeted mutant per trip keeps the kill
|
|
86
|
+
attributable; the `.bak` the in-place edit leaves is the revert.)
|
|
87
|
+
|
|
88
|
+
A **killed** mutant (the focused test went red) is the good outcome; a **survivor**
|
|
89
|
+
(still green on broken code) is the finding. Batching buys trips, never
|
|
90
|
+
experiments — the probe set itself never shrinks to save calls.
|
|
91
|
+
|
|
92
|
+
- **Say so in your verdict — the nudge.** When this path ran, note in your verdict that
|
|
93
|
+
the project can buy the cheaper batch: declaring a `test:mutation` script in
|
|
94
|
+
`package.json` (driving Stryker or any mutation tool) activates the scripted path on
|
|
95
|
+
the next review. The human reads it at the checkpoint, exactly while the artisanal
|
|
96
|
+
cost is being paid. (Only when this path actually ran — never as a standing line.)
|
|
97
|
+
|
|
98
|
+
## Classify each surviving mutant — the two-case model (both paths)
|
|
50
99
|
|
|
51
100
|
A diff-scoped run mutates changed _files_, but a changed file still contains
|
|
52
|
-
**pre-existing, untouched lines**.
|
|
101
|
+
**pre-existing, untouched lines**. Whichever path produced the survivor, where it
|
|
102
|
+
survives decides how you report it:
|
|
53
103
|
|
|
54
104
|
| Mutant survives on… | Nature | Channel |
|
|
55
105
|
| -------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------- |
|
|
@@ -60,20 +110,26 @@ Use the diff to decide which lines the change actually touched. When unsure whet
|
|
|
60
110
|
mutated line is new or pre-existing, treat it as in-scope (verdict) — over-reporting in
|
|
61
111
|
the verdict is cheaper than mis-routing a real gap to a dismissable offer.
|
|
62
112
|
|
|
63
|
-
|
|
113
|
+
## Report
|
|
64
114
|
|
|
65
115
|
- **In-scope surviving mutants** → list them in your review verdict as an **advisory**
|
|
66
116
|
block: file:line, the mutation that survived, and the assertion that would have
|
|
67
117
|
caught it — all of them when few (over-reporting here is cheaper than mis-routing,
|
|
68
|
-
per
|
|
69
|
-
the full list to the full-pass issue comment; on a per-step
|
|
70
|
-
comment — anything worth keeping goes in the verdict
|
|
71
|
-
records those in `progress.md`); what you leave out dies
|
|
72
|
-
|
|
73
|
-
|
|
118
|
+
per the two-case model). On a noisy run, carry the highest-value ones in the
|
|
119
|
+
verdict and route the full list to the full-pass issue comment; on a per-step
|
|
120
|
+
review — which posts no comment — anything worth keeping goes in the verdict
|
|
121
|
+
bullets (the Orchestrator records those in `progress.md`); what you leave out dies
|
|
122
|
+
with your context. This is **not** a REJECT on its own (decision: advisory). The
|
|
123
|
+
Implementer may strengthen the tests; the Reviewer may still REJECT by _judgment_
|
|
124
|
+
if a survivor exposes a genuinely dangerous untested path — but the mutation result
|
|
125
|
+
alone never auto-blocks.
|
|
74
126
|
- **Pre-existing surviving mutants** → run **`note-side-finding`**: one `## Side-findings`
|
|
75
127
|
bullet each (file:line + the gap), then keep going. The Orchestrator collects it and
|
|
76
128
|
offers the human a `/spinoff`. Don't fix it, don't reject on it — it isn't this task's.
|
|
129
|
+
- **Hand probes under declared risk also land in the ledger.** When the review's record
|
|
130
|
+
carries `basis: declared-risk`, every probe you ran belongs in the sidecar's
|
|
131
|
+
`mutants.files` entries per the Evidence-ledger section of the Reviewer contract —
|
|
132
|
+
this skill points at that contract and never duplicates it.
|
|
77
133
|
|
|
78
134
|
If no mutant survives on the changed lines, say so in one line — a clean diff-scoped
|
|
79
135
|
mutation run is a meaningful positive signal about the new tests.
|
|
@@ -83,6 +139,7 @@ mutation run is a meaningful positive signal about the new tests.
|
|
|
83
139
|
- **Equivalent mutants** (a mutation that can't change observable behavior, so no test
|
|
84
140
|
could kill it) are expected. Don't chase them; flag a survivor only when a real
|
|
85
141
|
assertion would have caught it.
|
|
86
|
-
- Mutation testing is **slow** — keep it diff-scoped
|
|
142
|
+
- Mutation testing is **slow** — keep it diff-scoped, and on the hand path run the
|
|
143
|
+
focused spec, never the whole suite per mutant. If a single review's scripted run is
|
|
87
144
|
blowing the time budget, report what completed and note the un-covered files rather
|
|
88
145
|
than silently truncating.
|
|
@@ -140,6 +140,47 @@ of the spec. Grouping criterion:
|
|
|
140
140
|
the group's rationale (e.g. "5 identical mechanical renames").
|
|
141
141
|
4. **Every group header carries a one-line boundary rationale** so the grouping is
|
|
142
142
|
auditable at the spec gate.
|
|
143
|
+
5. **A group whose risk has a name carries it as a machine-readable tag.** When the
|
|
144
|
+
damage a group can do is one the vocabulary below names, the header ends with
|
|
145
|
+
`[risk: <class>]` — the **last element of the line**, after the rationale; multiple
|
|
146
|
+
classes are comma-separated (`[risk: data-loss, secrets]`).
|
|
147
|
+
The vocabulary names what kind of damage the surface can do:
|
|
148
|
+
|
|
149
|
+
<!-- risk-vocabulary:start -->
|
|
150
|
+
|
|
151
|
+
| Class | Surface |
|
|
152
|
+
| ----------------- | ---------------------------------------------------------------- |
|
|
153
|
+
| `auth` | authentication, authorization, session or token handling |
|
|
154
|
+
| `payments` | money movement, billing, pricing |
|
|
155
|
+
| `shell-process` | shell invocation, subprocess spawning, argument construction |
|
|
156
|
+
| `data-loss` | writes, deletes, migrations, anything that can destroy user data |
|
|
157
|
+
| `secrets` | credentials, keys, tokens — at rest or in transit |
|
|
158
|
+
| `executable-mode` | file permissions, exec bits, anything that changes what can run |
|
|
159
|
+
|
|
160
|
+
<!-- risk-vocabulary:end -->
|
|
161
|
+
|
|
162
|
+
The tag keys on **damage, not on which rule made the group**. A group that isolates
|
|
163
|
+
a new seam or a hard-to-reverse decision is a risk isolate under rule 1 and still
|
|
164
|
+
carries **no tag**, because no class names that damage — and absence is itself the
|
|
165
|
+
declaration ("nothing here does one of those things"), never an omission.
|
|
166
|
+
|
|
167
|
+
If a group's damage has no class in the vocabulary, **leave the tag off and name that
|
|
168
|
+
damage in the group's rationale** — rule 4 already requires a rationale, and damage
|
|
169
|
+
named in prose beats a wrong class in a tag. Then say so in your handoff summary, in
|
|
170
|
+
one line, so the human can decide whether the vocabulary should grow. Do **not** pick
|
|
171
|
+
the nearest class (the self-check below rejects it), do not invent a one-off class,
|
|
172
|
+
and do not edit the table here: it is vendor-owned — generated into this skill and
|
|
173
|
+
`triage-issue` from one source, so the two always agree — and a local edit is
|
|
174
|
+
discarded on the next harness update.
|
|
175
|
+
|
|
176
|
+
The tag is what the **human approves at the spec gate**: approving the spec approves
|
|
177
|
+
the grouping _and its tags_, so a wrong tag is a spec defect, not a review defect.
|
|
178
|
+
That approval is where its force starts, and a reader now depends on it: at review the
|
|
179
|
+
tags are handed to `security-review`, which deepens the section named for each class
|
|
180
|
+
instead of inferring the surface from the diff. And a script reads them: the review
|
|
181
|
+
evidence ledger's validator (`lemony review-ledger validate`) checks every tag on the
|
|
182
|
+
group under review against the vocabulary and reports one outside it as
|
|
183
|
+
`unknown-risk-class` — a spec defect that reaches the human, never a silent drop.
|
|
143
184
|
|
|
144
185
|
```markdown
|
|
145
186
|
# Tasks — <topic>
|
|
@@ -149,7 +190,7 @@ of the spec. Grouping criterion:
|
|
|
149
190
|
- [ ] T1 — <smallest behavior that proves the path end-to-end> (R1)
|
|
150
191
|
- [ ] T2 — <next behavior> (R2)
|
|
151
192
|
|
|
152
|
-
## Group 2 — <name> _(<one-line boundary rationale>)_
|
|
193
|
+
## Group 2 — <name> _(<one-line boundary rationale>)_ [risk: data-loss]
|
|
153
194
|
|
|
154
195
|
- [ ] T3 — <error path> (R2, R3)
|
|
155
196
|
```
|
|
@@ -159,7 +200,10 @@ Rules: order so the first task is a tracer bullet; never "write all tests" then
|
|
|
159
200
|
changes task granularity — checkboxes stay atomic and TDD runs per task; only review
|
|
160
201
|
and checkpoint frequency follow the groups (all-at-once mode ignores the headers).
|
|
161
202
|
Tasks added mid-implementation (from a discovery) default to **their own group** —
|
|
162
|
-
they are risk by definition
|
|
203
|
+
they are risk by definition, and grouping rule 5 applies to them unchanged — but they
|
|
204
|
+
are added **after** the spec gate, and no later gate presents a group header, so their
|
|
205
|
+
tags get no human reading at all. Tag them for the record; do not rely on anyone
|
|
206
|
+
catching a wrong one.
|
|
163
207
|
|
|
164
208
|
### 5. Self-check before handing off
|
|
165
209
|
|
|
@@ -169,6 +213,8 @@ they are risk by definition.
|
|
|
169
213
|
- [ ] Every requirement is covered by at least one task.
|
|
170
214
|
- [ ] Every task sits in a group, and every group header carries a one-line
|
|
171
215
|
boundary rationale.
|
|
216
|
+
- [ ] Every group that can do damage the vocabulary names ends its header with a
|
|
217
|
+
`[risk: …]` tag; no group carries a class whose damage it cannot do.
|
|
172
218
|
- [ ] No closed PRD decision is contradicted.
|
|
173
219
|
|
|
174
220
|
### 6. Emit `spec_created`
|
|
@@ -32,6 +32,12 @@ human, and you do not resume yourself — the Orchestrator owns all of that (it
|
|
|
32
32
|
| **T5** | INFEASIBILITY | The plan is not implementable as written. |
|
|
33
33
|
| **T6** | PLAYBOOK_CONFLICT | Honoring the spec would contradict a client playbook. |
|
|
34
34
|
|
|
35
|
+
**Oversize / partition is a T2.** When the scope you are structuring or implementing
|
|
36
|
+
hides **≥2 independently mergeable units** the plan didn't reveal — each could leave the
|
|
37
|
+
default branch green and shippable without the other — the plan is silent on the cut
|
|
38
|
+
and both "partition" and "keep together" are valid: raise it as `T2 UNSPECIFIED_DECISION`
|
|
39
|
+
with the candidate cut in **Proposed**. Never partition silently from inside a task.
|
|
40
|
+
|
|
35
41
|
## When NOT to raise (resolve it yourself)
|
|
36
42
|
|
|
37
43
|
Escalation is for decisions with architectural or scope impact, not for everything
|
|
@@ -68,11 +68,12 @@ decision **stated in full** — at this moment it exists nowhere on disk; the en
|
|
|
68
68
|
`**Resolution**` block is only written at step 4 — plus the `discoveries.md` entry
|
|
69
69
|
**by path** for the surrounding context (it reads the entry itself):
|
|
70
70
|
|
|
71
|
-
| Artifact changed by the decision
|
|
72
|
-
|
|
|
73
|
-
| `spec/requirements.md`, `spec/design.md`, `spec/tasks.md`
|
|
74
|
-
| Implementation code, `progress.md`, `notes.md`
|
|
75
|
-
|
|
|
71
|
+
| Artifact changed by the decision | Owner to invoke |
|
|
72
|
+
| --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
73
|
+
| `spec/requirements.md`, `spec/design.md`, `spec/tasks.md` | **Spec Author** |
|
|
74
|
+
| Implementation code, `progress.md`, `notes.md` | **Implementer** (often just the resumed sub-agent) |
|
|
75
|
+
| A child issue's trace lines, a triage issue's fix plan, the parent partition-plan issue | **Orchestrator** (you) — `gh issue edit --body-file`, read-modify-write (an oversize discovery answered "partition": `.claude/agents/partition.md`) |
|
|
76
|
+
| `docs/adr/NNNN-<slug>.md`, `docs/architecture.md`, `docs/playbooks/` | **Architect** (on-demand) — `write-adr` to record the decision, `update-architecture` to keep the map true, `playbook-iterate` for a `T6 PLAYBOOK_CONFLICT` |
|
|
76
77
|
|
|
77
78
|
Not every resolution needs an artifact update first. If the decision is simply "do X"
|
|
78
79
|
with no change to the contract, skip straight to recording it and resuming. If it
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: security-review
|
|
3
|
-
description: Security review of a change — OWASP-style vectors plus AI/LLM-specific risks. Checks input validation, authorization/IDOR, injection, secrets & logging, transport/headers, dependencies,
|
|
3
|
+
description: Security review of a change — OWASP-style vectors plus AI/LLM-specific risks. Checks input validation, authorization/IDOR, injection, secrets & logging, transport/headers, dependencies, prompt-injection where an LLM is in the loop, shell and subprocess use, money movement, destructive writes, and executable surface. Every risk class a change can be declared under names one section, and a declaration handed over with the invocation deepens that section. Use after `senior-review` on any change touching user input, auth, data access, money, destructive writes, a subprocess, file permissions, or an LLM.
|
|
4
4
|
origin: vendor
|
|
5
5
|
vendor_version: '{{vendor_version}}'
|
|
6
6
|
phase: post-implementation
|
|
@@ -15,6 +15,38 @@ endpoint or an LLM call. Consult the project's **security playbook** for project
|
|
|
15
15
|
specific rules (deployment model, auth scheme, secret tooling); the checklist below is
|
|
16
16
|
the generic floor. Flag only findings you're confident are real, ranked by severity.
|
|
17
17
|
|
|
18
|
+
## Declared risk surfaces
|
|
19
|
+
|
|
20
|
+
Your invocation may carry a **declaration**: the kind of damage this change can do, named
|
|
21
|
+
at the structuring moment — before a line of it was written — and approved by a human, as a
|
|
22
|
+
class paired with the group or the file it was declared against. When one reaches you, it
|
|
23
|
+
is a **floor for depth, never a bound on scope**:
|
|
24
|
+
|
|
25
|
+
- Every section below still runs. The declaration says where to press harder, not what to
|
|
26
|
+
skip: it was written before the code existed, so it cannot know what the diff ended up
|
|
27
|
+
touching. Deriving the surface from the change itself remains your job.
|
|
28
|
+
- For each declared class, go to the section whose heading carries it. Every class in the
|
|
29
|
+
vocabulary names **exactly one** section below — the backticked class at the end of the
|
|
30
|
+
heading (`` ## 2. Authorization … — `auth` ``) — and a guard pins that the set of classes
|
|
31
|
+
in these headings and the vocabulary are the same set, so no heading ends in
|
|
32
|
+
`` — `token` `` unless `token` is a class. Go deep there: read the declared file whole
|
|
33
|
+
rather than sampling its diff, and follow the vector out of the change into the code it
|
|
34
|
+
reaches.
|
|
35
|
+
- A declared class that matches **no heading** is drift between the vocabulary and this
|
|
36
|
+
checklist — a guard should have caught it — not a class you may skip. Review it against
|
|
37
|
+
the project's security playbook and the generic floor, and **name it in the report** as
|
|
38
|
+
covered without a dedicated section, so the gap surfaces as a gap and not as silence.
|
|
39
|
+
- Damage the vocabulary could not name arrives as prose rather than a class — an
|
|
40
|
+
`unclassified: <path> — <what it can do>` line, or the rationale of a group that left its
|
|
41
|
+
tag off. That is a declaration, not a gap: take the prose as the brief and review that
|
|
42
|
+
path against it.
|
|
43
|
+
|
|
44
|
+
**No declaration is not a clean bill of health.** A declaration of `none` is a claim that
|
|
45
|
+
the change touches no such surface; a _missing_ declaration is no claim at all — the task
|
|
46
|
+
may simply predate the section. In both cases scope the depth from the change itself,
|
|
47
|
+
exactly as you would without one, and never report a surface as absent on the strength of a
|
|
48
|
+
declaration alone.
|
|
49
|
+
|
|
18
50
|
## 1. Input validation at boundaries
|
|
19
51
|
|
|
20
52
|
- Every entry point (request body, query/path params, headers, file, env, third-party
|
|
@@ -23,28 +55,41 @@ the generic floor. Flag only findings you're confident are real, ranked by sever
|
|
|
23
55
|
internal type.
|
|
24
56
|
- No mass-assignment: unexpected fields are rejected, not spread into a model.
|
|
25
57
|
|
|
26
|
-
## 2. Authorization (not just authentication)
|
|
58
|
+
## 2. Authorization (not just authentication) — `auth`
|
|
27
59
|
|
|
28
60
|
- **Authentication** (who are you?) and **authorization** (are you allowed?) are
|
|
29
61
|
distinct checks — a valid session is not permission.
|
|
30
62
|
- **Ownership / IDOR**: a resource fetched by id verifies it belongs to the caller. An
|
|
31
63
|
auth guard never substitutes for the ownership check.
|
|
32
64
|
- Privileged actions check role/scope server-side, never trusting a client claim.
|
|
65
|
+
- **Session & token handling**: a token is verified before it is trusted — signature and
|
|
66
|
+
algorithm pinned (never `none`, never caller-chosen), `exp`/`iss`/`aud` checked, claims
|
|
67
|
+
never read from an unverified decode. Sessions rotate on login and privilege change,
|
|
68
|
+
and logout/revocation invalidates server-side, not just client-side (cookie flags: §5).
|
|
33
69
|
|
|
34
70
|
## 3. Injection
|
|
35
71
|
|
|
36
72
|
- **SQL/NoSQL**: parameterized queries / validated ids; never interpolate user input
|
|
37
73
|
into a query, never pass a raw request object to a query (object-injection).
|
|
38
|
-
- **
|
|
39
|
-
(
|
|
74
|
+
- **Path**: a path built from user input is resolved and then proven to stay under a
|
|
75
|
+
fixed base — `join`/`resolve` alone are not that proof (`join` normalizes `..` through,
|
|
76
|
+
`resolve` lets an absolute segment discard the base); absolute and `..` segments are
|
|
77
|
+
rejected rather than stripped, and `realpath` is used where symlinks can exist.
|
|
78
|
+
Shell-command construction is §8.
|
|
40
79
|
- **Output/XSS**: rendered user content is escaped by default; raw-HTML sinks
|
|
41
80
|
(`dangerouslySetInnerHTML`, server-generated HTML/PDF) sanitize at render time.
|
|
42
81
|
|
|
43
|
-
## 4. Secrets & logging
|
|
82
|
+
## 4. Secrets & logging — `secrets`
|
|
44
83
|
|
|
45
84
|
- No secret, token, password, key, or PII in code, logs, or error messages (a
|
|
46
85
|
never-log redaction list is configured; emails in failure logs are hashed).
|
|
47
|
-
-
|
|
86
|
+
- **At rest**: a secret lives in the platform's secret store or a file only the app can
|
|
87
|
+
read (0600) — never a world-readable config, an image layer, or the repo (`.env`
|
|
88
|
+
gitignored; `.env.example` placeholders only). Passwords are stored under a slow KDF
|
|
89
|
+
(argon2/bcrypt/scrypt), never a bare or fast hash.
|
|
90
|
+
- **In transit**: a secret never rides in a URL or query string (proxies, browser history
|
|
91
|
+
and `Referer` capture those) — header or body over TLS only (§5) — and a leaked one has
|
|
92
|
+
a rotation path that does not require a deploy.
|
|
48
93
|
- Constant-time comparison for secrets/tokens (no `===` on a hash); strong randomness
|
|
49
94
|
for generated keys.
|
|
50
95
|
|
|
@@ -73,11 +118,79 @@ the generic floor. Flag only findings you're confident are real, ranked by sever
|
|
|
73
118
|
third-party model; tool/function access from the model is least-privilege.
|
|
74
119
|
- **Resource limits**: token/cost/loop bounds on agentic calls to prevent runaway use.
|
|
75
120
|
|
|
121
|
+
## 8. Shell & subprocess — `shell-process`
|
|
122
|
+
|
|
123
|
+
- **Argument construction**: no untrusted input concatenated into a command string;
|
|
124
|
+
arguments go as an array to an exec-style API (`execFile`/`spawn` without
|
|
125
|
+
`shell: true` — and on Windows a `.cmd`/`.bat` target is shell-parsed regardless),
|
|
126
|
+
never through a shell. An array is not sufficient on its own: a value that can begin
|
|
127
|
+
with `-` is read by the binary as a flag, so untrusted values sit after `--` or are
|
|
128
|
+
validated against a known form.
|
|
129
|
+
- **Spawn surface**: every new subprocess is deliberate — the binary resolved from a
|
|
130
|
+
known path rather than the inherited `PATH`, and the child running with no more
|
|
131
|
+
privilege than the caller (no `sudo`, setuid, or elevated container).
|
|
132
|
+
- **Secrets across the boundary**: a secret never travels as a command-line argument —
|
|
133
|
+
argv is world-readable in `ps`. It goes via the child's environment (pruned to what the
|
|
134
|
+
child needs), stdin, or a file descriptor.
|
|
135
|
+
- **Child I/O**: the child's output is untrusted input at a boundary (§1 — and §7 as well
|
|
136
|
+
if it reaches an LLM) before it drives anything; its exit code is checked, not assumed;
|
|
137
|
+
and it cannot hang or flood the parent unbounded (timeout, kill, output cap).
|
|
138
|
+
|
|
139
|
+
## 9. Money, billing & pricing — `payments`
|
|
140
|
+
|
|
141
|
+
- **Idempotency**: every charge, refund, or transfer is idempotent under retry
|
|
142
|
+
(idempotency key, provider-side dedup) — a replayed webhook or a double submit cannot
|
|
143
|
+
move money twice.
|
|
144
|
+
- **Amount & currency integrity**: amount and currency come from the server-side record,
|
|
145
|
+
never from the client; arithmetic is in integer minor units (no floats), the rounding
|
|
146
|
+
rule is explicit, and totals are recomputed rather than trusted.
|
|
147
|
+
- **Bounds & entitlement**: amounts, quantities and discounts are range- and sign-checked
|
|
148
|
+
server-side — no negative charge, no refund above what was captured, no coupon reused
|
|
149
|
+
or stacked past its rule — and access a payment buys is granted on settled state and
|
|
150
|
+
revoked when that state reverses.
|
|
151
|
+
- **Provider trust boundary**: webhooks are signature-verified against the **raw** request
|
|
152
|
+
body (a JSON body parser mounted ahead of the route silently breaks this) and
|
|
153
|
+
replay-windowed; state transitions (pending → paid → refunded) are enforced server-side
|
|
154
|
+
and written to an audit trail (who, what, when).
|
|
155
|
+
|
|
156
|
+
## 10. Destructive writes & data — `data-loss`
|
|
157
|
+
|
|
158
|
+
- **Scoped deletes and updates**: every delete or bulk update carries a bounding
|
|
159
|
+
predicate (tenant/owner/id) that is proven non-empty at runtime, not merely present in
|
|
160
|
+
the source — an `undefined` id the ORM silently drops, or a filter built from a request
|
|
161
|
+
that arrives `{}`, is a collection-wide call wearing a `where`.
|
|
162
|
+
- **Irreversibility**: a migration or schema change has a tested rollback, or is
|
|
163
|
+
explicitly one-way and says so; a destructive data migration runs behind a
|
|
164
|
+
backup/snapshot step. Soft-delete where a human may need to recover — and where it is
|
|
165
|
+
used, every read path filters deleted rows and records holding personal data still
|
|
166
|
+
have a real erasure path, not an indefinite tombstone.
|
|
167
|
+
- **Atomicity**: writes that must succeed together are transactional or idempotently
|
|
168
|
+
replayable — a partial failure cannot leave data half-written and report success.
|
|
169
|
+
- **On disk**: a write replaces a file atomically (temp + rename), never truncates or
|
|
170
|
+
deletes a path the change did not create, and never follows a symlink to do it. Any
|
|
171
|
+
destructive path is proven non-empty before use — a path variable that can be empty
|
|
172
|
+
turns a scoped delete into a root-relative one.
|
|
173
|
+
|
|
174
|
+
## 11. Executable surface — `executable-mode`
|
|
175
|
+
|
|
176
|
+
- **Permission changes**: no `chmod`/exec-bit change beyond what the feature needs;
|
|
177
|
+
neither the file nor the directory holding it becomes world-writable (a 0755 script in
|
|
178
|
+
a 0777 directory is replaceable), and no write-then-execute path — a file the app
|
|
179
|
+
writes is never later executed without a trust decision.
|
|
180
|
+
- **Install-time surface**: files laid down by an installer or updater carry the minimum
|
|
181
|
+
mode; a hook or script that gains executable mode is reviewed as code that runs, not
|
|
182
|
+
as data.
|
|
183
|
+
- **What can run**: a change to what the system executes unattended — a new entry point,
|
|
184
|
+
hook, cron or launch agent, service unit, `postinstall`, or a changed interpreter
|
|
185
|
+
shebang — is named in the report with who can write to it and what triggers it, even
|
|
186
|
+
when it looks routine. How this change spawns a process is §8.
|
|
187
|
+
|
|
76
188
|
## Report
|
|
77
189
|
|
|
78
190
|
```
|
|
79
191
|
## Security Review — <task name>
|
|
80
192
|
|
|
193
|
+
**Declared surfaces**: <class → where you pressed> | none declared | no declaration handed over
|
|
81
194
|
**Critical**: <vector — file:line — fix> | none
|
|
82
195
|
**High**: <…> | none
|
|
83
196
|
**Medium**: <…> | none
|
|
@@ -37,6 +37,8 @@ The body is the externalized spec — self-contained enough to read on GitHub, w
|
|
|
37
37
|
links back to the committed files for the full detail:
|
|
38
38
|
|
|
39
39
|
```markdown
|
|
40
|
+
<trace lines, if the skeleton carried any — verbatim, first>
|
|
41
|
+
|
|
40
42
|
## Summary
|
|
41
43
|
|
|
42
44
|
<one paragraph: the capability and why, from the PRD>
|
|
@@ -61,7 +63,11 @@ gh issue edit <id> --body-file <body>
|
|
|
61
63
|
```
|
|
62
64
|
|
|
63
65
|
This overwrites the `🚧 Spec in progress …` skeleton the Orchestrator wrote at issue
|
|
64
|
-
creation.
|
|
66
|
+
creation. **Preserve its trace lines**: a task that is a part of a partitioned feature
|
|
67
|
+
carries `Part <k> of #<parent>` (and `Depends on #<sibling> …`) at the top of the
|
|
68
|
+
skeleton — read the current body first and keep those lines, verbatim, as the first
|
|
69
|
+
lines of the new body. They are how the parent plan, `/resume`, and closeout find the
|
|
70
|
+
task's place in the partition; dropping them orphans the part. Labels are untouched — the issue still carries `harness:managed` +
|
|
65
71
|
`harness:sdd` + `harness:status:spec-in-progress`; the Orchestrator flips it to
|
|
66
72
|
`spec-ready` after you hand back.
|
|
67
73
|
|
|
@@ -27,7 +27,8 @@ Three moves, in this order:
|
|
|
27
27
|
`_archive/<id>/`; drop only `progress.md` (true scratch). The high-value memory stays
|
|
28
28
|
live and grep-able.
|
|
29
29
|
3. **Land via a PR** — the `history.md` append, the archival move, and any new ADR ride a
|
|
30
|
-
dedicated `harness/closeout-<id>` PR
|
|
30
|
+
dedicated `harness/closeout-<id>` PR that self-merges **only on green checks**
|
|
31
|
+
(through `.claude/hooks/lib/merge-pr.sh`). No direct push to the base.
|
|
31
32
|
|
|
32
33
|
Run this only when the **Reviewer has approved** and the task PR is merged. The merge is a
|
|
33
34
|
human decision (the merge gate) — closeout never merges the task; it **confirms** the
|
|
@@ -158,34 +159,74 @@ working scratch are gone. A UI task's **`ui-handoff.md`** is a sibling **inside*
|
|
|
158
159
|
`spec/`, so the single `git mv` of `spec/` archives it with the rest — no special
|
|
159
160
|
handling.
|
|
160
161
|
|
|
161
|
-
### 5. Open the closeout PR and
|
|
162
|
+
### 5. Open the closeout PR and self-merge on green
|
|
162
163
|
|
|
163
|
-
Commit the record, push the branch, open a PR, and
|
|
164
|
+
Commit the record, push the branch, open a PR, and self-merge it **through the
|
|
165
|
+
checks-precondition executor** — never a bare `gh pr merge`. There is no standing human
|
|
166
|
+
authorization on this path, so the precondition stands alone: **the executor merges
|
|
167
|
+
only on green; every other outcome takes one of the exits below** (some end in a
|
|
168
|
+
human's informed decision — what never happens is an autonomous merge on not-green).
|
|
169
|
+
The executor verifies the check status the platform reports for
|
|
170
|
+
the PR, waiting — bounded (`merge.checks_timeout_secs`, default ~10 min) — for pending
|
|
171
|
+
checks:
|
|
164
172
|
|
|
165
173
|
```bash
|
|
166
174
|
git commit -m "closeout(<id>): archive task state, record in history.md"
|
|
167
175
|
git push -u origin harness/closeout-<id>
|
|
168
176
|
gh pr create --base <default> --head harness/closeout-<id> \
|
|
169
177
|
--title "closeout(<id>): <topic>" --body "Closeout record for #<id>."
|
|
170
|
-
|
|
178
|
+
.claude/hooks/lib/merge-pr.sh harness/closeout-<id> --squash --delete-branch
|
|
171
179
|
```
|
|
172
180
|
|
|
181
|
+
(If the human recorded a standing merge answer at this task's merge gate — a
|
|
182
|
+
`merge.allow_no_checks: true` line sitting uncommitted in `harness.config.yml` —
|
|
183
|
+
stage it into the record commit above: the answer rides the closeout PR to the base.)
|
|
184
|
+
|
|
173
185
|
The closeout PR **must not** carry `Closes #<id>` — the task PR already auto-closed the
|
|
174
186
|
issue on merge; closeout only flips the label and finalizes.
|
|
175
187
|
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
- **
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
188
|
+
Act on the executor's exit code:
|
|
189
|
+
|
|
190
|
+
- **0 — merged.** Continue to step 6 (finalize).
|
|
191
|
+
- **10 — checks red.** Red does **not** automatically mean the base is broken — look at
|
|
192
|
+
what failed (`gh run view` / the check's output), then take one of three exits:
|
|
193
|
+
1. **Failure in the closeout's own files** (format/lint on the moved docs): fixable
|
|
194
|
+
by you alone — run the formatter, commit, push, re-run the executor; green ⇒
|
|
195
|
+
self-merge normally. No human interruption; fixing format on your own docs-only PR
|
|
196
|
+
is harmless and autonomous.
|
|
197
|
+
2. **Failure in the base** (main was already broken): **park** and surface with the
|
|
198
|
+
diagnosis — the human's informed merge-on-red decision is legitimate precisely
|
|
199
|
+
here, since the red isn't this PR's (on their explicit yes, re-run with
|
|
200
|
+
`--force`). Free side effect: every closeout is a broken-main detector.
|
|
201
|
+
3. **Can't tell:** **park** and surface what you saw — in doubt, the human.
|
|
202
|
+
|
|
203
|
+
The uniform rule stands: **never merge on red alone** — no "it's only docs"
|
|
204
|
+
exception (automatic exceptions are how the bug comes back).
|
|
205
|
+
|
|
206
|
+
- **20 — no checks reported (after the grace window).** Never merge alone, whatever the
|
|
207
|
+
cause — don't auto-classify (no CI, CI dead, path filters all look identical). The
|
|
208
|
+
once-per-repo memory is the **standing answer only**: if `merge.allow_no_checks: true`
|
|
209
|
+
is recorded (or sits uncommitted on this branch, staged per step 5), the executor
|
|
210
|
+
applies it itself — you won't see exit 20. A **one-shot** yes given at an earlier
|
|
211
|
+
merge (including this task's merge gate) authorized **that merge only** and does
|
|
212
|
+
**not** carry over — never re-apply it here on your own. Seeing exit 20 means no
|
|
213
|
+
standing answer exists, so ask: "I saw no checks on the closeout PR — merge?". A
|
|
214
|
+
standing answer ("this repo has no CI — merge without checks") is recorded as
|
|
215
|
+
`merge.allow_no_checks: true` in `harness.config.yml` — add the line on this very
|
|
216
|
+
closeout branch, push, and re-run the executor (it reads the working tree, and the
|
|
217
|
+
answer lands on the base with the record). A one-shot yes ⇒ re-run with `--force`.
|
|
218
|
+
No human available ⇒ **park**.
|
|
219
|
+
- **30 — checks still pending past the bound.** **No merge**: **park**. The closeout is
|
|
220
|
+
docs-only archival — leaving it pending blocks nothing already merged.
|
|
221
|
+
- **1 (or any other non-zero) — no verdict.** Read the executor's stderr: either the
|
|
222
|
+
check status could not be read (`gh`/auth/network — the precondition is
|
|
223
|
+
unverifiable), or the merge attempt itself was rejected (branch protection requires
|
|
224
|
+
human approval, a conflict). Both ⇒ **park**, surfacing that stderr — a
|
|
225
|
+
protection-parked record PR is merged by the human by hand.
|
|
186
226
|
|
|
187
227
|
**Park:** flip the issue to `harness:status:closeout-pending`, tell the human the closeout
|
|
188
|
-
PR is open
|
|
228
|
+
PR is open, why it did not merge (the executor's output — failing checks, no checks,
|
|
229
|
+
pending past the bound, or protection), and stop. The task issue is **already closed** (the
|
|
189
230
|
task PR's `Closes #<id>` fired on its merge), so `/resume` finds the parked closeout only
|
|
190
231
|
by listing closed issues too (`--state all`) — a default open-only queue would miss it. A
|
|
191
232
|
later `/resume` picks up at step 6 once the PR is merged. (Authority for the RESUME entry:
|
|
@@ -193,7 +234,10 @@ the Orchestrator.)
|
|
|
193
234
|
|
|
194
235
|
### 6. Finalize (once the closeout PR is merged)
|
|
195
236
|
|
|
196
|
-
|
|
237
|
+
On a `/resume` that finds the record PR **still open**, first re-run the executor on it
|
|
238
|
+
(step 5's `merge-pr.sh` line, verbatim): a park on pending or red checks is transient
|
|
239
|
+
and may have settled green — a repeat not-green outcome just re-parks. Then confirm the
|
|
240
|
+
closeout PR merged (`gh pr view <pr> --json state,mergedAt` → `MERGED`; on a
|
|
197
241
|
`/resume`, pass the deterministic branch `harness/closeout-<id>` as `<pr>` — the
|
|
198
242
|
merge-confirm accepts a branch name in place of a PR number), land the merged base
|
|
199
243
|
(`git checkout <default> && git pull`), then:
|
|
@@ -209,11 +253,31 @@ single emit point for either path). You compute the envelope (cycle time, review
|
|
|
209
253
|
rejections, level) as the Orchestrator running this skill; the fields and the `emit`
|
|
210
254
|
command line are in `orchestrator.md` §Closeout. `events.jsonl` is local-only/gitignored, so the emit never dirties the base.
|
|
211
255
|
|
|
256
|
+
**If the task is a part of a partitioned feature** — its issue body (read it with
|
|
257
|
+
`gh issue view <id> --json body`; the issue is closed by now) carries a
|
|
258
|
+
`Part <k> of #<parent>` trace line — update the parent **partition-plan** issue here, in
|
|
259
|
+
the same finalize, **read-modify-write**: read the parent body immediately before
|
|
260
|
+
editing (`gh issue view <parent> --json body`), tick only your row and set it to
|
|
261
|
+
`- [x] Part <k> — <slice> → #<id> merged`, write it back with
|
|
262
|
+
`gh issue edit <parent> --body-file`, then re-read to confirm the tick is present — if another row moved
|
|
263
|
+
underneath you (a sibling closing out elsewhere), redo it **once** on the fresh body
|
|
264
|
+
(match rows tolerant of a trailing `\r`: a body edited on the web may carry CRLF); still
|
|
265
|
+
absent → leave the tick to the human and say so. Idempotent on a retried finalize: an
|
|
266
|
+
already-ticked row is left alone, and the all-ticked check still runs. When that
|
|
267
|
+
leaves **every** row ticked (merged or dropped), close the parent with a summary comment
|
|
268
|
+
(`gh issue close <parent> --comment "<what shipped, part by part>"`) — mechanical, no
|
|
269
|
+
human prompt: each part's merge already passed the human gate; a parent the human already
|
|
270
|
+
closed takes the tick and the close is a no-op. A parent with unticked rows stays open;
|
|
271
|
+
`/resume` surfaces its next part. The contract is the Orchestrator's companion
|
|
272
|
+
`.claude/agents/partition.md`.
|
|
273
|
+
|
|
212
274
|
### 7. Report
|
|
213
275
|
|
|
214
276
|
Return a one-line summary: the task id, the `history.md` entry, the archive path
|
|
215
|
-
(`_archive/<id>/`), any ADR raised,
|
|
216
|
-
closeout is parked at `closeout-pending` awaiting the record PR's merge)
|
|
277
|
+
(`_archive/<id>/`), any ADR raised, confirmation that the issue is closed (or that
|
|
278
|
+
closeout is parked at `closeout-pending` awaiting the record PR's merge), and — for a
|
|
279
|
+
partition part — that the parent row was ticked (and the parent closed, on the last
|
|
280
|
+
part).
|
|
217
281
|
|
|
218
282
|
## Scope note
|
|
219
283
|
|