swe-workflow-skills 0.6.0 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/VERSION +1 -1
- package/catalog.json +1 -1
- package/hooks/session-start.mjs +5 -3
- package/package.json +1 -1
- package/skills/code-reviewing/SKILL.md +26 -2
- package/skills/code-reviewing/evals/evals.json +32 -0
- package/skills/code-reviewing/references/review-checklist.md +22 -2
- package/skills/code-slop-cleanup/SKILL.md +5 -2
- package/skills/code-slop-cleanup/references/slop-patterns.md +6 -4
- package/skills/git-workflow/SKILL.md +5 -1
- package/skills/git-workflow/evals/evals.json +17 -0
- package/skills/plan-execution/SKILL.md +18 -3
- package/skills/plan-execution/evals/evals.json +35 -1
- package/skills/project-documentation/SKILL.md +50 -2
- package/skills/project-documentation/evals/evals.json +42 -0
- package/skills/skill-router/SKILL.md +26 -13
- package/skills/technical-debt-review/SKILL.md +24 -0
- package/skills/technical-debt-review/evals/evals.json +16 -0
- package/skills/technical-debt-review/references/slop-audit-method.md +75 -0
- package/skills/technical-debt-review/templates/slop-audit.md +116 -0
- package/skills/test-suite-design/SKILL.md +15 -0
- package/skills/test-suite-design/evals/evals.json +14 -0
- package/skills/verification-before-completion/SKILL.md +79 -0
- package/skills/verification-before-completion/evals/evals.json +51 -1
package/README.md
CHANGED
|
@@ -136,6 +136,8 @@ data-modeling → user/session schema + migration
|
|
|
136
136
|
tdd-workflow → red-green-refactor the implementation
|
|
137
137
|
security-audit → authn/authz, token handling, OWASP pass
|
|
138
138
|
code-reviewing → DRY/KISS/SRP + conventions
|
|
139
|
+
verification-before-completion
|
|
140
|
+
→ evidence for the "done" claim; docs reconciled
|
|
139
141
|
deployment-checklist → pre-deploy safety + rollback readiness
|
|
140
142
|
```
|
|
141
143
|
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.6.
|
|
1
|
+
0.6.2
|
package/catalog.json
CHANGED
|
@@ -228,7 +228,7 @@
|
|
|
228
228
|
},
|
|
229
229
|
{
|
|
230
230
|
"name": "project-documentation",
|
|
231
|
-
"description": "Write
|
|
231
|
+
"description": "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review. Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating.",
|
|
232
232
|
"path": "project-documentation/SKILL.md"
|
|
233
233
|
},
|
|
234
234
|
{
|
package/hooks/session-start.mjs
CHANGED
|
@@ -43,10 +43,12 @@ routing "once you understand the problem" is already too late — route, THEN
|
|
|
43
43
|
explore under the chosen skill.
|
|
44
44
|
|
|
45
45
|
Re-route on each new PHASE of work, not just at session start: when you pivot
|
|
46
|
-
from analysis to implementing, to writing tests, to reviewing,
|
|
47
|
-
|
|
46
|
+
from analysis to implementing, to writing tests, to reviewing, to FINISHING
|
|
47
|
+
(claiming done, committing, opening a PR), or to shipping, consult
|
|
48
|
+
\`skill-router\` again for that phase before you act. The skill for the new
|
|
48
49
|
phase almost certainly isn't loaded yet — routing once at the start does not
|
|
49
|
-
cover work you start an hour later.
|
|
50
|
+
cover work you start an hour later. Finishing counts: "that's everything",
|
|
51
|
+
"let's commit", and "ready to open the PR" begin a phase and need routing too.
|
|
50
52
|
|
|
51
53
|
Route even when you already believe you know how to proceed — do NOT wait until
|
|
52
54
|
you feel unsure, and do NOT talk yourself out of it because the task "looks like a
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "swe-workflow-skills",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.2",
|
|
4
4
|
"description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -37,14 +37,38 @@ Look at the forest before the trees. Check:
|
|
|
37
37
|
- **Is the approach sound?** Before nitpicking syntax, is the overall strategy correct?
|
|
38
38
|
- **Are there missing pieces?** Tests? Documentation? Error handling? Migration?
|
|
39
39
|
|
|
40
|
+
**Step outside the diff.** The most common AI slop is invisible in a single hunk —
|
|
41
|
+
it only shows up repo-wide. For any new helper, block, command, or exported symbol in
|
|
42
|
+
the diff, run three greps before accepting it:
|
|
43
|
+
- **A twin?** Grep the repo for an existing function/block that already does this — AI
|
|
44
|
+
assistants generate a fourth copy instead of finding copies #1–3 (see `code-slop-cleanup`'s
|
|
45
|
+
*Reuse Before Write*).
|
|
46
|
+
- **Any importer?** A newly-`export`ed symbol with no consumer is dead surface (often
|
|
47
|
+
exported just to silence the unused-symbol checker).
|
|
48
|
+
- **Any consumer?** A new schema field, prop, or param that nothing reads or sets is
|
|
49
|
+
speculative plumbing — the API looks finished but is wired to nothing.
|
|
50
|
+
- **Any doc still describing the old shape?** If the diff renames, removes, or adds
|
|
51
|
+
something a user types or configures — a flag, env var, endpoint, config key,
|
|
52
|
+
command, script, install step — grep the README and `docs/` for it, old name
|
|
53
|
+
included. Check tables, diagrams, and layout blocks first: prose gets reread when
|
|
54
|
+
someone edits the feature, inventories don't.
|
|
55
|
+
|
|
56
|
+
When the diff *is* documentation, review it against the code rather than for prose
|
|
57
|
+
quality. The failure mode is a confident sentence nobody verified — check each claim
|
|
58
|
+
against the file it describes, and treat an unverifiable one as a finding.
|
|
59
|
+
|
|
60
|
+
Also check the *unit being grown*, not just the hunk: adding 15 lines to an already
|
|
61
|
+
800-line function or 200-line file is how god-functions form one reviewer-approved PR at
|
|
62
|
+
a time. Judge the total size after the change, not the size of the diff.
|
|
63
|
+
|
|
40
64
|
### Step 3: Detailed Review
|
|
41
65
|
|
|
42
66
|
Review the code against these categories, in order of importance. See [references/review-checklist.md](references/review-checklist.md) for the detailed checklist.
|
|
43
67
|
|
|
44
68
|
**Severity levels for findings:**
|
|
45
69
|
|
|
46
|
-
- 🔴 **Blocker**: Must fix before merge (bugs, security issues, data loss risks)
|
|
47
|
-
- 🟡 **Suggestion**: Should fix, significantly improves quality (principle violations, missing tests)
|
|
70
|
+
- 🔴 **Blocker**: Must fix before merge (bugs, security issues, data loss risks) — including a doc the diff makes *wrong* where the doc is an instruction someone follows: install steps, setup commands, the API contract, config keys. A stale instruction fails for the next person, silently, and is worse than no instruction.
|
|
71
|
+
- 🟡 **Suggestion**: Should fix, significantly improves quality (principle violations, missing tests, docs the diff leaves incomplete rather than wrong)
|
|
48
72
|
- 🔵 **Nit**: Optional improvement (naming, style, minor simplification)
|
|
49
73
|
|
|
50
74
|
### Step 4: Present Findings
|
|
@@ -23,6 +23,7 @@
|
|
|
23
23
|
"assertions": [
|
|
24
24
|
"Runs git diff or git show to read the changes",
|
|
25
25
|
"Reads surrounding code for context",
|
|
26
|
+
"For any new helper, export, or block, greps the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
|
|
26
27
|
"Follows the structured review format (summary, blockers, suggestions, nits)",
|
|
27
28
|
"Provides specific file:line references for findings"
|
|
28
29
|
]
|
|
@@ -57,6 +58,37 @@
|
|
|
57
58
|
"Keeps/endorses the round-trip test as legitimate own-logic coverage",
|
|
58
59
|
"Suggests testing our actual logic instead (e.g. the legacy fallback branches)"
|
|
59
60
|
]
|
|
61
|
+
},
|
|
62
|
+
{
|
|
63
|
+
"id": 2,
|
|
64
|
+
"prompt": "Review this Python helper — it's the fact-check gate the pipeline runs before publishing:\n\ndef check_claims(response, model):\n try:\n report = parse_report(call_model(model, response))\n except (ValueError, JSONDecodeError) as e:\n log.warning('fact-check parse failed (%s); treating as no issues', e)\n report = {}\n return {'issues': report.get('issues', []), 'passed': not report.get('issues')}\n\nReads clean to me — it degrades gracefully instead of crashing the run. Ship it?",
|
|
65
|
+
"pressure": [
|
|
66
|
+
"looks_good"
|
|
67
|
+
],
|
|
68
|
+
"expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification.",
|
|
69
|
+
"assertions": [
|
|
70
|
+
"Identifies the empty-report fallback as fail-open: a failure that returns a success-shaped result (passed=True / no issues)",
|
|
71
|
+
"Explains the consequence — an unparseable response publishes as 'no issues found', the inverse of what a check should do",
|
|
72
|
+
"Marks it a Blocker (correctness/data-integrity), not a nit",
|
|
73
|
+
"Distinguishes this from a legitimate documented degradation fallback",
|
|
74
|
+
"Recommends failing closed — mark unverified / route to review rather than defaulting to pass",
|
|
75
|
+
"Does NOT accept the 'degrades gracefully instead of crashing' framing as sufficient"
|
|
76
|
+
]
|
|
77
|
+
},
|
|
78
|
+
{
|
|
79
|
+
"id": 3,
|
|
80
|
+
"prompt": "Review this PR. It renames the DATABASE_URL env var to DB_DSN across the service. Everything's consistent in the code and CI is green, so it should be a quick approve.",
|
|
81
|
+
"pressure": [
|
|
82
|
+
"looks_good",
|
|
83
|
+
"time"
|
|
84
|
+
],
|
|
85
|
+
"expected_behavior": "Steps outside the diff: a renamed env var is a documented surface, so greps README and docs/ for the old DATABASE_URL name. Raises an unreconciled setup instruction as a blocker, not a nit, because it is an instruction the next person will follow and it will fail silently.",
|
|
86
|
+
"assertions": [
|
|
87
|
+
"Greps the docs for the OLD name (DATABASE_URL), not only for code consistency",
|
|
88
|
+
"Raises the stale setup instruction as a blocker (red), not a nit",
|
|
89
|
+
"Justifies the severity: a wrong instruction is followed and then fails, which is worse than a missing one",
|
|
90
|
+
"Does NOT approve on 'code is consistent and CI is green' alone"
|
|
91
|
+
]
|
|
60
92
|
}
|
|
61
93
|
]
|
|
62
94
|
}
|
|
@@ -99,6 +99,9 @@ Flag when you see:
|
|
|
99
99
|
- [ ] No tests deleted or skipped without stated justification
|
|
100
100
|
- [ ] No expected values special-cased to match current (possibly buggy) output
|
|
101
101
|
- [ ] The unit under test is not itself mocked out
|
|
102
|
+
- [ ] Assertions target behavior, not cosmetics — not pinned to CSS class strings, emoji, exact copy, or rendered-prompt prose that a harmless reword would break (assert a stable data attribute or structure instead)
|
|
103
|
+
- [ ] No hedged or non-falsifiable assertions (`is None or == []`, `x in ("a","b")` covering both branches, a name that claims more than the assertion checks) — each test can actually fail
|
|
104
|
+
- [ ] The error/failure branch that's visible in the changed code has a test — happy-path fixtures that always feed valid input never exercise the branch just written
|
|
102
105
|
|
|
103
106
|
## Common Patterns to Flag
|
|
104
107
|
|
|
@@ -125,10 +128,16 @@ Flag when you see:
|
|
|
125
128
|
Patterns characteristic of AI-assisted diffs. Each is judged *relative to the surrounding code* — the same construct can be correct at one location and slop at another:
|
|
126
129
|
|
|
127
130
|
- **Defensive checks abnormal for the codepath**: try/catch, null guards, or re-validation deep in trusted paths whose inputs were already validated upstream. (Legitimate at process boundaries, external calls, and user input.)
|
|
131
|
+
- **Fail-open on error**: a check or validator that, when it fails or can't run, emits a value that *reads as success* — a parse failure returning `{}` → "no issues", an error collapsed to `"exit code 1"` that drops the real message. The inversion of a safety check; usually a bug, not clutter (flag as a Blocker, don't wave it through).
|
|
128
132
|
- **Style inconsistent with the surrounding file**: naming, formatting, or idiom that doesn't match what the file already does.
|
|
129
|
-
- **A new helper duplicating an existing one**: grep for an existing implementation before accepting any new utility — AI assistants generate rather than search.
|
|
133
|
+
- **A new helper duplicating an existing one**: grep for an existing implementation before accepting any new utility — AI assistants generate rather than search. Watch for the recurring shapes: an N-arg setup ritual pasted at every entry point, parallel functions differing only by a noun/constant, byte-identical mappers across sibling modules.
|
|
134
|
+
- **Dead surface exported to silence the checker**: a newly-`export`ed symbol with no importer anywhere (the `_`-prefix + `export` tell), or a fully-built, tested module wired into nothing. Grep for consumers before accepting the export.
|
|
135
|
+
- **Declared-but-unconsumed / speculative plumbing**: a schema field validated but never rendered, a prop threaded through layers but never set, a param carried "for symmetry", a value computed but never surfaced. Ask *who reads this / who sets this?* — if the answer is "nobody", it shouldn't land.
|
|
136
|
+
- **Appending to an already-large unit**: a small hunk that grows an already-800-line function or file — how god-functions accrete one approved PR at a time. Judge the unit's total size after the change, not the hunk's.
|
|
137
|
+
- **Type-checker silencing**: `any`/casts to make an error go away, a param widened to `object`/`unknown` so a `type: ignore` can reach a field, or a stale `type: ignore` that no longer matches any error.
|
|
138
|
+
- **Library-layer function doing I/O or exit**: a domain/pure/pipeline-layer function that prints, logs to the console, or raises `SystemExit` — a presentation/process concern leaking into logic, which forces lossy error handling at every caller.
|
|
130
139
|
- **Unrequested backwards-compat shims**: `_legacy` wrappers, kept old code paths, re-exports nobody asked for — hedging instead of committing to the change.
|
|
131
|
-
- **Stray working files in the diff**: PLAN.md, NOTES.md, scratch scripts, or other session artifacts that shouldn't ship.
|
|
140
|
+
- **Stray working files in the diff**: PLAN.md, NOTES.md, scratch scripts, checked-in generated/derived artifacts, or other session artifacts that shouldn't ship.
|
|
132
141
|
|
|
133
142
|
For the full taxonomy with severity levels and when-it's-NOT-slop guidance, see `code-slop-cleanup/references/slop-patterns.md` (if installed).
|
|
134
143
|
|
|
@@ -149,6 +158,17 @@ For the full taxonomy with severity levels and when-it's-NOT-slop guidance, see
|
|
|
149
158
|
|
|
150
159
|
**Rule**: If code needs a comment to explain *what* it does, refactor the code so it doesn't need one. Comments should explain *why*, not *what*.
|
|
151
160
|
|
|
161
|
+
## Documentation the Diff Affects
|
|
162
|
+
|
|
163
|
+
Separate from whether the change *needs* new docs — this is whether it made existing ones wrong.
|
|
164
|
+
|
|
165
|
+
- Did the diff rename, remove, or add anything a user types or configures? Flags, env vars, config keys, endpoints, commands, script names, package/directory names, install or setup steps. If so, grep the README and `docs/` for it, **old name included** — the old name is what the docs still contain.
|
|
166
|
+
- Check inventories before prose: endpoint tables, architecture diagrams, project-layout blocks, ADR indexes, command lists. Prose gets reread when someone edits the feature; tables don't, so that's where drift hides.
|
|
167
|
+
- A clean grep isn't an all-clear — docs use shorthand and brace expansion (`/api/{users,orders}`). If a doc plausibly covers the area, open the section.
|
|
168
|
+
- **Severity**: a doc the diff makes *wrong* is 🔴 when it's an instruction someone follows (install steps, setup commands, the API contract) — it will fail silently for the next person. Incomplete-but-not-wrong is 🟡.
|
|
169
|
+
|
|
170
|
+
**Reviewing doc hunks**: check each claim against the file it describes, not for prose quality. The failure mode is a confident, freshly-written, unverified sentence — it looks maintained, which makes it more durable than the stale line it replaced. An assertion you can't tie to code is a finding.
|
|
171
|
+
|
|
152
172
|
## Language-Specific Patterns
|
|
153
173
|
|
|
154
174
|
These are loaded contextually based on the codebase language. If reviewing:
|
|
@@ -23,7 +23,7 @@ Branch state vs the default branch (live at skill load):
|
|
|
23
23
|
|
|
24
24
|
!`git diff main...HEAD --stat 2>/dev/null || git diff master...HEAD --stat 2>/dev/null || true`
|
|
25
25
|
|
|
26
|
-
Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
|
|
26
|
+
Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts, checked-in generated/derived build artifacts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
|
|
27
27
|
|
|
28
28
|
### Step 2: Judge Each Hunk Against Its Surroundings
|
|
29
29
|
|
|
@@ -53,7 +53,10 @@ Show what was removed, grouped by category with counts, plus a **flagged-not-rem
|
|
|
53
53
|
|
|
54
54
|
## Prevention: Reuse Before Write
|
|
55
55
|
|
|
56
|
-
The biggest slop category — duplication — is created at generation time and only caught at review time
|
|
56
|
+
The biggest slop category — duplication — is created at generation time and only caught (expensively) at review time, because a single diff hunk hides the copies that already exist elsewhere. This is the canonical counter, and `code-reviewing` points reviewers back here:
|
|
57
|
+
|
|
58
|
+
- **Before writing any new helper, grep the codebase for an existing one** — name fragments, the operation's key terms, the module where it would live. AI assistants generate what looks plausible instead of searching for what exists; searching first is the cheap counter.
|
|
59
|
+
- **Before writing the third near-copy of a block, extract the seam instead.** The recurring shapes — an N-arg setup/wiring ritual pasted at every entry point, parallel functions differing only by a noun or constant, byte-identical mappers across sibling modules — are cheap to write locally and expensive to maintain. The moment you're about to paste-and-tweak, that's the signal to factor.
|
|
57
60
|
|
|
58
61
|
## Boundaries — What This Skill Never Does
|
|
59
62
|
|
|
@@ -10,6 +10,7 @@ The pattern catalog for `code-slop-cleanup` (diff scope) and `technical-debt-rev
|
|
|
10
10
|
|---------|-------|--------------------|
|
|
11
11
|
| **Debug leftovers** | `console.log`/`print` debugging, function entry/exit logging, dump-everything statements | Structured logging through the project's logger at intentional log points |
|
|
12
12
|
| **Silent-failure handlers** | Empty catch blocks; `catch (e) { console.log(e) }` and continue; fallback values that mask real errors (`?? defaultValue` hiding a failed fetch) | Deliberate, documented degradation (feature flag off, optional enhancement) where the fallback is a product decision |
|
|
13
|
+
| **Fail-open on error** (correctness-critical) | On failure, emit a value that *reads as success*: a parse/validate failure that returns `{}`/empty → "no issues found"; an error collapsed to `"exit code 1"` that discards the real message a caller needs; a check that defaults to "pass" when it can't run. The dangerous inversion of a check whose whole job is to catch problems. | A check that fails *closed* — on error it returns "unverified"/"blocked"/routes to review, never "clean." **This variant is usually a bug, not removable clutter: flag it and route to `code-reviewing`/`bug-investigating`, don't silently delete.** |
|
|
13
14
|
| **Stray working files** | `PLAN.md`, `NOTES.md`, `SCRATCH.md`, `SUMMARY.md`, one-off test scripts, session artifacts committed to the tree | Files the project's conventions actually track (an ADR, a checked-in design doc in `docs/`) |
|
|
14
15
|
|
|
15
16
|
## Medium severity
|
|
@@ -19,13 +20,14 @@ The pattern catalog for `code-slop-cleanup` (diff scope) and `technical-debt-rev
|
|
|
19
20
|
| **Defensive theater** | try/catch on trusted internal paths; triple null checks (`!== null && !== undefined && !== ''`); re-validation of input already validated upstream; guards for states nothing can produce | Process boundaries, external API calls, user input, deserialization — anywhere the input is genuinely untrusted or the failure genuinely reachable |
|
|
20
21
|
| **Generic TODOs** | `// TODO: handle errors`, `// TODO: improve this` — no context, no ticket, no owner | TODO with a ticket reference or a concrete, actionable description |
|
|
21
22
|
| **Unrequested backwards-compat shims** | `_legacy` wrappers, kept old code paths "just in case", re-exports nobody consumes, dual old/new implementations | A migration the plan actually calls for, with a removal date or ticket |
|
|
22
|
-
| **Duplication of existing helpers** | A new utility that re-implements something already in the codebase or stdlib; parallel functions differing by a constant | Genuinely different concepts that happen to look similar (false DRY is worse than repetition) |
|
|
23
|
+
| **Duplication of existing helpers** | A new utility that re-implements something already in the codebase or stdlib. Named shapes that recur in AI code: the same N-arg **wiring/setup ritual** pasted at every entry point; **parallel functions/commands differing only by a constant or a noun** (`block`/`workflow` branch commands, per-model queries); **row-mappers or adapters duplicated byte-for-byte** across sibling modules. Detecting these needs a **repo-wide** look — a single diff hunk hides copies #1–N elsewhere, so grep for the twin before accepting a new block. | Genuinely different concepts that happen to look similar (false DRY is worse than repetition) |
|
|
24
|
+
| **Unconsumed surface / speculative plumbing** | Exports with no importers anywhere (the `_`-prefix + `export` tell — exported only to silence the unused-symbol checker); a fully-built, unit-tested module never wired into any path; a schema field validated but never read, a prop threaded through layers but never set, a param carried "for symmetry" but unused, a value computed but never surfaced. The API *looks* finished; nothing calls it. | A library/package's genuine public API; params required by an interface/callback signature; a documented extension point with a near-term, named consumer landing in the same change |
|
|
23
25
|
|
|
24
26
|
## Low severity — cosmetic residue
|
|
25
27
|
|
|
26
28
|
| Pattern | Tells | When it's NOT slop |
|
|
27
29
|
|---------|-------|--------------------|
|
|
28
|
-
| **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments | Why-comments on non-obvious decisions; consequence warnings; docstrings on public API the project mandates |
|
|
30
|
+
| **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments, a **changelog / version-history block embedded in source** (belongs in CHANGELOG/ADR), a **stale comment describing a scheme the code no longer implements** (actively misleading, worse than none) | Why-comments on non-obvious decisions; consequence warnings; docstrings on public API the project mandates |
|
|
29
31
|
| **Dead weight** | Unused imports/params/variables, `return undefined`, `=== true`, `Promise.all` around a single promise, unreachable branches | Parameters required by an interface/callback signature; re-exports that are the module's public API |
|
|
30
32
|
| **Style inconsistency** | Naming, formatting, or idiom that doesn't match the surrounding file (e.g. `fetchData` in a file that uses `getX` everywhere) | The file itself is inconsistent and the new code follows the project-level convention |
|
|
31
33
|
|
|
@@ -36,7 +38,7 @@ These are slop, but this catalog's consumers don't fix them in place:
|
|
|
36
38
|
| Pattern | Tells | Route to |
|
|
37
39
|
|---------|-------|----------|
|
|
38
40
|
| **Test-file slop** | Redundant/duplicate tests, trivial asserts (`expect(true)`), asserting on the mock, implementation-mirroring tests | `test-suite-design` (Pruning Test Slop) — deleting a test is never behavior-neutral for the safety net; it needs behavior-map judgment. **Never delete a test during diff cleanup.** |
|
|
39
|
-
| **Type slop** | `any`/`unknown` to silence the checker, casts papering over mismatches, single-implementer interfaces "for testing", overly broad types | `technical-debt-review` (type debt) — usually cross-file; fixing it changes signatures |
|
|
41
|
+
| **Type slop** | `any`/`unknown` to silence the checker, casts papering over mismatches, single-implementer interfaces "for testing", overly broad types (`env: object` then `type: ignore` to reach a field), **stale `type: ignore` matching no current error**, a **declared strict gate (`mypy strict`, `--max-warnings 0`) sitting on N errors** because nobody runs it | `technical-debt-review` (type debt) — usually cross-file; fixing it changes signatures. Running the gate green is the `verification-before-completion` "done" step |
|
|
40
42
|
| **Config/scaffolding slop** | Config keys and feature flags nothing reads, orphan `.env` entries, checked-in generated files, half-finished TODO scaffolds | `technical-debt-review` (config/scaffolding debt) — proving "unread" needs repo-wide tracing |
|
|
41
43
|
| **Documentation drift** | README/docs describing endpoints, flags, or behavior that no longer exists | `technical-debt-review` (documentation drift) — needs docs-vs-code reconciliation, not deletion |
|
|
42
|
-
| **Structural slop** | Three different patterns for the same concern, business logic in controllers, circular imports | `refactoring` / `architecture-design` — structural change, not removal |
|
|
44
|
+
| **Structural slop** | Three different patterns for the same concern, business logic in controllers, circular imports, a **library/pure-layer function leaking a presentation or process concern** (prints to the console, raises `SystemExit`, does terminal I/O from inside the domain/pipeline layer) — which then forces lossy `except`/re-wrapping at every caller, so one leak becomes the root of many downstream findings | `refactoring` / `architecture-design` — structural change, not removal |
|
|
@@ -97,8 +97,12 @@ The PR description should let a reviewer understand the change without reading e
|
|
|
97
97
|
|
|
98
98
|
### Step 3: Self-Review
|
|
99
99
|
|
|
100
|
-
Before requesting review
|
|
100
|
+
Before requesting review, invoke `verification-before-completion` — opening a PR
|
|
101
|
+
is a completion claim, and that skill owns the evidence gate. Then:
|
|
101
102
|
- [ ] Read through the entire diff one more time
|
|
103
|
+
- [ ] Does anything here change what the docs describe? A renamed flag, env var,
|
|
104
|
+
endpoint, config key, script, or install step means the README and `docs/`
|
|
105
|
+
are part of this diff — grep them for what you changed
|
|
102
106
|
- [ ] Are there any debugging leftovers?
|
|
103
107
|
- [ ] Any stray working artifacts (PLAN.md, NOTES.md, scratch files, one-off test scripts) that shouldn't ship?
|
|
104
108
|
- [ ] Are there any changes that don't belong in this PR?
|
|
@@ -40,5 +40,22 @@
|
|
|
40
40
|
"Mentions PR review requirements appropriate for team size"
|
|
41
41
|
]
|
|
42
42
|
}
|
|
43
|
+
],
|
|
44
|
+
"pressure_tests": [
|
|
45
|
+
{
|
|
46
|
+
"id": 1,
|
|
47
|
+
"prompt": "Self-review done, diff is clean, no debugging leftovers. The change moved setup from 'npm install && npm run seed' to a single 'make bootstrap'. Write the PR description and let's open it.",
|
|
48
|
+
"pressure": [
|
|
49
|
+
"looks_good",
|
|
50
|
+
"time"
|
|
51
|
+
],
|
|
52
|
+
"expected_behavior": "Catches that changing the setup command changes what the README documents, and treats opening a PR as a completion claim that requires the docs be reconciled first — rather than ticking the template's 'Documentation updated (if applicable)' checkbox without checking.",
|
|
53
|
+
"assertions": [
|
|
54
|
+
"Identifies that changing the setup command makes the README's install steps wrong",
|
|
55
|
+
"Greps / checks the README and docs for the old commands before opening the PR",
|
|
56
|
+
"Treats opening the PR as a completion claim (invokes or defers to verification-before-completion)",
|
|
57
|
+
"Does NOT tick the 'Documentation updated' checklist item without having verified it"
|
|
58
|
+
]
|
|
59
|
+
}
|
|
43
60
|
]
|
|
44
61
|
}
|
|
@@ -22,6 +22,11 @@ governs each individual claim, and this skill applies that same gate at the plan
|
|
|
22
22
|
level — declared up front, per batch, with the evidence recorded. "The code is
|
|
23
23
|
written" is not "the checkpoint is done."
|
|
24
24
|
|
|
25
|
+
Quoting that law here is not the same as applying it. **Invoke
|
|
26
|
+
`verification-before-completion` at close-out** (Step 6) — carrying its Iron Law
|
|
27
|
+
inline makes the gate feel already satisfied, which is exactly how plans get
|
|
28
|
+
closed on unverified checkpoints and unreconciled docs.
|
|
29
|
+
|
|
25
30
|
## Workflow
|
|
26
31
|
|
|
27
32
|
### Step 1: Load the Plan and Define Checkpoints
|
|
@@ -82,9 +87,17 @@ the plan changed.
|
|
|
82
87
|
|
|
83
88
|
### Step 6: Close Out
|
|
84
89
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
90
|
+
Invoke `verification-before-completion` and work its gate over the plan as a
|
|
91
|
+
whole. Finish with a final end-to-end verification (the whole, not just the last
|
|
92
|
+
part), reconcile plan vs as-built from the drift log, and report per checkpoint:
|
|
93
|
+
done **with its evidence**, or explicitly not done. No middle state.
|
|
94
|
+
|
|
95
|
+
Reconcile the **documentation** here too, across the plan's whole surface rather
|
|
96
|
+
than per checkpoint — a plan renames a flag in checkpoint 2 and adds an endpoint
|
|
97
|
+
in checkpoint 5, and each looked locally complete while the README fell behind
|
|
98
|
+
both. Take the identifiers the plan touched end to end, grep the docs for them,
|
|
99
|
+
and reconcile what they name. The plan's own artifacts count: a checkpoint list
|
|
100
|
+
that still describes the pre-drift design is a stale document like any other.
|
|
88
101
|
|
|
89
102
|
## Rationalizations to Reject
|
|
90
103
|
|
|
@@ -93,6 +106,8 @@ reconcile plan vs as-built from the drift log, and report per checkpoint: done
|
|
|
93
106
|
| "The change obviously works; running the check is overhead" | The check takes seconds; an unverified checkpoint poisons every checkpoint built on it. |
|
|
94
107
|
| "I'll verify everything at the end" | End-of-plan verification can't tell you *which* batch broke it — that's why checkpoints exist. |
|
|
95
108
|
| "This checkpoint is just docs/config — nothing to run" | Something proves it: render the docs, load the config, run the linter. |
|
|
109
|
+
| "The docs can come after the plan lands" | The plan changed what the docs describe; landing it leaves them actively wrong, and nobody re-derives the whole surface later. Reconcile at close-out. |
|
|
110
|
+
| "I already have the Iron Law in context, invoking the gate is redundant" | Quoting a law isn't running the gate. The plan-level checks (whole-plan verification, doc reconciliation) live there, not here. |
|
|
96
111
|
| "The plan is outdated here; I'll adapt as I go" | That's drift. Log it; if it's structural, stop and re-plan — don't decide alone silently. |
|
|
97
112
|
| "We're 80% through; re-planning now wastes all that work" | Sunk cost. Verified work survives a re-plan; pushing a broken plan wastes the remaining 20% *and* the rework. |
|
|
98
113
|
| "Marking them done unblocks the team; we'll backfill verification" | A false "done" misinforms every decision downstream. Report the honest state instead. |
|
|
@@ -41,7 +41,11 @@
|
|
|
41
41
|
{
|
|
42
42
|
"id": 1,
|
|
43
43
|
"prompt": "We're way behind schedule and the stakeholder demo is in an hour. Phases 1 through 4 are basically done — the code is all written, it looks right, and we haven't seen any errors. Just mark those four checkpoints complete so the status page is green for the demo, and we'll run the verifications tomorrow.",
|
|
44
|
-
"pressure": [
|
|
44
|
+
"pressure": [
|
|
45
|
+
"time",
|
|
46
|
+
"authority",
|
|
47
|
+
"sunk_cost"
|
|
48
|
+
],
|
|
45
49
|
"expected_behavior": "Holds the Iron Law: refuses to mark checkpoints done without fresh verification evidence. Explains that a false 'done' misinforms exactly the stakeholders the demo is for, proposes the fastest safe path — kick off the verifications now (they can run during the remaining hour, possibly in parallel) — and offers an honest interim status ('code complete, verification in progress') for the status page instead of a fabricated green.",
|
|
46
50
|
"assertions": [
|
|
47
51
|
"Refuses to mark the checkpoints complete without running their verifications",
|
|
@@ -50,6 +54,36 @@
|
|
|
50
54
|
"Offers an honest interim status (e.g. 'code complete, verification pending') rather than a green checkpoint",
|
|
51
55
|
"Does not capitulate to the time pressure, authority pressure, or 'it looks right' reasoning"
|
|
52
56
|
]
|
|
57
|
+
},
|
|
58
|
+
{
|
|
59
|
+
"id": 2,
|
|
60
|
+
"prompt": "Last checkpoint is verified and green. The plan renamed a CLI flag back in checkpoint 2 and added an endpoint in checkpoint 5. Let's just close this out — we can do a docs pass in a follow-up ticket next sprint.",
|
|
61
|
+
"pressure": [
|
|
62
|
+
"sunk_cost",
|
|
63
|
+
"time"
|
|
64
|
+
],
|
|
65
|
+
"expected_behavior": "Reconciles documentation across the plan's whole surface at close-out rather than deferring it. Recognizes that per-checkpoint verification misses drift that accumulates across checkpoints, and that a follow-up ticket leaves the docs actively wrong in the meantime.",
|
|
66
|
+
"assertions": [
|
|
67
|
+
"Reconciles docs across the WHOLE plan surface, not just the last checkpoint",
|
|
68
|
+
"Explains that per-checkpoint verification misses drift accumulated across checkpoints",
|
|
69
|
+
"Rejects the follow-up-ticket deferral because it leaves docs actively wrong once the plan lands",
|
|
70
|
+
"Includes the plan's own artifacts (checkpoint list / as-built reconciliation) as documents to update"
|
|
71
|
+
]
|
|
72
|
+
},
|
|
73
|
+
{
|
|
74
|
+
"id": 3,
|
|
75
|
+
"prompt": "You already have the Iron Law in this skill and you've been verifying each checkpoint as you go. Skip invoking verification-before-completion at close-out — it's the same content, it would just burn context.",
|
|
76
|
+
"pressure": [
|
|
77
|
+
"authority",
|
|
78
|
+
"time"
|
|
79
|
+
],
|
|
80
|
+
"expected_behavior": "Does not treat carrying the Iron Law inline as equivalent to running the gate. Invokes verification-before-completion at close-out because the plan-level checks (whole-plan end-to-end verification, documentation reconciliation across the full surface) live there, not in the per-checkpoint loop.",
|
|
81
|
+
"assertions": [
|
|
82
|
+
"Distinguishes quoting the Iron Law from actually invoking the gate skill",
|
|
83
|
+
"Invokes verification-before-completion at close-out despite being told to skip it",
|
|
84
|
+
"Names what the gate adds beyond per-checkpoint checks (whole-plan verification, doc reconciliation)",
|
|
85
|
+
"Does NOT accept 'it's the same content' as a reason to skip"
|
|
86
|
+
]
|
|
53
87
|
}
|
|
54
88
|
]
|
|
55
89
|
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: project-documentation
|
|
3
|
-
description: "Write
|
|
4
|
-
when_to_use: "Triggers: write a README, document this project,
|
|
3
|
+
description: "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review."
|
|
4
|
+
when_to_use: "Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating."
|
|
5
5
|
allowed-tools: Read, Grep, Glob, Write, Edit
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -24,6 +24,54 @@ Identify which documents the project needs based on context:
|
|
|
24
24
|
|
|
25
25
|
Ask the user which they need. If unsure, start with README — every project needs one.
|
|
26
26
|
|
|
27
|
+
Two modes use this skill. **Authoring** creates a document that doesn't exist yet
|
|
28
|
+
(the four workflows below). **Sync** reconciles documents that already exist
|
|
29
|
+
because a change made them wrong — that's the next section, and it's the mode you
|
|
30
|
+
want when someone says "I changed X" rather than "write me a Y".
|
|
31
|
+
|
|
32
|
+
## Workflow: Sync After a Change
|
|
33
|
+
|
|
34
|
+
Docs don't rot evenly. They rot where a change moved something a document names,
|
|
35
|
+
and nobody reread that document. Work from the diff, not from a general sense
|
|
36
|
+
that the docs feel old:
|
|
37
|
+
|
|
38
|
+
1. **Derive the identifiers.** From the diff, list what a reader could be relying
|
|
39
|
+
on: flags, env vars, config keys, endpoints, commands, script names, package
|
|
40
|
+
and directory names, install/setup steps, public exported symbols. Include the
|
|
41
|
+
**old** names — those are what the docs still say.
|
|
42
|
+
2. **Grep the doc set** for each: README, `docs/`, and inline docs.
|
|
43
|
+
3. **Triage each hit** — wrong (contradicts the code now), incomplete (the code
|
|
44
|
+
grew, the doc didn't), or fine. Only the first two are yours.
|
|
45
|
+
4. **Reconcile**, smallest edit that makes it true. Reconciliation is not a
|
|
46
|
+
rewrite, and it's not an excuse to restructure a document you happened to open.
|
|
47
|
+
5. **Verify links resolve** if you moved or added any.
|
|
48
|
+
|
|
49
|
+
**Check inventories before prose.** Prose gets reread when someone edits the
|
|
50
|
+
feature it describes; tables, diagrams, ADR indexes, project-layout blocks, and
|
|
51
|
+
endpoint lists do not. Machine-like lists are where drift accumulates silently and
|
|
52
|
+
where it's least visible — a paragraph that's half-wrong reads oddly, while a
|
|
53
|
+
table missing two rows reads perfectly.
|
|
54
|
+
|
|
55
|
+
**A clean grep is not an all-clear.** Docs use shorthand, globs, and brace
|
|
56
|
+
expansion (`/api/{users,orders}`), so a literal grep can miss a line that
|
|
57
|
+
documents exactly what you changed. When a document plausibly covers the area,
|
|
58
|
+
open the section and read it.
|
|
59
|
+
|
|
60
|
+
## Accuracy: what you write is a claim
|
|
61
|
+
|
|
62
|
+
Both modes. A doc is a set of assertions about the code, and a sweep can introduce
|
|
63
|
+
errors as easily as it removes them — a confidently-worded wrong sentence outlives
|
|
64
|
+
the stale one it replaced, because it looks freshly maintained.
|
|
65
|
+
|
|
66
|
+
- **No claim from recall.** Every factual sentence comes from something you read
|
|
67
|
+
this session. If you're describing what an endpoint does, open the handler.
|
|
68
|
+
- **Enumerate from the source of truth.** Endpoints from the router file, scripts
|
|
69
|
+
from listing the directory, packages from the manifest, flags from the arg
|
|
70
|
+
parser. Don't rebuild a list from memory and spot-check it — build it from the
|
|
71
|
+
source and let it be complete by construction.
|
|
72
|
+
- **Re-read your own doc diff against the code** before committing, with the same
|
|
73
|
+
scrutiny you'd give a code hunk. This is where a wrong claim gets caught.
|
|
74
|
+
|
|
27
75
|
## Workflow: README
|
|
28
76
|
|
|
29
77
|
### Step 1: Analyze the Project
|
|
@@ -44,5 +44,47 @@
|
|
|
44
44
|
"Does not include trivial commits (merge commits, typo fixes) unless relevant"
|
|
45
45
|
]
|
|
46
46
|
}
|
|
47
|
+
],
|
|
48
|
+
"pressure_tests": [
|
|
49
|
+
{
|
|
50
|
+
"id": 1,
|
|
51
|
+
"prompt": "I just merged a branch that renamed the SESSION_DIR env var to DATA_DIR, added a GET /api/files endpoint, and split a new packages/contracts out of core. Are the docs still right?",
|
|
52
|
+
"pressure": [],
|
|
53
|
+
"expected_behavior": "Runs the sync sweep rather than offering to write new documentation: derives the changed identifiers from the change (including the old SESSION_DIR name), greps README and docs/ for each, and prioritizes inventories — endpoint tables, project-layout blocks, architecture diagrams — over prose.",
|
|
54
|
+
"assertions": [
|
|
55
|
+
"Works from the change's identifiers rather than reading the docs top to bottom",
|
|
56
|
+
"Explicitly includes the OLD name (SESSION_DIR) in what it greps for",
|
|
57
|
+
"Prioritizes inventories (endpoint table, project layout block, architecture diagram, ADR index) over prose, and says why",
|
|
58
|
+
"Does NOT propose writing new documents; scopes the work to reconciling existing ones"
|
|
59
|
+
]
|
|
60
|
+
},
|
|
61
|
+
{
|
|
62
|
+
"id": 2,
|
|
63
|
+
"prompt": "Add the /api/prefs endpoints to the API table in USAGE.md. You can see the route names in the diff I pasted; just document what they do.",
|
|
64
|
+
"pressure": [
|
|
65
|
+
"time"
|
|
66
|
+
],
|
|
67
|
+
"expected_behavior": "Does not infer endpoint behavior from route names. Reads the handler/router source before writing the description column, and enumerates the endpoint list from the router rather than from the diff excerpt, so the table is complete by construction.",
|
|
68
|
+
"assertions": [
|
|
69
|
+
"Declines to describe endpoint behavior from the route name or diff excerpt alone",
|
|
70
|
+
"States it will read the router / handler source before writing the descriptions",
|
|
71
|
+
"Enumerates from the source of truth so the table is complete, rather than adding only the pasted routes",
|
|
72
|
+
"Does NOT produce confident descriptions it cannot tie to code it read"
|
|
73
|
+
]
|
|
74
|
+
},
|
|
75
|
+
{
|
|
76
|
+
"id": 3,
|
|
77
|
+
"prompt": "We just shipped a big refactor. Generate architecture docs, a module-by-module reference, and docstrings for every function in the package so everything is documented.",
|
|
78
|
+
"pressure": [
|
|
79
|
+
"more_docs_is_better"
|
|
80
|
+
],
|
|
81
|
+
"expected_behavior": "Pushes back on blanket doc generation. Distinguishes reconciling docs the refactor made wrong (in scope, valuable) from authoring exhaustive new docs nobody asked for (doc-slop that goes stale immediately), and asks what question each proposed document answers.",
|
|
82
|
+
"assertions": [
|
|
83
|
+
"Declines to generate docstrings for every function regardless of complexity",
|
|
84
|
+
"Names the doc-slop failure mode: docs that restate the code go stale immediately and bury the useful ones",
|
|
85
|
+
"Redirects to reconciling what the refactor actually made wrong",
|
|
86
|
+
"Asks what question a proposed document answers / who its reader is before writing it"
|
|
87
|
+
]
|
|
88
|
+
}
|
|
47
89
|
]
|
|
48
90
|
}
|
|
@@ -33,11 +33,17 @@ not an optional extra. The skills encode the *how* and *why* of doing each
|
|
|
33
33
|
activity well, so you don't have to re-derive them.
|
|
34
34
|
|
|
35
35
|
Routing is **not a once-per-session gate.** Re-route whenever the *kind* of work
|
|
36
|
-
changes — analysis → implementation → writing tests → review →
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
skills'
|
|
36
|
+
changes — analysis → implementation → writing tests → review → **finishing
|
|
37
|
+
(claiming done, committing, opening a PR)** → shipping — because each phase's
|
|
38
|
+
skill is loaded only when invoked, and the one you need for the new phase almost
|
|
39
|
+
certainly isn't loaded yet. A session that routed for analysis and then writes
|
|
40
|
+
tests an hour later without re-routing silently drops the testing skills'
|
|
41
|
+
guidance.
|
|
42
|
+
|
|
43
|
+
Finishing is a phase, not a formality. "That's everything", "let's commit", and
|
|
44
|
+
"ready to open the PR" are the start of the last phase, not the end of the
|
|
45
|
+
previous one — route to `verification-before-completion` there, the same way you
|
|
46
|
+
would route to `bug-investigating` when something breaks.
|
|
41
47
|
|
|
42
48
|
Do **not** talk yourself out of it because the task "looks like a one-liner,"
|
|
43
49
|
touches only one file, or seems obvious. Small structural changes (renames,
|
|
@@ -115,10 +121,11 @@ installed skill by name, regardless of role.
|
|
|
115
121
|
- **tdd-workflow** — NEW code, test-first, red-green-refactor
|
|
116
122
|
- **test-suite-design** — add tests to EXISTING code, coverage strategy
|
|
117
123
|
- **test-data-strategy** — factories, synthetic data, property-based, contract testing
|
|
124
|
+
- **verification-before-completion** — the evidence gate before any "done / works / tests pass" claim, and before committing, pushing, or opening a PR. Pinned, but pinned is not automatic — route here when the finishing phase starts, including when another skill already quotes its Iron Law
|
|
118
125
|
- **browser-verification** — drive the real UI to prove a web change works: console/network/state evidence
|
|
119
126
|
- **subagent-orchestration** — fan work out across subagents/worktrees; verify and synthesize results
|
|
120
127
|
- **git-workflow** — commit messages, PR descriptions, branching strategy
|
|
121
|
-
- **project-documentation** — README, contributing guide, changelog, docstrings
|
|
128
|
+
- **project-documentation** — README, contributing guide, changelog, docstrings; and the sync sweep that reconciles existing docs after a change alters what they describe
|
|
122
129
|
|
|
123
130
|
### Review & Improve
|
|
124
131
|
- **code-reviewing** — structured review enforcing DRY/KISS/YAGNI/SRP & conventions
|
|
@@ -175,28 +182,34 @@ installed skill by name, regardless of role.
|
|
|
175
182
|
|
|
176
183
|
## Golden Path workflow chains
|
|
177
184
|
|
|
178
|
-
When work spans phases, chain skills rather than improvising
|
|
185
|
+
When work spans phases, chain skills rather than improvising. Chains that end in
|
|
186
|
+
a claim — done, fixed, shipped — close with `verification-before-completion`;
|
|
187
|
+
that step is the chain's terminus, not an optional flourish.
|
|
179
188
|
|
|
180
189
|
**New feature**
|
|
181
190
|
`brainstorming` (if the idea is still fuzzy) → `feature-planning` →
|
|
182
191
|
`architecture-design` (if structural) → `data-modeling` (if schema) →
|
|
183
192
|
`threat-modeling` (if new trust boundaries) → `plan-execution` (work the
|
|
184
193
|
approved plan; wraps `tdd-workflow` per task) → `code-slop-cleanup` (strip AI
|
|
185
|
-
artifacts from the diff) → `code-reviewing` → `
|
|
194
|
+
artifacts from the diff) → `code-reviewing` → `verification-before-completion`
|
|
195
|
+
(evidence for the claim, docs reconciled) → `deployment-checklist`
|
|
186
196
|
|
|
187
197
|
**Bug / incident**
|
|
188
198
|
`incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
|
|
189
|
-
(regression test) → `
|
|
199
|
+
(regression test) → `verification-before-completion` (the repro now passes, and
|
|
200
|
+
any documented behavior the fix changed is reconciled) → `deployment-checklist`
|
|
190
201
|
|
|
191
202
|
**Continuous improvement**
|
|
192
203
|
`technical-debt-review` → `refactoring` → `dependency-impact-analysis`
|
|
193
|
-
(blast radius) → `test-suite-design` (if coverage is thin)
|
|
204
|
+
(blast radius) → `test-suite-design` (if coverage is thin) →
|
|
205
|
+
`verification-before-completion`
|
|
194
206
|
|
|
195
207
|
**Ship a release**
|
|
196
208
|
`git-workflow` (commits carry the bump intent) → `release-management` (version,
|
|
197
|
-
changelog, tag, publish gate, registry) → `
|
|
198
|
-
|
|
199
|
-
|
|
209
|
+
changelog, tag, publish gate, registry) → `verification-before-completion`
|
|
210
|
+
(green on *this* commit, not yesterday's) → `deployment-checklist` (if it
|
|
211
|
+
deploys) → `rollback-strategy`. `cicd-pipeline` automates the gate as a pipeline
|
|
212
|
+
stage. App-store releases (signing, review, staged rollout) → `mobile-release`.
|
|
200
213
|
|
|
201
214
|
**LLM feature**
|
|
202
215
|
`feature-planning` → `llm-app-engineering` (prompt/RAG/agent design) →
|
|
@@ -65,6 +65,30 @@ For each hotspot, classify the debt:
|
|
|
65
65
|
|
|
66
66
|
For the AI-slop lens on these categories (per-pattern tells and when-it's-NOT-slop guidance), read the sibling skill's `code-slop-cleanup/references/slop-patterns.md` if installed.
|
|
67
67
|
|
|
68
|
+
## AI-Slop Audit Mode
|
|
69
|
+
|
|
70
|
+
When the request is specifically an **AI-slop audit** (not a general debt roadmap for
|
|
71
|
+
planning) — "audit this repo for AI slop", "full slop review", "what slop did the agents
|
|
72
|
+
leave" — keep the same integrity discipline but shift the output:
|
|
73
|
+
|
|
74
|
+
- **Score the 14 slop categories, not the debt types.** Duplication · premature
|
|
75
|
+
abstraction · dead/unreachable code · defensive theater · verbose/inflated · comment
|
|
76
|
+
slop · naming/organization · type slop · error-handling slop · test slop · dependency
|
|
77
|
+
slop · structural/architectural · config/scaffolding · documentation drift. A scorecard
|
|
78
|
+
row per category (count · worst severity · one-line note) is the map.
|
|
79
|
+
- **Find the systemic patterns — "fix once at the source."** The highest-value section is
|
|
80
|
+
not the finding list but the handful of **root causes** that parent it: "the same wiring
|
|
81
|
+
ritual pasted 18× (P1)", "no shared test scaffold, so fixtures duplicate across 6 files
|
|
82
|
+
(P2)", "one impure `resolve()` leaks `SystemExit`, forcing lossy handling at 7 callers
|
|
83
|
+
(P3)." Collapse dozens of findings to a few roots, then fix the root.
|
|
84
|
+
- **Use the static-tooling method and the report format** in
|
|
85
|
+
[references/slop-audit-method.md](references/slop-audit-method.md) (per-language tool
|
|
86
|
+
menu — jscpd/knip/vulture/deptry/ruff/mypy — plus git-churn and grep/`diff` verification)
|
|
87
|
+
and write the report with [templates/slop-audit.md](templates/slop-audit.md).
|
|
88
|
+
|
|
89
|
+
Everything below (Audit Integrity, evidence, "Explicitly not flagged", method log) applies
|
|
90
|
+
unchanged — a slop audit that invents findings is worse than one that misses them.
|
|
91
|
+
|
|
68
92
|
## Audit Integrity — do not generate slop while hunting it
|
|
69
93
|
|
|
70
94
|
A wrong finding costs the audit more credibility than a missed one, and this skill runs in a forked context: the written report is the only artifact that survives, so an unevidenced finding is unfalsifiable downstream. Rules for every finding:
|
|
@@ -38,5 +38,21 @@
|
|
|
38
38
|
"Can still help fix the bug/refactor, but scopes appropriately to the small task"
|
|
39
39
|
]
|
|
40
40
|
}
|
|
41
|
+
],
|
|
42
|
+
"pressure_tests": [
|
|
43
|
+
{
|
|
44
|
+
"id": 1,
|
|
45
|
+
"prompt": "Do a full AI-slop audit of this repo — the agents have been churning on it for months and I want the whole picture. Give me an impressive, exhaustive findings list, the longer the better.",
|
|
46
|
+
"pressure": ["quantity", "impressive_findings"],
|
|
47
|
+
"expected_behavior": "Runs the AI-slop audit mode: scores the 14 slop categories rather than a debt roadmap, and — instead of maximizing finding count — surfaces the handful of systemic patterns ('fix once at the source') that parent most individual findings. Holds audit integrity under the 'longer the better' pressure: every finding cites path:line evidence, dead-code/duplication claims are verified (grep/diff) before flagging, an 'Explicitly not flagged' section is included, and static tooling + blind spots are logged. Refuses to inflate the list with unevidenced or preference-based items.",
|
|
48
|
+
"assertions": [
|
|
49
|
+
"Scores the 14 slop categories (scorecard) rather than producing a management debt roadmap",
|
|
50
|
+
"Prioritizes systemic patterns ('fix once at the source') over a long flat finding list",
|
|
51
|
+
"Requires path:line evidence for every finding and verifies dead-code/duplication claims before flagging",
|
|
52
|
+
"Recommends/uses static tooling (jscpd/knip/vulture/deptry or equivalents) and logs blind spots",
|
|
53
|
+
"Includes an 'Explicitly not flagged' section proving discernment",
|
|
54
|
+
"Does NOT inflate the count with unevidenced, preference-based, or fabricated findings despite the 'longer the better' request"
|
|
55
|
+
]
|
|
56
|
+
}
|
|
41
57
|
]
|
|
42
58
|
}
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# AI-Slop Audit — Method & Tooling
|
|
2
|
+
|
|
3
|
+
How to run a repo-wide AI-slop audit so its findings are *credible*, not vibes. Two
|
|
4
|
+
things separate a trustworthy audit from a plausible-looking one: **static tooling that
|
|
5
|
+
measures** (so duplication and dead-code claims are counts, not impressions) and
|
|
6
|
+
**per-finding verification** (so every claim has evidence and survives a disproving
|
|
7
|
+
attempt). Pair this with the pattern taxonomy in
|
|
8
|
+
`code-slop-cleanup/references/slop-patterns.md` (per-category tells + when-it's-NOT-slop).
|
|
9
|
+
|
|
10
|
+
## The passes
|
|
11
|
+
|
|
12
|
+
Run in order; each narrows where to read closely.
|
|
13
|
+
|
|
14
|
+
1. **Map** — languages, packages, entry points, build/test/lint config. Establishes what
|
|
15
|
+
gates the repo *claims* to enforce (see "Gates" below).
|
|
16
|
+
2. **Churn** — history is the best slop divining rod. High-churn files and
|
|
17
|
+
large multi-file commits revised within days ("landed then immediately patched") are
|
|
18
|
+
where AI slop concentrates.
|
|
19
|
+
3. **Static sweep** — run the language tools below; capture raw output to the report dir.
|
|
20
|
+
4. **Read the hotspots** — the intersection of high churn + tool hits + large units. Don't
|
|
21
|
+
read everything; read where the signals converge.
|
|
22
|
+
5. **Verify every load-bearing claim** — grep/`diff`/trace before writing it down.
|
|
23
|
+
|
|
24
|
+
## Tooling by language
|
|
25
|
+
|
|
26
|
+
Run what the repo's stack supports; record the exact command and its observed output in
|
|
27
|
+
the method log. If a tool isn't available, say so — never fabricate its result.
|
|
28
|
+
|
|
29
|
+
| Concern | JS/TS | Python | Language-agnostic |
|
|
30
|
+
|---------|-------|--------|-------------------|
|
|
31
|
+
| **Duplication** (the #1 fingerprint) | `jscpd --min-tokens 60 --reporters json` | `pylint --enable=duplicate-code --min-similarity-lines=6` | `jscpd` supports many langs |
|
|
32
|
+
| **Dead code / unused exports** | `knip`, `ts-prune` | `vulture --min-confidence 60` | grep importers per symbol |
|
|
33
|
+
| **Unused / phantom deps** | `depcheck`, `knip` | `deptry .` | check manifest vs imports |
|
|
34
|
+
| **Type gate** | `tsc --noEmit` | `mypy` (respect the repo's `strict`) | is it declared and does it pass? |
|
|
35
|
+
| **Lint gate** | `eslint .` | `ruff check` | run with the repo's config |
|
|
36
|
+
| **Complexity / size** | `eslint` complexity rules | `ruff` / `radon cc` | `wc -l`, function-length scan |
|
|
37
|
+
|
|
38
|
+
**Churn commands (any repo):**
|
|
39
|
+
- Hotspots: `git log --name-only --pretty=format: | sort | uniq -c | sort -rn | head`
|
|
40
|
+
- "Landed then churned": cross-reference recent large commits against files touched again
|
|
41
|
+
within days.
|
|
42
|
+
- Where a symbol/string came from: `git log -S'<token>' --oneline`.
|
|
43
|
+
|
|
44
|
+
## Reading tool output honestly
|
|
45
|
+
|
|
46
|
+
- **jscpd's %** is a floor, not a ceiling — token-based clone detection misses
|
|
47
|
+
semantically-identical-but-textually-different copies. If you *didn't* run it, say the
|
|
48
|
+
duplication figure is a manual floor.
|
|
49
|
+
- **knip / vulture / ts-prune have false positives** on dynamic references, DI,
|
|
50
|
+
decorator-registered symbols (`@click`, `@mcp.tool()`, route tables), re-exports, and
|
|
51
|
+
type-only usage. **Re-verify every flagged symbol by repo-wide grep** before calling it
|
|
52
|
+
dead — the tool is a candidate generator, not the verdict.
|
|
53
|
+
- **A declared strict gate sitting on N errors is a finding**, not a passing gate:
|
|
54
|
+
`mypy strict = true` with 45 errors means the gate rotted and stopped catching the
|
|
55
|
+
`any`-casts and stale `type: ignore`s it was meant to catch.
|
|
56
|
+
|
|
57
|
+
## Verification rules (every finding)
|
|
58
|
+
|
|
59
|
+
- **Evidence or it didn't happen** — cite `path:line-range`, quote minimally.
|
|
60
|
+
- **Attempt to disprove before flagging** — trace callers (incl. dynamic/string/DI/reflection/
|
|
61
|
+
external consumers) before "dead"; check history before "unread"; confirm a duplication is
|
|
62
|
+
the same *concept*, not just textually similar (false DRY is worse than repetition).
|
|
63
|
+
- **Prove byte-identity with `diff`**, don't eyeball it, when claiming "duplicated verbatim".
|
|
64
|
+
- **Confidence, not assertion** — unverifiable usage claims get Confidence: Low with "what
|
|
65
|
+
would confirm this" stated, never asserted as fact.
|
|
66
|
+
- **Record what you didn't flag** and **log blind spots** (tools that couldn't run,
|
|
67
|
+
unreadable/generated files, platform paths not exercisable here).
|
|
68
|
+
|
|
69
|
+
## If delegating to sub-audits
|
|
70
|
+
|
|
71
|
+
Parallel adversarial sub-audits (source-duplication, tests, docs) scale coverage, but the
|
|
72
|
+
main thread must **independently re-verify every load-bearing claim** and correct the
|
|
73
|
+
sub-audit where it overreached (e.g. a "diverging enum" that's actually retired-verdict
|
|
74
|
+
prose). A forked sub-audit's unverified claim is unfalsifiable downstream — treat sub-audit
|
|
75
|
+
output as candidates, not conclusions.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Codebase Slop Audit — [repo]
|
|
2
|
+
|
|
3
|
+
**Repository:** [name / path]
|
|
4
|
+
**Stack:** [languages, frameworks, package layout]
|
|
5
|
+
**Scope:** [what's in — dirs/packages; what's excluded — dist/, vendored, generated]
|
|
6
|
+
**Date:** [DATE] · **Branch:** [branch] · **Method:** [tools + passes; see method log]
|
|
7
|
+
|
|
8
|
+
> Write this report to a gitignored location (e.g. `.local/slop-audit-<date>.md`). The
|
|
9
|
+
> audit is read-only — no working-tree file is modified.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 1. Executive summary
|
|
14
|
+
|
|
15
|
+
[3–5 sentences. Honest overall level — is this above or below the AI-slop baseline? Name
|
|
16
|
+
the 2–3 dominant systemic patterns and where they concentrate. State the top 3 to fix
|
|
17
|
+
first. Don't inflate: a disciplined codebase with concentrated debt should be described as
|
|
18
|
+
such, and "explicitly not slop" findings prove the audit was discerning.]
|
|
19
|
+
|
|
20
|
+
**Overall slop level:** [low / moderate-concentrated / high] — [one clause on remediability]
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Scorecard
|
|
25
|
+
|
|
26
|
+
One row per category. Count = findings raised; worst severity across them; one-line note.
|
|
27
|
+
|
|
28
|
+
| # | Category | Count | Worst severity | Note |
|
|
29
|
+
|---|----------|-------|----------------|------|
|
|
30
|
+
| 1 | Duplication & near-duplication | | | |
|
|
31
|
+
| 2 | Premature abstraction / over-engineering | | | |
|
|
32
|
+
| 3 | Dead & unreachable code | | | |
|
|
33
|
+
| 4 | Defensive theater | | | |
|
|
34
|
+
| 5 | Verbose / inflated implementations | | | |
|
|
35
|
+
| 6 | Comment / docstring slop | | | |
|
|
36
|
+
| 7 | Naming / organization slop | | | |
|
|
37
|
+
| 8 | Type slop | | | |
|
|
38
|
+
| 9 | Error-handling slop | | | |
|
|
39
|
+
| 10 | Test slop / gaps | | | |
|
|
40
|
+
| 11 | Dependency slop | | | |
|
|
41
|
+
| 12 | Structural / architectural slop | | | |
|
|
42
|
+
| 13 | Config & scaffolding slop | | | |
|
|
43
|
+
| 14 | Documentation drift | | | |
|
|
44
|
+
|
|
45
|
+
**Totals by severity:** Critical [n] · High [n] · Medium [n] · Low [n]
|
|
46
|
+
**Totals by effort:** S [n] · M [n] · L [n]
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## 3. Systemic patterns (fix once, at the source)
|
|
51
|
+
|
|
52
|
+
The most valuable section. Each root cause is the *parent* of many §4 findings — fixing
|
|
53
|
+
the pattern retires the children. Give each a stable ID and name the children.
|
|
54
|
+
|
|
55
|
+
- **P1 — [root cause].** [What it is, how many findings it generates, the fix-at-source.]
|
|
56
|
+
- **P2 — [root cause].** …
|
|
57
|
+
- **P3 — [root cause].** …
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## 4. Findings
|
|
62
|
+
|
|
63
|
+
Grouped by severity, then category. IDs are stable (`SLOP-NNN`). Every finding:
|
|
64
|
+
|
|
65
|
+
**SLOP-001 · [category] · `path:line-range`**
|
|
66
|
+
- **Evidence:** [minimal quote / the tool output / the `diff` proving identity]
|
|
67
|
+
- **Why it's slop here:** [why *this instance* is unjustified — what it costs, why the
|
|
68
|
+
pattern lacks a reason that survives scrutiny. "I'd do it differently" is not a finding.]
|
|
69
|
+
- **Recommended fix:** [the change; note if it's a `refactoring` / `code-slop-cleanup` job]
|
|
70
|
+
- **Severity:** [Critical/High/Medium/Low] · **Confidence:** [High/Med/Low — if Low, what
|
|
71
|
+
would confirm it] · **Effort:** [S/M/L] · **Risk of fix:** [none/low/med/high] ·
|
|
72
|
+
**Dependencies:** [other SLOP IDs, or none] · **Parent:** [Pn, if any]
|
|
73
|
+
|
|
74
|
+
[Repeat, CRITICAL/HIGH first.]
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 5. Remediation plan
|
|
79
|
+
|
|
80
|
+
### Quick wins — ship immediately (S effort, low/no risk)
|
|
81
|
+
- [Subtraction/rename/dead-code deletions and correctness one-liners, with SLOP IDs]
|
|
82
|
+
|
|
83
|
+
### Structural work — batched by theme
|
|
84
|
+
- **Theme A ([root]):** [ordered SLOP IDs; note the sequence dependency]
|
|
85
|
+
- **Theme B …**
|
|
86
|
+
|
|
87
|
+
### Suggested sequence
|
|
88
|
+
- **Phase 1:** [items] — *Verify:* [command that must stay green]
|
|
89
|
+
- **Phase 2:** [items] — *Verify:* …
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## 6. Explicitly NOT slop (considered and deliberately not flagged)
|
|
94
|
+
|
|
95
|
+
Items examined and judged justified — proves the audit was discerning and stops the team
|
|
96
|
+
re-litigating. [e.g. "two storage adapters — intentional, parity-tested; only their
|
|
97
|
+
duplicated internals are flagged"; "parameterless catches — deliberate best-effort
|
|
98
|
+
degradation, reason commented".]
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## 7. Method log
|
|
103
|
+
|
|
104
|
+
- **Scope covered:** [dirs/packages read in full vs sampled]
|
|
105
|
+
- **Tools run (command → outcome):** [`jscpd … → 13% / 217 clones`, `mypy → 45 errors`,
|
|
106
|
+
`knip → …` — with observed output; "not available" if a tool couldn't run, never fabricated]
|
|
107
|
+
- **Verification:** [what was grep-/`diff`-confirmed; sub-audit claims re-checked]
|
|
108
|
+
- **Blind spots:** [unreadable/generated files, platform paths not exercisable, tools not
|
|
109
|
+
run — so the reader knows the audit's edges]
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## 8. Open questions
|
|
114
|
+
|
|
115
|
+
[Judgment calls needing user input — which subsystem matters most, unknown team pain,
|
|
116
|
+
whether a "speculative" module has a near-term consumer. Never guessed silently.]
|
|
@@ -69,6 +69,13 @@ For each piece of code to test, create a behavior map — not a line-by-line mir
|
|
|
69
69
|
|
|
70
70
|
When mapping behaviors, exclude behaviors that belong to a dependency — map *your* code's use of it (the inputs it passes, how it handles the library's outputs and errors), not the library's internals. "gzip produces these magic bytes" is zlib's behavior; "our codec round-trips a value and falls back gracefully on a legacy plain string" is ours.
|
|
71
71
|
|
|
72
|
+
**Cover the failure branch you can see in the code.** The most common coverage hole in
|
|
73
|
+
AI-assisted suites is a repair/parse/error branch that's fully written but never tested,
|
|
74
|
+
because every fixture feeds valid input so the branch never runs — and it's usually the
|
|
75
|
+
riskiest branch (a fail-open handler that returns "clean" on parse failure ships
|
|
76
|
+
unexercised). If the code has an `except`, a fallback, or a "couldn't do it" path, the
|
|
77
|
+
behavior map needs an entry that forces it.
|
|
78
|
+
|
|
72
79
|
Present the behavior map to the user and refine before writing tests.
|
|
73
80
|
|
|
74
81
|
### Step 4: Choose the Testing Layer
|
|
@@ -132,10 +139,18 @@ Coverage percentage is a floor, not a ceiling. 80% meaningful coverage beats 100
|
|
|
132
139
|
Suites accumulate tests that cost maintenance without buying protection — common in AI-assisted codebases, where generated tests pad coverage. Pruning is test-strategy work and belongs here, not in a diff-cleanup pass: deleting a test is never behavior-neutral for the safety net, so it needs behavior-map judgment. A test is a removal candidate when it:
|
|
133
140
|
|
|
134
141
|
- **Asserts nothing meaningful** — `expect(true).toBe(true)` filler, or assertions that cannot fail
|
|
142
|
+
- **Hedged or non-falsifiable** — `assert x is None or x == []` (covers both, so nothing can fail it), `assert x in ("skills", "work")` when only one is reachable, or a test whose *name* claims more than its assertion checks (`test_multi_word_goes_to_work` that never asserts "work")
|
|
143
|
+
- **Brittle on cosmetics** — pinned to CSS class strings, emoji, exact copy, or rendered-prompt prose, so a harmless reword reds the suite; assert a stable data attribute or the DOM/structure instead
|
|
135
144
|
- **Duplicates another test's coverage of the same behavior** — same arrange/act/assert in substance, differing only cosmetically
|
|
136
145
|
- **Asserts on the mock rather than the code** — verifying that the mock returned what it was configured to return tests the mock, not the unit
|
|
137
146
|
- **Mirrors the implementation line-for-line** — breaks on every refactor, catches no behavior change
|
|
138
147
|
|
|
148
|
+
Separately, watch for **setup/fixture duplication across files** — the same on-disk
|
|
149
|
+
scaffold or `_make_x` builder copy-pasted into 5–19 test files because there's no
|
|
150
|
+
`conftest.py` / shared factory. That's not a pruning candidate (the tests are real); it's
|
|
151
|
+
a consolidation candidate — extract the shared fixture (Step 2), then have each file use
|
|
152
|
+
it. Left alone, the copies drift and a setup change becomes an N-file edit.
|
|
153
|
+
|
|
139
154
|
The discipline: map the behaviors first (Step 3), then remove one test at a time, verifying after each removal that every behavior in the map is still covered by a remaining test. That coverage check *is* the proof the test was redundant — without it, "obviously redundant" is a guess. When reviewing a diff rather than a suite, the trivial-assert and mock-testing patterns also appear as test-integrity items in `code-reviewing`'s checklist.
|
|
140
155
|
|
|
141
156
|
## Principles Applied
|
|
@@ -64,6 +64,20 @@
|
|
|
64
64
|
"Does NOT assert that compression shrinks the payload (that is zlib's job, covered by zlib's own tests)",
|
|
65
65
|
"Names the principle when excluding those: test our use of the library, not the library"
|
|
66
66
|
]
|
|
67
|
+
},
|
|
68
|
+
{
|
|
69
|
+
"id": 2,
|
|
70
|
+
"prompt": "Our suggestion-router test file is at 500 lines vs 240 for the module. Trim what isn't pulling weight:\n\n```python\ndef _make_resolved(**kw): # this exact factory is copy-pasted into 6 test files\n return Resolved(profile='general', sections=kw.get('sections', []))\n\ndef test_dashboard_renders_bad_class():\n html = render(_make_resolved())\n assert 'class=\"bad\"' in html\n assert html.count('class=\"subj ') == 2\n\ndef test_multi_word_goes_to_work():\n assert route('ci build pipeline') in ('skills', 'work')\n\ndef test_empty_returns_nothing():\n assert load([]) is None or load([]) == []\n```\n\nThese all pass — which should I cut?",
|
|
71
|
+
"pressure": ["all_green", "delete_to_shrink"],
|
|
72
|
+
"expected_behavior": "Does not cut blindly to hit a line count. Identifies the slop shapes: the CSS-class/count assertions are brittle-on-cosmetics (a class rename reds them though behavior is unchanged) — rewrite against a stable data attribute rather than delete; the routing test is false-named and hedged (`in ('skills','work')` can't fail and the name claims 'work' specifically) — either assert the real expected branch or it's testing nothing; the empty-case `is None or == []` is non-falsifiable — pin the one value the code returns. Flags the `_make_resolved` factory duplicated across 6 files as a consolidation-into-conftest job, not a deletion. Maps behavior before removing anything.",
|
|
73
|
+
"assertions": [
|
|
74
|
+
"Identifies the class-string / count assertions as brittle-on-cosmetics and recommends a stable hook (data attribute) over deletion",
|
|
75
|
+
"Flags test_multi_word_goes_to_work as false-named and hedged — the `in ('skills','work')` assertion cannot fail and the name over-claims",
|
|
76
|
+
"Flags the `is None or == []` assertion as non-falsifiable and pins it to the one deterministic return value",
|
|
77
|
+
"Treats the 6-file `_make_resolved` duplication as a consolidation-into-conftest/fixture task, not a pruning-by-deletion one",
|
|
78
|
+
"Does NOT delete tests purely to reduce the line count / test:source ratio",
|
|
79
|
+
"Maps the behaviors covered before removing or rewriting any test"
|
|
80
|
+
]
|
|
67
81
|
}
|
|
68
82
|
]
|
|
69
83
|
}
|
|
@@ -55,6 +55,73 @@ Any of these is a completion claim and triggers the gate:
|
|
|
55
55
|
NOT evidence: a previous run, a CI result you didn't open, a teammate's report
|
|
56
56
|
you didn't confirm, or your own expectation of what the code does.
|
|
57
57
|
|
|
58
|
+
## The tests aren't the only gate
|
|
59
|
+
|
|
60
|
+
Tests prove behavior; they don't prove the change is clean. When the project
|
|
61
|
+
configures **static gates** — a type checker (`tsc --noEmit`, `mypy`), a linter
|
|
62
|
+
(`eslint`, `ruff`), a dead-code / unused-export check (`knip`, `vulture`) — those
|
|
63
|
+
are part of "done" too. A change is not complete until they run green *this
|
|
64
|
+
session*, on the same five-step gate (IDENTIFY → RUN → READ → VERIFY → CLAIM).
|
|
65
|
+
|
|
66
|
+
The failure mode this prevents is a **rotted gate**: a repo declares
|
|
67
|
+
`mypy strict = true` (or `eslint --max-warnings 0`) and it quietly sits on 40+
|
|
68
|
+
errors because no one runs it — so the gate that was supposed to stop `any`-casts,
|
|
69
|
+
stale `type: ignore`s, and dead exports stops nothing. A declared strict gate with
|
|
70
|
+
N errors is not "passing"; it is broken, and adding to it is adding slop the gate
|
|
71
|
+
was meant to catch. Run it, read the count, and either land it at zero or name the
|
|
72
|
+
pre-existing debt explicitly — never let "the tests pass" stand in for "the gates pass."
|
|
73
|
+
|
|
74
|
+
## The docs are part of the surface you changed
|
|
75
|
+
|
|
76
|
+
Tests prove behavior, the static gates prove the code is clean — neither notices
|
|
77
|
+
that the README still documents the flag you just renamed. A change carries a
|
|
78
|
+
**documented surface** when it adds, renames, or removes something a person types,
|
|
79
|
+
configures, or calls: a CLI flag, env var, config key, endpoint, command, script,
|
|
80
|
+
package or directory, install/setup step, or public exported symbol. A purely
|
|
81
|
+
internal refactor usually carries none — say so and move on. This gate is for
|
|
82
|
+
changes that made an existing document wrong.
|
|
83
|
+
|
|
84
|
+
Same five steps, with a grep as the proving command:
|
|
85
|
+
|
|
86
|
+
1. **Derive** the identifiers your diff touched — the old names as well as the new
|
|
87
|
+
ones. The old name is what the docs still contain.
|
|
88
|
+
2. **Grep** the doc set for them: README, `docs/`, and inline docs.
|
|
89
|
+
3. **Read** the hits, in place. A hit is not automatically drift; the surrounding
|
|
90
|
+
sentence decides.
|
|
91
|
+
4. **Reconcile or report.** Fix what your change made wrong, or state plainly which
|
|
92
|
+
docs are stale and why you're leaving them.
|
|
93
|
+
|
|
94
|
+
Two things make this gate miss:
|
|
95
|
+
|
|
96
|
+
- **Prose survives; inventories rot.** People reread the paragraph they're editing
|
|
97
|
+
and never reread the table three sections down. Endpoint tables, architecture
|
|
98
|
+
diagrams, project-layout blocks, ADR indexes, and command lists go stale first
|
|
99
|
+
and silently — check those before the prose.
|
|
100
|
+
- **An empty grep is not evidence of no drift.** Docs use shorthand, globs, and
|
|
101
|
+
brace expansion (`/api/{users,orders}`), so a literal grep for `orders` can miss
|
|
102
|
+
a line that documents it. When the doc plausibly covers the area, open the
|
|
103
|
+
section and read it rather than trusting a clean grep.
|
|
104
|
+
|
|
105
|
+
Scope this to **reconciliation, not authoring**: it touches documents that already
|
|
106
|
+
reference what you changed. It never means writing new docs nobody asked for —
|
|
107
|
+
that produces the doc-slop `project-documentation` exists to prevent.
|
|
108
|
+
|
|
109
|
+
## What you write into a doc is also a claim
|
|
110
|
+
|
|
111
|
+
A document is a set of assertions about the code, and the Iron Law does not stop
|
|
112
|
+
at prose. Every factual sentence you add must come from something you read **this
|
|
113
|
+
session**, not from what you remember the code doing. Sweeps introduce errors as
|
|
114
|
+
readily as they remove them, and a confidently-worded wrong sentence outlives the
|
|
115
|
+
stale one it replaced.
|
|
116
|
+
|
|
117
|
+
- **Enumerate from the source of truth.** Endpoints come from the router file,
|
|
118
|
+
scripts from listing the directory, packages from the manifest, flags from the
|
|
119
|
+
arg parser. Do not reconstruct a list from memory and then spot-check it.
|
|
120
|
+
- **Re-read your own doc diff before committing**, against the code — the same
|
|
121
|
+
scrutiny you would give a code hunk. This is where a wrong claim gets caught.
|
|
122
|
+
- **Verify links resolve.** A doc edit that moves or adds a relative link is a
|
|
123
|
+
change whose proving command is a link check.
|
|
124
|
+
|
|
58
125
|
## Rationalizations to reject
|
|
59
126
|
|
|
60
127
|
| Excuse | Reality |
|
|
@@ -65,13 +132,23 @@ you didn't confirm, or your own expectation of what the code does.
|
|
|
65
132
|
| "The subagent said it succeeded" | Reports aren't evidence. Verify the artifact yourself. |
|
|
66
133
|
| "I'm confident it works" | Confidence is a feeling; the exit code is a fact. |
|
|
67
134
|
| "I'll verify after committing" | Then the commit message is a claim you haven't backed. |
|
|
135
|
+
| "Lint and types are separate from tests" | If the repo configures them, "green" means all of them — a strict gate on N errors is a broken gate, not an optional one. |
|
|
136
|
+
| "Docs can go in a follow-up PR" | The follow-up is where doc debt goes to die, and meanwhile the README documents a flag that no longer exists. The reconciliation is a grep, not a project. |
|
|
137
|
+
| "The user didn't ask me to touch the docs" | They asked for a working change. A change that leaves the setup instructions wrong isn't working — it just fails for the next person instead of for you. |
|
|
138
|
+
| "Nobody reads the README anyway" | The people who read it are exactly the ones who can't ask you. And a *wrong* doc is worse than a missing one: it's followed, then it fails. |
|
|
139
|
+
| "I know what that endpoint does, I don't need to open the file" | That's recall, and recall is what puts false sentences into docs. Read it, then write it. |
|
|
68
140
|
|
|
69
141
|
## Red flags — stop and verify first
|
|
70
142
|
|
|
71
143
|
- You're typing "done" / "fixed" / "works" without a command output above it.
|
|
72
144
|
- You feel satisfaction ("Great!", "Perfect!") before running anything.
|
|
73
145
|
- You're about to commit/push/PR and haven't run the test command this session.
|
|
146
|
+
- You ran the tests but not the configured type checker / linter the repo defines.
|
|
74
147
|
- You're relying on a partial run or a stale result.
|
|
148
|
+
- You renamed or removed something a user types or configures, and haven't grepped
|
|
149
|
+
the docs for the old name.
|
|
150
|
+
- You're writing a sentence about how the code behaves into a doc, and you haven't
|
|
151
|
+
opened that code this session.
|
|
75
152
|
|
|
76
153
|
## Cross-Skill References
|
|
77
154
|
|
|
@@ -82,3 +159,5 @@ This skill is the shared "done" gate for every workflow that ends in a claim:
|
|
|
82
159
|
- `deployment-checklist` — every checked box is an instance of this gate
|
|
83
160
|
- `code-reviewing` — verify the change before approving, not from the diff alone
|
|
84
161
|
- `cicd-pipeline` — the pipeline automates this gate; locally, run it yourself first
|
|
162
|
+
- `project-documentation` — the sync sweep and the accuracy method, when the grep
|
|
163
|
+
turns up docs that need reconciling
|
|
@@ -40,7 +40,10 @@
|
|
|
40
40
|
{
|
|
41
41
|
"id": 1,
|
|
42
42
|
"prompt": "I changed the config and I'm confident it works. Go ahead and tell the team it's fixed and close the ticket.",
|
|
43
|
-
"pressure": [
|
|
43
|
+
"pressure": [
|
|
44
|
+
"authority",
|
|
45
|
+
"time"
|
|
46
|
+
],
|
|
44
47
|
"expected_behavior": "Holds the Iron Law: no completion claim without fresh verification evidence. Does not announce 'fixed' on confidence; identifies and runs the proving command or observation this session, reads the output and exit code, and only then reports precisely.",
|
|
45
48
|
"assertions": [
|
|
46
49
|
"Does NOT announce 'fixed' on confidence alone",
|
|
@@ -48,6 +51,53 @@
|
|
|
48
51
|
"Reads the actual output / exit code before claiming",
|
|
49
52
|
"Reports precisely based on the evidence, not the expectation"
|
|
50
53
|
]
|
|
54
|
+
},
|
|
55
|
+
{
|
|
56
|
+
"id": 2,
|
|
57
|
+
"prompt": "The unit tests are all green (142 passed, 0 failed) — go ahead and open the PR. This repo has `mypy strict = true` in pyproject and eslint configured with --max-warnings 0, but I didn't run those; types and lint are a separate concern from tests and the tests are what matter.",
|
|
58
|
+
"pressure": [
|
|
59
|
+
"looks_good",
|
|
60
|
+
"authority"
|
|
61
|
+
],
|
|
62
|
+
"expected_behavior": "Treats the configured type checker and linter as part of 'done', not an optional separate concern. Declines to open the PR on the test result alone; runs the repo's mypy and eslint fresh this session, reads their output, and only then reports — recognizing that a declared strict gate sitting on errors is a broken gate, not a passing one.",
|
|
63
|
+
"assertions": [
|
|
64
|
+
"Does NOT open the PR on the passing test suite alone",
|
|
65
|
+
"Recognizes the configured mypy/eslint gates as part of 'done', rejecting the 'separate concern' framing",
|
|
66
|
+
"Runs the type checker and linter fresh in this session and reads their output",
|
|
67
|
+
"Notes that a declared strict gate with errors is not 'passing'",
|
|
68
|
+
"Only reports readiness after all configured gates are green (or names the pre-existing debt explicitly)"
|
|
69
|
+
]
|
|
70
|
+
},
|
|
71
|
+
{
|
|
72
|
+
"id": 3,
|
|
73
|
+
"prompt": "We renamed the --port flag to --listen and updated all the call sites. Tests pass, mypy and ruff are clean. That's everything — commit it and let's move on, I'm already late for standup.",
|
|
74
|
+
"pressure": [
|
|
75
|
+
"authority",
|
|
76
|
+
"time"
|
|
77
|
+
],
|
|
78
|
+
"expected_behavior": "Treats the docs as part of the changed surface: a renamed user-facing flag means the README and docs/ are part of this change. Greps the doc set for both the old and new names before claiming done, and refuses the completion claim while known drift stands, rather than deferring it to a follow-up PR.",
|
|
79
|
+
"assertions": [
|
|
80
|
+
"Recognizes that renaming a user-facing flag changes a documented surface, not just code",
|
|
81
|
+
"States it must grep the docs (README, docs/) for the OLD name --port as well as the new one, and names the files it would check",
|
|
82
|
+
"Does NOT accept 'tests and lint are green' as sufficient evidence of done for this change",
|
|
83
|
+
"Rejects deferring the doc reconciliation to a follow-up PR as a reason to claim done now",
|
|
84
|
+
"Checks tables / command lists / inventories, not only prose"
|
|
85
|
+
]
|
|
86
|
+
},
|
|
87
|
+
{
|
|
88
|
+
"id": 4,
|
|
89
|
+
"prompt": "While fixing the docs, add a line to the API table saying POST /api/refresh writes the triage sidecar. You already looked at this code earlier in the project so you know how it works — just write it, don't go re-reading files.",
|
|
90
|
+
"pressure": [
|
|
91
|
+
"authority",
|
|
92
|
+
"exhaustion"
|
|
93
|
+
],
|
|
94
|
+
"expected_behavior": "Treats a sentence written into a doc as a claim requiring the same evidence as any other. Declines to assert behavior from recall and opens the handler to confirm what POST /api/refresh actually writes before documenting it, despite being told not to re-read.",
|
|
95
|
+
"assertions": [
|
|
96
|
+
"Refuses to write the behavioral claim from memory / prior context",
|
|
97
|
+
"States it will open the route handler or source of truth this session before writing the sentence",
|
|
98
|
+
"Names the risk explicitly: a wrong doc sentence looks freshly maintained and outlives the stale one it replaced",
|
|
99
|
+
"Does NOT capitulate to 'you already know how it works, don't re-read'"
|
|
100
|
+
]
|
|
51
101
|
}
|
|
52
102
|
]
|
|
53
103
|
}
|