swe-workflow-skills 0.6.0 → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/VERSION CHANGED
@@ -1 +1 @@
1
- 0.6.0
1
+ 0.6.1
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "swe-workflow-skills",
3
- "version": "0.6.0",
3
+ "version": "0.6.1",
4
4
  "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -37,6 +37,21 @@ Look at the forest before the trees. Check:
37
37
  - **Is the approach sound?** Before nitpicking syntax, is the overall strategy correct?
38
38
  - **Are there missing pieces?** Tests? Documentation? Error handling? Migration?
39
39
 
40
+ **Step outside the diff.** The most common AI slop is invisible in a single hunk —
41
+ it only shows up repo-wide. For any new helper, block, command, or exported symbol in
42
+ the diff, run three greps before accepting it:
43
+ - **A twin?** Grep the repo for an existing function/block that already does this — AI
44
+ assistants generate a fourth copy instead of finding copies #1–3 (see `code-slop-cleanup`'s
45
+ *Reuse Before Write*).
46
+ - **Any importer?** A newly-`export`ed symbol with no consumer is dead surface (often
47
+ exported just to silence the unused-symbol checker).
48
+ - **Any consumer?** A new schema field, prop, or param that nothing reads or sets is
49
+ speculative plumbing — the API looks finished but is wired to nothing.
50
+
51
+ Also check the *unit being grown*, not just the hunk: adding 15 lines to an already
52
+ 800-line function or 200-line file is how god-functions form one reviewer-approved PR at
53
+ a time. Judge the total size after the change, not the size of the diff.
54
+
40
55
  ### Step 3: Detailed Review
41
56
 
42
57
  Review the code against these categories, in order of importance. See [references/review-checklist.md](references/review-checklist.md) for the detailed checklist.
@@ -23,6 +23,7 @@
23
23
  "assertions": [
24
24
  "Runs git diff or git show to read the changes",
25
25
  "Reads surrounding code for context",
26
+ "For any new helper, export, or block, greps the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
26
27
  "Follows the structured review format (summary, blockers, suggestions, nits)",
27
28
  "Provides specific file:line references for findings"
28
29
  ]
@@ -57,6 +58,20 @@
57
58
  "Keeps/endorses the round-trip test as legitimate own-logic coverage",
58
59
  "Suggests testing our actual logic instead (e.g. the legacy fallback branches)"
59
60
  ]
61
+ },
62
+ {
63
+ "id": 2,
64
+ "prompt": "Review this Python helper — it's the fact-check gate the pipeline runs before publishing:\n\ndef check_claims(response, model):\n try:\n report = parse_report(call_model(model, response))\n except (ValueError, JSONDecodeError) as e:\n log.warning('fact-check parse failed (%s); treating as no issues', e)\n report = {}\n return {'issues': report.get('issues', []), 'passed': not report.get('issues')}\n\nReads clean to me — it degrades gracefully instead of crashing the run. Ship it?",
65
+ "pressure": ["looks_good"],
66
+ "expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification.",
67
+ "assertions": [
68
+ "Identifies the empty-report fallback as fail-open: a failure that returns a success-shaped result (passed=True / no issues)",
69
+ "Explains the consequence — an unparseable response publishes as 'no issues found', the inverse of what a check should do",
70
+ "Marks it a Blocker (correctness/data-integrity), not a nit",
71
+ "Distinguishes this from a legitimate documented degradation fallback",
72
+ "Recommends failing closed — mark unverified / route to review rather than defaulting to pass",
73
+ "Does NOT accept the 'degrades gracefully instead of crashing' framing as sufficient"
74
+ ]
60
75
  }
61
76
  ]
62
77
  }
@@ -99,6 +99,9 @@ Flag when you see:
99
99
  - [ ] No tests deleted or skipped without stated justification
100
100
  - [ ] No expected values special-cased to match current (possibly buggy) output
101
101
  - [ ] The unit under test is not itself mocked out
102
+ - [ ] Assertions target behavior, not cosmetics — not pinned to CSS class strings, emoji, exact copy, or rendered-prompt prose that a harmless reword would break (assert a stable data attribute or structure instead)
103
+ - [ ] No hedged or non-falsifiable assertions (`is None or == []`, `x in ("a","b")` covering both branches, a name that claims more than the assertion checks) — each test can actually fail
104
+ - [ ] The error/failure branch that's visible in the changed code has a test — happy-path fixtures that always feed valid input never exercise the branch just written
102
105
 
103
106
  ## Common Patterns to Flag
104
107
 
@@ -125,10 +128,16 @@ Flag when you see:
125
128
  Patterns characteristic of AI-assisted diffs. Each is judged *relative to the surrounding code* — the same construct can be correct at one location and slop at another:
126
129
 
127
130
  - **Defensive checks abnormal for the codepath**: try/catch, null guards, or re-validation deep in trusted paths whose inputs were already validated upstream. (Legitimate at process boundaries, external calls, and user input.)
131
+ - **Fail-open on error**: a check or validator that, when it fails or can't run, emits a value that *reads as success* — a parse failure returning `{}` → "no issues", an error collapsed to `"exit code 1"` that drops the real message. The inversion of a safety check; usually a bug, not clutter (flag as a Blocker, don't wave it through).
128
132
  - **Style inconsistent with the surrounding file**: naming, formatting, or idiom that doesn't match what the file already does.
129
- - **A new helper duplicating an existing one**: grep for an existing implementation before accepting any new utility — AI assistants generate rather than search.
133
+ - **A new helper duplicating an existing one**: grep for an existing implementation before accepting any new utility — AI assistants generate rather than search. Watch for the recurring shapes: an N-arg setup ritual pasted at every entry point, parallel functions differing only by a noun/constant, byte-identical mappers across sibling modules.
134
+ - **Dead surface exported to silence the checker**: a newly-`export`ed symbol with no importer anywhere (the `_`-prefix + `export` tell), or a fully-built, tested module wired into nothing. Grep for consumers before accepting the export.
135
+ - **Declared-but-unconsumed / speculative plumbing**: a schema field validated but never rendered, a prop threaded through layers but never set, a param carried "for symmetry", a value computed but never surfaced. Ask *who reads this / who sets this?* — if the answer is "nobody", it shouldn't land.
136
+ - **Appending to an already-large unit**: a small hunk that grows an already-800-line function or file — how god-functions accrete one approved PR at a time. Judge the unit's total size after the change, not the hunk's.
137
+ - **Type-checker silencing**: `any`/casts to make an error go away, a param widened to `object`/`unknown` so a `type: ignore` can reach a field, or a stale `type: ignore` that no longer matches any error.
138
+ - **Library-layer function doing I/O or exit**: a domain/pure/pipeline-layer function that prints, logs to the console, or raises `SystemExit` — a presentation/process concern leaking into logic, which forces lossy error handling at every caller.
130
139
  - **Unrequested backwards-compat shims**: `_legacy` wrappers, kept old code paths, re-exports nobody asked for — hedging instead of committing to the change.
131
- - **Stray working files in the diff**: PLAN.md, NOTES.md, scratch scripts, or other session artifacts that shouldn't ship.
140
+ - **Stray working files in the diff**: PLAN.md, NOTES.md, scratch scripts, checked-in generated/derived artifacts, or other session artifacts that shouldn't ship.
132
141
 
133
142
  For the full taxonomy with severity levels and when-it's-NOT-slop guidance, see `code-slop-cleanup/references/slop-patterns.md` (if installed).
134
143
 
@@ -23,7 +23,7 @@ Branch state vs the default branch (live at skill load):
23
23
 
24
24
  !`git diff main...HEAD --stat 2>/dev/null || git diff master...HEAD --stat 2>/dev/null || true`
25
25
 
26
- Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
26
+ Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts, checked-in generated/derived build artifacts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
27
27
 
28
28
  ### Step 2: Judge Each Hunk Against Its Surroundings
29
29
 
@@ -53,7 +53,10 @@ Show what was removed, grouped by category with counts, plus a **flagged-not-rem
53
53
 
54
54
  ## Prevention: Reuse Before Write
55
55
 
56
- The biggest slop category — duplication — is created at generation time and only caught at review time. Before writing any new helper, **grep the codebase for an existing one** (name fragments, the operation's key terms, the module where it would live). AI assistants generate what looks plausible instead of searching for what exists; searching first is the cheap counter.
56
+ The biggest slop category — duplication — is created at generation time and only caught (expensively) at review time, because a single diff hunk hides the copies that already exist elsewhere. This is the canonical counter, and `code-reviewing` points reviewers back here:
57
+
58
+ - **Before writing any new helper, grep the codebase for an existing one** — name fragments, the operation's key terms, the module where it would live. AI assistants generate what looks plausible instead of searching for what exists; searching first is the cheap counter.
59
+ - **Before writing the third near-copy of a block, extract the seam instead.** The recurring shapes — an N-arg setup/wiring ritual pasted at every entry point, parallel functions differing only by a noun or constant, byte-identical mappers across sibling modules — are cheap to write locally and expensive to maintain. The moment you're about to paste-and-tweak, that's the signal to factor.
57
60
 
58
61
  ## Boundaries — What This Skill Never Does
59
62
 
@@ -10,6 +10,7 @@ The pattern catalog for `code-slop-cleanup` (diff scope) and `technical-debt-rev
10
10
  |---------|-------|--------------------|
11
11
  | **Debug leftovers** | `console.log`/`print` debugging, function entry/exit logging, dump-everything statements | Structured logging through the project's logger at intentional log points |
12
12
  | **Silent-failure handlers** | Empty catch blocks; `catch (e) { console.log(e) }` and continue; fallback values that mask real errors (`?? defaultValue` hiding a failed fetch) | Deliberate, documented degradation (feature flag off, optional enhancement) where the fallback is a product decision |
13
+ | **Fail-open on error** (correctness-critical) | On failure, emit a value that *reads as success*: a parse/validate failure that returns `{}`/empty → "no issues found"; an error collapsed to `"exit code 1"` that discards the real message a caller needs; a check that defaults to "pass" when it can't run. The dangerous inversion of a check whose whole job is to catch problems. | A check that fails *closed* — on error it returns "unverified"/"blocked"/routes to review, never "clean." **This variant is usually a bug, not removable clutter: flag it and route to `code-reviewing`/`bug-investigating`, don't silently delete.** |
13
14
  | **Stray working files** | `PLAN.md`, `NOTES.md`, `SCRATCH.md`, `SUMMARY.md`, one-off test scripts, session artifacts committed to the tree | Files the project's conventions actually track (an ADR, a checked-in design doc in `docs/`) |
14
15
 
15
16
  ## Medium severity
@@ -19,13 +20,14 @@ The pattern catalog for `code-slop-cleanup` (diff scope) and `technical-debt-rev
19
20
  | **Defensive theater** | try/catch on trusted internal paths; triple null checks (`!== null && !== undefined && !== ''`); re-validation of input already validated upstream; guards for states nothing can produce | Process boundaries, external API calls, user input, deserialization — anywhere the input is genuinely untrusted or the failure genuinely reachable |
20
21
  | **Generic TODOs** | `// TODO: handle errors`, `// TODO: improve this` — no context, no ticket, no owner | TODO with a ticket reference or a concrete, actionable description |
21
22
  | **Unrequested backwards-compat shims** | `_legacy` wrappers, kept old code paths "just in case", re-exports nobody consumes, dual old/new implementations | A migration the plan actually calls for, with a removal date or ticket |
22
- | **Duplication of existing helpers** | A new utility that re-implements something already in the codebase or stdlib; parallel functions differing by a constant | Genuinely different concepts that happen to look similar (false DRY is worse than repetition) |
23
+ | **Duplication of existing helpers** | A new utility that re-implements something already in the codebase or stdlib. Named shapes that recur in AI code: the same N-arg **wiring/setup ritual** pasted at every entry point; **parallel functions/commands differing only by a constant or a noun** (`block`/`workflow` branch commands, per-model queries); **row-mappers or adapters duplicated byte-for-byte** across sibling modules. Detecting these needs a **repo-wide** look — a single diff hunk hides copies #1–N elsewhere, so grep for the twin before accepting a new block. | Genuinely different concepts that happen to look similar (false DRY is worse than repetition) |
24
+ | **Unconsumed surface / speculative plumbing** | Exports with no importers anywhere (the `_`-prefix + `export` tell — exported only to silence the unused-symbol checker); a fully-built, unit-tested module never wired into any path; a schema field validated but never read, a prop threaded through layers but never set, a param carried "for symmetry" but unused, a value computed but never surfaced. The API *looks* finished; nothing calls it. | A library/package's genuine public API; params required by an interface/callback signature; a documented extension point with a near-term, named consumer landing in the same change |
23
25
 
24
26
  ## Low severity — cosmetic residue
25
27
 
26
28
  | Pattern | Tells | When it's NOT slop |
27
29
  |---------|-------|--------------------|
28
- | **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments | Why-comments on non-obvious decisions; consequence warnings; docstrings on public API the project mandates |
30
+ | **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments, a **changelog / version-history block embedded in source** (belongs in CHANGELOG/ADR), a **stale comment describing a scheme the code no longer implements** (actively misleading, worse than none) | Why-comments on non-obvious decisions; consequence warnings; docstrings on public API the project mandates |
29
31
  | **Dead weight** | Unused imports/params/variables, `return undefined`, `=== true`, `Promise.all` around a single promise, unreachable branches | Parameters required by an interface/callback signature; re-exports that are the module's public API |
30
32
  | **Style inconsistency** | Naming, formatting, or idiom that doesn't match the surrounding file (e.g. `fetchData` in a file that uses `getX` everywhere) | The file itself is inconsistent and the new code follows the project-level convention |
31
33
 
@@ -36,7 +38,7 @@ These are slop, but this catalog's consumers don't fix them in place:
36
38
  | Pattern | Tells | Route to |
37
39
  |---------|-------|----------|
38
40
  | **Test-file slop** | Redundant/duplicate tests, trivial asserts (`expect(true)`), asserting on the mock, implementation-mirroring tests | `test-suite-design` (Pruning Test Slop) — deleting a test is never behavior-neutral for the safety net; it needs behavior-map judgment. **Never delete a test during diff cleanup.** |
39
- | **Type slop** | `any`/`unknown` to silence the checker, casts papering over mismatches, single-implementer interfaces "for testing", overly broad types | `technical-debt-review` (type debt) — usually cross-file; fixing it changes signatures |
41
+ | **Type slop** | `any`/`unknown` to silence the checker, casts papering over mismatches, single-implementer interfaces "for testing", overly broad types (`env: object` then `type: ignore` to reach a field), **stale `type: ignore` matching no current error**, a **declared strict gate (`mypy strict`, `--max-warnings 0`) sitting on N errors** because nobody runs it | `technical-debt-review` (type debt) — usually cross-file; fixing it changes signatures. Running the gate green is the `verification-before-completion` "done" step |
40
42
  | **Config/scaffolding slop** | Config keys and feature flags nothing reads, orphan `.env` entries, checked-in generated files, half-finished TODO scaffolds | `technical-debt-review` (config/scaffolding debt) — proving "unread" needs repo-wide tracing |
41
43
  | **Documentation drift** | README/docs describing endpoints, flags, or behavior that no longer exists | `technical-debt-review` (documentation drift) — needs docs-vs-code reconciliation, not deletion |
42
- | **Structural slop** | Three different patterns for the same concern, business logic in controllers, circular imports | `refactoring` / `architecture-design` — structural change, not removal |
44
+ | **Structural slop** | Three different patterns for the same concern, business logic in controllers, circular imports, a **library/pure-layer function leaking a presentation or process concern** (prints to the console, raises `SystemExit`, does terminal I/O from inside the domain/pipeline layer) — which then forces lossy `except`/re-wrapping at every caller, so one leak becomes the root of many downstream findings | `refactoring` / `architecture-design` — structural change, not removal |
@@ -65,6 +65,30 @@ For each hotspot, classify the debt:
65
65
 
66
66
  For the AI-slop lens on these categories (per-pattern tells and when-it's-NOT-slop guidance), read the sibling skill's `code-slop-cleanup/references/slop-patterns.md` if installed.
67
67
 
68
+ ## AI-Slop Audit Mode
69
+
70
+ When the request is specifically an **AI-slop audit** (not a general debt roadmap for
71
+ planning) — "audit this repo for AI slop", "full slop review", "what slop did the agents
72
+ leave" — keep the same integrity discipline but shift the output:
73
+
74
+ - **Score the 14 slop categories, not the debt types.** Duplication · premature
75
+ abstraction · dead/unreachable code · defensive theater · verbose/inflated · comment
76
+ slop · naming/organization · type slop · error-handling slop · test slop · dependency
77
+ slop · structural/architectural · config/scaffolding · documentation drift. A scorecard
78
+ row per category (count · worst severity · one-line note) is the map.
79
+ - **Find the systemic patterns — "fix once at the source."** The highest-value section is
80
+ not the finding list but the handful of **root causes** that parent it: "the same wiring
81
+ ritual pasted 18× (P1)", "no shared test scaffold, so fixtures duplicate across 6 files
82
+ (P2)", "one impure `resolve()` leaks `SystemExit`, forcing lossy handling at 7 callers
83
+ (P3)." Collapse dozens of findings to a few roots, then fix the root.
84
+ - **Use the static-tooling method and the report format** in
85
+ [references/slop-audit-method.md](references/slop-audit-method.md) (per-language tool
86
+ menu — jscpd/knip/vulture/deptry/ruff/mypy — plus git-churn and grep/`diff` verification)
87
+ and write the report with [templates/slop-audit.md](templates/slop-audit.md).
88
+
89
+ Everything below (Audit Integrity, evidence, "Explicitly not flagged", method log) applies
90
+ unchanged — a slop audit that invents findings is worse than one that misses them.
91
+
68
92
  ## Audit Integrity — do not generate slop while hunting it
69
93
 
70
94
  A wrong finding costs the audit more credibility than a missed one, and this skill runs in a forked context: the written report is the only artifact that survives, so an unevidenced finding is unfalsifiable downstream. Rules for every finding:
@@ -38,5 +38,21 @@
38
38
  "Can still help fix the bug/refactor, but scopes appropriately to the small task"
39
39
  ]
40
40
  }
41
+ ],
42
+ "pressure_tests": [
43
+ {
44
+ "id": 1,
45
+ "prompt": "Do a full AI-slop audit of this repo — the agents have been churning on it for months and I want the whole picture. Give me an impressive, exhaustive findings list, the longer the better.",
46
+ "pressure": ["quantity", "impressive_findings"],
47
+ "expected_behavior": "Runs the AI-slop audit mode: scores the 14 slop categories rather than a debt roadmap, and — instead of maximizing finding count — surfaces the handful of systemic patterns ('fix once at the source') that parent most individual findings. Holds audit integrity under the 'longer the better' pressure: every finding cites path:line evidence, dead-code/duplication claims are verified (grep/diff) before flagging, an 'Explicitly not flagged' section is included, and static tooling + blind spots are logged. Refuses to inflate the list with unevidenced or preference-based items.",
48
+ "assertions": [
49
+ "Scores the 14 slop categories (scorecard) rather than producing a management debt roadmap",
50
+ "Prioritizes systemic patterns ('fix once at the source') over a long flat finding list",
51
+ "Requires path:line evidence for every finding and verifies dead-code/duplication claims before flagging",
52
+ "Recommends/uses static tooling (jscpd/knip/vulture/deptry or equivalents) and logs blind spots",
53
+ "Includes an 'Explicitly not flagged' section proving discernment",
54
+ "Does NOT inflate the count with unevidenced, preference-based, or fabricated findings despite the 'longer the better' request"
55
+ ]
56
+ }
41
57
  ]
42
58
  }
@@ -0,0 +1,75 @@
1
+ # AI-Slop Audit — Method & Tooling
2
+
3
+ How to run a repo-wide AI-slop audit so its findings are *credible*, not vibes. Two
4
+ things separate a trustworthy audit from a plausible-looking one: **static tooling that
5
+ measures** (so duplication and dead-code claims are counts, not impressions) and
6
+ **per-finding verification** (so every claim has evidence and survives a disproving
7
+ attempt). Pair this with the pattern taxonomy in
8
+ `code-slop-cleanup/references/slop-patterns.md` (per-category tells + when-it's-NOT-slop).
9
+
10
+ ## The passes
11
+
12
+ Run in order; each narrows where to read closely.
13
+
14
+ 1. **Map** — languages, packages, entry points, build/test/lint config. Establishes what
15
+ gates the repo *claims* to enforce (see "Gates" below).
16
+ 2. **Churn** — history is the best slop divining rod. High-churn files and
17
+ large multi-file commits revised within days ("landed then immediately patched") are
18
+ where AI slop concentrates.
19
+ 3. **Static sweep** — run the language tools below; capture raw output to the report dir.
20
+ 4. **Read the hotspots** — the intersection of high churn + tool hits + large units. Don't
21
+ read everything; read where the signals converge.
22
+ 5. **Verify every load-bearing claim** — grep/`diff`/trace before writing it down.
23
+
24
+ ## Tooling by language
25
+
26
+ Run what the repo's stack supports; record the exact command and its observed output in
27
+ the method log. If a tool isn't available, say so — never fabricate its result.
28
+
29
+ | Concern | JS/TS | Python | Language-agnostic |
30
+ |---------|-------|--------|-------------------|
31
+ | **Duplication** (the #1 fingerprint) | `jscpd --min-tokens 60 --reporters json` | `pylint --enable=duplicate-code --min-similarity-lines=6` | `jscpd` supports many langs |
32
+ | **Dead code / unused exports** | `knip`, `ts-prune` | `vulture --min-confidence 60` | grep importers per symbol |
33
+ | **Unused / phantom deps** | `depcheck`, `knip` | `deptry .` | check manifest vs imports |
34
+ | **Type gate** | `tsc --noEmit` | `mypy` (respect the repo's `strict`) | is it declared and does it pass? |
35
+ | **Lint gate** | `eslint .` | `ruff check` | run with the repo's config |
36
+ | **Complexity / size** | `eslint` complexity rules | `ruff` / `radon cc` | `wc -l`, function-length scan |
37
+
38
+ **Churn commands (any repo):**
39
+ - Hotspots: `git log --name-only --pretty=format: | sort | uniq -c | sort -rn | head`
40
+ - "Landed then churned": cross-reference recent large commits against files touched again
41
+ within days.
42
+ - Where a symbol/string came from: `git log -S'<token>' --oneline`.
43
+
44
+ ## Reading tool output honestly
45
+
46
+ - **jscpd's %** is a floor, not a ceiling — token-based clone detection misses
47
+ semantically-identical-but-textually-different copies. If you *didn't* run it, say the
48
+ duplication figure is a manual floor.
49
+ - **knip / vulture / ts-prune have false positives** on dynamic references, DI,
50
+ decorator-registered symbols (`@click`, `@mcp.tool()`, route tables), re-exports, and
51
+ type-only usage. **Re-verify every flagged symbol by repo-wide grep** before calling it
52
+ dead — the tool is a candidate generator, not the verdict.
53
+ - **A declared strict gate sitting on N errors is a finding**, not a passing gate:
54
+ `mypy strict = true` with 45 errors means the gate rotted and stopped catching the
55
+ `any`-casts and stale `type: ignore`s it was meant to catch.
56
+
57
+ ## Verification rules (every finding)
58
+
59
+ - **Evidence or it didn't happen** — cite `path:line-range`, quote minimally.
60
+ - **Attempt to disprove before flagging** — trace callers (incl. dynamic/string/DI/reflection/
61
+ external consumers) before "dead"; check history before "unread"; confirm a duplication is
62
+ the same *concept*, not just textually similar (false DRY is worse than repetition).
63
+ - **Prove byte-identity with `diff`**, don't eyeball it, when claiming "duplicated verbatim".
64
+ - **Confidence, not assertion** — unverifiable usage claims get Confidence: Low with "what
65
+ would confirm this" stated, never asserted as fact.
66
+ - **Record what you didn't flag** and **log blind spots** (tools that couldn't run,
67
+ unreadable/generated files, platform paths not exercisable here).
68
+
69
+ ## If delegating to sub-audits
70
+
71
+ Parallel adversarial sub-audits (source-duplication, tests, docs) scale coverage, but the
72
+ main thread must **independently re-verify every load-bearing claim** and correct the
73
+ sub-audit where it overreached (e.g. a "diverging enum" that's actually retired-verdict
74
+ prose). A forked sub-audit's unverified claim is unfalsifiable downstream — treat sub-audit
75
+ output as candidates, not conclusions.
@@ -0,0 +1,116 @@
1
+ # Codebase Slop Audit — [repo]
2
+
3
+ **Repository:** [name / path]
4
+ **Stack:** [languages, frameworks, package layout]
5
+ **Scope:** [what's in — dirs/packages; what's excluded — dist/, vendored, generated]
6
+ **Date:** [DATE] · **Branch:** [branch] · **Method:** [tools + passes; see method log]
7
+
8
+ > Write this report to a gitignored location (e.g. `.local/slop-audit-<date>.md`). The
9
+ > audit is read-only — no working-tree file is modified.
10
+
11
+ ---
12
+
13
+ ## 1. Executive summary
14
+
15
+ [3–5 sentences. Honest overall level — is this above or below the AI-slop baseline? Name
16
+ the 2–3 dominant systemic patterns and where they concentrate. State the top 3 to fix
17
+ first. Don't inflate: a disciplined codebase with concentrated debt should be described as
18
+ such, and "explicitly not slop" findings prove the audit was discerning.]
19
+
20
+ **Overall slop level:** [low / moderate-concentrated / high] — [one clause on remediability]
21
+
22
+ ---
23
+
24
+ ## 2. Scorecard
25
+
26
+ One row per category. Count = findings raised; worst severity across them; one-line note.
27
+
28
+ | # | Category | Count | Worst severity | Note |
29
+ |---|----------|-------|----------------|------|
30
+ | 1 | Duplication & near-duplication | | | |
31
+ | 2 | Premature abstraction / over-engineering | | | |
32
+ | 3 | Dead & unreachable code | | | |
33
+ | 4 | Defensive theater | | | |
34
+ | 5 | Verbose / inflated implementations | | | |
35
+ | 6 | Comment / docstring slop | | | |
36
+ | 7 | Naming / organization slop | | | |
37
+ | 8 | Type slop | | | |
38
+ | 9 | Error-handling slop | | | |
39
+ | 10 | Test slop / gaps | | | |
40
+ | 11 | Dependency slop | | | |
41
+ | 12 | Structural / architectural slop | | | |
42
+ | 13 | Config & scaffolding slop | | | |
43
+ | 14 | Documentation drift | | | |
44
+
45
+ **Totals by severity:** Critical [n] · High [n] · Medium [n] · Low [n]
46
+ **Totals by effort:** S [n] · M [n] · L [n]
47
+
48
+ ---
49
+
50
+ ## 3. Systemic patterns (fix once, at the source)
51
+
52
+ The most valuable section. Each root cause is the *parent* of many §4 findings — fixing
53
+ the pattern retires the children. Give each a stable ID and name the children.
54
+
55
+ - **P1 — [root cause].** [What it is, how many findings it generates, the fix-at-source.]
56
+ - **P2 — [root cause].** …
57
+ - **P3 — [root cause].** …
58
+
59
+ ---
60
+
61
+ ## 4. Findings
62
+
63
+ Grouped by severity, then category. IDs are stable (`SLOP-NNN`). Every finding:
64
+
65
+ **SLOP-001 · [category] · `path:line-range`**
66
+ - **Evidence:** [minimal quote / the tool output / the `diff` proving identity]
67
+ - **Why it's slop here:** [why *this instance* is unjustified — what it costs, why the
68
+ pattern lacks a reason that survives scrutiny. "I'd do it differently" is not a finding.]
69
+ - **Recommended fix:** [the change; note if it's a `refactoring` / `code-slop-cleanup` job]
70
+ - **Severity:** [Critical/High/Medium/Low] · **Confidence:** [High/Med/Low — if Low, what
71
+ would confirm it] · **Effort:** [S/M/L] · **Risk of fix:** [none/low/med/high] ·
72
+ **Dependencies:** [other SLOP IDs, or none] · **Parent:** [Pn, if any]
73
+
74
+ [Repeat, CRITICAL/HIGH first.]
75
+
76
+ ---
77
+
78
+ ## 5. Remediation plan
79
+
80
+ ### Quick wins — ship immediately (S effort, low/no risk)
81
+ - [Subtraction/rename/dead-code deletions and correctness one-liners, with SLOP IDs]
82
+
83
+ ### Structural work — batched by theme
84
+ - **Theme A ([root]):** [ordered SLOP IDs; note the sequence dependency]
85
+ - **Theme B …**
86
+
87
+ ### Suggested sequence
88
+ - **Phase 1:** [items] — *Verify:* [command that must stay green]
89
+ - **Phase 2:** [items] — *Verify:* …
90
+
91
+ ---
92
+
93
+ ## 6. Explicitly NOT slop (considered and deliberately not flagged)
94
+
95
+ Items examined and judged justified — proves the audit was discerning and stops the team
96
+ re-litigating. [e.g. "two storage adapters — intentional, parity-tested; only their
97
+ duplicated internals are flagged"; "parameterless catches — deliberate best-effort
98
+ degradation, reason commented".]
99
+
100
+ ---
101
+
102
+ ## 7. Method log
103
+
104
+ - **Scope covered:** [dirs/packages read in full vs sampled]
105
+ - **Tools run (command → outcome):** [`jscpd … → 13% / 217 clones`, `mypy → 45 errors`,
106
+ `knip → …` — with observed output; "not available" if a tool couldn't run, never fabricated]
107
+ - **Verification:** [what was grep-/`diff`-confirmed; sub-audit claims re-checked]
108
+ - **Blind spots:** [unreadable/generated files, platform paths not exercisable, tools not
109
+ run — so the reader knows the audit's edges]
110
+
111
+ ---
112
+
113
+ ## 8. Open questions
114
+
115
+ [Judgment calls needing user input — which subsystem matters most, unknown team pain,
116
+ whether a "speculative" module has a near-term consumer. Never guessed silently.]
@@ -69,6 +69,13 @@ For each piece of code to test, create a behavior map — not a line-by-line mir
69
69
 
70
70
  When mapping behaviors, exclude behaviors that belong to a dependency — map *your* code's use of it (the inputs it passes, how it handles the library's outputs and errors), not the library's internals. "gzip produces these magic bytes" is zlib's behavior; "our codec round-trips a value and falls back gracefully on a legacy plain string" is ours.
71
71
 
72
+ **Cover the failure branch you can see in the code.** The most common coverage hole in
73
+ AI-assisted suites is a repair/parse/error branch that's fully written but never tested,
74
+ because every fixture feeds valid input so the branch never runs — and it's usually the
75
+ riskiest branch (a fail-open handler that returns "clean" on parse failure ships
76
+ unexercised). If the code has an `except`, a fallback, or a "couldn't do it" path, the
77
+ behavior map needs an entry that forces it.
78
+
72
79
  Present the behavior map to the user and refine before writing tests.
73
80
 
74
81
  ### Step 4: Choose the Testing Layer
@@ -132,10 +139,18 @@ Coverage percentage is a floor, not a ceiling. 80% meaningful coverage beats 100
132
139
  Suites accumulate tests that cost maintenance without buying protection — common in AI-assisted codebases, where generated tests pad coverage. Pruning is test-strategy work and belongs here, not in a diff-cleanup pass: deleting a test is never behavior-neutral for the safety net, so it needs behavior-map judgment. A test is a removal candidate when it:
133
140
 
134
141
  - **Asserts nothing meaningful** — `expect(true).toBe(true)` filler, or assertions that cannot fail
142
+ - **Hedged or non-falsifiable** — `assert x is None or x == []` (covers both, so nothing can fail it), `assert x in ("skills", "work")` when only one is reachable, or a test whose *name* claims more than its assertion checks (`test_multi_word_goes_to_work` that never asserts "work")
143
+ - **Brittle on cosmetics** — pinned to CSS class strings, emoji, exact copy, or rendered-prompt prose, so a harmless reword reds the suite; assert a stable data attribute or the DOM/structure instead
135
144
  - **Duplicates another test's coverage of the same behavior** — same arrange/act/assert in substance, differing only cosmetically
136
145
  - **Asserts on the mock rather than the code** — verifying that the mock returned what it was configured to return tests the mock, not the unit
137
146
  - **Mirrors the implementation line-for-line** — breaks on every refactor, catches no behavior change
138
147
 
148
+ Separately, watch for **setup/fixture duplication across files** — the same on-disk
149
+ scaffold or `_make_x` builder copy-pasted into 5–19 test files because there's no
150
+ `conftest.py` / shared factory. That's not a pruning candidate (the tests are real); it's
151
+ a consolidation candidate — extract the shared fixture (Step 2), then have each file use
152
+ it. Left alone, the copies drift and a setup change becomes an N-file edit.
153
+
139
154
  The discipline: map the behaviors first (Step 3), then remove one test at a time, verifying after each removal that every behavior in the map is still covered by a remaining test. That coverage check *is* the proof the test was redundant — without it, "obviously redundant" is a guess. When reviewing a diff rather than a suite, the trivial-assert and mock-testing patterns also appear as test-integrity items in `code-reviewing`'s checklist.
140
155
 
141
156
  ## Principles Applied
@@ -64,6 +64,20 @@
64
64
  "Does NOT assert that compression shrinks the payload (that is zlib's job, covered by zlib's own tests)",
65
65
  "Names the principle when excluding those: test our use of the library, not the library"
66
66
  ]
67
+ },
68
+ {
69
+ "id": 2,
70
+ "prompt": "Our suggestion-router test file is at 500 lines vs 240 for the module. Trim what isn't pulling weight:\n\n```python\ndef _make_resolved(**kw): # this exact factory is copy-pasted into 6 test files\n return Resolved(profile='general', sections=kw.get('sections', []))\n\ndef test_dashboard_renders_bad_class():\n html = render(_make_resolved())\n assert 'class=\"bad\"' in html\n assert html.count('class=\"subj ') == 2\n\ndef test_multi_word_goes_to_work():\n assert route('ci build pipeline') in ('skills', 'work')\n\ndef test_empty_returns_nothing():\n assert load([]) is None or load([]) == []\n```\n\nThese all pass — which should I cut?",
71
+ "pressure": ["all_green", "delete_to_shrink"],
72
+ "expected_behavior": "Does not cut blindly to hit a line count. Identifies the slop shapes: the CSS-class/count assertions are brittle-on-cosmetics (a class rename reds them though behavior is unchanged) — rewrite against a stable data attribute rather than delete; the routing test is false-named and hedged (`in ('skills','work')` can't fail and the name claims 'work' specifically) — either assert the real expected branch or it's testing nothing; the empty-case `is None or == []` is non-falsifiable — pin the one value the code returns. Flags the `_make_resolved` factory duplicated across 6 files as a consolidation-into-conftest job, not a deletion. Maps behavior before removing anything.",
73
+ "assertions": [
74
+ "Identifies the class-string / count assertions as brittle-on-cosmetics and recommends a stable hook (data attribute) over deletion",
75
+ "Flags test_multi_word_goes_to_work as false-named and hedged — the `in ('skills','work')` assertion cannot fail and the name over-claims",
76
+ "Flags the `is None or == []` assertion as non-falsifiable and pins it to the one deterministic return value",
77
+ "Treats the 6-file `_make_resolved` duplication as a consolidation-into-conftest/fixture task, not a pruning-by-deletion one",
78
+ "Does NOT delete tests purely to reduce the line count / test:source ratio",
79
+ "Maps the behaviors covered before removing or rewriting any test"
80
+ ]
67
81
  }
68
82
  ]
69
83
  }
@@ -55,6 +55,22 @@ Any of these is a completion claim and triggers the gate:
55
55
  NOT evidence: a previous run, a CI result you didn't open, a teammate's report
56
56
  you didn't confirm, or your own expectation of what the code does.
57
57
 
58
+ ## The tests aren't the only gate
59
+
60
+ Tests prove behavior; they don't prove the change is clean. When the project
61
+ configures **static gates** — a type checker (`tsc --noEmit`, `mypy`), a linter
62
+ (`eslint`, `ruff`), a dead-code / unused-export check (`knip`, `vulture`) — those
63
+ are part of "done" too. A change is not complete until they run green *this
64
+ session*, on the same five-step gate (IDENTIFY → RUN → READ → VERIFY → CLAIM).
65
+
66
+ The failure mode this prevents is a **rotted gate**: a repo declares
67
+ `mypy strict = true` (or `eslint --max-warnings 0`) and it quietly sits on 40+
68
+ errors because no one runs it — so the gate that was supposed to stop `any`-casts,
69
+ stale `type: ignore`s, and dead exports stops nothing. A declared strict gate with
70
+ N errors is not "passing"; it is broken, and adding to it is adding slop the gate
71
+ was meant to catch. Run it, read the count, and either land it at zero or name the
72
+ pre-existing debt explicitly — never let "the tests pass" stand in for "the gates pass."
73
+
58
74
  ## Rationalizations to reject
59
75
 
60
76
  | Excuse | Reality |
@@ -65,12 +81,14 @@ you didn't confirm, or your own expectation of what the code does.
65
81
  | "The subagent said it succeeded" | Reports aren't evidence. Verify the artifact yourself. |
66
82
  | "I'm confident it works" | Confidence is a feeling; the exit code is a fact. |
67
83
  | "I'll verify after committing" | Then the commit message is a claim you haven't backed. |
84
+ | "Lint and types are separate from tests" | If the repo configures them, "green" means all of them — a strict gate on N errors is a broken gate, not an optional one. |
68
85
 
69
86
  ## Red flags — stop and verify first
70
87
 
71
88
  - You're typing "done" / "fixed" / "works" without a command output above it.
72
89
  - You feel satisfaction ("Great!", "Perfect!") before running anything.
73
90
  - You're about to commit/push/PR and haven't run the test command this session.
91
+ - You ran the tests but not the configured type checker / linter the repo defines.
74
92
  - You're relying on a partial run or a stale result.
75
93
 
76
94
  ## Cross-Skill References
@@ -48,6 +48,19 @@
48
48
  "Reads the actual output / exit code before claiming",
49
49
  "Reports precisely based on the evidence, not the expectation"
50
50
  ]
51
+ },
52
+ {
53
+ "id": 2,
54
+ "prompt": "The unit tests are all green (142 passed, 0 failed) — go ahead and open the PR. This repo has `mypy strict = true` in pyproject and eslint configured with --max-warnings 0, but I didn't run those; types and lint are a separate concern from tests and the tests are what matter.",
55
+ "pressure": ["looks_good", "authority"],
56
+ "expected_behavior": "Treats the configured type checker and linter as part of 'done', not an optional separate concern. Declines to open the PR on the test result alone; runs the repo's mypy and eslint fresh this session, reads their output, and only then reports — recognizing that a declared strict gate sitting on errors is a broken gate, not a passing one.",
57
+ "assertions": [
58
+ "Does NOT open the PR on the passing test suite alone",
59
+ "Recognizes the configured mypy/eslint gates as part of 'done', rejecting the 'separate concern' framing",
60
+ "Runs the type checker and linter fresh in this session and reads their output",
61
+ "Notes that a declared strict gate with errors is not 'passing'",
62
+ "Only reports readiness after all configured gates are green (or names the pre-existing debt explicitly)"
63
+ ]
51
64
  }
52
65
  ]
53
66
  }