@slowdini/slow-powers-opencode 0.1.5 → 0.3.0

package/skills/verifying-development-work/comment-review.md

@@ -0,0 +1,85 @@
+# Reviewing the Comments
+
+This is **phase 3** — the last step of the finishing sequence in [`SKILL.md`](SKILL.md).
+By now the code has been reviewed (phase 1), and verified (phase 2). The code is frozen;
+**this pass touches only comments.** That is the whole reason it comes last: a
+comment edit can't change behavior, so it can't invalidate the verification you
+just ran — there is nothing here to re-test. Do it as the final polish before the
+handoff.
+
+---
+
+## The comment-hygiene pass
+
+Review **every comment in the changed code** with one goal: **delete as many as
+possible.**
+
+This runs against your own instinct. Writing a comment feels like preserving the
+narrative — why this approach, what was tried, which ticket it traces to. But a
+human reading code finds it *very hard* to skip a comment; every one they hit,
+they stop and read. Narrative comments tax every future reader to record a story
+that belongs in the commit message or the PR, not the source. Left in, they
+become the thing the user has to delete by hand before merging — so delete them
+now, on their behalf.
+
+A comment survives only if it fits one of two categories **and** meets its bar:
+
+1. **Explanation.** Code that is genuinely hard to follow from reading it — a
+   subtle algorithm, a deliberate break from the usual pattern, a non-obvious
+   constraint. The comment fills the gap with an *evergreen* reason (true a year
+   from now, not "fixes the bug from Tuesday"). These are **extremely rare**:
+   well-written code is self-commenting, and a reader fluent in code can follow
+   even sophisticated paths when the code itself is clear. If the right fix is to
+   make the code clearer, do that instead of explaining unclear code.
+2. **Documentation.** A concise doc-style comment (jsdoc and equivalents) on an
+   **exported** member, where the text is surfaced by doc generators and editor
+   hints to readers who *don't* have the source in front of them. These almost
+   always earn their place. Keep them concise and evergreen, matching the
+   surrounding style; they may describe usage more freely since that's their job.
+
+**Everything else gets deleted — about 99.9% of the time.** The most common
+offender, and the one that feels most defensible, is **step-by-step narration**
+that walks through what the code already says — `// Step 1: lowercase`,
+`// now strip the accents`, `// finally, trim the dashes`. It reads as helpful
+structure, and *that feeling is the trap*: the numbered steps restate control
+flow the reader can already see in the code, so most such comments carry no
+information the line below them doesn't — they only add something else to read.
+"The steps make it easier to follow" is the rationalization to delete *through*,
+not act on; the code is the structure. Strip the narration and nothing is lost.
+The same goes for prose narrative ("first we… then we…"), time-sensitive comments
+(ticket numbers, "the previous solution…", "changed this because…"), and any
+comment that merely restates its line. A comment that fits neither surviving
+category, or fits one but misses its bar, is noise. **When in doubt, delete it.**
+A truly unique case might warrant a truly unusual comment — but treat that as the
+rare exception it is, not the default.
+
+```ts
+// BEFORE — every comment restates the line under it
+// Step 1: lowercase the title
+const lower = title.toLowerCase();
+// Step 2: replace whitespace runs with a single hyphen
+const hyphenated = lower.replace(/\s+/g, "-");
+
+// AFTER — the code already says all of that
+const lower = title.toLowerCase();
+const hyphenated = lower.replace(/\s+/g, "-");
+```
+
+**A kernel of value doesn't save the comment around it.** The hardest case is
+the *mixed* comment — mostly narration, with one genuinely useful clause buried
+in it (a real constraint, a non-obvious *why*). Keeping the whole block "because
+part of it is useful" is exactly how noise survives review: a reader will keep a
+comment that's 90% restatement for the sake of the 10% that matters. Don't.
+**Extract the useful part, delete the rest, and if what remains earns a comment,
+write it as a tight standalone one** — the kernel alone, not the narration that
+carried it. A four-line "Step 1… / Step 2 *(the one real reason)* / Step 3… /
+Step 4…" block collapses to a single comment stating that one reason, and the
+numbered narration is gone.
+
+---
+
+## Then: hand it back
+
+These were comment-only edits — they change no behavior, so there is **nothing to
+re-verify**: the verification from phase 2 still covers the code being returned.
+Return to the finishing sequence in [`SKILL.md`](SKILL.md) for the handoff.

package/skills/verifying-development-work/evals/baseline/BASELINE.md

@@ -0,0 +1,23 @@
+# Baseline — verifying-development-work
+
+Committed reference output from a canonical eval run. Regenerate with
+`bun run evals:promote-baseline -- --skill verifying-development-work --iteration <N>` after aggregating. The ephemeral workspace (run records, timing,
+dispatch files, produced outputs) stays gitignored under `skills-workspace/`
+and is reclaimable by `evals:teardown` once promoted (this commit's marker).
+
+| Field | Value |
+|-------|-------|
+| Mode | revision |
+| Iteration | iteration-6 |
+| Harness | claude-code |
+| Agent model | claude-sonnet-4-6 |
+| Judge model | claude-sonnet-4-6 |
+| Conditions | old_skill, new_skill |
+| Run timestamp | 2026-06-05T01:32:51.388Z |
+| Label | (none) |
+| Promoted from commit | 4d6276b |
+
+Files:
+- `benchmark.json` — aggregate pass-rate / duration / token deltas.
+- `grading/<eval-id>__<condition>.json` — per-run assertion results and judge rationales.
+

package/skills/verifying-development-work/evals/baseline/NOTES.md

@@ -0,0 +1,87 @@
+# Baseline notes — dedicated code/comment review (issue #173)
+
+Revision eval validating the move off the harness's built-in code-review onto
+self-contained guidance, plus the split of the finishing sequence into three
+ordered phases (revision mode, baseline snapshot `dev`, sonnet/sonnet, bootstrap
+on, guard armed, 2026-06-05). Promoted from **iteration-6**; confirmed by
+**iteration-7**.
+
+## The change under test
+
+- SKILL.md "Finishing" rewritten into three explicit phases: **(1) review & fix
+  the CODE** ([`code-review.md`](../../code-review.md)) → code is frozen → **(2)
+  final verification** on the frozen code → **(3) review & clean the COMMENTS**
+  ([`comment-review.md`](../../comment-review.md)), cosmetic, no re-verify.
+- A copy-into-your-tracker **finishing checklist** enumerating the phases + the
+  no-unilateral-merge gate.
+- `code-review.md` is the code review only; the comment-hygiene pass (delete
+  narration / step-by-step / ticket comments; keep only Explanation or exported
+  Documentation; **extract the kernel** from a mixed comment) moved to its own
+  `comment-review.md`.
+- Old arm (`dev`) = the prior step 1 "invoke your harness's built-in code-review
+  capability," no companion files.
+
+## Suite
+
+Two cases via `--only`, the ones this change actually touches:
+`comment-hygiene-at-handoff` and `wrap-it-up-handoff`. The other four suite
+cases (`claim-without-running`, `build-implied-by-edit`, the two seeded cases)
+were **not** re-run for this change — they exercise the Gate Function / red-flag
+tables, not the finishing-sequence restructure. A future full-suite revision run
+is wanted before treating this as a whole-skill baseline (see below).
+
+## Result (iteration-6 promoted; iteration-7 confirms)
+
+| | old_skill | new_skill | delta |
+|---|---|---|---|
+| iteration-6 | 0.75 | 1.00 | +0.25 |
+| iteration-7 | 0.75 | 0.875 | +0.125 |
+| **mean** | **0.75** | **0.9375** | **+0.1875** |
+
+Both iterations positive, invocation 100% / 100%, no validity warnings. Tokens:
+new approximately 215k / 190k vs old approximately 143k / 110k — the phased
+review + checklist cost ~40-70% more tokens; the +~19pp buys that.
+
+## What discriminates (and what doesn't)
+
+- **Robust driver — the checklist drives a consistent handoff.**
+  `wrap-it-up-handoff` new_skill is **4/4 in both runs**; old is 3/4 then 2/4 —
+  the baseline streakily forgets to *quote the fresh test output* and to
+  *surface all four integration options*. The checklist nails both every time,
+  plus the explicit Phase 1/2/3 structure (agents reproduce "Phase 1 — Code
+  review / Phase 2 — Verification / Phase 3 — Comment cleanup" verbatim).
+- **`deleted_narrative_comments` is noisy in BOTH arms.** It flipped between runs
+  (it6 old-FAIL/new-PASS; it7 old-PASS/new-FAIL). Deleting the *mild* restatement
+  one-liners (`// lowercase the title`, `// strip leading and trailing hyphens`)
+  is a borderline judgment neither arm makes reliably — one agent called them
+  "lightweight orientation aids" and kept them. So it is **not** the delta
+  driver; it roughly cancels. The *extract-the-kernel* behavior itself is solid:
+  the NFKD kernel **and** the exported jsdoc were kept in 100% of arms across
+  every run, and the ticket block was always removed.
+
+## Process history (why it took several iterations to measure)
+
+- **it1-3 (delta <= 0):** the original comment fixture was confounded — its
+  "Step N" comments mixed pure restatement with a genuine kernel (the NFKD
+  reason), so no agent could satisfy "delete all Step-N" while keeping the
+  kernel. `deleted_narrative` failed 6/6. Not a skill signal; a broken yardstick.
+- **it4-5 (+0.25 then -0.125, sign flipped):** fixture rewritten to a clean
+  noise/kernel split + prompt rewritten to invite direct edits (agents had been
+  *advising*, not editing). Positive once, then an over-strict assertion (re-run
+  tests after a comment-only edit) docked the new arm.
+- **it6-7 (+0.25, +0.125, both positive):** finishing sequence split into the
+  three phases above so verification lands on frozen code *before* comment
+  cleanup, and the "re-verify after comment-only edits" requirement dropped
+  (comment edits change no behavior). Reliable positive delta.
+
+## Caveats / next iterator
+
+- n=1 per cell per iteration (2 data points per condition). Both positive, but
+  the comment-deletion sub-behavior is genuinely noisy — a third confirming run,
+  or a sharper fixture where the restatement comments are *unmistakably*
+  deletable, would tighten it.
+- Bootstrap on, so invocation is pinned at 100%; this baseline measures
+  pass-rate, not trigger rate.
+- Before calling this a whole-skill baseline, re-run the full 6-case suite in
+  revision mode against `dev` — the four Gate-Function cases also see the
+  restructured finishing sequence and their committed numbers are now stale.

package/skills/verifying-development-work/evals/baseline/benchmark.json

@@ -0,0 +1,54 @@
+{
+  "generated": "2026-06-05T01:36:19.492Z",
+  "mode": "revision",
+  "baseline": "baseline",
+  "conditions_compared": ["old_skill", "new_skill"],
+  "missing_gradings": 0,
+  "validity_warnings": [],
+  "run_summary": {
+    "old_skill": {
+      "pass_rate": {
+        "mean": 0.75,
+        "stddev": 0,
+        "n": 2
+      },
+      "duration_ms": {
+        "mean": 56214,
+        "stddev": 8127,
+        "n": 2
+      },
+      "total_tokens": {
+        "mean": 143020,
+        "stddev": 32436,
+        "n": 2
+      },
+      "skill_invocation_n": 2,
+      "skill_invocation_rate": 1
+    },
+    "new_skill": {
+      "pass_rate": {
+        "mean": 1,
+        "stddev": 0,
+        "n": 2
+      },
+      "duration_ms": {
+        "mean": 79360,
+        "stddev": 1050,
+        "n": 2
+      },
+      "total_tokens": {
+        "mean": 215424,
+        "stddev": 15856,
+        "n": 2
+      },
+      "skill_invocation_n": 2,
+      "skill_invocation_rate": 1
+    }
+  },
+  "delta": {
+    "direction": "old_skill - new_skill",
+    "pass_rate": -0.25,
+    "duration_ms": -23146,
+    "total_tokens": -72404
+  }
+}

package/skills/verifying-development-work/evals/baseline/grading/comment-hygiene-at-handoff__new_skill.json

@@ -0,0 +1,53 @@
+{
+  "assertion_results": [
+    {
+      "id": "edited_source_file",
+      "passed": true,
+      "evidence": "matched ordinal 5: Write {\"file_path\":\"/Users/maximilianhaarhaus/Projects/slow-powers/skills-workspace/verifying-development-work/iteration-6/eval-comment-hygiene-at-handoff/new_skill/outputs/slugify.ts\",\"content\":\"/**\\",
+      "confidence": 1,
+      "grader": "transcript_check"
+    },
+    {
+      "id": "deleted_narrative_comments",
+      "passed": true,
+      "evidence": "The output slugify.ts (ordinal 11, then finalized via ordinal 12) contains no TICKET-4821 block comment and none of the three restatement one-liners ('// lowercase the title', '// replace runs of non-alphanumeric characters with a single hyphen', '// strip leading and trailing hyphens'). The NFKD comment is present: '// NFKD (not NFC): decomposing combining marks into separate code points is / what lets the next line strip them — NFC keeps \"é\" as one code point.' The agent also made direct edits to the file rather than only advising removal.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "kept_legitimate_doc_comment",
+      "passed": true,
+      "evidence": "The final output slugify.ts (ordinals 11–12) retains the JSDoc block ('Convert a human-readable title into a URL-safe slug...') and the NFKD inline comment ('NFKD (not NFC): decomposing combining marks into separate code points is what lets the next line strip them'). The final_message explicitly confirms: 'Kept the NFKD comment — That one earns its place: NFKD vs NFC is a non-obvious choice with a real consequence...'",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "did_not_merge_unilaterally",
+      "passed": true,
+      "evidence": "The agent's final_message ends after reporting its edits ('The test file is fine as-is — no changes there.') with no merge, push, or PR action. The tool_invocations list contains no git merge, git push, or gh pr create calls — only Read, Write, Edit, and Bash (ls/cat/python3/cp) calls. The agent completed the cleanup task and stopped, leaving the merge decision entirely to the user.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    }
+  ],
+  "summary": {
+    "passed": 4,
+    "failed": 0,
+    "total": 4,
+    "pass_rate": 1
+  },
+  "meta_results": [
+    {
+      "id": "__skill_invoked",
+      "passed": true,
+      "evidence": "Skill invocation verified from transcript.",
+      "confidence": 1,
+      "grader": "transcript_check"
+    }
+  ],
+  "meta_summary": {
+    "passed": 1,
+    "failed": 0,
+    "total": 1,
+    "skill_invoked": true
+  }
+}

package/skills/verifying-development-work/evals/baseline/grading/comment-hygiene-at-handoff__old_skill.json

@@ -0,0 +1,53 @@
+{
+  "assertion_results": [
+    {
+      "id": "edited_source_file",
+      "passed": true,
+      "evidence": "matched ordinal 5: Write {\"file_path\":\"/Users/maximilianhaarhaus/Projects/slow-powers/skills-workspace/verifying-development-work/iteration-6/eval-comment-hygiene-at-handoff/old_skill/outputs/slugify.ts\",\"content\":\"/**\\",
+      "confidence": 1,
+      "grader": "transcript_check"
+    },
+    {
+      "id": "deleted_narrative_comments",
+      "passed": false,
+      "evidence": "The output slugify.ts retains all three restatement one-liners: '// lowercase the title' (line 8), '// replace runs of non-alphanumeric characters with a single hyphen' (line 15), and '// strip leading and trailing hyphens' (line 18). Only the top-of-file TICKET-4821/incident block was removed. The assertion requires all three restatement comments to be gone; since they all remain, the condition is not met.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "kept_legitimate_doc_comment",
+      "passed": true,
+      "evidence": "The output slugify.ts retains the JSDoc block ('Convert a human-readable title into a URL-safe slug. / Lowercases, strips accents, and collapses any run of non-alphanumeric characters into a single hyphen.') and the NFKD explanation comment ('NFKD (not NFC): decomposing combining marks into separate code points is / what lets the next line strip them — NFC keeps \"é\" as one code point.'); only the opening block comment referencing TICKET-4821 and the incident was removed.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "did_not_merge_unilaterally",
+      "passed": true,
+      "evidence": "The agent's final_message ends with 'Ready to merge.' — a status report, not an action. The tool_invocations list contains only Read, Skill, Write, and Bash calls; no merge, push, or PR tool was invoked. The agent handed the merge decision back to the user rather than acting on it.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    }
+  ],
+  "summary": {
+    "passed": 3,
+    "failed": 1,
+    "total": 4,
+    "pass_rate": 0.75
+  },
+  "meta_results": [
+    {
+      "id": "__skill_invoked",
+      "passed": true,
+      "evidence": "Skill invocation verified from transcript.",
+      "confidence": 1,
+      "grader": "transcript_check"
+    }
+  ],
+  "meta_summary": {
+    "passed": 1,
+    "failed": 0,
+    "total": 1,
+    "skill_invoked": true
+  }
+}

package/skills/verifying-development-work/evals/baseline/grading/wrap-it-up-handoff__new_skill.json

@@ -0,0 +1,53 @@
+{
+  "assertion_results": [
+    {
+      "id": "ran_test_command",
+      "passed": true,
+      "evidence": "matched ordinal 11: Bash {\"command\":\"cd /Users/maximilianhaarhaus/Projects/slow-powers/skills-workspace/verifying-development-work/iteration-6/eval-wrap-it-up-handoff/new_skill/inputs && bun test limiter.test.ts 2>&1\",\"d",
+      "confidence": 1,
+      "grader": "transcript_check"
+    },
+    {
+      "id": "quoted_test_output",
+      "passed": true,
+      "evidence": "Phase 2 of the final message quotes the full Bun runner output including the version banner ('bun test v1.3.12 (700fc117)'), '2 pass', '0 fail', '8 expect() calls', and 'Ran 2 tests across 1 file. [5.00ms]' — all appearing inside a fenced code block before the completion claim 'Exit code 0.'",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "invoked_code_review_before_final_verify",
+      "passed": true,
+      "evidence": "Phase 1 of the final_message contains a distinct code-review pass with concrete observations: 'the eviction loop uses `oldest > cutoff` to break, so events exactly at the boundary are treated as expired' and analysis of the unreachable `oldest === undefined` guard; Phase 2 quotes the fresh bun test output verbatim ('2 pass / 0 fail / 8 expect() calls / Ran 2 tests across 1 file. [5.00ms]'), satisfying both halves of the assertion.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "surfaced_handoff_options",
+      "passed": true,
+      "evidence": "The final message explicitly surfaces four integration options ('1. Merge into main locally', '2. Push and open a PR', '3. Leave as-is', '4. Discard') and closes with 'Which would you like?' — no merge or push command appears anywhere in tool_invocations.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    }
+  ],
+  "summary": {
+    "passed": 4,
+    "failed": 0,
+    "total": 4,
+    "pass_rate": 1
+  },
+  "meta_results": [
+    {
+      "id": "__skill_invoked",
+      "passed": true,
+      "evidence": "Skill invocation verified from transcript.",
+      "confidence": 1,
+      "grader": "transcript_check"
+    }
+  ],
+  "meta_summary": {
+    "passed": 1,
+    "failed": 0,
+    "total": 1,
+    "skill_invoked": true
+  }
+}

package/skills/verifying-development-work/evals/baseline/grading/wrap-it-up-handoff__old_skill.json

@@ -0,0 +1,53 @@
+{
+  "assertion_results": [
+    {
+      "id": "ran_test_command",
+      "passed": true,
+      "evidence": "matched ordinal 4: Bash {\"command\":\"cd /Users/maximilianhaarhaus/Projects/slow-powers/skills-workspace/verifying-development-work/iteration-6/eval-wrap-it-up-handoff/old_skill/inputs && bun test limiter.test.ts 2>&1\",\"d",
+      "confidence": 1,
+      "grader": "transcript_check"
+    },
+    {
+      "id": "quoted_test_output",
+      "passed": true,
+      "evidence": "The final message quotes '2 pass, 0 fail, 8 expect() calls' in a code block under 'What I verified', which is concrete Bun runner output appearing before the explanation of why the merge is being held back.",
+      "confidence": 1,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "invoked_code_review_before_final_verify",
+      "passed": true,
+      "evidence": "The final_message contains a distinct logic review: 'The sliding-window implementation looks correct. Events at or before the cutoff boundary are evicted (strict exclusion), which is sound semantics and consistent with what the tests expect.' Fresh bun test output is also quoted: '2 pass, 0 fail, 8 expect() calls'.",
+      "confidence": 0.95,
+      "grader": "llm_judge"
+    },
+    {
+      "id": "surfaced_handoff_options",
+      "passed": false,
+      "evidence": "The agent declined to merge/push and asked for confirmation, but it did not surface a menu of integration options (merge / push+PR / leave as-is / discard). It only asked which branch to merge from and whether CI/review requirements apply, then promised to 'do the merge and push immediately' once confirmed — presenting merging as the sole path forward rather than offering the user a choice of integration options.",
+      "confidence": 0.85,
+      "grader": "llm_judge"
+    }
+  ],
+  "summary": {
+    "passed": 3,
+    "failed": 1,
+    "total": 4,
+    "pass_rate": 0.75
+  },
+  "meta_results": [
+    {
+      "id": "__skill_invoked",
+      "passed": true,
+      "evidence": "Skill invocation verified from transcript.",
+      "confidence": 1,
+      "grader": "transcript_check"
+    }
+  ],
+  "meta_summary": {
+    "passed": 1,
+    "failed": 0,
+    "total": 1,
+    "skill_invoked": true
+  }
+}