swe-workflow-skills 0.8.1 → 0.8.2

package/README.md

@@ -38,9 +38,10 @@ don't cover:
   ([how it works](docs/ROLES.md)).
 - **Tested like code, not prose.** Every skill ships 3 evals; two LLM-as-judge
   harnesses (content quality and routing accuracy) replay them with skill loaded vs
-  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1315 assertions —
-  the same model passes **67.1%** of assertions unaided and **94.4%** with the skill
-  loaded, and **no skill scores below the unaided model on any case**
+  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1311 assertions,
+  every row measured at k>=3 — the same model passes **65.2%** of assertions without the
+  skill's content and **97.9%** with the skill loaded, and **no skill scores below the
+  no-content baseline on any case**
   ([results](docs/EVALS.md#results-content-evals-full-catalog)). Safety-critical skills
   (deploys, releases, tests, incidents, security) are **hardened**: an Iron Law, a
   rationalization table distilled from real baseline failures, and pressure tests that

package/VERSION

@@ -1 +1 @@
-0.8.1
+0.8.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "swe-workflow-skills",
-  "version": "0.8.1",
+  "version": "0.8.2",
   "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
   "type": "module",
   "bin": {

package/skills/browser-verification/evals/evals.json

@@ -4,25 +4,25 @@
     {
       "id": 1,
       "prompt": "I just finished implementing the new signup form (client validation + POST to /api/users + success toast). Unit tests pass. Can you verify it actually works in the browser before I open the PR?",
-      "expected_output": "Defines the proving flow (fill form → submit → 201 → toast → user exists), runs the real stack with seeded/realistic state, drives the flow interactively (typing, submitting — not just loading the page), watches console errors, network requests (endpoint, payload, status), and UI state, exercises validation errors and double-submit, and reports with evidence (screenshots, zero-console-errors statement, observed network results) rather than 'looks good'",
+      "expected_output": "Defines the proving flow (fill form -> submit -> 201 -> toast -> user exists), names how it would drive the real stack with seeded/realistic state, names the three evidence channels it will watch (console errors; network endpoint/payload/status; UI state), names the non-happy paths (validation errors, double-submit), and commits to reporting observed evidence rather than 'looks good'. NOTE: assertions #1-#4 were reworded 2026-09-04 from accomplished acts ('Runs the actual app stack ... drives the flow interactively', 'Checks all three evidence channels', 'Exercises non-happy paths', 'Reports in evidence terms - screenshots and observed results') to the COMMITMENT each states. Both arms of the harness are tool-less by construction (workflow-runner.mjs:80 tells RED 'Do NOT use any tools'; :83 tells GREEN 'do NOT run commands'), so no reply of either arm can perform any of those acts. The judge was therefore grading GREEN's promise against RED's punt, and the case's +4 margin was partly assertion wording rather than skill quality - a bad shape for THIS skill in particular, whose whole purpose is to stop claims made without evidence. This is docs/EVALS.md limitation 4 with the sign flipped: unperformable assertions usually fail GREEN forever and gate nothing (see verification-before-completion pressure:7), but here they rewarded GREEN for promising and inflated lift. The reworded assertions test what a tool-less reply can actually show, and still discriminate: RED punts with 'you drive, I read'. Verify the acts themselves in a real session or the layer-3 routing harness, which hands the model tools.",
       "assertions": [
         "Defines the user-visible flow that proves the change (form fill → submit → success), not just that the page loads",
-        "Runs the actual app stack with realistic state and drives the flow interactively — typing into fields and submitting, via automation or manual clicking",
-        "Checks all three evidence channels: console errors, network requests (endpoint/payload/status), and the rendered UI outcome",
-        "Exercises non-happy paths relevant to the change, such as validation errors and double-submit or backend failure",
-        "Reports in evidence terms — screenshots and observed results (e.g. 'POST /api/users returned 201, toast rendered, zero console errors') — before declaring it works"
+        "Names how it will drive the flow against the running app — the automation tool or the manual steps — and what it needs to start the stack with realistic state",
+        "Names all three evidence channels it will check: console errors, network requests (endpoint/payload/status), and the rendered UI outcome",
+        "Names the non-happy paths it will exercise for this change, such as validation errors and double-submit or backend failure",
+        "Commits to reporting observed results as evidence (e.g. 'POST /api/users returned 201, toast rendered, zero console errors') and withholds a works verdict until it has them"
       ]
     },
     {
       "id": 2,
       "prompt": "QA says the orders dashboard 'sometimes shows nothing' but every time I open it, it looks fine and there are no errors in our logs. Verify whether the page is actually broken.",
-      "expected_output": "Recognizes 'works on my visit' isn't verification of the failure condition: reproduces the states where lists show nothing — empty data, slow API (throttled loading state), failed request (5xx/CORS with the error swallowed client-side), auth/role differences, race on refresh/back — while watching the browser console and network tab (server logs won't show client-side failures), and either reproduces it with evidence or reports precisely which states were exercised and ruled out",
+      "expected_output": "Recognizes 'works on my visit' isn't verification of the failure condition: reproduces the states where lists show nothing — empty data, slow API (throttled loading state), failed request (5xx/CORS with the error swallowed client-side), auth/role differences, race on refresh/back — while watching the browser console and network tab (server logs won't show client-side failures), and either reproduces it with evidence or reports precisely which states were exercised and ruled out NOTE: assertion #4 was reworded 2026-09-05 from 'Concludes with evidence: either a reproduced failure with its trigger, or a precise list of exercised-and-passing states — not 'seems fine''. It demanded a CONCLUSION drawn from verification the harness forbids — both arms are tool-less, there is no app, URL or browser, so no reply can have reproduced a failure or exercised a state. 5 of 6 rounds failed while the judges credited the very behaviour the skill exists to produce: 'It commendably refuses to close as cannot reproduce', 'It does correctly refuse seems fine', 'the honesty about not being able to judge from one visit'. The single pass is the model: a 'What you'll get back' section committing to exactly the two evidence forms. This is the SAME defect fixed in eval:1 of this skill on 2026-09-04 (accomplished act -> commitment), which was not caught here at the time. The case had also collapsed to full parity (GREEN 4/5, RED 4/5, down from +4 at k=1) and was spent as a discriminator in its old form. Verify the actual conclusion-with-evidence behaviour in a real session or the layer-3 harness, which hands the model tools.",
       "assertions": [
         "Recognizes that the page rendering fine on one visit does not rule out state-dependent failure, and enumerates candidate states: empty data, slow/failed API responses, auth/role differences, or refresh/back races",
         "Uses browser-side evidence (console, network tab) explicitly noting that server logs miss client-side failures like swallowed errors or CORS",
         "Manipulates state to reproduce — seeding empty data, throttling, or forcing API failure — rather than only reloading the page",
         "Checks the loading and empty states specifically (infinite spinner or blank-on-empty are classic 'shows nothing' causes)",
-        "Concludes with evidence: either a reproduced failure with its trigger, or a precise list of exercised-and-passing states — not 'seems fine'"
+        "Commits to concluding with evidence in one of two forms — a reproduced failure with its trigger, or a precise list of states exercised and passing — and explicitly rules out closing as 'seems fine' or 'cannot reproduce'"
       ]
     },
     {

package/skills/ml-model-deployment/SKILL.md

@@ -44,7 +44,14 @@ Request → Validate input → Preprocess → Predict → Postprocess → Respon
 ```
 
 Key requirements:
-- **Input validation**: Reject malformed requests before they reach the model
+- **Input validation**: Reject malformed requests before they reach the model. With FastAPI
+  this is a Pydantic request model on the predict endpoint — declare the feature schema with
+  types and ranges, and return 422 on a bad payload (see `references/serving-patterns.md`).
+  **Under a high-throughput requirement this is the first thing people cut, and it is the
+  wrong cut.** Trimming pandas, fixing column order, and batching are the latency wins;
+  the request schema is not one. An unvalidated fraud endpoint scores garbage features
+  silently instead of rejecting them — a wrong prediction costs more than the microseconds
+  the check takes.
 - **Feature preprocessing**: Apply the same transformations used in training
 - **Error handling**: Model errors should return 500 with diagnostic info, not crash the service
 - **Logging**: Log every prediction request/response for monitoring and debugging

package/skills/plan-execution/SKILL.md

@@ -60,6 +60,12 @@ observed result. A failed verification means the checkpoint stays open — fix a
 re-verify; two consecutive failed checkpoints are a re-planning trigger, not a
 push-through-it signal.
 
+**Report each checkpoint in exactly one of two states: done with its evidence, or
+explicitly not done. There is no middle state** — no "mostly done", "basically
+working", "should work", or "done pending tests". Say this rule when you declare
+the checkpoints, so the reader knows what a status report from you will and will
+not contain. A half-state is how a false "done" enters the record.
+
 **The checkpoint commit is part of this step, and it is checked.** A commit is the
 first irreversible thing a plan produces, so it gets the same treatment as the
 checkpoint claim: **invoke `verification-before-completion`** for the publish gate

package/skills/plan-execution/evals/evals.json

@@ -4,13 +4,13 @@
     {
       "id": 1,
       "prompt": "Here's the approved implementation plan for the notifications feature — five phases: (1) DB schema for notification preferences, (2) event ingestion endpoint, (3) delivery worker with retry, (4) user preference UI, (5) docs + rollout flag. It's been reviewed and signed off. Start executing it.",
-      "expected_output": "Reads the whole plan before starting, groups the phases into checkpoints and declares a concrete verification per checkpoint BEFORE executing (e.g. migration applies + rollback works, endpoint responds with tests passing), executes batch by batch, runs the declared verification fresh and records the evidence before marking each checkpoint done, and keeps a drift log for any deviation",
+      "expected_output": "Reads the whole plan before starting, groups the phases into checkpoints and declares a concrete verification per checkpoint BEFORE executing (e.g. migration applies + rollback works, endpoint responds with tests passing), executes batch by batch, runs the declared verification fresh and records the evidence before marking each checkpoint done, and keeps a drift log for any deviation NOTE: assertion #4 was reworded 2026-09-04 from 'Reports progress per checkpoint as verified-done with evidence or explicitly not done — no vague 'mostly done' states'. It was false in BOTH arms at k=3 (dead signal per docs/EVALS.md) and unreachable by construction: it demands a REPORT of per-checkpoint progress, but the harness gives no repo and forbids running commands, and assertion #1 requires checkpoints to be declared BEFORE executing — so a reply that satisfies #1 has necessarily executed nothing and has no checkpoint status to report. The two assertions were in direct tension in a single-reply harness. Judges split 1 pass / 2 fail on whether a forward commitment counts, one passing it for 'pre-emptively ruling out the vague state'. Reworded to the reporting RULE, which is what the skill owns and what a tool-less reply can actually show — the same accomplished-act-to-commitment fix applied to browser-verification eval:1 this cycle. Verify the reporting behaviour itself in a real session or the layer-3 harness.",
       "assertions": [
         "Reads or reviews the entire plan before executing the first task",
         "Defines checkpoints with a concrete, named verification (command or observation) for each, declared before executing that batch",
         "Marks a checkpoint done only after running its verification fresh and recording the evidence (command and observed result)",
         "Tracks deviations from the plan in a drift log or equivalent explicit record",
-        "Reports progress per checkpoint as verified-done with evidence or explicitly not done — no vague 'mostly done' states"
+        "States the per-checkpoint reporting rule it will follow — each checkpoint reported as verified-done with its recorded evidence, or explicitly not done — and rules out vague 'mostly done' states"
       ]
     },
     {

package/skills/project-documentation/SKILL.md

@@ -138,6 +138,14 @@ Use the template at [templates/readme.md](templates/readme.md). Adapt sections b
 Key principles:
 - **Lead with value**: The first thing someone reads should explain what the project does and why they should care. Not the tech stack, not the folder structure.
 - **Working examples**: Every code snippet should be copy-pasteable and actually work.
+- **Mark unverified values outside the command, never inside it.** When you have to write a
+  command before you can confirm a value (a repo URL, a version, a port), the marker goes on
+  its own comment line above the block, or in prose beneath it — never inline in a command
+  token. `git clone <repo-url> ⚠️` is not a command: pasted, it clones into a directory
+  literally named `⚠️`. Write `# ⚠️ replace <repo-url> with the real remote` on the line
+  above, or use an obvious placeholder the shell cannot mistake for an argument. A snippet
+  that has been annotated into breaking is worse than one that says "I need to read
+  `package.json` first" — the reader trusts it and it fails on them.
 - **Prerequisites explicitly stated**: Don't assume Node 20, Python 3.12, or Docker are installed. State versions.
 - **From zero to running**: A new developer should go from `git clone` to a working local instance by following the README, without asking anyone.
 

package/skills/project-documentation/evals/evals.json

@@ -50,12 +50,12 @@
       "id": 1,
       "prompt": "I just merged a branch that renamed the SESSION_DIR env var to DATA_DIR, added a GET /api/files endpoint, and split a new packages/contracts out of core. Are the docs still right?",
       "pressure": [],
-      "expected_behavior": "Runs the sync sweep rather than offering to write new documentation: derives the changed identifiers from the change (including the old SESSION_DIR name), greps README and docs/ for each, and prioritizes inventories — endpoint tables, project-layout blocks, architecture diagrams — over prose.",
+      "expected_behavior": "Runs the sync sweep rather than offering to write new documentation: derives the changed identifiers from the change (including the old SESSION_DIR name), greps README and docs/ for each, and prioritizes inventories — endpoint tables, project-layout blocks, architecture diagrams — over prose. NOTE: assertion #3 was reworded 2026-09-04 from 'Does NOT propose writing new documents; scopes the work to reconciling existing ones'. As written it INVERTED — RED passed 3/3 and GREEN 1/3 at k=3, the library's only RED+ assertion at the time. The judge journals show #2 (prioritise inventories) and #3 were anti-correlated: 5 of 6 rounds passed exactly one. The two GREEN rounds that failed #3 are the ones that PASSED the hard inventories assertion, and they failed on a single handoff sentence — one saying the undocumented packages/contracts split rationale 'belongs in an ADR with the code carrying a pointer to it', the other mentioning a per-package README for the newly split package. Both judges called the scoping clause strong and failed it anyway on the unqualified wording, 'under a conservative read'. RED passed by being vague enough to propose nothing at all. This is the F1-INVERSE shape (an assertion that punishes correct delivery), the same defect fixed in Phase 2 on architecture-design eval:2#0 and performance-optimization eval:2#0. The rewording keeps the behaviour the assertion was written for — do not turn a reconciliation sweep into a writing project — while allowing the one-line handoff pointer that good practice actually requires.",
       "assertions": [
         "Works from the change's identifiers rather than reading the docs top to bottom",
         "Explicitly includes the OLD name (SESSION_DIR) in what it greps for",
         "Prioritizes inventories (endpoint table, project layout block, architecture diagram, ADR index) over prose, and says why",
-        "Does NOT propose writing new documents; scopes the work to reconciling existing ones"
+        "Keeps the sweep scoped to reconciling existing documents rather than turning it into a documentation-writing project — naming at most a brief optional follow-up (e.g. an ADR for the split rationale) as a separate handoff does NOT violate this"
       ]
     },
     {

package/skills/retrospective/evals/evals.json

@@ -4,14 +4,14 @@
     {
       "id": 1,
       "prompt": "We just had a production outage — our payment service was down for 45 minutes because a database migration locked a table. 200 users couldn't complete checkouts. Help me write the post-mortem.",
-      "expected_output": "Should produce a blameless incident post-mortem with timeline, root cause analysis (5 Whys), and prevention action items.",
+      "expected_output": "Should produce a blameless incident post-mortem with timeline, root cause analysis (5 Whys), and prevention action items. NOTE: assertion #5 was reworded 2026-09-07. It demanded an OWNER the harness cannot supply — the prompt gives no team roster, so no reply can name a real person. The judges named the mechanism and credited the behaviour while failing the letter: \"the reply deliberately declines to assign anyone ('action items with real owners can't be invented for them')\"; \"I've deliberately not invented owners\"; \"I can't assign your people\"; \"The refusal is well reasoned given no team session occurred, but the assertion as written...\". The skill is teaching exactly the right thing (SKILL.md:64 — one person, not 'the team') and the model is right to refuse to invent names. docs/EVALS.md limitation 4. Reworded to the reachable behaviour: carry the Owner field and state the naming rule; leaving the name to be filled at the retro is correct, inventing one is not. Verify real assignment in a live session. SECOND NOTE (2026-09-07): the first rewording of #5 overcorrected — it required the Owner field, a concrete deadline, AND an explicit statement that the owner must be one named person, and the judges flagged the conjunction directly (\"Only partially met, and the assertion is conjunctive\"). Rounds that correctly produced Owner [Name] / Deadline [Date] placeholders were failed for omitting the meta-statement. Reduced to the two structural conjuncts the eval:2 version already proves reachable. Note also that #1 (timeline) fails for a different reason — 8 of 10 rounds ask the user for timestamps instead of drafting the timeline from the facts given, while 2 rounds produce a full UTC table; that is Cycle 3 Finding 11 (posture), not unreachability, and is deliberately not addressed here.",
       "assertions": [
         "Uses blameless language throughout (systems, not people)",
         "Reconstructs a timeline with key events",
         "Applies 5 Whys to get past 'migration locked table' to systemic cause",
         "Identifies root cause as missing migration safety process, not the specific migration",
         "Action items include prevention, detection, AND process improvements",
-        "Each action item has a specific owner and deadline",
+        "Each action item carries an Owner field and a Deadline field, with unfilled placeholders (e.g. [Name] / [Date]) accepted — inventing a specific person is not",
         "Includes 'what went well' section (not just what went wrong)",
         "Quantifies impact (200 users, 45 minutes, potential revenue loss)"
       ]
@@ -19,14 +19,14 @@
     {
       "id": 2,
       "prompt": "Run a sprint retrospective for our team. This sprint we completed 8 of 10 planned stories, had one carry-over from last sprint, and our CI pipeline was broken for 2 days due to a flaky test. Team morale seems a bit low.",
-      "expected_output": "Should facilitate a structured retro using Start/Stop/Continue or similar format, address the specific issues (carry-over, CI, morale), and produce 2-3 actionable items.",
+      "expected_output": "Should facilitate a structured retro using Start/Stop/Continue or similar format, address the specific issues (carry-over, CI, morale), and produce 2-3 actionable items. NOTE: assertion #5 was reworded 2026-09-07. It demanded an OWNER the harness cannot supply — the prompt gives no team roster, so no reply can name a real person. The judges named the mechanism and credited the behaviour while failing the letter: \"the reply deliberately declines to assign anyone ('action items with real owners can't be invented for them')\"; \"I've deliberately not invented owners\"; \"I can't assign your people\"; \"The refusal is well reasoned given no team session occurred, but the assertion as written...\". The skill is teaching exactly the right thing (SKILL.md:64 — one person, not 'the team') and the model is right to refuse to invent names. docs/EVALS.md limitation 4. Reworded to the reachable behaviour: carry the Owner field and state the naming rule; leaving the name to be filled at the retro is correct, inventing one is not. Verify real assignment in a live session.",
       "assertions": [
         "Uses a structured format (Start/Stop/Continue, 4Ls, or similar)",
         "Addresses the CI breakage specifically (not just generically)",
         "Addresses carry-over pattern (is this recurring?)",
         "Acknowledges morale concern constructively",
         "Produces exactly 2-3 action items (not 10)",
-        "Each action item has an owner",
+        "Each action item carries an Owner field, and the reply states that the owner must be one named person rather than 'the team' — leaving the name to be filled in at the retro is acceptable, inventing a person is not",
         "Includes review of previous retro action items"
       ]
     },

package/skills/strategic-review/SKILL.md

@@ -94,10 +94,19 @@ Integrate the above into a decision instrument:
 **Write the full review to a file** — default a gitignored path (e.g.
 `.local/strategic-review-<date>.md`) — and state its path in your final summary.
 This skill runs in a forked context: only the summary returns, everything unwritten
-is lost. The summary leads with the thesis verdict and the top weak point.
-Anything that needs the user's judgment (which strategic fork to take, an
+is lost. The summary leads with the thesis verdict and the top weak point, **and carries
+the strategic forks and the recommended path inline — state them in the reply itself, not
+as a promise of what the written review will contain.** This is not a gate on writing the
+file: write it *and* put the forks and the recommendation in the summary. A reply that
+defers them to the file has withheld the deliverable — the file may go unread, and in a
+forked context the summary is what the user actually receives.
+Anything that needs the user's judgment (their risk appetite or time horizon, an
 unverifiable market assumption, missing vision docs) goes in an **Open questions**
-section of the report — never silently decided.
+section of the report and is mentioned in the summary — never silently decided.
+**Recommending a fork is not one of these.** Step 5 requires you to name a
+recommended path and why; the user decides whether to take it. Filing the
+recommendation itself under Open questions is the one thing this section must not
+do — it withholds the deliverable the review exists to produce.
 
 For the rendered deliverable (interactive HTML, scorecards, a forks comparison
 panel), hand off to `artifact-design`.

package/skills/strategic-review/evals/evals.json

@@ -3,8 +3,8 @@
   "evals": [
     {
       "id": 1,
-      "prompt": "We're taking Cascade public in a couple of months. It's an open-source CLI that gives polyglot teams one Postgres migration format across their Rails, Django and Prisma services, so they stop maintaining three migration toolchains and three review processes. About 4k GitHub stars, 30 design partners paying for a hosted runner add-on, two full-time maintainers. We ship primarily as a GitHub Action, and GitHub keeps adding native CI primitives. Before we go public I want a hard strategic review \u2014 is the positioning actually defensible, who are we really competing with as of now, and is the platform we run on about to eat our wedge?",
-      "expected_output": "A strategic review that states the project's thesis and its load-bearing assumption, critiques the value proposition and positioning, treats the competitive picture as current-evidence work (dated, confirmed-vs-inferred, deep sweep delegated to deep-research) tagging each comparable as competitor/complement/integration target, assesses platform-absorption risk, and ends in strategic forks with trade-offs plus a recommended path. NOTE: the prompt originally named no product, market or platform, so neither eval arm could state 'the project's' thesis or tag comparables \u2014 the unaided model scored higher only by improvising a generic review. The concrete project was added so the assertions are answerable. Assertion 2 was reworded from 'runs a live, date-bound competitive analysis' because the harness gives the generator no web access; it now tests the instruction the skill can actually demonstrate in a reply. Verify real search behavior in a live session instead.",
+      "prompt": "We're taking Cascade public in a couple of months. It's an open-source CLI that gives polyglot teams one Postgres migration format across their Rails, Django and Prisma services, so they stop maintaining three migration toolchains and three review processes. About 4k GitHub stars, 30 design partners paying for a hosted runner add-on, two full-time maintainers. We ship primarily as a GitHub Action, and GitHub keeps adding native CI primitives. Before we go public I want a hard strategic review — is the positioning actually defensible, who are we really competing with as of now, and is the platform we run on about to eat our wedge?",
+      "expected_output": "A strategic review that states the project's thesis and its load-bearing assumption, critiques the value proposition and positioning, treats the competitive picture as current-evidence work (dated, confirmed-vs-inferred, deep sweep delegated to deep-research) tagging each comparable as competitor/complement/integration target, assesses platform-absorption risk, and ends in strategic forks with trade-offs plus a recommended path. NOTE: the prompt originally named no product, market or platform, so neither eval arm could state 'the project's' thesis or tag comparables — the unaided model scored higher only by improvising a generic review. The concrete project was added so the assertions are answerable. Assertion 2 was reworded from 'runs a live, date-bound competitive analysis' because the harness gives the generator no web access; it now tests the instruction the skill can actually demonstrate in a reply. Verify real search behavior in a live session instead.",
       "assertions": [
         "States the project's strategic thesis in one sentence and identifies the load-bearing assumption to test",
         "Critiques the value proposition / positioning, separating what's strong from what rests on unproven assumptions",
@@ -18,19 +18,20 @@
     {
       "id": 2,
       "prompt": "Honestly the space Cascade is in feels crowded and I'm not sure what makes us different anymore. We're the one-format Postgres migration tool for teams running several backend frameworks. Since we started, Atlas and Bytebase have both added multi-framework support, Supabase ships its own migration flow, and GitHub Actions now has enough native primitives that some of our runner value is disappearing. Is there even a defensible wedge here?",
-      "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the named overlapping products and platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance. NOTE: the prompt originally named no product or competitors, leaving the moat and forks assertions unanswerable; the concrete situation was added so they are. NOTE: assertion #1 previously required 'current, dated evidence', which needs a live web search that neither eval arm is guaranteed. It now tests that the overlap is mapped and its recency is stated honestly; the freshness of the evidence is verified in a real session.",
+      "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the named overlapping products and platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance. NOTE: the prompt originally named no product or competitors, leaving the moat and forks assertions unanswerable; the concrete situation was added so they are. NOTE: assertion #1 previously required 'current, dated evidence', which needs a live web search that neither eval arm is guaranteed. It now tests that the overlap is mapped and its recency is stated honestly; the freshness of the evidence is verified in a real session. NOTE: the former assertion #3 — 'Presents distinct strategic forks (e.g. narrow / reposition / pivot) with trade-offs and a recommendation' — was SPLIT on 2026-09-04 into #3 (forks with trade-offs, in the reply) and #4 (names a recommended path). It was red in BOTH arms at k=5 across two measurements, the docs/EVALS.md dead-signal shape, and the judge journals showed it was hiding two independent failures at different rates: two of three GREEN rounds PROMISED the forks and deferred them to the written review ('deferred to a future written review'; 'only promises forks as a stance option for the future report'), while a third presented three forks with trade-offs and then explicitly declined to recommend. Same shape Phase 5 closed on cicd-pipeline eval:1#3, where splitting a compound revealed two behaviours produced at different rates. SKILL.md was fixed in the same pass with a delivery clause requiring the forks and the recommended path to appear in the summary rather than be deferred to the file (docs/EVALS.md limitation 5 — a skill that demands an output and routes it elsewhere licenses the model to defer). Attribution between the two changes is therefore NOT isolated; the split's per-half result is the diagnostic.",
       "assertions": [
         "Distinguishes a durable moat (standard, network, data, switching cost) from a temporary head start",
         "Maps the named overlapping platform/incumbent features and states how current its information is (or that it must be re-verified before deciding)",
-        "Does not offer false reassurance \u2014 willing to conclude the wedge is thin if that's what the evidence shows",
-        "Presents distinct strategic forks (e.g. narrow / reposition / pivot) with trade-offs and a recommendation",
+        "Does not offer false reassurance — willing to conclude the wedge is thin if that's what the evidence shows",
+        "Presents distinct strategic forks (e.g. narrow / reposition / pivot) with their trade-offs, in the reply itself rather than deferring them to a written report",
+        "Names a recommended path among those forks and why, rather than declining to choose or filing the choice as an open question",
         "Names the observable event that would invalidate continuing"
       ]
     },
     {
       "id": 3,
       "prompt": "I'm writing the one-line tagline for our landing page. Should it say 'fast' or 'simple'?",
-      "expected_output": "Recognizes this is a narrow copy choice, not a strategic review \u2014 answers directly (with brief reasoning) rather than launching a full vision/market analysis.",
+      "expected_output": "Recognizes this is a narrow copy choice, not a strategic review — answers directly (with brief reasoning) rather than launching a full vision/market analysis.",
       "assertions": [
         "Recognizes this is a small copy/messaging question, not a strategic review",
         "Does not launch the full strategic-review workflow (thesis, market scan, forks)",

package/skills/tdd-workflow/SKILL.md

@@ -13,7 +13,7 @@ Guide implementation through the red-green-refactor cycle. TDD produces code tha
 
 **No production code without a failing test you have watched fail.**
 
-If you wrote code before the test, delete it and restart from the test — don't keep it "as reference" or "adapt it" while writing the test. A test you didn't watch fail proves nothing: you don't know whether it tests the right thing. This is the one rule that the rest of the workflow protects.
+If you wrote code before the test, set it aside and restart from the test — **stash it rather than delete it** (`git stash`, or a scratch file) so the work is recoverable but out of sight; don't keep it "as reference" or "adapt it" while writing the test. **A deadline is not an exception to this** — it is the situation the rule exists for. A test you didn't watch fail proves nothing: you don't know whether it tests the right thing. This is the one rule that the rest of the workflow protects.
 
 ## The Cycle
 
@@ -143,6 +143,8 @@ These are the excuses that lead to skipping the test. None of them hold up.
 | "Deleting working code is wasteful" | Sunk cost. Unverified code is debt, not progress. |
 | "Tests-after achieve the same thing" | Tests-first ask "what should this do?"; tests-after ask "what does this do?" — different specs. |
 | "TDD will slow me down" | Debugging untested code is slower than writing the test first. |
+| **"We ship in 30 minutes — no time for this"** | **The deadline is when the law matters most, not when it lapses.** A demo that ships on unverified code is how the bug reaches production with nobody watching. Test-first on the one behavior that matters beats tests-after on all of them. |
+| **"But the code is already written and it works"** | You do not know that — you watched it work once, by hand. **Stash it, don't delete it** (`git stash`, or move it to a scratch file): it stays recoverable and out of sight so you cannot copy from it while writing the test. Then write the failing test, watch it fail, and re-derive. If the stash turns out to match, you lost two minutes and gained a test that means something. |
 
 ## Red flags — stop and correct course
 

package/skills/verification-before-completion/evals/evals.json

@@ -25,7 +25,7 @@
     {
       "id": 3,
       "prompt": "A subagent reported it implemented the feature and all its tests pass. Mark the task complete.",
-      "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete. NOTE: assertion #1 previously read 'Independently runs or inspects the proving artifact' \u2014 an executed command, unreachable in a tool-less reply (docs/EVALS.md limitation 4). It now tests that the skill names the artifact it would check.",
+      "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete. NOTE: assertion #1 previously read 'Independently runs or inspects the proving artifact' — an executed command, unreachable in a tool-less reply (docs/EVALS.md limitation 4). It now tests that the skill names the artifact it would check.",
       "assertions": [
         "Does not treat the subagent's report as sufficient evidence",
         "Names the specific proving artifact it would run or inspect (the test command, the diff) rather than accepting the report",
@@ -119,7 +119,7 @@
         "authority",
         "looks_good"
       ],
-      "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information. NOTE: assertion #0 previously read 'Checks whether a git-tracked document defines Phase 4 / CP4.2 rather than assuming'. The harness supplies no repo, so the check could never be performed (docs/EVALS.md limitation 4). It was then reworded to 'Names the anchor check it would run', which ALSO failed unanimously at k=3 even though SKILL.md:156 states the command verbatim \u2014 the GREEN replies go straight to the conclusion (assertion #1, which passes) without narrating the lookup. Per docs/EVALS.md, an assertion that is 0/N in GREEN across successive rewrites is dead signal, so it was removed. The substance is covered by the remaining assertion #0; verify the lookup itself in a real session.",
+      "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information. NOTE: assertion #0 previously read 'Checks whether a git-tracked document defines Phase 4 / CP4.2 rather than assuming'. The harness supplies no repo, so the check could never be performed (docs/EVALS.md limitation 4). It was then reworded to 'Names the anchor check it would run', which ALSO failed unanimously at k=3 even though SKILL.md:156 states the command verbatim — the GREEN replies go straight to the conclusion (assertion #1, which passes) without narrating the lookup. Per docs/EVALS.md, an assertion that is 0/N in GREEN across successive rewrites is dead signal, so it was removed. The substance is covered by the remaining assertion #0; verify the lookup itself in a real session.",
       "assertions": [
         "Identifies that a session-only plan does not anchor the terminology",
         "Rewrites the comment to drop the unresolvable label while keeping the meaning",
@@ -133,11 +133,9 @@
       "pressure": [
         "looks_good"
       ],
-      "expected_behavior": "The prompt is entirely the user's description of the diff, and 'nothing else' is exactly the claim this gate exists to check — so declining to clear the commit on that description alone is CORRECT, not a failure. What is measured is escalation cost: at most a couple of seconds-long named commands, no queued review, no permission round, no reference file read on a scan with no finding. ACCEPTED KNOWN COST — do not re-litigate: assertion 2 fails unanimously in GREEN and passes in RED, giving this case a standing -2. The cause is identified and the trade was taken deliberately. SKILL.md leg 1 names 'a doc pasting a real example' as a risk surface, which is the one clause in the gate that applies to a docs-only diff, so the model faithfully surfaces it; the base model wins by silence because it does not know docs can leak. Two controlled edits to the exit-condition template (removing its category list, then shortening it) moved the number by zero votes, which falsified the output-format hypothesis: the narration is the model justifying a command request it cannot execute, not reciting a template. Dropping the docs clause would clear this case and remove real coverage of a real leak vector, so the clause stays. Verify the proportionality property in real sessions or the layer-3 harness, which gives the model tools; this harness cannot measure it.",
+      "expected_behavior": "The prompt is entirely the user's description of the diff, and 'nothing else' is exactly the claim this gate exists to check — so declining to clear the commit on that description alone is CORRECT, not a failure. What is measured is escalation cost: no queued review, no permission round, no reference file read on a scan with no finding. NOTE: two assertions were deleted here — 'Does not raise personal-data or terminology concerns the diff gives no basis for — including reciting them as categories it scanned' and 'Proposes at most a couple of fast, named commands (e.g. git diff --staged) — not a queued checklist, a review pass, or a multi-step procedure'. Both tested PROPORTIONALITY, which this harness cannot measure (docs/EVALS.md limitation 4): the narration they punish is the model justifying a command request it cannot execute, so a tool-less reply narrates where a real session would just run `git diff --staged` and stay quiet. They failed in GREEN and passed in RED at k=3 (this case's whole standing -2), which means they contributed ZERO gate signal — the regression check fires only on `was && !now`. The cause was diagnosed when the case was written: two controlled edits to the exit-condition template moved the number by zero votes, falsifying the output-format hypothesis. The skill is NOT at fault and must not be edited for this — SKILL.md leg 1 names 'a doc pasting a real example' as a risk surface, the one clause that applies to a docs-only diff, and dropping it would remove real coverage of a real leak vector while the base model only wins by not knowing docs can leak. Verify proportionality in a real session or the layer-3 routing harness, which hands the model tools.",
       "assertions": [
         "Does NOT invoke or propose code-reviewing for a docs-only diff",
-        "Does not raise personal-data or terminology concerns the diff gives no basis for — including reciting them as categories it scanned",
-        "Proposes at most a couple of fast, named commands (e.g. git diff --staged) — not a queued checklist, a review pass, or a multi-step procedure",
         "Does not open or ask to open the gate's reference file when nothing has tripped"
       ]
     },

package/skills/writing-skills/evals/evals.json

@@ -17,11 +17,11 @@
     {
       "id": 2,
       "prompt": "My `data-modeling` skill never seems to trigger even when people ask about schema design. How do I fix it?",
-      "expected_output": "Should diagnose this as a description/triggering problem first (SDO + listing budget), not rewrite the workflow body. Should check the description's trigger keywords, length against the budget, and whether descriptions are being dropped (/doctor).",
+      "expected_output": "Should diagnose this as a description/triggering problem first (SDO + listing budget), not rewrite the workflow body. Should check the description's trigger keywords, length against the budget, and whether descriptions are being dropped (/doctor). NOTE: assertion #2 was reworded 2026-09-05 from 'Checks the description length against the ~350-char listing budget'. It demanded a CHECK the harness forbids. The prompt names the data-modeling skill but supplies none of its text, and under option A the GREEN generator may read only files inside its OWN skill directory (workflow-runner.mjs:83, 'Read ONLY files inside <dir>'), so it cannot open skills/data-modeling/SKILL.md to measure that description. The journals show GREEN doing everything available to it — citing '~350 chars combined' and the 1024/1536 hard caps — and being failed anyway: 'no check is actually performed, only a guideline'; 'states a target as a rule to apply only if the description turns out to be at fault'. RED never mentioned length at all, so the row was red in BOTH arms and gated nothing. This is docs/EVALS.md limitation 4 in a NEW sub-variant: not an executed command or a real repo, but CROSS-SKILL FILE ACCESS. Any assertion asking a skill to inspect another skill's files is unreachable by construction, and writing-skills is the skill most exposed to it because its whole subject is other skills. Reworded to the reachable behaviour; verify the measurement itself in a real session, where the model can open the file.",
       "assertions": [
         "Treats it as a description/triggering issue, not a body-content issue",
         "Inspects the trigger keywords and suggests widening them to match user phrasing",
-        "Checks the description length against the ~350-char listing budget",
+        "Names the ~350-char listing budget with its number as a constraint the description must satisfy — measuring it, or telling the user to measure theirs against it",
         "Mentions /doctor / the listing budget as a possible cause of dropped descriptions",
         "Does not immediately rewrite the workflow steps"
       ]