@harness-engineering/cli 1.25.4 → 1.25.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agents/skills/claude-code/enforce-architecture/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-api-design/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-architecture-advisor/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-auth/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-code-review/SKILL.md +30 -6
- package/dist/agents/skills/claude-code/harness-database/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-debugging/SKILL.md +38 -6
- package/dist/agents/skills/claude-code/harness-deployment/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-execution/SKILL.md +30 -6
- package/dist/agents/skills/claude-code/harness-planning/SKILL.md +20 -9
- package/dist/agents/skills/claude-code/harness-pre-commit-review/SKILL.md +59 -6
- package/dist/agents/skills/claude-code/harness-refactoring/SKILL.md +28 -7
- package/dist/agents/skills/claude-code/harness-security-scan/SKILL.md +12 -0
- package/dist/agents/skills/claude-code/harness-skill-authoring/SKILL.md +43 -8
- package/dist/agents/skills/claude-code/harness-soundness-review/SKILL.md +59 -7
- package/dist/agents/skills/claude-code/harness-tdd/SKILL.md +21 -0
- package/dist/agents/skills/claude-code/harness-verification/SKILL.md +31 -6
- package/dist/agents/skills/codex/enforce-architecture/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-api-design/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-architecture-advisor/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-auth/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-code-review/SKILL.md +30 -6
- package/dist/agents/skills/codex/harness-database/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-debugging/SKILL.md +38 -6
- package/dist/agents/skills/codex/harness-deployment/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-execution/SKILL.md +30 -6
- package/dist/agents/skills/codex/harness-planning/SKILL.md +20 -9
- package/dist/agents/skills/codex/harness-pre-commit-review/SKILL.md +59 -6
- package/dist/agents/skills/codex/harness-refactoring/SKILL.md +28 -7
- package/dist/agents/skills/codex/harness-security-scan/SKILL.md +12 -0
- package/dist/agents/skills/codex/harness-skill-authoring/SKILL.md +43 -8
- package/dist/agents/skills/codex/harness-soundness-review/SKILL.md +59 -7
- package/dist/agents/skills/codex/harness-tdd/SKILL.md +21 -0
- package/dist/agents/skills/codex/harness-verification/SKILL.md +31 -6
- package/dist/agents/skills/cursor/enforce-architecture/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-api-design/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-architecture-advisor/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-auth/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-code-review/SKILL.md +30 -6
- package/dist/agents/skills/cursor/harness-database/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-debugging/SKILL.md +38 -6
- package/dist/agents/skills/cursor/harness-deployment/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-execution/SKILL.md +30 -6
- package/dist/agents/skills/cursor/harness-planning/SKILL.md +20 -9
- package/dist/agents/skills/cursor/harness-pre-commit-review/SKILL.md +59 -6
- package/dist/agents/skills/cursor/harness-refactoring/SKILL.md +28 -7
- package/dist/agents/skills/cursor/harness-security-scan/SKILL.md +12 -0
- package/dist/agents/skills/cursor/harness-skill-authoring/SKILL.md +43 -8
- package/dist/agents/skills/cursor/harness-soundness-review/SKILL.md +59 -7
- package/dist/agents/skills/cursor/harness-tdd/SKILL.md +21 -0
- package/dist/agents/skills/cursor/harness-verification/SKILL.md +31 -6
- package/dist/agents/skills/gemini-cli/enforce-architecture/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-api-design/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-architecture-advisor/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-auth/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-code-review/SKILL.md +30 -6
- package/dist/agents/skills/gemini-cli/harness-database/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-debugging/SKILL.md +38 -6
- package/dist/agents/skills/gemini-cli/harness-deployment/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-execution/SKILL.md +30 -6
- package/dist/agents/skills/gemini-cli/harness-planning/SKILL.md +20 -9
- package/dist/agents/skills/gemini-cli/harness-pre-commit-review/SKILL.md +59 -6
- package/dist/agents/skills/gemini-cli/harness-refactoring/SKILL.md +28 -7
- package/dist/agents/skills/gemini-cli/harness-security-scan/SKILL.md +12 -0
- package/dist/agents/skills/gemini-cli/harness-skill-authoring/SKILL.md +43 -8
- package/dist/agents/skills/gemini-cli/harness-soundness-review/SKILL.md +59 -7
- package/dist/agents/skills/gemini-cli/harness-tdd/SKILL.md +21 -0
- package/dist/agents/skills/gemini-cli/harness-verification/SKILL.md +31 -6
- package/dist/{agents-md-MCUM4SIZ.js → agents-md-2PYJM2MK.js} +3 -3
- package/dist/{architecture-HNIO6AUX.js → architecture-VCLB7A23.js} +4 -4
- package/dist/{assess-project-6MV5TNY3.js → assess-project-64C6LIKN.js} +1 -1
- package/dist/bin/harness-mcp.js +16 -15
- package/dist/bin/harness.js +27 -115
- package/dist/business-knowledge-6RHYJOB3.js +7 -0
- package/dist/{check-phase-gate-VCBQHQAC.js → check-phase-gate-RT6PGEHY.js} +4 -4
- package/dist/{chunk-MI6MA6OP.js → chunk-42ZZLMYD.js} +169 -118
- package/dist/{chunk-BUYW3SA2.js → chunk-43VBX44J.js} +219 -107
- package/dist/{chunk-UV3BZMGT.js → chunk-4YEBV2FT.js} +2 -2
- package/dist/{chunk-5BQ5BOJL.js → chunk-7BAGSY5Q.js} +1 -1
- package/dist/{chunk-FSNPBT74.js → chunk-BUJOMC3O.js} +1 -1
- package/dist/{chunk-M55DGGF3.js → chunk-CBZECDCW.js} +5 -5
- package/dist/{chunk-XTITAVUR.js → chunk-EY6F2QXW.js} +1 -1
- package/dist/{chunk-6VZQJ5CX.js → chunk-FL6A72LV.js} +1 -1
- package/dist/{chunk-CXJTVICF.js → chunk-JQCS75DY.js} +4 -4
- package/dist/{chunk-WIQA4BSH.js → chunk-JUXFYB2K.js} +1 -1
- package/dist/{chunk-FES2YEQU.js → chunk-KJC4SE7C.js} +9 -9
- package/dist/{chunk-4NK7Z3BP.js → chunk-R2BI5UPK.js} +1 -1
- package/dist/{chunk-2KZAXESR.js → chunk-R7P2FMJT.js} +360 -183
- package/dist/chunk-RC5ZY3EV.js +82 -0
- package/dist/{chunk-K2SON7TI.js → chunk-RDQGCHKD.js} +252 -57
- package/dist/{chunk-EHRZMOQ2.js → chunk-RKZW3FDF.js} +1 -1
- package/dist/{chunk-BOQRTARD.js → chunk-SMY35HJM.js} +1 -1
- package/dist/{chunk-UQZBZINS.js → chunk-SZZ5UQL7.js} +6 -6
- package/dist/{chunk-AT74HEQM.js → chunk-U2OMWI7Z.js} +1 -1
- package/dist/{chunk-P7PANON5.js → chunk-VELT5VAG.js} +1 -1
- package/dist/{chunk-YTP2UDPV.js → chunk-WEOGCL7B.js} +3 -3
- package/dist/{chunk-TYV4EUAD.js → chunk-X2JJ3CPG.js} +8 -8
- package/dist/{chunk-GEEYCQDS.js → chunk-XLO4AXXM.js} +9 -9
- package/dist/{chunk-47N6R2F4.js → chunk-XMEEYMGE.js} +1 -1
- package/dist/{chunk-FIQL2HND.js → chunk-Y5JA4J2M.js} +2 -2
- package/dist/{chunk-F23H3U5U.js → chunk-ZAKUCM7O.js} +2 -2
- package/dist/{ci-workflow-RTM7VVTD.js → ci-workflow-SZL3KVUK.js} +3 -3
- package/dist/{dist-RADHOOXG.js → dist-GRW3X2ZQ.js} +3 -1
- package/dist/{dist-WCSJHQPK.js → dist-T3DGV5UN.js} +16 -2
- package/dist/{docs-UBOGGHTY.js → docs-WDLJORLK.js} +4 -4
- package/dist/{engine-MJJAP5CH.js → engine-2YWYRCKK.js} +3 -3
- package/dist/{entropy-EMSXF2PX.js → entropy-PJGTOORX.js} +3 -3
- package/dist/{feedback-ZLUX72HD.js → feedback-M5KCJKL2.js} +1 -1
- package/dist/{generate-agent-definitions-AWLPJ27C.js → generate-agent-definitions-MFDW6LZT.js} +4 -4
- package/dist/{graph-loader-JHQVQRUS.js → graph-loader-QMKXT454.js} +1 -1
- package/dist/hooks/telemetry-reporter.js +22 -1
- package/dist/index.d.ts +5 -5
- package/dist/index.js +26 -25
- package/dist/{loader-JVSJZSWZ.js → loader-6O52FYHE.js} +3 -3
- package/dist/{mcp-JZ7YB7TD.js → mcp-GGNFWKVC.js} +16 -15
- package/dist/{performance-7AGWJUY4.js → performance-337U5URQ.js} +4 -4
- package/dist/{review-pipeline-ZWVQJTJX.js → review-pipeline-DB5RD4SN.js} +3 -3
- package/dist/{runtime-D6YUQPP2.js → runtime-B74EN2WD.js} +3 -3
- package/dist/{scan-MPJ6JHUY.js → scan-DXQUHGTT.js} +1 -1
- package/dist/{security-JLZUAQYT.js → security-3AYN6FVU.js} +1 -1
- package/dist/{validate-TIIHRPMA.js → validate-35CD7VWN.js} +4 -4
- package/dist/{validate-cross-check-ZOWFA3DB.js → validate-cross-check-LJKXBQYH.js} +3 -3
- package/package.json +5 -5
|
@@ -342,17 +342,28 @@ Only apply when modifying existing documented behavior. When `docs/changes/` exi
|
|
|
342
342
|
- Human has reviewed and approved the plan
|
|
343
343
|
- Rigor level rules followed: fast skips skeleton; thorough always skeletons with approval; standard skeletons at >= 8 tasks
|
|
344
344
|
|
|
345
|
+
## Red Flags
|
|
346
|
+
|
|
347
|
+
| Flag | Corrective Action |
|
|
348
|
+
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
349
|
+
| "I know the implementation well enough to skip reading the spec" | STOP. Phase 1 SCOPE starts by reading the spec. Assumptions about spec content lead to plans that implement the wrong thing. |
|
|
350
|
+
| "This task is self-explanatory, no need for exact file paths and commands" | STOP. Iron Law: every task must contain exact file paths, exact commands, and complete code snippets. "Implement the service" is a wish, not a task. |
|
|
351
|
+
| "I'll plan the happy path now and add error handling tasks later" | STOP. Error handling is not optional. The spec's success criteria include error scenarios. Plan them alongside the happy path. |
|
|
352
|
+
| `// detailed steps TBD` or `// expand during execution` in task descriptions | STOP. A task that defers detail to execution is a vague task. If you cannot write the exact steps now, you do not understand the task well enough to plan it. |
|
|
353
|
+
|
|
345
354
|
## Rationalizations to Reject
|
|
346
355
|
|
|
347
|
-
| Rationalization | Reality
|
|
348
|
-
| ------------------------------------------------------------------------------------------------------------- |
|
|
349
|
-
| "The task is conceptually clear so I do not need to include exact code in the plan" | Every task must have exact file paths, exact code, and exact commands. If you cannot write the code in the plan, you do not understand the task well enough to plan it.
|
|
350
|
-
| "This task touches 5 files but it is logically one unit of work, so splitting it would add overhead" | Tasks touching more than 3 files must be split. The overhead of splitting is far less than the cost of a failed oversized task.
|
|
351
|
-
| "Tests for this task can be added in a follow-up task since the implementation is straightforward" | No skipping TDD in tasks. Every code-producing task must start with writing a test. "Add tests later" is explicitly forbidden.
|
|
352
|
-
| "The spec does not cover this edge case, but I can fill in the gap during planning" | When the spec is missing information, do not fill in the gaps yourself. Escalate. Filling gaps silently creates undocumented design decisions that no one reviewed.
|
|
353
|
-
| "I discovered we need an additional file during decomposition, but updating the file map is just bookkeeping" | The file map must be complete. Every file that will be created or modified must appear in the file map before task decomposition.
|
|
354
|
-
| "There are no real uncertainties — the spec is clear enough" | Every plan has unknowns. If you listed zero uncertainties, you skipped the step. Re-read the spec and list what is assumed but not stated.
|
|
355
|
-
| "I already know how to structure this, no need to finish scoping" | Premature decomposition anchors on the first approach found. Complete SCOPE (observable truths + uncertainties) before proposing any task structure.
|
|
356
|
+
| Rationalization | Reality |
|
|
357
|
+
| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
358
|
+
| "The task is conceptually clear so I do not need to include exact code in the plan" | Every task must have exact file paths, exact code, and exact commands. If you cannot write the code in the plan, you do not understand the task well enough to plan it. |
|
|
359
|
+
| "This task touches 5 files but it is logically one unit of work, so splitting it would add overhead" | Tasks touching more than 3 files must be split. The overhead of splitting is far less than the cost of a failed oversized task. |
|
|
360
|
+
| "Tests for this task can be added in a follow-up task since the implementation is straightforward" | No skipping TDD in tasks. Every code-producing task must start with writing a test. "Add tests later" is explicitly forbidden. |
|
|
361
|
+
| "The spec does not cover this edge case, but I can fill in the gap during planning" | When the spec is missing information, do not fill in the gaps yourself. Escalate. Filling gaps silently creates undocumented design decisions that no one reviewed. |
|
|
362
|
+
| "I discovered we need an additional file during decomposition, but updating the file map is just bookkeeping" | The file map must be complete. Every file that will be created or modified must appear in the file map before task decomposition. |
|
|
363
|
+
| "There are no real uncertainties — the spec is clear enough" | Every plan has unknowns. If you listed zero uncertainties, you skipped the step. Re-read the spec and list what is assumed but not stated. |
|
|
364
|
+
| "I already know how to structure this, no need to finish scoping" | Premature decomposition anchors on the first approach found. Complete SCOPE (observable truths + uncertainties) before proposing any task structure. |
|
|
365
|
+
| "The skeleton pass adds overhead for a plan this size — I will go straight to full tasks" | Rigor level rules are not optional. In thorough mode, the skeleton is always required. In standard mode, 8+ tasks require a skeleton. Skipping it risks task-level misalignment with the goal. |
|
|
366
|
+
| "I will write implementation code in the plan to make the tasks more concrete" | Planning produces a plan document, not code. Writing code during planning violates the phase boundary — code belongs in execution. Exact snippets in task descriptions are plan content, not executed code. |
|
|
356
367
|
|
|
357
368
|
## Examples
|
|
358
369
|
|
|
@@ -10,6 +10,16 @@
|
|
|
10
10
|
- NOT as a replacement for full peer review (use `harness-code-review` for that)
|
|
11
11
|
- NOT for commits that only update documentation or configuration (fast path skips AI review)
|
|
12
12
|
|
|
13
|
+
## Process
|
|
14
|
+
|
|
15
|
+
### Iron Law
|
|
16
|
+
|
|
17
|
+
**Mechanical checks gate AI review. No exceptions.**
|
|
18
|
+
|
|
19
|
+
If lint, typecheck, or tests fail, the pipeline stops. AI review does not run. Observations from AI review are advisory — they never block a commit. Only mechanical failures block. This ordering is non-negotiable.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
13
23
|
## Principle: Deterministic First
|
|
14
24
|
|
|
15
25
|
This skill follows the Deterministic-vs-LLM Responsibility Split principle. Mechanical checks run first and must pass before any AI review occurs. If a linter or type checker can catch the problem, the LLM should not be the one finding it.
|
|
@@ -262,6 +272,47 @@ fi
|
|
|
262
272
|
|
|
263
273
|
**Note:** AI review observations (WARN) do not block the commit — only mechanical check failures (FAIL) block. The author decides whether to address AI observations.
|
|
264
274
|
|
|
275
|
+
## Red Flags
|
|
276
|
+
|
|
277
|
+
| Flag | Corrective Action |
|
|
278
|
+
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
279
|
+
| "The lint errors are just warnings, I can proceed to AI review" | STOP. The gate is absolute. Any mechanical check failure means STOP. Warnings configured as errors are failures. |
|
|
280
|
+
| "I'll run the full test suite later in CI" | STOP. Pre-commit checks include tests. The purpose is to catch failures BEFORE they reach CI — not to defer them. |
|
|
281
|
+
| "This is just a config change, skip the security scan" | STOP. Phase 3 runs against all staged source files regardless of change type. Config files can contain hardcoded secrets. |
|
|
282
|
+
| `// quick fix, will clean up before PR` or `// TODO: handle error` in staged code | STOP. Pre-commit is the last line of defense before code enters the repository. Code committed with cleanup TODOs gets merged with cleanup TODOs. Fix it now. |
|
|
283
|
+
|
|
284
|
+
**Review-never-fixes:** Pre-commit review identifies observations. It never modifies staged code. AI review findings are reported for the author to decide — not applied automatically. A review that silently modifies code is not a review.
|
|
285
|
+
|
|
286
|
+
## Rubric Compression
|
|
287
|
+
|
|
288
|
+
Pre-commit review checklists MUST use compressed single-line format. Each check is one line with pipe-delimited fields:
|
|
289
|
+
|
|
290
|
+
```
|
|
291
|
+
phase|check-name|blocking|criterion
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
**Example:**
|
|
295
|
+
|
|
296
|
+
```
|
|
297
|
+
mechanical|lint|yes|Zero lint errors in staged files
|
|
298
|
+
mechanical|typecheck|yes|Zero type errors reported by tsc --noEmit
|
|
299
|
+
mechanical|tests|yes|All tests pass with exit code 0
|
|
300
|
+
mechanical|harness-health|yes|assess_project returns healthy: true
|
|
301
|
+
security|secrets|yes|No hardcoded secrets, API keys, or credentials in staged files
|
|
302
|
+
security|injection|yes|No eval(), exec(), or unparameterized SQL in staged files
|
|
303
|
+
security|advisory|no|CORS wildcards, HTTP URLs, disabled TLS — reported but non-blocking
|
|
304
|
+
ai|obvious-bugs|no|Null dereference, infinite loops, off-by-one, resource leaks
|
|
305
|
+
ai|debug-artifacts|no|No console.log, debugger statements, or TODO without issue ref
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
**Why:** Dense single-line rubrics minimize token consumption while preserving the same review signal. More budget for actual diff analysis.
|
|
309
|
+
|
|
310
|
+
**Rules:**
|
|
311
|
+
|
|
312
|
+
- Phase must be `mechanical`, `security`, or `ai`
|
|
313
|
+
- Blocking must be `yes` or `no`
|
|
314
|
+
- Maximum 80 characters per criterion text
|
|
315
|
+
|
|
265
316
|
## Gates
|
|
266
317
|
|
|
267
318
|
- **Mechanical checks must pass before AI review.** Do not run AI review if lint/typecheck/tests fail.
|
|
@@ -286,12 +337,14 @@ fi
|
|
|
286
337
|
|
|
287
338
|
## Rationalizations to Reject
|
|
288
339
|
|
|
289
|
-
| Rationalization | Why It Is Wrong
|
|
290
|
-
| ----------------------------------------------------------------------------- |
|
|
291
|
-
| "The lint errors are just warnings, so I can proceed to AI review" | The gate is absolute: any mechanical check failure means STOP. AI review does not run until lint, typecheck, and tests all pass.
|
|
292
|
-
| "This is a docs-only change but let me run AI review anyway for thoroughness" | The fast path is mandatory. If only docs/config files changed, AI review is skipped. Running it anyway wastes tokens.
|
|
293
|
-
| "The AI found a style issue, so I should block the commit" | AI review observations are advisory only. Only mechanical check failures block the commit.
|
|
294
|
-
| "I will skip the security scan since this is an internal endpoint" | Phase 3 runs the security scanner against all staged source files regardless of exposure. Hardcoded secrets and injection are blocking even in internal code.
|
|
340
|
+
| Rationalization | Why It Is Wrong |
|
|
341
|
+
| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
342
|
+
| "The lint errors are just warnings, so I can proceed to AI review" | The gate is absolute: any mechanical check failure means STOP. AI review does not run until lint, typecheck, and tests all pass. |
|
|
343
|
+
| "This is a docs-only change but let me run AI review anyway for thoroughness" | The fast path is mandatory. If only docs/config files changed, AI review is skipped. Running it anyway wastes tokens. |
|
|
344
|
+
| "The AI found a style issue, so I should block the commit" | AI review observations are advisory only. Only mechanical check failures block the commit. |
|
|
345
|
+
| "I will skip the security scan since this is an internal endpoint" | Phase 3 runs the security scanner against all staged source files regardless of exposure. Hardcoded secrets and injection are blocking even in internal code. |
|
|
346
|
+
| "The AI found an issue so I should fix it before reporting" | Pre-commit review reports findings — it does not apply fixes. The author decides what to act on. A review that silently modifies code is editing, not reviewing. |
|
|
347
|
+
| "The mechanical checks passed so the code is ready — skip the AI review" | If source files are staged (not docs/config only), AI review runs. Mechanical checks catch syntax and type errors; AI review catches semantic issues like null dereference and resource leaks. Both layers have value. |
|
|
295
348
|
|
|
296
349
|
## Examples
|
|
297
350
|
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
|
|
17
17
|
## Process
|
|
18
18
|
|
|
19
|
-
### Iron
|
|
19
|
+
### Iron Law
|
|
20
20
|
|
|
21
21
|
**All tests must pass BEFORE you start refactoring and AFTER every single change.**
|
|
22
22
|
|
|
@@ -41,6 +41,16 @@ When a knowledge graph exists at `.harness/graph/`, use graph queries for faster
|
|
|
41
41
|
|
|
42
42
|
Catches indirect consumers that grep misses. Fall back to file-based commands if no graph is available.
|
|
43
43
|
|
|
44
|
+
### Uncertainty Surfacing
|
|
45
|
+
|
|
46
|
+
When you encounter an unknown during refactoring, classify it immediately:
|
|
47
|
+
|
|
48
|
+
- **Blocking:** Cannot determine if the change is purely structural without resolving this (e.g., unclear whether callers rely on implementation detail). STOP and surface to human.
|
|
49
|
+
- **Assumption:** Can proceed if assumption is stated (e.g., "no external consumers of this internal API"). Document the assumption and continue. If wrong, revert.
|
|
50
|
+
- **Deferrable:** Does not affect the current refactoring step (e.g., whether a further refactoring would be beneficial). Note for future consideration.
|
|
51
|
+
|
|
52
|
+
Do not guess whether a change is behavioral or structural. If you are unsure, it is blocking.
|
|
53
|
+
|
|
44
54
|
### Phase 2: Execute — One Small Change at a Time
|
|
45
55
|
|
|
46
56
|
For EACH step in the plan:
|
|
@@ -134,14 +144,25 @@ Skipping this step means subsequent graph queries (impact analysis, dependency h
|
|
|
134
144
|
- No behavioral changes were introduced (the test suite is the proof)
|
|
135
145
|
- No dead code was left behind (run `harness cleanup` to verify)
|
|
136
146
|
|
|
147
|
+
## Red Flags
|
|
148
|
+
|
|
149
|
+
| Flag | Corrective Action |
|
|
150
|
+
| --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
151
|
+
| "This refactoring is safe, I don't need to run tests after this small change" | STOP. The Iron Law is absolute: tests after EVERY change. "Small" and "safe" are the changes that introduce subtle bugs. |
|
|
152
|
+
| "I'll combine these two renames into one commit since they're related" | STOP. One change per commit. Combined changes make it impossible to isolate which change caused a regression. |
|
|
153
|
+
| "The failing test is testing implementation details, so I'll fix the test" | STOP. Changing tests during refactoring is a warning sign. Verify the test is actually testing implementation details — not behavior your refactoring inadvertently changed. |
|
|
154
|
+
| `// removed old implementation` or `// TODO: move back later` replacing functional code | STOP. Either the code lives in its new location or it doesn't. Comments are not migration plans. Keep the code or delete it with a test proving the deletion is safe. |
|
|
155
|
+
|
|
137
156
|
## Rationalizations to Reject
|
|
138
157
|
|
|
139
|
-
| Rationalization | Reality
|
|
140
|
-
| ------------------------------------------------------------------------------------------------------------------- |
|
|
141
|
-
| "The tests are mostly passing, so I can start refactoring and fix the remaining failures as I go" | All tests must pass BEFORE refactoring starts. If tests are not green before you start, you are not refactoring -- you are debugging.
|
|
142
|
-
| "This refactoring changes a small amount of behavior, but it is a clear improvement" | Refactoring must not change behavior. The test suite is the proof. If the refactoring requires changing tests, you may be changing behavior.
|
|
143
|
-
| "I will make several changes at once and run tests at the end since each change is small" | Tests must run after EVERY single change. If a test breaks, you must undo the LAST change immediately.
|
|
144
|
-
| "The refactoring did not produce a measurable improvement, but the code is different so it must be somewhat better" | If the refactoring introduced no measurable improvement, revert the entire sequence. Refactoring for its own sake is churn.
|
|
158
|
+
| Rationalization | Reality |
|
|
159
|
+
| ------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
160
|
+
| "The tests are mostly passing, so I can start refactoring and fix the remaining failures as I go" | All tests must pass BEFORE refactoring starts. If tests are not green before you start, you are not refactoring -- you are debugging. |
|
|
161
|
+
| "This refactoring changes a small amount of behavior, but it is a clear improvement" | Refactoring must not change behavior. The test suite is the proof. If the refactoring requires changing tests, you may be changing behavior. |
|
|
162
|
+
| "I will make several changes at once and run tests at the end since each change is small" | Tests must run after EVERY single change. If a test breaks, you must undo the LAST change immediately. |
|
|
163
|
+
| "The refactoring did not produce a measurable improvement, but the code is different so it must be somewhat better" | If the refactoring introduced no measurable improvement, revert the entire sequence. Refactoring for its own sake is churn. |
|
|
164
|
+
| "I will refactor this and add the new feature in the same pass to be efficient" | Refactoring and feature work are separate tasks. Mixing them means test failures could be from the refactoring OR the new behavior — you cannot tell which. |
|
|
165
|
+
| "The test suite is slow, so I will run tests only at the end of the refactoring sequence" | Each step must be independently verified. A slow test suite is a separate problem to solve — it is not a reason to skip the safety net. |
|
|
145
166
|
|
|
146
167
|
## Examples
|
|
147
168
|
|
|
@@ -94,6 +94,18 @@ These apply to ALL skills. If you catch yourself doing any of these, STOP.
|
|
|
94
94
|
|
|
95
95
|
## Rationalizations to Reject
|
|
96
96
|
|
|
97
|
+
### Universal
|
|
98
|
+
|
|
99
|
+
These reasoning patterns sound plausible but lead to bad outcomes. Reject them.
|
|
100
|
+
|
|
101
|
+
- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
|
|
102
|
+
- **"This is best practice"** — Best practice in what context? Cite the source and
|
|
103
|
+
confirm it applies to this codebase.
|
|
104
|
+
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
|
|
105
|
+
with a concrete follow-up plan.
|
|
106
|
+
|
|
107
|
+
### Domain-Specific
|
|
108
|
+
|
|
97
109
|
| Rationalization | Reality |
|
|
98
110
|
| ----------------------------------- | -------------------------------------------------------------------------------------------------- |
|
|
99
111
|
| "No attacker would find this" | Security by obscurity. If the code is wrong, flag it regardless of discoverability. |
|
|
@@ -14,6 +14,14 @@
|
|
|
14
14
|
|
|
15
15
|
## Process
|
|
16
16
|
|
|
17
|
+
### Iron Law
|
|
18
|
+
|
|
19
|
+
**No skill ships without validation passing and test scenarios exercising every discipline section.**
|
|
20
|
+
|
|
21
|
+
A skill that passes happy-path execution but has untested discipline sections (Red Flags, Gates, Rationalizations) is a trap — agents activate it but have no guardrails when they encounter edge cases. Phase 5B is not optional.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
17
25
|
### Phase 1: DEFINE — Establish the Skill's Purpose
|
|
18
26
|
|
|
19
27
|
1. **Identify the recurring process.** What does the team do repeatedly? Name it. Describe it in one sentence. This becomes the skill's `description` in `skill.yaml` and the blockquote summary in `SKILL.md`.
|
|
@@ -344,16 +352,29 @@ Use this checklist as a final quality gate before declaring a skill complete.
|
|
|
344
352
|
- Rigid skills include Gates and Escalation sections with specific conditions and consequences
|
|
345
353
|
- The skill can be loaded and run with `harness skill run <name>`
|
|
346
354
|
|
|
355
|
+
## Red Flags
|
|
356
|
+
|
|
357
|
+
| Flag | Corrective Action |
|
|
358
|
+
| ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
359
|
+
| "This skill is simple enough to ship without test scenarios" | STOP. Phase 5B exists because untested discipline sections are decoration. If you cannot construct a scenario that triggers each discipline section, the section is too abstract — revise it. |
|
|
360
|
+
| "I'll add the Rationalizations/Gates/Escalation section after the skill is working" | STOP. Discipline sections are required sections. A skill that "works" without guardrails is a trap — agents activate but flounder at edge cases. Write the real content now. |
|
|
361
|
+
| "The skill works, I tested it by running it once" | STOP. A single happy-path run does not test discipline sections. Write scenarios that trigger Red Flags, Gates, and Rationalizations. |
|
|
362
|
+
| `// placeholder for future phases` or `// TODO: add gate conditions` in SKILL.md | STOP. Placeholder sections are stubs. A skill with stub discipline sections passes validation but fails in practice. Write the real content or remove the section entirely. |
|
|
363
|
+
|
|
364
|
+
**Review-never-fixes:** When reviewing or extending an existing skill, identify issues but do not fix them inline. Document findings, propose fixes, and let the skill author decide. Reviewing and editing are separate roles.
|
|
365
|
+
|
|
347
366
|
## Rationalizations to Reject
|
|
348
367
|
|
|
349
|
-
| Rationalization | Reality
|
|
350
|
-
| ----------------------------------------------------------------------- |
|
|
351
|
-
| "This skill is too simple to need all required sections" | Every section exists for a reason. A short section is fine; a missing section means the skill was not fully thought through.
|
|
352
|
-
| "The process section covers it — no need for explicit success criteria" | Process describes what to do. Success criteria describe how to know it worked. They serve different purposes.
|
|
353
|
-
| "Rationalizations to Reject is meta — this skill does not need it" | This section is required for all user-facing skills, including this one. No exceptions.
|
|
354
|
-
| "I will add examples later once the skill is proven" | Examples are a required section. A skill without examples forces the agent to guess at correct behavior. Write at least one example now.
|
|
355
|
-
| "The When to Use section is obvious from the name" | Negative conditions (when NOT to use) prevent misapplication. The skill name conveys nothing about boundary conditions.
|
|
356
|
-
| "The skill works — I tested it by running it once" | A single happy-path run does not test discipline sections. Write scenarios that trigger Red Flags, Gates, and Rationalizations. A skill that passes happy path but fails discipline scenarios is a trap.
|
|
368
|
+
| Rationalization | Reality |
|
|
369
|
+
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
370
|
+
| "This skill is too simple to need all required sections" | Every section exists for a reason. A short section is fine; a missing section means the skill was not fully thought through. |
|
|
371
|
+
| "The process section covers it — no need for explicit success criteria" | Process describes what to do. Success criteria describe how to know it worked. They serve different purposes. |
|
|
372
|
+
| "Rationalizations to Reject is meta — this skill does not need it" | This section is required for all user-facing skills, including this one. No exceptions. |
|
|
373
|
+
| "I will add examples later once the skill is proven" | Examples are a required section. A skill without examples forces the agent to guess at correct behavior. Write at least one example now. |
|
|
374
|
+
| "The When to Use section is obvious from the name" | Negative conditions (when NOT to use) prevent misapplication. The skill name conveys nothing about boundary conditions. |
|
|
375
|
+
| "The skill works — I tested it by running it once" | A single happy-path run does not test discipline sections. Write scenarios that trigger Red Flags, Gates, and Rationalizations. A skill that passes happy path but fails discipline scenarios is a trap. |
|
|
376
|
+
| "This is an internal skill, so discipline sections are unnecessary" | Agents rationalize skipping steps in internal skills too. Every user-facing skill requires the full discipline stack — no exemptions for internal or simple skills. |
|
|
377
|
+
| "I will copy the Rationalizations from a similar skill and adapt them" | Domain-specific means domain-specific. Copied rationalizations address the source skill's shortcuts, not this skill's. Write fresh entries based on what an agent executing THIS skill would try to skip. |
|
|
357
378
|
|
|
358
379
|
## Examples
|
|
359
380
|
|
|
@@ -471,3 +492,17 @@ Tools: Bash, Read, Glob.
|
|
|
471
492
|
"Rollback to version [X] failed. Current state: [description].
|
|
472
493
|
Manual intervention required."
|
|
473
494
|
```
|
|
495
|
+
|
|
496
|
+
## Gates
|
|
497
|
+
|
|
498
|
+
- **No shipping without validation.** `harness skill validate` must pass with zero errors before declaring a skill complete. Validation failures are not warnings — they are hard stops.
|
|
499
|
+
- **No shipping without test scenarios.** Phase 5B is mandatory. A skill without test scenarios that exercise its discipline sections (Red Flags, Gates, Rationalizations) is not complete, even if validation passes.
|
|
500
|
+
- **No placeholder sections.** Every required section must contain substantive content. A `## Rationalizations to Reject` section with generic entries or a `## Gates` section with no conditions is a stub, not a section.
|
|
501
|
+
- **No skipping negative conditions.** The `## When to Use` section must include both positive (when TO use) and negative (when NOT to use) conditions. Missing negatives cause misapplication.
|
|
502
|
+
|
|
503
|
+
## Escalation
|
|
504
|
+
|
|
505
|
+
- **When you cannot identify domain-specific rationalizations:** The skill's process section is too vague. If you cannot imagine an agent trying to shortcut the process, the process lacks enough prescriptive steps. Revise the process first, then rationalizations will become obvious.
|
|
506
|
+
- **When validation fails on a section you believe is present:** The section heading may not match the expected format exactly. Check for typos, extra whitespace, or wrong heading level.
|
|
507
|
+
- **When the skill spans more than 6 phases:** The skill is doing too much. Propose splitting into two skills with clear handoff points. A skill that tries to cover too many concerns becomes vague in each one.
|
|
508
|
+
- **When you cannot write a test scenario that triggers a Gate:** The Gate condition is too abstract. Revise it to include specific, observable conditions that an agent would encounter.
|
|
@@ -973,15 +973,67 @@ All checks work from document analysis and codebase reads alone. Graph adds prec
|
|
|
973
973
|
7. `harness validate` passes after all files are written
|
|
974
974
|
8. The skill test suite passes (structure, schema, platform-parity, references)
|
|
975
975
|
|
|
976
|
+
## Red Flags
|
|
977
|
+
|
|
978
|
+
| Flag | Corrective Action |
|
|
979
|
+
| ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
980
|
+
| "The spec looks internally consistent at a high level" | STOP. S1 requires checking each decision against Technical Design line by line. "High level" consistency misses contradictions in the details. |
|
|
981
|
+
| "This assumption is obvious and doesn't need to be stated" | STOP. S3 exists because unstated assumptions cause the most damage when wrong. If it's obvious, writing it down costs nothing. Skipping it costs debugging time later. |
|
|
982
|
+
| "The finding is minor so I'll auto-fix it without surfacing to the user" | STOP. Only inferrable fixes are auto-fixed. If the fix involves a design choice — even one you think is obvious — surface it. You are not the designer. |
|
|
983
|
+
| `// TODO: add traceability` or `// spec gap — fill later` in spec/plan files | STOP. TODOs in specs are unfinished review. The spec is not converged. Fix the gap or surface it as a finding — do not defer it. |
|
|
984
|
+
|
|
985
|
+
**Review-never-fixes:** Soundness review identifies structural issues in specs and plans. It applies inferrable fixes (formatting, missing links, obvious gaps) but NEVER makes design decisions. If a finding requires judgment, surface it to the user — even if the "right" answer seems obvious. A reviewer who makes design decisions has stopped reviewing and started designing without the authority to do so.
|
|
986
|
+
|
|
987
|
+
## Uncertainty Surfacing
|
|
988
|
+
|
|
989
|
+
When a check produces ambiguous results, classify the ambiguity immediately:
|
|
990
|
+
|
|
991
|
+
- **Blocking:** Cannot determine severity without user input (e.g., S1 finds a potential contradiction that might be intentional). Surface as a finding with `autoFixable: false`.
|
|
992
|
+
- **Assumption:** Can classify if assumption is stated (e.g., "the spec uses 'fast' to mean sub-second, not sub-minute"). Apply the assumption, log it, and continue. If wrong, the convergence loop will catch it.
|
|
993
|
+
- **Deferrable:** Ambiguity does not affect sign-off (e.g., unclear whether a non-goal is worth stating). Note as a suggestion-severity finding.
|
|
994
|
+
|
|
995
|
+
Do not auto-fix ambiguous findings. Ambiguity means you lack context — applying a "fix" without context is guessing.
|
|
996
|
+
|
|
997
|
+
## Rubric Compression
|
|
998
|
+
|
|
999
|
+
Soundness check rubrics used internally MUST use compressed single-line format. Each check is one line with pipe-delimited fields:
|
|
1000
|
+
|
|
1001
|
+
```
|
|
1002
|
+
mode|check-id|severity|criterion
|
|
1003
|
+
```
|
|
1004
|
+
|
|
1005
|
+
**Example (Spec Mode rubric):**
|
|
1006
|
+
|
|
1007
|
+
```
|
|
1008
|
+
spec|S1|error|No contradictions between decisions, technical design, and success criteria
|
|
1009
|
+
spec|S2|warning|Every goal has at least one success criterion; no orphan criteria
|
|
1010
|
+
spec|S3|warning|All implicit assumptions documented in Assumptions section
|
|
1011
|
+
spec|S4|warning|Error/edge cases covered; EARS unwanted-behavior gaps filled
|
|
1012
|
+
spec|S5|error|No references to nonexistent codebase capabilities or incompatible patterns
|
|
1013
|
+
spec|S6|error|No speculative features without requirement traceability
|
|
1014
|
+
spec|S7|warning|All success criteria are observable and measurable with concrete thresholds
|
|
1015
|
+
```
|
|
1016
|
+
|
|
1017
|
+
**Why:** Verbose check descriptions inflate review context without improving check accuracy. Dense single-line rubrics give the same signal in fewer tokens, leaving more budget for actual document analysis.
|
|
1018
|
+
|
|
1019
|
+
**Rules:**
|
|
1020
|
+
|
|
1021
|
+
- Mode prefix must be `spec` or `plan`
|
|
1022
|
+
- Check ID must match the defined check IDs (S1-S7, P1-P7)
|
|
1023
|
+
- Severity must be `error` or `warning`
|
|
1024
|
+
- Maximum 80 characters per criterion text
|
|
1025
|
+
|
|
976
1026
|
## Rationalizations to Reject
|
|
977
1027
|
|
|
978
|
-
| Rationalization | Reality
|
|
979
|
-
| -------------------------------------------------------------------------------------------------------- |
|
|
980
|
-
| "The spec looks coherent to me, so I can skip running the S1 internal coherence check" | Every check in the mode must run. S1 detects contradictions that human review frequently misses.
|
|
981
|
-
| "This unstated assumption is obvious, so documenting it would be pedantic" | S3 exists because "obvious" assumptions cause the most damage when wrong. Cheapest to document, most expensive to miss.
|
|
982
|
-
| "The success criterion is somewhat vague but the team will know what it means" | S7 flags vague criteria like "should be fast" because they are untestable. Vague criteria survive brainstorming only to fail at verification.
|
|
983
|
-
| "This auto-fixable finding is minor, so I will just note it rather than applying the fix" | Auto-fixable findings should be applied silently — that is the design intent. Skipping them ships known inferrable gaps.
|
|
984
|
-
| "The feasibility check found a signature mismatch but the code can probably be adapted during execution" | S5 red flags are always severity "error" and always surfaced. A spec referencing nonexistent modules produces a broken plan.
|
|
1028
|
+
| Rationalization | Reality |
|
|
1029
|
+
| -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
1030
|
+
| "The spec looks coherent to me, so I can skip running the S1 internal coherence check" | Every check in the mode must run. S1 detects contradictions that human review frequently misses. |
|
|
1031
|
+
| "This unstated assumption is obvious, so documenting it would be pedantic" | S3 exists because "obvious" assumptions cause the most damage when wrong. Cheapest to document, most expensive to miss. |
|
|
1032
|
+
| "The success criterion is somewhat vague but the team will know what it means" | S7 flags vague criteria like "should be fast" because they are untestable. Vague criteria survive brainstorming only to fail at verification. |
|
|
1033
|
+
| "This auto-fixable finding is minor, so I will just note it rather than applying the fix" | Auto-fixable findings should be applied silently — that is the design intent. Skipping them ships known inferrable gaps. |
|
|
1034
|
+
| "The feasibility check found a signature mismatch but the code can probably be adapted during execution" | S5 red flags are always severity "error" and always surfaced. A spec referencing nonexistent modules produces a broken plan. |
|
|
1035
|
+
| "The convergence loop is taking too long, so I will skip the re-check and declare converged" | Convergence requires the issue count to stop decreasing. Declaring convergence without a re-check is falsifying the exit condition. |
|
|
1036
|
+
| "This spec is well-written enough that a soundness review would not find anything" | Every spec gets a soundness review. Well-written specs still have unstated assumptions (S3) and vague criteria (S7). The review is not optional. |
|
|
985
1037
|
|
|
986
1038
|
## Examples
|
|
987
1039
|
|
|
@@ -85,6 +85,16 @@ harness scan [path]
|
|
|
85
85
|
|
|
86
86
|
Skipping this step means subsequent graph queries (impact analysis, dependency health, test advisor) may return stale results.
|
|
87
87
|
|
|
88
|
+
### Uncertainty Surfacing
|
|
89
|
+
|
|
90
|
+
When you encounter an unknown during a RED-GREEN-REFACTOR cycle, classify it immediately:
|
|
91
|
+
|
|
92
|
+
- **Blocking:** Cannot write a meaningful test without resolving this (e.g., unclear expected behavior). STOP and surface to human with options.
|
|
93
|
+
- **Assumption:** Can write the test if assumption is stated explicitly (e.g., "input is always non-null"). Document the assumption in a test comment and continue. If the assumption proves wrong, the test must be revised.
|
|
94
|
+
- **Deferrable:** Does not affect the current cycle (e.g., performance characteristics). Record for a future cycle.
|
|
95
|
+
|
|
96
|
+
Do not bury unknowns in test code. An unstated assumption in a test is a test that passes for the wrong reason.
|
|
97
|
+
|
|
88
98
|
### Cycle Rhythm
|
|
89
99
|
|
|
90
100
|
Repeat the 4 phases for each new behavior. A typical feature requires 3-10 cycles. Each cycle should take 2-15 minutes. If a cycle takes longer than 15 minutes, the step is too large — break it down.
|
|
@@ -122,6 +132,8 @@ Repeat the 4 phases for each new behavior. A typical feature requires 3-10 cycle
|
|
|
122
132
|
| "The test passed on the first run, so TDD is working" | If the test passed without implementing the production code, either the behavior already exists or the test is wrong. You must watch the test FAIL for the right reason before proceeding to GREEN. |
|
|
123
133
|
| "I will test multiple behaviors in this one test to be efficient" | One test, one assertion, one behavior. Multi-behavior tests make it impossible to pinpoint which behavior broke when the test fails. |
|
|
124
134
|
| "Harness validate can wait until the end of the feature since it slows down the cycle" | No skipping VALIDATE. Every cycle must end with harness check-deps and harness validate. A passing test with a failing validation means the implementation violated a project constraint. |
|
|
135
|
+
| "This edge case is unlikely, so I will skip writing a test for it" | If the edge case can happen, it needs a test. Unlikely is not impossible. The test is cheap; the production bug is expensive. |
|
|
136
|
+
| "The existing tests cover this behavior implicitly, so no new test is needed" | Implicit coverage is not TDD. If you cannot point to a specific test that asserts the specific behavior, write one. Implicit coverage breaks silently when the implying test changes. |
|
|
125
137
|
|
|
126
138
|
## Examples
|
|
127
139
|
|
|
@@ -166,6 +178,15 @@ git commit -m "feat(cart): calculate total from item price and quantity"
|
|
|
166
178
|
|
|
167
179
|
**Next cycle (RED):** Write a test for empty array input. Watch it fail (or pass — if it passes, the behavior is already handled). Continue.
|
|
168
180
|
|
|
181
|
+
## Red Flags
|
|
182
|
+
|
|
183
|
+
| Flag | Corrective Action |
|
|
184
|
+
| ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
185
|
+
| "I'll write the test after since I know what the code should do" | STOP. Test-after is not TDD. Delete the production code, write the test, watch it fail. |
|
|
186
|
+
| "The test is trivial/obvious so I don't need to watch it fail" | STOP. Observing failure proves the test catches the defect. A test you haven't seen fail might pass for the wrong reason. |
|
|
187
|
+
| "I'll batch these small tests together to save time" | STOP. Each RED-GREEN-REFACTOR cycle is atomic. Batching obscures which behavior broke when a test fails. |
|
|
188
|
+
| `// removed old validation` or `// TODO: re-add error handling` replacing functional code | STOP. Code-to-comment replacement is deletion with a fig leaf. Either keep the code or delete it cleanly with a test proving it is unnecessary. |
|
|
189
|
+
|
|
169
190
|
## Gates
|
|
170
191
|
|
|
171
192
|
These are hard stops. Violating any gate means the process has broken down.
|
|
@@ -43,6 +43,20 @@ When no arguments are provided (standalone invocation), session slug is unknown
|
|
|
43
43
|
|
|
44
44
|
---
|
|
45
45
|
|
|
46
|
+
### Uncertainty Surfacing
|
|
47
|
+
|
|
48
|
+
When you encounter an unknown during verification, classify it immediately:
|
|
49
|
+
|
|
50
|
+
- **Blocking:** Cannot determine pass/fail without resolving this (e.g., spec does not define expected behavior for a scenario, cannot run tests due to missing dependency). STOP and escalate.
|
|
51
|
+
- **Assumption:** Can verify if assumption is stated (e.g., "this module is internal-only so WIRED check against external consumers is not applicable"). Document the assumption in the report. If wrong, verification must be re-run.
|
|
52
|
+
- **Deferrable:** Does not affect current verification (e.g., whether additional test coverage would be beneficial). Note in report as an observation.
|
|
53
|
+
|
|
54
|
+
Do not mark PASS with unstated assumptions. An assumption-laden PASS is a false positive.
|
|
55
|
+
|
|
56
|
+
**Review-never-fixes:** Verification identifies gaps. Verification never fills them. If you find a stub, missing test, or unwired artifact, record it as a FAIL with evidence. Do not implement the fix — that is the executor's job. A verifier who fixes is no longer verifying independently.
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
46
60
|
### Context Loading
|
|
47
61
|
|
|
48
62
|
Before running verification levels, load session context:
|
|
@@ -284,6 +298,15 @@ L2|real-logic|Functions contain meaningful logic, not just hardcoded returns
|
|
|
284
298
|
- Maximum 80 characters per criterion text
|
|
285
299
|
- Rubric entries are guidance — the verification levels define the authoritative checks
|
|
286
300
|
|
|
301
|
+
## Red Flags
|
|
302
|
+
|
|
303
|
+
| Flag | Corrective Action |
|
|
304
|
+
| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
305
|
+
| "Tests passed earlier, so I just need to check the files exist" | STOP. Iron Law: fresh evidence in THIS session. "Earlier" is cached — run the checks now. |
|
|
306
|
+
| "The implementation looks substantive at a glance" | STOP. Level 2 requires thorough reading, not glancing. Stubs designed to look real (e.g., functions with only a log statement) are the whole reason L2 exists. |
|
|
307
|
+
| "The artifact is exported so it must be wired" | STOP. Export without import is dead code. Trace the actual usage chain: import, call, test, pass. "Must be" is not evidence. |
|
|
308
|
+
| `// stubbed for now` or `// implementation pending` in production code | STOP. These are Level 2 failures. Do not proceed to Level 3. Do not fix them yourself — record as FAIL and report. |
|
|
309
|
+
|
|
287
310
|
## Non-Determinism Tolerance
|
|
288
311
|
|
|
289
312
|
Mechanical checks (tests, lint, types) are binary pass/fail. No tolerance.
|
|
@@ -292,12 +315,14 @@ For behavioral verification (convention adherence, style guides), accept thresho
|
|
|
292
315
|
|
|
293
316
|
## Rationalizations to Reject
|
|
294
317
|
|
|
295
|
-
| Rationalization
|
|
296
|
-
|
|
|
297
|
-
| "Tests passed earlier, no need to re-run"
|
|
298
|
-
| "File exists and has code, skip thorough read for Level 2"
|
|
299
|
-
| "Artifact is imported by a test file, so passes Level 3"
|
|
300
|
-
| "Verification report probably looks fine from memory"
|
|
318
|
+
| Rationalization | Reality |
|
|
319
|
+
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
320
|
+
| "Tests passed earlier, no need to re-run" | Iron Law forbids cached results. All evidence must be fresh in THIS session. |
|
|
321
|
+
| "File exists and has code, skip thorough read for Level 2" | Level 2 requires thorough reading. Scanning for TODO, throw Error, empty functions catches stubs that look real. |
|
|
322
|
+
| "Artifact is imported by a test file, so passes Level 3" | Import is necessary but not sufficient. Test must assert on behavior and not be skipped. |
|
|
323
|
+
| "Verification report probably looks fine from memory" | "Should", "probably", "seems to", "I believe" are forbidden. Replace with "verified: [evidence]" or "not verified: [missing]." |
|
|
324
|
+
| "I found a stub so I'll quickly implement it to make verification pass" | Verification identifies gaps — verification never fills them. Record the stub as a FAIL. The executor fixes it. A verifier who implements is no longer independent. |
|
|
325
|
+
| "The spec only mentions 3 behaviors but I'll verify 5 to be thorough" | Verify what the spec requires, not what you think it should require. Extra verification against unstated requirements conflates verification with spec review. |
|
|
301
326
|
|
|
302
327
|
## Examples
|
|
303
328
|
|
|
@@ -272,6 +272,18 @@ These apply to ALL skills. If you catch yourself doing any of these, STOP.
|
|
|
272
272
|
|
|
273
273
|
## Rationalizations to Reject
|
|
274
274
|
|
|
275
|
+
### Universal
|
|
276
|
+
|
|
277
|
+
These reasoning patterns sound plausible but lead to bad outcomes. Reject them.
|
|
278
|
+
|
|
279
|
+
- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
|
|
280
|
+
- **"This is best practice"** — Best practice in what context? Cite the source and
|
|
281
|
+
confirm it applies to this codebase.
|
|
282
|
+
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
|
|
283
|
+
with a concrete follow-up plan.
|
|
284
|
+
|
|
285
|
+
### Domain-Specific
|
|
286
|
+
|
|
275
287
|
| Rationalization | Reality |
|
|
276
288
|
| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------ |
|
|
277
289
|
| "The violation is minor — just one import" | One violation sets a precedent. Enforce the constraint or document an explicit exception with rationale. |
|
|
@@ -332,6 +332,18 @@ These apply to ALL skills. If you catch yourself doing any of these, STOP.
|
|
|
332
332
|
|
|
333
333
|
## Rationalizations to Reject
|
|
334
334
|
|
|
335
|
+
### Universal
|
|
336
|
+
|
|
337
|
+
These reasoning patterns sound plausible but lead to bad outcomes. Reject them.
|
|
338
|
+
|
|
339
|
+
- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
|
|
340
|
+
- **"This is best practice"** — Best practice in what context? Cite the source and
|
|
341
|
+
confirm it applies to this codebase.
|
|
342
|
+
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
|
|
343
|
+
with a concrete follow-up plan.
|
|
344
|
+
|
|
345
|
+
### Domain-Specific
|
|
346
|
+
|
|
335
347
|
| Rationalization | Reality |
|
|
336
348
|
| ------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
337
349
|
| "It's an internal API, breaking changes are fine" | Internal consumers break too. Version the change or coordinate the migration explicitly. |
|
|
@@ -312,6 +312,18 @@ These apply to ALL skills. If you catch yourself doing any of these, STOP.
|
|
|
312
312
|
|
|
313
313
|
## Rationalizations to Reject
|
|
314
314
|
|
|
315
|
+
### Universal
|
|
316
|
+
|
|
317
|
+
These reasoning patterns sound plausible but lead to bad outcomes. Reject them.
|
|
318
|
+
|
|
319
|
+
- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
|
|
320
|
+
- **"This is best practice"** — Best practice in what context? Cite the source and
|
|
321
|
+
confirm it applies to this codebase.
|
|
322
|
+
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
|
|
323
|
+
with a concrete follow-up plan.
|
|
324
|
+
|
|
325
|
+
### Domain-Specific
|
|
326
|
+
|
|
315
327
|
| Rationalization | Reality |
|
|
316
328
|
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
317
329
|
| "This will be easier to maintain" | Easier for whom, and compared to what? Cite the maintenance burden with evidence from the codebase. |
|
|
@@ -307,6 +307,18 @@ These apply to ALL skills. If you catch yourself doing any of these, STOP.
|
|
|
307
307
|
|
|
308
308
|
## Rationalizations to Reject
|
|
309
309
|
|
|
310
|
+
### Universal
|
|
311
|
+
|
|
312
|
+
These reasoning patterns sound plausible but lead to bad outcomes. Reject them.
|
|
313
|
+
|
|
314
|
+
- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
|
|
315
|
+
- **"This is best practice"** — Best practice in what context? Cite the source and
|
|
316
|
+
confirm it applies to this codebase.
|
|
317
|
+
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
|
|
318
|
+
with a concrete follow-up plan.
|
|
319
|
+
|
|
320
|
+
### Domain-Specific
|
|
321
|
+
|
|
310
322
|
| Rationalization | Reality |
|
|
311
323
|
| -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
|
312
324
|
| "No one would guess this token format" | Security by obscurity. Tokens must be cryptographically secure regardless of format predictability. |
|