@harness-engineering/cli 3.1.0 → 4.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agents/personas/harness-pm.yaml +33 -0
- package/dist/agents/skills/claude-code/acceptance-eval/SKILL.md +92 -0
- package/dist/agents/skills/claude-code/acceptance-eval/skill.yaml +54 -0
- package/dist/agents/skills/claude-code/harness-brainstorming/SKILL.md +46 -38
- package/dist/agents/skills/claude-code/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-code-review/SKILL.md +10 -11
- package/dist/agents/skills/claude-code/harness-execution/SKILL.md +24 -26
- package/dist/agents/skills/claude-code/harness-integration/SKILL.md +21 -19
- package/dist/agents/skills/claude-code/harness-maintenance-pipeline/SKILL.md +173 -0
- package/dist/agents/skills/claude-code/harness-maintenance-pipeline/skill.yaml +43 -0
- package/dist/agents/skills/claude-code/harness-onboarding/SKILL.md +2 -0
- package/dist/agents/skills/claude-code/harness-planning/SKILL.md +17 -40
- package/dist/agents/skills/claude-code/harness-pulse/SKILL.md +1 -1
- package/dist/agents/skills/claude-code/harness-roadmap/SKILL.md +17 -0
- package/dist/agents/skills/claude-code/harness-roadmap-pilot/SKILL.md +8 -0
- package/dist/agents/skills/claude-code/harness-verification/SKILL.md +9 -12
- package/dist/agents/skills/claude-code/initialize-harness-project/SKILL.md +112 -126
- package/dist/agents/skills/claude-code/outcome-eval/SKILL.md +1 -0
- package/dist/agents/skills/claude-code/product-advisor/SKILL.md +202 -0
- package/dist/agents/skills/claude-code/product-advisor/skill.yaml +74 -0
- package/dist/agents/skills/codex/acceptance-eval/SKILL.md +92 -0
- package/dist/agents/skills/codex/acceptance-eval/skill.yaml +54 -0
- package/dist/agents/skills/codex/harness-brainstorming/SKILL.md +46 -38
- package/dist/agents/skills/codex/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/codex/harness-code-review/SKILL.md +10 -11
- package/dist/agents/skills/codex/harness-execution/SKILL.md +24 -26
- package/dist/agents/skills/codex/harness-integration/SKILL.md +21 -19
- package/dist/agents/skills/codex/harness-maintenance-pipeline/SKILL.md +173 -0
- package/dist/agents/skills/codex/harness-maintenance-pipeline/skill.yaml +43 -0
- package/dist/agents/skills/codex/harness-onboarding/SKILL.md +2 -0
- package/dist/agents/skills/codex/harness-planning/SKILL.md +17 -40
- package/dist/agents/skills/codex/harness-pulse/SKILL.md +1 -1
- package/dist/agents/skills/codex/harness-roadmap/SKILL.md +17 -0
- package/dist/agents/skills/codex/harness-roadmap-pilot/SKILL.md +8 -0
- package/dist/agents/skills/codex/harness-verification/SKILL.md +9 -12
- package/dist/agents/skills/codex/initialize-harness-project/SKILL.md +112 -126
- package/dist/agents/skills/codex/outcome-eval/SKILL.md +1 -0
- package/dist/agents/skills/codex/product-advisor/SKILL.md +202 -0
- package/dist/agents/skills/codex/product-advisor/skill.yaml +74 -0
- package/dist/agents/skills/cursor/acceptance-eval/SKILL.md +92 -0
- package/dist/agents/skills/cursor/acceptance-eval/skill.yaml +54 -0
- package/dist/agents/skills/cursor/harness-brainstorming/SKILL.md +46 -38
- package/dist/agents/skills/cursor/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/cursor/harness-code-review/SKILL.md +10 -11
- package/dist/agents/skills/cursor/harness-execution/SKILL.md +24 -26
- package/dist/agents/skills/cursor/harness-integration/SKILL.md +21 -19
- package/dist/agents/skills/cursor/harness-maintenance-pipeline/SKILL.md +173 -0
- package/dist/agents/skills/cursor/harness-maintenance-pipeline/skill.yaml +43 -0
- package/dist/agents/skills/cursor/harness-onboarding/SKILL.md +2 -0
- package/dist/agents/skills/cursor/harness-planning/SKILL.md +17 -40
- package/dist/agents/skills/cursor/harness-pulse/SKILL.md +1 -1
- package/dist/agents/skills/cursor/harness-roadmap/SKILL.md +17 -0
- package/dist/agents/skills/cursor/harness-roadmap-pilot/SKILL.md +8 -0
- package/dist/agents/skills/cursor/harness-verification/SKILL.md +9 -12
- package/dist/agents/skills/cursor/initialize-harness-project/SKILL.md +112 -126
- package/dist/agents/skills/cursor/outcome-eval/SKILL.md +1 -0
- package/dist/agents/skills/cursor/product-advisor/SKILL.md +202 -0
- package/dist/agents/skills/cursor/product-advisor/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/acceptance-eval/SKILL.md +92 -0
- package/dist/agents/skills/gemini-cli/acceptance-eval/skill.yaml +54 -0
- package/dist/agents/skills/gemini-cli/harness-brainstorming/SKILL.md +46 -38
- package/dist/agents/skills/gemini-cli/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-code-review/SKILL.md +10 -11
- package/dist/agents/skills/gemini-cli/harness-execution/SKILL.md +24 -26
- package/dist/agents/skills/gemini-cli/harness-integration/SKILL.md +21 -19
- package/dist/agents/skills/gemini-cli/harness-maintenance-pipeline/SKILL.md +173 -0
- package/dist/agents/skills/gemini-cli/harness-maintenance-pipeline/skill.yaml +43 -0
- package/dist/agents/skills/gemini-cli/harness-onboarding/SKILL.md +2 -0
- package/dist/agents/skills/gemini-cli/harness-planning/SKILL.md +17 -40
- package/dist/agents/skills/gemini-cli/harness-pulse/SKILL.md +1 -1
- package/dist/agents/skills/gemini-cli/harness-roadmap/SKILL.md +17 -0
- package/dist/agents/skills/gemini-cli/harness-roadmap-pilot/SKILL.md +8 -0
- package/dist/agents/skills/gemini-cli/harness-verification/SKILL.md +9 -12
- package/dist/agents/skills/gemini-cli/initialize-harness-project/SKILL.md +112 -126
- package/dist/agents/skills/gemini-cli/outcome-eval/SKILL.md +1 -0
- package/dist/agents/skills/gemini-cli/product-advisor/SKILL.md +202 -0
- package/dist/agents/skills/gemini-cli/product-advisor/skill.yaml +74 -0
- package/dist/agents/skills/tests/interaction-channel.test.ts +64 -0
- package/dist/{agents-md-YYCLGMBF.js → agents-md-VJLYEH2S.js} +4 -4
- package/dist/{analyzer-V633GUHY-MBCQWKDU.js → analyzer-V633GUHY-OV335HGM.js} +2 -2
- package/dist/{architecture-5KPP4ID4.js → architecture-YQH2NYXY.js} +5 -5
- package/dist/{assess-project-B6ENHUAU.js → assess-project-KOTAPAHD.js} +1 -1
- package/dist/bin/harness-mcp.js +19 -18
- package/dist/bin/harness.js +31 -29
- package/dist/{check-phase-gate-REG3HWYC.js → check-phase-gate-CHU2E3OU.js} +6 -5
- package/dist/{chunk-UBTMPO5H.js → chunk-36GQREOE.js} +3 -3
- package/dist/{chunk-CLI4K2CH.js → chunk-4U5LDFCY.js} +1 -1
- package/dist/{chunk-IF3JK3G5.js → chunk-4VXFZSYF.js} +1 -1
- package/dist/{chunk-TE5YHHHZ.js → chunk-5RSXUAMN.js} +1 -1
- package/dist/{chunk-YF3AY6KS.js → chunk-6SCYTIF4.js} +6 -6
- package/dist/{chunk-N55WOGN3.js → chunk-6YFHUIGT.js} +1 -1
- package/dist/{chunk-RGQXHCCH.js → chunk-7PJCK7UE.js} +8 -8
- package/dist/{chunk-LMZ2JTNP.js → chunk-7TIJXWG3.js} +5 -9
- package/dist/chunk-AAUY53CC.js +61 -0
- package/dist/{chunk-XEOSSKIB.js → chunk-AX5KGRM6.js} +48 -4
- package/dist/{chunk-FUH4J43G.js → chunk-E5EW5HVJ.js} +2 -2
- package/dist/{chunk-C2ZGWMTN.js → chunk-EYMOPFSD.js} +2 -2
- package/dist/{chunk-FL5SLUKZ.js → chunk-GHBFNLV2.js} +2374 -1095
- package/dist/{chunk-HIPBU2Q4.js → chunk-I3CGCZJF.js} +9 -9
- package/dist/{chunk-OQFEFVIN.js → chunk-JPVOI7GP.js} +7 -7
- package/dist/{chunk-DNX35Q5K.js → chunk-M7F2PPN3.js} +3 -3
- package/dist/{chunk-A65QO5KN.js → chunk-MMLQPHZ5.js} +3032 -1166
- package/dist/{chunk-ERZCC5VH.js → chunk-OQ6KOQEV.js} +9 -9
- package/dist/{chunk-ZOUJP3EY.js → chunk-RG3TNIUB.js} +1659 -808
- package/dist/{chunk-PZX2Y3RV.js → chunk-SDMV4QHM.js} +5 -5
- package/dist/{chunk-XZ2Q4UTW.js → chunk-SEA2PFVI.js} +1 -1
- package/dist/{chunk-TCVQVHEF.js → chunk-T746REAE.js} +1 -1
- package/dist/{chunk-6B6UN6SG.js → chunk-UTJJJDNN.js} +11 -1
- package/dist/{chunk-BXKNWSG4.js → chunk-UWVMJEUJ.js} +1 -1
- package/dist/{chunk-WWACIIBA.js → chunk-V3ZQUCRH.js} +1 -1
- package/dist/{chunk-27AJKSQY.js → chunk-WBVCN35T.js} +1 -1
- package/dist/{chunk-PP6ZRL5T.js → chunk-WZJR6SOI.js} +536 -422
- package/dist/{chunk-YQAFKOOL.js → chunk-XAVMTAUT.js} +2 -2
- package/dist/chunk-Y6SUQC32.js +9 -0
- package/dist/{chunk-ZRKFDE6M.js → chunk-YANR2RL7.js} +1 -1
- package/dist/{chunk-2N4OENGE.js → chunk-YYFSFSB3.js} +1 -1
- package/dist/{ci-workflow-JEYYCBOC.js → ci-workflow-HO4WKGNG.js} +4 -4
- package/dist/{dist-3HW4LCHE.js → dist-6D2F2J57.js} +73 -17
- package/dist/{dist-OWHMNL4W.js → dist-DWBTZXR5.js} +1 -1
- package/dist/{docs-YRMW7UED.js → docs-FP47JONV.js} +5 -5
- package/dist/{engine-E7H5U3AC.js → engine-YUA6IYOG.js} +4 -4
- package/dist/{entropy-3JBKCGJJ.js → entropy-DBW2INZU.js} +4 -4
- package/dist/{feedback-U6VJ6AUJ.js → feedback-BPHADTRQ.js} +1 -1
- package/dist/{generate-agent-definitions-S2FRZF4Y.js → generate-agent-definitions-KPVGBGDY.js} +6 -6
- package/dist/{glob-helper-GLZAURJC.js → glob-helper-VR6UCZZB.js} +1 -1
- package/dist/{graph-loader-HHXN5FLP.js → graph-loader-STFGYKZH.js} +1 -1
- package/dist/hooks/adoption-tracker.js +6 -5
- package/dist/hooks/protect-config.js +15 -5
- package/dist/{index-builder-3AOQ3ZII.js → index-builder-52IJJHJC.js} +2 -2
- package/dist/index.d.ts +8 -8
- package/dist/index.js +31 -29
- package/dist/{loader-RGQJYERV.js → loader-QOJQ6NZM.js} +4 -4
- package/dist/{mcp-NDTFZXWF.js → mcp-ZPX6O767.js} +21 -20
- package/dist/{performance-FPWKC4CN.js → performance-FXYTGJGQ.js} +5 -5
- package/dist/{review-pipeline-WUHG7PHO.js → review-pipeline-2TFQ5OAE.js} +4 -4
- package/dist/{runtime-Q3ORD7LW.js → runtime-XK6ZCHKE.js} +4 -4
- package/dist/{scan-P7YFKB77.js → scan-IYSVRPD2.js} +1 -1
- package/dist/{security-JDHTRBGC.js → security-4W2P5ZGI.js} +1 -1
- package/dist/{skill-executor-MOCUIAYS.js → skill-executor-BGUCAVBU.js} +2 -2
- package/dist/state-events-ECQAIZLU.js +25 -0
- package/dist/{validate-G3LKBOYA.js → validate-64KTEGIE.js} +5 -5
- package/dist/{validate-cross-check-4ITA72DF.js → validate-cross-check-J5PVGICB.js} +4 -4
- package/package.json +7 -7
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
version: 2
|
|
2
|
+
name: Harness PM
|
|
3
|
+
description: Owns acceptance-criteria and eval-coverage quality for specs
|
|
4
|
+
role: >
|
|
5
|
+
Owns acceptance-criteria and eval-coverage quality. Reviews every spec for
|
|
6
|
+
measurable, testable, complete success criteria; flags user-visible behaviors
|
|
7
|
+
with no covering test; blocks specs that ship with no measurable criteria.
|
|
8
|
+
The upstream twin of the outcome-eval ship gate.
|
|
9
|
+
skills:
|
|
10
|
+
- acceptance-eval
|
|
11
|
+
steps:
|
|
12
|
+
- command: validate
|
|
13
|
+
when: always
|
|
14
|
+
- skill: acceptance-eval
|
|
15
|
+
when: on_pr
|
|
16
|
+
output: auto
|
|
17
|
+
- skill: acceptance-eval
|
|
18
|
+
when: manual
|
|
19
|
+
output: auto
|
|
20
|
+
triggers:
|
|
21
|
+
- event: on_pr
|
|
22
|
+
conditions:
|
|
23
|
+
paths:
|
|
24
|
+
- 'docs/changes/**'
|
|
25
|
+
- event: manual
|
|
26
|
+
config:
|
|
27
|
+
severity: error
|
|
28
|
+
autoFix: false
|
|
29
|
+
timeout: 600000
|
|
30
|
+
outputs:
|
|
31
|
+
agents-md: true
|
|
32
|
+
ci-workflow: true
|
|
33
|
+
runtime-config: true
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Acceptance Eval
|
|
2
|
+
|
|
3
|
+
> Pre-execution LLM-judgment: does this spec carry measurable, testable, complete acceptance criteria before work begins? Resolves the spec's acceptance section, critiques observability / testability / completeness (advisory `criteriaFindings`), flags user-visible behaviors with no covering test (advisory `coverageFindings`), and emits a confidence-rated `AcceptanceVerdict` (`MEASURABLE | NOT_MEASURABLE | INCONCLUSIVE`) with a rationale. Merge authority is derived in TypeScript, never trusted from the LLM: a high-confidence `NOT_MEASURABLE` blocks merge; every other verdict is advisory. The upstream twin of the `outcome-eval` ship gate — it keeps the downstream gate fed with judgable specs.
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- On every spec entering the repo under `docs/changes/**` — triggered `on_pr` (and `manual`) — before execution begins.
|
|
8
|
+
- When you need a durable, structured answer to "can this spec's success be objectively judged later?"
|
|
9
|
+
- NOT for judging whether an implementation satisfied its spec post-execution (use `outcome-eval`, the downstream twin).
|
|
10
|
+
- NOT for authoring acceptance criteria — `acceptance-eval` reviews; humans own the thinking layer. It never writes the criteria it judges.
|
|
11
|
+
- NOT for rule-based floors (lint/architecture/entropy) or other craft ceilings — those run elsewhere.
|
|
12
|
+
- NOT when no judgable spec section exists — the verdict degrades to INCONCLUSIVE/advisory and never blocks.
|
|
13
|
+
|
|
14
|
+
## Process
|
|
15
|
+
|
|
16
|
+
### Phase 1: RESOLVE — Find the judgment section
|
|
17
|
+
|
|
18
|
+
The evaluator resolves the section internally via the fallback chain `## Success Criteria` -> `## User-Visible Behavior` -> `## Overview` (reusing `outcome-eval`'s resolver, not a fork), recording the match in `judgedAgainst`. No manual action — pass `specPath` and let `AcceptanceEvaluator` resolve. If no section is judgable, the verdict is INCONCLUSIVE/advisory.
|
|
19
|
+
|
|
20
|
+
### Phase 2: GATHER — Locate test evidence for coverage (optional)
|
|
21
|
+
|
|
22
|
+
For responsibility (b), supply test evidence so the judge can flag user-visible behaviors with no covering test: pass `testGlobs` (globs the tool reads) or `testContent` (snippets you already read). This evidence is optional: omitting it degrades `coverageFindings` to advisory-empty but NEVER affects the (c) measurability gate. Gather it when the spec describes user-visible behavior.
|
|
23
|
+
|
|
24
|
+
### Phase 3: JUDGE — Invoke the evaluator
|
|
25
|
+
|
|
26
|
+
1. Invoke the MCP tool `mcp__harness__acceptance_eval` with `{ specPath }` plus optional `{ testGlobs | testContent, model }`. The tool constructs `AcceptanceEvaluator` cli-side and returns the verdict; the supported v1 provider is the anthropic analysis provider (`ANTHROPIC_API_KEY`).
|
|
27
|
+
2. The LLM returns ONLY `measurability / confidence / criteriaFindings / coverageFindings / rationale`. `authority` is computed in TypeScript from `(measurability, confidence)` via `deriveAcceptanceAuthority` and is never read from the LLM — do not attempt to override it. The tool returns the verdict exactly as the evaluator derives it.
|
|
28
|
+
3. The call is degrade-safe: provider failure (incl. no `ANTHROPIC_API_KEY`), empty test evidence, or a missing judgable section yields INCONCLUSIVE/low/advisory. It never throws and never blocks.
|
|
29
|
+
|
|
30
|
+
### Phase 4: GATE — Render and (conditionally) halt
|
|
31
|
+
|
|
32
|
+
1. Render the verdict: `measurability`, `confidence`, `judgedAgainst`, `rationale`, `criteriaFindings` (a, advisory), and `coverageFindings` (b, advisory).
|
|
33
|
+
2. Authority rule (must match `deriveAcceptanceAuthority`): authority is `blocking` **iff** `measurability === 'NOT_MEASURABLE' && confidence === 'high'`; every other combination — including all `INCONCLUSIVE` and `MEASURABLE` cases, and all `medium`/`low` `NOT_MEASURABLE` — is `advisory`.
|
|
34
|
+
3. **On a blocking verdict: HALT before merge.** Report the missing measurable criteria and stop; the spec must not merge. Resolution requires a human adding measurable success criteria and re-running `acceptance-eval`.
|
|
35
|
+
4. On an advisory verdict: report the `criteriaFindings` and `coverageFindings` for human attention and proceed. Advisory findings do not stop the workflow.
|
|
36
|
+
|
|
37
|
+
## Harness Integration
|
|
38
|
+
|
|
39
|
+
- **`mcp__harness__acceptance_eval`** — MCP tool (the invocation surface). Inputs: `specPath` (required), `testGlobs` / `testContent` (optional (b) evidence), `model` (optional). The handler builds the cli `AnalysisProvider`, constructs `AcceptanceEvaluator`, and returns the `AcceptanceVerdict` with `authority` exactly as derived in TypeScript. (No graph persistence in v1 — the seam is documented in the tool header.)
|
|
40
|
+
- **Evaluator surface:** `AcceptanceEvaluator`, `deriveAcceptanceAuthority`, `acceptanceVerdictSchema`, `AcceptanceVerdict` are exported from `@harness-engineering/intelligence`. The section resolver is imported from `outcome-eval`, not duplicated.
|
|
41
|
+
- **Provider path (v1 supported):** the anthropic analysis provider (`ANTHROPIC_API_KEY`). When no provider is configured the call degrades to INCONCLUSIVE/advisory.
|
|
42
|
+
- **Relationship to `outcome-eval`:** `acceptance-eval` is the upstream twin. It guards spec measurability before execution; `outcome-eval` judges implementation satisfaction after. The same "authority is never read from the LLM" discipline spans both ends of the lifecycle.
|
|
43
|
+
|
|
44
|
+
## Known Limitations
|
|
45
|
+
|
|
46
|
+
- **Coverage findings (b) are heuristic and advisory.** Without `testGlobs`/`testContent` they are advisory-empty; even with evidence they are LLM-judgment over the behavior section plus located tests, not a graph-backed coverage map (deferred — see spec Decision D4). Heuristic misses are low-harm because (b) never blocks.
|
|
47
|
+
- **The (c) gate blocks only on high-confidence NOT_MEASURABLE.** A spec with weak-but-present criteria yields an advisory verdict, not a block — intentional, to avoid taste-blocks-merge false positives.
|
|
48
|
+
- **openai-compatible strict mode** is not the v1 path; the supported provider is anthropic/claude-cli.
|
|
49
|
+
|
|
50
|
+
## Success Criteria
|
|
51
|
+
|
|
52
|
+
See `docs/changes/harness-pm-persona/proposal.md` for the full criteria. This skill satisfies SC1 (skill exists, `tier: 2`, `type: rigid`, four-client artifacts), SC6 (blocking halt on high-confidence NOT_MEASURABLE; advisory-and-proceed otherwise), and SC7 (emits advisory `criteriaFindings` / `coverageFindings`).
|
|
53
|
+
|
|
54
|
+
## Examples
|
|
55
|
+
|
|
56
|
+
### Example: NOT_MEASURABLE with high confidence (blocks)
|
|
57
|
+
|
|
58
|
+
**Input:** a spec whose only "success criteria" are "the feature works well" and "users are happy" — no observable, testable assertions.
|
|
59
|
+
|
|
60
|
+
**Verdict:**
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
measurability: NOT_MEASURABLE
|
|
64
|
+
confidence: high
|
|
65
|
+
judgedAgainst: success-criteria
|
|
66
|
+
authority: blocking
|
|
67
|
+
criteriaFindings:
|
|
68
|
+
- { target: "'works well' / 'users are happy'", message: "No observable assertion — cannot be tested." }
|
|
69
|
+
rationale: "The success section contains only subjective statements; nothing can be judged at outcome time."
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
**Action:** HALT before merge. Report the missing measurable criteria; the spec must not merge until a human adds them.
|
|
73
|
+
|
|
74
|
+
### Example: measurable criteria (advisory, proceeds)
|
|
75
|
+
|
|
76
|
+
**Input:** a spec whose Success Criteria list concrete, testable assertions; one user-visible behavior lacks an obvious covering test.
|
|
77
|
+
|
|
78
|
+
**Verdict:** `measurability: MEASURABLE confidence: high authority: advisory`, with one `coverageFindings` `{target, message}` object (the one user-visible behavior lacking a covering test) surfaced for review. The workflow proceeds.
|
|
79
|
+
|
|
80
|
+
## Gates
|
|
81
|
+
|
|
82
|
+
- **Authority is never read from the LLM.** The verdict's `authority` is always `deriveAcceptanceAuthority(measurability, confidence)` computed in TypeScript. If you find yourself letting the model assert blocking/advisory, STOP — that defeats the entire purpose of this gate.
|
|
83
|
+
- **Block only on high-confidence NOT_MEASURABLE.** `authority === 'blocking'` iff `measurability === 'NOT_MEASURABLE' && confidence === 'high'`. Every other combination is advisory. Do not halt on an advisory verdict.
|
|
84
|
+
- **Never author criteria.** `acceptance-eval` reviews specs; it does not write the acceptance criteria it judges. Humans own the thinking layer.
|
|
85
|
+
- **Never block on infrastructure noise.** A provider failure, an unparseable response, or a missing spec section must resolve to INCONCLUSIVE/advisory, never a thrown error or a block. The evaluator enforces this; do not reintroduce a hard failure in the wrapper.
|
|
86
|
+
|
|
87
|
+
## Escalation
|
|
88
|
+
|
|
89
|
+
- **Blocking verdict the author disputes:** the resolution is for a human to add measurable success criteria (or fix the section the resolver judged) and re-run `acceptance-eval` — not to override the gate.
|
|
90
|
+
- **Repeated INCONCLUSIVE on a real spec:** usually means no judgable section exists. Confirm the spec has a Success Criteria / User-Visible Behavior / Overview section.
|
|
91
|
+
- **No `ANTHROPIC_API_KEY` configured:** every verdict degrades to INCONCLUSIVE/advisory and nothing blocks. Surface this to the human — the gate is effectively disabled until a provider is configured.
|
|
92
|
+
- **Verdict seems wrong (false positive/negative):** capture the spec section and verdict and route to the maintainers; do not loosen the conservative-confidence prompt ad hoc.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
name: acceptance-eval
|
|
2
|
+
version: '0.1.0'
|
|
3
|
+
description: >-
|
|
4
|
+
Pre-execution LLM-judgment skill: does a spec carry measurable, testable,
|
|
5
|
+
complete acceptance criteria before work begins? Resolves the spec's
|
|
6
|
+
acceptance section, critiques observability / testability / completeness
|
|
7
|
+
(advisory), flags user-visible behaviors with no covering test (advisory),
|
|
8
|
+
and emits a confidence-rated AcceptanceVerdict
|
|
9
|
+
(MEASURABLE | NOT_MEASURABLE | INCONCLUSIVE). Authority is derived in
|
|
10
|
+
TypeScript, never from the LLM: a high-confidence NOT_MEASURABLE blocks
|
|
11
|
+
merge; every other verdict is advisory. The upstream twin of the
|
|
12
|
+
outcome-eval ship gate.
|
|
13
|
+
stability: draft
|
|
14
|
+
cognitive_mode: constructive-architect
|
|
15
|
+
triggers:
|
|
16
|
+
- manual
|
|
17
|
+
- on_pr
|
|
18
|
+
platforms:
|
|
19
|
+
- claude-code
|
|
20
|
+
- cursor
|
|
21
|
+
- codex
|
|
22
|
+
- gemini-cli
|
|
23
|
+
tools:
|
|
24
|
+
- Bash
|
|
25
|
+
- Read
|
|
26
|
+
- Glob
|
|
27
|
+
- Grep
|
|
28
|
+
mcp:
|
|
29
|
+
tool: acceptance_eval
|
|
30
|
+
# specPath is the only required input. testGlobs/testContent are the (b)
|
|
31
|
+
# coverage evidence: omitting them degrades coverage findings to
|
|
32
|
+
# advisory-empty but NEVER affects the (c) measurability gate.
|
|
33
|
+
input:
|
|
34
|
+
specPath: 'string (required) — absolute or repo-relative path to the spec markdown to evaluate'
|
|
35
|
+
testGlobs: 'string[] (optional) — globs locating test files that evidence (b) coverage; omission degrades coverage findings to advisory-empty, never the (c) gate'
|
|
36
|
+
testContent: 'string (optional) — pre-read test snippets as (b) coverage evidence; an alternative to testGlobs'
|
|
37
|
+
model: 'string (optional) — model override for the acceptance-eval LLM call'
|
|
38
|
+
type: rigid
|
|
39
|
+
tier: 2
|
|
40
|
+
phases:
|
|
41
|
+
- name: resolve
|
|
42
|
+
description: Resolve the judgment section (Success Criteria -> User-Visible Behavior -> Overview), recorded in judgedAgainst
|
|
43
|
+
required: true
|
|
44
|
+
- name: gather
|
|
45
|
+
description: Locate test evidence for (b) coverage via testGlobs/testContent; optional — omission degrades coverage findings, never the (c) gate
|
|
46
|
+
required: false
|
|
47
|
+
- name: judge
|
|
48
|
+
description: Invoke AcceptanceEvaluator; the LLM returns measurability/confidence/criteriaFindings/coverageFindings/rationale; authority is derived in TS via deriveAcceptanceAuthority
|
|
49
|
+
required: true
|
|
50
|
+
- name: gate
|
|
51
|
+
description: Render the verdict; on a blocking verdict (NOT_MEASURABLE + high confidence) HALT before merge
|
|
52
|
+
required: true
|
|
53
|
+
state:
|
|
54
|
+
persistent: false
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
- When the problem space is ambiguous and needs exploration
|
|
9
9
|
- When multiple approaches exist and tradeoffs must be weighed
|
|
10
10
|
- When `on_new_feature` trigger fires and scope is non-trivial
|
|
11
|
-
- NOT when the implementation path is already clear (go straight to harness-planning)
|
|
11
|
+
- NOT when the implementation path is already clear (go straight to harness-planning, or harness-autopilot if the spec already exists)
|
|
12
12
|
- NOT when fixing a bug with an obvious root cause (use harness-debugging or harness-tdd)
|
|
13
13
|
- NOT for simple refactors with no design decisions (use harness-refactoring)
|
|
14
14
|
|
|
@@ -64,22 +64,23 @@ When no arguments are provided (standalone invocation), the skill operates exact
|
|
|
64
64
|
|
|
65
65
|
### Phase 2: EVALUATE -- Ask Questions and Narrow
|
|
66
66
|
|
|
67
|
-
1. **Ask ONE question at a time.** Ask the most important question first. Wait for the answer. Let it inform the next question.
|
|
67
|
+
1. **Ask ONE question at a time, in plain text.** Ask the most important question first. Wait for the answer. Let it inform the next question.
|
|
68
68
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
69
|
+
**Ask directly in your reply. Do NOT route the question through `emit_interaction`, `AskUserQuestion`, or any tool.** `emit_interaction` records the question in state but does not display it to the human — the client collapses it to "Called harness" and the rendered prompt only returns to the model, so the human sees nothing and you end up narrating a question they cannot answer. `AskUserQuestion` is Claude-Code-only and caps headers at 12 chars / 4 options. Plain text in your own message is the only channel that reliably reaches the human across every tool (Claude Code, Cursor, Codex, Gemini CLI).
|
|
70
|
+
|
|
71
|
+
Present the question as a markdown table so tradeoffs are scannable, state your recommendation, then STOP and wait for the human's reply:
|
|
72
|
+
|
|
73
|
+
```markdown
|
|
74
|
+
### Decision needed: For auth, which approach should we use?
|
|
75
|
+
|
|
76
|
+
| | A) Existing JWT middleware | B) OAuth2 via provider X | C) External auth service |
|
|
77
|
+
| ---------- | -------------------------- | ------------------------ | ------------------------ |
|
|
78
|
+
| **Pros** | Already in codebase | Industry standard | Zero maintenance |
|
|
79
|
+
| **Cons** | No refresh tokens | New dependency | Vendor lock-in; cost |
|
|
80
|
+
| **Risk** | Low | Medium | Medium |
|
|
81
|
+
| **Effort** | Low | Medium | Low |
|
|
82
|
+
|
|
83
|
+
**Recommendation:** A) Existing JWT middleware (confidence: high) — sufficient for current requirements.
|
|
83
84
|
```
|
|
84
85
|
|
|
85
86
|
2. **Prefer multiple choice over open-ended questions.** Give 2-4 concrete options with brief tradeoff notes.
|
|
@@ -169,24 +170,21 @@ These flow into `handoff.json` `contextKeywords` field. Select keywords that hel
|
|
|
169
170
|
|
|
170
171
|
5. **Run `harness validate`** to verify proper placement and project health.
|
|
171
172
|
|
|
172
|
-
6. **Request sign-off
|
|
173
|
+
6. **Request sign-off in plain text.** Ask directly in your reply — do NOT route the request through `emit_interaction` or `AskUserQuestion` (the human will not see it). Present it as:
|
|
173
174
|
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
risk: "low"
|
|
183
|
-
}
|
|
184
|
-
})
|
|
175
|
+
```markdown
|
|
176
|
+
Approve spec at <file-path>?
|
|
177
|
+
|
|
178
|
+
Context: <one-paragraph summary>
|
|
179
|
+
Impact: Spec approval unlocks implementation planning. No code changes yet.
|
|
180
|
+
Risk: low
|
|
181
|
+
|
|
182
|
+
Proceed? (yes/no)
|
|
185
183
|
```
|
|
186
184
|
|
|
187
|
-
The human must explicitly approve before this skill is complete.
|
|
185
|
+
The human must explicitly approve (a clear "yes") before this skill is complete.
|
|
188
186
|
|
|
189
|
-
7. **Promote the roadmap row.** If `docs/roadmap.md`
|
|
187
|
+
7. **Promote the roadmap row.** If a roadmap exists (a `docs/roadmap.md` aggregate **or** a `docs/roadmap.d/` shard directory), transition the named row to `planned` and link the spec in a single structured call (steps 7 and 8 are intentionally ordered so the commit captures the roadmap mutation). Skip silently only when no roadmap exists. In sharded mode `manage_roadmap promote` writes the single row's shard under `docs/roadmap.d/` **and** regenerates the `docs/roadmap.md` aggregate — both are staged in step 8.
|
|
190
188
|
- Derive the lookup key from the slash-command `ARGUMENTS` string (D1). Derive the summary from the spec title (the H1 heading).
|
|
191
189
|
- Call `manage_roadmap` with action `promote`, `feature: "<ARGUMENTS>"`, `spec: "docs/changes/<feature>/proposal.md"`, and `summary: "<H1>"`.
|
|
192
190
|
- Branch on the returned `PromoteResult` envelope:
|
|
@@ -210,11 +208,11 @@ These flow into `handoff.json` `contextKeywords` field. Select keywords that hel
|
|
|
210
208
|
8. **Commit spec artifacts and roadmap promotion.** After a successful promote (or a silent skip when no roadmap exists), commit the spec, skill recommendations, and the roadmap mutation together so the promotion ships atomically with the spec:
|
|
211
209
|
|
|
212
210
|
```bash
|
|
213
|
-
git add docs/changes/<feature>/proposal.md docs/changes/<feature>/SKILLS.md docs/roadmap.md
|
|
211
|
+
git add docs/changes/<feature>/proposal.md docs/changes/<feature>/SKILLS.md docs/roadmap.d/ docs/roadmap.md
|
|
214
212
|
git commit -m "docs(<feature>): add spec and promote to planned"
|
|
215
213
|
```
|
|
216
214
|
|
|
217
|
-
|
|
215
|
+
Stage the roadmap by mode: **sharded mode** (`docs/roadmap.d/` present) — stage both `docs/roadmap.d/` (the written shard) and the regenerated `docs/roadmap.md` aggregate; **monolith mode** — stage only `docs/roadmap.md`; **no roadmap** — omit both. Do not skip this step — `harness-execution` commits only implementation files, so a spec uncommitted here stays untracked until someone notices. If a pre-commit hook reformats a file, re-add and re-commit.
|
|
218
216
|
|
|
219
217
|
9. **Write handoff and suggest transition.** After approval:
|
|
220
218
|
|
|
@@ -241,14 +239,21 @@ These flow into `handoff.json` `contextKeywords` field. Select keywords that hel
|
|
|
241
239
|
}
|
|
242
240
|
```
|
|
243
241
|
|
|
244
|
-
|
|
242
|
+
**Choose the next skill with the human — ask in plain text, not via `emit_interaction`** (a `transition` records the handoff; it does not surface the choice). Present both paths and recommend autopilot:
|
|
243
|
+
|
|
244
|
+
> Spec approved. How do you want to build it?
|
|
245
|
+
>
|
|
246
|
+
> - **Autopilot (recommended)** — autonomously chains plan → execute → verify → review, pausing only at decision points. Best when the spec's `## Implementation Order` lays out clear phases.
|
|
247
|
+
> - **Planning** — work the implementation plan interactively before any execution. Choose this when phases are uncertain or you want to drive the breakdown by hand.
|
|
248
|
+
|
|
249
|
+
Record the answer as `<next>` (`autopilot` or `planning`), then call `emit_interaction`:
|
|
245
250
|
|
|
246
251
|
```json
|
|
247
252
|
{
|
|
248
253
|
"type": "transition",
|
|
249
254
|
"transition": {
|
|
250
255
|
"completedPhase": "brainstorming",
|
|
251
|
-
"suggestedNext": "
|
|
256
|
+
"suggestedNext": "<next>",
|
|
252
257
|
"reason": "Spec approved and written to docs/",
|
|
253
258
|
"artifacts": ["<spec path>"],
|
|
254
259
|
"requiresConfirmation": true,
|
|
@@ -269,7 +274,10 @@ These flow into `handoff.json` `contextKeywords` field. Select keywords that hel
|
|
|
269
274
|
}
|
|
270
275
|
```
|
|
271
276
|
|
|
272
|
-
If confirmed
|
|
277
|
+
If confirmed, dispatch by the recorded choice:
|
|
278
|
+
- `autopilot` → invoke harness-autopilot with `--spec <spec path>`.
|
|
279
|
+
- `planning` → invoke harness-planning with the spec path.
|
|
280
|
+
|
|
273
281
|
If declined: stop. The handoff is written for future invocation.
|
|
274
282
|
|
|
275
283
|
---
|
|
@@ -348,11 +356,11 @@ Technical claims about existing code, architecture, or tradeoffs MUST cite evide
|
|
|
348
356
|
- **`harness check-docs`** -- Verify spec does not conflict with existing docs.
|
|
349
357
|
- **`STRATEGY.md` grounding (Phase 1 step 0a)** -- When present at repo root and valid, loaded via the `read_strategy` MCP tool (which the harness MCP server backs with `validateStrategy` + `parseStrategyDoc` + `asStrategyDoc` from `@harness-engineering/core` — projects do not need core installed). Soft-fails when absent or invalid; cites strategy sections in the spec's evidence annotations when present. Boundary with `harness-strategy`: brainstorming READS; `harness-strategy` WRITES. Never modify `STRATEGY.md` from this skill.
|
|
350
358
|
- **Spec location** -- `docs/changes/<feature>/proposal.md`.
|
|
351
|
-
- **Handoff** -- Once approved,
|
|
359
|
+
- **Handoff** -- Once approved, hand off per the human's choice: harness-autopilot (recommended -- autonomous plan → execute → verify → review) or harness-planning (interactive plan only).
|
|
352
360
|
- **Session directory** — When session slug is known, handoff goes to `.harness/sessions/<slug>/handoff.json`. The session directory structure is: `handoff.json`, `state.json`, `artifacts.json` (registry of spec/plan paths and file lists). Do not write to `.harness/handoff.json` in session context.
|
|
353
|
-
- **Spec commit** -- After sign-off, the Phase 4 step 8 commit captures `docs/changes/<feature>/proposal.md`, `docs/changes/<feature>/SKILLS.md`, and the promoted `docs/roadmap.md`
|
|
361
|
+
- **Spec commit** -- After sign-off, the Phase 4 step 8 commit captures `docs/changes/<feature>/proposal.md`, `docs/changes/<feature>/SKILLS.md`, and the promoted roadmap together (in sharded mode: the `docs/roadmap.d/` shard plus the regenerated `docs/roadmap.md` aggregate; in monolith mode: `docs/roadmap.md`) so the spec enters git history at approval, not retroactively. `harness-execution` does not backfill these — see issue #487.
|
|
354
362
|
- **Roadmap promotion** -- After approval (Phase 4 step 7), call `manage_roadmap` action `promote` to transition the named row to `planned` and link the spec atomically with the spec commit. Falls back to `add` (create new row) when the row does not exist; refuses on `in-progress`/`done`/`ambiguous`. Skip silently if no roadmap.
|
|
355
|
-
- **`emit_interaction`** -- End of Phase 4 to
|
|
363
|
+
- **`emit_interaction`** -- End of Phase 4 to record the transition to harness-autopilot or harness-planning (the human picks; default autopilot). The choice itself is asked in plain text -- `emit_interaction` records the confirmed transition, it does not surface the question.
|
|
356
364
|
|
|
357
365
|
#### Requirement Phrasing
|
|
358
366
|
|
|
@@ -539,17 +539,16 @@ gh api repos/{owner}/{repo}/pulls/{pr}/comments \
|
|
|
539
539
|
|
|
540
540
|
### Review Acceptance
|
|
541
541
|
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
})
|
|
542
|
+
Ask for acceptance directly in your reply. Do NOT route this through `emit_interaction`, `AskUserQuestion`, or any tool. `emit_interaction` records the prompt but does not display it to the human (the client collapses the call to "Called harness" and the rendered text only returns to the model); `AskUserQuestion` is Claude-Code-only and caps headers at 12 chars / 4 options. Plain text in your own message is the only channel that reliably reaches the human across every tool (Claude Code, Cursor, Codex, Gemini CLI). Present it as:
|
|
543
|
+
|
|
544
|
+
```markdown
|
|
545
|
+
Review complete: <Assessment>. Accept review?
|
|
546
|
+
|
|
547
|
+
Context: <N critical, N important, N suggestion findings>
|
|
548
|
+
Impact: Accepting finalizes findings. Approve = ready for merge. Request-changes = fixes needed.
|
|
549
|
+
Risk: <low if approve, high if critical>
|
|
550
|
+
|
|
551
|
+
Proceed? (yes/no)
|
|
553
552
|
```
|
|
554
553
|
|
|
555
554
|
#### Handoff and Transition
|
|
@@ -186,40 +186,37 @@ Three checkpoint types. Each requires pausing execution.
|
|
|
186
186
|
|
|
187
187
|
**`[checkpoint:human-verify]` — Show and Confirm**
|
|
188
188
|
|
|
189
|
-
Stop. Present
|
|
189
|
+
Stop. Present the confirmation in plain text in your reply — do NOT route this through `emit_interaction`, `AskUserQuestion`, or any tool. `emit_interaction` records the prompt but does not display it to the human (the client collapses the call to "Called harness" and the rendered text only returns to the model); `AskUserQuestion` is Claude-Code-only and caps headers at 12 chars / 4 options. Plain text in your own message is the only channel that reliably reaches the human across every tool (Claude Code, Cursor, Codex, Gemini CLI).
|
|
190
190
|
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
risk: "low"
|
|
200
|
-
}
|
|
201
|
-
})
|
|
191
|
+
```markdown
|
|
192
|
+
Task N complete. Output: <summary>. Continue to Task N+1?
|
|
193
|
+
|
|
194
|
+
Context: <test output or diff summary>
|
|
195
|
+
Impact: Continuing proceeds to next task. Declining pauses for review.
|
|
196
|
+
Risk: low
|
|
197
|
+
|
|
198
|
+
Proceed? (yes/no)
|
|
202
199
|
```
|
|
203
200
|
|
|
204
201
|
Wait for human confirmation.
|
|
205
202
|
|
|
206
203
|
**`[checkpoint:decision]` — Present Options and Wait**
|
|
207
204
|
|
|
208
|
-
Stop. Present
|
|
205
|
+
Stop. Present the decision in plain text in your reply — do NOT route this through `emit_interaction`, `AskUserQuestion`, or any tool. `emit_interaction` records the prompt but does not display it to the human (the client collapses the call to "Called harness" and the rendered text only returns to the model); `AskUserQuestion` is Claude-Code-only and caps headers at 12 chars / 4 options. Plain text in your own message is the only channel that reliably reaches the human across every tool (Claude Code, Cursor, Codex, Gemini CLI).
|
|
209
206
|
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
207
|
+
Present the options as a markdown table so tradeoffs are scannable, state your recommendation, then STOP and wait for the human's reply:
|
|
208
|
+
|
|
209
|
+
```markdown
|
|
210
|
+
### Decision needed: Task N requires a decision: <description>
|
|
211
|
+
|
|
212
|
+
| | A) <option A> | B) <option B> |
|
|
213
|
+
| ---------- | --------------- | --------------- |
|
|
214
|
+
| **Pros** | <option A pros> | <option B pros> |
|
|
215
|
+
| **Cons** | <option A cons> | <option B cons> |
|
|
216
|
+
| **Risk** | low | medium |
|
|
217
|
+
| **Effort** | low | medium |
|
|
218
|
+
|
|
219
|
+
**Recommendation:** A) <option A> (confidence: medium) — <why>.
|
|
223
220
|
```
|
|
224
221
|
|
|
225
222
|
Wait for human choice.
|
|
@@ -378,6 +375,7 @@ Claims about task completion, test results, or code behavior MUST cite evidence:
|
|
|
378
375
|
- **`harness state learn "<message>"`** — Append a learning from CLI.
|
|
379
376
|
- **State/Learnings files** — Session-scoped when session known, otherwise `.harness/`. State updated after every task; learnings append-only.
|
|
380
377
|
- **Roadmap claim** — Phase 2 Step 0: `manage_roadmap update` with `status: in-progress` + `assignee: <currentUser>` claims the item at execution start (the assignee = "who is executing" invariant). Stop if `currentUser` is unresolvable. Marking the row `done` later auto-clears the assignee via the lifecycle `setStatus` chokepoint — never leave a non-in-progress row assigned (RMH005).
|
|
378
|
+
- **Auto-done (do NOT hand-mark rows done)** — A row reaches `done` automatically when the implementing PR merges and closes its linked issue: the merge-triggered auto-done reconciler flips exactly that shard via `External-ID` (CI Action authoritative; `harness roadmap reconcile` is the offline fallback). Execution should claim the row `in-progress` and let the merge drive it to `done` — do not call `manage_roadmap update status:done` by hand. See knowledge [`merge-triggered-auto-done.md`](../../../../docs/knowledge/roadmap/merge-triggered-auto-done.md). In sharded mode (`docs/roadmap.d/` present) the claim writes one shard and regenerates the aggregate.
|
|
381
379
|
- **Roadmap sync** — After plan completion, `manage_roadmap sync` with `apply: true`. Mandatory when roadmap exists. No `force_sync: true`.
|
|
382
380
|
- **`emit_interaction`** — Auto-transition to harness-verification at plan completion.
|
|
383
381
|
|
|
@@ -66,7 +66,7 @@ Before running sub-phases, resolve the effective integration tier:
|
|
|
66
66
|
Tier escalated from `small` to `medium`: 8 new exports detected.
|
|
67
67
|
```
|
|
68
68
|
|
|
69
|
-
|
|
69
|
+
Inform the human and get acknowledgment in plain text in your reply (do NOT route this through `emit_interaction` or `AskUserQuestion` — they record the prompt but do not display it to the human, so they won't see it).
|
|
70
70
|
|
|
71
71
|
---
|
|
72
72
|
|
|
@@ -213,7 +213,7 @@ Skip this sub-phase for `small` tier or `fast` rigor. For medium and large tiers
|
|
|
213
213
|
<What follows from this decision -- positive, negative, and neutral?>
|
|
214
214
|
```
|
|
215
215
|
|
|
216
|
-
d. **Present for review.** In `thorough` rigor, present each ADR draft
|
|
216
|
+
d. **Present for review.** In `thorough` rigor, present each ADR draft in plain text in your reply and wait for human approval — do NOT route it through `emit_interaction` or `AskUserQuestion`, which record the prompt but do not display it to the human (the client collapses the call to "Called harness" and the text only returns to the model). In `standard` rigor, auto-approve but log the draft for review.
|
|
217
217
|
|
|
218
218
|
#### Knowledge Graph Enrichment (medium + large tiers)
|
|
219
219
|
|
|
@@ -266,7 +266,7 @@ Skip this sub-phase for `small` tier or `fast` rigor. For medium and large tiers
|
|
|
266
266
|
|
|
267
267
|
---
|
|
268
268
|
|
|
269
|
-
### Sub-Phase 3: UPDATE (medium
|
|
269
|
+
### Sub-Phase 3: UPDATE (medium and large tiers)
|
|
270
270
|
|
|
271
271
|
Report progress: `**[UPDATE]** Checking project metadata updates`
|
|
272
272
|
|
|
@@ -364,22 +364,24 @@ After all applicable sub-phases complete, produce the combined integration repor
|
|
|
364
364
|
|
|
365
365
|
Immediately invoke harness-code-review without waiting for user input.
|
|
366
366
|
|
|
367
|
-
4. **On FAIL:** Do NOT emit a transition. Report incomplete items with fix instructions
|
|
367
|
+
4. **On FAIL:** Do NOT emit a transition. Report incomplete items with fix instructions, then ask the human how to proceed in plain text.
|
|
368
368
|
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
369
|
+
**Ask directly in your reply. Do NOT route the question through `emit_interaction`, `AskUserQuestion`, or any tool.** `emit_interaction` records the prompt but does not display it to the human — the client collapses the call to "Called harness" and the rendered text only returns to the model, so the human sees nothing and you end up narrating a question they cannot answer. `AskUserQuestion` is Claude-Code-only and caps headers at 12 chars / 4 options. Plain text in your own message is the only channel that reliably reaches the human across every tool (Claude Code, Cursor, Codex, Gemini CLI).
|
|
370
|
+
|
|
371
|
+
Present the options as a markdown table, state your recommendation, then STOP and wait for the human's reply:
|
|
372
|
+
|
|
373
|
+
```markdown
|
|
374
|
+
### Decision needed: Integration failed. <N> items incomplete. How to proceed?
|
|
375
|
+
|
|
376
|
+
| | fix | skip | stop |
|
|
377
|
+
| ---------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------ | ------------------------- |
|
|
378
|
+
| **What** | Re-enter EXECUTE with integration-specific fix tasks, then re-VERIFY, re-INTEGRATE | Record decision and proceed to REVIEW (human override) | Save state and exit |
|
|
379
|
+
| **Pros** | Closes gaps before review | Unblocks progress immediately | Preserves state for later |
|
|
380
|
+
| **Cons** | Another execute/verify cycle | Ships with known gaps | No forward progress |
|
|
381
|
+
| **Risk** | Low | Medium | Low |
|
|
382
|
+
| **Effort** | Medium | Low | Low |
|
|
383
|
+
|
|
384
|
+
**Recommendation:** fix (confidence: high) — fixing integration gaps now prevents review rework.
|
|
383
385
|
```
|
|
384
386
|
|
|
385
387
|
- **fix:** Record fix tasks in handoff. The autopilot re-enters EXECUTE with those tasks.
|
|
@@ -449,7 +451,7 @@ Every pass/fail assertion in the integration report MUST include concrete eviden
|
|
|
449
451
|
- **`harness check-deps`** -- Run during WIRE if new modules were added.
|
|
450
452
|
- **`ingest_source`** -- Run during MATERIALIZE to enrich knowledge graph with decision nodes.
|
|
451
453
|
- **`manage_roadmap`** -- Run during UPDATE to sync roadmap status. Use `sync` with `apply: true`.
|
|
452
|
-
- **`emit_interaction`** -- Tier escalation
|
|
454
|
+
- **`emit_interaction`** -- Used only for the transition to REVIEW on pass. Tier escalation, ADR review (thorough mode), and fail/skip/stop decisions are asked in plain text in your reply.
|
|
453
455
|
- **Session directory** -- `.harness/sessions/<slug>/` contains all report files. Do not write to global `.harness/` when session slug is known.
|
|
454
456
|
|
|
455
457
|
## Success Criteria
|