shapeup-sdlc 1.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.example.json +20 -0
- package/.claude-plugin/marketplace.json +16 -0
- package/.claude-plugin/plugin.json +18 -0
- package/.env.shapeup.example +14 -0
- package/AGENTS.md +133 -0
- package/LICENSE +21 -0
- package/README.md +362 -0
- package/SECURITY.md +72 -0
- package/bin/init.mjs +329 -0
- package/commands/build.md +14 -0
- package/commands/eval.md +15 -0
- package/commands/hammer.md +12 -0
- package/commands/orient.md +11 -0
- package/commands/qa.md +12 -0
- package/commands/retro.md +13 -0
- package/commands/scopes.md +14 -0
- package/commands/shape.md +12 -0
- package/commands/ship.md +53 -0
- package/commands/wire.md +11 -0
- package/hooks/anti-rationalization.mjs +244 -0
- package/hooks/compact-snapshot.mjs +47 -0
- package/hooks/gate-deadline.mjs +151 -0
- package/hooks/gate-intake.mjs +110 -0
- package/hooks/gate-l2.mjs +161 -0
- package/hooks/gate-zerowork.mjs +264 -0
- package/hooks/hooks.json +118 -0
- package/hooks/lib/decision.mjs +183 -0
- package/hooks/safety-spine.mjs +296 -0
- package/hooks/sandbox-guard.mjs +172 -0
- package/hooks/session-rehydrate.mjs +109 -0
- package/hooks/slop-cleaner.mjs +176 -0
- package/oracles/_shared.mjs +46 -0
- package/oracles/http-oracle.mjs +155 -0
- package/oracles/index.mjs +36 -0
- package/oracles/process-oracle.mjs +146 -0
- package/oracles/snapshot-oracle.mjs +119 -0
- package/oracles/test-oracle.mjs +138 -0
- package/package.json +49 -0
- package/skills/advisor-protocol/SKILL.md +171 -0
- package/skills/ba-pitch-analyzer/SKILL.md +175 -0
- package/skills/ba-pitch-analyzer/assets/templates/_index.tmpl.md +84 -0
- package/skills/ba-pitch-analyzer/assets/templates/api-feasibility.tmpl.md +65 -0
- package/skills/ba-pitch-analyzer/assets/templates/assess-report.tmpl.md +127 -0
- package/skills/ba-pitch-analyzer/assets/templates/contracts/be-service.contract.tmpl.md +62 -0
- package/skills/ba-pitch-analyzer/assets/templates/contracts/offline-storage.contract.tmpl.md +92 -0
- package/skills/ba-pitch-analyzer/assets/templates/contracts/third-party-api.contract.tmpl.md +66 -0
- package/skills/ba-pitch-analyzer/assets/templates/cross-context/context-map.tmpl.md +64 -0
- package/skills/ba-pitch-analyzer/assets/templates/cross-context/event-choreography.tmpl.md +77 -0
- package/skills/ba-pitch-analyzer/assets/templates/cross-context/migration-plan.tmpl.md +104 -0
- package/skills/ba-pitch-analyzer/assets/templates/cross-context/team-handoff.tmpl.md +79 -0
- package/skills/ba-pitch-analyzer/assets/templates/domain-model.tmpl.md +79 -0
- package/skills/ba-pitch-analyzer/assets/templates/feedback.tmpl.md +87 -0
- package/skills/ba-pitch-analyzer/assets/templates/integration.tmpl.md +64 -0
- package/skills/ba-pitch-analyzer/assets/templates/run-state.tmpl.md +77 -0
- package/skills/ba-pitch-analyzer/assets/templates/scope-summary.tmpl.md +105 -0
- package/skills/ba-pitch-analyzer/assets/templates/synthesis.tmpl.md +215 -0
- package/skills/ba-pitch-analyzer/assets/templates/task-board.tmpl.md +35 -0
- package/skills/ba-pitch-analyzer/assets/templates/task-spike.tmpl.md +86 -0
- package/skills/ba-pitch-analyzer/assets/templates/task.tmpl.md +84 -0
- package/skills/ba-pitch-analyzer/assets/templates/usecase.tmpl.md +105 -0
- package/skills/ba-pitch-analyzer/assets/templates/ux-behavior.tmpl.md +65 -0
- package/skills/ba-pitch-analyzer/references/contract-patterns.md +152 -0
- package/skills/ba-pitch-analyzer/references/ddd-patterns.md +173 -0
- package/skills/ba-pitch-analyzer/references/doc-schemas.md +275 -0
- package/skills/ba-pitch-analyzer/references/integration-analysis.md +152 -0
- package/skills/ba-pitch-analyzer/references/task-generation.md +623 -0
- package/skills/ba-pitch-analyzer/references/test-surface.md +102 -0
- package/skills/ba-pitch-analyzer/references/ux-behavior-patterns.md +269 -0
- package/skills/ba-pitch-analyzer/scripts/board-derive.mjs +241 -0
- package/skills/ba-pitch-analyzer/scripts/spec-lint.mjs +239 -0
- package/skills/coach/SKILL.md +202 -0
- package/skills/orient/SKILL.md +262 -0
- package/skills/qa-edge-hunter/SKILL.md +369 -0
- package/skills/scope-architect/SKILL.md +111 -0
- package/skills/scope-hammer/SKILL.md +186 -0
- package/skills/shapeup/SKILL.md +403 -0
- package/skills/shapeup/resources/breadboard-reflection.md +88 -0
- package/skills/shapeup/resources/breadboarding.md +334 -0
- package/skills/shapeup/resources/context-compaction.md +240 -0
- package/skills/shapeup/resources/framing-doc.md +92 -0
- package/skills/shapeup/resources/kickoff-doc.md +95 -0
- package/skills/shapeup/resources/shaping.md +194 -0
- package/skills/shapeup/resources/spike.md +86 -0
- package/skills/solution-architect/SKILL.md +129 -0
- package/skills/spec-evaluator/README.md +93 -0
- package/skills/spec-evaluator/SKILL.md +213 -0
- package/skills/spec-evaluator/references/anti-leniency.md +50 -0
- package/skills/spec-evaluator/references/dimension-contract.md +126 -0
- package/skills/spec-evaluator/references/dimensions/_registry.md +40 -0
- package/skills/spec-evaluator/references/dimensions/completeness.md +102 -0
- package/skills/spec-evaluator/references/dimensions/integration.md +129 -0
- package/skills/spec-evaluator/references/dimensions/performance.md +48 -0
- package/skills/spec-evaluator/references/dimensions/security.md +60 -0
- package/skills/spec-evaluator/references/dimensions/spec-conformance.md +134 -0
- package/skills/spec-evaluator/references/dimensions/tdd-surface.md +110 -0
- package/skills/spec-evaluator/references/dimensions/test-surface-conformance.md +100 -0
- package/skills/spec-evaluator/references/probing.md +177 -0
- package/skills/spec-evaluator/references/report-schema.md +101 -0
- package/skills/spec-evaluator/references/verdict-ledger.md +92 -0
- package/skills/spec-evaluator/scripts/verdict-ledger.mjs +166 -0
- package/skills/task-executor/SKILL.md +194 -0
- package/skills/tech-lead/README.md +71 -0
- package/skills/tech-lead/SKILL.md +459 -0
- package/skills/tech-lead/references/delegation.md +254 -0
- package/skills/tech-lead/references/gates.md +379 -0
- package/skills/tech-lead/references/invocation.md +45 -0
- package/skills/tech-lead/references/ledger-schema.md +214 -0
- package/skills/tech-lead/references/round-protocol.md +184 -0
- package/skills/tech-lead/references/state-model.md +66 -0
- package/skills/tech-lead/references/tiny-lane.md +52 -0
- package/skills/tech-lead/schemas/domain.schema.json +2294 -0
- package/skills/tech-lead/schemas/gate-answers.schema.json +92 -0
- package/skills/tech-lead/schemas/work-order.schema.json +21 -0
- package/skills/tech-lead/schemas/work-result.schema.json +40 -0
- package/skills/tech-lead/scripts/aegis-digest.mjs +124 -0
- package/skills/tech-lead/scripts/budget-check.mjs +156 -0
- package/skills/tech-lead/scripts/compile-order.mjs +518 -0
- package/skills/tech-lead/scripts/fit-check.mjs +196 -0
- package/skills/tech-lead/scripts/gate-answers.mjs +338 -0
- package/skills/tech-lead/scripts/ingest-result.mjs +270 -0
- package/skills/tech-lead/scripts/init-run.mjs +326 -0
- package/skills/tech-lead/scripts/lib/argv.mjs +224 -0
- package/skills/tech-lead/scripts/lib/contract-md.mjs +481 -0
- package/skills/tech-lead/scripts/lib/is-main.mjs +82 -0
- package/skills/tech-lead/scripts/lib/paths.mjs +278 -0
- package/skills/tech-lead/scripts/lib/ratchet-tree.mjs +112 -0
- package/skills/tech-lead/scripts/run-snapshot.mjs +269 -0
- package/skills/tech-lead/scripts/ship-report.mjs +275 -0
- package/skills/tech-lead/scripts/stats.mjs +423 -0
- package/skills/tech-lead/scripts/t0-verify.mjs +470 -0
- package/skills/tech-lead/scripts/trace-lint.mjs +367 -0
- package/skills/tech-lead/scripts/validate-envelope.mjs +283 -0
- package/skills/translator/README.md +66 -0
- package/skills/translator/SKILL.md +258 -0
- package/skills/translator/references/preservation-rules.md +102 -0
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# Report Schema — `evaluation/EVAL-<task_id>.md`
|
|
2
|
+
|
|
3
|
+
The handoff file. Communication between judge and doer is by file: the evaluator writes
|
|
4
|
+
this, the generator (`task-executor`) reads the bug list as its next input. Keep it
|
|
5
|
+
actionable enough that the generator never has to ask a follow-up question.
|
|
6
|
+
|
|
7
|
+
## Frontmatter
|
|
8
|
+
```yaml
|
|
9
|
+
---
|
|
10
|
+
type: eval-report
|
|
11
|
+
feature: [slug]
|
|
12
|
+
task: TASK-NNN(.variant)
|
|
13
|
+
verdict: pass | fail
|
|
14
|
+
dimensions_run: [spec-conformance] # the active set this run
|
|
15
|
+
dimensions_ignored: [security, performance]
|
|
16
|
+
bug_count: [N]
|
|
17
|
+
browser_mode: cli | mcp | none
|
|
18
|
+
evaluator: spec-evaluator v0.1
|
|
19
|
+
eval_at: [ISO date]
|
|
20
|
+
linked_docs: ["[[usecases/UC-Name]]", "[[scope-summary]]"] # + "[[tasks/TASK-NNN-slug]]" only when a local task file exists this run (V0.2b)
|
|
21
|
+
t0_citation: ~ | { scope_id: cart-creation, artifact: t0/verdicts/r2-a3.json, sha256: 1a2b3c… }
|
|
22
|
+
---
|
|
23
|
+
```
|
|
24
|
+
`t0_citation` is `~` (not applicable) only when the spec folder has no scope contracts
|
|
25
|
+
(pre-v0.3.0 spec). On any scoped spec it is required — GATE V0.7/V2.5 in SKILL.md — an absent
|
|
26
|
+
citation there means the report should not have been written yet.
|
|
27
|
+
|
|
28
|
+
## Body sections (in order)
|
|
29
|
+
|
|
30
|
+
### 1. Verdict line
|
|
31
|
+
```
|
|
32
|
+
OVERALL: FAIL — spec-conformance failed (1 AC, 1 non-go breach). 2 bugs. Ignored: security, performance.
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### 2. Per-dimension criteria table
|
|
36
|
+
One block per dimension run. N/A criteria shown as N/A, never PASS.
|
|
37
|
+
```
|
|
38
|
+
## spec-conformance — FAIL (threshold: all-pass)
|
|
39
|
+
| id | criterion | verdict | evidence |
|
|
40
|
+
|----|-----------|---------|----------|
|
|
41
|
+
| SC-AC | every UC Step/Error Case passes | ❌ FAIL | UC-Checkout Step 4 "Pay" click throws — apps/web/checkout/Pay.tsx:84 |
|
|
42
|
+
| SC-DONE-WHEN | "User pays successfully" | ❌ FAIL | redirect to /success never fires (see AC2) |
|
|
43
|
+
| SC-REQ | request matches contract | ✅ PASS | curl 200, fields match #Request |
|
|
44
|
+
| SC-RES | response mapping | ✅ PASS | field-by-field match to #Response |
|
|
45
|
+
| SC-ERR | error cases handled | ✅ PASS | 400/409/422 all returned as documented |
|
|
46
|
+
| SC-NONGO | non-go respected | ❌ FAIL | modified packages/shared/auth — out of scope |
|
|
47
|
+
| SC-LAYER | no layer leak | ✅ PASS | — |
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### 2b. Verdict stability (verdict-ledger.md)
|
|
51
|
+
After the dimension tables. Reports re-probe/confidence outcomes and any cross-run flips read from
|
|
52
|
+
`.verdicts-<task_id>.jsonl`. On the first run, state there is no history yet (the ledger is still
|
|
53
|
+
written, as the baseline).
|
|
54
|
+
```
|
|
55
|
+
## Verdict stability (run 3)
|
|
56
|
+
- ⚠ AC4 — FLIP PASS→FAIL vs run 2, no code change to checkout → judge non-deterministic here; confidence low. Re-run before trusting.
|
|
57
|
+
- AC2 — stable FAIL across runs 1–3 (confidence high).
|
|
58
|
+
- Stability: 6/7 criteria stable this round.
|
|
59
|
+
```
|
|
60
|
+
The per-dimension tables (§2) carry a `confidence` column when this is active.
|
|
61
|
+
|
|
62
|
+
### 3. Bug list
|
|
63
|
+
One entry per FAIL, using the failing dimension's bug template. This is the generator's
|
|
64
|
+
worklist.
|
|
65
|
+
```
|
|
66
|
+
## Bugs
|
|
67
|
+
### BUG-1 — critical
|
|
68
|
+
criterion: SC-AC / SC-DONE-WHEN
|
|
69
|
+
location: apps/web/checkout/Pay.tsx:84
|
|
70
|
+
repro: navigate /checkout → fill card → click "Pay"
|
|
71
|
+
expected: POST /payments fires, redirect to /success (Done when: "User pays successfully")
|
|
72
|
+
actual: console TypeError: onPay is not a function; no network call; stays on /checkout
|
|
73
|
+
fix_hint: onPay prop not passed from CheckoutPage; wire handler → Pay button
|
|
74
|
+
|
|
75
|
+
### BUG-2 — minor
|
|
76
|
+
criterion: SC-NONGO
|
|
77
|
+
location: apps/api/shared/auth.ts:12
|
|
78
|
+
repro: git diff --stat
|
|
79
|
+
expected: task touches apps/web only (## Non-go: do not change auth)
|
|
80
|
+
actual: auth.ts modified (added a field); revert or move to a separate task
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### 4. Next action
|
|
84
|
+
A single explicit handoff line the orchestrator/generator acts on.
|
|
85
|
+
```
|
|
86
|
+
## Next action
|
|
87
|
+
→ Generator: re-run `task-executor --spec <path> --task TASK-NNN` to fix BUG-1, BUG-2,
|
|
88
|
+
then re-run `spec-evaluator` on the same task. Do not close until verdict: pass.
|
|
89
|
+
```
|
|
90
|
+
For a PASS:
|
|
91
|
+
```
|
|
92
|
+
## Next action
|
|
93
|
+
→ TASK-NNN is verified against [spec-conformance]. Safe for the orchestrator to close.
|
|
94
|
+
(Note: security, performance were NOT evaluated this run.)
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Rules
|
|
98
|
+
- The report never sets `status: done` on the task — it sets `eval_verdict` only.
|
|
99
|
+
- A PASS report still names the dimensions that were *not* run, so "verified" is never read
|
|
100
|
+
as "verified for everything."
|
|
101
|
+
- Every bug is self-contained: location + repro + expected/actual, no external context needed.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Verdict Ledger — re-probe, confidence, and flip detection
|
|
2
|
+
|
|
3
|
+
Read this at GATE V2 (alongside `anti-leniency.md`) and act on it at Phase B. It exists because
|
|
4
|
+
the judge is the harness's least verifiable component: a single non-deterministic snapshot per
|
|
5
|
+
criterion, no second opinion, no record of whether last round's PASS would still pass. This turns
|
|
6
|
+
that blind oracle into a **measurable** one — without adding a second judge (the single-judge
|
|
7
|
+
invariant is untouched). Three mechanisms: **re-probe on FAIL**, **per-criterion confidence**, and
|
|
8
|
+
an append-only **verdict ledger** that flags when a criterion's verdict flips across runs.
|
|
9
|
+
|
|
10
|
+
Pure-worker note (v1.0): you READ the JSONL ledger with your normal file tools to detect flips,
|
|
11
|
+
but you never append it yourself — the new lines return in your WorkResult's
|
|
12
|
+
`verdict.criteria[]` and the orchestrator's ingest script performs the append. Wherever this
|
|
13
|
+
file says "write/append the ledger" or "annotate the task file", read it as "return the data".
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Re-probe on FAIL (within a run)
|
|
18
|
+
|
|
19
|
+
Before you finalize a `FAIL` at GATE V2, run that criterion's probe **once more** (same probe,
|
|
20
|
+
fresh invocation). This costs one extra probe per failing criterion and catches the two ways a
|
|
21
|
+
single snapshot lies:
|
|
22
|
+
|
|
23
|
+
- The two probes **agree** (both FAIL) → the failure is real. `confidence: high`, `reprobed: true`.
|
|
24
|
+
- The two probes **disagree** (one PASS, one FAIL) → the signal is *flaky*, not a clean defect.
|
|
25
|
+
Record `verdict: FAIL` still (absence of a stable pass = FAIL), but `confidence: low`,
|
|
26
|
+
`reprobed: true`, and say so in the evidence ("probe non-deterministic: 1 PASS / 1 FAIL across
|
|
27
|
+
2 runs — flaky or environment-dependent"). A flaky criterion is itself a finding the generator
|
|
28
|
+
must stabilize, not a clean pass.
|
|
29
|
+
|
|
30
|
+
Re-probe only FAILs (and any PASS you have specific reason to doubt) — re-probing every PASS
|
|
31
|
+
doubles cost for little signal. Deterministic oracles (`process`/`test`/`data`/`[cmd]`) rarely
|
|
32
|
+
flip; `[ui]` snapshots flip most, so weight re-probing toward them.
|
|
33
|
+
|
|
34
|
+
## 2. Per-criterion confidence
|
|
35
|
+
|
|
36
|
+
Every criterion verdict carries a confidence, assigned by this rule (do not free-form it):
|
|
37
|
+
|
|
38
|
+
| confidence | when |
|
|
39
|
+
|---|---|
|
|
40
|
+
| `high` | deterministic oracle (`process`/`test`/`[cmd]`/`[data]`) **or** a re-probe that agreed |
|
|
41
|
+
| `medium` | a single deterministic probe you did not re-probe (most PASSes) |
|
|
42
|
+
| `low` | re-probe disagreed (flaky), **or** a `[ui]`/`snapshot` read with any ambiguity, **or** the verdict flipped vs the prior run (§3 sets this for you) |
|
|
43
|
+
|
|
44
|
+
Confidence is reported, never overrides the verdict: a `low`-confidence FAIL is still a FAIL. It
|
|
45
|
+
tells the PO and the generator *how much to trust this row* and where re-running would help.
|
|
46
|
+
|
|
47
|
+
## 3. The verdict ledger (across runs)
|
|
48
|
+
|
|
49
|
+
**File:** `.shapeup/<slug>/evaluation/.verdicts-<task_id>.jsonl` — append-only, one JSON
|
|
50
|
+
object per criterion per run. Never rewrite prior lines; the history is the point.
|
|
51
|
+
|
|
52
|
+
```json
|
|
53
|
+
{"run":1,"task":"TASK-007","dimension":"spec-conformance","criterion":"AC4","verdict":"FAIL","confidence":"high","reprobed":true,"flip":false,"evidence":"Pay click throws — Pay.tsx:84","at":"2026-06-27T10:00:00Z"}
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Required keys: `run` (1-based, increment per evaluator invocation on this task), `task`,
|
|
57
|
+
`dimension`, `criterion`, `verdict` (`PASS|FAIL`), `confidence`, `reprobed` (bool), `flip` (bool),
|
|
58
|
+
`evidence`, `at` (ISO-8601).
|
|
59
|
+
|
|
60
|
+
**Procedure at Phase B (before writing the report):**
|
|
61
|
+
1. Read the existing ledger if present. Determine this run's number = (max prior `run`) + 1, else 1.
|
|
62
|
+
2. For each criterion you graded this run, find its **most recent prior line** (same `criterion` +
|
|
63
|
+
`dimension`). If one exists and `prior.verdict !== this.verdict`, set `flip: true` and force
|
|
64
|
+
`confidence: low` (a flip means the oracle is unstable for this row, whatever you thought).
|
|
65
|
+
Otherwise `flip: false`.
|
|
66
|
+
3. Append one line per criterion.
|
|
67
|
+
4. Surface every `flip: true` in the report (§4) — a flip is a calibration finding, not noise.
|
|
68
|
+
|
|
69
|
+
A flip is **PASS→FAIL** or **FAIL→PASS** for the same criterion across runs with no intervening
|
|
70
|
+
code change to that area. PASS→FAIL after the generator fixed an unrelated bug, or FAIL→PASS after
|
|
71
|
+
a targeted fix, is *expected* — note the cause. An *unexplained* flip means the judge is
|
|
72
|
+
non-deterministic on that row and the PO should not trust a single verdict there.
|
|
73
|
+
|
|
74
|
+
## 4. Reporting
|
|
75
|
+
|
|
76
|
+
Add to the EVAL report (see `report-schema.md`), after the per-dimension tables:
|
|
77
|
+
|
|
78
|
+
```
|
|
79
|
+
## Verdict stability (run 3)
|
|
80
|
+
- ⚠ AC4 — FLIP PASS→FAIL vs run 2, no code change to checkout → judge non-deterministic here; confidence low. Re-run before trusting.
|
|
81
|
+
- AC2 — stable FAIL across runs 1–3 (confidence high).
|
|
82
|
+
- Stability: 6/7 criteria stable this round.
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
If there are no prior runs, print `Verdict stability: first run — no history yet.` and still write
|
|
86
|
+
the ledger so the next run has a baseline.
|
|
87
|
+
|
|
88
|
+
## Why this stays single-judge
|
|
89
|
+
|
|
90
|
+
Nothing here adds a second grader or a knowledge base that could become a covert judge. Re-probe is
|
|
91
|
+
the *same* judge running the *same* probe twice; confidence and flips are bookkeeping over that one
|
|
92
|
+
judge's own outputs. The verdict authority is unchanged — this only makes its reliability visible.
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Verdict-ledger reference implementation (audit Stage D1) — spec-evaluator's own skill-local
|
|
3
|
+
// reference impl (ships beside SKILL.md; not invoked at runtime).
|
|
4
|
+
//
|
|
5
|
+
// The `spec-evaluator` skill performs verdict re-probe / confidence / flip-detection as a
|
|
6
|
+
// self-contained procedure (`references/verdict-ledger.md`) using its own file tools — it does NOT
|
|
7
|
+
// call this file. This is the executable proof that the documented grammar is well-defined and
|
|
8
|
+
// actually discriminates an unstable judge from a stable one, the same way oracles/*
|
|
9
|
+
// prove the probing grammar. Structural test #15 exercises it.
|
|
10
|
+
//
|
|
11
|
+
// Ledger line shape (one per criterion per run), see verdict-ledger.md:
|
|
12
|
+
// { run, task, dimension, criterion, verdict:"PASS"|"FAIL", confidence, reprobed, flip, evidence, at }
|
|
13
|
+
//
|
|
14
|
+
// Library use:
|
|
15
|
+
// import { reconcile, detectFlips, stability } from "skills/spec-evaluator/scripts/verdict-ledger.mjs";
|
|
16
|
+
// const { records, summary } = reconcile(priorLines, currentRecords);
|
|
17
|
+
|
|
18
|
+
// Most recent prior line for a (dimension, criterion), by highest run number.
|
|
19
|
+
import { isMain } from "../../tech-lead/scripts/lib/is-main.mjs";
|
|
20
|
+
import { runArgs } from "../../tech-lead/scripts/lib/argv.mjs";
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* Find the most recent prior ledger line for a record's (dimension, criterion), by highest run.
|
|
24
|
+
* @param {Array<{dimension:string, criterion:string, run:number, verdict:string}>} priorLines - Prior lines.
|
|
25
|
+
* @param {{dimension:string, criterion:string}} rec - The record to match.
|
|
26
|
+
* @returns {(object|null)} The highest-run matching prior line, or null when none exists.
|
|
27
|
+
*/
|
|
28
|
+
function priorFor(priorLines, rec) {
|
|
29
|
+
let best = null;
|
|
30
|
+
for (const p of priorLines) {
|
|
31
|
+
if (p.criterion !== rec.criterion || p.dimension !== rec.dimension) continue;
|
|
32
|
+
if (best === null || p.run > best.run) best = p;
|
|
33
|
+
}
|
|
34
|
+
return best;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
// Reconcile this run's records against the prior ledger: a verdict change vs the most recent prior
|
|
38
|
+
// line for the same criterion sets flip=true and FORCES confidence="low" (a flip means the oracle
|
|
39
|
+
// is unstable on that row, regardless of what confidence the judge proposed). Returns augmented
|
|
40
|
+
// records + a summary. Pure: no I/O, no clock (caller stamps `at`).
|
|
41
|
+
/**
|
|
42
|
+
* Reconcile this run's records against the prior ledger — a verdict change vs the most recent
|
|
43
|
+
* prior line for the same criterion sets flip=true and FORCES confidence="low". Pure: no I/O, no clock.
|
|
44
|
+
* @param {Array<object>} priorLines - Previously recorded ledger lines.
|
|
45
|
+
* @param {Array<{criterion:string, dimension:string, verdict:string, confidence:string}>}
|
|
46
|
+
* currentRecords - This run's per-criterion records.
|
|
47
|
+
* @returns {{records:Array<object>, summary:{total:number, flipped:number, stable:number,
|
|
48
|
+
* flips:Array<{criterion:string, dimension:string, to:string}>}}} The records augmented with
|
|
49
|
+
* `flip` (+ forced-low confidence) and a run summary.
|
|
50
|
+
*/
|
|
51
|
+
export function reconcile(priorLines, currentRecords) {
|
|
52
|
+
const records = currentRecords.map((rec) => {
|
|
53
|
+
const prior = priorFor(priorLines, rec);
|
|
54
|
+
const flip = !!prior && prior.verdict !== rec.verdict;
|
|
55
|
+
return { ...rec, flip, confidence: flip ? "low" : rec.confidence };
|
|
56
|
+
});
|
|
57
|
+
const flipped = records.filter((r) => r.flip);
|
|
58
|
+
return {
|
|
59
|
+
records,
|
|
60
|
+
summary: {
|
|
61
|
+
total: records.length,
|
|
62
|
+
flipped: flipped.length,
|
|
63
|
+
stable: records.length - flipped.length,
|
|
64
|
+
flips: flipped.map((r) => ({ criterion: r.criterion, dimension: r.dimension, to: r.verdict })),
|
|
65
|
+
},
|
|
66
|
+
};
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
// Walk the full ledger in run order; return every criterion whose verdict changed at any step,
|
|
70
|
+
// with the run pair and direction. Used for the report's stability block / audit of a whole task.
|
|
71
|
+
/**
|
|
72
|
+
* Walk the full ledger in run order and report every verdict change.
|
|
73
|
+
* @param {Array<{dimension:string, criterion:string, run:number, verdict:string}>} allLines - Every ledger line.
|
|
74
|
+
* @returns {Array<{criterion:string, dimension:string, from:string, to:string, runs:[number,number]}>}
|
|
75
|
+
* One entry per step where a criterion's verdict changed.
|
|
76
|
+
*/
|
|
77
|
+
export function detectFlips(allLines) {
|
|
78
|
+
const byKey = new Map();
|
|
79
|
+
for (const l of [...allLines].sort((a, b) => a.run - b.run)) {
|
|
80
|
+
const key = `${l.dimension}::${l.criterion}`;
|
|
81
|
+
const seq = byKey.get(key) || [];
|
|
82
|
+
seq.push(l);
|
|
83
|
+
byKey.set(key, seq);
|
|
84
|
+
}
|
|
85
|
+
const flips = [];
|
|
86
|
+
for (const [key, seq] of byKey) {
|
|
87
|
+
for (let i = 1; i < seq.length; i++) {
|
|
88
|
+
if (seq[i].verdict !== seq[i - 1].verdict) {
|
|
89
|
+
flips.push({
|
|
90
|
+
criterion: seq[i].criterion,
|
|
91
|
+
dimension: seq[i].dimension,
|
|
92
|
+
from: seq[i - 1].verdict,
|
|
93
|
+
to: seq[i].verdict,
|
|
94
|
+
runs: [seq[i - 1].run, seq[i].run],
|
|
95
|
+
});
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
}
|
|
99
|
+
return flips;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
// Stability of the latest run vs the run before it: fraction of the latest run's criteria whose
|
|
103
|
+
// verdict matches their immediately-prior line. 1.0 when there is only one run (nothing to contradict).
|
|
104
|
+
/**
|
|
105
|
+
* Measure stability of the latest run vs the run before it.
|
|
106
|
+
* @param {Array<{run:number, dimension:string, criterion:string, verdict:string}>} allLines - Every ledger line.
|
|
107
|
+
* @returns {{runs:number, stable:number, total:number, ratio:number}} The max run number, how many
|
|
108
|
+
* of the latest run's criteria match their prior verdict (new criteria count as stable), the
|
|
109
|
+
* latest run's criterion count, and their ratio (1.0 when there is only one run).
|
|
110
|
+
*/
|
|
111
|
+
export function stability(allLines) {
|
|
112
|
+
if (allLines.length === 0) return { runs: 0, stable: 0, total: 0, ratio: 1 };
|
|
113
|
+
const maxRun = Math.max(...allLines.map((l) => l.run));
|
|
114
|
+
const latest = allLines.filter((l) => l.run === maxRun);
|
|
115
|
+
if (maxRun === 1) return { runs: 1, stable: latest.length, total: latest.length, ratio: 1 };
|
|
116
|
+
const prior = allLines.filter((l) => l.run < maxRun);
|
|
117
|
+
let stable = 0;
|
|
118
|
+
for (const rec of latest) {
|
|
119
|
+
const p = priorFor(prior, rec);
|
|
120
|
+
if (p && p.verdict === rec.verdict) stable++;
|
|
121
|
+
else if (!p) stable++; // a brand-new criterion can't have flipped
|
|
122
|
+
}
|
|
123
|
+
return { runs: maxRun, stable, total: latest.length, ratio: latest.length ? stable / latest.length : 1 };
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
// Parse a .jsonl ledger string into records (blank lines ignored).
|
|
127
|
+
/**
|
|
128
|
+
* Parse a `.jsonl` verdict ledger into records.
|
|
129
|
+
* @param {string} text - The ledger file contents ("" / null → []).
|
|
130
|
+
* @returns {Array<object>} One parsed object per non-blank line.
|
|
131
|
+
* @throws {SyntaxError} If a non-blank line is not valid JSON.
|
|
132
|
+
*/
|
|
133
|
+
export function parseLedger(text) {
|
|
134
|
+
return (text || "")
|
|
135
|
+
.split(/\r?\n/)
|
|
136
|
+
.map((l) => l.trim())
|
|
137
|
+
.filter(Boolean)
|
|
138
|
+
.map((l) => JSON.parse(l));
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
// --- CLI entry: summarize a ledger file -------------------------------------
|
|
142
|
+
/** The typed argv contract (see `skills/tech-lead/scripts/lib/argv.mjs`). */
|
|
143
|
+
export const ARGV_SPEC = {
|
|
144
|
+
usage: "verdict-ledger.mjs <.verdicts-TASK.jsonl>",
|
|
145
|
+
_: { arity: 1, max: 1, name: ".verdicts-TASK.jsonl" },
|
|
146
|
+
};
|
|
147
|
+
|
|
148
|
+
if (isMain(import.meta.url)) {
|
|
149
|
+
const { readFileSync } = await import("node:fs");
|
|
150
|
+
const path = runArgs(ARGV_SPEC)._[0];
|
|
151
|
+
let lines;
|
|
152
|
+
try { lines = parseLedger(readFileSync(path, "utf8")); }
|
|
153
|
+
catch (e) { console.error(`cannot read ledger ${path}: ${e.message}`); process.exit(2); }
|
|
154
|
+
const flips = detectFlips(lines);
|
|
155
|
+
const s = stability(lines);
|
|
156
|
+
console.log(`ledger: ${lines.length} records over ${s.runs} run(s)`);
|
|
157
|
+
console.log(`stability (latest vs prior): ${s.stable}/${s.total} stable`);
|
|
158
|
+
if (flips.length) {
|
|
159
|
+
console.log(`⚠ ${flips.length} flip(s):`);
|
|
160
|
+
for (const f of flips) console.log(` ${f.dimension}/${f.criterion}: ${f.from}→${f.to} (runs ${f.runs.join("→")})`);
|
|
161
|
+
process.exit(1);
|
|
162
|
+
} else {
|
|
163
|
+
console.log("✅ no verdict flips");
|
|
164
|
+
process.exit(0);
|
|
165
|
+
}
|
|
166
|
+
}
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: task-executor
|
|
3
|
+
description: "Use this skill whenever a user wants to execute, implement, or run a specific task generated by the ba-pitch-analyzer skill. Triggers on: \"execute task TASK-NNN\", \"implement TASK-NNN\", \"run this task\", \"start working on TASK-NNN\", \"implement the task in [folder]\", \"run task from spec folder\", \"pick up the next task\", or a tech-lead --order dispatch. Implements a task's acceptance criteria exactly — minimum code, surgical diffs, verified observable outcomes."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Task Executor (pure worker, v2.0)
|
|
7
|
+
|
|
8
|
+
**Implement exactly what the acceptance criteria specify. Prove it. Report it.**
|
|
9
|
+
|
|
10
|
+
You are the *doer* in a planner → doer → judge harness. Your entire interface with the
|
|
11
|
+
pipeline is two JSON envelopes: a **WorkOrder** in, a **WorkResult** out. You know nothing
|
|
12
|
+
about boards, ledgers, run-state, rounds, or other workers — if the order doesn't carry it,
|
|
13
|
+
it does not exist for you.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Input contract — the WorkOrder
|
|
18
|
+
|
|
19
|
+
You are invoked as `--order <path>` pointing at a schema-valid WorkOrder. Fields you may
|
|
20
|
+
rely on (anything absent = **unknown**; never invent it):
|
|
21
|
+
|
|
22
|
+
| Field | What it is |
|
|
23
|
+
|---|---|
|
|
24
|
+
| `payload.tasks[]` | The task(s) to implement: `id`, `body_path` (read it fully), `acceptance_criteria[]` |
|
|
25
|
+
| `payload.scope_contract` | The active scope: `affordance_manifest`, `e2e_verification_fixtures`, topology |
|
|
26
|
+
| `substrate.allowed` / `substrate.shared` | The ONLY globs you may write. A needed file outside them → ESCALATE, never a write (a sandbox hook blocks it anyway) |
|
|
27
|
+
| `payload.decisions[]` | Adjudicated answers from prior escalations — binding precedent, apply them |
|
|
28
|
+
| `payload.digested_errors[]` | `{file, line, core_message}` triples from the previous attempt's failed verification — your starting bug list |
|
|
29
|
+
| `payload.verify.test_cmd` | The command that verifies your work. No test_cmd → command-verifiable ACs still need *some* observable check; say what you used |
|
|
30
|
+
| `payload.kb_rules_path` | Team guidelines (read if the file exists) — steering, never spec; conflict → the AC wins, note it in `deviations` |
|
|
31
|
+
| `payload.constraints` | Non-Go items and freezes (e.g. `ui_layers.layer3_frozen`) |
|
|
32
|
+
| `operation` | `execute` (fresh), `fix` (only the bugs in `payload.bugs` — touch nothing else), `spike` (produce a decision doc, not code) |
|
|
33
|
+
|
|
34
|
+
**Zero-memory rule.** You have no memory of prior attempts or sessions. Decisions that
|
|
35
|
+
mattered are in `payload.decisions`; errors that mattered are in `payload.digested_errors`.
|
|
36
|
+
Never ask anyone to fill in what a prior attempt "already established" — if it isn't in the
|
|
37
|
+
order, treat it as unknown: proceed on the spec alone or ESCALATE.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Core process
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
PLAN read every task body + linked spec docs from disk (never from memory)
|
|
45
|
+
→ surface EVERY assumption + one observable success criterion per AC
|
|
46
|
+
IMPLEMENT for each AC, one at a time:
|
|
47
|
+
1. state the minimum code needed (1 line)
|
|
48
|
+
2. write it — inside the substrate only
|
|
49
|
+
3. verify the OBSERVABLE outcome (run the command / drive the behavior)
|
|
50
|
+
4. record evidence, next AC
|
|
51
|
+
REPORT write the WorkResult envelope — the single thing the pipeline reads back
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
**PLAN — assumptions surfaced before code.** For each AC note any decision the spec leaves
|
|
55
|
+
open (naming, error format, validation order, defaults, edge handling). Two equally valid
|
|
56
|
+
interpretations = a genuine ambiguity: ESCALATE it (orchestrated) or ask, max 2 questions
|
|
57
|
+
(standalone). Everything you resolve yourself goes in `assumptions[]` — none stay silent.
|
|
58
|
+
Define per AC what success *observably* looks like, e.g.
|
|
59
|
+
`POST /orders {amount:-1} → 422 {error:'amount must be positive'}` — not just the AC text.
|
|
60
|
+
|
|
61
|
+
**IMPLEMENT — craft rules (this IS the skill):**
|
|
62
|
+
|
|
63
|
+
- **Minimum code.** Before each AC: "Minimum code needed: [what, ~lines]". If a simpler
|
|
64
|
+
alternative exists, name it and build the simpler one unless the contract/AC demands the
|
|
65
|
+
complex one. Senior-engineer test: would a seasoned engineer call this overcomplicated?
|
|
66
|
+
- **Surgical.** Touch only files the AC requires. Match the file's existing style. Remove
|
|
67
|
+
symbols your change made unused. Note adjacent code deliberately NOT touched. Unrelated
|
|
68
|
+
dead code → a `discoveries[]` entry, never a deletion.
|
|
69
|
+
- **Contract reference.** Every repository implementation references its contract file —
|
|
70
|
+
never redefine Request/Response/Error types inline.
|
|
71
|
+
- **Non-Go stop.** The moment an implementation starts touching a `constraints.non_go` item,
|
|
72
|
+
stop that path and report it — scope expansion is the caller's decision, not yours.
|
|
73
|
+
- **UI Layer 1 — affordance contract.** Every interactive element binds to the
|
|
74
|
+
`test_id`/`role` pairs in `scope_contract.affordance_manifest` (semantic HTML,
|
|
75
|
+
`data-testid`) and expresses `idle|loading|success|error|empty` via `data-state`. An
|
|
76
|
+
element the manifest lacks = spec gap → ESCALATE (`spec-ambiguity`); never extend the
|
|
77
|
+
manifest yourself.
|
|
78
|
+
- **UI Layer 2 — real data only.** Bind every element to the actual API/DB call this task
|
|
79
|
+
implements. **Hardcoded data arrays are banned** — a `const items = [...]` standing in for
|
|
80
|
+
a fetch is exactly what the T0 DB probe exists to catch.
|
|
81
|
+
- **UI Layer 3 — frozen.** No pixel/CSS polish beyond semantic HTML defaults. An AC that
|
|
82
|
+
demands visual polish is out of policy — flag it, don't build it.
|
|
83
|
+
- **SPIKE orders** produce a decision document (question, findings with `file:line`
|
|
84
|
+
citations, decision, residual unknowns) listed in `artifacts[]` — not production code.
|
|
85
|
+
|
|
86
|
+
**Verification is against the observable criterion, not exit codes.** Run the check; compare
|
|
87
|
+
what happened to what PLAN said success looks like. Evidence or it didn't happen — an AC with
|
|
88
|
+
no recorded evidence is reported `fail`, by your own hand.
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## Anti-rationalization table
|
|
93
|
+
|
|
94
|
+
| Excuse | Reality |
|
|
95
|
+
|---|---|
|
|
96
|
+
| "The AC is obviously satisfied, skip the test run" | Run it. Evidence or it didn't happen. |
|
|
97
|
+
| "This helper will be needed later" | Speculative code is scope creep. Minimum code only. |
|
|
98
|
+
| "The hardcoded array is temporary" | Layer-2 violation; the DB probe exists for you. Wire the real call. |
|
|
99
|
+
| "I remember what the last attempt decided" | You have no memory. If it's not in the order, ESCALATE. |
|
|
100
|
+
| "This file is just outside the substrate, it's clearly needed" | That's what substrate-expansion ESCALATE is for. The hook will deny the write anyway. |
|
|
101
|
+
| "The adjacent code is ugly, I'll clean it up while I'm here" | Surgical means surgical. Report it as a discovery. |
|
|
102
|
+
| "The test fails for an unrelated reason, so my AC still passes" | A red check is a red check. Report `fail` with the evidence; the digested error feeds the next attempt. |
|
|
103
|
+
| "Asking would slow things down, I'll just pick one" | A silent wrong guess costs a full round. Surface it — assumption or ESCALATE. |
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## ESCALATE protocol — the one outward port
|
|
108
|
+
|
|
109
|
+
When blocked on a decision that is not yours (design decision, spec ambiguity, substrate
|
|
110
|
+
expansion): stop work on **that AC only**, keep building every AC that doesn't depend on the
|
|
111
|
+
answer, and add a block to `escalates[]`:
|
|
112
|
+
|
|
113
|
+
```json
|
|
114
|
+
{ "kind": "design-decision | spec-ambiguity | substrate-expansion",
|
|
115
|
+
"question": "one checkable question",
|
|
116
|
+
"blocked_ac": "which AC waits on this",
|
|
117
|
+
"context": "the two interpretations / the file and why it's needed" }
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Orchestrated (`mode: orchestrated`): never ask an ad hoc question — there is no session to
|
|
121
|
+
answer it; the orchestrator adjudicates and the answer returns in the next order's
|
|
122
|
+
`decisions[]`. Standalone (`mode: standalone`): you may ask the user directly (max 2
|
|
123
|
+
questions) instead of escalating.
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## Output contract — the WorkResult
|
|
128
|
+
|
|
129
|
+
Write `.shapeup/<slug>/results/<order-suffix>.json` (mirror of the order path; slug and
|
|
130
|
+
suffix come from `order_id`) matching `work-result.schema.json`, and print its path:
|
|
131
|
+
|
|
132
|
+
```json
|
|
133
|
+
{
|
|
134
|
+
"schema_version": 1,
|
|
135
|
+
"order_id": "<copied from the order>",
|
|
136
|
+
"worker": "task-executor",
|
|
137
|
+
"status": "done | partial | escalated | failed",
|
|
138
|
+
"task_results": [
|
|
139
|
+
{ "task_id": "TASK-003", "status": "done",
|
|
140
|
+
"ac_results": [
|
|
141
|
+
{ "ac": "<the AC checkbox text, verbatim>", "result": "pass",
|
|
142
|
+
"evidence": "pnpm --filter api test → 14/14" } ] }
|
|
143
|
+
],
|
|
144
|
+
"files_touched": [ { "path": "apps/api/src/orders/create.ts", "change": "created" } ],
|
|
145
|
+
"escalates": [],
|
|
146
|
+
"discoveries": [ { "marker": "+", "line": "empty-cart edge case unhandled" } ],
|
|
147
|
+
"assumptions": [ "amount stored in minor units — inferred from schema" ],
|
|
148
|
+
"deviations": []
|
|
149
|
+
}
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
`ac` carries the checkbox text **verbatim** — the ingest script ticks boxes by matching it.
|
|
153
|
+
Failing/skipped ACs are reported as such, never omitted. Discovered work is *reported*, never
|
|
154
|
+
self-planned: no new task files, no spec edits — one raw line per discovery.
|
|
155
|
+
|
|
156
|
+
You do NOT: tick AC boxes, edit `tasks/_index.md`, write `run-state.md`, touch the discovery
|
|
157
|
+
ledger, mark anything done outside your result, or update any other spec document. The
|
|
158
|
+
orchestrator's `ingest-result.mjs` does all of that from your envelope.
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## Verification checklist — a task is not complete until this passes
|
|
163
|
+
|
|
164
|
+
- [ ] Every AC has an `ac_results` entry with concrete evidence (or an honest `fail`/`skipped`)
|
|
165
|
+
- [ ] Every file written matches a `substrate.allowed`/`substrate.shared` glob
|
|
166
|
+
- [ ] No hardcoded data arrays standing in for real API/DB calls
|
|
167
|
+
- [ ] Every interactive element built binds to a manifest `test_id` with `data-state`
|
|
168
|
+
- [ ] No unused imports/symbols left behind by your changes
|
|
169
|
+
- [ ] Assumptions and deviations are in the envelope, not in your head
|
|
170
|
+
- [ ] Blocked ACs have an `escalates[]` block; unrelated ACs were still built
|
|
171
|
+
- [ ] The WorkResult validates against `work-result.schema.json` and its path was printed
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Invocation
|
|
176
|
+
|
|
177
|
+
```bash
|
|
178
|
+
# Orchestrated (tech-lead's build loop) — the canonical form
|
|
179
|
+
/task-executor --order .shapeup/checkout-vnpay/orders/r2-a3.json
|
|
180
|
+
|
|
181
|
+
# Standalone — the preamble shim compiles a minimal WorkOrder from the flags, then the
|
|
182
|
+
# single code path above runs. Requires the harness scripts (plugin install):
|
|
183
|
+
# node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/compile-order.mjs" --task TASK-003 --slug checkout-vnpay
|
|
184
|
+
# node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/compile-order.mjs" --next --slug checkout-vnpay
|
|
185
|
+
/task-executor --spec shapeup/checkout-vnpay/spec/ --task TASK-003
|
|
186
|
+
/task-executor --spec shapeup/checkout-vnpay/spec/ --next
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Standalone shim: derive `<slug>` from the `--spec` path (`shapeup/<slug>/spec`),
|
|
190
|
+
run `compile-order.mjs` with the matching flags (mode becomes `standalone`), then proceed
|
|
191
|
+
against the compiled order exactly as if dispatched. After writing the WorkResult, run
|
|
192
|
+
`node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/ingest-result.mjs" <result path>` yourself and show the user its
|
|
193
|
+
summary — standalone has no orchestrator to ingest for you. One code path inside; two entry
|
|
194
|
+
points outside.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# tech-lead
|
|
2
|
+
|
|
3
|
+
The orchestrator over the harness. Acts as the tech lead that runs a feature end-to-end
|
|
4
|
+
across three skills, makes round decisions, and reports to the PO at gates. It is **thin** —
|
|
5
|
+
it sequences and decides, it does not plan, build, or judge itself.
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
ba-pitch-analyzer (planner) → task-executor (generator) → spec-evaluator (judge)
|
|
9
|
+
PLAN BUILD (loop all tasks) EVALUATE (once / round)
|
|
10
|
+
▲ │
|
|
11
|
+
└──── bugs, round r+1 ◄────────┘ on FAIL
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
**Core rule:** the evaluator runs exactly once per build round, only after the task board
|
|
15
|
+
is 100% done — never per task. That single end-of-round QA pass is the long-running harness
|
|
16
|
+
V2 lesson, and enforcing its timing is the reason this skill exists.
|
|
17
|
+
|
|
18
|
+
## Resource map
|
|
19
|
+
```
|
|
20
|
+
tech-lead/
|
|
21
|
+
├── SKILL.md # entry — GATE L0–L4, the round loop, flags, hard rules
|
|
22
|
+
├── README.md # this file
|
|
23
|
+
└── references/
|
|
24
|
+
├── round-protocol.md # ★ the loop: r=1 vs r>1, stop conditions, eval timing rule
|
|
25
|
+
├── delegation.md # how each of the 3 sub-skills is invoked + handoff files
|
|
26
|
+
└── ledger-schema.md # harness-run.md — the round table / decisions / escalation
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
| Resource | Loaded at | Purpose |
|
|
30
|
+
|----------|-----------|---------|
|
|
31
|
+
| `SKILL.md` | always | L-gates, phase sequence, invocation, authority rules |
|
|
32
|
+
| `references/round-protocol.md` | BUILD/EVAL | the loop semantics + the "eval once at end" rule |
|
|
33
|
+
| `references/delegation.md` | each phase | exact sub-skill commands + which handoff files to read |
|
|
34
|
+
| `references/ledger-schema.md` | throughout | the run ledger that carries state across rounds/sessions |
|
|
35
|
+
|
|
36
|
+
## Install
|
|
37
|
+
```bash
|
|
38
|
+
cp -r tech-lead <repo>/.claude/skills/ # or ~/.claude/skills/
|
|
39
|
+
```
|
|
40
|
+
Requires the harness skills installed: `ba-pitch-analyzer`, `task-executor`,
|
|
41
|
+
`spec-evaluator`, plus `translator` (the GATE L0 language gate for non-English intake).
|
|
42
|
+
The EVAL phase uses spec-evaluator's feature-level pass
|
|
43
|
+
(`--feature <slug>`) — see the dependency note in `references/delegation.md`.
|
|
44
|
+
|
|
45
|
+
## Invoke
|
|
46
|
+
```bash
|
|
47
|
+
/tech-lead --pitch shapeup/checkout/shaping/shaping.md --spec shapeup/checkout/spec/ --lens standard
|
|
48
|
+
/tech-lead --pitch ... --spec ... --auto # sub-skills unattended; pause at L1/L3/L4
|
|
49
|
+
/tech-lead --pitch ... --spec ... --unattended --max-rounds 3 # headless / CI (Agent SDK)
|
|
50
|
+
/tech-lead --spec shapeup/checkout/spec/ --from build # resume an existing run
|
|
51
|
+
/tech-lead --pitch ... --spec ... --no-eval # skip eval for a trivial feature
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Gate map
|
|
55
|
+
| Gate | When | Decision |
|
|
56
|
+
|------|------|----------|
|
|
57
|
+
| L0 | intake | language gate (`/translator --check`, translate if non-English) + run config: spec folder, lens, dims, max_rounds, auto level |
|
|
58
|
+
| L1a | after ORIENT | PO reviews the spiked area + 🗻 Hill unknowns before mapping scopes |
|
|
59
|
+
| L1b | after MAP SCOPES | PO accepts the task board (appetite guard) before any code |
|
|
60
|
+
| L2 | after BUILD round | board 100% done? → the single EVAL pass (hook warns if not; advisory) |
|
|
61
|
+
| L3 | after EVAL | PASS → /qa-edge-hunter pass → ship; FAIL → bug-only round r+1; max_rounds → escalate |
|
|
62
|
+
| L4 | after SHIP | PO sign-off, close the ledger, harvest metrics to metrics.jsonl |
|
|
63
|
+
|
|
64
|
+
## Auto levels
|
|
65
|
+
- **interactive** (default): pause at every L-gate; sub-skills keep their own gates.
|
|
66
|
+
- **--auto**: sub-skills run unattended; tech lead still pauses at L1a / L1b / L3 / L4.
|
|
67
|
+
- **--unattended**: auto-confirm all L-gates; stop only on PASS, max_rounds, or hard error.
|
|
68
|
+
This is the headless mode for an Agent SDK / CI driver.
|
|
69
|
+
|
|
70
|
+
## Version
|
|
71
|
+
0.10 — Two-root workspace split (`shapeup/` vs `.shapeup/`), automated discovered-task reconciliation (`/ba-pitch-analyzer --tasks-only --from-discovered`), regression rule (touched UCs re-run), QA edge-hunter wiring (exploratory findings, triage at SHIP S.0), metrics harvest, and split L1a/L1b gates.
|