shapeup-sdlc 1.6.2 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/AGENTS.md +36 -103
- package/README.md +60 -67
- package/SECURITY.md +31 -16
- package/bin/init.mjs +24 -69
- package/commands/build.md +19 -0
- package/commands/eval.md +5 -4
- package/commands/scopes.md +5 -4
- package/commands/shape.md +1 -1
- package/commands/ship.md +37 -4
- package/hooks/anti-rationalization.mjs +7 -13
- package/hooks/compact-snapshot.mjs +5 -5
- package/hooks/gate-deadline.mjs +12 -16
- package/hooks/gate-intake.mjs +13 -14
- package/hooks/gate-l2.mjs +3 -3
- package/hooks/gate-zerowork.mjs +97 -16
- package/hooks/lib/decision.mjs +21 -13
- package/hooks/safety-spine.mjs +1 -1
- package/hooks/sandbox-guard.mjs +83 -47
- package/hooks/session-rehydrate.mjs +8 -9
- package/hooks/slop-cleaner.mjs +1 -1
- package/oracles/_shared.mjs +1 -1
- package/oracles/http-oracle.mjs +1 -1
- package/oracles/index.mjs +1 -1
- package/oracles/process-oracle.mjs +1 -1
- package/oracles/snapshot-oracle.mjs +1 -1
- package/oracles/test-oracle.mjs +1 -1
- package/package.json +3 -7
- package/skills/ba-pitch-analyzer/SKILL.md +20 -15
- package/skills/ba-pitch-analyzer/references/doc-schemas.md +1 -1
- package/skills/ba-pitch-analyzer/references/task-generation.md +6 -6
- package/skills/ba-pitch-analyzer/references/test-surface.md +1 -1
- package/skills/ba-pitch-analyzer/scripts/board-derive.mjs +5 -5
- package/skills/ba-pitch-analyzer/scripts/spec-lint.mjs +2 -2
- package/skills/coach/SKILL.md +3 -3
- package/skills/orient/SKILL.md +2 -1
- package/skills/qa-edge-hunter/SKILL.md +9 -39
- package/skills/scope-architect/SKILL.md +17 -12
- package/skills/scope-hammer/SKILL.md +5 -5
- package/skills/shapeup/SKILL.md +3 -3
- package/skills/shapeup/resources/context-compaction.md +4 -3
- package/skills/solution-architect/SKILL.md +18 -11
- package/skills/spec-evaluator/SKILL.md +12 -5
- package/skills/spec-evaluator/references/dimensions/_registry.md +2 -2
- package/skills/spec-evaluator/references/dimensions/completeness.md +1 -1
- package/skills/spec-evaluator/references/dimensions/integration.md +98 -67
- package/skills/spec-evaluator/references/dimensions/spec-conformance.md +4 -4
- package/skills/spec-evaluator/references/dimensions/test-surface-conformance.md +1 -1
- package/skills/spec-evaluator/references/probing.md +42 -23
- package/skills/spec-evaluator/references/report-schema.md +2 -2
- package/skills/spec-evaluator/scripts/verdict-ledger.mjs +1 -1
- package/skills/task-executor/SKILL.md +25 -12
- package/skills/tech-lead/SKILL.md +115 -438
- package/skills/tech-lead/references/delegation.md +38 -31
- package/skills/tech-lead/references/gates.md +75 -33
- package/skills/tech-lead/references/hard-rules.md +34 -0
- package/skills/tech-lead/references/invocation.md +2 -2
- package/skills/tech-lead/references/ledger-schema.md +17 -18
- package/skills/tech-lead/references/round-protocol.md +77 -27
- package/skills/tech-lead/references/state-model.md +1 -1
- package/skills/tech-lead/schemas/domain.schema.json +320 -148
- package/skills/tech-lead/schemas/gate-answers.schema.json +2 -2
- package/skills/tech-lead/schemas/work-result.schema.json +55 -17
- package/skills/tech-lead/scripts/aegis-digest.mjs +1 -1
- package/skills/tech-lead/scripts/budget-check.mjs +7 -13
- package/skills/tech-lead/scripts/compile-order.mjs +69 -22
- package/skills/tech-lead/scripts/fit-check.mjs +10 -10
- package/skills/tech-lead/scripts/gate-answers.mjs +11 -12
- package/skills/tech-lead/scripts/hill-derive.mjs +133 -0
- package/skills/tech-lead/scripts/ingest-result.mjs +7 -19
- package/skills/tech-lead/scripts/init-run.mjs +60 -18
- package/skills/tech-lead/scripts/lib/argv.mjs +1 -1
- package/skills/tech-lead/scripts/lib/contract-md.mjs +10 -10
- package/skills/tech-lead/scripts/lib/is-main.mjs +11 -12
- package/skills/tech-lead/scripts/lib/paths.mjs +15 -13
- package/skills/tech-lead/scripts/resume-state.mjs +387 -0
- package/skills/tech-lead/scripts/run-snapshot.mjs +1 -6
- package/skills/tech-lead/scripts/run-workflow.mjs +381 -0
- package/skills/tech-lead/scripts/ship-report.mjs +33 -2
- package/skills/tech-lead/scripts/stats.mjs +9 -10
- package/skills/tech-lead/scripts/t0-verify.mjs +4 -4
- package/skills/tech-lead/scripts/trace-lint.mjs +2 -2
- package/skills/tech-lead/workflows/shapeup-run.js +911 -0
- package/skills/translator/SKILL.md +2 -2
- package/skills/advisor-protocol/SKILL.md +0 -171
|
@@ -31,10 +31,10 @@ The WorkOrder carries everything the worker may rely on (payload, decisions, dig
|
|
|
31
31
|
substrate write-contract); the WorkResult carries everything the worker used to write into
|
|
32
32
|
shared files. `validate-envelope.mjs` runs as a PreToolUse hook on Skill|Agent and DENIES a
|
|
33
33
|
dispatch whose `--order` file is missing or schema-invalid. Workers write only their own
|
|
34
|
-
domain artifacts inside their substrate — never boards, ledgers, or run-state
|
|
34
|
+
domain artifacts inside their substrate — never boards, ledgers, or run-state.
|
|
35
35
|
|
|
36
36
|
Role → model, resolved once at GATE L0.8 from the `orch`/`exec`/`eval`/`qa` matrix
|
|
37
|
-
(`t0-verify.mjs` is mechanical tooling run directly via Bash, never an Agent —
|
|
37
|
+
(`t0-verify.mjs` is mechanical tooling run directly via Bash, never an Agent — zero LLM
|
|
38
38
|
tokens):
|
|
39
39
|
|
|
40
40
|
| Skill | L0.8 role | Why this tier |
|
|
@@ -44,7 +44,6 @@ tokens):
|
|
|
44
44
|
| ba-pitch-analyzer | exec | planner — builder tier, not judgment |
|
|
45
45
|
| scope-architect | exec | scope-contract author (sole writer of scopes/*.md) — builder tier |
|
|
46
46
|
| task-executor | exec | the builder itself |
|
|
47
|
-
| advisor-protocol | exec | adjudicates ESCALATE by precedent/default, budget-capped — not the single judge |
|
|
48
47
|
| spec-evaluator | eval | the single judge (judge ≠ doer) — keep its own matrix key even if a PO points it at the same model as `exec`, so it can be split later without a harness change |
|
|
49
48
|
| qa-edge-hunter | qa | cheapest tier by design — exploratory breadth over depth |
|
|
50
49
|
| scope-hammer | exec | census + baseline comparison, proposes only — not a verdict |
|
|
@@ -95,7 +94,6 @@ Order A (the spec tree + board):
|
|
|
95
94
|
gen from reality), spike-<area>.md (feasibility/contracts).
|
|
96
95
|
Writes (its substrate): spec_folder → _index.md, domain-model.md, ux-behavior.md, usecases/*,
|
|
97
96
|
contracts/*.contract.md, scope-summary.md (+ api-feasibility.md if third-party) and the
|
|
98
|
-
LOCAL board .shapeup/<slug>/tasks/ (v3.2; regenerable via a generate-board order).
|
|
99
97
|
Returns: WorkResult (artifacts list + discoveries) → ingest-result.
|
|
100
98
|
Order B (the scope contracts):
|
|
101
99
|
compile-order --operation map-scopes --slug <slug> --worker scope-architect
|
|
@@ -121,11 +119,6 @@ Returns: WorkResult → ingest-result (which updates the board and bumps discove
|
|
|
121
119
|
Read back: updated tasks/_index.md + scope-summary.md before routing back to GATE L1b.
|
|
122
120
|
```
|
|
123
121
|
|
|
124
|
-
## 2c. BOOTSTRAP LOCAL BOARD → ba-pitch-analyzer (operation: generate-board, v3.2)
|
|
125
|
-
```
|
|
126
|
-
Only when GATE L1b's bootstrap check fires (LOCAL .shapeup/<slug>/tasks/_index.md missing
|
|
127
|
-
AND SHARED shapeup/<slug>/spec/usecases/ present):
|
|
128
|
-
compile-order --operation generate-board --slug <slug> --worker ba-pitch-analyzer
|
|
129
122
|
Agent (model: exec): Skill(shapeup-sdlc-plugin:ba-pitch-analyzer) --order <path>
|
|
130
123
|
Effect: regenerates the LOCAL task board fresh from the committed usecases/domain-model/scopes
|
|
131
124
|
— no ledger, no reconciliation. Status bootstraps from committed T0/hill facts at
|
|
@@ -149,40 +142,53 @@ r>1 (fix) per bug:
|
|
|
149
142
|
Scope contracts present (isolated attempt loop, per scope, per attempt):
|
|
150
143
|
compile-order --scope shapeup/<slug>/scopes/<id>.md --round <N> --attempt <M>
|
|
151
144
|
→ orders/r<N>-a<M>.json inlines the scope contract, this scope's tasks, promoted ledger
|
|
152
|
-
decisions,
|
|
153
|
-
|
|
145
|
+
decisions, the previous attempt's AEGIS triples, and `trial_history` — the last 8 trials
|
|
146
|
+
for this scope (score, status, delta, top-3 digest), CROSSING the round boundary so a fix
|
|
147
|
+
round cannot re-propose a change the build round already reverted. The zero-memory
|
|
148
|
+
handoff, compiled from facts only. Dispatch a fresh Agent per attempt (the isolation
|
|
149
|
+
boundary):
|
|
154
150
|
Skill(shapeup-sdlc-plugin:task-executor) --order <path>
|
|
155
|
-
ingest-result —
|
|
151
|
+
ingest-result — a WorkResult with status "escalated" leaves its artifact unwritten → see 3b below.
|
|
156
152
|
|
|
157
|
-
Read back: ingest-result's summary line (tasks updated, unblocked,
|
|
153
|
+
Read back: ingest-result's summary line (tasks updated, ACs ticked, unblocked, discoveries) — not raw board
|
|
158
154
|
files. SPIKE tasks close before the tasks they block can build (compile-order enforces the
|
|
159
155
|
dependency order).
|
|
160
156
|
```
|
|
161
157
|
|
|
162
|
-
## 3b.
|
|
158
|
+
## 3b. A worker that cannot finish (scope contracts present, mid-attempt)
|
|
163
159
|
```
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
160
|
+
There is NO adjudication dispatch. A worker has no port for "I cannot decide this": WorkResult
|
|
161
|
+
carries no escalates field, so the question never reaches you as data.
|
|
162
|
+
What you see: the phase produces no artifact. The workflow's post-condition
|
|
163
|
+
(resume-state.mjs --require <phase>) fails and the run ABORTS, naming the phase.
|
|
164
|
+
Do: read the phase's result file to find what it could not complete, resolve it yourself —
|
|
165
|
+
by amending the spec, widening the scope contract, or answering the ambiguity in the
|
|
166
|
+
round-ledger "Decisions" table — then relaunch. The fast-forward re-dispatches only
|
|
167
|
+
what is still unfinished.
|
|
168
|
+
Why it aborts rather than pauses: nothing persists an answer between launches, so a pause
|
|
169
|
+
would relaunch into the same order and hit the same wall. Aborting puts the question
|
|
170
|
+
in front of a human once instead of looping silently.
|
|
172
171
|
```
|
|
173
172
|
|
|
174
173
|
## 3c. T0 verify → scripts/t0-verify.mjs (skill-local; scope contracts present, every attempt)
|
|
175
174
|
```
|
|
176
|
-
Invoke via Bash directly — NOT an Agent, this is deterministic tooling, not a worker
|
|
175
|
+
Invoke via Bash directly — NOT an Agent, this is deterministic tooling, not a worker:
|
|
177
176
|
node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/t0-verify.mjs" shapeup/<slug>/scopes/<scope-id>.md
|
|
178
177
|
--round <N> --attempt <M> --seesaw-registry .shapeup/<slug>/seesaw/registry.json
|
|
179
178
|
Effect: runs the scope's e2e fixtures + DB probe, then (on green) the seesaw regression check
|
|
180
179
|
over every FINISHED scope's fixtures. Writes the verdict artifact spec-evaluator's
|
|
181
|
-
|
|
180
|
+
T0-citation rule will require a citation to, appends one row to t0/trials.jsonl, and — this
|
|
181
|
+
is the ratchet — scores the attempt against the last kept trial and snapshots or
|
|
182
|
+
restores the working tree ITSELF. Zero LLM tokens — deterministic tooling, not a
|
|
182
183
|
judge (this is what keeps "T1 once per round" true even though verification runs
|
|
183
|
-
every attempt
|
|
184
|
-
Read back:
|
|
185
|
-
|
|
184
|
+
every attempt).
|
|
185
|
+
Read back: the stdout JSON — {path, sha256, trial, overall, regression, score, status,
|
|
186
|
+
baseline_trial, delta, tree_ref}. `status` (kept|reverted|rebased|crash) is what
|
|
187
|
+
drives the attempt-loop branch in round-protocol.md "Isolated attempt loop"; by the
|
|
188
|
+
time you read it the tree action has already happened. Never branch on the process
|
|
189
|
+
exit code: it carries the T0 binary (0 green / 1 red / 2 bad argv, the oracles/*
|
|
190
|
+
convention), so a `kept` red-but-improved attempt, the ratchet's own signature case,
|
|
191
|
+
exits 1. On red, its `discovered_tasks` field is
|
|
186
192
|
the AEGIS digest to fold into the next brief — no separate digester dispatch needed
|
|
187
193
|
(t0-verify.mjs calls its sibling aegis-digest.mjs internally on failure).
|
|
188
194
|
```
|
|
@@ -245,10 +251,11 @@ Read back: the proposed cut list + verdict (SHIP now | SHIP after fixing ship-bl
|
|
|
245
251
|
| `scope-summary.md` | planner (analyze/reconcile orders) | tech lead (Done-when), evaluator (Done-when criteria) |
|
|
246
252
|
| `evaluation/EVAL-FEATURE-<slug>.md` + `.verdicts-*.jsonl` | evaluator (report) / **ingest-result.mjs** (verdict ledger, un-ticks) | tech lead (verdict), next fix order (bug list) |
|
|
247
253
|
| `harness-run.md` | **tech lead (sole writer)** | tech lead (round ledger + Hill + run-state), PO (audit) |
|
|
248
|
-
| `scopes/<scope-id>.md` | `scope-architect` (sole writer
|
|
249
|
-
| `t0/verdicts/r<N>-a<M>.json` | `scripts/t0-verify.mjs` (skill-local, mechanical — not a worker) | spec-evaluator (required citation), tech lead (hill derivation), compile-order (digested errors) |
|
|
250
|
-
| `
|
|
254
|
+
| `scopes/<scope-id>.md` | `scope-architect` (sole writer) | tech lead (substrate/sequence), sandbox hook (write-whitelist), compile-order (inlined into orders) |
|
|
255
|
+
| `t0/verdicts/r<N>-a<M>-t<T>.json` | `scripts/t0-verify.mjs` (skill-local, mechanical — not a worker) | spec-evaluator (required citation), tech lead (hill derivation), compile-order (digested errors) |
|
|
256
|
+
| `t0/trials.jsonl` (the ratchet ledger, append-only, `baseline_trial` as the parent link) | `scripts/t0-verify.mjs` (one row per attempt: score, status, delta, tree_ref) | compile-order (`trial_history` into the next order), ship-report (T0 + Ratchet sections), `stats.mjs --ratchet` |
|
|
257
|
+
| `round-ledger.md` | **tech lead (sole writer)** | compile-order (decisions into every order), PO (audit) |
|
|
251
258
|
| `hill/<scope-id>.yml` + `hill-chart.md` | **tech lead (sole writer)** | PO ("status without asking"), scope-hammer (H0 census) |
|
|
252
259
|
|
|
253
|
-
>
|
|
260
|
+
> The single-writer rule is closed mechanically: no worker writes `run-state.md`, the board, or the ledger. A worker
|
|
254
261
|
> performing a shared-state write is a defect — route it back through its WorkResult.
|
|
@@ -2,12 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
The full collect-lists, gate-output blocks, and delegation scripts for every gate and delegation
|
|
4
4
|
step, extracted from `SKILL.md` (progressive disclosure). **Read the relevant section when you
|
|
5
|
-
reach that gate** — `SKILL.md` carries the workflow spine
|
|
6
|
-
|
|
7
|
-
of where the prose lives.
|
|
5
|
+
reach that gate** — `SKILL.md` carries the workflow spine and the hard rules; this file carries the
|
|
6
|
+
step-by-step playbooks. The L2/L4 gates stay hook-enforced regardless of where the prose lives.
|
|
8
7
|
|
|
9
|
-
|
|
10
|
-
|
|
8
|
+
**Which lane these playbooks serve.** The gate blocks below are lane-independent: every
|
|
9
|
+
lane emits them verbatim, and on the scoped lane `shapeup-run.js` returns the `block` field for
|
|
10
|
+
`SKILL.md` to print unchanged. What differs is who runs the loop *between* the gates:
|
|
11
|
+
|
|
12
|
+
| lane | BUILD → GATE L2 → EVAL is | read |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| a spec with committed `scopes/*.md` (the common case) | **code** — `skills/tech-lead/workflows/shapeup-run.js`'s round loop | that script's comments |
|
|
15
|
+
| `--tiny`, or a spec with no scope contracts yet | **prose**, unchanged and non-regression | `references/round-protocol.md` (`:11-22` states the same split) |
|
|
16
|
+
|
|
17
|
+
Order matches the run: GATE L0 → ORIENT → GATE L1a → ANALYZE → WIRE/L1a.5 → MAP SCOPES →
|
|
18
|
+
GATE L1b → (BUILD → GATE L2 → EVAL — see the table above) → GATE L3 → SHIP → GATE L4.
|
|
19
|
+
|
|
20
|
+
⟐ **ANALYZE runs before WIRE.** MAP SCOPES' two dispatches split around
|
|
21
|
+
L1a.5: `analyze` writes the spec tree, then `wire` reads its `usecases/` — one wiring-map entry per
|
|
22
|
+
use case — then `map-scopes` slices. Dispatching WIRE first hands `solution-architect` an empty
|
|
23
|
+
spec folder, which its own input contract excludes (`skills/solution-architect/SKILL.md:43-44`); it
|
|
24
|
+
escalates, writes nothing, and every relaunch re-dispatches it.
|
|
11
25
|
|
|
12
26
|
---
|
|
13
27
|
|
|
@@ -37,11 +51,16 @@ Collect (explicit — never inferred):
|
|
|
37
51
|
- LOCAL .shapeup/<slug>/ — run-trace (hidden, gitignorable):
|
|
38
52
|
harness-run.md (this ledger), digest, orient/, evaluation/, qa/,
|
|
39
53
|
discovery/ledger.md, orders/ + results/ (the envelope port), tasks/ (the task
|
|
40
|
-
board, v3.2 — regenerable via a generate-board order on any machine)
|
|
41
54
|
spec_folder = shapeup/<slug>/spec/ (the deliverable arg passed to ba/eval/exec)
|
|
42
55
|
L0.3 lens: lite | standard | cross-context (passed to planner at step 8)
|
|
43
56
|
L0.4 stack hint (e.g. "pnpm, Next 16 web :3000") — aims orient's code-surface sweeps + run commands
|
|
44
|
-
L0.5 eval dimensions: default [spec-conformance]; only add if user asks
|
|
57
|
+
L0.5 eval dimensions: default [spec-conformance]; only add if user asks. An added dimension
|
|
58
|
+
must reach `init-run.mjs --dimensions <a,b>` — it is recorded in the ledger's
|
|
59
|
+
`eval_dimensions:` line and every EVAL order is compiled from there, so a set agreed
|
|
60
|
+
in conversation and not passed to the flag grades nothing. Shipped ids:
|
|
61
|
+
spec-conformance, tdd-surface, integration, completeness, test-surface-conformance
|
|
62
|
+
(security + performance ship disabled). Whatever is left out is reported at L4 as
|
|
63
|
+
`dims_not_evaluated` — "shipped" never silently means "verified for all".
|
|
45
64
|
L0.6 max_rounds: default 3, appetite-informed (see L0.1b)
|
|
46
65
|
L0.7 auto level:
|
|
47
66
|
interactive (default) — pause at every L-gate; sub-skills keep their own gates
|
|
@@ -49,7 +68,7 @@ Collect (explicit — never inferred):
|
|
|
49
68
|
at L1a/L1b (orient+plan), L3 (verdict), L4 (ship)
|
|
50
69
|
--unattended — auto-confirm all L-gates too; stop only on PASS,
|
|
51
70
|
max_rounds, or hard error (for headless / Agent SDK / CI)
|
|
52
|
-
L0.8 Model & budget resolution (
|
|
71
|
+
L0.8 Model & budget resolution (four layers, highest precedence
|
|
53
72
|
first) — resolve ONCE here, record the resulting matrix in the ledger header:
|
|
54
73
|
/ship flags → .claude/settings.local.json (per-member, Tier C) →
|
|
55
74
|
.claude/settings.json (team defaults, committed) → skill-shipped defaults
|
|
@@ -60,7 +79,7 @@ Collect (explicit — never inferred):
|
|
|
60
79
|
A requested model unavailable on the member's plan → degrade to the next tier down,
|
|
61
80
|
record the degrade in the ledger (R2 — invariants are code paths, so adherence
|
|
62
81
|
survives even when the model tier doesn't).
|
|
63
|
-
L0.9 attempt_budget: the INNER circuit breaker
|
|
82
|
+
L0.9 attempt_budget: the INNER circuit breaker, nested inside
|
|
64
83
|
max_rounds (the OUTER breaker, unchanged, L0.6). Default 5 — the number of T0 verify
|
|
65
84
|
attempts a single scope gets inside one round before its attempt loop trips and
|
|
66
85
|
queues a hammer PROPOSAL for GATE H rather than blocking the round. Only meaningful
|
|
@@ -84,8 +103,7 @@ none resolvable -> ABORT. Print the ✋ NO INTAKE block from SKILL.md and stop
|
|
|
84
103
|
Do NOT describe the pipeline that "will" run.
|
|
85
104
|
```
|
|
86
105
|
An orchestrator with no spec has nothing to orchestrate. Narrating the gate list in that state
|
|
87
|
-
produces output that reads exactly like a successful run and contains no work
|
|
88
|
-
acceptance, n=3, on the benchmark. Fail loudly instead.
|
|
106
|
+
produces output that reads exactly like a successful run and contains no work. Fail loudly instead.
|
|
89
107
|
|
|
90
108
|
**GATE L0 Output:**
|
|
91
109
|
```
|
|
@@ -149,7 +167,12 @@ Do NOT enter MAP SCOPES until Orient is accepted.
|
|
|
149
167
|
2. WIRE — compile-order --operation wire --slug <slug> (worker→solution-architect), payload
|
|
150
168
|
{project_profile}. Sole writer of committed wiring-map.md (per-UC engine → seam → entry-point
|
|
151
169
|
call site → affordance). ⏸ GATE L1a.5: confirm each UC has a declared seam before slicing.
|
|
152
|
-
|
|
170
|
+
⟐ PRECONDITION: MAP SCOPES step 1 (ANALYZE) has already run and usecases/ is
|
|
171
|
+
populated. WIRE writes one entry per use case, so dispatching it against an empty spec folder
|
|
172
|
+
is an escalation, not a wiring map — and a phase that writes no artifact is re-dispatched on
|
|
173
|
+
every relaunch. Verify: node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/resume-state.mjs"
|
|
174
|
+
--slug <slug> --require analyze (exit 0 = the spec tree is there; exit 6 = do not dispatch WIRE).
|
|
175
|
+
Post-check the same way after the dispatch: --require wire.
|
|
153
176
|
requirements.md registry (atomic REQ clauses, frozen ids).
|
|
154
177
|
4. trace-lint — node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/trace-lint.mjs" --slug <slug>. ADVISORY at L1b:
|
|
155
178
|
covers-closure (every covered REQ named by ≥1 AC's covers:) + reachability (every UC engine
|
|
@@ -160,6 +183,11 @@ Do NOT enter MAP SCOPES until Orient is accepted.
|
|
|
160
183
|
|
|
161
184
|
## MAP SCOPES (step 8) — delegate to ba-pitch-analyzer (orient-informed)
|
|
162
185
|
|
|
186
|
+
⟐ **Step 1 runs BEFORE the WIRE section above; step 2 runs after it.** The two
|
|
187
|
+
dispatches sit either side of L1a.5: ANALYZE writes the use cases WIRE reads, and MAP SCOPES slices
|
|
188
|
+
against the seams WIRE declared. Sequence: ORIENT → L1a → **ANALYZE** → **WIRE** → L1a.5 →
|
|
189
|
+
**MAP SCOPES** → L1b.
|
|
190
|
+
|
|
163
191
|
```
|
|
164
192
|
Two orders, two workers, one step (both model: exec — see references/delegation.md):
|
|
165
193
|
1. ANALYZE + BOARD — compile-order --operation analyze --slug <slug> --worker ba-pitch-analyzer
|
|
@@ -179,7 +207,7 @@ Record in ledger: planner duration + task count + scope count.
|
|
|
179
207
|
Faithful note: keep the planner ambitious on scope but high-level on tech — do not push it
|
|
180
208
|
to over-specify implementation. Errors baked into the spec cascade into every build round.
|
|
181
209
|
Honest deviation: `ba` is heavier than Shape Up's light "map scopes" bucketing — that extra
|
|
182
|
-
upfront spec-traceability is a deliberate trade for an LLM builder
|
|
210
|
+
upfront spec-traceability is a deliberate trade for an LLM builder, not
|
|
183
211
|
"pure Shape Up". State it; don't pretend otherwise.
|
|
184
212
|
|
|
185
213
|
---
|
|
@@ -203,7 +231,6 @@ v0.2.6 (non-regression).
|
|
|
203
231
|
`.shapeup/<slug>/tasks/_index.md` missing AND `shapeup/<slug>/spec/usecases/`
|
|
204
232
|
exists → a teammate (or a `--from build` resumed run) has the SHARED spec via git but no LOCAL
|
|
205
233
|
task board on this machine — `.shapeup/` is gitignored and never travels with a branch.
|
|
206
|
-
compile-order --operation generate-board --slug <slug> --worker ba-pitch-analyzer, then
|
|
207
234
|
Agent (model: exec): Skill(shapeup-sdlc-plugin:ba-pitch-analyzer) --order <path> + ingest —
|
|
208
235
|
regenerates the board from the committed usecases/domain-model/scopes. Record the bootstrap in
|
|
209
236
|
the ledger. No-op on a fresh r=1 run (MAP SCOPES just wrote the board on this same machine) or
|
|
@@ -224,13 +251,12 @@ No scope contracts (pre-v0.3.0, unchanged from v0.2.6):
|
|
|
224
251
|
- any SPIKE tasks (third-party feasibility) that block others
|
|
225
252
|
- scope-summary "Done when" headline statements
|
|
226
253
|
|
|
227
|
-
Substrate-disjointness assertion (
|
|
254
|
+
Substrate-disjointness assertion (only when
|
|
228
255
|
shapeup/<slug>/scopes/*.md exist — scope-architect's lint pass already ran;
|
|
229
256
|
this is the orchestrator's own re-confirmation before committing to a build sequence):
|
|
230
257
|
- Run `node "${CLAUDE_PLUGIN_ROOT}/skills/ba-pitch-analyzer/scripts/spec-lint.mjs" --slug <slug>`: DISJOINT (a file in two
|
|
231
258
|
scopes' `allowed_file_substrate` without BOTH declaring it `shared_substrate` — PA3
|
|
232
259
|
waiting to happen), PA1 (directory-aligned scope), PA2 (size cap). Any red → HARD STOP,
|
|
233
|
-
route a remap order to scope-architect before BUILD — a human may have hand-approved
|
|
234
260
|
past a 🔴 at the architect's own checkpoint.
|
|
235
261
|
- Lock the build SEQUENCE riskiest-first: order scopes by open-unknowns count (from
|
|
236
262
|
hill/<scope-id>.yml if present, else the orient hill signal), not by file count or
|
|
@@ -247,11 +273,27 @@ Do NOT enter BUILD until the board is accepted.
|
|
|
247
273
|
|
|
248
274
|
---
|
|
249
275
|
|
|
276
|
+
## GATE L2 — Board 100% Done (Pre-Eval)
|
|
277
|
+
|
|
278
|
+
**Purpose:** The single exit point from the BUILD loop into EVAL. It ensures every scope is green before invoking the evaluator.
|
|
279
|
+
|
|
280
|
+
**Output:**
|
|
281
|
+
```
|
|
282
|
+
⏸ GATE L2 — Board 100% ✅ + T0-green
|
|
283
|
+
Feature : [slug]
|
|
284
|
+
Round : [r]
|
|
285
|
+
Scopes : [N] green, [M] queued for hammer
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
Under `--interactive` / `--auto`, the hook warns if the board is not truly green (advisory) and requires explicit PO approval to proceed. Under `--unattended`, it automatically aborts on a red board or proceeds on a green one.
|
|
289
|
+
|
|
290
|
+
---
|
|
291
|
+
|
|
250
292
|
## GATE L3 — Verdict & Loop
|
|
251
293
|
|
|
252
294
|
```
|
|
253
295
|
Render the 🗻 Hill report (slice-level) — NOT a task count. Scope contracts present → read
|
|
254
|
-
committed hill/<scope-id>.yml shards (mechanical phases from GATE L2
|
|
296
|
+
committed hill/<scope-id>.yml shards (mechanical phases from GATE L2, never authored). No contracts →
|
|
255
297
|
fall back to the board + open-unknowns heuristic (uphill/crest/downhill/done). See
|
|
256
298
|
references/ledger-schema.md "Hill report". Roadmap rule unchanged either way: progress is
|
|
257
299
|
reported by hill position, never by "N/M tasks done".
|
|
@@ -267,7 +309,6 @@ PASS:
|
|
|
267
309
|
+ .shapeup/<slug>/qa/hunt-report.md. No verdict — the run's verdict stays this EVAL's PASS.
|
|
268
310
|
→ then proceed to SHIP (triage of QA findings happens at SHIP S.0/GATE L4).
|
|
269
311
|
→ subsequent PASS (a promoted-findings fix round): Agent (model: qa):
|
|
270
|
-
Skill(shapeup-sdlc-plugin:qa-edge-hunter) --recheck on the promoted items only, then SHIP.
|
|
271
312
|
→ --no-qa or skill absent: proceed straight to SHIP; ledger records `qa: skipped`.
|
|
272
313
|
|
|
273
314
|
FAIL:
|
|
@@ -289,7 +330,7 @@ Decision : [SHIP | re-build bugs in round r+1 | escalate: max rounds hit]
|
|
|
289
330
|
```
|
|
290
331
|
The Hill is the progress narrative; the board's `N/N ✅` is execution substrate (it gates
|
|
291
332
|
EVAL at L2), never the headline. Slices come from `ba`'s board; if slice IDs aren't present
|
|
292
|
-
yet
|
|
333
|
+
yet, report at task-group level and note the fallback in the ledger.
|
|
293
334
|
|
|
294
335
|
---
|
|
295
336
|
|
|
@@ -304,7 +345,7 @@ S.0 GATE H — delegate to scope-hammer (this IS Shape Up's "Decide When to Sto
|
|
|
304
345
|
exhausted scopes accumulated during BUILD)
|
|
305
346
|
(no --breaker flag) normal stop — all scopes FINISHED, post-QA-hunt
|
|
306
347
|
Feeds it: qa/hunt-report.md findings (when present), discovery/ledger.md open items,
|
|
307
|
-
|
|
348
|
+
the hammer-proposal queue from BUILD (attempt-budget exhaustions).
|
|
308
349
|
Reads back: its GATE H0/H1/H2 output — census, baseline comparison, cut list + verdict.
|
|
309
350
|
Authority: scope-hammer proposes; the tech lead records the PO's decision in
|
|
310
351
|
round-ledger.md and performs the actual close (S.1 onward). It never ships on its own.
|
|
@@ -326,35 +367,36 @@ S.2 Print a feature summary: tasks shipped, rounds used, final verdict, dims ev
|
|
|
326
367
|
(and explicitly: dims NOT evaluated, so "shipped" is never read as "verified for all").
|
|
327
368
|
S.3 Point to the traceability: tasks/_index.md (all ✅) + EVAL-FEATURE-<slug>.md (PASS) +
|
|
328
369
|
harness-run.md (the round ledger).
|
|
329
|
-
S.4 task-executor's
|
|
330
|
-
feature-level close.
|
|
370
|
+
S.4 task-executor's own verification checklist remains the formal per-task close; the tech
|
|
371
|
+
lead confirms the feature-level close.
|
|
331
372
|
S.5 Deploy truth — "done means deployed", honestly. Building stops at "built & verified";
|
|
332
373
|
deployment is an outward-facing action gated to the PO. Either:
|
|
333
|
-
- PO says yes → run the project deploy
|
|
374
|
+
- PO says yes → run the project's own deploy procedure and record "deployed".
|
|
334
375
|
- otherwise → record "built & verified — deploy pending (PO)".
|
|
335
376
|
NEVER auto-deploy; "shipped" must never silently mean "deployed".
|
|
336
|
-
(Baseline-anchored scope-hammering at ship time is
|
|
377
|
+
(Baseline-anchored scope-hammering at ship time is deliberately deferred; for now,
|
|
337
378
|
`ba`'s Appetite Guard covers overflow and cuts go to synthesis "Hammered Out".)
|
|
338
|
-
S.6 Harvest one signal row → append to
|
|
339
|
-
(
|
|
340
|
-
|
|
379
|
+
S.6 Harvest one signal row → append to `.shapeup/metrics/<machine-id>.jsonl`
|
|
380
|
+
(LOCAL root, gitignored since ADR-0001 — a committed shard keyed on a hostname only
|
|
381
|
+
grows and publishes a machine name; sharded per machine so shards can be pooled
|
|
382
|
+
deliberately without colliding on one filename. The read plane is
|
|
383
|
+
`stats.mjs`, or `cat .shapeup/metrics/*.jsonl`).
|
|
341
384
|
Copy fields that ALREADY exist as structured output (run-state, final EVAL report,
|
|
342
385
|
discovery ledger, qa/hunt-report, breadboard B5). Two hard rules:
|
|
343
386
|
1. Harvest only fields that already exist at ship time — never evaluate something new.
|
|
344
387
|
2. Record facts, never compute a new verdict (no `run_quality_score` — that would be
|
|
345
388
|
a second judge behind spec-evaluator). The eval suite interprets; harvest records.
|
|
346
389
|
`final_audit_score` is COPIED from the EVAL report, never re-graded.
|
|
347
|
-
ALSO copy
|
|
390
|
+
ALSO copy two run-quality measurements, both produced by scripts at zero model tokens:
|
|
348
391
|
node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/stats.mjs" --ratchet --slug <slug>
|
|
349
392
|
→ `ratchet`: {trials, scopes_multi_trial, improvement_rate, monotone_rate,
|
|
350
393
|
sawtooth_count, mean_trials_to_green}
|
|
351
394
|
node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/stats.mjs" --hooks
|
|
352
395
|
→ `hooks`: {evaluations, denials, errors, per_hook}
|
|
353
|
-
Harvest them HERE or lose them: both read
|
|
354
|
-
decisions.jsonl)
|
|
355
|
-
answers its question exactly once
|
|
356
|
-
|
|
357
|
-
criteria require, since neither is meaningful from a single run.
|
|
396
|
+
Harvest them HERE or lose them: both read per-run working ledgers (t0/trials.jsonl,
|
|
397
|
+
decisions.jsonl) that the next run's bookkeeping supersedes, so a number left there
|
|
398
|
+
answers its question exactly once. The metrics shard is the one record that
|
|
399
|
+
accumulates across runs — and neither measurement is meaningful from a single run.
|
|
358
400
|
→ full field list + row template: references/ledger-schema.md "Harvest row".
|
|
359
401
|
```
|
|
360
402
|
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Hard Rules (never override without explicit user instruction)
|
|
2
|
+
|
|
3
|
+
Moved out of `SKILL.md` when the round loop became a launchable script:
|
|
4
|
+
most of what this table used to guard against — a partial board reaching EVAL, a gate crossed
|
|
5
|
+
on the model's own authority, an evaluator called mid-BUILD — is now a property of
|
|
6
|
+
`skills/tech-lead/workflows/shapeup-run.js`'s code, not a rule a model has to remember to obey.
|
|
7
|
+
The table stays as the readable rationale for WHY the code is shaped the way it is; the runtime
|
|
8
|
+
guarantee lives in the script and, where noted, in a hook.
|
|
9
|
+
|
|
10
|
+
| Rule | Rationale |
|
|
11
|
+
|------|-----------|
|
|
12
|
+
| Orchestrates Building only (steps 7–11); shaping/betting/kick-off are PO-personal, upstream | Intake is a kicked-off pitch, not a raw idea — the tech lead does no shaping/planning-authority work |
|
|
13
|
+
| ORIENT (step 7) runs before MAP SCOPES (step 8) | Roadmap: no pre-divided tasks at kick-off; the team orients first so the board is reality-born |
|
|
14
|
+
| Intake must be English before ORIENT; tech lead does NOT translate — it delegates to `translator` at GATE L0 | Translation is a separate single-purpose skill; the intake conversation only detects + sequences it, before RunArgs is ever compiled |
|
|
15
|
+
| Every worker dispatch goes through the envelope port: compile-order → `--order` → ingest-result; shared state is written ONLY by ingest | The single-writer rule is mechanically true: a worker that writes boards/ledgers/run-state is a defect, and a malformed envelope is denied by the validate-envelope hook before it can corrupt run truth. `shapeup-run.js` uses this same shape for every operation in the central registry — orient, wire, analyze, map-scopes, evaluate, hunt, hammer alike |
|
|
16
|
+
| Progress is reported by Hill position, never by counting tasks | The roadmap forbids task-counting; a 90%-done slice can still be stuck uphill on the one unknown that matters |
|
|
17
|
+
| Evaluator runs once per round, only after GATE L2 (board 100% done) | The whole point: cheap end-of-round QA, never per task. `shapeup-run.js`'s round loop dispatches spec-evaluator exactly once per iteration, after the GATE L2 resolution — there is no code path that calls it from inside the scope attempt loop |
|
|
18
|
+
| Evaluator never called inside the BUILD loop | Keeps the build coherent and the run cheap |
|
|
19
|
+
| r>1 builds bugs only, never the whole board | Don't re-do passing work; minimize churn — see round-protocol.md's regression rule for what DOES re-run (touched UCs' full Test Surface) |
|
|
20
|
+
| Stop at max_rounds; escalate honestly | No infinite fix loops; `shapeup-run.js` returns `{status: "gate_h", breaker: "outer"}` rather than looping past the budget |
|
|
21
|
+
| Tech lead delegates, never reimplements a sub-skill | Stays thin; each skill keeps its own gates and authority |
|
|
22
|
+
| Every delegation to a sub-skill (except the mechanical `t0-verify.mjs`/`compile-order.mjs`/`ingest-result.mjs`) is a fresh Agent on the L0.8-resolved model | Isolation the zero-memory-handoff design assumes; a direct inline call would silently drop the model matrix — see references/delegation.md "Invocation mechanism" |
|
|
23
|
+
| Planner stays high-level on tech | Spec errors cascade into every build round |
|
|
24
|
+
| Never auto-deploy; "shipped" never silently means "deployed" | Deploy is outward-facing, PO-gated; record "deploy pending (PO)" otherwise |
|
|
25
|
+
| "Shipped" names the dims NOT evaluated | `RunReturn`'s `dims_not_evaluated` field carries this; the L4 sign-off block shows it, never silently drops it |
|
|
26
|
+
| Every gate emits the canonical `⏸ GATE LN — Title` block before any narrative | Composed by the workflow (`gateBlock()`), emitted VERBATIM by the skill — conversational re-summary is not a gate |
|
|
27
|
+
| In interactive/--auto: a `paused` return stops and waits for PO confirmation | Never auto-proceed past a gate; the PO must cross each threshold explicitly — see "The pause protocol" in SKILL.md |
|
|
28
|
+
| At GATE L3 FAIL: name scope (task + failed criterion), never prescribe fix options | Root cause analysis and fix paths belong to the implementer, not the orchestrator |
|
|
29
|
+
| SHIP harvest records facts only — copies existing structured output, never computes a new verdict/score | A self-computed score = a second judge behind spec-evaluator (breaks single-judge, invites Goodhart); the eval suite interprets, harvest records |
|
|
30
|
+
| Three-level circuit breaker: attempt_budget (inner, per scope) nests inside round_budget (outer), with an opt-in wall_clock_budget_s deadline | An exhausted scope queues a GATE H hammer proposal, it never blocks the round; only round_budget hitting 0 stops the whole run; the deadline breaker (checked every round boundary in `shapeup-run.js`) routes to GATE H so a run out of clock still ships what is green instead of being killed from outside |
|
|
31
|
+
| The tech lead never hand-edits a scope contract | scope-architect is its sole writer (single-writer-per-file) |
|
|
32
|
+
| Substrate-disjointness + PA1/PA2 lints are re-asserted at GATE L1b (spec-lint.mjs) even when scope-architect already checked them | A human may have hand-approved past a 🔴 at the architect's checkpoint; `shapeup-run.js` runs spec-lint itself, in code, before resolving L1b |
|
|
33
|
+
| Hill phase is read from mechanical facts (T0/T1/seesaw), never declared by a worker | Closes the self-reported-confidence risk outright — facts move dots, not authors |
|
|
34
|
+
| GATE H is delegated to scope-hammer, never adjudicated inline by the tech lead | Keeps the orchestrator thin; census/baseline-comparison/cut-list logic has one owner |
|
|
@@ -34,12 +34,12 @@ disclosure). Read this when you need the exact CLI shape or a flag's effect.
|
|
|
34
34
|
| `--unattended` | Auto-confirm all L-gates (headless / CI) |
|
|
35
35
|
| `--max-rounds N` | BUILD→EVAL cycles before escalating (default 3) — the OUTER breaker |
|
|
36
36
|
| `--attempts N` | Per-scope T0 attempts before queuing a GATE H hammer proposal (default 5) — the INNER breaker; no-op on specs without scope contracts |
|
|
37
|
-
| `--
|
|
37
|
+
| `--exec-model / --eval-model / --qa-model <name>` | Override L0.8's resolved model matrix (`RunArgs.models`: exec/eval/qa) for this run only — highest precedence. The orchestrator itself runs in-session; it has no model knob here |
|
|
38
38
|
| `--from orient\|plan\|build\|eval` | Resume an in-progress run at a build-phase step |
|
|
39
39
|
| `--no-eval` | Skip the evaluation pass this run (trivial feature) |
|
|
40
40
|
| `--no-qa` | Skip the post-PASS /qa-edge-hunter pass (ledger records `qa: skipped`) |
|
|
41
41
|
| `--tiny` | The small-change lane: orient (light) → single-task board → build → T0 → done. Implies `--no-eval --no-qa`, skips WIRE and scope contracts, collapses the gates to L0 + L4. See "The tiny lane" in SKILL.md — the L0 fit-check is mandatory and the ledger records `lane: tiny` |
|
|
42
|
-
| `--dimensions <list>` | Eval dimensions (default spec-conformance) |
|
|
42
|
+
| `--dimensions <list>` | Eval dimensions, comma-separated (default `spec-conformance`). Passed straight through to `init-run.mjs --dimensions`, which records the set in the ledger's `eval_dimensions:` — that line is what every EVAL order is compiled from, so a dimension named anywhere else does not run. Ids the evaluator ships: `spec-conformance`, `tdd-surface`, `integration`, `completeness`, `test-surface-conformance`, `security`, `performance` (the last two are disabled stubs); a dimension you inject yourself is named here the same way |
|
|
43
43
|
|
|
44
44
|
---
|
|
45
45
|
|
|
@@ -6,7 +6,7 @@ not a shared deliverable). It is the structured
|
|
|
6
6
|
artifact that carries state across rounds and across sessions (so `--from` can resume), and
|
|
7
7
|
the PO's audit of how the feature was built.
|
|
8
8
|
|
|
9
|
-
**The tech lead is the sole writer of this file
|
|
9
|
+
**The tech lead is the sole writer of this file.** It is the authoritative
|
|
10
10
|
run-state for the whole build phase — rounds, gate decisions, Hill positions, verdicts,
|
|
11
11
|
`discovered_rounds`, config, language record. Workers never write here; the tech lead passes
|
|
12
12
|
them what they need (`feature`, `spec`, `stack`, `discovered_rounds`) as args. The board
|
|
@@ -19,7 +19,7 @@ type: harness-run
|
|
|
19
19
|
feature: [slug]
|
|
20
20
|
spec_folder: [path to SHARED spec deliverable, e.g. shapeup/<slug>/spec/]
|
|
21
21
|
lens: lite | standard | cross-context
|
|
22
|
-
eval_dimensions: [spec-conformance]
|
|
22
|
+
eval_dimensions: [spec-conformance] # the set from GATE L0.5 (init-run --dimensions); every EVAL order is compiled from THIS line
|
|
23
23
|
max_rounds: 3
|
|
24
24
|
auto_level: interactive | auto | unattended
|
|
25
25
|
status: orienting | mapping | building | evaluating | shipped | escalated
|
|
@@ -71,7 +71,7 @@ Position triggers:
|
|
|
71
71
|
- ✅ **Done** — slice clickable-done.
|
|
72
72
|
|
|
73
73
|
Source: `orient/hill-signal.md` (area-level, at L1a) then the board + open SPIKE/contract state
|
|
74
|
-
(slice-level, L1b onward). If slice IDs aren't on the board yet
|
|
74
|
+
(slice-level, L1b onward). If slice IDs aren't on the board yet, report at
|
|
75
75
|
task-group level and note the fallback here.
|
|
76
76
|
|
|
77
77
|
## Decisions log
|
|
@@ -116,7 +116,7 @@ Decision owner: PO.
|
|
|
116
116
|
|
|
117
117
|
Lives at `shapeup/<slug>/round-ledger.md` (SHARED root, tracked). Not a replacement
|
|
118
118
|
for `harness-run.md` — a small, committed **subset** of it: the two things that must survive
|
|
119
|
-
a `.shapeup/` wipe or a crash
|
|
119
|
+
a `.shapeup/` wipe or a crash. Absent on specs with no scope
|
|
120
120
|
contracts; `harness-run.md`'s existing Decisions log stays the only ledger there.
|
|
121
121
|
|
|
122
122
|
```yaml
|
|
@@ -142,9 +142,9 @@ budgets:
|
|
|
142
142
|
| 2 | cart-creation | design-decision | cart-total rounding | round half-up | PO (interactive) |
|
|
143
143
|
| 2 | cart-creation | substrate-expansion | needs packages/shared/http.ts | approved → shared_substrate | PO (interactive) |
|
|
144
144
|
```
|
|
145
|
-
**Promotion timing:** a row is appended the INSTANT
|
|
145
|
+
**Promotion timing:** a row is appended the INSTANT the PO answers —
|
|
146
146
|
never batched to round close. This is the file `task-executor`'s isolated briefs read back
|
|
147
|
-
(zero-memory handoff
|
|
147
|
+
(the zero-memory handoff): an answer given once in round 2 must still be known in round 5's
|
|
148
148
|
fresh-context attempt without replaying any chat history.
|
|
149
149
|
|
|
150
150
|
---
|
|
@@ -156,16 +156,16 @@ FAIL-loop, and QA reconcile; worthless after ship. At SHIP the tech-lead **harve
|
|
|
156
156
|
the durable-mineable *signals* out of it into one append-only row:
|
|
157
157
|
|
|
158
158
|
```
|
|
159
|
-
shapeup/metrics/<machine-id>.jsonl # one row = one e2e run;
|
|
159
|
+
.shapeup/metrics/<machine-id>.jsonl # one row = one e2e run; LOCAL (gitignored)
|
|
160
160
|
```
|
|
161
161
|
|
|
162
|
-
Path note: the
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
`cat shapeup/metrics/*.jsonl`.
|
|
168
|
-
skill versions.
|
|
162
|
+
Path note: `metrics/` lives under the LOCAL root with the rest of the run trace — moved
|
|
163
|
+
there by ADR-0001, because a committed shard keyed on `$HOSTNAME` only grows and puts a
|
|
164
|
+
person's machine name in the repo. It outlives any single run (the per-slug run dirs are
|
|
165
|
+
superseded run by run; the shard only appends), which is what makes it the durable signal
|
|
166
|
+
feed. Sharded per machine so shards can be pooled deliberately without colliding on one
|
|
167
|
+
filename; the read plane is `stats.mjs`, or `cat .shapeup/metrics/*.jsonl`.
|
|
168
|
+
`schema_version` makes an old row readable by later skill versions.
|
|
169
169
|
|
|
170
170
|
### Two hard rules (same discipline as the Test Surface: *derived, never invented*)
|
|
171
171
|
1. **Harvest only fields that already exist as structured output at ship time.** If a
|
|
@@ -175,9 +175,8 @@ skill versions.
|
|
|
175
175
|
single-judge rule and invites Goodhart. The eval suite *interprets* downstream;
|
|
176
176
|
harvest *records*.
|
|
177
177
|
|
|
178
|
-
Scope: the harvest feeds **only
|
|
179
|
-
|
|
180
|
-
fixtures and do not consume it.
|
|
178
|
+
Scope: the harvest feeds **only the e2e pipeline measurement**. The per-skill functional
|
|
179
|
+
fixtures run in isolation and do not consume it.
|
|
181
180
|
|
|
182
181
|
### Row schema (one JSON object per line)
|
|
183
182
|
| Field | Existing source (copied, never re-graded) | Signal |
|
|
@@ -192,7 +191,7 @@ fixtures and do not consume it.
|
|
|
192
191
|
| `scope_cut_count` | `~` items cut at SHIP S.0 | appetite pressure / scope hammer |
|
|
193
192
|
| `qa_findings` | `.shapeup/<slug>/qa/hunt-report.md` + triage → `{total, promoted, held}` | edge quality |
|
|
194
193
|
| `slice_count` | breadboard B5 (≤9) | **normalizer / denominator** |
|
|
195
|
-
| `sources` | path to each **SHARED** source artifact — never a LOCAL `.shapeup/` path (
|
|
194
|
+
| `sources` | path to each **SHARED** source artifact — never a LOCAL `.shapeup/` path (the run-trace is superseded run by run, so a LOCAL path dangles by the time anyone reads the row; SHARED paths resolve on any clone — tier-direction rule) | auditability |
|
|
196
195
|
|
|
197
196
|
- `slice_count` is the **denominator**: `round_count=4` on a 2-slice feature is alarming,
|
|
198
197
|
on a 9-slice feature is normal. Without it, e2e comparisons are apples-to-oranges.
|