shapeup-sdlc 1.6.2 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/.claude-plugin/plugin.json +1 -1
  2. package/AGENTS.md +36 -103
  3. package/README.md +60 -67
  4. package/SECURITY.md +31 -16
  5. package/bin/init.mjs +24 -69
  6. package/commands/build.md +19 -0
  7. package/commands/eval.md +5 -4
  8. package/commands/scopes.md +5 -4
  9. package/commands/shape.md +1 -1
  10. package/commands/ship.md +37 -4
  11. package/hooks/anti-rationalization.mjs +7 -13
  12. package/hooks/compact-snapshot.mjs +5 -5
  13. package/hooks/gate-deadline.mjs +12 -16
  14. package/hooks/gate-intake.mjs +13 -14
  15. package/hooks/gate-l2.mjs +3 -3
  16. package/hooks/gate-zerowork.mjs +97 -16
  17. package/hooks/lib/decision.mjs +21 -13
  18. package/hooks/safety-spine.mjs +1 -1
  19. package/hooks/sandbox-guard.mjs +83 -47
  20. package/hooks/session-rehydrate.mjs +8 -9
  21. package/hooks/slop-cleaner.mjs +1 -1
  22. package/oracles/_shared.mjs +1 -1
  23. package/oracles/http-oracle.mjs +1 -1
  24. package/oracles/index.mjs +1 -1
  25. package/oracles/process-oracle.mjs +1 -1
  26. package/oracles/snapshot-oracle.mjs +1 -1
  27. package/oracles/test-oracle.mjs +1 -1
  28. package/package.json +3 -7
  29. package/skills/ba-pitch-analyzer/SKILL.md +20 -15
  30. package/skills/ba-pitch-analyzer/references/doc-schemas.md +1 -1
  31. package/skills/ba-pitch-analyzer/references/task-generation.md +6 -6
  32. package/skills/ba-pitch-analyzer/references/test-surface.md +1 -1
  33. package/skills/ba-pitch-analyzer/scripts/board-derive.mjs +5 -5
  34. package/skills/ba-pitch-analyzer/scripts/spec-lint.mjs +2 -2
  35. package/skills/coach/SKILL.md +3 -3
  36. package/skills/orient/SKILL.md +2 -1
  37. package/skills/qa-edge-hunter/SKILL.md +9 -39
  38. package/skills/scope-architect/SKILL.md +17 -12
  39. package/skills/scope-hammer/SKILL.md +5 -5
  40. package/skills/shapeup/SKILL.md +3 -3
  41. package/skills/shapeup/resources/context-compaction.md +4 -3
  42. package/skills/solution-architect/SKILL.md +18 -11
  43. package/skills/spec-evaluator/SKILL.md +12 -5
  44. package/skills/spec-evaluator/references/dimensions/_registry.md +2 -2
  45. package/skills/spec-evaluator/references/dimensions/completeness.md +1 -1
  46. package/skills/spec-evaluator/references/dimensions/integration.md +98 -67
  47. package/skills/spec-evaluator/references/dimensions/spec-conformance.md +4 -4
  48. package/skills/spec-evaluator/references/dimensions/test-surface-conformance.md +1 -1
  49. package/skills/spec-evaluator/references/probing.md +42 -23
  50. package/skills/spec-evaluator/references/report-schema.md +2 -2
  51. package/skills/spec-evaluator/scripts/verdict-ledger.mjs +1 -1
  52. package/skills/task-executor/SKILL.md +25 -12
  53. package/skills/tech-lead/SKILL.md +115 -438
  54. package/skills/tech-lead/references/delegation.md +38 -31
  55. package/skills/tech-lead/references/gates.md +75 -33
  56. package/skills/tech-lead/references/hard-rules.md +34 -0
  57. package/skills/tech-lead/references/invocation.md +2 -2
  58. package/skills/tech-lead/references/ledger-schema.md +17 -18
  59. package/skills/tech-lead/references/round-protocol.md +77 -27
  60. package/skills/tech-lead/references/state-model.md +1 -1
  61. package/skills/tech-lead/schemas/domain.schema.json +320 -148
  62. package/skills/tech-lead/schemas/gate-answers.schema.json +2 -2
  63. package/skills/tech-lead/schemas/work-result.schema.json +55 -17
  64. package/skills/tech-lead/scripts/aegis-digest.mjs +1 -1
  65. package/skills/tech-lead/scripts/budget-check.mjs +7 -13
  66. package/skills/tech-lead/scripts/compile-order.mjs +69 -22
  67. package/skills/tech-lead/scripts/fit-check.mjs +10 -10
  68. package/skills/tech-lead/scripts/gate-answers.mjs +11 -12
  69. package/skills/tech-lead/scripts/hill-derive.mjs +133 -0
  70. package/skills/tech-lead/scripts/ingest-result.mjs +7 -19
  71. package/skills/tech-lead/scripts/init-run.mjs +60 -18
  72. package/skills/tech-lead/scripts/lib/argv.mjs +1 -1
  73. package/skills/tech-lead/scripts/lib/contract-md.mjs +10 -10
  74. package/skills/tech-lead/scripts/lib/is-main.mjs +11 -12
  75. package/skills/tech-lead/scripts/lib/paths.mjs +15 -13
  76. package/skills/tech-lead/scripts/resume-state.mjs +387 -0
  77. package/skills/tech-lead/scripts/run-snapshot.mjs +1 -6
  78. package/skills/tech-lead/scripts/run-workflow.mjs +381 -0
  79. package/skills/tech-lead/scripts/ship-report.mjs +33 -2
  80. package/skills/tech-lead/scripts/stats.mjs +9 -10
  81. package/skills/tech-lead/scripts/t0-verify.mjs +4 -4
  82. package/skills/tech-lead/scripts/trace-lint.mjs +2 -2
  83. package/skills/tech-lead/workflows/shapeup-run.js +911 -0
  84. package/skills/translator/SKILL.md +2 -2
  85. package/skills/advisor-protocol/SKILL.md +0 -171
@@ -9,9 +9,9 @@ command line overrides this table for a single run.
9
9
  |----|---------|------|----------------------|-------|
10
10
  | `spec-conformance` | ✅ true | `dimensions/spec-conformance.md` | all tasks | baseline correctness: AC + Done-when + contract shapes + non-go |
11
11
  | `tdd-surface` | ✅ true | `dimensions/tdd-surface.md` | all tasks | suite green + companion test files for new code; TDD-1/2 critical, TDD-3 advisory |
12
- | `integration` | ✅ true | `dimensions/integration.md` | `.be` and `.e2e` variants only | full-stack integration test + auth boundary + RLS-JWT pattern; runs only when variant matches |
12
+ | `integration` | ✅ true | `dimensions/integration.md` | `.be` and `.e2e` variants only | full-stack integration test + access boundary + enforcement-path integrity; stack read from the project profile, not hard-coded; runs only when variant matches |
13
13
  | `completeness` | ⚙️ auto | `dimensions/completeness.md` | all tasks (lens lite/standard) | auto-ON when spec has UC `## Invariants` (v2.8+); no-op on older specs |
14
- | `test-surface-conformance` | ⚙️ auto | `dimensions/test-surface-conformance.md` | all tasks (lens lite/standard) | auto-ON when spec has UC `## Test Surface` (v2.9+ or `--surface-only` retrofit); no-op otherwise. Report must list every TS row probed — qa-edge-hunter's negative-space input |
14
+ | `test-surface-conformance` | ⚙️ auto | `dimensions/test-surface-conformance.md` | all tasks (lens lite/standard) | auto-ON when spec has UC `## Test Surface` (v2.9+ or a retrofit-surface order); no-op otherwise. Report must list every TS row probed — qa-edge-hunter's negative-space input |
15
15
  | `security` | ⛔ false | `dimensions/security.md` | be / shared | stub — flip on when ready |
16
16
  | `performance` | ⛔ false | `dimensions/performance.md` | be / web | stub — flip on when ready |
17
17
 
@@ -25,7 +25,7 @@ criteria?"* — it grades the tasks that exist. It cannot see the task that was
25
25
  written. Shape Up's discovered-task philosophy says the real bulk of work is found while
26
26
  building; an invariant declared on a UC but never backed by a regression task is exactly
27
27
  that blind spot. This dimension converts "absence of a task" from invisible into a graded
28
- **GAP** — the judge surfaces it; the planner (a ba-pitch-analyzer reconcile/generate-board order) fills it.
28
+ **GAP** — the judge surfaces it; the planner (a ba-pitch-analyzer `reconcile` order) fills it.
29
29
  Judge never fills it itself.
30
30
 
31
31
  **Distinction held:** conformance = *the tasks that exist are correct*; completeness =
@@ -15,28 +15,27 @@ requires_browser: false
15
15
 
16
16
  **Why this exists.** Unit tests verify isolated logic. `spec-conformance` drives the
17
17
  running app and confirms AC at the surface. Neither catches the class of bug where each
18
- layer works in isolation but the *seam* is broken: wrong auth header forwarded, RLS policy
19
- not invoked because the transaction is missing, Prisma schema migration not applied in the
20
- test database, or a contract mismatch between web's api-client and the API controller.
21
- This dimension fills that seam.
22
-
23
- **What "system integration" means here** (scoped to the IVS ADR-005 stack):
24
- the full request pathweb api-client API controller service → repository (Prisma +
25
- RLS) database exercised by tests that use a real database, not mocked layers.
26
- An integration test that mocks `PrismaClient` or replaces the repository with an in-memory
27
- stub is a unit test wearing integration clothing and does not satisfy this dimension.
28
-
29
- **Project context (read AGENTS.md Section 2 rules 3 & 4 before probing):**
30
- - All user-scoped queries must run inside a transaction that injects the JWT claims so RLS
31
- fires (`SET LOCAL request.jwt.claims`). An integration test that queries without this
32
- transaction bypasses RLS and produces a false-positive PASS.
33
- - The connection string must point to the **transaction-mode pooler** (port 6543), not the
34
- session-mode direct connection (port 5432). Session mode allows `SET LOCAL` to leak.
35
- - Never trust a `userId` sent in the request body — derive it from the verified JWT.
18
+ layer works in isolation but the *seam* is broken: the wrong auth header forwarded, an
19
+ authorization policy never invoked because the call bypassed the layer that enforces it, a
20
+ schema migration not applied to the test database, or a contract mismatch between the client
21
+ and the handler it calls. This dimension fills that seam.
22
+
23
+ **What "system integration" means here:** the full request path client → entry point →
24
+ application logic persistenceexercised by tests that use the project's **real**
25
+ dependencies, not mocked layers. An integration test that mocks the database client or
26
+ replaces the repository with an in-memory stub is a unit test wearing integration clothing
27
+ and does not satisfy this dimension.
28
+
29
+ **Read the project's stack from the artifacts, not from this file.** The commands, file
30
+ patterns, and enforcement mechanisms differ per project: resolve them from
31
+ `project-profile.md` (`archetype`, `entry_point`), the scope contract's
32
+ `e2e_verification_fixtures`, and the spec's `contracts/`. This file states the invariant;
33
+ the "Probing notes" section carries worked examples of what it looks like in common stacks.
34
+ A project whose stack matches none of them still owes the invariant.
36
35
 
37
36
  **`.e2e` variant:** the e2e suite inherently exercises the full stack. INT-1 (integration
38
37
  test existence) maps to the e2e test file(s) added by the task. INT-2 and INT-3 apply if
39
- the e2e task introduces any auth-scoped endpoint.
38
+ the e2e task introduces any access-controlled operation.
40
39
 
41
40
  ---
42
41
 
@@ -44,47 +43,59 @@ the e2e task introduces any auth-scoped endpoint.
44
43
 
45
44
  ```yaml
46
45
  - id: INT-1
47
- statement: "At least one integration/e2e test exercises the feature's main flow end-to-end with a real database — no mocked repository or Prisma layer."
46
+ statement: "At least one integration/e2e test exercises the feature's main flow end-to-end against the project's real dependencies — no mocked data layer."
48
47
  probe: static + cmd
49
48
  evidence_required: true
50
49
  pass_rule: >
51
- Locate integration test files for this task's package (typically `*.integration.spec.ts`,
52
- `*.e2e-spec.ts`, or a `test/` directory at the package root). Grep for test cases that
53
- touch the happy-path scenario the task implements. Verify the test does NOT mock the
54
- data layer: `grep -rE 'jest\.mock|vi\.mock' <integration-test-file>` should return no
55
- hits on repository, Prisma, or database imports. Then run `pnpm --filter <pkg> test:e2e`
56
- (or the integration-specific script) and capture exit code + test count.
57
- Zero integration test files FAIL at `critical`. Any integration test that mocks the
58
- DB layer FAIL at `critical` (evidence: the jest.mock / vi.mock import path).
50
+ Locate the integration test files for this task's package resolve the pattern and the
51
+ runner command from the scope contract's `e2e_verification_fixtures`, else the project's
52
+ test scripts (a target named for integration/e2e rather than unit). Grep for test cases
53
+ that touch the happy-path scenario the task implements. Verify the test does NOT mock the
54
+ data layer: search the file for the project's mocking primitive and inspect what it
55
+ replaces. Mocking outbound third-party calls (HTTP clients, email, payment sandboxes) is
56
+ acceptable and often required; mocking the database client, the repository, or the
57
+ persistence connection defeats the dimension. Then run the integration suite and capture
58
+ exit code + test count. Zero integration test files → FAIL at `critical`. Any integration
59
+ test that mocks the data layer → FAIL at `critical` (evidence: the mock call and the
60
+ module path it replaces).
59
61
  source: code
60
62
 
61
63
  - id: INT-2
62
- statement: "An unauthorized-access scenario is covered: a request from a different user/org is rejected with the correct error (403 or empty set), exercised by an integration test."
64
+ statement: "An unauthorized-access scenario is covered: a request from an unentitled caller is rejected with the documented outcome, exercised by an integration test."
63
65
  probe: cmd + data
64
66
  evidence_required: true
65
67
  pass_rule: >
66
- Find the integration test that exercises the unauthorized path: a user from a different
67
- org (or an unauthenticated caller) attempts to read/write the resource this task
68
- introduces. The test asserts the correct rejection: 403 Forbidden, an empty result set,
69
- or an RLS-filtered response. If the resource is read-only to viewers (role_tag check),
70
- also verify a `viewer` role cannot mutate it. Evidence = the test case text + the run
71
- output showing it passes. No such test FAIL at `critical` — this is the gap that lets
72
- real authorization bugs slip through conformance testing.
73
- source: code
68
+ FIRST establish whether the feature has an access boundary at all: does the spec
69
+ (`usecases/` Error Cases, `contracts/`, the Non-Go list) document any caller who must NOT
70
+ reach this operation a different tenant/user/org, a lower-privilege role, an
71
+ unauthenticated caller? If the spec documents none and the operation is genuinely public,
72
+ record INT-2 as N/A with that citation and exclude it from the denominator; do not invent
73
+ an authorization requirement the spec does not state. Otherwise, find the integration test
74
+ exercising the unauthorized path and assert it checks the outcome the spec documents
75
+ (a rejection status, an empty result set, a filtered response). If the spec also
76
+ distinguishes read from write for a role, verify the read-only role cannot mutate.
77
+ Evidence = the test case text + the run output showing it passes. A documented boundary
78
+ with no such test → FAIL at `critical` — this is the gap that lets real authorization
79
+ bugs slip through conformance testing.
80
+ source: usecases
74
81
 
75
82
  - id: INT-3
76
- statement: "Integration tests inject JWT claims via the RLS transaction pattern (set_config inside a transaction) and target the transaction-mode pooler (port 6543)."
83
+ statement: "Integration tests exercise the same enforcement path production uses — the test setup does not bypass or elevate past the mechanism that enforces the boundary."
77
84
  probe: static
78
85
  evidence_required: true
79
86
  pass_rule: >
80
- Inspect the integration test setup (beforeAll / test helper): it must execute
81
- `SET LOCAL request.jwt.claims` (or the equivalent `$executeRaw\`select set_config(...)\``)
82
- inside a Prisma `$transaction` block before any user-scoped query. Also check the
83
- DATABASE_URL used in the test config: it must include port 6543 (transaction pooler),
84
- not 5432 (session mode). A test that queries without the JWT injection calls the DB as
85
- the app role without RLS and will PASS rows it should not see → FAIL at `major`.
86
- A test using port 5432 risks SET LOCAL leaking across pooled connections → FAIL at `major`.
87
- If both problems are present, report as two separate findings.
87
+ Applies only when INT-2 is in scope AND the project enforces the boundary in a layer a
88
+ test could bypass (database row-level policies, a middleware/guard chain, a tenant-scoped
89
+ client, a signed-context requirement). Identify that mechanism from the project profile,
90
+ the contracts, or the production wiring then inspect the integration test setup
91
+ (suite-level hooks, test helpers, test env config) and confirm it goes through the same
92
+ path: the same credential/claim injection, the same connection or client mode, the same
93
+ guard chain. A setup that connects as an administrative or unscoped principal, disables
94
+ the guard layer, or seeds state through a back door calls the system with privileges the
95
+ real caller never has, and every access assertion above it is a false positive → FAIL at
96
+ `major`. If the project enforces the boundary only in code the test already traverses
97
+ (no bypassable layer exists), record INT-3 as N/A with that reasoning. Report each
98
+ distinct bypass as its own finding.
88
99
  source: code
89
100
  ```
90
101
 
@@ -93,12 +104,16 @@ the e2e task introduces any auth-scoped endpoint.
93
104
  ## Threshold
94
105
 
95
106
  `no-critical` — INT-1 and INT-2 are `critical`: a task that cannot prove end-to-end
96
- correctness and cannot prove its authorization boundary is not integration-tested and is
97
- not done. INT-3 (JWT transaction pattern + pooler port) is `major`: incorrect plumbing in
98
- the test setup produces false-positive passes (the test appears to work without RLS firing),
99
- which is a confidence-undermining failure — but it is one step removed from observable
107
+ correctness, and cannot prove a documented authorization boundary, is not integration-tested
108
+ and is not done. INT-3 (enforcement-path integrity) is `major`: a bypassed enforcement layer
109
+ produces false-positive passes the test appears to work while the boundary is never
110
+ exercised — which is confidence-undermining, but it is one step removed from observable
100
111
  breakage and is reported as major rather than critical.
101
112
 
113
+ A criterion recorded N/A under its own rule (INT-2 with no documented boundary, INT-3 with
114
+ no bypassable layer) leaves the denominator, exactly like `tdd-surface`'s TDD-2 on `.e2e`.
115
+ N/A requires the citation that justifies it; "no auth here, probably" is not a citation.
116
+
102
117
  ---
103
118
 
104
119
  ## Bug template
@@ -106,24 +121,40 @@ breakage and is reported as major rather than critical.
106
121
  ```
107
122
  severity: [critical|major]
108
123
  criterion: [INT-1|INT-2|INT-3]
109
- location: <test file path | package/test/setup.ts | .env.test | file:line>
110
- repro: <e.g. "grep -rE 'jest.mock.*repository' test/" | "grep DATABASE_URL .env.test" | "pnpm --filter api test:e2e">
111
- expected: <e.g. "integration test exists targeting BoardRepository with real DB" | "DATABASE_URL ends with :6543/...">
112
- actual: <e.g. "no *.integration.spec.ts in apps/api/src/boards/" | "DATABASE_URL uses port 5432" | "vi.mock('../../infrastructure/BoardRepository')">
113
- fix_hint: <e.g. "add boards/boards.integration.spec.ts; use PrismaService with a real test DB; wrap queries in $transaction with set_config">
124
+ location: <test file path | test setup/helper | test env config | file:line>
125
+ repro: <the command or search that shows it — e.g. the integration suite command, or the grep that finds the mock>
126
+ expected: <e.g. "integration test covering <flow> against the real database" | "test setup injects the caller's own credentials">
127
+ actual: <e.g. "no integration test found for <flow>" | "repository module is mocked in the suite" | "setup connects as the admin principal">
128
+ fix_hint: <e.g. "add <flow> integration test using the real test database; route setup through the same guard chain production uses">
114
129
  ```
115
130
 
116
131
  ## Probing notes
117
132
 
118
- - Integration test scripts are often separate from unit tests: `test:e2e` / `test:int` /
119
- `test:integration` in package.json. Check all scripts, not just `test`.
120
- - For mocking detection: `grep -rn "jest\.mock\|vi\.mock" <test-file>` and inspect what
121
- is being mocked. Mocking HTTP clients, external APIs, or filesystem utilities is fine;
122
- mocking the Prisma client or the repository class defeats integration testing.
123
- - For port check: the integration DB URL is typically in `.env.test` or `jest.config.ts`
124
- (globalSetup). Check `DATABASE_URL` and `DIRECT_URL` separately some setups split them.
125
- - For `.e2e` tasks: `supertest` or `axios` against the running NestJS server counts as
126
- an integration test if it hits a real DB. Verify the test server's DB config, not just
127
- the production server's config.
128
- - Do not skip INT-3 because the test "seems to work." RLS without `set_config` silently
129
- returns all rows as the app role the test passes but the auth boundary is untested.
133
+ - **Find the suite before judging it.** Integration targets are usually separate from unit
134
+ tests (`test:e2e`, `test:int`, `test:integration`, a tagged subset, a separate config).
135
+ Check every script the project defines, not just the default `test` "no integration
136
+ tests" is a FAIL, but "I only ran the unit target" is a probing error, not a finding.
137
+ - **Mock detection is about the target, not the call.** Find the project's mocking primitive
138
+ (`jest.mock`/`vi.mock`, a DI container override, a fake registered in a test module,
139
+ a stubbed client factory) and inspect *what it replaces*. Outbound third parties: fine.
140
+ The data layer: fatal to the dimension.
141
+ - **Worked example row-level policies with claim injection.** Where the database enforces
142
+ access (e.g. Postgres RLS), the enforcement is a per-connection claim set inside a
143
+ transaction; a test that queries outside that transaction runs as the application role and
144
+ silently returns rows the real caller could never see. INT-3 there means checking the setup
145
+ performs the claim injection, and that the connection mode preserves it — a session-mode
146
+ connection can leak transaction-scoped settings across pooled connections where a
147
+ transaction-mode one does not. Both are INT-3 findings, reported separately.
148
+ - **Worked example — guard/middleware chains.** Where a framework enforces access in a guard,
149
+ interceptor, or middleware, the bypass is a test that constructs the handler directly, or
150
+ registers a test module with the guard omitted. INT-3 means the test goes through the
151
+ composed application, not around it.
152
+ - **Worked example — tenant-scoped clients.** Where scoping is a client constructed per
153
+ caller, the bypass is a shared admin/service client in the test setup. INT-3 means the test
154
+ builds its client the way a request does.
155
+ - **For `.e2e` tasks**, a test driving the running server over HTTP counts as an integration
156
+ test if it reaches a real database. Verify the *test server's* configuration, not the
157
+ production server's — they routinely differ, and that difference is where INT-3 hides.
158
+ - **Do not skip INT-3 because the test "seems to work."** An enforcement layer that is never
159
+ invoked returns the same green as one that passes. Absence of a bypass must be evidenced,
160
+ not assumed.
@@ -26,16 +26,16 @@ dimensions, off by default.
26
26
  - id: SC-AC
27
27
  statement: "Every UC Step, Error Case, and (when present) Invariant/Test-Surface row for the
28
28
  use case(s) this task/feature implements is satisfied."
29
- probe: cmd # most are cmd; [ui]/[data] criteria use those probes per GATE V1.2 classification
29
+ probe: cmd # most are cmd; [ui]/[data] criteria use those probes per the CONTRACT-phase classification
30
30
  evidence_required: true
31
31
  pass_rule: >
32
32
  Each criterion is drawn from usecases/UC-*.md (numbered ## Steps, ## Error Cases rows,
33
33
  ## Invariants [INV-NN] entries, ## Test Surface derived rows when present) for every UC
34
- named in this task's use_case_refs (or confirmed with the user at V1.1 when no local task
35
- file exists). Run each via its classified probe and PASS. Zero failing criterion.
34
+ named in this task's use_case_refs (or confirmed with the user at CONTRACT when no local
35
+ task file exists). Run each via its classified probe and PASS. Zero failing criterion.
36
36
  A stubbed/display-only implementation that satisfies a local task file's own AC wording but
37
37
  not the UC's actual behavior FAILS — the task's AC (if present) is a paraphrase read only
38
- for traceability (GATE V0.2b), never the grading text itself.
38
+ for traceability, never the grading text itself.
39
39
  source: usecases
40
40
 
41
41
  - id: SC-DONE-WHEN
@@ -21,7 +21,7 @@ requires_browser: true
21
21
  > Explicit `--dimensions ...,test-surface-conformance` always wins over the auto rule.
22
22
 
23
23
  **Why this exists.** `spec-conformance` grades the AC that were written; it does not
24
- *expand* them. The QA meeting (2026-06-11) located the gap: nobody systematically derives
24
+ *expand* them. The settled division of labor locates the gap: nobody systematically derives
25
25
  the test matrix — boundary values, negative cases, error-code coverage, no-go breaches —
26
26
  that the spec already implies. The BA now derives that matrix into each UC's
27
27
  `## Test Surface` (D1 Invariants · D2 Error Cases · D3 Contract shapes · D4 No-gos,
@@ -43,7 +43,7 @@ The probe loop per [ui] criterion:
43
43
  the source file:line where the handler/wiring fails
44
44
  ```
45
45
 
46
- ## Oracle dispatch (evaluation contract — Stage G)
46
+ ## Oracle dispatch (evaluation contract)
47
47
 
48
48
  Each criterion / Test-Surface row carries an `oracle` tag — the dispatch key that declares *how*
49
49
  this criterion is verified. **Dispatch on it to choose the probe mechanism; default to `ui` when
@@ -124,34 +124,53 @@ This is a static read; no running required. Record whether each key AC has a mat
124
124
 
125
125
  ## Integration probing (for `integration` dimension)
126
126
 
127
- **Finding integration test files:** Check for files matching `*.integration.spec.ts`,
128
- `*.e2e-spec.ts`, or a top-level `test/` directory in the package. Also inspect
129
- `package.json` scripts for `test:e2e`, `test:int`, or `test:integration` targets.
127
+ The commands below are shaped for a JS/TS monorepo because that is the most common case, not
128
+ because the dimension assumes one. Resolve the real test pattern and runner from the scope
129
+ contract's `e2e_verification_fixtures` and the project's own test scripts; the criteria in
130
+ `dimensions/integration.md` state the invariant each probe is collecting evidence for.
130
131
 
131
- **INT-1 (no mocks):** After locating integration test files for the task's feature:
132
+ **Finding integration test files:** look for a naming convention that separates integration
133
+ from unit tests (e.g. `*.integration.spec.ts`, `*.e2e-spec.ts`, a top-level `test/`
134
+ directory) and inspect the project's script targets for an integration/e2e entry —
135
+ `test:e2e`, `test:int`, `test:integration`, a tagged subset, or a separate runner config.
136
+ Check every target the project defines: "I only ran the unit suite" is a probing error, not
137
+ a finding.
138
+
139
+ **INT-1 (no mocked data layer):** after locating the integration tests for the task's
140
+ feature, find the project's mocking primitive and inspect **what it replaces**:
132
141
  ```bash
133
- grep -rn "jest\.mock\|vi\.mock" <integration-test-file>
142
+ grep -rn "jest\.mock\|vi\.mock" <integration-test-file> # adjust to the project's primitive
134
143
  ```
135
- Mocking HTTP clients, email senders, or third-party APIs is acceptable. Mocking
136
- `PrismaClient`, a repository class, or the database connection defeats integration testing.
137
- Then run the integration suite: `pnpm --filter <pkg> test:e2e` (or equivalent). Capture
138
- exit code and test count.
139
-
140
- **INT-2 (auth boundary):** Locate the test case exercising the unauthorized path. Inspect
141
- its setup: it should create a different user's JWT (or use no auth) and assert a 403 or
142
- empty-set response. Capture the test body and the run output as evidence.
143
-
144
- **INT-3 (RLS-JWT pattern + pooler port):**
144
+ Mocking outbound third parties (HTTP clients, email senders, payment sandboxes) is
145
+ acceptable. Mocking the database client, a repository class, or the persistence connection
146
+ defeats integration testing. Then run the integration target and capture exit code and test
147
+ count.
148
+
149
+ **INT-2 (access boundary):** first confirm from `usecases/` Error Cases, `contracts/`, or the
150
+ Non-Go list that the spec documents a caller who must not reach this operation. If it
151
+ documents none, record N/A with that citation. Otherwise locate the test case exercising the
152
+ unauthorized path and inspect its setup: it should present an unentitled caller (different
153
+ tenant/user/org, lower-privilege role, or no credentials) and assert the outcome the spec
154
+ documents. Capture the test body and the run output as evidence.
155
+
156
+ **INT-3 (enforcement-path integrity):** identify the layer that actually enforces the
157
+ boundary, then read the test setup — suite-level hooks, shared helpers, test env config — and
158
+ confirm it traverses that layer rather than around it. What to grep for depends on the
159
+ mechanism:
145
160
  ```bash
146
- # Check for set_config call in test setup or helper
147
- grep -rn "set_config\|request\.jwt\.claims" <test-setup-files>
161
+ # database row-level policies: is the caller's claim set injected, and in the right scope?
162
+ grep -rn "set_config\|jwt\.claims\|SET LOCAL" <test-setup-files>
163
+
164
+ # connection mode: transaction-scoped settings can leak across pooled connections in
165
+ # session mode, so the configured port/mode is part of the evidence
166
+ grep -rn "DATABASE_URL\|DIRECT_URL" <test env config>
148
167
 
149
- # Check connection string in test env
150
- grep "DATABASE_URL" .env.test apps/api/.env.test
168
+ # guard / middleware chains: does the suite build the composed app, or the handler alone?
169
+ grep -rn "createTestingModule\|overrideGuard\|overrideProvider" <test-setup-files>
151
170
  ```
152
- The `DATABASE_URL` must end with `:6543/...` (transaction-mode pooler). If no `.env.test`
153
- exists, check `jest.config.ts` or `globalSetup` for a `DATABASE_URL` override. A missing
154
- `set_config` call means queries run without RLS false-positive auth passing silently.
171
+ A setup that connects as an administrative or unscoped principal, omits the guard layer, or
172
+ seeds state through a back door calls the system with privileges the real caller never has —
173
+ every access assertion above it is a false positive. Report each distinct bypass separately.
155
174
 
156
175
  ## By task variant
157
176
  - `.be` — no browser (`--browser none`). Start the API; send contract-shaped requests;
@@ -17,12 +17,12 @@ bug_count: [N]
17
17
  browser_mode: cli | mcp | none
18
18
  evaluator: spec-evaluator v0.1
19
19
  eval_at: [ISO date]
20
- linked_docs: ["[[usecases/UC-Name]]", "[[scope-summary]]"] # + "[[tasks/TASK-NNN-slug]]" only when a local task file exists this run (V0.2b)
20
+ linked_docs: ["[[usecases/UC-Name]]", "[[scope-summary]]"] # + "[[tasks/TASK-NNN-slug]]" only when a local task file exists this run
21
21
  t0_citation: ~ | { scope_id: cart-creation, artifact: t0/verdicts/r2-a3.json, sha256: 1a2b3c… }
22
22
  ---
23
23
  ```
24
24
  `t0_citation` is `~` (not applicable) only when the spec folder has no scope contracts
25
- (pre-v0.3.0 spec). On any scoped spec it is required — GATE V0.7/V2.5 in SKILL.md — an absent
25
+ (pre-v0.3.0 spec). On any scoped spec it is required — SKILL.md's T0-citation rule — an absent
26
26
  citation there means the report should not have been written yet.
27
27
 
28
28
  ## Body sections (in order)
@@ -1,5 +1,5 @@
1
1
  #!/usr/bin/env node
2
- // Verdict-ledger reference implementation (audit Stage D1) — spec-evaluator's own skill-local
2
+ // Verdict-ledger reference implementation — spec-evaluator's own skill-local
3
3
  // reference impl (ships beside SKILL.md; not invoked at runtime).
4
4
  //
5
5
  // The `spec-evaluator` skill performs verdict re-probe / confidence / flip-detection as a
@@ -26,6 +26,7 @@ rely on (anything absent = **unknown**; never invent it):
26
26
  | `substrate.allowed` / `substrate.shared` | The ONLY globs you may write. A needed file outside them → ESCALATE, never a write (a sandbox hook blocks it anyway) |
27
27
  | `payload.decisions[]` | Adjudicated answers from prior escalations — binding precedent, apply them |
28
28
  | `payload.digested_errors[]` | `{file, line, core_message}` triples from the previous attempt's failed verification — your starting bug list |
29
+ | `payload.trial_history[]` | Up to 8 prior attempts on this scope, oldest first, CROSSING the round boundary: `{score, status, delta, digest}`. `status: "reverted"` is a change that was tried and made things WORSE — do not re-propose it. `status: "kept"` with a still-red score is the tree you are building ON, not a failure to undo. Absent on the first attempt |
29
30
  | `payload.verify.test_cmd` | The command that verifies your work. No test_cmd → command-verifiable ACs still need *some* observable check; say what you used |
30
31
  | `payload.kb_rules_path` | Team guidelines (read if the file exists) — steering, never spec; conflict → the AC wins, note it in `deviations` |
31
32
  | `payload.constraints` | Non-Go items and freezes (e.g. `ui_layers.layer3_frozen`) |
@@ -104,28 +105,41 @@ no recorded evidence is reported `fail`, by your own hand.
104
105
 
105
106
  ---
106
107
 
107
- ## ESCALATE protocol — the one outward port
108
+ ## ESCALATE protocol — how to stop without guessing
108
109
 
109
110
  When blocked on a decision that is not yours (design decision, spec ambiguity, substrate
110
111
  expansion): stop work on **that AC only**, keep building every AC that doesn't depend on the
111
- answer, and add a block to `escalates[]`:
112
+ answer, then return `status: "escalated"` with the blocker as the **first** entry in
113
+ `deviations[]`, in this shape (one entry per blocked decision, blockers first):
112
114
 
113
- ```json
114
- { "kind": "design-decision | spec-ambiguity | substrate-expansion",
115
- "question": "one checkable question",
116
- "blocked_ac": "which AC waits on this",
117
- "context": "the two interpretations / the file and why it's needed" }
118
115
  ```
116
+ ESCALATE <design-decision | spec-ambiguity | substrate-expansion> [<blocked AC>]
117
+ <one checkable question>
118
+ <the two interpretations, or the file you need and why>
119
+ ```
120
+
121
+ `deviations[]` is the channel because there is **no `escalates[]` field** — the WorkResult
122
+ envelope carries no structured escalation, so a question written anywhere else does not reach a
123
+ human at all. `status: "escalated"` is what the run notices: the phase leaves its artifact
124
+ unwritten, the orchestrator's post-condition fails, and the run stops and names the phase
125
+ instead of looping. Your `deviations[]` text is the only record of *why*, so make it answerable
126
+ on its own — whoever reads it will not have your context.
119
127
 
120
128
  Orchestrated (`mode: orchestrated`): never ask an ad hoc question — there is no session to
121
- answer it; the orchestrator adjudicates and the answer returns in the next order's
122
- `decisions[]`. Standalone (`mode: standalone`): you may ask the user directly (max 2
123
- questions) instead of escalating.
129
+ answer it. Standalone (`mode: standalone`): you may ask the user directly (max 2 questions)
130
+ instead of escalating.
124
131
 
125
132
  ---
126
133
 
127
134
  ## Output contract — the WorkResult
128
135
 
136
+ **Escalation rule.** If you return `status: "escalated"`, the **first** entry in `deviations[]`
137
+ must be the blocker: one specific, answerable question plus the context needed to answer it.
138
+ Nothing else in the envelope carries it — there is no `escalates[]` field — so a vague entry, or
139
+ the question buried under other notes, reaches the human as "something went wrong" and costs a
140
+ round. Write it so someone without your context can answer it in one reply.
141
+
142
+
129
143
  Write `.shapeup/<slug>/results/<order-suffix>.json` (mirror of the order path; slug and
130
144
  suffix come from `order_id`) matching `work-result.schema.json`, and print its path:
131
145
 
@@ -142,7 +156,6 @@ suffix come from `order_id`) matching `work-result.schema.json`, and print its p
142
156
  "evidence": "pnpm --filter api test → 14/14" } ] }
143
157
  ],
144
158
  "files_touched": [ { "path": "apps/api/src/orders/create.ts", "change": "created" } ],
145
- "escalates": [],
146
159
  "discoveries": [ { "marker": "+", "line": "empty-cart edge case unhandled" } ],
147
160
  "assumptions": [ "amount stored in minor units — inferred from schema" ],
148
161
  "deviations": []
@@ -167,7 +180,7 @@ orchestrator's `ingest-result.mjs` does all of that from your envelope.
167
180
  - [ ] Every interactive element built binds to a manifest `test_id` with `data-state`
168
181
  - [ ] No unused imports/symbols left behind by your changes
169
182
  - [ ] Assumptions and deviations are in the envelope, not in your head
170
- - [ ] Blocked ACs have an `escalates[]` block; unrelated ACs were still built
183
+ - [ ] Blocked ACs are named in a `deviations[]` ESCALATE entry with `status: "escalated"`; unrelated ACs were still built
171
184
  - [ ] The WorkResult validates against `work-result.schema.json` and its path was printed
172
185
 
173
186
  ---