@metasession.co/devaudit-cli 0.1.61 → 0.1.62

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@metasession.co/devaudit-cli",
-  "version": "0.1.61",
+  "version": "0.1.62",
   "description": "DevAudit CLI — installs, syncs, and operates the Metasession SDLC across consumer projects.",
   "type": "module",
   "bin": {
@@ -33,7 +33,7 @@
   },
   "dependencies": {
     "@clack/prompts": "^0.8.2",
-    "@metasession.co/devaudit-plugin-sdk": "^0.1.61",
+    "@metasession.co/devaudit-plugin-sdk": "^0.1.62",
     "ajv": "^8.20.0",
     "commander": "^12.1.0",
     "consola": "^3.2.3",

package/sdlc/files/_common/skills/e2e-test-engineer/SKILL.md

@@ -189,6 +189,30 @@ For **visual regression** specifically:
 
 Do additions, updates, and (approved) deletions in the same change so the suite stays internally consistent.
 
+### Phase 5½ — Evidence wiring validation
+
+Before running the suite (Phase 6), verify that the evidence traceability wiring is in place. A spec can pass Phase 6's "AC covered" check by having a correct assertion without ever calling `evidenceShot()` or tagging `@requirement` — producing zero portal evidence and letting the release reach UAT with no traceable screenshots (DevAudit-Installer #170, #169).
+
+**For each in-scope REQ and its ACs from Phase 2's scenario table:**
+
+1. **Check `@requirement` annotation.** Grep the authored/modified spec files for `@requirement REQ-XXX` tags. Every spec file that covers an in-scope REQ must carry at least one `@requirement REQ-XXX` annotation so the CI's `detect-req` step and the portal's evidence-by-requirement view can find it.
+
+2. **Check `evidenceShot()` calls.** For each UI spec covering an in-scope REQ, grep for at least one `evidenceShot(page, 'REQ-XXX', <ac>, ...)` call per AC. The call must be placed **at the assertion that proves the AC**, before any further interaction or navigation. API-only specs that don't have a visual surface are exempt — note the exemption in the test-execution-summary.
+
+3. **Check test title annotations.** Each test for an in-scope REQ must carry a `[REQ-XXX]` tag in the test title or `test.info().annotations` so the Playwright JSON reporter emits the REQ association. The portal uses this to map test results to requirements.
+
+**If any check fails:**
+
+Halt and report the gap to the user:
+> Evidence wiring incomplete for REQ-XXX:
+> - Missing `@requirement REQ-XXX` annotation in `e2e/<area>/foo.spec.ts`
+> - Missing `evidenceShot()` call for AC2 in `e2e/<area>/bar.spec.ts`
+> - Missing `[REQ-XXX]` tag in test title for "should filter by status"
+>
+> These must be fixed before running the suite — without them the portal will show zero screenshots and zero tagged tests for this REQ, and the CI gate (#169) will block the release.
+
+Do **not** proceed to Phase 6 until all gaps are resolved. The user may choose to skip an AC (e.g. API-only, transport-only) — that's valid, but it must be an explicit decision recorded in the test-execution-summary, not an omission.
+
 ### Phase 6 — Execute and report
 
 Run the suite. Strategy:

package/sdlc/files/_common/skills/sdlc-implementer/SKILL.md

@@ -59,6 +59,39 @@ The only pauses in the whole workflow are the explicitly-named checkpoints:
 
 Everything else is silent continuation. The rule is **opt-in-to-pause, not opt-out-of-pause**. If you find yourself stopping after a sub-skill's "Return to the running `sdlc-implementer` context" line and waiting for the operator to ask _"is anything happening?"_ — that is the bug this section exists to prevent. Keep going.
 
+## Scope-expansion halt gate (devaudit-installer#171)
+
+The change-request loop's scope-expansion halt rule (see [`references/change-request-loop.md`](./references/change-request-loop.md) §"If the change-request is fundamentally a different REQ") only fires during Phase 5 — after a UAT reviewer clicks "Request Changes" and the portal state is `uat_changes_requested`. In practice, scope-expanding requests also arrive from the user at any point after Phase 1:
+
+- During Phase 2 (implementation) — user spots a related improvement and asks the agent to add it
+- During Phase 3 (evidence compile) — user asks for an additional evidence artefact not in the plan
+- During Phase 4 (UAT submission) — user exercises the deployed app and asks for a behaviour change
+- Between UAT submission and approval — same scenario
+- After approval but before merge — user makes a last-minute request
+
+In all these cases, the portal state is NOT `uat_changes_requested`, so the change-request loop's halt rule never fires. The agent implements the out-of-scope change without flagging it.
+
+**This gate fires on every user work request while a REQ is active, in any phase.**
+
+When the user asks for a change that goes beyond the current REQ's acceptance criteria:
+
+1. **Read the current REQ's `test-scope.md`** (or `implementation-plan.md` § Acceptance Criteria if no test-scope exists yet) to identify the defined ACs.
+
+2. **Check whether the user's request maps to an existing AC.** If yes — proceed (or enter the change-request loop if in Phase 5).
+
+3. **If no — halt with the scope-expansion message:**
+
+   > This request adds behaviour outside REQ-XXX's acceptance criteria (AC1–ACn defined in test-scope.md). Recommend filing a separate issue for \<subject\> and shipping REQ-XXX as originally scoped.
+
+4. **Wait for the user to confirm one of:**
+   - **(a) File a separate issue** (new REQ) — ship the current REQ as-is. The agent continues with the original scope.
+   - **(b) Amend REQ-XXX's scope** — explicitly expand `test-scope.md` / `implementation-plan.md`, update the plan, invalidate existing evidence, and re-walk Stage 3. This option carries a warning: *"Amending scope after evidence is compiled (Stage 3+) invalidates the existing test-execution-summary, screenshots, and UAT verification. All Stage 3 evidence must be re-compiled."*
+   - **(c) Abandon the request** — do nothing, continue with the original scope.
+
+**Do not implement the out-of-scope change before the user picks (a), (b), or (c).** The inertia trap is real: the agent is mid-flow, the codebase is open, and the request sounds reasonable. The gate exists to interrupt that inertia — STOP, surface the scope gap, and let the user decide.
+
+This gate is distinct from Phase 2 step 4's "any deviation from the plan must be noted" rule: that covers implementation *approach* deviations (how to build something the plan says to build), not *scope* expansions (building something the plan doesn't say to build at all).
+
 ## SDLC navigability — LAST/NEXT status sticky (devaudit#131)
 
 Long-running SDLC issues accumulate dozens of comments across multiple Claude Code sessions. The operator returning to the thread should be able to answer two questions in under five seconds:

package/sdlc/files/ci/ci.yml.template

@@ -549,6 +549,8 @@ jobs:
           # compliance/pending-releases/. Failures now bump UPLOAD_FAILURES
           # so the step exits non-zero and the release PR turns red.
           SHOT_REQS=()
+          ZERO_Screenshot_REQS=()
+          EVIDENCE_GAPS=0
           if [ -d compliance/pending-releases ]; then
             for TICKET in compliance/pending-releases/RELEASE-TICKET-REQ-*.md; do
               [ -f "$TICKET" ] || continue
@@ -580,6 +582,12 @@ jobs:
               REQ_SHOTS=(ci-evidence/compliance/evidence/"$REQ"/screenshots/*.png)
               if [ "${#REQ_SHOTS[@]}" -eq 0 ]; then
                 echo "No per-AC screenshots for ${REQ} (none captured by evidenceShot this run)"
+                # devaudit-installer#169 — track REQs with zero screenshots
+                # so the evidence-completeness gate below can fail the job
+                # if the REQ also has zero tagged tests in the Playwright
+                # report. Previously this was a silent continue — the release
+                # proceeded to UAT with no traceable evidence.
+                ZERO_Screenshot_REQS+=("$REQ")
                 continue
               fi
               echo "Uploading ${#REQ_SHOTS[@]} screenshot(s) for ${REQ}"
@@ -626,6 +634,52 @@ jobs:
           fi
           shopt -u nullglob
 
+          # devaudit-installer#169 — Evidence-completeness gate.
+          # For each in-scope REQ that had zero screenshots, check whether
+          # the Playwright JSON report contains at least one test tagged
+          # with that REQ. If a REQ has zero screenshots AND zero tagged
+          # tests, the release has no traceable evidence and must not
+          # proceed to UAT. Previously this was a silent pass — the release
+          # arrived in UAT review with "SCREENSHOTS: none" and "TESTS: no
+          # tests in report tagged with this REQ" on the portal.
+          if [ "${#ZERO_Screenshot_REQS[@]}" -gt 0 ]; then
+            for REQ in "${ZERO_Screenshot_REQS[@]}"; do
+              TAGGED_COUNT=0
+              if [ -f ci-evidence/e2e-results.json ]; then
+                TAGGED_COUNT=$(python3 -c "
+          import json, sys
+          with open('ci-evidence/e2e-results.json') as f:
+            data = json.load(f)
+          def count(specs):
+            n = 0
+            for s in specs:
+              tags = []
+              for a in s.get('annotations', []):
+                tags.append(a.get('type','') + ':' + a.get('description',''))
+              for t in s.get('tags', []):
+                tags.append(t)
+              title = s.get('title', '')
+              if '${REQ}' in title or any('${REQ}' in t for t in tags):
+                n += 1
+              n += count(s.get('suites', []))
+            return n
+          print(count(data.get('suites', [])))
+          " 2>/dev/null || echo "0")
+              fi
+              if [ "$TAGGED_COUNT" -eq 0 ]; then
+                echo "::error::Evidence completeness gate: ${REQ} has zero tagged tests AND zero per-AC screenshots. The release cannot proceed to UAT with no traceable evidence. Ensure specs carry @requirement ${REQ} annotations and evidenceShot() calls (skill Phase 5½, issue #170)."
+                EVIDENCE_GAPS=$((EVIDENCE_GAPS + 1))
+              else
+                echo "${REQ} has zero screenshots but ${TAGGED_COUNT} tagged test(s) — evidence-completeness gate passed (screenshot gap is non-blocking)"
+              fi
+            done
+          fi
+
+          if [ "$EVIDENCE_GAPS" -gt 0 ]; then
+            echo "::error::${EVIDENCE_GAPS} in-scope REQ(s) have zero traceable evidence (no tagged tests, no screenshots) — release cannot pass UAT review"
+            exit 1
+          fi
+
           # NOTE: committed compliance docs (planning category: RTM/test-plan/
           # test-cases, release tickets, and per-requirement
           # compliance/evidence/REQ-*/ folders) are intentionally NOT uploaded

package/sdlc/files/ci/feature-e2e.yml.template

@@ -43,22 +43,72 @@ jobs:
     needs: detect-req
     if: needs.detect-req.outputs.has_tests == 'true'
     runs-on: {{RUNNER}}
+
+    services:
+      {{DATABASE_SERVICE}}:
+        image: {{DATABASE_IMAGE}}
+        ports:
+          - {{DATABASE_PORT}}
+
     env:
+{{DATABASE_ENV}}
+{{APP_ENV}}
       DEVAUDIT_BASE_URL_VAR: ${{ vars.DEVAUDIT_BASE_URL }}
       DEVAUDIT_API_KEY: ${{ secrets.DEVAUDIT_API_KEY }}
+
     steps:
       - uses: actions/checkout@v6
         with:
           fetch-depth: 0
       - uses: actions/setup-node@v4
         with:
-          node-version: '20'
-      - run: npm ci
-      - run: npx playwright install --with-deps
-      - name: Run in-scope E2E
+          node-version: {{NODE_VERSION}}
+      - name: Install dependencies (skip if lockfile unchanged)
         run: |
-          REQ_ID="${{ needs.detect-req.outputs.req_id }}"
-          npx playwright test --grep "$REQ_ID"
+          LOCK_HASH=$(sha256sum package-lock.json | cut -d' ' -f1)
+          if [ -f node_modules/.lock-hash ] && [ "$(cat node_modules/.lock-hash)" = "$LOCK_HASH" ]; then
+            echo "node_modules up to date — skipping npm ci"
+          else
+            npm ci
+            echo "$LOCK_HASH" > node_modules/.lock-hash
+          fi
+      - name: Install Playwright (skip if already cached)
+        run: npx playwright install --with-deps chromium 2>/dev/null || npx playwright install chromium
+
+{{DATABASE_URI_STEP}}
+{{E2E_SETUP_STEP}}
+      - name: Kill stale dev server
+        run: lsof -ti:3000 | xargs kill -9 2>/dev/null || true
+
+{{E2E_DEV_SERVER_STEP}}
+
+      - name: Wait for dev server
+        run: npx wait-on http://localhost:3000 --timeout 120000
+
+      # Compute the set of e2e spec files added on this branch (relative
+      # to the merge base). The evidenceShot helper reads E2E_NEW_SPECS
+      # at capture time and tags each screenshot `origin=feature` when
+      # its spec appears in the list, else `origin=regression`.
+      - name: Compute new-spec list for E2E origin tagging
+        run: |
+          BASE_REF="${{ github.base_ref || 'develop' }}"
+          BASE_SHA=$(git merge-base "origin/${BASE_REF}" HEAD 2>/dev/null || git rev-parse HEAD)
+          NEW_SPECS=$(git diff --name-only --diff-filter=A "${BASE_SHA}...HEAD" \
+            -- 'tests/e2e/**/*.spec.ts' 'tests/e2e/**/*.spec.tsx' \
+               'e2e/**/*.spec.ts' 'e2e/**/*.spec.tsx' 2>/dev/null || true)
+          {
+            echo "E2E_NEW_SPECS<<EOF"
+            echo "$NEW_SPECS"
+            echo "EOF"
+          } >> "$GITHUB_ENV"
+          if [ -n "$NEW_SPECS" ]; then
+            echo "New e2e specs on this branch (origin=feature for captures):"
+            echo "$NEW_SPECS" | sed 's/^/  /'
+          else
+            echo "No new e2e specs detected — all captures will tag origin=regression."
+          fi
+
+{{E2E_FEATURE_TEST_STEP}}
       - name: Upload feature E2E evidence (stage 2, origin=feature)
         if: always()
         run: |