@metasession.co/devaudit-cli 0.2.0 → 0.3.0

package/sdlc/files/_common/skills/sdlc-implementer/SKILL.md

@@ -409,6 +409,15 @@ Reached only on the **tracked** route from Phase 0 (the issue is already fetched
    - **Requirements deviation** (an AC is wrong, incomplete, or missing) — trigger the [requirements gap flow](#requirements-gap-flow-devaudit-installer212). Do NOT just note it as a plan deviation. The AC table must be updated, `requirements-aligner` must be re-invoked to re-check SRS alignment, and the RTM must be updated.
    
    The classification question: "Did the implementation deviate from the plan's approach (how to build it), or did the plan's requirements deviate from reality (what to build)?" If the former, note and continue. If the latter, trigger the requirements gap flow.
+
+4b. **Reconcile test-plan.md file paths with actual files (devaudit-installer#241).** After writing/updating tests (both unit and E2E), diff the actual test file paths against `compliance/evidence/REQ-XXX/test-plan.md`. The test plan was authored during Stage 1 with predicted file paths — during implementation, tests are often added to existing files instead of creating new ones. This is a natural drift, but it must be reconciled before committing so `validate-compliance-artifacts.sh` doesn't fail at PR time.
+
+   For each file path referenced in `test-plan.md`:
+   - If the file exists on disk → OK, no action needed.
+   - If the file does not exist → check whether a test covering the same AC was added to a different file. If so, update `test-plan.md` to reference the actual file path.
+   - If the file does not exist and no equivalent test was found → **HALT**: "test-plan.md references <file> but no test file exists and no equivalent test covering the same AC was found. Either create the test file, update test-plan.md to point to the actual test file, or remove the entry if the AC is no longer relevant."
+
+   Commit the updated `test-plan.md` alongside the test code. This is a file-path reconciliation — it does not change the AC table (that's the step 5b plan ↔ test-scope AC consistency check, which is separate and checks AC drift, not file-path drift).
 5. **Run gates locally, cheap-first.** The gates are not equivalent-cost — `npm run lint` is seconds, `npx playwright test` is 30–60 minutes. Iterate on the fast gates; spend the e2e cost once.
 
    **Fast gates** (run on every change, ideally pre-commit):
@@ -428,7 +437,9 @@ Reached only on the **tracked** route from Phase 0 (the issue is already fetched
    git diff --name-only "$INTEGRATION_BRANCH"...HEAD -- 'app/**/*.tsx' 'src/**/*.tsx' 'pages/**/*.tsx' 'app/**/*.jsx' 'src/**/*.jsx' 'pages/**/*.jsx'
    ```
 
-   - **If UI-facing files are present:** check for `.e2e-gate-passed` sentinel file (written by `e2e-test-engineer` after a successful run) or `playwright-report/` directory with recent content. If neither exists, **HALT**: "E2E gate was not run. The change touches UI-facing files. Run `npx playwright test` (or invoke `e2e-test-engineer`) before committing. The pre-push hook will also block this push."
+   - **If UI-facing files are present:** check for `.e2e-gate-passed` sentinel file (written by `e2e-test-engineer` after a successful run) or `playwright-report/` directory with recent content. If neither exists, **HALT**: "E2E gate was not run. The change touches UI-facing files. Run `npx playwright test` (or invoke `e2e-test-engineer`) before committing. Do not defer to CI — install Playwright browsers with `npx playwright install` if needed. The pre-push hook will also block this push."
+
+     **Also check for `.e2e-evidence-wired` sentinel (devaudit-installer#226).** If `e2e/**/*.spec.ts` files were authored or modified, verify the `.e2e-evidence-wired` sentinel exists (written by `e2e-test-engineer` Phase 5½ after validating `tagTest()` and `evidenceShot()` calls). If missing, **HALT**: "Evidence wiring validation (Phase 5½) was not run. The change includes E2E spec files but no evidence wiring sentinel. Invoke `e2e-test-engineer` to validate `tagTest()` and `evidenceShot()` calls before committing. The pre-push hook will also block this push."
    - **If no UI-facing files (API-only, config, docs):** skip the check. Note the exemption in the commit body: "E2E gate skipped — no UI-facing files in this change."
    - **If `e2e-test-engineer` was invoked and determined e2e is not needed** (e.g. schema-only change): the skill writes `.e2e-gate-passed` with a `NOT_NEEDED` reason. The sentinel check passes. Note the exemption in the commit body: "E2E gate not needed — e2e-test-engineer assessed no UI surface (turn N)."
 
@@ -491,6 +502,14 @@ Reached only on the **tracked** route from Phase 0 (the issue is already fetched
    - The nil report uploads as `incident_report` evidence via `compliance-evidence.yml`'s `upload_incident_report` function, flipping `ISO29119.3.5.4` to COVERED for clean releases.
    - If incidents WERE closed, skip nil report generation — the populated incident report(s) from `incident-export.yml` serve as the evidence.
 
+5b. **Validate test-execution-summary.md gate states (devaudit-installer#240).** Before uploading evidence, verify that `compliance/evidence/REQ-XXX/test-execution-summary.md` does not contain invalid gate states. Run:
+
+   ```bash
+   bash scripts/validate-test-summary.sh origin/main
+   ```
+
+   If the validator fails, fix the summary before proceeding. E2E gate results must be one of: `PASS`, `FAIL`, `NOT_NEEDED` (with reason), or `SKIPPED` (with operator-approved rationale). The word "deferred" must never appear in `test-execution-summary.md` — not as a gate state, not in prose, not in final assessment. "Deferred to CI" and "Playwright browsers not installed locally" are environment issues, not gate states. The CI validator (`validate-test-summary.sh`) will reject any summary containing "deferred" or "browsers not installed" on PRs to main.
+
 6. **Organise artefacts** under `compliance/evidence/REQ-XXX/` with date-prefixed naming:
 
    ```

package/sdlc/files/ci/ci.yml.template

@@ -793,10 +793,35 @@ jobs:
                 # specs exist on disk. If specs are found but weren't run in
                 # this tier, emit a non-blocking warning instead of an error.
                 SPECS_ON_DISK=$(grep -rl --include='*.spec.ts' -e "@requirement ${REQ}" -e "tagTest.*${REQ}" -e "'${REQ}'" e2e/ 2>/dev/null | wc -l)
+
+                # devaudit-installer#237 — also scan unit test files for
+                # @requirement REQ-XXX annotations. A REQ with thorough unit
+                # test coverage but no E2E specs (legitimate for API-only or
+                # backend-only changes) should not be hard-blocked.
+                UNIT_TESTS_ON_DISK=$(grep -rl --include='*.test.ts' --include='*.test.tsx' -e "@requirement ${REQ}" -e "'${REQ}'" e2e/ __tests__/ tests/ src/ 2>/dev/null | wc -l)
+
+                # devaudit-installer#237 — check for test-execution-summary.md
+                # as a secondary signal. This is a human-authored document and
+                # weaker than machine-generated evidence, so it only converts
+                # a hard error to a warning when combined with unit test files
+                # on disk — never alone.
+                HAS_SUMMARY=false
+                if [ -f "compliance/evidence/${REQ}/test-execution-summary.md" ]; then
+                  HAS_SUMMARY=true
+                fi
+
                 if [ "$SPECS_ON_DISK" -gt 0 ]; then
+                  # Scenario B: E2E specs exist on disk but outside the smoke tier
                   echo "::warning::Evidence completeness gate: ${REQ} has zero tagged tests in this smoke run and zero per-AC screenshots, but ${SPECS_ON_DISK} spec file(s) on disk reference it (outside the smoke project). Evidence exists but was not executed in this CI tier. Consider running --project=critical or --grep \"${REQ}\" to capture evidence."
+                elif [ "$UNIT_TESTS_ON_DISK" -gt 0 ] && [ "$HAS_SUMMARY" = "true" ]; then
+                  # Scenario C: unit test files exist with @requirement annotations
+                  # and a test-execution-summary.md documents coverage
+                  echo "::warning::Evidence completeness gate: ${REQ} has zero E2E tagged tests and zero screenshots, but ${UNIT_TESTS_ON_DISK} unit test file(s) on disk reference it and a test-execution-summary.md exists. Unit test evidence found but execution was not verified in this run. If this is an API-only or backend-only change, this is expected."
                 else
-                  echo "::error::Evidence completeness gate: ${REQ} has zero tagged tests AND zero per-AC screenshots. The release cannot proceed to UAT with no traceable evidence. Ensure specs carry @requirement ${REQ} annotations and evidenceShot() calls (skill Phase 5½, issue #170)."
+                  # Scenario A: genuinely no evidence — no E2E specs, no unit tests,
+                  # no test-execution-summary. Hard error preserved (#237 Constraint 3).
+                  echo "::error::Evidence completeness gate: ${REQ} has zero traceable evidence — no tagged E2E tests, no per-AC screenshots, no unit test files with @requirement annotations, and no test-execution-summary.md. The release cannot proceed to UAT with no traceable per-REQ evidence."
+                  echo "::error::If E2E specs exist for this REQ, ensure they carry @requirement ${REQ} JSDoc annotations and tagTest() calls (e2e-test-engineer Phase 5½, issue #170). If unit tests exist, add @requirement ${REQ} annotations to the .test.ts files. If this is a pure config/docs change with no testable surface, document it in test-execution-summary.md."
                   EVIDENCE_GAPS=$((EVIDENCE_GAPS + 1))
                 fi
               else

package/sdlc/files/ci/compliance-validation.yml.template

@@ -30,5 +30,8 @@ jobs:
       - name: Validate compliance artifacts
         run: bash scripts/validate-compliance-artifacts.sh origin/main
 
+      - name: Validate test execution summaries
+        run: bash scripts/validate-test-summary.sh origin/main
+
       - name: Validate commit conventions
         run: bash scripts/validate-commits.sh origin/main

package/sdlc/files/stacks/node/hooks/pre-push

@@ -30,6 +30,8 @@ echo "Pre-push: TypeScript check passed."
 # Only fires if UI-facing files are in the push. Checks for either
 # playwright-report/ directory or .e2e-gate-passed sentinel (written by
 # e2e-test-engineer after a successful run).
+# Also checks .e2e-evidence-wired sentinel if e2e spec files changed
+# (written by e2e-test-engineer Phase 5½ after validating tagTest/evidenceShot).
 INTEGRATION_BRANCH=$(jq -r '.integration_branch // "develop"' sdlc-config.json 2>/dev/null || echo "develop")
 
 # Read stdin to get the pushed refs (husky pre-push protocol)
@@ -70,6 +72,23 @@ while read -r local_ref local_sha remote_ref remote_sha; do
     fi
     echo "Pre-push: E2E evidence check passed."
   fi
+
+  # Check for E2E spec file changes — require .e2e-evidence-wired sentinel
+  SPEC_FILES=$(git diff --name-only "$RANGE" -- 'e2e/**/*.spec.ts' 2>/dev/null || true)
+
+  if [ -n "$SPEC_FILES" ]; then
+    if [ ! -f .e2e-evidence-wired ]; then
+      echo ""
+      echo "ERROR: Evidence wiring validation (Phase 5½) was not run."
+      echo "       E2E spec files changed in this push:"
+      echo "$SPEC_FILES" | sed 's/^/         /'
+      echo ""
+      echo "       Invoke e2e-test-engineer to validate tagTest() and evidenceShot() calls."
+      echo "       Bypass with --no-verify (last resort, not a habit)."
+      exit 1
+    fi
+    echo "Pre-push: E2E evidence wiring check passed."
+  fi
 done
 
 # ── 3. Skill-invocation sentinel (devaudit-installer#226) ────────────

package/sdlc/package.json

@@ -1,6 +1,6 @@
 {
   "name": "devaudit-sdlc",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "DevAudit SDLC CLI Engine — cross-agent terminal-driven SDLC orchestrator.",
   "bin": {
     "devaudit-sdlc": "./src/bin/devaudit-sdlc.js"

package/sdlc/src/blueprints/2-implement-and-test.raw.md

@@ -174,6 +174,18 @@ If prerequisites are missing, do **not** start the full local suite. Run the tar
 
 **Do NOT proceed** until the scoped E2E/test-plan checks are complete and any local limitations are called out.
 
+### Step 4b: Reconcile Test Plan (devaudit-installer#241)
+
+After writing/updating tests (both unit and E2E), verify that `test-plan.md` file paths still match reality. The test plan was authored during Stage 1 with predicted file paths — during implementation, tests are often added to existing files instead of creating new ones. This natural drift must be reconciled before committing so `validate-compliance-artifacts.sh` doesn't fail at PR time.
+
+For each file path referenced in `test-plan.md`:
+
+- If the file exists on disk → OK, no action needed.
+- If the file does not exist → check whether a test covering the same AC was added to a different file. If so, update `test-plan.md` to reference the actual file path.
+- If the file does not exist and no equivalent test was found → **STOP**: "test-plan.md references `<file>` but no test file exists and no equivalent test covering the same AC was found. Either create the test file, update test-plan.md to point to the actual test file, or remove the entry if the AC is no longer relevant."
+
+Commit the updated `test-plan.md` alongside the test code. This is a file-path reconciliation — it does not change the AC table (the plan ↔ test-scope AC consistency check in the skill handles AC drift separately).
+
 ### Step 5: Stage Selectively
 
 ```bash

package/sdlc/.claude/settings.local.json

@@ -1,11 +0,0 @@
-{
-  "permissions": {
-    "allow": [
-      "Bash(git init:*)",
-      "Bash(git add:*)",
-      "Bash(git branch:*)",
-      "Bash(git commit:*)",
-      "Bash(gh repo:*)"
-    ]
-  }
-}