@metasession.co/devaudit-cli 0.1.52 → 0.1.53

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@metasession.co/devaudit-cli",
-  "version": "0.1.52",
+  "version": "0.1.53",
   "description": "DevAudit CLI — installs, syncs, and operates the Metasession SDLC across consumer projects.",
   "type": "module",
   "bin": {
@@ -33,7 +33,7 @@
   },
   "dependencies": {
     "@clack/prompts": "^0.8.2",
-    "@metasession.co/devaudit-plugin-sdk": "^0.1.52",
+    "@metasession.co/devaudit-plugin-sdk": "^0.1.53",
     "commander": "^12.1.0",
     "consola": "^3.2.3",
     "env-paths": "^3.0.0",

package/sdlc/files/_common/Test_Architecture.md

@@ -39,7 +39,7 @@ These standards apply to all Metasession products, client engagements, and inter
 ### Speed over Exhaustiveness
 - Fast feedback prioritized (unit tests < 30 seconds)
 - Parallelization and sharding for E2E suites
-- Strategic test selection based on code changes
+- Strategic test selection based on code changes — first concrete implementation is the three-tier E2E gating model (smoke / critical / regression), see Test_Strategy.md § *E2E gating model* (v0.1.53+)
 - Regression suites optimized for execution time
 
 ### Traceability

package/sdlc/files/_common/Test_Policy.md

@@ -150,6 +150,18 @@ Testing effort is prioritized by risk level, determined at planning time:
 
 AI involvement in Medium or High categories raises risk by one level. The Test Strategy defines specific testing depth requirements per level.
 
+### E2E gate enforcement (v0.1.53+)
+
+The MoSCoW prioritisation of acceptance criteria maps onto three E2E gates, each enforced at a different point in the workflow:
+
+- **Must-tier ACs in the smoke subset** must pass on every push to the integration branch. Blocking — a red smoke gate stops the integration hop.
+- **Must-tier ACs in the critical subset** must pass on every PR to the release branch. Blocking — a red critical gate stops the release.
+- **Should/Could-tier ACs (full regression)** must pass on the next post-merge run to the release branch OR a hotfix issue is auto-filed. Not pre-merge blocking — the safety net is post-hoc triage by the operator within working hours.
+
+Operator override on a hotfix issue (accept-with-rationale) is logged on the issue itself + carried in the next release's `test-execution-summary.md` design record (devaudit#50). The framework does not permit silently shipping a failing test — every red regression spec ends as either fixed, reverted, or accepted-with-recorded-rationale.
+
+See Test_Strategy.md § *System Testing (E2E)* — *E2E gating model* for the tier definitions + cost philosophy.
+
 ---
 
 ## Roles & Responsibilities

package/sdlc/files/_common/Test_Strategy.md

@@ -38,6 +38,24 @@ Validates interactions between system components — API contracts, service inte
 
 End-to-end validation of complete user workflows from UI to database. Primary responsibility of the QA team. Automated using BDD frameworks that map acceptance criteria to executable specifications. Covers 100% of critical user paths.
 
+#### E2E gating model — three tiers (devaudit#152 follow-up, v0.1.53)
+
+Full E2E regression on every PR is expensive — a 30+ minute wait per release-PR blocks velocity for diminishing marginal safety once smoke covers the headline flows. The framework's gating model maps the existing MoSCoW prioritisation onto three tiers, each gated at a different point in the workflow:
+
+| Tier | Location | When it runs | Wall-clock target | Audit role |
+|---|---|---|---|---|
+| **smoke** | `e2e/smoke/*.spec.ts` | every push to `$INTEGRATION_BRANCH` (via `ci.yml`) | ~3–5 min | fast feedback on every change |
+| **critical** | `e2e/smoke/` + `e2e/critical/*.spec.ts` | PR-to-`$RELEASE_BRANCH` (via `e2e-regression.yml`) | ~10–15 min | release-readiness Must gate |
+| **regression** | all `e2e/**/*.spec.ts` | nightly + push-to-`$RELEASE_BRANCH` + `workflow_dispatch` | ~35 min (or your project's full pack) | full audit trail + drift catch |
+
+The mapping to MoSCoW: **Must-priority SRS items live in `e2e/smoke/` (fast feedback) and `e2e/critical/` (release gate); Should/Could items live in `e2e/` and are covered by the regression tier.** The classifier is the developer authoring the spec — see `skills/e2e-test-engineer/SKILL.md` Phase 3 for the decision tree.
+
+**Cost philosophy.** Smoke protects every push from breaking the headline flow. Critical protects every release from a Must-tier regression. Full regression protects the audit trail + catches drift overnight. We accept that a Should/Could-tier regression *can* slip past the PR gate; we catch it on the next post-merge run + auto-file a hotfix issue. The framework prefers this over a 35-min wait on every release because operator velocity matters and the safety net stays intact.
+
+**Post-merge safety net.** Every push to `$RELEASE_BRANCH` re-runs the full regression. On failure, `e2e-regression.yml` auto-files a `bug, priority:high` issue tagging the merge commit + the failing specs. The operator triages within working hours — hotfix forward, revert the commit, or accept-with-rationale if the failure is environmental. No automated revert (false positives + flakes + UAT-data drift are real classes; an operator triages each individually).
+
+**Reference workflow.** A copy-pasteable `e2e-regression.yml` shape lives at `skills/e2e-test-engineer/references/e2e-regression-3-tier.yml`. Adoption is opt-in per consumer (the framework doesn't currently sync this workflow; consumers own their own `e2e-regression.yml`).
+
 ### Acceptance Testing
 
 Validates that requirements and acceptance criteria are met from a business perspective. Conducted in staging environments mirroring production. Requires sign-off from Product Managers. May include formal UAT with stakeholders for regulated features.

package/sdlc/files/_common/skills/e2e-test-engineer/SKILL.md

@@ -131,6 +131,26 @@ Resist padding. A new endpoint doesn't need a test that re-verifies login if log
 
 For each scenario, write a one-line description. Present the full grouped list to the user before writing any code: *"Here's the coverage I'd propose — anything to add or drop?"*
 
+#### Classify each spec into a tier (devaudit#152 follow-up, v0.1.53)
+
+When designing each scenario, also pick the tier it'll live in. Three tiers map to MoSCoW priority + gating point (see `Test_Strategy.md` § *E2E gating model*):
+
+| Tier | File location | Picks this when… |
+|---|---|---|
+| **smoke** | `e2e/smoke/*.spec.ts` | Cross-cutting sanity that proves the app is up: login, basic nav, one canonical CRUD per main domain. Runs on every push to the integration branch. Keep small — total smoke wall-clock target is ~3–5 min. |
+| **critical** | `e2e/critical/*.spec.ts` | Must-priority SRS item that breaks a headline flow if it regresses. Examples: payment authorisation, order completion, admin permission editing, RBAC enforcement on financial surfaces. Runs on PR-to-release-branch. Total critical wall-clock target ~10–15 min (includes smoke). |
+| **regression** | `e2e/<area>/*.spec.ts` | Should/Could-priority SRS item, edge cases, less-load-bearing flows. Runs nightly + post-merge + dispatch. Total full pack can be 30+ min; that's the point of the tier. |
+
+Decision tree, applied per scenario:
+
+1. **Does the spec prove a Must-priority SRS AC (or a baseline "app is up" sanity check)?** → smoke or critical.
+2. **Within Must: would a regression here break a headline business flow visible to a paying customer or stop a release from shipping?** → critical. Otherwise → smoke.
+3. **Should/Could priority, edge case, advanced flow?** → regression (file under `e2e/<area>/`, not under `e2e/smoke/` or `e2e/critical/`).
+
+When you can't decide between critical and regression, default to **regression** — promoting a spec from regression → critical later is cheap (move the file); demoting in the other direction is rarely needed but equally cheap. The cost of putting a Should spec in critical is everyone waiting longer on every PR-to-main for a low-value signal.
+
+Record the tier choice in the eventual `test-execution-summary.md` § *Test design* (devaudit#50) — Layers covered should name which tier each new spec landed in. Reviewers verify the tier choice is defensible during the WAIT CHECKPOINT.
+
 ### Phase 4 — Reconcile with existing tests
 
 For the area touched by the change, look at what's already there.

package/sdlc/files/_common/skills/e2e-test-engineer/references/e2e-regression-3-tier.yml

@@ -0,0 +1,178 @@
+# Reference: three-tier E2E gating workflow (devaudit#152 follow-up, v0.1.53)
+#
+# Copy this into your consumer-owned .github/workflows/e2e-regression.yml
+# to adopt the 3-tier model: smoke (every develop push, fast) / critical
+# (PR-to-main, ~10-15 min target) / regression (nightly + push-to-main +
+# dispatch, full audit trail with auto-issue on failure).
+#
+# The framework does NOT sync this file automatically — your consumer
+# owns its e2e-regression.yml. Apply the patterns below to your own
+# file; keep any consumer-specific env / matrix / runner customisations.
+#
+# Tier definitions:
+#   - smoke   — runs on develop push via ci.yml (no change here)
+#   - critical — Playwright project that selects e2e/smoke/ + e2e/critical/
+#   - regression — Playwright project that selects all e2e/**/*.spec.ts
+#
+# playwright.config.ts must define the `critical` project for this to
+# fire; if it doesn't, the gate falls back to the existing `smoke`
+# project so PR-to-main stays green during migration.
+
+name: E2E Regression
+
+on:
+  pull_request:
+    branches: [main] # critical-tier gate before merge
+  push:
+    branches: [main] # full regression after merge; auto-issues on failure
+  schedule:
+    - cron: '0 2 * * *' # nightly full regression
+  workflow_dispatch:
+    inputs:
+      specs:
+        description: 'Optional: space-separated spec paths or --grep pattern for a scoped run. Empty = full regression.'
+        required: false
+
+permissions:
+  contents: read
+  issues: write # post-merge auto-issue on regression failure
+
+concurrency:
+  group: e2e-regression-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  e2e:
+    name: E2E Regression Tests
+    runs-on: ubuntu-latest # adapt to your runner; e.g. self-hosted, ubuntu-24.04
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # for E2E_NEW_SPECS computation
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '22' # match your project
+          cache: 'npm'
+
+      - name: Install dependencies
+        run: npm ci --legacy-peer-deps
+
+      - name: Install Playwright browsers
+        run: npx playwright install --with-deps chromium
+
+      # Decide which Playwright project to run based on the trigger.
+      # PR-to-main uses critical with smoke fall-back; push-to-main and
+      # schedule run the full regression project; workflow_dispatch
+      # accepts an optional spec filter.
+      - name: Determine E2E project + spec selector
+        id: select
+        run: |
+          set -euo pipefail
+          EVENT="${{ github.event_name }}"
+          case "$EVENT" in
+            pull_request)
+              if grep -qE "name:\s*['\"]critical['\"]" playwright.config.ts 2>/dev/null; then
+                echo "project=critical" >> "$GITHUB_OUTPUT"
+                echo "Using critical-tier project (smoke + e2e/critical/)"
+              else
+                echo "project=smoke" >> "$GITHUB_OUTPUT"
+                echo "::warning::No 'critical' Playwright project defined; falling back to smoke. See e2e-test-engineer/references/e2e-regression-3-tier.yml + the Phase 3 tier-classification guide."
+              fi
+              echo "specs=" >> "$GITHUB_OUTPUT"
+              ;;
+            push|schedule)
+              echo "project=regression" >> "$GITHUB_OUTPUT"
+              echo "specs=" >> "$GITHUB_OUTPUT"
+              echo "Running full regression project"
+              ;;
+            workflow_dispatch)
+              echo "project=regression" >> "$GITHUB_OUTPUT"
+              echo "specs=${{ github.event.inputs.specs }}" >> "$GITHUB_OUTPUT"
+              if [ -n "${{ github.event.inputs.specs }}" ]; then
+                echo "Scoped dispatch: ${{ github.event.inputs.specs }}"
+              fi
+              ;;
+          esac
+
+      - name: Run E2E suite
+        id: run
+        env:
+          PLAYWRIGHT_HTML_REPORTER_OPEN: never
+          PLAYWRIGHT_JSON_OUTPUT_NAME: e2e-regression-results.json
+          # Add your e2e_env values here as needed (DEVAUDIT_BASE_URL etc.)
+        run: |
+          set -euo pipefail
+          PROJECT="${{ steps.select.outputs.project }}"
+          SPECS="${{ steps.select.outputs.specs }}"
+          if [ -n "$SPECS" ]; then
+            npx playwright test --project="$PROJECT" --reporter=json,html $SPECS
+          else
+            npx playwright test --project="$PROJECT" --reporter=json,html
+          fi
+
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: e2e-regression-report
+          path: |
+            e2e-regression-results.json
+            playwright-report/
+            test-results/
+
+      # ─────────────────────────────────────────────────────────────
+      # Post-merge auto-issue on regression failure (push:branches:[main])
+      #
+      # Catches regressions that slipped past the critical-tier PR gate.
+      # Opens a high-priority issue tagging the merge commit + the
+      # failing specs so the operator can triage within working hours.
+      # No auto-revert — that's intentionally an operator decision.
+      # ─────────────────────────────────────────────────────────────
+      - name: Open hotfix issue on post-merge regression
+        if: failure() && github.event_name == 'push' && github.ref == 'refs/heads/main'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+          MERGE_SHA="${{ github.sha }}"
+          MERGE_SHA_SHORT=$(echo "$MERGE_SHA" | cut -c1-7)
+          RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+
+          # Extract failing spec names from the JSON reporter output if available.
+          FAILING=""
+          if [ -f e2e-regression-results.json ]; then
+            FAILING=$(jq -r '
+              [.. | objects | select(.status == "failed" or .status == "timedOut") | .title // empty]
+              | unique | .[]
+            ' e2e-regression-results.json 2>/dev/null | head -20 || true)
+          fi
+          if [ -z "$FAILING" ]; then
+            FAILING="(see the failing run logs — could not parse spec titles from reporter output)"
+          fi
+
+          BODY=$(cat <<EOF
+          ## Post-merge regression caught on \`main\`
+
+          The full regression suite failed on the post-merge run for commit \`${MERGE_SHA_SHORT}\`. The critical-tier PR gate let this slip through.
+
+          **Failing specs (best-effort extracted from the JSON reporter):**
+
+          \`\`\`
+          ${FAILING}
+          \`\`\`
+
+          **Triage actions:**
+
+          - [ ] Read the run log: ${RUN_URL}
+          - [ ] Pull \`e2e-regression-report\` artifact from the run; inspect \`test-results/<spec>/error-context.md\` for page state at failure
+          - [ ] Decide: hotfix on \`main\`, revert \`${MERGE_SHA_SHORT}\`, or accept-with-rationale if the failure is environmental
+          - [ ] If the failing spec is a Must-tier candidate that should have caught this pre-merge, move it from \`e2e/\` to \`e2e/critical/\` so the next PR-to-main runs it
+
+          **Auto-filed by:** \`e2e-regression.yml\` (devaudit#152 3-tier gating, v0.1.53+)
+          EOF
+          )
+
+          gh issue create \
+            --title "[hotfix] Post-merge regression on \`${MERGE_SHA_SHORT}\` — full E2E failed" \
+            --body "$BODY" \
+            --label "bug,priority:high"