agentic-sdlc-wizard 1.30.0 → 1.32.0

package/.claude-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "sdlc-wizard",
       "source": ".",
       "description": "SDLC enforcement for AI agents — TDD, planning, self-review, CI shepherd",
-      "version": "1.30.0",
+      "version": "1.31.0",
       "author": {
         "name": "Stefan Ayala"
       },

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "sdlc-wizard",
-  "version": "1.30.0",
+  "version": "1.31.0",
   "description": "SDLC enforcement for AI agents — TDD, planning, self-review, CI shepherd",
   "author": {
     "name": "Stefan Ayala",

package/CHANGELOG.md

@@ -4,6 +4,46 @@ All notable changes to the SDLC Wizard.
 
 > **Note:** This changelog is for humans to read. Don't manually apply these changes - just run the wizard ("Check for SDLC wizard updates") and it handles everything automatically.
 
+## [1.32.0] - 2026-04-16
+
+### Added
+- Opus 4.7 support in benchmark workflow (#178)
+  - `claude-opus-4-7` added to model choices, `effort` input (high/xhigh/max)
+  - `--effort` passed via `claude_args`, effort recorded in artifacts + summaries
+  - Hard-fail when xhigh used with non-4.7 models (inputs resolved before shell)
+  - Artifact names include effort level to prevent collision
+  - Default: opus-4-7 + xhigh (matches CC's new default)
+  - 3 new tests (39 total model-comparison tests)
+- `xhigh` effort level documented in wizard (#178)
+  - New effort table: high → xhigh (recommended for coding) → max
+  - Opus 4.7 changes: stricter effort adherence, budget_tokens deprecated, 64k+ max_tokens guidance
+- Benchmark ceiling effect audit documented in wizard
+  - Cross-model audit (Codex GPT-5.4, xhigh) rated benchmark 2/10 NOT CERTIFIED
+  - 4 P0 findings: fake trials, answer key leaked, no independent verification, binary rubric
+  - 3 concrete fixes documented (remove coaching, add correctness scoring, real trials)
+  - External benchmark comparison (SWE-Bench, Aider methodology)
+- Automation Station community Discord link in README
+
+### Fixed
+- Orphaned `skills/gdlc/` causing test-doc-consistency failures (deleted)
+
+## [1.31.0] - 2026-04-14
+
+### Added
+- Ephemeral marketplace path detection in CLI `check` command (#174)
+  - Scans `~/.claude/settings.json` `extraKnownMarketplaces` for directory sources on ephemeral paths (`/tmp/`, `/private/tmp/`, `/var/folders/`)
+  - `EPHEMERAL` status (path exists but in ephemeral root) warns but doesn't fail check
+  - `DANGLING` status (path doesn't exist) errors with non-zero exit code
+  - Suggests moving to `~/.claude/plugins-local/<name>` for stable installs
+  - JSON output (`--json`) includes new `marketplace` field
+  - 10 new tests (51 total CLI tests)
+
+### Fixed
+- Hook false-positive "SETUP NOT COMPLETE" in non-SDLC directories (#173, PR #175)
+  - Three-way detection: both files (normal), one file (warn partial setup), neither (silent exit)
+  - Added `find_partial_sdlc_root` helper for partial-setup detection
+  - 2 new hook tests (60 total hook tests)
+
 ## [1.30.0] - 2026-04-12
 
 ### Added

package/CLAUDE_CODE_SDLC_WIZARD.md

@@ -99,6 +99,27 @@ This prevents both false positives (crying wolf) and false negatives (missing re
 - Green CI = safe to upgrade. Red = stay on current version until fixed
 - Results shown in PR with statistical confidence
 
+### Benchmark Ceiling Effect (Known Issue — April 2026)
+
+**Our E2E benchmark currently has zero discriminating power.** Both Opus 4.6 and 4.7 scored perfect 10/10 on the `add-feature` scenario (3 trials each, `high` effort). A cross-model audit (Codex GPT-5.4, xhigh reasoning) rated the benchmark methodology **2/10, NOT CERTIFIED** and identified 4 P0 critical issues:
+
+| Finding | Severity | Problem |
+|---------|----------|---------|
+| **Fake trials** | P0 | The workflow runs the simulation ONCE, then re-scores the same output N times. "Trials" measure judge jitter, not model variance |
+| **Answer key leaked** | P0 | The simulation prompt tells the model exactly what's scored ("You MUST use TodoWrite... scored by automated checks"). This tests obedience to rubric, not SDLC judgment |
+| **No independent verification** | P0 | "Tests pass" is self-reported from the transcript. The evaluator never re-runs `npm test` on the final code |
+| **Binary rubric** | P0 | Every criterion is YES/NO. The evaluator is explicitly designed for "near-zero variance." On an easy coached task, scores collapse to 10/10 |
+
+**Three concrete fixes to break the ceiling:**
+
+1. **Remove rubric leakage** — Don't tell the model what's scored in the simulation prompt. Let the wizard hooks and docs drive behavior naturally. Score hidden behaviors from traces, not coached compliance
+2. **Make correctness the majority of the score** — After simulation, run an external verifier: re-run `npm test` on the modified fixture, add hidden tests the model didn't know about, inspect the actual diff. Replace transcript-only `clean_code` with diff-based quality checks
+3. **Real trials on calibrated scenarios** — Each trial must be a fresh end-to-end simulation run on a fresh checkout. Select scenarios by pilot difficulty so top models don't all saturate (similar to Aider's hard-subset methodology). The current single-coached-toy-run approach is measuring nothing
+
+**What external benchmarks do differently:** SWE-Bench gives a real issue plus a full repo snapshot, applies the agent's patch, and runs the repo's actual tests to score `% resolved`. Aider's polyglot benchmark was explicitly rebuilt because the old one saturated — it uses 225 harder tasks chosen to preserve headroom. Our benchmark lacks real task difficulty calibration, independent execution-based correctness, multi-task breadth, and headroom management.
+
+**Status:** This is tracked as item #96 (E2E score audit) on the roadmap. Until fixed, the benchmark measures process compliance coaching, not model quality differentiation.
+
 ---
 
 ## Philosophy: Sensible Defaults, Smart Customization
@@ -221,12 +242,20 @@ Claude Code's **effort level** controls how much thinking the model does before
 
 | Level | When to Use | How to Set |
 |-------|-------------|------------|
-| `high` | **Default for all SDLC work.** Features, bug fixes, refactoring, tests, reviews | `effort: high` in skill frontmatter (already set) |
-| `max` | LOW confidence, FAILED 2x, architecture decisions, complex debugging, cross-model reviews | `/effort max` (session only — resets next session) |
+| `high` | Standard SDLC work. Features, bug fixes, refactoring, tests, reviews | `effort: high` in skill frontmatter (already set) |
+| `xhigh` | **Recommended default for coding and agentic work (Opus 4.7+).** Long-running tasks, repeated tool calls, deep exploration. Claude Code defaults to this on Opus 4.7 | `/effort xhigh` or set in skill frontmatter |
+| `max` | LOW confidence, FAILED 2x, architecture decisions, complex debugging, cross-model reviews. Reserve for genuinely frontier problems — on most workloads `max` adds cost for small quality gains | `/effort max` (session only — resets next session) |
+
+**Effort level changes in Opus 4.7 (April 2026):**
+- **`xhigh` is new** — sits between `high` and `max`, designed for coding and agentic work (30+ minute tasks with token budgets in the millions)
+- **Claude Code now defaults to `xhigh`** on Opus 4.7 for all plans
+- **Opus 4.7 respects effort levels more strictly** than 4.6 — at lower levels it scopes work tighter instead of going above and beyond. If you see shallow reasoning, raise effort rather than prompting around it
+- **`budget_tokens` is deprecated** on Opus 4.7 — use adaptive thinking with effort instead
+- When running at `xhigh` or `max`, set a large `max_tokens` (64k+) so the model has room to think across subagents and tool calls
 
-**Why `high` is the default:** Claude Code uses **adaptive thinking** to dynamically allocate reasoning budget per turn. On Pro and Max plans, the default effort level is **medium (85)**, which causes the model to under-allocate reasoning on complex multi-step tasks — leading to shallow analysis, missed edge cases, and "lazy" outputs. This was [confirmed by Anthropic engineer Boris Cherny](https://github.com/anthropics/claude-code/issues/42796) and is documented at [code.claude.com](https://code.claude.com/docs/en/model-config). API, Team, and Enterprise plans default to high effort and are not affected.
+**Why `high` was the previous default:** Claude Code uses **adaptive thinking** to dynamically allocate reasoning budget per turn. On Pro and Max plans, the default effort level was **medium (85)**, which causes the model to under-allocate reasoning on complex multi-step tasks — leading to shallow analysis, missed edge cases, and "lazy" outputs. This was [confirmed by Anthropic engineer Boris Cherny](https://github.com/anthropics/claude-code/issues/42796) and is documented at [code.claude.com](https://code.claude.com/docs/en/model-config). API, Team, and Enterprise plans default to high effort and are not affected.
 
-The `/sdlc` skill sets `effort: high` in its frontmatter, overriding the medium default on every SDLC invocation. This gives thorough reasoning without the unbounded token cost of `max`.
+The `/sdlc` skill sets `effort: high` in its frontmatter, overriding the medium default on every SDLC invocation. Consider upgrading to `effort: xhigh` on Opus 4.7+ for deeper reasoning on complex tasks.
 
 **Nuclear option — disable adaptive thinking entirely:** Set `CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1` in your environment or settings.json `env` block. This forces a fixed reasoning budget per turn instead of letting the model dynamically allocate. Use this if you observe persistent quality issues even with `effort: high`. See [Claude Code model config docs](https://code.claude.com/docs/en/model-config) for details.
 
@@ -2628,7 +2657,7 @@ If deployment fails or post-deploy verification catches issues:
 
 **SDLC.md:**
 ```markdown
-<!-- SDLC Wizard Version: 1.30.0 -->
+<!-- SDLC Wizard Version: 1.31.0 -->
 <!-- Setup Date: [DATE] -->
 <!-- Completed Steps: step-0.1, step-0.2, step-0.4, step-1, step-2, step-3, step-4, step-5, step-6, step-7, step-8, step-9 -->
 <!-- Git Workflow: [PRs or Solo] -->
@@ -3687,7 +3716,7 @@ Walk through updates? (y/n)
 Store wizard state in `SDLC.md` as metadata comments (invisible to readers, parseable by Claude):
 
 ```markdown
-<!-- SDLC Wizard Version: 1.30.0 -->
+<!-- SDLC Wizard Version: 1.31.0 -->
 <!-- Setup Date: 2026-01-24 -->
 <!-- Completed Steps: step-0.1, step-0.2, step-1, step-2, step-3, step-4, step-5, step-6, step-7, step-8, step-9 -->
 <!-- Git Workflow: PRs -->

package/README.md

@@ -87,7 +87,7 @@ Layer 4: STATISTICAL VALIDATION
   SDP normalizes for model quality. CUSUM catches drift.
 
 Layer 3: SCORING ENGINE
-  7 criteria, 10/11 points. Claude evaluates Claude.
+  Multi-criteria scoring, 10/11 points. Claude evaluates Claude.
   Before/after wizard A/B comparison in CI.
 
 Layer 2: ENFORCEMENT
@@ -229,12 +229,16 @@ This isn't the only Claude Code SDLC tool. Here's an honest comparison:
 | Document | What It Covers |
 |----------|---------------|
 | [ARCHITECTURE.md](ARCHITECTURE.md) | System design, 5-layer diagram, data flows, file structure |
-| [CI_CD.md](CI_CD.md) | All 7 workflows, E2E scoring, tier system, SDP, integrity checks |
+| [CI_CD.md](CI_CD.md) | All workflows, E2E scoring, tier system, SDP, integrity checks |
 | [SDLC.md](SDLC.md) | Version tracking, enforcement rules, SDLC configuration |
 | [TESTING.md](TESTING.md) | Testing philosophy, test diamond, TDD approach |
 | [CHANGELOG.md](CHANGELOG.md) | Version history, what changed and when |
 | [CONTRIBUTING.md](CONTRIBUTING.md) | How to contribute, evaluation methodology |
 
+## Community
+
+Come join **[Automation Station](https://discord.com/invite/fGPEF7GHrF)** — a community Discord packed with software engineers bringing 40+ years of combined experience across every area of the stack (frontend, backend, infra, embedded, data, QA, DevOps, you name it). Share patterns, ask questions, compare notes on AI agents, automation, and SDLC tooling.
+
 ## Contributing
 
 PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for evaluation methodology and testing.

package/cli/init.js

@@ -2,6 +2,7 @@
 
 const crypto = require('crypto');
 const fs = require('fs');
+const os = require('os');
 const path = require('path');
 
 const RESET = '\x1b[0m';
@@ -285,23 +286,37 @@ function check(targetDir, { json = false } = {}) {
     // Offline or npm unavailable — skip update check
   }
 
-  const hasDrift = results.some((r) => r.status === 'MISSING' || r.status === 'DRIFT');
+  const marketplace = checkMarketplacePaths();
+  const hasDrift = results.some((r) => r.status === 'MISSING' || r.status === 'DRIFT')
+    || marketplace.some((m) => m.status === 'DANGLING');
 
   if (json) {
-    console.log(JSON.stringify({ files: results, update: updateInfo }, null, 2));
+    console.log(JSON.stringify({ files: results, update: updateInfo, marketplace }, null, 2));
   } else {
     for (const r of results) {
       const color = r.status === 'MATCH' ? GREEN : r.status === 'MISSING' ? RED : YELLOW;
       console.log(`  ${color}${r.status}${RESET}  ${r.file}`);
       if (r.details) console.log(`         ${r.details}`);
     }
+    for (const m of marketplace) {
+      const color = m.status === 'DANGLING' ? RED : YELLOW;
+      const heading = m.status === 'EPHEMERAL'
+        ? `Marketplace '${m.name}' source path is ephemeral:`
+        : `Marketplace '${m.name}' source path does not exist:`;
+      console.log(`\n  ${color}${m.status}${RESET}  ${heading}`);
+      console.log(`         ${m.path}`);
+      console.log(`         ${m.details}`);
+      if (m.suggestion) {
+        console.log(`         Recommended: move to ${m.suggestion}`);
+      }
+    }
     if (updateInfo) {
       console.log(`\n  ${YELLOW}UPDATE${RESET}  v${updateInfo.current} -> v${updateInfo.latest}`);
       console.log('         Run: npx agentic-sdlc-wizard init --force');
     }
   }
 
-  return { results, updateInfo, hasDrift };
+  return { results, updateInfo, hasDrift, marketplace };
 }
 
 function checkFile(srcPath, destPath, relativeDest, shouldBeExecutable) {
@@ -341,4 +356,56 @@ function checkGitignore(gitignorePath) {
   return { file: '.gitignore', status: 'MATCH' };
 }
 
+const EPHEMERAL_ROOTS = /^(\/tmp\/|\/private\/tmp\/|\/var\/folders\/|\/private\/var\/folders\/)/;
+
+function checkMarketplacePaths() {
+  const results = [];
+  const globalSettings = path.join(os.homedir(), '.claude', 'settings.json');
+
+  if (!fs.existsSync(globalSettings)) return results;
+
+  let data;
+  try {
+    data = JSON.parse(fs.readFileSync(globalSettings, 'utf8'));
+  } catch (_) {
+    return results;
+  }
+
+  const marketplaces = data.extraKnownMarketplaces;
+  if (!marketplaces || typeof marketplaces !== 'object') return results;
+
+  for (const [name, entry] of Object.entries(marketplaces)) {
+    const source = entry && entry.source;
+    if (!source || source.source !== 'directory' || !source.path || typeof source.path !== 'string') continue;
+
+    const sourcePath = source.path;
+    const isEphemeral = EPHEMERAL_ROOTS.test(sourcePath);
+    const exists = fs.existsSync(sourcePath);
+    const basename = path.basename(sourcePath);
+    const suggestion = `~/.claude/plugins-local/${basename}`;
+
+    if (!exists) {
+      results.push({
+        name,
+        path: sourcePath,
+        status: 'DANGLING',
+        details: isEphemeral
+          ? 'Ephemeral path has been reaped — plugin is broken'
+          : 'Path does not exist — plugin may be silently broken',
+        suggestion: isEphemeral ? suggestion : undefined,
+      });
+    } else if (isEphemeral) {
+      results.push({
+        name,
+        path: sourcePath,
+        status: 'EPHEMERAL',
+        details: 'macOS reaps this path periodically — plugin may break silently',
+        suggestion,
+      });
+    }
+  }
+
+  return results;
+}
+
 module.exports = { init, check, planOperations, GITIGNORE_ENTRIES };

package/hooks/_find-sdlc-root.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# Shared helper: walk up from CWD to find nearest SDLC.md + TESTING.md pair
+# Sourced by sdlc-prompt-check.sh and instructions-loaded-check.sh
+# Fixes #171: false-positive "SETUP NOT COMPLETE" in monorepos / nested projects
+
+# find_sdlc_root — walks up from pwd, stops at $HOME (exclusive)
+# Sets SDLC_ROOT to the found directory, or empty string if not found
+find_sdlc_root() {
+    local check_dir
+    check_dir="$(pwd)"
+    SDLC_ROOT=""
+    while [ "$check_dir" != "/" ] && [ "$check_dir" != "$HOME" ] && [ -n "$check_dir" ]; do
+        if [ -f "$check_dir/SDLC.md" ] && [ -f "$check_dir/TESTING.md" ]; then
+            SDLC_ROOT="$check_dir"
+            return 0
+        fi
+        check_dir="$(dirname "$check_dir")"
+    done
+    return 1
+}
+
+# find_partial_sdlc_root — walks up looking for EITHER SDLC.md OR TESTING.md
+# Used to detect partial setup (one file exists but not both) vs not-an-SDLC-project
+find_partial_sdlc_root() {
+    local check_dir
+    check_dir="$(pwd)"
+    SDLC_ROOT=""
+    while [ "$check_dir" != "/" ] && [ "$check_dir" != "$HOME" ] && [ -n "$check_dir" ]; do
+        if [ -f "$check_dir/SDLC.md" ] || [ -f "$check_dir/TESTING.md" ]; then
+            SDLC_ROOT="$check_dir"
+            return 0
+        fi
+        check_dir="$(dirname "$check_dir")"
+    done
+    return 1
+}

package/hooks/instructions-loaded-check.sh

@@ -4,7 +4,21 @@
 # Available since Claude Code v2.1.69
 # Note: no set -e — this hook must always exit 0 to not block session start
 
-PROJECT_DIR="${CLAUDE_PROJECT_DIR:-.}"
+# Walk up from CWD to find nearest SDLC.md + TESTING.md (#171: monorepo support)
+HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "$HOOK_DIR/_find-sdlc-root.sh"
+
+# CWD walk-up finds nearest SDLC project (#173: silent exit for non-SDLC dirs)
+if find_sdlc_root; then
+    PROJECT_DIR="$SDLC_ROOT"
+elif find_partial_sdlc_root; then
+    # Partial setup — one file exists but not both. Warn about missing files
+    PROJECT_DIR="$SDLC_ROOT"
+else
+    # Not an SDLC project at all — exit silently
+    exit 0
+fi
+
 MISSING=""
 
 if [ ! -f "$PROJECT_DIR/SDLC.md" ]; then

package/hooks/sdlc-prompt-check.sh

@@ -2,8 +2,21 @@
 # Light SDLC hook - baseline reminder every prompt (~100 tokens)
 # Full guidance in skill: .claude/skills/sdlc/
 
-# Check if setup has been completed
-PROJECT_DIR="${CLAUDE_PROJECT_DIR:-.}"
+# Walk up from CWD to find nearest SDLC.md + TESTING.md (#171: monorepo support)
+HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "$HOOK_DIR/_find-sdlc-root.sh"
+
+# CWD walk-up finds nearest SDLC project (#173: silent exit for non-SDLC dirs)
+if find_sdlc_root; then
+    PROJECT_DIR="$SDLC_ROOT"
+elif find_partial_sdlc_root; then
+    # Partial setup — one file exists but not both. Warn about missing files
+    PROJECT_DIR="$SDLC_ROOT"
+else
+    # Not an SDLC project at all — exit silently
+    exit 0
+fi
+
 if [ ! -s "$PROJECT_DIR/SDLC.md" ] || [ ! -s "$PROJECT_DIR/TESTING.md" ]; then
     cat << 'SETUP'
 SETUP NOT COMPLETE: SDLC.md and/or TESTING.md are missing.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentic-sdlc-wizard",
-  "version": "1.30.0",
+  "version": "1.32.0",
   "description": "SDLC enforcement for Claude Code — hooks, skills, and wizard setup in one command",
   "bin": {
     "sdlc-wizard": "./cli/bin/sdlc-wizard.js"

package/skills/update/SKILL.md

@@ -46,9 +46,10 @@ Parse all CHANGELOG entries between the user's installed version and the latest.
 
 ```
 Installed: 1.24.0
-Latest:    1.30.0
+Latest:    1.31.0
 
 What changed:
+- [1.31.0] Hook false-positive fix for non-SDLC dirs, ephemeral marketplace path warning, ...
 - [1.30.0] Firmware fixture, model A/B comparison workflow, CC degradation detection, ...
 - [1.29.0] Node 24 compliance, autocompact in settings.json, effectiveness scoreboard, ...
 - [1.28.0] Autocompact benchmarking methodology, canary fact mechanism, benchmark harness, ...