@dzhechkov/harness-cli 0.3.226 → 0.3.227

package/README.md

@@ -389,7 +389,7 @@ Each pack is an npm package — click through for the **full per-skill documenta
 | `health` | 8 | Medical AI (diagnostics, drugs, labs, clinical decisions) |
 | `keysarium` | 9 | Full research toolkit (feature-adr, presentation, reverse-eng) |
 | `p-replicator` | 10 | AI product development (/replicate, SPARC PRD, pipeline-forge) |
-| `feature-adr` | 5 | Feature pipeline (feature-adr, explore, frontend-design) |
+| `feature-adr` | 8 | Feature pipeline (feature-adr, explore, frontend-design, code-critic, code-impl, system-grill) |
 | `reasoning` | 4 | Generic reasoning & code-quality (investigate, solid, karpathy-guidelines, agents-md-creator) — stack-neutral, zero coupling |
 | `devops` | 30 | DevOps skills (terraform, kubernetes, c4-architecture, incident-response, problem-management, risk-assessment, ...) |
 | `web3` | 12 | Web3/DeFi (quicknode, zerion, symbiosis, bankr, veil, neynar, ...) |
@@ -433,7 +433,7 @@ Get the whole set with `dz init --target claude-code --preset meta`, or pick one
 
 > **A skill and its npx toolkit are not duplicates — they're a graduation.** Several skills (e.g. `feature-adr`, `design-thinking`) exist BOTH as a skill inside a `dz` preset AND as a standalone `npx` package. The preset's SKILL.md is **fully functional on its own** (the whole methodology — modules + references — travels with it, and it auto-activates by description), and it's the only way to compile that capability to the **non-Claude platforms** (Codex/OpenCode/Hermes/OpenClaude) via `dz`. The npx package adds **project-level runtime governance** around the same skill: a slash command, governance rules, a context shard, and (for feature-adr) reward-learning + `/harvest`. So: pick the **skill/preset** for a working capability across platforms; pick the **npx toolkit** when you want it as a governed, command-driven fixture of one project.
 
-## All Commands (51)
+## All Commands (52)
 
 ```
 dz setup             --target <name> [--preset <name>] [--select id,id,...] [--skills-dir <dir>] [--memory agentdb] [--no-memory] [--no-hooks] [--install-driver] [--force]
@@ -487,6 +487,7 @@ dz feature-adr-setup [--plan] [--from-spec <f>] [--apply]   # guided project onb
 dz challenge --plan <plan.md> [--json] [--context-only] [--author <model>]   # adversarial plan-gate (behind the `challenge-panel` skill): assemble a WIDE context pack (plan + vision + testing + map + degradations) + the fixed C1-C8 "break it" brief for a FRESH adversary (≠ plan author); advisory, never blocks
 dz routing [--stage <s>] [--json]   # inspect the learned cost-optimal routing store: what `args.models.<stage>='auto-cost'` believes per (stage, complexity-tier, model) — gated attempts/successes/rate (feeds feature-adr model selection)
 dz bto-optimize --split|--plan|--select|--scope-check|--diff [--json]   # deterministic engine behind /bto-optimize: hold-out split + hard-capped budget + no-regress-on-holdout winner selection (defeats judge-gaming); prose-only, diff-confirmed, never auto-writes
+dz discrimination-check --test <f[,f]> [--base <ref>] [--name <filter>] [--runner <cmd>] [--json]   # §42 test-discrimination gate for feature-adr Step-8: run the ADR's property test in an isolated git worktree at pre-feature base — it MUST go red without the fix; a green is a false green (HIGH finding, advisory, never auto-aborts)
 dz publish           [--filter <name>] [--bump-only] [--claim-check <off|warn|error>]   (dry-run by default; pass --yes/--confirm to go live; claim-check gate defaults to warn — surfaces README claim findings, never blocks)
 dz auto-canonicalize --source <github-url> --pack <skills-pack>
 dz sync-upstream     [--package <dir>] [--list] [--all]
@@ -1558,6 +1559,30 @@ Verify: after a session, `.dz/agentdb.db` grows and `.dz/sessions.jsonl` stays e
 
 Bridge: with the agentdb backend enabled, `dz recall` itself is the bridge — it merges lexical FTS5 with vector similarity over the same store (see *Semantic recall* below). `dz recall --all --json` stays the portable SHARING form.
 
+### `dz discrimination-check` — when a test is green but you're not sure it actually catches anything
+
+feature-adr Step-8 asserts the ADR's load-bearing safety property **has** a test. But a green test can be a
+false green — it may never exercise the property, so it would stay green even if the property regressed. The
+`§42` gate (learned from rUv's `cve-bench/evaluate.mjs`) proves the test **discriminates**: it runs the property
+test in a throwaway git worktree at the pre-feature base (no fix) and requires it to go **red**.
+```bash
+# the Step-7 feature diff is uncommitted mid-pipeline, so HEAD is the pre-feature base
+dz discrimination-check --test packages/x/test/auth-property.test.ts --base HEAD --json
+```
+```
+discrimination-check @ HEAD — verdict: NON_DISCRIMINATING
+  ✗ packages/x/test/auth-property.test.ts: NON_DISCRIMINATING
+
+  [high] non-discriminating property test (false green)
+  1 property test(s) PASS at pre-feature base — they do not exercise the ADR safety property … (Advisory — the pipeline continues; the owner decides.)
+```
+Verdicts: `DISCRIMINATES` (red by assertion — good) · `DISCRIMINATES_VIA_ERROR` (couldn't load at base, e.g. a
+brand-new module — inferred) · `NON_DISCRIMINATING` (green at base — a false green, HIGH finding) ·
+`CANNOT_ISOLATE` (no runnable property test — folds into "property untested"). It **never auto-aborts** (dz's
+rule: a false gate kills trust) — exit 0 on any verdict, exit 2 only on a usage/setup error. The worktree +
+sanitation live in tested CLI code; base ref, paths, name filter, and runner are all injection-checked, and the
+worktree is always removed. Step-8 runs this on the ADR Confirmation's `Required automated check` automatically.
+
 ### Semantic recall (vector tier)
 
 `dz recall` is **hybrid** when the vector tier is available and **exactly the old lexical command** when it is not — enabling it never changes behavior for projects that skip it.

package/dist/cli.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAyOH,2EAA2E;AAC3E,MAAM,WAAW,KAAK;IACpB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACxC;;;;OAIG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;CACzB;AAmhJD,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAsI5E"}
+{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AA2OH,2EAA2E;AAC3E,MAAM,WAAW,KAAK;IACpB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACxC;;;;OAIG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;CACzB;AA+oJD,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAwI5E"}

package/dist/cli.js

@@ -3,13 +3,13 @@
  *
  * @packageDocumentation
  */
-import { existsSync, lstatSync, mkdirSync, readFileSync, readdirSync, readlinkSync, renameSync, rmdirSync, statSync, symlinkSync, writeFileSync } from 'node:fs';
+import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, readlinkSync, renameSync, rmdirSync, rmSync, statSync, symlinkSync, writeFileSync } from 'node:fs';
 import { basename, dirname, isAbsolute, join, relative, resolve, sep } from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { execSync } from 'node:child_process';
-import { homedir } from 'node:os';
+import { homedir, tmpdir } from 'node:os';
 import { createRequire } from 'node:module';
-import { createSkill, getSkillInfo, getWorkflow, isTargetName, listSkills, runDoctor, runInit, benchmarkSkill, benchmarkSkills, scanMcp, reconcileCapabilities, RECONCILE_BANNER, buildRegistry, discoverSkillPackDirs, checkUpstream, compareSkills, checkAllUpstream, sweepSkillDrift, syncCanonicalSkill, checkUpgrades, discoverPackages, discoverSourcePackages, fetchAllDownloads, filterByCategory, pretrain, recommend, generatePlugin, publishPackages, runSetup, runMigrate, searchRegistry, runSync, runVerify, runInitAgentsMd, runInitGeminiMd, TARGET_NAMES, WORKFLOW_NAMES, importEcc, recordPattern, resolveLearningBackend, storeStats, consolidateSessions, pruneNoisePatterns, lessonDeltaReport, removePatternsByIds, snapshotStore, recallHybrid, teachGuard, mirrorPatternsToVector, vectorMirrorEnabled, vectorTierStatus, resolveVectorEngine, reindexVectorStore, harmonizeVectorStore, importRvfCheckpoint, statuslineData, writeFeatureAdrState, computeUsage, deriveUsageCalibration, normalizeClaudeUsageModelKey, readUsageLimits, claimCheck, summarize, queryBookKnowledge, loadStorePatternsSync, patternRecordId, loadStoreRecords, recordToPattern, bundleSkills, brainHome, listBrain, promoteProjectToBrain, updateBrainSource, queryBrain, groundPrompt, expandKu, reindexBrainVectors, buildPrimer, exportBrainSlice, importBrainSlice, registerKusToBrain, RECALL_USAGE_LOG_RELATIVE, RECALL_USAGE_LOG_MAX_BYTES, parseRecallUsageLog, buildRecallUsageReport, buildManifest, buildSbom, resolveTrustRoot, decideVerifyPolicy, decideProvenance, isInsideTree, signManifest, verifyManifest, assertKeyOutsideTree, decidePublishGate, MANIFEST_NAME, SBOM_NAME, buildArchitectureMap, renderMapHuman, findArchitectureDrift, renderDriftReport, scanWorkspacePackages, loadSubsystemManifest, loadProductVision, checkFeatureAgainstArchitecture, renderArchCheck, planProjectSkills, guidanceForStage, renderInjectionReport, analyzeCorpus, renderRakeReport, renderCriticSection, rakeAsLesson, rakeReward, DEFAULT_RAKE_THRESHOLDS, streamSessionEvents, findLatestTranscript, detectProcessRakes, buildRetro, renderRetro, retroLessonText, PROCESS_SIGNATURES, RETRO_DOMAIN, scanForSetup, buildSetupPlan, scaffoldFromSpec, renderScaffoldPreview, readExistingForScaffold, assembleChallengeContext, buildChallengeBrief, pickAdversaryModel, CHALLENGE_QUESTIONS, loadOutcomes, renderOutcomes, statsForKey, selectAutoCost, recordProvisional, finalizeOutcome, COST_LADDER, splitScenarios, budgetPlan, selectWinner, proseScopeOk, renderProseDiff, readScenarioIds, DEFAULT_MAX_JUDGE_RUNS, } from '@dzhechkov/harness-core';
+import { createSkill, getSkillInfo, getWorkflow, isTargetName, listSkills, runDoctor, runInit, benchmarkSkill, benchmarkSkills, scanMcp, reconcileCapabilities, RECONCILE_BANNER, buildRegistry, discoverSkillPackDirs, checkUpstream, compareSkills, checkAllUpstream, sweepSkillDrift, syncCanonicalSkill, checkUpgrades, discoverPackages, discoverSourcePackages, fetchAllDownloads, filterByCategory, pretrain, recommend, generatePlugin, publishPackages, runSetup, runMigrate, searchRegistry, runSync, runVerify, runInitAgentsMd, runInitGeminiMd, TARGET_NAMES, WORKFLOW_NAMES, importEcc, recordPattern, resolveLearningBackend, storeStats, consolidateSessions, pruneNoisePatterns, lessonDeltaReport, removePatternsByIds, snapshotStore, recallHybrid, teachGuard, mirrorPatternsToVector, vectorMirrorEnabled, vectorTierStatus, resolveVectorEngine, reindexVectorStore, harmonizeVectorStore, importRvfCheckpoint, statuslineData, writeFeatureAdrState, computeUsage, deriveUsageCalibration, normalizeClaudeUsageModelKey, readUsageLimits, claimCheck, summarize, queryBookKnowledge, loadStorePatternsSync, patternRecordId, loadStoreRecords, recordToPattern, bundleSkills, brainHome, listBrain, promoteProjectToBrain, updateBrainSource, queryBrain, groundPrompt, expandKu, reindexBrainVectors, buildPrimer, exportBrainSlice, importBrainSlice, registerKusToBrain, RECALL_USAGE_LOG_RELATIVE, RECALL_USAGE_LOG_MAX_BYTES, parseRecallUsageLog, buildRecallUsageReport, buildManifest, buildSbom, resolveTrustRoot, decideVerifyPolicy, decideProvenance, isInsideTree, signManifest, verifyManifest, assertKeyOutsideTree, decidePublishGate, MANIFEST_NAME, SBOM_NAME, buildArchitectureMap, renderMapHuman, findArchitectureDrift, renderDriftReport, scanWorkspacePackages, loadSubsystemManifest, loadProductVision, checkFeatureAgainstArchitecture, renderArchCheck, planProjectSkills, guidanceForStage, renderInjectionReport, analyzeCorpus, renderRakeReport, renderCriticSection, rakeAsLesson, rakeReward, DEFAULT_RAKE_THRESHOLDS, streamSessionEvents, findLatestTranscript, detectProcessRakes, buildRetro, renderRetro, retroLessonText, PROCESS_SIGNATURES, RETRO_DOMAIN, scanForSetup, buildSetupPlan, scaffoldFromSpec, renderScaffoldPreview, readExistingForScaffold, assembleChallengeContext, buildChallengeBrief, planDiscriminationCheck, classifyDiscrimination, pickAdversaryModel, CHALLENGE_QUESTIONS, loadOutcomes, renderOutcomes, statsForKey, selectAutoCost, recordProvisional, finalizeOutcome, COST_LADDER, splitScenarios, budgetPlan, selectWinner, proseScopeOk, renderProseDiff, readScenarioIds, DEFAULT_MAX_JUDGE_RUNS, } from '@dzhechkov/harness-core';
 import { getPreset, PRESET_NAMES } from '@dzhechkov/harness-presets';
 import { scanGitHub, analyzeRepo, generateReport, deepAnalyze, scanAllSources, ScoutMemory } from '@dzhechkov/scout';
 const USAGE = `dz - DZ cross-platform harness CLI
@@ -4249,6 +4249,162 @@ function cmdChallenge(options, flags, cwd, write) {
     write(`\n── dispatch (panel ≠ plan author) ──\n${adversary.model}: ${adversary.note}`);
     return 0;
 }
+/**
+ * `dz discrimination-check` — the §42 test-discrimination gate (feature learned from cve-bench/evaluate.mjs).
+ * feature-adr Step-8 already asserts the ADR safety property HAS a test; this asserts that test DISCRIMINATES:
+ * run the property test(s) in an isolated git worktree at the pre-feature base (default HEAD, since the feature
+ * diff is uncommitted mid-pipeline). They MUST go red without the feature diff — a green is a false green.
+ *
+ *   --test <a.test.ts[,b.test.ts]>  the property test file(s) mapped from the ADR Confirmation (comma list)
+ *   --base <ref>                    the base ref to fail against (default HEAD)
+ *   --name '<filter>'               optional -t test-name filter applied to every target
+ *   --runner '<cmd>'                test runner (default `npx vitest run`)
+ *   --json                          machine-readable {plan, results, verdict, finding}
+ *
+ * NEVER auto-aborts: a non-discriminating (false-green) test is reported as a HIGH finding for the owner to
+ * decide (dz's rule — a false gate kills trust). Exit code is 0 on a clean run regardless of verdict; 2 only on
+ * a usage/setup error, so a caller distinguishes "gate ran" from "gate could not run".
+ */
+function cmdDiscriminationCheck(options, flags, cwd, write) {
+    let repoRoot = cwd;
+    try {
+        repoRoot = execSync('git rev-parse --show-toplevel', { cwd, encoding: 'utf-8' }).trim() || cwd;
+    }
+    catch { /* not git */ }
+    const testArg = options.get('test');
+    if (testArg === undefined || testArg.trim() === '') {
+        write('dz discrimination-check: pass --test <property-test.ts[,...]> (the test(s) the ADR Confirmation names).');
+        return 2;
+    }
+    const nameFilter = options.get('name');
+    const propertyTests = testArg.split(',').map((s) => s.trim()).filter(Boolean).map((file) => nameFilter !== undefined && nameFilter.trim() !== '' ? { file, name: nameFilter.trim() } : { file });
+    const baseRef = options.get('base') ?? 'HEAD';
+    const runnerOpt = options.get('runner');
+    const plan = planDiscriminationCheck(runnerOpt !== undefined ? { baseRef, propertyTests, runner: runnerOpt } : { baseRef, propertyTests });
+    if (!plan.runnable) {
+        // No safe target to run → this is the existing "property untested" finding (empty propertyTests classify).
+        const result = classifyDiscrimination({ propertyTests: [], results: [] });
+        if (flags.has('json')) {
+            write(JSON.stringify({ plan, results: [], ...result }, null, 2));
+            return 0;
+        }
+        write(`discrimination-check: NOT RUN (${plan.reason ?? 'unknown'})`);
+        if (plan.rejected.length)
+            write('  rejected: ' + plan.rejected.map((r) => `${r.file} (${r.reason})`).join(', '));
+        write(`  → ${result.finding?.detail ?? 'no property test to check'}`);
+        return 0;
+    }
+    // Execute the plan in a temp worktree WE own; substitute {{WORKTREE}} and always clean up.
+    // `git worktree add` must CREATE the path, so compute a fresh non-existent one (do NOT mkdtemp it).
+    const worktree = join(mkdtempSync(join(tmpdir(), 'dz-disc-')), 'wt');
+    const results = [];
+    try {
+        // 1) add the detached worktree at base (git creates `worktree`; its parent already exists).
+        const addCmd = plan.commands[0].replace(/\{\{WORKTREE\}\}/g, worktree);
+        execSync(addCmd, { cwd: repoRoot, stdio: 'pipe', encoding: 'utf-8' });
+        // 1b) a fresh worktree has NO node_modules — without this, every test fails to load (runner + deps
+        // unresolvable) and the gate collapses to always-VIA_ERROR, blind to false greens. Absolute-path
+        // symlinks point back at the main checkout's already-installed trees, robust across pnpm's layout.
+        const linkNodeModules = (relDir) => {
+            const srcNm = join(repoRoot, relDir, 'node_modules');
+            if (!existsSync(srcNm))
+                return;
+            const dstNm = join(worktree, relDir, 'node_modules');
+            if (existsSync(dstNm))
+                return;
+            try {
+                mkdirSync(dirname(dstNm), { recursive: true });
+                symlinkSync(srcNm, dstNm, 'dir');
+            }
+            catch { /* best effort */ }
+        };
+        linkNodeModules('.'); // root (hoisted deps + .bin)
+        const pkgDirs = new Set();
+        for (const t of plan.targets) {
+            let d = dirname(t.file);
+            while (d && d !== '.' && d !== sep) {
+                if (existsSync(join(repoRoot, d, 'package.json'))) {
+                    pkgDirs.add(d);
+                    break;
+                }
+                d = dirname(d);
+            }
+        }
+        for (const d of pkgDirs)
+            linkNodeModules(d);
+        // 2) copy each property test into the base worktree, then 3) run it and record pass/fail/error per target.
+        for (const t of plan.targets) {
+            try {
+                const src = resolve(repoRoot, t.file);
+                // containment guard (defense in depth beyond planDiscriminationCheck's path sanitation).
+                if (!resolve(src).startsWith(resolve(repoRoot) + sep)) {
+                    results.push(nameFor(t, 'error'));
+                    continue;
+                }
+                const dst = join(worktree, t.file);
+                mkdirSync(dirname(dst), { recursive: true });
+                writeFileSync(dst, readFileSync(src));
+            }
+            catch {
+                results.push(nameFor(t, 'error'));
+                continue;
+            }
+            const runner = (runnerOpt !== undefined && plan.commands.some((c) => c.includes(runnerOpt))) ? runnerOpt : 'npx vitest run';
+            // t.file + t.name already passed the engine's strict sanitation (no quotes/metacharacters/leading-dash);
+            // still quote + `--` so a path can never be read as a runner option or split a word.
+            const nameArg = t.name ? ` -t '${t.name}'` : '';
+            try {
+                execSync(`${runner}${nameArg} -- '${t.file}'`, { cwd: worktree, stdio: 'pipe', encoding: 'utf-8' });
+                results.push(nameFor(t, 'pass')); // exit 0 → test PASSED at base → false green
+            }
+            catch (e) {
+                // vitest exits non-zero on failure AND on load/compile error. Distinguish: a load error usually names
+                // "Cannot find module"/"Failed to load"/"No test files"; otherwise treat as an assertion failure (red).
+                const out = String(e.stdout ?? '') + String(e.stderr ?? '');
+                const isLoadError = /cannot find module|failed to load|no test (files )?found|error: cannot|transform failed|esbuild/i.test(out);
+                results.push(nameFor(t, isLoadError ? 'error' : 'fail'));
+            }
+        }
+    }
+    catch (e) {
+        if (flags.has('json')) {
+            write(JSON.stringify({ plan, error: 'worktree-setup-failed', detail: String(e.message).slice(0, 300) }, null, 2));
+        }
+        else
+            write(`discrimination-check: could not create worktree at ${baseRef}: ${String(e.message).slice(0, 200)}`);
+        return 2;
+    }
+    finally {
+        try {
+            execSync(`git worktree remove --force ${worktree}`, { cwd: repoRoot, stdio: 'pipe' });
+        }
+        catch { /* fall through to rm */ }
+        // remove the whole mkdtemp parent (worktree is `<mkdtemp>/wt`), so nothing leaks under tmp even on error.
+        try {
+            rmSync(dirname(worktree), { recursive: true, force: true });
+        }
+        catch { /* best effort */ }
+        try {
+            execSync('git worktree prune', { cwd: repoRoot, stdio: 'pipe' });
+        }
+        catch { /* best effort */ }
+    }
+    const result = classifyDiscrimination({ propertyTests, results });
+    if (flags.has('json')) {
+        write(JSON.stringify({ plan, results, ...result }, null, 2));
+        return 0;
+    }
+    write(`discrimination-check @ ${baseRef} — verdict: ${result.aggregate}`);
+    for (const p of result.perTest)
+        write(`  ${p.verdict === 'NON_DISCRIMINATING' ? '✗' : '✓'} ${p.file}${p.name ? ` (${p.name})` : ''}: ${p.verdict}`);
+    if (result.finding)
+        write(`\n  [${result.finding.severity}] ${result.finding.title}\n  ${result.finding.detail}`);
+    return 0;
+}
+/** small helper: build a result row, omitting `name` when absent (exactOptionalPropertyTypes). */
+function nameFor(t, outcome) {
+    return t.name !== undefined ? { file: t.file, name: t.name, outcome } : { file: t.file, outcome };
+}
 /**
  * `dz routing` — inspect the learned cost-optimal routing outcome store (feature learned-cost-routing). Shows
  * what `args.models[stage]='auto-cost'` currently believes per (stage, complexity-tier, model): gated
@@ -4676,6 +4832,8 @@ export async function runCli(argv, io = {}) {
                 return cmdFeatureAdrSetup(options, flags, cwd, write);
             case 'challenge':
                 return cmdChallenge(options, flags, cwd, write);
+            case 'discrimination-check':
+                return cmdDiscriminationCheck(options, flags, cwd, write);
             case 'routing':
                 return cmdRouting(options, flags, cwd, write);
             case 'bto-optimize':