@dailephd/my-dev-kit-lab 0.9.4 → 0.9.5

package/CHANGELOG.md

@@ -2,6 +2,21 @@
 
 All notable changes to my-dev-kit-lab are documented here.
 
+## [0.9.5] - 2026-10-27
+
+### Staleness experiment evidence correction
+
+- The `incremental-change-staleness` experiment now retrieves through an opt-in bounded multi-source protocol in the shared retrieval owner (`runMyDevKitBoundedRetrievalFromIndex`) so multi-file required evidence can be observed; default top-one retrieval and the meaning of its `filesRead` are unchanged for every other caller. Only files whose own `source` command succeeded count as read.
+- Required-file evidence reports `unknown` instead of `missing` when the bounded protocol cannot observe the requirement, and a full-refresh reference that did not itself observe the required files can no longer make equal results look like `same`/no observed regression: the relation is `unknown` and the classification `inconclusive`.
+- The V2 report labels the fake-agent score as synthetic, distinguishes read from merely surfaced files, and states that runtime command-log paths are not retained after cleanup. Persisted V1/V2 schemas, treatment ordering, scenarios, answer keys and the fake agent and scorer are unchanged.
+- The validation harness now gives disposable provider shims an explicit CommonJS package boundary; pins benchmark Git operations to their own initialized repository and rejects ancestor discovery and dangerous per-call redirects; and contains adversarial `--out`/`--index` traversal tests inside disposable enclosures before launching the tested CLI.
+- Tutorial and non-Git fixtures now establish their package and repository boundaries explicitly. On Windows, synthetic repository publication retries only transient `EPERM` rename failures, with a finite three-attempt limit; if staging cleanup also fails, its diagnostic is appended while the primary materialization error remains the reported error.
+- Packed-consumer staleness acceptance reuses the installed producer's reference-aware comparisons for all six scenarios and all three comparisons per scenario, including ordered reason codes; it retains the production artifact validator and does not change the scientific standard.
+- Replaced drive-root adversarial test probes with enclosed safe/unsafe witnesses. Network scanning recognizes two narrowly reviewed local HTTP sites only with strict source identities (CRLF-to-LF normalization only), expected fetch counts, caller evidence, and physical provenance; unreviewed calls and reportable provenance refusals remain major findings.
+- Corrected shared bounded source collection so ordinary files named like generated directories remain eligible, while actual generated directories stay excluded and case aliases cannot inherit reviewed trust.
+- Security command timeouts now use bounded process-tree termination; required nested security-suite failures retain actual timeout, exit, signal, start-error and cleanup evidence without weakening major-finding severity.
+- No Lab CLI flags, experiment schemas, or dependencies were added. Default retrieval behavior outside this experiment remains unchanged; persisted V1/V2 schemas, treatment ordering, scenarios, answer keys, fake agent, and scorer remain unchanged.
+
 ## [0.9.4] - 2026-10-23
 
 ### Kit schema-3 and npm 12 compatibility

package/README.md

@@ -10,7 +10,7 @@ The v0.9.3 release added hardened prompt contracts and output validation, extens
 
 v0.9.0 adds the `agent-success-rate` plugin, the eighth registered experiment plugin, which evaluates implementation patches under the `raw-full-file` and `context-pack` treatments.
 
-v0.9.3 is the previous package release.
+v0.9.4 is the previous package release.
 
 The v0.9.2 release corrected external npm package entrypoint validation, preserved strict Lab self-package validation, eliminated nine inherited false positives, corrected Semgrep failed/incomplete-scan evidence, and included campaign filesystem identity, path-safety, and cross-platform corrections. Security verdict strictness is unchanged. The v0.9.2 release-document state is final; actual npm publication is established separately through registry evidence.
 
@@ -18,11 +18,13 @@ The v0.9.3 release adds configurable Codex/Claude prompt contracts, strict struc
 
 The v0.9.4 release supports my-dev-kit symbol-index schema 3 alongside schema 2, preserves qualified symbol identities, and handles ambiguous names conservatively. Package inventory validation supports npm 11 and npm 12 formats, and the packed-package verifier reports early failures reliably. Development-only Vite and Vitest dependencies are patched; security-verdict policy is unchanged. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md), [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/COMMANDS.md](docs/COMMANDS.md), and [docs/WORKFLOWS.md](docs/WORKFLOWS.md).
 
+The v0.9.5 release corrects incremental-change-staleness evidence interpretation and strengthens disposable validation boundaries. Its bounded multi-source evidence path is opt-in; default top-one retrieval is unchanged. Insufficient evidence remains unknown or inconclusive, and synthetic fake-agent scores stay distinct from observed retrieval. No Lab CLI flags or experiment schemas were added.
+
 v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
 
 ## Current capabilities
 
-Package metadata is `0.9.4`. The v0.9.4 release supports my-dev-kit symbol-index schema 3 alongside schema 2, preserves qualified identities, handles ambiguity conservatively, supports npm 11 and npm 12 package inventories, corrects packed-package verifier behavior, and patches development-only security dependencies. Security-verdict policy is unchanged. The v0.9.3 release added configurable Codex/Claude prompt contracts, strict structured-diff output handling, generic report sections, HTML metric evidence and caveats, source-based gallery items including tutorial manifests, and prompt-template comparison. Registered metrics have evidence-based glossary definitions and links, and prompt comparison validates persisted identity and checkpoint evidence. Real-provider improvement remains NOT_VERIFIED because no paid smoke was authorized; its measurement is a separate opt-in empirical objective, not a v0.9.3 release gate. Bounded tool-use remains prompt guidance, not provider enforcement. See [Current State](docs/CURRENT_STATE.md) and [the roadmap](docs/ROADMAP.md).
+Package metadata is `0.9.5`. The v0.9.5 release corrects bounded evidence collection and uncertainty handling in the incremental-change-staleness experiment, preserves default retrieval behavior, and strengthens disposable test and validation boundaries. It adds no command, flag, experiment schema, or dependency. The v0.9.4 release supports my-dev-kit symbol-index schema 3 alongside schema 2, preserves qualified identities, handles ambiguity conservatively, supports npm 11 and npm 12 package inventories, corrects packed-package verifier behavior, and patches development-only security dependencies. Security-verdict policy is unchanged. The v0.9.3 release added configurable Codex/Claude prompt contracts, strict structured-diff output handling, generic report sections, HTML metric evidence and caveats, source-based gallery items including tutorial manifests, and prompt-template comparison. Registered metrics have evidence-based glossary definitions and links, and prompt comparison validates persisted identity and checkpoint evidence. Real-provider improvement remains NOT_VERIFIED because no paid smoke was authorized; its measurement is a separate opt-in empirical objective, not a v0.9.3 release gate. Bounded tool-use remains prompt guidance, not provider enforcement. See [Current State](docs/CURRENT_STATE.md) and [the roadmap](docs/ROADMAP.md).
 
 - **Run context-strategy experiments:** compare `raw-full-file` with `my-dev-kit-guided` using deterministic fixtures or locally configured Codex and Claude CLIs.
 - **Measure warm-index reuse (v0.5.0–v0.5.1, released):** the `warm-index-reuse` experiment plugin builds one my-dev-kit index per benchmark project, reuses it across that project's tasks, and compares every task with a matched `raw-full-file` baseline. Reports separate one-time index-build cost from per-task retrieval cost, show amortized index cost and cumulative measurements, and include deterministic fake-agent correctness and token evidence; `plots generate` renders four warm-index SVG charts. The v0.5.1 release adds a dedicated 12-task benchmark corpus (`benchmarks/contracts/warm-index-benchmark-cases.json`: six medium and six large/mixed tasks tagged `localized`, `cross-module`, or `broad-change`) for comparing warm-index behavior as task count grows. Estimated context tokens (a character-based context-size estimate) and fake-agent total tokens (simulated harness telemetry) are reported separately, and neither is provider billing telemetry.

package/dist/src/evaluation/benchmarkSandbox/createBenchmarkSandbox.js

@@ -3,7 +3,7 @@ import { copyFile, lstat, mkdir, realpath, rm, stat, writeFile } from "node:fs/p
 import path from "node:path";
 import { resolveWithinRoot } from "../../core/pathSafety.js";
 import { BenchmarkSandboxError } from "./errors.js";
-import { excerptGitFailure, runSandboxGit } from "./gitExecutor.js";
+import { excerptGitFailure, runSandboxGit, verifySandboxRepositoryBinding } from "./gitExecutor.js";
 import { BENCHMARK_SANDBOX_EXCLUDED_NAMES, isSameOrInside, resolvePhysicalPath, SANDBOX_ID_PATTERN } from "./pathPolicy.js";
 import { digestTreeSnapshot, scanProjectTree, snapshotProjectTree } from "./treeSnapshot.js";
 import { BENCHMARK_SANDBOX_BASELINE_BRANCH, BENCHMARK_SANDBOX_OWNER_MARKER_FILE, BENCHMARK_SANDBOX_SCHEMA_VERSION } from "./types.js";
@@ -93,6 +93,14 @@ export async function createBenchmarkSandbox(options) {
         }
         const git = { projectRoot, evidenceRoot };
         await mustGit(git, ["init", "-q", `--initial-branch=${BENCHMARK_SANDBOX_BASELINE_BRANCH}`], "init");
+        // A zero exit from init is not proof that later commands reach this repository: on some hosts Git discovery
+        // skips the new .git and selects an ancestor. Nothing may be configured, staged or committed before this passes.
+        const binding = await verifySandboxRepositoryBinding(git);
+        if (!binding.ok) {
+            throw binding.unavailable
+                ? new BenchmarkSandboxError("GIT_UNAVAILABLE", "the git executable is not available.")
+                : new BenchmarkSandboxError("GIT_REPOSITORY_MISMATCH", binding.reason);
+        }
         await mustGit(git, ["config", "user.name", "my-dev-kit-lab sandbox"], "config-name");
         await mustGit(git, ["config", "user.email", "sandbox@my-dev-kit-lab.invalid"], "config-email");
         await mustGit(git, ["config", "commit.gpgsign", "false"], "config-gpg");

package/dist/src/evaluation/benchmarkSandbox/gitExecutor.js

@@ -1,8 +1,13 @@
+import { realpath as realpathCallback } from "node:fs";
+import { lstat } from "node:fs/promises";
 import path from "node:path";
+import { promisify } from "node:util";
 import { runMeasuredCommand } from "../../core/runMeasuredCommand.js";
 import { buildMinimalHostEnv } from "./minimalHostEnv.js";
 // Git (including Git for Windows) opens this as an empty config file; os.devNull (\\.\nul) is rejected by Git on Windows.
 const NULL_CONFIG_PATH = process.platform === "win32" ? "NUL" : "/dev/null";
+/** Variables that can redirect which repository, index or object store a Git command operates on. */
+const REPOSITORY_REDIRECT_ENV = /^GIT_(DIR|WORK_TREE|COMMON_DIR|INDEX_FILE|OBJECT_DIRECTORY|ALTERNATE_OBJECT_DIRECTORIES|NAMESPACE|CEILING_DIRECTORIES|PREFIX|DISCOVERY_ACROSS_FILESYSTEM)$/i;
 const DEFAULT_GIT_TIMEOUT_MS = 60_000;
 const GIT_OUTPUT_LIMIT_BYTES = 8 * 1024 * 1024;
 const sequenceByEvidenceRoot = new Map();
@@ -17,6 +22,18 @@ function nextSequence(evidenceRoot) {
  * and the system's Git configuration so hashes and diffs do not depend on the developer machine.
  */
 export async function runSandboxGit(target, args, options = {}) {
+    const redirecting = Object.keys(options.env ?? {}).find((key) => REPOSITORY_REDIRECT_ENV.test(key));
+    if (redirecting !== undefined) {
+        return {
+            ok: false,
+            exitCode: null,
+            stdout: "",
+            stderr: "",
+            timedOut: false,
+            unavailable: false,
+            error: `sandbox git refuses an environment override that can redirect the repository (${redirecting}).`
+        };
+    }
     const label = (options.label ?? args[0] ?? "git").replace(/[^A-Za-z0-9_-]/g, "-").slice(0, 32);
     const commandId = `git-${String(nextSequence(target.evidenceRoot)).padStart(4, "0")}-${label}`;
     const env = buildMinimalHostEnv(process.env, {
@@ -26,7 +43,15 @@ export async function runSandboxGit(target, args, options = {}) {
         GIT_OPTIONAL_LOCKS: "0",
         HOME: target.evidenceRoot,
         USERPROFILE: target.evidenceRoot,
-        ...options.env
+        ...options.env,
+        // The sandbox project sits below other directories that may be Git repositories. Never let discovery walk
+        // above it, and (unless verifying) never rely on discovery at all. The pin is relative to the project working
+        // directory on purpose: Git for Windows rejects an absolute GIT_DIR of roughly 245 characters or more
+        // ("'$GIT_DIR' too big") although the repository itself works at those lengths.
+        GIT_CEILING_DIRECTORIES: path.dirname(target.projectRoot),
+        ...(options.repositoryBinding === "discovered"
+            ? {}
+            : { GIT_DIR: ".git", GIT_WORK_TREE: "." })
     });
     const result = await runMeasuredCommand({
         commandId,
@@ -60,3 +85,68 @@ export function excerptGitFailure(result) {
     const text = (result.stderr || result.stdout || result.error || "no output").trim().replace(/\s+/g, " ");
     return text.length > 300 ? `${text.slice(0, 300)}...` : text;
 }
+function samePhysicalPath(left, right) {
+    const caseFold = process.platform === "win32" || process.platform === "darwin";
+    const a = path.normalize(left);
+    const b = path.normalize(right);
+    return caseFold ? a.toLowerCase() === b.toLowerCase() : a === b;
+}
+// The native variant resolves Windows 8.3 short names and reports the on-disk case, matching what Git prints.
+const realpathNative = promisify(realpathCallback.native);
+async function physical(candidate) {
+    return realpathNative(path.resolve(candidate));
+}
+/**
+ * Proves that Git, discovering from the sandbox project (bounded by a ceiling at its parent), resolves to the
+ * project's own physically contained `.git` directory and to the project as work tree. Exit status of
+ * `git init` is not evidence of this; on some hosts `init` succeeds while discovery selects an ancestor.
+ */
+export async function verifySandboxRepositoryBinding(target) {
+    const fail = (reason, unavailable = false) => ({ ok: false, unavailable, reason });
+    const gitEntry = path.join(target.projectRoot, ".git");
+    let stats;
+    try {
+        stats = await lstat(gitEntry);
+    }
+    catch {
+        return fail("the sandbox project has no .git entry after initialization.");
+    }
+    if (stats.isSymbolicLink() || !stats.isDirectory()) {
+        return fail("the sandbox .git entry is not a real local directory.");
+    }
+    let expectedRoot;
+    try {
+        expectedRoot = await physical(target.projectRoot);
+        if (!samePhysicalPath(await physical(gitEntry), path.join(expectedRoot, ".git"))) {
+            return fail("the sandbox .git directory is not physically contained in the sandbox project.");
+        }
+    }
+    catch (error) {
+        return fail(`the sandbox repository paths could not be resolved: ${error instanceof Error ? error.message : String(error)}`);
+    }
+    const result = await runSandboxGit(target, ["rev-parse", "--absolute-git-dir", "--show-toplevel"], {
+        label: "verify-root",
+        repositoryBinding: "discovered"
+    });
+    if (result.unavailable)
+        return fail("the git executable is not available.", true);
+    if (!result.ok) {
+        return fail(`git does not recognize the new sandbox repository from its project directory: ${excerptGitFailure(result)}`);
+    }
+    const lines = result.stdout.split(/\r?\n/).filter((line) => line.trim() !== "");
+    if (lines.length !== 2)
+        return fail("git repository discovery returned an unexpected shape.");
+    try {
+        const [discoveredGitDir, discoveredTopLevel] = await Promise.all([physical(lines[0]), physical(lines[1])]);
+        if (!samePhysicalPath(discoveredGitDir, path.join(expectedRoot, ".git"))) {
+            return fail("git discovered a repository administration directory other than the sandbox project's own .git.");
+        }
+        if (!samePhysicalPath(discoveredTopLevel, expectedRoot)) {
+            return fail("git discovered a work tree other than the sandbox project.");
+        }
+    }
+    catch (error) {
+        return fail(`the discovered repository paths could not be resolved: ${error instanceof Error ? error.message : String(error)}`);
+    }
+    return { ok: true };
+}

package/dist/src/evaluation/runMyDevKitRetrieval.js

@@ -2,7 +2,7 @@ import path from "node:path";
 import { countEstimatedTokens, countTextChars, tokenCountMethod } from "../core/countTokens.js";
 import { runMeasuredCommand } from "../core/runMeasuredCommand.js";
 import { interpretToolVersionOutput } from "./indexSnapshot.js";
-import { buildRetrievalEvidenceFromCommands } from "./retrievalQuality/buildRetrievalEvidence.js";
+import { buildRetrievalEvidenceFromCommands, normalizeRetrievedRepositoryPath } from "./retrievalQuality/buildRetrievalEvidence.js";
 import { buildQueryStrategyEvidenceFromRetrievalEvidence } from "./retrievalQueryStrategyEvidence.js";
 const VERSION_PROBE_TIMEOUT_MS = 30_000;
 function parseJsonIfPossible(text) {
@@ -253,6 +253,39 @@ function selectRetrievalStrategyContextText(strategyId, outputs) {
                     : [outputs.source, outputs.slice, outputs.lookup, outputs.search];
     return ordered.find((text) => text && text.trim().length > 0) ?? "";
 }
+export const BOUNDED_SOURCE_PROTOCOL_ID = "bounded-multi-source-v1";
+export const DEFAULT_BOUNDED_SOURCE_OPTIONS = { maxSourceFiles: 8, searchLimit: 40 };
+export function isBoundedRetrievalResult(result) {
+    return result.boundedCoverage?.protocol === BOUNDED_SOURCE_PROTOCOL_ID;
+}
+/**
+ * Pure, deterministic candidate plan over search results. Only `file`/`symbol` entries (or legacy entries that
+ * carry no `kind`) with a node id and a safe repository-relative path are eligible; edges and other kinds are
+ * ignored. The first occurrence of each normalized file wins, rank order is preserved, and the plan is cut at
+ * `maxSourceFiles`. Never inspects an answer key.
+ */
+export function planBoundedSourceCandidates(results, maxSourceFiles) {
+    const budget = Number.isInteger(maxSourceFiles) && maxSourceFiles > 0 ? maxSourceFiles : 0;
+    const seen = new Set();
+    const eligible = [];
+    let rejectedCount = 0;
+    for (const entry of results) {
+        const kind = entry.kind;
+        if (kind !== undefined && kind !== "file" && kind !== "symbol")
+            continue;
+        const fields = pickCandidateFields(entry);
+        const file = normalizeRetrievedRepositoryPath(fields.file);
+        if (file === null || fields.nodeId === undefined) {
+            rejectedCount += 1;
+            continue;
+        }
+        if (seen.has(file))
+            continue;
+        seen.add(file);
+        eligible.push({ file, nodeId: fields.nodeId });
+    }
+    return { candidates: eligible.slice(0, budget), eligibleFileCount: eligible.length, rejectedCount };
+}
 const STRATEGY_DOWNSTREAM_COMMANDS = {
     "keyword-search": [],
     "symbol-lookup": ["lookup"],
@@ -275,7 +308,15 @@ export async function runMyDevKitRetrievalStrategyFromIndex(options) {
         commandString: options.kitCommand,
         cwd: process.cwd(),
         outDir: commandsDir,
-        extraArgs: ["search", "--index", indexDir, "--query", options.evaluationCase.query, "--json"]
+        extraArgs: [
+            "search",
+            "--index",
+            indexDir,
+            "--query",
+            options.evaluationCase.query,
+            ...(options.boundedSource ? ["--limit", String(options.boundedSource.searchLimit)] : []),
+            "--json"
+        ]
     });
     commands.push(searchCommand);
     if (!searchCommand.ok) {
@@ -324,8 +365,28 @@ export async function runMyDevKitRetrievalStrategyFromIndex(options) {
             warnings.push("No my-dev-kit node id was available after search.");
         }
     }
-    const contextText = selectRetrievalStrategyContextText(options.strategyId, outputs);
-    const filesRead = selectedFile ? [selectedFile] : [];
+    let contextText = selectRetrievalStrategyContextText(options.strategyId, outputs);
+    let filesRead = selectedFile ? [selectedFile] : [];
+    let boundedCoverage;
+    if (options.boundedSource && downstream.includes("source")) {
+        const bounded = await expandBoundedSourceCandidates({
+            bounded: options.boundedSource,
+            candidates,
+            selectedNodeId,
+            selectedSourceOk: commands.some((command) => command.commandId === "source" && command.ok),
+            kitCommand: options.kitCommand,
+            indexDir,
+            commandsDir,
+            commands,
+            warnings
+        });
+        boundedCoverage = bounded.coverage;
+        // Bounded mode: filesRead names only files whose own `source` command succeeded, in rank order.
+        filesRead = bounded.coverage.attempts.filter((attempt) => attempt.sourced).map((attempt) => attempt.file);
+        if (bounded.extraSourceTexts.length > 0) {
+            contextText = [contextText, ...bounded.extraSourceTexts].filter((text) => text.length > 0).join("\n\n");
+        }
+    }
     const retrievalEvidence = buildRetrievalEvidenceFromCommands(commands, { nodeId: selectedNodeId, file: selectedFile });
     return {
         caseId: options.evaluationCase.id,
@@ -342,9 +403,81 @@ export async function runMyDevKitRetrievalStrategyFromIndex(options) {
         selectedSymbol,
         retrievalEvidence,
         queryStrategyEvidence: buildQueryStrategyEvidenceFromRetrievalEvidence(options.strategyId, retrievalEvidence),
+        ...(boundedCoverage ? { boundedCoverage } : {}),
         durationMs: Date.now() - started
     };
 }
+async function expandBoundedSourceCandidates(args) {
+    const plan = planBoundedSourceCandidates(args.candidates, args.bounded.maxSourceFiles);
+    const attempts = [];
+    const extraSourceTexts = [];
+    for (const [index, candidate] of plan.candidates.entries()) {
+        if (candidate.nodeId === args.selectedNodeId) {
+            // The first search candidate already received its `source` command in the historical lifecycle.
+            attempts.push({ ...candidate, sourced: args.selectedSourceOk });
+            continue;
+        }
+        const command = await runMeasuredCommand({
+            commandId: `source-candidate-${index + 1}`,
+            commandString: args.kitCommand,
+            cwd: process.cwd(),
+            outDir: args.commandsDir,
+            extraArgs: ["source", "--index", args.indexDir, "--node", candidate.nodeId, "--max-lines", "160", "--format", "numbered"]
+        });
+        args.commands.push(command);
+        if (!command.ok) {
+            args.warnings.push(`my-dev-kit source command failed for bounded candidate ${index + 1}.`);
+        }
+        else if (command.stdout.length > 0) {
+            extraSourceTexts.push(command.stdout);
+        }
+        attempts.push({ ...candidate, sourced: command.ok });
+    }
+    const resultCount = args.candidates.length;
+    return {
+        coverage: {
+            protocol: BOUNDED_SOURCE_PROTOCOL_ID,
+            maxSourceFiles: args.bounded.maxSourceFiles,
+            searchLimit: args.bounded.searchLimit,
+            searchResultCount: resultCount,
+            searchTruncated: resultCount >= args.bounded.searchLimit,
+            eligibleCandidateFileCount: plan.eligibleFileCount,
+            candidateBudgetExhausted: plan.eligibleFileCount > plan.candidates.length,
+            rejectedCandidateCount: plan.rejectedCount,
+            attempts
+        },
+        extraSourceTexts
+    };
+}
+/**
+ * Opt-in bounded multi-source retrieval for evaluations that must observe more than one file (for example
+ * multi-file required-file evidence). Reuses the shared strategy executor; default top-one retrieval and the
+ * meaning of its `filesRead` are unchanged for every other caller. The result always carries
+ * `boundedCoverage`, including when the search yielded no usable candidate.
+ */
+export async function runMyDevKitBoundedRetrievalFromIndex(options) {
+    const boundedSource = options.boundedSource ?? DEFAULT_BOUNDED_SOURCE_OPTIONS;
+    const result = await runMyDevKitRetrievalStrategyFromIndex({ ...options, strategyId: "combined-graph-guided", boundedSource });
+    if (isBoundedRetrievalResult(result))
+        return result;
+    // Skipped before expansion (search failed or yielded no candidate): still declare the protocol, nothing sourced.
+    const skipped = {
+        ...result,
+        filesRead: [],
+        boundedCoverage: {
+            protocol: BOUNDED_SOURCE_PROTOCOL_ID,
+            maxSourceFiles: boundedSource.maxSourceFiles,
+            searchLimit: boundedSource.searchLimit,
+            searchResultCount: 0,
+            searchTruncated: false,
+            eligibleCandidateFileCount: 0,
+            candidateBudgetExhausted: false,
+            rejectedCandidateCount: 0,
+            attempts: []
+        }
+    };
+    return skipped;
+}
 /**
  * v0.8.0 compatibility entrypoint: the historical search -> lookup -> slice -> source lifecycle,
  * delegating to the `combined-graph-guided` strategy. Never invokes `index`.

package/dist/src/evaluation/syntheticRepository/materialize.js

@@ -20,6 +20,19 @@ export class SyntheticRepositoryMaterializationError extends Error {
     }
 }
 const STAGING_PREFIX = ".synthetic-staging-";
+/** Total attempts at the publication rename (initial try plus two retries); Windows EPERM only. */
+const PUBLISH_RENAME_MAX_ATTEMPTS = 3;
+/** Fixed waits before the second and third attempts (100 ms maximum for one materialization). */
+const PUBLISH_RENAME_RETRY_DELAYS_MS = [25, 75];
+/** Raised when the owned staging directory can no longer be proven to be ours; cleanup must not touch it. */
+class UnownedStagingError extends SyntheticRepositoryMaterializationError {
+}
+function sleepSync(milliseconds) {
+    Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, milliseconds);
+}
+function isTransientWindowsRenameError(error, platform) {
+    return platform === "win32" && typeof error === "object" && error !== null && error.code === "EPERM";
+}
 function withinRealRoot(realRoot, candidate) {
     try {
         relativeWithinRoot(realRoot, realpathSync(candidate));
@@ -63,12 +76,64 @@ function describeExisting(caseDirectory, expectedManifestText) {
         return { reusable: false, reasons: verification.issues };
     return { reusable: true };
 }
+/** Proves, before a rename retry, that the owned staging directory is still ours and the destination is still free. */
+function assertStagingStillPublishable(realRoot, staging, caseDirectory, caseId) {
+    let rootStillVerified = false;
+    try {
+        rootStillVerified = realpathSync(realRoot) === realRoot;
+    }
+    catch {
+        rootStillVerified = false;
+    }
+    if (!rootStillVerified) {
+        throw new UnownedStagingError("path-escape", ["output root no longer resolves to the verified physical root; the staging rename was not retried."]);
+    }
+    let stats;
+    try {
+        stats = lstatSync(staging);
+    }
+    catch {
+        throw new UnownedStagingError("io-failure", ["owned staging directory disappeared before the staging rename retry; nothing was published."]);
+    }
+    if (stats.isSymbolicLink() || !stats.isDirectory() || path.dirname(staging) !== realRoot || !withinRealRoot(realRoot, staging)) {
+        throw new UnownedStagingError("path-escape", ["owned staging directory is no longer a plain directory inside the output root; the staging rename was not retried."]);
+    }
+    if (exists(caseDirectory)) {
+        throw new SyntheticRepositoryMaterializationError("collision", [`output child ${caseId} appeared during materialization; nothing was overwritten.`]);
+    }
+}
+/**
+ * The single atomic publication of a verified staging directory. On Windows an externally induced, brief EPERM from this
+ * one rename is retried up to the fixed budget (3 attempts, 25 ms then 75 ms apart) after the ownership and destination
+ * guards are re-proven; every other error, and every non-Windows platform, gets exactly one attempt. Returns the number
+ * of attempts used.
+ */
+function publishStagingDirectory(options) {
+    for (let attempt = 1;; attempt += 1) {
+        try {
+            options.renameDirectory(options.staging, options.caseDirectory);
+            return attempt;
+        }
+        catch (error) {
+            if (!isTransientWindowsRenameError(error, options.platform))
+                throw error;
+            if (attempt >= PUBLISH_RENAME_MAX_ATTEMPTS) {
+                throw new SyntheticRepositoryMaterializationError("io-failure", [
+                    `final staging directory rename failed with EPERM after ${attempt} attempts (retry budget exhausted); nothing was copied, overwritten or published.`,
+                ]);
+            }
+            options.sleep(PUBLISH_RENAME_RETRY_DELAYS_MS[attempt - 1]);
+            assertStagingStillPublishable(options.realRoot, options.staging, options.caseDirectory, options.caseId);
+        }
+    }
+}
 /**
  * Materializes a frozen Batch 1 plan below `outputRoot/<caseId>/` as `repository/` plus a sibling
  * `synthetic-repository-manifest.json`. The plan is the single semantic source of truth: it is verified, never
  * re-planned or repaired. A deterministic existing directory is reused only when its manifest and every file
  * match the requested generation exactly; any other existing child is a collision. Writes go to an owned staging
- * directory that is verified and then renamed into place; on failure only that staging directory is removed.
+ * directory that is verified and then renamed into place (a brief, externally induced Windows EPERM at that single
+ * rename is retried within a fixed budget, see `publishStagingDirectory`); on failure only that staging directory is removed.
  */
 export function materializeSyntheticRepository(plan, outputRoot, io = {}) {
     try {
@@ -156,15 +221,46 @@ export function materializeSyntheticRepository(plan, outputRoot, io = {}) {
         if (exists(caseDirectory)) {
             throw new SyntheticRepositoryMaterializationError("collision", [`output child ${plan.caseId} appeared during materialization; nothing was overwritten.`]);
         }
-        renameSync(staging, caseDirectory);
+        const attempts = publishStagingDirectory({
+            staging,
+            caseDirectory,
+            caseId: plan.caseId,
+            realRoot,
+            renameDirectory: io.renameDirectory ?? renameSync,
+            sleep: io.sleep ?? sleepSync,
+            platform: io.platform ?? process.platform,
+        });
         staging = undefined;
+        if (attempts > 1) {
+            // The staged tree was verified before the rename; after a retried publication the destination is re-verified too.
+            // It now holds this invocation's output, so a failure here is reported and the destination is left in place.
+            const publishedManifest = path.join(caseDirectory, SYNTHETIC_REPOSITORY_MANIFEST_FILE_NAME);
+            const published = verifySyntheticRepositoryMaterialization({
+                manifestPath: publishedManifest,
+                repositoryRoot: path.join(caseDirectory, SYNTHETIC_REPOSITORY_REPOSITORY_DIRECTORY),
+            });
+            if (!published.ok || readFileSync(publishedManifest, "utf8") !== manifestText) {
+                throw new SyntheticRepositoryMaterializationError("io-failure", [
+                    `published output for ${plan.caseId} failed verification after the staging rename succeeded on attempt ${attempts}; the destination was left in place for inspection.`,
+                    ...published.issues,
+                ]);
+            }
+        }
     }
     catch (error) {
-        if (staging !== undefined)
-            rmSync(staging, { recursive: true, force: true });
-        if (error instanceof SyntheticRepositoryMaterializationError)
-            throw error;
-        throw new SyntheticRepositoryMaterializationError("io-failure", [error instanceof Error ? error.message : String(error)]);
+        let cleanupFailure;
+        if (staging !== undefined && !(error instanceof UnownedStagingError)) {
+            try {
+                rmSync(staging, { recursive: true, force: true });
+            }
+            catch (cleanupError) {
+                cleanupFailure = `cleanup of the owned staging directory failed: ${cleanupError instanceof Error ? cleanupError.message : String(cleanupError)}`;
+            }
+        }
+        const reported = error instanceof SyntheticRepositoryMaterializationError ? error : new SyntheticRepositoryMaterializationError("io-failure", [error instanceof Error ? error.message : String(error)]);
+        if (cleanupFailure === undefined)
+            throw reported;
+        throw new SyntheticRepositoryMaterializationError(reported.code, [...reported.errors, cleanupFailure]);
     }
     return result(false);
 }

package/dist/src/experiments/plugins/incrementalChangeStaleness/comparison.js

@@ -71,6 +71,16 @@ export function compareRequiredFileEvidence(stale, fullRefresh) {
         return "stale-better";
     return "same";
 }
+/**
+ * Reference-adequacy gate over the section 30 rules. Equal evidence only supports "same" when the full-refresh
+ * reference itself observed every required file: two treatments that both lack the required files are equally
+ * insufficient, which is not evidence that the candidate did not regress. That case is `unknown`. A candidate
+ * that observed files the insufficient reference did not is still reported as better; nothing else changes.
+ */
+export function compareRequiredFileEvidenceAgainstReference(candidate, reference) {
+    const relation = compareRequiredFileEvidence(candidate, reference);
+    return relation === "same" && reference.status !== "present" ? "unknown" : relation;
+}
 /**
  * Sections 31-35: the frozen precedence algorithm. Correctness and required-file evidence are the
  * only two inputs; affected-neighborhood evidence and reindexRecommendation are never inputs here.

package/dist/src/experiments/plugins/incrementalChangeStaleness/comparisonV2.js

@@ -1,4 +1,4 @@
-import { compareCorrectness, compareRequiredFileEvidence } from "./comparison.js";
+import { compareCorrectness, compareRequiredFileEvidence, compareRequiredFileEvidenceAgainstReference } from "./comparison.js";
 function neutral(relation) {
     if (relation === "stale-worse")
         return "candidate-worse";
@@ -14,6 +14,14 @@ export function compareCandidateCorrectness(candidate, reference) {
 export function compareCandidateRequiredFileEvidence(candidate, reference) {
     return neutral(compareRequiredFileEvidence(candidate, reference));
 }
+/**
+ * Candidate-vs-reference required-file relation with the reference-adequacy gate: equal weakness (both
+ * missing) is `unknown`, not `same`, because an insufficient full-refresh reference cannot support a
+ * no-regression statement. Every other outcome follows `compareCandidateRequiredFileEvidence`.
+ */
+export function compareCandidateRequiredFileEvidenceAgainstReference(candidate, reference) {
+    return neutral(compareRequiredFileEvidenceAgainstReference(candidate, reference));
+}
 /**
  * Frozen precedence. Rule 0 (realization gate) first: a full fallback is never evidence about
  * partial-refresh performance. Then observed regression (either dimension worse), then inconclusive

package/dist/src/experiments/plugins/incrementalChangeStaleness/execution.js

@@ -5,10 +5,10 @@ import { assessAffectedNeighborhood, mapAffectedNeighborhoodSeeds } from "../../
 import { classifyAgentRunOutcome } from "../../../evaluation/classifyAgentRunOutcome.js";
 import { parseAgentAnswer } from "../../../evaluation/parseAgentAnswer.js";
 import { readEvaluationCases } from "../../../evaluation/readEvaluationCases.js";
-import { runMyDevKitRetrievalFromIndex } from "../../../evaluation/runMyDevKitRetrieval.js";
+import { isBoundedRetrievalResult, runMyDevKitBoundedRetrievalFromIndex } from "../../../evaluation/runMyDevKitRetrieval.js";
 import { scoreCorrectness } from "../../../evaluation/scoreCorrectness.js";
 import { generatePromptVariants } from "../../../prompts/index.js";
-import { buildRequiredFileEvidence, classifyStaleRisk, compareCorrectness, compareRequiredFileEvidence } from "./comparison.js";
+import { buildRequiredFileEvidence, classifyStaleRisk, compareCorrectness, compareRequiredFileEvidenceAgainstReference } from "./comparison.js";
 import { WARM_INDEX_BENCHMARK_CASES_PATH } from "./scenarioCatalog.js";
 // ---------------------------------------------------------------------------
 // v0.6.2 Batch 4 -- treatment retrieval, correctness, and required-file
@@ -134,7 +134,9 @@ export function classifyIncrementalChangeStalenessRetrieval(result) {
     return result.commands.some((command) => !command.ok) ? "failed" : "skipped";
 }
 export async function runIncrementalChangeStalenessTreatmentRetrieval(args) {
-    return runMyDevKitRetrievalFromIndex({
+    // Opt-in bounded multi-source protocol: the staleness experiment must observe multi-file requirements,
+    // which the default first-candidate-only retrieval cannot. Candidates derive from the search output only.
+    return runMyDevKitBoundedRetrievalFromIndex({
         evaluationCase: args.derivedTask,
         kitCommand: args.kitCommand,
         indexDir: args.indexDir,
@@ -142,10 +144,41 @@ export async function runIncrementalChangeStalenessTreatmentRetrieval(args) {
         requireKit: false
     });
 }
+/**
+ * Required-file evidence under the bounded multi-source protocol. "Observed" means a successful `source`
+ * command for that file, never a search match. Fails closed: a protocol that cannot source as many files as
+ * are required, a failed source command, or rejected candidate identities leave unsourced required files
+ * `unknown` instead of `missing`. A `missing` result is relative to the fixed candidate budget and says so.
+ */
+function buildBoundedRequiredFileEvidence(args) {
+    const coverage = args.retrieval.boundedCoverage;
+    const observedFiles = coverage.attempts.filter((attempt) => attempt.sourced).map((attempt) => attempt.file);
+    const unknown = (unknownReason) => buildRequiredFileEvidence({ requiredFiles: args.requiredFiles, observedFiles, observedFilesComplete: false, unknownReason });
+    const required = buildRequiredFileEvidence({ requiredFiles: args.requiredFiles, observedFiles, observedFilesComplete: true });
+    if (required.requiredFiles.length > coverage.maxSourceFiles) {
+        return unknown(`Coverage-limited: the bounded retrieval protocol sources at most ${coverage.maxSourceFiles} candidate file(s) but ${required.requiredFiles.length} are required.`);
+    }
+    if (required.status === "present")
+        return required;
+    const failedAttempts = coverage.attempts.filter((attempt) => !attempt.sourced).length;
+    if (failedAttempts > 0) {
+        return unknown(`${failedAttempts} bounded source command(s) failed; unsourced required files cannot be classified as absent.`);
+    }
+    if (coverage.rejectedCandidateCount > 0) {
+        return unknown(`${coverage.rejectedCandidateCount} search candidate(s) were rejected (unsafe file identity or no node id); unsourced required files cannot be classified as absent.`);
+    }
+    return {
+        ...required,
+        reason: `Required file(s) were not sourced within the bounded retrieval protocol (${observedFiles.length} of up to ${coverage.maxSourceFiles} candidate file(s) sourced; ` +
+            `${coverage.eligibleCandidateFileCount} eligible; search truncated: ${coverage.searchTruncated ? "yes" : "no"}; candidate budget exhausted: ${coverage.candidateBudgetExhausted ? "yes" : "no"}). ` +
+            "This is not an exhaustive source-set assessment."
+    };
+}
 /**
  * Section 27: a `failed` retrieval status means the actual-files-read set cannot be trusted as
  * complete (unknown); `completed` and `skipped` (search ran and legitimately found no candidate)
- * both provide a complete, trustworthy files-read set, possibly empty.
+ * both provide a complete, trustworthy files-read set, possibly empty. A result produced by the opt-in
+ * bounded protocol is evaluated against its own declared coverage instead.
  */
 export function buildIncrementalChangeStalenessRequiredFileEvidence(args) {
     if (!args.retrieval) {
@@ -157,6 +190,9 @@ export function buildIncrementalChangeStalenessRequiredFileEvidence(args) {
         });
     }
     const status = classifyIncrementalChangeStalenessRetrieval(args.retrieval);
+    if (status !== "failed" && isBoundedRetrievalResult(args.retrieval)) {
+        return buildBoundedRequiredFileEvidence({ requiredFiles: args.requiredFiles, retrieval: args.retrieval });
+    }
     return buildRequiredFileEvidence({
         requiredFiles: args.requiredFiles,
         observedFiles: args.retrieval.filesRead,
@@ -412,7 +448,7 @@ export async function executeIncrementalChangeStalenessScenario(options) {
         deps
     });
     const correctnessRelation = compareCorrectness(correctnessComparable(stale.fakeAgent), correctnessComparable(fullRefresh.fakeAgent));
-    const requiredFileEvidenceRelation = compareRequiredFileEvidence(stale.requiredFileEvidence, fullRefresh.requiredFileEvidence);
+    const requiredFileEvidenceRelation = compareRequiredFileEvidenceAgainstReference(stale.requiredFileEvidence, fullRefresh.requiredFileEvidence);
     const classification = classifyStaleRisk(correctnessRelation, requiredFileEvidenceRelation);
     return {
         ...scenarioBase,