@dailephd/my-dev-kit-lab 0.4.8 → 0.5.0

package/CHANGELOG.md

@@ -2,6 +2,40 @@
 
 All notable changes to my-dev-kit-lab are documented here.
 
+## [Unreleased]
+
+## [0.5.0] - 2026-09-21
+
+Warm-index reuse experiment support.
+
+- Added the experimental `warm-index-reuse` experiment plugin, registered alongside `context-strategy-comparison` and reachable through `experiment list`, `experiment describe`, and `experiment run`.
+- Built exactly one my-dev-kit index per benchmark-project group and reused it across every task in that group, with a matched `raw-full-file` baseline for each task; cases that disagree on target or source roots fail structurally instead of being indexed.
+- Separated index construction from retrieval: one measured `index` command per project, and search/lookup/slice/source against the prepared index for each task, without re-indexing.
+- Added deterministic fake-agent evaluation of each task side with context evidence, reusing the existing prompt generator, answer parser, run classifier, and correctness scorer.
+- Reported index build duration, raw context-construction and warm retrieval durations, raw and retrieved context size, estimated context tokens, amortized index build duration (build duration / task ordinal), cumulative component durations, cumulative estimated context tokens, fake-agent correctness, fake-agent total tokens, and cumulative fake-agent total tokens, each with explicit `available`/`unavailable`/`not-applicable` availability and strict-prefix cumulative rules.
+- Added a bounded warm-index section to the plugin `report.json`, `report.txt`, and `report.html`, explaining cold-start versus warm-reuse cost, with a neutral interpretation and explicit limitations. The bounded `warm-index-execution.json` records execution evidence without context text.
+- Added four warm-index plots through the existing `plots generate` command: amortized index cost, raw versus retrieved context size, fake-agent correctness, and cumulative fake-agent token usage.
+- Added the warm-index-only `experiment run --kit-command <command>` option (default `npx @dailephd/my-dev-kit@latest`), grouped `experiment run --help` options by plugin, and corrected `plots generate` help to describe supported inputs.
+- Extended the exact packed-package acceptance gate with installed warm-index execution, reports, and plots while preserving installed-package and target immutability.
+- Limitations: agent evidence comes from the deterministic fake agent only; fake-agent correctness and token totals are simulated harness evidence, not real-model results or provider billing telemetry; estimated context tokens remain a separate context-size estimate; cumulative durations are component sums, not wall-clock latency; no token-savings percentage, break-even task, winner, or ranking is calculated; the bundled cases provide one task per benchmark project; real-agent warm-index campaigns, warm-index screenshots, and gallery integration are not included.
+
+## [0.4.9] - 2026-09-20
+
+Semantic native-select option selection for declarative browser tutorials.
+
+- Added the value-only `select-option` declarative tutorial action, anchored to one existing `TutorialLocatorV1` and identifying a native HTML `<select>` option by its stable HTML `value`.
+- Executed the action through Playwright `Locator.selectOption({ value })` using the existing tutorial timeout model, with no retry.
+- Verified the returned selection exactly: the action passes only when the browser reports one selected value equal to the requested value; empty, mismatched, and multiple returned values fail.
+- Added no keyboard fallback: `ArrowDown`/`Enter` emulation, click fallback, `page.evaluate`, and event dispatch are all absent, and `press` remains real keyboard input.
+- Applied closed validation: `locator` and a non-empty trimmed `value` are required, timeouts use the existing contract, and unknown fields — including `label`, `index`, and `values` — are rejected rather than ignored.
+- Moved the synthetic cursor to the select element before execution without fabricating a click ripple or animating the native popup menu.
+- Extended the generic fixture (`examples/tutorial-browser/`) with a visible native `Operation` select whose ordinary `change` handler updates visible application state, and extended the canonical scenario to ten steps.
+- Proved the action against real Chromium, including test-owned live DOM inspection of the selected value and the resulting visible state.
+- Proved the action from the exact packed npm candidate installed into a clean consumer through the new `TUTORIAL_SELECT_OPTION` gate, preserving installed-package, packaged-example, and target immutability.
+- Validated on Windows, macOS, and Linux across Node 24 and Node latest through the existing CI matrix.
+- Confirmed bounded read-only downstream `my-frontend-observer` compatibility: representative native selects are expressible through the generic value-only contract, with no Observer values added to lab production code and no Observer files changed.
+- Kept `TutorialScenarioV1`, `TutorialTargetContractV1`, `TutorialRunResultV1`, and `TutorialManifestV1` schemas at version `1.0.0`.
+
 ## [0.4.8] - 2026-09-19
 
 Locator-anchored pointer gestures for declarative browser tutorials.

package/README.md

@@ -4,18 +4,23 @@ my-dev-kit-lab is the experiment, audit, and evidence companion for [my-dev-kit]
 
 my-dev-kit provides local repository indexing and graph-guided retrieval. my-dev-kit-lab supplies the controlled benchmarks, agent adapters, metrics, security checks, and reports needed to evaluate when that retrieval is useful. Results are evidence for a specific target and configuration; they do not guarantee token savings or security.
 
-The latest release is v0.4.8 (locator-anchored pointer gestures for declarative browser tutorials, adding `pointer-click` and `pointer-drag` using normalized locator-relative fraction positions while existing `drag` remains element-to-element). The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md).
+Whole-ecosystem workflows are centralized in [my-dev-kit/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md). Its [command-surface compatibility map](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md#915-command-surface-compatibility-map) documents which Lab outputs can safely seed my-dev-kit investigation, Observer reference work, or Orchestrator evidence, and which look-alike artifacts are not direct inputs.
+
+The latest release is v0.5.0 (warm-index reuse experiment support). The previous release, v0.4.9, added semantic native-select option selection for declarative browser tutorials through the value-only `select-option` action backed by Playwright `Locator.selectOption({ value })`. The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md).
+
+The current release includes v0.5.0 warm-index reuse. The package and installed CLI are version 0.5.0; v0.4.9 remains the previous release. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for the validated release state.
 
 v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
 
 ## Current capabilities
 
 - **Run context-strategy experiments:** compare `raw-full-file` with `my-dev-kit-guided` using deterministic fixtures or locally configured Codex and Claude CLIs.
+- **Measure warm-index reuse (v0.5.0):** the `warm-index-reuse` experiment plugin builds one my-dev-kit index per benchmark project, reuses it across that project's tasks, and compares every task with a matched `raw-full-file` baseline. Reports separate one-time index-build cost from per-task retrieval cost, show amortized index cost and cumulative measurements, and include deterministic fake-agent correctness and token evidence; `plots generate` renders four warm-index SVG charts. Estimated context tokens (a character-based context-size estimate) and fake-agent total tokens (simulated harness telemetry) are reported separately, and neither is provider billing telemetry. See [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment) and [docs/METRICS.md](docs/METRICS.md#warm-index-reuse-metrics).
 - **Audit repository health:** run conservative code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, or adapt security findings into the common audit report.
 - **Validate CLI/package security:** inspect dependencies, package contents, path and subprocess boundaries, malformed inputs, optional static scanners, and bounded fuzz targets.
 - **Validate Android projects:** run nineteen static checks by default, with Gradle operations, external tools, and network access available only through explicit opt-in flags.
 - **Review evidence:** generate JSON and HTML reports, SVG plots, optional screenshots, visualization demos, and a static gallery.
-- **Generate declarative browser tutorials:** validate and execute bounded local browser scenarios with WebM recording, step screenshots, SRT/VTT subtitles, Markdown, and a tutorial manifest. Locator-anchored pointer gestures (`pointer-click` and `pointer-drag`) operate using normalized fraction coordinates inside a single located surface, while existing `drag` remains element-to-element. Tutorial execution requires a compatible locally installed Chromium; see [docs/COMMANDS.md](docs/COMMANDS.md) and [docs/TUTORIAL.md](docs/TUTORIAL.md) for details.
+- **Generate declarative browser tutorials:** validate and execute bounded local browser scenarios with WebM recording, step screenshots, SRT/VTT subtitles, Markdown, and a tutorial manifest. Locator-anchored pointer gestures (`pointer-click` and `pointer-drag`) operate using normalized fraction coordinates inside a single located surface, while existing `drag` remains element-to-element and `press` remains real keyboard input. `select-option` performs semantic native HTML `<select>` option selection by stable HTML value, executed through Playwright `Locator.selectOption({ value })` rather than keyboard navigation. Tutorial execution requires a compatible locally installed Chromium; see [docs/COMMANDS.md](docs/COMMANDS.md) and [docs/TUTORIAL.md](docs/TUTORIAL.md) for details.
 - **Evaluate stage-context strategies:** compare the two legacy strategies against six additional bounded stage-context strategies — `architecture-context-only`, `architecture-plus-implementation-refresh`, `architecture-plus-implementation-and-test-refresh`, `full-workflow-library`, `bounded-workflow-instruction-packet`, and `combined-bounded-stage-context` — selected through programmatic configuration, not CLI flags. Each strategy's evidence is reported through bounded `report.json`, `report.html`, and `report.txt` output with an explicit `available`/`unavailable`/`not-applicable` metric-availability model and no composite score, grade, ranking, or winning strategy.
 - **Evaluate the producer-readiness bridge (v0.4.4):** optionally extend `combined-bounded-stage-context` with the frozen my-dev-kit-orchestrator supplemental implementation/test-context packet and retrieval-report documents plus an observed readiness result, all supplied programmatically (there is no CLI flag), to measure owner, allocation, truncation-cause, supplemental/raw agreement, readiness-agreement, and criticality-overlay evidence without reimplementing upstream owner-selection, allocation, producer-parity, or readiness policy.
 - **Evaluate context integrity (v0.4.5):** compare condition-aware producer evidence from my-dev-kit v1.10.4 (role condition coverage, allocation/spillover, required-evidence-loss) against my-dev-kit-orchestrator v1.2.3 run-integrity evidence (run-integrity gate, judge integrity, final-report eligibility, artifact lifecycle state), reporting agreement or contradiction between them rather than re-deriving a verdict. Evaluation runs against a frozen, hash-verified regression fixture pair — a byte-exact real historical failed run and a hand-distilled corrected-replay counterpart representing the same validated contracts — and is programmatic/test-driven only. `npm run report:context-integrity-smoke` renders both fixtures' reports for manual inspection; it takes no arguments and is a developer convenience, not a configurable evaluation CLI.
@@ -151,6 +156,19 @@ npm run experiment:run -- `
   --no-screenshot
 ```
 
+Run the warm-index reuse plugin deterministically with the fake my-dev-kit fixture (omit `--kit-command` to use the default `npx @dailephd/my-dev-kit@latest`):
+
+```bash
+npm run experiment:run -- \
+  --experiment warm-index-reuse \
+  --cases tests/fixtures/warm-index-reuse/multi-task-cases.json \
+  --kit-command "node tests/fixtures/fake-my-dev-kit-cli.js" \
+  --out lab-output/warm-index-reuse
+npm run generate-experiment-plots -- --experiment lab-output/warm-index-reuse --out lab-output/warm-index-plots
+```
+
+The bundled `examples/token-savings-cases.json` currently has one task per benchmark project, so multi-task reuse needs a cases file with several tasks per project, such as the test fixture above; an expanded bundled warm-index benchmark suite is planned.
+
 When `--target` is omitted, the experiment runs in self mode against my-dev-kit-lab. When `--target <path>` is provided, the lab remains the tool root and the target project is inspected separately. Generated experiment outputs stay under lab-controlled output directories by default, not inside the target project.
 
 ---
@@ -168,6 +186,7 @@ When `--target` is omitted, the experiment runs in self mode against my-dev-kit-
 | Plugin experiment report JSON | `lab-output/experiments/<plugin>/<target>/<run>/report.json` |
 | Plugin experiment report HTML | `lab-output/experiments/<plugin>/<target>/<run>/report.html` |
 | Plugin experiment report text | `lab-output/experiments/<plugin>/<target>/<run>/report.txt` |
+| Warm-index execution evidence | `<experiment out>/warm-index-execution.json` (plus `indexes/`, `commands/`, and `agents/` beneath the same output root) |
 | Plot data | `lab-output/<plots>/plot-data.json` |
 | SVG charts | `lab-output/<plots>/charts/*.svg` |
 | Gallery manifest | `lab-output/<gallery>/gallery-manifest.json` |
@@ -201,7 +220,8 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 - Claude does not expose token totals; token savings comparisons are unavailable for Claude runs
 - Codex may expose token totals but can produce timeouts or invalid-output runs
 - Small projects may make raw-full-file cheaper than my-dev-kit-guided; larger localized tasks are where my-dev-kit is expected to become more useful
-- The generic experiment-plugin framework currently ships one plugin, `context-strategy-comparison`; future plugins such as warm-index reuse, incremental-change, and context-window scaling are not implemented yet
+- The generic experiment-plugin framework has two plugins: `context-strategy-comparison` and the current-release v0.5.0 `warm-index-reuse`. Later plugins such as incremental-change and context-window scaling are future roadmap work
+- `warm-index-reuse` uses the deterministic fake agent only; selectable Codex/Claude warm-index campaigns remain planned for v0.5.2, the expanded benchmark suite remains planned for v0.5.1, and warm-index screenshots/gallery integration are not implemented. Its component-duration sums are not wall-clock latency, and it calculates no token-savings percentage, break-even task, winner, or ranking
 - The current release does not guarantee token savings; it produces auditable evidence for specific cases, targets, agents, and strategies
 - Provider telemetry dashboards, semantic LLM judging, and cloud API billing integration are not yet implemented
 - The six new stage-context strategies have no CLI flags yet, are configured programmatically, and do not yet include plots, screenshots, or gallery integration
@@ -210,6 +230,7 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 - The v0.4.5 context-integrity evaluation is released. It has no CLI flags, no plots/screenshot/gallery integration, and produces no composite score, grade, ranking, or winner. Its corrected-replay fixture is a hand-distilled representation of the validated my-dev-kit v1.10.4 and my-dev-kit-orchestrator v1.2.3 contracts, not a live capture of a complete ten-stage workflow. The orchestrator agreement evidence does not read a literal upstream `promptMode` field; `stageMayRenderNormalPrompt`, derived from structured blocked-stage evidence, is the bounded substitute used instead.
 - v0.4.7 added declarative browser tutorial validation and execution with bounded actions/assertions, persistent Chromium sessions, WebM, screenshots, SRT/VTT, Markdown, and `tutorial-manifest.json`. Chromium is a separate local runtime prerequisite; the package does not download browser binaries automatically. Gallery consumption remains future scope.
 - v0.4.8 added `pointer-click` and `pointer-drag` for normalized fraction positions inside a single located element. Existing `drag` remains element-to-element between source and target locators.
+- v0.4.9 `select-option` identifies an option by HTML value only: label selection, index selection, multi-select arrays, and automatic selector fallback are not supported, and there is no keyboard (`ArrowDown`/`Enter`) fallback. Native popup/menu traversal is not animated. Chromium remains a separately installed local runtime prerequisite; neither package installation nor tutorial execution downloads it automatically.
 
 ---
 

package/dist/src/cli/help.js

@@ -142,7 +142,7 @@ export function renderExperimentRunHelp() {
         "Required:",
         "  --experiment <id>   Registered experiment plugin id (see \"my-dev-kit-lab experiment list\")",
         "",
-        "Options:",
+        "Common options (all plugins):",
         "  --target <path>                                    Local target project path (default: self)",
         "  --out <dir>                                         Output directory for experiment artifacts",
         "                                                      (default: beneath the workspace when installed)",
@@ -152,6 +152,15 @@ export function renderExperimentRunHelp() {
         "                                                      (default: the bundled package resource)",
         "  --case <ids>                                       Comma-separated case ids to run",
         "  --benchmark-project <ids>                          Comma-separated benchmark project ids to run",
+        "",
+        "warm-index-reuse only:",
+        "  --kit-command <command>                            my-dev-kit command used to build one index per benchmark",
+        "                                                      project and retrieve per task",
+        "                                                      (default: npx @dailephd/my-dev-kit@latest)",
+        "                                                      Agent evidence uses the deterministic fake agent only;",
+        "                                                      agent options below are not accepted by this plugin.",
+        "",
+        "context-strategy-comparison only:",
         "  --agents <list>                                    Comma-separated agent ids (default: fake-agent)",
         "  --strategies <list>                                Comma-separated strategies: raw-full-file,my-dev-kit-guided",
         "  --complexities <list>                               Comma-separated complexity levels: short,medium,long,multi-step",
@@ -238,19 +247,25 @@ export function renderPlotsHelp() {
         "  my-dev-kit-lab plots --help",
         "",
         "Commands:",
-        "  generate   Generate experiment plot artifacts from controlled-experiment artifacts.",
+        "  generate   Generate experiment plot artifacts from controlled-experiment or supported",
+        "             plugin experiment artifacts (warm-index-reuse).",
         "             Run \"my-dev-kit-lab plots generate --help\" for details."
     ].join("\n");
 }
 export function renderPlotsGenerateHelp() {
     return [
-        "my-dev-kit-lab plots generate - generate chart/plot artifacts from controlled-experiment artifacts",
+        "my-dev-kit-lab plots generate - generate chart/plot artifacts from experiment artifacts",
         "",
         "Usage:",
         "  my-dev-kit-lab plots generate --experiment <dir> --out <dir>",
         "",
+        "Supported inputs:",
+        "  - legacy controlled-experiment output directories",
+        "  - warm-index-reuse plugin output directories (containing its report.json)",
+        "  Other plugin outputs are not plotted by this command.",
+        "",
         "Required:",
-        "  --experiment <dir>   Path to a controlled-experiment output directory",
+        "  --experiment <dir>   Path to a supported experiment output directory",
         "  --out <dir>          Output directory for plot artifacts"
     ].join("\n");
 }

package/dist/src/commands/runExperimentDescribeCommand.js

@@ -106,6 +106,9 @@ function readConfigFields(plugin, required) {
     }));
 }
 function readSupportedVariants(plugin) {
+    if (plugin.supportedVariants) {
+        return [...plugin.supportedVariants];
+    }
     const strategies = readArrayField(plugin.defaultConfig, "strategies");
     return strategies.filter((value) => typeof value === "string");
 }
@@ -139,9 +142,15 @@ function describeExpectedReports(plugin) {
     return `Writes supported outputs: ${plugin.metadata.supportedOutputs.join(", ")}.`;
 }
 function describeExamples(plugin) {
+    // Only advertise agent-matrix options for plugins that declare them.
+    const agentOptions = hasConfigField(plugin, "agents") ? " --agents fake-agent --complexities short" : "";
+    const screenshotOption = hasConfigField(plugin, "agents") ? " --no-screenshot" : "";
     return [
         `my-dev-kit-lab experiment describe --experiment ${plugin.metadata.id}`,
-        `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --agents fake-agent --complexities short`,
-        `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target "Z:\\Users\\newuser\\Projects\\my-dev-kit-v1" --agents fake-agent --complexities short --no-screenshot`
+        `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id}${agentOptions}`,
+        `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target "Z:\\Users\\newuser\\Projects\\my-dev-kit-v1"${agentOptions}${screenshotOption}`
     ];
 }
+function hasConfigField(plugin, name) {
+    return (plugin.configDefinition?.fields ?? []).some((field) => field.name === name);
+}

package/dist/src/commands/runExperimentListCommand.js

@@ -29,6 +29,9 @@ function toListedExperiment(metadata, plugin) {
     };
 }
 function readSupportedVariants(plugin) {
+    if (plugin.supportedVariants) {
+        return [...plugin.supportedVariants];
+    }
     const strategies = readArrayField(plugin.defaultConfig, "strategies");
     return strategies.filter((value) => typeof value === "string");
 }

package/dist/src/commands/runExperimentRunCommand.js

@@ -2,7 +2,7 @@ import path from "node:path";
 import { parseAgentCommandTemplate } from "../agents/index.js";
 import { parseAgentId } from "../agents/agentRegistry.js";
 import { readBenchmarkProjectProfiles, readEvaluationCases } from "../evaluation/index.js";
-import { contextStrategyComparisonPlugin, createDefaultExperimentPluginRegistry, resolveExperimentTarget, runExperiment } from "../experiments/index.js";
+import { contextStrategyComparisonPlugin, createDefaultExperimentPluginRegistry, resolveExperimentTarget, runExperiment, warmIndexReusePlugin } from "../experiments/index.js";
 import { buildDefaultExperimentOutputRoot } from "../experiments/outputPaths.js";
 import { parsePromptComplexityLevel, parsePromptStrategy } from "../prompts/index.js";
 import { writePluginExperimentReports } from "../report/index.js";
@@ -100,6 +100,7 @@ export function parseRunExperimentArgs(argv) {
     let continueOnFailure;
     let requireAgents;
     let includeRealAgents;
+    let kitCommand;
     const commandTemplates = {};
     for (let index = 0; index < argv.length; index += 1) {
         const arg = argv[index];
@@ -157,6 +158,9 @@ export function parseRunExperimentArgs(argv) {
         else if (arg === "--command-template-claude") {
             commandTemplates.claude = parseAgentCommandTemplate(readRequiredValue(argv, ++index, "--command-template-claude"));
         }
+        else if (arg === "--kit-command") {
+            kitCommand = readRequiredValue(argv, ++index, "--kit-command");
+        }
         else if (arg === "--no-screenshot") {
             // The plugin-aware report path does not capture screenshots yet; accept this
             // flag so smoke commands can share the legacy demo option set.
@@ -168,6 +172,9 @@ export function parseRunExperimentArgs(argv) {
     if (!experimentId) {
         throw new Error("Usage: --experiment <id> [--target <path>] [--out <directory>]");
     }
+    if (kitCommand !== undefined && experimentId !== warmIndexReusePlugin.metadata.id) {
+        throw new Error(`--kit-command is only supported for --experiment ${warmIndexReusePlugin.metadata.id}.`);
+    }
     return {
         experimentId,
         targetPath,
@@ -185,7 +192,8 @@ export function parseRunExperimentArgs(argv) {
             continueOnFailure,
             requireAgents,
             includeRealAgents,
-            commandTemplates: Object.keys(commandTemplates).length > 0 ? commandTemplates : undefined
+            commandTemplates: Object.keys(commandTemplates).length > 0 ? commandTemplates : undefined,
+            kitCommand
         })
     };
 }
@@ -212,13 +220,25 @@ function generateExperimentRunId(pluginId) {
     return `${pluginId}-${timestamp}`;
 }
 async function loadPluginInputs(args, toolRoot, context) {
-    if (args.experimentId !== contextStrategyComparisonPlugin.metadata.id) {
-        return undefined;
+    if (args.experimentId === contextStrategyComparisonPlugin.metadata.id) {
+        const validation = contextStrategyComparisonPlugin.validateConfig(args.config);
+        if (!validation.valid || !validation.config) {
+            throw new Error(`Invalid context strategy comparison config: ${validation.errors.join("; ")}`);
+        }
+        return loadCasesAndProjectProfiles(args, toolRoot, context);
     }
-    const validation = contextStrategyComparisonPlugin.validateConfig(args.config);
-    if (!validation.valid || !validation.config) {
-        throw new Error(`Invalid context strategy comparison config: ${validation.errors.join("; ")}`);
+    if (args.experimentId === warmIndexReusePlugin.metadata.id) {
+        const validation = warmIndexReusePlugin.validateConfig(args.config);
+        if (!validation.valid || !validation.config) {
+            throw new Error(`Invalid warm index reuse config: ${validation.errors.join("; ")}`);
+        }
+        return loadCasesAndProjectProfiles(args, toolRoot, context);
     }
+    return undefined;
+}
+// Shared by both registered plugins: explicit paths resolve against toolRoot, defaults come from
+// bundled package resources so they do not depend on the invocation cwd.
+async function loadCasesAndProjectProfiles(args, toolRoot, context) {
     const projectProfilesPath = args.config.projectProfilesPath
         ? path.resolve(toolRoot, args.config.projectProfilesPath)
         : resolvePackageResource(context, DEFAULT_PROJECT_PROFILES_RESOURCE);

package/dist/src/evaluation/runMyDevKitRetrieval.js

@@ -39,52 +39,71 @@ function pickCandidateFields(candidate) {
         symbol: readString("symbol", "name", "label")
     };
 }
-export async function runMyDevKitRetrieval(options) {
-    const started = Date.now();
+function skippedRetrieval(caseId, warnings, commands, durationMs) {
+    return {
+        caseId,
+        skipped: true,
+        warnings,
+        totalChars: 0,
+        totalEstimatedTokens: 0,
+        tokenCountMethod,
+        contextText: "",
+        filesRead: [],
+        commands,
+        durationMs
+    };
+}
+/**
+ * Runs exactly one my-dev-kit `index` command for the target and writes the index only to
+ * `indexDir`. Performs no search/lookup/slice/source work. With `requireKit` a failed index
+ * throws; otherwise the failure is returned as `ok: false` with a warning.
+ */
+export async function buildMyDevKitIndex(options) {
     const warnings = [];
-    const commandsDir = path.join(options.outputDir, "commands", options.evaluationCase.id);
-    const indexesDir = path.join(options.outputDir, "indexes", options.evaluationCase.id);
-    const commands = [];
-    const indexCommand = await runMeasuredCommand({
+    const command = await runMeasuredCommand({
         commandId: "index",
         commandString: options.kitCommand,
         cwd: process.cwd(),
-        outDir: commandsDir,
+        outDir: options.commandsDir,
         extraArgs: [
             "index",
             "--root",
-            options.evaluationCase.absoluteTargetRoot,
-            ...options.evaluationCase.sourceRoots.flatMap((sourceRoot) => ["--src", sourceRoot]),
+            options.target.absoluteTargetRoot,
+            ...options.target.sourceRoots.flatMap((sourceRoot) => ["--src", sourceRoot]),
             "--out",
-            indexesDir,
+            options.indexDir,
             "--json"
         ]
     });
-    commands.push(indexCommand);
-    if (!indexCommand.ok) {
+    if (!command.ok) {
         if (options.requireKit) {
-            throw new Error(indexCommand.error || `my-dev-kit index failed with exit code ${indexCommand.exitCode}`);
+            throw new Error(command.error || `my-dev-kit index failed with exit code ${command.exitCode}`);
         }
         warnings.push("my-dev-kit index command was unavailable or failed.");
-        return {
-            caseId: options.evaluationCase.id,
-            skipped: true,
-            warnings,
-            totalChars: 0,
-            totalEstimatedTokens: 0,
-            tokenCountMethod,
-            contextText: "",
-            filesRead: [],
-            commands,
-            durationMs: Date.now() - started
-        };
     }
+    return {
+        ok: command.ok,
+        indexDir: options.indexDir,
+        durationMs: command.durationMs,
+        warnings,
+        command
+    };
+}
+/**
+ * Runs search -> lookup -> slice -> source against an already prepared index. Never invokes
+ * `index`; `durationMs` covers retrieval only.
+ */
+export async function runMyDevKitRetrievalFromIndex(options) {
+    const started = Date.now();
+    const warnings = [];
+    const commands = [];
+    const { commandsDir, indexDir } = options;
     const searchCommand = await runMeasuredCommand({
         commandId: "search",
         commandString: options.kitCommand,
         cwd: process.cwd(),
         outDir: commandsDir,
-        extraArgs: ["search", "--index", indexesDir, "--query", options.evaluationCase.query, "--json"]
+        extraArgs: ["search", "--index", indexDir, "--query", options.evaluationCase.query, "--json"]
     });
     commands.push(searchCommand);
     if (!searchCommand.ok) {
@@ -92,36 +111,14 @@ export async function runMyDevKitRetrieval(options) {
             throw new Error(searchCommand.error || `my-dev-kit search failed with exit code ${searchCommand.exitCode}`);
         }
         warnings.push("my-dev-kit search command failed.");
-        return {
-            caseId: options.evaluationCase.id,
-            skipped: true,
-            warnings,
-            totalChars: 0,
-            totalEstimatedTokens: 0,
-            tokenCountMethod,
-            contextText: "",
-            filesRead: [],
-            commands,
-            durationMs: Date.now() - started
-        };
+        return skippedRetrieval(options.evaluationCase.id, warnings, commands, Date.now() - started);
     }
     const searchPayload = parseJsonIfPossible(searchCommand.stdout);
     const candidates = readSearchResults(searchPayload);
     const selected = candidates[0];
     if (!selected) {
         warnings.push("No my-dev-kit search candidate was found.");
-        return {
-            caseId: options.evaluationCase.id,
-            skipped: true,
-            warnings,
-            totalChars: 0,
-            totalEstimatedTokens: 0,
-            tokenCountMethod,
-            contextText: "",
-            filesRead: [],
-            commands,
-            durationMs: Date.now() - started
-        };
+        return skippedRetrieval(options.evaluationCase.id, warnings, commands, Date.now() - started);
     }
     const candidate = pickCandidateFields(selected);
     const selectedNodeId = candidate.nodeId;
@@ -136,7 +133,7 @@ export async function runMyDevKitRetrieval(options) {
             commandString: options.kitCommand,
             cwd: process.cwd(),
             outDir: commandsDir,
-            extraArgs: ["lookup", "--index", indexesDir, "--node", selectedNodeId, "--json"]
+            extraArgs: ["lookup", "--index", indexDir, "--node", selectedNodeId, "--json"]
         });
         commands.push(lookupCommand);
         if (lookupCommand.ok) {
@@ -150,7 +147,7 @@ export async function runMyDevKitRetrieval(options) {
             commandString: options.kitCommand,
             cwd: process.cwd(),
             outDir: commandsDir,
-            extraArgs: ["slice", "--index", indexesDir, "--node", selectedNodeId, "--json"]
+            extraArgs: ["slice", "--index", indexDir, "--node", selectedNodeId, "--json"]
         });
         commands.push(sliceCommand);
         if (sliceCommand.ok) {
@@ -164,7 +161,7 @@ export async function runMyDevKitRetrieval(options) {
             commandString: options.kitCommand,
             cwd: process.cwd(),
             outDir: commandsDir,
-            extraArgs: ["source", "--index", indexesDir, "--node", selectedNodeId, "--max-lines", "160", "--format", "numbered"]
+            extraArgs: ["source", "--index", indexDir, "--node", selectedNodeId, "--max-lines", "160", "--format", "numbered"]
         });
         commands.push(sourceCommand);
         if (sourceCommand.ok) {
@@ -195,3 +192,35 @@ export async function runMyDevKitRetrieval(options) {
         durationMs: Date.now() - started
     };
 }
+/**
+ * Legacy per-case lifecycle: builds a case-specific index and retrieves from it. `commands`
+ * starts with the index command and `durationMs` covers the whole lifecycle including indexing.
+ */
+export async function runMyDevKitRetrieval(options) {
+    const started = Date.now();
+    const commandsDir = path.join(options.outputDir, "commands", options.evaluationCase.id);
+    const indexDir = path.join(options.outputDir, "indexes", options.evaluationCase.id);
+    const index = await buildMyDevKitIndex({
+        target: options.evaluationCase,
+        kitCommand: options.kitCommand,
+        indexDir,
+        commandsDir,
+        requireKit: options.requireKit
+    });
+    if (!index.ok) {
+        return skippedRetrieval(options.evaluationCase.id, [...index.warnings], [index.command], Date.now() - started);
+    }
+    const retrieval = await runMyDevKitRetrievalFromIndex({
+        evaluationCase: options.evaluationCase,
+        kitCommand: options.kitCommand,
+        indexDir,
+        commandsDir,
+        requireKit: options.requireKit
+    });
+    return {
+        ...retrieval,
+        warnings: [...index.warnings, ...retrieval.warnings],
+        commands: [index.command, ...retrieval.commands],
+        durationMs: Date.now() - started
+    };
+}

package/dist/src/experiments/defaultRegistry.js

@@ -1,7 +1,9 @@
 import { ExperimentPluginRegistry } from "./registry.js";
 import { contextStrategyComparisonPlugin } from "./plugins/contextStrategyComparison/index.js";
+import { warmIndexReusePlugin } from "./plugins/warmIndexReuse/index.js";
 export function createDefaultExperimentPluginRegistry() {
     const registry = new ExperimentPluginRegistry();
     registry.register(contextStrategyComparisonPlugin);
+    registry.register(warmIndexReusePlugin);
     return registry;
 }

package/dist/src/experiments/plugins/index.js

@@ -1 +1,2 @@
 export * from "./contextStrategyComparison/index.js";
+export * from "./warmIndexReuse/index.js";

package/dist/src/experiments/plugins/warmIndexReuse/config.js

@@ -0,0 +1,58 @@
+import { invalidExperimentConfig, isPlainObject, mergeConfig, validExperimentConfig } from "../../config.js";
+export const defaultWarmIndexReuseConfig = {
+    casesPath: "examples/token-savings-cases.json",
+    projectProfilesPath: "benchmarks/contracts/benchmark-project-profiles.json",
+    outDir: "lab-output/warm-index-reuse",
+    kitCommand: "npx @dailephd/my-dev-kit@latest",
+};
+const SUPPORTED_FIELDS = [
+    "casesPath",
+    "projectProfilesPath",
+    "outDir",
+    "kitCommand",
+    "caseIds",
+    "benchmarkProjects",
+];
+export const warmIndexReuseConfigDefinition = {
+    fields: [
+        { name: "casesPath", type: "string", required: true, description: "Evaluation cases file." },
+        { name: "outDir", type: "string", required: true, description: "Experiment output root." },
+        { name: "projectProfilesPath", type: "string", description: "Benchmark project profiles file." },
+        { name: "kitCommand", type: "string", description: "my-dev-kit command used for index and retrieval." },
+        { name: "caseIds", type: "array", description: "Evaluation case IDs to include." },
+        { name: "benchmarkProjects", type: "array", description: "Benchmark project IDs to include." },
+    ],
+};
+/**
+ * Closed validation: fields outside the warm-index contract (for example agent-matrix settings)
+ * are rejected instead of being silently ignored.
+ */
+export function validateWarmIndexReuseConfig(config) {
+    if (config !== undefined && !isPlainObject(config)) {
+        return invalidExperimentConfig(["warm index reuse config must be an object."]);
+    }
+    const errors = [];
+    const unsupported = Object.keys(config ?? {}).filter((key) => !SUPPORTED_FIELDS.includes(key));
+    if (unsupported.length > 0) {
+        errors.push(`Unsupported warm-index-reuse config field(s): ${unsupported.sort().join(", ")}.`);
+    }
+    const normalized = mergeConfig(defaultWarmIndexReuseConfig, config);
+    for (const field of ["casesPath", "projectProfilesPath", "outDir", "kitCommand"]) {
+        const value = normalized[field];
+        if (typeof value !== "string" || !value.trim()) {
+            errors.push(`${field} must be a non-empty string.`);
+        }
+    }
+    for (const field of ["caseIds", "benchmarkProjects"]) {
+        const value = normalized[field];
+        if (value === undefined)
+            continue;
+        if (!Array.isArray(value) || value.length === 0 || !value.every((item) => typeof item === "string" && item.trim())) {
+            errors.push(`${field} must be a non-empty array of non-empty strings when provided.`);
+        }
+    }
+    if (errors.length > 0) {
+        return invalidExperimentConfig(errors);
+    }
+    return validExperimentConfig(normalized);
+}