@dailephd/my-dev-kit-lab 0.5.1 → 0.6.0

package/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to my-dev-kit-lab are documented here.
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-09-24
+
+Index freshness and changed-file detection.
+
+- Added `IndexSnapshotV1` (schema `my-dev-kit-lab-index-snapshot-v1`): after a warm-index run's one index build, Lab interprets the my-dev-kit manifest and symbol index and records baseline index evidence — manifest metadata, the exact indexed-file set the upstream index contract lists (never a directory walk), per-file SHA-256, size, and modified-time metadata, the index command, and an inventory of generated index artifacts. Snapshots are `complete`, `partial`, or `unavailable` with explicit reasons; source contents are never persisted.
+- Added a configured my-dev-kit `--version` probe, run once per prepared index session outside the measured index build. An unsupported or failing probe is non-fatal and recorded as explicitly unavailable.
+- Added `IndexFreshnessAssessmentV1` (schema `my-dev-kit-lab-index-freshness-v1`): a read-only comparison of the files the snapshot lists against their current state. SHA-256 is the comparison identity; modified time is diagnostic only. Modified and missing indexed files are confirmed changes; unreadable or unresolvable files stay explicit unresolved evidence; files absent from the snapshot are not inferred to be newly indexable. Exactly four statuses: `fresh`, `stale`, `partially-stale`, `unknown`.
+- Freshness is assessed for every task immediately before its warm retrieval and is observational only: it never triggers or recommends reindexing, suppresses retrieval, or changes execution, provider, agent, correctness, or token-evidence status. One index is still built per benchmark project.
+- Extended `warm-index-execution.json` additively (schema `my-dev-kit-lab-warm-index-execution-v1` unchanged) with project-level `indexSnapshot` and task-level `indexFreshness`.
+- Extended the warm-index report additively (schema `my-dev-kit-lab-warm-index-report-v1` unchanged) with an index freshness summary (assessed, unassessed, fresh, stale, partially-stale, and unknown task counts) and per-task freshness evidence in `report.json`, `report.txt`, and `report.html`. Report evidence is bounded to 20 changed files and 10 unresolved entries per task, omits content hashes, and escapes dynamic strings; a task with no assessment is shown as not assessed, distinct from `unknown`. Report limitations and the run interpretation state what freshness does and does not establish.
+- No new CLI command or flag, `ExperimentMetric`, plot, gallery item, or experiment plugin was added: the warm-index metrics, exactly four plots, and the three-item campaign gallery are unchanged, and no reindex recommendation exists.
+- Extended the packed-package acceptance gate (`npm run verify:packed-package`) to prove the exact tarball ships the snapshot and freshness runtime and that an installed warm-index run produces snapshot, tool-version, freshness, and report freshness evidence while the target, installed package, and packaged examples stay unchanged.
+- Validation of the implementation candidate: exact-SHA ordinary CI passed on Ubuntu, macOS, and Windows with Node 24 and latest, and the full local suite passed with the packed-package gate.
+
+## [0.5.2] - 2026-09-23
+
+Warm-index real-agent campaigns.
+
+- Added three frozen `warm-index-reuse` campaign presets — `codex-full`, `claude-full`, and `codex-timeout-isolation` — each owning exactly one provider (Codex or Claude), the bundled production corpus, and project profiles. `codex-timeout-isolation` narrows to three large/mixed cases (`warm-large-health-label`, `warm-large-ts-leaderboard`, `warm-large-broad-analytics-comparison`) for provider timeout/partial-outcome exercises without the full 12-case corpus. Default per-agent timeout is 240000 ms, overridable with `--timeout-ms`.
+- Added the `experiment run --experiment warm-index-reuse` campaign surface: `--campaign <preset>`, `--include-real-agents` (required), `--case`, `--benchmark-project`, `--timeout-ms`, `--kit-command`, `--out`. Campaign mode rejects `--target`, explicit `--cases`/`--project-profiles`, and the agent-matrix flags owned by `context-strategy-comparison`. Legacy non-campaign warm-index runs are unchanged.
+- Real-agent evaluation consumes the exact `rawBaseline.contextText`/`warmRetrieval.contextText` the experiment already measured; providers never independently run my-dev-kit retrieval. The real-agent prompt explicitly forbids repository inspection, filesystem search, my-dev-kit execution, shell commands, external information, and file modification, and delivers the prompt over stdin. Each provider call runs from a neutral temporary working directory, never the repository, benchmark target, installed package, or agent artifact directory.
+- Added Codex JSONL and Claude JSON response parsing to the existing agent adapters, sharing the existing prompt/parser/classifier/scorer pipeline with the deterministic fake agent.
+- Extended provider outcome classification with `agent-unavailable`, `agent-limit-reached`, and `timeout`, alongside the existing `completed`, `failed`, `invalid-output`, and `skipped`. Warm-index infrastructure status (index build, raw/warm retrieval) and agent/provider outcome status are reported and reasoned about separately: infrastructure can report `completed` while campaign evidence is `partial` or `unavailable`.
+- Extended the warm-index report (schema unchanged: `my-dev-kit-lab-warm-index-report-v1`) with agent identity (`agent.id`, `agent.mode`) and an additive campaign summary (preset, provider, timeout, selected/scheduled/executed/not-run-for-missing-context side counts, closed provider outcome counts, `agentEvidenceStatus`, `tokenEvidenceStatus`). Generic agent metric IDs (`agent-correctness-score`, `agent-total-tokens`, `cumulative-agent-total-tokens`) and their formulas are unchanged; correctness remains deterministic answer-key scoring and missing token evidence remains explicitly unavailable, never zero, never substituted with an estimated context-token count. The existing four warm-index plots are unchanged in formula and gained provider-aware titles/labels; a v1 report from before this change still plots correctly as legacy fake-agent evidence.
+- A real-agent campaign run now automatically produces, beneath its output root: the existing report (`report.json`/`.txt`/`.html`), the same four warm-index SVG charts under `plots/`, a best-effort `report.png` (skipped without a Playwright/browser runtime — non-fatal — and reported as an unsuccessful command only on an unexpected capture failure, which still preserves the report/plots/gallery already written), and a narrow three-item campaign gallery (`gallery/gallery-manifest.json`, `gallery-index.html`: `warm-index-campaign-report`, `warm-index-campaign-plots`, `warm-index-execution`) with relative, bounded paths. Legacy non-campaign warm-index runs remain report-only. The `warm-index-reuse` plugin itself owns no presentation orchestration; the command layer sequences report → plots → screenshot → gallery.
+- Hardened `runMeasuredCommand`'s command-timeout path: a bounded fallback now resolves the measured-command promise even if a killed process's descendant outlives an intermediate shim, and its streams are explicitly detached so the calling process can exit; a provider evaluation's neutral-working-directory cleanup is now best-effort so an OS-level cleanup failure never masks already-classified agent evidence.
+- Extended the packed-package installed-execution acceptance gate (`npm run verify:packed-package`) with deterministic, cross-platform, local-only Codex/Claude provider fixtures proving, through the installed package: completed-with-tokens, token-unavailable, failed, invalid-output, agent-unavailable, agent-limit-reached, and timeout provider outcomes; the `codex-timeout-isolation` preset's continuation past a failed case; captured and skipped screenshot paths; the three-item campaign gallery; provider-argv privacy (no prompt/context in argv, frozen stdin transport flags); and target/installed-package immutability. No real Codex or Claude is ever invoked by this gate.
+
 ## [0.5.1] - 2026-09-22
 
 Expanded warm-index benchmark suite.
@@ -15,7 +43,6 @@ Expanded warm-index benchmark suite.
 - Regression tests prove the existing one-index-per-project warm-index runtime at six tasks per project: two index builds for the 12 tasks, task ordinals 1–6, amortized and cumulative metrics, per-project reset, strict-prefix availability, and isolation of single-task and single-project failures.
 - The existing warm-index report (schema `my-dev-kit-lab-warm-index-report-v1`) and the existing four plots cover the expanded suite: 2 projects, 12 tasks, and ordinals 1–6 per project.
 - The packed-package acceptance gate now checks that the corpus is in the tarball, readable from a clean install, and selectable through the installed CLI's existing `--cases` option.
-- No CLI command or flag, warm-index metric formula, report schema, or plot was added. The default cases file (`examples/token-savings-cases.json`) and the default my-dev-kit command are unchanged. Real-agent warm-index campaigns remain planned for v0.5.2.
 
 ## [0.5.0] - 2026-09-21
 

package/README.md

@@ -6,16 +6,16 @@ my-dev-kit provides local repository indexing and graph-guided retrieval. my-dev
 
 Whole-ecosystem workflows are centralized in [my-dev-kit/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md). Its [command-surface compatibility map](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md#915-command-surface-compatibility-map) documents which Lab outputs can safely seed my-dev-kit investigation, Observer reference work, or Orchestrator evidence, and which look-alike artifacts are not direct inputs.
 
-The latest release is v0.5.1 (expanded warm-index benchmark suite). The previous release, v0.5.0, added warm-index reuse experiment support. The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md).
-
-The current release includes v0.5.1 expanded warm-index benchmarks. The package and installed CLI are version 0.5.1; v0.5.0 remains the previous release. v0.5.2 real-agent warm-index campaigns are planned next. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for the validated release state.
+The latest release is v0.6.0 (index freshness and changed-file detection), and the package and installed CLI are version 0.6.0. The previous release is v0.5.2 (warm-index real-agent campaigns), and v0.6.1 (affected-neighborhood experiments) is next planned. The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
 
 v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
 
 ## Current capabilities
 
 - **Run context-strategy experiments:** compare `raw-full-file` with `my-dev-kit-guided` using deterministic fixtures or locally configured Codex and Claude CLIs.
-- **Measure warm-index reuse (v0.5.1):** the `warm-index-reuse` experiment plugin builds one my-dev-kit index per benchmark project, reuses it across that project's tasks, and compares every task with a matched `raw-full-file` baseline. Reports separate one-time index-build cost from per-task retrieval cost, show amortized index cost and cumulative measurements, and include deterministic fake-agent correctness and token evidence; `plots generate` renders four warm-index SVG charts. The v0.5.1 release adds a dedicated 12-task benchmark corpus (`benchmarks/contracts/warm-index-benchmark-cases.json`: six medium and six large/mixed tasks tagged `localized`, `cross-module`, or `broad-change`) for comparing warm-index behavior as task count grows. Estimated context tokens (a character-based context-size estimate) and fake-agent total tokens (simulated harness telemetry) are reported separately, and neither is provider billing telemetry. See [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment) and [docs/METRICS.md](docs/METRICS.md#warm-index-reuse-metrics).
+- **Measure warm-index reuse (v0.5.0–v0.5.1, released):** the `warm-index-reuse` experiment plugin builds one my-dev-kit index per benchmark project, reuses it across that project's tasks, and compares every task with a matched `raw-full-file` baseline. Reports separate one-time index-build cost from per-task retrieval cost, show amortized index cost and cumulative measurements, and include deterministic fake-agent correctness and token evidence; `plots generate` renders four warm-index SVG charts. The v0.5.1 release adds a dedicated 12-task benchmark corpus (`benchmarks/contracts/warm-index-benchmark-cases.json`: six medium and six large/mixed tasks tagged `localized`, `cross-module`, or `broad-change`) for comparing warm-index behavior as task count grows. Estimated context tokens (a character-based context-size estimate) and fake-agent total tokens (simulated harness telemetry) are reported separately, and neither is provider billing telemetry.
+- **Run selectable real-agent warm-index campaigns (v0.5.2, released):** `--campaign codex-full`, `claude-full`, or `codex-timeout-isolation` (with `--include-real-agents`) runs exactly one provider — Codex or Claude — against the exact raw and warm contexts the experiment already measured, over the frozen production corpus each preset owns. Provider outcomes (`completed`, `failed`, `timeout`, `invalid-output`, `agent-unavailable`, `agent-limit-reached`, `skipped`) are classified and reported separately from warm-index infrastructure status, so a campaign can complete its indexing/retrieval work even when provider evidence is partial or unavailable. A campaign run automatically produces the report, the same four warm-index SVG charts, a best-effort report screenshot (skipped without a browser runtime, never fatal), and a three-item gallery (report, plots, bounded execution evidence). Legacy runs without `--campaign` keep the existing deterministic fake-agent, report-only behavior unchanged. See [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment) and [docs/METRICS.md](docs/METRICS.md#warm-index-reuse-metrics).
+- **Record index freshness evidence (v0.6.0, current):** a warm-index run captures an index snapshot after its one index build — the exact indexed file set from the my-dev-kit index contract, each with SHA-256 content identity, size, and modified-time metadata, plus the my-dev-kit version and index-command evidence — and assesses freshness immediately before each task's warm retrieval as `fresh`, `stale`, `partially-stale`, or `unknown`. The evidence appears in `warm-index-execution.json` and in the existing `report.json`/`report.txt`/`report.html` warm-index reports. It is observational only: it covers only files the snapshot lists (not the whole repository), never changes retrieval, execution or provider status, and adds no CLI flag, metric, plot, gallery item, or reindex recommendation. See [docs/METRICS.md](docs/METRICS.md#index-freshness-evidence-v060) and [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment).
 - **Audit repository health:** run conservative code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, or adapt security findings into the common audit report.
 - **Validate CLI/package security:** inspect dependencies, package contents, path and subprocess boundaries, malformed inputs, optional static scanners, and bounded fuzz targets.
 - **Validate Android projects:** run nineteen static checks by default, with Gradle operations, external tools, and network access available only through explicit opt-in flags.
@@ -169,6 +169,19 @@ npm run generate-experiment-plots -- --experiment lab-output/warm-index-reuse --
 
 The default cases file, `examples/token-savings-cases.json`, is a small compatibility corpus with one task per benchmark project and remains the default when `--cases` is omitted. For multi-task reuse, select the dedicated corpus explicitly as above: it runs six tasks for each of the two benchmark projects against one index per project. Add `--benchmark-project task-workflow-medium-ts` or `--case <id>` to narrow the selection.
 
+Run a selectable real-agent warm-index campaign (v0.5.2; requires a locally available `codex` or `claude` CLI and `--include-real-agents`). A campaign preset owns its provider and corpus, so `--target`, explicit `--cases`/`--project-profiles`, and the agent-matrix flags below are rejected in campaign mode:
+
+```bash
+npm run experiment:run -- \
+  --experiment warm-index-reuse \
+  --campaign codex-full \
+  --include-real-agents \
+  --case warm-medium-complete-idempotent \
+  --out lab-output/warm-index-campaign
+```
+
+This automatically writes `report.json`/`.txt`/`.html`, the four warm-index charts under `plots/`, a best-effort `report.png`, and a three-item `gallery/` (report, plots, bounded execution evidence) beneath `--out`. `--timeout-ms` overrides the preset's default per-agent timeout (240000 ms). `codex-timeout-isolation` narrows the corpus to three larger cases for exercising provider timeout/partial-outcome handling without running the full 12-case corpus.
+
 When `--target` is omitted, the experiment runs in self mode against my-dev-kit-lab. When `--target <path>` is provided, the lab remains the tool root and the target project is inspected separately. Generated experiment outputs stay under lab-controlled output directories by default, not inside the target project.
 
 ---
@@ -186,7 +199,8 @@ When `--target` is omitted, the experiment runs in self mode against my-dev-kit-
 | Plugin experiment report JSON | `lab-output/experiments/<plugin>/<target>/<run>/report.json` |
 | Plugin experiment report HTML | `lab-output/experiments/<plugin>/<target>/<run>/report.html` |
 | Plugin experiment report text | `lab-output/experiments/<plugin>/<target>/<run>/report.txt` |
-| Warm-index execution evidence | `<experiment out>/warm-index-execution.json` (plus `indexes/`, `commands/`, and `agents/` beneath the same output root) |
+| Warm-index execution evidence | `<experiment out>/warm-index-execution.json` (plus `indexes/`, `commands/`, and `agents/` beneath the same output root); the v0.6.0 artifact carries project-level `indexSnapshot` and per-task `indexFreshness` evidence |
+| Warm-index campaign presentation (v0.5.2) | `<experiment out>/plots/` (four charts), `<experiment out>/report.png` (best-effort), `<experiment out>/gallery/gallery-manifest.json` and `gallery-index.html` — automatic for `--campaign` runs only |
 | Plot data | `lab-output/<plots>/plot-data.json` |
 | SVG charts | `lab-output/<plots>/charts/*.svg` |
 | Gallery manifest | `lab-output/<gallery>/gallery-manifest.json` |
@@ -217,11 +231,11 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 ## Current limitations
 
 - Token savings shown in fake-agent runs are based on estimated character counts, not provider billing telemetry
-- Claude does not expose token totals; token savings comparisons are unavailable for Claude runs
-- Codex may expose token totals but can produce timeouts or invalid-output runs
+- Claude and Codex token totals are used only when the CLI/run exposes them; missing provider token evidence stays explicitly unavailable (never zero, and never substituted with an estimated context-token count)
+- Codex and Claude campaign runs can produce timeouts, invalid-output, agent-unavailable, or agent-limit-reached outcomes; these are structured provider states, not warm-index infrastructure failures
 - Small projects may make raw-full-file cheaper than my-dev-kit-guided; larger localized tasks are where my-dev-kit is expected to become more useful
 - The generic experiment-plugin framework has two plugins: `context-strategy-comparison` and the v0.5.0-introduced `warm-index-reuse`. Later plugins such as incremental-change and context-window scaling are future roadmap work
-- `warm-index-reuse` uses the deterministic fake agent only; selectable Codex/Claude warm-index campaigns remain planned for v0.5.2, and warm-index screenshots/gallery integration are not implemented. The v0.5.1 expanded corpus is deterministic fake-agent benchmark evidence, not real-agent evidence. Its component-duration sums are not wall-clock latency, and it calculates no token-savings percentage, break-even task, winner, or ranking
+- `warm-index-reuse` without `--campaign` still uses the deterministic fake agent only. With `--campaign`, exactly one real provider (Codex or Claude) runs per campaign preset — there is no multi-agent matrix, retry, provider switching, or scheduler. The v0.5.1 expanded corpus is deterministic fake-agent benchmark evidence when run without `--campaign`. No warm-index mode calculates a token-savings percentage, break-even task, winner, or ranking, and component-duration sums are not wall-clock latency
 - The current release does not guarantee token savings; it produces auditable evidence for specific cases, targets, agents, and strategies
 - Provider telemetry dashboards, semantic LLM judging, and cloud API billing integration are not yet implemented
 - The six new stage-context strategies have no CLI flags yet, are configured programmatically, and do not yet include plots, screenshots, or gallery integration
@@ -231,6 +245,7 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 - v0.4.7 added declarative browser tutorial validation and execution with bounded actions/assertions, persistent Chromium sessions, WebM, screenshots, SRT/VTT, Markdown, and `tutorial-manifest.json`. Chromium is a separate local runtime prerequisite; the package does not download browser binaries automatically. Gallery consumption remains future scope.
 - v0.4.8 added `pointer-click` and `pointer-drag` for normalized fraction positions inside a single located element. Existing `drag` remains element-to-element between source and target locators.
 - v0.4.9 `select-option` identifies an option by HTML value only: label selection, index selection, multi-select arrays, and automatic selector fallback are not supported, and there is no keyboard (`ArrowDown`/`Enter`) fallback. Native popup/menu traversal is not animated. Chromium remains a separately installed local runtime prerequisite; neither package installation nor tutorial execution downloads it automatically.
+- v0.5.2 (released) selectable real-agent warm-index campaigns run exactly one provider per preset (`codex-full`, `claude-full`, `codex-timeout-isolation`); there is no retry, provider switching, or scheduler. Correctness remains deterministic answer-key scoring, not semantic LLM judging. No composite score, winner, ranking, or break-even task is calculated for campaign or legacy warm-index runs.
 
 ---
 

package/dist/src/agents/adapters/claudeAdapter.js

@@ -2,6 +2,20 @@ import { runCliAgent } from "./codexAdapter.js";
 import { applyPromptToCommandTemplate } from "../runAgentPrompt.js";
 import { parseAgentTokenUsage } from "../parseAgentTokenUsage.js";
 import { runMeasuredCommand } from "../../core/runMeasuredCommand.js";
+// v0.5.2 Batch 2 -- frozen stdin invocation for the real-agent evaluation transport. Every flag
+// here is planner-owned; --bare is intentionally not used (it changes auth behavior and still
+// exposes Bash/read/edit tools unless separately constrained).
+const CLAUDE_STDIN_ARGS = [
+    "--restricted",
+    "-p",
+    "--output-format",
+    "json",
+    "--no-session-persistence",
+    "--tools",
+    "",
+    "--disallowedTools",
+    "mcp__*"
+];
 export const claudeAdapter = {
     id: "claude",
     displayName: "Claude",
@@ -21,17 +35,46 @@ export const claudeAdapter = {
         return check.ok;
     },
     buildCommand(request) {
+        if (request.promptTransport === "stdin" && request.commandTemplate) {
+            throw new Error("claude agent: stdin prompt transport cannot be combined with a command template.");
+        }
         if (request.commandTemplate) {
             return applyPromptToCommandTemplate(request.commandTemplate, request.promptText);
         }
+        if (request.promptTransport === "stdin") {
+            return { command: "claude", args: [...CLAUDE_STDIN_ARGS], stdinText: request.promptText };
+        }
         return { command: "claude", args: ["-p", request.promptText] };
     },
     async runPrompt(request) {
         return runCliAgent(request, this);
     },
     parseTokenUsage: parseAgentTokenUsage,
-    parseFinalAnswer(text) {
-        const trimmed = text.trim();
-        return { finalAnswerText: trimmed, finalAnswerParseStatus: trimmed ? "parsed" : "empty" };
-    }
+    parseFinalAnswer: parseClaudeFinalAnswer
 };
+/**
+ * Parses a single JSON result envelope (`{ "result": "..." }`) when stdout is valid JSON, never
+ * the raw envelope itself. Falls back to the legacy plain-text behavior when stdout is not valid
+ * JSON, so existing argv-mode text output keeps working unchanged.
+ */
+export function parseClaudeFinalAnswer(text) {
+    const trimmedInput = text.trim();
+    if (trimmedInput) {
+        try {
+            const parsed = JSON.parse(trimmedInput);
+            if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
+                const payload = parsed;
+                if (typeof payload.result === "string") {
+                    const resultText = payload.result.trim();
+                    return { finalAnswerText: resultText, finalAnswerParseStatus: resultText ? "parsed" : "empty" };
+                }
+                // Valid JSON object without a string result: empty, never the raw envelope.
+                return { finalAnswerText: "", finalAnswerParseStatus: "empty" };
+            }
+        }
+        catch {
+            // Not valid JSON; fall through to the legacy plain-text behavior below.
+        }
+    }
+    return { finalAnswerText: trimmedInput, finalAnswerParseStatus: trimmedInput ? "parsed" : "empty" };
+}

package/dist/src/agents/adapters/codexAdapter.js

@@ -1,6 +1,17 @@
 import { runMeasuredCommand } from "../../core/runMeasuredCommand.js";
 import { applyPromptToCommandTemplate } from "../runAgentPrompt.js";
 import { parseAgentTokenUsage } from "../parseAgentTokenUsage.js";
+// v0.5.2 Batch 2 -- frozen stdin invocation for the real-agent evaluation transport. Every flag
+// here is planner-owned; do not add sandbox/model/approval-policy flags in this batch.
+const CODEX_STDIN_ARGS = [
+    "exec",
+    "--json",
+    "--ephemeral",
+    "--skip-git-repo-check",
+    "--ignore-user-config",
+    "--ignore-rules",
+    "-"
+];
 export const codexAdapter = {
     id: "codex",
     displayName: "Codex",
@@ -20,20 +31,67 @@ export const codexAdapter = {
         return check.ok;
     },
     buildCommand(request) {
+        if (request.promptTransport === "stdin" && request.commandTemplate) {
+            throw new Error("codex agent: stdin prompt transport cannot be combined with a command template.");
+        }
         if (request.commandTemplate) {
             return applyPromptToCommandTemplate(request.commandTemplate, request.promptText);
         }
+        if (request.promptTransport === "stdin") {
+            return { command: "codex", args: [...CODEX_STDIN_ARGS], stdinText: request.promptText };
+        }
         return { command: "codex", args: ["exec", "--json", request.promptText] };
     },
     async runPrompt(request) {
         return runCliAgent(request, this);
     },
     parseTokenUsage: parseAgentTokenUsage,
-    parseFinalAnswer(text) {
-        const trimmed = text.trim();
-        return { finalAnswerText: trimmed, finalAnswerParseStatus: trimmed ? "parsed" : "empty" };
-    }
+    parseFinalAnswer: parseCodexFinalAnswer
 };
+/**
+ * Recognizes a Codex JSONL event stream (at least one line parses to an object with a string
+ * `type`) and, when recognized, uses the LAST `item.completed` `agent_message` text as the final
+ * answer -- never the raw JSONL. Falls back to the legacy plain-text behavior when no event-shaped
+ * line is found, so non-JSONL Codex output (and existing tests) keep working unchanged.
+ */
+export function parseCodexFinalAnswer(text) {
+    let sawEventLine = false;
+    let lastAgentMessageText = null;
+    for (const line of text.split(/\r?\n/)) {
+        const trimmedLine = line.trim();
+        if (!trimmedLine)
+            continue;
+        let parsed;
+        try {
+            parsed = JSON.parse(trimmedLine);
+        }
+        catch {
+            // Agent output can mix JSONL with incidental non-JSON noise; ignore malformed lines.
+            continue;
+        }
+        if (!parsed || typeof parsed !== "object" || Array.isArray(parsed))
+            continue;
+        const event = parsed;
+        if (typeof event.type !== "string")
+            continue;
+        sawEventLine = true;
+        if (event.type === "item.completed") {
+            const item = event.item;
+            if (item && typeof item === "object" && item.type === "agent_message") {
+                const itemText = item.text;
+                if (typeof itemText === "string") {
+                    lastAgentMessageText = itemText;
+                }
+            }
+        }
+    }
+    if (sawEventLine) {
+        const finalAnswerText = lastAgentMessageText ?? "";
+        return { finalAnswerText, finalAnswerParseStatus: finalAnswerText ? "parsed" : "empty" };
+    }
+    const trimmed = text.trim();
+    return { finalAnswerText: trimmed, finalAnswerParseStatus: trimmed ? "parsed" : "empty" };
+}
 export async function runCliAgent(request, adapter) {
     const started = Date.now();
     const command = adapter.buildCommand(request);
@@ -74,7 +132,8 @@ export async function runCliAgent(request, adapter) {
         cwd: request.commandTemplate?.cwd ?? request.cwd,
         outDir: request.outDir,
         env: request.env,
-        timeoutMs: request.timeoutMs
+        timeoutMs: request.timeoutMs,
+        stdinText: command.stdinText
     });
     const ended = Date.now();
     const combinedOutput = `${measured.stdout}\n${measured.stderr}`;

package/dist/src/cli/help.js

@@ -157,8 +157,29 @@ export function renderExperimentRunHelp() {
         "  --kit-command <command>                            my-dev-kit command used to build one index per benchmark",
         "                                                      project and retrieve per task",
         "                                                      (default: npx @dailephd/my-dev-kit@latest)",
-        "                                                      Agent evidence uses the deterministic fake agent only;",
+        "                                                      Legacy/default mode uses the deterministic fake agent only;",
         "                                                      agent options below are not accepted by this plugin.",
+        "  --campaign <preset>                                 Runs the selected real-agent campaign preset: codex-full,",
+        "                                                      claude-full, codex-timeout-isolation. Selects Codex or",
+        "                                                      Claude as the preset's single provider and requires",
+        "                                                      --include-real-agents. The preset owns the bundled",
+        "                                                      production corpus and project profiles; explicit",
+        "                                                      --cases/--project-profiles and --target are rejected in",
+        "                                                      campaign mode. --case and --benchmark-project narrow the",
+        "                                                      preset's case set. --kit-command and --timeout-ms are",
+        "                                                      accepted. The report distinguishes warm-index",
+        "                                                      infrastructure status from agent/provider outcome status;",
+        "                                                      provider token telemetry may be unavailable and is never",
+        "                                                      replaced by a context-size estimate. A campaign run",
+        "                                                      automatically produces the report JSON/text/HTML, the",
+        "                                                      four warm-index SVG plots, a best-effort report PNG",
+        "                                                      screenshot, and a gallery manifest/index. A missing",
+        "                                                      Playwright/browser runtime makes screenshot capture skip",
+        "                                                      without invalidating the campaign; an unexpected",
+        "                                                      screenshot failure makes the command unsuccessful after",
+        "                                                      preserving the report, plots, and gallery. Legacy",
+        "                                                      non-campaign warm-index runs retain their existing",
+        "                                                      report-only behavior.",
         "",
         "context-strategy-comparison only:",
         "  --agents <list>                                    Comma-separated agent ids (default: fake-agent)",

package/dist/src/commands/runExperimentDescribeCommand.js

@@ -145,11 +145,16 @@ function describeExamples(plugin) {
     // Only advertise agent-matrix options for plugins that declare them.
     const agentOptions = hasConfigField(plugin, "agents") ? " --agents fake-agent --complexities short" : "";
     const screenshotOption = hasConfigField(plugin, "agents") ? " --no-screenshot" : "";
-    return [
+    const examples = [
         `my-dev-kit-lab experiment describe --experiment ${plugin.metadata.id}`,
         `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id}${agentOptions}`,
         `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target "Z:\\Users\\newuser\\Projects\\my-dev-kit-v1"${agentOptions}${screenshotOption}`
     ];
+    // Only advertise campaign examples for plugins that declare the campaignPreset config field.
+    if (hasConfigField(plugin, "campaignPreset")) {
+        examples.push(`my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --campaign codex-full --include-real-agents --out <dir>`, `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --campaign claude-full --include-real-agents --case warm-medium-complete-idempotent --out <dir>`);
+    }
+    return examples;
 }
 function hasConfigField(plugin, name) {
     return (plugin.configDefinition?.fields ?? []).some((field) => field.name === name);

package/dist/src/commands/runExperimentRunCommand.js

@@ -2,11 +2,12 @@ import path from "node:path";
 import { parseAgentCommandTemplate } from "../agents/index.js";
 import { parseAgentId } from "../agents/agentRegistry.js";
 import { readBenchmarkProjectProfiles, readEvaluationCases } from "../evaluation/index.js";
-import { contextStrategyComparisonPlugin, createDefaultExperimentPluginRegistry, resolveExperimentTarget, runExperiment, warmIndexReusePlugin } from "../experiments/index.js";
+import { contextStrategyComparisonPlugin, createDefaultExperimentPluginRegistry, getWarmIndexCampaignPreset, parseWarmIndexCampaignPresetId, resolveExperimentTarget, runExperiment, warmIndexReusePlugin } from "../experiments/index.js";
 import { buildDefaultExperimentOutputRoot } from "../experiments/outputPaths.js";
 import { parsePromptComplexityLevel, parsePromptStrategy } from "../prompts/index.js";
 import { writePluginExperimentReports } from "../report/index.js";
 import { createLabExecutionContext, resolvePackageResource } from "../runtime/index.js";
+import { runWarmIndexCampaignPresentation } from "./runWarmIndexCampaignPresentation.js";
 // ---------------------------------------------------------------------------
 // v0.4.6 Batch 4 -- reusable experiment-run command owner.
 //
@@ -61,7 +62,7 @@ export async function runExperimentRunCommandFromArgs(argv, options = {}) {
             plugin: registry.describe(result.pluginId),
             outputRoot: String(result.metadata?.outputRoot ?? "")
         });
-        console.log([
+        const outputLines = [
             `Experiment: ${result.pluginId}`,
             `Run ID: ${result.runId}`,
             `Status: ${result.status}`,
@@ -71,8 +72,51 @@ export async function runExperimentRunCommandFromArgs(argv, options = {}) {
             `Output: ${String(result.metadata?.outputRoot ?? "")}`,
             `Report JSON: ${reports.outputPaths.jsonPath}`,
             `Report HTML: ${reports.outputPaths.htmlPath}`
-        ].join("\n"));
-        return result.status === "failed" ? 1 : 0;
+        ];
+        // Automatic presentation (report already produced above) applies only to warm-index-reuse
+        // campaign runs; legacy fake-agent warm-index runs and context-strategy-comparison stay
+        // report-only, unchanged.
+        let screenshotFailed = false;
+        if (args.experimentId === warmIndexReusePlugin.metadata.id && args.config.campaignPreset) {
+            const executionArtifactPath = readMetadataString(result.metadata?.executionArtifactPath);
+            const campaignAgentId = readMetadataString(result.metadata?.campaignAgentId);
+            if (!executionArtifactPath) {
+                throw new Error("Warm-index campaign presentation requires an execution artifact path on the completed run.");
+            }
+            if (campaignAgentId !== "codex" && campaignAgentId !== "claude") {
+                throw new Error(`Warm-index campaign presentation requires a codex or claude campaign agent id on the completed run; received ${String(campaignAgentId)}.`);
+            }
+            const presentation = await runWarmIndexCampaignPresentation({
+                outputRoot: String(result.metadata?.outputRoot ?? ""),
+                reportPaths: reports.outputPaths,
+                executionArtifactPath,
+                campaignPreset: args.config.campaignPreset,
+                agentId: campaignAgentId,
+                captureScreenshot: options.presentation?.captureScreenshot
+            });
+            outputLines.push(`Plots: ${presentation.plots.artifactPaths.summaryPath}`);
+            if (presentation.screenshot.status === "captured") {
+                outputLines.push(`Screenshot: ${presentation.screenshot.pngPath}`);
+            }
+            else if (presentation.screenshot.status === "skipped") {
+                outputLines.push("Screenshot: skipped");
+                if (presentation.screenshot.warning)
+                    outputLines.push(presentation.screenshot.warning);
+            }
+            else {
+                outputLines.push("Screenshot: failed");
+                outputLines.push(presentation.screenshot.error ?? "Screenshot capture failed.");
+                screenshotFailed = true;
+            }
+            outputLines.push(`Gallery manifest: ${presentation.gallery.manifestPath}`);
+            outputLines.push(`Gallery index: ${presentation.gallery.indexPath}`);
+        }
+        console.log(outputLines.join("\n"));
+        if (result.status === "failed")
+            return 1;
+        if (screenshotFailed)
+            return 1;
+        return 0;
     }
     catch (error) {
         if (process.env.DEBUG) {
@@ -101,6 +145,7 @@ export function parseRunExperimentArgs(argv) {
     let requireAgents;
     let includeRealAgents;
     let kitCommand;
+    let campaignPreset;
     const commandTemplates = {};
     for (let index = 0; index < argv.length; index += 1) {
         const arg = argv[index];
@@ -161,6 +206,9 @@ export function parseRunExperimentArgs(argv) {
         else if (arg === "--kit-command") {
             kitCommand = readRequiredValue(argv, ++index, "--kit-command");
         }
+        else if (arg === "--campaign") {
+            campaignPreset = parseWarmIndexCampaignPresetId(readRequiredValue(argv, ++index, "--campaign"));
+        }
         else if (arg === "--no-screenshot") {
             // The plugin-aware report path does not capture screenshots yet; accept this
             // flag so smoke commands can share the legacy demo option set.
@@ -175,6 +223,20 @@ export function parseRunExperimentArgs(argv) {
     if (kitCommand !== undefined && experimentId !== warmIndexReusePlugin.metadata.id) {
         throw new Error(`--kit-command is only supported for --experiment ${warmIndexReusePlugin.metadata.id}.`);
     }
+    if (campaignPreset !== undefined) {
+        if (experimentId !== warmIndexReusePlugin.metadata.id) {
+            throw new Error(`--campaign is only supported for --experiment ${warmIndexReusePlugin.metadata.id}.`);
+        }
+        if (targetPath !== undefined) {
+            throw new Error("--campaign cannot be combined with --target; campaigns run only against bundled synthetic benchmark projects.");
+        }
+        if (casesPath !== undefined) {
+            throw new Error("--campaign cannot be combined with --cases; the selected preset owns the production corpus.");
+        }
+        if (projectProfilesPath !== undefined) {
+            throw new Error("--campaign cannot be combined with --project-profiles; the selected preset owns the project profiles.");
+        }
+    }
     return {
         experimentId,
         targetPath,
@@ -193,7 +255,8 @@ export function parseRunExperimentArgs(argv) {
             requireAgents,
             includeRealAgents,
             commandTemplates: Object.keys(commandTemplates).length > 0 ? commandTemplates : undefined,
-            kitCommand
+            kitCommand,
+            campaignPreset
         })
     };
 }
@@ -232,6 +295,9 @@ async function loadPluginInputs(args, toolRoot, context) {
         if (!validation.valid || !validation.config) {
             throw new Error(`Invalid warm index reuse config: ${validation.errors.join("; ")}`);
         }
+        if (args.config.campaignPreset) {
+            return loadCampaignCasesAndProjectProfiles(args.config.campaignPreset, toolRoot, context);
+        }
         return loadCasesAndProjectProfiles(args, toolRoot, context);
     }
     return undefined;
@@ -252,6 +318,19 @@ async function loadCasesAndProjectProfiles(args, toolRoot, context) {
     });
     return { cases, projectProfiles, env: process.env };
 }
+// Campaign resource resolution (v0.5.2 Batch 1): the preset owns both bundled resource paths;
+// explicit --cases/--project-profiles are already rejected for campaign mode by argument parsing.
+async function loadCampaignCasesAndProjectProfiles(campaignPresetId, toolRoot, context) {
+    const preset = getWarmIndexCampaignPreset(campaignPresetId);
+    const projectProfilesPath = resolvePackageResource(context, preset.projectProfilesResourcePath);
+    const casesPath = resolvePackageResource(context, preset.casesResourcePath);
+    const projectProfiles = await readBenchmarkProjectProfiles(projectProfilesPath, toolRoot);
+    const cases = await readEvaluationCases(casesPath, toolRoot, {
+        projectProfiles,
+        requireProjectProfileRef: true
+    });
+    return { cases, projectProfiles, env: process.env };
+}
 function splitList(value) {
     return value
         .split(",")
@@ -272,6 +351,9 @@ function parsePositiveInteger(label, value) {
     }
     return parsed;
 }
+function readMetadataString(value) {
+    return typeof value === "string" && value.trim() ? value : undefined;
+}
 function withoutUndefined(value) {
     return Object.fromEntries(Object.entries(value).filter(([, entryValue]) => entryValue !== undefined));
 }

package/dist/src/commands/runWarmIndexCampaignPresentation.js

@@ -0,0 +1,64 @@
+import path from "node:path";
+import { writeWarmIndexCampaignGallery } from "../gallery/writeWarmIndexCampaignGallery.js";
+import { captureReportScreenshot } from "../screenshot/captureReportScreenshot.js";
+import { writePlotArtifacts } from "../plots/writePlotArtifacts.js";
+// ---------------------------------------------------------------------------
+// v0.5.2 Batch 5 -- command-level warm-index campaign presentation owner.
+//
+// This is the single orchestrator for: existing report -> existing four warm-
+// index plots -> best-effort report screenshot -> narrow warm-index campaign
+// gallery. It reuses writePlotArtifacts, captureReportScreenshot, and
+// writeWarmIndexCampaignGallery unchanged; it builds no new plot, screenshot,
+// or gallery mechanism of its own. The warm-index-reuse plugin remains free of
+// this orchestration -- it is invoked only by the command layer, and only for
+// campaign runs.
+// ---------------------------------------------------------------------------
+const EXPECTED_WARM_INDEX_CHART_IDS = [
+    "warm-index-amortized-index-cost",
+    "warm-index-context-size",
+    "warm-index-correctness",
+    "warm-index-cumulative-token-usage",
+];
+export async function runWarmIndexCampaignPresentation(args) {
+    const plotsDir = path.join(args.outputRoot, "plots");
+    const plots = await writePlotArtifacts({ experimentDir: args.outputRoot, outDir: plotsDir });
+    const chartIds = Object.keys(plots.artifactPaths.charts);
+    const hasExactlyExpectedCharts = chartIds.length === EXPECTED_WARM_INDEX_CHART_IDS.length &&
+        EXPECTED_WARM_INDEX_CHART_IDS.every((id) => chartIds.includes(id));
+    if (!hasExactlyExpectedCharts) {
+        throw new Error(`Warm-index campaign presentation requires exactly the four warm-index plot charts (${EXPECTED_WARM_INDEX_CHART_IDS.join(", ")}); got: ${chartIds.join(", ") || "none"}.`);
+    }
+    const capture = args.captureScreenshot ?? captureReportScreenshot;
+    const pngPath = path.join(args.outputRoot, "report.png");
+    // Non-fatal by design: a missing Playwright/browser runtime is a normal, expected environment,
+    // never a reason to fail the campaign (see captureReportScreenshot's captured/skipped/failed contract).
+    const screenshot = await capture(args.reportPaths.htmlPath, pngPath);
+    const gallery = await writeWarmIndexCampaignGallery({
+        outDir: path.join(args.outputRoot, "gallery"),
+        campaignOutputRoot: args.outputRoot,
+        reportPaths: {
+            jsonPath: args.reportPaths.jsonPath,
+            htmlPath: args.reportPaths.htmlPath,
+            textPath: args.reportPaths.textPath,
+        },
+        executionArtifactPath: args.executionArtifactPath,
+        plotArtifacts: plots,
+        screenshot,
+        campaignPreset: args.campaignPreset,
+        agentId: args.agentId,
+    });
+    const warnings = [];
+    if (screenshot.status === "skipped" && screenshot.warning) {
+        warnings.push(screenshot.warning);
+    }
+    else if (screenshot.status === "failed") {
+        warnings.push(`Report screenshot capture failed: ${screenshot.error ?? "unknown error"}.`);
+    }
+    warnings.push(...plots.summary.warnings);
+    return {
+        plots,
+        screenshot,
+        gallery: { manifest: gallery.manifest, manifestPath: gallery.manifestPath, indexPath: gallery.indexPath },
+        warnings,
+    };
+}