@dailephd/my-dev-kit-lab 0.9.0 → 0.9.1

package/CHANGELOG.md

@@ -2,6 +2,15 @@
 
 All notable changes to my-dev-kit-lab are documented here.
 
+## [0.9.1] - 2026-10-12
+
+### Normalized provider telemetry and resumable campaigns
+
+- Normalized provider token and duration provenance while preserving legacy parser compatibility and separating unavailable telemetry from task outcomes.
+- Added deterministic agent-success-rate campaign queues, immutable checkpoint generations, version 1 compatibility and version 2 recovery epochs, explicit resume and verified completed-unit reuse, and bounded provider-limit continuation.
+- Added public campaign CLI options, Codex and Claude timeout presets, campaign progress summaries, and strong/partial/unavailable metric-specific evidence classifications.
+- Preserved partial evidence without presenting it as complete scientific analysis; uncertain provider invocations are not replayed automatically. Real-provider use remains explicitly opted in and depends on the existing provider adapters.
+
 ## [Unreleased]
 
 ## [0.9.0] - 2026-10-10

package/README.md

@@ -6,11 +6,11 @@ my-dev-kit provides local repository indexing and graph-guided retrieval. my-dev
 
 Whole-ecosystem workflows are centralized in [my-dev-kit/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md). Its [command-surface compatibility map](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md#915-command-surface-compatibility-map) documents which Lab outputs can safely seed my-dev-kit investigation, Observer reference work, or Orchestrator evidence, and which look-alike artifacts are not direct inputs.
 
-The latest release is v0.9.0 (agent-success-rate implementation-patch evaluation), and the repository package version is 0.9.0. The previous release is v0.8.2 (context-pack generation experiments), preceded by v0.8.1 (retrieval query strategy comparison). The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
+The latest release is v0.9.1 (normalized provider telemetry and resumable agent-success-rate campaigns), and the repository package version is 0.9.1. The previous release is v0.9.0 (agent-success-rate implementation-patch evaluation). The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
 
 v0.9.0 adds the `agent-success-rate` plugin, the eighth registered experiment plugin, which evaluates implementation patches under the `raw-full-file` and `context-pack` treatments.
 
-Planned after that: v0.9.1 (provider telemetry and campaign scheduling) and v0.9.2.
+v0.9.1 adds normalized provider telemetry and resumable campaign execution. v0.9.2 remains planned.
 
 v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
 
@@ -27,6 +27,7 @@ v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--vers
 - **Measure retrieval precision and recall (v0.8.0, released):** the `retrieval-precision-recall` plugin runs the existing my-dev-kit search, lookup, slice, and source lifecycle for each case and compares the retrieved files and symbols with a deterministic answer key, without invoking any agent. It reports file precision and recall, symbol precision and recall, fact coverage (from explicit fact-to-context mappings), irrelevant context ratio, the estimated retrieved token count, and the missed and irrelevant identities, as `retrieval-precision-recall-execution.json` plus `report.json`, `report.html`, and `report.txt`. By default it runs a bundled 12-case corpus over two benchmark projects against the Lab itself. It can also run over an explicitly selected local Git repository with `--target <repository> --local-subject-config <path>`; in that mode the repository is treated as read-only and durable outputs redact the repository path and file, symbol, and fact identities while keeping the numeric evidence. These are set-based measures with no ranking, winner, or composite score. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md).
 - **Compare retrieval query strategies (v0.8.1, released):** the `retrieval-query-strategy-comparison` plugin compares seven fixed my-dev-kit retrieval strategies without running any coding agent: `keyword-search`, `symbol-lookup`, `graph-neighborhood`, `source-slice`, `data-model-graph`, `model-view-lineage`, and `combined-graph-guided`. In bundled mode it uses the same frozen 12-case warm-index benchmark corpus as `retrieval-precision-recall`, and every selected case receives all seven strategies (filters narrow cases or projects, never strategies). File and symbol quality keeps the inherited v0.8.0 set-based semantics and adds balanced file and symbol F1 (`2TP / (2TP + FP + FN)`); fact coverage and the estimated retrieved token count complete the four comparison objectives. Strategies are compared over matched complete cases only, as macro means for four scopes (`overall`, `localized`, `cross-module`, `broad-change`, reusing each case's existing task-locality label), using Pareto dominance on mean file F1, mean symbol F1, mean fact coverage (all maximized) and mean retrieved tokens (minimized). A best strategy is named only when the Pareto front has a single member; when several strategies remain nondominated the report states a tradeoff. There is no scalar or composite score and no ordinal ranking. The plugin can also run over an explicitly selected local Git repository through the existing `LocalRepositorySubject` boundary with `--target <repository> --local-subject-config <path>`; the repository stays read-only and durable output redacts private file, symbol, fact, semantic-node, warning, and case-title identities while preserving the numeric results. It writes `retrieval-query-strategy-comparison-execution.json`, `retrieval-query-strategy-comparison-analysis.json`, and `report.json`, `report.html`, and `report.txt`. No context-pack behavior is included. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md#retrieval-query-strategy-comparison-metrics-v081).
 - **Generate auditable task context packs (v0.8.2):** `context-pack-generation` runs both fixed treatments, `raw-full-file` and `context-pack`, for each selected case. A pack contains a task summary, relevant files and symbols, bounded source slices, call relationships, tests, and evidence notes. Analysis reuses retrieval fact coverage and file/symbol precision, recall, and F1, reports estimated context size and paired raw-versus-pack tokens saved, percent saved, and objective deltas, and names no composite winner or ranking. Bundled runs persist pack artifacts and reports include a bounded preview. External-local runs calculate science on real identities, redact before persistence, omit durable pack bodies, and show redacted report previews. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md).
+- **Run resumable agent-success-rate campaigns (v0.9.1):** provider telemetry is normalized with provenance, and explicit campaign runs support deterministic case ordering, checkpoints, one-case execution, validated resume, safe completed-unit reuse, provider-limit pauses, authorized bounded continuation, and plugin-specific progress evidence. Campaigns use the existing Codex and Claude adapters only after explicit real-provider opt-in. Checkpoint/progress evidence is operational; complete scientific reports are available only after finalization.
 - **Evaluate implementation patches (v0.9.0):** the `agent-success-rate` plugin runs both `raw-full-file` and `context-pack` treatments over a bundled six-task implementation corpus in disposable benchmark copies and decides success from trusted checks on the changed code. The default deterministic mode applies reference patches and runs no coding agent; real Codex or Claude runs (with optional bounded repair) are explicit opt-in. It runs from the installed CLI or a built source checkout (see [docs/COMMANDS.md](docs/COMMANDS.md#agent-success-rate-v090)).
 - **Assess affected-neighborhood evidence (v0.6.1, released; retained in the current package):** using the baseline my-dev-kit graph of the same prepared warm index (loaded once per session), each task's confirmed modified or missing indexed files, and the baseline symbols those files contain, are mapped to graph nodes. The affected neighborhood is those seed nodes plus their direct one-hop neighbors (edges of any kind, either direction). The task's `expectedFiles` and file-scoped `expectedSymbols` are mapped to graph nodes and compared with that neighborhood, which yields six numeric warm-side metrics (`changedFileCount`, `changedSymbolCount`, `affectedNodeCount`, `affectedEdgeCount`, `taskOverlapCount`, `taskOverlapPercent`) plus categorical evidence: a `related`, `unrelated`, or `unknown` relationship and a `recommended`, `not-indicated`, or `unknown` `reindexRecommendation`. Incomplete evidence is reported as `unknown`, never as unrelated. The assessment is recorded additively in `warm-index-execution.json` and shown in the existing `report.json`/`report.txt`/`report.html` warm-index reports. It is observational only: retrieval, execution status, and provider status are unchanged, nothing is reindexed automatically, and no CLI flag, experiment plugin, plot, or gallery item was added. See [docs/METRICS.md](docs/METRICS.md#affected-neighborhood-metrics-v061) and [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment).
 - **Audit repository health:** run conservative code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, or adapt security findings into the common audit report.
@@ -259,7 +260,7 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 - Claude and Codex token totals are used only when the CLI/run exposes them; missing provider token evidence stays explicitly unavailable (never zero, and never substituted with an estimated context-token count)
 - Codex and Claude campaign runs can produce timeouts, invalid-output, agent-unavailable, or agent-limit-reached outcomes; these are structured provider states, not warm-index infrastructure failures
 - Small projects may make raw-full-file cheaper than my-dev-kit-guided; larger localized tasks are where my-dev-kit is expected to become more useful
-- The generic experiment-plugin framework has eight registered plugins: `context-strategy-comparison`, `warm-index-reuse`, `incremental-change-staleness`, `context-window-scaling`, `retrieval-precision-recall`, `retrieval-query-strategy-comparison`, `context-pack-generation`, and `agent-success-rate` (v0.9.0). The v0.7.0 context-window-scaling plugin is released; v0.7.1 added caller-supplied synthetic repositories; v0.7.2 added an explicitly selected local Git repository. The context-pack-generation plugin shipped in v0.8.2; the agent-success-rate plugin shipped in v0.9.0.
+- The generic experiment-plugin framework has eight registered plugins: `context-strategy-comparison`, `warm-index-reuse`, `incremental-change-staleness`, `context-window-scaling`, `retrieval-precision-recall`, `retrieval-query-strategy-comparison`, `context-pack-generation`, and `agent-success-rate` (v0.9.0). The v0.7.0 context-window-scaling plugin is released; v0.7.1 added caller-supplied synthetic repositories; v0.7.2 added an explicitly selected local Git repository. The context-pack-generation plugin shipped in v0.8.2; agent-success-rate shipped in v0.9.0 and gained resumable campaigns in v0.9.1.
 - `retrieval-precision-recall` (v0.8.0) measures set-based retrieval quality, not ranked retrieval: it has no ranking metrics, no retrieval-strategy comparison of its own (the separate `retrieval-query-strategy-comparison` plugin added in v0.8.1 compares strategies), no winner or composite score, and no real agents. Its lifecycle expands only the top search candidate. An external-local run fails as a whole if any case fails a safety or execution check, and then writes no normal artifact or report family. Durable external-local identities are redacted, although the console summary of a successful run can still show the physical output directory. Platform path-length limits can make a run with very long paths fail cleanly instead of executing.
 - `retrieval-query-strategy-comparison` (v0.8.1) runs no coding agent and does not evaluate answer quality. Its retrieved token count is a context-size estimate, not provider billing telemetry. Semantic strategies (`data-model-graph`, `model-view-lineage`) can truthfully be partial or unavailable for a case, and a case is then excluded from the matched cross-strategy comparison for every strategy while its per-strategy evidence is kept. The Pareto front is a set of nondominated strategies in canonical strategy order, not an ordinal ranking. The plugin reports no significance test or confidence interval and selects no strategy automatically. External-local mode rejects `--case` and `--benchmark-project`, builds one private base index per configured case, and fails as a whole, writing no normal artifact or report family, if any safety or privacy check fails.
 - `warm-index-reuse` without `--campaign` still uses the deterministic fake agent only. With `--campaign`, exactly one real provider (Codex or Claude) runs per campaign preset — there is no multi-agent matrix, retry, provider switching, or scheduler. The v0.5.1 expanded corpus is deterministic fake-agent benchmark evidence when run without `--campaign`. No warm-index mode calculates a token-savings percentage, break-even task, winner, or ranking, and component-duration sums are not wall-clock latency

package/dist/src/agents/adapters/claudeAdapter.js

@@ -54,8 +54,8 @@ export const claudeAdapter = {
 };
 /**
  * Parses a single JSON result envelope (`{ "result": "..." }`) when stdout is valid JSON, never
- * the raw envelope itself. Falls back to the legacy plain-text behavior when stdout is not valid
- * JSON, so existing argv-mode text output keeps working unchanged.
+ * the raw envelope itself. Malformed JSON-shaped output stays empty; legacy plain text output
+ * retains its fallback.
  */
 export function parseClaudeFinalAnswer(text) {
     const trimmedInput = text.trim();
@@ -71,9 +71,12 @@ export function parseClaudeFinalAnswer(text) {
                 // Valid JSON object without a string result: empty, never the raw envelope.
                 return { finalAnswerText: "", finalAnswerParseStatus: "empty" };
             }
+            return { finalAnswerText: "", finalAnswerParseStatus: "empty" };
         }
         catch {
-            // Not valid JSON; fall through to the legacy plain-text behavior below.
+            // A malformed envelope is not a legacy text answer.
+            if (/^[\[{]/.test(trimmedInput))
+                return { finalAnswerText: "", finalAnswerParseStatus: "empty" };
         }
     }
     return { finalAnswerText: trimmedInput, finalAnswerParseStatus: trimmedInput ? "parsed" : "empty" };

package/dist/src/agents/adapters/codexAdapter.js

@@ -1,6 +1,7 @@
 import { runMeasuredCommand } from "../../core/runMeasuredCommand.js";
 import { applyPromptToCommandTemplate } from "../runAgentPrompt.js";
 import { parseAgentTokenUsage } from "../parseAgentTokenUsage.js";
+import { trustedProviderLaunch } from "../providerBuildObservation.js";
 // v0.5.2 Batch 2 -- frozen stdin invocation for the real-agent evaluation transport. Every flag
 // here is planner-owned; do not add sandbox/model/approval-policy flags in this batch.
 const CODEX_STDIN_ARGS = [
@@ -51,13 +52,21 @@ export const codexAdapter = {
 /**
  * Recognizes a Codex JSONL event stream (at least one line parses to an object with a string
  * `type`) and, when recognized, uses the LAST `item.completed` `agent_message` text as the final
- * answer -- never the raw JSONL. Falls back to the legacy plain-text behavior when no event-shaped
- * line is found, so non-JSONL Codex output (and existing tests) keep working unchanged.
+ * answer -- never the raw JSONL. JSON-shaped output without an answer stays empty. Plain text
+ * retains the legacy fallback. A single formatted JSON event is supported as well.
  */
 export function parseCodexFinalAnswer(text) {
-    let sawEventLine = false;
+    let sawEventLine = /^[\s]*[\[{]/.test(text);
     let lastAgentMessageText = null;
-    for (const line of text.split(/\r?\n/)) {
+    const candidates = function* () {
+        const trimmed = text.trim();
+        if (trimmed.startsWith("{") || trimmed.startsWith("["))
+            yield trimmed;
+        for (const match of text.matchAll(/[^\r\n]+/g))
+            if (match[0].trim() !== trimmed)
+                yield match[0];
+    };
+    for (const line of candidates()) {
         const trimmedLine = line.trim();
         if (!trimmedLine)
             continue;
@@ -69,7 +78,10 @@ export function parseCodexFinalAnswer(text) {
             // Agent output can mix JSONL with incidental non-JSON noise; ignore malformed lines.
             continue;
         }
-        if (!parsed || typeof parsed !== "object" || Array.isArray(parsed))
+        if (!parsed || typeof parsed !== "object")
+            continue;
+        sawEventLine = true;
+        if (Array.isArray(parsed))
             continue;
         const event = parsed;
         if (typeof event.type !== "string")
@@ -95,7 +107,10 @@ export function parseCodexFinalAnswer(text) {
 export async function runCliAgent(request, adapter) {
     const started = Date.now();
     const command = adapter.buildCommand(request);
-    const available = await adapter.isAvailable(request);
+    if (request.trustedExecutable && (request.commandTemplate || request.promptTransport !== "stdin" || adapter.id === "fake-agent"))
+        throw new Error("INVALID_TRUSTED_PROVIDER_REQUEST");
+    const launch = request.trustedExecutable ? await trustedProviderLaunch(request.trustedExecutable, adapter.id) : null;
+    const available = launch !== null || await adapter.isAvailable(request);
     if (!available) {
         const ended = Date.now();
         const status = request.requireAvailable ? "failed" : "skipped";
@@ -127,18 +142,20 @@ export async function runCliAgent(request, adapter) {
     }
     const measured = await runMeasuredCommand({
         commandId: `${adapter.id}-agent-run`,
-        commandString: command.command,
-        extraArgs: command.args,
+        ...(launch ? { executable: launch.executable, args: [...launch.prefix, ...command.args], resolveCommand: false } : { commandString: command.command, extraArgs: command.args }),
         cwd: request.commandTemplate?.cwd ?? request.cwd,
         outDir: request.outDir,
         env: request.env,
         timeoutMs: request.timeoutMs,
-        stdinText: command.stdinText
+        stdinText: command.stdinText,
+        ...(launch ? { stdoutMaxBytes: 8 * 1024 * 1024, stderrMaxBytes: 1024 * 1024, terminateProcessTree: true } : {})
     });
     const ended = Date.now();
     const combinedOutput = `${measured.stdout}\n${measured.stderr}`;
     const parsedAnswer = adapter.parseFinalAnswer(measured.stdout || measured.stderr);
     const parsedUsage = adapter.parseTokenUsage(combinedOutput);
+    // Bound campaigns require process-channel diagnostics. Never search final agent message/patch bodies.
+    const condition = launch ? trustedProcessCondition(measured.stdout, measured.stderr, measured.timedOut === true) : null;
     return {
         runId: request.runId,
         agentId: adapter.id,
@@ -164,6 +181,30 @@ export async function runCliAgent(request, adapter) {
         tokenUsageSource: parsedUsage.tokenUsageSource,
         tokenUsageReliability: parsedUsage.tokenUsageReliability,
         warnings: parsedUsage.warnings,
-        errors: measured.ok ? [] : [measured.error ?? "Agent command failed."]
+        errors: [...(measured.ok ? [] : [measured.error ?? "Agent command failed."]), ...(condition === "limit" ? ["Provider rate limit reached."] : condition === "timeout" ? ["Provider command timed out."] : [])]
     };
 }
+function trustedProcessCondition(stdout, stderr, timedOut) {
+    if (timedOut)
+        return "timeout";
+    const diagnostics = [];
+    const isPatch = (text) => /^(?:diff --git |@@ |\+\+\+ |--- )/m.test(text);
+    // Non-interactive CLI diagnostic lines, not arbitrary substrings or diff additions.
+    if (!isPatch(stderr))
+        diagnostics.push(...stderr.split(/\r?\n/).filter(line => /^(?:error:\s*)?(?:rate limit (?:exceeded|reached)|usage limit (?:exceeded|reached)|quota (?:exceeded|exhausted)|too many requests|you(?:'ve| have) (?:hit|reached) (?:your |the )?(?:usage |session |rate )?limit)/i.test(line.trim())));
+    for (const line of stdout.split(/\r?\n/).slice(0, 1000)) {
+        try {
+            const event = JSON.parse(line);
+            if (!event || typeof event !== "object")
+                continue;
+            if (event.type !== "error" && event.type !== "turn.failed" && event.is_error !== true)
+                continue;
+            const error = event.error;
+            const message = typeof error === "string" ? error : error && typeof error === "object" ? error.message : event.message;
+            if (typeof message === "string" && message.length <= 2048 && !isPatch(message))
+                diagnostics.push(message);
+        }
+        catch { /* Not a structured error event; agent output is deliberately not diagnostic evidence. */ }
+    }
+    return diagnostics.some(text => /\b(?:rate\s+limit|usage\s+limit|quota|too many requests|session\s+limit)\b/i.test(text)) ? "limit" : null;
+}

package/dist/src/agents/index.js

@@ -2,6 +2,7 @@ export * from "./types.js";
 export * from "./agentRegistry.js";
 export * from "./runAgentPrompt.js";
 export * from "./parseAgentTokenUsage.js";
+export { normalizeAgentTokenUsageV091, normalizeAgentDurationV091 } from "./normalizeAgentTelemetry.js";
 export * from "./adapters/fakeAgentAdapter.js";
 export * from "./adapters/codexAdapter.js";
 export * from "./adapters/claudeAdapter.js";

package/dist/src/agents/normalizeAgentTelemetry.js

@@ -0,0 +1,50 @@
+const EVIDENCE_LEVEL = {
+    "provider-reported": "provider-reported",
+    "cli-json": "cli-reported",
+    "agent-reported": "parsed-from-output",
+    "estimated-from-text": "estimated",
+    otel: "unavailable",
+    unavailable: "unavailable"
+};
+function normalizeCount(value) {
+    return typeof value === "number" && Number.isFinite(value) && Number.isInteger(value) && value >= 0 ? value : null;
+}
+/**
+ * Classifies the legacy evidence channel, not the origin of each individual field.
+ * Parser-derived totals are preserved; missing totals are never calculated here.
+ * The existing contract has no verified OTel attribution, so its channel remains unavailable.
+ */
+export function normalizeAgentTokenUsageV091(usage, reliability) {
+    const counts = {
+        totalTokens: normalizeCount(usage.totalTokens),
+        inputTokens: normalizeCount(usage.inputTokens),
+        outputTokens: normalizeCount(usage.outputTokens),
+        cachedInputTokens: normalizeCount(usage.cachedInputTokens),
+        reasoningTokens: normalizeCount(usage.reasoningTokens)
+    };
+    const numericReasons = {};
+    for (const key of Object.keys(counts)) {
+        numericReasons[key] = counts[key] !== null ? null : usage[key] == null ? "missing-token-count" : "invalid-token-count";
+    }
+    const hasCount = Object.values(counts).some((count) => count !== null);
+    return {
+        source: usage.source,
+        reliability,
+        evidenceLevel: EVIDENCE_LEVEL[usage.source],
+        classificationReason: usage.source === "otel" ? "unattributed-source" : usage.source === "unavailable" ? "source-unavailable" : null,
+        numericAvailability: hasCount ? "available" : "unavailable",
+        condition: hasCount ? "token-available" : "token-unavailable",
+        ...counts,
+        numericReasons
+    };
+}
+/** Accepts the current Lab wall-clock measurement, never provider-reported timing. */
+export function normalizeAgentDurationV091(durationMs) {
+    if (durationMs == null) {
+        return { durationMs: null, source: "unavailable", reason: "missing-duration" };
+    }
+    if (!Number.isFinite(durationMs) || durationMs < 0) {
+        return { durationMs: null, source: "unavailable", reason: "invalid-duration" };
+    }
+    return { durationMs, source: "lab-wall-clock", reason: null };
+}

package/dist/src/agents/parseAgentTokenUsage.js

@@ -13,11 +13,11 @@ const numberFields = {
 };
 export function parseAgentTokenUsage(text) {
     const warnings = [];
-    const jsonUsage = parseJsonUsage(text);
+    const jsonUsage = parseJsonUsage(text, warnings);
     if (jsonUsage.found) {
         return buildResult(jsonUsage.usage, "cli-json", text, warnings);
     }
-    const plainTextUsage = parsePlainTextUsage(text);
+    const plainTextUsage = jsonUsage.structured ? { found: false, usage: {} } : parsePlainTextUsage(text);
     if (plainTextUsage.found) {
         return buildResult(plainTextUsage.usage, "agent-reported", text, warnings);
     }
@@ -29,57 +29,84 @@ export function parseAgentTokenUsage(text) {
         warnings
     };
 }
-function parseJsonUsage(text) {
-    const usage = {};
+function parseJsonUsage(text, warnings) {
+    let usage;
+    let structured = false;
+    let visited = 0;
+    const warn = (message) => { if (!warnings.includes(message))
+        warnings.push(message); };
+    const visit = (value, depth) => {
+        if (!value || typeof value !== "object")
+            return;
+        if (++visited > 2048 || depth > 16) {
+            warn("Token usage traversal limit reached; additional envelopes were ignored.");
+            return;
+        }
+        if (Array.isArray(value)) {
+            for (const child of value) {
+                if (visited > 2048)
+                    break;
+                visit(child, depth + 1);
+            }
+            return;
+        }
+        const record = value;
+        const local = {};
+        for (const [key, aliases] of Object.entries(numberFields)) {
+            for (const alias of aliases) {
+                const count = record[alias];
+                if (count === undefined)
+                    continue;
+                if (typeof count !== "number" || !Number.isFinite(count) || !Number.isInteger(count) || count < 0) {
+                    warn("Invalid token count ignored in agent output.");
+                    continue;
+                }
+                if (local[key] !== undefined && local[key] !== count)
+                    warn("Conflicting token aliases; legacy maximum precedence retained within one usage record.");
+                local[key] = Math.max(local[key] ?? 0, count);
+            }
+        }
+        if (hasAnyUsage(local)) {
+            if (usage === undefined)
+                usage = local;
+            else if (JSON.stringify(usage) !== JSON.stringify(local))
+                warn("Multiple token usage records; first valid record retained without combining attempts.");
+        }
+        // Supported envelope traversal only: patch/tool payload objects are not usage metadata.
+        for (const key of ["usage", "token_usage", "tokenUsage", "response", "message", "data", "event", "result"]) {
+            if (visited > 2048)
+                break;
+            visit(record[key], depth + 1);
+        }
+    };
+    let candidates = 0;
     for (const candidate of collectJsonCandidates(text)) {
+        if (++candidates > 4096) {
+            warn("Token usage candidate limit reached; additional lines were ignored.");
+            break;
+        }
         try {
-            mergeUsage(usage, extractUsage(JSON.parse(candidate)));
+            const value = JSON.parse(candidate);
+            structured = true;
+            visit(value, 0);
         }
         catch {
             // Agent output is often mixed JSONL plus text; malformed lines are ignored.
         }
     }
-    return { found: hasAnyUsage(usage), usage };
+    return { found: usage !== undefined, usage: usage ?? {}, structured };
 }
-function collectJsonCandidates(text) {
+function* collectJsonCandidates(text) {
     const trimmed = text.trim();
-    const candidates = [];
     if (trimmed.startsWith("{") || trimmed.startsWith("[")) {
-        candidates.push(trimmed);
+        yield trimmed;
     }
-    for (const line of text.split(/\r?\n/)) {
-        const lineTrimmed = line.trim();
-        if (lineTrimmed.startsWith("{") || lineTrimmed.startsWith("[")) {
-            candidates.push(lineTrimmed);
+    for (const match of text.matchAll(/[^\r\n]+/g)) {
+        const lineTrimmed = match[0].trim();
+        if (lineTrimmed !== trimmed && (lineTrimmed.startsWith("{") || lineTrimmed.startsWith("["))) {
+            yield lineTrimmed;
         }
     }
-    return [...new Set(candidates)];
-}
-function extractUsage(value) {
-    const usage = {};
-    visit(value, usage);
-    return usage;
-}
-function visit(value, usage) {
-    if (!value || typeof value !== "object") {
-        return;
-    }
-    if (Array.isArray(value)) {
-        value.forEach((item) => visit(item, usage));
-        return;
-    }
-    const record = value;
-    for (const [usageKey, fieldNames] of Object.entries(numberFields)) {
-        for (const fieldName of fieldNames) {
-            const fieldValue = record[fieldName];
-            if (typeof fieldValue === "number" && Number.isFinite(fieldValue)) {
-                usage[usageKey] = Math.max(usage[usageKey] ?? 0, fieldValue);
-            }
-        }
-    }
-    for (const child of Object.values(record)) {
-        visit(child, usage);
-    }
 }
 function parsePlainTextUsage(text) {
     const usage = {};
@@ -91,20 +118,15 @@ function parsePlainTextUsage(text) {
         ["totalTokens", /total\s+tokens?\s*[:=]\s*(\d+)/i]
     ];
     for (const [key, pattern] of patterns) {
-        const match = text.match(pattern);
+        const match = text.match(new RegExp(pattern.source.replace("(\\d+)", "(\\d+)(?![\\d.eE])"), pattern.flags));
         if (match?.[1]) {
-            usage[key] = Number(match[1]);
+            const count = Number(match[1]);
+            if (Number.isFinite(count) && Number.isInteger(count))
+                usage[key] = count;
         }
     }
     return { found: hasAnyUsage(usage), usage };
 }
-function mergeUsage(target, source) {
-    for (const key of Object.keys(numberFields)) {
-        if (source[key] !== undefined) {
-            target[key] = Math.max(target[key] ?? 0, source[key]);
-        }
-    }
-}
 function hasAnyUsage(usage) {
     return Object.values(usage).some((value) => typeof value === "number");
 }

package/dist/src/agents/providerBuildObservation.js

@@ -0,0 +1,97 @@
+import { createHash } from "node:crypto";
+import { lstat, mkdtemp, readFile, realpath, rm } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { resolveCommand } from "../core/resolveCommand.js";
+import { runMeasuredCommand } from "../core/runMeasuredCommand.js";
+const launches = new WeakMap();
+const hash = (bytes) => createHash("sha256").update(bytes).digest("hex");
+const fail = () => { throw new Error("PROVIDER_BUILD_UNVERIFIABLE: use a native executable or an attributable node entrypoint; unsupported wrappers cannot authorize a campaign."); };
+async function inspect(alias) {
+    const actual = await realpath(alias);
+    const stat = await lstat(actual);
+    if (!stat.isFile() || stat.isSymbolicLink() || stat.size > 256 * 1024 * 1024 || stat.size === 0)
+        return fail();
+    const bytes = await readFile(actual);
+    if (bytes.length !== stat.size || await realpath(alias) !== actual)
+        return fail();
+    return { proof: { alias: path.resolve(alias), actual, digest: hash(bytes) }, bytes };
+}
+function native(bytes) {
+    return bytes.subarray(0, 2).toString() === "MZ" || bytes.subarray(0, 4).equals(Buffer.from([0x7f, 0x45, 0x4c, 0x46])) || ["feedface", "feedfacf", "cefaedfe", "cffaedfe", "cafebabe"].includes(bytes.subarray(0, 4).toString("hex"));
+}
+/** Read-only task-free observation. All temporary version diagnostics are removed. No shell templates. */
+export async function observeProviderBuild(providerId, options = {}) {
+    if (providerId !== "codex" && providerId !== "claude")
+        return fail();
+    const cwd = await mkdtemp(path.join(os.tmpdir(), "lab-provider-build-"));
+    try {
+        const env = { ...process.env, ...options.env };
+        const resolved = resolveCommand(providerId, { cwd, env, allowPowerShellShim: false });
+        if (!resolved.resolvedPath || resolved.resolutionKind === "unavailable")
+            return fail();
+        const selected = await inspect(resolved.resolvedPath);
+        let executable = selected.proof.actual;
+        const prefix = [];
+        const files = [selected.proof];
+        let limitation = "Observed launch bytes only; transitive dependencies are not attested.";
+        if (!native(selected.bytes)) {
+            const text = selected.bytes.toString("utf8");
+            // A deliberately closed, attributable wrapper subset. Never execute arbitrary wrapper text.
+            const cmd = /^@echo off\r?\n(?:node|"([^"\r\n]+)") "([^"\r\n]+)" %\*\r?\n?$/i.exec(text);
+            const shebang = /^#!\/usr\/bin\/env node\r?\n/.test(text);
+            if (!cmd && !shebang)
+                return fail();
+            const node = resolveCommand(cmd?.[1] ?? "node", { cwd, env, allowPowerShellShim: false });
+            if (!node.resolvedPath || node.resolutionKind === "unavailable")
+                return fail();
+            const interpreter = await inspect(node.resolvedPath);
+            if (!native(interpreter.bytes))
+                return fail();
+            files.push(interpreter.proof);
+            executable = interpreter.proof.actual;
+            if (cmd) {
+                if (!path.isAbsolute(cmd[2]))
+                    return fail();
+                const entry = await inspect(cmd[2]);
+                files.push(entry.proof);
+                prefix.push(entry.proof.actual);
+            }
+            else
+                prefix.push(selected.proof.actual);
+            limitation = "Restricted node wrapper, interpreter and entrypoint bytes observed; transitive dependencies are not attested.";
+        }
+        const version = await runMeasuredCommand({ commandId: `${providerId}-build-version`, executable, args: [...prefix, "--version"], resolveCommand: false, cwd, outDir: cwd, env, timeoutMs: 5000, stdoutMaxBytes: 4096, stderrMaxBytes: 4096, terminateProcessTree: true });
+        const observedVersion = version.stdout.trim();
+        if (!version.ok || version.timedOut || version.outputLimit || !/^[\x20-\x7e]{1,512}$/.test(observedVersion) || !/\d+\.\d+/.test(observedVersion))
+            return fail();
+        const capability = Object.freeze({ providerId });
+        const bytes = Buffer.from(JSON.stringify({ contract: "local-provider-launch-v1", providerId, origin: "inspected-local-executable", executable, prefix, files, version: observedVersion, adapterContract: `${providerId}-fixed-stdin-v1`, limitation }));
+        launches.set(capability, { executable, prefix, files, bytes });
+        await trustedProviderObservation(capability, providerId);
+        return capability;
+    }
+    catch {
+        return fail();
+    }
+    finally {
+        await rm(cwd, { recursive: true, force: true });
+    }
+}
+export async function trustedProviderObservation(capability, providerId) {
+    const launch = launches.get(capability);
+    if (!launch || capability.providerId !== providerId)
+        return fail();
+    for (const file of launch.files) {
+        const current = await inspect(file.alias);
+        if (current.proof.actual !== file.actual || current.proof.digest !== file.digest)
+            throw new Error("PROVIDER_BUILD_CHANGED");
+    }
+    return Buffer.from(launch.bytes);
+}
+/** Fixed adapter args are appended by the adapter owner; no PATH lookup occurs on this launch. */
+export async function trustedProviderLaunch(capability, providerId) {
+    await trustedProviderObservation(capability, providerId);
+    const launch = launches.get(capability);
+    return { executable: launch.executable, prefix: [...launch.prefix] };
+}

package/dist/src/audits/core/projectInventory.js

@@ -145,6 +145,7 @@ const EXCLUDED_DIR_NAMES = new Set([
     ".my-dev-kit",
     ".my-dev-kit-v1",
     ".my-dev-kit-lab",
+    ".my-dev-kit-context",
     "__pycache__",
     ".idea",
     ".vscode",

package/dist/src/cli/help.js

@@ -306,11 +306,16 @@ export function renderExperimentRunHelp() {
         "                                                      with fixed-category feedback and the untrusted previous diff; it",
         "                                                      never follows a provider, sandbox or verification infrastructure",
         "                                                      problem.",
-        "                                                      Writes report.json, report.html and report.txt plus",
+        "                                                      Ordinary noncampaign execution writes report.json, report.html and report.txt plus",
         "                                                      agent-success-rate-execution.json and",
         "                                                      agent-success-rate-analysis.json. Initial-attempt and final success",
         "                                                      are reported separately. Results are descriptive for the finite",
-        "                                                      corpus: no winner, ranking, composite score or significance claim."
+        "                                                      corpus: no winner, ranking, composite score or significance claim.",
+        "",
+        "agent-success-rate resumable campaigns and timeouts only:",
+        ...AGENT_CAMPAIGN_OPTION_HELP.map(line => `  ${line}`),
+        "Examples (replace bracketed values; explicit source-root is required when package source proof is unavailable):",
+        ...AGENT_CAMPAIGN_EXAMPLES.map(line => `  ${line}`)
     ].join("\n");
 }
 export function renderExperimentControlledHelp() {
@@ -527,3 +532,4 @@ export function renderTutorialRunHelp() {
         "  2   Invalid command usage"
     ].join("\n");
 }
+import { AGENT_CAMPAIGN_EXAMPLES, AGENT_CAMPAIGN_OPTION_HELP } from "../commands/runExperimentDescribeCommand.js";

package/dist/src/commands/runExperimentDescribeCommand.js

@@ -1,5 +1,30 @@
 import { createDefaultExperimentPluginRegistry } from "../experiments/index.js";
 import { AGENT_SUCCESS_RATE_DEFAULT_AGENT_TIMEOUT_MS, AGENT_SUCCESS_RATE_MAX_AGENT_TIMEOUT_MS, AGENT_SUCCESS_RATE_REAL_AGENT_IDS, AGENT_SUCCESS_RATE_REPAIR_ATTEMPT_VALUES, AGENT_SUCCESS_RATE_TREATMENT_IDS } from "../experiments/plugins/agentSuccessRate/index.js";
+import { AGENT_TIMEOUT_PRESETS } from "../experiments/plugins/agentSuccessRate/timeoutPresets.js";
+/** Public command-owned operational contract, shared by human, JSON and run help. */
+export const AGENT_CAMPAIGN_OPTION_HELP = [
+    "--agent-campaign <new|resume|continue-limit>: explicit version 2 campaign; requires stable --out <directory>.",
+    "--one-case: campaign-only; process at most the first unfinished selected case, with both treatments mandatory.",
+    "--campaign-source-root <directory>: campaign-only verified current source checkout; required when installed source proof is unavailable.",
+    "--timeout-preset <standard|extended|maximum>: codex/claude real-agent mode only; 240000/900000/1800000 ms per attempt.",
+    "--timeout-ms <ms>: numeric override; mutually exclusive with --timeout-preset. Neither supplied preserves 240000 ms default.",
+    "--expected-checkpoint-revision <positive-integer>: continue-limit only; exact committed checkpoint generation.",
+    "--limit-case <case-id> --limit-treatment <raw-full-file|context-pack> --limit-epoch <epoch-id>: continue-limit only; exact terminal limit unit and epoch.",
+    "--authorize-additional-invocation: mandatory boolean with no value for continue-limit; at most two execution epochs per treatment.",
+    "Resume revalidates identity, ownership and artifact digests and skips completed work, including completed task failures.",
+    "Real campaigns require --agent <codex|claude> and --include-real-agents plus trusted local build observation bound to invocation.",
+    "Native executables and attributable restricted node entrypoints are supported; unsupported wrappers are UNVERIFIABLE. No universal installation support or transitive attestation.",
+    "Uncertain invocations block resume; no force replay, marker stealing, automatic quota waiting, polling or provider switching.",
+    "Partial campaign work is persisted without final scientific reports; complete campaigns write the existing execution/analysis v1 artifacts.",
+    "Campaign exits: 0 consistent bounded work (including one-case partial), 1 paused/blocked/failed execution, 2 invalid CLI usage. Legacy behavior is preserved.",
+    "--campaign belongs to warm-index-reuse and is rejected by agent-success-rate; other plugins reject these campaign options."
+];
+export const AGENT_CAMPAIGN_EXAMPLES = [
+    "my-dev-kit-lab experiment run --experiment agent-success-rate --agent-campaign new --one-case --out <campaign-output-root> --campaign-source-root <source-checkout>",
+    "my-dev-kit-lab experiment run --experiment agent-success-rate --agent-campaign resume --one-case --out <same-campaign-output-root> --campaign-source-root <source-checkout>",
+    "my-dev-kit-lab experiment run --experiment agent-success-rate --agent codex --include-real-agents --agent-campaign new --timeout-preset extended --out <campaign-output-root> --campaign-source-root <source-checkout>",
+    "my-dev-kit-lab experiment run --experiment agent-success-rate --agent codex --include-real-agents --agent-campaign continue-limit --out <campaign-output-root> --campaign-source-root <source-checkout> --expected-checkpoint-revision <revision> --limit-case <case-id> --limit-treatment <treatment> --limit-epoch <epoch-id> --authorize-additional-invocation"
+];
 // ---------------------------------------------------------------------------
 // v0.4.6 Batch 4 -- reusable experiment-describe command owner.
 //
@@ -72,6 +97,8 @@ function describeAgentSuccessRate() {
         realAgentMode: "Opt-in only: --agent <codex|claude> together with --include-real-agents. Neither flag alone enables it; real execution is never the default.",
         providers: [...AGENT_SUCCESS_RATE_REAL_AGENT_IDS],
         timeoutMs: { default: AGENT_SUCCESS_RATE_DEFAULT_AGENT_TIMEOUT_MS, maximum: AGENT_SUCCESS_RATE_MAX_AGENT_TIMEOUT_MS, scope: "each provider attempt" },
+        timeoutPresets: AGENT_TIMEOUT_PRESETS,
+        campaign: { modes: ["new", "resume", "continue-limit"], options: [...AGENT_CAMPAIGN_OPTION_HELP], examples: [...AGENT_CAMPAIGN_EXAMPLES] },
         repairAttempts: { allowed: [...AGENT_SUCCESS_RATE_REPAIR_ATTEMPT_VALUES], default: 0, maximumTotalAttemptsPerTreatment: 3, mode: "real-agent only" },
         agentContext: "Source-only: the provider sees the public task text and the treatment's source context, never tests, trusted checks or reference patches, and runs in a neutral working directory.",
         patchApplication: "Guarded: the provider returns a unified diff only. The Lab validates it against a patch policy and applies it in a fresh disposable sandbox; canonical benchmark projects are never modified.",
@@ -115,6 +142,8 @@ function printDescription(plugin) {
         console.log(`  Real-agent mode: ${agentSuccessRate.realAgentMode}`);
         console.log(`  Providers: ${agentSuccessRate.providers.join(", ")}`);
         console.log(`  Timeout per attempt: default ${agentSuccessRate.timeoutMs.default} ms, maximum ${agentSuccessRate.timeoutMs.maximum} ms`);
+        for (const option of agentSuccessRate.campaign.options)
+            console.log(`  ${option}`);
         console.log(`  Repair attempts: ${agentSuccessRate.repairAttempts.allowed.join(", ")} (default ${agentSuccessRate.repairAttempts.default}; at most ${agentSuccessRate.repairAttempts.maximumTotalAttemptsPerTreatment} attempts per treatment; real-agent mode only)`);
         console.log(`  Agent context: ${agentSuccessRate.agentContext}`);
         console.log(`  Patch application: ${agentSuccessRate.patchApplication}`);
@@ -212,6 +241,8 @@ function describeExamples(plugin) {
         `my-dev-kit-lab experiment describe --experiment ${plugin.metadata.id}`,
         `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id}${agentOptions}`
     ];
+    if (plugin.metadata.id === AGENT_SUCCESS_RATE_ID)
+        examples.push(...AGENT_CAMPAIGN_EXAMPLES);
     // Only advertise an explicit --target example for plugins that support external targets.
     if (plugin.metadata.supportedTargets.includes("external-local") &&
         plugin.metadata.id !== "context-window-scaling" &&