@dailephd/my-dev-kit-lab 0.8.1 → 0.8.2

package/CHANGELOG.md

@@ -4,7 +4,16 @@ All notable changes to my-dev-kit-lab are documented here.
 
 ## [Unreleased]
 
-## [0.8.1] - 2026-10-05
+## [0.8.2] - 2026-10-07
+
+### Context-pack generation experiments
+
+- Added the experimental `context-pack-generation` plugin, which always compares the fixed `raw-full-file` and `context-pack` treatments using reproducible, bounded task-specific context packs.
+- Reused retrieval-quality coverage metrics and estimated context-size semantics, with paired token savings, percent savings, and objective deltas; reports include bounded JSON, text, and HTML pack previews without a composite winner or ranking.
+- Added external-local execution that calculates science on real identities before privacy projection, persists redacted reports, and writes no durable context-pack body.
+- Extended installed-package acceptance and verified compatibility with the published `@dailephd/my-dev-kit@1.12.5` package.
+
+## [0.8.1] - 2026-10-06
 
 ### Retrieval query strategy comparison
 

package/README.md

@@ -6,9 +6,9 @@ my-dev-kit provides local repository indexing and graph-guided retrieval. my-dev
 
 Whole-ecosystem workflows are centralized in [my-dev-kit/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md). Its [command-surface compatibility map](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md#915-command-surface-compatibility-map) documents which Lab outputs can safely seed my-dev-kit investigation, Observer reference work, or Orchestrator evidence, and which look-alike artifacts are not direct inputs.
 
-The latest release is v0.8.1 (retrieval query strategy comparison), and the published package and installed CLI are version 0.8.1. The previous release is v0.8.0 (retrieval precision/recall). The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
+The latest published release is v0.8.2 (context-pack generation experiments), and the repository package version is 0.8.2. The previous published release is v0.8.1 (retrieval query strategy comparison). The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
 
-v0.8.2 (context-pack generation experiments) is the next planned milestone.
+The next planned milestone after v0.8.2 is v0.9.0 (agent-success-rate experiments).
 
 v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
 
@@ -24,6 +24,7 @@ v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--vers
 - **Run context-window scaling over a local Git repository (v0.7.2, released):** `context-window-scaling` can also run against an explicitly selected local Git repository with `--target <repository> --local-subject-config <path>` (a `LocalRepositorySubjectConfigV1` JSON file). The Lab treats the repository as read-only and compares its state before and after the run, builds contexts only from files Git does not ignore (ignored, oversized (over 1 MiB), symbolic-link, and other unsafe entries are excluded), requires the output directory to be outside the repository, and keeps the repository path, source text, and file names out of durable artifacts (file identities appear as numbered `<redacted file N>` placeholders). A `local-repository-subject-manifest.json` records the logical subject ID, the full Git commit, the branch, and aggregate size metadata. This measures operational context fit only, not retrieval precision, recall, or ranking quality. It is part of the current package. See [docs/COMMANDS.md](docs/COMMANDS.md) and [docs/WORKFLOWS.md](docs/WORKFLOWS.md).
 - **Measure retrieval precision and recall (v0.8.0, released):** the `retrieval-precision-recall` plugin runs the existing my-dev-kit search, lookup, slice, and source lifecycle for each case and compares the retrieved files and symbols with a deterministic answer key, without invoking any agent. It reports file precision and recall, symbol precision and recall, fact coverage (from explicit fact-to-context mappings), irrelevant context ratio, the estimated retrieved token count, and the missed and irrelevant identities, as `retrieval-precision-recall-execution.json` plus `report.json`, `report.html`, and `report.txt`. By default it runs a bundled 12-case corpus over two benchmark projects against the Lab itself. It can also run over an explicitly selected local Git repository with `--target <repository> --local-subject-config <path>`; in that mode the repository is treated as read-only and durable outputs redact the repository path and file, symbol, and fact identities while keeping the numeric evidence. These are set-based measures with no ranking, winner, or composite score. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md).
 - **Compare retrieval query strategies (v0.8.1, released):** the `retrieval-query-strategy-comparison` plugin compares seven fixed my-dev-kit retrieval strategies without running any coding agent: `keyword-search`, `symbol-lookup`, `graph-neighborhood`, `source-slice`, `data-model-graph`, `model-view-lineage`, and `combined-graph-guided`. In bundled mode it uses the same frozen 12-case warm-index benchmark corpus as `retrieval-precision-recall`, and every selected case receives all seven strategies (filters narrow cases or projects, never strategies). File and symbol quality keeps the inherited v0.8.0 set-based semantics and adds balanced file and symbol F1 (`2TP / (2TP + FP + FN)`); fact coverage and the estimated retrieved token count complete the four comparison objectives. Strategies are compared over matched complete cases only, as macro means for four scopes (`overall`, `localized`, `cross-module`, `broad-change`, reusing each case's existing task-locality label), using Pareto dominance on mean file F1, mean symbol F1, mean fact coverage (all maximized) and mean retrieved tokens (minimized). A best strategy is named only when the Pareto front has a single member; when several strategies remain nondominated the report states a tradeoff. There is no scalar or composite score and no ordinal ranking. The plugin can also run over an explicitly selected local Git repository through the existing `LocalRepositorySubject` boundary with `--target <repository> --local-subject-config <path>`; the repository stays read-only and durable output redacts private file, symbol, fact, semantic-node, warning, and case-title identities while preserving the numeric results. It writes `retrieval-query-strategy-comparison-execution.json`, `retrieval-query-strategy-comparison-analysis.json`, and `report.json`, `report.html`, and `report.txt`. No context-pack behavior is included. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md#retrieval-query-strategy-comparison-metrics-v081).
+- **Generate auditable task context packs (v0.8.2):** `context-pack-generation` runs both fixed treatments, `raw-full-file` and `context-pack`, for each selected case. A pack contains a task summary, relevant files and symbols, bounded source slices, call relationships, tests, and evidence notes. Analysis reuses retrieval fact coverage and file/symbol precision, recall, and F1, reports estimated context size and paired raw-versus-pack tokens saved, percent saved, and objective deltas, and names no composite winner or ranking. Bundled runs persist pack artifacts and reports include a bounded preview. External-local runs calculate science on real identities, redact before persistence, omit durable pack bodies, and show redacted report previews. See [docs/COMMANDS.md](docs/COMMANDS.md), [docs/WORKFLOWS.md](docs/WORKFLOWS.md), and [docs/METRICS.md](docs/METRICS.md).
 - **Assess affected-neighborhood evidence (v0.6.1, released; retained in the current package):** using the baseline my-dev-kit graph of the same prepared warm index (loaded once per session), each task's confirmed modified or missing indexed files, and the baseline symbols those files contain, are mapped to graph nodes. The affected neighborhood is those seed nodes plus their direct one-hop neighbors (edges of any kind, either direction). The task's `expectedFiles` and file-scoped `expectedSymbols` are mapped to graph nodes and compared with that neighborhood, which yields six numeric warm-side metrics (`changedFileCount`, `changedSymbolCount`, `affectedNodeCount`, `affectedEdgeCount`, `taskOverlapCount`, `taskOverlapPercent`) plus categorical evidence: a `related`, `unrelated`, or `unknown` relationship and a `recommended`, `not-indicated`, or `unknown` `reindexRecommendation`. Incomplete evidence is reported as `unknown`, never as unrelated. The assessment is recorded additively in `warm-index-execution.json` and shown in the existing `report.json`/`report.txt`/`report.html` warm-index reports. It is observational only: retrieval, execution status, and provider status are unchanged, nothing is reindexed automatically, and no CLI flag, experiment plugin, plot, or gallery item was added. See [docs/METRICS.md](docs/METRICS.md#affected-neighborhood-metrics-v061) and [docs/WORKFLOWS.md](docs/WORKFLOWS.md#warm-index-reuse-experiment).
 - **Audit repository health:** run conservative code-rot detectors for TypeScript/JavaScript, Python, Java, and Kotlin, or adapt security findings into the common audit report.
 - **Validate CLI/package security:** inspect dependencies, package contents, path and subprocess boundaries, malformed inputs, optional static scanners, and bounded fuzz targets.
@@ -243,9 +244,9 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
 - Claude and Codex token totals are used only when the CLI/run exposes them; missing provider token evidence stays explicitly unavailable (never zero, and never substituted with an estimated context-token count)
 - Codex and Claude campaign runs can produce timeouts, invalid-output, agent-unavailable, or agent-limit-reached outcomes; these are structured provider states, not warm-index infrastructure failures
 - Small projects may make raw-full-file cheaper than my-dev-kit-guided; larger localized tasks are where my-dev-kit is expected to become more useful
-- The generic experiment-plugin framework has six registered plugins: `context-strategy-comparison`, `warm-index-reuse`, `incremental-change-staleness`, `context-window-scaling`, `retrieval-precision-recall`, and `retrieval-query-strategy-comparison`. The v0.7.0 context-window-scaling plugin is released; v0.7.1 lets it also run over caller-supplied synthetic repositories; v0.7.2 lets it run over an explicitly selected local Git repository. The fifth, `retrieval-precision-recall`, was added in v0.8.0. The sixth, `retrieval-query-strategy-comparison`, was added in v0.8.1; agent-success plugins remain planned.
+- The generic experiment-plugin framework has seven registered plugins: `context-strategy-comparison`, `warm-index-reuse`, `incremental-change-staleness`, `context-window-scaling`, `retrieval-precision-recall`, `retrieval-query-strategy-comparison`, and `context-pack-generation`. The v0.7.0 context-window-scaling plugin is released; v0.7.1 added caller-supplied synthetic repositories; v0.7.2 added an explicitly selected local Git repository. The context-pack-generation plugin shipped in v0.8.2; agent-success plugins remain planned.
 - `retrieval-precision-recall` (v0.8.0) measures set-based retrieval quality, not ranked retrieval: it has no ranking metrics, no retrieval-strategy comparison of its own (the separate `retrieval-query-strategy-comparison` plugin added in v0.8.1 compares strategies), no winner or composite score, and no real agents. Its lifecycle expands only the top search candidate. An external-local run fails as a whole if any case fails a safety or execution check, and then writes no normal artifact or report family. Durable external-local identities are redacted, although the console summary of a successful run can still show the physical output directory. Platform path-length limits can make a run with very long paths fail cleanly instead of executing.
-- `retrieval-query-strategy-comparison` (v0.8.1) runs no coding agent and does not evaluate answer quality. Its retrieved token count is a context-size estimate, not provider billing telemetry. Semantic strategies (`data-model-graph`, `model-view-lineage`) can truthfully be partial or unavailable for a case, and a case is then excluded from the matched cross-strategy comparison for every strategy while its per-strategy evidence is kept. The Pareto front is a set of nondominated strategies in canonical strategy order, not an ordinal ranking. The plugin reports no significance test or confidence interval, selects no strategy automatically, and generates no context pack. External-local mode rejects `--case` and `--benchmark-project`, builds one private base index per configured case, and fails as a whole, writing no normal artifact or report family, if any safety or privacy check fails.
+- `retrieval-query-strategy-comparison` (v0.8.1) runs no coding agent and does not evaluate answer quality. Its retrieved token count is a context-size estimate, not provider billing telemetry. Semantic strategies (`data-model-graph`, `model-view-lineage`) can truthfully be partial or unavailable for a case, and a case is then excluded from the matched cross-strategy comparison for every strategy while its per-strategy evidence is kept. The Pareto front is a set of nondominated strategies in canonical strategy order, not an ordinal ranking. The plugin reports no significance test or confidence interval and selects no strategy automatically. External-local mode rejects `--case` and `--benchmark-project`, builds one private base index per configured case, and fails as a whole, writing no normal artifact or report family, if any safety or privacy check fails.
 - `warm-index-reuse` without `--campaign` still uses the deterministic fake agent only. With `--campaign`, exactly one real provider (Codex or Claude) runs per campaign preset — there is no multi-agent matrix, retry, provider switching, or scheduler. The v0.5.1 expanded corpus is deterministic fake-agent benchmark evidence when run without `--campaign`. No warm-index mode calculates a token-savings percentage, break-even task, winner, or ranking, and component-duration sums are not wall-clock latency
 - The current release does not guarantee token savings; it produces auditable evidence for specific cases, targets, agents, and strategies
 - Context-window scaling uses estimated tokens (`ceil(characters / 4)`) and experiment budgets, not actual provider tokenizer or model-window limits. Its deterministic fake-agent correctness check does not measure semantic sensitivity to the supplied context.

package/dist/src/cli/help.js

@@ -156,7 +156,7 @@ export function renderExperimentRunHelp() {
         "  --case <ids>                                       Comma-separated case ids to run",
         "  --benchmark-project <ids>                          Comma-separated benchmark project ids to run",
         "",
-        "my-dev-kit command override (warm-index-reuse, incremental-change-staleness, context-window-scaling, retrieval-precision-recall, and retrieval-query-strategy-comparison):",
+        "my-dev-kit command override (warm-index-reuse, incremental-change-staleness, context-window-scaling, retrieval-precision-recall, retrieval-query-strategy-comparison, and context-pack-generation):",
         "  --kit-command <command>                            my-dev-kit command used by the selected experiment plugin",
         "                                                      (defaults: warm-index-reuse and context-window-scaling use",
         "                                                      npx @dailephd/my-dev-kit@latest;",
@@ -219,6 +219,21 @@ export function renderExperimentRunHelp() {
         "                                                      strategies are reported as a tradeoff. No scalar score or total",
         "                                                      ranking exists.",
         "",
+        "context-pack-generation only:",
+        "                                                      Bundled/self mode runs the frozen 12-case corpus and accepts only",
+        "                                                      --out, --case, --benchmark-project and --kit-command (default",
+        "                                                      npx @dailephd/my-dev-kit@latest); --case and --benchmark-project",
+        "                                                      narrow the case set. Both treatments, raw-full-file and",
+        "                                                      context-pack, always run for every case; there is no treatment,",
+        "                                                      strategy, or selection-policy option. External-local mode requires",
+        "                                                      --target together with --local-subject-config and accepts only",
+        "                                                      --out, --target, --local-subject-config and --kit-command; --case",
+        "                                                      and --benchmark-project are rejected because the local-subject",
+        "                                                      config owns the case set. The repository is never modified and",
+        "                                                      durable output withholds private file, symbol, fact, semantic-node,",
+        "                                                      call-relationship, test, source-text, warning, task and case-title",
+        "                                                      identities; no context-pack body is written for an external run.",
+        "",
         "warm-index-reuse only:",
         "                                                      Legacy/default mode uses the deterministic fake agent only;",
         "                                                      agent options below are not accepted by this plugin.",

package/dist/src/commands/runExperimentDescribeCommand.js

@@ -139,6 +139,9 @@ function describeTargetBehavior(plugin) {
     if (plugin.metadata.id === "retrieval-query-strategy-comparison") {
         return "Two subject modes. Bundled (default): the frozen 12-case warm-index corpus compares all seven retrieval query strategies and may be narrowed with --case and --benchmark-project. External local repository: an explicitly selected local Git worktree via --target <path> together with --local-subject-config <path>; the local-subject config owns the case set, one private base index is built per configured case, core strategies share that base, semantic strategies use isolated copies, the repository is never modified, and durable output withholds private file, symbol, fact, semantic-node, warning and case-title identities while preserving numeric scientific results. Bundled filters are not accepted in external-local mode.";
     }
+    if (plugin.metadata.id === "context-pack-generation") {
+        return "Two subject modes. Bundled (default): the frozen 12-case warm-index corpus against the Lab itself, narrowable with --case and --benchmark-project. External local repository: an explicitly selected local Git worktree via --target <path> together with --local-subject-config <path>; the local-subject config owns the case set, the repository is never modified, the output directory must be outside it, one private base index is built per configured case, and durable output withholds private file, symbol, fact, semantic-node, call-relationship, test, source-text, warning, task and case-title identities while preserving numeric scientific results; no context-pack body is written externally. Bundled filters are not accepted in external-local mode. Both treatments (raw-full-file and context-pack) always run for every case; there is no treatment, strategy, or selection-policy option.";
+    }
     if (supportsExternal) {
         return "Runs against the current lab repository when --target is omitted, or against an explicit local target project when --target <path> is provided. Outputs stay under lab-controlled output directories by default.";
     }
@@ -165,7 +168,8 @@ function describeExamples(plugin) {
     if (plugin.metadata.supportedTargets.includes("external-local") &&
         plugin.metadata.id !== "context-window-scaling" &&
         plugin.metadata.id !== "retrieval-precision-recall" &&
-        plugin.metadata.id !== "retrieval-query-strategy-comparison") {
+        plugin.metadata.id !== "retrieval-query-strategy-comparison" &&
+        plugin.metadata.id !== "context-pack-generation") {
         examples.push(`my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target "Z:\\Users\\newuser\\Projects\\my-dev-kit-v1"${agentOptions}${screenshotOption}`);
     }
     // Only advertise campaign examples for plugins that declare the campaignPreset config field.
@@ -178,6 +182,9 @@ function describeExamples(plugin) {
     if (plugin.metadata.id === "retrieval-query-strategy-comparison") {
         examples.push(`my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --case <case-id> --out <run-dir>`, `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target <local-git-repository> --local-subject-config <path-to-local-subject-config.json> --out <run-dir-outside-the-repository>`);
     }
+    if (plugin.metadata.id === "context-pack-generation") {
+        examples.push(`my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --case <case-id> --out <run-dir>`, `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target <local-git-repository> --local-subject-config <path-to-local-subject-config.json> --out <run-dir-outside-the-repository>`);
+    }
     if (plugin.metadata.id === "context-window-scaling") {
         examples.push(`my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --synthetic-config <path-to-config.json> --out <run-dir>`, `my-dev-kit-lab experiment run --experiment ${plugin.metadata.id} --target <local-git-repository> --local-subject-config <path-to-local-subject-config.json> --out <run-dir-outside-the-repository>`);
     }

package/dist/src/commands/runExperimentRunCommand.js

@@ -6,6 +6,7 @@ import { readBenchmarkProjectProfiles, readEvaluationCases } from "../evaluation
 import { summarizeRetrievalGroundTruthIssuesSafely, validateRetrievalPrecisionRecallCorpus } from "../evaluation/retrievalQuality/index.js";
 import { loadLocalRepositorySubject } from "../evaluation/localRepositorySubject/index.js";
 import { contextStrategyComparisonPlugin, contextWindowScalingPlugin, createDefaultExperimentPluginRegistry, getWarmIndexCampaignPreset, incrementalChangeStalenessPlugin, parseContextBudgetsCliValue, parseWarmIndexCampaignPresetId, resolveExperimentTarget, retrievalPrecisionRecallPlugin, retrievalQueryStrategyComparisonPlugin, runExperiment, warmIndexReusePlugin } from "../experiments/index.js";
+import { contextPackGenerationPlugin } from "../experiments/plugins/contextPackGeneration/index.js";
 import { prepareSyntheticContextWindowScalingInputs } from "../experiments/plugins/contextWindowScaling/index.js";
 import { projectExternalLocalTarget, projectRunForExternalLocalPersistence, } from "../experiments/plugins/contextWindowScaling/localSubjectPrivacy.js";
 import { assertWorkRootOutsideTarget } from "../experiments/plugins/contextWindowScaling/localSubjectScratch.js";
@@ -64,6 +65,12 @@ const RETRIEVAL_PRECISION_RECALL_EXTERNAL_ALLOWED_FLAGS = ["--experiment", "--ou
 const RETRIEVAL_QUERY_STRATEGY_COMPARISON_CASES_RESOURCE = "benchmarks/contracts/warm-index-benchmark-cases.json";
 const RETRIEVAL_QUERY_STRATEGY_COMPARISON_ALLOWED_FLAGS = ["--experiment", "--out", "--case", "--benchmark-project", "--kit-command"];
 const RETRIEVAL_QUERY_STRATEGY_COMPARISON_EXTERNAL_ALLOWED_FLAGS = ["--experiment", "--out", "--target", "--local-subject-config", "--kit-command"];
+// context-pack-generation (v0.8.2) reuses the same frozen bundled corpus. Both treatments (raw-full-file and context-pack)
+// always run, so there is no treatment or strategy option. External-local mode (--target with --local-subject-config) is a
+// different subject source: the local-subject config owns the case set, so the bundled corpus filters are not accepted there.
+const CONTEXT_PACK_GENERATION_CASES_RESOURCE = "benchmarks/contracts/warm-index-benchmark-cases.json";
+const CONTEXT_PACK_GENERATION_ALLOWED_FLAGS = ["--experiment", "--out", "--case", "--benchmark-project", "--kit-command"];
+const CONTEXT_PACK_GENERATION_EXTERNAL_ALLOWED_FLAGS = ["--experiment", "--out", "--target", "--local-subject-config", "--kit-command"];
 export async function runExperimentRunCommandFromArgs(argv, options = {}) {
     try {
         const args = parseRunExperimentArgs(argv);
@@ -315,7 +322,8 @@ export function parseRunExperimentArgs(argv) {
         incrementalChangeStalenessPlugin.metadata.id,
         contextWindowScalingPlugin.metadata.id,
         retrievalPrecisionRecallPlugin.metadata.id,
-        retrievalQueryStrategyComparisonPlugin.metadata.id
+        retrievalQueryStrategyComparisonPlugin.metadata.id,
+        contextPackGenerationPlugin.metadata.id
     ];
     if (experimentId === retrievalPrecisionRecallPlugin.metadata.id) {
         // Mode matrix, no precedence rules: neither flag = bundled; both = external-local; exactly one is rejected.
@@ -367,6 +375,30 @@ export function parseRunExperimentArgs(argv) {
             throw new Error("--benchmark-project must list at least one benchmark project id.");
         }
     }
+    if (experimentId === contextPackGenerationPlugin.metadata.id) {
+        // Mode matrix, no precedence rules: neither flag = bundled; both = external-local; exactly one is rejected.
+        if (targetPath !== undefined && localSubjectConfigPath === undefined) {
+            throw new Error(`External ${experimentId} targets require --local-subject-config.`);
+        }
+        if (localSubjectConfigPath !== undefined && targetPath === undefined) {
+            throw new Error(`--local-subject-config requires an external --target for ${experimentId}.`);
+        }
+        const externalMode = targetPath !== undefined;
+        if (externalMode && (seenFlags.includes("--case") || seenFlags.includes("--benchmark-project"))) {
+            throw new Error("--case and --benchmark-project cannot be combined with --local-subject-config; the local subject config owns the case set.");
+        }
+        const allowedFlags = externalMode ? CONTEXT_PACK_GENERATION_EXTERNAL_ALLOWED_FLAGS : CONTEXT_PACK_GENERATION_ALLOWED_FLAGS;
+        const unsupported = [...new Set(seenFlags.filter((flag) => !allowedFlags.includes(flag)))];
+        if (unsupported.length > 0) {
+            throw new Error(`${unsupported.join(", ")} ${unsupported.length === 1 ? "is" : "are"} not supported for --experiment ${experimentId}${externalMode ? " in external-local mode" : ""}; supported options: ${allowedFlags.join(", ")}.`);
+        }
+        if (seenFlags.includes("--case") && caseIds.length === 0) {
+            throw new Error("--case must list at least one case id.");
+        }
+        if (seenFlags.includes("--benchmark-project") && benchmarkProjects.length === 0) {
+            throw new Error("--benchmark-project must list at least one benchmark project id.");
+        }
+    }
     if (contextBudgets !== undefined && experimentId !== contextWindowScalingPlugin.metadata.id) {
         throw new Error(`--context-budgets is only supported for --experiment ${contextWindowScalingPlugin.metadata.id}.`);
     }
@@ -376,8 +408,9 @@ export function parseRunExperimentArgs(argv) {
     if (localSubjectConfigPath !== undefined &&
         experimentId !== contextWindowScalingPlugin.metadata.id &&
         experimentId !== retrievalPrecisionRecallPlugin.metadata.id &&
-        experimentId !== retrievalQueryStrategyComparisonPlugin.metadata.id) {
-        throw new Error(`--local-subject-config is only supported for --experiment ${contextWindowScalingPlugin.metadata.id}, ${retrievalPrecisionRecallPlugin.metadata.id}, or ${retrievalQueryStrategyComparisonPlugin.metadata.id}.`);
+        experimentId !== retrievalQueryStrategyComparisonPlugin.metadata.id &&
+        experimentId !== contextPackGenerationPlugin.metadata.id) {
+        throw new Error(`--local-subject-config is only supported for --experiment ${contextWindowScalingPlugin.metadata.id}, ${retrievalPrecisionRecallPlugin.metadata.id}, ${retrievalQueryStrategyComparisonPlugin.metadata.id}, or ${contextPackGenerationPlugin.metadata.id}.`);
     }
     if (experimentId === contextWindowScalingPlugin.metadata.id) {
         const unsupported = [...new Set(seenFlags.filter((flag) => !CONTEXT_WINDOW_SCALING_ALLOWED_FLAGS.includes(flag)))];
@@ -499,6 +532,11 @@ async function loadPluginInputs(args, toolRoot, context, outputRoot) {
             ? loadRetrievalPrecisionRecallLocalSubjectInputs(args, toolRoot, context, outputRoot)
             : loadRetrievalPrecisionRecallInputs(args, toolRoot, context);
     }
+    if (args.experimentId === contextPackGenerationPlugin.metadata.id) {
+        return args.localSubjectConfigPath !== undefined
+            ? loadContextPackGenerationLocalSubjectInputs(args, toolRoot, context, outputRoot)
+            : loadContextPackGenerationInputs(args, toolRoot, context);
+    }
     if (args.experimentId === retrievalQueryStrategyComparisonPlugin.metadata.id) {
         return args.localSubjectConfigPath !== undefined
             ? loadRetrievalQueryStrategyComparisonLocalSubjectInputs(args, toolRoot, context, outputRoot)
@@ -506,6 +544,55 @@ async function loadPluginInputs(args, toolRoot, context, outputRoot) {
     }
     return undefined;
 }
+// context-pack-generation (bundled/self): the frozen warm-index corpus through the established resource resolver, profile
+// and corpus validation. The plugin applies --case/--benchmark-project selection and owns execution, science and artifacts.
+async function loadContextPackGenerationInputs(args, toolRoot, context) {
+    const validation = contextPackGenerationPlugin.validateConfig(args.config);
+    if (!validation.valid || !validation.config) {
+        throw new Error(`Invalid context pack generation config: ${validation.errors.join("; ")}`);
+    }
+    const projectProfiles = await readBenchmarkProjectProfiles(resolvePackageResource(context, DEFAULT_PROJECT_PROFILES_RESOURCE), toolRoot);
+    const cases = await readEvaluationCases(resolvePackageResource(context, CONTEXT_PACK_GENERATION_CASES_RESOURCE), toolRoot, {
+        projectProfiles,
+        requireProjectProfileRef: true
+    });
+    const corpusErrors = validateRetrievalPrecisionRecallCorpus(cases);
+    if (corpusErrors.length > 0) {
+        throw new Error(`Invalid context-pack-generation corpus: ${corpusErrors.join(" ")}`);
+    }
+    return { cases };
+}
+// External-local context-pack-generation: same loader and safety order as the other external-local retrieval experiments,
+// with the plugin's own config validation. Ground-truth issues carry logical case ids and counts only.
+async function loadContextPackGenerationLocalSubjectInputs(args, toolRoot, context, outputRoot) {
+    const validation = contextPackGenerationPlugin.validateConfig(args.config);
+    if (!validation.valid || !validation.config) {
+        throw new Error(`Invalid context pack generation config: ${validation.errors.join("; ")}`);
+    }
+    if (outputRoot === undefined || args.localSubjectConfigPath === undefined) {
+        throw new Error("Internal error: external-local runs require a resolved experiment output root and config path.");
+    }
+    const target = resolveExperimentTarget(args.targetPath, toolRoot);
+    if (target.isSelf || target.kind !== "external-local") {
+        throw new Error("Local-repository subject mode requires an external-local target; --target must not be the Lab repository.");
+    }
+    try {
+        await assertWorkRootOutsideTarget(outputRoot, target.targetRoot);
+    }
+    catch {
+        throw new Error("Experiment output root must not be inside the external target project.");
+    }
+    const configPath = path.resolve(context.invocationCwd, args.localSubjectConfigPath);
+    const localSubjectConfig = await readJsonConfigFile("--local-subject-config", configPath, args.localSubjectConfigPath);
+    const localSubject = await loadLocalRepositorySubject({ config: localSubjectConfig, repositoryPath: target.targetRoot });
+    const groundTruthIssues = summarizeRetrievalGroundTruthIssuesSafely(localSubject.evaluationCases);
+    if (groundTruthIssues.length > 0) {
+        throw new Error(`Invalid context-pack-generation ground truth for the local subject (${groundTruthIssues
+            .map((issue) => `case ${issue.caseId}: ${issue.issueCount} issue${issue.issueCount === 1 ? "" : "s"}`)
+            .join("; ")}); details withheld.`);
+    }
+    return { cases: localSubject.evaluationCases, localSubject, env: process.env };
+}
 // External-local retrieval-query-strategy-comparison: same loader and safety order as retrieval-precision-recall, with
 // the plugin's own config validation. Ground-truth issues carry logical case ids and counts only.
 async function loadRetrievalQueryStrategyComparisonLocalSubjectInputs(args, toolRoot, context, outputRoot) {

package/dist/src/evaluation/runMyDevKitRetrieval.js

@@ -183,6 +183,7 @@ export async function buildMyDevKitIndex(options) {
             ...(options.excludePaths ?? []).flatMap((excludePath) => ["--exclude", excludePath]),
             "--out",
             options.indexDir,
+            ...(options.callGraph === true ? ["--call-graph"] : []),
             ...(mode.kind === "incremental" ? ["--incremental", "--refresh-scope", mode.refreshScope] : []),
             "--json"
         ]

package/dist/src/experiments/defaultRegistry.js

@@ -1,5 +1,6 @@
 import { ExperimentPluginRegistry } from "./registry.js";
 import { contextStrategyComparisonPlugin } from "./plugins/contextStrategyComparison/index.js";
+import { contextPackGenerationPlugin } from "./plugins/contextPackGeneration/index.js";
 import { contextWindowScalingPlugin } from "./plugins/contextWindowScaling/index.js";
 import { incrementalChangeStalenessPlugin } from "./plugins/incrementalChangeStaleness/index.js";
 import { retrievalPrecisionRecallPlugin } from "./plugins/retrievalPrecisionRecall/index.js";
@@ -13,5 +14,6 @@ export function createDefaultExperimentPluginRegistry() {
     registry.register(contextWindowScalingPlugin);
     registry.register(retrievalPrecisionRecallPlugin);
     registry.register(retrievalQueryStrategyComparisonPlugin);
+    registry.register(contextPackGenerationPlugin);
     return registry;
 }

package/dist/src/experiments/plugins/contextPackGeneration/analysis.js

@@ -0,0 +1,157 @@
+import { calculateRetrievalQualityMetricsFromIdentityEvidence } from "../../../evaluation/retrievalQuality/metrics.js";
+import { calculateBalancedIdentityF1 } from "../retrievalQueryStrategyComparison/analysis.js";
+import { RETRIEVAL_QUERY_STRATEGY_COMPARISON_SCOPE_IDS } from "../retrievalQueryStrategyComparison/analysisTypes.js";
+import { CONTEXT_PACK_GENERATION_TREATMENT_IDS } from "./metadata.js";
+const noMeasurementF1 = () => ({
+    availability: "unavailable",
+    numerator: null,
+    denominator: null,
+    value: null,
+    reason: "no-measurement"
+});
+/** Scores already-completed treatment evidence with the frozen retrieval-quality science. It never influences retrieval. */
+export function analyzeContextPackGenerationTreatment(evaluationCase, treatment) {
+    if (treatment.identityEvidence === null || treatment.size === null) {
+        return {
+            treatmentId: treatment.treatmentId,
+            executionStatus: treatment.status,
+            availability: treatment.availability,
+            quality: null,
+            fileF1: noMeasurementF1(),
+            symbolF1: noMeasurementF1(),
+            estimatedTokens: null,
+            tokenCountMethod: null
+        };
+    }
+    const quality = calculateRetrievalQualityMetricsFromIdentityEvidence({
+        evaluationCase,
+        retrieval: {
+            identityEvidence: treatment.identityEvidence,
+            totalEstimatedTokens: treatment.size.totalEstimatedTokens,
+            tokenCountMethod: treatment.size.tokenCountMethod
+        }
+    });
+    return {
+        treatmentId: treatment.treatmentId,
+        executionStatus: treatment.status,
+        availability: treatment.availability,
+        quality,
+        fileF1: calculateBalancedIdentityF1(quality.file.relevantRetrievedFiles, quality.file.irrelevantRetrievedFiles, quality.file.missedFiles),
+        symbolF1: calculateBalancedIdentityF1(quality.symbol.relevantRetrievedSymbols, quality.symbol.irrelevantRetrievedSymbols, quality.symbol.missedSymbols),
+        estimatedTokens: treatment.size.totalEstimatedTokens,
+        tokenCountMethod: treatment.size.tokenCountMethod
+    };
+}
+/**
+ * Same arithmetic as the existing repository token-savings comparison (tokensSaved = raw - other; percentSaved =
+ * tokensSaved / raw * 100, 0 when raw is 0). Negative savings are valid and never clamped.
+ */
+export function calculateContextPackTokenSavings(rawTokens, packTokens) {
+    const tokensSaved = rawTokens - packTokens;
+    return { tokensSaved, percentSaved: rawTokens === 0 ? 0 : (tokensSaved / rawTokens) * 100 };
+}
+const valueOf = (metric) => (metric && metric.availability === "available" ? metric.value : null);
+function compareTreatments(raw, pack) {
+    const delta = (packValue, rawValue) => (packValue === null || rawValue === null ? null : packValue - rawValue);
+    const factPack = valueOf(pack.quality?.fact.coverage);
+    const factRaw = valueOf(raw.quality?.fact.coverage);
+    const sizes = raw.estimatedTokens !== null && pack.estimatedTokens !== null ? calculateContextPackTokenSavings(raw.estimatedTokens, pack.estimatedTokens) : null;
+    return {
+        fileF1Delta: delta(valueOf(pack.fileF1), valueOf(raw.fileF1)),
+        symbolF1Delta: delta(valueOf(pack.symbolF1), valueOf(raw.symbolF1)),
+        factCoverageDelta: delta(factPack, factRaw),
+        estimatedTokenDelta: delta(pack.estimatedTokens, raw.estimatedTokens),
+        tokensSaved: sizes?.tokensSaved ?? null,
+        percentSaved: sizes?.percentSaved ?? null
+    };
+}
+export function analyzeContextPackGenerationCase(evaluationCase, executionCase) {
+    if (evaluationCase.id !== executionCase.caseId)
+        throw new Error("Context pack generation analysis case identity mismatch.");
+    const order = executionCase.treatments.map((treatment) => treatment.treatmentId);
+    if (order.length !== CONTEXT_PACK_GENERATION_TREATMENT_IDS.length || order.some((id, index) => id !== CONTEXT_PACK_GENERATION_TREATMENT_IDS[index])) {
+        throw new Error("Context pack generation analysis requires the two canonical treatments in canonical order.");
+    }
+    const treatments = executionCase.treatments.map((treatment) => analyzeContextPackGenerationTreatment(evaluationCase, treatment));
+    return {
+        caseId: executionCase.caseId,
+        benchmarkProject: executionCase.benchmarkProject,
+        taskLocality: executionCase.taskLocality,
+        treatments,
+        comparison: compareTreatments(treatments[0], treatments[1])
+    };
+}
+function isComparable(treatment) {
+    const quality = treatment.quality;
+    return (quality !== null &&
+        treatment.fileF1.availability === "available" &&
+        treatment.symbolF1.availability === "available" &&
+        quality.fact.coverage.availability === "available" &&
+        typeof quality.retrievedTokenCount === "number" &&
+        Number.isFinite(quality.retrievedTokenCount) &&
+        quality.retrievedTokenCount >= 0);
+}
+/** Matched design: a case counts only when BOTH treatments expose all four required measures. */
+export function isContextPackCaseComparable(caseAnalysis) {
+    return caseAnalysis.treatments.length === CONTEXT_PACK_GENERATION_TREATMENT_IDS.length && caseAnalysis.treatments.every(isComparable);
+}
+const mean = (values) => values.reduce((sum, value) => sum + value, 0) / values.length;
+function casesForScope(cases, scopeId) {
+    return scopeId === "overall" ? [...cases] : cases.filter((entry) => entry.taskLocality === scopeId);
+}
+export function aggregateContextPackGenerationScope(scopeId, cases) {
+    const scopeCases = casesForScope(cases, scopeId);
+    const included = scopeCases.filter(isContextPackCaseComparable);
+    const excluded = scopeCases.filter((entry) => !isContextPackCaseComparable(entry));
+    const treatmentSummaries = CONTEXT_PACK_GENERATION_TREATMENT_IDS.map((treatmentId, index) => {
+        const countStatus = (status) => scopeCases.filter((entry) => entry.treatments[index].executionStatus === status).length;
+        const rows = included.map((entry) => entry.treatments[index]);
+        return {
+            treatmentId,
+            completedCaseCount: countStatus("completed"),
+            partialCaseCount: countStatus("partial"),
+            failedCaseCount: countStatus("failed"),
+            objectives: rows.length === 0
+                ? null
+                : {
+                    meanFileF1: mean(rows.map((row) => row.fileF1.value)),
+                    meanSymbolF1: mean(rows.map((row) => row.symbolF1.value)),
+                    meanFactCoverage: mean(rows.map((row) => row.quality.fact.coverage.value)),
+                    meanEstimatedTokenCount: mean(rows.map((row) => row.quality.retrievedTokenCount))
+                }
+        };
+    });
+    const [rawSummary, packSummary] = treatmentSummaries;
+    const rawObjectives = rawSummary.objectives;
+    const packObjectives = packSummary.objectives;
+    const paired = rawObjectives && packObjectives;
+    return {
+        scopeId,
+        caseCount: scopeCases.length,
+        includedCaseCount: included.length,
+        excludedCaseCount: excluded.length,
+        includedCaseIds: included.map((entry) => entry.caseId),
+        excludedCaseIds: excluded.map((entry) => entry.caseId),
+        treatmentSummaries,
+        pairedDeltas: paired
+            ? {
+                meanFileF1Delta: mean(included.map((entry) => entry.comparison.fileF1Delta)),
+                meanSymbolF1Delta: mean(included.map((entry) => entry.comparison.symbolF1Delta)),
+                meanFactCoverageDelta: mean(included.map((entry) => entry.comparison.factCoverageDelta)),
+                meanEstimatedTokenDelta: mean(included.map((entry) => entry.comparison.estimatedTokenDelta))
+            }
+            : null,
+        tokenSavings: paired
+            ? {
+                meanTokensSaved: mean(included.map((entry) => entry.comparison.tokensSaved)),
+                percentSavedOfMeans: calculateContextPackTokenSavings(rawObjectives.meanEstimatedTokenCount, packObjectives.meanEstimatedTokenCount).percentSaved
+            }
+            : null
+    };
+}
+export function analyzeContextPackGeneration(evaluationCases, executionCases) {
+    if (evaluationCases.length !== executionCases.length)
+        throw new Error("Context pack generation analysis requires one execution case per evaluation case.");
+    const cases = evaluationCases.map((evaluationCase, index) => analyzeContextPackGenerationCase(evaluationCase, executionCases[index]));
+    return { cases, scopes: RETRIEVAL_QUERY_STRATEGY_COMPARISON_SCOPE_IDS.map((scopeId) => aggregateContextPackGenerationScope(scopeId, cases)) };
+}

package/dist/src/experiments/plugins/contextPackGeneration/analysisArtifact.js

@@ -0,0 +1,22 @@
+export const CONTEXT_PACK_GENERATION_ANALYSIS_ARTIFACT_FILE = "context-pack-generation-analysis.json";
+export const CONTEXT_PACK_GENERATION_ANALYSIS_SCHEMA_VERSION = "my-dev-kit-lab-context-pack-generation-analysis-v1";
+/** Single owner of the frozen methodology. No ranking, winner, composite score or Pareto front exists for this experiment. */
+export const CONTEXT_PACK_GENERATION_METHODOLOGY = {
+    fileF1: "balanced-f1",
+    symbolF1: "balanced-f1",
+    aggregation: "matched-complete-case-macro-mean",
+    treatmentComparison: "paired-descriptive-delta",
+    sizeMeasure: "estimated-tokens-of-rendered-text"
+};
+export function buildContextPackGenerationAnalysisArtifact(args) {
+    return {
+        schemaVersion: CONTEXT_PACK_GENERATION_ANALYSIS_SCHEMA_VERSION,
+        runId: args.runId,
+        pluginId: args.pluginId,
+        pluginSchemaVersion: args.pluginSchemaVersion,
+        startedAt: args.startedAt,
+        completedAt: args.completedAt,
+        methodology: { ...CONTEXT_PACK_GENERATION_METHODOLOGY },
+        analysis: structuredClone(args.analysis)
+    };
+}

package/dist/src/experiments/plugins/contextPackGeneration/analysisTypes.js

@@ -0,0 +1 @@
+export {};

package/dist/src/experiments/plugins/contextPackGeneration/buildContextPack.js

@@ -0,0 +1,84 @@
+import { countEstimatedTokens, countTextChars, tokenCountMethod } from "../../../core/countTokens.js";
+import { compareCodeUnits } from "../../../evaluation/indexSnapshot.js";
+import { buildContextPackSelectionPolicy, selectCallRelationships, selectRelevantFiles, selectRelevantSymbols, selectSourceSlices, selectTestFiles } from "./packSelectionPolicy.js";
+import { renderContextPack } from "./renderContextPack.js";
+import { CONTEXT_PACK_SCHEMA_VERSION, CONTEXT_PACK_SECTION_IDS } from "./types.js";
+/**
+ * Pure context-pack composition. No filesystem, process, environment, network, clock or randomness: identical input
+ * yields structurally identical output. It selects within the frozen caps, renders deterministically, and measures size
+ * from the exact rendered text.
+ */
+export function buildContextPack(input) {
+    const files = selectRelevantFiles(input.fileCandidates);
+    const symbols = selectRelevantSymbols(input.symbolCandidates);
+    const sourceSlices = selectSourceSlices(input.sourceSliceCandidates);
+    const calls = selectCallRelationships(input.callCandidates);
+    const tests = selectTestFiles(input.testCandidates);
+    const evidenceNotes = [
+        ...new Set([
+            ...(input.evidenceNotes ?? []),
+            ...files.notes,
+            ...symbols.notes,
+            ...sourceSlices.notes,
+            ...calls.notes,
+            ...tests.notes
+        ])
+    ].sort(compareCodeUnits);
+    const sectionAvailability = {
+        task: { availability: "available", reason: null },
+        files: { ...input.sectionAvailability.files },
+        symbols: { ...input.sectionAvailability.symbols },
+        sourceSlices: { ...input.sectionAvailability.sourceSlices },
+        callRelationships: { ...input.sectionAvailability.callRelationships },
+        tests: { ...input.sectionAvailability.tests },
+        evidenceNotes: { ...input.sectionAvailability.evidenceNotes }
+    };
+    const itemCounts = {
+        task: 1,
+        files: files.items.length,
+        symbols: symbols.items.length,
+        sourceSlices: sourceSlices.items.length,
+        callRelationships: calls.items.length,
+        tests: tests.items.length,
+        evidenceNotes: evidenceNotes.length
+    };
+    const rendered = renderContextPack({
+        task: input.task,
+        files: files.items,
+        symbols: symbols.items,
+        sourceSlices: sourceSlices.items,
+        callRelationships: calls.items,
+        tests: tests.items,
+        evidenceNotes,
+        sectionAvailability
+    });
+    return {
+        schemaVersion: CONTEXT_PACK_SCHEMA_VERSION,
+        caseId: input.caseId,
+        benchmarkProject: input.benchmarkProject,
+        taskLocality: input.taskLocality,
+        task: { title: input.task.title, summary: input.task.summary },
+        selectionPolicy: buildContextPackSelectionPolicy(input.myDevKitVersion),
+        files: files.items,
+        symbols: symbols.items,
+        sourceSlices: sourceSlices.items,
+        callRelationships: calls.items,
+        tests: tests.items,
+        evidenceNotes,
+        sections: CONTEXT_PACK_SECTION_IDS.map((id) => ({
+            id,
+            availability: sectionAvailability[id].availability,
+            reason: sectionAvailability[id].reason,
+            itemCount: itemCounts[id],
+            estimatedTokens: countEstimatedTokens(rendered.sections[id])
+        })),
+        size: {
+            totalChars: countTextChars(rendered.renderedText),
+            totalEstimatedTokens: countEstimatedTokens(rendered.renderedText),
+            tokenCountMethod
+        },
+        availability: input.availability,
+        reason: input.reason,
+        renderedText: rendered.renderedText
+    };
+}

package/dist/src/experiments/plugins/contextPackGeneration/config.js

@@ -0,0 +1,39 @@
+import { invalidExperimentConfig, isPlainObject, mergeConfig, validExperimentConfig } from "../../config.js";
+export const defaultContextPackGenerationConfig = {
+    outDir: "lab-output/context-pack-generation",
+    kitCommand: "npx @dailephd/my-dev-kit@latest"
+};
+const SUPPORTED_FIELDS = ["outDir", "kitCommand", "caseIds", "benchmarkProjects"];
+export const contextPackGenerationConfigDefinition = {
+    fields: [
+        { name: "outDir", type: "string", required: true, description: "Experiment output root." },
+        { name: "kitCommand", type: "string", description: "my-dev-kit command used for index and retrieval." },
+        { name: "caseIds", type: "array", description: "Bundled evaluation case IDs to include; corpus order is preserved." },
+        { name: "benchmarkProjects", type: "array", description: "Bundled benchmark project IDs to include; corpus order is preserved." }
+    ]
+};
+export function validateContextPackGenerationConfig(config) {
+    if (config !== undefined && !isPlainObject(config)) {
+        return invalidExperimentConfig(["context pack generation config must be an object."]);
+    }
+    const errors = [];
+    const unsupported = Object.keys(config ?? {}).filter((key) => !SUPPORTED_FIELDS.includes(key));
+    if (unsupported.length > 0) {
+        errors.push(`Unsupported context-pack-generation config field(s): ${unsupported.sort().join(", ")}.`);
+    }
+    const normalized = mergeConfig(defaultContextPackGenerationConfig, config);
+    for (const field of ["outDir", "kitCommand"]) {
+        const value = normalized[field];
+        if (typeof value !== "string" || !value.trim())
+            errors.push(`${field} must be a non-empty string.`);
+    }
+    for (const field of ["caseIds", "benchmarkProjects"]) {
+        const value = normalized[field];
+        if (value === undefined)
+            continue;
+        if (!Array.isArray(value) || value.length === 0 || !value.every((item) => typeof item === "string" && item.trim())) {
+            errors.push(`${field} must be a non-empty array of non-empty strings when provided.`);
+        }
+    }
+    return errors.length > 0 ? invalidExperimentConfig(errors) : validExperimentConfig(normalized);
+}