@wix/pathgrade 1.0.39 → 1.0.40

package/README.md

@@ -178,7 +178,7 @@ Use `check()` for binary requirements, `score()` for partial credit, `judge()` f
 check('tests-pass', async ({ runCommand }) => {
     const { exitCode } = await runCommand('npm test');
     return exitCode === 0;
-});
+}, { revision: { contract: 1 } });
 ```
 
 ### `score()` - Partial credit
@@ -187,7 +187,7 @@ check('tests-pass', async ({ runCommand }) => {
 score('coverage', async ({ runCommand }) => {
     const { stdout } = await runCommand('npx coverage-summary');
     return parseFloat(stdout) / 100;
-});
+}, { revision: { contract: 1 } });
 ```
 
 ### `judge()` - Rubric evaluation
@@ -199,6 +199,7 @@ Was the fix minimal and correct? (0-0.5)`,
 });
 ```
 
+Function-bearing `check()` and `score()` scorers require explicit canonical-JSON `revision` data for authoritative baseline comparisons. A `judge()` with a function-valued `input` requires the same declaration. Imported scorer helpers belong in `comparisonInputs`; closed-over values belong in `revision`.
 Judge scorers also support:
 
 - `retry` for transient judge failures
@@ -440,9 +441,8 @@ Useful details:
 - `pathgrade preview browser` starts a local viewer on `http://localhost:3847`.
 - `pathgrade report` posts or updates a PR comment in GitHub Actions; locally it prints the markdown report and then the numeric pass rate. Provider orchestrators can add a `--details-url`, suppress stale updates with `--expected-head-sha`, and opt into surfaced API failures with `--strict`.
 - `pathgrade validate --affected` is a strict mode for CI: every discovered eval must either live under a `SKILL.md` anchor or export valid `__pathgradeMeta`.
-
+Declare methodology files separately from affected-selection dependencies with `__pathgradeMeta.comparisonInputs`, for example `comparisonInputs: ['evals/fixtures/**', 'evals/scorers/**']`. An empty array explicitly declares a self-contained eval. Pathgrade hashes the raw eval source plus each declared glob, matched repository-relative path, and raw file bytes; `.git` internals and `.pathgrade` outputs are excluded. Missing, invalid, unmatched, escaping, or unreadable inputs suppress numeric deltas. `deps`, `extraDeps`, inferred skill roots, and global triggers affect selection only and never definition identity.
 Run `pathgrade --help` for the full help text.
-
 ## Configuration
 
 ```typescript
@@ -461,7 +461,7 @@ export default {
 
 Pathgrade reads `pathgrade.config.*` for CLI and affected-selection behavior. `runner.adapter` and `--adapter=<name|path>` select the runner; `--adapter` wins over config. `attempts` defaults to `1` and must be a positive integer. Built-in adapters `vitest`, `jest`, and `node-test` support repeated attempts; third-party invocation adapters must advertise `supportsRepeatedAttempts: true` and produce the normalized child snapshot contract.
 
-New reports use schema version 2. They preserve every attempt with `case_id`, `attempt_id`, and `attempt_index`; include the display-safe `runner_outcome` that determined binary success; publish `mean_reward` for partial scores; and publish finite-sample pass@k only for complete binary attempts. `runner_status` and `threshold_status` expose the two gates independently, and canonical `status` passes only when runner assertions pass and any configured threshold passes. Runner assertions, runner diagnostics, and native runner references remain in the normalized run model and are not copied into public report artifacts; evaluation diagnostics retain their existing report behavior. A partial reward or incomplete attempt makes pass@k explicitly unavailable. Version-1 and older version-2 reports remain readable with missing gate fields derived during loading, but Pathgrade no longer computes pass@k or pass^k by pooling heterogeneous cases.
+New reports use schema version 3. They add a versioned task inventory with every selected eval file, collection completeness, stable task keys, and bounded non-scoring reasons. They also preserve every attempt with `case_id`, `attempt_id`, and `attempt_index`; include the display-safe `runner_outcome`; publish `mean_reward`; and publish finite-sample pass@k only for complete binary attempts. `runner_status` and `threshold_status` expose the two gates independently, and canonical `status` passes only when runner assertions pass and any configured threshold passes. Runner assertions, runner diagnostics, native runner references, and full flow traces remain transient or in trace artifacts rather than being copied into consolidated public reports. A partial reward or incomplete attempt makes pass@k explicitly unavailable. Report versions 1 and 2 remain readable, but they cannot be promoted as authoritative task-accounted baselines and must be rerun.
 
 Third-party runner adapters are supported through `@wix/pathgrade/adapter-kit`. Adapter names resolve as follows:
 

package/dist/adapters/jest/results.d.ts

@@ -23,6 +23,7 @@ export declare function normalizeJestRunResults(input: {
     run: AdapterRunHandle;
     results: JestAggregatedResult;
     metadataByCaseId?: Map<string, PathgradeTestMeta[]>;
+    discoveredFiles?: readonly string[];
     cwd?: string;
 }): NormalizedRunSnapshot;
 export declare function jestCaseId(input: {

package/dist/adapters/jest/results.js

@@ -1,20 +1,39 @@
-import { resolveSourceMetadata } from '../../reporting/source-metadata.js';
+import { createSourceMetadataResolver } from '../../reporting/source-metadata.js';
 export function normalizeJestRunResults(input) {
     const runId = `${input.run.adapterName}:run`;
+    const cwd = input.cwd ?? process.cwd();
+    const resolveSourceMetadata = createSourceMetadataResolver(cwd);
     const files = input.results.testResults ?? [];
-    const units = files.map((fileResult, index) => ({
-        id: unitId(index),
-        runId,
-        displayName: fileResult.testFilePath,
-        ...resolveSourceMetadata(fileResult.testFilePath, input.cwd ?? process.cwd()),
-        groupingHints: [{
-                kind: 'source',
-                key: fileResult.testFilePath,
-                label: fileResult.testFilePath,
-                order: index,
-            }],
-        nativeReferences: [{ kind: 'jest-file', id: fileResult.testFilePath }],
-    }));
+    const collection = input.run.status === 'completed'
+        ? { state: 'complete' }
+        : { state: 'incomplete', reason: 'run-incomplete' };
+    const units = [
+        ...files.map((fileResult, index) => ({
+            id: unitId(index),
+            runId,
+            displayName: fileResult.testFilePath,
+            collection,
+            ...resolveSourceMetadata(fileResult.testFilePath),
+            groupingHints: [{
+                    kind: 'source',
+                    key: fileResult.testFilePath,
+                    label: fileResult.testFilePath,
+                    order: index,
+                }],
+            nativeReferences: [{ kind: 'jest-file', id: fileResult.testFilePath }],
+        })),
+        ...(input.discoveredFiles ?? [])
+            .filter(file => !files.some(result => normalizePath(result.testFilePath, resolveSourceMetadata) === file))
+            .map((file, index) => ({
+            id: unitId(files.length + index),
+            runId,
+            displayName: file,
+            collection,
+            ...resolveSourceMetadata(file),
+            groupingHints: [{ kind: 'source', key: file, label: file, order: files.length + index }],
+            nativeReferences: [{ kind: 'jest-file', id: file }],
+        })),
+    ];
     const occurrenceCounts = new Map();
     return {
         version: 1,
@@ -45,6 +64,9 @@ export function normalizeJestRunResults(input) {
         },
     };
 }
+function normalizePath(file, resolveSourceMetadata) {
+    return resolveSourceMetadata(file).sourceFile ?? file.replaceAll('\\', '/');
+}
 function assertionsForFile(fileResult) {
     return fileResult.assertionResults ?? fileResult.testResults ?? [];
 }

package/dist/adapters/jest/runner-adapter.d.ts

@@ -3,6 +3,7 @@ import { type PathgradeTestMeta, RunnerAdapter } from '@wix/pathgrade/adapter-ki
 interface JestRunNative {
     results?: JestAggregatedResult;
     metadataByCaseId?: Map<string, PathgradeTestMeta[]>;
+    discoveredFiles?: string[];
 }
 export declare function createJestAdapter(options?: JestRunNative): RunnerAdapter;
 export declare function createPathgradeAdapter(): RunnerAdapter;

package/dist/adapters/jest/runner-adapter.js

@@ -27,7 +27,10 @@ export function createJestAdapter(options = {}) {
                 adapterName: this.name,
                 status: input.signal?.aborted ? 'cancelled' : 'completed',
                 exitCode: input.signal?.aborted ? 1 : 0,
-                native: options,
+                native: {
+                    ...options,
+                    discoveredFiles: input.discovered.units.flatMap(unit => unit.sourceRef ? [unit.sourceRef] : []),
+                },
             };
         },
         async collectNormalizedRunSnapshot(run) {
@@ -36,6 +39,7 @@ export function createJestAdapter(options = {}) {
                 run,
                 results: native.results ?? { testResults: [] },
                 metadataByCaseId: native.metadataByCaseId,
+                discoveredFiles: native.discoveredFiles,
                 cwd: discoveryCwd,
             });
         },

package/dist/adapters/node-test/runner-adapter.js

@@ -37,7 +37,7 @@ export function createNodeTestAdapter() {
                     adapterName: this.name,
                     status: 'completed',
                     exitCode: 0,
-                    native: { resultsPath, resultsDir, cwd },
+                    native: { resultsPath, resultsDir, cwd, discoveredFiles: files },
                 };
             }
             const exitCode = await spawnNodeTest({
@@ -54,7 +54,7 @@ export function createNodeTestAdapter() {
                 adapterName: this.name,
                 status: input.signal?.aborted ? 'cancelled' : (exitCode === 0 ? 'completed' : 'failed'),
                 exitCode,
-                native: { resultsPath, resultsDir, cwd },
+                native: { resultsPath, resultsDir, cwd, discoveredFiles: files },
             };
         },
         async collectNormalizedRunSnapshot(run) {
@@ -71,7 +71,7 @@ export function createNodeTestAdapter() {
                 return buildNormalizedRunSnapshotFromReportGroups(run, Array.from(groupMap.entries()).map(([groupName, groupedCases]) => ({
                     groupName,
                     cases: groupedCases,
-                })), { cwd: native.cwd });
+                })), { cwd: native.cwd, discoveredFiles: native.discoveredFiles });
             }
             finally {
                 if (native.resultsDir)
@@ -104,10 +104,11 @@ function readNodeTestRunNative(run) {
         && run.native !== null
         && typeof run.native.resultsPath === 'string'
         && typeof run.native.resultsDir === 'string'
-        && typeof run.native.cwd === 'string') {
+        && typeof run.native.cwd === 'string'
+        && Array.isArray(run.native.discoveredFiles)) {
         return run.native;
     }
-    return { resultsPath: '', resultsDir: '', cwd: process.cwd() };
+    return { resultsPath: '', resultsDir: '', cwd: process.cwd(), discoveredFiles: [] };
 }
 async function readCases(resultsPath) {
     if (!resultsPath || !(await fs.pathExists(resultsPath)))

package/dist/affected/meta.d.ts

@@ -17,10 +17,13 @@
 export interface ParsedMeta {
     deps?: string[];
     extraDeps?: string[];
+    comparisonInputs?: string[];
     alwaysRun?: boolean;
 }
 /**
  * Parse `__pathgradeMeta` from an eval file's AST. Returns `null` if the
  * export is not present.
  */
-export declare function parsePathgradeMeta(evalFile: string): ParsedMeta | null;
+export declare function parsePathgradeMeta(evalFile: string, options?: {
+    validateComparisonInputs?: boolean;
+}): ParsedMeta | null;

package/dist/affected/meta.js

@@ -21,14 +21,14 @@ import picomatch from 'picomatch';
  * Parse `__pathgradeMeta` from an eval file's AST. Returns `null` if the
  * export is not present.
  */
-export function parsePathgradeMeta(evalFile) {
+export function parsePathgradeMeta(evalFile, options = {}) {
     const source = fs.readFileSync(evalFile, 'utf-8');
     const sourceFile = ts.createSourceFile(evalFile, source, ts.ScriptTarget.Latest, 
     /* setParentNodes */ true, ts.ScriptKind.TS);
     for (const stmt of sourceFile.statements) {
         const initializer = findMetaInitializer(stmt);
         if (initializer) {
-            return extractMeta(initializer, evalFile);
+            return extractMeta(initializer, evalFile, options);
         }
     }
     return null;
@@ -50,7 +50,7 @@ function findMetaInitializer(stmt) {
     }
     return null;
 }
-function extractMeta(expr, evalFile) {
+function extractMeta(expr, evalFile, options) {
     // Optionally unwrap a type assertion: `{...} as PathgradeMeta` / `<PathgradeMeta>{...}`.
     let node = expr;
     while (ts.isAsExpression(node) || ts.isTypeAssertionExpression(node) || ts.isSatisfiesExpression(node)) {
@@ -64,7 +64,9 @@ function extractMeta(expr, evalFile) {
         if (!ts.isPropertyAssignment(prop))
             continue;
         const name = propertyKeyName(prop.name);
-        if (name === 'deps' || name === 'extraDeps') {
+        if (name === 'deps' || name === 'extraDeps' || name === 'comparisonInputs') {
+            if (name === 'comparisonInputs' && options.validateComparisonInputs === false)
+                continue;
             const globs = extractStringArray(prop.initializer, evalFile, name);
             for (const g of globs)
                 validateGlob(g, evalFile, name);
@@ -89,7 +91,9 @@ function validateGlob(glob, evalFile, field) {
     if (glob.length === 0) {
         throw new Error(`${evalFile}: __pathgradeMeta.${field} contains an empty glob.`);
     }
-    if (glob.startsWith('../') || glob.includes('/../') || glob === '..') {
+    const normalized = glob.replaceAll('\\', '/');
+    if (normalized.startsWith('/') || /^[A-Za-z]:\//.test(normalized)
+        || normalized.startsWith('../') || normalized.includes('/../') || normalized === '..') {
         throw new Error(`${evalFile}: __pathgradeMeta.${field} entry "${glob}" escapes the repo root — ` +
             `globs must be repo-root-relative.`);
     }

package/dist/affected/select.js

@@ -44,7 +44,7 @@ export function selectAffected(input) {
     for (const evalFile of evalFiles) {
         const absEval = path.resolve(repoRoot, evalFile);
         const skillRoot = findSkillRoot(absEval, repoRoot);
-        const meta = parsePathgradeMeta(absEval);
+        const meta = parsePathgradeMeta(absEval, { validateComparisonInputs: false });
         // Precedence #2: alwaysRun wins over dep matching.
         if (meta?.alwaysRun === true) {
             selected.push({ file: evalFile, reason: 'always-run' });

package/dist/commands/report.js

@@ -44,9 +44,8 @@ async function loadReport(resolvedPath) {
     if (!(await fs.pathExists(resolvedPath))) {
         throw new Error(`results file not found at ${resolvedPath}`);
     }
-    const raw = await fs.readJSON(resolvedPath);
     try {
-        return parsePathgradeReport(raw);
+        return parsePathgradeReport(await fs.readJSON(resolvedPath));
     }
     catch {
         throw new Error(`results file at ${resolvedPath} is not a valid pathgrade report`);

package/dist/reporting/comparison-contract.d.ts

@@ -1,10 +1,11 @@
-import type { ComparisonContract, EvalReport } from '../types.js';
+import type { EvalReport } from '../types.js';
 import type { Scorer } from '../sdk/types.js';
 import type { ReportGroupInput } from './types.js';
+import type { ComparisonContractV2 } from './reliability-contract.js';
 export declare function buildComparisonContract(input: {
     group: ReportGroupInput;
     report: EvalReport;
     attemptsRequested: number;
     attemptsCompleted: number;
-}): ComparisonContract;
+}): ComparisonContractV2;
 export declare function createScorerRevision(scorers: readonly Scorer[]): string | undefined;

package/dist/reporting/comparison-contract.js

@@ -1,7 +1,6 @@
 import { createHash } from 'node:crypto';
 import { canonicalizeJson } from '../core/canonical-json.js';
 export function buildComparisonContract(input) {
-    const unavailable = new Set();
     const evaluations = input.group.cases.flatMap(testCase => {
         const caseIdentity = testCase.repeatKey ?? testCase.caseId ?? testCase.name;
         return (testCase.attempts?.flatMap(attempt => attempt.evaluations ?? [])
@@ -15,16 +14,16 @@ export function buildComparisonContract(input) {
             runtime: runtimeIdentity(evaluation),
         }));
     });
-    const definitionRevision = input.group.sourceRevision
+    const comparisonInputs = input.group.comparisonInputs ?? { state: 'missing' };
+    const definitionRevision = input.group.sourceRevision && comparisonInputs.state === 'resolved'
         ? revision('definition', {
             source: input.group.sourceRevision,
+            comparison_inputs: comparisonInputs.revision,
             evaluations: uniqueSorted(evaluations.map(evaluation => ({
                 case: evaluation.case, definition: evaluation.definition,
             }))),
         })
         : undefined;
-    if (!definitionRevision)
-        unavailable.add('definition_metadata_missing');
     const scorerRevision = evaluations.length > 0 && evaluations.every(evaluation => evaluation.scorer)
         ? revision('scorers', uniqueSorted(evaluations.map(evaluation => ({
             case: evaluation.case,
@@ -32,20 +31,10 @@ export function buildComparisonContract(input) {
             scorer: evaluation.scorer,
         }))))
         : undefined;
-    if (!scorerRevision)
-        unavailable.add('scorer_metadata_missing');
     const runtimeComplete = evaluations.length > 0
-        && evaluations.every(evaluation => Object.values(evaluation.runtime)
-            .every(value => value !== undefined));
-    const runtimeRevision = runtimeComplete
-        ? revision('runtime', uniqueSorted(evaluations.map(evaluation => ({
-            case: evaluation.case,
-            definition: evaluation.definition,
-            runtime: evaluation.runtime,
-        }))))
-        : undefined;
-    if (!runtimeRevision)
-        unavailable.add('runtime_metadata_missing');
+        && evaluations.every(evaluation => runtimeIdentityComplete(evaluation.runtime));
+    const runtimeComponents = runtimeComplete ? runtimeComponentRevisions(evaluations) : undefined;
+    const runtimeRevision = runtimeComponents ? revision('runtime', runtimeComponents) : undefined;
     const reportableCases = input.group.cases.filter(testCase => testCase.reportable !== false
         && testCase.state !== 'skipped' && testCase.state !== 'pending');
     const samplingComplete = input.attemptsCompleted === input.attemptsRequested
@@ -59,16 +48,41 @@ export function buildComparisonContract(input) {
             cases: reportableCases.map(testCase => testCase.repeatKey ?? testCase.caseId ?? testCase.name).toSorted(),
         })
         : undefined;
-    if (!samplingRevision)
-        unavailable.add('sampling_incomplete');
     return {
-        version: 1,
-        ...(definitionRevision ? { definition_revision: definitionRevision } : {}),
-        ...(scorerRevision ? { scorer_revision: scorerRevision } : {}),
-        ...(runtimeRevision ? { runtime_revision: runtimeRevision } : {}),
-        ...(samplingRevision ? { sampling_revision: samplingRevision } : {}),
-        report_schema_revision: 'pathgrade-results-v2',
-        ...(unavailable.size > 0 ? { unavailable_reasons: [...unavailable].sort() } : {}),
+        version: 2,
+        ...(definitionRevision
+            ? { comparison_inputs: comparisonInputs, definition_revision: definitionRevision }
+            : { comparison_inputs: comparisonInputs }),
+        ...(scorerRevision
+            ? { scorer_revision_state: { state: 'complete' }, scorer_revision: scorerRevision }
+            : { scorer_revision_state: { state: 'incomplete', reason: 'explicit-revision-required' } }),
+        ...(runtimeRevision && runtimeComponents
+            ? {
+                runtime_revision_state: { state: 'complete' },
+                runtime_revision: runtimeRevision,
+                runtime_components: runtimeComponents,
+            }
+            : { runtime_revision_state: { state: 'incomplete', reason: 'metadata-missing' } }),
+        ...(samplingRevision
+            ? { sampling_revision_state: { state: 'complete' }, sampling_revision: samplingRevision }
+            : { sampling_revision_state: { state: 'incomplete', reason: 'sampling-incomplete' } }),
+        report_schema_revision: 'pathgrade-results-v3',
+    };
+}
+function runtimeComponentRevisions(evaluations) {
+    const component = (name) => revision(`runtime-${name}`, uniqueSorted(evaluations.map(evaluation => ({
+        case: evaluation.case,
+        definition: evaluation.definition,
+        value: evaluation.runtime[name] ?? null,
+    }))));
+    return {
+        agent_name: component('agent_name'),
+        model: component('model'),
+        transport: component('transport'),
+        interaction_mode: component('interaction_mode'),
+        ...(evaluations.some(evaluation => evaluation.runtime.flow)
+            ? { flow: component('flow') }
+            : {}),
     };
 }
 function hasTerminalEvaluation(evaluations) {
@@ -85,16 +99,20 @@ export function createScorerRevision(scorers) {
 }
 function runtimeIdentity(evaluation) {
     const agent = evaluation.agent ?? evaluation.trial?.agent;
-    if (agent) {
+    if (agent)
         return {
-            name: agent.name,
+            agent_name: agent.name,
             model: agent.resolvedModel,
             transport: agent.transport,
-            interaction: agent.interactionMode,
+            interaction_mode: agent.interactionMode,
+            flow: undefined,
         };
-    }
     const flow = evaluation.trial?.flow_trace;
     return {
+        agent_name: undefined,
+        model: undefined,
+        transport: undefined,
+        interaction_mode: undefined,
         flow: flow?.completeness.runtimeIdentity === 'complete'
             && flow.participants.every(participant => participant.runtime) ? {
             ...(flow.protocol ? { protocol: flow.protocol } : {}),
@@ -105,10 +123,22 @@ function runtimeIdentity(evaluation) {
         } : undefined,
     };
 }
+function runtimeIdentityComplete(identity) {
+    return identity.flow !== undefined || [
+        identity.agent_name, identity.model, identity.transport, identity.interaction_mode,
+    ].every(value => value !== undefined);
+}
 function scorerDeclaration(scorer) {
-    const common = { type: scorer.type, name: scorer.name.normalize('NFC'), weight: scorer.weight };
+    const common = {
+        type: scorer.type,
+        name: scorer.name.normalize('NFC'),
+        weight: scorer.weight,
+        ...(scorer.revision === undefined ? {} : { revision: scorer.revision }),
+    };
     if (scorer.type === 'check' || scorer.type === 'score') {
-        return { ...common, function: scorer.fn.toString() };
+        if (scorer.revision === undefined)
+            throw new Error('explicit scorer revision required');
+        return common;
     }
     if (scorer.type === 'tool_usage') {
         return {
@@ -117,13 +147,16 @@ function scorerDeclaration(scorer) {
                 .toSorted((left, right) => canonicalizeJson(left).localeCompare(canonicalizeJson(right))),
         };
     }
+    if (typeof scorer.input === 'function' && scorer.revision === undefined) {
+        throw new Error('explicit scorer revision required');
+    }
     return {
         ...common,
         rubric: scorer.rubric,
         model: scorer.model ?? null,
         retry: scorer.retry ?? null,
         includeToolEvents: scorer.includeToolEvents ?? null,
-        input: typeof scorer.input === 'function' ? scorer.input.toString() : scorer.input ?? null,
+        input: typeof scorer.input === 'function' ? null : scorer.input ?? null,
         tools: scorer.tools?.toSorted() ?? null,
         maxRounds: scorer.maxRounds ?? null,
         cacheControl: scorer.cacheControl ?? null,

package/dist/reporting/core.js

@@ -1,6 +1,7 @@
 import { buildDiagnosticsReport } from '../sdk/diagnostics.js';
 import { extractSkillsFromLog } from '../tool-events.js';
 import { buildComparisonContract } from './comparison-contract.js';
+import { TASK_INVENTORY_VERSION } from './reliability-contract.js';
 export function buildPathgradeReport(input) {
     const warnings = [];
     const consolidatedGroups = [];
@@ -60,7 +61,7 @@ export function buildPathgradeReport(input) {
         : overallMeanReward >= input.threshold ? 'pass' : 'fail';
     return {
         report: {
-            version: 2,
+            version: 3,
             timestamp: new Date().toISOString(),
             ...(input.threshold != null ? { threshold: input.threshold } : {}),
             attempts_requested: attemptsRequested,
@@ -71,6 +72,15 @@ export function buildPathgradeReport(input) {
             threshold_status: thresholdStatus,
             status: runnerStatus === 'fail' || thresholdStatus === 'fail' ? 'fail' : 'pass',
             groups: consolidatedGroups,
+            task_inventory: input.taskInventory ?? {
+                version: TASK_INVENTORY_VERSION,
+                files: input.groups.flatMap(group => group.sourceFile ? [{
+                        eval_file: group.sourceFile,
+                        completeness: 'incomplete',
+                        reason: 'adapter-cannot-prove-completeness',
+                        tasks: [taskInventoryEntry(group)],
+                    }] : []),
+            },
             ...(input.selection ? { selection: input.selection } : {}),
         },
         traces,
@@ -78,6 +88,13 @@ export function buildPathgradeReport(input) {
         warnings,
     };
 }
+function taskInventoryEntry(group) {
+    const scored = group.cases.some(testCase => testCase.state !== 'skipped' && testCase.state !== 'pending');
+    if (scored)
+        return { task_key: group.groupName, state: 'scored' };
+    const reason = group.cases.every(testCase => testCase.state === 'skipped') ? 'skipped' : 'pending';
+    return { task_key: group.groupName, state: 'not-scored', reason };
+}
 function reportableBuiltCase(testCase) {
     if (testCase.reportable !== undefined)
         return testCase.reportable;

package/dist/reporting/reliability-contract.d.ts

@@ -53,6 +53,7 @@ export interface RuntimeComponentRevisions {
     model: string;
     transport: string;
     interaction_mode: string;
+    flow?: string;
 }
 type DefinitionRevisionContract = {
     comparison_inputs: Extract<ComparisonInputsState, {

package/dist/reporting/report-parser.js

@@ -1,9 +1,9 @@
 export function parsePathgradeReport(value) {
     if (!isRecord(value)
-        || (value.version !== 1 && value.version !== 2)
+        || (value.version !== 1 && value.version !== 2 && value.version !== 3)
         || typeof value.timestamp !== 'string'
         || typeof value.overall_pass_rate !== 'number'
-        || (value.version === 2 && (typeof value.overall_mean_reward !== 'number' || !validAttemptCounts(value)))
+        || (value.version !== 1 && (typeof value.overall_mean_reward !== 'number' || !validAttemptCounts(value)))
         || !optionalNumber(value.threshold)
         || (value.status !== 'pass' && value.status !== 'fail')
         || !optionalGateStatus(value.runner_status)
@@ -11,6 +11,7 @@ export function parsePathgradeReport(value) {
         || (value.run_kind !== undefined && value.run_kind !== 'evaluation' && value.run_kind !== 'no-affected')
         || !Array.isArray(value.groups)
         || !value.groups.every(group => isReportGroup(group, value.version))
+        || (value.version === 3 && !isTaskInventory(value.task_inventory))
         || !isSelection(value.selection)) {
         throw new Error('PathGrade results.json is missing or has an unsupported schema');
     }
@@ -50,7 +51,9 @@ function isReportGroup(value, version) {
         && value.skills_used.every(item => typeof item === 'string')
         && Array.isArray(value.trials)
         && value.trials.every(isTrial)
-        && (value.comparison_contract === undefined || isComparisonContract(value.comparison_contract));
+        && (version === 3
+            ? isComparisonContractV2(value.comparison_contract)
+            : value.comparison_contract === undefined || isComparisonContract(value.comparison_contract));
 }
 function deriveRunnerStatus(value) {
     if (value.runner_status === 'fail')
@@ -137,6 +140,9 @@ function isSelection(value) {
             && item.reason === 'no-matching-deps'));
 }
 function isComparisonContract(value) {
+    return isComparisonContractV1(value) || isComparisonContractV2(value);
+}
+function isComparisonContractV1(value) {
     return isRecord(value)
         && value.version === 1
         && optionalString(value.definition_revision)
@@ -150,6 +156,65 @@ function isComparisonContract(value) {
                 || reason === 'runtime_metadata_missing'
                 || reason === 'sampling_incomplete')));
 }
+function isComparisonContractV2(value) {
+    if (!isRecord(value) || value.version !== 2 || value.report_schema_revision !== 'pathgrade-results-v3')
+        return false;
+    const comparisonInputs = value.comparison_inputs;
+    const definitionComplete = isRecord(comparisonInputs) && comparisonInputs.state === 'resolved';
+    const scorerComplete = isState(value.scorer_revision_state, 'complete');
+    const runtimeComplete = isState(value.runtime_revision_state, 'complete');
+    const samplingComplete = isState(value.sampling_revision_state, 'complete');
+    return isComparisonInputs(comparisonInputs)
+        && (definitionComplete ? typeof value.definition_revision === 'string' : value.definition_revision === undefined)
+        && isRevisionState(value.scorer_revision_state, ['explicit-revision-required', 'metadata-missing'])
+        && (scorerComplete ? typeof value.scorer_revision === 'string' : value.scorer_revision === undefined)
+        && isRevisionState(value.runtime_revision_state, ['metadata-missing'])
+        && (runtimeComplete
+            ? typeof value.runtime_revision === 'string' && isRuntimeComponents(value.runtime_components)
+            : value.runtime_revision === undefined && value.runtime_components === undefined)
+        && isRevisionState(value.sampling_revision_state, ['sampling-incomplete'])
+        && (samplingComplete ? typeof value.sampling_revision === 'string' : value.sampling_revision === undefined);
+}
+function isComparisonInputs(value) {
+    if (!isRecord(value))
+        return false;
+    if (value.state === 'missing')
+        return true;
+    if (value.state === 'invalid') {
+        return value.reason === 'malformed-declaration' || value.reason === 'no-matches'
+            || value.reason === 'outside-repository' || value.reason === 'unreadable-input';
+    }
+    return value.state === 'resolved'
+        && Array.isArray(value.declarations) && value.declarations.every(item => typeof item === 'string')
+        && Array.isArray(value.files) && value.files.every(file => isRecord(file)
+        && typeof file.path === 'string' && typeof file.revision === 'string')
+        && typeof value.revision === 'string';
+}
+function isRevisionState(value, reasons) {
+    return isState(value, 'complete') || (isRecord(value) && value.state === 'incomplete'
+        && typeof value.reason === 'string' && reasons.includes(value.reason));
+}
+function isState(value, state) {
+    return isRecord(value) && value.state === state;
+}
+function isRuntimeComponents(value) {
+    return isRecord(value) && ['agent_name', 'model', 'transport', 'interaction_mode']
+        .every(key => typeof value[key] === 'string')
+        && optionalString(value.flow);
+}
+function isTaskInventory(value) {
+    return isRecord(value) && value.version === 1 && Array.isArray(value.files)
+        && value.files.every(file => isRecord(file)
+            && typeof file.eval_file === 'string'
+            && (file.completeness === 'complete' || (file.completeness === 'incomplete'
+                && (file.reason === 'adapter-cannot-prove-completeness'
+                    || file.reason === 'collection-failed' || file.reason === 'run-incomplete')))
+            && Array.isArray(file.tasks)
+            && file.tasks.every(task => isRecord(task) && typeof task.task_key === 'string'
+                && (task.state === 'scored' || (task.state === 'not-scored'
+                    && (task.reason === 'skipped' || task.reason === 'pending'
+                        || task.reason === 'timed-out' || task.reason === 'failed-before-evaluation')))));
+}
 function optionalString(value) {
     return value === undefined || typeof value === 'string';
 }

package/dist/reporting/source-metadata.d.ts

@@ -1,5 +1,8 @@
+import type { ComparisonInputsState } from './reliability-contract.js';
 export interface SourceMetadata {
     sourceFile?: string;
     sourceRevision?: string;
+    comparisonInputs?: ComparisonInputsState;
 }
 export declare function resolveSourceMetadata(filePath: string | undefined, cwd: string): SourceMetadata;
+export declare function createSourceMetadataResolver(cwd: string): (filePath: string | undefined) => SourceMetadata;

package/dist/reporting/source-metadata.js

@@ -1,7 +1,18 @@
 import { createHash } from 'node:crypto';
 import { readFileSync } from 'node:fs';
+import { readdirSync } from 'node:fs';
 import * as path from 'node:path';
+import picomatch from 'picomatch';
+import { canonicalizeJson } from '../core/canonical-json.js';
+import { parsePathgradeMeta } from '../affected/meta.js';
 export function resolveSourceMetadata(filePath, cwd) {
+    return resolveSourceMetadataWithFiles(filePath, cwd, () => listRepositoryFiles(cwd));
+}
+export function createSourceMetadataResolver(cwd) {
+    let repositoryFiles;
+    return filePath => resolveSourceMetadataWithFiles(filePath, cwd, () => repositoryFiles ??= listRepositoryFiles(cwd));
+}
+function resolveSourceMetadataWithFiles(filePath, cwd, repositoryFiles) {
     if (!filePath)
         return {};
     const absolute = path.resolve(cwd, filePath);
@@ -11,10 +22,64 @@ export function resolveSourceMetadata(filePath, cwd) {
     try {
         return {
             sourceFile: relative,
-            sourceRevision: `sha256:${createHash('sha256').update(readFileSync(absolute)).digest('hex')}`,
+            sourceRevision: digest(readFileSync(absolute)),
+            comparisonInputs: resolveComparisonInputs(absolute, cwd, repositoryFiles),
+        };
+    }
+    catch {
+        return { sourceFile: relative, comparisonInputs: { state: 'invalid', reason: 'unreadable-input' } };
+    }
+}
+function resolveComparisonInputs(evalFile, cwd, repositoryFiles) {
+    let declarations;
+    try {
+        declarations = parsePathgradeMeta(evalFile)?.comparisonInputs;
+    }
+    catch (error) {
+        const message = error instanceof Error ? error.message : '';
+        return {
+            state: 'invalid',
+            reason: message.includes('escapes the repo root') ? 'outside-repository' : 'malformed-declaration',
+        };
+    }
+    if (declarations === undefined)
+        return { state: 'missing' };
+    const normalized = declarations.map(value => value.replaceAll('\\', '/').normalize('NFC'));
+    const matched = [...new Set(normalized.flatMap(declaration => {
+            const matches = picomatch(declaration, { dot: true });
+            return repositoryFiles().filter(file => matches(file));
+        }))].toSorted();
+    if (normalized.length > 0 && matched.length === 0)
+        return { state: 'invalid', reason: 'no-matches' };
+    try {
+        const files = matched.map(file => ({ path: file, revision: digest(readFileSync(path.join(cwd, file))) }));
+        return {
+            state: 'resolved',
+            declarations: normalized,
+            files,
+            revision: digest(canonicalizeJson({ declarations: normalized, files })),
         };
     }
     catch {
-        return { sourceFile: relative };
+        return { state: 'invalid', reason: 'unreadable-input' };
     }
 }
+function listRepositoryFiles(cwd) {
+    const files = [];
+    const walk = (directory, relative) => {
+        for (const entry of readdirSync(directory, { withFileTypes: true })) {
+            const childRelative = relative ? `${relative}/${entry.name}` : entry.name;
+            if (!relative && (entry.name === '.git' || entry.name === '.pathgrade'))
+                continue;
+            if (entry.isDirectory())
+                walk(path.join(directory, entry.name), childRelative);
+            else if (entry.isFile())
+                files.push(childRelative);
+        }
+    };
+    walk(path.resolve(cwd), '');
+    return files.toSorted();
+}
+function digest(value) {
+    return `sha256:${createHash('sha256').update(value).digest('hex')}`;
+}

package/dist/reporting/types.d.ts

@@ -2,6 +2,7 @@ import type { DiagnosticsReport } from '../sdk/diagnostics.js';
 import type { AgentExecutionMetadata, EvaluationResultKind } from '../sdk/types.js';
 import type { AttemptOutcome, RunStatus } from '../runners/model.js';
 import type { PathgradeReport, PathgradeSelectionReport, TrialResult } from '../types.js';
+import type { ComparisonInputsState, TaskInventory } from './reliability-contract.js';
 export type ReportCaseState = 'passed' | 'failed' | 'skipped' | 'pending';
 export interface ReportRunInput {
     runStatus?: RunStatus;
@@ -9,12 +10,14 @@ export interface ReportRunInput {
     selection?: PathgradeSelectionReport;
     attemptsRequested?: number;
     attemptsCompleted?: number;
+    taskInventory?: TaskInventory;
     groups: ReportGroupInput[];
 }
 export interface ReportGroupInput {
     groupName: string;
     sourceFile?: string;
     sourceRevision?: string;
+    comparisonInputs?: ComparisonInputsState;
     cases: ReportCaseInput[];
 }
 export interface ReportCaseInput {

package/dist/runners/model-builders.d.ts

@@ -3,4 +3,5 @@ import type { AdapterRunHandle } from './adapter.js';
 import type { NormalizedRunSnapshot } from './model.js';
 export declare function buildNormalizedRunSnapshotFromReportGroups(run: AdapterRunHandle, groups: ReportGroupInput[], options?: {
     cwd?: string;
+    discoveredFiles?: readonly string[];
 }): NormalizedRunSnapshot;

package/dist/runners/model-builders.js

@@ -1,6 +1,7 @@
-import { resolveSourceMetadata } from '../reporting/source-metadata.js';
+import { createSourceMetadataResolver } from '../reporting/source-metadata.js';
 export function buildNormalizedRunSnapshotFromReportGroups(run, groups, options = {}) {
     const cwd = options.cwd ?? process.cwd();
+    const resolveSourceMetadata = createSourceMetadataResolver(cwd);
     return {
         version: 1,
         completeness: 'final',
@@ -14,18 +15,41 @@ export function buildNormalizedRunSnapshotFromReportGroups(run, groups, options
                         message: diagnostic.message,
                     })) } : {}),
             },
-            units: groups.map((group, index) => {
-                const source = group.sourceFile && group.sourceRevision
-                    ? { sourceFile: group.sourceFile, sourceRevision: group.sourceRevision }
-                    : resolveSourceMetadata(group.sourceFile ?? group.cases.map(testCase => testCase.filePath ?? testCase.sourceRef).find(Boolean), cwd);
-                return {
-                    id: unitId(group.groupName, index),
+            units: [
+                ...groups.map((group, index) => {
+                    const source = group.sourceFile && group.sourceRevision
+                        ? {
+                            sourceFile: group.sourceFile,
+                            sourceRevision: group.sourceRevision,
+                            ...(group.comparisonInputs ? { comparisonInputs: group.comparisonInputs } : {}),
+                        }
+                        : resolveSourceMetadata(group.sourceFile ?? group.cases.map(testCase => testCase.filePath ?? testCase.sourceRef).find(Boolean));
+                    return {
+                        id: unitId(group.groupName, index),
+                        runId: `${run.adapterName}:run`,
+                        displayName: group.groupName,
+                        collection: run.status === 'completed' || run.status === 'failed'
+                            ? { state: 'complete' }
+                            : { state: 'incomplete', reason: 'run-incomplete' },
+                        ...source,
+                        groupingHints: [{ kind: 'suite', key: group.groupName, label: group.groupName, order: index }],
+                    };
+                }),
+                ...(options.discoveredFiles ?? [])
+                    .filter(file => !groups.some(group => group.sourceFile === file
+                    || group.cases.some(testCase => testCase.filePath === file || testCase.sourceRef === file)))
+                    .map((file, index) => ({
+                    id: unitId(file, groups.length + index),
                     runId: `${run.adapterName}:run`,
-                    displayName: group.groupName,
-                    ...source,
-                    groupingHints: [{ kind: 'suite', key: group.groupName, label: group.groupName, order: index }],
-                };
-            }),
+                    displayName: file,
+                    collection: {
+                        state: 'incomplete',
+                        reason: 'adapter-cannot-prove-completeness',
+                    },
+                    ...resolveSourceMetadata(file),
+                    groupingHints: [{ kind: 'source', key: file, label: file, order: groups.length + index }],
+                })),
+            ],
             cases: groups.flatMap((group, groupIndex) => group.cases.map((testCase, caseIndex) => {
                 const caseId = testCase.caseId ?? `${unitId(group.groupName, groupIndex)}:case-${caseIndex + 1}`;
                 const attemptId = `${caseId}:attempt-1`;

package/dist/runners/model.d.ts

@@ -1,6 +1,7 @@
 import type { DiagnosticsReport } from '../sdk/diagnostics.js';
 import type { TrialResult } from '../types.js';
 import type { CollectionIncompleteReason } from '../reporting/reliability-contract.js';
+import type { ComparisonInputsState } from '../reporting/reliability-contract.js';
 export interface NormalizedRunSnapshot {
     version: 1;
     completeness: SnapshotCompleteness;
@@ -29,6 +30,7 @@ export interface EvalUnitRecord {
     collection?: EvalUnitCollection;
     sourceFile?: string;
     sourceRevision?: string;
+    comparisonInputs?: ComparisonInputsState;
     diagnostics?: Diagnostic[];
     nativeReferences?: NativeReference[];
     groupingHints?: GroupingHint[];

package/dist/runners/report-projection.js

@@ -1,3 +1,4 @@
+import { TASK_INVENTORY_VERSION } from '../reporting/reliability-contract.js';
 import { validateNormalizedRunSnapshot } from './model-validation.js';
 export function projectNormalizedRunSnapshotToReportInput(snapshot, options = {}) {
     const validation = validateNormalizedRunSnapshot(snapshot, { completeness: 'final' });
@@ -20,6 +21,7 @@ export function projectNormalizedRunSnapshotToReportInput(snapshot, options = {}
             groupName: key,
             ...(unit?.sourceFile ? { sourceFile: unit.sourceFile } : {}),
             ...(unit?.sourceRevision ? { sourceRevision: unit.sourceRevision } : {}),
+            ...(unit?.comparisonInputs ? { comparisonInputs: unit.comparisonInputs } : {}),
             cases: [],
         };
         group.cases.push({
@@ -44,9 +46,90 @@ export function projectNormalizedRunSnapshotToReportInput(snapshot, options = {}
         runStatus: snapshot.model.run.status,
         attemptsRequested: snapshot.model.run.attemptsRequested ?? 1,
         attemptsCompleted: snapshot.model.run.attemptsCompleted ?? 1,
+        taskInventory: buildTaskInventory(snapshot, [...groups.values()]),
         groups: [...groups.values()],
     };
 }
+function buildTaskInventory(snapshot, groups) {
+    const unitsBySource = new Map(snapshot.model.units
+        .filter(unit => unit.sourceFile)
+        .map(unit => [unit.sourceFile, unit]));
+    const files = new Map();
+    for (const unit of snapshot.model.units) {
+        if (!unit.sourceFile)
+            continue;
+        const collection = unit.collection ?? {
+            state: 'incomplete',
+            reason: 'adapter-cannot-prove-completeness',
+        };
+        const existing = files.get(unit.sourceFile);
+        if (!existing) {
+            files.set(unit.sourceFile, {
+                tasks: [],
+                completeness: collection.state,
+                ...(collection.state === 'incomplete' ? { reason: collection.reason } : {}),
+            });
+        }
+        else if (collection.state === 'incomplete') {
+            existing.completeness = 'incomplete';
+            existing.reason = collection.reason;
+        }
+    }
+    for (const group of groups) {
+        if (!group.sourceFile)
+            continue;
+        const unit = unitsBySource.get(group.sourceFile);
+        const collection = unit?.collection ?? {
+            state: 'incomplete',
+            reason: 'adapter-cannot-prove-completeness',
+        };
+        const file = files.get(group.sourceFile) ?? {
+            tasks: [],
+            completeness: collection.state,
+            ...(collection.state === 'incomplete' ? { reason: collection.reason } : {}),
+        };
+        file.tasks.push(taskInventoryEntry(group));
+        if (collection.state === 'incomplete') {
+            file.completeness = 'incomplete';
+            file.reason = collection.reason;
+        }
+        files.set(group.sourceFile, file);
+    }
+    return {
+        version: TASK_INVENTORY_VERSION,
+        files: [...files.entries()].toSorted(([left], [right]) => left.localeCompare(right)).map(([eval_file, file]) => ({
+            eval_file,
+            tasks: file.tasks.toSorted((left, right) => left.task_key.localeCompare(right.task_key)),
+            ...(file.completeness === 'complete'
+                ? { completeness: 'complete' }
+                : { completeness: 'incomplete', reason: file.reason ?? 'adapter-cannot-prove-completeness' }),
+        })),
+    };
+}
+function taskInventoryEntry(group) {
+    const states = group.cases.map(testCase => testCase.state);
+    if (states.every(state => state === 'skipped')) {
+        return { task_key: group.groupName, state: 'not-scored', reason: 'skipped' };
+    }
+    if (states.every(state => state === 'pending')) {
+        return { task_key: group.groupName, state: 'not-scored', reason: 'pending' };
+    }
+    if (group.cases.some(testCase => testCase.attempts?.some(attempt => attempt.outcome.kind === 'timed-out'))) {
+        return { task_key: group.groupName, state: 'not-scored', reason: 'timed-out' };
+    }
+    if (group.cases.some(isPromotableCase))
+        return { task_key: group.groupName, state: 'scored' };
+    return { task_key: group.groupName, state: 'not-scored', reason: 'failed-before-evaluation' };
+}
+function isPromotableCase(testCase) {
+    if (testCase.state === 'skipped' || testCase.state === 'pending')
+        return false;
+    const evaluations = testCase.attempts?.flatMap(attempt => attempt.evaluations ?? []) ?? testCase.evaluations;
+    if (evaluations === undefined)
+        return true;
+    return evaluations.some(evaluation => evaluation.score !== undefined
+        && evaluation.resultKind !== 'synthetic_no_evaluation');
+}
 function totalDurationMs(runCase) {
     return runCase.attempts.reduce((sum, attempt) => sum + (attempt.durationMs ?? 0), 0);
 }

package/dist/runners/vitest-adapter.js

@@ -31,11 +31,15 @@ export function createVitestAdapter(options = {}) {
                 native: {
                     testModules: options.testModules ?? [],
                     cwd: discoveryCwd,
+                    discoveredFiles: input.discovered.units.flatMap(unit => unit.sourceRef ? [unit.sourceRef] : []),
                 },
             };
         },
         async collectNormalizedRunSnapshot(run) {
-            return buildNormalizedRunSnapshotFromReportGroups(run, collectVitestReportGroups(readVitestRunNative(run).testModules), { cwd: readVitestRunNative(run).cwd });
+            return buildNormalizedRunSnapshotFromReportGroups(run, collectVitestReportGroups(readVitestRunNative(run).testModules), {
+                cwd: readVitestRunNative(run).cwd,
+                discoveredFiles: readVitestRunNative(run).discoveredFiles,
+            });
         },
     };
 }
@@ -59,7 +63,7 @@ export function collectVitestReportGroups(testModules) {
 function readVitestRunNative(run) {
     if (isVitestRunNative(run.native))
         return run.native;
-    return { testModules: [], cwd: process.cwd() };
+    return { testModules: [], cwd: process.cwd(), discoveredFiles: [] };
 }
 function isVitestRunNative(native) {
     return typeof native === 'object'

package/dist/sdk/scorers.d.ts

@@ -1,15 +1,18 @@
 import type { CheckScorer, CodeJudgeToolName, ScorerContext, JudgeScorer, ScoreScorer, ScoreResult, ToolExpectation, ToolUsageScorer } from './types.js';
+import type { JsonValue } from '../internal/direct-mcp-v2/types.js';
 /**
  * Create a check scorer — a boolean gate that passes (1.0) or fails (0.0).
  */
 export declare function check(name: string, fn: (ctx: ScorerContext) => boolean | Promise<boolean>, opts?: {
     weight?: number;
+    revision?: JsonValue;
 }): CheckScorer;
 /**
  * Create a score scorer — returns a number (0-1) or { score, details } for partial credit.
  */
 export declare function score(name: string, fn: (ctx: ScorerContext) => number | ScoreResult | Promise<number | ScoreResult>, opts?: {
     weight?: number;
+    revision?: JsonValue;
 }): ScoreScorer;
 /**
  * Create a judge scorer — uses an LLM to evaluate against a rubric.
@@ -55,10 +58,12 @@ export declare function judge(name: string, opts: {
     maxRounds?: number;
     /** Anthropic prompt caching. Default: true when tools is set, unchanged otherwise. */
     cacheControl?: boolean;
+    revision?: JsonValue;
 }): JudgeScorer;
 /**
  * Create a tool usage scorer — matches tool events against expectations.
  */
 export declare function toolUsage(name: string, expectations: ToolExpectation[], opts?: {
     weight?: number;
+    revision?: JsonValue;
 }): ToolUsageScorer;

package/dist/sdk/scorers.js

@@ -7,6 +7,7 @@ export function check(name, fn, opts) {
         name,
         weight: opts?.weight ?? 1,
         fn,
+        ...(opts?.revision === undefined ? {} : { revision: opts.revision }),
     };
 }
 /**
@@ -18,6 +19,7 @@ export function score(name, fn, opts) {
         name,
         weight: opts?.weight ?? 1,
         fn,
+        ...(opts?.revision === undefined ? {} : { revision: opts.revision }),
     };
 }
 /**
@@ -65,6 +67,7 @@ export function judge(name, opts) {
         tools,
         maxRounds: opts.maxRounds,
         cacheControl,
+        ...(opts.revision === undefined ? {} : { revision: opts.revision }),
     };
 }
 /**
@@ -76,5 +79,6 @@ export function toolUsage(name, expectations, opts) {
         name,
         weight: opts?.weight ?? 1,
         expectations,
+        ...(opts?.revision === undefined ? {} : { revision: opts.revision }),
     };
 }

package/dist/types.d.ts

@@ -207,7 +207,7 @@ export interface EvalReport {
     skills_used: string[];
 }
 export type ComparisonUnavailableReason = 'definition_metadata_missing' | 'scorer_metadata_missing' | 'runtime_metadata_missing' | 'sampling_incomplete';
-export interface ComparisonContract {
+export interface ComparisonContractV1 {
     version: 1;
     definition_revision?: string;
     scorer_revision?: string;
@@ -216,6 +216,7 @@ export interface ComparisonContract {
     report_schema_revision: 'pathgrade-results-v2';
     unavailable_reasons?: ComparisonUnavailableReason[];
 }
+export type ComparisonContract = ComparisonContractV1 | import('./reporting/reliability-contract.js').ComparisonContractV2;
 /**
  * TrialResult with large trace fields stripped. These fields
  * live only in the per-group trace files; the consolidated results.json keeps
@@ -260,7 +261,7 @@ export interface PathgradeSelectionReport {
  * external consumers can gate on schema revisions.
  */
 export interface PathgradeReport {
-    version: 1 | 2;
+    version: 1 | 2 | 3;
     timestamp: string;
     /** `ci.threshold` from Pathgrade config or legacy plugin config, if configured. */
     threshold?: number;
@@ -279,6 +280,8 @@ export interface PathgradeReport {
     /** Agent Evals compatibility manifest when changed selection finds no runnable evals. */
     run_kind?: 'evaluation' | 'no-affected';
     groups: PathgradeGroupReport[];
+    /** Current selected-task accounting, required for schema-v3 reports. */
+    task_inventory?: import('./reporting/reliability-contract.js').TaskInventory;
     /**
      * Present when `pathgrade run --changed` produced the run. Absent on
      * plain `pathgrade run`. Backward compatible — older consumers ignore

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@wix/pathgrade",
-  "version": "1.0.39",
+  "version": "1.0.40",
   "packageManager": "yarn@4.12.0",
   "description": "Evaluate whether AI agents discover and use your skills correctly",
   "exports": {
@@ -142,5 +142,5 @@
     "typescript": "^5.9.3",
     "zod": "4.3.6"
   },
-  "falconPackageHash": "df08d3b446a37c3224730fca89a408f6238eef420358705733692eee"
+  "falconPackageHash": "4c4d43a7c8d439f02756404accbc72b44db02c52db4d0c2b2d18452f"
 }