@openpond/evals 0.2.0 → 0.4.0

package/CONTRACT.md

@@ -1,21 +1,29 @@
 # Portable contract map
 
-`@openpond/evals` owns the portable wire formats and pure protocol logic. Host
-applications own persistence, provider sessions, secret leases, connected-app
-authorization, model streaming, artifact bytes, and runtime processes.
+`@openpond/harness` owns immutable Harness identities, workspaces,
+model-driven refinement/review policy, improvements, traces, tools, model
+identities, and shared hashing.
+`@openpond/evals` owns Tasksets, graders, evaluation runs and receipts,
+execution orchestration, conformance fixtures, and Work-evidence eligibility.
+Evals depends on Harness for exact release identities but does not re-export
+Harness APIs; Harness never depends on Evals. Host applications own persistence, provider sessions, secret
+leases, connected-app authorization, model streaming, artifact bytes, and
+runtime processes.
 
 | Existing object | Portable object | Migration rule |
 | --- | --- | --- |
-| `Taskset` | `TasksetRelease` | Project only the released tasks, policy, environment, tools, capabilities, graders, and immutable Harness binding. Authoring state and UI readiness remain host state. |
+| `Taskset` | `TasksetRelease` | Project only the released tasks, policy, environment, tools, capabilities, and graders. A Taskset is deliberately independent of any Harness so the same workload can run against local or hosted execution. Authoring state and UI readiness remain host state. |
 | `HarnessRunManifest` (`openpond.harnessRunManifest.v1`) | `RunManifest` (`openpond.runManifest.v1`) | Treat the old object as a legacy training projection. Normalize its release/model/runtime identities into one new manifest; recipe, compute, engine, secret leases, and approval records remain host bindings referenced by hashes. |
 | `HarnessRunTrace` | `HarnessTrace` | Preserve ordered actions, observations, lifecycle events, terminal state, failure class, and trace hash. Learning-signal envelopes remain a training projection of the trace and receipt. |
 | `TaskAttemptResult` | `AttemptReceipt` | Preserve the old record for application persistence while adding a lossless receipt reference. Output becomes `outputHash`; trace and artifacts are separately hash-bound. |
 | `GradeResult` | `GraderEvidence[]` | Preserve component score, pass, reward eligibility, failure class, feedback, and visible/private evidence references. Aggregate UI results remain host projections. |
 | managed-RL local receipt | `AttemptReceipt` | Submit canonical manifest/task/trace/artifact/grader identities; policy token responses and provider request IDs remain host-private trace data. |
-| resolved training bundle | `HarnessRelease` + host training binding | Environment, tools, program, policy, files, and grader interface belong to the Harness. Dataset/evidence, recipe, compute, engine, approval, and opaque leases remain explicit host-side bindings. |
+| resolved training bundle | `HarnessRelease` + host training binding | Agent snapshot, program, lifecycle, tool declarations, files, and grader interface belong to the Harness. Taskset environment/policy/graders and dataset/evidence, recipe, compute, engine, approval, and opaque leases remain explicit host-side bindings. |
 | completed Work or Development turn | `WorkEvidenceReceipt` | Project the authoritative terminal turn, immutable Agent snapshot when available, model/runtime identity, sanitized trace reference, exact output revisions, validation evidence, interventions, timing, usage, and explicit consent provenance. Keep the raw source and trace host-private. |
 | Agent plus environment runtime events | `WorkProcessTrace` | Emit one ordered trace with `agent` and `environment` layers. Bind every environment step to its outer Agent tool call or stable Agent-turn receipt hash. Hash inputs/outputs and expose only enumerated, bounded attributes. |
 | user feedback on Work output | `WorkFeedbackReceipt` | Append a new receipt bound to the evidence receipt and, when selected, the exact content-addressed output-revision descriptor. Corrections are separate artifacts and never mutate prior receipts. |
+| bounded cross-Work review | `HarnessEvaluationReviewReceipt` | Select only currently authorized immutable evidence, advance one watermark, group one stable claim, route to the smallest correct layer, and name the next authority without performing downstream effects. |
+| model-improvement qualification | `ModelImprovementQualificationReceipt` | Bind the originating review, exact Harness, Taskset, real baseline Evaluation, Model, Environment/tool/permission/policy hashes, Verifier, source policies, privacy, budget, and non-frozen signal. Weak or confounded evidence emits `no_training`; training and activation remain host effects. |
 
 ## Compatibility policy
 
@@ -26,6 +34,17 @@ authorization, model streaming, artifact bytes, and runtime processes.
 - Compatible package releases may add optional helpers and exports. Changing a
   required field, identity hash, or privacy boundary requires a new schema
   literal and an explicit normalizer.
+- `openpond.agentSnapshot.v2`, `openpond.harnessRelease.v2`, and
+  `openpond.tasksetRelease.v2` define the Harness-first boundary. The v2
+  contracts remove the Profile reference from the Agent snapshot and keep the
+  Taskset independent of a concrete Harness, environment, or policy binding.
+- Runs with different Harness releases require an explicit
+  `HarnessCompatibilityReceipt` binding both Harnesses to the same Taskset and
+  recording environment, tool, policy, and grader-interface contract hashes.
+  Callers with materialized releases should use
+  `createVerifiedHarnessCompatibilityReceipt`; it derives those hashes from the
+  immutable objects and rejects lifecycle, tool, grader-interface, or required
+  Environment-tool drift before issuing the receipt.
 - The initial support target is Node.js ESM on Node 22.14 through Node 24.
 - Portable paths are relative and at most 2,000 characters. Individual assets
   are at most 250 MB. Tasksets, traces, and evidence arrays have schema-level

package/README.md

@@ -1,18 +1,24 @@
 # `@openpond/evals`
 
-Portable OpenPond contracts and reference helpers for immutable Agent snapshots,
-Harness releases, Taskset releases, run manifests, attempt receipts, observational
-Work evidence, deterministic graders, and conformance fixtures.
+Portable evaluation contracts and pure helpers for Tasksets, graders, run
+manifests, attempt and evaluation receipts, execution adapters, conformance
+fixtures, Work-evidence eligibility, and no-training/SFT/preference/RL
+qualification receipts. The package depends on
+[`@openpond/harness`](../harness/README.md) for exact Harness identities but
+does not re-export Harness APIs. Applications import the two packages directly,
+which keeps refinement and evaluation authority visibly separate.
 
 ```ts
 import {
   AttemptReceiptSchema,
-  HarnessReleaseSchema,
   TasksetReleaseSchema,
+  ModelImprovementQualificationReceiptSchema,
   validateTasksetRelease,
   verifyAttemptReceipt,
 } from "@openpond/evals";
 
+import { HarnessReleaseSchema } from "@openpond/harness";
+
 import {
   WorkEvidenceReceiptSchema,
   classifyWorkEvidence,
@@ -21,9 +27,22 @@ import {
 ```
 
 Subpath exports are available at `/harness`, `/tasksets`, `/graders`, `/runs`,
-`/conformance`, and `/evidence`. The package is a protocol library, not a hosted client. It
-does not execute OpenPond Desktop or Sandbox sessions, resolve credentials, or
-persist artifacts.
+`/conformance`, `/evidence`, `/review`, and
+`/model-improvement-qualification`. The package is an evaluation protocol library,
+not a hosted client. It does not execute OpenPond Desktop or Sandbox sessions,
+resolve credentials, or persist artifacts.
+
+## Harness Evaluation review
+
+`@openpond/harness` owns the public model-driven Refiner and continuous-review
+policy plus immutable bounded decisions and receipts. Evals does not own
+learning prompts, evidence selection, scheduling, or Harness mutation. It
+binds an accepted review to the exact Harness, Taskset, scored
+baseline Evaluation, base Model, Environment/tool/permission/policy hashes,
+Verifier, source-policy checks, privacy approval, budget approval, and eligible
+non-frozen learning signal. Frozen Evaluation evidence is rejected as training
+evidence. Weak, constant, confounded, uncalibrated, revoked, or incompletely
+authorized signal produces `no_training`; it cannot qualify a managed run.
 
 ## Work evidence
 
@@ -66,7 +85,7 @@ overrides that state.
 Implement `HarnessRuntime` for environment state and `HarnessExecutor` around
 the host's existing model/Agent loop. Do not reimplement prompting, tool dispatch,
 session persistence, cost accounting, cancellation, or cleanup in this package.
-A Prime-style environment maps reset/step/observation semantics to the same
+A step-based environment maps reset/step/observation semantics to the same
 interfaces and keeps provider allocation, authentication, and cleanup in its host.
 
 ## Conformance

package/RELEASING.md

@@ -1,54 +1,22 @@
 # Releasing `@openpond/evals`
 
-The package version is independent from OpenPond application and schema
-versions. Version `0.1.1` is the current provenance-backed baseline and supports
-these initial schema literals:
-
-- `openpond.agentSnapshot.v1`
-- `openpond.harnessRelease.v1`
-- `openpond.tasksetRelease.v1`
-- `openpond.runManifest.v1`
-- `openpond.attemptReceipt.v1`
-- `openpond.harnessTrace.v1`
-- `openpond.graderEvidence.v1`
-- `openpond.evaluationResult.v1`
-
-This branch prepares the additive Work evidence schemas for `0.2.0`:
-
-- `openpond.workEvidenceReceipt.v1`
-- `openpond.workProcessTrace.v1`
-- `openpond.workFeedbackReceipt.v1`
-- `openpond.workEvidenceEligibility.v1`
-
-## Trusted publishing
-
-The package already exists on npm and the `npm-production` environment is
-configured for trusted publishing through `release-evals.yml`. Do not run a
-manual `npm publish` or repeat the original bootstrap. The workflow uses its own
-pinned npm version for trusted publication and provenance.
-
-To inspect an already-published version:
+`@openpond/evals` is the public evaluation package for OpenPond Harnesses. It
+depends on `@openpond/harness` for exact Harness identities but exposes only
+evaluation APIs. Publish the matching Harness peer before Evals so npm can
+satisfy the declared range.
 
 ```bash
 pnpm evals:check
-npm trust list @openpond/evals
-npm view @openpond/evals version dist.integrity dist.attestations
-```
-
-## Later releases
-
-For releases that do not already carry an intentional version bump, merge
-feature work without changing the package version. When the intended package
-changes are on `master`, prepare a separate release from a clean current
-`master` checkout:
-
-```bash
 pnpm release:evals:patch
 # or release:evals:minor / release:evals:major
 ```
 
-The release helper creates an Evals-only release PR and updates the package
-version and lockfile. Merging it triggers trusted publishing with provenance,
-registry verification, and a package-specific tag. Record the registry
-integrity/provenance evidence and merge commit in the release notes after
-publication.
+Merging a release PR triggers `release-evals.yml`, trusted npm publishing,
+registry verification, provenance verification, and the package-specific tag.
+Release notes must call out removal or movement of a public subpath and use the
+appropriate semver increment.
+
+Version `0.4.0` removes the retired Harness root barrel and
+`./harness-improvements` / `./harness-workspaces` compatibility subpaths.
+Consumers must import those public contracts from `@openpond/harness`; this is
+an intentional breaking package-boundary correction.

package/dist/compatibility.js

@@ -0,0 +1,45 @@
+import { assertContentHash, contentHash } from "@openpond/harness";
+import { HarnessReleaseSchema, } from "@openpond/harness";
+import { createHarnessCompatibilityReceipt, } from "./runs.js";
+import { TasksetReleaseSchema, } from "./tasksets.js";
+export function createVerifiedHarnessCompatibilityReceipt(input) {
+    const base = HarnessReleaseSchema.parse(input.baseHarnessRelease);
+    const candidate = HarnessReleaseSchema.parse(input.candidateHarnessRelease);
+    const taskset = TasksetReleaseSchema.parse(input.tasksetRelease);
+    assertContentHash(base, "Base Harness release");
+    assertContentHash(candidate, "Candidate Harness release");
+    assertContentHash(taskset, "Taskset release");
+    requireSameContract("lifecycle", base.lifecycle, candidate.lifecycle);
+    requireSameContract("tool", base.tools, candidate.tools);
+    requireSameContract("grader interface", base.graderInterface, candidate.graderInterface);
+    const environmentTools = new Set(taskset.tools.map((tool) => tool.name));
+    const unsupportedTools = base.tools
+        .map((tool) => tool.name)
+        .filter((name) => !environmentTools.has(name));
+    if (unsupportedTools.length) {
+        throw new Error(`Harness compatibility failed: Taskset Environment does not provide ${unsupportedTools.join(", ")}.`);
+    }
+    return createHarnessCompatibilityReceipt({
+        schemaVersion: "openpond.harnessCompatibility.v1",
+        id: input.id,
+        baseHarnessRelease: { id: base.id, contentHash: base.contentHash },
+        candidateHarnessRelease: {
+            id: candidate.id,
+            contentHash: candidate.contentHash,
+        },
+        tasksetRelease: { id: taskset.id, contentHash: taskset.contentHash },
+        environmentHash: contentHash(taskset.environment),
+        toolContractHash: contentHash(taskset.tools),
+        policyHash: contentHash(taskset.policy),
+        graderInterfaceHash: contentHash({
+            harness: base.graderInterface,
+            taskset: taskset.graders,
+        }),
+        metadata: input.metadata ?? {},
+    });
+}
+function requireSameContract(label, base, candidate) {
+    if (contentHash(base) !== contentHash(candidate)) {
+        throw new Error(`Harness compatibility failed: ${label} contract changed.`);
+    }
+}

package/dist/conformance.js

@@ -1,5 +1,4 @@
-import { contentHash, withContentHash } from "./common.js";
-import { createAgentSnapshot, createHarnessRelease } from "./harness.js";
+import { contentHash, createAgentSnapshot, createHarnessRelease, withContentHash, } from "@openpond/harness";
 import { createRunManifest } from "./runs.js";
 import { TasksetReleaseSchema } from "./tasksets.js";
 const EMPTY_HASH = contentHash("");
@@ -14,32 +13,29 @@ export const marketingPortfolioConformance = fixture("marketing-portfolio-v1", [
 ]);
 function fixture(id, tools) {
     const snapshot = createAgentSnapshot({
-        schemaVersion: "openpond.agentSnapshot.v1",
+        schemaVersion: "openpond.agentSnapshot.v2",
         id: `${id}-agent`,
-        profileRelease: null,
+        sourceRelease: null,
         instructions: [], skills: [], agents: [], toolDeclarations: tools,
         capabilityRequirements: [], dependencyLock,
         portability: { portable: true, blockers: [], localOnlyAssetRefs: [], hostPrivateAssetRefs: [] },
         metadata: { conformanceFixture: id },
     });
     const harness = createHarnessRelease({
-        schemaVersion: "openpond.harnessRelease.v1",
+        schemaVersion: "openpond.harnessRelease.v2",
         id: `${id}-harness`,
         agentSnapshot: { id: snapshot.id, contentHash: snapshot.contentHash },
         program,
-        environment: { protocolVersion: "openpond.environment.v1", kind: "agent", entrypoint: id, stateful: true, deterministicSeeds: true, lifecycle: ["create", "reset", "step", "collect", "destroy"], networkPolicy: "none", defaultTimeoutMs: 5_000 },
         tools,
         lifecycle: { create: true, reset: true, step: true, collect: true, destroy: true, resetScope: "attempt" },
         graderInterface: { visibleEvidence: ["output"], privilegedEvidence: ["expected"], privateVerifierIsolation: true },
-        policy: { policyVisibleFields: ["input"], privilegedFields: ["expectedOutput"], hiddenGraderRefs: [], connectedAppScopes: [] },
         files: [], metadata: { conformanceFixture: id },
     });
     const tasksetContent = {
-        schemaVersion: "openpond.tasksetRelease.v1",
+        schemaVersion: "openpond.tasksetRelease.v2",
         id: `${id}-taskset`, revision: 1,
-        harnessRelease: { id: harness.id, contentHash: harness.contentHash },
-        policy: harness.policy,
-        environment: harness.environment,
+        policy: { policyVisibleFields: ["input"], privilegedFields: ["expectedOutput"], hiddenGraderRefs: [], connectedAppScopes: [] },
+        environment: { protocolVersion: "openpond.environment.v1", kind: "agent", entrypoint: id, stateful: true, deterministicSeeds: true, lifecycle: ["create", "reset", "step", "collect", "destroy"], networkPolicy: "none", defaultTimeoutMs: 5_000 },
         tools,
         capabilities: [],
         tasks: [

package/dist/evidence/authoring.js

@@ -1,5 +1,5 @@
 import { z } from "zod";
-import { ImmutableReleaseRefSchema, ReleaseHashSchema } from "../common.js";
+import { ImmutableReleaseRefSchema, ReleaseHashSchema } from "@openpond/harness";
 import { EvidenceArtifactRefSchema, WorkEvidenceReceiptSchema, evidenceArtifactRef, } from "./contracts.js";
 import { WorkEvidenceEligibilitySchema } from "./eligibility.js";
 export const WorkEvidenceAuthoringInputSchema = z.object({

package/dist/evidence/conformance.js

@@ -1,4 +1,4 @@
-import { canonicalJson, contentHash } from "../common.js";
+import { canonicalJson, contentHash } from "@openpond/harness";
 import { createWorkEvidenceReceipt, createWorkFeedbackReceipt, createWorkProcessTrace, evidenceArtifactRef, workEvidenceReceiptRef, workSourceOpaqueRef, } from "./contracts.js";
 import { classifyWorkEvidence } from "./eligibility.js";
 const startedAt = "2026-08-04T12:00:00.000Z";

package/dist/evidence/contracts.js

@@ -1,6 +1,5 @@
 import { z } from "zod";
-import { ImmutableArtifactRefSchema, ImmutableReleaseRefSchema, ReleaseHashSchema, ReleaseIdSchema, ReleaseTimestampSchema, contentHash, } from "../common.js";
-import { ModelRefSchema } from "../runs.js";
+import { ImmutableArtifactRefSchema, ImmutableReleaseRefSchema, ModelRefSchema, ReleaseHashSchema, ReleaseIdSchema, ReleaseTimestampSchema, contentHash, } from "@openpond/harness";
 export const WORK_EVIDENCE_SCHEMA_VERSION = "openpond.workEvidenceReceipt.v1";
 export const WORK_PROCESS_TRACE_SCHEMA_VERSION = "openpond.workProcessTrace.v1";
 export const WORK_FEEDBACK_SCHEMA_VERSION = "openpond.workFeedbackReceipt.v1";

package/dist/evidence/eligibility.js

@@ -1,5 +1,5 @@
 import { z } from "zod";
-import { contentHash, ReleaseHashSchema } from "../common.js";
+import { contentHash, ReleaseHashSchema } from "@openpond/harness";
 import { AttemptReceiptSchema, verifyAttemptReceipt } from "../runs.js";
 import { WORK_EVIDENCE_ELIGIBILITY_SCHEMA_VERSION, WorkFeedbackReceiptSchema, createWorkFeedbackReceipt, verifyWorkEvidenceReceipt, } from "./contracts.js";
 export const WorkEvidencePolicyStateSchema = z.enum([

package/dist/graders.js

@@ -1,5 +1,5 @@
 import { z } from "zod";
-import { FailureClassSchema, ReleaseHashSchema, ReleaseIdSchema, contentHash } from "./common.js";
+import { FailureClassSchema, ReleaseHashSchema, ReleaseIdSchema, contentHash } from "@openpond/harness";
 export const GraderEvidenceContentSchema = z.object({
     schemaVersion: z.literal("openpond.graderEvidence.v1"),
     graderId: ReleaseIdSchema,

package/dist/harness.js

@@ -1,73 +1,6 @@
-import { z } from "zod";
-import { FailureClassSchema, ImmutableArtifactRefSchema, ImmutableAssetRefSchema, ImmutableReleaseRefSchema, MetadataSchema, ReleaseHashSchema, ReleaseIdSchema, contentHash, } from "./common.js";
-import { EnvironmentContractSchema, PolicyBoundarySchema, ToolDeclarationSchema } from "./tasksets.js";
-import { createAttemptReceipt } from "./runs.js";
-export const PortabilityReportSchema = z.object({
-    portable: z.boolean(),
-    blockers: z.array(z.string().trim().min(1).max(2_000)).max(1_000),
-    localOnlyAssetRefs: z.array(ReleaseIdSchema).max(10_000),
-    hostPrivateAssetRefs: z.array(ReleaseIdSchema).max(10_000),
-}).strict();
-export const AgentSnapshotContentSchema = z.object({
-    schemaVersion: z.literal("openpond.agentSnapshot.v1"),
-    id: ReleaseIdSchema,
-    profileRelease: ImmutableReleaseRefSchema.nullable(),
-    instructions: z.array(ImmutableAssetRefSchema).max(10_000),
-    skills: z.array(ImmutableAssetRefSchema).max(10_000),
-    agents: z.array(ImmutableAssetRefSchema).max(10_000),
-    toolDeclarations: z.array(ToolDeclarationSchema).max(200),
-    capabilityRequirements: z.array(z.object({
-        id: ReleaseIdSchema,
-        required: z.boolean(),
-        scopes: z.array(z.string().trim().min(1).max(500)).max(100),
-    }).strict()).max(200),
-    dependencyLock: ImmutableAssetRefSchema,
-    portability: PortabilityReportSchema,
-    metadata: MetadataSchema,
-}).strict();
-export const AgentSnapshotSchema = AgentSnapshotContentSchema.extend({ contentHash: ReleaseHashSchema }).strict();
-export const LifecycleContractSchema = z.object({
-    create: z.literal(true),
-    reset: z.literal(true),
-    step: z.literal(true),
-    collect: z.literal(true),
-    destroy: z.literal(true),
-    resetScope: z.enum(["task", "attempt"]),
-}).strict();
-export const GraderInterfaceContractSchema = z.object({
-    visibleEvidence: z.array(ReleaseIdSchema).max(1_000),
-    privilegedEvidence: z.array(ReleaseIdSchema).max(1_000),
-    privateVerifierIsolation: z.boolean(),
-}).strict();
-export const HarnessReleaseContentSchema = z.object({
-    schemaVersion: z.literal("openpond.harnessRelease.v1"),
-    id: ReleaseIdSchema,
-    agentSnapshot: ImmutableReleaseRefSchema,
-    program: ImmutableAssetRefSchema,
-    environment: EnvironmentContractSchema,
-    tools: z.array(ToolDeclarationSchema).max(200),
-    lifecycle: LifecycleContractSchema,
-    graderInterface: GraderInterfaceContractSchema,
-    policy: PolicyBoundarySchema,
-    files: z.array(ImmutableAssetRefSchema).max(100_000),
-    metadata: MetadataSchema,
-}).strict();
-export const HarnessReleaseSchema = HarnessReleaseContentSchema.extend({ contentHash: ReleaseHashSchema }).strict();
-export const ModelActionSchema = z.object({
-    id: ReleaseIdSchema,
-    turn: z.number().int().nonnegative(),
-    kind: z.enum(["message", "tool_call", "terminal"]),
-    name: ReleaseIdSchema.nullable(),
-    arguments: z.record(z.string(), z.unknown()),
-    content: z.string().max(1_000_000).nullable(),
-}).strict();
-export const ToolObservationSchema = z.object({
-    actionId: ReleaseIdSchema,
-    turn: z.number().int().nonnegative(),
-    terminal: z.boolean(),
-    output: z.record(z.string(), z.unknown()),
-    artifactRefs: z.array(ImmutableArtifactRefSchema).max(100_000),
-}).strict();
+import { HarnessTraceSchema, ModelActionSchema, ToolObservationSchema, contentHash, } from "@openpond/harness";
+import { createAttemptReceipt, } from "./runs.js";
+export * from "@openpond/harness/harness";
 export async function executeRuntimeProtocol(input) {
     const now = input.now ?? (() => new Date().toISOString());
     const startedAt = now();
@@ -86,12 +19,25 @@ export async function executeRuntimeProtocol(input) {
     let output = {};
     let artifactRefs = [];
     const event = (type, payload, metadata = {}) => {
-        events.push({ sequence: sequence++, type, payloadHash: contentHash(payload), metadata });
+        events.push({
+            sequence: sequence++,
+            type,
+            payloadHash: contentHash(payload),
+            metadata,
+        });
     };
     try {
-        lease = await input.runtime.create({ manifest: input.manifest, taskId: input.taskId, seed: input.seed, signal: controller.signal });
+        lease = await input.runtime.create({
+            manifest: input.manifest,
+            taskId: input.taskId,
+            seed: input.seed,
+            signal: controller.signal,
+        });
         event("created", { leaseId: lease.id });
-        await input.runtime.reset(lease, { seed: input.seed, signal: controller.signal });
+        await input.runtime.reset(lease, {
+            seed: input.seed,
+            signal: controller.signal,
+        });
         event("reset", { seed: input.seed });
         for (const candidate of input.actions.slice(0, input.manifest.limits.maxTurns)) {
             if (controller.signal.aborted)
@@ -154,12 +100,23 @@ export async function executeRuntimeProtocol(input) {
         failureClass,
         output,
     };
-    const trace = HarnessTraceSchema.parse({ ...traceContent, contentHash: contentHash(traceContent) });
+    const trace = HarnessTraceSchema.parse({
+        ...traceContent,
+        contentHash: contentHash(traceContent),
+    });
     const completedAt = now();
     const receipt = createAttemptReceipt({
         schemaVersion: "openpond.attemptReceipt.v1",
-        id: `attempt-${contentHash([input.manifest.contentHash, input.taskId, input.seed, trace.contentHash]).slice(0, 24)}`,
-        runManifest: { id: input.manifest.id, contentHash: input.manifest.contentHash },
+        id: `attempt-${contentHash([
+            input.manifest.contentHash,
+            input.taskId,
+            input.seed,
+            trace.contentHash,
+        ]).slice(0, 24)}`,
+        runManifest: {
+            id: input.manifest.id,
+            contentHash: input.manifest.contentHash,
+        },
         taskId: input.taskId,
         seed: input.seed,
         terminal,
@@ -177,30 +134,3 @@ export async function executeRuntimeProtocol(input) {
     });
     return { receipt, trace, output };
 }
-export function createAgentSnapshot(input) {
-    const content = AgentSnapshotContentSchema.parse(input);
-    return AgentSnapshotSchema.parse({ ...content, contentHash: contentHash(content) });
-}
-export function createHarnessRelease(input) {
-    const content = HarnessReleaseContentSchema.parse(input);
-    return HarnessReleaseSchema.parse({ ...content, contentHash: contentHash(content) });
-}
-export const HarnessLifecycleEventSchema = z.object({
-    sequence: z.number().int().nonnegative(),
-    type: z.enum(["created", "reset", "action", "observation", "terminal", "failure", "collected", "destroyed"]),
-    payloadHash: ReleaseHashSchema,
-    metadata: MetadataSchema,
-}).strict();
-export const HarnessTraceSchema = z.object({
-    schemaVersion: z.literal("openpond.harnessTrace.v1"),
-    manifest: ImmutableReleaseRefSchema,
-    taskId: ReleaseIdSchema,
-    seed: z.string().trim().min(1).max(500),
-    events: z.array(HarnessLifecycleEventSchema).max(1_000_000),
-    actions: z.array(ModelActionSchema).max(100_000),
-    observations: z.array(ToolObservationSchema).max(100_000),
-    terminal: z.boolean(),
-    failureClass: FailureClassSchema.nullable(),
-    output: z.record(z.string(), z.unknown()),
-    contentHash: ReleaseHashSchema,
-}).strict();

package/dist/index.js

@@ -1,6 +1,8 @@
-export * from "./common.js";
+export * from "./compatibility.js";
 export * from "./evidence/index.js";
 export * from "./graders.js";
 export * from "./harness.js";
 export * from "./runs.js";
+export * from "./model-improvement-qualification.js";
+export * from "./review-conformance.js";
 export * from "./tasksets.js";

package/dist/model-improvement-qualification.js

@@ -0,0 +1,125 @@
+import { z } from "zod";
+import { HarnessReviewSourcePolicyRefSchema, ImmutableReleaseRefSchema, MetadataSchema, ModelRefSchema, ReleaseHashSchema, ReleaseIdSchema, ReleaseTimestampSchema, contentHash, } from "@openpond/harness";
+const BoundedTextSchema = z.string().trim().min(1).max(100_000);
+export const ModelImprovementDecisionSchema = z.enum([
+    "no_training",
+    "sft",
+    "preference",
+    "rl",
+]);
+export const ModelImprovementSignalSchema = z
+    .object({
+    kind: z.enum(["none", "demonstrations", "chosen_rejected", "scalar_reward"]),
+    strength: z.enum(["absent", "weak", "usable"]),
+    calibrated: z.boolean(),
+    confounded: z.boolean(),
+    variance: z.number().finite().nonnegative().nullable(),
+    evidenceRefs: z.array(ImmutableReleaseRefSchema).max(100_000),
+})
+    .strict();
+export const ModelImprovementQualificationReceiptContentSchema = z
+    .object({
+    schemaVersion: z.literal("openpond.modelImprovementQualificationReceipt.v1"),
+    id: ReleaseIdSchema,
+    review: ImmutableReleaseRefSchema,
+    harnessRelease: ImmutableReleaseRefSchema,
+    tasksetRelease: ImmutableReleaseRefSchema.nullable(),
+    baselineEvaluation: ImmutableReleaseRefSchema.nullable(),
+    model: ModelRefSchema,
+    environmentHash: ReleaseHashSchema.nullable(),
+    toolContractHash: ReleaseHashSchema.nullable(),
+    permissionContractHash: ReleaseHashSchema.nullable(),
+    policyHash: ReleaseHashSchema.nullable(),
+    verifierRef: ImmutableReleaseRefSchema.nullable(),
+    sourcePolicies: z.array(HarnessReviewSourcePolicyRefSchema).max(10_000),
+    trainingEvidenceRefs: z.array(ImmutableReleaseRefSchema).max(100_000),
+    frozenEvaluationEvidenceRefs: z.array(ImmutableReleaseRefSchema).max(100_000),
+    privacyApproval: ImmutableReleaseRefSchema.nullable(),
+    budgetApproval: ImmutableReleaseRefSchema.nullable(),
+    maximumCostUsd: z.number().finite().nonnegative(),
+    signal: ModelImprovementSignalSchema,
+    decision: ModelImprovementDecisionSchema,
+    reasons: z.array(BoundedTextSchema).min(1).max(100),
+    createdAt: ReleaseTimestampSchema,
+    metadata: MetadataSchema,
+})
+    .strict()
+    .superRefine((receipt, context) => {
+    const trainingKeys = new Set(receipt.trainingEvidenceRefs.map(refKey));
+    if (receipt.frozenEvaluationEvidenceRefs.some((reference) => trainingKeys.has(refKey(reference)))) {
+        context.addIssue({
+            code: "custom",
+            message: "frozen Evaluation evidence cannot be used as training evidence",
+            path: ["frozenEvaluationEvidenceRefs"],
+        });
+    }
+    if (receipt.decision === "no_training")
+        return;
+    const missingGate = !receipt.tasksetRelease ||
+        !receipt.baselineEvaluation ||
+        !receipt.environmentHash ||
+        !receipt.toolContractHash ||
+        !receipt.permissionContractHash ||
+        !receipt.policyHash ||
+        !receipt.verifierRef ||
+        !receipt.privacyApproval ||
+        !receipt.budgetApproval ||
+        receipt.sourcePolicies.length === 0 ||
+        receipt.sourcePolicies.some((policy) => policy.state !== "authorized") ||
+        receipt.trainingEvidenceRefs.length === 0 ||
+        receipt.signal.strength !== "usable" ||
+        !receipt.signal.calibrated ||
+        receipt.signal.confounded;
+    if (missingGate) {
+        context.addIssue({
+            code: "custom",
+            message: "qualified model improvement requires frozen lineage, authorized signal, privacy, and budget gates",
+        });
+    }
+    if (receipt.decision === "sft" &&
+        receipt.signal.kind !== "demonstrations") {
+        context.addIssue({
+            code: "custom",
+            message: "SFT qualification requires demonstration signal",
+            path: ["signal", "kind"],
+        });
+    }
+    if (receipt.decision === "preference" &&
+        receipt.signal.kind !== "chosen_rejected") {
+        context.addIssue({
+            code: "custom",
+            message: "preference qualification requires chosen/rejected signal",
+            path: ["signal", "kind"],
+        });
+    }
+    if (receipt.decision === "rl" &&
+        (receipt.signal.kind !== "scalar_reward" ||
+            receipt.signal.variance === null ||
+            receipt.signal.variance <= 0)) {
+        context.addIssue({
+            code: "custom",
+            message: "RL qualification requires a usable scalar reward with variance",
+            path: ["signal"],
+        });
+    }
+});
+export const ModelImprovementQualificationReceiptSchema = ModelImprovementQualificationReceiptContentSchema.extend({
+    contentHash: ReleaseHashSchema,
+}).strict();
+export function createModelImprovementQualificationReceipt(input) {
+    const content = ModelImprovementQualificationReceiptContentSchema.parse(input);
+    return ModelImprovementQualificationReceiptSchema.parse({
+        ...content,
+        contentHash: contentHash(content),
+    });
+}
+export function verifyModelImprovementQualificationReceipt(value) {
+    const parsed = ModelImprovementQualificationReceiptSchema.safeParse(value);
+    if (!parsed.success)
+        return false;
+    const { contentHash: actual, ...content } = parsed.data;
+    return (contentHash(ModelImprovementQualificationReceiptContentSchema.parse(content)) === actual);
+}
+function refKey(reference) {
+    return `${reference.id}:${reference.contentHash}`;
+}