@hue-run/sdk 0.5.1 → 0.7.0

package/CLI.md

@@ -42,8 +42,8 @@ project manifest are refused because managers can update ancestor locks; Python
 The generated `hue.setup.mjs` or `hue_setup.py` always selects `captureContent: false` /
 `capture_content=False`. For a supported application, setup installs the dependency and adds the
 managed import and middleware registration to the existing entrypoint; an unreferenced helper is
-not a completed integration. TypeScript uses `@hue-run/sdk@0.5.1`, `@opentelemetry/api@1.9.1` and
-`@opentelemetry/context-async-hooks@2.11.0`; Python uses published `hue-run==0.2.2`.
+not a completed integration. TypeScript uses `@hue-run/sdk@0.7.0`, `@opentelemetry/api@1.9.1` and
+`@opentelemetry/context-async-hooks@2.11.0`; Python setup uses its separately tested package pin.
 Content capture requires an ordinary account-managed key and a later explicit application decision.
 
 The generated bootstrap supplies standard active SERVER-span context across asynchronous/streaming
@@ -231,7 +231,8 @@ capability `setup_telemetry_write`.
 Normal project credentials and unknown token shapes are refused in setup responses and private
 managed state. These credentials authorize metadata-only OTLP at `/api/v1/otlp/v1/traces` and exact
 content-free receipt verification at `/api/v1/setup/traces/{traceId}/receipt`. They do not authorize
-generic project, receipt, evaluation, log or browsing APIs. Published Python `0.2.2` can export with the setup
+generic project, receipt, evaluation, log or browsing APIs. The Python package version
+pinned by setup can export with the setup
 credential; the CLI verifies the dedicated setup receipt instead of Python's generic receipt helper.
 
 Each command uses bounded timeouts and retries. Provisioning records at most five attempts per local
@@ -300,7 +301,7 @@ node packages/sdk-typescript/scripts/verify-package.mjs --artifacts-dir .artifac
 # Set project to an existing supported fixture; use the same directory on resume.
 project=/absolute/path/to/supported-fixture
 node packages/sdk-typescript/scripts/verify-setup-live.mjs \
-  --archive .artifacts/typescript/hue-run-sdk-0.5.1.tgz \
+  --archive .artifacts/typescript/hue-run-sdk-0.7.0.tgz \
   --origin https://STAGING_ORIGIN \
   --project "$project" --command setup \
   --evidence .context/setup-staging-before-claim.json
@@ -311,7 +312,7 @@ the private local handoff and finish the real browser claim, then reconcile the
 
 ```sh
 node packages/sdk-typescript/scripts/verify-setup-live.mjs \
-  --archive .artifacts/typescript/hue-run-sdk-0.5.1.tgz \
+  --archive .artifacts/typescript/hue-run-sdk-0.7.0.tgz \
   --origin https://STAGING_ORIGIN \
   --project "$project" --command claim \
   --evidence .context/setup-staging-after-claim.json
@@ -333,14 +334,14 @@ Publication under a candidate dist-tag, registry acceptance, production activati
 the same version to `latest`, and clean-project literal `@latest` smoke remain separate gates; see
 [RELEASING.md](../../RELEASING.md).
 
-## Evaluate an agent against a Scenario
+## Evaluate an agent against a case
 
-`hue eval` runs a developer's local agent against a published Hue Scenario or a saved eval set
+`hue eval` runs a developer's local agent against a published Hue case or a saved eval set
 and prints Hue's verdicts. It is the command-line form of `runSimulation()` (one-shot) and
 `runLocalAgent()` (worker): the agent, its prompts and its provider credentials stay in the local
 process, Hue creates one isolated simulated world per case, and Hue-owned outcome checks grade
 the sealed world. Hue never executes the agent. `HUE_API_KEY` must be a **Read and
-write** project key (a **Read** or **Tracing only** key cannot read Scenarios or create experiments);
+write** project key (a **Read** or **Tracing only** key cannot read cases or create runs);
 the CLI never prints it. The optional `zod` peer of `@hue-run/sdk/evals` must be installed.
 
 Write an adapter module that hands the case inputs and the world's tools or MCP connection to
@@ -362,16 +363,18 @@ export default function runMyAgent(inputs: JsonValue, context: SimulationTargetC
 ```
 
 ```sh
-hue eval --scenario "Refund an eligible charge" ./hue-agent.ts --env-file .env.hue
-hue eval --scenario https://app.hue.run/projects/demo/scenarios/<id> ./hue-agent.ts --baseline <experiment id>
+hue eval --case "Refund an eligible charge" ./hue-agent.ts --env-file .env.hue
+hue eval --case https://app.hue.run/projects/demo/scenarios/<id> ./hue-agent.ts --baseline <run id>
 hue eval --set "Billing regressions" --scorer-version <id> ./hue-agent.ts --save-version
-hue eval --scenario "Refund an eligible charge" --command "python agent.py" --timeout 120 --content
+hue eval --case "Refund an eligible charge" --command "python agent.py" --timeout 120 --content
 hue eval --worker ./hue-agent.ts --agent-key support-agent --env-file .env.hue
 ```
 
-The one-shot mode resolves the selection (`--scenario` by name, ID or URL; `--set` by name, ID or
+`--scenario` remains an alias for `--case` for existing scripts. Pass one selection flag.
+
+The one-shot mode resolves the selection (`--case` by name, ID or URL; `--set` by name, ID or
 URL with explicit `--scorer-version` pins; or `--dataset-version` with `--scorer-version`),
-creates a fresh experiment from those immutable pins named `<scenario> · <agent key> · <revision>`
+creates a fresh run from those immutable pins named `<case> · <agent key> · <revision>`
 (`--name` overrides), prints `Run: <url>` and `Experiment: <id>` as soon as the experiment exists,
 one line per case event (world created, agent started, world sealed), then
 `Waiting for Hue checks...` and a table with one row per case: boolean metrics as `PASS`/`FAIL`,
@@ -421,7 +424,7 @@ on any Node.js version.
 
 Eval sets whose cases pin no simulated world — a task plus pinned input files, answered with
 generated documents — run as **direct** cases through `runExperiment()`. `hue eval` detects this
-from the saved version (`--mode direct|simulation` overrides the detection; `--scenario` is always
+from the saved version (`--mode direct|simulation` overrides the detection; `--case` is always
 a simulation). `--set` accepts the eval set's slug, name, ID or URL; `--set-version <n>` pins a
 saved version other than the latest; `--scorer <slug|name|id>` pins an evaluator at its newest
 published version, beside or instead of explicit `--scorer-version` IDs.

package/ENVIRONMENTS.md

@@ -79,7 +79,7 @@ until every item has a terminal result for every pin or the budget elapses (`com
 `summarizeVerdicts` turns results into per-case rows with `passed` and totals; `compareVerdicts`
 diffs two summaries by case key; and `collectExperimentVerdicts` combines those reads for one
 experiment. The `hue eval` command uses the same path; see
-[Evaluate an agent against a Scenario](CLI.md#evaluate-an-agent-against-a-scenario).
+[Evaluate an agent against a case](CLI.md#evaluate-an-agent-against-a-case).
 
 ### Pinned provider-profile preflight
 

package/EVALUATIONS.md

@@ -70,6 +70,25 @@ try {
 }
 ```
 
+For new registry code, use `createEvalSet`, `getEvalSet`, `listEvalSets`,
+`createEvalSetVersion`, `getEvalSetVersion`, `listEvalSetCases`, `addEvalSetCase`,
+and `freezeEvalSetVersion`. Evaluators use `createEvaluator`, `getEvaluator`,
+`listEvaluators`, `publishEvaluatorVersion`, and `getEvaluatorVersion`. These
+methods return product fields such as `evalSetId` and `evalSetVersionId` alongside
+the existing v1 fields. Evaluator versions include `evaluatorId` when the server
+supplies their owning identity; older v1 responses may omit it. They use the existing v1 paths;
+the earlier method names remain callable for existing integrations.
+
+For new run and scoring code, use `createRun`, `getRun`, `listRunItems`,
+`getRunCase`, `startRunExecution`, `getRunExecution`,
+`completeRunExecution`, and `finishRun`. Use `createScoring`, `getScoring`,
+`listScorings`, `listScoringItems`, `getScoringSubject`,
+`submitScoringResults`, `listScoringResults`, and `getScoringResult` to score
+saved subjects. `createRun` accepts `evalSetVersionId` and
+`evaluatorVersionIds`; `createScoring` and `submitScoringResults` use evaluator
+version IDs. A run ID and a scoring ID identify different records. The methods
+use the existing v1 paths and leave existing runner entry points callable.
+
 Create another experiment with the same frozen version and different `config` to compare configurations. The runner reads the exact experiment case/version and scorer definitions; it never resolves a mutable latest version. `rescore` accepts an existing evaluation-run ID and has no target callback. Subject IDs refer to immutable saved outputs and trace evidence.
 
 `rescore` preserves terminal scores already recorded for each item and evaluator version,

package/README.md

@@ -329,7 +329,8 @@ const hue = createHue({
 
 ## Delivery behavior
 
-Exports use official OTel retry handling for temporary HTTP/network failures. Each
+Exports retry temporary HTTP/network failures (429, 502, 503, 504 and connection errors,
+honoring `Retry-After`) within the export timeout, by OpenTelemetry's OTLP/HTTP exporter rules. Each
 request is limited to 1 MiB before gzip (with space reserved for gzip overhead) and each content value to 256 KiB. Batches
 split at record boundaries. Each signal queues at most 2,048 records, including
 exports in flight; overflow is reported through the callback, counters and next
@@ -349,6 +350,36 @@ drain, including records emitted before its call. Stop request production
 before shutdown so late spans cannot race it. A client does not own instrumented
 operations still running in the application.
 
+## Live spans
+
+OpenTelemetry exports a span only when it ends, so a long streamed turn would otherwise stay
+invisible until it finishes. When a Hue or AI span is still open at the transport's next 500 ms
+tick, the transport queues a placeholder: an ordinary OTLP span whose parent is the running span,
+with its name, kind, start time and current attributes, an end time of 0,
+`hue.span_type = "pending_span"` and `hue.pending_parent_id` (the running span's own parent,
+omitted for a root). Hue shows the span as running and replaces the placeholder when the real span
+arrives. A placeholder whose span has ended by the time it is exported is not sent.
+
+- Only spans from the client's tracer (`withSpan`, `tool`, `model`, `hue.tracer` and the AI SDK
+  adapters) and spans with a `gen_ai.`, `ai.`, `llm.` or `traceloop.` attribute at start, or a
+  name starting with `ai.`, are announced. HTTP, database and other framework spans are not.
+- Placeholder attributes follow `captureContent` and `redact` like the real span. Tool
+  definitions, system instructions and any value over 64 KiB are left out.
+- Placeholders are advisory. They are queued only while the queue is under a quarter of its
+  record and byte budgets, and skipped silently otherwise. While queued they count in
+  `pendingSpans` and `pendingBytes`, but never as accepted, rejected, failed or dropped records.
+  Losing only placeholders records a warning and does not make `flush()` throw.
+- A Hue server that accepts placeholders sends `Hue-Pending-Spans: 1` on trace acknowledgements.
+  When a response to a request carrying placeholders lacks it, the receiver predates them: the
+  transport attributes up to one rejection per placeholder to them, records one warning and stops
+  sending placeholders for that client. Other rejections count against real spans as usual.
+- Opt out with `liveSpans: false`. Setup credentials never send placeholders.
+- With an existing provider, announcements start in `spanProcessor.onStart`. A wrapping processor
+  that forwards `onStart` should forward `onEnd` for the same spans: a span that ends without
+  reaching Hue is forgotten at the next tick, but a placeholder already sent keeps it showing as
+  running until Hue marks the trace stalled. The filtering wrapper above forwards no starts, so it
+  sends no placeholders.
+
 ## Verify a stored application trace
 
 After exercising a real application request and finishing its stream, flush the
@@ -445,12 +476,12 @@ their agents with `runLocalAgent()`; setup does not register workers or launch s
 ### Command-line evaluation
 
 The unreleased `hue eval` command wraps `runSimulation()` and `runLocalAgent()` for an adapter
-file or a shell command: `hue eval --scenario "<name>" ./hue-agent.ts` creates a fresh experiment
-from a published Scenario's immutable pins, runs the agent in one isolated world per case, waits
+file or a shell command: `hue eval --case "<name>" ./hue-agent.ts` creates a fresh run
+from a published case's immutable pins, runs the agent in one isolated world per case, waits
 for Hue's outcome checks and prints the run URL and per-case PASS/FAIL verdicts with an exit code;
 `--worker` registers the same adapter for runs launched from Hue. It needs a Read and
 write key in `HUE_API_KEY` (never printed) and keeps content capture off unless `--content`
-is passed. See [Evaluate an agent against a Scenario](CLI.md#evaluate-an-agent-against-a-scenario).
+is passed. See [Evaluate an agent against a case](CLI.md#evaluate-an-agent-against-a-case).
 Eval sets whose cases pin files instead of a world run as direct cases through `runExperiment()`:
 `hue eval --set <slug> --scorer <slug> --command "…"` hands the agent each case's pinned files in a
 private directory, uploads the documents it writes and waits for Hue's grading executor to score

package/dist/cli/eval.js

@@ -18,10 +18,11 @@ import { runSimulation, } from "../evals/simulation.js";
 import { collectExperimentVerdicts, compareVerdicts, metricPassed, } from "../evals/verdicts.js";
 const USAGE = `Usage: hue eval [adapter-file] [options]
 
-Run a local agent against a Hue Scenario or eval set, then print Hue's verdicts.
+Run a local agent against a published case or eval set, then print Hue's verdicts.
 
 Selection (exactly one, not used with --worker):
-  --scenario <name|id|url>        Published Scenario to run
+  --case <name|id|url>            Published case to run
+  --scenario <name|id|url>        Legacy alias for --case
   --set <name|id|url>             Saved eval set; requires --scorer or --scorer-version
   --set-version <n>               Saved version number of the eval set (default: latest saved)
   --dataset-version <id>          Frozen dataset version; requires --scorer or --scorer-version
@@ -51,7 +52,7 @@ Connection:
   --origin <url>                  Hue origin (default: HUE_BASE_URL or https://app.hue.run)
 
 Output and limits:
-  --name <run name>               Experiment name (default: <scenario> · <agent key> · <revision>)
+  --name <run name>               Run name (default: <case> · <agent key> · <revision>)
   --baseline <experiment id|url>  Compare verdicts with a previous experiment
   --json                          Print one JSON document on stdout; progress goes to stderr
   --content                       Capture telemetry content; one-shot also persists
@@ -82,6 +83,7 @@ function parse(argv) {
             allowPositionals: true,
             strict: true,
             options: {
+                case: { type: "string" },
                 scenario: { type: "string" },
                 set: { type: "string" },
                 "set-version": { type: "string" },
@@ -508,15 +510,16 @@ async function resolveSelection(client, values) {
     const extra = [...(values["scorer-version"] ?? [])];
     for (const selector of values.scorer ?? [])
         extra.push(await resolveScorerVersion(client, selector));
-    if (values.scenario) {
+    const caseSelector = values.case ?? values.scenario;
+    if (caseSelector) {
         if (values["set-version"])
             throw new UsageError("--set-version applies to --set only");
-        const pins = await resolveScenarioPins(client, values.scenario);
+        const pins = await resolveScenarioPins(client, caseSelector);
         pins.scorerVersionIds = [...new Set([...pins.scorerVersionIds, ...extra])];
         return pins;
     }
     if (!extra.length)
-        throw new UsageError(`${values.set ? "--set" : "--dataset-version"} needs at least one --scorer or --scorer-version; use --scenario for published pins`);
+        throw new UsageError(`${values.set ? "--set" : "--dataset-version"} needs at least one --scorer or --scorer-version; use --case for published pins`);
     if (values.set) {
         const pins = await resolveEvalSetPins(client, values.set, { scorerVersionIds: extra });
         if (values["set-version"] === undefined)
@@ -918,11 +921,11 @@ export async function runEvalCommand(argv) {
         const adapterFile = positionals[0];
         if ((adapterFile ? 1 : 0) + (values.command ? 1 : 0) !== 1)
             throw new UsageError("Pass exactly one agent: an adapter file or --command");
-        const selections = [values.scenario, values.set, values["dataset-version"]].filter((value) => value !== undefined).length;
+        const selections = [values.case, values.scenario, values.set, values["dataset-version"]].filter((value) => value !== undefined).length;
         if (values.worker && selections)
             throw new UsageError("--worker takes no selection; Hue chooses the run to execute");
         if (!values.worker && selections !== 1)
-            throw new UsageError("Pass exactly one of --scenario, --set or --dataset-version");
+            throw new UsageError("Pass exactly one of --case, --set or --dataset-version");
         if (values["env-file"]) {
             try {
                 process.loadEnvFile(resolve(values["env-file"]));

package/dist/cli/login.js

@@ -739,6 +739,6 @@ export async function runLoginCommand(argv, io = {}) {
         // `hue mcp install` defaults to production; a non-default origin needs its own endpoint.
         out(`  hue mcp install --client claude-code${mcpUrl === mcpUrlForOrigin(DEFAULT_ORIGIN) ? "" : ` --url ${mcpUrl}`}`);
     if (stored.includes("evaluations"))
-        out(`  hue eval --scenario "<name>" ./hue-agent.ts --env-file ${envDisplay}`);
+        out(`  hue eval --case "<name>" ./hue-agent.ts --env-file ${envDisplay}`);
     return 0;
 }

package/dist/client.js

@@ -6,6 +6,7 @@ import { W3CTraceContextPropagator } from "@opentelemetry/core";
 import { LoggerProvider } from "@opentelemetry/sdk-logs";
 import { TracerProvider } from "@opentelemetry/sdk-trace";
 import { defaultResource, resourceFromAttributes } from "@opentelemetry/resources";
+import { HUE_SCOPE } from "./config.js";
 import { encodeContent, noopSpan, safeSpan } from "./safety.js";
 import { createHueTransport, HueExportError } from "./transport.js";
 import { verifyTrace } from "./receipt.js";
@@ -197,8 +198,8 @@ export class HueClient {
         }
         this.captureContent = this.transport.options.captureContent;
         this.enabled = this.transport.options.enabled !== false;
-        this.tracer = new ContextualTracer(this.tracerProvider.getTracer("@hue-run/sdk", sdkVersion), this.storage, () => this.enabled && !this.closed, () => this.transport.instrumentationFailure());
-        this.logger = this.loggerProvider.getLogger("@hue-run/sdk", sdkVersion);
+        this.tracer = new ContextualTracer(this.tracerProvider.getTracer(HUE_SCOPE, sdkVersion), this.storage, () => this.enabled && !this.closed, () => this.transport.instrumentationFailure());
+        this.logger = this.loggerProvider.getLogger(HUE_SCOPE, sdkVersion);
     }
     /**
      * Verifies that Hue stored a trace by ID, optionally waiting for expected span IDs and normalized

package/dist/config.d.ts

@@ -1,11 +1,13 @@
 import type { HueOptions, SharedHueOptions } from "./types.js";
 export declare const MAX_BODY_BYTES: number;
 export declare const MAX_CONTENT_BYTES: number;
+/** Instrumentation scope of the client's own tracer and logger. */
+export declare const HUE_SCOPE = "@hue-run/sdk";
 /** Loopback hostnames that may use plain HTTP without opting in. */
 export declare function isLoopbackHost(hostname: string): boolean;
 /** True when a validated origin exports over plain HTTP to a host other than loopback. */
 export declare function isInsecureOrigin(baseUrl: string): boolean;
-export declare function validateOptions(options: HueOptions): HueOptions & Required<Pick<SharedHueOptions, "captureContent" | "baseUrl" | "timeoutMillis" | "maxQueueBytes">> & {
+export declare function validateOptions(options: HueOptions): HueOptions & Required<Pick<SharedHueOptions, "captureContent" | "baseUrl" | "timeoutMillis" | "maxQueueBytes" | "liveSpans">> & {
     /** Project key after validation; empty for a disabled client. */
     apiKey: string;
     /** Service name after validation; `hue-disabled` for a disabled client. */

package/dist/config.js

@@ -1,5 +1,7 @@
 export const MAX_BODY_BYTES = 1024 * 1024;
 export const MAX_CONTENT_BYTES = 256 * 1024;
+/** Instrumentation scope of the client's own tracer and logger. */
+export const HUE_SCOPE = "@hue-run/sdk";
 /** Loopback hostnames that may use plain HTTP without opting in. */
 export function isLoopbackHost(hostname) {
     return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "[::1]";
@@ -30,6 +32,7 @@ export function validateOptions(options) {
             baseUrl: "https://app.hue.run",
             timeoutMillis: 10000,
             maxQueueBytes: 8 * 1024 * 1024,
+            liveSpans: false,
         };
     }
     if (typeof options.captureContent !== "boolean")
@@ -74,5 +77,9 @@ export function validateOptions(options) {
         maxQueueBytes < 1024 ||
         maxQueueBytes > 64 * 1024 * 1024)
         throw new TypeError("maxQueueBytes must be 1024–67108864");
-    return { ...options, baseUrl: url.origin, timeoutMillis, maxQueueBytes };
+    if (options.liveSpans !== undefined && typeof options.liveSpans !== "boolean")
+        throw new TypeError("liveSpans must be a boolean");
+    // Setup credentials send installer telemetry only, never in-progress placeholders.
+    const liveSpans = options.liveSpans !== false && !options.apiKey.startsWith("hue_setup_");
+    return { ...options, baseUrl: url.origin, timeoutMillis, maxQueueBytes, liveSpans };
 }

package/dist/evals/client.d.ts

@@ -1,6 +1,6 @@
 import type { ProjectConnection } from "../types.js";
 import { type AttemptConnectionBundleV2, type PrepareAttemptRequestV2 } from "./attempt.js";
-import type { ArtifactReservation, ArtifactUpload, CaseConversion, CaseConversionSummary, CaseWrite, CompleteExecution, Completion, Dataset, DatasetCase, DatasetVersion, EvaluationItem, EvaluationRun, EvaluationRunSummary, EnvironmentEvidenceSnapshot, Execution, Experiment, ExperimentCase, ExperimentItem, Identity, JsonValue, LocalAgentClaim, LocalAgentRegistration, RegisteredLocalAgent, JudgeBudget, JudgeJob, Page, PageOptions, RegistryPageOptions, Result, ResultSummary, Scorer, ScorerDefinition, ScorerVersion, SimulationMcpCapability, StartExecution, Subject, StoredResult } from "./types.js";
+import type { ArtifactReservation, ArtifactUpload, CaseConversion, CaseConversionSummary, CaseWrite, CompleteExecution, Completion, Dataset, DatasetCase, DatasetVersion, EvalSet, EvalSetCase, EvalSetVersion, EvaluationItem, EvaluationRun, EvaluationRunSummary, EnvironmentEvidenceSnapshot, Execution, Experiment, ExperimentCase, ExperimentItem, Evaluator, EvaluatorVersion, Identity, JsonValue, LocalAgentClaim, LocalAgentRegistration, RegisteredLocalAgent, JudgeBudget, JudgeJob, Page, PageOptions, RegistryPageOptions, Result, ResultSummary, Run, RunCase, Scorer, ScorerDefinition, ScorerVersion, Scoring, ScoringResultInput, ScoringResultSummary, ScoringSubject, ScoringSummary, SimulationMcpCapability, StartExecution, Subject, StoredResult, StoredScoringResult } from "./types.js";
 /** Connection options for {@link createEvaluationClient}. */
 export interface EvaluationClientOptions {
     /** Project service key sent as a Bearer token; server side only. */
@@ -71,6 +71,39 @@ export declare class EvaluationClient {
     publishScorerVersion(id: string, definition: ScorerDefinition): Promise<ScorerVersion>;
     /** Reads a published scorer version. */
     getScorerVersion(id: string): Promise<ScorerVersion>;
+    /** Creates an eval set using the existing v1 registry path. */
+    createEvalSet(input: Identity): Promise<EvalSet>;
+    /** Reads an eval set and its versions. */
+    getEvalSet(id: string): Promise<EvalSet>;
+    /** Lists eval sets. */
+    listEvalSets(page?: RegistryPageOptions): Promise<Page<Omit<EvalSet, "versions">>>;
+    /** Creates a draft eval set version, optionally copying cases from another version. */
+    createEvalSetVersion(id: string, input?: {
+        fromVersionId?: string;
+    }): Promise<EvalSetVersion>;
+    /** Reads an eval set version. */
+    getEvalSetVersion(id: string): Promise<EvalSetVersion>;
+    /** Lists cases in an eval set version. */
+    listEvalSetCases(id: string, page?: PageOptions): Promise<Page<EvalSetCase>>;
+    /** Adds a case to a draft eval set version at its expected revision. */
+    addEvalSetCase(id: string, input: CaseWrite): Promise<{
+        /** The stored case. */
+        item: EvalSetCase;
+        /** The version with its new revision. */
+        version: EvalSetVersion;
+    }>;
+    /** Freezes a draft eval set version at its expected revision. */
+    freezeEvalSetVersion(id: string, expectedRevision: number): Promise<EvalSetVersion>;
+    /** Creates an evaluator identity. */
+    createEvaluator(input: Identity): Promise<Evaluator>;
+    /** Reads an evaluator and its published versions. */
+    getEvaluator(id: string): Promise<Evaluator>;
+    /** Lists evaluators. */
+    listEvaluators(page?: RegistryPageOptions): Promise<Page<Evaluator>>;
+    /** Publishes an immutable evaluator version. */
+    publishEvaluatorVersion(id: string, definition: ScorerDefinition): Promise<EvaluatorVersion>;
+    /** Reads a published evaluator version. */
+    getEvaluatorVersion(id: string): Promise<EvaluatorVersion>;
     /** Creates an experiment over a frozen dataset version with pinned scorer versions and a configuration. */
     createExperiment(input: {
         idempotencyKey: string;
@@ -175,6 +208,70 @@ export declare class EvaluationClient {
     listResults(id: string, page?: PageOptions): Promise<Page<ResultSummary>>;
     /** Reads a full stored result. */
     getResult(id: string): Promise<StoredResult>;
+    /** Creates a run using product request fields on the existing v1 path. */
+    createRun(input: {
+        idempotencyKey: string;
+        name: string;
+        evalSetVersionId: string;
+        evaluatorVersionIds: string[];
+        config: JsonValue;
+    }): Promise<{
+        /** Run ID. */
+        id: string;
+        /** ID of the run's scoring pass. */
+        scoringId: string;
+        /** Existing v1 field for the scoring pass ID. */
+        evaluationRunId: string;
+    }>;
+    /** Reads a run with its scoring progress. */
+    getRun(id: string): Promise<Run>;
+    /** Lists a run's cases with their latest executions. */
+    listRunItems(id: string, page?: PageOptions): Promise<Page<ExperimentItem>>;
+    /** Reads one frozen case of a run. */
+    getRunCase(id: string, caseId: string): Promise<RunCase>;
+    /** Starts or replays a target execution for a run case. */
+    startRunExecution(id: string, caseId: string, input: StartExecution): Promise<Execution>;
+    /** Reads one run execution. */
+    getRunExecution(id: string): Promise<Execution>;
+    /** Saves a run execution's outcome. */
+    completeRunExecution(id: string, input: CompleteExecution): Promise<Completion>;
+    /** Marks a run finished. */
+    finishRun(id: string, idempotencyKey: string): Promise<{
+        /** Run ID. */
+        id: string;
+        /** When the run finished. */
+        finishedAt: string;
+    }>;
+    /** Creates a standalone scoring pass over saved subjects. */
+    createScoring(input: {
+        idempotencyKey: string;
+        name: string;
+        subjectIds: string[];
+        evaluatorVersionIds: string[];
+    }): Promise<{
+        /** Scoring pass ID. */
+        id: string;
+    }>;
+    /** Reads a scoring pass and its pinned evaluators. */
+    getScoring(id: string): Promise<Scoring>;
+    /** Lists scoring passes in the project. */
+    listScorings(page?: PageOptions): Promise<Page<ScoringSummary>>;
+    /** Lists the subjects of a scoring pass. */
+    listScoringItems(id: string, page?: PageOptions): Promise<Page<EvaluationItem>>;
+    /** Reads a saved subject with product-named source fields. */
+    getScoringSubject(id: string): Promise<ScoringSubject>;
+    /** Uploads evaluator results for a scoring pass using product request fields. */
+    submitScoringResults(id: string, input: {
+        idempotencyKey: string;
+        results: ScoringResultInput[];
+    }): Promise<{
+        /** Stored result IDs, in input order. */
+        ids: string[];
+    }>;
+    /** Lists result summaries for a scoring pass. */
+    listScoringResults(id: string, page?: PageOptions): Promise<Page<ScoringResultSummary>>;
+    /** Reads a stored evaluator result. */
+    getScoringResult(id: string): Promise<StoredScoringResult>;
     /** Dispatches hosted judge jobs for `llm_judge` pins; check {@link getJudgeBudget} first. */
     createJudgeJobs(id: string, input: {
         idempotencyKey: string;