@eval-studio/flue 0.0.0-stage → 1.1.0

package/CHANGELOG.md

@@ -0,0 +1,40 @@
+# Changelog
+
+All notable changes to `@eval-studio/flue`, released as tags of
+[oghaAI/flue-eval-reporter](https://github.com/oghaAI/flue-eval-reporter).
+Install a release with `bun add github:oghaAI/flue-eval-reporter#vX.Y.Z`.
+
+## 1.1.0
+
+The first release of the Eval Studio SDK for Flue 2.3 apps, replacing the
+v0.x backbone reporter. It works beside `@flue/opentelemetry`: runs reach
+Eval Studio as OpenTelemetry spans over OTLP, and the SDK adds what spans
+alone cannot carry. There is no 1.0.0 tag; 1.1.0 follows 0.4.0.
+
+- **Agent versions.** `createEvalStudio()` and `registerAgentVersion()` hash
+  the agent name, rendered system instructions, tool manifest and model into
+  one version identity, register it with `POST /v1/subject-versions`, and
+  stamp every `invoke_agent` span with `eval.subject_version_id` and
+  `eval.version_hash`. The stamp is computed locally, so spans carry it from
+  the first request. `serviceVersion` (default `OTEL_SERVICE_VERSION`, else
+  `GIT_SHA`) is recorded beside each version. `sendInstructions: false`
+  keeps instruction text out; the hash still covers it.
+- **Cost.** `evalStudio.instrumentation({ otel })` wraps Flue's OpenTelemetry
+  instrumentation and puts Flue's own per-call cost (`usage.cost.total`) on
+  each `chat` span as `flue.cost.usd`.
+- **Human feedback.** `feedback()` reports approvals, rejections with a
+  reason code, revisions with their edit size, take-overs, hand-backs and
+  holds to `POST /v1/feedback`. Events are validated locally, batched (cut by
+  count and size, split on a `413`), retried with backoff on network errors,
+  `408`, `429` and `5xx`, and kept queued through an outage, up to
+  `maxQueueSize` (default 10 000; the oldest are dropped past it and reported
+  to `onError`). One bad event no longer sinks its batch. `hashActor()` and `editStats()` keep reviewer ids
+  and edited text in your process; note text is never sent.
+- **Inert without configuration.** With no `EVAL_STUDIO_URL` /
+  `EVAL_STUDIO_PROJECT_KEY` nothing is sent; versions are still hashed and
+  stamped.
+- **Peers:** `@flue/runtime` ^2.3, `@opentelemetry/api` ^1.9, `valibot` ^1.1.
+  `@valibot/to-json-schema` is the one regular dependency.
+- **Deprecated:** the v0.x push reporter (`initEvalStudioReporter`,
+  `agentSubject`, `reportFeedback`) still ships at `@eval-studio/flue/legacy`
+  for apps mid-migration. See the README's migration section.

package/README.md

@@ -1,3 +1,255 @@
-# Temporary Holding Version
+# @eval-studio/flue
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The Eval Studio SDK for Flue 2.3 apps. It works beside
+[`@flue/opentelemetry`](https://flueframework.com/docs/ecosystem/tooling/opentelemetry/),
+which exports the spans Eval Studio turns into runs, and adds the three things
+spans alone cannot carry:
+
+- **Agent version.** `registerAgentVersion()` hashes the agent name, the
+  rendered system instructions, the tool manifest and the model into one
+  version identity, registers it with Eval Studio, and every `invoke_agent`
+  span is stamped with it (`eval.subject_version_id`, `eval.version_hash`).
+  Each run in Eval Studio then names the exact version it ran.
+- **Cost.** Flue knows what each model call cost (`usage.cost.total` from its
+  model catalog). The SDK puts that number on each `chat` span as
+  `flue.cost.usd`. Eval Studio uses it before falling back to its own token ×
+  price table.
+- **Human feedback.** `feedback()` reports what a reviewer did with an agent's
+  output: approved, rejected (with a reason code), revised (with how much was
+  edited), taken over, handed back. Events are batched and retried.
+
+The SDK never creates spans, a tracer provider or an exporter. Your
+OpenTelemetry setup stays as it is.
+
+## Install
+
+```sh
+bun add github:oghaAI/flue-eval-reporter#v1.1.0
+# or: npm install github:oghaAI/flue-eval-reporter#v1.1.0
+```
+
+The package installs as `@eval-studio/flue`. Peer dependencies, which a Flue
+app with OpenTelemetry already has: `@flue/runtime` ^2.3, `@opentelemetry/api`
+^1.9, `valibot` ^1.1. Its one regular dependency, `@valibot/to-json-schema`
+(which turns a tool's valibot input into the JSON Schema the tools hash
+covers), installs with it and uses your `valibot`. Everything else, including
+the shared hashing and validation code, is bundled into `dist/`. Always pin a
+`#vX.Y.Z` tag.
+
+## Setup
+
+You need a project key with the `ingest` scope (Eval Studio → **Settings → API
+keys**). The key decides the project; nothing in a payload can change it.
+
+```ts title="src/telemetry.ts"
+import { createEvalStudio } from "@eval-studio/flue"
+import { createOpenTelemetryInstrumentation } from "@flue/opentelemetry"
+import { instrument } from "@flue/runtime"
+import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto"
+import { NodeSDK } from "@opentelemetry/sdk-node"
+
+import { SUPPORT_INSTRUCTIONS, lookupOrder, refund } from "./agents/support.ts"
+
+new NodeSDK({ traceExporter: new OTLPTraceExporter() }).start()
+
+export const evalStudio = createEvalStudio({
+  endpoint: process.env.EVAL_STUDIO_URL, // https://evals.example.com
+  projectKey: process.env.EVAL_STUDIO_PROJECT_KEY, // es_ingest_…
+  serviceVersion: process.env.GIT_SHA, // recorded beside each version
+})
+instrument(
+  evalStudio.instrumentation({ otel: createOpenTelemetryInstrumentation() })
+)
+
+void evalStudio.registerAgentVersion({
+  agentName: "support",
+  instructions: SUPPORT_INSTRUCTIONS, // what the agent function returns
+  tools: [lookupOrder, refund], // the defineTool() values it mounts
+  model: "openrouter/deepseek/deepseek-v4-flash", // what it passes to useModel()
+})
+```
+
+```ts title="src/app.ts"
+import "./telemetry.ts" // first, before anything that makes model calls
+```
+
+Point the OpenTelemetry exporter at the same Eval Studio with the same key:
+
+```sh
+OTEL_EXPORTER_OTLP_ENDPOINT=https://evals.example.com
+OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer es_ingest_…"
+OTEL_SERVICE_NAME=support-app
+```
+
+Things to know:
+
+- **Install one instrumentation.** `evalStudio.instrumentation({ otel })`
+  wraps the OpenTelemetry instrumentation; install it **instead of**
+  `createOpenTelemetryInstrumentation()`, not beside it. The wrapping puts the
+  version stamp inside the agent span and lets the cost bridge set
+  `flue.cost.usd` before the chat span ends. Installed on its own (after the
+  OpenTelemetry one) it still stamps versions, but cannot bridge cost.
+- **Register at boot.** The stamp is computed locally, so spans carry
+  `eval.version_hash` from the first request, even before Eval Studio has
+  answered. Eval Studio resolves the version by hash in that case.
+- **Keep the definition in step with the agent.** The definition is what the
+  agent renders. If it drifts, you register a version the agent is not
+  running.
+- **Inert without configuration.** With no `endpoint` or `projectKey` (and no
+  `EVAL_STUDIO_URL` / `EVAL_STUDIO_PROJECT_KEY` in the environment) nothing is
+  sent. Versions are still hashed and stamped. It is safe to leave in
+  production code paths.
+- **Flush on shutdown.** Call `await evalStudio.shutdown()` from your
+  shutdown path, next to your OpenTelemetry SDK's `shutdown()`.
+
+`apps/flue-test` in the eval-studio repo is a complete working example
+(`src/telemetry.ts`, `src/eval-studio.ts`, `scripts/send-feedback.ts`).
+
+## Version identity
+
+```text
+instructions_hash = sha256(instructions), 12 hex
+tools_hash        = sha256(tools sorted by name, JSON Schema keys sorted), 12 hex
+version_hash      = sha256({ agentName, instructionsHash, model, toolManifestHash }), 12 hex
+```
+
+The order you list tools in never changes the hash. A renamed tool, a reworded
+description, a changed input schema, a different model or one changed word in
+the instructions does. `agentVersion(def)` and `agentVersionHash(def)` compute
+it without registering. Eval Studio recomputes `version_hash` from the parts
+and refuses a mismatch.
+
+The SDK sends the instruction text with the registration, so Eval Studio's
+spec sheet can show it. Eval Studio stores it only when the project's
+telemetry content mode is `redacted` or `full` (new projects default to
+`none`, which keeps the hash and drops the text), and only when it hashes to
+`instructions_hash`. Pass `sendInstructions: false` to keep it in the
+process either way; the hashes are still sent.
+
+## Human feedback
+
+```ts
+import { editStats, hashActor } from "@eval-studio/flue"
+
+const salt = process.env.FEEDBACK_SALT! // one secret per deployment
+
+// approved as is
+evalStudio.feedback({
+  type: "approval.approved",
+  conversationId: caseId, // your conversation (Runbook: the case id)
+  submissionId, // Flue's submission id: links the event to its run
+  actorHash: hashActor(reviewer.id, salt),
+  decisionLatencyMs: Date.now() - shownAt,
+})
+
+// rejected: a reason code is required
+evalStudio.feedback({
+  type: "approval.rejected",
+  conversationId: caseId,
+  submissionId,
+  reasonCode: "wrong_facts", // wrong_facts | wrong_action | tone | policy | duplicate | other
+  notePresent: note.length > 0, // the note itself is never sent
+  actorHash: hashActor(reviewer.id, salt),
+})
+
+// revised before sending: send the size of the edit, not the text
+evalStudio.feedback({
+  type: "approval.revised",
+  conversationId: caseId,
+  submissionId,
+  edit: editStats(agentDraft, sentText), // { charsBefore, charsAfter, editDistance, editRatio }
+})
+
+// several at once
+evalStudio.feedbackBatch([
+  { type: "case.takeover", conversationId: caseId, actorHash },
+  { type: "case.handback", conversationId: caseId, actorHash },
+])
+```
+
+Event types: `approval.approved`, `approval.rejected`, `approval.revised`,
+`case.takeover`, `case.handback`, `case.hold`, `edit.sent`. Optional fields:
+`id` (a UUID is generated; it is the idempotency key, so set your own when you
+may resend), `occurredAt`, `toolCallId`, `outputId`, `traceId`, `spanId`,
+`agentName`, `actionType`, `decisionLatencyMs`, and `attributes` (a flat map of
+up to 32 string, number or boolean values).
+
+Eval Studio links each event to its run by `submissionId`, or by `traceId` /
+`spanId` when you have those instead. Feedback that arrives before its run is
+linked once the run's spans land.
+
+`feedback()` validates the event and throws a `TypeError` on a mistake the
+server would refuse, such as a rejection with no reason or a raw id in
+`actorHash`. Valid events are queued and sent in batches (every
+`flushIntervalMs`, default 2 s, or as soon as `batchSize`, default 100, are
+waiting). `await evalStudio.flush()` sends what is queued now.
+
+### Privacy
+
+- **Never send note text.** Report `notePresent: true`. The API has no field
+  for the note and refuses events that carry `note`, `comment` or similar.
+- **Never send raw user ids or emails.** Use `hashActor(id, salt)`: a salted
+  sha256 that counts distinct reviewers without saying who they are. Use one
+  secret salt per deployment, not per event, so the same person hashes the
+  same way. The API only accepts a 64-hex digest in `actor_hash`.
+- **Edits travel as numbers.** `editStats(before, after)` runs in your process
+  and returns four numbers; neither text leaves it.
+
+## Delivery
+
+- Requests are retried with exponential backoff (500 ms doubling, with
+  jitter, capped at 30 s, 5 attempts) on network errors, `408`, `429` and
+  `5xx`. A `Retry-After` header is honoured.
+- Other `4xx` answers (an invalid event, a bad key, a body over the cap) are
+  not retried. The batch is dropped and reported to `onError`.
+- If a batch still fails after every retry, it stays queued for the next
+  flush. The queue holds up to `maxQueueSize` events (default 10 000); past
+  that the oldest are dropped and reported.
+- `shutdown()` stops the timer, flushes, and refuses further events.
+
+| Option                                     | Default                                              |
+| ------------------------------------------ | ---------------------------------------------------- |
+| `endpoint`                                 | `EVAL_STUDIO_URL`                                    |
+| `projectKey`                               | `EVAL_STUDIO_PROJECT_KEY` (or `EVAL_STUDIO_API_KEY`) |
+| `serviceVersion`                           | `OTEL_SERVICE_VERSION`, else `GIT_SHA`               |
+| `batchSize`                                | 100 (server cap 500)                                 |
+| `flushIntervalMs`                          | 2000                                                 |
+| `maxQueueSize`                             | 10 000                                               |
+| `maxAttempts`, `baseDelayMs`, `maxDelayMs` | 5, 500, 30 000                                       |
+| `timeoutMs`                                | 10 000 per attempt                                   |
+| `sendInstructions`                         | `true`                                               |
+| `onError`                                  | logs a warning                                       |
+
+## API reference
+
+The two endpoints the SDK calls, `POST /v1/subject-versions` and
+`POST /v1/feedback`, are documented in Eval Studio's
+`docs/sdk/api-reference.md`, for apps that send without this SDK.
+
+## Migrating from the reporter (`initEvalStudioReporter`)
+
+The backbone push reporter still ships, deprecated, at
+`@eval-studio/flue/legacy`. Eval Studio still serves `/api/ingest-push`, and
+answers a beta.9 snapshot with `422`. To move:
+
+1. Set up `@flue/opentelemetry` with an OTLP exporter as above. Runs now come
+   from spans.
+2. Replace `initEvalStudioReporter({ subjects })` with `createEvalStudio()` and
+   one `registerAgentVersion()` per agent. The object you passed to
+   `agentSubject()` is the definition `registerAgentVersion()` takes, and the
+   hashes are identical, so existing versions are re-used, not duplicated.
+3. Replace `reportFeedback({ verdict })` with `feedback({ type })`.
+
+## Releasing (maintainers)
+
+```sh
+packages/flue-eval-reporter/scripts/release.sh 1.1.0          # prepare: build, stage, commit, tag locally
+packages/flue-eval-reporter/scripts/release.sh 1.1.0 --push   # then publish the prepared tag
+```
+
+The script builds with `tsup` (ESM + types, the shared `@repo/schema` slices
+inlined, `@flue/runtime` and `@opentelemetry/api` external) and lays down
+`package.json` (name `@eval-studio/flue`), `dist/` and this README in a clone
+of the `oghaAI/flue-eval-reporter` dist repo, tagged `v<version>`. Without
+`--push` it stops before pushing and prints the push command. The source of
+truth stays in the eval-studio monorepo; never edit the dist repo by hand.

package/dist/chunk-LKO4T54N.js

@@ -0,0 +1,131 @@
+// ../schema/src/stamping.ts
+import { createHash } from "crypto";
+var sha12 = (s) => createHash("sha256").update(s).digest("hex").slice(0, 12);
+function promptHash(instructions) {
+  return sha12(instructions);
+}
+function instructionsHash(instructions) {
+  return promptHash(instructions);
+}
+function toolManifestHash(tools) {
+  const canonical = [...tools].map((t) => ({
+    name: t.name,
+    description: t.description,
+    ...t.inputSchema !== void 0 ? { inputSchema: t.inputSchema } : {}
+  })).sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0);
+  return sha12(stableStringify(canonical));
+}
+var HASH12 = /^[0-9a-f]{12}$/;
+function subjectProblem(subject) {
+  if (!subject.agentName.trim()) return "agentName is empty";
+  if (!HASH12.test(subject.instructionsHash))
+    return "instructionsHash must be 12 lowercase hex characters";
+  if (!HASH12.test(subject.toolManifestHash))
+    return "toolManifestHash must be 12 lowercase hex characters";
+  if (typeof subject.model !== "string") return "model must be a string";
+  return null;
+}
+function subjectVersionHash(subject) {
+  return sha12(
+    stableStringify({
+      agentName: subject.agentName,
+      instructionsHash: subject.instructionsHash,
+      model: subject.model,
+      toolManifestHash: subject.toolManifestHash
+    })
+  );
+}
+function subjectRegistration(subject) {
+  return {
+    agentName: subject.agentName,
+    promptHash: subject.instructionsHash,
+    skillHash: subject.toolManifestHash,
+    model: subject.model,
+    ...subject.instructions != null ? { instructions: subject.instructions } : {},
+    ...subject.toolNames != null ? { skillNames: [...subject.toolNames].sort() } : {},
+    ...subject.scope ? { scopeName: subject.scope.name, scopeTemplate: subject.scope.template } : {},
+    ...subject.bom ? { bom: subject.bom } : {}
+  };
+}
+function manifestBom(tools) {
+  return {
+    partial: true,
+    tools: [...tools].map((t) => ({
+      name: t.name,
+      description: t.description,
+      ...t.inputSchema !== void 0 ? { inputSchema: t.inputSchema } : {}
+    })).sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0),
+    actions: [],
+    skills: [],
+    subagents: []
+  };
+}
+function stableStringify(value) {
+  if (value === void 0) return "null";
+  if (value === null || typeof value !== "object") return JSON.stringify(value);
+  if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`;
+  const keys = Object.keys(value).sort();
+  return `{${keys.map((k) => `${JSON.stringify(k)}:${stableStringify(value[k])}`).join(",")}}`;
+}
+var SCOPE_CASE_PLACEHOLDER = "{case}";
+function isValidScopeTemplate(template) {
+  return template.includes(SCOPE_CASE_PLACEHOLDER);
+}
+
+// src/version.ts
+import { toJsonSchema } from "@valibot/to-json-schema";
+function toolManifestEntry(tool) {
+  let inputSchema;
+  if (tool.input != null) {
+    try {
+      inputSchema = toJsonSchema(tool.input, {
+        errorMode: "ignore"
+      });
+    } catch {
+      inputSchema = void 0;
+    }
+  }
+  return {
+    name: tool.name,
+    description: tool.description,
+    ...inputSchema !== void 0 ? { inputSchema } : {}
+  };
+}
+var isManifestEntry = (t) => "inputSchema" in t || !("input" in t);
+function agentVersion(def) {
+  if (!def.agentName.trim()) throw new TypeError("agentName is empty");
+  if (typeof def.model !== "string" || !def.model.trim())
+    throw new TypeError("model is empty");
+  const manifest = def.tools.map(
+    (t) => isManifestEntry(t) ? t : toolManifestEntry(t)
+  );
+  const iHash = instructionsHash(def.instructions);
+  const tHash = toolManifestHash(manifest);
+  return {
+    agentName: def.agentName,
+    model: def.model,
+    instructionsHash: iHash,
+    toolsHash: tHash,
+    versionHash: subjectVersionHash({
+      agentName: def.agentName,
+      instructionsHash: iHash,
+      toolManifestHash: tHash,
+      model: def.model
+    }),
+    toolNames: manifest.map((t) => t.name).sort()
+  };
+}
+var agentVersionHash = (def) => agentVersion(def).versionHash;
+
+export {
+  instructionsHash,
+  toolManifestHash,
+  subjectProblem,
+  subjectVersionHash,
+  subjectRegistration,
+  manifestBom,
+  isValidScopeTemplate,
+  toolManifestEntry,
+  agentVersion,
+  agentVersionHash
+};

package/dist/index.d.ts

@@ -0,0 +1,195 @@
+import { FlueInstrumentation } from '@flue/runtime';
+import { A as AgentDefinition, a as AgentVersion } from './version-DU6_w67Z.js';
+export { T as ToolLike, b as agentVersion, c as agentVersionHash, t as toolManifestEntry } from './version-DU6_w67Z.js';
+export { ToolManifestEntry } from '@repo/schema/stamping';
+
+declare const FEEDBACK_TYPES: readonly ["approval.approved", "approval.rejected", "approval.revised", "case.takeover", "case.handback", "case.hold", "edit.sent"];
+type FeedbackType = (typeof FEEDBACK_TYPES)[number];
+declare const REASON_CODES: readonly ["wrong_facts", "wrong_action", "tone", "policy", "duplicate", "other"];
+type ReasonCode = (typeof REASON_CODES)[number];
+type EditStats = {
+    charsBefore: number;
+    charsAfter: number;
+    editDistance: number;
+    /** editDistance / max(charsBefore, charsAfter): 0 = untouched, 1 = rewritten */
+    editRatio: number;
+};
+/** One human signal about an agent output. */
+type FeedbackEvent = {
+    type: FeedbackType;
+    /** The conversation the output belongs to (Runbook: the case id). */
+    conversationId: string;
+    /** Idempotency key; a UUID is generated when absent. */
+    id?: string;
+    /** When the human acted; now when absent. */
+    occurredAt?: Date | string;
+    submissionId?: string;
+    toolCallId?: string;
+    outputId?: string;
+    /** The run's trace and span, when the app has them (hex32 / hex16). */
+    traceId?: string;
+    spanId?: string;
+    agentName?: string;
+    actionType?: string;
+    /** hashActor(userId, salt) — never a raw id or email. */
+    actorHash?: string;
+    /** Required for approval.rejected. */
+    reasonCode?: ReasonCode;
+    /** Whether the human wrote a note; the note itself is never sent. */
+    notePresent?: boolean;
+    /** edit.sent / approval.revised: how much the human changed (editStats). */
+    edit?: Partial<EditStats>;
+    decisionLatencyMs?: number;
+    /** Flat string/number/boolean map, at most 32 keys. */
+    attributes?: Record<string, string | number | boolean>;
+};
+/** The snake_case event POST /v1/feedback takes. */
+type WireFeedbackEvent = {
+    id: string;
+    type: FeedbackType;
+    occurred_at: string;
+    conversation_id: string;
+    submission_id?: string;
+    tool_call_id?: string;
+    output_id?: string;
+    trace_id?: string;
+    span_id?: string;
+    agent_name?: string;
+    action_type?: string;
+    actor_hash?: string;
+    reason_code?: ReasonCode;
+    note_present?: boolean;
+    edit?: {
+        chars_before?: number;
+        chars_after?: number;
+        edit_distance?: number;
+        edit_ratio?: number;
+    };
+    decision_latency_ms?: number;
+    attributes?: Record<string, string | number | boolean>;
+};
+/**
+ * A salted sha256 of an actor id, as lowercase hex. Use one secret salt per
+ * deployment (not per event) so the same person hashes the same way and
+ * Eval Studio can count distinct reviewers without learning who they are.
+ */
+declare function hashActor(id: string, salt: string): string;
+/**
+ * How much a human changed an agent's draft, computed locally — the texts
+ * never leave the process, only these four numbers do.
+ */
+declare function editStats(before: string, after: string): EditStats;
+/**
+ * The wire form of an event, or a TypeError naming what is wrong. The final
+ * check is the server's own validator (@repo/schema/feedback-wire, bundled
+ * into the SDK), so an event that gets queued is an event POST /v1/feedback
+ * accepts: a bad one throws here, at the call site, instead of sinking the
+ * batch it would have been sent in.
+ */
+declare function toWireEvent(e: FeedbackEvent): WireFeedbackEvent;
+
+declare const EVAL_STUDIO_INSTRUMENTATION_KEY: unique symbol;
+declare const ATTR_SUBJECT_VERSION_ID = "eval.subject_version_id";
+declare const ATTR_VERSION_HASH = "eval.version_hash";
+declare const ATTR_COST_USD = "flue.cost.usd";
+/** What the stamp needs per agent, kept current by the client. */
+type VersionLookup = (agentName: string) => {
+    versionHash: string;
+    subjectVersionId: string | null;
+} | undefined;
+type InstrumentationOptions = {
+    /**
+     * The @flue/opentelemetry instrumentation to compose with
+     * (createOpenTelemetryInstrumentation()). Install the result INSTEAD of it.
+     * Without it, install this after the OpenTelemetry one: stamping still
+     * works, the cost bridge cannot (the chat span has already ended).
+     */
+    otel?: FlueInstrumentation;
+    /** Set flue.cost.usd on chat spans that lack it. Default true. */
+    costBridge?: boolean;
+};
+declare function createInstrumentation(lookup: VersionLookup, options?: InstrumentationOptions): FlueInstrumentation;
+
+type TransportOptions = {
+    fetch?: typeof fetch;
+    /** Attempts per request, first try included. Default 5. */
+    maxAttempts?: number;
+    /** First backoff delay; doubles per attempt, with jitter. Default 500 ms. */
+    baseDelayMs?: number;
+    /** Backoff ceiling. Default 30 s. */
+    maxDelayMs?: number;
+    /** Per-attempt timeout. Default 10 s. */
+    timeoutMs?: number;
+    /** Injectable for tests. */
+    sleep?: (ms: number) => Promise<void>;
+    random?: () => number;
+};
+type PostResult = {
+    ok: true;
+    status: number;
+    body: unknown;
+} | {
+    ok: false;
+    status: number | null;
+    error: string;
+    retryable: boolean;
+    /** The parsed error body, when the server sent one (e.g. 400 `invalid`). */
+    body?: unknown;
+};
+declare function backoffDelay(attempt: number, opts: Pick<TransportOptions, "baseDelayMs" | "maxDelayMs" | "random">, retryAfterSeconds?: number): number;
+/** POST JSON with the project key, retrying what is worth retrying. */
+declare function postWithRetry(url: string, projectKey: string, body: unknown, opts?: TransportOptions): Promise<PostResult>;
+
+type EvalStudioOptions = {
+    /** Eval Studio base URL. Default EVAL_STUDIO_URL; absent → inert. */
+    endpoint?: string;
+    /** Project API key with the `ingest` scope. Default EVAL_STUDIO_PROJECT_KEY (or EVAL_STUDIO_API_KEY). */
+    projectKey?: string;
+    /**
+     * The build this process runs (decision 5: the git SHA, recorded beside
+     * the version). Default OTEL_SERVICE_VERSION, else GIT_SHA. A hex SHA is
+     * also sent as the version's git_sha.
+     */
+    serviceVersion?: string;
+    /** Events per POST. Default 100 (server cap 500); a batch also stays under ~900 KB. */
+    batchSize?: number;
+    /** Auto-flush period for queued feedback. Default 2000 ms. */
+    flushIntervalMs?: number;
+    /** Queue bound; the oldest events are dropped past it. Default 10 000. */
+    maxQueueSize?: number;
+    /** Send each version's instruction text as provenance. Default true. */
+    sendInstructions?: boolean;
+    /** Called on every dropped event or failed request (never throws into the app). */
+    onError?: (error: EvalStudioError) => void;
+} & TransportOptions;
+type EvalStudioError = {
+    kind: "register" | "feedback" | "dropped";
+    message: string;
+    status?: number | null;
+    count?: number;
+};
+type RegisteredAgentVersion = AgentVersion & {
+    /** Eval Studio's id for the version; null when inert or unreachable. */
+    subjectVersionId: string | null;
+};
+type EvalStudio = {
+    /** Hash an agent version, stamp it from now on, and register it. */
+    registerAgentVersion: (def: AgentDefinition) => Promise<RegisteredAgentVersion>;
+    /** The Flue instrumentation: version stamps + the cost bridge. */
+    instrumentation: (options?: InstrumentationOptions) => FlueInstrumentation;
+    /** Queue one human feedback event (validated now, sent in a batch). */
+    feedback: (event: FeedbackEvent) => void;
+    /** Queue several. */
+    feedbackBatch: (events: FeedbackEvent[]) => void;
+    /** Send everything queued; resolves when the queue is empty or a send failed. */
+    flush: () => Promise<void>;
+    /** Stop the timer, flush, and refuse further events. */
+    shutdown: () => Promise<void>;
+    /** Versions registered in this process, by agent name. */
+    versions: () => ReadonlyMap<string, RegisteredAgentVersion>;
+    /** Whether this client sends anything. */
+    readonly enabled: boolean;
+};
+declare function createEvalStudio(options?: EvalStudioOptions): EvalStudio;
+
+export { ATTR_COST_USD, ATTR_SUBJECT_VERSION_ID, ATTR_VERSION_HASH, AgentDefinition, AgentVersion, EVAL_STUDIO_INSTRUMENTATION_KEY, type EditStats, type EvalStudio, type EvalStudioError, type EvalStudioOptions, FEEDBACK_TYPES, type FeedbackEvent, type FeedbackType, type InstrumentationOptions, type PostResult, REASON_CODES, type ReasonCode, type RegisteredAgentVersion, type TransportOptions, type VersionLookup, type WireFeedbackEvent, backoffDelay, createEvalStudio, createInstrumentation, editStats, hashActor, postWithRetry, toWireEvent };