@apo-ai/sdk 0.4.0 → 0.6.0

package/README.md

@@ -1,14 +1,15 @@
 # apo SDK
 
 TypeScript/JavaScript SDK for the [apo](.) backend — an agent testing and
-observability platform. The SDK has three entry points:
+observability platform. The SDK's entry points:
 
-- **`@apo-ai/sdk`** — config + error types shared across the surface.
 - **`@apo-ai/sdk/otel`** — OpenTelemetry-native tracing. Wrap your LLM calls and
   agent steps so they land in apo as structured spans, then attach scores.
 - **`@apo-ai/sdk/agent-task`** — the agent-task evaluation framework. Define tasks,
   adapters, and checks; run them against an agent and collect structured
   results.
+- **`@apo-ai/sdk/agent-task/integrations/{ai-sdk,openai,anthropic}`** —
+  per-framework tracing wrappers re-exported for direct import.
 
 ## Installation
 
@@ -36,57 +37,23 @@ no long-lived npm token is stored in the repository.
 
 ## Configuration
 
-`readConfig()` reads backend/project/credentials from environment variables.
-
-| Field       | Env var (server)            | Env var (Next.js, browser-safe)      | Required |
-|-------------|-----------------------------|--------------------------------------|----------|
-| `endpoint`  | `APO_BACKEND_URL`           | `NEXT_PUBLIC_APO_BACKEND_URL`        | yes      |
-| `project`   | `APO_PROJECT`               | `NEXT_PUBLIC_APO_PROJECT`            | yes      |
-| `publicKey` | `APO_PUBLIC_KEY`            | — *(server-side only — see below)*   | two-key  |
-| `secretKey` | `APO_SECRET_KEY`            | —                                    | two-key  |
-| `apiKey`    | `APO_API_KEY`               | —                                    | legacy   |
-
-The Next.js (`NEXT_PUBLIC_*`) variants are read first, then the plain server
-variants. `apiKey` is the legacy single-key format (`sk-…`); prefer the
-two-key model (`publicKey` + `secretKey`, generated as `pk-apo-…` / `sk-apo-…`
-in the dashboard).
-
-there is no browser-safe variant of `APO_PUBLIC_KEY`. The public
-identifier alone does not authorize ingestion, and exposing it in a
-browser bundle creates a misleading direct-browser integration.
-`NEXT_PUBLIC_APO_PUBLIC_KEY` is intentionally not read. Telemetry
-submission always requires both halves of an API-key pair sent as HTTP
-Basic (`base64("pk-apo-…:sk-apo-…")`); the SDK never synthesizes partial
-Basic credentials from only one half.
+The tracing surface (`@apo-ai/sdk/otel`) reads credentials from environment
+variables when they are not passed explicitly:
 
-```ts
-import { readConfig, type EnvConfig } from "@apo-ai/sdk";
-
-const config: EnvConfig = readConfig();
-```
-
-## API reference — `@apo-ai/sdk`
-
-The root entry is intentionally thin: shared types + config + errors. The
-actual tracing surface lives at `@apo-ai/sdk/otel`.
-
-### Types
-
-The trace/observation parameter types used by the OTel helpers:
-`CreateTraceParams`, `CreateSpanParams`, `EndSpanParams`, `TraceRunContext`,
-`TraceRunOptions`, `TraceStepOptions`, `TraceEventOptions`, `IngestionEvent`,
-`CreateScoreParams`, `ClientOptions`, `ParameterOverrides`, `ObserveOptions`,
-`ObservationContext`.
-
-### `readConfig(): EnvConfig`
-
-Read endpoint/project/keys from environment variables (table above).
-
-### Errors
+| Purpose                  | Env var             | Used for                                   |
+|--------------------------|---------------------|--------------------------------------------|
+| OTLP traces endpoint     | `APO_OTLP_ENDPOINT` | Where spans are exported                   |
+| Project (diagnostic)     | `APO_PROJECT`       | Resource attribute only — auth owns tenancy |
+| Two-key auth (Basic)     | `APO_PUBLIC_KEY` + `APO_SECRET_KEY` | `Authorization: Basic base64(pk:sk)` |
+| Bearer token             | `APO_AUTH_TOKEN`    | Short-lived task-run tokens; used only when the key pair is absent |
 
-`ClientError` (Effect `Data.TaggedError`), `ConfigurationError`,
-`ClientErrorCode`. `ClientError` carries a `code` field for categorising
-failures.
+Telemetry submission always requires both halves of an API-key pair sent as
+HTTP Basic (`base64("pk-apo-…:sk-apo-…")`); the SDK never synthesizes partial
+Basic credentials from only one half. There is deliberately no
+`NEXT_PUBLIC_*` browser variant — a public identifier alone does not
+authorize ingestion, and exposing it in a browser bundle creates a misleading
+direct-browser integration. Use `buildApoAuthHeaders` (exported from
+`@apo-ai/sdk/otel`) to build the headers explicitly.
 
 ## Tracing — `@apo-ai/sdk/otel`
 
@@ -115,7 +82,7 @@ call this if your app already configures OTel itself).
 ```ts
 const handle = await configureApoTelemetry({
   takeOwnership: true,
-  endpoint: process.env.APO_OTLP_ENDPOINT ?? "http://localhost:8000/v1/otlp",
+  endpoint: process.env.APO_OTLP_ENDPOINT ?? "http://localhost:8000/api/public/otel/v1/traces",
   // headers, resource attributes, batch/simple processor, etc.
 });
 
@@ -199,12 +166,10 @@ import {
 - **`test(id, fn)`** — register a test through the scope returned by `task()`.
   The callback receives `t` (flat, eve-style assertions over the run's flow)
   and the task-selected deliverables, inferred from the adapter.
-- **`runTask(task, adapter, options?)`** — execute a single task and return a
-  `TaskRunResult` (transcript + evaluation).
+- **`runTask(taskDir, options?)`** — execute a task and return a
+  `TaskRunResult` (transcript + evaluation). The adapter comes from the task
+  definition.
 - **`loadTask(dir)` / `discoverAgentTaskDirs()`** — load tasks from disk.
-- **`createAgentTaskTraceClient(config)`** — the adapter-side trace client
-  passed into runs; the auth token is supplied by the backend runner via
-  `APO_AUTH_TOKEN`.
 
 ### Testing (`*.eval.ts`)
 
@@ -277,8 +242,8 @@ anything). Matchers: `includes(substring|RegExp)`, `equals(value)` (deep),
 `satisfies(predicate, label)`, `similarity(expected, threshold = 0.8)` (fuzzy,
 normalized Levenshtein).
 
-For apo tasks the flow is built automatically from the run's trace (via
-`createFlowTee`) — no extra wiring.
+For apo tasks the flow is built automatically from the run's trace
+projection — no extra wiring.
 
 **`t.judge(value, instruction)` — LLM-backed assertions.** Configure the judge
 with `runTask(dir, { judge: { model, apiKey?, baseURL? } })`, or set
@@ -297,21 +262,22 @@ never revealed. The SDK appends its JSON response contract to any custom
 `system` constant per task and vary only `user` to preserve the cached prompt
 prefix across a task's criteria (issue #161).
 
-**Testing agents not built on apo's adapter.** Build a `Flow` from the
-framework's own output with a normalizer, then run the same checks:
+**Agents not built on apo's adapter.** `fromOpenAIMessages(messages)`,
+`fromAnthropicMessages(messages)`, and `fromAISDK(result)` convert another
+framework's output into the neutral `Flow` format so it can be inspected with
+`FlowView`:
 
 ```ts
-import { runFlowChecks, test } from "@apo-ai/sdk/agent-task";
-import { fromAISDK } from "@apo-ai/sdk/agent-task";        // or fromOpenAIMessages / fromAnthropicMessages
+import { fromAISDK, FlowView } from "@apo-ai/sdk/agent-task";
 
-const flow = fromAISDK(myGenerateTextResult);          // their framework → Flow
-test("used-tools", (t) => { t.calledTool("read_file"); });
-const results = await runFlowChecks({ flow, deliverables: { reply: myGenerateTextResult.text } });
+const view = new FlowView(fromAISDK(myGenerateTextResult));
+view.toolNamesInOrder; // e.g. ["read_file", "search_content"]
+view.reply;            // last assistant message text
 ```
 
-`fromOpenAIMessages(messages)`, `fromAnthropicMessages(messages)`, and
-`fromAISDK(result)` each convert one source into the neutral `Flow` the checks
-read — the plugs that make the testing framework reusable across agent stacks.
+These normalizers are **deprecated** compatibility adapters — the canonical
+path emits standard OTel spans (see the tracing integrations above), which
+apo projects into the same view without a Flow intermediary.
 
 ### CLI / runtime
 

package/dist/agent-task/integrations/ai-sdk.d.ts

@@ -1,4 +1,4 @@
-import { A as AgentTaskTraceContext } from '../../tracing-BE53Q8Ru.js';
+import { A as AgentTaskTraceContext } from '../../tracing-CTmq4PSw.js';
 
 /**
  * Vercel AI SDK tracing integration for apo.
@@ -6,9 +6,9 @@ import { A as AgentTaskTraceContext } from '../../tracing-BE53Q8Ru.js';
  * Pass the result of {@link createApoTracer} to `generateText`'s
  * `experimental_telemetry` option. The AI SDK already emits OpenTelemetry
  * spans natively (e.g. `ai.toolCall`, `ai.generateText`); this tracer
- * intercepts them and translates them into apo trace spans. The FlowTee
- * picks up TOOL/GENERATION observations and builds the Flow that
- * `t.calledTool`, `t.noFailedActions`, and `t.messageIncludes` read.
+ * intercepts them and translates them into apo trace spans, where they
+ * become the TOOL/GENERATION observations that `t.calledTool`,
+ * `t.noFailedActions`, and `t.messageIncludes` read.
  *
  * The adapter author writes zero manual span code.
  *
@@ -109,8 +109,8 @@ interface CreateApoTracerOptions {
  * - `ai.generateText` → a GENERATION span (so `t.messageIncludes(text)` works)
  *   and error status → `t.noFailedActions()` catches it
  *
- * Other spans (`ai.generateText.doGenerate`, etc.) are ignored — the FlowTee
- * only cares about TOOL/AGENT/GENERATION observation types.
+ * Other spans (`ai.generateText.doGenerate`, etc.) are ignored — only
+ * TOOL/AGENT/GENERATION observation types are visible to `t.*` assertions.
  */
 declare function createApoTracer(options: CreateApoTracerOptions): {
     isEnabled: true;

package/dist/agent-task/integrations/ai-sdk.js

@@ -2,7 +2,7 @@
 // Do not edit by hand — edit the src/ originals and rebuild.
 import {
   createApoTracer
-} from "../../chunk-HKLBPIB4.js";
+} from "../../chunk-H7ITLBWU.js";
 import "../../chunk-HWZVL6TS.js";
 export {
   createApoTracer

package/dist/agent-task/integrations/anthropic.d.ts

@@ -1,4 +1,4 @@
-import { A as AgentTaskTraceContext } from '../../tracing-BE53Q8Ru.js';
+import { A as AgentTaskTraceContext } from '../../tracing-CTmq4PSw.js';
 
 /**
  * Anthropic JS SDK tracing integration for apo.

package/dist/agent-task/integrations/openai.d.ts

@@ -1,4 +1,4 @@
-import { A as AgentTaskTraceContext } from '../../tracing-BE53Q8Ru.js';
+import { A as AgentTaskTraceContext } from '../../tracing-CTmq4PSw.js';
 
 /**
  * OpenAI JS SDK tracing integration for apo.
@@ -8,8 +8,8 @@ import { A as AgentTaskTraceContext } from '../../tracing-BE53Q8Ru.js';
  * any manual span code in the adapter.
  *
  * Uses a Proxy to intercept the `create` method, then emits GENERATION +
- * TOOL spans through the trace context. The FlowTee picks these up and
- * builds the Flow that `t.calledTool` reads.
+ * TOOL spans through the trace context, where they become the observations
+ * that `t.calledTool` reads.
  *
  * @example
  * ```ts

package/dist/agent-task/integrations/openai.js

@@ -2,7 +2,7 @@
 // Do not edit by hand — edit the src/ originals and rebuild.
 import {
   createApoOpenAI
-} from "../../chunk-DT2LMB7O.js";
+} from "../../chunk-6ZRW5IW6.js";
 import "../../chunk-GDB6SDGN.js";
 export {
   createApoOpenAI

package/dist/agent-task/public.d.ts

@@ -1,4 +1,4 @@
-import { A as AgentTaskTraceContext, a as AgentTaskTraceOptions } from '../tracing-BE53Q8Ru.js';
+import { A as AgentTaskTraceContext, a as AgentTaskTraceOptions } from '../tracing-CTmq4PSw.js';
 export { CreateApoTracerOptions, createApoTracer } from './integrations/ai-sdk.js';
 export { CreateApoOpenAIOptions, createApoOpenAI } from './integrations/openai.js';
 export { CreateApoAnthropicOptions, createApoAnthropic } from './integrations/anthropic.js';
@@ -323,6 +323,13 @@ type AssertionOutcome = "pass" | "fail" | "unsupported";
 type JudgeMetadata = {
     /** Model identifier, e.g. ``"google/gemini-2.5-flash-lite"``. */
     model?: string;
+    /**
+     * Which response contract elicited this judgment (#163): reasoning-first
+     * (the default) or the legacy verdict-first (``pass`` before
+     * ``reasoning``, via ``APO_JUDGE_VERDICT_FIRST``). Group comparisons
+     * across the default flip on this field, not on time.
+     */
+    contract?: "verdict-first" | "reasoning-first";
     /** The messages sent to the judge LLM. */
     prompt?: {
         system?: string;
@@ -443,10 +450,10 @@ type EvaluationItemResult = {
      */
     assertions?: AssertionResult[];
     /**
-     * the snapshot source the checks ran against. Set to
-     * ``"legacy-flow"`` by the deprecated {@link runFlowChecks} wrapper so
-     * consumers can detect they're on the compatibility path. Absent on
-     * projection-first results (the default).
+     * the snapshot source the checks ran against. ``"local"`` marks locally
+     * recorded snapshots; ``"legacy-flow"`` survives only on results recorded
+     * by the removed Flow-path loader. Absent on projection-first results
+     * (the default).
      */
     source?: "canonical" | "local" | "legacy-flow";
     /**
@@ -927,6 +934,40 @@ interface PreparedTaskResult {
  */
 declare function persistFileArtifacts(deliverables: Record<string, unknown>, config: ArtifactUploadConfig): Promise<PreparedTaskResult>;
 
+/**
+ * Check submission compaction (issue #175).
+ *
+ * The backend's ``check_report_storage.normalize_check_report`` truncates
+ * every ``received`` over 4 KiB and every judge prompt/response segment over
+ * 16 KiB into a marker before storing anything — the full value never
+ * survives server-side. This module applies the same limits at the producer,
+ * so a task judging N criteria against one large document sends N tiny
+ * markers instead of N copies of the document (a 43 MB result body becomes
+ * single-digit MB). Local rendering keeps the uncompacted values.
+ *
+ * The markers are byte-compatible with the backend's (same field names,
+ * preview length, and hash subject), so server-side normalization passes
+ * them through unchanged and old servers store them as-is.
+ */
+
+/** Mirrors backend RECEIVED_VALUE_LIMIT (check_report_storage.py). */
+declare const RECEIVED_VALUE_LIMIT: number;
+/** Mirrors backend JUDGE_SEGMENT_LIMIT. */
+declare const JUDGE_SEGMENT_LIMIT: number;
+interface CheckCompaction {
+    /** Copy of ``checks`` with oversized fields replaced by markers. */
+    checks: EvaluationItemResult[];
+    /** How many ``received`` values were replaced. */
+    truncatedValues: number;
+    /** How many judge prompt/response segments were replaced. */
+    truncatedSegments: number;
+}
+/**
+ * Return a submission-safe copy of ``checks``. Touched paths are copied;
+ * untouched values are shared. Input is never mutated.
+ */
+declare function compactChecksForSubmission(checks: readonly EvaluationItemResult[]): CheckCompaction;
+
 /**
  * Flow — the canonical, source-agnostic recording of "what happened" during
  * an agent run. This is the single substrate the testing framework reads.
@@ -1018,8 +1059,7 @@ declare class FlowView {
  * not the primary testing architecture. The canonical path is now the Trace
  * Projection snapshot (`TraceProjectionSnapshot` → `TraceView` →
  * `runTraceChecks`). These normalizers remain available during the
- * deprecation period; `snapshotFromFlow` bridges their output into the
- * projection contract. New integrations should emit standard OTel spans
+ * deprecation period. New integrations should emit standard OTel spans
  * instead of producing a Flow.
  */
 
@@ -1241,7 +1281,7 @@ type DeliverableValidationResult = {
  *
  * The processor is global, but run isolation is maintained via
  * {@link AsyncLocalStorage}: {@link getActiveApoRun} returns the current
- * run's trace context, so each span is routed to the correct run's FlowTee.
+ * run's trace context, so each span is routed to the correct run's trace.
  *
  * **Design: eager creation, deferred completion.** The apo span is created
  * at `onStart` (when initial attributes like tool name and model are
@@ -1465,4 +1505,4 @@ declare function canonicalManifestJson(manifest: TaskRevisionManifestV1): string
 /** SHA-256 (lowercase hex) of the canonical manifest JSON's UTF-8 bytes. */
 declare function contentSha256(manifest: TaskRevisionManifestV1): string;
 
-export { type AISDKResult, type AdapterDefinition, type AdapterRuntimeState, type AdapterSession, type AgentTaskCliOptions, type AgentTaskRunConfiguration, AgentTaskRunError, type AgentTaskRunSummary, type AgentTaskRuntime, AgentTaskTraceContext, AgentTaskTraceOptions, type AgentTurnResult, type AnthropicMessage, type ApoRunContext, ApoSpanProcessor, type ArtifactUploadConfig, type CheckContext, type CheckGroup, type CleanupContext, type CollectDeliverablesContext, type CollectedDeliverables, type DeliverableDefinition, type DeliverableSummary, type DeliverableValidationResult, type DescribeRegistration, type FileArtifact, type FileArtifactOptions, type FileEntry, type Flow, type FlowEvent, FlowView, type InitializeContext, type JudgeConfig, type JudgeContext, type JudgePromptBuilder, type JudgeScope, type LoadedTask, type ManifestFile, type ManifestFileInput, type Matcher, type ModeClass, type NameMatcher, type OpenAIMessage, type PreparedTaskResult, type RegisterApoTracingOptions, type RejudgeCheckStability, type RejudgeEndpoints, RejudgeError, type RejudgeOptions, type RejudgeOutcome, type RunTaskOptions, type StartSessionContext, TEST_METHOD_NAMES, type TaskConfig, type TaskDefinition, type TaskEvaluationResult, TaskFiles, type TaskRevisionManifestV1, type TaskRunResult, type TaskScope, type TaskTranscript, type TaskTranscriptTurn, type TestContext, type TestRegistration, type ToolCallOptions, type ToolCallStatus, type TurnContext, type TurnFn, type TurnRecord, type TypedAdapterDefinition, type ValueMatcher, buildManifest, canonicalManifestJson, contentSha256, defineAdapter, defineTask, describe, discoverAgentTaskDirs, equals, fileArtifact, filePaths, fromAISDK, fromAnthropicMessages, fromOpenAIMessages, getActiveApoRun, getTaskTurn, includes, isFileArtifact, loadTask, loadTaskRuntime, matchValue, matches, normalizeManifestPath, parseAgentTaskCliArgs, persistFileArtifacts, registerApoTracing, rejudgeTaskRun, resetApoTracing, resetFlowChecks, resetTaskRegistry, resetTaskTurn, resolveTurn, runAgentTaskCli, runTask, runTaskDir, satisfies, sha256Hex, similarity, task, defineCheck as test, turn, withApoRun, withApoRunSync };
+export { type AISDKResult, type AdapterDefinition, type AdapterRuntimeState, type AdapterSession, type AgentTaskCliOptions, type AgentTaskRunConfiguration, AgentTaskRunError, type AgentTaskRunSummary, type AgentTaskRuntime, AgentTaskTraceContext, AgentTaskTraceOptions, type AgentTurnResult, type AnthropicMessage, type ApoRunContext, ApoSpanProcessor, type ArtifactUploadConfig, type CheckCompaction, type CheckContext, type CheckGroup, type CleanupContext, type CollectDeliverablesContext, type CollectedDeliverables, type DeliverableDefinition, type DeliverableSummary, type DeliverableValidationResult, type DescribeRegistration, type FileArtifact, type FileArtifactOptions, type FileEntry, type Flow, type FlowEvent, FlowView, type InitializeContext, JUDGE_SEGMENT_LIMIT, type JudgeConfig, type JudgeContext, type JudgePromptBuilder, type JudgeScope, type LoadedTask, type ManifestFile, type ManifestFileInput, type Matcher, type ModeClass, type NameMatcher, type OpenAIMessage, type PreparedTaskResult, RECEIVED_VALUE_LIMIT, type RegisterApoTracingOptions, type RejudgeCheckStability, type RejudgeEndpoints, RejudgeError, type RejudgeOptions, type RejudgeOutcome, type RunTaskOptions, type StartSessionContext, TEST_METHOD_NAMES, type TaskConfig, type TaskDefinition, type TaskEvaluationResult, TaskFiles, type TaskRevisionManifestV1, type TaskRunResult, type TaskScope, type TaskTranscript, type TaskTranscriptTurn, type TestContext, type TestRegistration, type ToolCallOptions, type ToolCallStatus, type TurnContext, type TurnFn, type TurnRecord, type TypedAdapterDefinition, type ValueMatcher, buildManifest, canonicalManifestJson, compactChecksForSubmission, contentSha256, defineAdapter, defineTask, describe, discoverAgentTaskDirs, equals, fileArtifact, filePaths, fromAISDK, fromAnthropicMessages, fromOpenAIMessages, getActiveApoRun, getTaskTurn, includes, isFileArtifact, loadTask, loadTaskRuntime, matchValue, matches, normalizeManifestPath, parseAgentTaskCliArgs, persistFileArtifacts, registerApoTracing, rejudgeTaskRun, resetApoTracing, resetFlowChecks, resetTaskRegistry, resetTaskTurn, resolveTurn, runAgentTaskCli, runTask, runTaskDir, satisfies, sha256Hex, similarity, task, defineCheck as test, turn, withApoRun, withApoRunSync };