nexus-ai-pro 2.0.0 → 2.1.0

package/API_STABILITY.md

@@ -41,6 +41,33 @@ Both are covered by this policy:
 Deep imports into `dist`, `dist-cjs`, or `src` are not supported. Import only from `nexus-ai-pro` or
 one of its explicit subpaths.
 
+## 2.1 stage (2.1.0)
+
+Everything 2.1 adds is stable from the start and additive: no default changes meaning.
+
+- graph durability modes, `flush()`, `maxPendingWrites`, `onError` with `NodeFailure` and
+  `RecoveredFailure`, `nodeDefaults`, `NodeTimeout`, `context.heartbeat()`, `SendOptions`,
+  `RunControl` and `RunControlLike`, and `GraphDrainedError` (`nexus-ai-pro/graph`);
+- `lintGraph()` and its finding types (`nexus-ai-pro/graph/lint`). New rules may be added in a minor
+  release; a rule's code and severity do not change within 2.x;
+- `OperationContext.previousHeartbeat` and `OperationRecord.heartbeatDetails`;
+- `AssistantRunContext.control`;
+- the graph event stream: `events()`, `resumeEvents()`, `continueEvents()`, `GraphEventStream`, its
+  event and option types, `GraphStreamOverflowError`, `context.message()` and `context.tool()`, and
+  `subgraphEvents`. New projections may be added in a minor release;
+- `TracerOptions.incremental` and `exporters`, `TraceExporter`, `traceparent` in and out, `flush()`,
+  `closeAbandonedRuns()`, `w3cTraceId()` and `w3cSpanId()` (`nexus-ai-pro/tracing`);
+- `OtlpTraceExporter` and `runToOtlpSpan()` (`nexus-ai-pro/tracing/otlp`). Attributes may be added as
+  the GenAI conventions settle; the ones listed in the tracing guide stay;
+- rollups (`nexus-ai-pro/tracing/rollups`), timed replay (`timing`, `pace`), evaluator provenance
+  (`withProvenance()`, `Experiment.evaluators` and `framework`, `LLMJudge.asEvaluator()`),
+  `createAgent({ streamTokens })`, and `graphAssistant(graph, { events })`;
+- `rateLimit.algorithm` and `burst`, `RateLimitStore.gcra()`, `RateLimitDecision`, and `gcraDecide()`;
+  `health.observationTtlMs`, `ProviderHealthStatus`, a snapshot's `status` and `stale`, and
+  `checkProviders({ staleOnly })`.
+
+The studio's `rollups` source follows the studio's experimental label.
+
 ## 2.0 stage (2.0.0)
 
 2.0 removes what 1.x deprecated and promotes the surfaces that have settled.

package/CHANGELOG.md

@@ -4,6 +4,127 @@ Notable changes to this project are recorded here. The format follows [Keep a Ch
 
 ## [Unreleased]
 
+## [2.1.0] - 2026-10-04
+
+Fault-tolerant graphs and crash-safe observability. A graph now chooses when its checkpoints are
+written, recovers from a node whose retries ran out, tells a hung task from a slow one, and stops
+cleanly when its worker has to go. Traces survive the process that wrote them and reach any
+OpenTelemetry backend, every consumer of a run reads one event stream, and dashboards read hourly
+totals instead of every trace. Nothing changes until it is opted into: the defaults keep their 2.0
+meaning.
+
+Two pieces planned for 2.3 come early: a rate limiter with no window edge to burst across, and
+provider health that expires.
+
+### Added
+
+- **Durability modes.** `compile({ durability })` takes `sync` (the default), `async`, or `exit`.
+  `async` writes checkpoints in the background, in order, while the next superstep runs, and waits
+  for them before the run returns, pauses, fails, or drains. `maxPendingWrites` (8) bounds the writes
+  in flight. `exit` writes only where the run stops. `flush()` waits for every run's writes.
+- **Recovery after retries.** A node's `onError` receives a `NodeFailure` once its retries run out
+  and returns an update or a `Command`, such as a route to a refund node. The decision is
+  checkpointed, as `recovered` on the next checkpoint, before anything it routes to runs, so a crash
+  after it resumes into the same path. `compile({ nodeDefaults })` sets retries, timeouts, and
+  `onError` for every node.
+- **Run and idle timeouts.** `timeout: { runMs, idleMs }` caps an attempt and the time between signs
+  of progress. `context.heartbeat()`, `emit()`, and `report()` refresh the idle timer.
+  `GraphNodeTimeoutError.kind` says which limit fired. A `Send` can override the timeout of the task
+  it creates, and the override is checkpointed with the task.
+- **Cooperative drain.** `RunControl`, passed as `control` to a run, stops it after its current
+  superstep with its checkpoint written, ending in `GraphDrainedError`; `continue()` resumes it
+  anywhere. The agent server drains its graph runs this way when a replica drains with a queue, so
+  they are handed off at once instead of after the drain timeout.
+- **`lintGraph()`** on the new `nexus-ai-pro/graph/lint`, and `nexus graph lint`: static cycles,
+  retried nodes that are not `idempotent` or have no timeout, deferred durability with side effects,
+  an in-process checkpointer under a deployment, unbounded concurrency, and unchecked routes. Nodes
+  can declare `idempotent: true`, and `describe()` reports durability, the checkpointer, and each
+  node's retries, timeouts, and recovery.
+- **Progress that survives a retry.** An operation's `context.heartbeat(details)` stores `details` as
+  `heartbeatDetails`, and the next attempt reads them as `context.previousHeartbeat`.
+- A `task_failed` graph event, and a semantics table in the graph guide that says what each
+  execution primitive repeats after a crash.
+- **Crash-safe traces.** `new Tracer({ incremental: true })` writes each run when it starts and when it
+  ends, so a crashed run stays in the store with every run beneath it that finished, and a live run
+  can be watched. Sampling is decided at the root's start, and an unsampled trace is still kept whole
+  when tail sampling wants it. `closeAbandonedRuns()` closes what a dead process left running, as a
+  `RunAbandoned` error. `tracer.flush()` waits for writes and exports.
+- **Traces across processes.** `handle.traceparent()` and `StartRunOptions.traceparent` carry a trace
+  through a W3C `traceparent` header: an operation's `traceContext`, a remote call, any OpenTelemetry
+  service. `w3cTraceId()` and `w3cSpanId()` give the ids every process derives alike.
+- **OpenTelemetry export**, on the new `nexus-ai-pro/tracing/otlp`. `TracerOptions.exporters` takes any
+  `TraceExporter`, and `OtlpTraceExporter` sends runs to a collector as OTLP/HTTP JSON with no SDK:
+  model calls as `chat <model>` client spans with the GenAI semantic conventions, tools, agents, and
+  graph nodes with theirs, and Nexus-only facts under `nexus.*`. `runToOtlpSpan()` is the conversion.
+- **One event stream.** `graph.events()`, `resumeEvents()`, and `continueEvents()` run a graph as
+  typed events with projections — `values`, `updates`, `messages`, `tools`, `tasks`, `checkpoints`,
+  `custom` — read together or one at a time through `messages()`, `tools()`, `values()`, and
+  `updates()`, with `result` for the outcome. Nodes stream model output with `context.message()`
+  and report tool calls with `context.tool()`. Subgraph events come through with a `namespace` when
+  asked, and `GraphRunOptions.subgraphEvents` does the same for `onEvent`.
+- **Bounded readers.** Every reader of a stream has its own buffer (`maxBuffered`, 1,000), and the run
+  waits for the slowest at each superstep. Within a step, `overflow` coalesces model chunks and state
+  snapshots (the default), drops the oldest or newest, or fails with `GraphStreamOverflowError`.
+  `streamStats()` reports the peak and what was merged or dropped.
+- **Agents stream.** `createAgent()` reports its model's output and every tool call on the stream,
+  token by token with `streamTokens` and a client with `stream()`. `traceGraph()` records the tool
+  calls as `tool` runs inside their node. `graphAssistant(graph, { events })` records chosen
+  projections on each server run.
+- **Rollups**, on the new `nexus-ai-pro/tracing/rollups`: `rollupTraceStore()` adds every finished run
+  to hourly rows per kind, model, provider, and tenant, in `MemoryRollupStore` or any `RollupStore`,
+  with latency buckets for `rollupPercentile()` and `sumRollups()` to total them. The studio's costs
+  view reads them when given `rollups`.
+- **Timed replay.** `recordingFetch({ timing: true })` keeps each chunk of a response and when it came,
+  and `replayFetch({ pace })` replays it at once, at the original pace, scaled, or released chunk by
+  chunk by a function.
+- **GCRA rate limiting.** `rateLimit.algorithm: 'gcra'` spreads calls at `maxRequests` per `windowMs`
+  with at most `burst` at once, so a burst across a window edge can no longer pass twice the limit.
+  Both stores decide it through `gcra()`; `RedisRateLimitStore` runs it as one Lua script on Redis's
+  clock. `gcraDecide()` is the arithmetic, for a store of your own.
+- **Health that expires.** `health.observationTtlMs` makes a provider with no recent calls `unknown`,
+  routed to as one never seen instead of kept down by an old failure or up by an old success.
+  Snapshots carry a `status` (`healthy`, `degraded`, `unhealthy`, `unknown`) and `stale`, and
+  `ai.checkProviders({ staleOnly: true })` refreshes the unknown ones.
+- **Evaluator provenance.** An experiment records each evaluator's position, name, the score keys it
+  produced, and what `withProvenance()` declared — version, judge model, prompt version, temperature,
+  and rubric — and the framework version that ran it. `LLMJudge.asEvaluator()` brings the judge to
+  `evaluate()`, declaring its own provenance.
+- A weekly `live-providers` workflow runs real conformance against every provider with a key in the
+  repository's secrets, DeepSeek included, and opens an issue when one breaks.
+
+### Fixed
+
+- **A manual operation heartbeat no longer strands the result.** `context.heartbeat()` wrote the
+  record behind the runner's back, so the runner's final write failed its sequence check and a
+  successful result was not saved. Heartbeats now go through the runner, one at a time.
+- `continue()` on a finished thread left it marked `running` with an empty result. It now leaves it
+  completed and returns its state.
+- A run started under a parent from another tracer, or another process, was never written: its trace
+  waited for a root that was not there. It is now written as its trace's root in this process.
+
+### Proof
+
+- A crash-injection suite kills a run before, during, and after every checkpoint write and inside
+  nodes, before and after their side effects, in `sync` and `async` mode, on the memory, operation
+  store, SQLite, Postgres, and Redis checkpointers. A fresh process always reaches the state of an
+  uninterrupted run.
+- A saga test: a charge exhausts its retries, `onError` routes to a refund, the process dies during
+  the refund, and the resumed run refunds without charging again.
+- A property test generates 40 graphs with branches, loops, `Send`, retries, and interrupts, and
+  replays each from every checkpoint of its history to the same state.
+- The graph benchmark runs 50 supersteps against an 8 ms store: `async` finishes in about half the
+  time of `sync`, in the same state, and CI fails if it stops beating it by a quarter.
+- A long agent's process dies at its fourteenth step: a second process reads every finished step from
+  the same JSONL store, and `closeAbandonedRuns()` closes the rest.
+- A server and a worker each export to a local OTLP collector, and the collector receives one trace,
+  the worker's run under the server's, with the GenAI attributes on the model call.
+- A reader a hundred times slower than a node that streams 100,000 chunks never holds more than its
+  buffer, under every overflow policy, and coalescing loses no text.
+- The studio's costs view answers over a month of a million runs, held as rollups, in under 200 ms,
+  reading the traces once.
+- In the two milliseconds around a window's edge, fixed windows pass 199 calls of a 100-call limit and
+  GCRA passes 100. A provider down on Monday and idle since is `unknown`, and routed to, on Wednesday.
+
 ## [2.0.0] - 2026-10-02
 
 Consolidation. One lifecycle runs every call of every family, the root import holds the core client
@@ -1373,7 +1494,8 @@ shared policy vocabulary is in place for a future release that revisits this.
 - URL fetching rejects unsafe private-network targets and limits response reads.
 - CLI and audit findings no longer disclose detected secret values.
 
-[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.0.0...HEAD
+[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.1.0...HEAD
+[2.1.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.0.0...v2.1.0
 [2.0.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v1.25.0...v2.0.0
 [1.25.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v1.24.0...v1.25.0
 [1.24.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v1.23.0...v1.24.0

package/MIGRATING.md

@@ -69,7 +69,7 @@ It cannot split a namespace import (`import * as nexus from 'nexus-ai-pro'`) or
 so it lists them. Import each name you use from its subpath instead. Every subpath, and what it costs,
 is in the [packaging guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/packaging.md).
 
-The root import now costs about 323 KB, down from 572 KB.
+The root import now costs about 325 KB, down from 572 KB.
 
 ## Validators and Node types are optional
 

package/README.md

@@ -8,7 +8,7 @@ team can share.
 Import only what you use. The root import is the client and nothing else, and every other capability
 has its own entry point with a size budget CI enforces; [the packaging guide](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/packaging.md)
 publishes what each one costs. The package has no required dependencies: a graph-only application
-loads 51 KB, and installs nothing but this package.
+loads 73 KB, and installs nothing but this package.
 
 - NPM: https://www.npmjs.com/package/nexus-ai-pro
 - GitHub: https://github.com/mkhitar-abrahamyan/nexus-ai
@@ -243,12 +243,16 @@ Shipped so far:
   a codemod that moves your imports;
 - 2.0 consolidation: one lifecycle for every call of every family, a shared spend budget, a root
   import that holds only the client, no required dependencies, and a model registry checked against
-  every provider.
+  every provider;
+- 2.1 fault tolerance and crash-safe observability: checkpoints written on your terms, recovery after
+  retries, idle timeouts and a clean drain, traces that survive a crash and reach any OpenTelemetry
+  backend, one event stream, and dashboard rollups.
 
 Upgrading from 1.x: `npx nexus migrate src --write` moves your imports, and
 [MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers the rest.
-Next, the image family leaves experimental once recorded live conformance passes on all three
-backends.
+Next, 2.2 removes the scaling cliffs the guides document: versioned database migrations, indexed
+queue dispatch, a circuit breaker that counts across workers, a routing index, and one tenant model. The [roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists the
+releases after it.
 
 ## Before Production
 

package/dist/agent/create-agent.d.ts

@@ -2,7 +2,7 @@ import type { OperationLifecycleLike } from '../types/lifecycle.js';
 import { type CompiledGraph } from '../graph/graph.js';
 import type { GraphCheckpointer, NodeContext } from '../types/graph.js';
 import type { CompletionRequest, Message, ToolDefinition } from '../types/messages.js';
-import type { NexusResponse } from '../types/response.js';
+import type { NexusResponse, StreamChunk } from '../types/response.js';
 import type { Store } from '../types/store.js';
 /**
  * An agent as a graph.
@@ -17,6 +17,8 @@ import type { Store } from '../types/store.js';
 export interface AgentModelClient {
     /** Runs one completion. */
     complete(request: CompletionRequest): Promise<NexusResponse>;
+    /** Streams one completion. Used when the agent is created with `streamTokens`. */
+    stream?(request: CompletionRequest): AsyncIterable<StreamChunk>;
 }
 /** A tool call the agent is about to make. */
 export interface AgentToolCall {
@@ -108,6 +110,12 @@ export interface CreateAgentOptions {
      * `ai.lifecycle`. Its model calls are operations of their own.
      */
     lifecycle?: OperationLifecycleLike;
+    /**
+     * Streams the model's output token by token onto the run's `messages` events, through the client's
+     * `stream()`. Without it, or with a client that cannot stream, each answer arrives as one message
+     * event once the model returns. Middleware sees the assembled response either way.
+     */
+    streamTokens?: boolean;
 }
 declare const agentChannels: () => {
     messages: import("../types/graph.js").Channel<Message[]>;

package/dist/agent/create-agent.js

@@ -37,7 +37,12 @@ export function createAgent(options) {
         for (const item of middleware) {
             request = (await item.beforeModel?.({ request, state, store: options.store })) ?? request;
         }
-        let response = await options.client.complete(request);
+        let response = options.streamTokens && options.client.stream
+            ? await streamed(options.client.stream(request), (chunk) => context.message(chunk))
+            : await options.client.complete(request);
+        if (!(options.streamTokens && options.client.stream) && response.content) {
+            context.message({ content: response.content });
+        }
         for (const item of middleware) {
             response = (await item.afterModel?.({ response, state, store: options.store })) ?? response;
         }
@@ -88,9 +93,13 @@ export function createAgent(options) {
                 args = verdict.args;
         }
         const run = async () => executor.execute(description.name, args);
+        context.tool({ phase: 'start', id: call.id, name: description.name, args });
         const result = await middleware.reduceRight((next, item) => item.wrapToolCall
             ? () => Promise.resolve(item.wrapToolCall?.({ ...description, args }, next))
             : next, run)();
+        context.tool(result.ok
+            ? { phase: 'result', id: call.id, name: description.name, result: result.result }
+            : { phase: 'error', id: call.id, name: description.name, error: result.error ?? 'The tool failed' });
         return { messages: [toolMessage(call, result)] };
     }, { ends: ['model'] })
         .setEntry('model')
@@ -156,3 +165,33 @@ function parseArguments(raw) {
         return { ok: false, error: `Tool arguments are not valid JSON (${reason}); the tool was not run` };
     }
 }
+async function streamed(chunks, onChunk) {
+    let content = '';
+    const toolCalls = [];
+    let meta = {};
+    for await (const chunk of chunks) {
+        if (chunk.type === 'text' && chunk.content) {
+            content += chunk.content;
+            onChunk({ kind: 'text', content: chunk.content });
+        }
+        else if (chunk.type === 'reasoning' && chunk.content) {
+            onChunk({ kind: 'reasoning', content: chunk.content });
+        }
+        else if (chunk.type === 'tool_call' && chunk.toolCall) {
+            toolCalls.push(chunk.toolCall);
+        }
+        else if (chunk.type === 'done') {
+            meta = chunk.meta ?? meta;
+        }
+        else if (chunk.type === 'error') {
+            throw new Error(chunk.error ?? 'The model stream failed');
+        }
+    }
+    return {
+        content,
+        role: 'assistant',
+        ...(toolCalls.length ? { toolCalls } : {}),
+        finishReason: toolCalls.length ? 'tool_calls' : 'stop',
+        meta: meta,
+    };
+}

package/dist/cli/graph.d.ts

@@ -0,0 +1,9 @@
+import { type ParsedArgs } from './args.js';
+/**
+ * `nexus graph lint <module>[#export]`.
+ *
+ * The module exports a compiled graph, or a description of one as `describe()` returns it, as the
+ * named export, as `graph`, or as its default. Exits 1 when a finding is an error, or with `--strict`
+ * when any finding is a warning too.
+ */
+export declare function runGraphCommand(subcommand: string | undefined, args: ParsedArgs): Promise<void>;

package/dist/cli/graph.js

@@ -0,0 +1,47 @@
+import { lintGraph } from '../graph/lint.js';
+import { CliUsageError, flagBool, isRecord, listFlag, loadModule, printTable, writeJson, } from './args.js';
+export async function runGraphCommand(subcommand, args) {
+    if (subcommand !== 'lint') {
+        throw new CliUsageError(`Unknown graph command "${subcommand ?? ''}". Use lint.`);
+    }
+    const [target] = args.positionals;
+    if (!target)
+        throw new CliUsageError('nexus graph lint needs a module: nexus graph lint ./graph.mjs[#exportName].');
+    const [file, exportName] = target.split('#');
+    const loaded = await loadModule(file);
+    const graph = (exportName ? loaded[exportName] : (loaded.graph ?? loaded.default ?? loaded));
+    if (!isLintable(graph)) {
+        throw new CliUsageError(`${target} must export a compiled graph, or a graph description, ${exportName ? `as "${exportName}"` : 'as `graph` or its default'}.`);
+    }
+    const findings = lintGraph(graph, {
+        deployed: flagBool(args.flags, 'deployed'),
+        ignore: listFlag(args.flags, 'ignore'),
+    });
+    const failing = findings.filter((finding) => finding.severity === 'error' || (flagBool(args.flags, 'strict') && finding.severity === 'warning'));
+    if (flagBool(args.flags, 'json')) {
+        writeJson({ findings, failed: failing.length > 0 });
+    }
+    else if (findings.length === 0) {
+        console.log('No findings.');
+    }
+    else {
+        printTable(findings.map((finding) => ({
+            severity: finding.severity,
+            code: finding.code,
+            node: finding.node ?? '',
+            message: finding.message,
+        })), ['severity', 'code', 'node', 'message']);
+        console.log('');
+        for (const finding of findings)
+            console.log(`${finding.code}: ${finding.fix}`);
+    }
+    if (failing.length > 0)
+        process.exitCode = 1;
+}
+function isLintable(value) {
+    if (!isRecord(value))
+        return false;
+    if (typeof value.describe === 'function')
+        return true;
+    return Array.isArray(value.nodes) && Array.isArray(value.edges) && Array.isArray(value.dynamic);
+}

package/dist/cli.js

@@ -54,6 +54,12 @@ async function main() {
             await runMigrateCommand(parsed);
             return;
         }
+        case 'graph': {
+            const [subcommand, ...args] = parsed.positionals;
+            const { runGraphCommand } = await import('./cli/graph.js');
+            await runGraphCommand(subcommand, { positionals: args, flags: parsed.flags });
+            return;
+        }
         case 'deploy': {
             const [subcommand, ...args] = parsed.positionals;
             const { runDeployCommand } = await import('./cli/deploy.js');
@@ -402,6 +408,7 @@ Usage:
   nexus traces export --store <runs.jsonl|store.mjs> [--out runs.jsonl] [same filters as list]
   nexus db sql [--adapters operations,store,traces,evaluation,circuits,prompts] [--vector-dimensions 1536]
   nexus migrate [paths...] [--write] [--check] [--json]
+  nexus graph lint <module>[#export] [--deployed] [--strict] [--ignore CODE,CODE] [--json]
   nexus deploy status [assistant] --url <server> [--token t] [--json]
   nexus deploy canary <assistant> <revision> <percent> --url <server> [--reason r] [--version n]
   nexus deploy promote <assistant> <revision> --url <server> [--reason r] [--version n]
@@ -413,7 +420,8 @@ Evaluation modules for "eval run" export { target, dataset, evaluators, summary?
 "eval gate" exits 1 when a metric got worse beyond noise, a new failure appeared, or the datasets differ.
 Trace store modules export any TraceStore as { store }, such as a PostgresTraceStore over the application's own pool.
 "deploy" reads NEXUS_SERVER_URL and NEXUS_SERVER_TOKEN when --url and --token are not given.
-"migrate" moves 1.x root imports to their 2.0 subpaths and reports removed options; --check exits 1 while any remain.`);
+"migrate" moves 1.x root imports to their 2.0 subpaths and reports removed options; --check exits 1 while any remain.
+"graph lint" reads a compiled graph's shape and exits 1 on an error finding, or on a warning with --strict.`);
 }
 main().catch((error) => {
     const message = error instanceof Error ? error.message : String(error);

package/dist/core/nexus.d.ts

@@ -248,9 +248,13 @@ export declare class NexusAI {
     /** Health of every provider seen so far, when health tracking is on. */
     getProviderHealth(): import("../ops/health.js").ProviderHealthSnapshot[];
     /**
-     * Calls each provider's health check and records the result.
+     * Calls each provider's health check and records the result. With `staleOnly`, only the providers
+     * whose health is `unknown` are checked: never called, or not since `health.observationTtlMs`. Run
+     * it on a schedule to keep an idle provider's health current.
      */
-    checkProviders(): Promise<Array<{
+    checkProviders(options?: {
+        staleOnly?: boolean;
+    }): Promise<Array<{
         providerName: string;
         ok: boolean;
         error?: string;

package/dist/core/nexus.js

@@ -547,9 +547,11 @@ export class NexusAI {
     getProviderHealth() {
         return this.health.snapshot();
     }
-    async checkProviders() {
+    async checkProviders(options = {}) {
         const results = [];
         for (const [providerName, provider] of this.providers) {
+            if (options.staleOnly && !this.health.isUnknown(providerName))
+                continue;
             const started = Date.now();
             try {
                 const ok = await provider.healthCheck();

package/dist/evals/judge.d.ts

@@ -1,3 +1,4 @@
+import type { EvaluationContext, Evaluator } from '../types/evaluate.js';
 import type { CompletionRequest } from '../types/messages.js';
 import type { NexusResponse } from '../types/response.js';
 import type { EvalCase, EvalJudge } from './runner.js';
@@ -75,6 +76,16 @@ export declare class LLMJudge<Response = NexusResponse> {
     constructor(options: LLMJudgeOptions<Response>);
     /** Judges one answer. */
     evaluate(input: LLMJudgeInput): Promise<LLMJudgeResult>;
+    /**
+     * The judge as an evaluator for `evaluate()`, scoring each output under `key` (`judge` by
+     * default). `input` turns an evaluation context into what the judge sees; by default the output is
+     * the answer, the example's expected output the reference, and its inputs the question. The
+     * experiment records the judge's model, temperature, rubric, and a hash of its instructions.
+     */
+    asEvaluator(options?: {
+        key?: string;
+        input?: (context: EvaluationContext) => LLMJudgeInput;
+    }): Evaluator;
     /**
      * Adapts the judge to an `EvalRunner` case, optionally with a mapper that picks what the judge
      * sees.

package/dist/evals/judge.js

@@ -1,3 +1,5 @@
+import { createHash } from 'node:crypto';
+import { withProvenance } from '../evaluate/provenance.js';
 const DEFAULT_SYSTEM_PROMPT = [
     'You are a careful evaluation judge.',
     'Return only JSON with these fields: score, passed, rationale, labels.',
@@ -43,6 +45,34 @@ export class LLMJudge {
             modelUsed: modelOf(response),
         });
     }
+    asEvaluator(options = {}) {
+        const evaluator = async (context) => {
+            const input = options.input?.(context) ?? {
+                actual: text(context.output),
+                ...(context.example.expected === undefined ? {} : { expected: text(context.example.expected) }),
+                query: text(context.example.inputs),
+            };
+            const verdict = await this.evaluate(input);
+            return {
+                key: options.key ?? 'judge',
+                score: verdict.score,
+                passed: verdict.passed,
+                ...(verdict.rationale ? { comment: verdict.rationale } : {}),
+            };
+        };
+        return withProvenance(evaluator, {
+            name: options.key ?? 'judge',
+            judge: {
+                model: this.options.model,
+                promptVersion: createHash('sha256')
+                    .update(this.options.systemPrompt ?? 'default')
+                    .digest('hex')
+                    .slice(0, 12),
+                temperature: this.options.temperature ?? 0,
+            },
+            ...(this.options.rubric ? { rubric: this.options.rubric } : {}),
+        });
+    }
     asEvalJudge(mapper) {
         return async (response, testCase) => {
             const input = mapper
@@ -150,3 +180,6 @@ function clamp(value, [min, max]) {
 function midpoint([min, max]) {
     return min + (max - min) / 2;
 }
+function text(value) {
+    return typeof value === 'string' ? value : JSON.stringify(value ?? '');
+}

package/dist/evaluate/index.d.ts

@@ -5,3 +5,4 @@ export { AnnotationQueue, type AnnotationQueueOptions, evaluateOnline, type Onli
 export { evaluate, type EvaluateOptions, type EvaluationTarget, stats, summarize } from './run.js';
 export { type CachedOutput, type EvaluationCache, evaluationCacheKey, FileEvaluationCache, fingerprintOf, MemoryEvaluationCache, type MemoryEvaluationCacheOptions, } from './cache.js';
 export type { Dataset, DatasetExample, DatasetStore, EvaluationContext, EvaluationScore, Evaluator, ExampleResult, Experiment, ExperimentStore, MetricSummary, SummaryEvaluator, } from '../types/evaluate.js';
+export { withProvenance } from './provenance.js';

package/dist/evaluate/index.js

@@ -4,3 +4,4 @@ export { completed, contains, embeddingSimilarity, exactMatch, mustNotMatch, pai
 export { AnnotationQueue, evaluateOnline, } from './review.js';
 export { evaluate, stats, summarize } from './run.js';
 export { evaluationCacheKey, FileEvaluationCache, fingerprintOf, MemoryEvaluationCache, } from './cache.js';
+export { withProvenance } from './provenance.js';

package/dist/evaluate/provenance.d.ts

@@ -0,0 +1,22 @@
+import type { Evaluator, EvaluatorProvenance } from '../types/evaluate.js';
+/**
+ * Says where an evaluator's scores come from: its name and version, the judge model and prompt
+ * version behind it, and its rubric. `evaluate()` records it on the experiment, so a score of 0.87
+ * can be traced to exactly what produced it, and two experiments scored by different judges are not
+ * compared as if they were alike.
+ *
+ * Returns the same evaluator, marked; it runs exactly as before.
+ *
+ * @example
+ * ```ts
+ * const grounded = withProvenance(groundedness, {
+ *   name: 'groundedness',
+ *   version: '3',
+ *   judge: { model: 'claude-sonnet-5-5', promptVersion: 'grounded-v3', temperature: 0 },
+ *   rubric: 'Every claim is supported by a cited passage.',
+ * });
+ * ```
+ */
+export declare function withProvenance<I = unknown, O = unknown>(evaluator: Evaluator<I, O>, provenance: Omit<EvaluatorProvenance, 'keys' | 'position'>): Evaluator<I, O>;
+/** What an evaluator declared through `withProvenance()`, if anything. */
+export declare function declaredProvenance(evaluator: unknown): Omit<EvaluatorProvenance, 'keys' | 'position'> | undefined;

package/dist/evaluate/provenance.js

@@ -0,0 +1,10 @@
+const PROVENANCE = Symbol('nexus.evaluate.provenance');
+export function withProvenance(evaluator, provenance) {
+    Object.defineProperty(evaluator, PROVENANCE, { value: provenance, configurable: true });
+    return evaluator;
+}
+export function declaredProvenance(evaluator) {
+    if (typeof evaluator !== 'function')
+        return undefined;
+    return evaluator[PROVENANCE];
+}

package/dist/evaluate/run.js

@@ -1,5 +1,7 @@
 import { randomBytes } from 'node:crypto';
+import { NEXUS_VERSION } from '../version.js';
 import { evaluationCacheKey } from './cache.js';
+import { declaredProvenance } from './provenance.js';
 export async function evaluate(target, dataset, evaluators, options = {}) {
     if (dataset.examples.length === 0)
         throw new RangeError(`Dataset "${dataset.name}" has no examples`);
@@ -14,8 +16,9 @@ export async function evaluate(target, dataset, evaluators, options = {}) {
         example,
         run,
     })));
+    const produced = evaluators.map(() => new Set());
     await runConcurrently(jobs, options.concurrency ?? 4, async ({ example, run }) => {
-        const result = await evaluateOne(target, example, run, evaluators, options);
+        const result = await evaluateOne(target, example, run, evaluators, options, produced);
         results.push(result);
         options.onResult?.(result);
     }, options.signal);
@@ -35,6 +38,16 @@ export async function evaluate(target, dataset, evaluators, options = {}) {
         results,
         metrics: summarize(results),
         summary,
+        evaluators: evaluators.map((evaluator, position) => {
+            const declared = declaredProvenance(evaluator);
+            return {
+                ...declared,
+                position,
+                name: declared?.name ?? (evaluator.name || `evaluator-${position}`),
+                keys: [...(produced[position] ?? [])],
+            };
+        }),
+        framework: { name: 'nexus-ai-pro', version: NEXUS_VERSION },
         ...(options.metadata ? { metadata: options.metadata } : {}),
         ...(options.cache
             ? {
@@ -57,7 +70,7 @@ function repetitionsOf(example, setting) {
     }
     return count;
 }
-async function evaluateOne(target, example, run, evaluators, options) {
+async function evaluateOne(target, example, run, evaluators, options, produced) {
     const started = Date.now();
     let output;
     let error;
@@ -97,9 +110,12 @@ async function evaluateOne(target, example, run, evaluators, options) {
         });
     }
     const scores = [];
-    for (const evaluator of evaluators) {
+    for (const [position, evaluator] of evaluators.entries()) {
         try {
-            scores.push(...normalizeScores(await evaluator(context), evaluator));
+            const made = normalizeScores(await evaluator(context), evaluator);
+            for (const score of made)
+                produced[position]?.add(score.key);
+            scores.push(...made);
         }
         catch (caught) {
             scores.push({

package/dist/graph/durability.d.ts

@@ -0,0 +1,27 @@
+import type { DurabilityMode } from '../types/graph.js';
+/**
+ * Writes one run's checkpoints under a durability mode.
+ *
+ * `sync` writes each one before returning. `async` chains writes behind each other, so they land in
+ * order, and makes the run wait only when `maxPending` are already in flight. `exit` keeps only the
+ * latest and writes it when the run stops. A write that fails is kept and thrown from the next call,
+ * so a background failure still fails the run.
+ */
+export declare class CheckpointWriter<T> {
+    private readonly write;
+    private readonly mode;
+    private readonly maxPending;
+    private chain;
+    private readonly inFlight;
+    private failure;
+    private unwritten;
+    constructor(write: (checkpoint: T) => Promise<void>, mode: DurabilityMode, maxPending?: number);
+    /**
+     * Hands over a checkpoint. A `durable` one is persisted, with everything before it, before this
+     * resolves, whatever the mode: a pause, a failure, the end of a run, or a recovery decision.
+     */
+    persist(checkpoint: T, durable?: boolean): Promise<void>;
+    /** Waits for every write handed over so far, and writes what `exit` mode was holding. */
+    flush(): Promise<void>;
+    private throwFailure;
+}