@noetaris/harness 0.6.0 → 0.8.0

package/README.md

@@ -249,18 +249,24 @@ paused in its store — clean it up or reuse it the next time the step runs.
 An `Observer` receives telemetry hooks for a run. Pass it as `observer` in the resources
 of `agent.run()` or `agent.resume()`. Combine several with `composeObservers([a, b])`;
 a hook that throws is reported to `onObserverError` (or `console.error`) and never stops
-the run. Every hook is optional.
+the run. The observer that LLM adapters call is isolated the same way, so an observer that
+throws on an adapter event never fails a step. Every hook is optional.
 
 | Hook | Fires |
 |---|---|
 | `onRunStart(ctx)` | once when a run starts |
-| `onRunEnd(ctx, { signal, durationMs })` | once when a run settles (`$stopped` for a stop) |
+| `onRunEnd(ctx, { signal, durationMs, error? })` | once when a run settles — completed, stopped (`$stopped`), paused, or rejected by a thrown error (`$error`). `error` is set when the run ends with `$error` because of an error no `onError` step handled (paused) or a rejection; a route that returns `'$error'` itself gives no `error`, so test for `error`, not the signal |
 | `onStepStart(ctx)` | before each step |
 | `onStepEnd(ctx, { durationMs })` | after a step's `run` succeeds, before its `route` |
 | `onStepError(ctx, { error, durationMs })` | when a step's `run` throws |
 | `onInterrupt(ctx, { prompt, interruptId })` | when a step calls `ctx.interrupt()` |
 | `onEvent(ctx, type, payload)` | on `ctx.emit()` and adapter events such as `llm.response` |
 | `onStepSettled(ctx, event)` | once per step, when its outcome and destination are final |
+| `wrap(ctx, scope, fn)` | around a step's `run`, and around calls made with `ctx.within` |
+
+Step hooks receive a `StepContext`: `agentId`, `sessionId`, `runId`, `stepName` and, inside a
+fork branch, `branchPath`. `runId` is the same value as `RunContext.runId`. Steps inside a branch
+carry the top-level run's id, and a resumed run has a new one.
 
 **`onStepSettled`** is the one hook that tells you, for every step, how it ended and
 where the run goes next. It fires exactly once for each `onStepStart`, on every path:
@@ -271,7 +277,8 @@ onStepStart → onStepEnd | onStepError | onInterrupt → onStepSettled → onRu
 
 It also fires for a fork node and for each step inside a branch (`ctx.branchPath` names the
 branch). A graph-definition error (`UnknownSignalError`, `NoNextStepError`,
-`MissingReducerError`) settles with `next.kind === 'throw'` before the run rejects. It does
+`MissingReducerError`) settles with `next.kind === 'throw'` before the run rejects; `onRunEnd`
+then fires with `'$error'` and the run rejects with the same error. It does
 not fire when a run stops before starting a step, or for a finished branch that a resume
 skips.
 
@@ -307,6 +314,68 @@ await createAgent('demo', h, {}).run({}, { observer: logger })
 // decide: ok end (signal done)
 ```
 
+### Running code inside a span
+
+Hooks are notifications: they return before the step runs, so they cannot make a span the
+active context for the code inside it. `wrap` is the one hook that encloses work.
+
+```ts
+const timing: Observer = {
+  async wrap(ctx, scope, fn) {
+    const label = scope.kind === 'step' ? ctx.stepName : `${ctx.stepName}/${scope.type}`
+    console.log(`enter ${label}`)
+    try {
+      return await fn()
+    } finally {
+      console.log(`exit ${label}`)
+    }
+  },
+}
+
+const h = createHarness()({ done: field({ default: () => false }) }).loop(l =>
+  l.start()
+   .step('work', {
+     run: async (_s, ctx) => {
+       await ctx.within('tool.call', { toolCallId: 't1' }, async () => { /* call the tool */ })
+       return { done: true }
+     },
+     route: () => 'done',
+   })
+   .on('done').end()
+)
+
+await createAgent('demo', h, {}).run({}, { observer: timing })
+// enter work
+// enter work/tool.call
+// exit work/tool.call
+// exit work
+```
+
+- `scope` is `{ kind: 'step' }` for a step's `run`, or `{ kind: 'event', type, payload }` for code
+  that asks for a scope with `ctx.within(type, payload, fn)`. `type` is the name of the event you
+  emit for that call (for example `'tool.call'`) and `payload` carries its id (`{ toolCallId }`).
+- `ctx.within` only runs `fn` when no observer has `wrap`. For tool calls, `runTool` from
+  [`@noetaris/harness-types`](https://github.com/noetaris-lab/harness-types) emits the tool events
+  and calls `ctx.within` for you.
+- Only a step's `run` is wrapped. `route` runs after the wrap has exited, and a fork node is not
+  wrapped; each step inside a branch is wrapped with its own `ctx.branchPath`.
+- With `composeObservers([a, b])`, `a` is the outer scope.
+- LLM adapters use the same primitive: the `TelemetryContext` an `ObserverAware` slot receives from
+  `withTelemetry` carries a `within` bound to the current step.
+
+The harness never trusts a wrapper. `fn` always runs exactly once, and the caller always gets
+`fn`'s own result:
+
+| The wrapper... | What happens |
+|---|---|
+| throws before calling `fn` | the error goes to `onObserverError`; `fn` runs without the scope |
+| never calls `fn` | `fn` runs without the scope |
+| calls `fn` twice | `fn` runs once |
+| returns another value, or swallows `fn`'s error | ignored; the caller gets `fn`'s result or error |
+| does not await `fn` | the harness still waits for `fn` |
+| throws after `fn` settled | reported to `onObserverError`, unless it is `fn`'s own error passed through |
+| never settles | the step never finishes — a wrapper must settle |
+
 ## API
 
 | Export | Description |
@@ -322,9 +391,11 @@ await createAgent('demo', h, {}).run({}, { observer: logger })
 | `Observer` | Interface for telemetry hooks on run and step lifecycle events (see [Observers](#observers)). |
 | `StepSettledEvent` | Payload of `Observer.onStepSettled` — outcome, duration, update, signal, next, error, interrupt, fork. |
 | `StepSettledNext` | Where the run goes after a step settles: `step`, `end`, `pause`, or `throw`. |
-| `ObserverAware` | Interface for provider objects that accept an `Observer` binding via `bindObserver()`. |
-| `RunContext` | Context passed to run-level observer hooks — `agentId`, `sessionId`. |
-| `StepContext` | Context passed to step-level observer hooks — `agentId`, `sessionId`, `stepName`. |
+| `ObserverAware` | Interface for provider objects that want per-step telemetry: the harness calls `withTelemetry(telemetry)` once per step and installs the returned view on `ctx`. |
+| `TelemetryContext` | What `withTelemetry` receives: `observer` (the run's observer, fault-isolated), `stepContext`, and `within(type, payload, fn)` bound to that step. |
+| `WrapScope` | What `Observer.wrap` is asked to enclose: `{ kind: 'step' }` or `{ kind: 'event', type, payload }`. |
+| `RunContext` | Context passed to run-level observer hooks — `agentId`, `sessionId`, `runId`, `parentRunId?`, `instanceId?`, `branchPath?`. |
+| `StepContext` | Context passed to step-level observer hooks — `agentId`, `sessionId`, `runId`, `stepName`, `branchPath?`. |
 | `NoInterruptError` | Thrown when `resume()` is called but the session is not paused on a matching interrupt. |
 | `SessionInFlightError` | Thrown when a session is already running. |
 | `SessionPendingInterruptError` | Thrown when a session is paused on a pending interrupt — use `agent.resume()` instead of `agent.run()`. |
@@ -353,6 +424,7 @@ See [LIMITATIONS.md](LIMITATIONS.md) for known limitations.
 - [`@noetaris/harness-openai`](https://github.com/noetaris-lab/harness-openai) — OpenAI adapter
 - [`@noetaris/harness-google`](https://github.com/noetaris-lab/harness-google) — Google Gemini adapter
 - [`@noetaris/harness-otel`](https://github.com/noetaris-lab/harness-otel) — OpenTelemetry observer bridge
+- [`@noetaris/harness-otel-genai`](https://github.com/noetaris-lab/harness-otel-genai) — GenAI semantic-conventions extension for the bridge
 
 ## License
 

package/dist/index.d.ts

@@ -137,6 +137,13 @@ interface RunContext {
 interface StepContext {
     readonly agentId: string;
     readonly sessionId: string;
+    /**
+     * The runId of the run this step belongs to — the same value as `RunContext.runId`. Steps
+     * inside fork branches carry the top-level run's id (tell branches apart with `branchPath`).
+     * A resumed run has a new id. Empty string only when the loop is driven without a runId
+     * (direct `runLoop` use), as for `RunContext.runId`.
+     */
+    readonly runId: string;
     readonly stepName: string;
     /**
      * Branch-name path (outermost to innermost), present only when this step runs inside a
@@ -201,6 +208,21 @@ interface StepSettledEvent {
         }[];
     };
 }
+/**
+ * What {@link Observer.wrap} is asked to enclose.
+ *
+ * - `step` — one step's `run()`. Hooks (`onStepStart`, `onStepEnd`, …) and `route()` run outside it.
+ * - `event` — a call inside a step that an event already opened, named by that event's `type`
+ *   (e.g. `'tool.call'`, `'llm.request'`), with a `payload` carrying the id the event used
+ *   (e.g. `{ toolCallId }`, `{ callId }`). Requested via `ctx.within()` or `TelemetryContext.within()`.
+ */
+type WrapScope = {
+    readonly kind: 'step';
+} | {
+    readonly kind: 'event';
+    readonly type: string;
+    readonly payload: unknown;
+};
 /**
  * Observability hook interface.  All methods are optional — implement only
  * the hooks you need.
@@ -223,10 +245,21 @@ interface StepSettledEvent {
 interface Observer {
     /** Called once when a run begins, before the first step executes. */
     onRunStart?: (ctx: RunContext) => void;
-    /** Called once when a run settles (completed or stopped). */
+    /**
+     * Called once when a run settles — completed, stopped, paused, or rejected by a thrown error
+     * (a graph-definition error or a throwing callback).
+     *
+     * When the run ends because of an error, `signal` is `'$error'` and `error` holds it: the error
+     * a step's `run` or `route` threw, or a failed fork's branch error, when no `onError` step
+     * handled it (the run resolves paused); or the original thrown value when the run rejects (it
+     * rejects right after). `error` is absent on every other ending. A route may also return
+     * `'$error'` as an ordinary signal; that ending has no `error` key, so test for `error`, not
+     * for the signal.
+     */
     onRunEnd?: (ctx: RunContext, event: {
         signal: string;
         durationMs: number;
+        error?: unknown;
     }) => void;
     /** Called immediately before each step's `run` function is invoked. */
     onStepStart?: (ctx: StepContext) => void;
@@ -256,6 +289,21 @@ interface Observer {
      * (with `ctx.branchPath`). Not fired when a run stops before starting a step.
      */
     onStepSettled?: (ctx: StepContext, event: StepSettledEvent) => void;
+    /**
+     * Run `fn` inside this observer's scope — e.g. make a span the active context so code in
+     * `fn` (auto-instrumented clients, sub agents) nests under it. The only hook that encloses
+     * work rather than being notified about it.
+     *
+     * Call `fn` at most once and return its promise. The harness never trusts the wrapper:
+     * `fn` always runs exactly once, and the caller always gets `fn`'s own result. A wrapper
+     * that throws before calling `fn` is reported via `onObserverError` (hook name `'wrap'`) and
+     * `fn` runs unwrapped; one that never calls `fn` just loses its scope; one that swallows or
+     * replaces `fn`'s result is ignored. A wrapper error after `fn` was called is reported too,
+     * unless it is `fn`'s own rejection passed through.
+     *
+     * Not guarded: a wrapper that never settles hangs the step it encloses.
+     */
+    wrap?: <T>(ctx: StepContext, scope: WrapScope, fn: () => Promise<T>) => Promise<T>;
 }
 /**
  * The telemetry a resource needs to attribute an `invoke()` call to the correct
@@ -266,8 +314,18 @@ interface Observer {
  * each call carries its own context by value.
  */
 interface TelemetryContext {
+    /**
+     * The run's observer, fault-isolated: a hook that throws is reported via `onObserverError`
+     * and never reaches the caller. Not the object passed to `agent.run()` itself.
+     */
     readonly observer: Observer;
     readonly stepContext: StepContext;
+    /**
+     * Run `fn` inside the observer's scope for the event `type` (e.g. `'llm.request'`), whose
+     * `payload` carries that event's id (e.g. `{ callId }`). Bound to this step. A passthrough
+     * when the observer has no `wrap`; wrapper faults are isolated as for {@link Observer.wrap}.
+     */
+    within<T>(type: string, payload: unknown, fn: () => Promise<T>): Promise<T>;
 }
 /**
  * Implemented by resources (e.g. LLM adapters) that emit per-step telemetry.
@@ -613,6 +671,12 @@ type RunFn<S, Ctx> = (state: StepState<S>, ctx: Ctx & {
     readonly signal: AbortSignal;
     readonly interrupt: (prompt: unknown, id?: string) => Promise<unknown>;
     readonly emit: (name: string, payload?: unknown) => void;
+    /**
+     * Run `fn` inside the observer's scope for an event this step already emitted — `type` is
+     * that event's name (e.g. `'tool.call'`), `payload` carries its id (e.g. `{ toolCallId }`).
+     * Returns `fn`'s own result. A passthrough when no observer implements `wrap`.
+     */
+    readonly within: <T>(type: string, payload: unknown, fn: () => Promise<T>) => Promise<T>;
     readonly keepAlive: KeepAliveFn;
 }) => Promise<Partial<Omit<S, '$error' | '$interrupt'>>> | Partial<Omit<S, '$error' | '$interrupt'>>;
 /**
@@ -1046,4 +1110,4 @@ declare class LeaseExpiredError extends Error {
     constructor(sessionId: string);
 }
 
-export { type Agent, type BranchCursor, type BranchDef, type ClaimOptions, type Cursor, type DeepWithMarkers, type FieldDefinition, type ForkCursor, type ForkDef, type FrameworkState, type Harness, type Lease, LeaseExpiredError, type LoopDefinition, type LoopNode, LoopNotDefinedError, NoInterruptError, type Observer, type ObserverAware, type ObserverErrorContext, type ObserverErrorSink, REQUIRED_TAG, RUNTIME_TAG, type RequiredMarker, type RouteFn, type RunContext, type RunFn, type RuntimeMarker, SessionBusyError, SessionInFlightError, SessionPendingInterruptError, type SessionStore, type SignalTransition, type StateFromSchema, type StepContext, type StepDef, type StepSettledEvent, type StepSettledNext, type StepState, StoreLoadError, type StoredRun, type StoredRunMetadata, type TelemetryContext, type TransitionTarget, composeObservers, createAgent, createHarness, field, isForkCursor, isForkDef, isRequiredMarker, isRuntimeMarker, required, runtime };
+export { type Agent, type BranchCursor, type BranchDef, type ClaimOptions, type Cursor, type DeepWithMarkers, type FieldDefinition, type ForkCursor, type ForkDef, type FrameworkState, type Harness, type Lease, LeaseExpiredError, type LoopDefinition, type LoopNode, LoopNotDefinedError, NoInterruptError, type Observer, type ObserverAware, type ObserverErrorContext, type ObserverErrorSink, REQUIRED_TAG, RUNTIME_TAG, type RequiredMarker, type RouteFn, type RunContext, type RunFn, type RuntimeMarker, SessionBusyError, SessionInFlightError, SessionPendingInterruptError, type SessionStore, type SignalTransition, type StateFromSchema, type StepContext, type StepDef, type StepSettledEvent, type StepSettledNext, type StepState, StoreLoadError, type StoredRun, type StoredRunMetadata, type TelemetryContext, type TransitionTarget, type WrapScope, composeObservers, createAgent, createHarness, field, isForkCursor, isForkDef, isRequiredMarker, isRuntimeMarker, required, runtime };