fascicle 0.12.3 → 0.12.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +35 -6
- package/README.md +3 -2
- package/dist/agents.d.ts +1 -1
- package/dist/{engine-J4lIxpw5.js → engine-CdXegeqC.js} +164 -26
- package/dist/engine-CdXegeqC.js.map +1 -0
- package/dist/index.d.ts +28 -2
- package/dist/index.js +46 -2
- package/dist/index.js.map +1 -1
- package/dist/mcp.d.ts +1 -1
- package/dist/otel.d.ts +20 -2
- package/dist/otel.js +228 -8
- package/dist/otel.js.map +1 -1
- package/dist/testing.d.ts +1 -1
- package/dist/testing.js +1 -1
- package/dist/{types-DU6cvoA4.d.ts → types-CLigYA62.d.ts} +28 -2
- package/package.json +6 -2
- package/dist/engine-J4lIxpw5.js.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,34 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v0.12.5 — 2026-08-21
|
|
4
|
+
|
|
5
|
+
Everything in v0.12.4 ships here for the first time: that version was tagged but never reached npm, because its publish run failed on the gate described below. Upgrading from v0.12.3 picks up both sections.
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- **A type error only CI could see.** `examples/otel-grafana/main.ts` imports `fascicle/otel`, which had no `tsconfig` `paths` entry, so TypeScript fell through to the package `exports` map and resolved the subpath from `./dist/otel.d.ts`. That file is present on any machine that has built before, so the `types` slot passed locally and failed on CI's clean checkout, where `pnpm check` runs ahead of `pnpm build`. `fascicle/viewer` carried the same gap with no importer to expose it yet. Both point at source now, and the `deps` slot fails if a published `exports` subpath ever loses its mapping again.
|
|
10
|
+
|
|
11
|
+
### Internal
|
|
12
|
+
|
|
13
|
+
- **A failed check job keeps its evidence.** checkride prints one pass/fail line per slot and writes each tool's real output to `.check/`, so a red run named the failing slot without saying why. Every workflow that runs the suite now uploads that directory as an artifact on failure, minus the coverage and mutation report trees that account for most of its size.
|
|
14
|
+
- **The `claude_cli` spawn tests no longer race their own mock.** Both SIGTERM escalator tests slept 150ms to let the mock install its handler before signalling it, which is a bet on how long a cold Node start takes: measured spawn to handler ran from 42ms idle to 127ms under a process spawn storm against that budget. The mock now signals readiness through a file and the tests wait for it.
|
|
15
|
+
|
|
16
|
+
## v0.12.4 — 2026-08-21
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **Per-step model timing, and tokens per second derived from it.** Every step the tool loop produces carries a `StepTiming` block (`started_at`, `duration_ms`, and `first_chunk_ms` on streamed turns), stamped so only the successful attempt is measured: failed attempts, backoff waits, and tool execution never inflate the number. `throughput()` derives tokens per second from a `GenerateResult`, a step array, or a single step, and names its basis (`decode` when every turn streamed, `blended` when any window includes network and prefill). The ollama native adapter passes the daemon's own nanosecond durations through `provider_reported.ollama` verbatim.
|
|
21
|
+
|
|
22
|
+
- **The trajectory stream is dashboard-ready: retry events and opt-in OTel metrics.** `response_received` carries the full flat per-turn record, every absorbed retry becomes a `turn_retry` event (attempt, classified failure kind, HTTP status, and the honored backoff delay), and the closing `engine.generate` span reports step and tool-call counts. On the `fascicle/otel` side, span meta flattens one level so a dashboard can query `fascicle.usage.input_tokens`, and a new opt-in `metrics` option records the GenAI semconv client histograms plus cost and retry instruments, tagged with provider and model.
|
|
23
|
+
|
|
24
|
+
- **A Grafana walkthrough.** `docs/grafana.md` tells the end-to-end story for the `fascicle/otel` bridge: the mental model, a three-step keyless demo against the `grafana/otel-lgtm` container, wiring for a real app, what each metric means, and hand-built panel queries, with an importable dashboard JSON beside the example.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- **Every example lives in its own folder now, with a README and a screenshot of a real run.** The 28 single-file demos moved to `examples/<name>/main.ts`, the seven app workspaces kept their layout and gained screenshots, and the shared agents library got a README. The screenshots are captures rather than mockups: keyless demos as terminal renders, local Ollama and `claude` sessions for the model-backed ones, the viewer UI, and Grafana dashboard and Tempo trace views fed by the demo. Every cross-reference in `docs/`, the README, and the site follows the new paths.
|
|
29
|
+
|
|
30
|
+
- **`structured-output` and `tool-loop` run on the `openrouter` provider.** The pair used to require `ANTHROPIC_API_KEY`; they now read `OPENROUTER_API_KEY`, matching `live-smoke`, and their hand-rolled key guards are gone because `create_engine` already rejects an empty `api_key` with a message that names the fix.
|
|
31
|
+
|
|
3
32
|
## v0.12.3 — 2026-08-19
|
|
4
33
|
|
|
5
34
|
### Changed
|
|
@@ -132,9 +161,9 @@
|
|
|
132
161
|
|
|
133
162
|
- **Compile-time joint checking for literal `sequence` tuples.** Each child of a literal tuple must accept its predecessor's output, and a mismatch errors on the offending element. Arrays built at runtime still degrade to unknown boundaries; annotate the outer `Step<In, Out>` to keep the flow's boundary checked.
|
|
134
163
|
|
|
135
|
-
- **Docs for the layering.** `docs/leaf-arm-spine.md` names the leaf / arm / spine shape and the decision rules at each layer, `docs/advanced-composition.md` covers the demoted tier (`scope`/`stash`/`use`, plain `ensemble`, `tournament`, `improve`/`learn`), and `examples/newsroom.ts` is the vocabulary tour that uses every primary primitive once.
|
|
164
|
+
- **Docs for the layering.** `docs/leaf-arm-spine.md` names the leaf / arm / spine shape and the decision rules at each layer, `docs/advanced-composition.md` covers the demoted tier (`scope`/`stash`/`use`, plain `ensemble`, `tournament`, `improve`/`learn`), and `examples/newsroom/main.ts` is the vocabulary tour that uses every primary primitive once.
|
|
136
165
|
|
|
137
|
-
- **A one-file style pair in the examples.** `examples/
|
|
166
|
+
- **A one-file style pair in the examples.** `examples/release-notes/main.ts` and `examples/release-notes-direct/main.ts` build the same release-notes flow two ways (composed through `chain` and arms, and written straight through as one function), so the cost and the benefit of composing are readable side by side rather than asserted. `docs/blueprint.md` is rebuilt around `chain` and points at the pair.
|
|
138
167
|
|
|
139
168
|
### Changed
|
|
140
169
|
|
|
@@ -392,14 +421,14 @@ work is in the docs surface, the example apps, and the check suite.
|
|
|
392
421
|
|
|
393
422
|
- Releases are now published from CI with npm Trusted Publishing (OIDC) and a signed provenance attestation, gated by a required-reviewer GitHub Environment (`.github/workflows/publish.yaml`). No long-lived npm token exists anywhere; verify a release with `npm audit signatures`. This makes v0.9.0 the first provenance-attested Fascicle publish (see `research/provenance-publish-spec.md`).
|
|
394
423
|
- Agent-layer boundary ADR: Fascicle stays on the single-turn seam (`generateText`/`streamText`) and declines the v7 agent layer (`ToolLoopAgent`, `WorkflowAgent`, `HarnessAgent`, `toolApproval`, `@ai-sdk/otel`), recorded in `research/explorations/2026-07-ai-sdk-agent-layer-boundary.md` and linked from `docs/providers.md`.
|
|
395
|
-
- `examples/
|
|
424
|
+
- `examples/live-smoke/main.ts`: a manual release smoke gate that runs one tool-loop flow streamed and non-streamed against OpenRouter and an OpenAI-compatible backend, checking the tool round trip, stream/record text parity, and usage/cost accounting. Live network, so it stays out of the test suite by design.
|
|
396
425
|
- Dev toolchain sweep: vitest 4.1.10, oxlint 1.73, oxlint-tsgolint 0.24, tsdown 0.22.3, zod 4.4.3, `@types/node` 26, fallow 3, and friends.
|
|
397
426
|
|
|
398
427
|
## v0.8.16 — 2026-07-07
|
|
399
428
|
|
|
400
429
|
### Added
|
|
401
430
|
|
|
402
|
-
- `fascicle/ui` subpath for streaming a run into an AI SDK `useChat` UI: `to_ui_message_response`, `pipe_ui_message_stream_to_response`, and `to_ui_message_chunks` translate trajectory `model_chunk` events onto the AI SDK UI message wire. Ships with `docs/human-in-the-loop.md` and `examples/
|
|
431
|
+
- `fascicle/ui` subpath for streaming a run into an AI SDK `useChat` UI: `to_ui_message_response`, `pipe_ui_message_stream_to_response`, and `to_ui_message_chunks` translate trajectory `model_chunk` events onto the AI SDK UI message wire. Ships with `docs/human-in-the-loop.md` and `examples/hitl-http/main.ts` covering suspend/confirm/resume over HTTP.
|
|
403
432
|
|
|
404
433
|
### Changed
|
|
405
434
|
|
|
@@ -691,7 +720,7 @@ work is in the docs surface, the example apps, and the check suite.
|
|
|
691
720
|
- `judge_equals`, `judge_with`, `judge_llm` in `@repo/composites`: stock judges over `{ input, output, meta }`. `judge_llm` takes a `Step<string, string>` model so composites stays engine-free; users wire their own `model_call({...})` into the judge.
|
|
692
721
|
- `regression_compare`, `read_baseline`, `write_baseline`: diff two `BenchReport`s against `pass_rate`, per-judge means, and a relative cost threshold (default 10%). Doesn't short-circuit; full delta + per-case report.
|
|
693
722
|
- `tee_logger` adapter in `@repo/observability`: fan one `TrajectoryLogger` contract out to N sinks. First sink's `start_span` id is canonical, and per-sink ids are translated back on `end_span`. Sinks that throw don't derail the others.
|
|
694
|
-
- `examples/
|
|
723
|
+
- `examples/bench-reviewer/main.ts` + `bench/reviewer/{cases.json,baseline.json}`: end-to-end driver against `@repo/agents`'s `reviewer`. `WRITE_BASELINE=1` records, subsequent runs compare and exit 1 on regression.
|
|
695
724
|
- Cost rendering in the viewer: per-span cost badges that roll up the tree, plus a header running total (`<n> events · <m> errors · $<total>`). Run filter narrows the total. Cost attribution uses an open-span stack per `run_id`; format is 4 decimals under $0.01, 2 decimals otherwise.
|
|
696
725
|
- `examples/amplify` opt-in viewer push via `AMPLIFY_VIEWER_URL`: tees the on-disk trajectory with `http_logger` when set; standalone runs without the env var keep the existing single-sink behaviour.
|
|
697
726
|
|
|
@@ -724,7 +753,7 @@ work is in the docs surface, the example apps, and the check suite.
|
|
|
724
753
|
## v0.3.3 — 2026-04-29
|
|
725
754
|
|
|
726
755
|
### Added
|
|
727
|
-
- `improve` composite in `@repo/composites`: bounded online self-improvement loop with parallel proposers, structured lessons accumulator, plateau detection, and configurable wall-clock + round budgets. Online counterpart to `learn`. Example at `examples/improve.ts`.
|
|
756
|
+
- `improve` composite in `@repo/composites`: bounded online self-improvement loop with parallel proposers, structured lessons accumulator, plateau detection, and configurable wall-clock + round budgets. Online counterpart to `learn`. Example at `examples/improve/main.ts`.
|
|
728
757
|
- `ensemble_step` composite: Step-based sibling of `ensemble` for cases where scoring is itself a `Step`. Returns `winner_id`, `winner`, structured `winner_scored`, and the full `scored` map.
|
|
729
758
|
|
|
730
759
|
## v0.3.2 — 2026-04-29
|
package/README.md
CHANGED
|
@@ -14,7 +14,7 @@ pnpm add fascicle
|
|
|
14
14
|
|
|
15
15
|
Fascicle is ESM-only and requires Node >= 24. `ai`, `zod`, and every provider SDK are optional peers — install only the ones you use. `pnpm add fascicle` alone builds and runs a flow against `transport: 'native'` or `claude_cli`, with no AI SDK package and no schema library installed at all; schemas accept any [Standard Schema](https://standardschema.dev) — zod, ArkType, Valibot, whichever you already use. See [docs/providers.md](./docs/providers.md).
|
|
16
16
|
|
|
17
|
-
No API key on hand? Several runnable examples need zero keys: [examples/hello.ts](./examples/hello.ts), [examples/
|
|
17
|
+
No API key on hand? Several runnable examples need zero keys: [examples/hello/main.ts](./examples/hello/main.ts), [examples/suspend-resume/main.ts](./examples/suspend-resume/main.ts), and [examples/viewer-demo/main.ts](./examples/viewer-demo/main.ts) use no engine at all, and [examples/newsroom/main.ts](./examples/newsroom/main.ts) runs the full vocabulary against the bundled stub engine from `fascicle/testing`.
|
|
18
18
|
|
|
19
19
|
## A 60-second tour
|
|
20
20
|
|
|
@@ -221,8 +221,9 @@ The canonical worked example is [examples/pr-improve/](./examples/pr-improve/),
|
|
|
221
221
|
- [docs/comparison.md](./docs/comparison.md) — how Fascicle compares to LangChain, Mastra, and others
|
|
222
222
|
- [docs/adoption-decision.md](./docs/adoption-decision.md) weighs whether to adopt Fascicle: the honest case, the risks, and when to reach for something else
|
|
223
223
|
- [docs/roadmap.md](./docs/roadmap.md) — what has shipped, what is being considered, and what Fascicle deliberately will not do
|
|
224
|
-
- [examples/](./examples/) — runnable reference flows; start with [examples/newsroom.ts](./examples/newsroom.ts), the vocabulary tour
|
|
224
|
+
- [examples/](./examples/) — runnable reference flows; start with [examples/newsroom/main.ts](./examples/newsroom/main.ts), the vocabulary tour
|
|
225
225
|
- [docs/viewer.md](./docs/viewer.md) — viewer details and transport options
|
|
226
|
+
- [docs/grafana.md](./docs/grafana.md) — shipping a run's traces and metrics to a Grafana dashboard over OpenTelemetry
|
|
226
227
|
|
|
227
228
|
## Contributing
|
|
228
229
|
|
package/dist/agents.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { a as Step } from "./types-DBpU27sQ.js";
|
|
2
2
|
import { r as ToolSchema } from "./types-BK0UW28o.js";
|
|
3
|
-
import { a as Engine, i as EffortLevel } from "./types-
|
|
3
|
+
import { a as Engine, i as EffortLevel } from "./types-CLigYA62.js";
|
|
4
4
|
|
|
5
5
|
//#region src/agents/define_agent.d.ts
|
|
6
6
|
type AgentBuiltPrompt = string | {
|
|
@@ -387,6 +387,22 @@ function next_backoff(retryable, backoff_policy, attempt) {
|
|
|
387
387
|
return { delay };
|
|
388
388
|
}
|
|
389
389
|
/**
|
|
390
|
+
* Assemble the `on_retry` payload from the classified failure and the backoff
|
|
391
|
+
* the retry will actually wait.
|
|
392
|
+
*/
|
|
393
|
+
function build_attempt_info(retryable, attempt, delay_ms) {
|
|
394
|
+
const info = {
|
|
395
|
+
attempt,
|
|
396
|
+
failure_kind: retryable.kind,
|
|
397
|
+
delay_ms
|
|
398
|
+
};
|
|
399
|
+
if (retryable.kind === "rate_limit" || retryable.kind === "provider_5xx") {
|
|
400
|
+
if (retryable.status !== void 0) info.status = retryable.status;
|
|
401
|
+
}
|
|
402
|
+
if (retryable.kind === "rate_limit" && retryable.retry_after_ms !== void 0) info.retry_after_ms = retryable.retry_after_ms;
|
|
403
|
+
return info;
|
|
404
|
+
}
|
|
405
|
+
/**
|
|
390
406
|
* Retry `fn` under `policy`. `fn` must throw a RetryableError-shaped object to
|
|
391
407
|
* trigger retry; any other thrown value short-circuits as a permanent failure.
|
|
392
408
|
*
|
|
@@ -394,8 +410,13 @@ function next_backoff(retryable, backoff_policy, attempt) {
|
|
|
394
410
|
* `aborted_error`. Returns the value from the last successful `fn()` call. On
|
|
395
411
|
* exhaustion, throws `rate_limit_error` (for 429s) or `provider_error` (for
|
|
396
412
|
* 5xx/network/timeout).
|
|
413
|
+
*
|
|
414
|
+
* `on_retry` fires once per retried failure, just before its backoff wait.
|
|
415
|
+
* It does NOT fire for a first attempt that succeeds, a non-retryable
|
|
416
|
+
* failure, or the final failure that exhausts the policy (those two throw
|
|
417
|
+
* instead), so its call count is exactly the number of extra attempts made.
|
|
397
418
|
*/
|
|
398
|
-
async function retry_with_policy(fn, policy = DEFAULT_RETRY, abort) {
|
|
419
|
+
async function retry_with_policy(fn, policy = DEFAULT_RETRY, abort, on_retry) {
|
|
399
420
|
const backoff_policy = {
|
|
400
421
|
initial_delay_ms: policy.initial_delay_ms,
|
|
401
422
|
max_delay_ms: policy.max_delay_ms,
|
|
@@ -415,6 +436,7 @@ async function retry_with_policy(fn, policy = DEFAULT_RETRY, abort) {
|
|
|
415
436
|
if (attempt >= policy.max_attempts) throw build_exhaustion_error(retryable, attempt, last_rate_limit_after);
|
|
416
437
|
const backoff = next_backoff(retryable, backoff_policy, attempt);
|
|
417
438
|
last_rate_limit_after = backoff.retry_after_ms ?? last_rate_limit_after;
|
|
439
|
+
on_retry?.(build_attempt_info(retryable, attempt, backoff.delay));
|
|
418
440
|
await wait_with_abort(backoff.delay, signal, to_abort_error);
|
|
419
441
|
}
|
|
420
442
|
}
|
|
@@ -2071,6 +2093,28 @@ function map_ollama_usage(payload) {
|
|
|
2071
2093
|
output_tokens: typeof evaluated === "number" ? evaluated : 0
|
|
2072
2094
|
};
|
|
2073
2095
|
}
|
|
2096
|
+
const REPORTED_DURATION_FIELDS = [
|
|
2097
|
+
"total_duration",
|
|
2098
|
+
"load_duration",
|
|
2099
|
+
"prompt_eval_duration",
|
|
2100
|
+
"eval_duration"
|
|
2101
|
+
];
|
|
2102
|
+
/**
|
|
2103
|
+
* Collect the daemon's nanosecond duration fields for verbatim passthrough
|
|
2104
|
+
* under `provider_reported.ollama`: the daemon's own decode/prefill timing,
|
|
2105
|
+
* which is what an honest local tokens-per-second number derives from
|
|
2106
|
+
* (eval_count over eval_duration). Units and names stay the wire's own, per
|
|
2107
|
+
* the provider_reported contract; returns undefined when none are present
|
|
2108
|
+
* (mid-proxy shapes that strip them), never an empty object.
|
|
2109
|
+
*/
|
|
2110
|
+
function map_ollama_reported(payload) {
|
|
2111
|
+
const out = {};
|
|
2112
|
+
for (const key of REPORTED_DURATION_FIELDS) {
|
|
2113
|
+
const value = Reflect.get(payload, key);
|
|
2114
|
+
if (typeof value === "number") out[key] = value;
|
|
2115
|
+
}
|
|
2116
|
+
return Object.keys(out).length > 0 ? out : void 0;
|
|
2117
|
+
}
|
|
2074
2118
|
/**
|
|
2075
2119
|
* This wire sends no tool-call id, so one is synthesized from the step index
|
|
2076
2120
|
* and the call's ordinal within the turn: deterministic, so the streamed and
|
|
@@ -2105,11 +2149,13 @@ function parse_ollama_chat(payload, step_index) {
|
|
|
2105
2149
|
const content = Reflect.get(message, "content");
|
|
2106
2150
|
const raw_tool_calls = Reflect.get(message, "tool_calls");
|
|
2107
2151
|
const tool_calls = Array.isArray(raw_tool_calls) ? raw_tool_calls.map((entry, ordinal) => parse_ollama_tool_call(entry, ordinal, step_index)) : [];
|
|
2152
|
+
const reported = map_ollama_reported(payload);
|
|
2108
2153
|
return {
|
|
2109
2154
|
text: typeof content === "string" ? content : "",
|
|
2110
2155
|
tool_calls,
|
|
2111
2156
|
finish_reason: map_ollama_finish_reason(Reflect.get(payload, "done_reason"), tool_calls.length > 0),
|
|
2112
|
-
usage: map_ollama_usage(payload)
|
|
2157
|
+
usage: map_ollama_usage(payload),
|
|
2158
|
+
...reported !== void 0 ? { provider_reported: { ollama: reported } } : {}
|
|
2113
2159
|
};
|
|
2114
2160
|
}
|
|
2115
2161
|
/**
|
|
@@ -2145,6 +2191,7 @@ function create_ollama_stream_aggregator(step_index, dispatch) {
|
|
|
2145
2191
|
input_tokens: 0,
|
|
2146
2192
|
output_tokens: 0
|
|
2147
2193
|
};
|
|
2194
|
+
let reported;
|
|
2148
2195
|
const on_message = async (message) => {
|
|
2149
2196
|
const content = Reflect.get(message, "content");
|
|
2150
2197
|
if (typeof content === "string" && content.length > 0) {
|
|
@@ -2190,6 +2237,7 @@ function create_ollama_stream_aggregator(step_index, dispatch) {
|
|
|
2190
2237
|
done = true;
|
|
2191
2238
|
done_reason = Reflect.get(frame, "done_reason");
|
|
2192
2239
|
usage = map_ollama_usage(frame);
|
|
2240
|
+
reported = map_ollama_reported(frame);
|
|
2193
2241
|
await dispatch({
|
|
2194
2242
|
kind: "step_finish",
|
|
2195
2243
|
step_index,
|
|
@@ -2216,7 +2264,8 @@ function create_ollama_stream_aggregator(step_index, dispatch) {
|
|
|
2216
2264
|
done: true,
|
|
2217
2265
|
done_reason: done_reason ?? null,
|
|
2218
2266
|
prompt_eval_count: usage.input_tokens,
|
|
2219
|
-
eval_count: usage.output_tokens
|
|
2267
|
+
eval_count: usage.output_tokens,
|
|
2268
|
+
...reported
|
|
2220
2269
|
}, step_index);
|
|
2221
2270
|
}
|
|
2222
2271
|
};
|
|
@@ -4061,7 +4110,10 @@ function start_generate_span(trajectory, meta) {
|
|
|
4061
4110
|
}
|
|
4062
4111
|
/**
|
|
4063
4112
|
* Close the `engine.generate` span with usage, finish reason, resolved model,
|
|
4064
|
-
* or the error message on failure.
|
|
4113
|
+
* or the error message on failure. `step_count` and `tool_call_count` are the
|
|
4114
|
+
* call-level totals (model turns and tool executions in the call), the
|
|
4115
|
+
* span-attribute analog of the GenAI semconv agent metrics
|
|
4116
|
+
* `inference_calls` / `tool_calls`.
|
|
4065
4117
|
*/
|
|
4066
4118
|
function end_generate_span(trajectory, span_id, meta) {
|
|
4067
4119
|
if (trajectory === void 0 || span_id === void 0) return;
|
|
@@ -4082,6 +4134,64 @@ function end_step_span(trajectory, span_id, meta) {
|
|
|
4082
4134
|
trajectory.end_span(span_id, { ...meta });
|
|
4083
4135
|
}
|
|
4084
4136
|
/**
|
|
4137
|
+
* Record a provider response: finish reason, flat token counts, and the
|
|
4138
|
+
* engine-measured turn timing when the turn carried one.
|
|
4139
|
+
*
|
|
4140
|
+
* Every field is a flat number or string on purpose. An OTel span event or a
|
|
4141
|
+
* log pipeline can aggregate `input_tokens` / `duration_ms` /
|
|
4142
|
+
* `first_chunk_ms` directly, where a nested usage object would arrive as an
|
|
4143
|
+
* opaque JSON string. This is the one per-turn event a metrics dashboard
|
|
4144
|
+
* needs: tokens by direction, the cache and reasoning splits when the
|
|
4145
|
+
* provider reports them, and the model-only latency window.
|
|
4146
|
+
*/
|
|
4147
|
+
function record_response_received(trajectory, step_index, turn) {
|
|
4148
|
+
if (trajectory === void 0) return;
|
|
4149
|
+
const { usage, timing } = turn;
|
|
4150
|
+
const event = {
|
|
4151
|
+
kind: "response_received",
|
|
4152
|
+
step_index,
|
|
4153
|
+
input_tokens: usage.input_tokens,
|
|
4154
|
+
output_tokens: usage.output_tokens,
|
|
4155
|
+
finish_reason: turn.finish_reason
|
|
4156
|
+
};
|
|
4157
|
+
if (usage.reasoning_tokens !== void 0) event["reasoning_tokens"] = usage.reasoning_tokens;
|
|
4158
|
+
if (usage.cached_input_tokens !== void 0) event["cached_input_tokens"] = usage.cached_input_tokens;
|
|
4159
|
+
if (usage.cache_write_tokens !== void 0) event["cache_write_tokens"] = usage.cache_write_tokens;
|
|
4160
|
+
if (timing !== void 0) {
|
|
4161
|
+
event["started_at"] = timing.started_at;
|
|
4162
|
+
event["duration_ms"] = timing.duration_ms;
|
|
4163
|
+
if (timing.first_chunk_ms !== void 0) event["first_chunk_ms"] = timing.first_chunk_ms;
|
|
4164
|
+
}
|
|
4165
|
+
trajectory.record({
|
|
4166
|
+
kind: "response_received",
|
|
4167
|
+
...event
|
|
4168
|
+
});
|
|
4169
|
+
}
|
|
4170
|
+
/**
|
|
4171
|
+
* Record one failed-then-retried provider attempt as a `turn_retry` event:
|
|
4172
|
+
* the 1-based attempt count, the classified failure kind, the backoff this
|
|
4173
|
+
* retry waits, and the HTTP status / `Retry-After` when the failure carried
|
|
4174
|
+
* them. This is the event a rate-limit or error-budget dashboard counts;
|
|
4175
|
+
* without it, only the final exhaustion is visible (as a span error), and
|
|
4176
|
+
* every absorbed 429 or 5xx goes unrecorded.
|
|
4177
|
+
*/
|
|
4178
|
+
function record_turn_retry(trajectory, step_index, info) {
|
|
4179
|
+
if (trajectory === void 0) return;
|
|
4180
|
+
const event = {
|
|
4181
|
+
kind: "turn_retry",
|
|
4182
|
+
step_index,
|
|
4183
|
+
attempt: info.attempt,
|
|
4184
|
+
failure_kind: info.failure_kind,
|
|
4185
|
+
delay_ms: info.delay_ms
|
|
4186
|
+
};
|
|
4187
|
+
if (info.status !== void 0) event["status"] = info.status;
|
|
4188
|
+
if (info.retry_after_ms !== void 0) event["retry_after_ms"] = info.retry_after_ms;
|
|
4189
|
+
trajectory.record({
|
|
4190
|
+
kind: "turn_retry",
|
|
4191
|
+
...event
|
|
4192
|
+
});
|
|
4193
|
+
}
|
|
4194
|
+
/**
|
|
4085
4195
|
* Record a tool invocation request: name, call id, input, and duration.
|
|
4086
4196
|
*/
|
|
4087
4197
|
function record_tool_call(trajectory, meta) {
|
|
@@ -5597,12 +5707,7 @@ async function invoke_turn(config, step_index, step_span) {
|
|
|
5597
5707
|
abort: config.abort,
|
|
5598
5708
|
stream: config.stream
|
|
5599
5709
|
});
|
|
5600
|
-
config.trajectory
|
|
5601
|
-
kind: "response_received",
|
|
5602
|
-
step_index,
|
|
5603
|
-
output_tokens: turn.usage.output_tokens,
|
|
5604
|
-
finish_reason: turn.finish_reason
|
|
5605
|
-
});
|
|
5710
|
+
record_response_received(config.trajectory, step_index, turn);
|
|
5606
5711
|
return turn;
|
|
5607
5712
|
} catch (err) {
|
|
5608
5713
|
const message = err instanceof Error ? err.message : String(err);
|
|
@@ -5984,8 +6089,9 @@ function resolve_turn_finish_reason(ctx, salvaged, turn_finish) {
|
|
|
5984
6089
|
return turn_finish;
|
|
5985
6090
|
}
|
|
5986
6091
|
/**
|
|
5987
|
-
* Build one StepRecord (cost derived and recorded,
|
|
5988
|
-
* passed through), push it, notify on_finish_step, and
|
|
6092
|
+
* Build one StepRecord (cost derived and recorded, timing and
|
|
6093
|
+
* provider_reported passed through), push it, notify on_finish_step, and
|
|
6094
|
+
* close the step span.
|
|
5989
6095
|
*/
|
|
5990
6096
|
function push_step_record(config, acc, step_index, turn, tool_calls, finish_reason, step_span) {
|
|
5991
6097
|
const record = {
|
|
@@ -5997,6 +6103,7 @@ function push_step_record(config, acc, step_index, turn, tool_calls, finish_reas
|
|
|
5997
6103
|
};
|
|
5998
6104
|
const breakdown = compute_and_record_cost(config, step_index, turn.usage);
|
|
5999
6105
|
if (breakdown !== void 0) record.cost = breakdown;
|
|
6106
|
+
if (turn.timing !== void 0) record.timing = turn.timing;
|
|
6000
6107
|
if (turn.provider_reported !== void 0) record.provider_reported = turn.provider_reported;
|
|
6001
6108
|
acc.steps.push(record);
|
|
6002
6109
|
if (config.on_finish_step !== void 0) config.on_finish_step(record);
|
|
@@ -6241,12 +6348,24 @@ function arm_turn_timeout(user_abort, turn_timeout_ms) {
|
|
|
6241
6348
|
* actually cancels the in-flight request. Adapters may swap `classify`, but
|
|
6242
6349
|
* never the ladder itself: the engine owns retry, so a hidden adapter-level
|
|
6243
6350
|
* retry would be invisible to anything relying on this ladder.
|
|
6244
|
-
|
|
6245
|
-
|
|
6351
|
+
*
|
|
6352
|
+
* The wrapper also stamps StepTiming here, and only here, because this is the
|
|
6353
|
+
* one place that can see attempt boundaries: started_at re-stamps at every
|
|
6354
|
+
* attempt entry, so failed attempts and backoff waits never inflate
|
|
6355
|
+
* duration_ms. `first_chunk_at` (the builder's Date.now() from its first
|
|
6356
|
+
* dispatched chunk) can only belong to the returning attempt, since any
|
|
6357
|
+
* failure after a chunk has flowed is non-retryable by the ladder below.
|
|
6358
|
+
*/
|
|
6359
|
+
function retry_turn(call_once, args, first_chunk_at, retry_policy, classify, turn_timeout_ms, trajectory) {
|
|
6360
|
+
const has_streamed = () => first_chunk_at() !== void 0;
|
|
6246
6361
|
return retry_with_policy(async () => {
|
|
6362
|
+
const started_at = Date.now();
|
|
6247
6363
|
const deadline = arm_turn_timeout(args.abort, turn_timeout_ms);
|
|
6248
6364
|
try {
|
|
6249
|
-
return
|
|
6365
|
+
return {
|
|
6366
|
+
...await call_once(deadline.signal),
|
|
6367
|
+
timing: build_step_timing(started_at, first_chunk_at())
|
|
6368
|
+
};
|
|
6250
6369
|
} catch (err) {
|
|
6251
6370
|
if (err instanceof on_chunk_error) throw err;
|
|
6252
6371
|
if (args.abort.aborted) throw new aborted_error("aborted", {
|
|
@@ -6260,28 +6379,43 @@ function retry_turn(call_once, args, has_streamed, retry_policy, classify, turn_
|
|
|
6260
6379
|
} finally {
|
|
6261
6380
|
deadline.dispose();
|
|
6262
6381
|
}
|
|
6263
|
-
}, retry_policy, args.abort)
|
|
6382
|
+
}, retry_policy, args.abort, (info) => {
|
|
6383
|
+
record_turn_retry(trajectory, args.step_index, info);
|
|
6384
|
+
});
|
|
6385
|
+
}
|
|
6386
|
+
/**
|
|
6387
|
+
* Assemble one turn's StepTiming from the attempt's start stamp and the
|
|
6388
|
+
* builder's first-chunk stamp (undefined on non-streamed turns).
|
|
6389
|
+
*/
|
|
6390
|
+
function build_step_timing(started_at, first_chunk) {
|
|
6391
|
+
const timing = {
|
|
6392
|
+
started_at,
|
|
6393
|
+
duration_ms: Date.now() - started_at
|
|
6394
|
+
};
|
|
6395
|
+
if (first_chunk !== void 0) timing.first_chunk_ms = first_chunk - started_at;
|
|
6396
|
+
return timing;
|
|
6264
6397
|
}
|
|
6265
6398
|
/**
|
|
6266
6399
|
* Build the ai_sdk-transport InvokeOnce: the SDK turn built by
|
|
6267
6400
|
* providers/ai_sdk/invoke.ts behind the same engine-owned retry_turn wrapper
|
|
6268
6401
|
* the native path uses. The SDK call body lives behind the seam; this builder
|
|
6269
6402
|
* owns only retry and the streamed-output tracking that makes a mid-stream
|
|
6270
|
-
* failure a non-retryable interruption
|
|
6403
|
+
* failure a non-retryable interruption (the same first-chunk stamp doubles as
|
|
6404
|
+
* StepTiming.first_chunk_ms).
|
|
6271
6405
|
*/
|
|
6272
6406
|
function build_ai_sdk_invoke(cfg) {
|
|
6273
6407
|
return async (args) => {
|
|
6274
|
-
let
|
|
6408
|
+
let first_chunk_at;
|
|
6275
6409
|
const call_once = (turn_abort) => cfg.invoke_turn({
|
|
6276
6410
|
step_index: args.step_index,
|
|
6277
6411
|
messages: args.messages,
|
|
6278
6412
|
abort: turn_abort,
|
|
6279
6413
|
stream: args.stream,
|
|
6280
6414
|
on_first_chunk: () => {
|
|
6281
|
-
|
|
6415
|
+
first_chunk_at ??= Date.now();
|
|
6282
6416
|
}
|
|
6283
6417
|
});
|
|
6284
|
-
return await retry_turn(call_once, args, () =>
|
|
6418
|
+
return await retry_turn(call_once, args, () => first_chunk_at, cfg.retry_policy, classify_provider_error, cfg.turn_timeout_ms, cfg.trajectory);
|
|
6285
6419
|
};
|
|
6286
6420
|
}
|
|
6287
6421
|
/**
|
|
@@ -6295,7 +6429,7 @@ function build_ai_sdk_invoke(cfg) {
|
|
|
6295
6429
|
function build_native_invoke(cfg) {
|
|
6296
6430
|
const classify = cfg.adapter.classify_error ?? classify_provider_error;
|
|
6297
6431
|
return async (args) => {
|
|
6298
|
-
let
|
|
6432
|
+
let first_chunk_at;
|
|
6299
6433
|
const call_once = async (turn_abort) => {
|
|
6300
6434
|
const internal_controller = new AbortController();
|
|
6301
6435
|
const cancel_on_turn_abort = () => {
|
|
@@ -6304,7 +6438,7 @@ function build_native_invoke(cfg) {
|
|
|
6304
6438
|
if (turn_abort.aborted) internal_controller.abort(turn_abort.reason);
|
|
6305
6439
|
else turn_abort.addEventListener("abort", cancel_on_turn_abort, { once: true });
|
|
6306
6440
|
const dispatch_chunk = async (chunk) => {
|
|
6307
|
-
|
|
6441
|
+
first_chunk_at ??= Date.now();
|
|
6308
6442
|
try {
|
|
6309
6443
|
await cfg.dispatcher.dispatch(chunk);
|
|
6310
6444
|
} catch (err) {
|
|
@@ -6335,7 +6469,7 @@ function build_native_invoke(cfg) {
|
|
|
6335
6469
|
turn_abort.removeEventListener("abort", cancel_on_turn_abort);
|
|
6336
6470
|
}
|
|
6337
6471
|
};
|
|
6338
|
-
return await retry_turn(call_once, args, () =>
|
|
6472
|
+
return await retry_turn(call_once, args, () => first_chunk_at, cfg.retry_policy, classify, cfg.turn_timeout_ms, cfg.trajectory);
|
|
6339
6473
|
};
|
|
6340
6474
|
}
|
|
6341
6475
|
/**
|
|
@@ -6470,7 +6604,8 @@ async function build_ai_sdk_transport(cfg) {
|
|
|
6470
6604
|
telemetry: cfg.telemetry
|
|
6471
6605
|
}),
|
|
6472
6606
|
retry_policy: cfg.retry_policy,
|
|
6473
|
-
turn_timeout_ms: cfg.turn_timeout_ms
|
|
6607
|
+
turn_timeout_ms: cfg.turn_timeout_ms,
|
|
6608
|
+
trajectory: cfg.trajectory
|
|
6474
6609
|
});
|
|
6475
6610
|
}
|
|
6476
6611
|
/**
|
|
@@ -6684,6 +6819,7 @@ async function generate(opts_in, engine) {
|
|
|
6684
6819
|
model_id: target.model_id,
|
|
6685
6820
|
retry_policy: turn.retry_policy,
|
|
6686
6821
|
turn_timeout_ms: turn.turn_timeout_ms,
|
|
6822
|
+
trajectory,
|
|
6687
6823
|
dispatcher,
|
|
6688
6824
|
effort: turn.effort,
|
|
6689
6825
|
schema: opts.schema,
|
|
@@ -6729,7 +6865,9 @@ async function generate(opts_in, engine) {
|
|
|
6729
6865
|
end_generate_span(trajectory, generate_span, {
|
|
6730
6866
|
usage: result.usage,
|
|
6731
6867
|
finish_reason: outcome.finish_reason,
|
|
6732
|
-
model_resolved: result.model_resolved
|
|
6868
|
+
model_resolved: result.model_resolved,
|
|
6869
|
+
step_count: result.steps.length,
|
|
6870
|
+
tool_call_count: result.tool_calls.length
|
|
6733
6871
|
});
|
|
6734
6872
|
return result;
|
|
6735
6873
|
} catch (err) {
|
|
@@ -6997,4 +7135,4 @@ function create_engine(config) {
|
|
|
6997
7135
|
//#endregion
|
|
6998
7136
|
export { create_engine as t };
|
|
6999
7137
|
|
|
7000
|
-
//# sourceMappingURL=engine-
|
|
7138
|
+
//# sourceMappingURL=engine-CdXegeqC.js.map
|