agent-inspect 2.1.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,32 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.3.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 22cad5a: Release v2.3.0 with hardened framework adapter paths.
8
+
9
+ This train strengthens the official AI SDK, OpenAI Agents JS, and LangChain/LangGraph integrations with no-network fixtures, clearer local-only defaults, adapter conformance evidence, and adoption-ready recipes. Mastra and NestJS framework packages remain demand-gated; NestJS stays on the structured-log ingestion recipe path for this release.
10
+
11
+ ## 2.2.0
12
+
13
+ ### Minor Changes
14
+
15
+ - efb3fef: Release v2.2.0 with local test reporter artifacts and CI summaries.
16
+
17
+ Adds the public optional `@agent-inspect/vitest` and `@agent-inspect/jest` reporter packages, the shared experimental `agent-inspect/reporters` helpers, and the `agent-inspect ci-summary` workflow for deterministic local reporter manifests and CI artifacts.
18
+
19
+ ## Unreleased
20
+
21
+ ### Draft v2.3.0 Notes
22
+
23
+ - Hardened the official adapter paths for AI SDK, OpenAI Agents JS, and LangChain/LangGraph with no-network recipes, local-only defaults, clearer lifecycle coverage, and executable adapter conformance evidence.
24
+ - AI SDK coverage now includes route-style telemetry factory guidance, per-request integration isolation, tool/stream/error/parallel fixtures, token metadata, and the required `recordInputs: false` / `recordOutputs: false` host settings.
25
+ - OpenAI Agents JS documentation and fixtures distinguish local-only replacement via `setTraceProcessors()` from advanced additional processor usage.
26
+ - LangGraph support remains through `@agent-inspect/langchain`, with graph/node identity, subgraphs, checkpoint/session IDs, stream modes, handoffs, and parallel branch hints covered through callback metadata.
27
+ - Mastra and NestJS framework packages remain explicitly deferred. NestJS support stays on structured-log ingestion unless future demand proves a narrow local-only helper is worth maintaining.
28
+ - No root/core framework dependency, hosted upload, provider call, schema change, or public breaking change is added in this train.
29
+
3
30
  ## 2.1.0
4
31
 
5
32
  ### Minor Changes
package/README.md CHANGED
@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
22
22
 
23
23
  ## Install
24
24
 
25
- Current npm release line: **2.0.x** for the existing public packages. v2.0.0 is the stable trace-contract release: small root API, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit non-destructive migration workflow.
25
+ Current npm release line: **2.1.x** for the existing public packages. v2.1.0 adds deterministic local eval and reusable redaction utilities on top of the stable v2 trace contract: small root API, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit non-destructive migration workflow.
26
26
 
27
27
  ```bash
28
28
  npm install agent-inspect
@@ -240,6 +240,7 @@ AGENT_INSPECT=1 node eval-runner.mjs
240
240
  - **Redact local files** with `agent-inspect redact` or `@agent-inspect/redact` before creating shareable copies.
241
241
  - **Migrate explicitly** with `agent-inspect migrate <trace.jsonl> --to 1.0 --dry-run` or `--output <file>`; originals are never overwritten by default.
242
242
  - **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
243
+ - **Create local CI artifacts** with `agent-inspect artifacts`, and summarize local test-reporter manifests with `agent-inspect ci-summary`.
243
244
  - **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
244
245
  - **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
245
246
  - **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
@@ -308,6 +309,7 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
308
309
  | `report` | Markdown/HTML inspection report (what + timeline + tree) |
309
310
  | `check` / `scan` / `verify-safe` | Deterministic local trace checks and best-effort safety verification |
310
311
  | `artifacts` | Safe local CI artifact bundles and optional step-summary file output |
312
+ | `ci-summary` | Summarize local Vitest/Jest reporter artifact manifests for CI |
311
313
 
312
314
  ![Timeline with slow-step focus for one run](https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/demos/timeline.gif)
313
315
 
@@ -338,6 +340,12 @@ AgentInspect is the **local-first trace workbench** for TypeScript AI agents:
338
340
 
339
341
  Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
340
342
 
343
+ **v2.3 train status:** adapter hardening is ready for release prep. AI SDK, OpenAI Agents JS, and LangChain/LangGraph paths have no-network recipes and executable conformance coverage. Mastra and NestJS framework packages remain demand-gated; NestJS is covered through structured-log ingestion.
344
+
345
+ **Shipped in 2.2.0:** public optional Vitest/Jest reporter packages, shared `agent-inspect/reporters` helpers, and `agent-inspect ci-summary` for deterministic local reporter artifact summaries. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, `@agent-inspect/openai-agents`, `@agent-inspect/redact`, `@agent-inspect/eval`, `@agent-inspect/vitest`, and `@agent-inspect/jest` at **2.2.0**.
346
+
347
+ **Shipped in 2.1.0:** deterministic local eval and redaction utilities. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, `@agent-inspect/openai-agents`, `@agent-inspect/redact`, and `@agent-inspect/eval` at **2.1.0**.
348
+
341
349
  **Shipped in 2.0.0:** stable root API contract, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit trace migration workflow. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, and `@agent-inspect/openai-agents` at **2.0.0**.
342
350
 
343
351
  **Shipped in 1.9.0:** private harness workspace foundation, explain dry-run/local analysis, promoted adapter adoption paths, and the v2 root API slimming plan.
@@ -350,7 +358,7 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
350
358
 
351
359
  **Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
352
360
 
353
- **Roadmap beyond current release work:** v2.1 starts the eval/redact utility triangle, followed by reporters/CI, adapter hardening, sessions/MCP telemetry, guardrails, optional viewer/IDE surfaces, and conditional v3 extensibility. See [ROADMAP.md](ROADMAP.md).
361
+ **Roadmap beyond current release work:** v2.4 adds sessions/MCP telemetry, followed by guardrails, optional viewer/IDE surfaces, and conditional v3 extensibility. See [ROADMAP.md](ROADMAP.md).
354
362
 
355
363
  **Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
356
364
 
@@ -366,6 +374,16 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
366
374
 
367
375
  ## Optional packages
368
376
 
377
+ ### Framework adapters (`@agent-inspect/ai-sdk`, `@agent-inspect/openai-agents`, `@agent-inspect/langchain`)
378
+
379
+ Official framework adapters are optional packages and stay explicit, local-first, and metadata-only by default:
380
+
381
+ - **AI SDK:** pass `agentInspect(...)` through AI SDK telemetry with `recordInputs: false` and `recordOutputs: false`.
382
+ - **OpenAI Agents JS:** use `setTraceProcessors([agentInspectProcessor(...)])` for the documented local-only replacement path.
383
+ - **LangChain/LangGraph:** pass `new AgentInspectCallback(...)` through callbacks; LangGraph support rides through the LangChain callback boundary.
384
+
385
+ No-network recipes: [ai-sdk-local-telemetry](examples/recipes/ai-sdk-local-telemetry/), [ai-sdk-next-route](examples/recipes/ai-sdk-next-route/), [openai-agents-local-tracing](examples/recipes/openai-agents-local-tracing/), and [langgraph-callback-local](examples/recipes/langgraph-callback-local/). Conformance and limits are documented in [docs/ADAPTERS.md](docs/ADAPTERS.md) and [docs/ADAPTER-CONFORMANCE.md](docs/ADAPTER-CONFORMANCE.md).
386
+
369
387
  ### LangChain callback adapter (`@agent-inspect/langchain`)
370
388
 
371
389
  Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. Optional **`stream: true`** records chunk counts and stream duration **without storing full token text by default**. The LangChain adapter ships with 1.x; its programmatic API remains experimental and may evolve independently of the stable core tracing API.
@@ -405,6 +423,12 @@ npx agent-inspect view <run-id> --tui
405
423
 
406
424
  The TUI is available as a separate optional package; its programmatic API is experimental, while the CLI integration (`view --tui`) is the intended usage. Details: [docs/ADAPTERS.md](docs/ADAPTERS.md).
407
425
 
426
+ ### Test reporter artifacts (`@agent-inspect/vitest`, `@agent-inspect/jest`)
427
+
428
+ Optional Vitest/Jest reporter packages are public as of v2.2. They write shared `schemaVersion: "0.1"` reporter manifests with safe relative artifact paths and bounded structural metadata. Use `agent-inspect ci-summary` to summarize those local manifests in CI without reading trace contents or calling GitHub APIs.
429
+
430
+ Reporter artifact behavior and API details are documented in [docs/API.md](docs/API.md) and [docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md).
431
+
408
432
  ## Examples and recipes
409
433
 
410
434
  | Example | Shows |
@@ -431,7 +455,7 @@ The TUI is available as a separate optional package; its programmatic API is exp
431
455
  | [examples/recipes/eval-local-checks](examples/recipes/eval-local-checks) | v2.1 deterministic local eval checks |
432
456
  | [examples/recipes/redact-share-safe-file](examples/recipes/redact-share-safe-file) | v2.1 share-safe local redaction copy |
433
457
  | [examples/recipes/eval-ci-artifacts](examples/recipes/eval-ci-artifacts) | v2.1 eval before safe CI artifacts |
434
- | [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
458
+ | [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | Vitest/Jest reporter artifact patterns |
435
459
  | [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
436
460
  | [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
437
461
 
@@ -2,7 +2,7 @@
2
2
 
3
3
  AgentInspect optional framework adapters must stay local-first, dependency-isolated, and metadata-only by default.
4
4
 
5
- The machine-readable matrix lives at [docs/implementation/adapter-conformance-matrix.json](./implementation/adapter-conformance-matrix.json). It tracks expected coverage for:
5
+ The machine-readable matrix lives at [docs/implementation/adapter-conformance-matrix.json](./implementation/adapter-conformance-matrix.json). The release-evidence fixture manifest lives at [docs/implementation/adapter-conformance-fixtures.json](./implementation/adapter-conformance-fixtures.json). These files track expected coverage for:
6
6
 
7
7
  - run lifecycle
8
8
  - generic steps
@@ -12,19 +12,39 @@ The machine-readable matrix lives at [docs/implementation/adapter-conformance-ma
12
12
  - streaming metadata
13
13
  - metadata bounds and privacy controls
14
14
 
15
- The v1.7 matrix is declarative coverage guidance. v1.8 makes conformance executable and requires canonical-reader round trips before adapter output is used by checks.
15
+ The v1.7 matrix began as declarative coverage guidance. v1.8 made the core conformance path executable and requires canonical-reader round trips before adapter output is used by checks. v2.3 uses that foundation to harden official adapters without adding shallow new adapter packages.
16
16
 
17
17
  Executable shared assertions live in `packages/core/test/adapter-executable-conformance.test.ts` and `packages/core/test/adapter-conformance-utils.ts`. Adapter-specific suites may add deeper fixture coverage, but the shared suite owns the cross-adapter defaults: local-only execution, no raw payload persistence, lifecycle identity, parentage, streaming summaries, token usage where exposed, and reader round trips.
18
18
 
19
+ Release evidence gate:
20
+
21
+ ```bash
22
+ pnpm exec vitest run packages/core/test/adapter-executable-conformance.test.ts packages/core/test/adapter-conformance-matrix.test.ts
23
+ ```
24
+
25
+ This is an internal official-adapter gate, not a third-party certification program.
26
+
19
27
  ## Current matrix
20
28
 
21
29
  | Adapter | Package | Status | Default install mode | Boundary |
22
30
  | --- | --- | --- | --- | --- |
23
- | AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v1.8 correctness hardening pending | AI SDK telemetry integration | optional package peer dependency |
24
- | LangChain | `@agent-inspect/langchain` | implemented experimental | explicit callback | optional package peer dependency |
25
- | OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; public package, with v1.9 publication recovery tracked separately | `setTraceProcessors()` replacement | optional package peer dependency |
31
+ | AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v2.3 priority 1 | AI SDK telemetry integration | optional package peer dependency |
32
+ | OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; v2.3 priority 2 | `setTraceProcessors()` replacement | optional package peer dependency |
33
+ | LangChain | `@agent-inspect/langchain` | implemented experimental; v2.3 priority 3 | explicit callback | optional package peer dependency |
26
34
  | LangGraph | `@agent-inspect/langchain` | fixture-backed through LangChain callback | explicit LangChain callback | existing LangChain adapter first |
27
35
 
36
+ ## v2.3 scorecard
37
+
38
+ | Adapter path | Current coverage | Hardening gap | Decision |
39
+ | ------------ | ---------------- | ------------- | -------- |
40
+ | AI SDK | Shared conformance marks run, step, tool, LLM, error, streaming, and metadata bounds covered. | More adoption-grade fixtures for `generateText`, `streamText`, tool calls, Next.js route usage, parallel calls, abort/error lifecycle, and token metadata. | Harden first. |
41
+ | OpenAI Agents JS | Shared conformance covers run, step, tool, LLM, error, metadata bounds, and local-only replacement metadata; adapter fixtures cover agents, generations, tools, handoffs, guardrails, response, MCP tools, custom, transcription, and speech shapes without provider calls. | Streaming remains planned in the matrix; `addTraceProcessor()` remains documented only as an advanced user-owned additional mode. | Hardened second. |
42
+ | LangChain/LangGraph | LangChain and LangGraph-through-LangChain fixtures cover shared signals plus graph/node identity, subgraphs, checkpoint/session IDs, stream modes, handoffs, and parallel branch hints through callback metadata. | Callback-surface limits remain documented; no separate package is justified yet. | Hardened third. |
43
+ | Mastra | No official package, no conformance fixture, no root dependency. | Demand and extension-point evidence are not yet sufficient for local-only useful traces. | Defer. |
44
+ | NestJS | Structured-log recipe exists; no official framework adapter or conformance fixture. | Current evidence supports log ingestion. Demand may justify a narrow harness/bootstrap helper, not a broad monkey-patching adapter. | Defer package; keep recipe/helper gate. |
45
+
46
+ `@agent-inspect/vitest` and `@agent-inspect/jest` are public reporter packages, not framework trace adapters. Their artifact-manifest behavior is covered by reporter tests and CI-summary tests rather than this adapter conformance matrix.
47
+
28
48
  ## Required defaults
29
49
 
30
50
  - No network behavior.
package/docs/ADAPTERS.md CHANGED
@@ -2,11 +2,25 @@
2
2
 
3
3
  AgentInspect is **framework-agnostic** at its core. Optional adapter packages integrate specific frameworks without monkey-patching, vendor sinks, or network upload.
4
4
 
5
+ ## v2.3 hardening scorecard
6
+
7
+ v2.3 hardens existing official adapters before adding new ones. Priority is based on the current package set, open issue/adoption signals, conformance coverage, and how directly a framework can produce useful local traces without root/core dependencies.
8
+
9
+ | Priority | Adapter path | Decision | v2.3 focus |
10
+ | -------- | ------------ | -------- | ---------- |
11
+ | 1 | AI SDK (`@agent-inspect/ai-sdk`) | Harden first | Improve low-friction `generateText`, `streamText`, tool-call, parallel-call, abort/error, token metadata, and Next.js route coverage while keeping `recordInputs: false` and `recordOutputs: false` as required host controls. |
12
+ | 2 | OpenAI Agents JS (`@agent-inspect/openai-agents`) | Harden second | Make local-only replacement vs additional processor modes unmistakable, with fixtures for agents, generations, tools, handoffs, guardrails, and no default upload confusion. |
13
+ | 3 | LangChain/LangGraph (`@agent-inspect/langchain`) | Harden third | Improve LangGraph-through-LangChain mapping for node identity, subgraphs, checkpoints, stream modes, branches, handoffs, and session/thread IDs without adding a separate package unless the callback surface proves insufficient. |
14
+ | Defer | Mastra | No package in v2.3 | Current evidence does not justify an official package. Revisit only when there is explicit user demand and a verified extension point that avoids hidden monkey-patching, hosted sinks, or root dependencies. |
15
+ | Defer | NestJS | No framework adapter in v2.3 | Keep the supported path at structured-log ingestion via the existing NestJS JSON logging recipe. Revisit a narrow harness/bootstrap helper only with concrete demand; do not add a package only to wrap app bootstrap. |
16
+
17
+ Reporters (`@agent-inspect/vitest` and `@agent-inspect/jest`) are public packages as of v2.2, but they are CI/test artifact reporters rather than framework trace adapters. They stay outside the v2.3 adapter-hardening priority order.
18
+
5
19
  ## Vercel AI SDK (`@agent-inspect/ai-sdk`)
6
20
 
7
- **Status:** experimental adapter — optional package published in the aligned v1.8.0 package set.
21
+ **Status:** experimental adapter — optional package published in the aligned v2.2.0 package set and hardened in the v2.3 adapter train.
8
22
 
9
- The v1.8 train has hardened lifecycle identity and parallel integration isolation. The adapter remains metadata-only: `capture: "preview"` and preview-only redaction options emit diagnostics and fall back to metadata-only capture until bounded free-text previews are implemented.
23
+ The adapter has hardened lifecycle identity and parallel integration isolation. It remains metadata-only: `capture: "preview"` and preview-only redaction options emit diagnostics and fall back to metadata-only capture until bounded free-text previews are implemented.
10
24
 
11
25
  ### Install
12
26
 
@@ -49,9 +63,11 @@ const result = await generateText({
49
63
 
50
64
  [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/) uses AI SDK test utilities only (`MockLanguageModelV3`, `simulateReadableStream`) and writes local v0.2 adapter events for `agent-inspect open`.
51
65
 
66
+ [examples/recipes/ai-sdk-next-route](../examples/recipes/ai-sdk-next-route/) shows a route-style telemetry factory that creates one AgentInspect integration per request while keeping the same no-network, metadata-only defaults.
67
+
52
68
  ### Common host shapes
53
69
 
54
- Use the same explicit telemetry block for route handlers, streaming, and tool calls. The adapter does not wrap providers or change host-call settings.
70
+ Use the same explicit telemetry block shape for route handlers, streaming, and tool calls. Create a fresh `agentInspect(...)` integration per concurrent request/generation. The adapter does not wrap providers or change host-call settings.
55
71
 
56
72
  ```ts
57
73
  const telemetry = {
@@ -83,7 +99,7 @@ Full API: [API.md](./API.md) §11.
83
99
 
84
100
  ## LangChain.js (`@agent-inspect/langchain`)
85
101
 
86
- **Status:** experimental — programmatic API may evolve independently of stable core tracing.
102
+ **Status:** experimental adapter optional package published in the aligned v2.2.0 package set; programmatic API may evolve independently of stable core tracing.
87
103
 
88
104
  ### Install
89
105
 
@@ -209,7 +225,7 @@ Full API: [API.md](./API.md) §9.
209
225
 
210
226
  ### LangGraph boundary
211
227
 
212
- LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. v1.8 adds executable no-network fixtures before claiming broader LangGraph support. A dedicated LangGraph package remains deferred until fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
228
+ LangGraph support rides through this same `@agent-inspect/langchain` callback boundary first. The v2.3 fixtures cover graph/node identity, subgraphs, checkpoint/session IDs, stream modes, handoffs, and parallel branch hints without adding a separate package. A dedicated LangGraph package remains deferred until fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
213
229
 
214
230
  Future LangGraph examples must keep the same safety defaults: explicit callback installation, metadata-only capture, no raw prompt/output/tool payload capture by default, no hosted sink, and local persistence only when `persist: true` is set.
215
231
 
@@ -236,7 +252,7 @@ Requires an interactive terminal. See [API.md](./API.md) §10.
236
252
 
237
253
  ## Vitest (`@agent-inspect/vitest`)
238
254
 
239
- **Status:** experimental workspace package, private/unpublished.
255
+ **Status:** experimental reporter package — optional package published in the aligned v2.2.0 package set.
240
256
 
241
257
  ```bash
242
258
  npm install agent-inspect @agent-inspect/vitest vitest
@@ -285,7 +301,7 @@ Full API: [API.md](./API.md) §12.
285
301
 
286
302
  ## Jest (`@agent-inspect/jest`)
287
303
 
288
- **Status:** experimental workspace package, private/unpublished.
304
+ **Status:** experimental reporter package — optional package published in the aligned v2.2.0 package set.
289
305
 
290
306
  ```bash
291
307
  npm install agent-inspect @agent-inspect/jest jest
@@ -328,7 +344,7 @@ Full API: [API.md](./API.md) §13.
328
344
 
329
345
  ## OpenAI Agents JS (`@agent-inspect/openai-agents`)
330
346
 
331
- **Status:** experimental adapter — optional package published in the aligned v1.8.0 package set.
347
+ **Status:** experimental adapter — optional package published in the aligned v2.2.0 package set.
332
348
 
333
349
  The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md). Install the AgentInspect processor by replacing processors:
334
350
 
@@ -348,14 +364,15 @@ Do not use `addTraceProcessor()` as the default AgentInspect path; that preserve
348
364
 
349
365
  Integration modes:
350
366
 
351
- - **Local-only replacement:** `setTraceProcessors([agentInspectProcessor(...)])` replaces existing processors for the current process. This is the documented safe default when you want AgentInspect to own local trace output.
352
- - **Additional processor:** `addTraceProcessor(agentInspectProcessor(...))` is an advanced, user-owned choice. It can preserve existing/default processors and any backend export behavior they already perform.
367
+ - **Local-only replacement:** `setTraceProcessors([agentInspectProcessor(...)])` replaces existing processors for the current process. This is the documented safe default when you want AgentInspect to own local trace output and avoid preserving the SDK default exporter.
368
+ - **Additional processor:** `addTraceProcessor(agentInspectProcessor(...))` is an advanced, user-owned choice. It can preserve existing/default processors and any backend export behavior they already perform; use it only when that is intentional.
353
369
 
354
370
  - **No auto-install** — importing or constructing `agentInspectProcessor()` never calls `setTraceProcessors()` or `addTraceProcessor()`.
355
371
  - **No upload behavior** — the processor writes only to an explicit local writer or `traceDir`.
356
372
  - **Metadata-only by default** — records trace/span IDs, parentage, names, timing, status, errors, safe model/tool names, token counts, and bounded summaries.
357
373
  - **No raw payload capture by default** — prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, and hosted tool payloads are not persisted.
358
374
  - **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
375
+ - **Fixture-backed lifecycle coverage** — local tests and the recipe cover agent, generation, function tool, handoff, guardrail, response, MCP tools, custom, transcription, and speech span shapes without provider calls.
359
376
 
360
377
  Full API: [API.md](./API.md) §14.
361
378
 
@@ -363,11 +380,38 @@ Runnable local recipe: [openai-agents-local-tracing](../examples/recipes/openai-
363
380
 
364
381
  ---
365
382
 
383
+ ## Demand-gated framework decisions
384
+
385
+ ### Mastra
386
+
387
+ **v2.3 decision:** no official package, recipe, or conformance fixture.
388
+
389
+ The v2.3 evidence review found no open adapter request and no verified extension point in this repository that would produce useful local AgentInspect traces without hidden framework patching or new root/core dependencies. A future Mastra path must first prove:
390
+
391
+ - explicit user demand, such as an issue, design-partner request, or retained external recipe;
392
+ - a stable, framework-native callback/export hook;
393
+ - metadata-only local trace output with no hosted upload or provider calls by default;
394
+ - no dependency leakage into `agent-inspect` root or `@agent-inspect/core`;
395
+ - no raw prompt, output, tool payload, header, or request/response capture by default.
396
+
397
+ Until those are true, Mastra stays outside the official adapter set rather than shipping a shallow package.
398
+
399
+ ### NestJS
400
+
401
+ **v2.3 decision:** no official framework adapter package.
402
+
403
+ NestJS remains covered through structured-log ingestion, not app bootstrap wrapping. The supported recipe is [examples/recipes/nestjs-json-logging](../examples/recipes/nestjs-json-logging), which maps Nest-shaped JSON lines into local execution trees without importing `@nestjs/*`, starting an HTTP server, or changing application behavior.
404
+
405
+ A future Nest helper remains demand-gated and must be narrower than a framework adapter. Acceptable evidence would be a repeated need to reduce logging/harness setup friction while preserving explicit opt-in, local-only output, and zero root/core Nest dependency. Broad interceptors, automatic module scanning, monkey-patching, request body capture, or default telemetry upload remain out of scope.
406
+
407
+ ---
408
+
366
409
  ## Future adapters (not shipped)
367
410
 
368
411
  Direction only — see [ROADMAP.md](../ROADMAP.md):
369
412
 
370
- - **NestJS / logging bridges** — deeper recipes or helper patterns beyond [LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md)
413
+ - **NestJS helper patterns** — only if demand proves a narrow harness/bootstrap helper is worth maintaining beyond [LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md) and the current NestJS logging recipe.
414
+ - **Mastra** — deferred until demand and extension-point evidence justify a narrow explicit integration.
371
415
 
372
416
  No automatic universal instrumentation. Integrations remain explicit and opt-in.
373
417
 
package/docs/API.md CHANGED
@@ -222,9 +222,32 @@ No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-pa
222
222
 
223
223
  Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
224
224
 
225
+ ## 11.1 Experimental `agent-inspect/reporters` APIs
226
+
227
+ `agent-inspect/reporters` contains shared, dependency-free helpers for local test reporter artifacts. The subpath does not import Vitest, Jest, GitHub SDKs, provider SDKs, or upload clients.
228
+
229
+ Import from `agent-inspect/reporters`:
230
+
231
+ ```ts
232
+ import {
233
+ TRACE_ARTIFACT_MANIFEST_SCHEMA_VERSION,
234
+ createReporterArtifactPath,
235
+ createTraceArtifactManifest,
236
+ validateReporterArtifactPath,
237
+ type TraceArtifactManifest,
238
+ } from "agent-inspect/reporters";
239
+ ```
240
+
241
+ - **`TRACE_ARTIFACT_MANIFEST_SCHEMA_VERSION`**: currently `"0.1"` for local reporter manifests.
242
+ - **`createTraceArtifactManifest(options)`**: clones, sorts, and deduplicates reporter results/artifacts into deterministic manifest JSON.
243
+ - **`createReporterArtifactPath(options)`**: creates a safe relative artifact path under a caller-provided output directory.
244
+ - **`validateReporterArtifactPath(options)`**: rejects empty, absolute, traversal, Windows-absolute, and symlink-escape style paths before reporters or `ci-summary` trust artifact links.
245
+
246
+ The manifest records framework, generation time, bounded test results, artifact descriptors, redaction profile, and diagnostics. It is an artifact index only; it should not contain raw trace contents, prompts, model outputs, request/response bodies, headers, API keys, secrets, or full tool payloads.
247
+
225
248
  ## 12. Experimental `@agent-inspect/vitest` APIs
226
249
 
227
- `@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
250
+ `@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished pending maintainer first-publication setup. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
228
251
 
229
252
  Import from `@agent-inspect/vitest`:
230
253
 
@@ -237,6 +260,7 @@ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
237
260
  - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
238
261
  - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
239
262
  - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
263
+ - **`redactionProfile`**: manifest artifact profile, `local` (default), `share`, or `strict`.
240
264
  - **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
241
265
  - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
242
266
  - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
@@ -252,11 +276,11 @@ ctx.task.meta.agentInspect = {
252
276
  };
253
277
  ```
254
278
 
255
- Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
279
+ Artifacts are safe structural summaries. The reporter writes a shared `schemaVersion: "0.1"` manifest wrapper with package metadata, generated time, framework, test results, artifact descriptors, relative paths, and redaction profile. It includes bounded test identity, status, trace run id, and trace filename, but it does not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
256
280
 
257
281
  ## 13. Experimental `@agent-inspect/jest` APIs
258
282
 
259
- `@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
283
+ `@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished pending maintainer first-publication setup. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
260
284
 
261
285
  Import from `@agent-inspect/jest`:
262
286
 
@@ -270,6 +294,7 @@ import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent
270
294
  - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
271
295
  - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
272
296
  - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
297
+ - **`redactionProfile`**: manifest artifact profile, `local` (default), `share`, or `strict`.
273
298
  - **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
274
299
  - **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
275
300
  - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
@@ -294,7 +319,7 @@ reporters: [
294
319
  ],
295
320
  ```
296
321
 
297
- Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
322
+ Artifacts are safe structural summaries. The reporter writes a shared `schemaVersion: "0.1"` manifest wrapper with package metadata, generated time, framework, test results, artifact descriptors, relative paths, and redaction profile. It includes bounded test identity, status, trace run id, and trace filename, but it does not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
298
323
 
299
324
  ## 14. Experimental `@agent-inspect/openai-agents` APIs
300
325
 
package/docs/CLI.md CHANGED
@@ -33,6 +33,7 @@ Core commands:
33
33
  - `scan` — best-effort local safety scan for trace capture risks
34
34
  - `verify-safe` — best-effort local trace safety verification
35
35
  - `artifacts` — create safe local CI trace artifact bundles and optional step summaries
36
+ - `ci-summary` — summarize local reporter artifact manifests for CI
36
37
  - `diff` — compare two manual traces (local, read-only)
37
38
  - `timeline` — chronological view of one run (local JSONL)
38
39
  - `stats` — local aggregate stats over a trace directory
@@ -489,7 +490,33 @@ npx agent-inspect artifacts candidate.jsonl --baseline baseline.jsonl --output-d
489
490
 
490
491
  Recipe and sample workflow: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
491
492
 
492
- ### 6.14 `diff`
493
+ ### 6.14 `ci-summary`
494
+
495
+ Summarize local Vitest/Jest reporter artifact manifests into deterministic Markdown or JSON. This command reads shared `schemaVersion: "0.1"` manifest JSON files only, including the reporter package wrapper emitted by the workspace reporters. It does not read trace contents, rerun tests, upload artifacts, call GitHub APIs, or mutate repository state. `--output` and `--github-summary` write local files.
496
+
497
+ ```bash
498
+ agent-inspect ci-summary <manifest...> [options]
499
+ ```
500
+
501
+ Options:
502
+
503
+ - `-o, --output <path>`: write the Markdown summary to a local file
504
+ - `--github-summary <path>`: append the Markdown summary to a local file, such as `$GITHUB_STEP_SUMMARY`
505
+ - `--json`: print deterministic JSON summary
506
+
507
+ Example:
508
+
509
+ ```bash
510
+ npx agent-inspect ci-summary .agent-inspect/jest-artifacts/tests/**/report.json \
511
+ --output ./artifacts/reporter-summary.md \
512
+ --github-summary "$GITHUB_STEP_SUMMARY"
513
+ ```
514
+
515
+ Reporter artifact paths in the summary are kept relative and validated conservatively. The summary includes bounded package/framework metadata, test identity, status counts, trace filenames, artifact paths, redaction profiles, and diagnostic counts only.
516
+
517
+ Recipe and sample workflow: [examples/recipes/github-actions-artifact](../examples/recipes/github-actions-artifact/README.md)
518
+
519
+ ### 6.15 `diff`
493
520
 
494
521
  Compare two manual trace runs. Diff is **local** and **read-only** (does not rerun agents).
495
522
 
@@ -553,7 +580,7 @@ Differences:
553
580
 
554
581
  More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
555
582
 
556
- ### 6.15 `timeline`
583
+ ### 6.16 `timeline`
557
584
 
558
585
  Chronological step list for one manual trace. Read-only; does not mutate JSONL files.
559
586
 
@@ -569,7 +596,7 @@ Options:
569
596
 
570
597
  ![Timeline with slow-step focus](../assets/demos/timeline.gif)
571
598
 
572
- ### 6.16 `stats`
599
+ ### 6.17 `stats`
573
600
 
574
601
  Local aggregate statistics over trace files in a directory. Read-only.
575
602
 
@@ -589,7 +616,7 @@ Options:
589
616
 
590
617
  Use `--correlation-id` or `--group-id` to filter runs by `run_started` metadata (see [API.md](./API.md)).
591
618
 
592
- ### 6.17 `search`
619
+ ### 6.18 `search`
593
620
 
594
621
  Deterministic search over local traces (substring / exact filters). No semantic search.
595
622
 
@@ -619,7 +646,7 @@ npx agent-inspect search --duration ">100ms" --json
619
646
 
620
647
  ![Search traces by status error](../assets/demos/search.gif)
621
648
 
622
- ### 6.18 `what`
649
+ ### 6.19 `what`
623
650
 
624
651
  Concise human-readable summary of one local trace run. Read-only; accepts v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path. Vocabulary: [TRACE-VOCABULARY-V1.5.md](./proposals/TRACE-VOCABULARY-V1.5.md).
625
652
 
@@ -648,7 +675,7 @@ Outcome: Completed successfully.
648
675
  Slowest: plan (100ms, logic)
649
676
  ```
650
677
 
651
- ### 6.19 `report`
678
+ ### 6.20 `report`
652
679
 
653
680
  Generate a local inspection report combining **what happened**, **timeline**, and **execution tree** sections. The command reads local v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path without mutating them. Distinct from `export` (which targets shareable tree snapshots and standards formats).
654
681
 
@@ -673,7 +700,7 @@ Example:
673
700
  npx agent-inspect report minimal-success --dir fixtures/traces --format html -o report.html
674
701
  ```
675
702
 
676
- ### 6.20 `explain`
703
+ ### 6.21 `explain`
677
704
 
678
705
  Explain a local trace using deterministic facts and local inference labels. This command reads through the same local reader pipeline as `open` / `check`; it does not call a model provider, upload traces, replay agents, or mutate input files.
679
706
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-inspect",
3
- "version": "2.1.0",
3
+ "version": "2.3.0",
4
4
  "license": "MIT",
5
5
  "type": "module",
6
6
  "description": "Local-first execution-tree debugger for TypeScript AI agents",
@@ -105,6 +105,16 @@
105
105
  "types": "./packages/core/dist/checks.d.cts",
106
106
  "default": "./packages/core/dist/checks.cjs"
107
107
  }
108
+ },
109
+ "./reporters": {
110
+ "import": {
111
+ "types": "./packages/core/dist/reporters.d.ts",
112
+ "default": "./packages/core/dist/reporters.mjs"
113
+ },
114
+ "require": {
115
+ "types": "./packages/core/dist/reporters.d.cts",
116
+ "default": "./packages/core/dist/reporters.cjs"
117
+ }
108
118
  }
109
119
  },
110
120
  "bin": {