agent-inspect 1.6.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/CHANGELOG.md +18 -0
  2. package/README.md +12 -5
  3. package/docs/ADAPTER-CONFORMANCE.md +39 -0
  4. package/docs/ADAPTERS.md +194 -1
  5. package/docs/API.md +171 -10
  6. package/docs/CLI.md +154 -6
  7. package/docs/KNOWN-ISSUES.md +9 -0
  8. package/docs/LIMITATIONS.md +14 -0
  9. package/docs/SCHEMA.md +2 -1
  10. package/package.json +13 -2
  11. package/packages/cli/dist/index.cjs +2057 -33
  12. package/packages/cli/dist/index.cjs.map +1 -1
  13. package/packages/cli/dist/index.mjs +2057 -33
  14. package/packages/cli/dist/index.mjs.map +1 -1
  15. package/packages/core/dist/advanced.d.cts +4 -4
  16. package/packages/core/dist/advanced.d.ts +4 -4
  17. package/packages/core/dist/checks.cjs +1535 -0
  18. package/packages/core/dist/checks.cjs.map +1 -0
  19. package/packages/core/dist/checks.d.cts +585 -0
  20. package/packages/core/dist/checks.d.ts +585 -0
  21. package/packages/core/dist/checks.mjs +1512 -0
  22. package/packages/core/dist/checks.mjs.map +1 -0
  23. package/packages/core/dist/diff.d.cts +3 -3
  24. package/packages/core/dist/diff.d.ts +3 -3
  25. package/packages/core/dist/exporters.d.cts +3 -3
  26. package/packages/core/dist/exporters.d.ts +3 -3
  27. package/packages/core/dist/index.d.cts +6 -6
  28. package/packages/core/dist/index.d.ts +6 -6
  29. package/packages/core/dist/{inspect-event-Des4JDHo.d.cts → inspect-event-CevRYp58.d.cts} +1 -1
  30. package/packages/core/dist/{inspect-event-Des4JDHo.d.ts → inspect-event-CevRYp58.d.ts} +1 -1
  31. package/packages/core/dist/{log-config-C1GcJPIM.d.ts → log-config-BPHS4Sds.d.ts} +1 -1
  32. package/packages/core/dist/{log-config-BnH8Ykcb.d.cts → log-config-DanPV3P9.d.cts} +1 -1
  33. package/packages/core/dist/logs.d.cts +3 -3
  34. package/packages/core/dist/logs.d.ts +3 -3
  35. package/packages/core/dist/{persisted-inspect-event-DiFto0K2.d.ts → persisted-inspect-event-Cw7TeYGr.d.ts} +1 -1
  36. package/packages/core/dist/{persisted-inspect-event-0kaRADsp.d.cts → persisted-inspect-event-DHPfzUd8.d.cts} +1 -1
  37. package/packages/core/dist/persisted.d.cts +5 -5
  38. package/packages/core/dist/persisted.d.ts +5 -5
  39. package/packages/core/dist/readers.d.cts +2 -2
  40. package/packages/core/dist/readers.d.ts +2 -2
  41. package/packages/core/dist/{types-tSix7tfv.d.ts → types-Ap9uMdx_.d.ts} +1 -1
  42. package/packages/core/dist/{types-DB8jB6Jg.d.cts → types-B2-BU5CS.d.cts} +1 -1
  43. package/packages/core/dist/writers.d.cts +2 -2
  44. package/packages/core/dist/writers.d.ts +2 -2
package/CHANGELOG.md CHANGED
@@ -1,5 +1,23 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.8.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 0bee42c: Release v1.8.0 with OpenAI Agents trace processor support, optional Vitest/Jest reporter packages kept private, deterministic CI release checks, and the validated local-first reporting improvements from the v1.8 release train.
8
+
9
+ ## 1.7.0
10
+
11
+ Released **2026-06-26**.
12
+
13
+ ### Minor Changes
14
+
15
+ - 94a7220: Release v1.7.0 framework-native adoption with the experimental AI SDK telemetry adapter, declarative adapter conformance matrix, and local-first adapter documentation.
16
+
17
+ ### Notes
18
+
19
+ - The v1.8 train carries the remaining adapter correctness work: AI SDK logical lifecycle identity, parallel integration isolation, explicit capture/redaction behavior, executable conformance fixtures, OpenAI Agents runtime mapping, and LangGraph no-network fixtures. v1.7.0 should not be read as claiming those deferred behaviors.
20
+
3
21
  ## 1.6.0
4
22
 
5
23
  Released **2026-06-25**.
package/README.md CHANGED
@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
22
22
 
23
23
  ## Install
24
24
 
25
- Current npm release: **1.6.0** (`agent-inspect`, `@agent-inspect/langchain`, `@agent-inspect/tui` — all aligned).
25
+ Current npm release: **1.7.0** (`agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui` — all aligned).
26
26
 
27
27
  ```bash
28
28
  npm install agent-inspect
@@ -111,7 +111,7 @@ await maybeInspectRun("eval-case-42", async () => runAgent());
111
111
  AGENT_INSPECT=1 node eval-runner.mjs
112
112
  ```
113
113
 
114
- ## What you can do today (v1.6.0)
114
+ ## What you can do today (v1.7.0)
115
115
 
116
116
  - **Trace manually** with `inspectRun`, `step`, `step.llm`, `step.tool`, and `observe` — local JSONL under `.agent-inspect/` by default.
117
117
  - **Toggle tracing** with `maybeInspectRun` and `AGENT_INSPECT=1` in eval harnesses or CI.
@@ -121,6 +121,7 @@ AGENT_INSPECT=1 node eval-runner.mjs
121
121
  - **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
122
122
  - **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
123
123
  - **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
124
+ - **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
124
125
  - **Optional TUI** — `view --tui` when `@agent-inspect/tui` is installed.
125
126
  - **Persisted-event foundation (v1.2.0+)** — in-memory `PersistedInspectEvent` converters; manual writing stays `schemaVersion: "0.1"`.
126
127
  - **Experimental v1.6.0 APIs** — `agent-inspect/writers`, `agent-inspect/readers`, `createInspector()`, and `agent-inspect open` for local AgentInspect/OpenInference/OTLP ingestion.
@@ -180,6 +181,8 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
180
181
  | `search` | Deterministic search over local traces |
181
182
  | `what` | Concise summary of one run |
182
183
  | `report` | Markdown/HTML inspection report (what + timeline + tree) |
184
+ | `check` / `scan` / `verify-safe` | Deterministic local trace checks and best-effort safety verification |
185
+ | `artifacts` | Safe local CI artifact bundles and optional step-summary file output |
183
186
 
184
187
  ![Timeline with slow-step focus for one run](https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/demos/timeline.gif)
185
188
 
@@ -198,7 +201,7 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
198
201
 
199
202
  ## Stable foundation (AgentInspect 1.x)
200
203
 
201
- **agent-inspect 1.x** (current: **1.6.0**) is the **local-first trace workbench** for TypeScript AI agents:
204
+ **agent-inspect 1.x** (current: **1.7.0**) is the **local-first trace workbench** for TypeScript AI agents:
202
205
 
203
206
  - Instrument runs with `inspectRun` and `step`
204
207
  - Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
@@ -208,11 +211,13 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
208
211
 
209
212
  Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
210
213
 
211
- **Shipped in 1.6.0:** experimental writer subpath (`agent-inspect/writers`), isolated `createInspector()` API via `agent-inspect/advanced`, local trace readers via `agent-inspect/readers`, OpenInference/OTLP JSON readers, universal `agent-inspect open`, and deterministic [runtime-and-ingestion recipe](examples/recipes/runtime-and-ingestion/). These remain local-only and do not add upload behavior. Linked release aligns all three npm packages at **1.6.0**.
214
+ **Shipped in 1.7.0:** experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6 with a local no-network [ai-sdk-local-telemetry recipe](examples/recipes/ai-sdk-local-telemetry/), adapter conformance fixtures, OpenAI Agents/LangGraph support decisions, and local-first adapter docs. Examples keep `recordInputs: false`, `recordOutputs: false`, metadata-only capture, and no upload behavior. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, and `@agent-inspect/tui` at **1.7.0**.
215
+
216
+ **Shipped in 1.6.0:** experimental writer subpath (`agent-inspect/writers`), isolated `createInspector()` API via `agent-inspect/advanced`, local trace readers via `agent-inspect/readers`, OpenInference/OTLP JSON readers, universal `agent-inspect open`, and deterministic [runtime-and-ingestion recipe](examples/recipes/runtime-and-ingestion/). These remain local-only and do not add upload behavior. Linked release aligns all three then-published npm packages at **1.6.0**.
212
217
 
213
218
  **Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
214
219
 
215
- **Roadmap beyond current release work:** future work continues from the local runtime and universal ingestion foundation before broadening framework adapters. See [ROADMAP.md](ROADMAP.md).
220
+ **Roadmap beyond current release work:** future work continues from the local runtime, universal ingestion, and optional adapter foundations. See [ROADMAP.md](ROADMAP.md).
216
221
 
217
222
  **Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
218
223
 
@@ -289,6 +294,8 @@ The TUI is available as a separate optional package; its programmatic API is exp
289
294
  | [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
290
295
  | [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
291
296
  | [examples/recipes/github-actions-artifact](examples/recipes/github-actions-artifact) | CI trace artifacts |
297
+ | [examples/recipes/deterministic-ci-checks](examples/recipes/deterministic-ci-checks) | v1.8 checks, baseline, and safe CI artifacts |
298
+ | [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
292
299
  | [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
293
300
  | [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
294
301
 
@@ -0,0 +1,39 @@
1
+ # Adapter conformance
2
+
3
+ AgentInspect optional framework adapters must stay local-first, dependency-isolated, and metadata-only by default.
4
+
5
+ The machine-readable matrix lives at [docs/implementation/adapter-conformance-matrix.json](./implementation/adapter-conformance-matrix.json). It tracks expected coverage for:
6
+
7
+ - run lifecycle
8
+ - generic steps
9
+ - tools
10
+ - LLM calls
11
+ - errors
12
+ - streaming metadata
13
+ - metadata bounds and privacy controls
14
+
15
+ The v1.7 matrix is declarative coverage guidance. v1.8 makes conformance executable and requires canonical-reader round trips before adapter output is used by checks.
16
+
17
+ Executable shared assertions live in `packages/core/test/adapter-executable-conformance.test.ts` and `packages/core/test/adapter-conformance-utils.ts`. Adapter-specific suites may add deeper fixture coverage, but the shared suite owns the cross-adapter defaults: local-only execution, no raw payload persistence, lifecycle identity, parentage, streaming summaries, token usage where exposed, and reader round trips.
18
+
19
+ ## Current matrix
20
+
21
+ | Adapter | Package | Status | Default install mode | Boundary |
22
+ | --- | --- | --- | --- | --- |
23
+ | AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v1.8 correctness hardening pending | AI SDK telemetry integration | optional package peer dependency |
24
+ | LangChain | `@agent-inspect/langchain` | implemented experimental | explicit callback | optional package peer dependency |
25
+ | OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; private until first publication gate | `setTraceProcessors()` replacement | optional package peer dependency |
26
+ | LangGraph | `@agent-inspect/langchain` | fixture-backed through LangChain callback | explicit LangChain callback | existing LangChain adapter first |
27
+
28
+ ## Required defaults
29
+
30
+ - No network behavior.
31
+ - No default upload behavior.
32
+ - No root/core dependency on framework SDKs.
33
+ - Metadata-only capture by default.
34
+ - Raw prompts, messages, outputs, tool payloads, headers, request bodies, and response bodies must not be persisted by default.
35
+ - Framework-specific fixtures must be no-network and dependency-isolated.
36
+
37
+ ## Before claiming support
38
+
39
+ An adapter can be documented as supported only after no-network fixtures cover run, step, tool, LLM, error, streaming, and metadata-bound expectations for that framework path.
package/docs/ADAPTERS.md CHANGED
@@ -2,6 +2,57 @@
2
2
 
3
3
  AgentInspect is **framework-agnostic** at its core. Optional adapter packages integrate specific frameworks without monkey-patching, vendor sinks, or network upload.
4
4
 
5
+ ## Vercel AI SDK (`@agent-inspect/ai-sdk`)
6
+
7
+ **Status:** experimental v1.7 adapter — optional package published in the v1.7.0 linked release.
8
+
9
+ The v1.8 train has hardened lifecycle identity and parallel integration isolation. The adapter remains metadata-only: `capture: "preview"` and preview-only redaction options emit diagnostics and fall back to metadata-only capture until bounded free-text previews are implemented.
10
+
11
+ ### Install
12
+
13
+ ```bash
14
+ npm install agent-inspect @agent-inspect/ai-sdk ai
15
+ ```
16
+
17
+ ### Local telemetry integration
18
+
19
+ ```ts
20
+ import { generateText } from "ai";
21
+ import { agentInspect } from "@agent-inspect/ai-sdk";
22
+
23
+ const result = await generateText({
24
+ model,
25
+ prompt,
26
+ experimental_telemetry: {
27
+ isEnabled: true,
28
+ recordInputs: false,
29
+ recordOutputs: false,
30
+ integrations: [
31
+ agentInspect({
32
+ traceDir: "./.agent-inspect",
33
+ runName: "support-agent",
34
+ capture: "metadata-only",
35
+ }),
36
+ ],
37
+ },
38
+ });
39
+ ```
40
+
41
+ - **No monkey-patching** — pass the integration explicitly through AI SDK telemetry.
42
+ - **No upload behavior** — the adapter writes only to an explicit local writer or `traceDir`.
43
+ - **Metadata-only by default** — records model, finish reason, token usage, timing, and safe counts/summaries.
44
+ - **Required safe telemetry settings** — set `recordInputs: false` and `recordOutputs: false` on every AI SDK call using this adapter.
45
+ - **No raw payload capture by default** — prompts, messages, generated text, stream chunks, tool inputs/outputs, headers, request bodies, and response bodies are not persisted.
46
+ - **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
47
+
48
+ ### Local no-network recipe
49
+
50
+ [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/) uses AI SDK test utilities only (`MockLanguageModelV3`, `simulateReadableStream`) and writes local v0.2 adapter events for `agent-inspect open`.
51
+
52
+ Full API: [API.md](./API.md) §11.
53
+
54
+ ---
55
+
5
56
  ## LangChain.js (`@agent-inspect/langchain`)
6
57
 
7
58
  **Status:** experimental — programmatic API may evolve independently of stable core tracing.
@@ -64,6 +115,16 @@ npx agent-inspect export <run-id> --format markdown --redaction-profile share
64
115
 
65
116
  Written events use `schemaVersion: "0.1"` manual trace names.
66
117
 
118
+ ### LangGraph through LangChain callbacks
119
+
120
+ LangGraph-shaped callback metadata is covered through the existing `@agent-inspect/langchain` callback boundary. No separate `@agent-inspect/langgraph` package is shipped.
121
+
122
+ - **Explicit callback only** — pass `new AgentInspectCallback(...)` through LangChain/LangGraph callback configuration.
123
+ - **Bounded graph metadata** — known graph, node, subgraph, task, branch, checkpoint, retry, handoff, thread, and session identifiers are copied into `attributes.langGraph` when present.
124
+ - **No full graph state** — checkpoint/task/branch containers are summarized by type/count; raw graph state, prompts, tool payloads, outputs, and stream tokens are not stored in `metadata-only` mode.
125
+ - **Conservative parent mapping** — in-memory events preserve framework `parentRunId`; persisted JSONL maps parents only when the parent callback was seen, and marks unresolved parent mappings in step metadata.
126
+ - **No hosted tracing requirement** — fixtures use structural no-network callback payloads and do not require LangSmith, provider calls, or LangGraph platform services.
127
+
67
128
  ### Capture modes
68
129
 
69
130
  | `capture` | Behavior |
@@ -114,6 +175,14 @@ Full API: [API.md](./API.md) §9.
114
175
 
115
176
  [examples/08-langchain-adapter](../examples/08-langchain-adapter/README.md)
116
177
 
178
+ ### LangGraph boundary
179
+
180
+ LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. v1.8 adds executable no-network fixtures before claiming broader LangGraph support. A dedicated LangGraph package remains deferred until fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
181
+
182
+ Future LangGraph examples must keep the same safety defaults: explicit callback installation, metadata-only capture, no raw prompt/output/tool payload capture by default, no hosted sink, and local persistence only when `persist: true` is set.
183
+
184
+ Decision note: [LANGGRAPH-ADAPTER-BOUNDARY.md](./proposals/LANGGRAPH-ADAPTER-BOUNDARY.md).
185
+
117
186
  ---
118
187
 
119
188
  ## TUI (`@agent-inspect/tui`)
@@ -131,11 +200,134 @@ Requires an interactive terminal. See [API.md](./API.md) §10.
131
200
 
132
201
  ---
133
202
 
203
+ ## Vitest (`@agent-inspect/vitest`)
204
+
205
+ **Status:** experimental v1.8 reporter — optional workspace package, private/unpublished until release readiness.
206
+
207
+ ```bash
208
+ npm install agent-inspect @agent-inspect/vitest vitest
209
+ ```
210
+
211
+ After publication, the reporter creates safe, structural artifacts for failed tests that explicitly attach AgentInspect trace metadata. It never guesses trace files by timestamp and does not read trace contents into artifacts.
212
+
213
+ ```ts
214
+ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
215
+
216
+ export default {
217
+ test: {
218
+ reporters: [
219
+ "default",
220
+ createAgentInspectVitestReporter({
221
+ artifactDir: ".agent-inspect/vitest-artifacts",
222
+ retainSuccessful: 5,
223
+ }),
224
+ ],
225
+ },
226
+ };
227
+ ```
228
+
229
+ Attach explicit metadata from the test harness, or provide `resolveTrace(test)`:
230
+
231
+ ```ts
232
+ test("agent workflow", async (ctx) => {
233
+ ctx.task.meta.agentInspect = {
234
+ runId: "support-agent",
235
+ tracePath: ".agent-inspect/support-agent.jsonl",
236
+ artifactLabel: "support-agent",
237
+ };
238
+ });
239
+ ```
240
+
241
+ - **Explicit association only** — no timestamp matching or directory guessing.
242
+ - **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
243
+ - **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
244
+ - **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Vitest failures.
245
+ - **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Vitest dependency.
246
+ - **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
247
+
248
+ Full API: [API.md](./API.md) §12.
249
+
250
+ ---
251
+
252
+ ## Jest (`@agent-inspect/jest`)
253
+
254
+ **Status:** experimental v1.8 reporter — optional workspace package, private/unpublished until release readiness.
255
+
256
+ ```bash
257
+ npm install agent-inspect @agent-inspect/jest jest
258
+ ```
259
+
260
+ After publication, the reporter creates safe, structural artifacts for failed Jest assertions that explicitly attach AgentInspect trace metadata through a map or resolver. It never guesses trace files by timestamp and does not read trace contents into artifacts.
261
+
262
+ ```js
263
+ module.exports = {
264
+ reporters: [
265
+ "default",
266
+ [
267
+ "@agent-inspect/jest",
268
+ {
269
+ artifactDir: ".agent-inspect/jest-artifacts",
270
+ retainSuccessful: 5,
271
+ associations: {
272
+ "agent.test.cjs::agent suite agent workflow": {
273
+ runId: "support-agent",
274
+ tracePath: ".agent-inspect/support-agent.jsonl",
275
+ },
276
+ },
277
+ },
278
+ ],
279
+ ],
280
+ };
281
+ ```
282
+
283
+ - **Explicit association only** — use `associations` or `resolveTrace(test)`; no timestamp matching or directory guessing.
284
+ - **Jest lifecycle** — processes assertion results from `onTestResult` and aggregated file results from `onRunComplete`.
285
+ - **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
286
+ - **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
287
+ - **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Jest failures.
288
+ - **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Jest dependency.
289
+ - **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
290
+
291
+ Full API: [API.md](./API.md) §13.
292
+
293
+ ---
294
+
295
+ ## OpenAI Agents JS (`@agent-inspect/openai-agents`)
296
+
297
+ **Status:** experimental v1.8 adapter — optional workspace package remains private/unpublished until the manual first-publication gate.
298
+
299
+ The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md). Install the AgentInspect processor by replacing processors:
300
+
301
+ ```ts
302
+ import { setTraceProcessors } from "@openai/agents";
303
+ import { agentInspectProcessor } from "@agent-inspect/openai-agents";
304
+
305
+ setTraceProcessors([
306
+ agentInspectProcessor({
307
+ traceDir: "./.agent-inspect",
308
+ capture: "metadata-only",
309
+ }),
310
+ ]);
311
+ ```
312
+
313
+ Do not use `addTraceProcessor()` as the default AgentInspect path; that preserves the OpenAI default exporter in server runtimes. The processor does not auto-install itself, does not upload, and does not add OpenAI Agents dependencies to root/core.
314
+
315
+ - **No auto-install** — importing or constructing `agentInspectProcessor()` never calls `setTraceProcessors()` or `addTraceProcessor()`.
316
+ - **No upload behavior** — the processor writes only to an explicit local writer or `traceDir`.
317
+ - **Metadata-only by default** — records trace/span IDs, parentage, names, timing, status, errors, safe model/tool names, token counts, and bounded summaries.
318
+ - **No raw payload capture by default** — prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, and hosted tool payloads are not persisted.
319
+ - **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
320
+
321
+ Full API: [API.md](./API.md) §14.
322
+
323
+ Runnable local recipe: [openai-agents-local-tracing](../examples/recipes/openai-agents-local-tracing).
324
+
325
+ ---
326
+
134
327
  ## Future adapters (not shipped)
135
328
 
136
329
  Direction only — see [ROADMAP.md](../ROADMAP.md):
137
330
 
138
- - **Vercel AI SDK** — optional callback-style adapter (metadata-first, no vendor sink)
139
331
  - **NestJS / logging bridges** — deeper recipes or helper patterns beyond [LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md)
140
332
 
141
333
  No automatic universal instrumentation. Integrations remain explicit and opt-in.
@@ -145,5 +337,6 @@ No automatic universal instrumentation. Integrations remain explicit and opt-in.
145
337
  ## Related docs
146
338
 
147
339
  - [API.md](./API.md) — adapter options and stability policy
340
+ - [ADAPTER-CONFORMANCE.md](./ADAPTER-CONFORMANCE.md) — no-network fixture matrix and shared expectations
148
341
  - [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) — review exports before sharing
149
342
  - [LIMITATIONS.md](./LIMITATIONS.md) — LangChain streaming and metadata boundaries
package/docs/API.md CHANGED
@@ -150,7 +150,148 @@ Rationale: v1.x includes one official adapter and **zero production sinks**, so
150
150
 
151
151
  - `runTraceViewer`, `loadTraceForTui`, `buildTuiTraceModel`, etc.
152
152
 
153
- ## 11. Experimental persisted-event foundation (v1.2.0)
153
+ ## 11. Experimental `@agent-inspect/ai-sdk` APIs
154
+
155
+ `@agent-inspect/ai-sdk` is an optional v1.7 adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental and published as part of the v1.7.0 linked release.
156
+
157
+ Import from `@agent-inspect/ai-sdk`:
158
+
159
+ ```ts
160
+ import { agentInspect } from "@agent-inspect/ai-sdk";
161
+ ```
162
+
163
+ - **`agentInspect(options?)`**: returns an AI SDK `TelemetryIntegration` bound with `bindTelemetryIntegration()`.
164
+ - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
165
+ - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
166
+ - **`runName`**: optional local run name.
167
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
168
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
169
+ - **`getDiagnostics()`**: exposes isolated adapter write, lifecycle/configuration, flush, and close failures without throwing into AI SDK callbacks.
170
+ - **`getWriterStats()`**, **`flush()`**, and **`close()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
171
+
172
+ Every AI SDK call using the adapter must keep telemetry local and metadata-only:
173
+
174
+ ```ts
175
+ experimental_telemetry: {
176
+ isEnabled: true,
177
+ recordInputs: false,
178
+ recordOutputs: false,
179
+ integrations: [agentInspect({ traceDir: "./.agent-inspect" })],
180
+ }
181
+ ```
182
+
183
+ The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. It does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`. Unsupported preview capture options are explicit diagnostics and keep this metadata-only behavior.
184
+
185
+ No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-patch is part of this package.
186
+
187
+ Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
188
+
189
+ ## 12. Experimental `@agent-inspect/vitest` APIs
190
+
191
+ `@agent-inspect/vitest` is an optional experimental package for local Vitest failure artifacts. In the v1.8 train it remains private/unpublished until release readiness. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
192
+
193
+ Import from `@agent-inspect/vitest`:
194
+
195
+ ```ts
196
+ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
197
+ ```
198
+
199
+ - **`createAgentInspectVitestReporter(options?)`**: returns a structural Vitest reporter facade with `onTestCaseResult`, `onTaskUpdate`, and `onFinished` hooks.
200
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/vitest-artifacts`.
201
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
202
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
203
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
204
+ - **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
205
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
206
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
207
+ - **`agentInspectVitestReporter`**: alias for `createAgentInspectVitestReporter`.
208
+
209
+ Trace association must be explicit. The default resolver reads `meta.agentInspect`, `meta["agent-inspect"]`, `meta.trace`, `context.meta.agentInspect`, or result metadata when present:
210
+
211
+ ```ts
212
+ ctx.task.meta.agentInspect = {
213
+ runId: "support-agent",
214
+ tracePath: ".agent-inspect/support-agent.jsonl",
215
+ artifactLabel: "support-agent",
216
+ };
217
+ ```
218
+
219
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
220
+
221
+ ## 13. Experimental `@agent-inspect/jest` APIs
222
+
223
+ `@agent-inspect/jest` is an optional experimental package for local Jest failure artifacts. In the v1.8 train it remains private/unpublished until release readiness. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
224
+
225
+ Import from `@agent-inspect/jest`:
226
+
227
+ ```ts
228
+ import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent-inspect/jest";
229
+ ```
230
+
231
+ - **`AgentInspectJestReporter`**: default Jest custom reporter class for `reporters: [["@agent-inspect/jest", options]]`.
232
+ - **`createAgentInspectJestReporter(options?)`**: returns a structural reporter facade with `onTestResult` and `onRunComplete` hooks for tests and custom harnesses.
233
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/jest-artifacts`.
234
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
235
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
236
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
237
+ - **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
238
+ - **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
239
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
240
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
241
+ - **`agentInspectJestReporter`**: alias for `createAgentInspectJestReporter`.
242
+
243
+ Jest association is explicit because Jest assertion results do not expose Vitest-style mutable task metadata:
244
+
245
+ ```js
246
+ reporters: [
247
+ [
248
+ "@agent-inspect/jest",
249
+ {
250
+ associations: {
251
+ "agent.test.cjs::agent suite agent workflow": {
252
+ runId: "support-agent",
253
+ tracePath: ".agent-inspect/support-agent.jsonl",
254
+ },
255
+ },
256
+ },
257
+ ],
258
+ ],
259
+ ```
260
+
261
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
262
+
263
+ ## 14. Experimental `@agent-inspect/openai-agents` APIs
264
+
265
+ `@agent-inspect/openai-agents` is an optional experimental package for OpenAI Agents JS tracing processor integration. In the v1.8 train it remains private/unpublished until the manual first-publication gate, but runtime metadata mapping is implemented locally.
266
+
267
+ Import from `@agent-inspect/openai-agents`:
268
+
269
+ ```ts
270
+ import { agentInspectProcessor } from "@agent-inspect/openai-agents";
271
+ ```
272
+
273
+ - **`agentInspectProcessor(options?)`**: returns a local-only OpenAI Agents `TracingProcessor`.
274
+ - **`installMode`**: always `"setTraceProcessors"` to document the safe replacement install path.
275
+ - **`localOnly`**: always `true`; the processor performs no network I/O and does not install itself globally.
276
+ - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
277
+ - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
278
+ - **`workflowName`**: optional local run name overriding the SDK trace name.
279
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
280
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
281
+ - **`getDiagnostics()`**: exposes isolated processor write, lifecycle/configuration, flush, and shutdown failures without throwing into OpenAI Agents callbacks.
282
+ - **`getWriterStats()`**, **`forceFlush()`**, and **`shutdown()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
283
+
284
+ Safe future usage must replace processors explicitly:
285
+
286
+ ```ts
287
+ setTraceProcessors([agentInspectProcessor({ traceDir: "./.agent-inspect" })]);
288
+ ```
289
+
290
+ Do not use `addTraceProcessor()` as the default AgentInspect path; that leaves existing/default processors in place and can preserve backend export behavior in server runtimes.
291
+
292
+ The processor records local v0.2 persisted events for trace/run, agent, generation/response, function/tool, handoff, guardrail, MCP tools, custom, transcription, and speech span metadata where safely representable. It does not persist raw prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, or hosted tool payloads by default.
293
+
294
+ ## 15. Experimental persisted-event foundation (v1.2.0)
154
295
 
155
296
  These helpers expose the **source-agnostic `PersistedInspectEvent` model** (`schemaVersion: "0.2"`). They are **local-only**, **in-memory**, and **do not change** storage write/read or CLI behavior in v1.2.0.
156
297
 
@@ -176,7 +317,7 @@ Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEv
176
317
  - v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
177
318
  - Inspection read paths normalize v0.1 and v0.2 JSONL for local CLI/API use. v0.2 remains experimental as a persisted-event foundation and is not the default writer.
178
319
 
179
- ## 12. Local observability helpers (v1.4.0+)
320
+ ## 16. Local observability helpers (v1.4.0+)
180
321
 
181
322
  Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1 manual traces remain the default writer; v0.2 persisted-event files are accepted where the shared dual-format read path is used. Local files only.
182
323
 
@@ -186,7 +327,7 @@ Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1
186
327
 
187
328
  CLI wrappers: `agent-inspect timeline`, `stats`, `search` — see [CLI.md](./CLI.md).
188
329
 
189
- ## 13. Report and what helpers (v1.5.0+)
330
+ ## 17. Report and what helpers (v1.5.0+)
190
331
 
191
332
  Read-only helpers for concise inspection summaries and local reports:
192
333
 
@@ -195,7 +336,7 @@ Read-only helpers for concise inspection summaries and local reports:
195
336
 
196
337
  Report redaction profiles are key-based safeguards applied to the complete rendered report input, not only to the tree section. Review generated reports before sharing; this is not compliance-grade DLP.
197
338
 
198
- ## 14. Experimental trace writers (v1.6)
339
+ ## 18. Experimental trace writers (v1.6)
199
340
 
200
341
  Trace writers are the first slice of the v1.6 runtime foundation. They are experimental during v1.x and intended for tests, adapters, and future `createInspector` work.
201
342
 
@@ -227,7 +368,7 @@ import type {
227
368
 
228
369
  No network writer or vendor sink exists in this package.
229
370
 
230
- ## 15. Experimental inspector API/runtime (v1.6)
371
+ ## 19. Experimental inspector API/runtime (v1.6)
231
372
 
232
373
  `createInspector()` is the experimental public instance API for local-first tracing with explicit writers. It owns an instance-specific runtime context, records v0.2 persisted inspect events, preserves application return values/errors, and exposes diagnostics plus deterministic `flush()`/`close()` lifecycle hooks.
233
374
 
@@ -269,7 +410,7 @@ Public methods:
269
410
 
270
411
  These APIs are experimental during v1.x. They do not add a default network writer or vendor sink.
271
412
 
272
- ## 16. Experimental trace readers (v1.6)
413
+ ## 20. Experimental trace readers (v1.6)
273
414
 
274
415
  `agent-inspect/readers` exposes the experimental local trace reader contract and detection pipeline. It includes AgentInspect JSONL for v0.1, v0.2, and mixed local trace files, plus local OpenInference JSON and OTLP JSON compatibility readers.
275
416
 
@@ -300,23 +441,43 @@ import type { TraceReader } from "agent-inspect/readers";
300
441
 
301
442
  The reader contract does not silently accept arbitrary JSON and does not add OTel SDK, database, hosted ingestion, or network upload dependencies.
302
443
 
303
- ## 17. Deprecated APIs
444
+ ## 21. Experimental Checks
445
+
446
+ `agent-inspect/checks` exposes the experimental deterministic trace-check engine foundation. It consumes normalized reader output, runs supplied pure rules in stable order, and returns aggregate findings/diagnostics. It does not read files, discover config, call providers, perform network I/O, mutate inputs, or create a new persisted schema.
447
+
448
+ Import from `agent-inspect/checks`:
449
+
450
+ ```ts
451
+ import { runTraceChecks } from "agent-inspect/checks";
452
+ import type { TraceCheckRule, TraceCheckResult } from "agent-inspect/checks";
453
+ ```
454
+
455
+ - **`runTraceChecks({ read }, { rules?, select?, runId? })`**: executes provided rules against a `TraceReadResult` from `agent-inspect/readers`.
456
+ - **Built-in rule factories**: run, tool, LLM, structure, retrieval, guardrail, decision, safety, and baseline helpers including `createRunStatusRule`, `createToolUsageRule`, `createLlmUsageRule`, `createStructureOrphanRule`, `createStructureCycleRule`, `createStructureRelationshipRule`, `createRetrievalRule`, `createGuardrailRule`, `createDecisionRule`, `createSafetyRawContentRule`, `createSafetySecretPatternRule`, and `createBaselineRegressionRule`.
457
+ - **`TraceCheckRule`**: synchronous pure rule contract.
458
+ - **`TraceCheckResult`**: deterministic aggregate result with findings, evidence, summary counts, and execution diagnostics.
459
+
460
+ The checks API is experimental in v1.x. The `agent-inspect check` CLI uses this API for local reader-backed checks and deterministic JSON output; `agent-inspect artifacts` reuses the same safe findings for local CI artifact bundles and optional step-summary file output. Built-in rules operate on normalized event metadata, tree relationships, bounded summaries, token counts, and normalized baseline facts; safety and baseline findings identify event IDs and field paths rather than emitting raw prompts, outputs, secrets, headers, request/response bodies, or full tool payloads.
461
+
462
+ Recipes: [deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for check/baseline/artifact workflows, and [test-reporter-artifacts](../examples/recipes/test-reporter-artifacts/README.md) for Vitest/Jest reporter configuration patterns.
463
+
464
+ ## 22. Deprecated APIs
304
465
 
305
466
  No deprecated APIs are declared as of 1.4.0.
306
467
 
307
- ## 18. Removal / deprecation policy
468
+ ## 23. Removal / deprecation policy
308
469
 
309
470
  - Stable APIs are not removed in v1.x.
310
471
  - If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
311
472
 
312
- ## 19. Backward compatibility policy
473
+ ## 24. Backward compatibility policy
313
474
 
314
475
  - Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
315
476
  - Additive schema changes are allowed in minor versions.
316
477
  - Breaking changes require a major version.
317
478
  - Unknown fields should be ignored where safe.
318
479
 
319
- ## 20. Examples
480
+ ## 25. Examples
320
481
 
321
482
  ### Minimal manual trace
322
483