agent-inspect 1.7.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/CHANGELOG.md +13 -1
  2. package/README.md +11 -6
  3. package/docs/ADAPTER-CONFORMANCE.md +7 -3
  4. package/docs/ADAPTERS.md +120 -5
  5. package/docs/API.md +123 -21
  6. package/docs/CLI.md +154 -6
  7. package/docs/KNOWN-ISSUES.md +7 -1
  8. package/docs/LIMITATIONS.md +7 -1
  9. package/docs/SCHEMA.md +1 -0
  10. package/package.json +12 -2
  11. package/packages/cli/dist/index.cjs +2057 -33
  12. package/packages/cli/dist/index.cjs.map +1 -1
  13. package/packages/cli/dist/index.mjs +2057 -33
  14. package/packages/cli/dist/index.mjs.map +1 -1
  15. package/packages/core/dist/advanced.d.cts +4 -4
  16. package/packages/core/dist/advanced.d.ts +4 -4
  17. package/packages/core/dist/checks.cjs +1535 -0
  18. package/packages/core/dist/checks.cjs.map +1 -0
  19. package/packages/core/dist/checks.d.cts +585 -0
  20. package/packages/core/dist/checks.d.ts +585 -0
  21. package/packages/core/dist/checks.mjs +1512 -0
  22. package/packages/core/dist/checks.mjs.map +1 -0
  23. package/packages/core/dist/diff.d.cts +3 -3
  24. package/packages/core/dist/diff.d.ts +3 -3
  25. package/packages/core/dist/exporters.d.cts +3 -3
  26. package/packages/core/dist/exporters.d.ts +3 -3
  27. package/packages/core/dist/index.d.cts +6 -6
  28. package/packages/core/dist/index.d.ts +6 -6
  29. package/packages/core/dist/{inspect-event-Des4JDHo.d.cts → inspect-event-CevRYp58.d.cts} +1 -1
  30. package/packages/core/dist/{inspect-event-Des4JDHo.d.ts → inspect-event-CevRYp58.d.ts} +1 -1
  31. package/packages/core/dist/{log-config-C1GcJPIM.d.ts → log-config-BPHS4Sds.d.ts} +1 -1
  32. package/packages/core/dist/{log-config-BnH8Ykcb.d.cts → log-config-DanPV3P9.d.cts} +1 -1
  33. package/packages/core/dist/logs.d.cts +3 -3
  34. package/packages/core/dist/logs.d.ts +3 -3
  35. package/packages/core/dist/{persisted-inspect-event-DiFto0K2.d.ts → persisted-inspect-event-Cw7TeYGr.d.ts} +1 -1
  36. package/packages/core/dist/{persisted-inspect-event-0kaRADsp.d.cts → persisted-inspect-event-DHPfzUd8.d.cts} +1 -1
  37. package/packages/core/dist/persisted.d.cts +5 -5
  38. package/packages/core/dist/persisted.d.ts +5 -5
  39. package/packages/core/dist/readers.d.cts +2 -2
  40. package/packages/core/dist/readers.d.ts +2 -2
  41. package/packages/core/dist/{types-tSix7tfv.d.ts → types-Ap9uMdx_.d.ts} +1 -1
  42. package/packages/core/dist/{types-DB8jB6Jg.d.cts → types-B2-BU5CS.d.cts} +1 -1
  43. package/packages/core/dist/writers.d.cts +2 -2
  44. package/packages/core/dist/writers.d.ts +2 -2
package/CHANGELOG.md CHANGED
@@ -1,10 +1,22 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.8.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 0bee42c: Release v1.8.0 with OpenAI Agents trace processor support, optional Vitest/Jest reporter packages kept private, deterministic CI release checks, and the validated local-first reporting improvements from the v1.8 release train.
8
+
3
9
  ## 1.7.0
4
10
 
11
+ Released **2026-06-26**.
12
+
5
13
  ### Minor Changes
6
14
 
7
- - 94a7220: Release v1.7.0 framework-native adoption with the experimental AI SDK telemetry adapter, adapter conformance coverage, and local-first adapter documentation.
15
+ - 94a7220: Release v1.7.0 framework-native adoption with the experimental AI SDK telemetry adapter, declarative adapter conformance matrix, and local-first adapter documentation.
16
+
17
+ ### Notes
18
+
19
+ - The v1.8 train carries the remaining adapter correctness work: AI SDK logical lifecycle identity, parallel integration isolation, explicit capture/redaction behavior, executable conformance fixtures, OpenAI Agents runtime mapping, and LangGraph no-network fixtures. v1.7.0 should not be read as claiming those deferred behaviors.
8
20
 
9
21
  ## 1.6.0
10
22
 
package/README.md CHANGED
@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
22
22
 
23
23
  ## Install
24
24
 
25
- Current npm release: **1.6.0** (`agent-inspect`, `@agent-inspect/langchain`, `@agent-inspect/tui` — all aligned).
25
+ Current npm release: **1.7.0** (`agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui` — all aligned).
26
26
 
27
27
  ```bash
28
28
  npm install agent-inspect
@@ -111,7 +111,7 @@ await maybeInspectRun("eval-case-42", async () => runAgent());
111
111
  AGENT_INSPECT=1 node eval-runner.mjs
112
112
  ```
113
113
 
114
- ## What you can do today (v1.6.0)
114
+ ## What you can do today (v1.7.0)
115
115
 
116
116
  - **Trace manually** with `inspectRun`, `step`, `step.llm`, `step.tool`, and `observe` — local JSONL under `.agent-inspect/` by default.
117
117
  - **Toggle tracing** with `maybeInspectRun` and `AGENT_INSPECT=1` in eval harnesses or CI.
@@ -121,6 +121,7 @@ AGENT_INSPECT=1 node eval-runner.mjs
121
121
  - **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
122
122
  - **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
123
123
  - **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
124
+ - **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
124
125
  - **Optional TUI** — `view --tui` when `@agent-inspect/tui` is installed.
125
126
  - **Persisted-event foundation (v1.2.0+)** — in-memory `PersistedInspectEvent` converters; manual writing stays `schemaVersion: "0.1"`.
126
127
  - **Experimental v1.6.0 APIs** — `agent-inspect/writers`, `agent-inspect/readers`, `createInspector()`, and `agent-inspect open` for local AgentInspect/OpenInference/OTLP ingestion.
@@ -180,6 +181,8 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
180
181
  | `search` | Deterministic search over local traces |
181
182
  | `what` | Concise summary of one run |
182
183
  | `report` | Markdown/HTML inspection report (what + timeline + tree) |
184
+ | `check` / `scan` / `verify-safe` | Deterministic local trace checks and best-effort safety verification |
185
+ | `artifacts` | Safe local CI artifact bundles and optional step-summary file output |
183
186
 
184
187
  ![Timeline with slow-step focus for one run](https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/demos/timeline.gif)
185
188
 
@@ -198,7 +201,7 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
198
201
 
199
202
  ## Stable foundation (AgentInspect 1.x)
200
203
 
201
- **agent-inspect 1.x** (current: **1.6.0**) is the **local-first trace workbench** for TypeScript AI agents:
204
+ **agent-inspect 1.x** (current: **1.7.0**) is the **local-first trace workbench** for TypeScript AI agents:
202
205
 
203
206
  - Instrument runs with `inspectRun` and `step`
204
207
  - Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
@@ -208,13 +211,13 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
208
211
 
209
212
  Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
210
213
 
211
- **Shipped in 1.6.0:** experimental writer subpath (`agent-inspect/writers`), isolated `createInspector()` API via `agent-inspect/advanced`, local trace readers via `agent-inspect/readers`, OpenInference/OTLP JSON readers, universal `agent-inspect open`, and deterministic [runtime-and-ingestion recipe](examples/recipes/runtime-and-ingestion/). These remain local-only and do not add upload behavior. Linked release aligns all three npm packages at **1.6.0**.
214
+ **Shipped in 1.7.0:** experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6 with a local no-network [ai-sdk-local-telemetry recipe](examples/recipes/ai-sdk-local-telemetry/), adapter conformance fixtures, OpenAI Agents/LangGraph support decisions, and local-first adapter docs. Examples keep `recordInputs: false`, `recordOutputs: false`, metadata-only capture, and no upload behavior. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, and `@agent-inspect/tui` at **1.7.0**.
212
215
 
213
- **In progress for 1.7.0 on main:** experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6 with a local no-network [ai-sdk-local-telemetry recipe](examples/recipes/ai-sdk-local-telemetry/). Examples keep `recordInputs: false`, `recordOutputs: false`, metadata-only capture, and no upload behavior.
216
+ **Shipped in 1.6.0:** experimental writer subpath (`agent-inspect/writers`), isolated `createInspector()` API via `agent-inspect/advanced`, local trace readers via `agent-inspect/readers`, OpenInference/OTLP JSON readers, universal `agent-inspect open`, and deterministic [runtime-and-ingestion recipe](examples/recipes/runtime-and-ingestion/). These remain local-only and do not add upload behavior. Linked release aligns all three then-published npm packages at **1.6.0**.
214
217
 
215
218
  **Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
216
219
 
217
- **Roadmap beyond current release work:** future work continues from the local runtime and universal ingestion foundation before broadening framework adapters. See [ROADMAP.md](ROADMAP.md).
220
+ **Roadmap beyond current release work:** future work continues from the local runtime, universal ingestion, and optional adapter foundations. See [ROADMAP.md](ROADMAP.md).
218
221
 
219
222
  **Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
220
223
 
@@ -291,6 +294,8 @@ The TUI is available as a separate optional package; its programmatic API is exp
291
294
  | [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
292
295
  | [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
293
296
  | [examples/recipes/github-actions-artifact](examples/recipes/github-actions-artifact) | CI trace artifacts |
297
+ | [examples/recipes/deterministic-ci-checks](examples/recipes/deterministic-ci-checks) | v1.8 checks, baseline, and safe CI artifacts |
298
+ | [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
294
299
  | [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
295
300
  | [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
296
301
 
@@ -12,14 +12,18 @@ The machine-readable matrix lives at [docs/implementation/adapter-conformance-ma
12
12
  - streaming metadata
13
13
  - metadata bounds and privacy controls
14
14
 
15
+ The v1.7 matrix is declarative coverage guidance. v1.8 makes conformance executable and requires canonical-reader round trips before adapter output is used by checks.
16
+
17
+ Executable shared assertions live in `packages/core/test/adapter-executable-conformance.test.ts` and `packages/core/test/adapter-conformance-utils.ts`. Adapter-specific suites may add deeper fixture coverage, but the shared suite owns the cross-adapter defaults: local-only execution, no raw payload persistence, lifecycle identity, parentage, streaming summaries, token usage where exposed, and reader round trips.
18
+
15
19
  ## Current matrix
16
20
 
17
21
  | Adapter | Package | Status | Default install mode | Boundary |
18
22
  | --- | --- | --- | --- | --- |
19
- | AI SDK | `@agent-inspect/ai-sdk` | implemented experimental | AI SDK telemetry integration | optional package peer dependency |
23
+ | AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v1.8 correctness hardening pending | AI SDK telemetry integration | optional package peer dependency |
20
24
  | LangChain | `@agent-inspect/langchain` | implemented experimental | explicit callback | optional package peer dependency |
21
- | OpenAI Agents JS | `@agent-inspect/openai-agents` | scaffold only | `setTraceProcessors()` replacement | optional package peer dependency |
22
- | LangGraph | `@agent-inspect/langchain` | decision only | explicit LangChain callback | existing LangChain adapter first |
25
+ | OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; private until first publication gate | `setTraceProcessors()` replacement | optional package peer dependency |
26
+ | LangGraph | `@agent-inspect/langchain` | fixture-backed through LangChain callback | explicit LangChain callback | existing LangChain adapter first |
23
27
 
24
28
  ## Required defaults
25
29
 
package/docs/ADAPTERS.md CHANGED
@@ -4,7 +4,9 @@ AgentInspect is **framework-agnostic** at its core. Optional adapter packages in
4
4
 
5
5
  ## Vercel AI SDK (`@agent-inspect/ai-sdk`)
6
6
 
7
- **Status:** experimental v1.7 train — optional workspace package; publication waits for v1.7 release readiness.
7
+ **Status:** experimental v1.7 adapter — optional package published in the v1.7.0 linked release.
8
+
9
+ The v1.8 train has hardened lifecycle identity and parallel integration isolation. The adapter remains metadata-only: `capture: "preview"` and preview-only redaction options emit diagnostics and fall back to metadata-only capture until bounded free-text previews are implemented.
8
10
 
9
11
  ### Install
10
12
 
@@ -41,6 +43,7 @@ const result = await generateText({
41
43
  - **Metadata-only by default** — records model, finish reason, token usage, timing, and safe counts/summaries.
42
44
  - **Required safe telemetry settings** — set `recordInputs: false` and `recordOutputs: false` on every AI SDK call using this adapter.
43
45
  - **No raw payload capture by default** — prompts, messages, generated text, stream chunks, tool inputs/outputs, headers, request bodies, and response bodies are not persisted.
46
+ - **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
44
47
 
45
48
  ### Local no-network recipe
46
49
 
@@ -112,6 +115,16 @@ npx agent-inspect export <run-id> --format markdown --redaction-profile share
112
115
 
113
116
  Written events use `schemaVersion: "0.1"` manual trace names.
114
117
 
118
+ ### LangGraph through LangChain callbacks
119
+
120
+ LangGraph-shaped callback metadata is covered through the existing `@agent-inspect/langchain` callback boundary. No separate `@agent-inspect/langgraph` package is shipped.
121
+
122
+ - **Explicit callback only** — pass `new AgentInspectCallback(...)` through LangChain/LangGraph callback configuration.
123
+ - **Bounded graph metadata** — known graph, node, subgraph, task, branch, checkpoint, retry, handoff, thread, and session identifiers are copied into `attributes.langGraph` when present.
124
+ - **No full graph state** — checkpoint/task/branch containers are summarized by type/count; raw graph state, prompts, tool payloads, outputs, and stream tokens are not stored in `metadata-only` mode.
125
+ - **Conservative parent mapping** — in-memory events preserve framework `parentRunId`; persisted JSONL maps parents only when the parent callback was seen, and marks unresolved parent mappings in step metadata.
126
+ - **No hosted tracing requirement** — fixtures use structural no-network callback payloads and do not require LangSmith, provider calls, or LangGraph platform services.
127
+
115
128
  ### Capture modes
116
129
 
117
130
  | `capture` | Behavior |
@@ -164,7 +177,7 @@ Full API: [API.md](./API.md) §9.
164
177
 
165
178
  ### LangGraph boundary
166
179
 
167
- LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. A dedicated LangGraph package is deferred until no-network fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
180
+ LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. v1.8 adds executable no-network fixtures before claiming broader LangGraph support. A dedicated LangGraph package remains deferred until fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
168
181
 
169
182
  Future LangGraph examples must keep the same safety defaults: explicit callback installation, metadata-only capture, no raw prompt/output/tool payload capture by default, no hosted sink, and local persistence only when `persist: true` is set.
170
183
 
@@ -187,11 +200,103 @@ Requires an interactive terminal. See [API.md](./API.md) §10.
187
200
 
188
201
  ---
189
202
 
203
+ ## Vitest (`@agent-inspect/vitest`)
204
+
205
+ **Status:** experimental v1.8 reporter — optional workspace package, private/unpublished until release readiness.
206
+
207
+ ```bash
208
+ npm install agent-inspect @agent-inspect/vitest vitest
209
+ ```
210
+
211
+ After publication, the reporter creates safe, structural artifacts for failed tests that explicitly attach AgentInspect trace metadata. It never guesses trace files by timestamp and does not read trace contents into artifacts.
212
+
213
+ ```ts
214
+ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
215
+
216
+ export default {
217
+ test: {
218
+ reporters: [
219
+ "default",
220
+ createAgentInspectVitestReporter({
221
+ artifactDir: ".agent-inspect/vitest-artifacts",
222
+ retainSuccessful: 5,
223
+ }),
224
+ ],
225
+ },
226
+ };
227
+ ```
228
+
229
+ Attach explicit metadata from the test harness, or provide `resolveTrace(test)`:
230
+
231
+ ```ts
232
+ test("agent workflow", async (ctx) => {
233
+ ctx.task.meta.agentInspect = {
234
+ runId: "support-agent",
235
+ tracePath: ".agent-inspect/support-agent.jsonl",
236
+ artifactLabel: "support-agent",
237
+ };
238
+ });
239
+ ```
240
+
241
+ - **Explicit association only** — no timestamp matching or directory guessing.
242
+ - **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
243
+ - **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
244
+ - **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Vitest failures.
245
+ - **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Vitest dependency.
246
+ - **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
247
+
248
+ Full API: [API.md](./API.md) §12.
249
+
250
+ ---
251
+
252
+ ## Jest (`@agent-inspect/jest`)
253
+
254
+ **Status:** experimental v1.8 reporter — optional workspace package, private/unpublished until release readiness.
255
+
256
+ ```bash
257
+ npm install agent-inspect @agent-inspect/jest jest
258
+ ```
259
+
260
+ After publication, the reporter creates safe, structural artifacts for failed Jest assertions that explicitly attach AgentInspect trace metadata through a map or resolver. It never guesses trace files by timestamp and does not read trace contents into artifacts.
261
+
262
+ ```js
263
+ module.exports = {
264
+ reporters: [
265
+ "default",
266
+ [
267
+ "@agent-inspect/jest",
268
+ {
269
+ artifactDir: ".agent-inspect/jest-artifacts",
270
+ retainSuccessful: 5,
271
+ associations: {
272
+ "agent.test.cjs::agent suite agent workflow": {
273
+ runId: "support-agent",
274
+ tracePath: ".agent-inspect/support-agent.jsonl",
275
+ },
276
+ },
277
+ },
278
+ ],
279
+ ],
280
+ };
281
+ ```
282
+
283
+ - **Explicit association only** — use `associations` or `resolveTrace(test)`; no timestamp matching or directory guessing.
284
+ - **Jest lifecycle** — processes assertion results from `onTestResult` and aggregated file results from `onRunComplete`.
285
+ - **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
286
+ - **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
287
+ - **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Jest failures.
288
+ - **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Jest dependency.
289
+ - **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
290
+
291
+ Full API: [API.md](./API.md) §13.
292
+
293
+ ---
294
+
190
295
  ## OpenAI Agents JS (`@agent-inspect/openai-agents`)
191
296
 
192
- **Status:** scaffolded for v1.7 train — optional workspace package; runtime mapping is not implemented yet.
297
+ **Status:** experimental v1.8 adapter — optional workspace package remains private/unpublished until the manual first-publication gate.
193
298
 
194
- The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md). Future examples must install the AgentInspect processor by replacing processors:
299
+ The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md). Install the AgentInspect processor by replacing processors:
195
300
 
196
301
  ```ts
197
302
  import { setTraceProcessors } from "@openai/agents";
@@ -205,7 +310,17 @@ setTraceProcessors([
205
310
  ]);
206
311
  ```
207
312
 
208
- Do not use `addTraceProcessor()` as the default AgentInspect path; that preserves the OpenAI default exporter in server runtimes. The scaffold does not auto-install a processor, does not upload, and does not add OpenAI Agents dependencies to root/core.
313
+ Do not use `addTraceProcessor()` as the default AgentInspect path; that preserves the OpenAI default exporter in server runtimes. The processor does not auto-install itself, does not upload, and does not add OpenAI Agents dependencies to root/core.
314
+
315
+ - **No auto-install** — importing or constructing `agentInspectProcessor()` never calls `setTraceProcessors()` or `addTraceProcessor()`.
316
+ - **No upload behavior** — the processor writes only to an explicit local writer or `traceDir`.
317
+ - **Metadata-only by default** — records trace/span IDs, parentage, names, timing, status, errors, safe model/tool names, token counts, and bounded summaries.
318
+ - **No raw payload capture by default** — prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, and hosted tool payloads are not persisted.
319
+ - **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
320
+
321
+ Full API: [API.md](./API.md) §14.
322
+
323
+ Runnable local recipe: [openai-agents-local-tracing](../examples/recipes/openai-agents-local-tracing).
209
324
 
210
325
  ---
211
326
 
package/docs/API.md CHANGED
@@ -152,7 +152,7 @@ Rationale: v1.x includes one official adapter and **zero production sinks**, so
152
152
 
153
153
  ## 11. Experimental `@agent-inspect/ai-sdk` APIs
154
154
 
155
- `@agent-inspect/ai-sdk` is an optional v1.7 adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental during the v1.7 train and remains private until release readiness.
155
+ `@agent-inspect/ai-sdk` is an optional v1.7 adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental and published as part of the v1.7.0 linked release.
156
156
 
157
157
  Import from `@agent-inspect/ai-sdk`:
158
158
 
@@ -164,9 +164,10 @@ import { agentInspect } from "@agent-inspect/ai-sdk";
164
164
  - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
165
165
  - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
166
166
  - **`runName`**: optional local run name.
167
- - **`capture`**: `"metadata-only"` (default) or `"preview"` (reserved for explicit bounded capture).
168
- - **`redactionProfile`** and **`maxPreviewChars`**: reserved for bounded/redacted future preview behavior.
169
- - **`getDiagnostics()`**: exposes isolated adapter write failures without throwing into AI SDK callbacks.
167
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
168
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
169
+ - **`getDiagnostics()`**: exposes isolated adapter write, lifecycle/configuration, flush, and close failures without throwing into AI SDK callbacks.
170
+ - **`getWriterStats()`**, **`flush()`**, and **`close()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
170
171
 
171
172
  Every AI SDK call using the adapter must keep telemetry local and metadata-only:
172
173
 
@@ -179,15 +180,89 @@ experimental_telemetry: {
179
180
  }
180
181
  ```
181
182
 
182
- The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. By default it does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`.
183
+ The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. It does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`. Unsupported preview capture options are explicit diagnostics and keep this metadata-only behavior.
183
184
 
184
185
  No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-patch is part of this package.
185
186
 
186
187
  Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
187
188
 
188
- ## 12. Experimental `@agent-inspect/openai-agents` APIs
189
+ ## 12. Experimental `@agent-inspect/vitest` APIs
189
190
 
190
- `@agent-inspect/openai-agents` is an optional v1.7 scaffold for future OpenAI Agents JS tracing processor integration. Runtime span mapping is not implemented yet.
191
+ `@agent-inspect/vitest` is an optional experimental package for local Vitest failure artifacts. In the v1.8 train it remains private/unpublished until release readiness. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
192
+
193
+ Import from `@agent-inspect/vitest`:
194
+
195
+ ```ts
196
+ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
197
+ ```
198
+
199
+ - **`createAgentInspectVitestReporter(options?)`**: returns a structural Vitest reporter facade with `onTestCaseResult`, `onTaskUpdate`, and `onFinished` hooks.
200
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/vitest-artifacts`.
201
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
202
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
203
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
204
+ - **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
205
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
206
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
207
+ - **`agentInspectVitestReporter`**: alias for `createAgentInspectVitestReporter`.
208
+
209
+ Trace association must be explicit. The default resolver reads `meta.agentInspect`, `meta["agent-inspect"]`, `meta.trace`, `context.meta.agentInspect`, or result metadata when present:
210
+
211
+ ```ts
212
+ ctx.task.meta.agentInspect = {
213
+ runId: "support-agent",
214
+ tracePath: ".agent-inspect/support-agent.jsonl",
215
+ artifactLabel: "support-agent",
216
+ };
217
+ ```
218
+
219
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
220
+
221
+ ## 13. Experimental `@agent-inspect/jest` APIs
222
+
223
+ `@agent-inspect/jest` is an optional experimental package for local Jest failure artifacts. In the v1.8 train it remains private/unpublished until release readiness. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
224
+
225
+ Import from `@agent-inspect/jest`:
226
+
227
+ ```ts
228
+ import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent-inspect/jest";
229
+ ```
230
+
231
+ - **`AgentInspectJestReporter`**: default Jest custom reporter class for `reporters: [["@agent-inspect/jest", options]]`.
232
+ - **`createAgentInspectJestReporter(options?)`**: returns a structural reporter facade with `onTestResult` and `onRunComplete` hooks for tests and custom harnesses.
233
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/jest-artifacts`.
234
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
235
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
236
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
237
+ - **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
238
+ - **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
239
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
240
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
241
+ - **`agentInspectJestReporter`**: alias for `createAgentInspectJestReporter`.
242
+
243
+ Jest association is explicit because Jest assertion results do not expose Vitest-style mutable task metadata:
244
+
245
+ ```js
246
+ reporters: [
247
+ [
248
+ "@agent-inspect/jest",
249
+ {
250
+ associations: {
251
+ "agent.test.cjs::agent suite agent workflow": {
252
+ runId: "support-agent",
253
+ tracePath: ".agent-inspect/support-agent.jsonl",
254
+ },
255
+ },
256
+ },
257
+ ],
258
+ ],
259
+ ```
260
+
261
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
262
+
263
+ ## 14. Experimental `@agent-inspect/openai-agents` APIs
264
+
265
+ `@agent-inspect/openai-agents` is an optional experimental package for OpenAI Agents JS tracing processor integration. In the v1.8 train it remains private/unpublished until the manual first-publication gate, but runtime metadata mapping is implemented locally.
191
266
 
192
267
  Import from `@agent-inspect/openai-agents`:
193
268
 
@@ -195,11 +270,16 @@ Import from `@agent-inspect/openai-agents`:
195
270
  import { agentInspectProcessor } from "@agent-inspect/openai-agents";
196
271
  ```
197
272
 
198
- - **`agentInspectProcessor(options?)`**: returns a local-only scaffold object with diagnostics.
273
+ - **`agentInspectProcessor(options?)`**: returns a local-only OpenAI Agents `TracingProcessor`.
199
274
  - **`installMode`**: always `"setTraceProcessors"` to document the safe replacement install path.
200
- - **`localOnly`**: always `true`; the scaffold performs no network I/O and does not install itself globally.
201
- - **`writer`**, **`traceDir`**, **`workflowName`**, **`capture`**, **`redactionProfile`**, and **`maxPreviewChars`** are reserved for the later runtime mapping chunk.
202
- - **`getDiagnostics()`**: exposes scaffold diagnostics and reports `runtimeMappingImplemented: false`.
275
+ - **`localOnly`**: always `true`; the processor performs no network I/O and does not install itself globally.
276
+ - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
277
+ - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
278
+ - **`workflowName`**: optional local run name overriding the SDK trace name.
279
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
280
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
281
+ - **`getDiagnostics()`**: exposes isolated processor write, lifecycle/configuration, flush, and shutdown failures without throwing into OpenAI Agents callbacks.
282
+ - **`getWriterStats()`**, **`forceFlush()`**, and **`shutdown()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
203
283
 
204
284
  Safe future usage must replace processors explicitly:
205
285
 
@@ -209,7 +289,9 @@ setTraceProcessors([agentInspectProcessor({ traceDir: "./.agent-inspect" })]);
209
289
 
210
290
  Do not use `addTraceProcessor()` as the default AgentInspect path; that leaves existing/default processors in place and can preserve backend export behavior in server runtimes.
211
291
 
212
- ## 13. Experimental persisted-event foundation (v1.2.0)
292
+ The processor records local v0.2 persisted events for trace/run, agent, generation/response, function/tool, handoff, guardrail, MCP tools, custom, transcription, and speech span metadata where safely representable. It does not persist raw prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, or hosted tool payloads by default.
293
+
294
+ ## 15. Experimental persisted-event foundation (v1.2.0)
213
295
 
214
296
  These helpers expose the **source-agnostic `PersistedInspectEvent` model** (`schemaVersion: "0.2"`). They are **local-only**, **in-memory**, and **do not change** storage write/read or CLI behavior in v1.2.0.
215
297
 
@@ -235,7 +317,7 @@ Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEv
235
317
  - v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
236
318
  - Inspection read paths normalize v0.1 and v0.2 JSONL for local CLI/API use. v0.2 remains experimental as a persisted-event foundation and is not the default writer.
237
319
 
238
- ## 14. Local observability helpers (v1.4.0+)
320
+ ## 16. Local observability helpers (v1.4.0+)
239
321
 
240
322
  Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1 manual traces remain the default writer; v0.2 persisted-event files are accepted where the shared dual-format read path is used. Local files only.
241
323
 
@@ -245,7 +327,7 @@ Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1
245
327
 
246
328
  CLI wrappers: `agent-inspect timeline`, `stats`, `search` — see [CLI.md](./CLI.md).
247
329
 
248
- ## 15. Report and what helpers (v1.5.0+)
330
+ ## 17. Report and what helpers (v1.5.0+)
249
331
 
250
332
  Read-only helpers for concise inspection summaries and local reports:
251
333
 
@@ -254,7 +336,7 @@ Read-only helpers for concise inspection summaries and local reports:
254
336
 
255
337
  Report redaction profiles are key-based safeguards applied to the complete rendered report input, not only to the tree section. Review generated reports before sharing; this is not compliance-grade DLP.
256
338
 
257
- ## 16. Experimental trace writers (v1.6)
339
+ ## 18. Experimental trace writers (v1.6)
258
340
 
259
341
  Trace writers are the first slice of the v1.6 runtime foundation. They are experimental during v1.x and intended for tests, adapters, and future `createInspector` work.
260
342
 
@@ -286,7 +368,7 @@ import type {
286
368
 
287
369
  No network writer or vendor sink exists in this package.
288
370
 
289
- ## 17. Experimental inspector API/runtime (v1.6)
371
+ ## 19. Experimental inspector API/runtime (v1.6)
290
372
 
291
373
  `createInspector()` is the experimental public instance API for local-first tracing with explicit writers. It owns an instance-specific runtime context, records v0.2 persisted inspect events, preserves application return values/errors, and exposes diagnostics plus deterministic `flush()`/`close()` lifecycle hooks.
292
374
 
@@ -328,7 +410,7 @@ Public methods:
328
410
 
329
411
  These APIs are experimental during v1.x. They do not add a default network writer or vendor sink.
330
412
 
331
- ## 18. Experimental trace readers (v1.6)
413
+ ## 20. Experimental trace readers (v1.6)
332
414
 
333
415
  `agent-inspect/readers` exposes the experimental local trace reader contract and detection pipeline. It includes AgentInspect JSONL for v0.1, v0.2, and mixed local trace files, plus local OpenInference JSON and OTLP JSON compatibility readers.
334
416
 
@@ -359,23 +441,43 @@ import type { TraceReader } from "agent-inspect/readers";
359
441
 
360
442
  The reader contract does not silently accept arbitrary JSON and does not add OTel SDK, database, hosted ingestion, or network upload dependencies.
361
443
 
362
- ## 19. Deprecated APIs
444
+ ## 21. Experimental Checks
445
+
446
+ `agent-inspect/checks` exposes the experimental deterministic trace-check engine foundation. It consumes normalized reader output, runs supplied pure rules in stable order, and returns aggregate findings/diagnostics. It does not read files, discover config, call providers, perform network I/O, mutate inputs, or create a new persisted schema.
447
+
448
+ Import from `agent-inspect/checks`:
449
+
450
+ ```ts
451
+ import { runTraceChecks } from "agent-inspect/checks";
452
+ import type { TraceCheckRule, TraceCheckResult } from "agent-inspect/checks";
453
+ ```
454
+
455
+ - **`runTraceChecks({ read }, { rules?, select?, runId? })`**: executes provided rules against a `TraceReadResult` from `agent-inspect/readers`.
456
+ - **Built-in rule factories**: run, tool, LLM, structure, retrieval, guardrail, decision, safety, and baseline helpers including `createRunStatusRule`, `createToolUsageRule`, `createLlmUsageRule`, `createStructureOrphanRule`, `createStructureCycleRule`, `createStructureRelationshipRule`, `createRetrievalRule`, `createGuardrailRule`, `createDecisionRule`, `createSafetyRawContentRule`, `createSafetySecretPatternRule`, and `createBaselineRegressionRule`.
457
+ - **`TraceCheckRule`**: synchronous pure rule contract.
458
+ - **`TraceCheckResult`**: deterministic aggregate result with findings, evidence, summary counts, and execution diagnostics.
459
+
460
+ The checks API is experimental in v1.x. The `agent-inspect check` CLI uses this API for local reader-backed checks and deterministic JSON output; `agent-inspect artifacts` reuses the same safe findings for local CI artifact bundles and optional step-summary file output. Built-in rules operate on normalized event metadata, tree relationships, bounded summaries, token counts, and normalized baseline facts; safety and baseline findings identify event IDs and field paths rather than emitting raw prompts, outputs, secrets, headers, request/response bodies, or full tool payloads.
461
+
462
+ Recipes: [deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for check/baseline/artifact workflows, and [test-reporter-artifacts](../examples/recipes/test-reporter-artifacts/README.md) for Vitest/Jest reporter configuration patterns.
463
+
464
+ ## 22. Deprecated APIs
363
465
 
364
466
  No deprecated APIs are declared as of 1.4.0.
365
467
 
366
- ## 20. Removal / deprecation policy
468
+ ## 23. Removal / deprecation policy
367
469
 
368
470
  - Stable APIs are not removed in v1.x.
369
471
  - If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
370
472
 
371
- ## 21. Backward compatibility policy
473
+ ## 24. Backward compatibility policy
372
474
 
373
475
  - Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
374
476
  - Additive schema changes are allowed in minor versions.
375
477
  - Breaking changes require a major version.
376
478
  - Unknown fields should be ignored where safe.
377
479
 
378
- ## 22. Examples
480
+ ## 25. Examples
379
481
 
380
482
  ### Minimal manual trace
381
483