agent-inspect 1.7.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -1
- package/README.md +145 -21
- package/docs/ADAPTER-CONFORMANCE.md +7 -3
- package/docs/ADAPTERS.md +155 -5
- package/docs/API.md +214 -26
- package/docs/CLI.md +189 -7
- package/docs/GETTING-STARTED.md +71 -16
- package/docs/KNOWN-ISSUES.md +7 -1
- package/docs/LIMITATIONS.md +7 -1
- package/docs/LOG-TO-TREE-QUICKSTART.md +1 -2
- package/docs/MIGRATION.md +67 -0
- package/docs/SCHEMA.md +1 -0
- package/package.json +12 -2
- package/packages/cli/dist/index.cjs +2454 -140
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +2454 -140
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +4 -4
- package/packages/core/dist/advanced.d.ts +4 -4
- package/packages/core/dist/checks.cjs +1535 -0
- package/packages/core/dist/checks.cjs.map +1 -0
- package/packages/core/dist/checks.d.cts +585 -0
- package/packages/core/dist/checks.d.ts +585 -0
- package/packages/core/dist/checks.mjs +1512 -0
- package/packages/core/dist/checks.mjs.map +1 -0
- package/packages/core/dist/diff.d.cts +3 -3
- package/packages/core/dist/diff.d.ts +3 -3
- package/packages/core/dist/exporters.d.cts +3 -3
- package/packages/core/dist/exporters.d.ts +3 -3
- package/packages/core/dist/index.cjs +146 -0
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +44 -7
- package/packages/core/dist/index.d.ts +44 -7
- package/packages/core/dist/index.mjs +148 -1
- package/packages/core/dist/index.mjs.map +1 -1
- package/packages/core/dist/{inspect-event-Des4JDHo.d.cts → inspect-event-CevRYp58.d.cts} +1 -1
- package/packages/core/dist/{inspect-event-Des4JDHo.d.ts → inspect-event-CevRYp58.d.ts} +1 -1
- package/packages/core/dist/{log-config-C1GcJPIM.d.ts → log-config-BPHS4Sds.d.ts} +1 -1
- package/packages/core/dist/{log-config-BnH8Ykcb.d.cts → log-config-DanPV3P9.d.cts} +1 -1
- package/packages/core/dist/logs.d.cts +3 -3
- package/packages/core/dist/logs.d.ts +3 -3
- package/packages/core/dist/{persisted-inspect-event-DiFto0K2.d.ts → persisted-inspect-event-Cw7TeYGr.d.ts} +1 -1
- package/packages/core/dist/{persisted-inspect-event-0kaRADsp.d.cts → persisted-inspect-event-DHPfzUd8.d.cts} +1 -1
- package/packages/core/dist/persisted.d.cts +5 -5
- package/packages/core/dist/persisted.d.ts +5 -5
- package/packages/core/dist/readers.d.cts +2 -2
- package/packages/core/dist/readers.d.ts +2 -2
- package/packages/core/dist/{types-tSix7tfv.d.ts → types-Ap9uMdx_.d.ts} +1 -1
- package/packages/core/dist/{types-DB8jB6Jg.d.cts → types-B2-BU5CS.d.cts} +1 -1
- package/packages/core/dist/writers.d.cts +2 -2
- package/packages/core/dist/writers.d.ts +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,30 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.9.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 309350e: Release v1.9.0 adoption leverage with the private harness workspace, explain dry-run/local analysis, promoted adapter adoption paths, and the v2 root API slimming plan.
|
|
8
|
+
|
|
9
|
+
## 1.8.0
|
|
10
|
+
|
|
11
|
+
Released **2026-06-27**.
|
|
12
|
+
|
|
13
|
+
### Minor Changes
|
|
14
|
+
|
|
15
|
+
- 0bee42c: Release v1.8.0 with OpenAI Agents trace processor support, optional Vitest/Jest reporter packages kept private, deterministic CI release checks, and the validated local-first reporting improvements from the v1.8 release train.
|
|
16
|
+
|
|
3
17
|
## 1.7.0
|
|
4
18
|
|
|
19
|
+
Released **2026-06-26**.
|
|
20
|
+
|
|
5
21
|
### Minor Changes
|
|
6
22
|
|
|
7
|
-
- 94a7220: Release v1.7.0 framework-native adoption with the experimental AI SDK telemetry adapter, adapter conformance
|
|
23
|
+
- 94a7220: Release v1.7.0 framework-native adoption with the experimental AI SDK telemetry adapter, declarative adapter conformance matrix, and local-first adapter documentation.
|
|
24
|
+
|
|
25
|
+
### Notes
|
|
26
|
+
|
|
27
|
+
- The v1.8 train carries the remaining adapter correctness work: AI SDK logical lifecycle identity, parallel integration isolation, explicit capture/redaction behavior, executable conformance fixtures, OpenAI Agents runtime mapping, and LangGraph no-network fixtures. v1.7.0 should not be read as claiming those deferred behaviors.
|
|
8
28
|
|
|
9
29
|
## 1.6.0
|
|
10
30
|
|
package/README.md
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# agent-inspect
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**Trace, check, and safely share TypeScript AI agent runs locally.**
|
|
4
4
|
|
|
5
|
-
agent-inspect helps you understand what happened inside an AI agent run
|
|
5
|
+
agent-inspect helps you understand what happened inside an AI agent run without sending traces to a hosted service. It turns framework events, observed objects/classes, manual steps, tool calls, LLM calls, structured logs, failures, durations, and run metadata into readable local execution trees.
|
|
6
6
|
|
|
7
7
|
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it for **local TypeScript agent debugging**, **eval iteration**, and **CI trace artifacts**. It **complements** production observability platforms; it does **not** replace them.
|
|
8
8
|
|
|
9
|
-
The
|
|
9
|
+
The default loop is local-first: capture a trace, inspect/report/diff it, run deterministic checks in CI, then export a redacted copy only when you choose to share.
|
|
10
10
|
|
|
11
11
|
**No account. No cloud upload. No dashboard required.**
|
|
12
12
|
|
|
@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
|
|
|
22
22
|
|
|
23
23
|
## Install
|
|
24
24
|
|
|
25
|
-
Current npm release: **1.
|
|
25
|
+
Current npm release: **1.8.0** (`agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, `@agent-inspect/openai-agents` — all aligned).
|
|
26
26
|
|
|
27
27
|
```bash
|
|
28
28
|
npm install agent-inspect
|
|
@@ -40,10 +40,110 @@ npx agent-inspect --help
|
|
|
40
40
|
|
|
41
41
|
For a clean npm/pnpm install checklist with ESM, CJS, and CLI checks, see [Clean install smoke test](docs/INSTALL-SMOKE-TEST.md).
|
|
42
42
|
|
|
43
|
-
##
|
|
43
|
+
## Three adoption paths
|
|
44
|
+
|
|
45
|
+
Already using AI SDK, OpenAI Agents JS, LangChain, or LangGraph-through-LangChain? Start with **Path B** for framework-native local traces before adding manual instrumentation.
|
|
46
|
+
|
|
47
|
+
### Path A — Observe an existing object/class
|
|
48
|
+
|
|
49
|
+
Use `observe()` when you already have an agent-like object with a `run`, `execute`, or `invoke` method.
|
|
44
50
|
|
|
45
51
|
Create `demo.mjs`:
|
|
46
52
|
|
|
53
|
+
```js
|
|
54
|
+
import { observe } from "agent-inspect";
|
|
55
|
+
|
|
56
|
+
class SupportAgent {
|
|
57
|
+
async run(input) {
|
|
58
|
+
return {
|
|
59
|
+
answer: `Answering: ${input.question}`,
|
|
60
|
+
};
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
const agent = observe(new SupportAgent(), {
|
|
65
|
+
traceDir: "./.agent-inspect",
|
|
66
|
+
});
|
|
67
|
+
|
|
68
|
+
await agent.run({
|
|
69
|
+
question: "How do refunds work?",
|
|
70
|
+
});
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Run it, then inspect the trace:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
node demo.mjs
|
|
77
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
78
|
+
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Path B — Use a framework adapter
|
|
82
|
+
|
|
83
|
+
Optional adapters keep framework dependencies out of the root package and write local traces only when configured.
|
|
84
|
+
|
|
85
|
+
AI SDK local telemetry:
|
|
86
|
+
|
|
87
|
+
```ts
|
|
88
|
+
import { generateText } from "ai";
|
|
89
|
+
import { agentInspect } from "@agent-inspect/ai-sdk";
|
|
90
|
+
|
|
91
|
+
await generateText({
|
|
92
|
+
model,
|
|
93
|
+
prompt,
|
|
94
|
+
experimental_telemetry: {
|
|
95
|
+
isEnabled: true,
|
|
96
|
+
recordInputs: false,
|
|
97
|
+
recordOutputs: false,
|
|
98
|
+
integrations: [
|
|
99
|
+
agentInspect({
|
|
100
|
+
traceDir: "./.agent-inspect",
|
|
101
|
+
runName: "support-agent",
|
|
102
|
+
capture: "metadata-only",
|
|
103
|
+
}),
|
|
104
|
+
],
|
|
105
|
+
},
|
|
106
|
+
});
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
OpenAI Agents local-only processor:
|
|
110
|
+
|
|
111
|
+
```ts
|
|
112
|
+
import { setTraceProcessors } from "@openai/agents";
|
|
113
|
+
import { agentInspectProcessor } from "@agent-inspect/openai-agents";
|
|
114
|
+
|
|
115
|
+
setTraceProcessors([
|
|
116
|
+
agentInspectProcessor({
|
|
117
|
+
traceDir: "./.agent-inspect",
|
|
118
|
+
workflowName: "support-agent",
|
|
119
|
+
capture: "metadata-only",
|
|
120
|
+
}),
|
|
121
|
+
]);
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
LangChain callback adapter:
|
|
125
|
+
|
|
126
|
+
```ts
|
|
127
|
+
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
128
|
+
|
|
129
|
+
const callback = new AgentInspectCallback({
|
|
130
|
+
runName: "support-agent",
|
|
131
|
+
traceDir: "./.agent-inspect",
|
|
132
|
+
persist: true,
|
|
133
|
+
capture: "metadata-only",
|
|
134
|
+
});
|
|
135
|
+
|
|
136
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
See [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
140
|
+
|
|
141
|
+
No-network recipes: [ai-sdk-local-telemetry](examples/recipes/ai-sdk-local-telemetry/), [openai-agents-local-tracing](examples/recipes/openai-agents-local-tracing/), and [langgraph-callback-local](examples/recipes/langgraph-callback-local/).
|
|
142
|
+
|
|
143
|
+
### Path C — Manually instrument custom flows
|
|
144
|
+
|
|
145
|
+
Use `inspectRun` and `step` when you want explicit names, custom nesting, or flows that are not object/class shaped.
|
|
146
|
+
|
|
47
147
|
```js
|
|
48
148
|
import { inspectRun, step } from "agent-inspect";
|
|
49
149
|
|
|
@@ -71,15 +171,6 @@ await inspectRun(
|
|
|
71
171
|
);
|
|
72
172
|
```
|
|
73
173
|
|
|
74
|
-
Run it, then inspect the trace:
|
|
75
|
-
|
|
76
|
-
```bash
|
|
77
|
-
node demo.mjs
|
|
78
|
-
npx agent-inspect list --dir ./.agent-inspect
|
|
79
|
-
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
80
|
-
npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
|
|
81
|
-
```
|
|
82
|
-
|
|
83
174
|
Full flow:
|
|
84
175
|
|
|
85
176
|
```bash
|
|
@@ -99,6 +190,31 @@ support-agent
|
|
|
99
190
|
|
|
100
191
|
A runnable copy lives in [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
101
192
|
|
|
193
|
+
Use the root import for stable beginner APIs:
|
|
194
|
+
|
|
195
|
+
```ts
|
|
196
|
+
import {
|
|
197
|
+
observe,
|
|
198
|
+
inspectRun,
|
|
199
|
+
maybeInspectRun,
|
|
200
|
+
step,
|
|
201
|
+
getCurrentCorrelationMetadata,
|
|
202
|
+
} from "agent-inspect";
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
Use subpaths for advanced, experimental, or lower-level workflows:
|
|
206
|
+
|
|
207
|
+
```ts
|
|
208
|
+
import { openTrace } from "agent-inspect/readers";
|
|
209
|
+
import { memoryWriter } from "agent-inspect/writers";
|
|
210
|
+
import { runTraceChecks } from "agent-inspect/checks";
|
|
211
|
+
import { diffTraceEvents } from "agent-inspect/diff";
|
|
212
|
+
import { exportMarkdown } from "agent-inspect/exporters";
|
|
213
|
+
import { parseLogsToTrees } from "agent-inspect/logs";
|
|
214
|
+
import { traceEventsToPersistedInspectEvents } from "agent-inspect/persisted";
|
|
215
|
+
import { createInspector } from "agent-inspect/advanced";
|
|
216
|
+
```
|
|
217
|
+
|
|
102
218
|
**Env-gated tracing** (eval harnesses, CI): use `maybeInspectRun` and set `AGENT_INSPECT=1` when you want a trace — otherwise no files are written.
|
|
103
219
|
|
|
104
220
|
```ts
|
|
@@ -111,7 +227,7 @@ await maybeInspectRun("eval-case-42", async () => runAgent());
|
|
|
111
227
|
AGENT_INSPECT=1 node eval-runner.mjs
|
|
112
228
|
```
|
|
113
229
|
|
|
114
|
-
## What you can do today (v1.
|
|
230
|
+
## What you can do today (v1.8.0)
|
|
115
231
|
|
|
116
232
|
- **Trace manually** with `inspectRun`, `step`, `step.llm`, `step.tool`, and `observe` — local JSONL under `.agent-inspect/` by default.
|
|
117
233
|
- **Toggle tracing** with `maybeInspectRun` and `AGENT_INSPECT=1` in eval harnesses or CI.
|
|
@@ -121,9 +237,11 @@ AGENT_INSPECT=1 node eval-runner.mjs
|
|
|
121
237
|
- **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
|
|
122
238
|
- **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
|
|
123
239
|
- **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
|
|
240
|
+
- **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
|
|
241
|
+
- **Optional OpenAI Agents adapter** — experimental `@agent-inspect/openai-agents` trace processor for local OpenAI Agents JS trace processing.
|
|
124
242
|
- **Optional TUI** — `view --tui` when `@agent-inspect/tui` is installed.
|
|
125
243
|
- **Persisted-event foundation (v1.2.0+)** — in-memory `PersistedInspectEvent` converters; manual writing stays `schemaVersion: "0.1"`.
|
|
126
|
-
- **Experimental
|
|
244
|
+
- **Experimental subpaths** — `agent-inspect/readers`, `/writers`, `/checks`, `/diff`, `/exporters`, `/logs`, `/persisted`, and `/advanced` for advanced local workflows.
|
|
127
245
|
|
|
128
246
|
Nothing uploads traces by default. Review exports before sharing — see [safe trace sharing](docs/SAFE-TRACE-SHARING.md).
|
|
129
247
|
|
|
@@ -135,7 +253,7 @@ Each run produces a **JSONL** trace: `run_started` / `run_completed`, `step_star
|
|
|
135
253
|
|
|
136
254
|
*Synthetic demo — [examples/02-nested-steps](examples/02-nested-steps/README.md). More visuals: [SCREENSHOTS.md](docs/SCREENSHOTS.md).*
|
|
137
255
|
|
|
138
|
-
##
|
|
256
|
+
## Advanced ingestion: use this when your app already emits structured logs
|
|
139
257
|
|
|
140
258
|
Many production systems already emit **line-delimited JSON** or text logs with embedded JSON (e.g. via **pino**, **winston**, **log4js**, **NestJS** loggers, job runners, or custom event streams). agent-inspect can turn those into **local grouped timelines/trees** without wrapping every function.
|
|
141
259
|
|
|
@@ -180,6 +298,8 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
|
|
|
180
298
|
| `search` | Deterministic search over local traces |
|
|
181
299
|
| `what` | Concise summary of one run |
|
|
182
300
|
| `report` | Markdown/HTML inspection report (what + timeline + tree) |
|
|
301
|
+
| `check` / `scan` / `verify-safe` | Deterministic local trace checks and best-effort safety verification |
|
|
302
|
+
| `artifacts` | Safe local CI artifact bundles and optional step-summary file output |
|
|
183
303
|
|
|
184
304
|

|
|
185
305
|
|
|
@@ -198,7 +318,7 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
|
198
318
|
|
|
199
319
|
## Stable foundation (AgentInspect 1.x)
|
|
200
320
|
|
|
201
|
-
**agent-inspect 1.x** (current: **1.
|
|
321
|
+
**agent-inspect 1.x** (current: **1.8.0**) is the **local-first trace workbench** for TypeScript AI agents:
|
|
202
322
|
|
|
203
323
|
- Instrument runs with `inspectRun` and `step`
|
|
204
324
|
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
@@ -208,13 +328,15 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
|
208
328
|
|
|
209
329
|
Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
|
|
210
330
|
|
|
211
|
-
**Shipped in 1.
|
|
331
|
+
**Shipped in 1.8.0:** experimental deterministic checks (`agent-inspect/checks` and `agent-inspect check`), safe-sharing workflows (`scan`, `verify-safe`, safe artifacts), and first public `@agent-inspect/openai-agents` package. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, and `@agent-inspect/openai-agents` at **1.8.0**.
|
|
332
|
+
|
|
333
|
+
**Shipped in 1.7.0:** experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6 with a local no-network [ai-sdk-local-telemetry recipe](examples/recipes/ai-sdk-local-telemetry/), adapter conformance fixtures, OpenAI Agents/LangGraph support decisions, and local-first adapter docs. Examples keep `recordInputs: false`, `recordOutputs: false`, metadata-only capture, and no upload behavior. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, and `@agent-inspect/tui` at **1.7.0**.
|
|
212
334
|
|
|
213
|
-
**
|
|
335
|
+
**Shipped in 1.6.0:** experimental writer subpath (`agent-inspect/writers`), isolated `createInspector()` API via `agent-inspect/advanced`, local trace readers via `agent-inspect/readers`, OpenInference/OTLP JSON readers, universal `agent-inspect open`, and deterministic [runtime-and-ingestion recipe](examples/recipes/runtime-and-ingestion/). These remain local-only and do not add upload behavior. Linked release aligns all three then-published npm packages at **1.6.0**.
|
|
214
336
|
|
|
215
337
|
**Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
|
|
216
338
|
|
|
217
|
-
**Roadmap beyond current release work:** future work continues from the local runtime
|
|
339
|
+
**Roadmap beyond current release work:** future work continues from the local runtime, universal ingestion, and optional adapter foundations. See [ROADMAP.md](ROADMAP.md).
|
|
218
340
|
|
|
219
341
|
**Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
|
|
220
342
|
|
|
@@ -291,6 +413,8 @@ The TUI is available as a separate optional package; its programmatic API is exp
|
|
|
291
413
|
| [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
|
|
292
414
|
| [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
|
|
293
415
|
| [examples/recipes/github-actions-artifact](examples/recipes/github-actions-artifact) | CI trace artifacts |
|
|
416
|
+
| [examples/recipes/deterministic-ci-checks](examples/recipes/deterministic-ci-checks) | v1.8 checks, baseline, and safe CI artifacts |
|
|
417
|
+
| [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
|
|
294
418
|
| [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
|
|
295
419
|
| [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
|
|
296
420
|
|
|
@@ -12,14 +12,18 @@ The machine-readable matrix lives at [docs/implementation/adapter-conformance-ma
|
|
|
12
12
|
- streaming metadata
|
|
13
13
|
- metadata bounds and privacy controls
|
|
14
14
|
|
|
15
|
+
The v1.7 matrix is declarative coverage guidance. v1.8 makes conformance executable and requires canonical-reader round trips before adapter output is used by checks.
|
|
16
|
+
|
|
17
|
+
Executable shared assertions live in `packages/core/test/adapter-executable-conformance.test.ts` and `packages/core/test/adapter-conformance-utils.ts`. Adapter-specific suites may add deeper fixture coverage, but the shared suite owns the cross-adapter defaults: local-only execution, no raw payload persistence, lifecycle identity, parentage, streaming summaries, token usage where exposed, and reader round trips.
|
|
18
|
+
|
|
15
19
|
## Current matrix
|
|
16
20
|
|
|
17
21
|
| Adapter | Package | Status | Default install mode | Boundary |
|
|
18
22
|
| --- | --- | --- | --- | --- |
|
|
19
|
-
| AI SDK | `@agent-inspect/ai-sdk` | implemented experimental | AI SDK telemetry integration | optional package peer dependency |
|
|
23
|
+
| AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v1.8 correctness hardening pending | AI SDK telemetry integration | optional package peer dependency |
|
|
20
24
|
| LangChain | `@agent-inspect/langchain` | implemented experimental | explicit callback | optional package peer dependency |
|
|
21
|
-
| OpenAI Agents JS | `@agent-inspect/openai-agents` |
|
|
22
|
-
| LangGraph | `@agent-inspect/langchain` |
|
|
25
|
+
| OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; private until first publication gate | `setTraceProcessors()` replacement | optional package peer dependency |
|
|
26
|
+
| LangGraph | `@agent-inspect/langchain` | fixture-backed through LangChain callback | explicit LangChain callback | existing LangChain adapter first |
|
|
23
27
|
|
|
24
28
|
## Required defaults
|
|
25
29
|
|
package/docs/ADAPTERS.md
CHANGED
|
@@ -4,7 +4,9 @@ AgentInspect is **framework-agnostic** at its core. Optional adapter packages in
|
|
|
4
4
|
|
|
5
5
|
## Vercel AI SDK (`@agent-inspect/ai-sdk`)
|
|
6
6
|
|
|
7
|
-
**Status:** experimental
|
|
7
|
+
**Status:** experimental adapter — optional package published in the aligned v1.8.0 package set.
|
|
8
|
+
|
|
9
|
+
The v1.8 train has hardened lifecycle identity and parallel integration isolation. The adapter remains metadata-only: `capture: "preview"` and preview-only redaction options emit diagnostics and fall back to metadata-only capture until bounded free-text previews are implemented.
|
|
8
10
|
|
|
9
11
|
### Install
|
|
10
12
|
|
|
@@ -41,11 +43,40 @@ const result = await generateText({
|
|
|
41
43
|
- **Metadata-only by default** — records model, finish reason, token usage, timing, and safe counts/summaries.
|
|
42
44
|
- **Required safe telemetry settings** — set `recordInputs: false` and `recordOutputs: false` on every AI SDK call using this adapter.
|
|
43
45
|
- **No raw payload capture by default** — prompts, messages, generated text, stream chunks, tool inputs/outputs, headers, request bodies, and response bodies are not persisted.
|
|
46
|
+
- **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
|
|
44
47
|
|
|
45
48
|
### Local no-network recipe
|
|
46
49
|
|
|
47
50
|
[examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/) uses AI SDK test utilities only (`MockLanguageModelV3`, `simulateReadableStream`) and writes local v0.2 adapter events for `agent-inspect open`.
|
|
48
51
|
|
|
52
|
+
### Common host shapes
|
|
53
|
+
|
|
54
|
+
Use the same explicit telemetry block for route handlers, streaming, and tool calls. The adapter does not wrap providers or change host-call settings.
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
const telemetry = {
|
|
58
|
+
isEnabled: true,
|
|
59
|
+
recordInputs: false,
|
|
60
|
+
recordOutputs: false,
|
|
61
|
+
integrations: [agentInspect({ traceDir: "./.agent-inspect", capture: "metadata-only" })],
|
|
62
|
+
};
|
|
63
|
+
|
|
64
|
+
// Next.js route or local handler
|
|
65
|
+
await generateText({ model, prompt, experimental_telemetry: telemetry });
|
|
66
|
+
|
|
67
|
+
// Streaming
|
|
68
|
+
await streamText({ model, prompt, experimental_telemetry: telemetry });
|
|
69
|
+
|
|
70
|
+
// Tool calls
|
|
71
|
+
await generateText({ model, prompt, tools, experimental_telemetry: telemetry });
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Review recipe output with:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
npx agent-inspect open ./examples/recipes/ai-sdk-local-telemetry/.agent-inspect-runs
|
|
78
|
+
```
|
|
79
|
+
|
|
49
80
|
Full API: [API.md](./API.md) §11.
|
|
50
81
|
|
|
51
82
|
---
|
|
@@ -112,6 +143,16 @@ npx agent-inspect export <run-id> --format markdown --redaction-profile share
|
|
|
112
143
|
|
|
113
144
|
Written events use `schemaVersion: "0.1"` manual trace names.
|
|
114
145
|
|
|
146
|
+
### LangGraph through LangChain callbacks
|
|
147
|
+
|
|
148
|
+
LangGraph-shaped callback metadata is covered through the existing `@agent-inspect/langchain` callback boundary. No separate `@agent-inspect/langgraph` package is shipped.
|
|
149
|
+
|
|
150
|
+
- **Explicit callback only** — pass `new AgentInspectCallback(...)` through LangChain/LangGraph callback configuration.
|
|
151
|
+
- **Bounded graph metadata** — known graph, node, subgraph, task, branch, checkpoint, retry, handoff, thread, and session identifiers are copied into `attributes.langGraph` when present.
|
|
152
|
+
- **No full graph state** — checkpoint/task/branch containers are summarized by type/count; raw graph state, prompts, tool payloads, outputs, and stream tokens are not stored in `metadata-only` mode.
|
|
153
|
+
- **Conservative parent mapping** — in-memory events preserve framework `parentRunId`; persisted JSONL maps parents only when the parent callback was seen, and marks unresolved parent mappings in step metadata.
|
|
154
|
+
- **No hosted tracing requirement** — fixtures use structural no-network callback payloads and do not require LangSmith, provider calls, or LangGraph platform services.
|
|
155
|
+
|
|
115
156
|
### Capture modes
|
|
116
157
|
|
|
117
158
|
| `capture` | Behavior |
|
|
@@ -164,10 +205,12 @@ Full API: [API.md](./API.md) §9.
|
|
|
164
205
|
|
|
165
206
|
### LangGraph boundary
|
|
166
207
|
|
|
167
|
-
LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. A dedicated LangGraph package
|
|
208
|
+
LangGraph support is expected to ride through this same `@agent-inspect/langchain` callback boundary first. v1.8 adds executable no-network fixtures before claiming broader LangGraph support. A dedicated LangGraph package remains deferred until fixtures prove that LangGraph exposes important lifecycle data unavailable through LangChain callbacks.
|
|
168
209
|
|
|
169
210
|
Future LangGraph examples must keep the same safety defaults: explicit callback installation, metadata-only capture, no raw prompt/output/tool payload capture by default, no hosted sink, and local persistence only when `persist: true` is set.
|
|
170
211
|
|
|
212
|
+
Runnable local recipe: [langgraph-callback-local](../examples/recipes/langgraph-callback-local).
|
|
213
|
+
|
|
171
214
|
Decision note: [LANGGRAPH-ADAPTER-BOUNDARY.md](./proposals/LANGGRAPH-ADAPTER-BOUNDARY.md).
|
|
172
215
|
|
|
173
216
|
---
|
|
@@ -187,11 +230,103 @@ Requires an interactive terminal. See [API.md](./API.md) §10.
|
|
|
187
230
|
|
|
188
231
|
---
|
|
189
232
|
|
|
233
|
+
## Vitest (`@agent-inspect/vitest`)
|
|
234
|
+
|
|
235
|
+
**Status:** experimental workspace package, private/unpublished.
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
npm install agent-inspect @agent-inspect/vitest vitest
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
The reporter creates safe, structural artifacts for failed tests that explicitly attach AgentInspect trace metadata. It never guesses trace files by timestamp and does not read trace contents into artifacts.
|
|
242
|
+
|
|
243
|
+
```ts
|
|
244
|
+
import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
|
|
245
|
+
|
|
246
|
+
export default {
|
|
247
|
+
test: {
|
|
248
|
+
reporters: [
|
|
249
|
+
"default",
|
|
250
|
+
createAgentInspectVitestReporter({
|
|
251
|
+
artifactDir: ".agent-inspect/vitest-artifacts",
|
|
252
|
+
retainSuccessful: 5,
|
|
253
|
+
}),
|
|
254
|
+
],
|
|
255
|
+
},
|
|
256
|
+
};
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Attach explicit metadata from the test harness, or provide `resolveTrace(test)`:
|
|
260
|
+
|
|
261
|
+
```ts
|
|
262
|
+
test("agent workflow", async (ctx) => {
|
|
263
|
+
ctx.task.meta.agentInspect = {
|
|
264
|
+
runId: "support-agent",
|
|
265
|
+
tracePath: ".agent-inspect/support-agent.jsonl",
|
|
266
|
+
artifactLabel: "support-agent",
|
|
267
|
+
};
|
|
268
|
+
});
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
- **Explicit association only** — no timestamp matching or directory guessing.
|
|
272
|
+
- **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
|
|
273
|
+
- **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
|
|
274
|
+
- **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Vitest failures.
|
|
275
|
+
- **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Vitest dependency.
|
|
276
|
+
- **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
|
|
277
|
+
|
|
278
|
+
Full API: [API.md](./API.md) §12.
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## Jest (`@agent-inspect/jest`)
|
|
283
|
+
|
|
284
|
+
**Status:** experimental workspace package, private/unpublished.
|
|
285
|
+
|
|
286
|
+
```bash
|
|
287
|
+
npm install agent-inspect @agent-inspect/jest jest
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
The reporter creates safe, structural artifacts for failed Jest assertions that explicitly attach AgentInspect trace metadata through a map or resolver. It never guesses trace files by timestamp and does not read trace contents into artifacts.
|
|
291
|
+
|
|
292
|
+
```js
|
|
293
|
+
module.exports = {
|
|
294
|
+
reporters: [
|
|
295
|
+
"default",
|
|
296
|
+
[
|
|
297
|
+
"@agent-inspect/jest",
|
|
298
|
+
{
|
|
299
|
+
artifactDir: ".agent-inspect/jest-artifacts",
|
|
300
|
+
retainSuccessful: 5,
|
|
301
|
+
associations: {
|
|
302
|
+
"agent.test.cjs::agent suite agent workflow": {
|
|
303
|
+
runId: "support-agent",
|
|
304
|
+
tracePath: ".agent-inspect/support-agent.jsonl",
|
|
305
|
+
},
|
|
306
|
+
},
|
|
307
|
+
},
|
|
308
|
+
],
|
|
309
|
+
],
|
|
310
|
+
};
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
- **Explicit association only** — use `associations` or `resolveTrace(test)`; no timestamp matching or directory guessing.
|
|
314
|
+
- **Jest lifecycle** — processes assertion results from `onTestResult` and aggregated file results from `onRunComplete`.
|
|
315
|
+
- **Failure artifacts by default** — passing-test artifacts are kept only when `retainSuccessful` is configured.
|
|
316
|
+
- **Bounded success retention** — successful artifacts are capped by `maxSuccessfulTraces`.
|
|
317
|
+
- **Failure-preserving** — reporter/artifact errors are surfaced through diagnostics and do not replace original Jest failures.
|
|
318
|
+
- **Local-only** — no network I/O, no hosted upload, no GitHub API, and no root/core Jest dependency.
|
|
319
|
+
- **Safe rendering** — artifacts include structural test/run/file references only, not raw trace contents, prompts, outputs, request/response bodies, headers, API keys, secrets, or tool payloads.
|
|
320
|
+
|
|
321
|
+
Full API: [API.md](./API.md) §13.
|
|
322
|
+
|
|
323
|
+
---
|
|
324
|
+
|
|
190
325
|
## OpenAI Agents JS (`@agent-inspect/openai-agents`)
|
|
191
326
|
|
|
192
|
-
**Status:**
|
|
327
|
+
**Status:** experimental adapter — optional package published in the aligned v1.8.0 package set.
|
|
193
328
|
|
|
194
|
-
The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md).
|
|
329
|
+
The safe integration boundary is documented in [OPENAI-AGENTS-JS-TRACING.md](./proposals/OPENAI-AGENTS-JS-TRACING.md). Install the AgentInspect processor by replacing processors:
|
|
195
330
|
|
|
196
331
|
```ts
|
|
197
332
|
import { setTraceProcessors } from "@openai/agents";
|
|
@@ -205,7 +340,22 @@ setTraceProcessors([
|
|
|
205
340
|
]);
|
|
206
341
|
```
|
|
207
342
|
|
|
208
|
-
Do not use `addTraceProcessor()` as the default AgentInspect path; that preserves the OpenAI default exporter in server runtimes. The
|
|
343
|
+
Do not use `addTraceProcessor()` as the default AgentInspect path; that preserves the OpenAI default exporter in server runtimes. The processor does not auto-install itself, does not upload, and does not add OpenAI Agents dependencies to root/core.
|
|
344
|
+
|
|
345
|
+
Integration modes:
|
|
346
|
+
|
|
347
|
+
- **Local-only replacement:** `setTraceProcessors([agentInspectProcessor(...)])` replaces existing processors for the current process. This is the documented safe default when you want AgentInspect to own local trace output.
|
|
348
|
+
- **Additional processor:** `addTraceProcessor(agentInspectProcessor(...))` is an advanced, user-owned choice. It can preserve existing/default processors and any backend export behavior they already perform.
|
|
349
|
+
|
|
350
|
+
- **No auto-install** — importing or constructing `agentInspectProcessor()` never calls `setTraceProcessors()` or `addTraceProcessor()`.
|
|
351
|
+
- **No upload behavior** — the processor writes only to an explicit local writer or `traceDir`.
|
|
352
|
+
- **Metadata-only by default** — records trace/span IDs, parentage, names, timing, status, errors, safe model/tool names, token counts, and bounded summaries.
|
|
353
|
+
- **No raw payload capture by default** — prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, and hosted tool payloads are not persisted.
|
|
354
|
+
- **Preview capture is not enabled yet** — `capture: "preview"`, `redactionProfile`, and `maxPreviewChars` are diagnosed through `getDiagnostics()` and do not persist raw previews.
|
|
355
|
+
|
|
356
|
+
Full API: [API.md](./API.md) §14.
|
|
357
|
+
|
|
358
|
+
Runnable local recipe: [openai-agents-local-tracing](../examples/recipes/openai-agents-local-tracing).
|
|
209
359
|
|
|
210
360
|
---
|
|
211
361
|
|