agent-inspect 3.5.3 → 3.5.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -2
- package/README.md +83 -41
- package/docs/ADOPTION.md +3 -1
- package/docs/AI-SDK-ADOPTION.md +109 -0
- package/docs/ARCHITECTURE.md +2 -0
- package/docs/CI-ARTIFACTS.md +2 -0
- package/docs/COMPARE.md +2 -0
- package/docs/DEMO-SCRIPT.md +87 -0
- package/docs/DESIGN-PARTNER-GUIDE.md +71 -0
- package/docs/FIRST-TRACE-IN-5-MINUTES.md +3 -1
- package/docs/GETTING-STARTED.md +2 -0
- package/docs/NESTJS.md +46 -0
- package/docs/OPENAI-AGENTS-LOCAL.md +52 -0
- package/docs/SAFE-TRACE-SHARING.md +2 -0
- package/docs/SCREENSHOTS.md +13 -1
- package/docs/TEAM-WORKFLOWS.md +47 -0
- package/docs/TECHNICAL-GUIDE.md +526 -0
- package/docs/VSCODE.md +21 -0
- package/docs/assets/agent-inspect-logo-dark.svg +13 -0
- package/docs/assets/agent-inspect-logo.svg +15 -0
- package/docs/assets/readme-product-loop.svg +31 -0
- package/package.json +19 -5
- package/packages/cli/dist/index.cjs +1 -1
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 3.5.4
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 1ffe989: v3.5.4 README adoption polish: centered brand header, product-loop visual, npm package files for linked docs/assets. Docs-only; no runtime API changes.
|
|
8
|
+
|
|
3
9
|
## 3.5.3
|
|
4
10
|
|
|
5
11
|
### Patch Changes
|
|
@@ -50,9 +56,9 @@
|
|
|
50
56
|
|
|
51
57
|
## Unreleased
|
|
52
58
|
|
|
53
|
-
|
|
59
|
+
README adoption polish (brand header, product-loop visual, npm link hygiene). Docs-only; no runtime feature changes.
|
|
54
60
|
|
|
55
|
-
See [docs/implementation/
|
|
61
|
+
See [docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md](docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md).
|
|
56
62
|
|
|
57
63
|
---
|
|
58
64
|
|
package/README.md
CHANGED
|
@@ -1,11 +1,37 @@
|
|
|
1
|
-
|
|
1
|
+
<p align="center">
|
|
2
|
+
<picture>
|
|
3
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/agent-inspect-logo-dark.svg">
|
|
4
|
+
<img src="docs/assets/agent-inspect-logo.svg" width="240" alt="AgentInspect">
|
|
5
|
+
</picture>
|
|
6
|
+
</p>
|
|
2
7
|
|
|
3
|
-
|
|
8
|
+
<h1 align="center">agent-inspect</h1>
|
|
4
9
|
|
|
5
|
-
|
|
10
|
+
<p align="center">
|
|
11
|
+
<strong>Trace, check, and safely share TypeScript AI-agent runs locally.</strong>
|
|
12
|
+
</p>
|
|
6
13
|
|
|
7
|
-
|
|
8
|
-
|
|
14
|
+
<p align="center">
|
|
15
|
+
<sub>No account · no upload · no hosted dashboard · metadata-only by default</sub>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<a href="https://agentinspect.vercel.app/">Website</a> ·
|
|
20
|
+
<a href="https://agentinspect.vercel.app/docs/">Docs</a> ·
|
|
21
|
+
<a href="https://www.npmjs.com/package/agent-inspect">npm</a> ·
|
|
22
|
+
<a href="https://github.com/rajudandigam/agent-inspect">GitHub</a>
|
|
23
|
+
</p>
|
|
24
|
+
|
|
25
|
+
<p align="center">
|
|
26
|
+
<a href="https://www.npmjs.com/package/agent-inspect"><img src="https://img.shields.io/npm/v/agent-inspect.svg" alt="npm version"></a>
|
|
27
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="MIT license"></a>
|
|
28
|
+
<a href="package.json"><img src="https://img.shields.io/badge/node-%3E%3D20-brightgreen" alt="Node.js >= 20"></a>
|
|
29
|
+
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178c6" alt="TypeScript"></a>
|
|
30
|
+
</p>
|
|
31
|
+
|
|
32
|
+
agent-inspect turns AI-agent runs into readable **local execution trees**: framework events, observed objects, tool calls, LLM steps, retries, failures, timings, sessions, and CI artifacts — without an account, collector, or hosted dashboard.
|
|
33
|
+
|
|
34
|
+
**Default loop:** capture locally → inspect / report / diff → check in CI → redact before sharing.
|
|
9
35
|
|
|
10
36
|
```bash
|
|
11
37
|
npm install agent-inspect
|
|
@@ -22,46 +48,56 @@ npx agent-inspect report <run-id> --dir .agent-inspect
|
|
|
22
48
|
npx agent-inspect verify-safe --dir .agent-inspect
|
|
23
49
|
```
|
|
24
50
|
|
|
25
|
-
|
|
51
|
+
Install → one trace → one failure check → one share-safe artifact in under five minutes.
|
|
52
|
+
Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](docs/FIRST-TRACE-IN-5-MINUTES.md)
|
|
53
|
+
|
|
54
|
+
<p align="center">
|
|
55
|
+
<img src="docs/assets/readme-product-loop.svg" alt="Capture, inspect, check, redact — local JSONL only" width="720">
|
|
56
|
+
</p>
|
|
26
57
|
|
|
27
58
|
## Choose your path
|
|
28
59
|
|
|
29
|
-
| Path |
|
|
30
|
-
| ---- |
|
|
60
|
+
| Path | Use when | Start |
|
|
61
|
+
| ---- | -------- | ----- |
|
|
31
62
|
| **AI SDK** | Vercel AI SDK `generateText` / `streamText` | [`@agent-inspect/ai-sdk`](packages/ai-sdk/README.md) · [guide](docs/AI-SDK-ADOPTION.md) |
|
|
32
|
-
| **OpenAI Agents** | OpenAI Agents JS
|
|
33
|
-
| **LangChain** |
|
|
34
|
-
| **Observe** |
|
|
35
|
-
| **Manual** | Custom
|
|
63
|
+
| **OpenAI Agents** | OpenAI Agents JS | [`@agent-inspect/openai-agents`](packages/openai-agents/README.md) · [guide](docs/OPENAI-AGENTS-LOCAL.md) |
|
|
64
|
+
| **LangChain** | Callbacks / LangGraph-via-LangChain | [`@agent-inspect/langchain`](packages/langchain/README.md) |
|
|
65
|
+
| **Observe** | Object/class with `run` / `execute` / `invoke` | [Getting started](docs/GETTING-STARTED.md) |
|
|
66
|
+
| **Manual** | Custom spans and nesting | `inspectRun` + `step` in [API](docs/API.md) |
|
|
36
67
|
| **Logs** | Structured logs already emitted | [Log-to-tree](docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
37
|
-
| **CI / tests** | Failed
|
|
38
|
-
| **Real projects** | Fixture
|
|
68
|
+
| **CI / tests** | Failed-test artifacts | [`@agent-inspect/vitest`](packages/vitest/README.md) · [`@agent-inspect/jest`](packages/jest/README.md) |
|
|
69
|
+
| **Real projects** | Fixture runner / bootstrap | [`@agent-inspect/harness`](packages/harness/README.md) · [NestJS](docs/NESTJS.md) |
|
|
39
70
|
|
|
40
71
|
Blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
41
72
|
|
|
42
73
|
## What it helps with
|
|
43
74
|
|
|
44
|
-
- **Wrong tool call** —
|
|
45
|
-
- **
|
|
46
|
-
- **
|
|
47
|
-
- **PR
|
|
75
|
+
- **Wrong tool call** — tool steps, args metadata, and parent run in one tree
|
|
76
|
+
- **Failed eval / test** — `check`, `eval`, Vitest/Jest reporters on failure
|
|
77
|
+
- **Baseline vs candidate** — local `diff` of two runs
|
|
78
|
+
- **PR trace artifact** — [CI artifacts](docs/CI-ARTIFACTS.md) + `redact --profile share`
|
|
79
|
+
- **Safe incident handoff** — [safe sharing](https://agentinspect.vercel.app/docs/safe-sharing/) before Slack or GitHub
|
|
48
80
|
- **Multi-agent / sessions** — `sessions`, `search`, handoff metadata
|
|
49
|
-
- **MCP
|
|
50
|
-
- **VS Code** — in-repo extension (
|
|
81
|
+
- **MCP tool calls** — [`@agent-inspect/mcp`](packages/mcp/README.md) client tracing
|
|
82
|
+
- **VS Code review** — in-repo extension ([dev guide](docs/VSCODE.md); Marketplace not published yet)
|
|
83
|
+
- **Existing logs** — parse structured logs when you cannot instrument
|
|
51
84
|
|
|
52
85
|
## Real-world scenarios
|
|
53
86
|
|
|
54
|
-
| Scenario |
|
|
55
|
-
| -------- |
|
|
56
|
-
| Local debugging | [
|
|
57
|
-
| CI failure review | [
|
|
58
|
-
| Team adoption | [
|
|
87
|
+
| Scenario | Where to start |
|
|
88
|
+
| -------- | -------------- |
|
|
89
|
+
| Local debugging | [Use cases](docs/USE-CASES.md) · [broken-agent starter](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
90
|
+
| CI failure review | [CI artifacts](docs/CI-ARTIFACTS.md) · [ci-eval-redact](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/ci-eval-redact) |
|
|
91
|
+
| Team adoption | [Team workflows](docs/TEAM-WORKFLOWS.md) · [Design partners](docs/DESIGN-PARTNER-GUIDE.md) |
|
|
92
|
+
| Safe sharing | [Safe sharing](https://agentinspect.vercel.app/docs/safe-sharing/) · [repo](docs/SAFE-TRACE-SHARING.md) |
|
|
93
|
+
| Framework-native tracing | [Adapters](docs/ADAPTERS.md) · package READMEs above |
|
|
94
|
+
| Design partner trial | [Design partner guide](docs/DESIGN-PARTNER-GUIDE.md) · [Demo script](docs/DEMO-SCRIPT.md) |
|
|
59
95
|
|
|
60
96
|
## Package map
|
|
61
97
|
|
|
62
98
|
| Package | Purpose |
|
|
63
99
|
| ------- | ------- |
|
|
64
|
-
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core + CLI |
|
|
100
|
+
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI |
|
|
65
101
|
| [`@agent-inspect/ai-sdk`](packages/ai-sdk/README.md) | AI SDK telemetry |
|
|
66
102
|
| [`@agent-inspect/openai-agents`](packages/openai-agents/README.md) | OpenAI Agents processor |
|
|
67
103
|
| [`@agent-inspect/langchain`](packages/langchain/README.md) | LangChain callbacks |
|
|
@@ -73,45 +109,51 @@ Blessed starters (no API keys): [examples/starters](https://github.com/rajudandi
|
|
|
73
109
|
| [`@agent-inspect/mcp`](packages/mcp/README.md) | MCP client tracing |
|
|
74
110
|
| [`@agent-inspect/mcp-server`](packages/mcp-server/README.md) | Read-only trace MCP server |
|
|
75
111
|
| [`@agent-inspect/guardrails`](packages/guardrails/README.md) | Deterministic guardrail rules |
|
|
76
|
-
| [`@agent-inspect/circuit`](packages/circuit/README.md) | Loop/retry/timeout analyzers |
|
|
112
|
+
| [`@agent-inspect/circuit`](packages/circuit/README.md) | Loop / retry / timeout analyzers |
|
|
77
113
|
| [`@agent-inspect/viewer`](packages/viewer/README.md) | Localhost viewer |
|
|
78
114
|
| [`@agent-inspect/adapter-sdk`](packages/adapter-sdk/README.md) | Third-party adapters |
|
|
79
115
|
| [`@agent-inspect/tui`](packages/tui/README.md) | Optional terminal UI |
|
|
80
|
-
| `agent-inspect-vscode` | VS Code extension (in-repo
|
|
116
|
+
| `agent-inspect-vscode` | VS Code extension (in-repo; not on Marketplace yet) |
|
|
81
117
|
|
|
82
118
|
## Safety model
|
|
83
119
|
|
|
84
|
-
- Traces are **local JSONL
|
|
120
|
+
- Traces are **local JSONL** under `.agent-inspect/` (or `AGENT_INSPECT_TRACE_DIR`)
|
|
85
121
|
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
86
122
|
- **No hidden upload** — AgentInspect does not send traces to the cloud
|
|
87
|
-
- **Redaction profiles** — `local` / `share` / `strict` via [`@agent-inspect/redact`](packages/redact/README.md)
|
|
123
|
+
- **Redaction profiles** — `local` / `share` / `strict` via CLI or [`@agent-inspect/redact`](packages/redact/README.md)
|
|
88
124
|
- **`scan` / `verify-safe`** — check artifacts before sharing
|
|
89
|
-
- **Not** a chain-of-thought recorder or compliance engine
|
|
125
|
+
- **Not** a chain-of-thought recorder or compliance engine — review exports before posting
|
|
90
126
|
|
|
91
|
-
Details: [Safe
|
|
127
|
+
Details: [Safe sharing](https://agentinspect.vercel.app/docs/safe-sharing/) · [repo](docs/SAFE-TRACE-SHARING.md) · [Security](SECURITY.md)
|
|
92
128
|
|
|
93
129
|
## Documentation
|
|
94
130
|
|
|
95
|
-
|
|
|
96
|
-
|
|
|
97
|
-
|
|
|
98
|
-
|
|
|
99
|
-
|
|
|
100
|
-
|
|
|
131
|
+
| | Website | Repo |
|
|
132
|
+
| - | ------- | ---- |
|
|
133
|
+
| Getting started | [docs/getting-started](https://agentinspect.vercel.app/docs/getting-started/) | [GETTING-STARTED.md](docs/GETTING-STARTED.md) |
|
|
134
|
+
| Safe sharing | [docs/safe-sharing](https://agentinspect.vercel.app/docs/safe-sharing/) | [SAFE-TRACE-SHARING.md](docs/SAFE-TRACE-SHARING.md) |
|
|
135
|
+
| Compare | [docs/compare](https://agentinspect.vercel.app/docs/compare/) | [COMPARE.md](docs/COMPARE.md) |
|
|
136
|
+
| API / CLI | — | [API.md](docs/API.md) · [CLI.md](docs/CLI.md) |
|
|
137
|
+
| Adoption | — | [ADOPTION.md](docs/ADOPTION.md) · [USE-CASES.md](docs/USE-CASES.md) |
|
|
138
|
+
| Technical guide | — | [TECHNICAL-GUIDE.md](docs/TECHNICAL-GUIDE.md) |
|
|
139
|
+
| Examples | — | [starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters) |
|
|
140
|
+
| Visual demos | — | [SCREENSHOTS.md](docs/SCREENSHOTS.md) |
|
|
101
141
|
|
|
102
|
-
Full index: [docs/README.md](docs/README.md)
|
|
142
|
+
Full index: [docs/README.md](docs/README.md)
|
|
103
143
|
|
|
104
144
|
## What AgentInspect is not
|
|
105
145
|
|
|
106
146
|
- Hosted SaaS or dashboard product
|
|
107
147
|
- Production APM replacement (use LangSmith, Langfuse, OTel, etc. alongside)
|
|
108
148
|
- Eval dataset platform or LLM-as-judge service
|
|
109
|
-
- Prompt registry or pricing engine
|
|
149
|
+
- Prompt registry or provider pricing engine
|
|
110
150
|
- Default telemetry uploader or replay engine
|
|
111
151
|
|
|
152
|
+
See [Compare](https://agentinspect.vercel.app/docs/compare/).
|
|
153
|
+
|
|
112
154
|
## Install details
|
|
113
155
|
|
|
114
|
-
Current release: **3.5.
|
|
156
|
+
Current release: **3.5.3** (sixteen linked npm packages). Persisted trace schema **1.0**. Requires **Node.js >= 20**.
|
|
115
157
|
|
|
116
158
|
```bash
|
|
117
159
|
pnpm add agent-inspect
|
package/docs/ADOPTION.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
AgentInspect is **local-first**: traces stay on disk, checks run in CI, and sharing is opt-in via redaction. This guide is the blessed path from zero to a working team workflow.
|
|
4
4
|
|
|
5
|
+
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) · **Docs:** [https://agentinspect.vercel.app/docs/](https://agentinspect.vercel.app/docs/) · **Getting started:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
|
|
6
|
+
|
|
5
7
|
## Who this is for
|
|
6
8
|
|
|
7
9
|
- TypeScript/Node teams shipping AI agents (not toy demos)
|
|
@@ -10,7 +12,7 @@ AgentInspect is **local-first**: traces stay on disk, checks run in CI, and shar
|
|
|
10
12
|
|
|
11
13
|
## 5-minute path
|
|
12
14
|
|
|
13
|
-
See [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md).
|
|
15
|
+
See [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md) or the [docs site getting started](https://agentinspect.vercel.app/docs/getting-started/).
|
|
14
16
|
|
|
15
17
|
```bash
|
|
16
18
|
npm install agent-inspect
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# AI SDK adoption guide
|
|
2
|
+
|
|
3
|
+
Blessed path for **Vercel AI SDK** + AgentInspect — local traces only, metadata-first by default.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install agent-inspect @agent-inspect/ai-sdk ai
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Or bootstrap a project:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
npx agent-inspect init --framework ai-sdk
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Minimal `generateText`
|
|
18
|
+
|
|
19
|
+
```ts
|
|
20
|
+
import { generateText } from "ai";
|
|
21
|
+
import { agentInspect } from "@agent-inspect/ai-sdk";
|
|
22
|
+
|
|
23
|
+
await generateText({
|
|
24
|
+
model: yourModel,
|
|
25
|
+
prompt: "Hello",
|
|
26
|
+
experimental_telemetry: {
|
|
27
|
+
isEnabled: true,
|
|
28
|
+
recordInputs: false,
|
|
29
|
+
recordOutputs: false,
|
|
30
|
+
integrations: [
|
|
31
|
+
agentInspect({
|
|
32
|
+
traceDir: ".agent-inspect",
|
|
33
|
+
runName: "support-agent",
|
|
34
|
+
capture: "metadata-only",
|
|
35
|
+
}),
|
|
36
|
+
],
|
|
37
|
+
},
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## `streamText` (metadata-only)
|
|
42
|
+
|
|
43
|
+
```ts
|
|
44
|
+
import { streamText } from "ai";
|
|
45
|
+
import { agentInspect } from "@agent-inspect/ai-sdk";
|
|
46
|
+
|
|
47
|
+
const result = streamText({
|
|
48
|
+
model: yourModel,
|
|
49
|
+
prompt: "Hello",
|
|
50
|
+
experimental_telemetry: {
|
|
51
|
+
isEnabled: true,
|
|
52
|
+
recordInputs: false,
|
|
53
|
+
recordOutputs: false,
|
|
54
|
+
integrations: [
|
|
55
|
+
agentInspect({
|
|
56
|
+
traceDir: ".agent-inspect",
|
|
57
|
+
runName: "stream-demo",
|
|
58
|
+
capture: "metadata-only",
|
|
59
|
+
}),
|
|
60
|
+
],
|
|
61
|
+
},
|
|
62
|
+
});
|
|
63
|
+
|
|
64
|
+
for await (const _chunk of result.textStream) {
|
|
65
|
+
// consume stream
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Streaming lifecycle metadata is captured; raw token streams are not persisted by default.
|
|
70
|
+
|
|
71
|
+
## Tool calls
|
|
72
|
+
|
|
73
|
+
Tool spans appear when the model invokes tools. Keep `recordInputs` / `recordOutputs` false unless you explicitly accept content capture risk.
|
|
74
|
+
|
|
75
|
+
## Next.js route handler
|
|
76
|
+
|
|
77
|
+
See [examples/recipes/ai-sdk-next-route](../../examples/recipes/ai-sdk-next-route/) — one `agentInspect()` integration per request, no network, metadata-only defaults.
|
|
78
|
+
|
|
79
|
+
## Privacy controls
|
|
80
|
+
|
|
81
|
+
| Setting | Required default | Why |
|
|
82
|
+
| ------- | ---------------- | --- |
|
|
83
|
+
| `recordInputs: false` | yes | Prevents AI SDK from recording prompts into telemetry payloads |
|
|
84
|
+
| `recordOutputs: false` | yes | Prevents model output capture in telemetry |
|
|
85
|
+
| `capture: "metadata-only"` | yes (adapter) | AgentInspect adapter redacts/bounds persisted fields |
|
|
86
|
+
|
|
87
|
+
## Inspect locally
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
npx agent-inspect list --dir .agent-inspect
|
|
91
|
+
npx agent-inspect open .agent-inspect/<run>.jsonl
|
|
92
|
+
npx agent-inspect check .agent-inspect/<run>.jsonl
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
## Troubleshooting
|
|
96
|
+
|
|
97
|
+
| Symptom | Fix |
|
|
98
|
+
| ------- | --- |
|
|
99
|
+
| No trace file | Ensure `experimental_telemetry.isEnabled: true` and integrations include `agentInspect()` |
|
|
100
|
+
| Empty trace | Confirm `AGENT_INSPECT` is not `0` |
|
|
101
|
+
| Prompts in trace | Set `recordInputs: false` and `recordOutputs: false` on the AI SDK call |
|
|
102
|
+
| Wrong directory | Pass `traceDir` to `agentInspect()` or set `AGENT_INSPECT_TRACE_DIR` |
|
|
103
|
+
|
|
104
|
+
## Recipes (no network)
|
|
105
|
+
|
|
106
|
+
- [ai-sdk-local-telemetry](../../examples/recipes/ai-sdk-local-telemetry/)
|
|
107
|
+
- [ai-sdk-next-route](../../examples/recipes/ai-sdk-next-route/)
|
|
108
|
+
|
|
109
|
+
See also [ADAPTERS.md](./ADAPTERS.md) and [ADAPTER-CONFORMANCE.md](./ADAPTER-CONFORMANCE.md).
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
## Architecture
|
|
2
2
|
|
|
3
|
+
> **Adoption-focused overview:** see [TECHNICAL-GUIDE.md](./TECHNICAL-GUIDE.md) for a comprehensive technical guide (v3.5.3) suitable for blogs and outreach.
|
|
4
|
+
|
|
3
5
|
AgentInspect is a **local-first execution-tree debugger**: manual traces and log ingest produce inspectable step trees stored as JSONL on disk (default `.agent-inspect-runs/`), with a CLI for list/view/clean/logs/tail/export/diff workflows.
|
|
4
6
|
|
|
5
7
|
### Package layout
|
package/docs/CI-ARTIFACTS.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# CI trace artifacts
|
|
2
2
|
|
|
3
|
+
**Docs site:** [https://agentinspect.vercel.app/docs/ci/](https://agentinspect.vercel.app/docs/ci/)
|
|
4
|
+
|
|
3
5
|
AgentInspect helps you **write and export traces locally** in CI. Uploading artifacts is done by **your CI platform** (e.g. GitHub Actions `upload-artifact`) — AgentInspect does not upload anywhere.
|
|
4
6
|
|
|
5
7
|
## Quick pattern
|
package/docs/COMPARE.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
AgentInspect is a local-first execution-tree debugger for TypeScript AI agents. It’s designed for inner-loop debugging, deterministic local eval heuristics, redaction, and quick inspection — not as a replacement for hosted observability, dataset evaluation, or production monitoring platforms.
|
|
4
4
|
|
|
5
|
+
**Docs site:** [https://agentinspect.vercel.app/docs/compare/](https://agentinspect.vercel.app/docs/compare/)
|
|
6
|
+
|
|
5
7
|
## AgentInspect vs console.log
|
|
6
8
|
|
|
7
9
|
- **console.log is flat**: logs are a stream of lines without run grouping or step boundaries.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Live demo script (~3 minutes)
|
|
2
|
+
|
|
3
|
+
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
+
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect`. Monorepo: `pnpm build` at repo root.
|
|
5
|
+
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/)
|
|
6
|
+
|
|
7
|
+
**Blessed demo starter:** [broken-agent-debugging](../examples/starters/broken-agent-debugging/) — intentional tool failure, no API keys.
|
|
8
|
+
|
|
9
|
+
## Setup (before the call)
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
cd examples/starters/broken-agent-debugging
|
|
13
|
+
pnpm install && pnpm start
|
|
14
|
+
npx agent-inspect list --dir .agent-inspect
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Copy a `<run-id>` for beats below.
|
|
18
|
+
|
|
19
|
+
## Beat 1 — Problem (30s)
|
|
20
|
+
|
|
21
|
+
"Console logs are flat. When an agent picks the wrong tool or a step throws, you can't see parent/child relationships or which step failed first."
|
|
22
|
+
|
|
23
|
+
## Beat 2 — Capture (45s)
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
pnpm start # writes failing trace to .agent-inspect/
|
|
27
|
+
npx agent-inspect list --dir .agent-inspect
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Point out: run id, status, duration — **no upload**, metadata-only by default.
|
|
31
|
+
|
|
32
|
+
## Beat 3 — Inspect (45s)
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
36
|
+
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
37
|
+
npx agent-inspect timeline <run-id> --dir .agent-inspect
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Optional: `npx agent-inspect serve --dir .agent-inspect` for browser viewer.
|
|
41
|
+
|
|
42
|
+
## Beat 4 — Verify (30s)
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
npx agent-inspect check .agent-inspect/*.jsonl --require-completed
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Mention `--detect-stalls` and `--max-step-duration 30s` for long-running agents.
|
|
49
|
+
|
|
50
|
+
## Beat 5 — Share safely (30s)
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
npx agent-inspect redact .agent-inspect/*.jsonl --profile share -o safe.jsonl
|
|
54
|
+
npx agent-inspect verify-safe safe.jsonl
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
"Redacted copy is safe to attach to a GitHub issue or Slack."
|
|
58
|
+
|
|
59
|
+
## Beat 6 — Fix and diff (optional, 30s)
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
pnpm run fixed
|
|
63
|
+
npx agent-inspect diff .agent-inspect/<broken-run>.jsonl .agent-inspect/<fixed-run>.jsonl
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## Beat 7 — Close (15s)
|
|
67
|
+
|
|
68
|
+
"Starters for AI SDK, OpenAI Agents, LangChain, CI, and NestJS harness are in `examples/starters/`. Fresh repos: `npx agent-inspect init --yes`."
|
|
69
|
+
|
|
70
|
+
## Alternative opener (zero clone)
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
npm install agent-inspect
|
|
74
|
+
npx agent-inspect init --yes
|
|
75
|
+
node examples/agent-inspect-demo.mjs
|
|
76
|
+
npx agent-inspect list --dir .agent-inspect
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
See [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md).
|
|
80
|
+
|
|
81
|
+
## Optional beats
|
|
82
|
+
|
|
83
|
+
- **VS Code:** F5 from `packages/vscode` (dev host) — Marketplace listing pending
|
|
84
|
+
- **Doctor:** `npx agent-inspect doctor` when onboarding fails
|
|
85
|
+
- **Framework:** switch to `examples/starters/ai-sdk` for adapter path
|
|
86
|
+
|
|
87
|
+
Related: [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) · [SCREENSHOTS.md](./SCREENSHOTS.md)
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Design partner guide
|
|
2
|
+
|
|
3
|
+
Thank you for trying AgentInspect early. This doc is the **evaluation contract** — what we ask you to try, what we measure, and what we won't do.
|
|
4
|
+
|
|
5
|
+
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) · **Docs:** [https://agentinspect.vercel.app/docs/](https://agentinspect.vercel.app/docs/)
|
|
6
|
+
|
|
7
|
+
## What we're validating
|
|
8
|
+
|
|
9
|
+
1. **First trace < 30 minutes** from `npm install` (with `init` + a starter)
|
|
10
|
+
2. **First CI check** on a real or fixture trace
|
|
11
|
+
3. **Share-safe workflow** — redacted artifact attached to an issue or PR
|
|
12
|
+
4. **Framework fit** — AI SDK, OpenAI Agents, or LangChain path feels native enough
|
|
13
|
+
|
|
14
|
+
## Your path
|
|
15
|
+
|
|
16
|
+
| Week | Task | Success signal |
|
|
17
|
+
| ---- | ---- | -------------- |
|
|
18
|
+
| 1 | Run `init` + one starter | `list` shows a run |
|
|
19
|
+
| 1 | `doctor` clean | No failed checks |
|
|
20
|
+
| 2 | Wire adapter or `observe()` in your app | Real trace captured |
|
|
21
|
+
| 2 | `check` or `eval` in CI | Job fails on bad fixture |
|
|
22
|
+
| 3 | `redact --profile share` | Comfortable posting trace externally |
|
|
23
|
+
| 4 | Feedback session | Case study draft or structured notes |
|
|
24
|
+
|
|
25
|
+
## What we provide
|
|
26
|
+
|
|
27
|
+
- Starters: [examples/starters/](../examples/starters/README.md)
|
|
28
|
+
- Office hours async via GitHub Discussions / Issues
|
|
29
|
+
- [DEMO-SCRIPT.md](./DEMO-SCRIPT.md) for internal team demos
|
|
30
|
+
|
|
31
|
+
## What we ask from you
|
|
32
|
+
|
|
33
|
+
- **Structured feedback** (template below) — not vague "looks good"
|
|
34
|
+
- **Redacted traces** only — never raw secrets
|
|
35
|
+
- Permission to quote anonymously unless you approve public case study
|
|
36
|
+
|
|
37
|
+
## Feedback template
|
|
38
|
+
|
|
39
|
+
```markdown
|
|
40
|
+
### Environment
|
|
41
|
+
- Node version:
|
|
42
|
+
- Framework:
|
|
43
|
+
- AgentInspect version:
|
|
44
|
+
|
|
45
|
+
### First trace
|
|
46
|
+
- Time to first trace:
|
|
47
|
+
- Blockers:
|
|
48
|
+
|
|
49
|
+
### CI
|
|
50
|
+
- check/eval used?
|
|
51
|
+
- Rules that mattered:
|
|
52
|
+
|
|
53
|
+
### Gaps
|
|
54
|
+
- Missing docs:
|
|
55
|
+
- Missing framework coverage:
|
|
56
|
+
- Would not adopt because:
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Metrics (no hidden telemetry)
|
|
60
|
+
|
|
61
|
+
We track adoption from **public signals** and **your reports** only. See [product/ADOPTION-METRICS.md](./product/ADOPTION-METRICS.md).
|
|
62
|
+
|
|
63
|
+
## Out of scope for partners
|
|
64
|
+
|
|
65
|
+
- Hosted dashboards or SaaS
|
|
66
|
+
- Custom adapter development on our roadmap without demand gates
|
|
67
|
+
- SLAs — this is open-source MIT software
|
|
68
|
+
|
|
69
|
+
## Contact
|
|
70
|
+
|
|
71
|
+
Open a GitHub issue with label `design-partner` or email the maintainer listed in the repo.
|
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
Goal: install → one trace → one check → one share-safe artifact.
|
|
4
4
|
|
|
5
|
+
**Docs site:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
|
|
6
|
+
|
|
5
7
|
```bash
|
|
6
8
|
npm install agent-inspect && npx agent-inspect init --yes && node examples/agent-inspect-demo.mjs
|
|
7
9
|
npx agent-inspect list --dir .agent-inspect && npx agent-inspect report <run-id> --dir .agent-inspect
|
|
@@ -59,4 +61,4 @@ Attach redacted JSONL to a PR or issue — not raw traces.
|
|
|
59
61
|
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) |
|
|
60
62
|
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
61
63
|
|
|
62
|
-
Full index: [docs/README.md](./README.md)
|
|
64
|
+
Full index: [docs/README.md](./README.md) · Website docs: [agentinspect.vercel.app/docs](https://agentinspect.vercel.app/docs/)
|
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
AgentInspect is a **local-first execution-tree debugger** for TypeScript AI agents. It helps you produce and inspect an execution tree of steps, safely and deterministically, without uploading data anywhere.
|
|
4
4
|
|
|
5
|
+
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) · **Docs site:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
|
|
6
|
+
|
|
5
7
|
**Visual demos:** [SCREENSHOTS.md](./SCREENSHOTS.md)
|
|
6
8
|
|
|
7
9
|
## 1. Install
|
package/docs/NESTJS.md
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# NestJS + AgentInspect (harness path)
|
|
2
|
+
|
|
3
|
+
AgentInspect does **not** ship a production NestJS interceptor in v3.2. Use **`@agent-inspect/harness`** to bootstrap real Nest apps for local fixture runs.
|
|
4
|
+
|
|
5
|
+
## Recommended path (v3.2)
|
|
6
|
+
|
|
7
|
+
1. `npx agent-inspect init` or use [examples/starters/harness-nestjs](../examples/starters/harness-nestjs/)
|
|
8
|
+
2. Wire `createFixtureRunner()` with your `TestingModule` / app bootstrap
|
|
9
|
+
3. Run targets with JSON fixtures — no real model calls by default
|
|
10
|
+
|
|
11
|
+
```ts
|
|
12
|
+
import { createFixtureRunner, defineTarget } from "@agent-inspect/harness";
|
|
13
|
+
|
|
14
|
+
await createFixtureRunner({
|
|
15
|
+
name: "nestjs-support",
|
|
16
|
+
trace: { mode: "run-if-enabled", traceDir: ".agent-inspect" },
|
|
17
|
+
bootstrap: async () => {
|
|
18
|
+
const moduleRef = await Test.createTestingModule({ /* ... */ }).compile();
|
|
19
|
+
return moduleRef.createNestApplication();
|
|
20
|
+
},
|
|
21
|
+
shutdown: async (app) => {
|
|
22
|
+
await app?.close?.();
|
|
23
|
+
},
|
|
24
|
+
targets: {
|
|
25
|
+
ask: defineTarget({
|
|
26
|
+
resolve: (app) => app.get(SupportAgent),
|
|
27
|
+
invoke: (agent, input) => agent.run(input),
|
|
28
|
+
}),
|
|
29
|
+
},
|
|
30
|
+
}).runFromArgv();
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Out of scope (v3.2)
|
|
34
|
+
|
|
35
|
+
- `@agent-inspect/nestjs` production package (demand-gated)
|
|
36
|
+
- Redis/SQS mocking in harness
|
|
37
|
+
- Global monkey-patching of Nest providers
|
|
38
|
+
|
|
39
|
+
## Structured logs alternative
|
|
40
|
+
|
|
41
|
+
If you only need log ingestion, see [examples/recipes/nestjs-json-logging](../examples/recipes/nestjs-json-logging/).
|
|
42
|
+
|
|
43
|
+
## Related
|
|
44
|
+
|
|
45
|
+
- [`@agent-inspect/harness`](../packages/harness/README.md)
|
|
46
|
+
- [examples/starters/harness-nestjs](../examples/starters/harness-nestjs/)
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# OpenAI Agents JS — local-only tracing
|
|
2
|
+
|
|
3
|
+
How to use `@agent-inspect/openai-agents` without sending traces to OpenAI's default export pipeline.
|
|
4
|
+
|
|
5
|
+
## Two processor modes
|
|
6
|
+
|
|
7
|
+
| Mode | API | OpenAI default export | AgentInspect traces |
|
|
8
|
+
| ---- | --- | --------------------- | ------------------- |
|
|
9
|
+
| **Additional processor** | `addTraceProcessor(agentInspect(...))` | May still run | Local JSONL via AgentInspect |
|
|
10
|
+
| **Replacement list** | `setTraceProcessors([agentInspect(...)])` | Disabled when list is only AgentInspect | Local only |
|
|
11
|
+
|
|
12
|
+
For **local-only** debugging, prefer **`setTraceProcessors`** with only the AgentInspect processor unless you explicitly need OpenAI's export.
|
|
13
|
+
|
|
14
|
+
```ts
|
|
15
|
+
import { setTraceProcessors } from "@openai/agents";
|
|
16
|
+
import { agentInspect } from "@agent-inspect/openai-agents";
|
|
17
|
+
|
|
18
|
+
setTraceProcessors([
|
|
19
|
+
agentInspect({
|
|
20
|
+
traceDir: ".agent-inspect",
|
|
21
|
+
capture: "metadata-only",
|
|
22
|
+
}),
|
|
23
|
+
]);
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## What maps to AgentInspect steps
|
|
27
|
+
|
|
28
|
+
- Agent runs → run / step spans
|
|
29
|
+
- Tool calls → `TOOL` steps with metadata
|
|
30
|
+
- Handoffs → correlated steps when metadata is present
|
|
31
|
+
- Guardrails → bounded metadata (no certification claims)
|
|
32
|
+
- MCP tool calls → tool steps when the Agents SDK exposes MCP spans
|
|
33
|
+
|
|
34
|
+
## Privacy
|
|
35
|
+
|
|
36
|
+
- Default `metadata-only` capture
|
|
37
|
+
- No automatic upload from AgentInspect
|
|
38
|
+
- Replacing processors does not by itself redact OpenAI SDK network traffic — review OpenAI SDK settings separately
|
|
39
|
+
|
|
40
|
+
## Recipes
|
|
41
|
+
|
|
42
|
+
- [openai-agents-local-processor](../../examples/recipes/openai-agents-local-processor/)
|
|
43
|
+
|
|
44
|
+
## Troubleshooting
|
|
45
|
+
|
|
46
|
+
| Symptom | Check |
|
|
47
|
+
| ------- | ----- |
|
|
48
|
+
| Traces also appear in OpenAI dashboard | You may be using `addTraceProcessor` alongside default export — switch to `setTraceProcessors` for local-only |
|
|
49
|
+
| No local files | Confirm processor is registered before agent runs; check `traceDir` |
|
|
50
|
+
| Missing tool spans | Ensure tools run inside a traced agent session |
|
|
51
|
+
|
|
52
|
+
See [ADAPTERS.md](./ADAPTERS.md).
|