agent-inspect 1.0.1 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +151 -419
- package/package.json +1 -1
- package/docs/MIGRATION.md +0 -109
package/CHANGELOG.md
CHANGED
package/README.md
CHANGED
|
@@ -1,10 +1,22 @@
|
|
|
1
1
|
# agent-inspect
|
|
2
2
|
|
|
3
|
-
Local execution trees for TypeScript AI agents
|
|
3
|
+
**Local execution trees for TypeScript AI agents.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
agent-inspect helps you understand what happened inside an AI agent run — **locally**. It turns manual steps, tool calls, LLM calls, structured logs, failures, durations, and run metadata into **readable execution trees** you can inspect from the terminal.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it **before** a hosted observability platform, **alongside** one, or as the **local debugging layer** underneath enterprise observability.
|
|
8
|
+
|
|
9
|
+
The tool starts with **manual traces** and **existing structured logs**, and extends into **optional framework callbacks** and **standards-aligned local export** — without turning the core into a SaaS or a vendor pipeline.
|
|
10
|
+
|
|
11
|
+
**No account. No cloud upload. No dashboard required.**
|
|
12
|
+
|
|
13
|
+
## Why agent-inspect exists
|
|
14
|
+
|
|
15
|
+
AI agents are no longer single function calls. They plan, call tools, invoke LLMs, branch, retry, fail, and run work in parallel. **Console logs are flat**; reconstructing causality from a wall of lines is slow and error-prone.
|
|
16
|
+
|
|
17
|
+
**Hosted observability** is valuable in production, but it can be heavy for the **inner loop**: local runs, fast iteration, and debugging before anything reaches a collector or dashboard.
|
|
18
|
+
|
|
19
|
+
agent-inspect gives those runs **structure**: an **execution tree** you can read and diff on disk, with a **CLI-first** workflow and **no vendor lock-in**.
|
|
8
20
|
|
|
9
21
|
## Install
|
|
10
22
|
|
|
@@ -16,6 +28,12 @@ npm install agent-inspect
|
|
|
16
28
|
pnpm add agent-inspect
|
|
17
29
|
```
|
|
18
30
|
|
|
31
|
+
Verify the CLI is available:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
npx agent-inspect --help
|
|
35
|
+
```
|
|
36
|
+
|
|
19
37
|
## 60-second quickstart
|
|
20
38
|
|
|
21
39
|
Create `demo.mjs`:
|
|
@@ -29,25 +47,25 @@ await inspectRun(
|
|
|
29
47
|
"support-agent",
|
|
30
48
|
async () => {
|
|
31
49
|
const plan = await step("plan", async () => {
|
|
32
|
-
await delay(
|
|
33
|
-
return {
|
|
50
|
+
await delay(40);
|
|
51
|
+
return { intent: "refund-policy", needsPolicy: true };
|
|
34
52
|
});
|
|
35
53
|
|
|
36
|
-
const
|
|
37
|
-
await delay(
|
|
38
|
-
return
|
|
54
|
+
const policy = await step.tool("retrieve-policy", async () => {
|
|
55
|
+
await delay(60);
|
|
56
|
+
return { text: "Refunds are available within 30 days of purchase." };
|
|
39
57
|
});
|
|
40
58
|
|
|
41
|
-
return step.llm("answer", async () => {
|
|
42
|
-
await delay(
|
|
43
|
-
return `
|
|
59
|
+
return step.llm("generate-answer", async () => {
|
|
60
|
+
await delay(80);
|
|
61
|
+
return `Policy: ${policy.text} (intent: ${plan.intent})`;
|
|
44
62
|
});
|
|
45
63
|
},
|
|
46
64
|
{ traceDir: "./.agent-inspect" }
|
|
47
65
|
);
|
|
48
66
|
```
|
|
49
67
|
|
|
50
|
-
Run it, then inspect
|
|
68
|
+
Run it, then inspect the trace:
|
|
51
69
|
|
|
52
70
|
```bash
|
|
53
71
|
node demo.mjs
|
|
@@ -56,33 +74,32 @@ npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
|
56
74
|
npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
|
|
57
75
|
```
|
|
58
76
|
|
|
59
|
-
|
|
77
|
+
Full flow:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
npm install agent-inspect
|
|
81
|
+
node demo.mjs
|
|
82
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
**Simplified example output** (actual CLI formatting may differ slightly):
|
|
60
86
|
|
|
61
87
|
```text
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
Summary:
|
|
68
|
-
Steps: 3 (0 error)
|
|
69
|
-
Duration: 225ms
|
|
88
|
+
support-agent
|
|
89
|
+
✔ plan
|
|
90
|
+
✔ tool:retrieve-policy
|
|
91
|
+
✔ llm:generate-answer
|
|
70
92
|
```
|
|
71
93
|
|
|
72
|
-
|
|
94
|
+
A runnable copy lives in [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
73
95
|
|
|
74
|
-
##
|
|
96
|
+
## What the trace shows
|
|
75
97
|
|
|
76
|
-
|
|
98
|
+
Each run produces a **JSONL** trace: `run_started` / `run_completed`, `step_started` / `step_completed`, with **nested steps**, **tool/LLM** types where you use `step.tool` / `step.llm`, and **durations** on completed steps. Failures are recorded on `step_completed` with `status: "error"` (there is no separate `step_failed` event). See [docs/SCHEMA.md](docs/SCHEMA.md).
|
|
77
99
|
|
|
78
|
-
|
|
79
|
-
- explicit step boundaries (including nesting)
|
|
80
|
-
- step types (`tool:*`, `llm:*`)
|
|
81
|
-
- status + duration summaries
|
|
82
|
-
- a CLI to list/view/export/diff runs
|
|
83
|
-
- log ingestion workflows (`logs`, `tail`) when you already have structured logs
|
|
100
|
+
## Works with structured logs you already have
|
|
84
101
|
|
|
85
|
-
|
|
102
|
+
Many production systems already emit **line-delimited JSON** or text logs with embedded JSON (e.g. via **pino**, **winston**, **log4js**, **NestJS** loggers, job runners, or custom event streams). agent-inspect can turn those into **local grouped timelines/trees** without wrapping every function.
|
|
86
103
|
|
|
87
104
|
```bash
|
|
88
105
|
npx agent-inspect logs ./agent.log \
|
|
@@ -92,452 +109,167 @@ npx agent-inspect logs ./agent.log \
|
|
|
92
109
|
--timestamp-key timestamp
|
|
93
110
|
```
|
|
94
111
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
## When to use AgentInspect
|
|
98
|
-
|
|
99
|
-
Use AgentInspect when:
|
|
100
|
-
|
|
101
|
-
- you are building TypeScript/Node.js AI agents
|
|
102
|
-
- you want local debugging before a hosted observability setup
|
|
103
|
-
- console logs are too flat for multi-step execution
|
|
104
|
-
- you want to inspect tool calls, LLM calls, failures, and durations locally
|
|
105
|
-
- you want a lightweight CLI workflow with no account and no cloud upload
|
|
106
|
-
- you want to compare two local runs
|
|
107
|
-
- you want to turn structured logs into readable execution trees
|
|
108
|
-
|
|
109
|
-
## When not to use AgentInspect
|
|
110
|
-
|
|
111
|
-
Do not use AgentInspect as a replacement for:
|
|
112
|
-
|
|
113
|
-
- production monitoring or alerting
|
|
114
|
-
- hosted observability dashboards
|
|
115
|
-
- long-term trace storage
|
|
116
|
-
- eval dataset management
|
|
117
|
-
- prompt management
|
|
118
|
-
- cost analytics
|
|
119
|
-
- replay/fork execution
|
|
120
|
-
- vendor telemetry pipelines
|
|
121
|
-
|
|
122
|
-
AgentInspect can complement tools like LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, etc. It does not replace their production/eval/dashboard workflows.
|
|
123
|
-
|
|
124
|
-
## Security and privacy posture
|
|
125
|
-
|
|
126
|
-
- local files only by default (no upload)
|
|
127
|
-
- no vendor sinks
|
|
128
|
-
- no API keys required
|
|
129
|
-
- small root dependency footprint
|
|
130
|
-
- traces can include **user-provided metadata**; review exports before sharing
|
|
131
|
-
|
|
132
|
-
See `SECURITY.md`.
|
|
133
|
-
|
|
134
|
-
## Documentation
|
|
135
|
-
|
|
136
|
-
- **Getting started**: [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md)
|
|
137
|
-
- **API**: [docs/API.md](docs/API.md)
|
|
138
|
-
- **CLI**: [docs/CLI.md](docs/CLI.md)
|
|
139
|
-
- **Schema**: [docs/SCHEMA.md](docs/SCHEMA.md)
|
|
140
|
-
- **Logs**: [docs/LOGS.md](docs/LOGS.md) and [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
141
|
-
- **Exports**: [docs/EXPORTS.md](docs/EXPORTS.md)
|
|
142
|
-
- **Diff**: [docs/DIFF.md](docs/DIFF.md)
|
|
143
|
-
- **Adapters**: [docs/ADAPTERS.md](docs/ADAPTERS.md)
|
|
144
|
-
- **Compare with other tools**: [docs/COMPARE.md](docs/COMPARE.md)
|
|
145
|
-
- **Known issues**: [docs/KNOWN-ISSUES.md](docs/KNOWN-ISSUES.md)
|
|
146
|
-
- **Limitations**: [docs/LIMITATIONS.md](docs/LIMITATIONS.md)
|
|
147
|
-
|
|
148
|
-
Screenshots/GIFs are planned; see [docs/SCREENSHOTS.md](docs/SCREENSHOTS.md).
|
|
149
|
-
|
|
150
|
-
## Minimal API
|
|
151
|
-
|
|
152
|
-
```ts
|
|
153
|
-
import { inspectRun, step } from "agent-inspect";
|
|
154
|
-
|
|
155
|
-
await inspectRun("my-agent-run", async () => {
|
|
156
|
-
const plan = await step("plan", async () => ({ task: "research" }));
|
|
157
|
-
return step("act", async () => plan);
|
|
158
|
-
});
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
## LLM and tool helpers
|
|
162
|
-
|
|
163
|
-
```ts
|
|
164
|
-
await step.llm("mock-gpt", async () => {
|
|
165
|
-
return planner.run();
|
|
166
|
-
});
|
|
167
|
-
|
|
168
|
-
await step.tool("searchHotels", async () => {
|
|
169
|
-
return searchHotels();
|
|
170
|
-
});
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
Helpers only label steps in the trace.
|
|
174
|
-
|
|
175
|
-
They do not import or call vendor SDKs.
|
|
176
|
-
|
|
177
|
-
## observe()
|
|
178
|
-
|
|
179
|
-
`observe()` wraps top-level `run`, `execute`, and `invoke`.
|
|
180
|
-
|
|
181
|
-
For internal detail, add manual `step()` calls inside the agent.
|
|
182
|
-
|
|
183
|
-
```ts
|
|
184
|
-
import { observe } from "agent-inspect";
|
|
185
|
-
|
|
186
|
-
class MyAgent {
|
|
187
|
-
async run(input: string) {
|
|
188
|
-
return `ok: ${input}`;
|
|
189
|
-
}
|
|
190
|
-
}
|
|
191
|
-
|
|
192
|
-
const agent = observe(new MyAgent());
|
|
193
|
-
await agent.run("hello");
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
See [examples/05-observe-wrapper](examples/05-observe-wrapper) for a top-level observed run with internal `step()`, `step.tool()`, and `step.llm()` calls.
|
|
197
|
-
|
|
198
|
-
## Usage examples
|
|
199
|
-
|
|
200
|
-
### Example 1: Basic workflow
|
|
201
|
-
|
|
202
|
-
```ts
|
|
203
|
-
import { inspectRun, step } from "agent-inspect";
|
|
112
|
+
With a reusable ingest config:
|
|
204
113
|
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
return ["Tokyo Grand Hotel", "Tokyo Central Inn"];
|
|
208
|
-
});
|
|
209
|
-
|
|
210
|
-
const availability = await step("check-availability", async () => {
|
|
211
|
-
return { hotel: hotels[0], rooms: 2 };
|
|
212
|
-
});
|
|
213
|
-
|
|
214
|
-
return step("finalize-booking", async () => {
|
|
215
|
-
return `confirmed:${availability.hotel}`;
|
|
216
|
-
});
|
|
217
|
-
});
|
|
218
|
-
|
|
219
|
-
console.log(result);
|
|
220
|
-
```
|
|
221
|
-
|
|
222
|
-
Expected tree:
|
|
223
|
-
|
|
224
|
-
```text
|
|
225
|
-
hotel-booking
|
|
226
|
-
✔ search-hotels
|
|
227
|
-
✔ check-availability
|
|
228
|
-
✔ finalize-booking
|
|
114
|
+
```bash
|
|
115
|
+
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
229
116
|
```
|
|
230
117
|
|
|
231
|
-
|
|
118
|
+
- **JSON logs** are first-class.
|
|
119
|
+
- **log4js-style** lines are **best-effort** when a recoverable JSON payload is present.
|
|
120
|
+
- **No `eval`**, no JavaScript object-literal parsing as a log interchange format.
|
|
121
|
+
- **Flat timeline by default**; nesting when parent relationships are explicit or configured.
|
|
122
|
+
- **Confidence labels** (`explicit`, `correlated`, `heuristic`, `unknown`) describe how attribution was inferred.
|
|
232
123
|
|
|
233
|
-
|
|
234
|
-
import { inspectRun, step } from "agent-inspect";
|
|
235
|
-
|
|
236
|
-
await inspectRun("trip-planner", async () => {
|
|
237
|
-
const plan = await step("plan-trip", async () => {
|
|
238
|
-
const draft = await step.llm("mock-gpt", async () => {
|
|
239
|
-
return "Plan: museum, dinner, evening walk.";
|
|
240
|
-
});
|
|
241
|
-
|
|
242
|
-
return step("parse-plan", async () => {
|
|
243
|
-
return draft.replace("Plan: ", "").split(", ");
|
|
244
|
-
});
|
|
245
|
-
});
|
|
246
|
-
|
|
247
|
-
const hotels = await step.tool("searchHotels", async () => {
|
|
248
|
-
return [{ id: "h1", city: "Kyoto" }];
|
|
249
|
-
});
|
|
250
|
-
|
|
251
|
-
return step("finalize", async () => {
|
|
252
|
-
return { plan, hotel: hotels[0] };
|
|
253
|
-
});
|
|
254
|
-
});
|
|
255
|
-
```
|
|
124
|
+
More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md).
|
|
256
125
|
|
|
257
|
-
|
|
126
|
+
## CLI at a glance
|
|
258
127
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
128
|
+
| Command | Use it for |
|
|
129
|
+
| -------- | ---------- |
|
|
130
|
+
| `list` | Find recent runs |
|
|
131
|
+
| `view` | Inspect one run as a tree |
|
|
132
|
+
| `clean` | Safely remove old trace files |
|
|
133
|
+
| `logs` | Turn existing structured logs into a local tree/timeline |
|
|
134
|
+
| `tail` | Watch structured logs while the app runs |
|
|
135
|
+
| `export` | Write Markdown / HTML / OpenInference-compatible JSON / OTLP JSON **locally** |
|
|
136
|
+
| `diff` | Compare two local runs (read-only) |
|
|
267
137
|
|
|
268
|
-
|
|
138
|
+
Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
269
139
|
|
|
270
|
-
|
|
271
|
-
import { inspectRun, step } from "agent-inspect";
|
|
140
|
+
## Real-world workflows
|
|
272
141
|
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
142
|
+
- Debug a **failed tool call** or thrown error in a support or ops agent.
|
|
143
|
+
- See **which step dominated latency** in a multi-step planner or RAG pipeline.
|
|
144
|
+
- **Diff two runs** after a prompt, model, or routing change.
|
|
145
|
+
- Point **`logs`** / **`tail`** at existing job or service logs to get a **local execution view** without shipping data upstream.
|
|
146
|
+
- **Export** a run to Markdown for a PR, postmortem, or internal thread — then review before sharing.
|
|
147
|
+
- Keep traces **on disk** while still using enterprise observability elsewhere.
|
|
276
148
|
|
|
277
|
-
|
|
278
|
-
throw new Error("Pricing API timeout");
|
|
279
|
-
});
|
|
149
|
+
## What v1.0 stabilizes
|
|
280
150
|
|
|
281
|
-
|
|
282
|
-
return "this step will not run";
|
|
283
|
-
});
|
|
284
|
-
});
|
|
285
|
-
} catch (error) {
|
|
286
|
-
console.error("Original error still propagated:", error);
|
|
287
|
-
}
|
|
288
|
-
```
|
|
151
|
+
**agent-inspect 1.0** stabilizes the **local debugging foundation**:
|
|
289
152
|
|
|
290
|
-
|
|
153
|
+
- Instrument a run with `inspectRun` and `step`
|
|
154
|
+
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
155
|
+
- Inspect runs with **`list`** and **`view`**
|
|
156
|
+
- Safely remove old trace files with **`clean`**
|
|
291
157
|
|
|
292
|
-
|
|
158
|
+
**Stable APIs:** `inspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`.
|
|
293
159
|
|
|
294
|
-
|
|
295
|
-
import { observe, step } from "agent-inspect";
|
|
296
|
-
|
|
297
|
-
class CustomerSupportAgent {
|
|
298
|
-
async run(question: string): Promise<string> {
|
|
299
|
-
const category = await step("triage-question", async () => {
|
|
300
|
-
return question.toLowerCase().includes("password")
|
|
301
|
-
? "account-access"
|
|
302
|
-
: "general";
|
|
303
|
-
});
|
|
160
|
+
**Stable CLI workflows:** `agent-inspect list`, `agent-inspect view`, `agent-inspect clean`.
|
|
304
161
|
|
|
305
|
-
|
|
306
|
-
return ["Reset your password from the login page."];
|
|
307
|
-
});
|
|
162
|
+
**Also included in 1.0** as local-first extensions:
|
|
308
163
|
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
164
|
+
- Structured log inspection: **`logs`**
|
|
165
|
+
- Live log tailing: **`tail`**
|
|
166
|
+
- Local exports: **`export`** (Markdown, HTML, OpenInference-compatible JSON, OTLP JSON — files only)
|
|
167
|
+
- Local run comparison: **`diff`**
|
|
168
|
+
- Optional **`@agent-inspect/langchain`** callback adapter
|
|
169
|
+
- Optional **`@agent-inspect/tui`** terminal viewer
|
|
170
|
+
- **Fixtures** and **recipes** for deterministic checks and adoption patterns
|
|
314
171
|
|
|
315
|
-
|
|
316
|
-
await agent.run("How do I reset my password?");
|
|
317
|
-
```
|
|
172
|
+
**Honest boundaries:** programmatic log parsing, export, and diff APIs; LangChain and TUI programmatic surfaces; and OpenInference/OTLP JSON exports are **experimental or compatibility-oriented**. Nothing performs **vendor upload** by default.
|
|
318
173
|
|
|
319
|
-
|
|
174
|
+
## Optional packages
|
|
320
175
|
|
|
321
|
-
|
|
176
|
+
### LangChain callback adapter (`@agent-inspect/langchain`)
|
|
322
177
|
|
|
323
|
-
|
|
178
|
+
Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. The LangChain adapter is available in 1.0, but its programmatic API remains experimental and may evolve independently of the stable core tracing API.
|
|
324
179
|
|
|
325
180
|
```bash
|
|
326
181
|
pnpm add agent-inspect @agent-inspect/langchain @langchain/core
|
|
327
182
|
```
|
|
328
183
|
|
|
329
|
-
`@langchain/core` is a **peer dependency** of `@agent-inspect/langchain`. The adapter uses official LangChain.js **callbacks** only (extends `BaseCallbackHandler`): **no** monkey-patching, **no** `agent-inspect/auto`, **no** vendor observability sinks.
|
|
330
|
-
|
|
331
184
|
```ts
|
|
332
185
|
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
333
186
|
|
|
334
187
|
const callback = new AgentInspectCallback({
|
|
335
|
-
runName: "
|
|
188
|
+
runName: "my-run",
|
|
336
189
|
capture: "metadata-only",
|
|
337
190
|
});
|
|
338
191
|
|
|
339
|
-
await agent.invoke(input, {
|
|
340
|
-
callbacks: [callback],
|
|
341
|
-
});
|
|
342
|
-
|
|
192
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
343
193
|
const events = callback.getEvents();
|
|
344
194
|
```
|
|
345
195
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
- **Metadata-only** capture by default (model, tags, token usage when present, counts). **No** full prompt/output capture by default.
|
|
349
|
-
- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
|
|
350
|
-
- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
|
|
351
|
-
- **No** cost calculation; token fields are informational only.
|
|
352
|
-
- Events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into the manual JSONL trace format.
|
|
353
|
-
|
|
354
|
-
The LangChain adapter API remains **experimental** even though the core AgentInspect tracing API is stable in 1.0. See [examples/08-langchain-adapter](examples/08-langchain-adapter).
|
|
355
|
-
|
|
356
|
-
## CLI
|
|
357
|
-
|
|
358
|
-
List recent runs:
|
|
359
|
-
|
|
360
|
-
```bash
|
|
361
|
-
npx agent-inspect list
|
|
362
|
-
```
|
|
363
|
-
|
|
364
|
-
Common filters:
|
|
365
|
-
|
|
366
|
-
```bash
|
|
367
|
-
npx agent-inspect list --status success
|
|
368
|
-
npx agent-inspect list --status error
|
|
369
|
-
npx agent-inspect list --status running
|
|
370
|
-
npx agent-inspect list --status unknown
|
|
371
|
-
npx agent-inspect list --name hotel
|
|
372
|
-
npx agent-inspect list --since 24h
|
|
373
|
-
npx agent-inspect list --json
|
|
374
|
-
```
|
|
375
|
-
|
|
376
|
-
View a run:
|
|
377
|
-
|
|
378
|
-
```bash
|
|
379
|
-
npx agent-inspect view run_abc123
|
|
380
|
-
```
|
|
381
|
-
|
|
382
|
-
Alternate view modes:
|
|
383
|
-
|
|
384
|
-
```bash
|
|
385
|
-
npx agent-inspect view run_abc123 --summary
|
|
386
|
-
npx agent-inspect view run_abc123 --metadata
|
|
387
|
-
npx agent-inspect view run_abc123 --errors-only
|
|
388
|
-
npx agent-inspect view run_abc123 --json --summary
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
Safely clean up old traces (recommended: start with `--dry-run`):
|
|
392
|
-
|
|
393
|
-
```bash
|
|
394
|
-
npx agent-inspect clean --older-than 7d --dry-run
|
|
395
|
-
npx agent-inspect clean --older-than 7d
|
|
396
|
-
npx agent-inspect clean --keep 100 --dry-run
|
|
397
|
-
npx agent-inspect clean --keep 100 --yes
|
|
398
|
-
npx agent-inspect clean --dir ./traces --older-than 7d --dry-run
|
|
399
|
-
```
|
|
400
|
-
|
|
401
|
-
Safety notes:
|
|
402
|
-
|
|
403
|
-
- `clean` **verifies each file** as an AgentInspect trace before deleting.
|
|
404
|
-
- Arbitrary JSONL files are **not deleted**.
|
|
405
|
-
- Malformed JSONL files are **not deleted**.
|
|
406
|
-
- Without `--dry-run`, `clean` requires confirmation unless `--yes` is provided.
|
|
407
|
-
- In non-interactive terminals, deletion requires `--yes`.
|
|
408
|
-
|
|
409
|
-
Inspect structured logs:
|
|
410
|
-
|
|
411
|
-
```bash
|
|
412
|
-
npx agent-inspect logs ./agent.log --format json
|
|
413
|
-
npx agent-inspect logs ./agent.log --format log4js
|
|
414
|
-
npx agent-inspect logs ./agent.log --format auto
|
|
415
|
-
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
416
|
-
npx agent-inspect logs ./agent.log --json
|
|
417
|
-
npx agent-inspect logs ./agent.log --summary
|
|
418
|
-
npx agent-inspect logs ./agent.log --warnings all
|
|
419
|
-
```
|
|
420
|
-
|
|
421
|
-
Log ingestion notes:
|
|
196
|
+
See [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) and [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
422
197
|
|
|
423
|
-
|
|
424
|
-
- log4js text logs are best-effort: only embedded **valid JSON payloads** are supported.
|
|
425
|
-
- JavaScript object-literal payloads are intentionally unsupported.
|
|
426
|
-
- No eval is used.
|
|
427
|
-
- Flat timeline is default (nesting only with explicit `parentId`).
|
|
428
|
-
- Confidence labels explain attribution.
|
|
429
|
-
- Redaction is applied to sensitive attributes (based on config).
|
|
198
|
+
### TUI viewer (`@agent-inspect/tui`)
|
|
430
199
|
|
|
431
|
-
|
|
200
|
+
Optional **Ink/React** package, installed separately. Use with an interactive terminal:
|
|
432
201
|
|
|
433
202
|
```bash
|
|
434
|
-
|
|
435
|
-
npx agent-inspect
|
|
436
|
-
npm run dev 2>&1 | npx agent-inspect tail --format log4js --config agent-inspect.logs.json
|
|
437
|
-
npx agent-inspect tail --file ./agent.log --format auto --once
|
|
438
|
-
npx agent-inspect tail --file ./agent.log --json --once
|
|
203
|
+
pnpm add agent-inspect @agent-inspect/tui
|
|
204
|
+
npx agent-inspect view <run-id> --tui
|
|
439
205
|
```
|
|
440
206
|
|
|
441
|
-
|
|
207
|
+
The TUI is available as a separate optional package; its programmatic API is experimental, while the CLI integration (`view --tui`) is the intended usage. Details: [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
442
208
|
|
|
443
|
-
|
|
444
|
-
npx agent-inspect list --dir ./traces
|
|
445
|
-
npx agent-inspect view run_abc123 --dir ./traces
|
|
446
|
-
```
|
|
447
|
-
|
|
448
|
-
By default, traces are stored under `~/.agent-inspect/runs`.
|
|
449
|
-
|
|
450
|
-
You can also set a default trace directory with:
|
|
209
|
+
## Examples and recipes
|
|
451
210
|
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
agent-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
```text
|
|
470
|
-
~/.agent-inspect/runs
|
|
471
|
-
```
|
|
211
|
+
| Example | Shows |
|
|
212
|
+
| ------- | ----- |
|
|
213
|
+
| [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md) | Fast install-and-try trace |
|
|
214
|
+
| [examples/01-basic](examples/01-basic) | `inspectRun` + `step` |
|
|
215
|
+
| [examples/02-nested-steps](examples/02-nested-steps) | Nested tree |
|
|
216
|
+
| [examples/03-parallel-steps](examples/03-parallel-steps) | Parallel siblings |
|
|
217
|
+
| [examples/04-error-handling](examples/04-error-handling) | Failed steps |
|
|
218
|
+
| [examples/05-observe-wrapper](examples/05-observe-wrapper) | `observe()` |
|
|
219
|
+
| [examples/06-log-to-tree](examples/06-log-to-tree) | `logs` / `tail` |
|
|
220
|
+
| [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) | LangChain callbacks |
|
|
221
|
+
| [examples/recipes/rag-pipeline](examples/recipes/rag-pipeline) | RAG-shaped flow |
|
|
222
|
+
| [examples/recipes/tool-failure-retry](examples/recipes/tool-failure-retry) | Tool failure + retry |
|
|
223
|
+
| [examples/recipes/multi-agent-handoff](examples/recipes/multi-agent-handoff) | Handoff |
|
|
224
|
+
| [examples/recipes/proactive-agent-logs](examples/recipes/proactive-agent-logs) | Structured logs |
|
|
225
|
+
| [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
|
|
226
|
+
| [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
|
|
472
227
|
|
|
473
|
-
|
|
228
|
+
**Recipes** are deterministic and require **no external services** by default. Index: [examples/README.md](examples/README.md), [examples/recipes/README.md](examples/recipes/README.md).
|
|
474
229
|
|
|
475
|
-
|
|
476
|
-
{"schemaVersion":"0.1","event":"step_started","name":"search-hotels","type":"logic"}
|
|
477
|
-
```
|
|
478
|
-
|
|
479
|
-
You can inspect traces with standard tools:
|
|
480
|
-
|
|
481
|
-
```bash
|
|
482
|
-
cat ~/.agent-inspect/runs/run_abc123.jsonl
|
|
483
|
-
cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
|
|
484
|
-
```
|
|
485
|
-
|
|
486
|
-
## Runnable examples
|
|
487
|
-
|
|
488
|
-
The repo includes runnable examples for manual tracing, log-to-tree, and the optional LangChain adapter:
|
|
489
|
-
|
|
490
|
-
- `examples/00-quickstart-demo` — minimal install-and-try demo
|
|
491
|
-
- `examples/01-basic` — `inspectRun()` + `step()`
|
|
492
|
-
- `examples/02-nested-steps` — nested execution tree hierarchy
|
|
493
|
-
- `examples/03-parallel-steps` — `Promise.all` sibling isolation
|
|
494
|
-
- `examples/04-error-handling` — failed steps and error traces
|
|
495
|
-
- `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
|
|
496
|
-
- `examples/06-log-to-tree` — structured log-to-tree example (`agent-inspect logs`, `tail`)
|
|
497
|
-
- `examples/08-langchain-adapter` — optional LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
|
|
498
|
-
|
|
499
|
-
Run one locally:
|
|
500
|
-
|
|
501
|
-
```bash
|
|
502
|
-
pnpm build
|
|
503
|
-
cd examples/01-basic
|
|
504
|
-
pnpm install
|
|
505
|
-
pnpm start
|
|
506
|
-
```
|
|
230
|
+
## Security and privacy posture
|
|
507
231
|
|
|
508
|
-
|
|
232
|
+
- **Local files by default** — no upload, no vendor sinks in core workflows.
|
|
233
|
+
- **No API keys** required for core tracing and CLI inspection.
|
|
234
|
+
- **Manual metadata** is user-controlled; traces and exports can contain sensitive data if you put it there.
|
|
235
|
+
- **Review exports** before sharing (especially with richer attribute flags).
|
|
509
236
|
|
|
510
|
-
|
|
511
|
-
node ../../packages/cli/dist/index.cjs list
|
|
512
|
-
node ../../packages/cli/dist/index.cjs view run_abc123
|
|
513
|
-
```
|
|
237
|
+
See [SECURITY.md](SECURITY.md).
|
|
514
238
|
|
|
515
|
-
|
|
239
|
+
## agent-inspect comparison
|
|
516
240
|
|
|
517
|
-
|
|
241
|
+
It can **complement** LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, and similar platforms — but it does **not** replace their production or eval workflows.
|
|
518
242
|
|
|
519
|
-
|
|
243
|
+
For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
520
244
|
|
|
521
|
-
|
|
245
|
+
## Documentation
|
|
522
246
|
|
|
523
|
-
-
|
|
524
|
-
-
|
|
525
|
-
-
|
|
526
|
-
-
|
|
527
|
-
-
|
|
528
|
-
-
|
|
529
|
-
-
|
|
530
|
-
-
|
|
531
|
-
-
|
|
532
|
-
-
|
|
533
|
-
-
|
|
534
|
-
-
|
|
247
|
+
- [Getting started](docs/GETTING-STARTED.md)
|
|
248
|
+
- [API stability & experimental surfaces](docs/API.md)
|
|
249
|
+
- [CLI reference](docs/CLI.md)
|
|
250
|
+
- [Schema (`schemaVersion: "0.1"`)](docs/SCHEMA.md)
|
|
251
|
+
- [Architecture (links to deeper design notes)](docs/ARCHITECTURE.md)
|
|
252
|
+
- [Logs & tail](docs/LOGS.md)
|
|
253
|
+
- [Log-to-tree quickstart](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
254
|
+
- [Exports](docs/EXPORTS.md)
|
|
255
|
+
- [Diff](docs/DIFF.md)
|
|
256
|
+
- [Adapters](docs/ADAPTERS.md)
|
|
257
|
+
- [Compare with other tools](docs/COMPARE.md)
|
|
258
|
+
- [Security](SECURITY.md)
|
|
259
|
+
- [Changelog](CHANGELOG.md)
|
|
260
|
+
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
261
|
+
- [Limitations](docs/LIMITATIONS.md)
|
|
262
|
+
- [Screenshot checklist (planned assets)](docs/SCREENSHOTS.md)
|
|
535
263
|
|
|
536
264
|
## Development
|
|
537
265
|
|
|
266
|
+
From a clone of this repo:
|
|
267
|
+
|
|
538
268
|
```bash
|
|
539
269
|
pnpm install
|
|
540
270
|
pnpm build
|
|
541
271
|
pnpm test
|
|
542
272
|
pnpm test:all
|
|
543
273
|
```
|
|
274
|
+
|
|
275
|
+
To run the CLI from source after a build: `node packages/cli/dist/index.cjs --help`.
|
package/package.json
CHANGED
package/docs/MIGRATION.md
DELETED
|
@@ -1,109 +0,0 @@
|
|
|
1
|
-
# Migration guide (to AgentInspect 1.0)
|
|
2
|
-
|
|
3
|
-
This guide summarizes how to move from early AgentInspect MVP usage to **AgentInspect 1.0**.
|
|
4
|
-
|
|
5
|
-
AgentInspect remains local-first: it does not introduce any network upload or vendor sink workflows.
|
|
6
|
-
|
|
7
|
-
## Scope
|
|
8
|
-
|
|
9
|
-
Covered:
|
|
10
|
-
|
|
11
|
-
- manual tracing (`inspectRun`, `step`, `observe`)
|
|
12
|
-
- trace directory behavior
|
|
13
|
-
- CLI commands (`list`, `view`, `clean`, `logs`, `tail`, `export`, `diff`)
|
|
14
|
-
- optional packages (`@agent-inspect/langchain`, `@agent-inspect/tui`)
|
|
15
|
-
- schema compatibility guarantees
|
|
16
|
-
|
|
17
|
-
Not covered:
|
|
18
|
-
|
|
19
|
-
- publish/version bump workflows (see `docs-local/RELEASE-CHECKLIST.md`)
|
|
20
|
-
- vendor sinks (not implemented)
|
|
21
|
-
- replay (not implemented)
|
|
22
|
-
- cost engine (not implemented)
|
|
23
|
-
|
|
24
|
-
## Manual tracing API
|
|
25
|
-
|
|
26
|
-
If you were using:
|
|
27
|
-
|
|
28
|
-
```ts
|
|
29
|
-
import { inspectRun, step } from "agent-inspect";
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
that remains the recommended stable path. AgentInspect 1.0 is specifically about keeping these entry points compatible.
|
|
33
|
-
|
|
34
|
-
### Event names and failure representation
|
|
35
|
-
|
|
36
|
-
Manual JSONL event names remain stable:
|
|
37
|
-
|
|
38
|
-
- `run_started`
|
|
39
|
-
- `run_completed`
|
|
40
|
-
- `step_started`
|
|
41
|
-
- `step_completed`
|
|
42
|
-
|
|
43
|
-
There is **no `step_failed` event**. Step failures are represented as:
|
|
44
|
-
|
|
45
|
-
- `step_completed` with `status: "error"`
|
|
46
|
-
|
|
47
|
-
Existing `schemaVersion: "0.1"` traces remain readable. No migration command is required.
|
|
48
|
-
|
|
49
|
-
## Trace directory behavior
|
|
50
|
-
|
|
51
|
-
- `AGENT_INSPECT_TRACE_DIR` is supported.
|
|
52
|
-
- When unset, AgentInspect uses its default local directory (see `docs/CLI.md` and `docs/API.md`).
|
|
53
|
-
- Trace files are JSONL and are not automatically rewritten.
|
|
54
|
-
|
|
55
|
-
## CLI changes and additive commands
|
|
56
|
-
|
|
57
|
-
Manual inspection commands (`list`, `view`) are stable and local-only.
|
|
58
|
-
|
|
59
|
-
The following commands are additive workflows that remain local-only:
|
|
60
|
-
|
|
61
|
-
- `logs`: parse structured logs into normalized trees
|
|
62
|
-
- `tail`: live-tail structured logs in the terminal
|
|
63
|
-
- `export`: export a manual trace as Markdown/HTML/OpenInference-compatible JSON/OTLP JSON (local-only)
|
|
64
|
-
- `diff`: compare two manual traces (local, read-only)
|
|
65
|
-
|
|
66
|
-
### `clean` is safety-critical
|
|
67
|
-
|
|
68
|
-
`clean` verifies traces before deletion using conservative heuristics. If a file cannot be verified as an AgentInspect trace, it is skipped.
|
|
69
|
-
|
|
70
|
-
## Logs and tail
|
|
71
|
-
|
|
72
|
-
- JSON logs are first-class.
|
|
73
|
-
- log4js parsing is best-effort.
|
|
74
|
-
- No JS object literal parsing or `eval`.
|
|
75
|
-
- Redaction is applied to log-derived attributes based on config/default rules.
|
|
76
|
-
|
|
77
|
-
## Export and share safety
|
|
78
|
-
|
|
79
|
-
- Exports are **generated locally** and do not upload anywhere.
|
|
80
|
-
- Exporters default to **redacted** and **bounded** attribute previews.
|
|
81
|
-
- Always review exports before sharing.
|
|
82
|
-
|
|
83
|
-
## Diff and compare
|
|
84
|
-
|
|
85
|
-
- Diff compares two existing local traces.
|
|
86
|
-
- Diff does not rerun agents, mutate trace files, or call an LLM.
|
|
87
|
-
|
|
88
|
-
## Optional LangChain adapter
|
|
89
|
-
|
|
90
|
-
`@agent-inspect/langchain` is optional and separate from core. It uses `@langchain/core` as a **peer dependency**.
|
|
91
|
-
|
|
92
|
-
## Optional TUI
|
|
93
|
-
|
|
94
|
-
`@agent-inspect/tui` is optional and separate from core. It contains `ink`/`react` dependencies so the main `agent-inspect` install remains lean.
|
|
95
|
-
|
|
96
|
-
## Breaking changes
|
|
97
|
-
|
|
98
|
-
This stabilization effort aims to avoid breaking changes. If a breaking change is ever required:
|
|
99
|
-
|
|
100
|
-
- it requires a major version
|
|
101
|
-
- it must preserve trace readability where possible
|
|
102
|
-
- it must be explicitly documented
|
|
103
|
-
|
|
104
|
-
## Known non-migrations
|
|
105
|
-
|
|
106
|
-
- No vendor sink migration exists because there are **no vendor sinks** in core.
|
|
107
|
-
- No replay migration exists because replay is **not implemented**.
|
|
108
|
-
- No cost engine migration exists because cost calculation is **not implemented**.
|
|
109
|
-
|