agent-inspect 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +161 -415
- package/docs/API.md +4 -2
- package/docs/CLI.md +1 -0
- package/docs/GETTING-STARTED.md +22 -0
- package/package.json +9 -4
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/index.cjs +24 -0
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +20 -2
- package/packages/core/dist/index.d.ts +20 -2
- package/packages/core/dist/index.mjs +23 -1
- package/packages/core/dist/index.mjs.map +1 -1
- package/docs/MIGRATION.md +0 -109
package/README.md
CHANGED
|
@@ -1,10 +1,22 @@
|
|
|
1
1
|
# agent-inspect
|
|
2
2
|
|
|
3
|
-
Local execution trees for TypeScript AI agents
|
|
3
|
+
**Local execution trees for TypeScript AI agents.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
agent-inspect helps you understand what happened inside an AI agent run — **locally**. It turns manual steps, tool calls, LLM calls, structured logs, failures, durations, and run metadata into **readable execution trees** you can inspect from the terminal.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it **before** a hosted observability platform, **alongside** one, or as the **local debugging layer** underneath enterprise observability.
|
|
8
|
+
|
|
9
|
+
The tool starts with **manual traces** and **existing structured logs**, and extends into **optional framework callbacks** and **standards-aligned local export** — without turning the core into a SaaS or a vendor pipeline.
|
|
10
|
+
|
|
11
|
+
**No account. No cloud upload. No dashboard required.**
|
|
12
|
+
|
|
13
|
+
## Why agent-inspect exists
|
|
14
|
+
|
|
15
|
+
AI agents are no longer single function calls. They plan, call tools, invoke LLMs, branch, retry, fail, and run work in parallel. **Console logs are flat**; reconstructing causality from a wall of lines is slow and error-prone.
|
|
16
|
+
|
|
17
|
+
**Hosted observability** is valuable in production, but it can be heavy for the **inner loop**: local runs, fast iteration, and debugging before anything reaches a collector or dashboard.
|
|
18
|
+
|
|
19
|
+
agent-inspect gives those runs **structure**: an **execution tree** you can read and diff on disk, with a **CLI-first** workflow and **no vendor lock-in**.
|
|
8
20
|
|
|
9
21
|
## Install
|
|
10
22
|
|
|
@@ -16,6 +28,12 @@ npm install agent-inspect
|
|
|
16
28
|
pnpm add agent-inspect
|
|
17
29
|
```
|
|
18
30
|
|
|
31
|
+
Verify the CLI is available:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
npx agent-inspect --help
|
|
35
|
+
```
|
|
36
|
+
|
|
19
37
|
## 60-second quickstart
|
|
20
38
|
|
|
21
39
|
Create `demo.mjs`:
|
|
@@ -29,25 +47,25 @@ await inspectRun(
|
|
|
29
47
|
"support-agent",
|
|
30
48
|
async () => {
|
|
31
49
|
const plan = await step("plan", async () => {
|
|
32
|
-
await delay(
|
|
33
|
-
return {
|
|
50
|
+
await delay(40);
|
|
51
|
+
return { intent: "refund-policy", needsPolicy: true };
|
|
34
52
|
});
|
|
35
53
|
|
|
36
|
-
const
|
|
37
|
-
await delay(
|
|
38
|
-
return
|
|
54
|
+
const policy = await step.tool("retrieve-policy", async () => {
|
|
55
|
+
await delay(60);
|
|
56
|
+
return { text: "Refunds are available within 30 days of purchase." };
|
|
39
57
|
});
|
|
40
58
|
|
|
41
|
-
return step.llm("answer", async () => {
|
|
42
|
-
await delay(
|
|
43
|
-
return `
|
|
59
|
+
return step.llm("generate-answer", async () => {
|
|
60
|
+
await delay(80);
|
|
61
|
+
return `Policy: ${policy.text} (intent: ${plan.intent})`;
|
|
44
62
|
});
|
|
45
63
|
},
|
|
46
64
|
{ traceDir: "./.agent-inspect" }
|
|
47
65
|
);
|
|
48
66
|
```
|
|
49
67
|
|
|
50
|
-
Run it, then inspect
|
|
68
|
+
Run it, then inspect the trace:
|
|
51
69
|
|
|
52
70
|
```bash
|
|
53
71
|
node demo.mjs
|
|
@@ -56,488 +74,216 @@ npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
|
56
74
|
npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
|
|
57
75
|
```
|
|
58
76
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
```text
|
|
62
|
-
Run run_abc123 (support-agent)
|
|
63
|
-
├─ ✔ plan (50ms)
|
|
64
|
-
├─ ✔ tool:search-docs (75ms)
|
|
65
|
-
└─ ✔ llm:answer (100ms)
|
|
66
|
-
|
|
67
|
-
Summary:
|
|
68
|
-
Steps: 3 (0 error)
|
|
69
|
-
Duration: 225ms
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
Want a runnable demo folder? See [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
73
|
-
|
|
74
|
-
## Why not just console.log?
|
|
75
|
-
|
|
76
|
-
Console logs are great for quick values, but they’re flat. AgentInspect gives you:
|
|
77
|
-
|
|
78
|
-
- run grouping and local trace files
|
|
79
|
-
- explicit step boundaries (including nesting)
|
|
80
|
-
- step types (`tool:*`, `llm:*`)
|
|
81
|
-
- status + duration summaries
|
|
82
|
-
- a CLI to list/view/export/diff runs
|
|
83
|
-
- log ingestion workflows (`logs`, `tail`) when you already have structured logs
|
|
84
|
-
|
|
85
|
-
## Inspect existing structured logs
|
|
77
|
+
Full flow:
|
|
86
78
|
|
|
87
79
|
```bash
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
--event-key event \
|
|
92
|
-
--timestamp-key timestamp
|
|
80
|
+
npm install agent-inspect
|
|
81
|
+
node demo.mjs
|
|
82
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
93
83
|
```
|
|
94
84
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
## When to use AgentInspect
|
|
98
|
-
|
|
99
|
-
Use AgentInspect when:
|
|
100
|
-
|
|
101
|
-
- you are building TypeScript/Node.js AI agents
|
|
102
|
-
- you want local debugging before a hosted observability setup
|
|
103
|
-
- console logs are too flat for multi-step execution
|
|
104
|
-
- you want to inspect tool calls, LLM calls, failures, and durations locally
|
|
105
|
-
- you want a lightweight CLI workflow with no account and no cloud upload
|
|
106
|
-
- you want to compare two local runs
|
|
107
|
-
- you want to turn structured logs into readable execution trees
|
|
108
|
-
|
|
109
|
-
## When not to use AgentInspect
|
|
110
|
-
|
|
111
|
-
Do not use AgentInspect as a replacement for:
|
|
112
|
-
|
|
113
|
-
- production monitoring or alerting
|
|
114
|
-
- hosted observability dashboards
|
|
115
|
-
- long-term trace storage
|
|
116
|
-
- eval dataset management
|
|
117
|
-
- prompt management
|
|
118
|
-
- cost analytics
|
|
119
|
-
- replay/fork execution
|
|
120
|
-
- vendor telemetry pipelines
|
|
85
|
+
**Simplified example output** (actual CLI formatting may differ slightly):
|
|
121
86
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
-
|
|
127
|
-
|
|
128
|
-
- no API keys required
|
|
129
|
-
- small root dependency footprint
|
|
130
|
-
- traces can include **user-provided metadata**; review exports before sharing
|
|
131
|
-
|
|
132
|
-
See `SECURITY.md`.
|
|
133
|
-
|
|
134
|
-
## Documentation
|
|
135
|
-
|
|
136
|
-
- **Getting started**: [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md)
|
|
137
|
-
- **API**: [docs/API.md](docs/API.md)
|
|
138
|
-
- **CLI**: [docs/CLI.md](docs/CLI.md)
|
|
139
|
-
- **Schema**: [docs/SCHEMA.md](docs/SCHEMA.md)
|
|
140
|
-
- **Logs**: [docs/LOGS.md](docs/LOGS.md) and [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
141
|
-
- **Exports**: [docs/EXPORTS.md](docs/EXPORTS.md)
|
|
142
|
-
- **Diff**: [docs/DIFF.md](docs/DIFF.md)
|
|
143
|
-
- **Adapters**: [docs/ADAPTERS.md](docs/ADAPTERS.md)
|
|
144
|
-
- **Compare with other tools**: [docs/COMPARE.md](docs/COMPARE.md)
|
|
145
|
-
- **Known issues**: [docs/KNOWN-ISSUES.md](docs/KNOWN-ISSUES.md)
|
|
146
|
-
- **Limitations**: [docs/LIMITATIONS.md](docs/LIMITATIONS.md)
|
|
87
|
+
```text
|
|
88
|
+
support-agent
|
|
89
|
+
✔ plan
|
|
90
|
+
✔ tool:retrieve-policy
|
|
91
|
+
✔ llm:generate-answer
|
|
92
|
+
```
|
|
147
93
|
|
|
148
|
-
|
|
94
|
+
A runnable copy lives in [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
149
95
|
|
|
150
|
-
|
|
96
|
+
**Env-gated tracing** (eval harnesses, CI): use `maybeInspectRun` and set `AGENT_INSPECT=1` when you want a trace — otherwise no files are written.
|
|
151
97
|
|
|
152
98
|
```ts
|
|
153
|
-
import {
|
|
99
|
+
import { maybeInspectRun } from "agent-inspect";
|
|
154
100
|
|
|
155
|
-
await
|
|
156
|
-
const plan = await step("plan", async () => ({ task: "research" }));
|
|
157
|
-
return step("act", async () => plan);
|
|
158
|
-
});
|
|
101
|
+
await maybeInspectRun("eval-case-42", async () => runAgent());
|
|
159
102
|
```
|
|
160
103
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
```ts
|
|
164
|
-
await step.llm("mock-gpt", async () => {
|
|
165
|
-
return planner.run();
|
|
166
|
-
});
|
|
167
|
-
|
|
168
|
-
await step.tool("searchHotels", async () => {
|
|
169
|
-
return searchHotels();
|
|
170
|
-
});
|
|
104
|
+
```bash
|
|
105
|
+
AGENT_INSPECT=1 node eval-runner.mjs
|
|
171
106
|
```
|
|
172
107
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
They do not import or call vendor SDKs.
|
|
176
|
-
|
|
177
|
-
## observe()
|
|
108
|
+
## What the trace shows
|
|
178
109
|
|
|
179
|
-
`
|
|
110
|
+
Each run produces a **JSONL** trace: `run_started` / `run_completed`, `step_started` / `step_completed`, with **nested steps**, **tool/LLM** types where you use `step.tool` / `step.llm`, and **durations** on completed steps. Failures are recorded on `step_completed` with `status: "error"` (there is no separate `step_failed` event). See [docs/SCHEMA.md](docs/SCHEMA.md).
|
|
180
111
|
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
```ts
|
|
184
|
-
import { observe } from "agent-inspect";
|
|
112
|
+
## Works with structured logs you already have
|
|
185
113
|
|
|
186
|
-
|
|
187
|
-
async run(input: string) {
|
|
188
|
-
return `ok: ${input}`;
|
|
189
|
-
}
|
|
190
|
-
}
|
|
114
|
+
Many production systems already emit **line-delimited JSON** or text logs with embedded JSON (e.g. via **pino**, **winston**, **log4js**, **NestJS** loggers, job runners, or custom event streams). agent-inspect can turn those into **local grouped timelines/trees** without wrapping every function.
|
|
191
115
|
|
|
192
|
-
|
|
193
|
-
|
|
116
|
+
```bash
|
|
117
|
+
npx agent-inspect logs ./agent.log \
|
|
118
|
+
--format json \
|
|
119
|
+
--run-id-key requestId \
|
|
120
|
+
--event-key event \
|
|
121
|
+
--timestamp-key timestamp
|
|
194
122
|
```
|
|
195
123
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
## Usage examples
|
|
199
|
-
|
|
200
|
-
### Example 1: Basic workflow
|
|
124
|
+
With a reusable ingest config:
|
|
201
125
|
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
const result = await inspectRun("hotel-booking", async () => {
|
|
206
|
-
const hotels = await step("search-hotels", async () => {
|
|
207
|
-
return ["Tokyo Grand Hotel", "Tokyo Central Inn"];
|
|
208
|
-
});
|
|
209
|
-
|
|
210
|
-
const availability = await step("check-availability", async () => {
|
|
211
|
-
return { hotel: hotels[0], rooms: 2 };
|
|
212
|
-
});
|
|
213
|
-
|
|
214
|
-
return step("finalize-booking", async () => {
|
|
215
|
-
return `confirmed:${availability.hotel}`;
|
|
216
|
-
});
|
|
217
|
-
});
|
|
218
|
-
|
|
219
|
-
console.log(result);
|
|
126
|
+
```bash
|
|
127
|
+
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
220
128
|
```
|
|
221
129
|
|
|
222
|
-
|
|
130
|
+
- **JSON logs** are first-class.
|
|
131
|
+
- **log4js-style** lines are **best-effort** when a recoverable JSON payload is present.
|
|
132
|
+
- **No `eval`**, no JavaScript object-literal parsing as a log interchange format.
|
|
133
|
+
- **Flat timeline by default**; nesting when parent relationships are explicit or configured.
|
|
134
|
+
- **Confidence labels** (`explicit`, `correlated`, `heuristic`, `unknown`) describe how attribution was inferred.
|
|
223
135
|
|
|
224
|
-
|
|
225
|
-
hotel-booking
|
|
226
|
-
✔ search-hotels
|
|
227
|
-
✔ check-availability
|
|
228
|
-
✔ finalize-booking
|
|
229
|
-
```
|
|
136
|
+
More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md).
|
|
230
137
|
|
|
231
|
-
|
|
138
|
+
## CLI at a glance
|
|
232
139
|
|
|
233
|
-
|
|
234
|
-
|
|
140
|
+
| Command | Use it for |
|
|
141
|
+
| -------- | ---------- |
|
|
142
|
+
| `list` | Find recent runs |
|
|
143
|
+
| `view` | Inspect one run as a tree |
|
|
144
|
+
| `clean` | Safely remove old trace files |
|
|
145
|
+
| `logs` | Turn existing structured logs into a local tree/timeline |
|
|
146
|
+
| `tail` | Watch structured logs while the app runs |
|
|
147
|
+
| `export` | Write Markdown / HTML / OpenInference-compatible JSON / OTLP JSON **locally** |
|
|
148
|
+
| `diff` | Compare two local runs (read-only) |
|
|
235
149
|
|
|
236
|
-
|
|
237
|
-
const plan = await step("plan-trip", async () => {
|
|
238
|
-
const draft = await step.llm("mock-gpt", async () => {
|
|
239
|
-
return "Plan: museum, dinner, evening walk.";
|
|
240
|
-
});
|
|
150
|
+
Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
241
151
|
|
|
242
|
-
|
|
243
|
-
return draft.replace("Plan: ", "").split(", ");
|
|
244
|
-
});
|
|
245
|
-
});
|
|
152
|
+
## Real-world workflows
|
|
246
153
|
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
154
|
+
- Debug a **failed tool call** or thrown error in a support or ops agent.
|
|
155
|
+
- See **which step dominated latency** in a multi-step planner or RAG pipeline.
|
|
156
|
+
- **Diff two runs** after a prompt, model, or routing change.
|
|
157
|
+
- Point **`logs`** / **`tail`** at existing job or service logs to get a **local execution view** without shipping data upstream.
|
|
158
|
+
- **Export** a run to Markdown for a PR, postmortem, or internal thread — then review before sharing.
|
|
159
|
+
- Keep traces **on disk** while still using enterprise observability elsewhere.
|
|
250
160
|
|
|
251
|
-
|
|
252
|
-
return { plan, hotel: hotels[0] };
|
|
253
|
-
});
|
|
254
|
-
});
|
|
255
|
-
```
|
|
161
|
+
## What v1.0 stabilizes
|
|
256
162
|
|
|
257
|
-
|
|
163
|
+
**agent-inspect 1.0** stabilizes the **local debugging foundation**:
|
|
258
164
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
✔ parse-plan
|
|
264
|
-
✔ tool:searchHotels
|
|
265
|
-
✔ finalize
|
|
266
|
-
```
|
|
165
|
+
- Instrument a run with `inspectRun` and `step`
|
|
166
|
+
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
167
|
+
- Inspect runs with **`list`** and **`view`**
|
|
168
|
+
- Safely remove old trace files with **`clean`**
|
|
267
169
|
|
|
268
|
-
|
|
170
|
+
**Stable APIs:** `inspectRun()`, `maybeInspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`.
|
|
269
171
|
|
|
270
|
-
|
|
271
|
-
import { inspectRun, step } from "agent-inspect";
|
|
172
|
+
Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` (or `true` / `yes` / `on` / `enabled`) to toggle tracing in eval or CI jobs — see [docs/API.md](docs/API.md).
|
|
272
173
|
|
|
273
|
-
|
|
274
|
-
await inspectRun("pricing-flow", async () => {
|
|
275
|
-
await step("load-catalog", async () => ["sku-a", "sku-b"]);
|
|
174
|
+
**Stable CLI workflows:** `agent-inspect list`, `agent-inspect view`, `agent-inspect clean`.
|
|
276
175
|
|
|
277
|
-
|
|
278
|
-
throw new Error("Pricing API timeout");
|
|
279
|
-
});
|
|
176
|
+
**Also included in 1.0** as local-first extensions:
|
|
280
177
|
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
```
|
|
289
|
-
|
|
290
|
-
agent-inspect records the failed step, writes it to the trace file, and still rethrows the original error.
|
|
178
|
+
- Structured log inspection: **`logs`**
|
|
179
|
+
- Live log tailing: **`tail`**
|
|
180
|
+
- Local exports: **`export`** (Markdown, HTML, OpenInference-compatible JSON, OTLP JSON — files only)
|
|
181
|
+
- Local run comparison: **`diff`**
|
|
182
|
+
- Optional **`@agent-inspect/langchain`** callback adapter
|
|
183
|
+
- Optional **`@agent-inspect/tui`** terminal viewer
|
|
184
|
+
- **Fixtures** and **recipes** for deterministic checks and adoption patterns
|
|
291
185
|
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
```ts
|
|
295
|
-
import { observe, step } from "agent-inspect";
|
|
296
|
-
|
|
297
|
-
class CustomerSupportAgent {
|
|
298
|
-
async run(question: string): Promise<string> {
|
|
299
|
-
const category = await step("triage-question", async () => {
|
|
300
|
-
return question.toLowerCase().includes("password")
|
|
301
|
-
? "account-access"
|
|
302
|
-
: "general";
|
|
303
|
-
});
|
|
186
|
+
**Honest boundaries:** programmatic log parsing, export, and diff APIs; LangChain and TUI programmatic surfaces; and OpenInference/OTLP JSON exports are **experimental or compatibility-oriented**. Nothing performs **vendor upload** by default.
|
|
304
187
|
|
|
305
|
-
|
|
306
|
-
return ["Reset your password from the login page."];
|
|
307
|
-
});
|
|
308
|
-
|
|
309
|
-
return step.llm("mock-support-model", async () => {
|
|
310
|
-
return `Category: ${category}. ${articles[0]}`;
|
|
311
|
-
});
|
|
312
|
-
}
|
|
313
|
-
}
|
|
314
|
-
|
|
315
|
-
const agent = observe(new CustomerSupportAgent());
|
|
316
|
-
await agent.run("How do I reset my password?");
|
|
317
|
-
```
|
|
188
|
+
## Optional packages
|
|
318
189
|
|
|
319
|
-
|
|
190
|
+
### LangChain callback adapter (`@agent-inspect/langchain`)
|
|
320
191
|
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
Install:
|
|
192
|
+
Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. The LangChain adapter is available in 1.0, but its programmatic API remains experimental and may evolve independently of the stable core tracing API.
|
|
324
193
|
|
|
325
194
|
```bash
|
|
326
195
|
pnpm add agent-inspect @agent-inspect/langchain @langchain/core
|
|
327
196
|
```
|
|
328
197
|
|
|
329
|
-
`@langchain/core` is a **peer dependency** of `@agent-inspect/langchain`. The adapter uses official LangChain.js **callbacks** only (extends `BaseCallbackHandler`): **no** monkey-patching, **no** `agent-inspect/auto`, **no** vendor observability sinks.
|
|
330
|
-
|
|
331
198
|
```ts
|
|
332
199
|
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
333
200
|
|
|
334
201
|
const callback = new AgentInspectCallback({
|
|
335
|
-
runName: "
|
|
202
|
+
runName: "my-run",
|
|
336
203
|
capture: "metadata-only",
|
|
337
204
|
});
|
|
338
205
|
|
|
339
|
-
await agent.invoke(input, {
|
|
340
|
-
callbacks: [callback],
|
|
341
|
-
});
|
|
342
|
-
|
|
206
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
343
207
|
const events = callback.getEvents();
|
|
344
208
|
```
|
|
345
209
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
- **Metadata-only** capture by default (model, tags, token usage when present, counts). **No** full prompt/output capture by default.
|
|
349
|
-
- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
|
|
350
|
-
- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
|
|
351
|
-
- **No** cost calculation; token fields are informational only.
|
|
352
|
-
- Events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into the manual JSONL trace format.
|
|
353
|
-
|
|
354
|
-
The LangChain adapter API remains **experimental** even though the core AgentInspect tracing API is stable in 1.0. See [examples/08-langchain-adapter](examples/08-langchain-adapter).
|
|
355
|
-
|
|
356
|
-
## CLI
|
|
357
|
-
|
|
358
|
-
List recent runs:
|
|
359
|
-
|
|
360
|
-
```bash
|
|
361
|
-
npx agent-inspect list
|
|
362
|
-
```
|
|
363
|
-
|
|
364
|
-
Common filters:
|
|
365
|
-
|
|
366
|
-
```bash
|
|
367
|
-
npx agent-inspect list --status success
|
|
368
|
-
npx agent-inspect list --status error
|
|
369
|
-
npx agent-inspect list --status running
|
|
370
|
-
npx agent-inspect list --status unknown
|
|
371
|
-
npx agent-inspect list --name hotel
|
|
372
|
-
npx agent-inspect list --since 24h
|
|
373
|
-
npx agent-inspect list --json
|
|
374
|
-
```
|
|
375
|
-
|
|
376
|
-
View a run:
|
|
377
|
-
|
|
378
|
-
```bash
|
|
379
|
-
npx agent-inspect view run_abc123
|
|
380
|
-
```
|
|
381
|
-
|
|
382
|
-
Alternate view modes:
|
|
383
|
-
|
|
384
|
-
```bash
|
|
385
|
-
npx agent-inspect view run_abc123 --summary
|
|
386
|
-
npx agent-inspect view run_abc123 --metadata
|
|
387
|
-
npx agent-inspect view run_abc123 --errors-only
|
|
388
|
-
npx agent-inspect view run_abc123 --json --summary
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
Safely clean up old traces (recommended: start with `--dry-run`):
|
|
392
|
-
|
|
393
|
-
```bash
|
|
394
|
-
npx agent-inspect clean --older-than 7d --dry-run
|
|
395
|
-
npx agent-inspect clean --older-than 7d
|
|
396
|
-
npx agent-inspect clean --keep 100 --dry-run
|
|
397
|
-
npx agent-inspect clean --keep 100 --yes
|
|
398
|
-
npx agent-inspect clean --dir ./traces --older-than 7d --dry-run
|
|
399
|
-
```
|
|
400
|
-
|
|
401
|
-
Safety notes:
|
|
402
|
-
|
|
403
|
-
- `clean` **verifies each file** as an AgentInspect trace before deleting.
|
|
404
|
-
- Arbitrary JSONL files are **not deleted**.
|
|
405
|
-
- Malformed JSONL files are **not deleted**.
|
|
406
|
-
- Without `--dry-run`, `clean` requires confirmation unless `--yes` is provided.
|
|
407
|
-
- In non-interactive terminals, deletion requires `--yes`.
|
|
408
|
-
|
|
409
|
-
Inspect structured logs:
|
|
410
|
-
|
|
411
|
-
```bash
|
|
412
|
-
npx agent-inspect logs ./agent.log --format json
|
|
413
|
-
npx agent-inspect logs ./agent.log --format log4js
|
|
414
|
-
npx agent-inspect logs ./agent.log --format auto
|
|
415
|
-
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
416
|
-
npx agent-inspect logs ./agent.log --json
|
|
417
|
-
npx agent-inspect logs ./agent.log --summary
|
|
418
|
-
npx agent-inspect logs ./agent.log --warnings all
|
|
419
|
-
```
|
|
420
|
-
|
|
421
|
-
Log ingestion notes:
|
|
210
|
+
See [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) and [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
422
211
|
|
|
423
|
-
|
|
424
|
-
- log4js text logs are best-effort: only embedded **valid JSON payloads** are supported.
|
|
425
|
-
- JavaScript object-literal payloads are intentionally unsupported.
|
|
426
|
-
- No eval is used.
|
|
427
|
-
- Flat timeline is default (nesting only with explicit `parentId`).
|
|
428
|
-
- Confidence labels explain attribution.
|
|
429
|
-
- Redaction is applied to sensitive attributes (based on config).
|
|
212
|
+
### TUI viewer (`@agent-inspect/tui`)
|
|
430
213
|
|
|
431
|
-
|
|
214
|
+
Optional **Ink/React** package, installed separately. Use with an interactive terminal:
|
|
432
215
|
|
|
433
216
|
```bash
|
|
434
|
-
|
|
435
|
-
npx agent-inspect
|
|
436
|
-
npm run dev 2>&1 | npx agent-inspect tail --format log4js --config agent-inspect.logs.json
|
|
437
|
-
npx agent-inspect tail --file ./agent.log --format auto --once
|
|
438
|
-
npx agent-inspect tail --file ./agent.log --json --once
|
|
217
|
+
pnpm add agent-inspect @agent-inspect/tui
|
|
218
|
+
npx agent-inspect view <run-id> --tui
|
|
439
219
|
```
|
|
440
220
|
|
|
441
|
-
|
|
221
|
+
The TUI is available as a separate optional package; its programmatic API is experimental, while the CLI integration (`view --tui`) is the intended usage. Details: [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
442
222
|
|
|
443
|
-
|
|
444
|
-
npx agent-inspect list --dir ./traces
|
|
445
|
-
npx agent-inspect view run_abc123 --dir ./traces
|
|
446
|
-
```
|
|
447
|
-
|
|
448
|
-
By default, traces are stored under `~/.agent-inspect/runs`.
|
|
449
|
-
|
|
450
|
-
You can also set a default trace directory with:
|
|
223
|
+
## Examples and recipes
|
|
451
224
|
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
agent-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
```text
|
|
470
|
-
~/.agent-inspect/runs
|
|
471
|
-
```
|
|
225
|
+
| Example | Shows |
|
|
226
|
+
| ------- | ----- |
|
|
227
|
+
| [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md) | Fast install-and-try trace |
|
|
228
|
+
| [examples/01-basic](examples/01-basic) | `inspectRun` + `step` |
|
|
229
|
+
| [examples/02-nested-steps](examples/02-nested-steps) | Nested tree |
|
|
230
|
+
| [examples/03-parallel-steps](examples/03-parallel-steps) | Parallel siblings |
|
|
231
|
+
| [examples/04-error-handling](examples/04-error-handling) | Failed steps |
|
|
232
|
+
| [examples/05-observe-wrapper](examples/05-observe-wrapper) | `observe()` |
|
|
233
|
+
| [examples/06-log-to-tree](examples/06-log-to-tree) | `logs` / `tail` |
|
|
234
|
+
| [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) | LangChain callbacks |
|
|
235
|
+
| [examples/recipes/rag-pipeline](examples/recipes/rag-pipeline) | RAG-shaped flow |
|
|
236
|
+
| [examples/recipes/tool-failure-retry](examples/recipes/tool-failure-retry) | Tool failure + retry |
|
|
237
|
+
| [examples/recipes/multi-agent-handoff](examples/recipes/multi-agent-handoff) | Handoff |
|
|
238
|
+
| [examples/recipes/proactive-agent-logs](examples/recipes/proactive-agent-logs) | Structured logs |
|
|
239
|
+
| [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
|
|
240
|
+
| [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
|
|
472
241
|
|
|
473
|
-
|
|
242
|
+
**Recipes** are deterministic and require **no external services** by default. Index: [examples/README.md](examples/README.md), [examples/recipes/README.md](examples/recipes/README.md).
|
|
474
243
|
|
|
475
|
-
|
|
476
|
-
{"schemaVersion":"0.1","event":"step_started","name":"search-hotels","type":"logic"}
|
|
477
|
-
```
|
|
478
|
-
|
|
479
|
-
You can inspect traces with standard tools:
|
|
480
|
-
|
|
481
|
-
```bash
|
|
482
|
-
cat ~/.agent-inspect/runs/run_abc123.jsonl
|
|
483
|
-
cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
|
|
484
|
-
```
|
|
485
|
-
|
|
486
|
-
## Runnable examples
|
|
487
|
-
|
|
488
|
-
The repo includes runnable examples for manual tracing, log-to-tree, and the optional LangChain adapter:
|
|
489
|
-
|
|
490
|
-
- `examples/00-quickstart-demo` — minimal install-and-try demo
|
|
491
|
-
- `examples/01-basic` — `inspectRun()` + `step()`
|
|
492
|
-
- `examples/02-nested-steps` — nested execution tree hierarchy
|
|
493
|
-
- `examples/03-parallel-steps` — `Promise.all` sibling isolation
|
|
494
|
-
- `examples/04-error-handling` — failed steps and error traces
|
|
495
|
-
- `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
|
|
496
|
-
- `examples/06-log-to-tree` — structured log-to-tree example (`agent-inspect logs`, `tail`)
|
|
497
|
-
- `examples/08-langchain-adapter` — optional LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
|
|
498
|
-
|
|
499
|
-
Run one locally:
|
|
500
|
-
|
|
501
|
-
```bash
|
|
502
|
-
pnpm build
|
|
503
|
-
cd examples/01-basic
|
|
504
|
-
pnpm install
|
|
505
|
-
pnpm start
|
|
506
|
-
```
|
|
244
|
+
## Security and privacy posture
|
|
507
245
|
|
|
508
|
-
|
|
246
|
+
- **Local files by default** — no upload, no vendor sinks in core workflows.
|
|
247
|
+
- **No API keys** required for core tracing and CLI inspection.
|
|
248
|
+
- **Manual metadata** is user-controlled; traces and exports can contain sensitive data if you put it there.
|
|
249
|
+
- **Review exports** before sharing (especially with richer attribute flags).
|
|
509
250
|
|
|
510
|
-
|
|
511
|
-
node ../../packages/cli/dist/index.cjs list
|
|
512
|
-
node ../../packages/cli/dist/index.cjs view run_abc123
|
|
513
|
-
```
|
|
251
|
+
See [SECURITY.md](SECURITY.md).
|
|
514
252
|
|
|
515
|
-
|
|
253
|
+
## agent-inspect comparison
|
|
516
254
|
|
|
517
|
-
|
|
255
|
+
It can **complement** LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, and similar platforms — but it does **not** replace their production or eval workflows.
|
|
518
256
|
|
|
519
|
-
|
|
257
|
+
For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
520
258
|
|
|
521
|
-
|
|
259
|
+
## Documentation
|
|
522
260
|
|
|
523
|
-
-
|
|
524
|
-
-
|
|
525
|
-
-
|
|
526
|
-
-
|
|
527
|
-
-
|
|
528
|
-
-
|
|
529
|
-
-
|
|
530
|
-
-
|
|
531
|
-
-
|
|
532
|
-
-
|
|
533
|
-
-
|
|
534
|
-
-
|
|
261
|
+
- [Getting started](docs/GETTING-STARTED.md)
|
|
262
|
+
- [API stability & experimental surfaces](docs/API.md)
|
|
263
|
+
- [CLI reference](docs/CLI.md)
|
|
264
|
+
- [Schema (`schemaVersion: "0.1"`)](docs/SCHEMA.md)
|
|
265
|
+
- [Architecture (links to deeper design notes)](docs/ARCHITECTURE.md)
|
|
266
|
+
- [Logs & tail](docs/LOGS.md)
|
|
267
|
+
- [Log-to-tree quickstart](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
268
|
+
- [Exports](docs/EXPORTS.md)
|
|
269
|
+
- [Diff](docs/DIFF.md)
|
|
270
|
+
- [Adapters](docs/ADAPTERS.md)
|
|
271
|
+
- [Compare with other tools](docs/COMPARE.md)
|
|
272
|
+
- [Security](SECURITY.md)
|
|
273
|
+
- [Changelog](CHANGELOG.md)
|
|
274
|
+
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
275
|
+
- [Limitations](docs/LIMITATIONS.md)
|
|
276
|
+
- [Screenshot checklist (planned assets)](docs/SCREENSHOTS.md)
|
|
535
277
|
|
|
536
278
|
## Development
|
|
537
279
|
|
|
280
|
+
From a clone of this repo:
|
|
281
|
+
|
|
538
282
|
```bash
|
|
539
283
|
pnpm install
|
|
540
284
|
pnpm build
|
|
541
285
|
pnpm test
|
|
542
286
|
pnpm test:all
|
|
543
287
|
```
|
|
288
|
+
|
|
289
|
+
To run the CLI from source after a build: `node packages/cli/dist/index.cjs --help`.
|