agent-inspect 1.0.0 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +105 -0
- package/README.md +168 -418
- package/SECURITY.md +77 -0
- package/docs/ADAPTERS.md +15 -0
- package/docs/API.md +152 -0
- package/docs/ARCHITECTURE.md +13 -0
- package/docs/CLI.md +212 -0
- package/docs/COMPARE.md +69 -0
- package/docs/DIFF.md +11 -0
- package/docs/EXPORTS.md +12 -0
- package/docs/GETTING-STARTED.md +128 -0
- package/docs/KNOWN-ISSUES.md +34 -0
- package/docs/LIMITATIONS.md +32 -0
- package/docs/LOG-TO-TREE-QUICKSTART.md +54 -0
- package/docs/LOGS.md +13 -0
- package/docs/SCHEMA.md +199 -0
- package/docs/SCREENSHOTS.md +14 -0
- package/package.json +36 -13
- package/packages/cli/dist/index.cjs +1 -1
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
package/README.md
CHANGED
|
@@ -1,525 +1,275 @@
|
|
|
1
1
|
# agent-inspect
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Local execution trees for TypeScript AI agents.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
agent-inspect helps you understand what happened inside an AI agent run — **locally**. It turns manual steps, tool calls, LLM calls, structured logs, failures, durations, and run metadata into **readable execution trees** you can inspect from the terminal.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it **before** a hosted observability platform, **alongside** one, or as the **local debugging layer** underneath enterprise observability.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
The tool starts with **manual traces** and **existing structured logs**, and extends into **optional framework callbacks** and **standards-aligned local export** — without turning the core into a SaaS or a vendor pipeline.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
**No account. No cloud upload. No dashboard required.**
|
|
12
12
|
|
|
13
|
-
##
|
|
13
|
+
## Why agent-inspect exists
|
|
14
14
|
|
|
15
|
-
|
|
16
|
-
- Nested `step()` support with parent-child relationships
|
|
17
|
-
- `step.llm()` and `step.tool()` helpers for agent-aware traces
|
|
18
|
-
- Local JSONL trace files
|
|
19
|
-
- Real-time terminal output while the agent runs
|
|
20
|
-
- CLI commands to inspect previous runs
|
|
21
|
-
- No accounts, API keys, dashboards, or cloud ingestion
|
|
15
|
+
AI agents are no longer single function calls. They plan, call tools, invoke LLMs, branch, retry, fail, and run work in parallel. **Console logs are flat**; reconstructing causality from a wall of lines is slow and error-prone.
|
|
22
16
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
npm install agent-inspect
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
## Documentation (v1.0 stabilization)
|
|
17
|
+
**Hosted observability** is valuable in production, but it can be heavy for the **inner loop**: local runs, fast iteration, and debugging before anything reaches a collector or dashboard.
|
|
30
18
|
|
|
31
|
-
-
|
|
32
|
-
- [API reference](docs/API.md)
|
|
33
|
-
- [CLI reference](docs/CLI.md)
|
|
34
|
-
- [Schema reference](docs/SCHEMA.md)
|
|
35
|
-
- [Security policy](SECURITY.md)
|
|
36
|
-
- [Migration guide](docs/MIGRATION.md)
|
|
37
|
-
- [Release checklist](docs/RELEASE-CHECKLIST.md)
|
|
38
|
-
- [Changelog](CHANGELOG.md)
|
|
39
|
-
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
40
|
-
- [Limitations](docs/LIMITATIONS.md)
|
|
41
|
-
- [V1 readiness checklist (non-binding)](docs/V1-READINESS-CHECKLIST.md)
|
|
19
|
+
agent-inspect gives those runs **structure**: an **execution tree** you can read and diff on disk, with a **CLI-first** workflow and **no vendor lock-in**.
|
|
42
20
|
|
|
43
|
-
##
|
|
44
|
-
|
|
45
|
-
Run a traced workflow, then inspect it with the CLI.
|
|
46
|
-
|
|
47
|
-
```ts
|
|
48
|
-
import { inspectRun, step } from "agent-inspect";
|
|
49
|
-
|
|
50
|
-
await inspectRun("hello-agent", async () => {
|
|
51
|
-
const plan = await step("plan", async () => "search hotels");
|
|
52
|
-
return step("finalize", async () => ({ plan, status: "done" }));
|
|
53
|
-
});
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
```bash
|
|
57
|
-
npx agent-inspect list
|
|
58
|
-
npx agent-inspect view run_abc123
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
Replace `run_abc123` with the run id printed by `agent-inspect list`.
|
|
62
|
-
|
|
63
|
-
### Optional TUI viewer
|
|
64
|
-
|
|
65
|
-
The core `agent-inspect` package stays lightweight and does not bundle Ink or React. For a keyboard-driven terminal UI over existing traces, install the optional package:
|
|
21
|
+
## Install
|
|
66
22
|
|
|
67
23
|
```bash
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
npx agent-inspect view run_abc123 --tui
|
|
24
|
+
npm install agent-inspect
|
|
71
25
|
```
|
|
72
26
|
|
|
73
|
-
The plain CLI remains the default. `--tui` requires an interactive terminal; for scripts or CI, use `agent-inspect view` or `agent-inspect view --json`. There is no live tail TUI yet.
|
|
74
|
-
|
|
75
|
-
### Export traces
|
|
76
|
-
|
|
77
|
-
Export existing manual JSONL traces locally — **no upload**, **no vendor SDKs**. Markdown is handy for PRs and issues; HTML is a single offline file. OpenInference export is **OpenInference-compatible JSON** (not a guarantee for every backend). OTLP JSON uses **OTel GenAI-aligned attributes** where applicable and is **experimental** until validated against a specific collector.
|
|
78
|
-
|
|
79
27
|
```bash
|
|
80
|
-
|
|
81
|
-
npx agent-inspect export run_abc123 --format html -o run.html
|
|
82
|
-
npx agent-inspect export run_abc123 --format openinference -o trace.openinference.json
|
|
83
|
-
npx agent-inspect export run_abc123 --format otlp-json -o trace.otlp.json
|
|
84
|
-
npx agent-inspect export run_abc123 --format openinference --validate
|
|
28
|
+
pnpm add agent-inspect
|
|
85
29
|
```
|
|
86
30
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
### Compare runs
|
|
90
|
-
|
|
91
|
-
Diff is **local** and **read-only**: it compares two existing AgentInspect JSONL traces and does **not** rerun agents, mutate trace files, or write output traces. It does **not** claim semantic equivalence and does **not** call an LLM.
|
|
92
|
-
|
|
93
|
-
Finding differences does **not** change the exit code by default (exit code `1` is reserved for command errors such as a missing run).
|
|
31
|
+
Verify the CLI is available:
|
|
94
32
|
|
|
95
33
|
```bash
|
|
96
|
-
npx agent-inspect
|
|
97
|
-
npx agent-inspect diff run_a run_b --json
|
|
98
|
-
npx agent-inspect diff run_a run_b --ignore-duration
|
|
99
|
-
npx agent-inspect diff run_a run_b --duration-threshold 500ms
|
|
100
|
-
npx agent-inspect diff run_a run_b --focus errors
|
|
101
|
-
npx agent-inspect diff run_a run_b --check structure
|
|
34
|
+
npx agent-inspect --help
|
|
102
35
|
```
|
|
103
36
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
### Fixtures and hardening (v0.9)
|
|
107
|
-
|
|
108
|
-
**v0.9** adds canonical [**fixtures/**](fixtures/README.md), validation (`pnpm fixtures:check`), **recipe examples** under [**examples/recipes/**](examples/recipes/README.md) (`pnpm recipes:check`), and docs aimed at adoption—not new tracing features. Recipes use mocks only and require no API keys or external services by default. Good starting points: **rag-pipeline**, **tool-failure-retry**, **proactive-agent-logs**. See [**Known issues**](docs/KNOWN-ISSUES.md), [**Limitations**](docs/LIMITATIONS.md), and the non-binding [**v1 readiness checklist**](docs/V1-READINESS-CHECKLIST.md).
|
|
37
|
+
## 60-second quickstart
|
|
109
38
|
|
|
110
|
-
|
|
39
|
+
Create `demo.mjs`:
|
|
111
40
|
|
|
112
|
-
```
|
|
41
|
+
```js
|
|
113
42
|
import { inspectRun, step } from "agent-inspect";
|
|
114
43
|
|
|
115
|
-
|
|
116
|
-
const plan = await step("plan", async () => ({ task: "research" }));
|
|
117
|
-
return step("act", async () => plan);
|
|
118
|
-
});
|
|
119
|
-
```
|
|
44
|
+
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
120
45
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
await step
|
|
125
|
-
|
|
126
|
-
}
|
|
127
|
-
|
|
128
|
-
await step.tool("searchHotels", async () => {
|
|
129
|
-
return searchHotels();
|
|
130
|
-
});
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
Helpers only label steps in the trace.
|
|
134
|
-
|
|
135
|
-
They do not import or call vendor SDKs.
|
|
136
|
-
|
|
137
|
-
## observe()
|
|
138
|
-
|
|
139
|
-
`observe()` wraps top-level `run`, `execute`, and `invoke`.
|
|
140
|
-
|
|
141
|
-
For internal detail, add manual `step()` calls inside the agent.
|
|
142
|
-
|
|
143
|
-
```ts
|
|
144
|
-
import { observe } from "agent-inspect";
|
|
145
|
-
|
|
146
|
-
class MyAgent {
|
|
147
|
-
async run(input: string) {
|
|
148
|
-
return `ok: ${input}`;
|
|
149
|
-
}
|
|
150
|
-
}
|
|
151
|
-
|
|
152
|
-
const agent = observe(new MyAgent());
|
|
153
|
-
await agent.run("hello");
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
See [examples/05-observe-wrapper](examples/05-observe-wrapper) for a top-level observed run with internal `step()`, `step.tool()`, and `step.llm()` calls.
|
|
157
|
-
|
|
158
|
-
## Usage examples
|
|
159
|
-
|
|
160
|
-
### Example 1: Basic workflow
|
|
161
|
-
|
|
162
|
-
```ts
|
|
163
|
-
import { inspectRun, step } from "agent-inspect";
|
|
164
|
-
|
|
165
|
-
const result = await inspectRun("hotel-booking", async () => {
|
|
166
|
-
const hotels = await step("search-hotels", async () => {
|
|
167
|
-
return ["Tokyo Grand Hotel", "Tokyo Central Inn"];
|
|
168
|
-
});
|
|
169
|
-
|
|
170
|
-
const availability = await step("check-availability", async () => {
|
|
171
|
-
return { hotel: hotels[0], rooms: 2 };
|
|
172
|
-
});
|
|
173
|
-
|
|
174
|
-
return step("finalize-booking", async () => {
|
|
175
|
-
return `confirmed:${availability.hotel}`;
|
|
176
|
-
});
|
|
177
|
-
});
|
|
178
|
-
|
|
179
|
-
console.log(result);
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
Expected tree:
|
|
183
|
-
|
|
184
|
-
```text
|
|
185
|
-
hotel-booking
|
|
186
|
-
✔ search-hotels
|
|
187
|
-
✔ check-availability
|
|
188
|
-
✔ finalize-booking
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
### Example 2: Nested LLM and tool steps
|
|
192
|
-
|
|
193
|
-
```ts
|
|
194
|
-
import { inspectRun, step } from "agent-inspect";
|
|
195
|
-
|
|
196
|
-
await inspectRun("trip-planner", async () => {
|
|
197
|
-
const plan = await step("plan-trip", async () => {
|
|
198
|
-
const draft = await step.llm("mock-gpt", async () => {
|
|
199
|
-
return "Plan: museum, dinner, evening walk.";
|
|
46
|
+
await inspectRun(
|
|
47
|
+
"support-agent",
|
|
48
|
+
async () => {
|
|
49
|
+
const plan = await step("plan", async () => {
|
|
50
|
+
await delay(40);
|
|
51
|
+
return { intent: "refund-policy", needsPolicy: true };
|
|
200
52
|
});
|
|
201
53
|
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
});
|
|
206
|
-
|
|
207
|
-
const hotels = await step.tool("searchHotels", async () => {
|
|
208
|
-
return [{ id: "h1", city: "Kyoto" }];
|
|
209
|
-
});
|
|
210
|
-
|
|
211
|
-
return step("finalize", async () => {
|
|
212
|
-
return { plan, hotel: hotels[0] };
|
|
213
|
-
});
|
|
214
|
-
});
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
Expected tree:
|
|
218
|
-
|
|
219
|
-
```text
|
|
220
|
-
trip-planner
|
|
221
|
-
✔ plan-trip
|
|
222
|
-
✔ llm:mock-gpt
|
|
223
|
-
✔ parse-plan
|
|
224
|
-
✔ tool:searchHotels
|
|
225
|
-
✔ finalize
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
### Example 3: Error handling
|
|
229
|
-
|
|
230
|
-
```ts
|
|
231
|
-
import { inspectRun, step } from "agent-inspect";
|
|
232
|
-
|
|
233
|
-
try {
|
|
234
|
-
await inspectRun("pricing-flow", async () => {
|
|
235
|
-
await step("load-catalog", async () => ["sku-a", "sku-b"]);
|
|
236
|
-
|
|
237
|
-
await step("fetch-dynamic-pricing", async () => {
|
|
238
|
-
throw new Error("Pricing API timeout");
|
|
54
|
+
const policy = await step.tool("retrieve-policy", async () => {
|
|
55
|
+
await delay(60);
|
|
56
|
+
return { text: "Refunds are available within 30 days of purchase." };
|
|
239
57
|
});
|
|
240
58
|
|
|
241
|
-
|
|
242
|
-
|
|
59
|
+
return step.llm("generate-answer", async () => {
|
|
60
|
+
await delay(80);
|
|
61
|
+
return `Policy: ${policy.text} (intent: ${plan.intent})`;
|
|
243
62
|
});
|
|
244
|
-
}
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
}
|
|
63
|
+
},
|
|
64
|
+
{ traceDir: "./.agent-inspect" }
|
|
65
|
+
);
|
|
248
66
|
```
|
|
249
67
|
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
### Example 4: `observe()` wrapper
|
|
68
|
+
Run it, then inspect the trace:
|
|
253
69
|
|
|
254
|
-
```
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
const category = await step("triage-question", async () => {
|
|
260
|
-
return question.toLowerCase().includes("password")
|
|
261
|
-
? "account-access"
|
|
262
|
-
: "general";
|
|
263
|
-
});
|
|
264
|
-
|
|
265
|
-
const articles = await step.tool("retrieveArticles", async () => {
|
|
266
|
-
return ["Reset your password from the login page."];
|
|
267
|
-
});
|
|
268
|
-
|
|
269
|
-
return step.llm("mock-support-model", async () => {
|
|
270
|
-
return `Category: ${category}. ${articles[0]}`;
|
|
271
|
-
});
|
|
272
|
-
}
|
|
273
|
-
}
|
|
274
|
-
|
|
275
|
-
const agent = observe(new CustomerSupportAgent());
|
|
276
|
-
await agent.run("How do I reset my password?");
|
|
70
|
+
```bash
|
|
71
|
+
node demo.mjs
|
|
72
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
73
|
+
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
74
|
+
npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
|
|
277
75
|
```
|
|
278
76
|
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
## LangChain adapter (v0.5, experimental)
|
|
282
|
-
|
|
283
|
-
Install:
|
|
77
|
+
Full flow:
|
|
284
78
|
|
|
285
79
|
```bash
|
|
286
|
-
|
|
80
|
+
npm install agent-inspect
|
|
81
|
+
node demo.mjs
|
|
82
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
287
83
|
```
|
|
288
84
|
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
```ts
|
|
292
|
-
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
293
|
-
|
|
294
|
-
const callback = new AgentInspectCallback({
|
|
295
|
-
runName: "support-agent-eval",
|
|
296
|
-
capture: "metadata-only",
|
|
297
|
-
});
|
|
298
|
-
|
|
299
|
-
await agent.invoke(input, {
|
|
300
|
-
callbacks: [callback],
|
|
301
|
-
});
|
|
85
|
+
**Simplified example output** (actual CLI formatting may differ slightly):
|
|
302
86
|
|
|
303
|
-
|
|
87
|
+
```text
|
|
88
|
+
support-agent
|
|
89
|
+
✔ plan
|
|
90
|
+
✔ tool:retrieve-policy
|
|
91
|
+
✔ llm:generate-answer
|
|
304
92
|
```
|
|
305
93
|
|
|
306
|
-
|
|
94
|
+
A runnable copy lives in [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
307
95
|
|
|
308
|
-
|
|
309
|
-
- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
|
|
310
|
-
- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
|
|
311
|
-
- **No** cost calculation; token fields are informational only.
|
|
312
|
-
- In this pass, events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into v0.1 JSONL manual traces.
|
|
96
|
+
## What the trace shows
|
|
313
97
|
|
|
314
|
-
|
|
98
|
+
Each run produces a **JSONL** trace: `run_started` / `run_completed`, `step_started` / `step_completed`, with **nested steps**, **tool/LLM** types where you use `step.tool` / `step.llm`, and **durations** on completed steps. Failures are recorded on `step_completed` with `status: "error"` (there is no separate `step_failed` event). See [docs/SCHEMA.md](docs/SCHEMA.md).
|
|
315
99
|
|
|
316
|
-
##
|
|
100
|
+
## Works with structured logs you already have
|
|
317
101
|
|
|
318
|
-
|
|
102
|
+
Many production systems already emit **line-delimited JSON** or text logs with embedded JSON (e.g. via **pino**, **winston**, **log4js**, **NestJS** loggers, job runners, or custom event streams). agent-inspect can turn those into **local grouped timelines/trees** without wrapping every function.
|
|
319
103
|
|
|
320
104
|
```bash
|
|
321
|
-
npx agent-inspect
|
|
105
|
+
npx agent-inspect logs ./agent.log \
|
|
106
|
+
--format json \
|
|
107
|
+
--run-id-key requestId \
|
|
108
|
+
--event-key event \
|
|
109
|
+
--timestamp-key timestamp
|
|
322
110
|
```
|
|
323
111
|
|
|
324
|
-
|
|
112
|
+
With a reusable ingest config:
|
|
325
113
|
|
|
326
114
|
```bash
|
|
327
|
-
npx agent-inspect
|
|
328
|
-
npx agent-inspect list --status error
|
|
329
|
-
npx agent-inspect list --status running
|
|
330
|
-
npx agent-inspect list --status unknown
|
|
331
|
-
npx agent-inspect list --name hotel
|
|
332
|
-
npx agent-inspect list --since 24h
|
|
333
|
-
npx agent-inspect list --json
|
|
115
|
+
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
334
116
|
```
|
|
335
117
|
|
|
336
|
-
|
|
118
|
+
- **JSON logs** are first-class.
|
|
119
|
+
- **log4js-style** lines are **best-effort** when a recoverable JSON payload is present.
|
|
120
|
+
- **No `eval`**, no JavaScript object-literal parsing as a log interchange format.
|
|
121
|
+
- **Flat timeline by default**; nesting when parent relationships are explicit or configured.
|
|
122
|
+
- **Confidence labels** (`explicit`, `correlated`, `heuristic`, `unknown`) describe how attribution was inferred.
|
|
337
123
|
|
|
338
|
-
|
|
339
|
-
npx agent-inspect view run_abc123
|
|
340
|
-
```
|
|
124
|
+
More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md).
|
|
341
125
|
|
|
342
|
-
|
|
126
|
+
## CLI at a glance
|
|
343
127
|
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
128
|
+
| Command | Use it for |
|
|
129
|
+
| -------- | ---------- |
|
|
130
|
+
| `list` | Find recent runs |
|
|
131
|
+
| `view` | Inspect one run as a tree |
|
|
132
|
+
| `clean` | Safely remove old trace files |
|
|
133
|
+
| `logs` | Turn existing structured logs into a local tree/timeline |
|
|
134
|
+
| `tail` | Watch structured logs while the app runs |
|
|
135
|
+
| `export` | Write Markdown / HTML / OpenInference-compatible JSON / OTLP JSON **locally** |
|
|
136
|
+
| `diff` | Compare two local runs (read-only) |
|
|
350
137
|
|
|
351
|
-
|
|
138
|
+
Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
352
139
|
|
|
353
|
-
|
|
354
|
-
npx agent-inspect clean --older-than 7d --dry-run
|
|
355
|
-
npx agent-inspect clean --older-than 7d
|
|
356
|
-
npx agent-inspect clean --keep 100 --dry-run
|
|
357
|
-
npx agent-inspect clean --keep 100 --yes
|
|
358
|
-
npx agent-inspect clean --dir ./traces --older-than 7d --dry-run
|
|
359
|
-
```
|
|
140
|
+
## Real-world workflows
|
|
360
141
|
|
|
361
|
-
|
|
142
|
+
- Debug a **failed tool call** or thrown error in a support or ops agent.
|
|
143
|
+
- See **which step dominated latency** in a multi-step planner or RAG pipeline.
|
|
144
|
+
- **Diff two runs** after a prompt, model, or routing change.
|
|
145
|
+
- Point **`logs`** / **`tail`** at existing job or service logs to get a **local execution view** without shipping data upstream.
|
|
146
|
+
- **Export** a run to Markdown for a PR, postmortem, or internal thread — then review before sharing.
|
|
147
|
+
- Keep traces **on disk** while still using enterprise observability elsewhere.
|
|
362
148
|
|
|
363
|
-
|
|
364
|
-
- Arbitrary JSONL files are **not deleted**.
|
|
365
|
-
- Malformed JSONL files are **not deleted**.
|
|
366
|
-
- Without `--dry-run`, `clean` requires confirmation unless `--yes` is provided.
|
|
367
|
-
- In non-interactive terminals, deletion requires `--yes`.
|
|
149
|
+
## What v1.0 stabilizes
|
|
368
150
|
|
|
369
|
-
|
|
151
|
+
**agent-inspect 1.0** stabilizes the **local debugging foundation**:
|
|
370
152
|
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
|
|
376
|
-
npx agent-inspect logs ./agent.log --json
|
|
377
|
-
npx agent-inspect logs ./agent.log --summary
|
|
378
|
-
npx agent-inspect logs ./agent.log --warnings all
|
|
379
|
-
```
|
|
380
|
-
|
|
381
|
-
Log ingestion notes:
|
|
382
|
-
|
|
383
|
-
- JSON logs are first-class.
|
|
384
|
-
- log4js text logs are best-effort: only embedded **valid JSON payloads** are supported.
|
|
385
|
-
- JavaScript object-literal payloads are intentionally unsupported.
|
|
386
|
-
- No eval is used.
|
|
387
|
-
- Flat timeline is default (nesting only with explicit `parentId`).
|
|
388
|
-
- Confidence labels explain attribution.
|
|
389
|
-
- Redaction is applied to sensitive attributes (based on config).
|
|
153
|
+
- Instrument a run with `inspectRun` and `step`
|
|
154
|
+
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
155
|
+
- Inspect runs with **`list`** and **`view`**
|
|
156
|
+
- Safely remove old trace files with **`clean`**
|
|
390
157
|
|
|
391
|
-
|
|
158
|
+
**Stable APIs:** `inspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`.
|
|
392
159
|
|
|
393
|
-
|
|
394
|
-
npx agent-inspect tail --file ./agent.log --format json
|
|
395
|
-
npx agent-inspect tail --file ./agent.log --format log4js --config agent-inspect.logs.json
|
|
396
|
-
npm run dev 2>&1 | npx agent-inspect tail --format log4js --config agent-inspect.logs.json
|
|
397
|
-
npx agent-inspect tail --file ./agent.log --format auto --once
|
|
398
|
-
npx agent-inspect tail --file ./agent.log --json --once
|
|
399
|
-
```
|
|
160
|
+
**Stable CLI workflows:** `agent-inspect list`, `agent-inspect view`, `agent-inspect clean`.
|
|
400
161
|
|
|
401
|
-
|
|
162
|
+
**Also included in 1.0** as local-first extensions:
|
|
402
163
|
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
164
|
+
- Structured log inspection: **`logs`**
|
|
165
|
+
- Live log tailing: **`tail`**
|
|
166
|
+
- Local exports: **`export`** (Markdown, HTML, OpenInference-compatible JSON, OTLP JSON — files only)
|
|
167
|
+
- Local run comparison: **`diff`**
|
|
168
|
+
- Optional **`@agent-inspect/langchain`** callback adapter
|
|
169
|
+
- Optional **`@agent-inspect/tui`** terminal viewer
|
|
170
|
+
- **Fixtures** and **recipes** for deterministic checks and adoption patterns
|
|
407
171
|
|
|
408
|
-
|
|
172
|
+
**Honest boundaries:** programmatic log parsing, export, and diff APIs; LangChain and TUI programmatic surfaces; and OpenInference/OTLP JSON exports are **experimental or compatibility-oriented**. Nothing performs **vendor upload** by default.
|
|
409
173
|
|
|
410
|
-
|
|
174
|
+
## Optional packages
|
|
411
175
|
|
|
412
|
-
|
|
413
|
-
AGENT_INSPECT_TRACE_DIR=./traces npx agent-inspect list
|
|
414
|
-
```
|
|
176
|
+
### LangChain callback adapter (`@agent-inspect/langchain`)
|
|
415
177
|
|
|
416
|
-
|
|
178
|
+
Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. The LangChain adapter is available in 1.0, but its programmatic API remains experimental and may evolve independently of the stable core tracing API.
|
|
417
179
|
|
|
418
180
|
```bash
|
|
419
|
-
|
|
420
|
-
node packages/cli/dist/index.cjs view run_abc123
|
|
421
|
-
```
|
|
422
|
-
|
|
423
|
-
## Local traces
|
|
424
|
-
|
|
425
|
-
agent-inspect writes one JSONL file per run.
|
|
426
|
-
|
|
427
|
-
Default location:
|
|
428
|
-
|
|
429
|
-
```text
|
|
430
|
-
~/.agent-inspect/runs
|
|
181
|
+
pnpm add agent-inspect @agent-inspect/langchain @langchain/core
|
|
431
182
|
```
|
|
432
183
|
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
```json
|
|
436
|
-
{"schemaVersion":"0.1","event":"step_started","name":"search-hotels","type":"logic"}
|
|
437
|
-
```
|
|
184
|
+
```ts
|
|
185
|
+
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
438
186
|
|
|
439
|
-
|
|
187
|
+
const callback = new AgentInspectCallback({
|
|
188
|
+
runName: "my-run",
|
|
189
|
+
capture: "metadata-only",
|
|
190
|
+
});
|
|
440
191
|
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
|
|
192
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
193
|
+
const events = callback.getEvents();
|
|
444
194
|
```
|
|
445
195
|
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
The repo includes five runnable MVP manual-tracing examples, the v0.3 structured log-to-tree example, and the v0.5 LangChain adapter example:
|
|
196
|
+
See [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) and [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
449
197
|
|
|
450
|
-
|
|
451
|
-
- `examples/02-nested-steps` — nested execution tree hierarchy
|
|
452
|
-
- `examples/03-parallel-steps` — `Promise.all` sibling isolation
|
|
453
|
-
- `examples/04-error-handling` — failed steps and error traces
|
|
454
|
-
- `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
|
|
455
|
-
- `examples/06-log-to-tree` — v0.3 structured log-to-tree example (includes historical spike prototype and production `agent-inspect logs` usage)
|
|
456
|
-
- `examples/08-langchain-adapter` — v0.5 LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
|
|
198
|
+
### TUI viewer (`@agent-inspect/tui`)
|
|
457
199
|
|
|
458
|
-
|
|
200
|
+
Optional **Ink/React** package, installed separately. Use with an interactive terminal:
|
|
459
201
|
|
|
460
202
|
```bash
|
|
461
|
-
pnpm
|
|
462
|
-
|
|
463
|
-
pnpm install
|
|
464
|
-
pnpm start
|
|
203
|
+
pnpm add agent-inspect @agent-inspect/tui
|
|
204
|
+
npx agent-inspect view <run-id> --tui
|
|
465
205
|
```
|
|
466
206
|
|
|
467
|
-
|
|
207
|
+
The TUI is available as a separate optional package; its programmatic API is experimental, while the CLI integration (`view --tui`) is the intended usage. Details: [docs/ADAPTERS.md](docs/ADAPTERS.md).
|
|
468
208
|
|
|
469
|
-
|
|
470
|
-
node ../../packages/cli/dist/index.cjs list
|
|
471
|
-
node ../../packages/cli/dist/index.cjs view run_abc123
|
|
472
|
-
```
|
|
209
|
+
## Examples and recipes
|
|
473
210
|
|
|
474
|
-
|
|
211
|
+
| Example | Shows |
|
|
212
|
+
| ------- | ----- |
|
|
213
|
+
| [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md) | Fast install-and-try trace |
|
|
214
|
+
| [examples/01-basic](examples/01-basic) | `inspectRun` + `step` |
|
|
215
|
+
| [examples/02-nested-steps](examples/02-nested-steps) | Nested tree |
|
|
216
|
+
| [examples/03-parallel-steps](examples/03-parallel-steps) | Parallel siblings |
|
|
217
|
+
| [examples/04-error-handling](examples/04-error-handling) | Failed steps |
|
|
218
|
+
| [examples/05-observe-wrapper](examples/05-observe-wrapper) | `observe()` |
|
|
219
|
+
| [examples/06-log-to-tree](examples/06-log-to-tree) | `logs` / `tail` |
|
|
220
|
+
| [examples/08-langchain-adapter](examples/08-langchain-adapter/README.md) | LangChain callbacks |
|
|
221
|
+
| [examples/recipes/rag-pipeline](examples/recipes/rag-pipeline) | RAG-shaped flow |
|
|
222
|
+
| [examples/recipes/tool-failure-retry](examples/recipes/tool-failure-retry) | Tool failure + retry |
|
|
223
|
+
| [examples/recipes/multi-agent-handoff](examples/recipes/multi-agent-handoff) | Handoff |
|
|
224
|
+
| [examples/recipes/proactive-agent-logs](examples/recipes/proactive-agent-logs) | Structured logs |
|
|
225
|
+
| [examples/recipes/retry-fallback](examples/recipes/retry-fallback) | Fallback pattern |
|
|
226
|
+
| [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
|
|
475
227
|
|
|
476
|
-
|
|
228
|
+
**Recipes** are deterministic and require **no external services** by default. Index: [examples/README.md](examples/README.md), [examples/recipes/README.md](examples/recipes/README.md).
|
|
477
229
|
|
|
478
|
-
|
|
230
|
+
## Security and privacy posture
|
|
479
231
|
|
|
480
|
-
|
|
232
|
+
- **Local files by default** — no upload, no vendor sinks in core workflows.
|
|
233
|
+
- **No API keys** required for core tracing and CLI inspection.
|
|
234
|
+
- **Manual metadata** is user-controlled; traces and exports can contain sensitive data if you put it there.
|
|
235
|
+
- **Review exports** before sharing (especially with richer attribute flags).
|
|
481
236
|
|
|
482
|
-
|
|
237
|
+
See [SECURITY.md](SECURITY.md).
|
|
483
238
|
|
|
484
|
-
-
|
|
485
|
-
- `step()`
|
|
486
|
-
- `step.llm()`
|
|
487
|
-
- `step.tool()`
|
|
488
|
-
- `observe()`
|
|
489
|
-
- JSONL traces
|
|
490
|
-
- CLI `list` and `view`
|
|
239
|
+
## agent-inspect comparison
|
|
491
240
|
|
|
492
|
-
|
|
241
|
+
It can **complement** LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, and similar platforms — but it does **not** replace their production or eval workflows.
|
|
493
242
|
|
|
494
|
-
|
|
495
|
-
- CLI `logs` (structured log-to-tree)
|
|
496
|
-
- CLI `tail` (live log tailing into grouped timelines)
|
|
497
|
-
- LangChain callback adapter via `@agent-inspect/langchain`
|
|
498
|
-
- Optional TUI viewer via `@agent-inspect/tui`
|
|
499
|
-
- Standards-aligned **local** exports (`export`: Markdown, HTML, OpenInference-compatible JSON, OTLP JSON mapping)
|
|
500
|
-
- Run diff / compare (`diff`: two local traces, read-only)
|
|
501
|
-
- Canonical **fixtures** under [`fixtures/`](fixtures/README.md) plus `pnpm fixtures:check` for deterministic samples
|
|
243
|
+
For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
502
244
|
|
|
503
|
-
|
|
245
|
+
## Documentation
|
|
504
246
|
|
|
505
|
-
-
|
|
506
|
-
-
|
|
507
|
-
-
|
|
508
|
-
-
|
|
509
|
-
-
|
|
510
|
-
-
|
|
511
|
-
-
|
|
512
|
-
-
|
|
513
|
-
-
|
|
514
|
-
-
|
|
515
|
-
-
|
|
516
|
-
-
|
|
247
|
+
- [Getting started](docs/GETTING-STARTED.md)
|
|
248
|
+
- [API stability & experimental surfaces](docs/API.md)
|
|
249
|
+
- [CLI reference](docs/CLI.md)
|
|
250
|
+
- [Schema (`schemaVersion: "0.1"`)](docs/SCHEMA.md)
|
|
251
|
+
- [Architecture (links to deeper design notes)](docs/ARCHITECTURE.md)
|
|
252
|
+
- [Logs & tail](docs/LOGS.md)
|
|
253
|
+
- [Log-to-tree quickstart](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
254
|
+
- [Exports](docs/EXPORTS.md)
|
|
255
|
+
- [Diff](docs/DIFF.md)
|
|
256
|
+
- [Adapters](docs/ADAPTERS.md)
|
|
257
|
+
- [Compare with other tools](docs/COMPARE.md)
|
|
258
|
+
- [Security](SECURITY.md)
|
|
259
|
+
- [Changelog](CHANGELOG.md)
|
|
260
|
+
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
261
|
+
- [Limitations](docs/LIMITATIONS.md)
|
|
262
|
+
- [Screenshot checklist (planned assets)](docs/SCREENSHOTS.md)
|
|
517
263
|
|
|
518
264
|
## Development
|
|
519
265
|
|
|
266
|
+
From a clone of this repo:
|
|
267
|
+
|
|
520
268
|
```bash
|
|
521
269
|
pnpm install
|
|
522
270
|
pnpm build
|
|
523
271
|
pnpm test
|
|
524
272
|
pnpm test:all
|
|
525
273
|
```
|
|
274
|
+
|
|
275
|
+
To run the CLI from source after a build: `node packages/cli/dist/index.cjs --help`.
|