agent-inspect 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -4
- package/README.md +39 -37
- package/docs/ADAPTERS.md +111 -31
- package/docs/API.md +15 -5
- package/docs/CLI.md +62 -1
- package/docs/EXPORTS.md +73 -15
- package/docs/GETTING-STARTED.md +29 -1
- package/docs/KNOWN-ISSUES.md +6 -0
- package/docs/LIMITATIONS.md +8 -2
- package/docs/SCHEMA.md +5 -4
- package/package.json +1 -1
- package/packages/cli/dist/index.cjs +748 -0
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +748 -0
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/index.cjs +575 -0
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +126 -1
- package/packages/core/dist/index.d.ts +126 -1
- package/packages/core/dist/index.mjs +569 -1
- package/packages/core/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,12 +1,27 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## 1.
|
|
3
|
+
## 1.4.0
|
|
4
|
+
|
|
5
|
+
### Added
|
|
4
6
|
|
|
5
|
-
|
|
7
|
+
- Added `docs/CI-ARTIFACTS.md` and `examples/recipes/github-actions-artifact/` for CI trace artifact workflows.
|
|
8
|
+
- Added `agent-inspect timeline <runId>` — chronological local run view (`--json`, `--focus slow`).
|
|
9
|
+
- Added `agent-inspect stats` — local aggregate stats (`--since`, `--correlation-id`, `--group-id`, `--json`).
|
|
10
|
+
- Added `agent-inspect search` — deterministic local trace search (`--status`, `--kind`, `--name`, `--tool`, `--duration`, `--json`).
|
|
11
|
+
- Added core helpers: `buildRunTimeline`, `buildTraceStats`, `searchTraces`.
|
|
6
12
|
|
|
7
|
-
|
|
13
|
+
### Notes
|
|
14
|
+
|
|
15
|
+
- CI artifact upload is configured in user CI (e.g. GitHub Actions `upload-artifact`) — AgentInspect does not upload.
|
|
16
|
+
- Search is exact/contains matching only — no semantic search or index database.
|
|
17
|
+
- Stats/search scan local files linearly — intended for developer-machine scale.
|
|
18
|
+
- No Vitest/Jest reporter package in this release.
|
|
19
|
+
- Manual trace writing remains `schemaVersion: "0.1"`.
|
|
20
|
+
- Linked release aligns `@agent-inspect/tui` with `agent-inspect` and `@agent-inspect/langchain` (all **1.4.0**).
|
|
21
|
+
|
|
22
|
+
## 1.3.0
|
|
8
23
|
|
|
9
|
-
|
|
24
|
+
Released **2026-06-12**.
|
|
10
25
|
|
|
11
26
|
### Added
|
|
12
27
|
|
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
|
|
|
20
20
|
|
|
21
21
|
## Install
|
|
22
22
|
|
|
23
|
-
Current npm release: **1.2.
|
|
23
|
+
Current npm release: **1.3.0** (`agent-inspect`, `@agent-inspect/langchain`; `@agent-inspect/tui` is **1.2.1** — patch-only alignment release).
|
|
24
24
|
|
|
25
25
|
```bash
|
|
26
26
|
npm install agent-inspect
|
|
@@ -109,6 +109,21 @@ await maybeInspectRun("eval-case-42", async () => runAgent());
|
|
|
109
109
|
AGENT_INSPECT=1 node eval-runner.mjs
|
|
110
110
|
```
|
|
111
111
|
|
|
112
|
+
## What you can do today (v1.3.0)
|
|
113
|
+
|
|
114
|
+
- **Trace manually** with `inspectRun`, `step`, `step.llm`, `step.tool`, and `observe` — local JSONL under `.agent-inspect/` by default.
|
|
115
|
+
- **Toggle tracing** with `maybeInspectRun` and `AGENT_INSPECT=1` in eval harnesses or CI.
|
|
116
|
+
- **Correlate runs** with optional `correlationId`, `requestId`, `decisionId`, and `groupId` on `run_started` metadata.
|
|
117
|
+
- **Redact before disk** with default key-based redaction, or choose `redactionProfile`: `local`, `share`, or `strict`.
|
|
118
|
+
- **Inspect from the CLI** — `list`, `view`, `clean`, `logs`, `tail`, `export`, `diff`, `timeline`, `stats`, `search`.
|
|
119
|
+
- **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
|
|
120
|
+
- **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
|
|
121
|
+
- **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
|
|
122
|
+
- **Optional TUI** — `view --tui` when `@agent-inspect/tui` is installed.
|
|
123
|
+
- **Persisted-event foundation (v1.2.0+)** — in-memory `PersistedInspectEvent` converters; manual writing stays `schemaVersion: "0.1"`.
|
|
124
|
+
|
|
125
|
+
Nothing uploads traces by default. Review exports before sharing — see [safe trace sharing](docs/SAFE-TRACE-SHARING.md).
|
|
126
|
+
|
|
112
127
|
## What the trace shows
|
|
113
128
|
|
|
114
129
|
Each run produces a **JSONL** trace: `run_started` / `run_completed`, `step_started` / `step_completed`, with **nested steps**, **tool/LLM** types where you use `step.tool` / `step.llm`, and **durations** on completed steps. Failures are recorded on `step_completed` with `status: "error"` (there is no separate `step_failed` event). See [docs/SCHEMA.md](docs/SCHEMA.md).
|
|
@@ -150,6 +165,9 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
|
|
|
150
165
|
| `tail` | Watch structured logs while the app runs |
|
|
151
166
|
| `export` | Write Markdown / HTML / OpenInference-compatible JSON / OTLP JSON **locally** |
|
|
152
167
|
| `diff` | Compare two local runs (read-only) |
|
|
168
|
+
| `timeline` | Chronological view of one run |
|
|
169
|
+
| `stats` | Local aggregates over a trace directory |
|
|
170
|
+
| `search` | Deterministic search over local traces |
|
|
153
171
|
|
|
154
172
|
Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
155
173
|
|
|
@@ -164,31 +182,25 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
|
164
182
|
|
|
165
183
|
## Stable foundation (AgentInspect 1.x)
|
|
166
184
|
|
|
167
|
-
**agent-inspect 1.x** (current: **1.
|
|
185
|
+
**agent-inspect 1.x** (current: **1.3.0**) is the **local-first trace workbench** for TypeScript AI agents:
|
|
168
186
|
|
|
169
|
-
- Instrument
|
|
187
|
+
- Instrument runs with `inspectRun` and `step`
|
|
170
188
|
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
171
|
-
- Inspect
|
|
172
|
-
- Safely remove old trace files with **`clean`**
|
|
189
|
+
- Inspect with **`list`**, **`view`**, **`clean`**, **`logs`**, **`tail`**, **`export`**, **`diff`**
|
|
173
190
|
|
|
174
|
-
**Stable APIs:** `inspectRun()`, `maybeInspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`.
|
|
191
|
+
**Stable APIs:** `inspectRun()`, `maybeInspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`, `getCurrentCorrelationMetadata()`.
|
|
175
192
|
|
|
176
|
-
Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1`
|
|
193
|
+
Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
|
|
177
194
|
|
|
178
|
-
**
|
|
195
|
+
**Shipped in 1.3.0:** correlation metadata; redaction profiles (`local` / `share` / `strict`); `export --redaction-profile`; LangChain `stream: true` metadata (chunk counts, duration — no full token capture by default).
|
|
179
196
|
|
|
180
|
-
**Also
|
|
197
|
+
**Also in 1.x** (local-first extensions):
|
|
181
198
|
|
|
182
|
-
-
|
|
183
|
-
-
|
|
184
|
-
-
|
|
185
|
-
- Local run comparison: **`diff`**
|
|
186
|
-
- Optional **`@agent-inspect/langchain`** callback adapter
|
|
187
|
-
- Optional **`@agent-inspect/tui`** terminal viewer
|
|
188
|
-
- **Fixtures** and **recipes** for deterministic checks and adoption patterns
|
|
189
|
-
- **v1.2.0** — experimental persisted-event foundation (`PersistedInspectEvent`, converters, in-memory tree bridge) for future source-agnostic local inspection. Manual trace writing remains **`schemaVersion: "0.1"`**; v0.2 is **not written by default**; CLI behavior unchanged; no vendor upload.
|
|
199
|
+
- **v1.2.0** — experimental persisted-event foundation (`PersistedInspectEvent`, converters, in-memory tree bridge). Manual writing remains **`schemaVersion: "0.1"`**; v0.2 is **not written by default**.
|
|
200
|
+
- Optional **`@agent-inspect/langchain`** and **`@agent-inspect/tui`**
|
|
201
|
+
- **Fixtures** and **recipes** for deterministic adoption patterns
|
|
190
202
|
|
|
191
|
-
**Honest boundaries:**
|
|
203
|
+
**Honest boundaries:** log parsing, export, diff, LangChain/TUI programmatic APIs, and OpenInference/OTLP JSON exports are **experimental or compatibility-oriented**. Nothing performs **vendor upload** by default.
|
|
192
204
|
|
|
193
205
|
## Optional packages
|
|
194
206
|
|
|
@@ -272,25 +284,15 @@ For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
|
272
284
|
|
|
273
285
|
## Documentation
|
|
274
286
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
- [API
|
|
278
|
-
- [CLI
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
- [Exports](docs/EXPORTS.md)
|
|
285
|
-
- [Diff](docs/DIFF.md)
|
|
286
|
-
- [Adapters](docs/ADAPTERS.md)
|
|
287
|
-
- [Compare with other tools](docs/COMPARE.md)
|
|
288
|
-
- [Security](SECURITY.md)
|
|
289
|
-
- [Safe trace sharing checklist](docs/SAFE-TRACE-SHARING.md)
|
|
290
|
-
- [Changelog](CHANGELOG.md) — **upcoming [1.3.0](CHANGELOG.md#130--unreleased)** on `main` (not yet on npm): correlation metadata, share-safe export profiles, LangChain streaming metadata
|
|
291
|
-
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
292
|
-
- [Limitations](docs/LIMITATIONS.md)
|
|
293
|
-
- [Screenshot checklist (planned assets)](docs/SCREENSHOTS.md)
|
|
287
|
+
| Start here | Reference | Safety & boundaries |
|
|
288
|
+
| ---------- | --------- | ------------------- |
|
|
289
|
+
| [Getting started](docs/GETTING-STARTED.md) | [API](docs/API.md) | [Safe trace sharing](docs/SAFE-TRACE-SHARING.md) |
|
|
290
|
+
| [Install smoke test](docs/INSTALL-SMOKE-TEST.md) | [CLI](docs/CLI.md) | [Security](SECURITY.md) |
|
|
291
|
+
| [Log-to-tree quickstart](docs/LOG-TO-TREE-QUICKSTART.md) | [Schema](docs/SCHEMA.md) | [Limitations](docs/LIMITATIONS.md) |
|
|
292
|
+
| [Logging playbook](docs/LOGGING-PLAYBOOK.md) | [Exports](docs/EXPORTS.md) | [Known issues](docs/KNOWN-ISSUES.md) |
|
|
293
|
+
| [Examples](examples/README.md) | [Adapters](docs/ADAPTERS.md) | [Compare with other tools](docs/COMPARE.md) |
|
|
294
|
+
|
|
295
|
+
Also: [Architecture](docs/ARCHITECTURE.md) · [Logs & tail](docs/LOGS.md) · [Diff](docs/DIFF.md) · [Changelog](CHANGELOG.md) · [Roadmap](ROADMAP.md) · [Contributing](CONTRIBUTING.md) · [Good first issues](GOOD-FIRST-ISSUES.md)
|
|
294
296
|
|
|
295
297
|
## Contributing
|
|
296
298
|
|
package/docs/ADAPTERS.md
CHANGED
|
@@ -1,27 +1,38 @@
|
|
|
1
|
-
|
|
1
|
+
# Adapters
|
|
2
2
|
|
|
3
|
-
AgentInspect is framework-agnostic at its core
|
|
3
|
+
AgentInspect is **framework-agnostic** at its core. Optional adapter packages integrate specific frameworks without monkey-patching, vendor sinks, or network upload.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
- Documented as **experimental** in `docs/API.md`
|
|
7
|
-
- Requires `@langchain/core` as a peer dependency
|
|
8
|
-
- **In-memory by default** (`getEvents()` / `clear()`)
|
|
9
|
-
- **Optional JSONL persistence** (`persist: true`) maps callback lifecycle to schemaVersion `"0.1"` manual events (`run_started`, `step_started`, `step_completed`, `run_completed`) so `list` / `view` / `export` / `diff` work without a separate schema
|
|
10
|
-
- **Metadata-only capture by default**; `capture: "preview"` is opt-in with truncation
|
|
11
|
-
- **Optional streaming metadata** (`stream: true`) — chunk counts, timing, and bounded previews only; **no full token stream** captured by default
|
|
12
|
-
- **No monkey-patching**, **no vendor sinks**, **no network upload**
|
|
13
|
-
- **Interactive TUI** (optional): `@agent-inspect/tui`
|
|
14
|
-
- Documented as **experimental** in `docs/API.md`
|
|
15
|
-
- Intended for CLI integration; programmatic TUI APIs may change
|
|
5
|
+
## LangChain.js (`@agent-inspect/langchain`)
|
|
16
6
|
|
|
17
|
-
|
|
7
|
+
**Status:** experimental — programmatic API may evolve independently of stable core tracing.
|
|
18
8
|
|
|
19
|
-
|
|
9
|
+
### Install
|
|
20
10
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
11
|
+
```bash
|
|
12
|
+
npm install agent-inspect @agent-inspect/langchain @langchain/core
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
### Basic callback (in-memory)
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
import { AgentInspectCallback } from "@agent-inspect/langchain";
|
|
19
|
+
|
|
20
|
+
const callback = new AgentInspectCallback({
|
|
21
|
+
runName: "support-agent",
|
|
22
|
+
capture: "metadata-only", // default
|
|
23
|
+
});
|
|
24
|
+
|
|
25
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
26
|
+
|
|
27
|
+
const events = callback.getEvents();
|
|
28
|
+
callback.clear();
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
- **No monkey-patching** — pass the callback explicitly to LangChain.
|
|
32
|
+
- **Metadata-only by default** — does not capture full prompts/outputs unless you opt into `capture: "preview"`.
|
|
33
|
+
- **No vendor sink** — events stay in memory unless you set `persist: true`.
|
|
34
|
+
|
|
35
|
+
### Persist to local JSONL
|
|
25
36
|
|
|
26
37
|
```ts
|
|
27
38
|
const callback = new AgentInspectCallback({
|
|
@@ -30,23 +41,34 @@ const callback = new AgentInspectCallback({
|
|
|
30
41
|
persist: true,
|
|
31
42
|
capture: "metadata-only",
|
|
32
43
|
});
|
|
44
|
+
|
|
45
|
+
await agent.invoke(input, { callbacks: [callback] });
|
|
33
46
|
```
|
|
34
47
|
|
|
35
48
|
```bash
|
|
36
49
|
npx agent-inspect list --dir ./.agent-inspect
|
|
37
50
|
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
51
|
+
npx agent-inspect export <run-id> --format markdown --redaction-profile share
|
|
38
52
|
```
|
|
39
53
|
|
|
40
|
-
|
|
54
|
+
**Persistence model (Strategy A):**
|
|
41
55
|
|
|
42
|
-
|
|
56
|
+
1. **Standalone session** — one AgentInspect run per callback instance until the root LangChain run completes.
|
|
57
|
+
2. **Inside `inspectRun`** — callback steps append to the active manual run (no extra `run_started` / `run_completed`).
|
|
58
|
+
3. **Parent mapping** — LangChain `parentRunId` maps to AgentInspect `parentId` when the parent step was persisted; unknown parents stay at run root.
|
|
59
|
+
4. **Step types** — `LLM` → `llm`, `TOOL` → `tool`, `DECISION` → `decision`, other kinds → `logic`.
|
|
60
|
+
|
|
61
|
+
Written events use `schemaVersion: "0.1"` manual trace names.
|
|
62
|
+
|
|
63
|
+
### Capture modes
|
|
43
64
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
-
|
|
48
|
-
|
|
49
|
-
|
|
65
|
+
| `capture` | Behavior |
|
|
66
|
+
| --------- | -------- |
|
|
67
|
+
| `none` | Minimal metadata |
|
|
68
|
+
| `metadata-only` | **Default** — model names, token usage, timing; no full text |
|
|
69
|
+
| `preview` | Truncated previews via `maxPreviewChars` — review before sharing |
|
|
70
|
+
|
|
71
|
+
### Streaming metadata (v1.3.0+)
|
|
50
72
|
|
|
51
73
|
```ts
|
|
52
74
|
const callback = new AgentInspectCallback({
|
|
@@ -56,8 +78,66 @@ const callback = new AgentInspectCallback({
|
|
|
56
78
|
});
|
|
57
79
|
```
|
|
58
80
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
- `
|
|
62
|
-
-
|
|
81
|
+
When `stream: true`:
|
|
82
|
+
|
|
83
|
+
- `handleLLMNewToken` updates **in-memory stats only** — no per-token JSONL lines.
|
|
84
|
+
- On LLM end/error: `chunkCount`, `firstChunkAt`, `lastChunkAt`, `streamDurationMs`, `streamedCharCount`.
|
|
85
|
+
- **`capture: "metadata-only"`** does **not** store raw token text.
|
|
86
|
+
- **`capture: "preview"`** may include bounded `streamPreview` via `maxStreamPreviewChars`.
|
|
87
|
+
- With **`persist: true`**, streaming metadata is written on the LLM step at completion (deferred write for streaming LLM steps).
|
|
88
|
+
|
|
89
|
+
Streaming metadata is for **local inspection and timing** — not replay or cassette playback.
|
|
90
|
+
|
|
91
|
+
### Correlation inside `inspectRun`
|
|
92
|
+
|
|
93
|
+
When the callback runs inside `inspectRun` / `maybeInspectRun`, correlation fields from `getCurrentCorrelationMetadata()` attach to LLM lifecycle events.
|
|
94
|
+
|
|
95
|
+
### Options reference
|
|
96
|
+
|
|
97
|
+
| Option | Default | Notes |
|
|
98
|
+
| ------ | ------- | ----- |
|
|
99
|
+
| `persist` | `false` | Write local JSONL |
|
|
100
|
+
| `runName` | `"langchain-agent"` | Standalone persisted run name |
|
|
101
|
+
| `traceDir` | from env / `.agent-inspect` | |
|
|
102
|
+
| `capture` | `"metadata-only"` | |
|
|
103
|
+
| `stream` | `false` | Streaming lifecycle metadata |
|
|
104
|
+
| `maxStreamPreviewChars` | `maxPreviewChars` | Bounds preview when `capture: "preview"` |
|
|
105
|
+
| `redact` | — | Custom `RedactionRule[]` before disk |
|
|
106
|
+
|
|
107
|
+
Full API: [API.md](./API.md) §9.
|
|
108
|
+
|
|
109
|
+
### Example
|
|
110
|
+
|
|
111
|
+
[examples/08-langchain-adapter](../examples/08-langchain-adapter/README.md)
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## TUI (`@agent-inspect/tui`)
|
|
116
|
+
|
|
117
|
+
**Status:** experimental programmatic API; CLI integration is the intended usage.
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
npm install agent-inspect @agent-inspect/tui
|
|
121
|
+
npx agent-inspect view <run-id> --tui
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Requires an interactive terminal. See [API.md](./API.md) §10.
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## Future adapters (not shipped)
|
|
129
|
+
|
|
130
|
+
Direction only — see [ROADMAP.md](../ROADMAP.md):
|
|
131
|
+
|
|
132
|
+
- **Vercel AI SDK** — optional callback-style adapter (metadata-first, no vendor sink)
|
|
133
|
+
- **NestJS / logging bridges** — deeper recipes or helper patterns beyond [LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md)
|
|
134
|
+
|
|
135
|
+
No automatic universal instrumentation. Integrations remain explicit and opt-in.
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## Related docs
|
|
63
140
|
|
|
141
|
+
- [API.md](./API.md) — adapter options and stability policy
|
|
142
|
+
- [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) — review exports before sharing
|
|
143
|
+
- [LIMITATIONS.md](./LIMITATIONS.md) — LangChain streaming and metadata boundaries
|
package/docs/API.md
CHANGED
|
@@ -166,23 +166,33 @@ Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEv
|
|
|
166
166
|
- v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
|
|
167
167
|
- Storage dual-read and CLI integration are future work.
|
|
168
168
|
|
|
169
|
-
## 12.
|
|
169
|
+
## 12. Local observability helpers (v1.4.0)
|
|
170
170
|
|
|
171
|
-
|
|
171
|
+
Read-only helpers for timeline, stats, and search over manual `TraceEvent` JSONL. Local files only.
|
|
172
172
|
|
|
173
|
-
|
|
173
|
+
- **`buildRunTimeline`**, **`renderTimeline`** — chronological run view; types `RunTimeline`, `TimelineEntry`
|
|
174
|
+
- **`buildTraceStats`**, **`renderTraceStats`** — directory aggregates; type `TraceStats`
|
|
175
|
+
- **`searchTraces`**, **`parseDurationFilter`**, **`loadTraceMetadataList`** — deterministic search; types `TraceSearchResult`, `TraceSearchOptions`
|
|
176
|
+
|
|
177
|
+
CLI wrappers: `agent-inspect timeline`, `stats`, `search` — see [CLI.md](./CLI.md).
|
|
178
|
+
|
|
179
|
+
## 13. Deprecated APIs
|
|
180
|
+
|
|
181
|
+
No deprecated APIs are declared as of 1.4.0.
|
|
182
|
+
|
|
183
|
+
## 14. Removal / deprecation policy
|
|
174
184
|
|
|
175
185
|
- Stable APIs are not removed in v1.x.
|
|
176
186
|
- If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
|
|
177
187
|
|
|
178
|
-
##
|
|
188
|
+
## 15. Backward compatibility policy
|
|
179
189
|
|
|
180
190
|
- Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
|
|
181
191
|
- Additive schema changes are allowed in minor versions.
|
|
182
192
|
- Breaking changes require a major version.
|
|
183
193
|
- Unknown fields should be ignored where safe.
|
|
184
194
|
|
|
185
|
-
##
|
|
195
|
+
## 16. Examples
|
|
186
196
|
|
|
187
197
|
### Minimal manual trace
|
|
188
198
|
|
package/docs/CLI.md
CHANGED
|
@@ -26,6 +26,9 @@ Core commands:
|
|
|
26
26
|
- `tail` — live-tail logs into updating local trees
|
|
27
27
|
- `export` — export manual traces to Markdown/HTML/OpenInference/OTLP JSON (local only)
|
|
28
28
|
- `diff` — compare two manual traces (local, read-only)
|
|
29
|
+
- `timeline` — chronological view of one run (local JSONL)
|
|
30
|
+
- `stats` — local aggregate stats over a trace directory
|
|
31
|
+
- `search` — deterministic local search over traces
|
|
29
32
|
|
|
30
33
|
## 2. Environment variables
|
|
31
34
|
|
|
@@ -162,7 +165,7 @@ Important: `tail` is a local developer tool, not a production monitor.
|
|
|
162
165
|
|
|
163
166
|
### 6.6 `export`
|
|
164
167
|
|
|
165
|
-
Export a manual trace run to local formats. **No upload
|
|
168
|
+
Export a manual trace run to local formats. **No upload.** Export redaction operates on a **copy** of the run tree — original JSONL files are not modified. Review every export before sharing, even with `--redaction-profile strict`.
|
|
166
169
|
|
|
167
170
|
```bash
|
|
168
171
|
agent-inspect export <run-id> --format <markdown|html|openinference|otlp-json> [options]
|
|
@@ -251,6 +254,64 @@ Differences:
|
|
|
251
254
|
|
|
252
255
|
More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
|
|
253
256
|
|
|
257
|
+
### 6.8 `timeline`
|
|
258
|
+
|
|
259
|
+
Chronological step list for one manual trace. Read-only; does not mutate JSONL files.
|
|
260
|
+
|
|
261
|
+
```bash
|
|
262
|
+
agent-inspect timeline <run-id> [options]
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Options:
|
|
266
|
+
|
|
267
|
+
- `--dir <path>`
|
|
268
|
+
- `--json` — structured `RunTimeline` JSON
|
|
269
|
+
- `--focus slow` — show only the slowest steps by duration (top N)
|
|
270
|
+
|
|
271
|
+
### 6.9 `stats`
|
|
272
|
+
|
|
273
|
+
Local aggregate statistics over trace files in a directory. Read-only.
|
|
274
|
+
|
|
275
|
+
```bash
|
|
276
|
+
agent-inspect stats [options]
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
Options:
|
|
280
|
+
|
|
281
|
+
- `--dir <path>`
|
|
282
|
+
- `--since <duration>` — e.g. `7d`, `24h`
|
|
283
|
+
- `--correlation-id <id>` — filter by `run_started.metadata.correlationId`
|
|
284
|
+
- `--group-id <id>` — filter by `run_started.metadata.groupId`
|
|
285
|
+
- `--json`
|
|
286
|
+
|
|
287
|
+
### 6.10 `search`
|
|
288
|
+
|
|
289
|
+
Deterministic search over local traces (substring / exact filters). No semantic search.
|
|
290
|
+
|
|
291
|
+
```bash
|
|
292
|
+
agent-inspect search [options]
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
Options:
|
|
296
|
+
|
|
297
|
+
- `--dir <path>`
|
|
298
|
+
- `--since <duration>`
|
|
299
|
+
- `--status <success|error|running|unknown>`
|
|
300
|
+
- `--kind <kind>` / `--type <type>` — manual step type (`llm`, `tool`, `logic`, …)
|
|
301
|
+
- `--name <query>` — substring on run or step name
|
|
302
|
+
- `--tool <query>` — substring on tool step name or `metadata.toolName`
|
|
303
|
+
- `--duration <expr>` — e.g. `>5s`, `>=500ms`
|
|
304
|
+
- `--limit <number>` — default 50
|
|
305
|
+
- `--json`
|
|
306
|
+
|
|
307
|
+
Examples:
|
|
308
|
+
|
|
309
|
+
```bash
|
|
310
|
+
npx agent-inspect search --status error --dir ./.agent-inspect
|
|
311
|
+
npx agent-inspect search --kind tool --name search
|
|
312
|
+
npx agent-inspect search --duration ">100ms" --json
|
|
313
|
+
```
|
|
314
|
+
|
|
254
315
|
## 7. Optional TUI behavior
|
|
255
316
|
|
|
256
317
|
`view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.
|
package/docs/EXPORTS.md
CHANGED
|
@@ -1,26 +1,84 @@
|
|
|
1
|
-
|
|
1
|
+
# Exports
|
|
2
2
|
|
|
3
|
-
AgentInspect
|
|
3
|
+
AgentInspect exports are **local-only**: they produce strings or files on disk. Nothing uploads to a vendor, collector, or hosted dashboard.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
- **Schema + compatibility guarantees**: `docs/SCHEMA.md`
|
|
7
|
-
- **Safety / redaction notes**: `docs/CLI.md`, `SECURITY.md`, and `docs/SCHEMA.md` (redaction section)
|
|
5
|
+
Use exports to share run summaries in PRs, postmortems, or internal threads — **after reviewing** the output. See [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md).
|
|
8
6
|
|
|
9
|
-
##
|
|
7
|
+
## CLI
|
|
10
8
|
|
|
11
|
-
|
|
9
|
+
```bash
|
|
10
|
+
agent-inspect export <run-id> --format <markdown|html|openinference|otlp-json> [options]
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Full flags: [CLI.md](./CLI.md) §6.6.
|
|
14
|
+
|
|
15
|
+
## Formats
|
|
16
|
+
|
|
17
|
+
| Format | Purpose |
|
|
18
|
+
| ------ | ------- |
|
|
19
|
+
| `markdown` | Human-readable tree for PRs, issues, docs |
|
|
20
|
+
| `html` | Standalone HTML snapshot for local viewing |
|
|
21
|
+
| `openinference` | OpenInference-compatible JSON (compatibility-oriented) |
|
|
22
|
+
| `otlp-json` | OTLP JSON shape (experimental; validate per backend) |
|
|
23
|
+
|
|
24
|
+
All formats are **compatibility-oriented** for local inspection and handoff — not guaranteed to match every vendor schema version.
|
|
25
|
+
|
|
26
|
+
## Common options
|
|
27
|
+
|
|
28
|
+
| Flag | Notes |
|
|
29
|
+
| ---- | ----- |
|
|
30
|
+
| `--dir <path>` | Trace directory (default from `AGENT_INSPECT_TRACE_DIR` or `.agent-inspect`) |
|
|
31
|
+
| `-o, --output <path>` | Write to file instead of stdout |
|
|
32
|
+
| `--json` | JSON wrapper output (includes content when not writing to file) |
|
|
33
|
+
| `--validate` | Validate exported payload shape before writing |
|
|
34
|
+
| `--include-attributes` | Include bounded attributes — **review before sharing** |
|
|
35
|
+
| `--no-metadata` | Omit summary/metadata sections |
|
|
36
|
+
| `--no-errors` | Omit error sections |
|
|
37
|
+
| `--redaction-profile <local\|share\|strict>` | Key-based redaction on the **export copy** (v1.3.0+) |
|
|
38
|
+
|
|
39
|
+
## Redaction profiles (share-safe exports)
|
|
40
|
+
|
|
41
|
+
Profiles apply to the **exported copy** only. Original JSONL traces on disk are **not mutated**.
|
|
42
|
+
|
|
43
|
+
| Profile | Typical use |
|
|
44
|
+
| ------- | ------------- |
|
|
45
|
+
| `local` | Default — same key-based defaults as trace writing |
|
|
46
|
+
| `share` | PRs, GitHub issues, internal Slack/email threads |
|
|
47
|
+
| `strict` | External or public sharing — also redacts prompt/output/message-like keys |
|
|
12
48
|
|
|
13
49
|
```bash
|
|
14
|
-
agent-inspect export <run-id> --format markdown --redaction-profile share
|
|
50
|
+
npx agent-inspect export <run-id> --format markdown --redaction-profile share
|
|
51
|
+
npx agent-inspect export <run-id> --format html --redaction-profile strict
|
|
52
|
+
npx agent-inspect export <run-id> --format openinference --redaction-profile share --validate
|
|
15
53
|
```
|
|
16
54
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
55
|
+
Redaction profiles are **key-based safeguards**, not compliance-grade PII detection. Always review exports manually.
|
|
56
|
+
|
|
57
|
+
## Programmatic API
|
|
58
|
+
|
|
59
|
+
Experimental helpers (local-only):
|
|
60
|
+
|
|
61
|
+
- `exportRunTree`, `redactRunTreeForExport`
|
|
62
|
+
- `exportMarkdown`, `exportHtml`, `exportOpenInference`, `exportOtlpJson`
|
|
63
|
+
- `validateExport`, `validateExportContent`
|
|
64
|
+
|
|
65
|
+
See [API.md](./API.md) §7.
|
|
66
|
+
|
|
67
|
+
## What not to share
|
|
68
|
+
|
|
69
|
+
Even with `--redaction-profile strict`, review exports for:
|
|
70
|
+
|
|
71
|
+
- API keys, tokens, cookies, session IDs
|
|
72
|
+
- Customer IDs, emails, phone numbers, internal hostnames
|
|
73
|
+
- Full prompts, completions, tool I/O (especially with `--include-attributes`)
|
|
74
|
+
- File paths that expose usernames or internal project names
|
|
20
75
|
|
|
21
|
-
|
|
76
|
+
Prefer the smallest useful artifact — a summary or synthetic reproduction beats a full production trace.
|
|
22
77
|
|
|
23
|
-
|
|
24
|
-
- Export formats (Markdown/HTML/OpenInference/OTLP JSON) are documented as **experimental / compatibility-oriented** in `docs/API.md`.
|
|
25
|
-
- AgentInspect does **not** upload anywhere; exports write local strings/files only.
|
|
78
|
+
## Related docs
|
|
26
79
|
|
|
80
|
+
- [CLI.md](./CLI.md) — `export` command reference
|
|
81
|
+
- [SCHEMA.md](./SCHEMA.md) — trace event model and redaction notes
|
|
82
|
+
- [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) — checklist before posting
|
|
83
|
+
- [API.md](./API.md) — `ExportOptions`, experimental export APIs
|
|
84
|
+
- [LIMITATIONS.md](./LIMITATIONS.md) — compatibility and boundary notes
|
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -60,6 +60,24 @@ AGENT_INSPECT=1 node eval-runner.mjs
|
|
|
60
60
|
|
|
61
61
|
Enable tokens: `1`, `true`, `yes`, `on`, `enabled` (case-insensitive). Explicit `enabled: true | false` in options overrides the env var.
|
|
62
62
|
|
|
63
|
+
### Correlation metadata (v1.3.0+)
|
|
64
|
+
|
|
65
|
+
Attach optional correlation fields to `run_started` metadata — useful for eval cases, CI job IDs, and future stats views. They are **metadata only**, not run IDs. Review before sharing exports.
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
await inspectRun(
|
|
69
|
+
"support-agent",
|
|
70
|
+
async () => runAgent(),
|
|
71
|
+
{
|
|
72
|
+
correlationId: "eval-suite-2026-06",
|
|
73
|
+
requestId: "req-abc123",
|
|
74
|
+
groupId: "ci-job-42",
|
|
75
|
+
}
|
|
76
|
+
);
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Inside a traced run, adapters can read active fields via `getCurrentCorrelationMetadata()`.
|
|
80
|
+
|
|
63
81
|
### Install compatibility
|
|
64
82
|
|
|
65
83
|
If `import` or `require` fails after install, see [KNOWN-ISSUES.md — Common install/runtime compatibility checks](./KNOWN-ISSUES.md#common-installruntime-compatibility-checks).
|
|
@@ -109,7 +127,14 @@ agent-inspect export minimal-success --dir fixtures/traces --format markdown
|
|
|
109
127
|
agent-inspect export minimal-success --dir fixtures/traces --format openinference --validate
|
|
110
128
|
```
|
|
111
129
|
|
|
112
|
-
|
|
130
|
+
Share-safe copy for a PR or issue (v1.3.0+):
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
agent-inspect export minimal-success --dir fixtures/traces \
|
|
134
|
+
--format markdown --redaction-profile share
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Exports are **local-only** and do not upload anywhere. Review output before sharing — see [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md).
|
|
113
138
|
|
|
114
139
|
## 8. Diff two runs
|
|
115
140
|
|
|
@@ -151,6 +176,9 @@ agent-inspect view <runId> --tui
|
|
|
151
176
|
- [docs/API.md](./API.md)
|
|
152
177
|
- [docs/CLI.md](./CLI.md)
|
|
153
178
|
- [docs/SCHEMA.md](./SCHEMA.md)
|
|
179
|
+
- [docs/EXPORTS.md](./EXPORTS.md)
|
|
180
|
+
- [docs/SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md)
|
|
181
|
+
- [docs/ADAPTERS.md](./ADAPTERS.md)
|
|
154
182
|
- [docs/LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md)
|
|
155
183
|
- [docs/COMPARE.md](./COMPARE.md)
|
|
156
184
|
- [docs/LOG-TO-TREE-QUICKSTART.md](./LOG-TO-TREE-QUICKSTART.md)
|
package/docs/KNOWN-ISSUES.md
CHANGED
|
@@ -16,6 +16,12 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
|
|
|
16
16
|
|
|
17
17
|
- **Vendor sinks** (hosted dashboards, Langfuse/Braintrust/New Relic/Datadog native uploads, OTLP gRPC streaming, etc.) are **not implemented** in the core packages described here.
|
|
18
18
|
- **LangChain adapter** captures **metadata-oriented** signals by default; it does not replace full framework observability.
|
|
19
|
+
- **LangChain `stream: true`** records chunk counts and timing only — not a full token replay. Per-token JSONL events are not emitted.
|
|
20
|
+
- **Correlation metadata** (`correlationId`, `requestId`, `decisionId`, `groupId`) is written on `run_started` but **CLI list/view does not filter by correlation fields** yet.
|
|
21
|
+
|
|
22
|
+
## Structured log parsing
|
|
23
|
+
|
|
24
|
+
- Ingest requires **line-delimited JSON** or a **recoverable JSON payload** in the log line (log4js-style is best-effort). Arbitrary printf-style text without JSON is not supported.
|
|
19
25
|
|
|
20
26
|
## UI / replay
|
|
21
27
|
|
package/docs/LIMITATIONS.md
CHANGED
|
@@ -9,9 +9,9 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
9
9
|
- **No vendor upload pipeline**: no built-in Langfuse/Braintrust/New Relic/Datadog direct exporters as live sinks.
|
|
10
10
|
- **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
|
|
11
11
|
|
|
12
|
-
## Correlation metadata (v1.3.0
|
|
12
|
+
## Correlation metadata (v1.3.0)
|
|
13
13
|
|
|
14
|
-
-
|
|
14
|
+
- **`list` / `view` / `export`** do not filter by correlation fields yet — `stats --correlation-id` / `--group-id` and `search` provide targeted read paths; full CLI filtering remains incremental.
|
|
15
15
|
|
|
16
16
|
## Persisted event model (v1.2.0 foundation)
|
|
17
17
|
|
|
@@ -48,6 +48,12 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
48
48
|
|
|
49
49
|
- **No cost engine**: no pricing tables, invoice-grade usage, or provider billing reconciliation.
|
|
50
50
|
|
|
51
|
+
## Local observability commands (v1.4.0)
|
|
52
|
+
|
|
53
|
+
- **`timeline`**, **`stats`**, and **`search`** scan local JSONL files — no database index; large directories may be slow.
|
|
54
|
+
- **`search`** is deterministic exact/contains matching only — no semantic or fuzzy search.
|
|
55
|
+
- **`stats`** is local file aggregation — not production fleet analytics.
|
|
56
|
+
|
|
51
57
|
## Scale
|
|
52
58
|
|
|
53
59
|
- Designed for **developer machines** and **inner-loop debugging**, not petabyte log warehouses.
|