agent-inspect 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -4
- package/README.md +24 -7
- package/SECURITY.md +1 -1
- package/docs/API.md +35 -9
- package/docs/ARCHITECTURE.md +1 -0
- package/docs/CLI.md +47 -0
- package/docs/DIFF.md +127 -0
- package/docs/GETTING-STARTED.md +7 -2
- package/docs/KNOWN-ISSUES.md +53 -0
- package/docs/LIMITATIONS.md +6 -0
- package/docs/SCHEMA.md +40 -2
- package/package.json +2 -1
- package/packages/cli/dist/index.cjs +619 -95
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +613 -90
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/index.cjs +1312 -122
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +140 -1
- package/packages/core/dist/index.d.ts +140 -1
- package/packages/core/dist/index.mjs +1299 -119
- package/packages/core/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,12 +1,32 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## 1.
|
|
3
|
+
## 1.2.0
|
|
4
4
|
|
|
5
5
|
### Minor Changes
|
|
6
6
|
|
|
7
|
-
-
|
|
7
|
+
- 5a7f785: v1.2.0: experimental persisted-event foundation (`PersistedInspectEvent` schemaVersion 0.2), validators, TraceEvent/InspectEvent converters, in-memory tree bridge, v0.2 fixtures, and docs. Manual trace writing remains schemaVersion 0.1; no storage or CLI behavior change.
|
|
8
|
+
|
|
9
|
+
## 1.2.0 — Unreleased
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- Added experimental `PersistedInspectEvent` model (`schemaVersion: "0.2"`) as a source-agnostic event foundation.
|
|
14
|
+
- Added validator for persisted events (`isPersistedInspectEvent`).
|
|
15
|
+
- Added converters from legacy `schemaVersion: "0.1"` manual trace events to persisted events.
|
|
16
|
+
- Added converters between `InspectEvent` and `PersistedInspectEvent`.
|
|
17
|
+
- Added in-memory helpers to build run trees from persisted events (`persistedInspectEventsToRunTrees`, `traceEventsToPersistedRunTrees`).
|
|
18
|
+
- Added canonical v0.2 fixture samples under `fixtures/traces-v0.2/`.
|
|
19
|
+
|
|
20
|
+
### Notes
|
|
21
|
+
|
|
22
|
+
- Existing manual trace writing remains `schemaVersion: "0.1"`.
|
|
23
|
+
- v0.2 is not written by default in this release.
|
|
24
|
+
- CLI read/write behavior is unchanged.
|
|
25
|
+
- No vendor upload, hosted dashboard, OTLP HTTP sink, replay engine, or cost engine was added.
|
|
26
|
+
|
|
27
|
+
## 1.1.0
|
|
8
28
|
|
|
9
|
-
|
|
29
|
+
Changeset `21ecc6f`: env-gated tracing, trace safety (redaction + size bounds), LangChain JSONL persistence, logging recipes, CJS/ESM type export compatibility, community docs.
|
|
10
30
|
|
|
11
31
|
### Added
|
|
12
32
|
|
|
@@ -34,7 +54,7 @@
|
|
|
34
54
|
### Documentation
|
|
35
55
|
|
|
36
56
|
- Added/updated logging playbook for structured JSON logs ([docs/LOGGING-PLAYBOOK.md](docs/LOGGING-PLAYBOOK.md)).
|
|
37
|
-
- Updated public roadmap
|
|
57
|
+
- Updated public roadmap after the 1.1.0 release (Released recently / Now / Next / Future).
|
|
38
58
|
- Updated contributor/community docs for package boundaries and optional packages.
|
|
39
59
|
- Added clearer community onboarding and issue-draft guidance.
|
|
40
60
|
|
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
agent-inspect helps you understand what happened inside an AI agent run — **locally**. It turns manual steps, tool calls, LLM calls, structured logs, failures, durations, and run metadata into **readable execution trees** you can inspect from the terminal.
|
|
6
6
|
|
|
7
|
-
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it **
|
|
7
|
+
It is built for TypeScript/Node.js developers and teams shipping real agentic products — not just toy demos. Use it for **local TypeScript agent debugging**, **eval iteration**, and **CI trace artifacts**. It **complements** production observability platforms; it does **not** replace them.
|
|
8
8
|
|
|
9
9
|
The tool starts with **manual traces** and **existing structured logs**, and extends into **optional framework callbacks** and **standards-aligned local export** — without turning the core into a SaaS or a vendor pipeline.
|
|
10
10
|
|
|
@@ -20,6 +20,8 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
|
|
|
20
20
|
|
|
21
21
|
## Install
|
|
22
22
|
|
|
23
|
+
Current npm release: **1.1.0**.
|
|
24
|
+
|
|
23
25
|
```bash
|
|
24
26
|
npm install agent-inspect
|
|
25
27
|
```
|
|
@@ -34,6 +36,8 @@ Verify the CLI is available:
|
|
|
34
36
|
npx agent-inspect --help
|
|
35
37
|
```
|
|
36
38
|
|
|
39
|
+
For a clean npm/pnpm install checklist with ESM, CJS, and CLI checks, see [Clean install smoke test](docs/INSTALL-SMOKE-TEST.md).
|
|
40
|
+
|
|
37
41
|
## 60-second quickstart
|
|
38
42
|
|
|
39
43
|
Create `demo.mjs`:
|
|
@@ -153,14 +157,14 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
|
153
157
|
|
|
154
158
|
- Debug a **failed tool call** or thrown error in a support or ops agent.
|
|
155
159
|
- See **which step dominated latency** in a multi-step planner or RAG pipeline.
|
|
156
|
-
- **Diff two runs** after a prompt, model, or routing change.
|
|
160
|
+
- **Diff two runs** after a prompt, model, or routing change (see [diff examples](docs/DIFF.md)).
|
|
157
161
|
- Point **`logs`** / **`tail`** at existing job or service logs to get a **local execution view** without shipping data upstream.
|
|
158
162
|
- **Export** a run to Markdown for a PR, postmortem, or internal thread — then review before sharing.
|
|
159
163
|
- Keep traces **on disk** while still using enterprise observability elsewhere.
|
|
160
164
|
|
|
161
|
-
##
|
|
165
|
+
## Stable foundation (AgentInspect 1.x)
|
|
162
166
|
|
|
163
|
-
**agent-inspect 1.0** stabilizes the **local debugging foundation**:
|
|
167
|
+
**agent-inspect 1.x** (current: **1.1.0**) stabilizes the **local debugging foundation**:
|
|
164
168
|
|
|
165
169
|
- Instrument a run with `inspectRun` and `step`
|
|
166
170
|
- Write **local JSONL traces** (`schemaVersion: "0.1"` — compatibility retained)
|
|
@@ -173,7 +177,7 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
|
|
|
173
177
|
|
|
174
178
|
**Stable CLI workflows:** `agent-inspect list`, `agent-inspect view`, `agent-inspect clean`.
|
|
175
179
|
|
|
176
|
-
**Also included in 1.
|
|
180
|
+
**Also included in 1.x** as local-first extensions:
|
|
177
181
|
|
|
178
182
|
- Structured log inspection: **`logs`**
|
|
179
183
|
- Live log tailing: **`tail`**
|
|
@@ -189,7 +193,7 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
|
|
|
189
193
|
|
|
190
194
|
### LangChain callback adapter (`@agent-inspect/langchain`)
|
|
191
195
|
|
|
192
|
-
Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. The LangChain adapter
|
|
196
|
+
Optional package: official **LangChain.js callbacks** (`BaseCallbackHandler`), **metadata-oriented by default**, **no monkey-patching**, **no vendor sink**. The LangChain adapter ships with 1.x; its programmatic API remains experimental and may evolve independently of the stable core tracing API.
|
|
193
197
|
|
|
194
198
|
```bash
|
|
195
199
|
pnpm add agent-inspect @agent-inspect/langchain @langchain/core
|
|
@@ -257,7 +261,7 @@ The TUI is available as a separate optional package; its programmatic API is exp
|
|
|
257
261
|
- **Manual metadata** is user-controlled. By default, common sensitive keys are **redacted before disk**; pass `redact: false` to opt out. Long metadata is truncated and events are capped at 64 KiB per JSONL line. Review traces and exports before sharing.
|
|
258
262
|
- **Review exports** before sharing (especially with richer attribute flags).
|
|
259
263
|
|
|
260
|
-
See [SECURITY.md](SECURITY.md).
|
|
264
|
+
See [SECURITY.md](SECURITY.md) and the [safe trace sharing checklist](docs/SAFE-TRACE-SHARING.md).
|
|
261
265
|
|
|
262
266
|
## agent-inspect comparison
|
|
263
267
|
|
|
@@ -268,6 +272,7 @@ For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
|
268
272
|
## Documentation
|
|
269
273
|
|
|
270
274
|
- [Getting started](docs/GETTING-STARTED.md)
|
|
275
|
+
- [Clean install smoke test](docs/INSTALL-SMOKE-TEST.md)
|
|
271
276
|
- [API stability & experimental surfaces](docs/API.md)
|
|
272
277
|
- [CLI reference](docs/CLI.md)
|
|
273
278
|
- [Schema (`schemaVersion: "0.1"`)](docs/SCHEMA.md)
|
|
@@ -280,11 +285,23 @@ For a detailed comparison, see [Compare with other tools](docs/COMPARE.md).
|
|
|
280
285
|
- [Adapters](docs/ADAPTERS.md)
|
|
281
286
|
- [Compare with other tools](docs/COMPARE.md)
|
|
282
287
|
- [Security](SECURITY.md)
|
|
288
|
+
- [Safe trace sharing checklist](docs/SAFE-TRACE-SHARING.md)
|
|
283
289
|
- [Changelog](CHANGELOG.md)
|
|
284
290
|
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
285
291
|
- [Limitations](docs/LIMITATIONS.md)
|
|
286
292
|
- [Screenshot checklist (planned assets)](docs/SCREENSHOTS.md)
|
|
287
293
|
|
|
294
|
+
## Contributing
|
|
295
|
+
|
|
296
|
+
AgentInspect welcomes docs, fixtures, examples, and carefully scoped CLI improvements.
|
|
297
|
+
|
|
298
|
+
- **Good first issues:** [GOOD-FIRST-ISSUES.md](GOOD-FIRST-ISSUES.md) — live batches [#7–#14](https://github.com/rajudandigam/agent-inspect/issues?q=is%3Aissue+is%3Aopen) and [#18–#30](https://github.com/rajudandigam/agent-inspect/issues/18) (comment on an issue before opening a PR)
|
|
299
|
+
- **Discussions:** [github.com/rajudandigam/agent-inspect/discussions](https://github.com/rajudandigam/agent-inspect/discussions) — feedback, stack survey, integration ideas
|
|
300
|
+
- **Roadmap:** [ROADMAP.md](ROADMAP.md) — Now / Next / Future direction (non-committal)
|
|
301
|
+
- **Contributing guide:** [CONTRIBUTING.md](CONTRIBUTING.md) — validation commands, PR expectations, scope boundaries
|
|
302
|
+
|
|
303
|
+
**Security:** Traces and logs may contain secrets. **Redact before sharing** in issues, Discussions, PRs, or exports. See [SECURITY.md](SECURITY.md) and the [safe trace sharing checklist](docs/SAFE-TRACE-SHARING.md).
|
|
304
|
+
|
|
288
305
|
## Development
|
|
289
306
|
|
|
290
307
|
From a clone of this repo:
|
package/SECURITY.md
CHANGED
|
@@ -70,7 +70,7 @@ Important limitations:
|
|
|
70
70
|
- Always **review exported files** before sharing them externally.
|
|
71
71
|
- Avoid committing trace directories (`.agent-inspect-runs/`) to source control.
|
|
72
72
|
|
|
73
|
-
For schema-level redaction notes, see `docs/SCHEMA.md` (redaction section) and `docs/API.md`.
|
|
73
|
+
For a practical pre-share workflow, see `docs/SAFE-TRACE-SHARING.md`. For schema-level redaction notes, see `docs/SCHEMA.md` (redaction section) and `docs/API.md`.
|
|
74
74
|
|
|
75
75
|
## Dependency and security review process
|
|
76
76
|
|
package/docs/API.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# API (AgentInspect 1.
|
|
1
|
+
# API (AgentInspect 1.x)
|
|
2
2
|
|
|
3
3
|
This document describes the **public TypeScript API surface** of AgentInspect and classifies each area as **stable** or **experimental**.
|
|
4
4
|
|
|
@@ -11,9 +11,9 @@ AgentInspect is a **local-first execution-tree debugger**. It is not a SaaS, not
|
|
|
11
11
|
|
|
12
12
|
Notes:
|
|
13
13
|
|
|
14
|
-
- The core guarantee of v1.
|
|
14
|
+
- The core guarantee of v1.x is **stable local debugging**: manual tracing + CLI inspection.
|
|
15
15
|
- Export formats (OpenInference / OTLP JSON) are **local-only** and **compatibility-oriented**. They do **not** upload anywhere.
|
|
16
|
-
- There are **zero production sinks** in v1.
|
|
16
|
+
- There are **zero production sinks** in v1.x; sink/uploader APIs are not stable.
|
|
17
17
|
|
|
18
18
|
## 2. Stable core APIs (manual tracing)
|
|
19
19
|
|
|
@@ -124,7 +124,7 @@ Diff is local and read-only. Programmatic diff surfaces are experimental until t
|
|
|
124
124
|
- In-memory **`getEvents()`** / **`clear()`** unchanged when `persist` is false
|
|
125
125
|
- Metadata helpers: `extractModelName`, `extractTokenUsage`, `safePreview`, `toPlainMetadata`
|
|
126
126
|
|
|
127
|
-
Rationale: v1.
|
|
127
|
+
Rationale: v1.x includes one official adapter and **zero production sinks**, so adapter surfaces remain experimental.
|
|
128
128
|
|
|
129
129
|
## 10. Experimental `@agent-inspect/tui` APIs
|
|
130
130
|
|
|
@@ -132,23 +132,49 @@ Rationale: v1.0 includes one official adapter and **zero production sinks**, so
|
|
|
132
132
|
|
|
133
133
|
- `runTraceViewer`, `loadTraceForTui`, `buildTuiTraceModel`, etc.
|
|
134
134
|
|
|
135
|
-
## 11.
|
|
135
|
+
## 11. Experimental persisted-event foundation (v1.2.0)
|
|
136
136
|
|
|
137
|
-
|
|
137
|
+
These helpers expose the **source-agnostic `PersistedInspectEvent` model** (`schemaVersion: "0.2"`). They are **local-only**, **in-memory**, and **do not change** storage write/read or CLI behavior in v1.2.0.
|
|
138
138
|
|
|
139
|
-
|
|
139
|
+
Import from `agent-inspect`:
|
|
140
|
+
|
|
141
|
+
| API | Role |
|
|
142
|
+
| --- | ---- |
|
|
143
|
+
| `isPersistedInspectEvent` | Runtime validator for v0.2 persisted events |
|
|
144
|
+
| `traceEventToPersistedInspectEvent` | Convert one v0.1 `TraceEvent` |
|
|
145
|
+
| `traceEventsToPersistedInspectEvents` | Batch v0.1 → v0.2 |
|
|
146
|
+
| `inspectEventToPersistedInspectEvent` | Convert one in-memory `InspectEvent` |
|
|
147
|
+
| `inspectEventsToPersistedInspectEvents` | Batch `InspectEvent` → v0.2 |
|
|
148
|
+
| `persistedInspectEventToInspectEvent` | Convert one v0.2 event to `InspectEvent` |
|
|
149
|
+
| `persistedInspectEventsToInspectEvents` | Batch v0.2 → `InspectEvent` |
|
|
150
|
+
| `persistedInspectEventsToRunTrees` | Build `InspectRunTree[]` from v0.2 events (via `TreeBuilder`) |
|
|
151
|
+
| `traceEventsToPersistedRunTrees` | v0.1 `TraceEvent[]` → persisted model → trees |
|
|
152
|
+
|
|
153
|
+
Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEventStatus`, `TraceEventToPersistedOptions`, `InspectEventToPersistedOptions`, `PersistedToInspectEventOptions`, `PersistedTreeBridgeOptions`.
|
|
154
|
+
|
|
155
|
+
**Notes:**
|
|
156
|
+
|
|
157
|
+
- Manual trace **writing** remains `schemaVersion: "0.1"`.
|
|
158
|
+
- v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
|
|
159
|
+
- Storage dual-read and CLI integration are future work.
|
|
160
|
+
|
|
161
|
+
## 12. Deprecated APIs
|
|
162
|
+
|
|
163
|
+
No deprecated APIs are declared as of 1.1.0.
|
|
164
|
+
|
|
165
|
+
## 13. Removal / deprecation policy
|
|
140
166
|
|
|
141
167
|
- Stable APIs are not removed in v1.x.
|
|
142
168
|
- If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
|
|
143
169
|
|
|
144
|
-
##
|
|
170
|
+
## 14. Backward compatibility policy
|
|
145
171
|
|
|
146
172
|
- Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
|
|
147
173
|
- Additive schema changes are allowed in minor versions.
|
|
148
174
|
- Breaking changes require a major version.
|
|
149
175
|
- Unknown fields should be ignored where safe.
|
|
150
176
|
|
|
151
|
-
##
|
|
177
|
+
## 15. Examples
|
|
152
178
|
|
|
153
179
|
### Minimal manual trace
|
|
154
180
|
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -19,6 +19,7 @@ Root `agent-inspect` uses **conditional exports** for ESM/CJS TypeScript consume
|
|
|
19
19
|
- Manual traces use **`schemaVersion: "0.1"`** JSONL events (`run_started`, `step_started`, `step_completed`, `run_completed`).
|
|
20
20
|
- Failures use `step_completed` with `status: "error"` — there is no `step_failed` event.
|
|
21
21
|
- Log-derived runs use confidence labels (`explicit`, `correlated`, `heuristic`, `unknown`) and conservative tree-building rules.
|
|
22
|
+
- **v1.2.0 persisted-event foundation (release-readiness):** source-agnostic `PersistedInspectEvent` helpers (`schemaVersion: "0.2"`) — types, validators, converters, and in-memory tree bridge. Existing **`TreeBuilder`** remains the canonical tree builder. v0.2 bridge works **in memory only**; storage dual-read and CLI integration are future work. See [proposals/UNIFIED-PERSISTED-INSPECT-EVENT.md](./proposals/UNIFIED-PERSISTED-INSPECT-EVENT.md) and [implementation/V1.2.0-RELEASE-READINESS.md](./implementation/V1.2.0-RELEASE-READINESS.md).
|
|
22
23
|
|
|
23
24
|
See [SCHEMA.md](./SCHEMA.md) for field reference and [API.md](./API.md) for public surfaces (stable vs experimental).
|
|
24
25
|
|
package/docs/CLI.md
CHANGED
|
@@ -196,6 +196,53 @@ Options:
|
|
|
196
196
|
- `--focus <all|errors|structure|outputs>`
|
|
197
197
|
- `--check <all|structure|outputs|errors|timing>`
|
|
198
198
|
|
|
199
|
+
Fixture examples:
|
|
200
|
+
|
|
201
|
+
```bash
|
|
202
|
+
agent-inspect diff minimal-success minimal-error --dir fixtures/traces
|
|
203
|
+
agent-inspect diff minimal-success long-running --dir fixtures/traces --check timing --duration-threshold 1ms
|
|
204
|
+
agent-inspect diff minimal-success nested-3-levels --dir fixtures/traces --check structure --ignore-duration
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
**Simplified example output** (actual CLI formatting may differ slightly):
|
|
208
|
+
|
|
209
|
+
```text
|
|
210
|
+
Run diff
|
|
211
|
+
Left: minimal-success
|
|
212
|
+
Right: minimal-error
|
|
213
|
+
|
|
214
|
+
Summary:
|
|
215
|
+
Differences: 4
|
|
216
|
+
Errors: 0
|
|
217
|
+
Warnings: 3
|
|
218
|
+
Info: 1
|
|
219
|
+
|
|
220
|
+
First divergence:
|
|
221
|
+
run-status at (run)
|
|
222
|
+
left: success
|
|
223
|
+
right: error
|
|
224
|
+
|
|
225
|
+
Differences:
|
|
226
|
+
[warning] run-status
|
|
227
|
+
Run completion status differs
|
|
228
|
+
left: success
|
|
229
|
+
right: error
|
|
230
|
+
[info] duration
|
|
231
|
+
Run duration differs
|
|
232
|
+
left: 120
|
|
233
|
+
right: 70
|
|
234
|
+
[warning] step-removed plan
|
|
235
|
+
Step only in left run: plan
|
|
236
|
+
left: step_root
|
|
237
|
+
right: (undefined)
|
|
238
|
+
[warning] step-added failing-step
|
|
239
|
+
Step only in right run: failing-step
|
|
240
|
+
left: (undefined)
|
|
241
|
+
right: step_fail
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
|
|
245
|
+
|
|
199
246
|
## 7. Optional TUI behavior
|
|
200
247
|
|
|
201
248
|
`view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.
|
package/docs/DIFF.md
CHANGED
|
@@ -8,4 +8,131 @@ AgentInspect can compare two runs (manual traces) locally and render a diff summ
|
|
|
8
8
|
Notes:
|
|
9
9
|
- Programmatic diff APIs are documented as **experimental** in `docs/API.md`.
|
|
10
10
|
- `diff` is read-only and does **not** rerun agents; “differences” do not necessarily imply failure (command errors do).
|
|
11
|
+
- `diff` reads local trace files only. It is useful for local debugging, but it is not production APM or hosted observability.
|
|
11
12
|
|
|
13
|
+
## CLI examples
|
|
14
|
+
|
|
15
|
+
The examples below use the checked-in fixture traces, so they are safe to copy and run locally:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
agent-inspect diff minimal-success minimal-error --dir fixtures/traces
|
|
19
|
+
agent-inspect diff minimal-success long-running --dir fixtures/traces --check timing --duration-threshold 1ms
|
|
20
|
+
agent-inspect diff minimal-success nested-3-levels --dir fixtures/traces --check structure --ignore-duration
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
If you are running from a source checkout before installing the package globally, use the built CLI path instead:
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
node packages/cli/dist/index.cjs diff minimal-success minimal-error --dir fixtures/traces
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### Success run vs error run
|
|
30
|
+
|
|
31
|
+
This compares a successful run with a run that ended in an error. It shows the run status change, a duration difference, and the step-level structure difference (`plan` removed, `failing-step` added).
|
|
32
|
+
|
|
33
|
+
**Simplified example output** (actual CLI formatting may differ slightly):
|
|
34
|
+
|
|
35
|
+
```text
|
|
36
|
+
Run diff
|
|
37
|
+
Left: minimal-success
|
|
38
|
+
Right: minimal-error
|
|
39
|
+
|
|
40
|
+
Summary:
|
|
41
|
+
Differences: 4
|
|
42
|
+
Errors: 0
|
|
43
|
+
Warnings: 3
|
|
44
|
+
Info: 1
|
|
45
|
+
|
|
46
|
+
First divergence:
|
|
47
|
+
run-status at (run)
|
|
48
|
+
left: success
|
|
49
|
+
right: error
|
|
50
|
+
|
|
51
|
+
Differences:
|
|
52
|
+
[warning] run-status
|
|
53
|
+
Run completion status differs
|
|
54
|
+
left: success
|
|
55
|
+
right: error
|
|
56
|
+
[info] duration
|
|
57
|
+
Run duration differs
|
|
58
|
+
left: 120
|
|
59
|
+
right: 70
|
|
60
|
+
[warning] step-removed plan
|
|
61
|
+
Step only in left run: plan
|
|
62
|
+
left: step_root
|
|
63
|
+
right: (undefined)
|
|
64
|
+
[warning] step-added failing-step
|
|
65
|
+
Step only in right run: failing-step
|
|
66
|
+
left: (undefined)
|
|
67
|
+
right: step_fail
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Duration-only timing check
|
|
71
|
+
|
|
72
|
+
Use `--check timing` when you only want timing differences. `--duration-threshold` can hide tiny deltas.
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
agent-inspect diff minimal-success long-running --dir fixtures/traces --check timing --duration-threshold 1ms
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
**Simplified example output**:
|
|
79
|
+
|
|
80
|
+
```text
|
|
81
|
+
Run diff
|
|
82
|
+
Left: minimal-success
|
|
83
|
+
Right: long-running
|
|
84
|
+
|
|
85
|
+
Summary:
|
|
86
|
+
Differences: 1
|
|
87
|
+
Errors: 0
|
|
88
|
+
Warnings: 0
|
|
89
|
+
Info: 1
|
|
90
|
+
|
|
91
|
+
First divergence:
|
|
92
|
+
duration at (run)
|
|
93
|
+
left: 120
|
|
94
|
+
right: 45
|
|
95
|
+
|
|
96
|
+
Differences:
|
|
97
|
+
[info] duration
|
|
98
|
+
Run duration differs
|
|
99
|
+
left: 120
|
|
100
|
+
right: 45
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### Step structure / rename-style changes
|
|
104
|
+
|
|
105
|
+
When a prompt, model, or routing change renames or restructures steps, the current diff output represents that as `step-removed` plus `step-added`. Use `--ignore-duration` if the structure is what matters.
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
agent-inspect diff minimal-success nested-3-levels --dir fixtures/traces --check structure --ignore-duration
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
**Simplified example output**:
|
|
112
|
+
|
|
113
|
+
```text
|
|
114
|
+
Run diff
|
|
115
|
+
Left: minimal-success
|
|
116
|
+
Right: nested-3-levels
|
|
117
|
+
|
|
118
|
+
Summary:
|
|
119
|
+
Differences: 2
|
|
120
|
+
Errors: 0
|
|
121
|
+
Warnings: 2
|
|
122
|
+
Info: 0
|
|
123
|
+
|
|
124
|
+
First divergence:
|
|
125
|
+
step-removed at plan
|
|
126
|
+
left: step_root
|
|
127
|
+
right: (undefined)
|
|
128
|
+
|
|
129
|
+
Differences:
|
|
130
|
+
[warning] step-removed plan
|
|
131
|
+
Step only in left run: plan
|
|
132
|
+
left: step_root
|
|
133
|
+
right: (undefined)
|
|
134
|
+
[warning] step-added outer
|
|
135
|
+
Step only in right run: outer
|
|
136
|
+
left: (undefined)
|
|
137
|
+
right: step_outer
|
|
138
|
+
```
|
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -14,6 +14,8 @@ The `agent-inspect` package includes the CLI binary via its `bin` field:
|
|
|
14
14
|
npx agent-inspect --help
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
+
For a clean install verification path covering npm, pnpm, ESM import, CJS require, and CLI help, see [Clean install smoke test](./INSTALL-SMOKE-TEST.md).
|
|
18
|
+
|
|
17
19
|
For local repo development (this monorepo), build and run the CLI from `packages/cli`:
|
|
18
20
|
|
|
19
21
|
```bash
|
|
@@ -33,7 +35,7 @@ await inspectRun("demo-agent", async () => {
|
|
|
33
35
|
});
|
|
34
36
|
```
|
|
35
37
|
|
|
36
|
-
This writes a local JSONL trace with stable
|
|
38
|
+
This writes a local JSONL trace with stable event names (`schemaVersion: "0.1"`):
|
|
37
39
|
|
|
38
40
|
- `run_started`, `run_completed`
|
|
39
41
|
- `step_started`, `step_completed`
|
|
@@ -58,6 +60,10 @@ AGENT_INSPECT=1 node eval-runner.mjs
|
|
|
58
60
|
|
|
59
61
|
Enable tokens: `1`, `true`, `yes`, `on`, `enabled` (case-insensitive). Explicit `enabled: true | false` in options overrides the env var.
|
|
60
62
|
|
|
63
|
+
### Install compatibility
|
|
64
|
+
|
|
65
|
+
If `import` or `require` fails after install, see [KNOWN-ISSUES.md — Common install/runtime compatibility checks](./KNOWN-ISSUES.md#common-installruntime-compatibility-checks).
|
|
66
|
+
|
|
61
67
|
To skip tracing in code without env vars: `inspectRun(name, fn, { enabled: false })`.
|
|
62
68
|
|
|
63
69
|
## 3. View runs
|
|
@@ -151,4 +157,3 @@ agent-inspect view <runId> --tui
|
|
|
151
157
|
- [docs/KNOWN-ISSUES.md](./KNOWN-ISSUES.md)
|
|
152
158
|
- [docs/LIMITATIONS.md](./LIMITATIONS.md)
|
|
153
159
|
- [ROADMAP.md](../ROADMAP.md)
|
|
154
|
-
|
package/docs/KNOWN-ISSUES.md
CHANGED
|
@@ -32,3 +32,56 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
|
|
|
32
32
|
- Log-derived trees carry **confidence** (`explicit`, `correlated`, `heuristic`, `unknown`). Labels explain **how** relationships were inferred—they are not semantic proof.
|
|
33
33
|
|
|
34
34
|
When in doubt, treat AgentInspect as a **debugging aid**, not a compliance or billing system of record.
|
|
35
|
+
|
|
36
|
+
## Common install/runtime compatibility checks
|
|
37
|
+
|
|
38
|
+
Use these before filing a packaging bug. AgentInspect targets **Node.js ≥ 20**.
|
|
39
|
+
|
|
40
|
+
### Quick self-check (clean temp project)
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
mkdir /tmp/agent-inspect-check && cd /tmp/agent-inspect-check
|
|
44
|
+
npm init -y
|
|
45
|
+
npm install agent-inspect@latest
|
|
46
|
+
node -e "import('agent-inspect').then(m => console.log('esm', typeof m.inspectRun))"
|
|
47
|
+
node -e "const m=require('agent-inspect'); console.log('cjs', typeof m.inspectRun, typeof m.maybeInspectRun)"
|
|
48
|
+
npx agent-inspect --help
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
From a **repo clone** (maintainers / contributors):
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
pnpm install --frozen-lockfile
|
|
55
|
+
pnpm build
|
|
56
|
+
pnpm compat:smoke
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### ESM import
|
|
60
|
+
|
|
61
|
+
- Consumer `package.json` should use `"type": "module"` (or `.mjs` entry files).
|
|
62
|
+
- Import: `import { inspectRun, step, maybeInspectRun } from "agent-inspect"`.
|
|
63
|
+
- TypeScript: `module` / `moduleResolution` `NodeNext` or `Node16` with matching `type` field.
|
|
64
|
+
|
|
65
|
+
### CJS require
|
|
66
|
+
|
|
67
|
+
- Consumer `package.json` should use `"type": "commonjs"` (or `.cjs` entry files).
|
|
68
|
+
- Require: `const { inspectRun, step, maybeInspectRun } = require("agent-inspect")`.
|
|
69
|
+
- TypeScript: compile `.cts` with `module` / `moduleResolution` `Node16`.
|
|
70
|
+
|
|
71
|
+
**Note:** `chalk@5` and `nanoid@5` are ESM-only upstream. Published `agent-inspect` bundles them into CJS output so Jest 29 / `require()` consumers do not resolve ESM-only deps at runtime. If you still see `ERR_REQUIRE_ESM`, report Node version, bundler, and full stack trace.
|
|
72
|
+
|
|
73
|
+
### Jest / ts-jest (CJS)
|
|
74
|
+
|
|
75
|
+
- Prefer `require("agent-inspect")` in CJS test files, or ensure your Jest config supports the package's export conditions.
|
|
76
|
+
- For tracing off in unit tests: `maybeInspectRun(name, fn, { enabled: false })` or unset `AGENT_INSPECT`.
|
|
77
|
+
- Fixture pattern: [test/consumer-fixtures/jest-cjs/](../../test/consumer-fixtures/jest-cjs/).
|
|
78
|
+
- Full Jest runner smoke in CI is a documented follow-up — root package does not ship Jest as a devDependency.
|
|
79
|
+
|
|
80
|
+
### What to include in a bug report
|
|
81
|
+
|
|
82
|
+
- Node.js version (`node -v`)
|
|
83
|
+
- Package manager and lockfile type (npm / pnpm / yarn)
|
|
84
|
+
- ESM vs CJS vs TypeScript + ts-jest
|
|
85
|
+
- Minimal repro repo or snippet
|
|
86
|
+
- Full error stack (not only the top line)
|
|
87
|
+
- Whether `AGENT_INSPECT` is set in the environment
|
package/docs/LIMITATIONS.md
CHANGED
|
@@ -9,6 +9,12 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
9
9
|
- **No vendor upload pipeline**: no built-in Langfuse/Braintrust/New Relic/Datadog direct exporters as live sinks.
|
|
10
10
|
- **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
|
|
11
11
|
|
|
12
|
+
## Persisted event model (v1.2.0 foundation)
|
|
13
|
+
|
|
14
|
+
- **v0.2 is not the default persisted trace file format.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL.
|
|
15
|
+
- **CLI commands** (`list`, `view`, `export`, `diff`, `logs`, `tail`) still primarily operate on current v0.1 trace and log paths.
|
|
16
|
+
- **v0.2 read/write integration** (dual-format storage, CLI consumption) is future work — v1.2.0 ships in-memory converters and canonical fixtures only.
|
|
17
|
+
|
|
12
18
|
## Data fidelity
|
|
13
19
|
|
|
14
20
|
- **No full prompt/output capture by default** for manual traces (by design: safety and PII risk).
|
package/docs/SCHEMA.md
CHANGED
|
@@ -198,8 +198,46 @@ Always review any exported content (Markdown/HTML/OpenInference/OTLP JSON) befor
|
|
|
198
198
|
|
|
199
199
|
See: `SECURITY.md` and `docs/API.md` (`InspectRunOptions.redact`, size bound options).
|
|
200
200
|
|
|
201
|
-
## 14.
|
|
201
|
+
## 14. Persisted InspectEvent schemaVersion "0.2" — experimental foundation
|
|
202
|
+
|
|
203
|
+
v1.2.0 introduces a **source-agnostic persisted event model** as an experimental foundation. It is **not** the default on-disk format.
|
|
204
|
+
|
|
205
|
+
| Topic | Rule |
|
|
206
|
+
| ----- | ---- |
|
|
207
|
+
| Default write format | Manual traces still use **`schemaVersion: "0.1"`** (`run_started`, `step_started`, `step_completed`, `run_completed`). |
|
|
208
|
+
| v0.2 role | Unified persisted shape for manual traces, log-derived events, adapter events, and future AI SDK / OTel mappings. |
|
|
209
|
+
| v0.2 file writing | **Not enabled by default** in v1.2.0 — converters and fixtures only. |
|
|
210
|
+
| v0.1 compatibility | v0.2 does **not** replace v0.1 in this release; existing `0.1` files remain canonical for CLI write/read today. |
|
|
211
|
+
| Failures | Still **no** `step_failed`; use `status: "error"` on the persisted event. |
|
|
212
|
+
|
|
213
|
+
Canonical samples: `fixtures/traces-v0.2/*.jsonl` (validated by `pnpm fixtures:check`).
|
|
214
|
+
|
|
215
|
+
### 14.1 Field reference (compact)
|
|
216
|
+
|
|
217
|
+
| Field | Required | Notes |
|
|
218
|
+
| ----- | -------- | ----- |
|
|
219
|
+
| `schemaVersion` | yes | `"0.2"` |
|
|
220
|
+
| `eventId` | yes | Stable per-event identifier |
|
|
221
|
+
| `runId` | yes | Run grouping key |
|
|
222
|
+
| `parentId` | no | Explicit nesting only when present |
|
|
223
|
+
| `kind` | yes | `InspectKind` (`RUN`, `LLM`, `TOOL`, `LOGIC`, …) |
|
|
224
|
+
| `name` | yes | Human-readable step label |
|
|
225
|
+
| `status` | no | `running` \| `ok` \| `error` \| `unknown` |
|
|
226
|
+
| `timestamp` | yes | ISO-8601 string (event time) |
|
|
227
|
+
| `startedAt` / `endedAt` | no | ISO-8601 bounds when known |
|
|
228
|
+
| `durationMs` | no | Non-negative milliseconds when known |
|
|
229
|
+
| `confidence` | yes | `explicit` \| `correlated` \| `heuristic` \| `unknown` |
|
|
230
|
+
| `source` | yes | `{ type, name?, version? }` — `manual`, `json-log`, `log4js`, `adapter`, `ai-sdk`, `otel` |
|
|
231
|
+
| `attributes` | no | Shallow metadata bag (redaction-ready) |
|
|
232
|
+
| `inputSummary` / `outputSummary` | no | Truncated previews when explicitly captured |
|
|
233
|
+
| `error` | no | `{ name?, message, code? }` when `status: "error"` |
|
|
234
|
+
| `tokenUsage` | no | `{ input?, output?, total? }` when known |
|
|
235
|
+
| `trace` | no | Optional `{ traceId?, spanId?, parentSpanId? }` for future OTel alignment |
|
|
236
|
+
|
|
237
|
+
Programmatic helpers: see [API.md](./API.md) §15 (experimental persisted-event foundation).
|
|
238
|
+
|
|
239
|
+
## 15. Migration notes
|
|
202
240
|
|
|
203
241
|
- Minor releases may add optional fields/events, but must keep existing v0.1 traces readable.
|
|
204
|
-
- Migration guides (v0.1 →
|
|
242
|
+
- Migration guides (v0.1 → v0.2 file format) are future work; storage/CLI dual-read is not in the v1.2.0 foundation release.
|
|
205
243
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.2.0",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Local-first execution-tree debugger for TypeScript AI agents",
|
|
@@ -97,6 +97,7 @@
|
|
|
97
97
|
"prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run pack:smoke",
|
|
98
98
|
"pack:dry-run": "pnpm run build && npm pack --dry-run",
|
|
99
99
|
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs",
|
|
100
|
+
"compat:smoke": "node scripts/compat-smoke.mjs",
|
|
100
101
|
"fixtures:check": "node scripts/validate-fixtures.mjs",
|
|
101
102
|
"recipes:check": "node scripts/validate-recipes.mjs",
|
|
102
103
|
"perf:baseline": "node scripts/performance-baseline.mjs",
|