agent-inspect 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +99 -0
- package/README.md +116 -98
- package/SECURITY.md +77 -0
- package/docs/ADAPTERS.md +15 -0
- package/docs/API.md +152 -0
- package/docs/ARCHITECTURE.md +13 -0
- package/docs/CLI.md +212 -0
- package/docs/COMPARE.md +69 -0
- package/docs/DIFF.md +11 -0
- package/docs/EXPORTS.md +12 -0
- package/docs/GETTING-STARTED.md +128 -0
- package/docs/KNOWN-ISSUES.md +34 -0
- package/docs/LIMITATIONS.md +32 -0
- package/docs/LOG-TO-TREE-QUICKSTART.md +54 -0
- package/docs/LOGS.md +13 -0
- package/docs/MIGRATION.md +109 -0
- package/docs/SCHEMA.md +199 -0
- package/docs/SCREENSHOTS.md +14 -0
- package/package.json +36 -13
- package/packages/cli/dist/index.cjs +1 -1
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 1.0.1
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 575b093: docs: onboarding polish
|
|
8
|
+
|
|
9
|
+
## Unreleased
|
|
10
|
+
|
|
11
|
+
### Documentation
|
|
12
|
+
|
|
13
|
+
- Improved README onboarding for new users.
|
|
14
|
+
- Added comparison guide.
|
|
15
|
+
- Added log-to-tree quickstart.
|
|
16
|
+
- Added a quickstart demo example.
|
|
17
|
+
- Clarified docs organization between user-facing `docs/` and internal `docs-local/`.
|
|
18
|
+
|
|
19
|
+
## 1.0.0
|
|
20
|
+
|
|
21
|
+
### Stable local tracing
|
|
22
|
+
|
|
23
|
+
- Stable manual tracing entry points: `inspectRun`, `step`, `step.llm`, `step.tool`, `observe`
|
|
24
|
+
- v0.1 JSONL trace compatibility retained (schemaVersion `"0.1"`)
|
|
25
|
+
|
|
26
|
+
### Local inspection CLI
|
|
27
|
+
|
|
28
|
+
- Stable CLI workflows: `list`, `view`, `clean`
|
|
29
|
+
- Safety-critical cleanup verifies traces before deletion
|
|
30
|
+
|
|
31
|
+
### Structured logs and live tail
|
|
32
|
+
|
|
33
|
+
- Local log-to-tree parsing and live tail workflows (`logs`, `tail`) with confidence labeling
|
|
34
|
+
- Best-effort log4js parsing; JSON logs first-class; no unsafe object parsing
|
|
35
|
+
|
|
36
|
+
### Optional LangChain adapter
|
|
37
|
+
|
|
38
|
+
- `@agent-inspect/langchain` optional adapter package (experimental surface)
|
|
39
|
+
|
|
40
|
+
### Optional TUI
|
|
41
|
+
|
|
42
|
+
- `@agent-inspect/tui` optional Ink/React viewer (experimental programmatic surface)
|
|
43
|
+
|
|
44
|
+
### Standards-aligned local export
|
|
45
|
+
|
|
46
|
+
- Markdown/HTML exports for sharing traces locally
|
|
47
|
+
- OpenInference-compatible JSON export (experimental; verify against backends)
|
|
48
|
+
- OTLP JSON export (experimental; JSON mapping only, no OTLP gRPC)
|
|
49
|
+
|
|
50
|
+
### Diff and compare
|
|
51
|
+
|
|
52
|
+
- Local, read-only diff of two manual traces (`diff`)
|
|
53
|
+
|
|
54
|
+
### Fixtures, recipes, and hardening (v0.9)
|
|
55
|
+
|
|
56
|
+
- Canonical fixtures under `fixtures/` plus validation scripts
|
|
57
|
+
- Runnable recipes under `examples/recipes/` with deterministic expected output markers
|
|
58
|
+
- Package smoke checks and adoption hardening tests
|
|
59
|
+
|
|
60
|
+
### Documentation and stability
|
|
61
|
+
|
|
62
|
+
- Added/updated API/CLI/schema/getting-started docs for v1.0 stabilization
|
|
63
|
+
- Added stability and compatibility tests to prevent accidental surface breaks
|
|
64
|
+
|
|
65
|
+
### Known limitations
|
|
66
|
+
|
|
67
|
+
- Local-first only; no SaaS/dashboard; no vendor sinks; no replay; no cost engine
|
|
68
|
+
|
|
69
|
+
## Historical notes (v0.1–v0.9)
|
|
70
|
+
|
|
71
|
+
AgentInspect started as a minimal manual tracing MVP (v0.1) and evolved through:
|
|
72
|
+
|
|
73
|
+
- local inspection improvements (metadata, filtering, safety checks)
|
|
74
|
+
- structured log ingestion (JSON first-class, log4js best-effort)
|
|
75
|
+
- conservative tree building rules with confidence labels
|
|
76
|
+
- incremental live tail rendering
|
|
77
|
+
- standards-aligned local exports (experimental)
|
|
78
|
+
- run diff and compare
|
|
79
|
+
- fixtures, recipes, and hardening focused on adoption
|
|
80
|
+
|
|
81
|
+
For detailed intent and sequencing (planning docs), see:
|
|
82
|
+
|
|
83
|
+
- `docs-local/roadmap/VERSION-ROADMAP.md`
|
|
84
|
+
- `docs-local/strategy/PRODUCT-PRINCIPLES.md`
|
|
85
|
+
|
|
86
|
+
# agent-inspect
|
|
87
|
+
|
|
88
|
+
## 0.1.2
|
|
89
|
+
|
|
90
|
+
### Patch Changes
|
|
91
|
+
|
|
92
|
+
- 62afb94: fix CI/publish smoke + vitest config
|
|
93
|
+
|
|
94
|
+
## 0.1.1
|
|
95
|
+
|
|
96
|
+
### Patch Changes
|
|
97
|
+
|
|
98
|
+
- bd719ef: Prepare npm publishing (Trusted Publishing via GitHub Actions OIDC) and polish documentation.
|
|
99
|
+
- 76791b8: Improve README, release docs, and npm publishing guidance.
|
package/README.md
CHANGED
|
@@ -1,24 +1,10 @@
|
|
|
1
1
|
# agent-inspect
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Local execution trees for TypeScript AI agents.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
AgentInspect helps you debug multi-step AI workflows locally by turning manual steps, structured logs, and agent callbacks into readable execution trees.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
agent-inspect turns runs into structured execution trees with JSONL traces and CLI inspection.
|
|
10
|
-
|
|
11
|
-
agent-inspect is designed for inner-loop debugging, not as a replacement for production observability platforms.
|
|
12
|
-
|
|
13
|
-
## What you get
|
|
14
|
-
|
|
15
|
-
- Execution-tree tracing for TypeScript agent workflows
|
|
16
|
-
- Nested `step()` support with parent-child relationships
|
|
17
|
-
- `step.llm()` and `step.tool()` helpers for agent-aware traces
|
|
18
|
-
- Local JSONL trace files
|
|
19
|
-
- Real-time terminal output while the agent runs
|
|
20
|
-
- CLI commands to inspect previous runs
|
|
21
|
-
- No accounts, API keys, dashboards, or cloud ingestion
|
|
7
|
+
No account. No cloud upload. No dashboard required.
|
|
22
8
|
|
|
23
9
|
## Install
|
|
24
10
|
|
|
@@ -26,86 +12,140 @@ agent-inspect is designed for inner-loop debugging, not as a replacement for pro
|
|
|
26
12
|
npm install agent-inspect
|
|
27
13
|
```
|
|
28
14
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
- [API reference](docs/API.md)
|
|
33
|
-
- [CLI reference](docs/CLI.md)
|
|
34
|
-
- [Schema reference](docs/SCHEMA.md)
|
|
35
|
-
- [Security policy](SECURITY.md)
|
|
36
|
-
- [Migration guide](docs/MIGRATION.md)
|
|
37
|
-
- [Release checklist](docs/RELEASE-CHECKLIST.md)
|
|
38
|
-
- [Changelog](CHANGELOG.md)
|
|
39
|
-
- [Known issues](docs/KNOWN-ISSUES.md)
|
|
40
|
-
- [Limitations](docs/LIMITATIONS.md)
|
|
41
|
-
- [V1 readiness checklist (non-binding)](docs/V1-READINESS-CHECKLIST.md)
|
|
15
|
+
```bash
|
|
16
|
+
pnpm add agent-inspect
|
|
17
|
+
```
|
|
42
18
|
|
|
43
|
-
##
|
|
19
|
+
## 60-second quickstart
|
|
44
20
|
|
|
45
|
-
|
|
21
|
+
Create `demo.mjs`:
|
|
46
22
|
|
|
47
|
-
```
|
|
23
|
+
```js
|
|
48
24
|
import { inspectRun, step } from "agent-inspect";
|
|
49
25
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
26
|
+
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
27
|
+
|
|
28
|
+
await inspectRun(
|
|
29
|
+
"support-agent",
|
|
30
|
+
async () => {
|
|
31
|
+
const plan = await step("plan", async () => {
|
|
32
|
+
await delay(50);
|
|
33
|
+
return { query: "refund policy", intent: "support" };
|
|
34
|
+
});
|
|
35
|
+
|
|
36
|
+
const docs = await step.tool("search-docs", async () => {
|
|
37
|
+
await delay(75);
|
|
38
|
+
return ["Refunds are available within 30 days."];
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
return step.llm("answer", async () => {
|
|
42
|
+
await delay(100);
|
|
43
|
+
return `Based on ${docs.length} document(s), refunds are available within 30 days.`;
|
|
44
|
+
});
|
|
45
|
+
},
|
|
46
|
+
{ traceDir: "./.agent-inspect" }
|
|
47
|
+
);
|
|
54
48
|
```
|
|
55
49
|
|
|
50
|
+
Run it, then inspect it:
|
|
51
|
+
|
|
56
52
|
```bash
|
|
57
|
-
|
|
58
|
-
npx agent-inspect
|
|
53
|
+
node demo.mjs
|
|
54
|
+
npx agent-inspect list --dir ./.agent-inspect
|
|
55
|
+
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
56
|
+
npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
|
|
59
57
|
```
|
|
60
58
|
|
|
61
|
-
|
|
59
|
+
Simplified example output:
|
|
62
60
|
|
|
63
|
-
|
|
61
|
+
```text
|
|
62
|
+
Run run_abc123 (support-agent)
|
|
63
|
+
├─ ✔ plan (50ms)
|
|
64
|
+
├─ ✔ tool:search-docs (75ms)
|
|
65
|
+
└─ ✔ llm:answer (100ms)
|
|
64
66
|
|
|
65
|
-
|
|
67
|
+
Summary:
|
|
68
|
+
Steps: 3 (0 error)
|
|
69
|
+
Duration: 225ms
|
|
70
|
+
```
|
|
66
71
|
|
|
67
|
-
|
|
68
|
-
pnpm add agent-inspect @agent-inspect/tui
|
|
72
|
+
Want a runnable demo folder? See [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
|
|
69
73
|
|
|
70
|
-
|
|
71
|
-
```
|
|
74
|
+
## Why not just console.log?
|
|
72
75
|
|
|
73
|
-
|
|
76
|
+
Console logs are great for quick values, but they’re flat. AgentInspect gives you:
|
|
74
77
|
|
|
75
|
-
|
|
78
|
+
- run grouping and local trace files
|
|
79
|
+
- explicit step boundaries (including nesting)
|
|
80
|
+
- step types (`tool:*`, `llm:*`)
|
|
81
|
+
- status + duration summaries
|
|
82
|
+
- a CLI to list/view/export/diff runs
|
|
83
|
+
- log ingestion workflows (`logs`, `tail`) when you already have structured logs
|
|
76
84
|
|
|
77
|
-
|
|
85
|
+
## Inspect existing structured logs
|
|
78
86
|
|
|
79
87
|
```bash
|
|
80
|
-
npx agent-inspect
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
88
|
+
npx agent-inspect logs ./agent.log \
|
|
89
|
+
--format json \
|
|
90
|
+
--run-id-key requestId \
|
|
91
|
+
--event-key event \
|
|
92
|
+
--timestamp-key timestamp
|
|
85
93
|
```
|
|
86
94
|
|
|
87
|
-
|
|
95
|
+
See the log-to-tree guide: [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md).
|
|
88
96
|
|
|
89
|
-
|
|
97
|
+
## When to use AgentInspect
|
|
90
98
|
|
|
91
|
-
|
|
99
|
+
Use AgentInspect when:
|
|
92
100
|
|
|
93
|
-
|
|
101
|
+
- you are building TypeScript/Node.js AI agents
|
|
102
|
+
- you want local debugging before a hosted observability setup
|
|
103
|
+
- console logs are too flat for multi-step execution
|
|
104
|
+
- you want to inspect tool calls, LLM calls, failures, and durations locally
|
|
105
|
+
- you want a lightweight CLI workflow with no account and no cloud upload
|
|
106
|
+
- you want to compare two local runs
|
|
107
|
+
- you want to turn structured logs into readable execution trees
|
|
94
108
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
109
|
+
## When not to use AgentInspect
|
|
110
|
+
|
|
111
|
+
Do not use AgentInspect as a replacement for:
|
|
112
|
+
|
|
113
|
+
- production monitoring or alerting
|
|
114
|
+
- hosted observability dashboards
|
|
115
|
+
- long-term trace storage
|
|
116
|
+
- eval dataset management
|
|
117
|
+
- prompt management
|
|
118
|
+
- cost analytics
|
|
119
|
+
- replay/fork execution
|
|
120
|
+
- vendor telemetry pipelines
|
|
121
|
+
|
|
122
|
+
AgentInspect can complement tools like LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, etc. It does not replace their production/eval/dashboard workflows.
|
|
103
123
|
|
|
104
|
-
|
|
124
|
+
## Security and privacy posture
|
|
105
125
|
|
|
106
|
-
|
|
126
|
+
- local files only by default (no upload)
|
|
127
|
+
- no vendor sinks
|
|
128
|
+
- no API keys required
|
|
129
|
+
- small root dependency footprint
|
|
130
|
+
- traces can include **user-provided metadata**; review exports before sharing
|
|
107
131
|
|
|
108
|
-
|
|
132
|
+
See `SECURITY.md`.
|
|
133
|
+
|
|
134
|
+
## Documentation
|
|
135
|
+
|
|
136
|
+
- **Getting started**: [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md)
|
|
137
|
+
- **API**: [docs/API.md](docs/API.md)
|
|
138
|
+
- **CLI**: [docs/CLI.md](docs/CLI.md)
|
|
139
|
+
- **Schema**: [docs/SCHEMA.md](docs/SCHEMA.md)
|
|
140
|
+
- **Logs**: [docs/LOGS.md](docs/LOGS.md) and [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md)
|
|
141
|
+
- **Exports**: [docs/EXPORTS.md](docs/EXPORTS.md)
|
|
142
|
+
- **Diff**: [docs/DIFF.md](docs/DIFF.md)
|
|
143
|
+
- **Adapters**: [docs/ADAPTERS.md](docs/ADAPTERS.md)
|
|
144
|
+
- **Compare with other tools**: [docs/COMPARE.md](docs/COMPARE.md)
|
|
145
|
+
- **Known issues**: [docs/KNOWN-ISSUES.md](docs/KNOWN-ISSUES.md)
|
|
146
|
+
- **Limitations**: [docs/LIMITATIONS.md](docs/LIMITATIONS.md)
|
|
147
|
+
|
|
148
|
+
Screenshots/GIFs are planned; see [docs/SCREENSHOTS.md](docs/SCREENSHOTS.md).
|
|
109
149
|
|
|
110
150
|
## Minimal API
|
|
111
151
|
|
|
@@ -278,7 +318,7 @@ await agent.run("How do I reset my password?");
|
|
|
278
318
|
|
|
279
319
|
`observe()` wraps top-level `run`, `execute`, and `invoke` methods. For internal detail, add manual `step()` calls inside the agent.
|
|
280
320
|
|
|
281
|
-
## LangChain adapter (
|
|
321
|
+
## LangChain adapter (experimental)
|
|
282
322
|
|
|
283
323
|
Install:
|
|
284
324
|
|
|
@@ -309,9 +349,9 @@ Behavior:
|
|
|
309
349
|
- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
|
|
310
350
|
- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
|
|
311
351
|
- **No** cost calculation; token fields are informational only.
|
|
312
|
-
-
|
|
352
|
+
- Events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into the manual JSONL trace format.
|
|
313
353
|
|
|
314
|
-
The API
|
|
354
|
+
The LangChain adapter API remains **experimental** even though the core AgentInspect tracing API is stable in 1.0. See [examples/08-langchain-adapter](examples/08-langchain-adapter).
|
|
315
355
|
|
|
316
356
|
## CLI
|
|
317
357
|
|
|
@@ -445,15 +485,16 @@ cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
|
|
|
445
485
|
|
|
446
486
|
## Runnable examples
|
|
447
487
|
|
|
448
|
-
The repo includes
|
|
488
|
+
The repo includes runnable examples for manual tracing, log-to-tree, and the optional LangChain adapter:
|
|
449
489
|
|
|
490
|
+
- `examples/00-quickstart-demo` — minimal install-and-try demo
|
|
450
491
|
- `examples/01-basic` — `inspectRun()` + `step()`
|
|
451
492
|
- `examples/02-nested-steps` — nested execution tree hierarchy
|
|
452
493
|
- `examples/03-parallel-steps` — `Promise.all` sibling isolation
|
|
453
494
|
- `examples/04-error-handling` — failed steps and error traces
|
|
454
495
|
- `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
|
|
455
|
-
- `examples/06-log-to-tree` —
|
|
456
|
-
- `examples/08-langchain-adapter` —
|
|
496
|
+
- `examples/06-log-to-tree` — structured log-to-tree example (`agent-inspect logs`, `tail`)
|
|
497
|
+
- `examples/08-langchain-adapter` — optional LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
|
|
457
498
|
|
|
458
499
|
Run one locally:
|
|
459
500
|
|
|
@@ -477,29 +518,6 @@ Supporting material:
|
|
|
477
518
|
|
|
478
519
|
- [examples/README.md](examples/README.md)
|
|
479
520
|
|
|
480
|
-
## Original MVP scope
|
|
481
|
-
|
|
482
|
-
Included:
|
|
483
|
-
|
|
484
|
-
- `inspectRun()`
|
|
485
|
-
- `step()`
|
|
486
|
-
- `step.llm()`
|
|
487
|
-
- `step.tool()`
|
|
488
|
-
- `observe()`
|
|
489
|
-
- JSONL traces
|
|
490
|
-
- CLI `list` and `view`
|
|
491
|
-
|
|
492
|
-
Current scope also includes:
|
|
493
|
-
|
|
494
|
-
- CLI `clean` (safe deletion with verification)
|
|
495
|
-
- CLI `logs` (structured log-to-tree)
|
|
496
|
-
- CLI `tail` (live log tailing into grouped timelines)
|
|
497
|
-
- LangChain callback adapter via `@agent-inspect/langchain`
|
|
498
|
-
- Optional TUI viewer via `@agent-inspect/tui`
|
|
499
|
-
- Standards-aligned **local** exports (`export`: Markdown, HTML, OpenInference-compatible JSON, OTLP JSON mapping)
|
|
500
|
-
- Run diff / compare (`diff`: two local traces, read-only)
|
|
501
|
-
- Canonical **fixtures** under [`fixtures/`](fixtures/README.md) plus `pnpm fixtures:check` for deterministic samples
|
|
502
|
-
|
|
503
521
|
Not included:
|
|
504
522
|
|
|
505
523
|
- Live TUI / streaming trace updates in the TUI
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
AgentInspect is a **local-first** debugging tool. It does **not** upload your traces or logs anywhere, and it does not include vendor sink integrations in the core package.
|
|
4
|
+
|
|
5
|
+
This document describes how to report vulnerabilities and how to think about data safety when using AgentInspect.
|
|
6
|
+
|
|
7
|
+
## Supported status
|
|
8
|
+
|
|
9
|
+
AgentInspect is in active development. Security fixes are accepted and prioritized based on impact and exploitability.
|
|
10
|
+
|
|
11
|
+
There is **no formal SLA**. If a fix is needed, it should land as a patch release once verified.
|
|
12
|
+
|
|
13
|
+
## Reporting vulnerabilities
|
|
14
|
+
|
|
15
|
+
- If **GitHub Security Advisories** are enabled for this repository, please report via a private advisory.
|
|
16
|
+
- Otherwise, open a GitHub issue with **minimal sensitive detail** and request a private contact path for the full report.
|
|
17
|
+
|
|
18
|
+
### What to include in a report
|
|
19
|
+
|
|
20
|
+
- A clear description of the issue and expected impact
|
|
21
|
+
- Steps to reproduce (prefer a small repro repo or minimized snippet)
|
|
22
|
+
- Affected package(s) and version(s)
|
|
23
|
+
- Platform info (Node version, OS)
|
|
24
|
+
- Any mitigations/workarounds you found
|
|
25
|
+
|
|
26
|
+
### Please do not include
|
|
27
|
+
|
|
28
|
+
- real API keys or tokens
|
|
29
|
+
- production log files
|
|
30
|
+
- customer/user PII
|
|
31
|
+
- full unredacted traces
|
|
32
|
+
|
|
33
|
+
If you need to include sample data, use synthetic placeholders (for example, `example.test` emails and fake tokens).
|
|
34
|
+
|
|
35
|
+
## Scope
|
|
36
|
+
|
|
37
|
+
In scope:
|
|
38
|
+
|
|
39
|
+
- vulnerabilities in trace/log parsing that could lead to code execution, path traversal, or data disclosure
|
|
40
|
+
- unsafe redaction defaults (obvious secrets displayed where the product promises safety)
|
|
41
|
+
- package supply-chain / dependency boundary issues that cause heavy or unsafe dependencies to be pulled into the main `agent-inspect` install
|
|
42
|
+
|
|
43
|
+
## Out of scope
|
|
44
|
+
|
|
45
|
+
- production monitoring SLAs or uptime guarantees (AgentInspect is not a production observability platform)
|
|
46
|
+
- vendor sink behavior (not implemented in core)
|
|
47
|
+
- network upload security (AgentInspect does not upload)
|
|
48
|
+
- replay sandboxing (replay is not implemented)
|
|
49
|
+
- cost calculation correctness (cost engine is not implemented)
|
|
50
|
+
|
|
51
|
+
## Data handling model (local-first)
|
|
52
|
+
|
|
53
|
+
- Manual tracing writes local JSONL files (see `docs/SCHEMA.md`).
|
|
54
|
+
- The CLI reads and renders local files.
|
|
55
|
+
- Exports generate local strings/files only (Markdown/HTML/OpenInference/OTLP JSON); they do not upload anywhere.
|
|
56
|
+
|
|
57
|
+
## Redaction expectations
|
|
58
|
+
|
|
59
|
+
AgentInspect aims to be safe-by-default for **log-derived attributes** and **exported payloads**:
|
|
60
|
+
|
|
61
|
+
- Log ingestion applies redaction to parsed attributes using configured rules (with conservative defaults).
|
|
62
|
+
- Exporters default to redacted output and bounded attribute previews.
|
|
63
|
+
|
|
64
|
+
Important limitations:
|
|
65
|
+
|
|
66
|
+
- **Manual trace metadata is user-controlled.** If you attach secrets to `inspectRun({ metadata })` or step metadata, those values may appear in local trace files and could appear in some views/exports depending on your settings and what you choose to include.
|
|
67
|
+
- Always **review exported files** before sharing them externally.
|
|
68
|
+
- Avoid committing trace directories (`.agent-inspect-runs/`) to source control.
|
|
69
|
+
|
|
70
|
+
For redaction design details, see `docs-local/architecture/REDACTION.md`.
|
|
71
|
+
|
|
72
|
+
## Dependency and security review process
|
|
73
|
+
|
|
74
|
+
- Prefer Node.js built-ins over new dependencies.
|
|
75
|
+
- Do not add vendor SDKs, OpenTelemetry SDKs, or framework dependencies to the main `agent-inspect` package.
|
|
76
|
+
- Keep optional integrations (`@agent-inspect/langchain`, `@agent-inspect/tui`) separate so users do not pull them in by default.
|
|
77
|
+
|
package/docs/ADAPTERS.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
## Adapters
|
|
2
|
+
|
|
3
|
+
AgentInspect is framework-agnostic at its core, but can optionally integrate with frameworks via adapter packages.
|
|
4
|
+
|
|
5
|
+
- **LangChain.js adapter** (optional): `@agent-inspect/langchain`
|
|
6
|
+
- Documented as **experimental** in `docs/API.md`
|
|
7
|
+
- Requires `@langchain/core` as a peer dependency
|
|
8
|
+
- **Interactive TUI** (optional): `@agent-inspect/tui`
|
|
9
|
+
- Documented as **experimental** in `docs/API.md`
|
|
10
|
+
- Intended for CLI integration; programmatic TUI APIs may change
|
|
11
|
+
|
|
12
|
+
See also:
|
|
13
|
+
- `docs/MIGRATION.md` (what changed from early versions)
|
|
14
|
+
- `docs-local/RELEASE-CHECKLIST.md` (maintainer-only release steps)
|
|
15
|
+
|
package/docs/API.md
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# API (AgentInspect 1.0)
|
|
2
|
+
|
|
3
|
+
This document describes the **public TypeScript API surface** of AgentInspect and classifies each area as **stable** or **experimental**.
|
|
4
|
+
|
|
5
|
+
AgentInspect is a **local-first execution-tree debugger**. It is not a SaaS, not a production APM, not a sink/uploader, and not a replay engine.
|
|
6
|
+
|
|
7
|
+
## 1. Stability policy
|
|
8
|
+
|
|
9
|
+
- **Stable**: intended to be compatible across v1.x. Breaking changes require v2.0.
|
|
10
|
+
- **Experimental**: available for adoption, but subject to refinement (including naming/shape changes) before a future stability declaration. Experimental APIs may change in v1.x.
|
|
11
|
+
|
|
12
|
+
Notes:
|
|
13
|
+
|
|
14
|
+
- The core guarantee of v1.0 is **stable local debugging**: manual tracing + CLI inspection.
|
|
15
|
+
- Export formats (OpenInference / OTLP JSON) are **local-only** and **compatibility-oriented**. They do **not** upload anywhere.
|
|
16
|
+
- There are **zero production sinks** in v1.0; sink/uploader APIs are not stable.
|
|
17
|
+
|
|
18
|
+
## 2. Stable core APIs (manual tracing)
|
|
19
|
+
|
|
20
|
+
These are the recommended entry points for manual instrumentation. They are designed to be dependency-light and safe-by-default.
|
|
21
|
+
|
|
22
|
+
Import from `agent-inspect`:
|
|
23
|
+
|
|
24
|
+
```ts
|
|
25
|
+
import { inspectRun, step, observe } from "agent-inspect";
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown).
|
|
29
|
+
- **`step(name, fn, options?)`**: traces a named unit of work inside `inspectRun` (`step_started` / `step_completed`).
|
|
30
|
+
- **`step.llm(model, fn)`**: convenience wrapper (`type: "llm"`, `metadata.model`).
|
|
31
|
+
- **`step.tool(toolName, fn)`**: convenience wrapper (`type: "tool"`, `metadata.toolName`).
|
|
32
|
+
- **`observe(agent, options?)`**: proxy wrapper that traces top-level `run` / `execute` / `invoke` methods via `inspectRun`.
|
|
33
|
+
|
|
34
|
+
## 3. Stable local inspection APIs
|
|
35
|
+
|
|
36
|
+
These APIs support local workflows like listing traces, extracting metadata/summaries, and safety checks for deletion.
|
|
37
|
+
|
|
38
|
+
- **`TraceDirectory`**, **`resolveTraceDir`**
|
|
39
|
+
- **`extractMetadata`**, **`buildRunSummary`**
|
|
40
|
+
- **`filterTraces`**
|
|
41
|
+
- **`isAgentInspectTrace`** (conservative trace verifier for cleanup)
|
|
42
|
+
- **`parseDuration`**, **`formatDuration`**
|
|
43
|
+
- Types: **`TraceMetadata`**, **`RunSummary`**
|
|
44
|
+
|
|
45
|
+
## 4. Stable event/config model types
|
|
46
|
+
|
|
47
|
+
### Manual trace JSONL model (stable)
|
|
48
|
+
|
|
49
|
+
- **`TraceSchemaVersion`** (`"0.1"`)
|
|
50
|
+
- **`TraceEvent`** union and specific event types:
|
|
51
|
+
- `RunStartedEvent` (`event: "run_started"`)
|
|
52
|
+
- `RunCompletedEvent` (`event: "run_completed"`)
|
|
53
|
+
- `StepStartedEvent` (`event: "step_started"`)
|
|
54
|
+
- `StepCompletedEvent` (`event: "step_completed"`)
|
|
55
|
+
- Related types: `StepType`, `StepStatus`, `RunStatus`, `ErrorInfo`, `StepMetadata`, `TokenMetadata`
|
|
56
|
+
|
|
57
|
+
### Log-derived normalized model (stable as a model, not necessarily the parsing APIs)
|
|
58
|
+
|
|
59
|
+
- **`InspectKind`**
|
|
60
|
+
- **`AttributionConfidence`**
|
|
61
|
+
- **`InspectEvent`**, **`InspectNode`**, **`InspectRunTree`**
|
|
62
|
+
- **`EventSource`**
|
|
63
|
+
|
|
64
|
+
### Log ingest config types (stable)
|
|
65
|
+
|
|
66
|
+
- **`LogIngestConfig`**
|
|
67
|
+
- **`LogEventMapping`**
|
|
68
|
+
- **`RedactionRule`**, **`RedactionStrategy`**
|
|
69
|
+
|
|
70
|
+
## 5. Experimental log parsing APIs
|
|
71
|
+
|
|
72
|
+
These are compatibility-oriented utilities for turning structured logs into normalized `InspectEvent` and grouped trees. They remain conservative: **no eval**, **no parsing JS object literals**, JSON logs first-class, log4js best-effort.
|
|
73
|
+
|
|
74
|
+
- **`parseLogsToTrees`**
|
|
75
|
+
- **`JsonLogParser`**, **`Log4jsParser`**
|
|
76
|
+
- **`EventNormalizer`**
|
|
77
|
+
- **`TreeBuilder`**
|
|
78
|
+
- **`Redactor`**
|
|
79
|
+
- **`renderRunTree`**, **`renderRunTrees`**
|
|
80
|
+
- **`parseLogLine`**
|
|
81
|
+
|
|
82
|
+
## 6. Experimental live tail APIs
|
|
83
|
+
|
|
84
|
+
The CLI `tail` workflow is supported. The programmatic accumulator is experimental.
|
|
85
|
+
|
|
86
|
+
- **`LiveLogAccumulator`**
|
|
87
|
+
|
|
88
|
+
## 7. Experimental standards export APIs
|
|
89
|
+
|
|
90
|
+
Exports are **read-only**, **local-only**, and **compatibility-oriented**. They do not upload, stream, or integrate vendor SDKs.
|
|
91
|
+
|
|
92
|
+
- **`exportRunTree`**
|
|
93
|
+
- **`exportMarkdown`**, **`exportHtml`**
|
|
94
|
+
- **`exportOpenInference`** (OpenInference-compatible JSON)
|
|
95
|
+
- **`exportOtlpJson`** (OTLP JSON, experimental until verified per backend)
|
|
96
|
+
- **`validateExport`**, `validateExportContent` (validation helpers)
|
|
97
|
+
|
|
98
|
+
## 8. Experimental diff APIs
|
|
99
|
+
|
|
100
|
+
Diff is local and read-only. Programmatic diff surfaces are experimental until the comparison semantics are explicitly frozen.
|
|
101
|
+
|
|
102
|
+
- **`diffRuns`**
|
|
103
|
+
- **`diffTraceEvents`**
|
|
104
|
+
- **`renderRunDiff`**
|
|
105
|
+
- **`manualTraceEventsToComparableRun`**
|
|
106
|
+
|
|
107
|
+
## 9. Experimental `@agent-inspect/langchain` APIs
|
|
108
|
+
|
|
109
|
+
`@agent-inspect/langchain` is an optional adapter package.
|
|
110
|
+
|
|
111
|
+
- **`AgentInspectCallback`** (experimental)
|
|
112
|
+
- Metadata helpers: `extractModelName`, `extractTokenUsage`, `safePreview`, `toPlainMetadata`
|
|
113
|
+
|
|
114
|
+
Rationale: v1.0 includes one official adapter and **zero production sinks**, so adapter surfaces remain experimental.
|
|
115
|
+
|
|
116
|
+
## 10. Experimental `@agent-inspect/tui` APIs
|
|
117
|
+
|
|
118
|
+
`@agent-inspect/tui` is an optional package. CLI integration via `agent-inspect view --tui` is supported; programmatic TUI APIs remain experimental.
|
|
119
|
+
|
|
120
|
+
- `runTraceViewer`, `loadTraceForTui`, `buildTuiTraceModel`, etc.
|
|
121
|
+
|
|
122
|
+
## 11. Deprecated APIs
|
|
123
|
+
|
|
124
|
+
No deprecated APIs are declared in v1.0 Pass 1.
|
|
125
|
+
|
|
126
|
+
## 12. Removal / deprecation policy
|
|
127
|
+
|
|
128
|
+
- Stable APIs are not removed in v1.x.
|
|
129
|
+
- If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
|
|
130
|
+
|
|
131
|
+
## 13. Backward compatibility policy
|
|
132
|
+
|
|
133
|
+
- Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
|
|
134
|
+
- Additive schema changes are allowed in minor versions.
|
|
135
|
+
- Breaking changes require a major version.
|
|
136
|
+
- Unknown fields should be ignored where safe.
|
|
137
|
+
|
|
138
|
+
## 14. Examples
|
|
139
|
+
|
|
140
|
+
### Minimal manual trace
|
|
141
|
+
|
|
142
|
+
```ts
|
|
143
|
+
import { inspectRun, step } from "agent-inspect";
|
|
144
|
+
|
|
145
|
+
await inspectRun("demo-agent", async () => {
|
|
146
|
+
const plan = await step("plan", async () => "ok");
|
|
147
|
+
const hits = await step.tool("search", async () => ({ count: 2 }));
|
|
148
|
+
const answer = await step.llm("fixture-model", async () => "done");
|
|
149
|
+
return { plan, hits, answer };
|
|
150
|
+
});
|
|
151
|
+
```
|
|
152
|
+
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
## Architecture
|
|
2
|
+
|
|
3
|
+
This doc is a stable entry point for AgentInspect’s internal architecture and product boundaries.
|
|
4
|
+
|
|
5
|
+
- **Canonical architecture overview**: see `docs-local/architecture/ARCHITECTURE.md`
|
|
6
|
+
- **Event model**: `docs-local/architecture/EVENT-MODEL.md`
|
|
7
|
+
- **Tree building rules**: `docs-local/architecture/TREE-BUILDING-RULES.md`
|
|
8
|
+
- **Schema evolution policy**: `docs-local/architecture/SCHEMA-EVOLUTION.md`
|
|
9
|
+
- **Dependency policy**: `docs-local/architecture/DEPENDENCY-POLICY.md`
|
|
10
|
+
- **Redaction design**: `docs-local/architecture/REDACTION.md`
|
|
11
|
+
|
|
12
|
+
If you’re new, start with `docs/GETTING-STARTED.md`, then `docs/API.md`, `docs/CLI.md`, and `docs/SCHEMA.md`.
|
|
13
|
+
|