agent-inspect 6.7.0 → 6.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -10
- package/README.md +148 -93
- package/docs/ADOPTION.md +1 -1
- package/docs/BUNDLES.md +67 -0
- package/docs/COMPARE.md +3 -2
- package/docs/DEMO-SCRIPT.md +6 -5
- package/docs/FIRST-TRACE-IN-5-MINUTES.md +28 -10
- package/docs/GOLDEN-PATH.md +40 -0
- package/docs/INDEX.md +69 -0
- package/docs/NETWORK-BEHAVIOR.md +32 -0
- package/docs/PRE-V7-PILOT-KIT.md +45 -0
- package/docs/SCREENSHOTS.md +7 -6
- package/docs/SELF-HOSTING.md +130 -0
- package/docs/SESSIONS-AND-OUTCOMES.md +19 -0
- package/docs/STANDARDS.md +45 -0
- package/docs/SUITES-COHORTS-GATES.md +34 -0
- package/docs/SUPPORT-LEVELS.md +47 -0
- package/docs/TRACE-CONTRACTS.md +39 -0
- package/docs/assets/readme-product-loop.svg +27 -18
- package/package.json +20 -4
- package/packages/cli/dist/index.cjs +1 -1
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 6.7.2
|
|
6
|
+
|
|
7
|
+
### Patch Changes
|
|
8
|
+
|
|
9
|
+
- 9c1f54c: Product presentation patch: rebuild README/docs/package guides/website for the 6.7.x launch candidate, fix public CLI command targets, add docs/link/public-truth validation, and slim the public roadmap. Docs and presentation only — no runtime product expansion.
|
|
10
|
+
|
|
11
|
+
## 6.7.1
|
|
12
|
+
|
|
13
|
+
### Patch Changes
|
|
14
|
+
|
|
15
|
+
- dea3d91: Release tooling and public-truth patch: Changesets fixed group for all 18 public packages, linked-versions CI check, README/ROADMAP aligned to 6.7.x launch candidate, MCP server initialize version from package metadata.
|
|
16
|
+
|
|
3
17
|
## 6.7.0
|
|
4
18
|
|
|
5
19
|
### Minor Changes
|
|
@@ -193,17 +207,15 @@
|
|
|
193
207
|
|
|
194
208
|
- 70f3fb2: v3.1 adoption train: public `@agent-inspect/harness`, `agent-inspect init` and `doctor` commands, adoption starters, and onboarding docs.
|
|
195
209
|
|
|
196
|
-
##
|
|
197
|
-
|
|
198
|
-
README adoption polish (brand header, product-loop visual, npm link hygiene). Docs-only; no runtime feature changes.
|
|
210
|
+
## Historical (pre-3.5 train notes)
|
|
199
211
|
|
|
200
|
-
|
|
212
|
+
The v3.0→v3.5 feature train is complete. Older in-progress notes below are kept for history.
|
|
201
213
|
|
|
202
|
-
|
|
214
|
+
### README adoption polish (shipped in 3.5.x docs)
|
|
203
215
|
|
|
204
|
-
|
|
216
|
+
README adoption polish (brand header, product-loop visual, npm link hygiene). Docs-only; no runtime feature changes.
|
|
205
217
|
|
|
206
|
-
|
|
218
|
+
See [docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md](docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md).
|
|
207
219
|
|
|
208
220
|
### v3.1 (shipped in 3.1.0)
|
|
209
221
|
|
|
@@ -257,9 +269,9 @@ The v3.0→v3.5 feature train is complete. Older in-progress notes below are kep
|
|
|
257
269
|
|
|
258
270
|
Adds the public optional `@agent-inspect/vitest` and `@agent-inspect/jest` reporter packages, the shared experimental `agent-inspect/reporters` helpers, and the `agent-inspect ci-summary` workflow for deterministic local reporter manifests and CI artifacts.
|
|
259
271
|
|
|
260
|
-
|
|
272
|
+
### Draft notes (superseded by 2.3.0 / 2.4.0 releases)
|
|
261
273
|
|
|
262
|
-
|
|
274
|
+
#### Draft v2.4.0 Notes
|
|
263
275
|
|
|
264
276
|
- Added session/workflow causality model and `agent-inspect/advanced` session index helpers (`buildSessionIndex`, scope/cohort helpers, session fixtures).
|
|
265
277
|
- Added `agent-inspect sessions` and `agent-inspect session` CLI for multi-run handoff/retry navigation with timeline, critical-path, diagnostics, and JSON output.
|
|
@@ -267,7 +279,7 @@ The v3.0→v3.5 feature train is complete. Older in-progress notes below are kep
|
|
|
267
279
|
- Added public optional `@agent-inspect/mcp` for local MCP **client** `tools/list` and `tools/call` telemetry with bounded summaries and `source.type: mcp-client` metadata.
|
|
268
280
|
- No schema version change, no MCP gateway/server, no timestamp-only causality inference, and no root/core dependency on MCP SDKs.
|
|
269
281
|
|
|
270
|
-
|
|
282
|
+
#### Draft v2.3.0 Notes
|
|
271
283
|
|
|
272
284
|
- Hardened the official adapter paths for AI SDK, OpenAI Agents JS, and LangChain/LangGraph with no-network recipes, local-only defaults, clearer lifecycle coverage, and executable adapter conformance evidence.
|
|
273
285
|
- AI SDK coverage now includes route-style telemetry factory guidance, per-request integration isolation, tool/stream/error/parallel fixtures, token metadata, and the required `recordInputs: false` / `recordOutputs: false` host settings.
|
package/README.md
CHANGED
|
@@ -8,11 +8,16 @@
|
|
|
8
8
|
<h1 align="center">agent-inspect</h1>
|
|
9
9
|
|
|
10
10
|
<p align="center">
|
|
11
|
-
<strong>
|
|
11
|
+
<strong>Debug, regression-test, and safely share TypeScript AI-agent behavior—locally.</strong>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
14
|
<p align="center">
|
|
15
|
-
<sub>No account · no upload ·
|
|
15
|
+
<sub>No account · no default upload · metadata-only by default · optional customer-owned Studio</sub>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
AgentInspect turns agent runs into <strong>customer-owned evidence</strong>: execution trees,
|
|
20
|
+
deterministic contracts, CI gates, verified-safe bundles, and optional self-hosted review.
|
|
16
21
|
</p>
|
|
17
22
|
|
|
18
23
|
<p align="center">
|
|
@@ -29,141 +34,191 @@
|
|
|
29
34
|
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178c6" alt="TypeScript"></a>
|
|
30
35
|
</p>
|
|
31
36
|
|
|
32
|
-
agent-inspect turns AI-agent runs into readable **local execution trees**: framework events, observed objects, tool calls, LLM steps, retries, failures, timings, sessions, and CI artifacts — without an account, collector, or hosted dashboard.
|
|
33
|
-
|
|
34
|
-
**Default loop:** capture locally → inspect / report / diff → check in CI → redact before sharing.
|
|
35
|
-
|
|
36
37
|
```bash
|
|
37
38
|
npm install agent-inspect
|
|
38
39
|
```
|
|
39
40
|
|
|
40
|
-
##
|
|
41
|
+
## Three workflows
|
|
42
|
+
|
|
43
|
+
| Workflow | What you do |
|
|
44
|
+
| -------- | ----------- |
|
|
45
|
+
| **Debug one run** | Capture/import → tree / timeline / report → first causal failure |
|
|
46
|
+
| **Prevent one regression** | TraceContract / `check` → suite / cohort → CI gate |
|
|
47
|
+
| **Share one safe artifact** | Redact → `verify-safe` → offline bundle → optional Studio review |
|
|
48
|
+
|
|
49
|
+
**Review as a team (optional):** workspace + optional SQLite index → customer-owned [Studio Beta](https://github.com/rajudandigam/agent-inspect/tree/main/packages/studio) (no AgentInspect-hosted cloud).
|
|
50
|
+
|
|
51
|
+
<p align="center">
|
|
52
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture or import → understand → enforce → verify and bundle → review locally or in customer-owned Studio" width="900">
|
|
53
|
+
</p>
|
|
54
|
+
|
|
55
|
+
## Five-minute path
|
|
56
|
+
|
|
57
|
+
Commands below match the packed quickstart. Replace `<run-id>` with a value from `list`.
|
|
41
58
|
|
|
42
59
|
```bash
|
|
60
|
+
npm install agent-inspect
|
|
43
61
|
npx agent-inspect init --yes
|
|
44
62
|
node examples/agent-inspect-demo.mjs
|
|
45
63
|
npx agent-inspect list --dir .agent-inspect
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
# After copying a run id from list:
|
|
46
68
|
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
47
69
|
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
70
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
71
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
48
72
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
49
73
|
```
|
|
50
74
|
|
|
51
|
-
|
|
52
|
-
Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](docs/FIRST-TRACE-IN-5-MINUTES.md)
|
|
75
|
+
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md)
|
|
53
76
|
|
|
54
|
-
|
|
55
|
-
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture, inspect, check, redact — local JSONL only" width="720">
|
|
56
|
-
</p>
|
|
57
|
-
|
|
58
|
-
## Choose your path
|
|
77
|
+
## Choose your capture path
|
|
59
78
|
|
|
60
79
|
| Path | Use when | Start |
|
|
61
80
|
| ---- | -------- | ----- |
|
|
62
|
-
| **
|
|
63
|
-
| **
|
|
64
|
-
| **
|
|
65
|
-
| **
|
|
66
|
-
| **
|
|
67
|
-
| **
|
|
68
|
-
| **CI
|
|
69
|
-
| **
|
|
81
|
+
| **Manual / observe** | Custom nesting or object methods | [Getting started](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GETTING-STARTED.md) |
|
|
82
|
+
| **AI SDK** | Vercel AI SDK `generateText` / `streamText` | [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) |
|
|
83
|
+
| **OpenAI Agents** | OpenAI Agents JS | [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) |
|
|
84
|
+
| **LangChain** | Callbacks / LangGraph-via-LangChain | [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) |
|
|
85
|
+
| **Structured logs** | Logs already emitted | [Log-to-tree](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
86
|
+
| **Harness** | Fixture runner for real projects | [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) |
|
|
87
|
+
| **CI reporters** | Failed-test artifacts | [`vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) · [`jest`](https://www.npmjs.com/package/@agent-inspect/jest) |
|
|
88
|
+
| **Standards files** | OpenInference / OTLP JSON | [Standards](https://github.com/rajudandigam/agent-inspect/blob/main/docs/STANDARDS.md) |
|
|
70
89
|
|
|
71
90
|
Blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
72
91
|
|
|
73
|
-
## What
|
|
92
|
+
## What you can do after capture
|
|
93
|
+
|
|
94
|
+
**Understand** — `what` / `view` / tree · timeline · report · diff · sessions / activity
|
|
95
|
+
|
|
96
|
+
**Prevent regressions** — deterministic checks · typed TraceContract (Beta) · suites · cohorts · CI gates · Vitest/Jest reporters (artifact reporters today; TraceContract matchers not shipped)
|
|
97
|
+
|
|
98
|
+
**Share safely** — redaction profiles · `scan` · `verify-safe` · offline bundles · CI artifacts
|
|
99
|
+
|
|
100
|
+
**Scale locally** — workspace · optional SQLite index (Beta) · observed outcomes · viewer / TUI / VS Code
|
|
74
101
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
- **Multi-agent / sessions** — `sessions`, `search`, handoff metadata
|
|
81
|
-
- **MCP tool calls** — [`@agent-inspect/mcp`](packages/mcp/README.md) client tracing
|
|
82
|
-
- **VS Code review** — in-repo extension ([dev guide](docs/VSCODE.md); Marketplace not published yet)
|
|
83
|
-
- **Existing logs** — parse structured logs when you cannot instrument
|
|
102
|
+
**Review as a team** — customer-owned Studio Beta · explicit file / GitHub / HTTP ingest (disabled by default) · no maintainer cloud
|
|
103
|
+
|
|
104
|
+
**Interoperate** — read-only MCP server (Preview) · OpenInference-compatible / OTLP GenAI-aligned mapping with known-loss reporting
|
|
105
|
+
|
|
106
|
+
Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md) · Network: [NETWORK-BEHAVIOR.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md)
|
|
84
107
|
|
|
85
108
|
## Real-world scenarios
|
|
86
109
|
|
|
87
|
-
| Scenario |
|
|
88
|
-
| -------- |
|
|
89
|
-
|
|
|
90
|
-
| CI
|
|
91
|
-
|
|
|
92
|
-
|
|
|
93
|
-
|
|
|
94
|
-
| Design partner trial | [
|
|
110
|
+
| Scenario | Start |
|
|
111
|
+
| -------- | ----- |
|
|
112
|
+
| Wrong tool call | [broken-agent starter](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
113
|
+
| CI trajectory gate | [CI artifacts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CI-ARTIFACTS.md) |
|
|
114
|
+
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
115
|
+
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
116
|
+
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio |
|
|
117
|
+
| Design partner trial | [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) |
|
|
118
|
+
|
|
119
|
+
## Safety and network behavior
|
|
120
|
+
|
|
121
|
+
- Traces are **local JSONL** under `.agent-inspect/` (or `AGENT_INSPECT_TRACE_DIR`)
|
|
122
|
+
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
123
|
+
- **No hidden upload** — core does not send traces to AgentInspect
|
|
124
|
+
- **Customer-owned Studio ingestion** is disabled by default and explicit when enabled
|
|
125
|
+
- **MCP server** exposes configured local evidence to the connected client (Preview)
|
|
126
|
+
- **Standards export** only when you run/configure it
|
|
127
|
+
- Redaction is **best-effort**, not certification — review before posting
|
|
128
|
+
- **Not** a chain-of-thought recorder
|
|
129
|
+
|
|
130
|
+
Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) · [Network behavior](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md) · [Security](SECURITY.md)
|
|
131
|
+
|
|
132
|
+
## Project status
|
|
133
|
+
|
|
134
|
+
**Current release:** **6.7.2** (eighteen linked npm packages). Technical launch candidate; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
135
|
+
|
|
136
|
+
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
137
|
+
|
|
138
|
+
## What AgentInspect is not
|
|
95
139
|
|
|
96
|
-
|
|
140
|
+
- Hosted SaaS or maintainer-hosted dashboard
|
|
141
|
+
- Production APM replacement
|
|
142
|
+
- Eval dataset platform or LLM-as-judge by default
|
|
143
|
+
- Prompt registry, pricing engine, or replay engine
|
|
144
|
+
- Universal standards exporter or completed external adoption proof
|
|
145
|
+
|
|
146
|
+
See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPARE.md).
|
|
147
|
+
|
|
148
|
+
<details>
|
|
149
|
+
<summary><strong>Package family (18 public packages)</strong></summary>
|
|
150
|
+
|
|
151
|
+
### Framework adapters
|
|
97
152
|
|
|
98
153
|
| Package | Purpose |
|
|
99
154
|
| ------- | ------- |
|
|
100
155
|
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI |
|
|
101
|
-
| [`@agent-inspect/ai-sdk`](
|
|
102
|
-
| [`@agent-inspect/openai-agents`](
|
|
103
|
-
| [`@agent-inspect/langchain`](
|
|
104
|
-
| [`@agent-inspect/harness`](packages/harness/README.md) | Fixture runner for real projects |
|
|
105
|
-
| [`@agent-inspect/redact`](packages/redact/README.md) | Deterministic redaction |
|
|
106
|
-
| [`@agent-inspect/eval`](packages/eval/README.md) | Local eval heuristics |
|
|
107
|
-
| [`@agent-inspect/vitest`](packages/vitest/README.md) | Vitest reporter |
|
|
108
|
-
| [`@agent-inspect/jest`](packages/jest/README.md) | Jest reporter |
|
|
109
|
-
| [`@agent-inspect/mcp`](packages/mcp/README.md) | MCP client tracing |
|
|
110
|
-
| [`@agent-inspect/mcp-server`](packages/mcp-server/README.md) | Read-only trace MCP server |
|
|
111
|
-
| [`@agent-inspect/guardrails`](packages/guardrails/README.md) | Deterministic guardrail rules |
|
|
112
|
-
| [`@agent-inspect/circuit`](packages/circuit/README.md) | Loop / retry / timeout analyzers |
|
|
113
|
-
| [`@agent-inspect/viewer`](packages/viewer/README.md) | Localhost viewer |
|
|
114
|
-
| [`@agent-inspect/adapter-sdk`](packages/adapter-sdk/README.md) | Third-party adapters |
|
|
115
|
-
| [`@agent-inspect/tui`](packages/tui/README.md) | Optional terminal UI |
|
|
116
|
-
| `agent-inspect-vscode` | VS Code extension (in-repo; not on Marketplace yet) |
|
|
117
|
-
|
|
118
|
-
## Safety model
|
|
156
|
+
| [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) | AI SDK telemetry |
|
|
157
|
+
| [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) | OpenAI Agents processor |
|
|
158
|
+
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain callbacks |
|
|
119
159
|
|
|
120
|
-
|
|
121
|
-
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
122
|
-
- **No hidden upload** — AgentInspect does not send traces to the cloud
|
|
123
|
-
- **Redaction profiles** — `local` / `share` / `strict` via CLI or [`@agent-inspect/redact`](packages/redact/README.md)
|
|
124
|
-
- **`scan` / `verify-safe`** — check artifacts before sharing
|
|
125
|
-
- **Not** a chain-of-thought recorder or compliance engine — review exports before posting
|
|
160
|
+
### Testing / evaluation
|
|
126
161
|
|
|
127
|
-
|
|
162
|
+
| Package | Purpose |
|
|
163
|
+
| ------- | ------- |
|
|
164
|
+
| [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) | Fixture runner |
|
|
165
|
+
| [`@agent-inspect/eval`](https://www.npmjs.com/package/@agent-inspect/eval) | Local eval heuristics |
|
|
166
|
+
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter |
|
|
167
|
+
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter |
|
|
128
168
|
|
|
129
|
-
|
|
169
|
+
### Safety
|
|
130
170
|
|
|
131
|
-
|
|
|
132
|
-
|
|
|
133
|
-
|
|
|
134
|
-
|
|
|
135
|
-
|
|
|
136
|
-
| API / CLI | — | [API.md](docs/API.md) · [CLI.md](docs/CLI.md) |
|
|
137
|
-
| Adoption | — | [ADOPTION.md](docs/ADOPTION.md) · [USE-CASES.md](docs/USE-CASES.md) |
|
|
138
|
-
| Technical guide | — | [TECHNICAL-GUIDE.md](docs/TECHNICAL-GUIDE.md) |
|
|
139
|
-
| Examples | — | [starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters) |
|
|
140
|
-
| Visual demos | — | [SCREENSHOTS.md](docs/SCREENSHOTS.md) |
|
|
171
|
+
| Package | Purpose |
|
|
172
|
+
| ------- | ------- |
|
|
173
|
+
| [`@agent-inspect/redact`](https://www.npmjs.com/package/@agent-inspect/redact) | Deterministic redaction |
|
|
174
|
+
| [`@agent-inspect/guardrails`](https://www.npmjs.com/package/@agent-inspect/guardrails) | Deterministic guardrail rules |
|
|
175
|
+
| [`@agent-inspect/circuit`](https://www.npmjs.com/package/@agent-inspect/circuit) | Loop / retry / timeout analyzers |
|
|
141
176
|
|
|
142
|
-
|
|
177
|
+
### Developer surfaces
|
|
143
178
|
|
|
144
|
-
|
|
179
|
+
| Package | Purpose |
|
|
180
|
+
| ------- | ------- |
|
|
181
|
+
| [`@agent-inspect/viewer`](https://www.npmjs.com/package/@agent-inspect/viewer) | Localhost viewer |
|
|
182
|
+
| [`@agent-inspect/tui`](https://www.npmjs.com/package/@agent-inspect/tui) | Optional terminal UI |
|
|
183
|
+
| [`@agent-inspect/mcp`](https://www.npmjs.com/package/@agent-inspect/mcp) | MCP client tracing |
|
|
184
|
+
| [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) | Read-only MCP server (Preview) |
|
|
185
|
+
|
|
186
|
+
### Team / self-hosted
|
|
187
|
+
|
|
188
|
+
| Package | Purpose |
|
|
189
|
+
| ------- | ------- |
|
|
190
|
+
| [`@agent-inspect/index-sqlite`](https://www.npmjs.com/package/@agent-inspect/index-sqlite) | Optional SQLite index (Beta) |
|
|
191
|
+
| [`@agent-inspect/studio`](https://www.npmjs.com/package/@agent-inspect/studio) | Customer-owned Studio (Beta) |
|
|
145
192
|
|
|
146
|
-
|
|
147
|
-
- Production APM replacement (use LangSmith, Langfuse, OTel, etc. alongside)
|
|
148
|
-
- Eval dataset platform or LLM-as-judge service
|
|
149
|
-
- Prompt registry or provider pricing engine
|
|
150
|
-
- Default telemetry uploader or replay engine
|
|
193
|
+
### Extension / interop
|
|
151
194
|
|
|
152
|
-
|
|
195
|
+
| Package | Purpose |
|
|
196
|
+
| ------- | ------- |
|
|
197
|
+
| [`@agent-inspect/adapter-sdk`](https://www.npmjs.com/package/@agent-inspect/adapter-sdk) | Third-party adapters (Beta) |
|
|
153
198
|
|
|
154
|
-
|
|
199
|
+
`agent-inspect-vscode` is in-repo (Marketplace not published yet).
|
|
155
200
|
|
|
156
|
-
|
|
201
|
+
</details>
|
|
157
202
|
|
|
158
|
-
|
|
159
|
-
pnpm add agent-inspect
|
|
160
|
-
npx agent-inspect doctor
|
|
161
|
-
```
|
|
203
|
+
## Documentation
|
|
162
204
|
|
|
163
|
-
|
|
205
|
+
| | Website | Repo |
|
|
206
|
+
| - | ------- | ---- |
|
|
207
|
+
| Getting started | [docs](https://agentinspect.vercel.app/docs/getting-started/) | [FIRST-TRACE](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) |
|
|
208
|
+
| Contracts / suites / gates | — | [TRACE-CONTRACTS](https://github.com/rajudandigam/agent-inspect/blob/main/docs/TRACE-CONTRACTS.md) · [SUITES-COHORTS-GATES](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUITES-COHORTS-GATES.md) |
|
|
209
|
+
| Safe sharing | [safe-sharing](https://agentinspect.vercel.app/docs/safe-sharing/) | [SAFE-TRACE-SHARING](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
210
|
+
| API / CLI | — | [API](docs/API.md) · [CLI](docs/CLI.md) (packed with npm) |
|
|
211
|
+
| Full index | — | [docs/README.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/README.md) |
|
|
164
212
|
|
|
165
213
|
## Contributing
|
|
166
214
|
|
|
167
|
-
[CONTRIBUTING.md](CONTRIBUTING.md) · [Good first issues](GOOD-FIRST-ISSUES.md) · [Discussions](https://github.com/rajudandigam/agent-inspect/discussions)
|
|
215
|
+
[CONTRIBUTING.md](https://github.com/rajudandigam/agent-inspect/blob/main/CONTRIBUTING.md) · [Good first issues](https://github.com/rajudandigam/agent-inspect/blob/main/GOOD-FIRST-ISSUES.md) · [Discussions](https://github.com/rajudandigam/agent-inspect/discussions)
|
|
168
216
|
|
|
169
217
|
**Redact traces before posting issues or PRs.**
|
|
218
|
+
|
|
219
|
+
```bash
|
|
220
|
+
pnpm add agent-inspect
|
|
221
|
+
npx agent-inspect doctor
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Monorepo: `pnpm install && pnpm build && pnpm test`
|
package/docs/ADOPTION.md
CHANGED
|
@@ -20,7 +20,7 @@ npx agent-inspect init --yes
|
|
|
20
20
|
node examples/agent-inspect-demo.mjs
|
|
21
21
|
npx agent-inspect list --dir .agent-inspect
|
|
22
22
|
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
23
|
-
npx agent-inspect verify-safe --dir .agent-inspect
|
|
23
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
No API keys required for the init demo. See [examples/starters/](../examples/starters/README.md).
|
package/docs/BUNDLES.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Shareable trace bundles
|
|
2
|
+
|
|
3
|
+
AgentInspect **bundles** are local, offline folders you can attach to PRs, incident threads, or internal reviews. They combine redacted trace copies, HTML reports, safety check results, and a human summary — without mutating source traces or calling the network.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
# Single run (default share profile + automatic verify-safe)
|
|
9
|
+
npx agent-inspect bundle <runId> --dir ./.agent-inspect
|
|
10
|
+
|
|
11
|
+
# Session or time window
|
|
12
|
+
npx agent-inspect bundle --session <sessionId> --dir ./.agent-inspect
|
|
13
|
+
npx agent-inspect bundle --since 24h --profile strict --dir ./.agent-inspect
|
|
14
|
+
|
|
15
|
+
# Explicit output folder (.zip suffix is stripped — folder-first MVP)
|
|
16
|
+
npx agent-inspect bundle <runId> --out ./my-bundle --json
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
When a workspace exists, default output goes under `.agent-inspect/bundles/`.
|
|
20
|
+
|
|
21
|
+
## Bundle layout
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
trace.html # offline HTML (index for multi-run)
|
|
25
|
+
trace.jsonl # redacted JSONL copy
|
|
26
|
+
summary.md # human overview
|
|
27
|
+
metadata.json # manifest (version, profile, safe status)
|
|
28
|
+
check-results.json # verify-safe results per run
|
|
29
|
+
redaction-report.json # detector summary (no secret values)
|
|
30
|
+
eval-results.json # placeholder unless eval artifacts are added later
|
|
31
|
+
performance-summary.json # placeholder unless perf artifacts are added later
|
|
32
|
+
assets/runs/<runId>.* # per-run HTML + JSONL mirrors
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Safety defaults
|
|
36
|
+
|
|
37
|
+
| Setting | Default |
|
|
38
|
+
| ------- | ------- |
|
|
39
|
+
| Redaction profile | `share` |
|
|
40
|
+
| verify-safe | Runs automatically before write |
|
|
41
|
+
| UNSAFE traces | Command fails unless `--allow-unsafe` |
|
|
42
|
+
| Source traces | Read-only; never modified |
|
|
43
|
+
|
|
44
|
+
Profiles:
|
|
45
|
+
|
|
46
|
+
- **`share`** — PR / internal support (default)
|
|
47
|
+
- **`strict`** — external or public sharing
|
|
48
|
+
- **`local`** — minimal redaction for local archives only
|
|
49
|
+
|
|
50
|
+
## When to use bundles vs other commands
|
|
51
|
+
|
|
52
|
+
| Need | Command |
|
|
53
|
+
| ---- | ------- |
|
|
54
|
+
| One redacted file | `redact` |
|
|
55
|
+
| Safety scan only | `verify-safe` |
|
|
56
|
+
| CI artifact set | `artifacts` |
|
|
57
|
+
| PR-ready evidence folder | **`bundle`** |
|
|
58
|
+
|
|
59
|
+
## Review before sharing
|
|
60
|
+
|
|
61
|
+
Bundles are **derived copies**, not compliance certification. Always review `summary.md`, `check-results.json`, and `trace.html` before attaching to tickets or PRs.
|
|
62
|
+
|
|
63
|
+
See also [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) and [CLI.md §6.24](./CLI.md#624-bundle).
|
|
64
|
+
|
|
65
|
+
## RFC
|
|
66
|
+
|
|
67
|
+
Design details: [proposals/SHAREABLE-BUNDLES-V4.3.md](./proposals/SHAREABLE-BUNDLES-V4.3.md).
|
package/docs/COMPARE.md
CHANGED
|
@@ -10,11 +10,12 @@ AgentInspect is a local-first trace workbench for TypeScript AI agents: capture
|
|
|
10
10
|
| --------- | ------------ | --------------------------------------- | --------------------------- | ------------------------------- |
|
|
11
11
|
| Where traces live | Local JSONL on your disk | Vendor or self-hosted backend | Vendor backend | Your collector + backend |
|
|
12
12
|
| Account / setup | None; `npm install` + CLI | Account or deployment | Account | SDK + collector + backend |
|
|
13
|
-
| Primary surface | CLI (`view`, `report`, `diff`, `check
|
|
13
|
+
| Primary surface | CLI (`view`, `report`, `diff`, `check`, suites/gates), local viewer, optional Studio Beta | Web dashboards | Web dashboards, eval UI | Dashboards, alerting |
|
|
14
14
|
| Evals | Deterministic local heuristics and CI gates (`check`, `eval`, `circuit`, `guardrails`) | Platform evals | Datasets, scoring, regressions at scale | Not the focus |
|
|
15
15
|
| Retention / fleet view | Not the goal | Strong | Strong | Strong |
|
|
16
16
|
| Data sharing | Explicit `redact` + `scan` / `verify-safe` before you share a file | Team access controls | Team access controls | Org pipelines |
|
|
17
|
-
| Network behavior | No upload;
|
|
17
|
+
| Network behavior | No default upload; explicit customer-owned ingest only | Uploads traces by design | Uploads by design | Ships telemetry by design |
|
|
18
|
+
| Team review | Optional customer-owned Studio Beta (not maintainer-hosted) | Strong | Strong | Strong |
|
|
18
19
|
|
|
19
20
|
If you need hosted retention, fleet dashboards, dataset management, or org-wide pipelines, use one of those platforms alongside AgentInspect. Boundaries are listed in [LIMITATIONS.md](./LIMITATIONS.md); concrete inner-loop workflows in [USE-CASES.md](./USE-CASES.md).
|
|
20
21
|
|
package/docs/DEMO-SCRIPT.md
CHANGED
|
@@ -42,19 +42,20 @@ Optional: `npx agent-inspect serve --dir .agent-inspect` for browser viewer.
|
|
|
42
42
|
## Beat 4 — Verify (30s)
|
|
43
43
|
|
|
44
44
|
```bash
|
|
45
|
-
npx agent-inspect check .agent-inspect
|
|
45
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
-
|
|
48
|
+
Optional flags: `--require-completed`, `--detect-stalls`, `--max-step-duration 30s`.
|
|
49
49
|
|
|
50
50
|
## Beat 5 — Share safely (30s)
|
|
51
51
|
|
|
52
52
|
```bash
|
|
53
|
-
npx agent-inspect redact .agent-inspect
|
|
54
|
-
npx agent-inspect verify-safe
|
|
53
|
+
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl
|
|
54
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
55
|
+
# or: npx agent-inspect verify-safe safe.jsonl
|
|
55
56
|
```
|
|
56
57
|
|
|
57
|
-
"Redacted
|
|
58
|
+
"Redacted / verified-safe artifacts are safe to attach to a GitHub issue or Slack."
|
|
58
59
|
|
|
59
60
|
## Beat 6 — Fix and diff (optional, 30s)
|
|
60
61
|
|
|
@@ -1,13 +1,23 @@
|
|
|
1
1
|
# First trace in 5 minutes
|
|
2
2
|
|
|
3
|
-
Goal: install → one trace → one check → one share-safe
|
|
3
|
+
Goal: install → one trace → one check → one share-safe bundle.
|
|
4
4
|
|
|
5
5
|
**Docs site:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
|
|
6
6
|
|
|
7
7
|
```bash
|
|
8
|
-
npm install agent-inspect
|
|
9
|
-
npx agent-inspect
|
|
10
|
-
|
|
8
|
+
npm install agent-inspect
|
|
9
|
+
npx agent-inspect init --yes
|
|
10
|
+
node examples/agent-inspect-demo.mjs
|
|
11
|
+
npx agent-inspect list --dir .agent-inspect
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Copy a `<run-id>` from `list`, then:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
18
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
19
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
20
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
11
21
|
```
|
|
12
22
|
|
|
13
23
|
## Minutes 0–1: Install
|
|
@@ -18,7 +28,8 @@ npm install agent-inspect
|
|
|
18
28
|
npx agent-inspect init --yes
|
|
19
29
|
```
|
|
20
30
|
|
|
21
|
-
Creates `agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`.
|
|
31
|
+
Creates `agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`.
|
|
32
|
+
`init` scaffolds files; it does **not** write a trace by itself.
|
|
22
33
|
|
|
23
34
|
## Minutes 1–2: Run
|
|
24
35
|
|
|
@@ -39,17 +50,23 @@ npx agent-inspect report <run-id> --dir .agent-inspect
|
|
|
39
50
|
## Minutes 3–4: Check
|
|
40
51
|
|
|
41
52
|
```bash
|
|
42
|
-
npx agent-inspect check .agent-inspect
|
|
53
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
43
54
|
```
|
|
44
55
|
|
|
45
56
|
## Minutes 4–5: Share-safe artifact
|
|
46
57
|
|
|
47
58
|
```bash
|
|
48
|
-
npx agent-inspect
|
|
49
|
-
npx agent-inspect verify-safe --dir .agent-inspect
|
|
59
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
60
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
50
61
|
```
|
|
51
62
|
|
|
52
|
-
Attach redacted
|
|
63
|
+
Attach the share-profile bundle (or a redacted file) to a PR or issue — not raw traces.
|
|
64
|
+
|
|
65
|
+
Optional file redaction:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redacted.jsonl
|
|
69
|
+
```
|
|
53
70
|
|
|
54
71
|
## Next steps
|
|
55
72
|
|
|
@@ -58,7 +75,8 @@ Attach redacted JSONL to a PR or issue — not raw traces.
|
|
|
58
75
|
| AI SDK | [AI SDK adoption](./AI-SDK-ADOPTION.md) |
|
|
59
76
|
| OpenAI Agents | [OpenAI Agents local](./OPENAI-AGENTS-LOCAL.md) |
|
|
60
77
|
| LangChain | [Adapters](./ADAPTERS.md) |
|
|
61
|
-
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) |
|
|
78
|
+
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) · [Suites / gates](./SUITES-COHORTS-GATES.md) |
|
|
79
|
+
| Golden path | [GOLDEN-PATH.md](./GOLDEN-PATH.md) |
|
|
62
80
|
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
63
81
|
|
|
64
82
|
Full index: [docs/README.md](./README.md) · Website docs: [agentinspect.vercel.app/docs](https://agentinspect.vercel.app/docs/)
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Golden path (technical launch candidate)
|
|
2
|
+
|
|
3
|
+
The 6.7.x technical launch candidate path. Be precise about what is automated today.
|
|
4
|
+
|
|
5
|
+
## Automated packed path (CI)
|
|
6
|
+
|
|
7
|
+
From a packed `agent-inspect` install (`scripts/packed-quickstart-e2e.mjs` via `pnpm pack:smoke`):
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
init --yes → demo → list → verify-safe <runId> --dir .agent-inspect
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
This proves install, scaffold, one local run, and safety verification.
|
|
14
|
+
|
|
15
|
+
## Recommended developer path (manual)
|
|
16
|
+
|
|
17
|
+
```text
|
|
18
|
+
init → demo → list → view/report → check → bundle --profile share → verify-safe
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Use required positional targets (`<run-id>` or file path). See the root README five-minute path.
|
|
22
|
+
|
|
23
|
+
## Technical-but-not-fully-automated
|
|
24
|
+
|
|
25
|
+
`scripts/golden-path-e2e.mjs` covers a subset (init → inspect → verify-safe → report). It does **not** yet automate the full roadmap chain:
|
|
26
|
+
|
|
27
|
+
```text
|
|
28
|
+
broken → contract fail → suite → CI gate → fix → diff/cohort → Studio review
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## External pilot step
|
|
32
|
+
|
|
33
|
+
Three external teams + design-partner Studio trial remain **pending**. Do not mark launch adoption complete without rows in [implementation/PRE-V7-ADOPTION-EVIDENCE.md](./implementation/PRE-V7-ADOPTION-EVIDENCE.md). Pilot kit: [PRE-V7-PILOT-KIT.md](./PRE-V7-PILOT-KIT.md).
|
|
34
|
+
|
|
35
|
+
## Pending product gaps (honest)
|
|
36
|
+
|
|
37
|
+
- TraceContract matchers / full contract depth
|
|
38
|
+
- Studio HTML productization for some pages
|
|
39
|
+
- Cross-platform consumer matrix evidence
|
|
40
|
+
- External Collector/Phoenix verification
|