agent-inspect 6.11.0 → 6.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +59 -20
- package/docs/ADOPTION.md +5 -1
- package/docs/COMPARE.md +22 -0
- package/docs/DEMO-SCRIPT.md +30 -13
- package/docs/DESIGN-PARTNER-GUIDE.md +23 -10
- package/docs/FIRST-TRACE-IN-5-MINUTES.md +5 -2
- package/docs/GOLDEN-PATH.md +2 -2
- package/docs/PRE-V7-PILOT-KIT.md +41 -15
- package/docs/SUPPORT-LEVELS.md +1 -1
- package/docs/TEAM-WORKFLOWS.md +1 -1
- package/docs/TECHNICAL-GUIDE.md +2 -2
- package/docs/USE-CASES.md +34 -10
- package/package.json +1 -1
- package/packages/cli/dist/index.cjs +1 -1
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1 -1
- package/packages/cli/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.12.1
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 2a53751: Example-heavy presentation patch: align website/README/starters/docs with the 6.12 hero flow (causal failure, MCP coding-agent loop, Evidence v2), fix incomplete redact/verify-safe command examples, and extend docs:commands to starters. Docs and presentation only — no runtime product expansion.
|
|
8
|
+
|
|
9
|
+
## 6.12.0
|
|
10
|
+
|
|
11
|
+
### Minor Changes
|
|
12
|
+
|
|
13
|
+
- 3ee1692: Consolidation and stable launch candidate: positioning/portfolio tiers, install kits, honest packed/native/MCP matrices, PARTIAL design-partner trial worksheets, package maintenance audit (keep fixed group through v6), comparison/interop handoff, and launch demo checklist. No schema break; no new packages; no default upload; trial results not fabricated.
|
|
14
|
+
|
|
3
15
|
## 6.11.0
|
|
4
16
|
|
|
5
17
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -8,18 +8,13 @@
|
|
|
8
8
|
<h1 align="center">agent-inspect</h1>
|
|
9
9
|
|
|
10
10
|
<p align="center">
|
|
11
|
-
<strong>
|
|
11
|
+
<strong>The local evidence debugger for TypeScript agents—faithful execution trees, deterministic regression checks, share-checked evidence, and coding-agent access without a collector or account.</strong>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
14
|
<p align="center">
|
|
15
15
|
<sub>No account · no default upload · metadata-only by default · optional customer-owned Studio</sub>
|
|
16
16
|
</p>
|
|
17
17
|
|
|
18
|
-
<p align="center">
|
|
19
|
-
AgentInspect turns agent runs into <strong>customer-owned evidence</strong>: execution trees,
|
|
20
|
-
deterministic contracts, CI gates, verified-safe bundles, and optional self-hosted review.
|
|
21
|
-
</p>
|
|
22
|
-
|
|
23
18
|
<p align="center">
|
|
24
19
|
<a href="https://agentinspect.vercel.app/">Website</a> ·
|
|
25
20
|
<a href="https://agentinspect.vercel.app/docs/">Docs</a> ·
|
|
@@ -38,6 +33,16 @@
|
|
|
38
33
|
npm install agent-inspect
|
|
39
34
|
```
|
|
40
35
|
|
|
36
|
+
## Hero flow
|
|
37
|
+
|
|
38
|
+
```text
|
|
39
|
+
1. Capture one real run
|
|
40
|
+
2. Find the causal failure
|
|
41
|
+
3. Ask your coding agent to inspect it
|
|
42
|
+
4. Lock the fix with a contract
|
|
43
|
+
5. Attach the share-checked evidence
|
|
44
|
+
```
|
|
45
|
+
|
|
41
46
|
## Three workflows
|
|
42
47
|
|
|
43
48
|
| Workflow | What you do |
|
|
@@ -46,7 +51,9 @@ npm install agent-inspect
|
|
|
46
51
|
| **Prevent one regression** | TraceContract / `check` → suite / cohort → CI gate |
|
|
47
52
|
| **Share one safe artifact** | Redact → `verify-safe` → offline bundle → optional Studio review |
|
|
48
53
|
|
|
49
|
-
**
|
|
54
|
+
**Coding-agent loop (optional):** local read-only [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) — see [CODING-AGENT-LOOP.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md).
|
|
55
|
+
|
|
56
|
+
**Review as a team (optional):** workspace + optional SQLite index → customer-owned [Studio Beta](https://github.com/rajudandigam/agent-inspect/tree/main/packages/studio) (no AgentInspect-hosted cloud). Portfolio tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/POSITIONING-AND-PORTFOLIO.md).
|
|
50
57
|
|
|
51
58
|
<p align="center">
|
|
52
59
|
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture or import → understand → enforce → verify and bundle → review locally or in customer-owned Studio" width="900">
|
|
@@ -65,14 +72,32 @@ npx agent-inspect list --dir .agent-inspect
|
|
|
65
72
|
|
|
66
73
|
```bash
|
|
67
74
|
# After copying a run id from list:
|
|
68
|
-
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
69
75
|
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
70
76
|
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
71
77
|
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
72
78
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
79
|
+
npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
80
|
+
# Optional coding-agent loop (dry-run by default):
|
|
81
|
+
npx agent-inspect mcp configure --client cursor
|
|
73
82
|
```
|
|
74
83
|
|
|
75
|
-
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md)
|
|
84
|
+
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md) · [Coding-agent loop](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md)
|
|
85
|
+
|
|
86
|
+
## Why AgentInspect
|
|
87
|
+
|
|
88
|
+
**Unique position:** a local **evidence debugger** for TypeScript agents—not a hosted APM, eval host, or prompt registry. Complements LangSmith/Langfuse/Phoenix; owns the laptop → PR loop.
|
|
89
|
+
|
|
90
|
+
**Proven mechanisms (shipped):**
|
|
91
|
+
|
|
92
|
+
| Mechanism | What you get |
|
|
93
|
+
| --------- | ------------ |
|
|
94
|
+
| Faithful execution trees | Nested steps, tools, LLMs, status, duration on disk |
|
|
95
|
+
| First causal failure | Conservative ordered engine for what failed first |
|
|
96
|
+
| Deterministic checks / TraceContract (Beta) | Trajectory expectations without an LLM judge |
|
|
97
|
+
| Share-checked Evidence v2 | `bundle` + `bundle verify` offline artifacts |
|
|
98
|
+
| Read-only MCP coding-agent loop (Preview) | `mcp configure` + `@agent-inspect/mcp-server` over local traces |
|
|
99
|
+
|
|
100
|
+
No fabricated adoption counts. External pilot evidence is still pending—see [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md).
|
|
76
101
|
|
|
77
102
|
## Choose your capture path
|
|
78
103
|
|
|
@@ -109,13 +134,16 @@ Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspec
|
|
|
109
134
|
|
|
110
135
|
| Scenario | Start |
|
|
111
136
|
| -------- | ----- |
|
|
112
|
-
| Wrong tool
|
|
113
|
-
|
|
|
137
|
+
| Wrong tool / intentional failure | [broken-agent-debugging](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
138
|
+
| Coding-agent MCP debug loop | [coding-agent-debug-loop](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/coding-agent-debug-loop) · [CODING-AGENT-LOOP](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) |
|
|
139
|
+
| CI trajectory gate + redact | [ci-eval-redact](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/ci-eval-redact) · [CI artifacts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CI-ARTIFACTS.md) |
|
|
114
140
|
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
115
141
|
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
116
|
-
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio |
|
|
142
|
+
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio (optional) |
|
|
117
143
|
| Design partner trial | [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) |
|
|
118
144
|
|
|
145
|
+
All blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
146
|
+
|
|
119
147
|
## Safety and network behavior
|
|
120
148
|
|
|
121
149
|
- Traces are **local JSONL** under `.agent-inspect/` (or `AGENT_INSPECT_TRACE_DIR`)
|
|
@@ -131,7 +159,7 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
131
159
|
|
|
132
160
|
## Project status
|
|
133
161
|
|
|
134
|
-
**Current release:** **6.
|
|
162
|
+
**Current release:** **6.12.1** (eighteen linked npm packages). Stable launch candidate; eight-week adoption checkpoint in progress; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
135
163
|
|
|
136
164
|
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
137
165
|
|
|
@@ -146,18 +174,29 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
146
174
|
See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPARE.md).
|
|
147
175
|
|
|
148
176
|
<details>
|
|
149
|
-
<summary><strong>Package family (18
|
|
177
|
+
<summary><strong>Package family (18 linked packages · tiered presentation)</strong></summary>
|
|
178
|
+
|
|
179
|
+
Canonical tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/POSITIONING-AND-PORTFOLIO.md) · [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md).
|
|
180
|
+
|
|
181
|
+
### Tier A — Flagship
|
|
182
|
+
|
|
183
|
+
| Package | Purpose |
|
|
184
|
+
| ------- | ------- |
|
|
185
|
+
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI + evidence |
|
|
186
|
+
| [`@agent-inspect/redact`](https://www.npmjs.com/package/@agent-inspect/redact) | Deterministic redaction |
|
|
187
|
+
| [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) | Read-only MCP coding-agent loop (Preview) |
|
|
188
|
+
|
|
189
|
+
### Tier B — Official integrations
|
|
150
190
|
|
|
151
191
|
### Framework adapters
|
|
152
192
|
|
|
153
193
|
| Package | Purpose |
|
|
154
194
|
| ------- | ------- |
|
|
155
|
-
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI |
|
|
156
195
|
| [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) | AI SDK telemetry |
|
|
157
196
|
| [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) | OpenAI Agents processor |
|
|
158
197
|
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain callbacks |
|
|
159
198
|
|
|
160
|
-
### Testing / evaluation
|
|
199
|
+
### Tier B — Testing / evaluation
|
|
161
200
|
|
|
162
201
|
| Package | Purpose |
|
|
163
202
|
| ------- | ------- |
|
|
@@ -166,7 +205,7 @@ See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPA
|
|
|
166
205
|
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter |
|
|
167
206
|
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter |
|
|
168
207
|
|
|
169
|
-
### Safety
|
|
208
|
+
### Tier C — Optional supporting (Safety)
|
|
170
209
|
|
|
171
210
|
| Package | Purpose |
|
|
172
211
|
| ------- | ------- |
|
|
@@ -174,7 +213,7 @@ See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPA
|
|
|
174
213
|
| [`@agent-inspect/guardrails`](https://www.npmjs.com/package/@agent-inspect/guardrails) | Deterministic guardrail rules |
|
|
175
214
|
| [`@agent-inspect/circuit`](https://www.npmjs.com/package/@agent-inspect/circuit) | Loop / retry / timeout analyzers |
|
|
176
215
|
|
|
177
|
-
### Developer surfaces
|
|
216
|
+
### Tier C — Optional supporting (Developer surfaces)
|
|
178
217
|
|
|
179
218
|
| Package | Purpose |
|
|
180
219
|
| ------- | ------- |
|
|
@@ -183,14 +222,14 @@ See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPA
|
|
|
183
222
|
| [`@agent-inspect/mcp`](https://www.npmjs.com/package/@agent-inspect/mcp) | MCP client tracing |
|
|
184
223
|
| [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) | Read-only MCP server (Preview) |
|
|
185
224
|
|
|
186
|
-
### Team / self-hosted
|
|
225
|
+
### Tier C — Optional supporting (Team / self-hosted)
|
|
187
226
|
|
|
188
227
|
| Package | Purpose |
|
|
189
228
|
| ------- | ------- |
|
|
190
229
|
| [`@agent-inspect/index-sqlite`](https://www.npmjs.com/package/@agent-inspect/index-sqlite) | Optional SQLite index (Beta) |
|
|
191
230
|
| [`@agent-inspect/studio`](https://www.npmjs.com/package/@agent-inspect/studio) | Customer-owned Studio (Beta) |
|
|
192
231
|
|
|
193
|
-
### Extension / interop
|
|
232
|
+
### Tier C — Optional supporting (Extension / interop)
|
|
194
233
|
|
|
195
234
|
| Package | Purpose |
|
|
196
235
|
| ------- | ------- |
|
package/docs/ADOPTION.md
CHANGED
|
@@ -25,6 +25,10 @@ npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
|
25
25
|
|
|
26
26
|
No API keys required for the init demo. See [examples/starters/](../examples/starters/README.md).
|
|
27
27
|
|
|
28
|
+
## Install kits
|
|
29
|
+
|
|
30
|
+
Copyable task kits (LangGraph, portable evidence, AI SDK, OpenAI Agents, CI): [INSTALL-KITS.md](./INSTALL-KITS.md).
|
|
31
|
+
|
|
28
32
|
## Choose a starter
|
|
29
33
|
|
|
30
34
|
| Goal | Starter |
|
|
@@ -43,7 +47,7 @@ No API keys required for the init demo. See [examples/starters/](../examples/sta
|
|
|
43
47
|
1. **Capture** — `observe()` / adapter / manual steps → `.agent-inspect/*.jsonl`
|
|
44
48
|
2. **Inspect** — `list`, `view`, `timeline`, `report`, optional `serve` viewer
|
|
45
49
|
3. **Verify** — `check`, `eval`, `@agent-inspect/circuit` rules in CI
|
|
46
|
-
4. **Share safely** — `redact --profile share` before attaching traces to issues/PRs
|
|
50
|
+
4. **Share safely** — `redact <run-id> --dir .agent-inspect --profile share` before attaching traces to issues/PRs
|
|
47
51
|
5. **Scale** — when directories grow, see [SCALE-LIMITS.md](./SCALE-LIMITS.md) and `agent-inspect index build`
|
|
48
52
|
|
|
49
53
|
## CI checklist
|
package/docs/COMPARE.md
CHANGED
|
@@ -4,6 +4,28 @@ AgentInspect is a local-first trace workbench for TypeScript AI agents: capture
|
|
|
4
4
|
|
|
5
5
|
**Docs site:** [https://agentinspect.vercel.app/docs/compare/](https://agentinspect.vercel.app/docs/compare/)
|
|
6
6
|
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
## Complementary handoff (v6.12)
|
|
10
|
+
|
|
11
|
+
Position AgentInspect as the **inner-loop local evidence** layer:
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
local evidence and inner-loop debugging → AgentInspect
|
|
15
|
+
production observability → LangSmith / Langfuse / MLflow / Phoenix / APM
|
|
16
|
+
prompt/output and red-team eval → Promptfoo / Evalite / other eval tools
|
|
17
|
+
generic OTel trace access via MCP → OTel MCP servers
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Handoff story:
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
debug locally with AgentInspect
|
|
24
|
+
export OpenInference/OTLP when production tooling is needed
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Portfolio presentation: [POSITIONING-AND-PORTFOLIO.md](./POSITIONING-AND-PORTFOLIO.md).
|
|
28
|
+
|
|
7
29
|
## Category comparison
|
|
8
30
|
|
|
9
31
|
| Dimension | AgentInspect | Hosted dashboards (LangSmith, Langfuse) | Eval platforms (Braintrust) | Production APM / OTel pipelines |
|
package/docs/DEMO-SCRIPT.md
CHANGED
|
@@ -1,10 +1,13 @@
|
|
|
1
|
-
# Live demo script (~
|
|
1
|
+
# Live demo script (~4 minutes)
|
|
2
2
|
|
|
3
|
-
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
-
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect`. Monorepo: `pnpm build` at repo root.
|
|
3
|
+
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
+
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect@6.12.1`. Monorepo: `pnpm build` at repo root.
|
|
5
5
|
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/)
|
|
6
6
|
|
|
7
|
-
**Blessed demo
|
|
7
|
+
**Blessed demo starters:**
|
|
8
|
+
|
|
9
|
+
- [broken-agent-debugging](../examples/starters/broken-agent-debugging/) — intentional tool failure, no API keys
|
|
10
|
+
- [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/) — MCP inspect + share-checked evidence, no API keys
|
|
8
11
|
|
|
9
12
|
## Setup (before the call)
|
|
10
13
|
|
|
@@ -39,39 +42,52 @@ npx agent-inspect timeline <run-id> --dir .agent-inspect
|
|
|
39
42
|
|
|
40
43
|
Optional: `npx agent-inspect serve --dir .agent-inspect` for browser viewer.
|
|
41
44
|
|
|
42
|
-
## Beat 4 — Verify (30s)
|
|
45
|
+
## Beat 4 — Verify / CI gate (30s)
|
|
43
46
|
|
|
44
47
|
```bash
|
|
45
48
|
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
46
49
|
```
|
|
47
50
|
|
|
48
51
|
Optional flags: `--require-completed`, `--detect-stalls`, `--max-step-duration 30s`.
|
|
52
|
+
For CI artifact demos: `npx agent-inspect artifacts <run-id> --dir .agent-inspect -o ./ci-out`.
|
|
49
53
|
|
|
50
|
-
## Beat 5 — Share
|
|
54
|
+
## Beat 5 — Share-checked evidence (Evidence v2) (45s)
|
|
51
55
|
|
|
52
56
|
```bash
|
|
57
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --format html -o ./evidence-out
|
|
58
|
+
npx agent-inspect bundle verify ./evidence-out
|
|
59
|
+
# lighter alternative:
|
|
53
60
|
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl
|
|
54
61
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
55
|
-
# or: npx agent-inspect verify-safe safe.jsonl
|
|
56
62
|
```
|
|
57
63
|
|
|
58
|
-
"
|
|
64
|
+
"Share-checked evidence is best-effort local policy — not a compliance certification. Safe to attach to a GitHub issue or Slack when assessment allows."
|
|
65
|
+
|
|
66
|
+
## Beat 6 — MCP coding-agent loop (optional, 45s)
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
npx agent-inspect mcp configure --client cursor
|
|
70
|
+
cd ../coding-agent-debug-loop
|
|
71
|
+
pnpm install && pnpm start && pnpm run inspect-mcp
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Point to [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md). Stay read-only; do not claim a live partner session unless one is on the call.
|
|
59
75
|
|
|
60
|
-
## Beat
|
|
76
|
+
## Beat 7 — Fix and diff (optional, 30s)
|
|
61
77
|
|
|
62
78
|
```bash
|
|
63
79
|
pnpm run fixed
|
|
64
80
|
npx agent-inspect diff .agent-inspect/<broken-run>.jsonl .agent-inspect/<fixed-run>.jsonl
|
|
65
81
|
```
|
|
66
82
|
|
|
67
|
-
## Beat
|
|
83
|
+
## Beat 8 — Close (15s)
|
|
68
84
|
|
|
69
|
-
"Starters for AI SDK, OpenAI Agents, LangChain, CI,
|
|
85
|
+
"Starters for AI SDK, OpenAI Agents, LangChain, CI, NestJS harness, and the MCP coding-agent loop are in `examples/starters/`. Fresh repos: `npx agent-inspect init --yes`."
|
|
70
86
|
|
|
71
87
|
## Alternative opener (zero clone)
|
|
72
88
|
|
|
73
89
|
```bash
|
|
74
|
-
npm install agent-inspect
|
|
90
|
+
npm install agent-inspect@6.12.1
|
|
75
91
|
npx agent-inspect init --yes
|
|
76
92
|
node examples/agent-inspect-demo.mjs
|
|
77
93
|
npx agent-inspect list --dir .agent-inspect
|
|
@@ -84,5 +100,6 @@ See [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md).
|
|
|
84
100
|
- **VS Code:** F5 from `packages/vscode` (dev host) — Marketplace listing pending
|
|
85
101
|
- **Doctor:** `npx agent-inspect doctor` when onboarding fails
|
|
86
102
|
- **Framework:** switch to `examples/starters/ai-sdk` for adapter path
|
|
103
|
+
- **CI starter:** `examples/starters/ci-eval-redact`
|
|
87
104
|
|
|
88
|
-
Related: [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) · [SCREENSHOTS.md](./SCREENSHOTS.md)
|
|
105
|
+
Related: [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) · [SCREENSHOTS.md](./SCREENSHOTS.md) · [PRE-V7-PILOT-KIT.md](./PRE-V7-PILOT-KIT.md) · [EVIDENCE-FORMAT.md](./EVIDENCE-FORMAT.md)
|
|
@@ -3,13 +3,15 @@
|
|
|
3
3
|
Thank you for trying AgentInspect early. This doc is the **evaluation contract** — what we ask you to try, what we measure, and what we won't do.
|
|
4
4
|
|
|
5
5
|
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) · **Docs:** [https://agentinspect.vercel.app/docs/](https://agentinspect.vercel.app/docs/)
|
|
6
|
+
**Baseline:** `agent-inspect@6.12.0` (stable launch candidate; adoption checkpoint in progress)
|
|
6
7
|
|
|
7
8
|
## What we're validating
|
|
8
9
|
|
|
9
10
|
1. **First trace < 30 minutes** from `npm install` (with `init` + a starter)
|
|
10
11
|
2. **First CI check** on a real or fixture trace
|
|
11
|
-
3. **Share-
|
|
12
|
-
4. **
|
|
12
|
+
3. **Share-checked evidence** — `bundle` / `verify-safe` artifact attached to an issue or PR (not a compliance certification)
|
|
13
|
+
4. **MCP coding-agent loop** (optional) — read-only inspect → fix → re-inspect with `@agent-inspect/mcp-server`
|
|
14
|
+
5. **Framework fit** — AI SDK, OpenAI Agents, or LangChain path feels native enough
|
|
13
15
|
|
|
14
16
|
## Your path
|
|
15
17
|
|
|
@@ -18,20 +20,24 @@ Thank you for trying AgentInspect early. This doc is the **evaluation contract**
|
|
|
18
20
|
| 1 | Run `init` + one starter | `list` shows a run |
|
|
19
21
|
| 1 | `doctor` clean | No failed checks |
|
|
20
22
|
| 2 | Wire adapter or `observe()` in your app | Real trace captured |
|
|
21
|
-
| 2 | `check` or `
|
|
22
|
-
| 3 | `redact --profile share` | Comfortable posting
|
|
23
|
+
| 2 | `check` or `artifacts` in CI | Job fails on bad fixture |
|
|
24
|
+
| 3 | `bundle` + `bundle verify` (or `redact <run-id> --profile share` + `verify-safe <run-id>`) | Comfortable posting evidence externally |
|
|
25
|
+
| 3–4 | Optional: `mcp configure` + [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/) | Repeated local MCP debug loop |
|
|
23
26
|
| 4 | Feedback session | Case study draft or structured notes |
|
|
24
27
|
|
|
25
28
|
## What we provide
|
|
26
29
|
|
|
27
|
-
- Starters: [examples/starters/](../examples/starters/README.md)
|
|
30
|
+
- Starters: [examples/starters/](../examples/starters/README.md) (incl. `broken-agent-debugging`, `coding-agent-debug-loop`, `ci-eval-redact`)
|
|
31
|
+
- Evidence format: [EVIDENCE-FORMAT.md](./EVIDENCE-FORMAT.md)
|
|
32
|
+
- MCP loop: [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md)
|
|
28
33
|
- Office hours async via GitHub Discussions / Issues
|
|
29
34
|
- [DEMO-SCRIPT.md](./DEMO-SCRIPT.md) for internal team demos
|
|
35
|
+
- Pilot kit overview: [PRE-V7-PILOT-KIT.md](./PRE-V7-PILOT-KIT.md)
|
|
30
36
|
|
|
31
37
|
## What we ask from you
|
|
32
38
|
|
|
33
39
|
- **Structured feedback** (template below) — not vague "looks good"
|
|
34
|
-
- **
|
|
40
|
+
- **Share-checked / redacted artifacts only** — never raw secrets
|
|
35
41
|
- Permission to quote anonymously unless you approve public case study
|
|
36
42
|
|
|
37
43
|
## Feedback template
|
|
@@ -40,15 +46,21 @@ Thank you for trying AgentInspect early. This doc is the **evaluation contract**
|
|
|
40
46
|
### Environment
|
|
41
47
|
- Node version:
|
|
42
48
|
- Framework:
|
|
43
|
-
- AgentInspect version:
|
|
49
|
+
- AgentInspect version: (please use 6.12.0)
|
|
44
50
|
|
|
45
51
|
### First trace
|
|
46
52
|
- Time to first trace:
|
|
47
53
|
- Blockers:
|
|
48
54
|
|
|
49
|
-
### CI
|
|
50
|
-
- check/
|
|
51
|
-
- Rules that mattered:
|
|
55
|
+
### CI / evidence
|
|
56
|
+
- check / artifacts / bundle used?
|
|
57
|
+
- Rules or gates that mattered:
|
|
58
|
+
- Attached share-checked evidence? (yes/no)
|
|
59
|
+
|
|
60
|
+
### MCP (if tried)
|
|
61
|
+
- Client (Cursor / Claude Code / Codex / Gemini / other):
|
|
62
|
+
- Repeated debug loop? (yes/no):
|
|
63
|
+
- Blockers:
|
|
52
64
|
|
|
53
65
|
### Gaps
|
|
54
66
|
- Missing docs:
|
|
@@ -65,6 +77,7 @@ We track adoption from **public signals** and **your reports** only. See [produc
|
|
|
65
77
|
- Hosted dashboards or SaaS
|
|
66
78
|
- Custom adapter development on our roadmap without demand gates
|
|
67
79
|
- SLAs — this is open-source MIT software
|
|
80
|
+
- Treating `verify-safe` / evidence assessment as a compliance certification
|
|
68
81
|
|
|
69
82
|
## Contact
|
|
70
83
|
|
|
@@ -58,6 +58,7 @@ npx agent-inspect check <run-id> --dir .agent-inspect
|
|
|
58
58
|
```bash
|
|
59
59
|
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
60
60
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
61
|
+
npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
61
62
|
```
|
|
62
63
|
|
|
63
64
|
Attach the share-profile bundle (or a redacted file) to a PR or issue — not raw traces.
|
|
@@ -72,11 +73,13 @@ npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redact
|
|
|
72
73
|
|
|
73
74
|
| If you use… | Go to |
|
|
74
75
|
| ----------- | ----- |
|
|
76
|
+
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
77
|
+
| Coding-agent MCP loop | [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) · [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/README.md) |
|
|
78
|
+
| Contracts / CI gates | [TRACE-CONTRACTS.md](./TRACE-CONTRACTS.md) · [SUITES-COHORTS-GATES.md](./SUITES-COHORTS-GATES.md) |
|
|
75
79
|
| AI SDK | [AI SDK adoption](./AI-SDK-ADOPTION.md) |
|
|
76
80
|
| OpenAI Agents | [OpenAI Agents local](./OPENAI-AGENTS-LOCAL.md) |
|
|
77
81
|
| LangChain | [Adapters](./ADAPTERS.md) |
|
|
78
|
-
| CI tests | [CI artifacts](./CI-ARTIFACTS.md)
|
|
82
|
+
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) |
|
|
79
83
|
| Golden path | [GOLDEN-PATH.md](./GOLDEN-PATH.md) |
|
|
80
|
-
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
81
84
|
|
|
82
85
|
Full index: [docs/README.md](./README.md) · Website docs: [agentinspect.vercel.app/docs](https://agentinspect.vercel.app/docs/)
|
package/docs/GOLDEN-PATH.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Golden path (
|
|
1
|
+
# Golden path (stable launch candidate)
|
|
2
2
|
|
|
3
|
-
The 6.
|
|
3
|
+
The **6.12.0** stable launch candidate path. Be precise about what is automated today.
|
|
4
4
|
|
|
5
5
|
## Automated packed path (CI)
|
|
6
6
|
|
package/docs/PRE-V7-PILOT-KIT.md
CHANGED
|
@@ -1,45 +1,71 @@
|
|
|
1
1
|
# Pre-v7 Pilot Kit
|
|
2
2
|
|
|
3
|
-
**Purpose:** External design-partner and three-team pilot evidence for the 6.
|
|
3
|
+
**Purpose:** External design-partner and three-team pilot evidence for the **6.12.0** stable launch candidate during the v6.12 adoption checkpoint.
|
|
4
4
|
**Do not fabricate results.** Record only real partner outcomes in [implementation/PRE-V7-ADOPTION-EVIDENCE.md](implementation/PRE-V7-ADOPTION-EVIDENCE.md).
|
|
5
|
+
**Do not invent partner names, emails, or acceptance rows.** Outreach templates (placeholders only) live in [implementation/ADOPTION-OUTREACH-CHECKLIST.md](implementation/ADOPTION-OUTREACH-CHECKLIST.md) — do not send outreach from this kit automatically.
|
|
5
6
|
|
|
6
7
|
## What is shipping for partners
|
|
7
8
|
|
|
8
9
|
| Item | Version / path |
|
|
9
10
|
|------|----------------|
|
|
10
|
-
| npm | `agent-inspect@6.
|
|
11
|
+
| npm | `agent-inspect@6.12.0` (and fixed-group packages) |
|
|
11
12
|
| Quickstart | `npx agent-inspect init --yes` → demo → `list` → `verify-safe` |
|
|
12
13
|
| Packed E2E (maintainers) | `pnpm run pack:smoke` |
|
|
13
14
|
| Demo script | [DEMO-SCRIPT.md](DEMO-SCRIPT.md) |
|
|
14
15
|
| Design partner guide | [DESIGN-PARTNER-GUIDE.md](DESIGN-PARTNER-GUIDE.md) |
|
|
15
16
|
| Broken-agent starter | `examples/starters/broken-agent-debugging` |
|
|
17
|
+
| Coding-agent MCP loop | `examples/starters/coding-agent-debug-loop` · [CODING-AGENT-LOOP.md](CODING-AGENT-LOOP.md) |
|
|
18
|
+
| Evidence v2 | `agent-inspect bundle` / `bundle verify` · [EVIDENCE-FORMAT.md](EVIDENCE-FORMAT.md) |
|
|
19
|
+
| MCP configure | `npx agent-inspect mcp configure --client cursor` (also claude-code, codex, gemini) |
|
|
20
|
+
| CI artifacts | `agent-inspect check` / `artifacts` · `examples/starters/ci-eval-redact` |
|
|
16
21
|
| Studio (Beta) | `@agent-inspect/studio` — customer-owned, local |
|
|
17
22
|
|
|
18
23
|
## Partner trial checklist (copy per team)
|
|
19
24
|
|
|
20
|
-
1. Install `agent-inspect@6.
|
|
25
|
+
1. Install `agent-inspect@6.12.0` on Node ≥ 20.
|
|
21
26
|
2. Complete five-minute quickstart (init → one run → verify-safe).
|
|
22
27
|
3. Run at least one framework path (AI SDK, OpenAI Agents, or LangChain) **or** observe/manual path.
|
|
23
|
-
4.
|
|
24
|
-
|
|
25
|
-
|
|
28
|
+
4. **Evidence path:** create a share-checked bundle and verify integrity:
|
|
29
|
+
```bash
|
|
30
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --format html -o ./evidence-out
|
|
31
|
+
npx agent-inspect bundle verify ./evidence-out
|
|
32
|
+
```
|
|
33
|
+
5. **MCP path (optional but preferred for §19 MCP indicator):** configure a client (dry-run first), then run the coding-agent debug loop starter:
|
|
34
|
+
```bash
|
|
35
|
+
npx agent-inspect mcp configure --client cursor
|
|
36
|
+
cd examples/starters/coding-agent-debug-loop && pnpm install && pnpm start && pnpm run inspect-mcp
|
|
37
|
+
```
|
|
38
|
+
6. **CI path (optional):** retain `check` / `artifacts` (or `ci-eval-redact` starter) on a PR.
|
|
39
|
+
7. Optionally run Studio against a local workspace.
|
|
40
|
+
8. Return dated findings: blockers, what worked, whether they will keep using it.
|
|
26
41
|
|
|
27
|
-
##
|
|
42
|
+
## Blessed demo flows (6.12.0)
|
|
28
43
|
|
|
29
|
-
|
|
44
|
+
| Flow | Starter / commands | Shows |
|
|
45
|
+
|------|-------------------|--------|
|
|
46
|
+
| Broken → inspect → share | `broken-agent-debugging` + [DEMO-SCRIPT.md](DEMO-SCRIPT.md) | Tree, check, redact, verify-safe |
|
|
47
|
+
| Evidence v2 | `bundle` → `bundle verify` | Share-checked offline evidence |
|
|
48
|
+
| MCP coding-agent loop | `coding-agent-debug-loop` + `mcp configure` | Read-only inspect → fix → re-inspect → evidence |
|
|
49
|
+
| CI retention | `ci-eval-redact` or `check`/`artifacts` in CI | Contract/gate on a PR |
|
|
30
50
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
-
|
|
51
|
+
## Evidence required for a strong v7 go input
|
|
52
|
+
|
|
53
|
+
From the adoption checkpoint / canonical roadmap §19:
|
|
54
|
+
|
|
55
|
+
- 3–5 design partners with dated findings
|
|
56
|
+
- 2–3 retained CI/evidence workflows (external)
|
|
57
|
+
- 2 repeated MCP debug loops (external)
|
|
58
|
+
- 1 public external integration / case study
|
|
59
|
+
- Organic usage that persists beyond release spikes
|
|
60
|
+
|
|
61
|
+
Empty / `_pending_` ledger rows mean “not yet evidenced,” not product failure.
|
|
36
62
|
|
|
37
63
|
## Maintainer stop condition
|
|
38
64
|
|
|
39
|
-
|
|
65
|
+
While the adoption checkpoint is active and external rows remain pending:
|
|
40
66
|
|
|
41
67
|
```text
|
|
42
|
-
trainStatus:
|
|
68
|
+
trainStatus: in-progress (v6.12-adoption-checkpoint)
|
|
43
69
|
```
|
|
44
70
|
|
|
45
71
|
Do **not** schedule or implement v7 until adoption gates in [implementation/release-trains/V7.0.0-READINESS-ASSESSMENT.md](implementation/release-trains/V7.0.0-READINESS-ASSESSMENT.md) are met and a maintainer explicitly authorizes a v7 train.
|
package/docs/SUPPORT-LEVELS.md
CHANGED
package/docs/TEAM-WORKFLOWS.md
CHANGED
|
@@ -7,7 +7,7 @@ How teams adopt AgentInspect without a hosted platform.
|
|
|
7
7
|
1. `npm install agent-inspect && npx agent-inspect init --yes`
|
|
8
8
|
2. Run demo: `node examples/agent-inspect-demo.mjs`
|
|
9
9
|
3. `npx agent-inspect view <run-id>`
|
|
10
|
-
4. Before posting issues: `npx agent-inspect redact --profile share`
|
|
10
|
+
4. Before posting issues: `npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl`
|
|
11
11
|
|
|
12
12
|
## PR / code review
|
|
13
13
|
|
package/docs/TECHNICAL-GUIDE.md
CHANGED
|
@@ -408,7 +408,7 @@ See [LIMITATIONS.md](./LIMITATIONS.md).
|
|
|
408
408
|
| 2 | Eval failure — which step? | `check`, `@agent-inspect/vitest` | [ci-eval-redact](../examples/starters/ci-eval-redact/) |
|
|
409
409
|
| 3 | CI PR artifact | upload + `redact` + `verify-safe` | [CI-ARTIFACTS.md](./CI-ARTIFACTS.md) |
|
|
410
410
|
| 4 | Framework-native trace | adapter packages | [starters](../examples/starters/) |
|
|
411
|
-
| 5 | Safe incident handoff | `redact --profile share` | [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) |
|
|
411
|
+
| 5 | Safe incident handoff | `redact <run-id> --profile share` | [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) |
|
|
412
412
|
| 6 | Multi-agent / sessions | `sessions`, `search`, `diff` | [USE-CASES.md](./USE-CASES.md) |
|
|
413
413
|
| 7 | MCP tool tracing | `@agent-inspect/mcp` | [ADAPTERS.md](./ADAPTERS.md) |
|
|
414
414
|
| 8 | Team adoption sprint | one agent, one check, one artifact | [DESIGN-PARTNER-GUIDE.md](./DESIGN-PARTNER-GUIDE.md) |
|
|
@@ -446,7 +446,7 @@ AgentInspect is a local-first TypeScript toolkit for AI agents. It writes execut
|
|
|
446
446
|
When an agent calls the wrong tool or a test fails, flat logs are not enough. AgentInspect captures runs and nested steps locally, lets you `report` and `diff` failures, and gates CI with deterministic `check` rules — no LLM judge. Framework adapters exist for AI SDK, OpenAI Agents, and LangChain. Metadata-only by default; redact before posting traces anywhere.
|
|
447
447
|
|
|
448
448
|
**120 seconds (technical):**
|
|
449
|
-
Add `observe()` or a framework adapter, get JSONL under `.agent-inspect/`, then use `agent-inspect list`, `view`, `report`, `check`, and `redact --profile share`. Sixteen npm packages cover adapters, eval, redaction, MCP tracing, harness fixtures, and test reporters. Schema 1.0 persisted events; readers handle 0.1/0.2/1.0. Not production APM — local debug and share-safe artifacts first.
|
|
449
|
+
Add `observe()` or a framework adapter, get JSONL under `.agent-inspect/`, then use `agent-inspect list`, `view`, `report`, `check`, and `redact <run-id> --profile share`. Sixteen npm packages cover adapters, eval, redaction, MCP tracing, harness fixtures, and test reporters. Schema 1.0 persisted events; readers handle 0.1/0.2/1.0. Not production APM — local debug and share-safe artifacts first.
|
|
450
450
|
|
|
451
451
|
### 16.2 Pull quotes (factual)
|
|
452
452
|
|