agent-inspect 6.12.0 → 6.12.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +27 -6
- package/docs/ADOPTION.md +1 -1
- package/docs/DEMO-SCRIPT.md +30 -25
- package/docs/DESIGN-PARTNER-GUIDE.md +23 -10
- package/docs/FIRST-TRACE-IN-5-MINUTES.md +5 -2
- package/docs/GOLDEN-PATH.md +2 -2
- package/docs/KNOWN-ISSUES.md +1 -0
- package/docs/PRE-V7-PILOT-KIT.md +41 -15
- package/docs/SUPPORT-LEVELS.md +1 -1
- package/docs/TEAM-WORKFLOWS.md +1 -1
- package/docs/TECHNICAL-GUIDE.md +2 -2
- package/docs/USE-CASES.md +34 -10
- package/package.json +2 -2
- package/packages/cli/dist/{chunk-XTAA733P.mjs → chunk-4WA7DQCM.mjs} +316 -55
- package/packages/cli/dist/chunk-4WA7DQCM.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +337 -70
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +4 -4
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-G7PM24W2.mjs → src-YAASTXGE.mjs} +3 -3
- package/packages/cli/dist/{src-G7PM24W2.mjs.map → src-YAASTXGE.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +290 -30
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +1 -1
- package/packages/core/dist/advanced.d.ts +1 -1
- package/packages/core/dist/advanced.mjs +1 -1
- package/packages/core/dist/checks.cjs +295 -30
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.d.cts +2 -2
- package/packages/core/dist/checks.d.ts +2 -2
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-QT6CQ2XA.mjs → chunk-KJICERSD.mjs} +296 -33
- package/packages/core/dist/chunk-KJICERSD.mjs.map +1 -0
- package/packages/core/dist/{index-mdFcxSOR.d.cts → index-BOP33T4O.d.cts} +72 -1
- package/packages/core/dist/{index-BO7l0iAe.d.ts → index-CgCH8NE9.d.ts} +72 -1
- package/packages/cli/dist/chunk-XTAA733P.mjs.map +0 -1
- package/packages/core/dist/chunk-QT6CQ2XA.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.12.2
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- a3c0daa: Logical lifecycle projection for built-in checks and TraceContract: additive experimental `logicalEvents` (raw `events` unchanged), v0.1 start/complete pairing, stepId parent normalization, nested tool identity, and `metadata.tokens.*` safe metrics. Eval/gate/Evidence/MCP inherit via `runTraceChecks`. Includes anonymized pilot-shaped fixture and packed check→gate→bundle→verify E2E. No schema break; no new packages; no default network.
|
|
8
|
+
|
|
9
|
+
## Unreleased
|
|
10
|
+
|
|
11
|
+
## 6.12.1
|
|
12
|
+
|
|
13
|
+
### Patch Changes
|
|
14
|
+
|
|
15
|
+
- 2a53751: Example-heavy presentation patch: align website/README/starters/docs with the 6.12 hero flow (causal failure, MCP coding-agent loop, Evidence v2), fix incomplete redact/verify-safe command examples, and extend docs:commands to starters. Docs and presentation only — no runtime product expansion.
|
|
16
|
+
|
|
3
17
|
## 6.12.0
|
|
4
18
|
|
|
5
19
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -72,14 +72,32 @@ npx agent-inspect list --dir .agent-inspect
|
|
|
72
72
|
|
|
73
73
|
```bash
|
|
74
74
|
# After copying a run id from list:
|
|
75
|
-
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
76
75
|
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
77
76
|
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
78
77
|
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
79
78
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
79
|
+
npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
80
|
+
# Optional coding-agent loop (dry-run by default):
|
|
81
|
+
npx agent-inspect mcp configure --client cursor
|
|
80
82
|
```
|
|
81
83
|
|
|
82
|
-
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md)
|
|
84
|
+
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md) · [Coding-agent loop](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md)
|
|
85
|
+
|
|
86
|
+
## Why AgentInspect
|
|
87
|
+
|
|
88
|
+
**Unique position:** a local **evidence debugger** for TypeScript agents—not a hosted APM, eval host, or prompt registry. Complements LangSmith/Langfuse/Phoenix; owns the laptop → PR loop.
|
|
89
|
+
|
|
90
|
+
**Proven mechanisms (shipped):**
|
|
91
|
+
|
|
92
|
+
| Mechanism | What you get |
|
|
93
|
+
| --------- | ------------ |
|
|
94
|
+
| Faithful execution trees | Nested steps, tools, LLMs, status, duration on disk |
|
|
95
|
+
| First causal failure | Conservative ordered engine for what failed first |
|
|
96
|
+
| Deterministic checks / TraceContract (Beta) | Trajectory expectations without an LLM judge |
|
|
97
|
+
| Share-checked Evidence v2 | `bundle` + `bundle verify` offline artifacts |
|
|
98
|
+
| Read-only MCP coding-agent loop (Preview) | `mcp configure` + `@agent-inspect/mcp-server` over local traces |
|
|
99
|
+
|
|
100
|
+
No fabricated adoption counts. External pilot evidence is still pending—see [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md).
|
|
83
101
|
|
|
84
102
|
## Choose your capture path
|
|
85
103
|
|
|
@@ -116,13 +134,16 @@ Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspec
|
|
|
116
134
|
|
|
117
135
|
| Scenario | Start |
|
|
118
136
|
| -------- | ----- |
|
|
119
|
-
| Wrong tool
|
|
120
|
-
|
|
|
137
|
+
| Wrong tool / intentional failure | [broken-agent-debugging](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
138
|
+
| Coding-agent MCP debug loop | [coding-agent-debug-loop](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/coding-agent-debug-loop) · [CODING-AGENT-LOOP](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) |
|
|
139
|
+
| CI trajectory gate + redact | [ci-eval-redact](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/ci-eval-redact) · [CI artifacts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CI-ARTIFACTS.md) |
|
|
121
140
|
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
122
141
|
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
123
|
-
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio |
|
|
142
|
+
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio (optional) |
|
|
124
143
|
| Design partner trial | [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) |
|
|
125
144
|
|
|
145
|
+
All blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
146
|
+
|
|
126
147
|
## Safety and network behavior
|
|
127
148
|
|
|
128
149
|
- Traces are **local JSONL** under `.agent-inspect/` (or `AGENT_INSPECT_TRACE_DIR`)
|
|
@@ -138,7 +159,7 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
138
159
|
|
|
139
160
|
## Project status
|
|
140
161
|
|
|
141
|
-
**Current release:** **6.12.
|
|
162
|
+
**Current release:** **6.12.2** (eighteen linked npm packages). Stable launch candidate; eight-week adoption checkpoint in progress; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
142
163
|
|
|
143
164
|
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
144
165
|
|
package/docs/ADOPTION.md
CHANGED
|
@@ -47,7 +47,7 @@ Copyable task kits (LangGraph, portable evidence, AI SDK, OpenAI Agents, CI): [I
|
|
|
47
47
|
1. **Capture** — `observe()` / adapter / manual steps → `.agent-inspect/*.jsonl`
|
|
48
48
|
2. **Inspect** — `list`, `view`, `timeline`, `report`, optional `serve` viewer
|
|
49
49
|
3. **Verify** — `check`, `eval`, `@agent-inspect/circuit` rules in CI
|
|
50
|
-
4. **Share safely** — `redact --profile share` before attaching traces to issues/PRs
|
|
50
|
+
4. **Share safely** — `redact <run-id> --dir .agent-inspect --profile share` before attaching traces to issues/PRs
|
|
51
51
|
5. **Scale** — when directories grow, see [SCALE-LIMITS.md](./SCALE-LIMITS.md) and `agent-inspect index build`
|
|
52
52
|
|
|
53
53
|
## CI checklist
|
package/docs/DEMO-SCRIPT.md
CHANGED
|
@@ -1,10 +1,13 @@
|
|
|
1
|
-
# Live demo script (~
|
|
1
|
+
# Live demo script (~4 minutes)
|
|
2
2
|
|
|
3
|
-
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
-
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect`. Monorepo: `pnpm build` at repo root.
|
|
3
|
+
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
+
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect@6.12.1`. Monorepo: `pnpm build` at repo root.
|
|
5
5
|
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/)
|
|
6
6
|
|
|
7
|
-
**Blessed demo
|
|
7
|
+
**Blessed demo starters:**
|
|
8
|
+
|
|
9
|
+
- [broken-agent-debugging](../examples/starters/broken-agent-debugging/) — intentional tool failure, no API keys
|
|
10
|
+
- [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/) — MCP inspect + share-checked evidence, no API keys
|
|
8
11
|
|
|
9
12
|
## Setup (before the call)
|
|
10
13
|
|
|
@@ -39,39 +42,52 @@ npx agent-inspect timeline <run-id> --dir .agent-inspect
|
|
|
39
42
|
|
|
40
43
|
Optional: `npx agent-inspect serve --dir .agent-inspect` for browser viewer.
|
|
41
44
|
|
|
42
|
-
## Beat 4 — Verify (30s)
|
|
45
|
+
## Beat 4 — Verify / CI gate (30s)
|
|
43
46
|
|
|
44
47
|
```bash
|
|
45
48
|
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
46
49
|
```
|
|
47
50
|
|
|
48
51
|
Optional flags: `--require-completed`, `--detect-stalls`, `--max-step-duration 30s`.
|
|
52
|
+
For CI artifact demos: `npx agent-inspect artifacts <run-id> --dir .agent-inspect -o ./ci-out`.
|
|
49
53
|
|
|
50
|
-
## Beat 5 — Share
|
|
54
|
+
## Beat 5 — Share-checked evidence (Evidence v2) (45s)
|
|
51
55
|
|
|
52
56
|
```bash
|
|
57
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --format html -o ./evidence-out
|
|
58
|
+
npx agent-inspect bundle verify ./evidence-out
|
|
59
|
+
# lighter alternative:
|
|
53
60
|
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl
|
|
54
61
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
55
|
-
# or: npx agent-inspect verify-safe safe.jsonl
|
|
56
62
|
```
|
|
57
63
|
|
|
58
|
-
"
|
|
64
|
+
"Share-checked evidence is best-effort local policy — not a compliance certification. Safe to attach to a GitHub issue or Slack when assessment allows."
|
|
59
65
|
|
|
60
|
-
## Beat 6 —
|
|
66
|
+
## Beat 6 — MCP coding-agent loop (optional, 45s)
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
npx agent-inspect mcp configure --client cursor
|
|
70
|
+
cd ../coding-agent-debug-loop
|
|
71
|
+
pnpm install && pnpm start && pnpm run inspect-mcp
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Point to [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md). Stay read-only; do not claim a live partner session unless one is on the call.
|
|
75
|
+
|
|
76
|
+
## Beat 7 — Fix and diff (optional, 30s)
|
|
61
77
|
|
|
62
78
|
```bash
|
|
63
79
|
pnpm run fixed
|
|
64
80
|
npx agent-inspect diff .agent-inspect/<broken-run>.jsonl .agent-inspect/<fixed-run>.jsonl
|
|
65
81
|
```
|
|
66
82
|
|
|
67
|
-
## Beat
|
|
83
|
+
## Beat 8 — Close (15s)
|
|
68
84
|
|
|
69
|
-
"Starters for AI SDK, OpenAI Agents, LangChain, CI,
|
|
85
|
+
"Starters for AI SDK, OpenAI Agents, LangChain, CI, NestJS harness, and the MCP coding-agent loop are in `examples/starters/`. Fresh repos: `npx agent-inspect init --yes`."
|
|
70
86
|
|
|
71
87
|
## Alternative opener (zero clone)
|
|
72
88
|
|
|
73
89
|
```bash
|
|
74
|
-
npm install agent-inspect
|
|
90
|
+
npm install agent-inspect@6.12.1
|
|
75
91
|
npx agent-inspect init --yes
|
|
76
92
|
node examples/agent-inspect-demo.mjs
|
|
77
93
|
npx agent-inspect list --dir .agent-inspect
|
|
@@ -84,17 +100,6 @@ See [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md).
|
|
|
84
100
|
- **VS Code:** F5 from `packages/vscode` (dev host) — Marketplace listing pending
|
|
85
101
|
- **Doctor:** `npx agent-inspect doctor` when onboarding fails
|
|
86
102
|
- **Framework:** switch to `examples/starters/ai-sdk` for adapter path
|
|
103
|
+
- **CI starter:** `examples/starters/ci-eval-redact`
|
|
87
104
|
|
|
88
|
-
Related: [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) · [SCREENSHOTS.md](./SCREENSHOTS.md)
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
## Optional MCP beat (30s)
|
|
92
|
-
|
|
93
|
-
If time allows after inspect:
|
|
94
|
-
|
|
95
|
-
```bash
|
|
96
|
-
npx agent-inspect mcp configure --client cursor --dry-run
|
|
97
|
-
npx @agent-inspect/mcp-server --dir .agent-inspect --help
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
Point to [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) and the coding-agent-debug-loop starter. Stay read-only; do not claim a live partner session unless one is on the call.
|
|
105
|
+
Related: [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) · [SCREENSHOTS.md](./SCREENSHOTS.md) · [PRE-V7-PILOT-KIT.md](./PRE-V7-PILOT-KIT.md) · [EVIDENCE-FORMAT.md](./EVIDENCE-FORMAT.md)
|
|
@@ -3,13 +3,15 @@
|
|
|
3
3
|
Thank you for trying AgentInspect early. This doc is the **evaluation contract** — what we ask you to try, what we measure, and what we won't do.
|
|
4
4
|
|
|
5
5
|
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) · **Docs:** [https://agentinspect.vercel.app/docs/](https://agentinspect.vercel.app/docs/)
|
|
6
|
+
**Baseline:** `agent-inspect@6.12.0` (stable launch candidate; adoption checkpoint in progress)
|
|
6
7
|
|
|
7
8
|
## What we're validating
|
|
8
9
|
|
|
9
10
|
1. **First trace < 30 minutes** from `npm install` (with `init` + a starter)
|
|
10
11
|
2. **First CI check** on a real or fixture trace
|
|
11
|
-
3. **Share-
|
|
12
|
-
4. **
|
|
12
|
+
3. **Share-checked evidence** — `bundle` / `verify-safe` artifact attached to an issue or PR (not a compliance certification)
|
|
13
|
+
4. **MCP coding-agent loop** (optional) — read-only inspect → fix → re-inspect with `@agent-inspect/mcp-server`
|
|
14
|
+
5. **Framework fit** — AI SDK, OpenAI Agents, or LangChain path feels native enough
|
|
13
15
|
|
|
14
16
|
## Your path
|
|
15
17
|
|
|
@@ -18,20 +20,24 @@ Thank you for trying AgentInspect early. This doc is the **evaluation contract**
|
|
|
18
20
|
| 1 | Run `init` + one starter | `list` shows a run |
|
|
19
21
|
| 1 | `doctor` clean | No failed checks |
|
|
20
22
|
| 2 | Wire adapter or `observe()` in your app | Real trace captured |
|
|
21
|
-
| 2 | `check` or `
|
|
22
|
-
| 3 | `redact --profile share` | Comfortable posting
|
|
23
|
+
| 2 | `check` or `artifacts` in CI | Job fails on bad fixture |
|
|
24
|
+
| 3 | `bundle` + `bundle verify` (or `redact <run-id> --profile share` + `verify-safe <run-id>`) | Comfortable posting evidence externally |
|
|
25
|
+
| 3–4 | Optional: `mcp configure` + [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/) | Repeated local MCP debug loop |
|
|
23
26
|
| 4 | Feedback session | Case study draft or structured notes |
|
|
24
27
|
|
|
25
28
|
## What we provide
|
|
26
29
|
|
|
27
|
-
- Starters: [examples/starters/](../examples/starters/README.md)
|
|
30
|
+
- Starters: [examples/starters/](../examples/starters/README.md) (incl. `broken-agent-debugging`, `coding-agent-debug-loop`, `ci-eval-redact`)
|
|
31
|
+
- Evidence format: [EVIDENCE-FORMAT.md](./EVIDENCE-FORMAT.md)
|
|
32
|
+
- MCP loop: [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md)
|
|
28
33
|
- Office hours async via GitHub Discussions / Issues
|
|
29
34
|
- [DEMO-SCRIPT.md](./DEMO-SCRIPT.md) for internal team demos
|
|
35
|
+
- Pilot kit overview: [PRE-V7-PILOT-KIT.md](./PRE-V7-PILOT-KIT.md)
|
|
30
36
|
|
|
31
37
|
## What we ask from you
|
|
32
38
|
|
|
33
39
|
- **Structured feedback** (template below) — not vague "looks good"
|
|
34
|
-
- **
|
|
40
|
+
- **Share-checked / redacted artifacts only** — never raw secrets
|
|
35
41
|
- Permission to quote anonymously unless you approve public case study
|
|
36
42
|
|
|
37
43
|
## Feedback template
|
|
@@ -40,15 +46,21 @@ Thank you for trying AgentInspect early. This doc is the **evaluation contract**
|
|
|
40
46
|
### Environment
|
|
41
47
|
- Node version:
|
|
42
48
|
- Framework:
|
|
43
|
-
- AgentInspect version:
|
|
49
|
+
- AgentInspect version: (please use 6.12.0)
|
|
44
50
|
|
|
45
51
|
### First trace
|
|
46
52
|
- Time to first trace:
|
|
47
53
|
- Blockers:
|
|
48
54
|
|
|
49
|
-
### CI
|
|
50
|
-
- check/
|
|
51
|
-
- Rules that mattered:
|
|
55
|
+
### CI / evidence
|
|
56
|
+
- check / artifacts / bundle used?
|
|
57
|
+
- Rules or gates that mattered:
|
|
58
|
+
- Attached share-checked evidence? (yes/no)
|
|
59
|
+
|
|
60
|
+
### MCP (if tried)
|
|
61
|
+
- Client (Cursor / Claude Code / Codex / Gemini / other):
|
|
62
|
+
- Repeated debug loop? (yes/no):
|
|
63
|
+
- Blockers:
|
|
52
64
|
|
|
53
65
|
### Gaps
|
|
54
66
|
- Missing docs:
|
|
@@ -65,6 +77,7 @@ We track adoption from **public signals** and **your reports** only. See [produc
|
|
|
65
77
|
- Hosted dashboards or SaaS
|
|
66
78
|
- Custom adapter development on our roadmap without demand gates
|
|
67
79
|
- SLAs — this is open-source MIT software
|
|
80
|
+
- Treating `verify-safe` / evidence assessment as a compliance certification
|
|
68
81
|
|
|
69
82
|
## Contact
|
|
70
83
|
|
|
@@ -58,6 +58,7 @@ npx agent-inspect check <run-id> --dir .agent-inspect
|
|
|
58
58
|
```bash
|
|
59
59
|
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
60
60
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
61
|
+
npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
61
62
|
```
|
|
62
63
|
|
|
63
64
|
Attach the share-profile bundle (or a redacted file) to a PR or issue — not raw traces.
|
|
@@ -72,11 +73,13 @@ npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redact
|
|
|
72
73
|
|
|
73
74
|
| If you use… | Go to |
|
|
74
75
|
| ----------- | ----- |
|
|
76
|
+
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
77
|
+
| Coding-agent MCP loop | [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) · [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/README.md) |
|
|
78
|
+
| Contracts / CI gates | [TRACE-CONTRACTS.md](./TRACE-CONTRACTS.md) · [SUITES-COHORTS-GATES.md](./SUITES-COHORTS-GATES.md) |
|
|
75
79
|
| AI SDK | [AI SDK adoption](./AI-SDK-ADOPTION.md) |
|
|
76
80
|
| OpenAI Agents | [OpenAI Agents local](./OPENAI-AGENTS-LOCAL.md) |
|
|
77
81
|
| LangChain | [Adapters](./ADAPTERS.md) |
|
|
78
|
-
| CI tests | [CI artifacts](./CI-ARTIFACTS.md)
|
|
82
|
+
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) |
|
|
79
83
|
| Golden path | [GOLDEN-PATH.md](./GOLDEN-PATH.md) |
|
|
80
|
-
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
81
84
|
|
|
82
85
|
Full index: [docs/README.md](./README.md) · Website docs: [agentinspect.vercel.app/docs](https://agentinspect.vercel.app/docs/)
|
package/docs/GOLDEN-PATH.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Golden path (
|
|
1
|
+
# Golden path (stable launch candidate)
|
|
2
2
|
|
|
3
|
-
The 6.
|
|
3
|
+
The **6.12.0** stable launch candidate path. Be precise about what is automated today.
|
|
4
4
|
|
|
5
5
|
## Automated packed path (CI)
|
|
6
6
|
|
package/docs/KNOWN-ISSUES.md
CHANGED
|
@@ -34,6 +34,7 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
|
|
|
34
34
|
- **LangChain adapter** captures **metadata-oriented** signals by default; it does not replace full framework observability.
|
|
35
35
|
- **LangChain `stream: true`** records chunk counts and timing only — not a full token replay. Per-token JSONL events are not emitted.
|
|
36
36
|
- **Correlation metadata** (`correlationId`, `requestId`, `decisionId`, `groupId`) is written on `run_started` but **CLI list/view does not filter by correlation fields** yet.
|
|
37
|
+
- **Built-in check/contract semantics (6.12.2+)** use an additive `logicalEvents` projection that pairs v0.1 start/complete rows. Custom rules that only read raw `context.events` still see bridged `running` TOOL starts and separate LOGIC completions—opt into `logicalEvents` for lifecycle-aware matching. Formal `TraceFacts` promotion is planned for 6.13.
|
|
37
38
|
|
|
38
39
|
## Structured log parsing
|
|
39
40
|
|
package/docs/PRE-V7-PILOT-KIT.md
CHANGED
|
@@ -1,45 +1,71 @@
|
|
|
1
1
|
# Pre-v7 Pilot Kit
|
|
2
2
|
|
|
3
|
-
**Purpose:** External design-partner and three-team pilot evidence for the 6.
|
|
3
|
+
**Purpose:** External design-partner and three-team pilot evidence for the **6.12.0** stable launch candidate during the v6.12 adoption checkpoint.
|
|
4
4
|
**Do not fabricate results.** Record only real partner outcomes in [implementation/PRE-V7-ADOPTION-EVIDENCE.md](implementation/PRE-V7-ADOPTION-EVIDENCE.md).
|
|
5
|
+
**Do not invent partner names, emails, or acceptance rows.** Outreach templates (placeholders only) live in [implementation/ADOPTION-OUTREACH-CHECKLIST.md](implementation/ADOPTION-OUTREACH-CHECKLIST.md) — do not send outreach from this kit automatically.
|
|
5
6
|
|
|
6
7
|
## What is shipping for partners
|
|
7
8
|
|
|
8
9
|
| Item | Version / path |
|
|
9
10
|
|------|----------------|
|
|
10
|
-
| npm | `agent-inspect@6.
|
|
11
|
+
| npm | `agent-inspect@6.12.0` (and fixed-group packages) |
|
|
11
12
|
| Quickstart | `npx agent-inspect init --yes` → demo → `list` → `verify-safe` |
|
|
12
13
|
| Packed E2E (maintainers) | `pnpm run pack:smoke` |
|
|
13
14
|
| Demo script | [DEMO-SCRIPT.md](DEMO-SCRIPT.md) |
|
|
14
15
|
| Design partner guide | [DESIGN-PARTNER-GUIDE.md](DESIGN-PARTNER-GUIDE.md) |
|
|
15
16
|
| Broken-agent starter | `examples/starters/broken-agent-debugging` |
|
|
17
|
+
| Coding-agent MCP loop | `examples/starters/coding-agent-debug-loop` · [CODING-AGENT-LOOP.md](CODING-AGENT-LOOP.md) |
|
|
18
|
+
| Evidence v2 | `agent-inspect bundle` / `bundle verify` · [EVIDENCE-FORMAT.md](EVIDENCE-FORMAT.md) |
|
|
19
|
+
| MCP configure | `npx agent-inspect mcp configure --client cursor` (also claude-code, codex, gemini) |
|
|
20
|
+
| CI artifacts | `agent-inspect check` / `artifacts` · `examples/starters/ci-eval-redact` |
|
|
16
21
|
| Studio (Beta) | `@agent-inspect/studio` — customer-owned, local |
|
|
17
22
|
|
|
18
23
|
## Partner trial checklist (copy per team)
|
|
19
24
|
|
|
20
|
-
1. Install `agent-inspect@6.
|
|
25
|
+
1. Install `agent-inspect@6.12.0` on Node ≥ 20.
|
|
21
26
|
2. Complete five-minute quickstart (init → one run → verify-safe).
|
|
22
27
|
3. Run at least one framework path (AI SDK, OpenAI Agents, or LangChain) **or** observe/manual path.
|
|
23
|
-
4.
|
|
24
|
-
|
|
25
|
-
|
|
28
|
+
4. **Evidence path:** create a share-checked bundle and verify integrity:
|
|
29
|
+
```bash
|
|
30
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --format html -o ./evidence-out
|
|
31
|
+
npx agent-inspect bundle verify ./evidence-out
|
|
32
|
+
```
|
|
33
|
+
5. **MCP path (optional but preferred for §19 MCP indicator):** configure a client (dry-run first), then run the coding-agent debug loop starter:
|
|
34
|
+
```bash
|
|
35
|
+
npx agent-inspect mcp configure --client cursor
|
|
36
|
+
cd examples/starters/coding-agent-debug-loop && pnpm install && pnpm start && pnpm run inspect-mcp
|
|
37
|
+
```
|
|
38
|
+
6. **CI path (optional):** retain `check` / `artifacts` (or `ci-eval-redact` starter) on a PR.
|
|
39
|
+
7. Optionally run Studio against a local workspace.
|
|
40
|
+
8. Return dated findings: blockers, what worked, whether they will keep using it.
|
|
26
41
|
|
|
27
|
-
##
|
|
42
|
+
## Blessed demo flows (6.12.0)
|
|
28
43
|
|
|
29
|
-
|
|
44
|
+
| Flow | Starter / commands | Shows |
|
|
45
|
+
|------|-------------------|--------|
|
|
46
|
+
| Broken → inspect → share | `broken-agent-debugging` + [DEMO-SCRIPT.md](DEMO-SCRIPT.md) | Tree, check, redact, verify-safe |
|
|
47
|
+
| Evidence v2 | `bundle` → `bundle verify` | Share-checked offline evidence |
|
|
48
|
+
| MCP coding-agent loop | `coding-agent-debug-loop` + `mcp configure` | Read-only inspect → fix → re-inspect → evidence |
|
|
49
|
+
| CI retention | `ci-eval-redact` or `check`/`artifacts` in CI | Contract/gate on a PR |
|
|
30
50
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
-
|
|
51
|
+
## Evidence required for a strong v7 go input
|
|
52
|
+
|
|
53
|
+
From the adoption checkpoint / canonical roadmap §19:
|
|
54
|
+
|
|
55
|
+
- 3–5 design partners with dated findings
|
|
56
|
+
- 2–3 retained CI/evidence workflows (external)
|
|
57
|
+
- 2 repeated MCP debug loops (external)
|
|
58
|
+
- 1 public external integration / case study
|
|
59
|
+
- Organic usage that persists beyond release spikes
|
|
60
|
+
|
|
61
|
+
Empty / `_pending_` ledger rows mean “not yet evidenced,” not product failure.
|
|
36
62
|
|
|
37
63
|
## Maintainer stop condition
|
|
38
64
|
|
|
39
|
-
|
|
65
|
+
While the adoption checkpoint is active and external rows remain pending:
|
|
40
66
|
|
|
41
67
|
```text
|
|
42
|
-
trainStatus:
|
|
68
|
+
trainStatus: in-progress (v6.12-adoption-checkpoint)
|
|
43
69
|
```
|
|
44
70
|
|
|
45
71
|
Do **not** schedule or implement v7 until adoption gates in [implementation/release-trains/V7.0.0-READINESS-ASSESSMENT.md](implementation/release-trains/V7.0.0-READINESS-ASSESSMENT.md) are met and a maintainer explicitly authorizes a v7 train.
|
package/docs/SUPPORT-LEVELS.md
CHANGED
package/docs/TEAM-WORKFLOWS.md
CHANGED
|
@@ -7,7 +7,7 @@ How teams adopt AgentInspect without a hosted platform.
|
|
|
7
7
|
1. `npm install agent-inspect && npx agent-inspect init --yes`
|
|
8
8
|
2. Run demo: `node examples/agent-inspect-demo.mjs`
|
|
9
9
|
3. `npx agent-inspect view <run-id>`
|
|
10
|
-
4. Before posting issues: `npx agent-inspect redact --profile share`
|
|
10
|
+
4. Before posting issues: `npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl`
|
|
11
11
|
|
|
12
12
|
## PR / code review
|
|
13
13
|
|
package/docs/TECHNICAL-GUIDE.md
CHANGED
|
@@ -408,7 +408,7 @@ See [LIMITATIONS.md](./LIMITATIONS.md).
|
|
|
408
408
|
| 2 | Eval failure — which step? | `check`, `@agent-inspect/vitest` | [ci-eval-redact](../examples/starters/ci-eval-redact/) |
|
|
409
409
|
| 3 | CI PR artifact | upload + `redact` + `verify-safe` | [CI-ARTIFACTS.md](./CI-ARTIFACTS.md) |
|
|
410
410
|
| 4 | Framework-native trace | adapter packages | [starters](../examples/starters/) |
|
|
411
|
-
| 5 | Safe incident handoff | `redact --profile share` | [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) |
|
|
411
|
+
| 5 | Safe incident handoff | `redact <run-id> --profile share` | [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) |
|
|
412
412
|
| 6 | Multi-agent / sessions | `sessions`, `search`, `diff` | [USE-CASES.md](./USE-CASES.md) |
|
|
413
413
|
| 7 | MCP tool tracing | `@agent-inspect/mcp` | [ADAPTERS.md](./ADAPTERS.md) |
|
|
414
414
|
| 8 | Team adoption sprint | one agent, one check, one artifact | [DESIGN-PARTNER-GUIDE.md](./DESIGN-PARTNER-GUIDE.md) |
|
|
@@ -446,7 +446,7 @@ AgentInspect is a local-first TypeScript toolkit for AI agents. It writes execut
|
|
|
446
446
|
When an agent calls the wrong tool or a test fails, flat logs are not enough. AgentInspect captures runs and nested steps locally, lets you `report` and `diff` failures, and gates CI with deterministic `check` rules — no LLM judge. Framework adapters exist for AI SDK, OpenAI Agents, and LangChain. Metadata-only by default; redact before posting traces anywhere.
|
|
447
447
|
|
|
448
448
|
**120 seconds (technical):**
|
|
449
|
-
Add `observe()` or a framework adapter, get JSONL under `.agent-inspect/`, then use `agent-inspect list`, `view`, `report`, `check`, and `redact --profile share`. Sixteen npm packages cover adapters, eval, redaction, MCP tracing, harness fixtures, and test reporters. Schema 1.0 persisted events; readers handle 0.1/0.2/1.0. Not production APM — local debug and share-safe artifacts first.
|
|
449
|
+
Add `observe()` or a framework adapter, get JSONL under `.agent-inspect/`, then use `agent-inspect list`, `view`, `report`, `check`, and `redact <run-id> --profile share`. Sixteen npm packages cover adapters, eval, redaction, MCP tracing, harness fixtures, and test reporters. Schema 1.0 persisted events; readers handle 0.1/0.2/1.0. Not production APM — local debug and share-safe artifacts first.
|
|
450
450
|
|
|
451
451
|
### 16.2 Pull quotes (factual)
|
|
452
452
|
|
package/docs/USE-CASES.md
CHANGED
|
@@ -17,7 +17,12 @@ npx agent-inspect report <run-id> --dir .agent-inspect
|
|
|
17
17
|
|
|
18
18
|
**Output:** Step tree with tool metadata (not raw payloads by default).
|
|
19
19
|
|
|
20
|
-
**Safety:** Metadata-only default. Redact before sharing:
|
|
20
|
+
**Safety:** Metadata-only default. Redact before sharing:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl
|
|
24
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
25
|
+
```
|
|
21
26
|
|
|
22
27
|
**Starter:** [broken-agent-debugging](../examples/starters/broken-agent-debugging/README.md) (intentional failure) or [custom-observe](../examples/starters/custom-observe/README.md)
|
|
23
28
|
|
|
@@ -32,7 +37,7 @@ npx agent-inspect report <run-id> --dir .agent-inspect
|
|
|
32
37
|
**Use:** `@agent-inspect/vitest` or `npx agent-inspect check`.
|
|
33
38
|
|
|
34
39
|
```bash
|
|
35
|
-
npx agent-inspect check .agent-inspect
|
|
40
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
36
41
|
```
|
|
37
42
|
|
|
38
43
|
**Starter:** [ci-eval-redact](../examples/starters/ci-eval-redact/README.md)
|
|
@@ -43,7 +48,7 @@ npx agent-inspect check .agent-inspect/*.jsonl --require-completed
|
|
|
43
48
|
|
|
44
49
|
**Problem:** Reviewers need safe evidence without cloning your laptop.
|
|
45
50
|
|
|
46
|
-
**Use:** CI upload + `redact --profile share` + `verify-safe`.
|
|
51
|
+
**Use:** CI upload + `redact <run-id> --dir … --profile share` + `verify-safe <run-id>` + optional `bundle` / `bundle verify`.
|
|
47
52
|
|
|
48
53
|
**Doc:** [CI artifacts](./CI-ARTIFACTS.md)
|
|
49
54
|
|
|
@@ -63,7 +68,7 @@ npx agent-inspect check .agent-inspect/*.jsonl --require-completed
|
|
|
63
68
|
|
|
64
69
|
**Problem:** You need to paste a trace into Slack or a GitHub issue.
|
|
65
70
|
|
|
66
|
-
**Use:** `@agent-inspect/redact` or CLI `redact` / `scan
|
|
71
|
+
**Use:** `@agent-inspect/redact` or CLI `redact` / `scan` / `bundle` with a run id or path target.
|
|
67
72
|
|
|
68
73
|
**Doc:** [Safe trace sharing](./SAFE-TRACE-SHARING.md)
|
|
69
74
|
|
|
@@ -77,23 +82,42 @@ npx agent-inspect check .agent-inspect/*.jsonl --require-completed
|
|
|
77
82
|
|
|
78
83
|
---
|
|
79
84
|
|
|
80
|
-
## 7.
|
|
85
|
+
## 7. Ask a coding agent what failed first (MCP)
|
|
86
|
+
|
|
87
|
+
**Problem:** You want Cursor/Claude/Codex to inspect a local failing run without uploading traces.
|
|
88
|
+
|
|
89
|
+
**Use:** `@agent-inspect/mcp-server` + `agent-inspect mcp configure` (dry-run by default).
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
npx agent-inspect mcp configure --client cursor
|
|
93
|
+
cd examples/starters/coding-agent-debug-loop && pnpm start && pnpm run inspect-mcp
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Ask the agent for `get_first_causal_failure` / `create_share_checked_evidence`.
|
|
97
|
+
|
|
98
|
+
**Docs:** [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) · **Starter:** [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/README.md)
|
|
99
|
+
|
|
100
|
+
**Not:** Remote MCP gateway or default network upload.
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## 8. MCP client tool tracing
|
|
81
105
|
|
|
82
106
|
**Problem:** Which MCP tools were listed, called, and where did they fail?
|
|
83
107
|
|
|
84
|
-
**Use:** `@agent-inspect/mcp` (client tracing only).
|
|
108
|
+
**Use:** `@agent-inspect/mcp` (client tracing only — distinct from the coding-agent MCP **server** loop above).
|
|
85
109
|
|
|
86
110
|
---
|
|
87
111
|
|
|
88
|
-
##
|
|
112
|
+
## 9. Design partner adoption
|
|
89
113
|
|
|
90
114
|
**Problem:** Team wants one real agent instrumented in one sprint.
|
|
91
115
|
|
|
92
|
-
**Doc:** [Design partner guide](./DESIGN-PARTNER-GUIDE.md)
|
|
116
|
+
**Doc:** [Design partner guide](./DESIGN-PARTNER-GUIDE.md) · [Pilot kit](./PRE-V7-PILOT-KIT.md)
|
|
93
117
|
|
|
94
118
|
---
|
|
95
119
|
|
|
96
|
-
##
|
|
120
|
+
## 10. VS Code trace review
|
|
97
121
|
|
|
98
122
|
**Problem:** Browse `.agent-inspect/` from the editor.
|
|
99
123
|
|
|
@@ -103,7 +127,7 @@ npx agent-inspect check .agent-inspect/*.jsonl --require-completed
|
|
|
103
127
|
|
|
104
128
|
---
|
|
105
129
|
|
|
106
|
-
##
|
|
130
|
+
## 11. Existing structured logs
|
|
107
131
|
|
|
108
132
|
**Problem:** You cannot change app code; logs already exist.
|
|
109
133
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.12.
|
|
3
|
+
"version": "6.12.2",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Debug, regression-test, and safely share TypeScript AI-agent behavior locally — no account, no default upload, metadata-only by default",
|
|
@@ -226,7 +226,7 @@
|
|
|
226
226
|
"test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run build && pnpm run test && pnpm run size",
|
|
227
227
|
"prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run pack:smoke",
|
|
228
228
|
"pack:dry-run": "pnpm run build && npm pack --dry-run",
|
|
229
|
-
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-quickstart-e2e.mjs",
|
|
229
|
+
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs",
|
|
230
230
|
"linked-versions:check": "node scripts/check-linked-versions.mjs",
|
|
231
231
|
"docs:commands": "node scripts/validate-doc-commands.mjs",
|
|
232
232
|
"docs:links": "node scripts/validate-doc-links.mjs",
|