agent-inspect 6.13.0 → 6.14.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -8
- package/README.md +47 -45
- package/docs/DEMO-SCRIPT.md +2 -2
- package/docs/GOLDEN-PATH.md +18 -18
- package/docs/SUPPORT-LEVELS.md +2 -2
- package/docs/TRACE-CONTRACTS.md +13 -3
- package/package.json +14 -5
- package/packages/cli/dist/{chunk-QOKTMZAY.mjs → chunk-MEO2Z3KE.mjs} +30 -4
- package/packages/cli/dist/chunk-MEO2Z3KE.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +69 -6
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +47 -8
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-MU4PS6GO.mjs → src-6VZNGIF6.mjs} +3 -3
- package/packages/cli/dist/{src-MU4PS6GO.mjs.map → src-6VZNGIF6.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +3 -1
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +23 -0
- package/packages/core/dist/advanced.d.ts +23 -0
- package/packages/core/dist/advanced.mjs +3 -1
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/cli/dist/chunk-QOKTMZAY.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.14.1
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 44f80dd: Public positioning and AI discoverability patch: align README/docs/website/package metadata with shipped 6.14 TraceFacts, Evidence, MCP, and experimental matchers; add llms/AI manifests, Agent Skill, and public-truth validators. Docs and presentation only — no schema/runtime change.
|
|
8
|
+
|
|
9
|
+
## Unreleased
|
|
10
|
+
|
|
11
|
+
### Patch
|
|
12
|
+
|
|
13
|
+
- (pending 6.14.1) Public positioning, documentation truth, and AI discoverability refresh.
|
|
14
|
+
|
|
15
|
+
## 6.14.0
|
|
16
|
+
|
|
17
|
+
### Minor Changes
|
|
18
|
+
|
|
19
|
+
- 52a3e23: Evidence-first CI and no-egress launch candidate: optional Evidence `semantics` TraceFacts summary on CI packages, MCP `get_trace_facts`, `init --framework langgraph`, langgraph-gate-evidence recipe, and no-egress/acceptance docs. No schema break; no new packages; no default network.
|
|
20
|
+
|
|
3
21
|
## 6.13.0
|
|
4
22
|
|
|
5
23
|
### Minor Changes
|
|
@@ -12,14 +30,6 @@
|
|
|
12
30
|
|
|
13
31
|
- a3c0daa: Logical lifecycle projection for built-in checks and TraceContract: additive experimental `logicalEvents` (raw `events` unchanged), v0.1 start/complete pairing, stepId parent normalization, nested tool identity, and `metadata.tokens.*` safe metrics. Eval/gate/Evidence/MCP inherit via `runTraceChecks`. Includes anonymized pilot-shaped fixture and packed check→gate→bundle→verify E2E. No schema break; no new packages; no default network.
|
|
14
32
|
|
|
15
|
-
## Unreleased
|
|
16
|
-
|
|
17
|
-
### Patch (6.12.3) / Minor (6.13.0) candidates
|
|
18
|
-
|
|
19
|
-
- Cross-surface semantic parity: `summarizeSemanticParity`, MCP adapter diagnostics parity payload, scaffolding-root parent diagnostics, TraceContract `requiredTools`/`forbiddenTools` aliases.
|
|
20
|
-
- Experimental `TraceFacts` (`buildTraceFacts`) with tool/LLM/outcome indexes over logicalEvents.
|
|
21
|
-
- Experimental Vitest/Jest matchers: `toPassTraceContract`, `toHaveRequiredTool`.
|
|
22
|
-
|
|
23
33
|
## 6.12.1
|
|
24
34
|
|
|
25
35
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -8,11 +8,19 @@
|
|
|
8
8
|
<h1 align="center">agent-inspect</h1>
|
|
9
9
|
|
|
10
10
|
<p align="center">
|
|
11
|
-
<strong>
|
|
11
|
+
<strong>Debug and regression-test TypeScript AI agents from local evidence.</strong>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
14
|
<p align="center">
|
|
15
|
-
|
|
15
|
+
AgentInspect captures framework-faithful execution trees, evaluates them with deterministic TraceFacts and TraceContract rules, creates integrity-verifiable Evidence v2, and lets coding assistants inspect the same local facts over read-only MCP—without a collector, account, or default upload.
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<em>See what your agent did. Prove the fix. Keep the evidence.</em>
|
|
20
|
+
</p>
|
|
21
|
+
|
|
22
|
+
<p align="center">
|
|
23
|
+
<sub>No account · no collector · no default upload · metadata-only by default</sub>
|
|
16
24
|
</p>
|
|
17
25
|
|
|
18
26
|
<p align="center">
|
|
@@ -33,7 +41,14 @@
|
|
|
33
41
|
npm install agent-inspect
|
|
34
42
|
```
|
|
35
43
|
|
|
36
|
-
##
|
|
44
|
+
## Four pillars
|
|
45
|
+
|
|
46
|
+
| Pillar | What you get |
|
|
47
|
+
| ------ | ------------ |
|
|
48
|
+
| **Capture faithfully** | Framework-aware execution trees and local JSONL you own |
|
|
49
|
+
| **Test behavior deterministically** | TraceFacts, TraceContract, checks, suites, gates, and experimental Vitest/Jest matchers |
|
|
50
|
+
| **Produce portable evidence** | Offline Evidence v2 with integrity verification and share-policy disclosure |
|
|
51
|
+
| **Debug with coding assistants locally** | Read-only MCP over the same TraceFacts — no collector required |
|
|
37
52
|
|
|
38
53
|
```text
|
|
39
54
|
1. Capture one real run
|
|
@@ -43,18 +58,6 @@ npm install agent-inspect
|
|
|
43
58
|
5. Attach the share-checked evidence
|
|
44
59
|
```
|
|
45
60
|
|
|
46
|
-
## Three workflows
|
|
47
|
-
|
|
48
|
-
| Workflow | What you do |
|
|
49
|
-
| -------- | ----------- |
|
|
50
|
-
| **Debug one run** | Capture/import → tree / timeline / report → first causal failure |
|
|
51
|
-
| **Prevent one regression** | TraceContract / `check` → suite / cohort → CI gate |
|
|
52
|
-
| **Share one safe artifact** | Redact → `verify-safe` → offline bundle → optional Studio review |
|
|
53
|
-
|
|
54
|
-
**Coding-agent loop (optional):** local read-only [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) — see [CODING-AGENT-LOOP.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md).
|
|
55
|
-
|
|
56
|
-
**Review as a team (optional):** workspace + optional SQLite index → customer-owned [Studio Beta](https://github.com/rajudandigam/agent-inspect/tree/main/packages/studio) (no AgentInspect-hosted cloud). Portfolio tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/POSITIONING-AND-PORTFOLIO.md).
|
|
57
|
-
|
|
58
61
|
<p align="center">
|
|
59
62
|
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture or import → understand → enforce → verify and bundle → review locally or in customer-owned Studio" width="900">
|
|
60
63
|
</p>
|
|
@@ -66,6 +69,7 @@ Commands below match the packed quickstart. Replace `<run-id>` with a value from
|
|
|
66
69
|
```bash
|
|
67
70
|
npm install agent-inspect
|
|
68
71
|
npx agent-inspect init --yes
|
|
72
|
+
# or: npx agent-inspect init --framework langgraph --yes
|
|
69
73
|
node examples/agent-inspect-demo.mjs
|
|
70
74
|
npx agent-inspect list --dir .agent-inspect
|
|
71
75
|
```
|
|
@@ -81,23 +85,22 @@ npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
|
81
85
|
npx agent-inspect mcp configure --client cursor
|
|
82
86
|
```
|
|
83
87
|
|
|
84
|
-
|
|
88
|
+
Guides: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md) · [Coding-agent loop](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md)
|
|
85
89
|
|
|
86
90
|
## Why AgentInspect
|
|
87
91
|
|
|
88
|
-
**
|
|
92
|
+
**Category:** the local evidence debugger and trajectory-test toolkit for TypeScript AI agents — not a hosted APM, eval host, or prompt registry. Complements LangSmith/Langfuse/Phoenix; owns the laptop → PR loop.
|
|
89
93
|
|
|
90
|
-
**
|
|
94
|
+
**Proof (public-safe):** Validated against production-shaped NestJS/LangGraph integrations. Fixture-backed across official adapters and packed consumer workflows.
|
|
91
95
|
|
|
92
96
|
| Mechanism | What you get |
|
|
93
97
|
| --------- | ------------ |
|
|
94
98
|
| Faithful execution trees | Nested steps, tools, LLMs, status, duration on disk |
|
|
95
|
-
|
|
|
99
|
+
| TraceFacts / logical projection | Canonical tool identity and semantic parity without inventing hierarchy |
|
|
96
100
|
| Deterministic checks / TraceContract (Beta) | Trajectory expectations without an LLM judge |
|
|
97
|
-
|
|
|
98
|
-
|
|
|
99
|
-
|
|
100
|
-
No fabricated adoption counts. External pilot evidence is still pending—see [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md).
|
|
101
|
+
| Experimental Vitest/Jest matchers | `toPassTraceContract` · `toHaveRequiredTool` |
|
|
102
|
+
| Share-checked Evidence v2 | `bundle` + `bundle verify` with optional TraceFacts semantics |
|
|
103
|
+
| Read-only MCP coding-agent loop (Preview) | `get_trace_facts` and related tools over local traces |
|
|
101
104
|
|
|
102
105
|
## Choose your capture path
|
|
103
106
|
|
|
@@ -106,29 +109,27 @@ No fabricated adoption counts. External pilot evidence is still pending—see [P
|
|
|
106
109
|
| **Manual / observe** | Custom nesting or object methods | [Getting started](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GETTING-STARTED.md) |
|
|
107
110
|
| **AI SDK** | Vercel AI SDK `generateText` / `streamText` | [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) |
|
|
108
111
|
| **OpenAI Agents** | OpenAI Agents JS | [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) |
|
|
109
|
-
| **LangChain** | Callbacks / LangGraph
|
|
112
|
+
| **LangChain / LangGraph** | Callbacks / LangGraph | [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) · `init --framework langgraph` |
|
|
110
113
|
| **Structured logs** | Logs already emitted | [Log-to-tree](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
111
114
|
| **Harness** | Fixture runner for real projects | [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) |
|
|
112
|
-
| **CI reporters** | Failed-test artifacts | [`vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) · [`jest`](https://www.npmjs.com/package/@agent-inspect/jest) |
|
|
115
|
+
| **CI reporters + matchers** | Failed-test artifacts and contracts | [`vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) · [`jest`](https://www.npmjs.com/package/@agent-inspect/jest) |
|
|
113
116
|
| **Standards files** | OpenInference / OTLP JSON | [Standards](https://github.com/rajudandigam/agent-inspect/blob/main/docs/STANDARDS.md) |
|
|
114
117
|
|
|
115
|
-
Blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
118
|
+
Blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters) · LangGraph gate/evidence recipe: [langgraph-gate-evidence](https://github.com/rajudandigam/agent-inspect/tree/main/examples/recipes/langgraph-gate-evidence)
|
|
116
119
|
|
|
117
120
|
## What you can do after capture
|
|
118
121
|
|
|
119
|
-
**Understand** — `what` / `view` / tree · timeline · report · diff ·
|
|
122
|
+
**Understand** — `what` / `view` / tree · timeline · report · diff · `buildTraceFacts`
|
|
120
123
|
|
|
121
|
-
**Prevent regressions** — deterministic checks ·
|
|
124
|
+
**Prevent regressions** — deterministic checks · TraceContract (Beta) · suites · cohorts · CI gates · Vitest/Jest reporters · experimental matchers (`toPassTraceContract`, `toHaveRequiredTool`)
|
|
122
125
|
|
|
123
|
-
**Share safely** — redaction profiles · `scan` · `verify-safe` ·
|
|
126
|
+
**Share safely** — redaction profiles · `scan` · `verify-safe` · Evidence v2 bundles · CI artifacts
|
|
124
127
|
|
|
125
|
-
**
|
|
128
|
+
**Debug with coding assistants** — `@agent-inspect/mcp-server` · `get_trace_facts` · first-causal-failure
|
|
126
129
|
|
|
127
|
-
**
|
|
130
|
+
**Scale locally** — workspace · optional SQLite index (Beta) · viewer / TUI / VS Code · customer-owned Studio (Beta)
|
|
128
131
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md) · Network: [NETWORK-BEHAVIOR.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md)
|
|
132
|
+
Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md) · Network: [NETWORK-BEHAVIOR.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md) · No-egress: [NO-EGRESS-POLICY.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NO-EGRESS-POLICY.md)
|
|
132
133
|
|
|
133
134
|
## Real-world scenarios
|
|
134
135
|
|
|
@@ -136,13 +137,10 @@ Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspec
|
|
|
136
137
|
| -------- | ----- |
|
|
137
138
|
| Wrong tool / intentional failure | [broken-agent-debugging](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
138
139
|
| Coding-agent MCP debug loop | [coding-agent-debug-loop](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/coding-agent-debug-loop) · [CODING-AGENT-LOOP](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) |
|
|
139
|
-
| CI trajectory gate +
|
|
140
|
+
| CI trajectory gate + Evidence | [ci-eval-redact](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/ci-eval-redact) · [langgraph-gate-evidence](https://github.com/rajudandigam/agent-inspect/tree/main/examples/recipes/langgraph-gate-evidence) |
|
|
140
141
|
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
141
142
|
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
142
143
|
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio (optional) |
|
|
143
|
-
| Design partner trial | [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) |
|
|
144
|
-
|
|
145
|
-
All blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
146
144
|
|
|
147
145
|
## Safety and network behavior
|
|
148
146
|
|
|
@@ -159,9 +157,11 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
159
157
|
|
|
160
158
|
## Project status
|
|
161
159
|
|
|
162
|
-
**Current release:** **6.
|
|
160
|
+
**Current release:** **6.14.1** · schema **1.0** · Node.js **≥ 20** · **MIT** · **actively maintained** (eighteen linked npm packages).
|
|
161
|
+
|
|
162
|
+
The 6.14 line is actively maintained for correctness, compatibility, documentation, security, and framework evolution.
|
|
163
163
|
|
|
164
|
-
[Roadmap](ROADMAP.md) · [
|
|
164
|
+
[Roadmap](ROADMAP.md) · [Changelog](CHANGELOG.md) · [Public product facts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/product/PUBLIC-PRODUCT-FACTS.md)
|
|
165
165
|
|
|
166
166
|
## What AgentInspect is not
|
|
167
167
|
|
|
@@ -169,7 +169,8 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
169
169
|
- Production APM replacement
|
|
170
170
|
- Eval dataset platform or LLM-as-judge by default
|
|
171
171
|
- Prompt registry, pricing engine, or replay engine
|
|
172
|
-
- Universal standards exporter
|
|
172
|
+
- Universal standards exporter
|
|
173
|
+
- Compliance certification
|
|
173
174
|
|
|
174
175
|
See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPARE.md).
|
|
175
176
|
|
|
@@ -194,7 +195,7 @@ Canonical tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/
|
|
|
194
195
|
| ------- | ------- |
|
|
195
196
|
| [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) | AI SDK telemetry |
|
|
196
197
|
| [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) | OpenAI Agents processor |
|
|
197
|
-
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain callbacks |
|
|
198
|
+
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain / LangGraph callbacks |
|
|
198
199
|
|
|
199
200
|
### Tier B — Testing / evaluation
|
|
200
201
|
|
|
@@ -202,8 +203,8 @@ Canonical tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/
|
|
|
202
203
|
| ------- | ------- |
|
|
203
204
|
| [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) | Fixture runner |
|
|
204
205
|
| [`@agent-inspect/eval`](https://www.npmjs.com/package/@agent-inspect/eval) | Local eval heuristics |
|
|
205
|
-
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter |
|
|
206
|
-
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter |
|
|
206
|
+
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter + experimental matchers |
|
|
207
|
+
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter + experimental matchers |
|
|
207
208
|
|
|
208
209
|
### Tier C — Optional supporting (Safety)
|
|
209
210
|
|
|
@@ -244,7 +245,8 @@ Canonical tiers: [POSITIONING-AND-PORTFOLIO.md](https://github.com/rajudandigam/
|
|
|
244
245
|
| | Website | Repo |
|
|
245
246
|
| - | ------- | ---- |
|
|
246
247
|
| Getting started | [docs](https://agentinspect.vercel.app/docs/getting-started/) | [FIRST-TRACE](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) |
|
|
247
|
-
|
|
|
248
|
+
| TraceFacts / contracts | — | [TRACE-FACTS](https://github.com/rajudandigam/agent-inspect/blob/main/docs/TRACE-FACTS.md) · [TRACE-CONTRACTS](https://github.com/rajudandigam/agent-inspect/blob/main/docs/TRACE-CONTRACTS.md) |
|
|
249
|
+
| Evidence / MCP / no-egress | — | [EVIDENCE-FORMAT](https://github.com/rajudandigam/agent-inspect/blob/main/docs/EVIDENCE-FORMAT.md) · [CODING-AGENT-LOOP](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) · [NO-EGRESS-POLICY](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NO-EGRESS-POLICY.md) |
|
|
248
250
|
| Safe sharing | [safe-sharing](https://agentinspect.vercel.app/docs/safe-sharing/) | [SAFE-TRACE-SHARING](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
249
251
|
| API / CLI | — | [API](docs/API.md) · [CLI](docs/CLI.md) (packed with npm) |
|
|
250
252
|
| Full index | — | [docs/README.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/README.md) |
|
package/docs/DEMO-SCRIPT.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Live demo script (~4 minutes)
|
|
2
2
|
|
|
3
3
|
**Audience:** TypeScript developers debugging AI agents.
|
|
4
|
-
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect@6.
|
|
4
|
+
**Prereqs:** Node 20+. Consumers: `npm install agent-inspect@6.14.1`. Monorepo: `pnpm build` at repo root.
|
|
5
5
|
**Website:** [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/)
|
|
6
6
|
|
|
7
7
|
**Blessed demo starters:**
|
|
@@ -87,7 +87,7 @@ npx agent-inspect diff .agent-inspect/<broken-run>.jsonl .agent-inspect/<fixed-r
|
|
|
87
87
|
## Alternative opener (zero clone)
|
|
88
88
|
|
|
89
89
|
```bash
|
|
90
|
-
npm install agent-inspect@6.
|
|
90
|
+
npm install agent-inspect@6.14.1
|
|
91
91
|
npx agent-inspect init --yes
|
|
92
92
|
node examples/agent-inspect-demo.mjs
|
|
93
93
|
npx agent-inspect list --dir .agent-inspect
|
package/docs/GOLDEN-PATH.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Golden path
|
|
1
|
+
# Golden path
|
|
2
2
|
|
|
3
|
-
The
|
|
3
|
+
The recommended local evidence path for the current **6.14** release line.
|
|
4
4
|
|
|
5
5
|
## Automated packed path (CI)
|
|
6
6
|
|
|
@@ -10,31 +10,31 @@ From a packed `agent-inspect` install (`scripts/packed-quickstart-e2e.mjs` via `
|
|
|
10
10
|
init --yes → demo → list → verify-safe <runId> --dir .agent-inspect
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
Semantic loop smoke (`scripts/packed-semantic-loop-e2e.mjs`) exercises check → gate → bundle → verify on a pilot-shaped fixture.
|
|
14
14
|
|
|
15
|
-
## Recommended developer path
|
|
15
|
+
## Recommended developer path
|
|
16
16
|
|
|
17
17
|
```text
|
|
18
|
-
init → demo → list → view/report → check → bundle --profile share → verify-safe
|
|
18
|
+
init → demo → list → view/report → check → bundle --profile share → verify-safe → bundle verify
|
|
19
19
|
```
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
## Technical-but-not-fully-automated
|
|
24
|
-
|
|
25
|
-
`scripts/golden-path-e2e.mjs` covers a subset (init → inspect → verify-safe → report). It does **not** yet automate the full roadmap chain:
|
|
21
|
+
LangGraph-oriented path:
|
|
26
22
|
|
|
27
23
|
```text
|
|
28
|
-
|
|
24
|
+
init --framework langgraph → capture → TraceContract / gate → Evidence v2 → optional MCP get_trace_facts
|
|
29
25
|
```
|
|
30
26
|
|
|
31
|
-
|
|
27
|
+
Use required positional targets (`<run-id>` or file path). See the root README five-minute path.
|
|
28
|
+
|
|
29
|
+
## Optional extensions
|
|
32
30
|
|
|
33
|
-
|
|
31
|
+
- Suites / cohorts / CI gates — [SUITES-COHORTS-GATES.md](./SUITES-COHORTS-GATES.md)
|
|
32
|
+
- Experimental Vitest/Jest matchers — [TRACE-CONTRACTS.md](./TRACE-CONTRACTS.md)
|
|
33
|
+
- Coding-agent MCP loop — [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md)
|
|
34
|
+
- Customer-owned Studio — [SELF-HOSTING.md](./SELF-HOSTING.md)
|
|
34
35
|
|
|
35
|
-
##
|
|
36
|
+
## Honest boundaries
|
|
36
37
|
|
|
37
|
-
-
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
- External Collector/Phoenix verification
|
|
38
|
+
- Full broken→fix→Studio productization is not a single automated script.
|
|
39
|
+
- External partner retention attestation is tracked internally; do not invent adoption rows.
|
|
40
|
+
- Redaction is best-effort, not compliance certification.
|
package/docs/SUPPORT-LEVELS.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Support levels
|
|
2
2
|
|
|
3
|
-
Canonical maturity labels for AgentInspect public packages and major surfaces (6.
|
|
3
|
+
Canonical maturity labels for AgentInspect public packages and major surfaces (6.14.x fixed release line).
|
|
4
4
|
|
|
5
5
|
## Definitions
|
|
6
6
|
|
|
@@ -32,7 +32,7 @@ Canonical maturity labels for AgentInspect public packages and major surfaces (6
|
|
|
32
32
|
| Studio HTTP / GitHub ingest | Preview |
|
|
33
33
|
| `@agent-inspect/mcp-server` | Preview |
|
|
34
34
|
| Standards round-trip / Collector–Phoenix external proof | Preview |
|
|
35
|
-
| Vitest/Jest TraceContract matchers |
|
|
35
|
+
| Vitest/Jest TraceContract matchers (`toPassTraceContract`, `toHaveRequiredTool`) | Experimental |
|
|
36
36
|
|
|
37
37
|
Part of the fixed AgentInspect release line — see the npm badge for the current version.
|
|
38
38
|
|
package/docs/TRACE-CONTRACTS.md
CHANGED
|
@@ -9,17 +9,27 @@ Typed trajectory expectations over local AgentInspect traces via `defineTraceCon
|
|
|
9
9
|
Contracts compile to deterministic check rules for common cases:
|
|
10
10
|
|
|
11
11
|
- run status / completion / max duration
|
|
12
|
-
- tool required / forbidden / allowed / maxCalls / order
|
|
12
|
+
- tool required / forbidden / allowed / maxCalls / order (`requiredTools` / `forbiddenTools` aliases)
|
|
13
13
|
- LLM maxCalls / maxTotalTokens / allowedModels
|
|
14
14
|
- evidence-bearing findings on failures
|
|
15
|
+
- evaluation over **logical** TraceFacts (raw events remain available)
|
|
15
16
|
|
|
16
|
-
|
|
17
|
+
### Experimental Vitest / Jest matchers (shipped)
|
|
18
|
+
|
|
19
|
+
| Package | Export | Matchers |
|
|
20
|
+
| ------- | ------ | -------- |
|
|
21
|
+
| `@agent-inspect/vitest` | `agentInspectVitestMatchers` | `toPassTraceContract`, `toHaveRequiredTool` |
|
|
22
|
+
| `@agent-inspect/jest` | `agentInspectJestMatchers` | `toPassTraceContract`, `toHaveRequiredTool` |
|
|
23
|
+
|
|
24
|
+
These are **Experimental** — API names may evolve. There is no `expectTrace(...).toSatisfyTraceContract` helper.
|
|
25
|
+
|
|
26
|
+
See [API.md](./API.md), [TRACE-FACTS.md](./TRACE-FACTS.md), and `packages/core/src/checks/contract.ts`.
|
|
17
27
|
|
|
18
28
|
## What is not shipped (yet)
|
|
19
29
|
|
|
20
30
|
Do **not** document these as available:
|
|
21
31
|
|
|
22
|
-
-
|
|
32
|
+
- `expectTrace(...).toSatisfyTraceContract` (different API shape than the shipped matchers)
|
|
23
33
|
- Full workflow handoff / approval / MCP protocol contract rules
|
|
24
34
|
- Per-tool argument schema / regex validators on the contract surface
|
|
25
35
|
- Every structure rule (orphan/cycle/depth) exposed on the contract API (many exist as standalone check rules)
|
package/package.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.
|
|
3
|
+
"version": "6.14.1",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
|
-
"description": "
|
|
6
|
+
"description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
|
|
7
7
|
"repository": {
|
|
8
8
|
"type": "git",
|
|
9
9
|
"url": "git+https://github.com/rajudandigam/agent-inspect.git"
|
|
@@ -187,12 +187,20 @@
|
|
|
187
187
|
"keywords": [
|
|
188
188
|
"ai",
|
|
189
189
|
"agents",
|
|
190
|
-
"debugging",
|
|
191
190
|
"typescript",
|
|
191
|
+
"debugging",
|
|
192
|
+
"trajectory-testing",
|
|
193
|
+
"langgraph",
|
|
194
|
+
"langchain",
|
|
195
|
+
"tracecontract",
|
|
196
|
+
"tracefacts",
|
|
197
|
+
"evidence",
|
|
198
|
+
"mcp",
|
|
192
199
|
"llm",
|
|
193
200
|
"devtools",
|
|
194
201
|
"jsonl",
|
|
195
|
-
"cli"
|
|
202
|
+
"cli",
|
|
203
|
+
"local-first"
|
|
196
204
|
],
|
|
197
205
|
"publishConfig": {
|
|
198
206
|
"access": "public"
|
|
@@ -231,7 +239,8 @@
|
|
|
231
239
|
"docs:commands": "node scripts/validate-doc-commands.mjs",
|
|
232
240
|
"docs:links": "node scripts/validate-doc-links.mjs",
|
|
233
241
|
"public-truth:check": "node scripts/validate-public-truth.mjs",
|
|
234
|
-
"
|
|
242
|
+
"ai-assets:check": "node scripts/validate-ai-assets.mjs",
|
|
243
|
+
"docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check",
|
|
235
244
|
"compat:smoke": "node scripts/compat-smoke.mjs",
|
|
236
245
|
"fixtures:check": "node scripts/validate-fixtures.mjs",
|
|
237
246
|
"recipes:check": "node scripts/validate-recipes.mjs",
|
|
@@ -4004,6 +4004,7 @@ function buildEvidenceManifest(parts) {
|
|
|
4004
4004
|
verificationPolicy: parts.verificationPolicy ?? parts.redactionProfile
|
|
4005
4005
|
},
|
|
4006
4006
|
assessment,
|
|
4007
|
+
...parts.semantics !== void 0 ? { semantics: { ...parts.semantics } } : {},
|
|
4007
4008
|
files: buildEvidenceFileEntries(parts.files)
|
|
4008
4009
|
};
|
|
4009
4010
|
}
|
|
@@ -5568,7 +5569,8 @@ function buildEvidenceCiPackage(input) {
|
|
|
5568
5569
|
sourceStatus: input.sourceStatus,
|
|
5569
5570
|
files: packaged,
|
|
5570
5571
|
createdAt,
|
|
5571
|
-
note: EVIDENCE_ASSESSMENT_NOTE
|
|
5572
|
+
note: EVIDENCE_ASSESSMENT_NOTE,
|
|
5573
|
+
...input.semantics !== void 0 ? { semantics: input.semantics } : {}
|
|
5572
5574
|
});
|
|
5573
5575
|
return {
|
|
5574
5576
|
"evidence.html": evidenceHtml,
|
|
@@ -6109,6 +6111,30 @@ function pickString(record, keys) {
|
|
|
6109
6111
|
return void 0;
|
|
6110
6112
|
}
|
|
6111
6113
|
|
|
6114
|
+
// packages/core/src/checks/trace-facts.ts
|
|
6115
|
+
function summarizeSemanticParity(events) {
|
|
6116
|
+
const projection = projectLogicalEvents(events);
|
|
6117
|
+
const logical = projection.logicalEvents;
|
|
6118
|
+
const finishedTools = logical.filter(
|
|
6119
|
+
(event) => event.kind === "TOOL" && event.status !== "running"
|
|
6120
|
+
);
|
|
6121
|
+
const finishedToolNames = Object.freeze(
|
|
6122
|
+
finishedTools.map((event) => resolveCanonicalToolName(event)).sort((a, b) => a.localeCompare(b))
|
|
6123
|
+
);
|
|
6124
|
+
return {
|
|
6125
|
+
rawEventCount: events.length,
|
|
6126
|
+
logicalEventCount: logical.length,
|
|
6127
|
+
runningLogicalCount: logical.filter((event) => event.status === "running").length,
|
|
6128
|
+
finishedToolNames,
|
|
6129
|
+
finishedToolCount: finishedTools.length,
|
|
6130
|
+
pairedCount: logical.filter((event) => event.projection.paired).length,
|
|
6131
|
+
parentRemapCount: projection.diagnostics.filter(
|
|
6132
|
+
(item) => item.code === "AI_LOGICAL_PARENT_REMAPPED"
|
|
6133
|
+
).length,
|
|
6134
|
+
diagnostics: projection.diagnostics
|
|
6135
|
+
};
|
|
6136
|
+
}
|
|
6137
|
+
|
|
6112
6138
|
// packages/core/src/checks/index.ts
|
|
6113
6139
|
var SEVERITY_RANK = {
|
|
6114
6140
|
error: 0,
|
|
@@ -11019,6 +11045,6 @@ function renderGateReport(result, options = {}) {
|
|
|
11019
11045
|
return renderGateSummaryMarkdown(result);
|
|
11020
11046
|
}
|
|
11021
11047
|
|
|
11022
|
-
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
|
|
11023
|
-
//# sourceMappingURL=chunk-
|
|
11024
|
-
//# sourceMappingURL=chunk-
|
|
11048
|
+
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
|
|
11049
|
+
//# sourceMappingURL=chunk-MEO2Z3KE.mjs.map
|
|
11050
|
+
//# sourceMappingURL=chunk-MEO2Z3KE.mjs.map
|