agent-inspect 6.7.1 → 6.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -10
- package/README.md +147 -102
- package/docs/ADOPTION.md +1 -1
- package/docs/BUNDLES.md +67 -0
- package/docs/COMPARE.md +3 -2
- package/docs/DEMO-SCRIPT.md +6 -5
- package/docs/EXPORTS.md +9 -0
- package/docs/FIRST-TRACE-IN-5-MINUTES.md +28 -10
- package/docs/GOLDEN-PATH.md +40 -0
- package/docs/INDEX.md +69 -0
- package/docs/NETWORK-BEHAVIOR.md +32 -0
- package/docs/PERFORMANCE.md +18 -0
- package/docs/PRE-V7-PILOT-KIT.md +45 -0
- package/docs/SCREENSHOTS.md +7 -6
- package/docs/SELF-HOSTING.md +130 -0
- package/docs/SESSIONS-AND-OUTCOMES.md +19 -0
- package/docs/STANDARDS.md +49 -0
- package/docs/SUITES-COHORTS-GATES.md +34 -0
- package/docs/SUPPORT-LEVELS.md +47 -0
- package/docs/TRACE-CONTRACTS.md +39 -0
- package/docs/assets/readme-product-loop.svg +27 -18
- package/package.json +18 -3
- package/packages/cli/dist/{chunk-M6PC5KUT.mjs → chunk-EMYDJREA.mjs} +26 -10
- package/packages/cli/dist/chunk-EMYDJREA.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +46 -21
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +23 -14
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-27LC2BB3.mjs → src-IPJNFV37.mjs} +3 -3
- package/packages/cli/dist/{src-27LC2BB3.mjs.map → src-IPJNFV37.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +24 -8
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.mjs +25 -9
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/checks.cjs +56 -4
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-EWHN6NA4.mjs → chunk-EQVXDGGU.mjs} +58 -6
- package/packages/core/dist/chunk-EQVXDGGU.mjs.map +1 -0
- package/packages/core/dist/diff.cjs +4 -0
- package/packages/core/dist/diff.cjs.map +1 -1
- package/packages/core/dist/diff.mjs +4 -0
- package/packages/core/dist/diff.mjs.map +1 -1
- package/packages/core/dist/exporters.cjs +9 -4
- package/packages/core/dist/exporters.cjs.map +1 -1
- package/packages/core/dist/exporters.d.cts +3 -2
- package/packages/core/dist/exporters.d.ts +3 -2
- package/packages/core/dist/exporters.mjs +9 -4
- package/packages/core/dist/exporters.mjs.map +1 -1
- package/packages/core/dist/logs.cjs +3 -2
- package/packages/core/dist/logs.cjs.map +1 -1
- package/packages/core/dist/logs.mjs +3 -2
- package/packages/core/dist/logs.mjs.map +1 -1
- package/packages/cli/dist/chunk-M6PC5KUT.mjs.map +0 -1
- package/packages/core/dist/chunk-EWHN6NA4.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.7.3
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- ac6747d: Corrective patch after 6.7.2: TraceContract allowed-status handling, OpenInference nano timestamp strings, cohort maxRelativeDelta for all numeric metrics, Windows path portability, CLI search --session exit codes, diff/normalizer correctness, Studio auth timing / ingest / bundle dedup, and related test hardening. No schema break; no new product surface.
|
|
8
|
+
|
|
9
|
+
## Unreleased
|
|
10
|
+
|
|
11
|
+
## 6.7.2
|
|
12
|
+
|
|
13
|
+
### Patch Changes
|
|
14
|
+
|
|
15
|
+
- 9c1f54c: Product presentation patch: rebuild README/docs/package guides/website for the 6.7.x launch candidate, fix public CLI command targets, add docs/link/public-truth validation, and slim the public roadmap. Docs and presentation only — no runtime product expansion.
|
|
16
|
+
|
|
3
17
|
## 6.7.1
|
|
4
18
|
|
|
5
19
|
### Patch Changes
|
|
@@ -199,17 +213,15 @@
|
|
|
199
213
|
|
|
200
214
|
- 70f3fb2: v3.1 adoption train: public `@agent-inspect/harness`, `agent-inspect init` and `doctor` commands, adoption starters, and onboarding docs.
|
|
201
215
|
|
|
202
|
-
##
|
|
203
|
-
|
|
204
|
-
README adoption polish (brand header, product-loop visual, npm link hygiene). Docs-only; no runtime feature changes.
|
|
216
|
+
## Historical (pre-3.5 train notes)
|
|
205
217
|
|
|
206
|
-
|
|
218
|
+
The v3.0→v3.5 feature train is complete. Older in-progress notes below are kept for history.
|
|
207
219
|
|
|
208
|
-
|
|
220
|
+
### README adoption polish (shipped in 3.5.x docs)
|
|
209
221
|
|
|
210
|
-
|
|
222
|
+
README adoption polish (brand header, product-loop visual, npm link hygiene). Docs-only; no runtime feature changes.
|
|
211
223
|
|
|
212
|
-
|
|
224
|
+
See [docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md](docs/implementation/reviews/README-ADOPTION-POLISH-REVIEW.md).
|
|
213
225
|
|
|
214
226
|
### v3.1 (shipped in 3.1.0)
|
|
215
227
|
|
|
@@ -263,9 +275,9 @@ The v3.0→v3.5 feature train is complete. Older in-progress notes below are kep
|
|
|
263
275
|
|
|
264
276
|
Adds the public optional `@agent-inspect/vitest` and `@agent-inspect/jest` reporter packages, the shared experimental `agent-inspect/reporters` helpers, and the `agent-inspect ci-summary` workflow for deterministic local reporter manifests and CI artifacts.
|
|
265
277
|
|
|
266
|
-
|
|
278
|
+
### Draft notes (superseded by 2.3.0 / 2.4.0 releases)
|
|
267
279
|
|
|
268
|
-
|
|
280
|
+
#### Draft v2.4.0 Notes
|
|
269
281
|
|
|
270
282
|
- Added session/workflow causality model and `agent-inspect/advanced` session index helpers (`buildSessionIndex`, scope/cohort helpers, session fixtures).
|
|
271
283
|
- Added `agent-inspect sessions` and `agent-inspect session` CLI for multi-run handoff/retry navigation with timeline, critical-path, diagnostics, and JSON output.
|
|
@@ -273,7 +285,7 @@ The v3.0→v3.5 feature train is complete. Older in-progress notes below are kep
|
|
|
273
285
|
- Added public optional `@agent-inspect/mcp` for local MCP **client** `tools/list` and `tools/call` telemetry with bounded summaries and `source.type: mcp-client` metadata.
|
|
274
286
|
- No schema version change, no MCP gateway/server, no timestamp-only causality inference, and no root/core dependency on MCP SDKs.
|
|
275
287
|
|
|
276
|
-
|
|
288
|
+
#### Draft v2.3.0 Notes
|
|
277
289
|
|
|
278
290
|
- Hardened the official adapter paths for AI SDK, OpenAI Agents JS, and LangChain/LangGraph with no-network recipes, local-only defaults, clearer lifecycle coverage, and executable adapter conformance evidence.
|
|
279
291
|
- AI SDK coverage now includes route-style telemetry factory guidance, per-request integration isolation, tool/stream/error/parallel fixtures, token metadata, and the required `recordInputs: false` / `recordOutputs: false` host settings.
|
package/README.md
CHANGED
|
@@ -8,11 +8,16 @@
|
|
|
8
8
|
<h1 align="center">agent-inspect</h1>
|
|
9
9
|
|
|
10
10
|
<p align="center">
|
|
11
|
-
<strong>
|
|
11
|
+
<strong>Debug, regression-test, and safely share TypeScript AI-agent behavior—locally.</strong>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
14
|
<p align="center">
|
|
15
|
-
<sub>No account · no upload ·
|
|
15
|
+
<sub>No account · no default upload · metadata-only by default · optional customer-owned Studio</sub>
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
AgentInspect turns agent runs into <strong>customer-owned evidence</strong>: execution trees,
|
|
20
|
+
deterministic contracts, CI gates, verified-safe bundles, and optional self-hosted review.
|
|
16
21
|
</p>
|
|
17
22
|
|
|
18
23
|
<p align="center">
|
|
@@ -29,151 +34,191 @@
|
|
|
29
34
|
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178c6" alt="TypeScript"></a>
|
|
30
35
|
</p>
|
|
31
36
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
37
|
+
```bash
|
|
38
|
+
npm install agent-inspect
|
|
39
|
+
```
|
|
35
40
|
|
|
36
41
|
## Three workflows
|
|
37
42
|
|
|
38
|
-
| Workflow | What you do |
|
|
39
|
-
| -------- | ----------- |
|
|
40
|
-
| **Debug one run** | Capture →
|
|
41
|
-
| **Prevent one regression** |
|
|
42
|
-
| **Share one safe artifact** | Redact → verify-safe → bundle
|
|
43
|
+
| Workflow | What you do |
|
|
44
|
+
| -------- | ----------- |
|
|
45
|
+
| **Debug one run** | Capture/import → tree / timeline / report → first causal failure |
|
|
46
|
+
| **Prevent one regression** | TraceContract / `check` → suite / cohort → CI gate |
|
|
47
|
+
| **Share one safe artifact** | Redact → `verify-safe` → offline bundle → optional Studio review |
|
|
43
48
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
49
|
+
**Review as a team (optional):** workspace + optional SQLite index → customer-owned [Studio Beta](https://github.com/rajudandigam/agent-inspect/tree/main/packages/studio) (no AgentInspect-hosted cloud).
|
|
50
|
+
|
|
51
|
+
<p align="center">
|
|
52
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture or import → understand → enforce → verify and bundle → review locally or in customer-owned Studio" width="900">
|
|
53
|
+
</p>
|
|
54
|
+
|
|
55
|
+
## Five-minute path
|
|
47
56
|
|
|
48
|
-
|
|
57
|
+
Commands below match the packed quickstart. Replace `<run-id>` with a value from `list`.
|
|
49
58
|
|
|
50
59
|
```bash
|
|
60
|
+
npm install agent-inspect
|
|
51
61
|
npx agent-inspect init --yes
|
|
52
62
|
node examples/agent-inspect-demo.mjs
|
|
53
63
|
npx agent-inspect list --dir .agent-inspect
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
# After copying a run id from list:
|
|
54
68
|
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
55
69
|
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
70
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
71
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
56
72
|
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
57
73
|
```
|
|
58
74
|
|
|
59
|
-
|
|
60
|
-
Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](docs/FIRST-TRACE-IN-5-MINUTES.md)
|
|
75
|
+
`init` scaffolds config and a demo script; the **demo** writes the trace. Guide: [First trace in 5 minutes](https://agentinspect.vercel.app/docs/getting-started/) · [repo](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) · [Golden path](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GOLDEN-PATH.md)
|
|
61
76
|
|
|
62
|
-
|
|
63
|
-
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture, inspect, check, redact — local JSONL only" width="720">
|
|
64
|
-
</p>
|
|
65
|
-
|
|
66
|
-
## Choose your path
|
|
77
|
+
## Choose your capture path
|
|
67
78
|
|
|
68
79
|
| Path | Use when | Start |
|
|
69
80
|
| ---- | -------- | ----- |
|
|
70
|
-
| **
|
|
71
|
-
| **
|
|
72
|
-
| **
|
|
73
|
-
| **
|
|
74
|
-
| **
|
|
75
|
-
| **
|
|
76
|
-
| **CI
|
|
77
|
-
| **
|
|
81
|
+
| **Manual / observe** | Custom nesting or object methods | [Getting started](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GETTING-STARTED.md) |
|
|
82
|
+
| **AI SDK** | Vercel AI SDK `generateText` / `streamText` | [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) |
|
|
83
|
+
| **OpenAI Agents** | OpenAI Agents JS | [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) |
|
|
84
|
+
| **LangChain** | Callbacks / LangGraph-via-LangChain | [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) |
|
|
85
|
+
| **Structured logs** | Logs already emitted | [Log-to-tree](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
86
|
+
| **Harness** | Fixture runner for real projects | [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) |
|
|
87
|
+
| **CI reporters** | Failed-test artifacts | [`vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) · [`jest`](https://www.npmjs.com/package/@agent-inspect/jest) |
|
|
88
|
+
| **Standards files** | OpenInference / OTLP JSON | [Standards](https://github.com/rajudandigam/agent-inspect/blob/main/docs/STANDARDS.md) |
|
|
78
89
|
|
|
79
90
|
Blessed starters (no API keys): [examples/starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters)
|
|
80
91
|
|
|
81
|
-
## What
|
|
92
|
+
## What you can do after capture
|
|
93
|
+
|
|
94
|
+
**Understand** — `what` / `view` / tree · timeline · report · diff · sessions / activity
|
|
95
|
+
|
|
96
|
+
**Prevent regressions** — deterministic checks · typed TraceContract (Beta) · suites · cohorts · CI gates · Vitest/Jest reporters (artifact reporters today; TraceContract matchers not shipped)
|
|
97
|
+
|
|
98
|
+
**Share safely** — redaction profiles · `scan` · `verify-safe` · offline bundles · CI artifacts
|
|
99
|
+
|
|
100
|
+
**Scale locally** — workspace · optional SQLite index (Beta) · observed outcomes · viewer / TUI / VS Code
|
|
101
|
+
|
|
102
|
+
**Review as a team** — customer-owned Studio Beta · explicit file / GitHub / HTTP ingest (disabled by default) · no maintainer cloud
|
|
103
|
+
|
|
104
|
+
**Interoperate** — read-only MCP server (Preview) · OpenInference-compatible / OTLP GenAI-aligned mapping with known-loss reporting
|
|
82
105
|
|
|
83
|
-
|
|
84
|
-
- **Failed eval / test** — `check`, `eval`, Vitest/Jest reporters on failure
|
|
85
|
-
- **Baseline vs candidate** — local `diff` of two runs
|
|
86
|
-
- **PR trace artifact** — [CI artifacts](docs/CI-ARTIFACTS.md) + `redact --profile share`
|
|
87
|
-
- **Safe incident handoff** — [safe sharing](https://agentinspect.vercel.app/docs/safe-sharing/) before Slack or GitHub
|
|
88
|
-
- **Multi-agent / sessions** — `sessions`, `search`, handoff metadata
|
|
89
|
-
- **MCP tool calls** — [`@agent-inspect/mcp`](packages/mcp/README.md) client tracing
|
|
90
|
-
- **VS Code review** — in-repo extension ([dev guide](docs/VSCODE.md); Marketplace not published yet)
|
|
91
|
-
- **Existing logs** — parse structured logs when you cannot instrument
|
|
106
|
+
Support labels: [SUPPORT-LEVELS.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md) · Network: [NETWORK-BEHAVIOR.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md)
|
|
92
107
|
|
|
93
108
|
## Real-world scenarios
|
|
94
109
|
|
|
95
|
-
| Scenario |
|
|
96
|
-
| -------- |
|
|
97
|
-
|
|
|
98
|
-
| CI
|
|
99
|
-
|
|
|
100
|
-
|
|
|
101
|
-
|
|
|
102
|
-
| Design partner trial | [
|
|
110
|
+
| Scenario | Start |
|
|
111
|
+
| -------- | ----- |
|
|
112
|
+
| Wrong tool call | [broken-agent starter](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
113
|
+
| CI trajectory gate | [CI artifacts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CI-ARTIFACTS.md) |
|
|
114
|
+
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
115
|
+
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
116
|
+
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio |
|
|
117
|
+
| Design partner trial | [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) |
|
|
103
118
|
|
|
104
|
-
##
|
|
119
|
+
## Safety and network behavior
|
|
120
|
+
|
|
121
|
+
- Traces are **local JSONL** under `.agent-inspect/` (or `AGENT_INSPECT_TRACE_DIR`)
|
|
122
|
+
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
123
|
+
- **No hidden upload** — core does not send traces to AgentInspect
|
|
124
|
+
- **Customer-owned Studio ingestion** is disabled by default and explicit when enabled
|
|
125
|
+
- **MCP server** exposes configured local evidence to the connected client (Preview)
|
|
126
|
+
- **Standards export** only when you run/configure it
|
|
127
|
+
- Redaction is **best-effort**, not certification — review before posting
|
|
128
|
+
- **Not** a chain-of-thought recorder
|
|
129
|
+
|
|
130
|
+
Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) · [Network behavior](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md) · [Security](SECURITY.md)
|
|
131
|
+
|
|
132
|
+
## Project status
|
|
133
|
+
|
|
134
|
+
**Current release:** **6.7.3** (eighteen linked npm packages). Technical launch candidate; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
135
|
+
|
|
136
|
+
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
137
|
+
|
|
138
|
+
## What AgentInspect is not
|
|
139
|
+
|
|
140
|
+
- Hosted SaaS or maintainer-hosted dashboard
|
|
141
|
+
- Production APM replacement
|
|
142
|
+
- Eval dataset platform or LLM-as-judge by default
|
|
143
|
+
- Prompt registry, pricing engine, or replay engine
|
|
144
|
+
- Universal standards exporter or completed external adoption proof
|
|
145
|
+
|
|
146
|
+
See [Compare](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPARE.md).
|
|
147
|
+
|
|
148
|
+
<details>
|
|
149
|
+
<summary><strong>Package family (18 public packages)</strong></summary>
|
|
150
|
+
|
|
151
|
+
### Framework adapters
|
|
105
152
|
|
|
106
153
|
| Package | Purpose |
|
|
107
154
|
| ------- | ------- |
|
|
108
155
|
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI |
|
|
109
|
-
| [`@agent-inspect/ai-sdk`](
|
|
110
|
-
| [`@agent-inspect/openai-agents`](
|
|
111
|
-
| [`@agent-inspect/langchain`](
|
|
112
|
-
| [`@agent-inspect/harness`](packages/harness/README.md) | Fixture runner for real projects |
|
|
113
|
-
| [`@agent-inspect/redact`](packages/redact/README.md) | Deterministic redaction |
|
|
114
|
-
| [`@agent-inspect/eval`](packages/eval/README.md) | Local eval heuristics |
|
|
115
|
-
| [`@agent-inspect/vitest`](packages/vitest/README.md) | Vitest reporter |
|
|
116
|
-
| [`@agent-inspect/jest`](packages/jest/README.md) | Jest reporter |
|
|
117
|
-
| [`@agent-inspect/mcp`](packages/mcp/README.md) | MCP client tracing |
|
|
118
|
-
| [`@agent-inspect/mcp-server`](packages/mcp-server/README.md) | Read-only trace MCP server |
|
|
119
|
-
| [`@agent-inspect/guardrails`](packages/guardrails/README.md) | Deterministic guardrail rules |
|
|
120
|
-
| [`@agent-inspect/circuit`](packages/circuit/README.md) | Loop / retry / timeout analyzers |
|
|
121
|
-
| [`@agent-inspect/viewer`](packages/viewer/README.md) | Localhost viewer |
|
|
122
|
-
| [`@agent-inspect/adapter-sdk`](packages/adapter-sdk/README.md) | Third-party adapters |
|
|
123
|
-
| [`@agent-inspect/index-sqlite`](packages/index-sqlite/README.md) | Optional SQLite index |
|
|
124
|
-
| [`@agent-inspect/studio`](packages/studio/README.md) | Customer-owned Studio (Beta) |
|
|
125
|
-
| [`@agent-inspect/tui`](packages/tui/README.md) | Optional terminal UI |
|
|
126
|
-
| `agent-inspect-vscode` | VS Code extension (in-repo; not on Marketplace yet) |
|
|
127
|
-
|
|
128
|
-
## Safety model
|
|
156
|
+
| [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) | AI SDK telemetry |
|
|
157
|
+
| [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) | OpenAI Agents processor |
|
|
158
|
+
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain callbacks |
|
|
129
159
|
|
|
130
|
-
|
|
131
|
-
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
132
|
-
- **No hidden upload** — AgentInspect does not send traces to the cloud
|
|
133
|
-
- **Redaction profiles** — `local` / `share` / `strict` via CLI or [`@agent-inspect/redact`](packages/redact/README.md)
|
|
134
|
-
- **`scan` / `verify-safe`** — check artifacts before sharing
|
|
135
|
-
- **Not** a chain-of-thought recorder or compliance engine — review exports before posting
|
|
160
|
+
### Testing / evaluation
|
|
136
161
|
|
|
137
|
-
|
|
162
|
+
| Package | Purpose |
|
|
163
|
+
| ------- | ------- |
|
|
164
|
+
| [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) | Fixture runner |
|
|
165
|
+
| [`@agent-inspect/eval`](https://www.npmjs.com/package/@agent-inspect/eval) | Local eval heuristics |
|
|
166
|
+
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter |
|
|
167
|
+
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter |
|
|
138
168
|
|
|
139
|
-
|
|
169
|
+
### Safety
|
|
140
170
|
|
|
141
|
-
|
|
|
142
|
-
|
|
|
143
|
-
|
|
|
144
|
-
|
|
|
145
|
-
|
|
|
146
|
-
| API / CLI | — | [API.md](docs/API.md) · [CLI.md](docs/CLI.md) |
|
|
147
|
-
| Adoption | — | [ADOPTION.md](docs/ADOPTION.md) · [USE-CASES.md](docs/USE-CASES.md) |
|
|
148
|
-
| Technical guide | — | [TECHNICAL-GUIDE.md](docs/TECHNICAL-GUIDE.md) |
|
|
149
|
-
| Examples | — | [starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters) |
|
|
150
|
-
| Visual demos | — | [SCREENSHOTS.md](docs/SCREENSHOTS.md) |
|
|
171
|
+
| Package | Purpose |
|
|
172
|
+
| ------- | ------- |
|
|
173
|
+
| [`@agent-inspect/redact`](https://www.npmjs.com/package/@agent-inspect/redact) | Deterministic redaction |
|
|
174
|
+
| [`@agent-inspect/guardrails`](https://www.npmjs.com/package/@agent-inspect/guardrails) | Deterministic guardrail rules |
|
|
175
|
+
| [`@agent-inspect/circuit`](https://www.npmjs.com/package/@agent-inspect/circuit) | Loop / retry / timeout analyzers |
|
|
151
176
|
|
|
152
|
-
|
|
177
|
+
### Developer surfaces
|
|
153
178
|
|
|
154
|
-
|
|
179
|
+
| Package | Purpose |
|
|
180
|
+
| ------- | ------- |
|
|
181
|
+
| [`@agent-inspect/viewer`](https://www.npmjs.com/package/@agent-inspect/viewer) | Localhost viewer |
|
|
182
|
+
| [`@agent-inspect/tui`](https://www.npmjs.com/package/@agent-inspect/tui) | Optional terminal UI |
|
|
183
|
+
| [`@agent-inspect/mcp`](https://www.npmjs.com/package/@agent-inspect/mcp) | MCP client tracing |
|
|
184
|
+
| [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) | Read-only MCP server (Preview) |
|
|
155
185
|
|
|
156
|
-
|
|
157
|
-
- Production APM replacement (use LangSmith, Langfuse, OTel, etc. alongside)
|
|
158
|
-
- Eval dataset platform or LLM-as-judge service
|
|
159
|
-
- Prompt registry or provider pricing engine
|
|
160
|
-
- Default telemetry uploader or replay engine
|
|
186
|
+
### Team / self-hosted
|
|
161
187
|
|
|
162
|
-
|
|
188
|
+
| Package | Purpose |
|
|
189
|
+
| ------- | ------- |
|
|
190
|
+
| [`@agent-inspect/index-sqlite`](https://www.npmjs.com/package/@agent-inspect/index-sqlite) | Optional SQLite index (Beta) |
|
|
191
|
+
| [`@agent-inspect/studio`](https://www.npmjs.com/package/@agent-inspect/studio) | Customer-owned Studio (Beta) |
|
|
163
192
|
|
|
164
|
-
|
|
193
|
+
### Extension / interop
|
|
165
194
|
|
|
166
|
-
|
|
195
|
+
| Package | Purpose |
|
|
196
|
+
| ------- | ------- |
|
|
197
|
+
| [`@agent-inspect/adapter-sdk`](https://www.npmjs.com/package/@agent-inspect/adapter-sdk) | Third-party adapters (Beta) |
|
|
167
198
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
```
|
|
199
|
+
`agent-inspect-vscode` is in-repo (Marketplace not published yet).
|
|
200
|
+
|
|
201
|
+
</details>
|
|
172
202
|
|
|
173
|
-
|
|
203
|
+
## Documentation
|
|
204
|
+
|
|
205
|
+
| | Website | Repo |
|
|
206
|
+
| - | ------- | ---- |
|
|
207
|
+
| Getting started | [docs](https://agentinspect.vercel.app/docs/getting-started/) | [FIRST-TRACE](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md) |
|
|
208
|
+
| Contracts / suites / gates | — | [TRACE-CONTRACTS](https://github.com/rajudandigam/agent-inspect/blob/main/docs/TRACE-CONTRACTS.md) · [SUITES-COHORTS-GATES](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUITES-COHORTS-GATES.md) |
|
|
209
|
+
| Safe sharing | [safe-sharing](https://agentinspect.vercel.app/docs/safe-sharing/) | [SAFE-TRACE-SHARING](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
210
|
+
| API / CLI | — | [API](docs/API.md) · [CLI](docs/CLI.md) (packed with npm) |
|
|
211
|
+
| Full index | — | [docs/README.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/README.md) |
|
|
174
212
|
|
|
175
213
|
## Contributing
|
|
176
214
|
|
|
177
|
-
[CONTRIBUTING.md](CONTRIBUTING.md) · [Good first issues](GOOD-FIRST-ISSUES.md) · [Discussions](https://github.com/rajudandigam/agent-inspect/discussions)
|
|
215
|
+
[CONTRIBUTING.md](https://github.com/rajudandigam/agent-inspect/blob/main/CONTRIBUTING.md) · [Good first issues](https://github.com/rajudandigam/agent-inspect/blob/main/GOOD-FIRST-ISSUES.md) · [Discussions](https://github.com/rajudandigam/agent-inspect/discussions)
|
|
178
216
|
|
|
179
217
|
**Redact traces before posting issues or PRs.**
|
|
218
|
+
|
|
219
|
+
```bash
|
|
220
|
+
pnpm add agent-inspect
|
|
221
|
+
npx agent-inspect doctor
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Monorepo: `pnpm install && pnpm build && pnpm test`
|
package/docs/ADOPTION.md
CHANGED
|
@@ -20,7 +20,7 @@ npx agent-inspect init --yes
|
|
|
20
20
|
node examples/agent-inspect-demo.mjs
|
|
21
21
|
npx agent-inspect list --dir .agent-inspect
|
|
22
22
|
npx agent-inspect view <run-id> --dir .agent-inspect
|
|
23
|
-
npx agent-inspect verify-safe --dir .agent-inspect
|
|
23
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
No API keys required for the init demo. See [examples/starters/](../examples/starters/README.md).
|
package/docs/BUNDLES.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Shareable trace bundles
|
|
2
|
+
|
|
3
|
+
AgentInspect **bundles** are local, offline folders you can attach to PRs, incident threads, or internal reviews. They combine redacted trace copies, HTML reports, safety check results, and a human summary — without mutating source traces or calling the network.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
# Single run (default share profile + automatic verify-safe)
|
|
9
|
+
npx agent-inspect bundle <runId> --dir ./.agent-inspect
|
|
10
|
+
|
|
11
|
+
# Session or time window
|
|
12
|
+
npx agent-inspect bundle --session <sessionId> --dir ./.agent-inspect
|
|
13
|
+
npx agent-inspect bundle --since 24h --profile strict --dir ./.agent-inspect
|
|
14
|
+
|
|
15
|
+
# Explicit output folder (.zip suffix is stripped — folder-first MVP)
|
|
16
|
+
npx agent-inspect bundle <runId> --out ./my-bundle --json
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
When a workspace exists, default output goes under `.agent-inspect/bundles/`.
|
|
20
|
+
|
|
21
|
+
## Bundle layout
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
trace.html # offline HTML (index for multi-run)
|
|
25
|
+
trace.jsonl # redacted JSONL copy
|
|
26
|
+
summary.md # human overview
|
|
27
|
+
metadata.json # manifest (version, profile, safe status)
|
|
28
|
+
check-results.json # verify-safe results per run
|
|
29
|
+
redaction-report.json # detector summary (no secret values)
|
|
30
|
+
eval-results.json # placeholder unless eval artifacts are added later
|
|
31
|
+
performance-summary.json # placeholder unless perf artifacts are added later
|
|
32
|
+
assets/runs/<runId>.* # per-run HTML + JSONL mirrors
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Safety defaults
|
|
36
|
+
|
|
37
|
+
| Setting | Default |
|
|
38
|
+
| ------- | ------- |
|
|
39
|
+
| Redaction profile | `share` |
|
|
40
|
+
| verify-safe | Runs automatically before write |
|
|
41
|
+
| UNSAFE traces | Command fails unless `--allow-unsafe` |
|
|
42
|
+
| Source traces | Read-only; never modified |
|
|
43
|
+
|
|
44
|
+
Profiles:
|
|
45
|
+
|
|
46
|
+
- **`share`** — PR / internal support (default)
|
|
47
|
+
- **`strict`** — external or public sharing
|
|
48
|
+
- **`local`** — minimal redaction for local archives only
|
|
49
|
+
|
|
50
|
+
## When to use bundles vs other commands
|
|
51
|
+
|
|
52
|
+
| Need | Command |
|
|
53
|
+
| ---- | ------- |
|
|
54
|
+
| One redacted file | `redact` |
|
|
55
|
+
| Safety scan only | `verify-safe` |
|
|
56
|
+
| CI artifact set | `artifacts` |
|
|
57
|
+
| PR-ready evidence folder | **`bundle`** |
|
|
58
|
+
|
|
59
|
+
## Review before sharing
|
|
60
|
+
|
|
61
|
+
Bundles are **derived copies**, not compliance certification. Always review `summary.md`, `check-results.json`, and `trace.html` before attaching to tickets or PRs.
|
|
62
|
+
|
|
63
|
+
See also [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) and [CLI.md §6.24](./CLI.md#624-bundle).
|
|
64
|
+
|
|
65
|
+
## RFC
|
|
66
|
+
|
|
67
|
+
Design details: [proposals/SHAREABLE-BUNDLES-V4.3.md](./proposals/SHAREABLE-BUNDLES-V4.3.md).
|
package/docs/COMPARE.md
CHANGED
|
@@ -10,11 +10,12 @@ AgentInspect is a local-first trace workbench for TypeScript AI agents: capture
|
|
|
10
10
|
| --------- | ------------ | --------------------------------------- | --------------------------- | ------------------------------- |
|
|
11
11
|
| Where traces live | Local JSONL on your disk | Vendor or self-hosted backend | Vendor backend | Your collector + backend |
|
|
12
12
|
| Account / setup | None; `npm install` + CLI | Account or deployment | Account | SDK + collector + backend |
|
|
13
|
-
| Primary surface | CLI (`view`, `report`, `diff`, `check
|
|
13
|
+
| Primary surface | CLI (`view`, `report`, `diff`, `check`, suites/gates), local viewer, optional Studio Beta | Web dashboards | Web dashboards, eval UI | Dashboards, alerting |
|
|
14
14
|
| Evals | Deterministic local heuristics and CI gates (`check`, `eval`, `circuit`, `guardrails`) | Platform evals | Datasets, scoring, regressions at scale | Not the focus |
|
|
15
15
|
| Retention / fleet view | Not the goal | Strong | Strong | Strong |
|
|
16
16
|
| Data sharing | Explicit `redact` + `scan` / `verify-safe` before you share a file | Team access controls | Team access controls | Org pipelines |
|
|
17
|
-
| Network behavior | No upload;
|
|
17
|
+
| Network behavior | No default upload; explicit customer-owned ingest only | Uploads traces by design | Uploads by design | Ships telemetry by design |
|
|
18
|
+
| Team review | Optional customer-owned Studio Beta (not maintainer-hosted) | Strong | Strong | Strong |
|
|
18
19
|
|
|
19
20
|
If you need hosted retention, fleet dashboards, dataset management, or org-wide pipelines, use one of those platforms alongside AgentInspect. Boundaries are listed in [LIMITATIONS.md](./LIMITATIONS.md); concrete inner-loop workflows in [USE-CASES.md](./USE-CASES.md).
|
|
20
21
|
|
package/docs/DEMO-SCRIPT.md
CHANGED
|
@@ -42,19 +42,20 @@ Optional: `npx agent-inspect serve --dir .agent-inspect` for browser viewer.
|
|
|
42
42
|
## Beat 4 — Verify (30s)
|
|
43
43
|
|
|
44
44
|
```bash
|
|
45
|
-
npx agent-inspect check .agent-inspect
|
|
45
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
-
|
|
48
|
+
Optional flags: `--require-completed`, `--detect-stalls`, `--max-step-duration 30s`.
|
|
49
49
|
|
|
50
50
|
## Beat 5 — Share safely (30s)
|
|
51
51
|
|
|
52
52
|
```bash
|
|
53
|
-
npx agent-inspect redact .agent-inspect
|
|
54
|
-
npx agent-inspect verify-safe
|
|
53
|
+
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o safe.jsonl
|
|
54
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
55
|
+
# or: npx agent-inspect verify-safe safe.jsonl
|
|
55
56
|
```
|
|
56
57
|
|
|
57
|
-
"Redacted
|
|
58
|
+
"Redacted / verified-safe artifacts are safe to attach to a GitHub issue or Slack."
|
|
58
59
|
|
|
59
60
|
## Beat 6 — Fix and diff (optional, 30s)
|
|
60
61
|
|
package/docs/EXPORTS.md
CHANGED
|
@@ -36,6 +36,14 @@ agent-inspect export minimal-success --dir fixtures/traces --format markdown
|
|
|
36
36
|
- **HTML:** export writes a local file; a rendered-report visual is pending re-record ([RECORDING.md](./assets/demos/RECORDING.md)). Use `agent-inspect export <run-id> --format html -o report.html` and open the file locally.
|
|
37
37
|
- **OpenInference / OTLP JSON:** compatibility-oriented shapes for local handoff — validate with `--validate` before sharing. No dedicated GIF; see format tables below.
|
|
38
38
|
|
|
39
|
+
### Fixture-backed export evidence
|
|
40
|
+
|
|
41
|
+
The canonical OpenInference export of a persisted schema **1.0** trace is committed at
|
|
42
|
+
[`fixtures/standards/openinference-export-golden.json`](../fixtures/standards/openinference-export-golden.json)
|
|
43
|
+
(generated from `fixtures/traces-v1.0/manual-basic.jsonl`; regeneration is asserted byte-stable in
|
|
44
|
+
`packages/core/test/exporters/openinference-export-golden.test.ts`). Nanosecond timestamps are decimal strings,
|
|
45
|
+
matching the import fixtures and the OTLP exporter.
|
|
46
|
+
|
|
39
47
|
## Common options
|
|
40
48
|
|
|
41
49
|
| Flag | Notes |
|
|
@@ -90,6 +98,7 @@ Prefer the smallest useful artifact — a summary or synthetic reproduction beat
|
|
|
90
98
|
|
|
91
99
|
## Related docs
|
|
92
100
|
|
|
101
|
+
- [STANDARDS-GRADUATION.md](./STANDARDS-GRADUATION.md) — export → review → optional import path
|
|
93
102
|
- [CLI.md](./CLI.md) — `export` command reference
|
|
94
103
|
- [SCHEMA.md](./SCHEMA.md) — trace event model and redaction notes
|
|
95
104
|
- [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) — checklist before posting
|
|
@@ -1,13 +1,23 @@
|
|
|
1
1
|
# First trace in 5 minutes
|
|
2
2
|
|
|
3
|
-
Goal: install → one trace → one check → one share-safe
|
|
3
|
+
Goal: install → one trace → one check → one share-safe bundle.
|
|
4
4
|
|
|
5
5
|
**Docs site:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
|
|
6
6
|
|
|
7
7
|
```bash
|
|
8
|
-
npm install agent-inspect
|
|
9
|
-
npx agent-inspect
|
|
10
|
-
|
|
8
|
+
npm install agent-inspect
|
|
9
|
+
npx agent-inspect init --yes
|
|
10
|
+
node examples/agent-inspect-demo.mjs
|
|
11
|
+
npx agent-inspect list --dir .agent-inspect
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Copy a `<run-id>` from `list`, then:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
18
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
19
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
20
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
11
21
|
```
|
|
12
22
|
|
|
13
23
|
## Minutes 0–1: Install
|
|
@@ -18,7 +28,8 @@ npm install agent-inspect
|
|
|
18
28
|
npx agent-inspect init --yes
|
|
19
29
|
```
|
|
20
30
|
|
|
21
|
-
Creates `agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`.
|
|
31
|
+
Creates `agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`.
|
|
32
|
+
`init` scaffolds files; it does **not** write a trace by itself.
|
|
22
33
|
|
|
23
34
|
## Minutes 1–2: Run
|
|
24
35
|
|
|
@@ -39,17 +50,23 @@ npx agent-inspect report <run-id> --dir .agent-inspect
|
|
|
39
50
|
## Minutes 3–4: Check
|
|
40
51
|
|
|
41
52
|
```bash
|
|
42
|
-
npx agent-inspect check .agent-inspect
|
|
53
|
+
npx agent-inspect check <run-id> --dir .agent-inspect
|
|
43
54
|
```
|
|
44
55
|
|
|
45
56
|
## Minutes 4–5: Share-safe artifact
|
|
46
57
|
|
|
47
58
|
```bash
|
|
48
|
-
npx agent-inspect
|
|
49
|
-
npx agent-inspect verify-safe --dir .agent-inspect
|
|
59
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
60
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
50
61
|
```
|
|
51
62
|
|
|
52
|
-
Attach redacted
|
|
63
|
+
Attach the share-profile bundle (or a redacted file) to a PR or issue — not raw traces.
|
|
64
|
+
|
|
65
|
+
Optional file redaction:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redacted.jsonl
|
|
69
|
+
```
|
|
53
70
|
|
|
54
71
|
## Next steps
|
|
55
72
|
|
|
@@ -58,7 +75,8 @@ Attach redacted JSONL to a PR or issue — not raw traces.
|
|
|
58
75
|
| AI SDK | [AI SDK adoption](./AI-SDK-ADOPTION.md) |
|
|
59
76
|
| OpenAI Agents | [OpenAI Agents local](./OPENAI-AGENTS-LOCAL.md) |
|
|
60
77
|
| LangChain | [Adapters](./ADAPTERS.md) |
|
|
61
|
-
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) |
|
|
78
|
+
| CI tests | [CI artifacts](./CI-ARTIFACTS.md) · [Suites / gates](./SUITES-COHORTS-GATES.md) |
|
|
79
|
+
| Golden path | [GOLDEN-PATH.md](./GOLDEN-PATH.md) |
|
|
62
80
|
| Broken agent demo | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
|
|
63
81
|
|
|
64
82
|
Full index: [docs/README.md](./README.md) · Website docs: [agentinspect.vercel.app/docs](https://agentinspect.vercel.app/docs/)
|