agent-inspect 6.17.1 → 6.17.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +146 -182
- package/docs/CLI.md +1 -0
- package/docs/GOLDEN-PATH.md +1 -1
- package/docs/SAFE-TRACE-SHARING.md +2 -0
- package/docs/SCREENSHOTS.md +36 -204
- package/docs/SUPPORT-LEVELS.md +16 -5
- package/docs/assets/agent-inspect-logo-mark.svg +12 -0
- package/docs/assets/readme-product-loop.svg +17 -26
- package/package.json +2 -1
- package/packages/cli/dist/{chunk-LFSBMUCD.mjs → chunk-5GGYDIZD.mjs} +26 -6
- package/packages/cli/dist/chunk-5GGYDIZD.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +89 -25
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +68 -24
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-5J2HPOJ3.mjs → src-IDMCWKRH.mjs} +3 -3
- package/packages/cli/dist/{src-5J2HPOJ3.mjs.map → src-IDMCWKRH.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +24 -4
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.mjs +7 -2
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/checks.cjs +18 -3
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-NF4ZHNTC.mjs → chunk-UFP54T7F.mjs} +20 -5
- package/packages/core/dist/chunk-UFP54T7F.mjs.map +1 -0
- package/packages/cli/dist/chunk-LFSBMUCD.mjs.map +0 -1
- package/packages/core/dist/chunk-NF4ZHNTC.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.17.3
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 737bb03: Docs updates (README mark/loop, case-study template, Evidence retention guidance, support reproduction), TraceFacts/Evidence/OTLP/CLI regression tests, and exclude the RUN boundary from `run.slowestNode`.
|
|
8
|
+
|
|
9
|
+
## 6.17.2
|
|
10
|
+
|
|
11
|
+
### Patch Changes
|
|
12
|
+
|
|
13
|
+
- a7a7ea8: Union CLI check shorthands with preset select, resolve nested v0.1 LLM metadata in checks, and present Debug / Prevent / Share with curated showcase media.
|
|
14
|
+
|
|
3
15
|
## 6.17.1
|
|
4
16
|
|
|
5
17
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -1,27 +1,24 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<
|
|
3
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/agent-inspect-logo-dark.svg?sanitize=true">
|
|
4
|
-
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/agent-inspect-logo.svg?sanitize=true" width="240" alt="AgentInspect">
|
|
5
|
-
</picture>
|
|
2
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/agent-inspect-logo-mark.svg?sanitize=true" width="56" height="56" alt="AgentInspect">
|
|
6
3
|
</p>
|
|
4
|
+
<p align="center"><strong>agent-inspect</strong></p>
|
|
5
|
+
<p align="center">Local-first evidence for TypeScript AI agents</p>
|
|
7
6
|
|
|
8
|
-
<h1 align="center">
|
|
7
|
+
<h1 align="center">
|
|
8
|
+
See what your agent did.<br>
|
|
9
|
+
Catch the wrong path in CI.<br>
|
|
10
|
+
Keep the evidence local.
|
|
11
|
+
</h1>
|
|
9
12
|
|
|
10
13
|
<p align="center">
|
|
11
|
-
|
|
14
|
+
AgentInspect turns TypeScript agent runs into readable execution trees,
|
|
15
|
+
deterministic trajectory checks, and portable Evidence v2—without requiring
|
|
16
|
+
an account, collector, or default upload.
|
|
12
17
|
</p>
|
|
13
18
|
|
|
14
|
-
<p align="center">
|
|
15
|
-
AgentInspect captures framework-faithful execution trees, evaluates them with deterministic TraceFacts and TraceContract rules, creates integrity-verifiable Evidence v2, and lets coding assistants inspect the same local facts over read-only MCP—without a collector, account, or default upload.
|
|
16
|
-
</p>
|
|
19
|
+
<p align="center"><strong>Capture once. Debug, prevent, and share from the same local trace.</strong></p>
|
|
17
20
|
|
|
18
|
-
<p align="center">
|
|
19
|
-
<em>See what your agent did. Prove the fix. Keep the evidence.</em>
|
|
20
|
-
</p>
|
|
21
|
-
|
|
22
|
-
<p align="center">
|
|
23
|
-
<sub>No account · no collector · no default upload · metadata-only by default</sub>
|
|
24
|
-
</p>
|
|
21
|
+
<p align="center"><sub>No account · no collector · no default upload · metadata-only by default</sub></p>
|
|
25
22
|
|
|
26
23
|
<p align="center">
|
|
27
24
|
<a href="https://agentinspect.vercel.app/">Website</a> ·
|
|
@@ -32,234 +29,201 @@
|
|
|
32
29
|
|
|
33
30
|
<p align="center">
|
|
34
31
|
<a href="https://www.npmjs.com/package/agent-inspect"><img src="https://img.shields.io/npm/v/agent-inspect.svg" alt="npm version"></a>
|
|
35
|
-
<a href="LICENSE"><img src="https://img.shields.io/badge/
|
|
36
|
-
<a href="package.json"><img src="https://img.shields.io/badge/node-%3E%3D20-brightgreen" alt="Node.js
|
|
37
|
-
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-strict-3178c6" alt="TypeScript"></a>
|
|
32
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT license"></a>
|
|
33
|
+
<a href="package.json"><img src="https://img.shields.io/badge/node-%3E%3D20-brightgreen" alt="Node.js 20 or newer"></a>
|
|
38
34
|
</p>
|
|
39
35
|
|
|
40
36
|
```bash
|
|
41
37
|
npm install agent-inspect
|
|
42
38
|
```
|
|
43
39
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
| **Test behavior deterministically** | TraceFacts, TraceContract, checks, suites, gates, and experimental Vitest/Jest matchers |
|
|
50
|
-
| **Produce portable evidence** | Offline Evidence v2 with integrity verification and share-policy disclosure |
|
|
51
|
-
| **Debug with coding assistants locally** | Read-only MCP over the same TraceFacts — no collector required |
|
|
52
|
-
|
|
53
|
-
```text
|
|
54
|
-
1. Capture one real run
|
|
55
|
-
2. Find the causal failure
|
|
56
|
-
3. Ask your coding agent to inspect it
|
|
57
|
-
4. Lock the fix with a contract
|
|
58
|
-
5. Attach the share-checked evidence
|
|
59
|
-
```
|
|
40
|
+
<p align="center">
|
|
41
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/showcase/gif/debug-tree.gif" alt="AgentInspect lists a local demo-good run, then you inspect the execution tree from the same JSONL" width="900">
|
|
42
|
+
</p>
|
|
43
|
+
|
|
44
|
+
Agent code rarely fails as one function call. It plans, retrieves, calls tools, invokes a model, retries, and produces side effects. Flat logs show fragments. AgentInspect keeps the run as local JSONL and gives you one evidence loop from the same trace.
|
|
60
45
|
|
|
61
46
|
<p align="center">
|
|
62
|
-
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture
|
|
47
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/readme-product-loop.svg?sanitize=true" alt="Capture one local trace, then debug, prevent, and share from the same evidence loop" width="900">
|
|
63
48
|
</p>
|
|
64
49
|
|
|
65
|
-
##
|
|
50
|
+
## One trace. Three jobs.
|
|
66
51
|
|
|
67
|
-
|
|
52
|
+
### Debug — read the execution path
|
|
68
53
|
|
|
69
|
-
|
|
70
|
-
npm install agent-inspect
|
|
71
|
-
npx agent-inspect init --yes
|
|
72
|
-
# or: npx agent-inspect init --framework langgraph --yes
|
|
73
|
-
node examples/agent-inspect-demo.mjs
|
|
74
|
-
npx agent-inspect list --dir .agent-inspect
|
|
75
|
-
```
|
|
54
|
+
See nested steps, tool calls, LLM calls, model and token metadata, durations, errors, and the first causal failure without sending the trace to an AgentInspect service.
|
|
76
55
|
|
|
77
56
|
```bash
|
|
78
|
-
|
|
57
|
+
npx agent-inspect view <run-id> --dir .agent-inspect --summary
|
|
79
58
|
npx agent-inspect report <run-id> --dir .agent-inspect
|
|
80
|
-
npx agent-inspect
|
|
81
|
-
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
|
|
82
|
-
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
83
|
-
npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
|
|
84
|
-
# Optional coding-agent loop (dry-run by default):
|
|
85
|
-
npx agent-inspect mcp configure --client cursor
|
|
59
|
+
npx agent-inspect explain <run-id> --dir .agent-inspect
|
|
86
60
|
```
|
|
87
61
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
## Why AgentInspect
|
|
91
|
-
|
|
92
|
-
**Category:** the local evidence debugger and trajectory-test toolkit for TypeScript AI agents — not a hosted APM, eval host, or prompt registry. Complements LangSmith/Langfuse/Phoenix; owns the laptop → PR loop.
|
|
62
|
+
Use manual instrumentation, official adapters, supported standards files, or structured logs you already emit.
|
|
93
63
|
|
|
94
|
-
|
|
64
|
+
### Prevent — fail CI on the wrong trajectory
|
|
95
65
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
| TraceFacts / logical projection | Canonical tool identity and semantic parity without inventing hierarchy |
|
|
100
|
-
| Deterministic checks / TraceContract (Beta) | Trajectory expectations without an LLM judge |
|
|
101
|
-
| Experimental Vitest/Jest matchers | `toPassTraceContract` · `toHaveRequiredTool` |
|
|
102
|
-
| Share-checked Evidence v2 | `bundle` + `bundle verify` with optional TraceFacts semantics |
|
|
103
|
-
| Read-only MCP coding-agent loop (Preview) | `get_trace_facts` and related tools over local traces |
|
|
104
|
-
|
|
105
|
-
## Choose your capture path
|
|
66
|
+
<p align="center">
|
|
67
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/showcase/gif/check-pass-fail.gif" alt="The same deterministic AgentInspect check exits zero for demo-good and one for demo-regression" width="900">
|
|
68
|
+
</p>
|
|
106
69
|
|
|
107
|
-
|
|
108
|
-
| ---- | -------- | ----- |
|
|
109
|
-
| **Manual / observe** | Custom nesting or object methods | [Getting started](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GETTING-STARTED.md) |
|
|
110
|
-
| **AI SDK** | Vercel AI SDK `generateText` / `streamText` | [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) |
|
|
111
|
-
| **OpenAI Agents** | OpenAI Agents JS | [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) |
|
|
112
|
-
| **LangChain / LangGraph** | Callbacks / LangGraph | [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) · `init --framework langgraph` |
|
|
113
|
-
| **Structured logs** | Logs already emitted | [Log-to-tree](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
114
|
-
| **Harness** | Fixture runner for real projects | [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) |
|
|
115
|
-
| **CI reporters + matchers** | Failed-test artifacts and contracts | [`vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) · [`jest`](https://www.npmjs.com/package/@agent-inspect/jest) |
|
|
116
|
-
| **Standards files** | OpenInference / OTLP JSON | [Standards](https://github.com/rajudandigam/agent-inspect/blob/main/docs/STANDARDS.md) |
|
|
70
|
+
Checks are deterministic and provider-free: the same trace and rules produce the same result. Start with a preset, then extend it with the expectations that matter to the workflow.
|
|
117
71
|
|
|
118
|
-
|
|
72
|
+
```bash
|
|
73
|
+
npx agent-inspect check <run-id> --dir .agent-inspect \
|
|
74
|
+
--preset trajectory \
|
|
75
|
+
--required-tool retrieve_policy \
|
|
76
|
+
--fail-on-observation failed
|
|
77
|
+
```
|
|
119
78
|
|
|
120
|
-
|
|
79
|
+
A passing check exits `0`; a rule failure exits `1`. Invalid configuration and unreadable/unsupported inputs use separate documented exit codes. Use TraceContract, suites, cohorts, Vitest/Jest reporters, or `--evidence-on fail` when the workflow needs more than one CLI check.
|
|
121
80
|
|
|
122
|
-
|
|
81
|
+
### Share — create reviewable offline evidence
|
|
123
82
|
|
|
124
|
-
|
|
83
|
+
<p align="center">
|
|
84
|
+
<img src="https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/showcase/gif/evidence-bundle.gif" alt="AgentInspect writes a share-checked Evidence v2 bundle to ./evidence and verifies the listed file hashes offline" width="900">
|
|
85
|
+
</p>
|
|
125
86
|
|
|
126
|
-
|
|
87
|
+
```bash
|
|
88
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
89
|
+
npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share --out ./evidence
|
|
90
|
+
npx agent-inspect bundle verify ./evidence
|
|
91
|
+
```
|
|
127
92
|
|
|
128
|
-
|
|
93
|
+
`verify-safe` is a best-effort local assessment and can report `SAFE`, `SAFE WITH WARNINGS`, `UNSAFE`, or `UNKNOWN`. `bundle` writes a redacted artifact and manifest; `bundle verify` rechecks the listed file hashes offline. “Share-checked” is not compliance certification—review the artifact before attaching it to a PR, incident, or support handoff.
|
|
129
94
|
|
|
130
|
-
|
|
95
|
+
## First local trace
|
|
131
96
|
|
|
132
|
-
|
|
97
|
+
The generated demo is synthetic, keyless, and local. `npm install agent-inspect` does not copy repository examples; `init` writes them into the project.
|
|
133
98
|
|
|
134
|
-
|
|
99
|
+
```bash
|
|
100
|
+
npm install agent-inspect
|
|
101
|
+
npx agent-inspect init --yes
|
|
102
|
+
node examples/agent-inspect-demo.mjs
|
|
103
|
+
npx agent-inspect list --dir .agent-inspect
|
|
104
|
+
```
|
|
135
105
|
|
|
136
|
-
|
|
137
|
-
| -------- | ----- |
|
|
138
|
-
| Wrong tool / intentional failure | [broken-agent-debugging](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging) |
|
|
139
|
-
| Coding-agent MCP debug loop | [coding-agent-debug-loop](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/coding-agent-debug-loop) · [CODING-AGENT-LOOP](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) |
|
|
140
|
-
| CI trajectory gate + Evidence | [ci-eval-redact](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/ci-eval-redact) · [langgraph-gate-evidence](https://github.com/rajudandigam/agent-inspect/tree/main/examples/recipes/langgraph-gate-evidence) |
|
|
141
|
-
| Safe incident handoff | [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md) |
|
|
142
|
-
| Multi-agent / session retry | [Sessions & outcomes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SESSIONS-AND-OUTCOMES.md) |
|
|
143
|
-
| Customer-owned team review | [Self-hosting](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SELF-HOSTING.md) · Studio (optional) |
|
|
106
|
+
Copy the printed run ID, then inspect it:
|
|
144
107
|
|
|
145
|
-
|
|
108
|
+
```bash
|
|
109
|
+
npx agent-inspect view <run-id> --dir .agent-inspect --summary
|
|
110
|
+
npx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory
|
|
111
|
+
npx agent-inspect verify-safe <run-id> --dir .agent-inspect
|
|
112
|
+
```
|
|
146
113
|
|
|
147
|
-
-
|
|
148
|
-
- **Metadata-only by default** — no raw prompts/outputs unless you opt in
|
|
149
|
-
- **No hidden upload** — core does not send traces to AgentInspect
|
|
150
|
-
- **Customer-owned Studio ingestion** is disabled by default and explicit when enabled
|
|
151
|
-
- **MCP server** exposes configured local evidence to the connected client (Preview)
|
|
152
|
-
- **Standards export** only when you run/configure it
|
|
153
|
-
- Redaction is **best-effort**, not certification — review before posting
|
|
154
|
-
- **Not** a chain-of-thought recorder
|
|
114
|
+
For a fixed good/regression/PII walkthrough with stable run ids (`demo-good`, `demo-regression`, `demo-pii`), use the [canonical keyless showcase](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging). The richer [blessed starters](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters) cover framework-specific paths.
|
|
155
115
|
|
|
156
|
-
|
|
116
|
+
## Use it with your stack
|
|
157
117
|
|
|
158
|
-
|
|
118
|
+
| Stack | Install / command | Start |
|
|
119
|
+
| --- | --- | --- |
|
|
120
|
+
| Custom functions or classes | `agent-inspect` | [Getting started](https://github.com/rajudandigam/agent-inspect/blob/main/docs/GETTING-STARTED.md) |
|
|
121
|
+
| LangChain / LangGraph | `agent-inspect` + `@agent-inspect/langchain` | [LangGraph guide](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LANGGRAPH.md) |
|
|
122
|
+
| AI SDK | `agent-inspect` + `@agent-inspect/ai-sdk` | [AI SDK guide](https://github.com/rajudandigam/agent-inspect/blob/main/docs/AI-SDK-ADOPTION.md) |
|
|
123
|
+
| OpenAI Agents JS | `agent-inspect` + `@agent-inspect/openai-agents` | [Local processor guide](https://github.com/rajudandigam/agent-inspect/blob/main/docs/OPENAI-AGENTS-LOCAL.md) |
|
|
124
|
+
| Existing structured logs | `npx agent-inspect logs ...` | [Log-to-tree quickstart](https://github.com/rajudandigam/agent-inspect/blob/main/docs/LOG-TO-TREE-QUICKSTART.md) |
|
|
125
|
+
| OpenInference / OTLP JSON | `npx agent-inspect open ...` | [Standards](https://github.com/rajudandigam/agent-inspect/blob/main/docs/STANDARDS.md) |
|
|
126
|
+
| Vitest / Jest CI | `@agent-inspect/vitest` / `@agent-inspect/jest` | [CI artifacts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CI-ARTIFACTS.md) |
|
|
159
127
|
|
|
160
|
-
|
|
128
|
+
See [support levels](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SUPPORT-LEVELS.md) for Stable, Supported, Beta, Preview, and Experimental surfaces.
|
|
161
129
|
|
|
162
|
-
|
|
130
|
+
## Programmatic trajectory checks
|
|
163
131
|
|
|
164
|
-
|
|
132
|
+
```ts
|
|
133
|
+
import { openTraceFile } from "agent-inspect/readers";
|
|
134
|
+
import {
|
|
135
|
+
defineTraceContract,
|
|
136
|
+
evaluateTraceContractRead,
|
|
137
|
+
} from "agent-inspect/checks";
|
|
165
138
|
|
|
166
|
-
|
|
139
|
+
const read = await openTraceFile("./trace.jsonl");
|
|
140
|
+
const contract = defineTraceContract({
|
|
141
|
+
run: { requireCompleted: true },
|
|
142
|
+
tools: { required: ["retrieve_policy"] },
|
|
143
|
+
});
|
|
167
144
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
- Prompt registry, pricing engine, or replay engine
|
|
172
|
-
- Universal standards exporter
|
|
173
|
-
- Compliance certification
|
|
145
|
+
const result = evaluateTraceContractRead(read, contract);
|
|
146
|
+
if (result.status !== "pass") process.exitCode = 1;
|
|
147
|
+
```
|
|
174
148
|
|
|
175
|
-
See [
|
|
149
|
+
The full API also exposes TraceFacts for bounded, logical analysis of supported traces. See [programmatic trace analysis](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PROGRAMMATIC-TRACE-ANALYSIS.md).
|
|
176
150
|
|
|
177
|
-
|
|
178
|
-
<summary><strong>Package family (18 linked packages · tiered presentation)</strong></summary>
|
|
151
|
+
## Where AgentInspect fits
|
|
179
152
|
|
|
180
|
-
|
|
153
|
+
AgentInspect owns the laptop-to-PR evidence loop. It complements hosted observability, production trace retention, and broad eval platforms rather than replacing them.
|
|
181
154
|
|
|
182
|
-
|
|
155
|
+
| Need | Best fit |
|
|
156
|
+
| --- | --- |
|
|
157
|
+
| Inspect one local TypeScript agent run immediately | AgentInspect |
|
|
158
|
+
| Enforce deterministic tool/order/completion expectations in CI | AgentInspect |
|
|
159
|
+
| Attach a redacted, hash-verifiable offline artifact | AgentInspect |
|
|
160
|
+
| Monitor a production fleet with hosted dashboards and retention | Hosted observability / APM |
|
|
161
|
+
| Manage prompts, datasets, and online LLM-judge experiments | Eval / prompt platforms |
|
|
183
162
|
|
|
184
|
-
|
|
185
|
-
| ------- | ------- |
|
|
186
|
-
| [`agent-inspect`](https://www.npmjs.com/package/agent-inspect) | Core APIs + CLI + evidence |
|
|
187
|
-
| [`@agent-inspect/redact`](https://www.npmjs.com/package/@agent-inspect/redact) | Deterministic redaction |
|
|
188
|
-
| [`@agent-inspect/mcp-server`](https://www.npmjs.com/package/@agent-inspect/mcp-server) | Read-only MCP coding-agent loop (Preview) |
|
|
163
|
+
See the factual [comparison guide](https://github.com/rajudandigam/agent-inspect/blob/main/docs/COMPARE.md).
|
|
189
164
|
|
|
190
|
-
|
|
165
|
+
## Safety and network behavior
|
|
191
166
|
|
|
192
|
-
|
|
167
|
+
- Traces are local JSONL under `.agent-inspect/` or `AGENT_INSPECT_TRACE_DIR`.
|
|
168
|
+
- Capture is metadata-only by default; raw prompts and outputs require opt-in.
|
|
169
|
+
- Write-time redaction is enabled by default. The `redact` command creates a separate redacted copy and does not mutate its source.
|
|
170
|
+
- Core capture and inspection do not require an AgentInspect account, collector, or default upload.
|
|
171
|
+
- Redaction and `verify-safe` are best-effort controls, not privacy, security, or compliance certification.
|
|
172
|
+
- Optional MCP, standards export, and customer-owned Studio surfaces are explicit and have their own network boundaries.
|
|
173
|
+
- AgentInspect is not a chain-of-thought recorder.
|
|
193
174
|
|
|
194
|
-
|
|
195
|
-
| ------- | ------- |
|
|
196
|
-
| [`@agent-inspect/ai-sdk`](https://www.npmjs.com/package/@agent-inspect/ai-sdk) | AI SDK telemetry |
|
|
197
|
-
| [`@agent-inspect/openai-agents`](https://www.npmjs.com/package/@agent-inspect/openai-agents) | OpenAI Agents processor |
|
|
198
|
-
| [`@agent-inspect/langchain`](https://www.npmjs.com/package/@agent-inspect/langchain) | LangChain / LangGraph callbacks |
|
|
175
|
+
Read [network behavior](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NETWORK-BEHAVIOR.md), [no-egress policy](https://github.com/rajudandigam/agent-inspect/blob/main/docs/NO-EGRESS-POLICY.md), and [safe trace sharing](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SAFE-TRACE-SHARING.md).
|
|
199
176
|
|
|
200
|
-
|
|
177
|
+
## Optional coding-agent loop
|
|
201
178
|
|
|
202
|
-
|
|
203
|
-
| ------- | ------- |
|
|
204
|
-
| [`@agent-inspect/harness`](https://www.npmjs.com/package/@agent-inspect/harness) | Fixture runner |
|
|
205
|
-
| [`@agent-inspect/eval`](https://www.npmjs.com/package/@agent-inspect/eval) | Local eval heuristics |
|
|
206
|
-
| [`@agent-inspect/vitest`](https://www.npmjs.com/package/@agent-inspect/vitest) | Vitest reporter + experimental matchers |
|
|
207
|
-
| [`@agent-inspect/jest`](https://www.npmjs.com/package/@agent-inspect/jest) | Jest reporter + experimental matchers |
|
|
179
|
+
`@agent-inspect/mcp-server` exposes configured local evidence to an MCP client through bounded, read-only tools. It is optional and currently Preview; the core debug/check/evidence loop does not require it.
|
|
208
180
|
|
|
209
|
-
|
|
181
|
+
```bash
|
|
182
|
+
npx agent-inspect mcp configure --client cursor
|
|
183
|
+
```
|
|
210
184
|
|
|
211
|
-
|
|
212
|
-
| ------- | ------- |
|
|
213
|
-
| [`@agent-inspect/redact`](https://www.npmjs.com/package/@agent-inspect/redact) | Deterministic redaction |
|
|
214
|
-
| [`@agent-inspect/guardrails`](https://www.npmjs.com/package/@agent-inspect/guardrails) | Deterministic guardrail rules |
|
|
215
|
-
| [`@agent-inspect/circuit`](https://www.npmjs.com/package/@agent-inspect/circuit) | Loop / retry / timeout analyzers |
|
|
185
|
+
Review the generated dry-run configuration and the [coding-agent loop guide](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CODING-AGENT-LOOP.md) before enabling it.
|
|
216
186
|
|
|
217
|
-
|
|
187
|
+
<details>
|
|
188
|
+
<summary><strong>Package family (18-package fixed release group)</strong></summary>
|
|
218
189
|
|
|
219
|
-
|
|
|
220
|
-
|
|
|
221
|
-
|
|
|
222
|
-
|
|
|
223
|
-
|
|
|
224
|
-
|
|
|
190
|
+
| Group | Packages |
|
|
191
|
+
| --- | --- |
|
|
192
|
+
| Core product | `agent-inspect`, `@agent-inspect/redact` |
|
|
193
|
+
| Framework integrations | `@agent-inspect/ai-sdk`, `@agent-inspect/openai-agents`, `@agent-inspect/langchain` |
|
|
194
|
+
| Testing and evaluation | `@agent-inspect/harness`, `@agent-inspect/eval`, `@agent-inspect/vitest`, `@agent-inspect/jest` |
|
|
195
|
+
| Safety analysis | `@agent-inspect/guardrails`, `@agent-inspect/circuit` |
|
|
196
|
+
| Developer surfaces | `@agent-inspect/viewer`, `@agent-inspect/tui`, `@agent-inspect/mcp`, `@agent-inspect/mcp-server` |
|
|
197
|
+
| Local/team optional surfaces | `@agent-inspect/index-sqlite`, `@agent-inspect/studio` |
|
|
198
|
+
| Extension SDK | `@agent-inspect/adapter-sdk` |
|
|
225
199
|
|
|
226
|
-
|
|
200
|
+
The root package is enough for custom capture, the CLI, checks, and Evidence workflows. Install optional packages only for the integration or surface you use. `agent-inspect-vscode` is currently in the repository and is not presented as a published Marketplace extension.
|
|
227
201
|
|
|
228
|
-
|
|
229
|
-
| ------- | ------- |
|
|
230
|
-
| [`@agent-inspect/index-sqlite`](https://www.npmjs.com/package/@agent-inspect/index-sqlite) | Optional SQLite index (Beta) |
|
|
231
|
-
| [`@agent-inspect/studio`](https://www.npmjs.com/package/@agent-inspect/studio) | Customer-owned Studio (Beta) |
|
|
202
|
+
</details>
|
|
232
203
|
|
|
233
|
-
|
|
204
|
+
## What it is not
|
|
234
205
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
206
|
+
- A maintainer-hosted SaaS or production APM replacement
|
|
207
|
+
- Hosted trace retention, a team dashboard service, or a prompt registry
|
|
208
|
+
- An LLM-as-judge or dataset platform by default
|
|
209
|
+
- A replay engine, automatic remediation system, or compliance certification
|
|
238
210
|
|
|
239
|
-
|
|
211
|
+
## Status and documentation
|
|
240
212
|
|
|
241
|
-
|
|
213
|
+
**Current published baseline:** **6.17.2** · persisted schema `1.0` · Node.js `>=20` · MIT.
|
|
242
214
|
|
|
243
|
-
|
|
215
|
+
Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
|
|
244
216
|
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
| Full index | — | [docs/README.md](https://github.com/rajudandigam/agent-inspect/blob/main/docs/README.md) |
|
|
217
|
+
- [First trace in five minutes](https://github.com/rajudandigam/agent-inspect/blob/main/docs/FIRST-TRACE-IN-5-MINUTES.md)
|
|
218
|
+
- [CLI reference](https://github.com/rajudandigam/agent-inspect/blob/main/docs/CLI.md)
|
|
219
|
+
- [Trace contracts](https://github.com/rajudandigam/agent-inspect/blob/main/docs/TRACE-CONTRACTS.md)
|
|
220
|
+
- [Evidence format](https://github.com/rajudandigam/agent-inspect/blob/main/docs/EVIDENCE-FORMAT.md)
|
|
221
|
+
- [Use cases](https://github.com/rajudandigam/agent-inspect/blob/main/docs/USE-CASES.md)
|
|
222
|
+
- [Demos and screenshots](https://github.com/rajudandigam/agent-inspect/blob/main/docs/SCREENSHOTS.md)
|
|
223
|
+
- [Full documentation index](https://github.com/rajudandigam/agent-inspect/blob/main/docs/README.md)
|
|
253
224
|
|
|
254
225
|
## Contributing
|
|
255
226
|
|
|
256
|
-
[CONTRIBUTING.md](
|
|
257
|
-
|
|
258
|
-
**Redact traces before posting issues or PRs.**
|
|
259
|
-
|
|
260
|
-
```bash
|
|
261
|
-
pnpm add agent-inspect
|
|
262
|
-
npx agent-inspect doctor
|
|
263
|
-
```
|
|
227
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md), [good first issues](GOOD-FIRST-ISSUES.md), and [GitHub Discussions](https://github.com/rajudandigam/agent-inspect/discussions).
|
|
264
228
|
|
|
265
|
-
|
|
229
|
+
**Redact and review traces before posting them in issues or pull requests.**
|
package/docs/CLI.md
CHANGED
|
@@ -322,6 +322,7 @@ Options:
|
|
|
322
322
|
- `trajectory`: completion/structure/relationship focus; excludes share-safety findings
|
|
323
323
|
- `safety`: raw-content / secret / redaction focus
|
|
324
324
|
- `comprehensive`: union of trajectory and safety
|
|
325
|
+
- Presets are a base select set. CLI shorthands on the same invocation (`--fail-on-observation`, `--required-tool`, `--forbidden-tool`, `--allowed-model`, `--max-total-tokens`, `--max-duration-ms`, `--max-step-duration`, `--detect-stalls`) extend that set; they are not dropped because the preset already selected rules. Config `checks.select` is not silently expanded with unrelated configured rules.
|
|
325
326
|
- `--evidence-on <fail|always|never>`: write local Evidence v2 (no upload); omitted = never
|
|
326
327
|
- `--evidence-dir <path>`: Evidence output directory or base path
|
|
327
328
|
- `--evidence-profile <local|share|strict>`: redaction profile for Evidence (default `share`)
|
package/docs/GOLDEN-PATH.md
CHANGED
|
@@ -28,6 +28,8 @@ Manual traces redact common sensitive keys **before disk** by default. Pass `red
|
|
|
28
28
|
|
|
29
29
|
## Before sharing
|
|
30
30
|
|
|
31
|
+
When a maintainer or support responder needs reproducible evidence, follow the [safe support reproduction workflow](./SUPPORT-REPRODUCTION.md) to create and review a minimized Evidence bundle. Do not attach a raw trace directory.
|
|
32
|
+
|
|
31
33
|
- Use **`--redaction-profile share`** for PR/issue attachments; use **`strict`** when sharing outside your team.
|
|
32
34
|
- **Review** the exported file — profiles do not detect all sensitive data.
|
|
33
35
|
- Treat traces written with `redact: false` as sensitive. Review every event before sharing them outside your team.
|