agent-inspect 1.7.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -1
- package/README.md +145 -21
- package/docs/ADAPTER-CONFORMANCE.md +7 -3
- package/docs/ADAPTERS.md +155 -5
- package/docs/API.md +214 -26
- package/docs/CLI.md +189 -7
- package/docs/GETTING-STARTED.md +71 -16
- package/docs/KNOWN-ISSUES.md +7 -1
- package/docs/LIMITATIONS.md +7 -1
- package/docs/LOG-TO-TREE-QUICKSTART.md +1 -2
- package/docs/MIGRATION.md +67 -0
- package/docs/SCHEMA.md +1 -0
- package/package.json +12 -2
- package/packages/cli/dist/index.cjs +2454 -140
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +2454 -140
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +4 -4
- package/packages/core/dist/advanced.d.ts +4 -4
- package/packages/core/dist/checks.cjs +1535 -0
- package/packages/core/dist/checks.cjs.map +1 -0
- package/packages/core/dist/checks.d.cts +585 -0
- package/packages/core/dist/checks.d.ts +585 -0
- package/packages/core/dist/checks.mjs +1512 -0
- package/packages/core/dist/checks.mjs.map +1 -0
- package/packages/core/dist/diff.d.cts +3 -3
- package/packages/core/dist/diff.d.ts +3 -3
- package/packages/core/dist/exporters.d.cts +3 -3
- package/packages/core/dist/exporters.d.ts +3 -3
- package/packages/core/dist/index.cjs +146 -0
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +44 -7
- package/packages/core/dist/index.d.ts +44 -7
- package/packages/core/dist/index.mjs +148 -1
- package/packages/core/dist/index.mjs.map +1 -1
- package/packages/core/dist/{inspect-event-Des4JDHo.d.cts → inspect-event-CevRYp58.d.cts} +1 -1
- package/packages/core/dist/{inspect-event-Des4JDHo.d.ts → inspect-event-CevRYp58.d.ts} +1 -1
- package/packages/core/dist/{log-config-C1GcJPIM.d.ts → log-config-BPHS4Sds.d.ts} +1 -1
- package/packages/core/dist/{log-config-BnH8Ykcb.d.cts → log-config-DanPV3P9.d.cts} +1 -1
- package/packages/core/dist/logs.d.cts +3 -3
- package/packages/core/dist/logs.d.ts +3 -3
- package/packages/core/dist/{persisted-inspect-event-DiFto0K2.d.ts → persisted-inspect-event-Cw7TeYGr.d.ts} +1 -1
- package/packages/core/dist/{persisted-inspect-event-0kaRADsp.d.cts → persisted-inspect-event-DHPfzUd8.d.cts} +1 -1
- package/packages/core/dist/persisted.d.cts +5 -5
- package/packages/core/dist/persisted.d.ts +5 -5
- package/packages/core/dist/readers.d.cts +2 -2
- package/packages/core/dist/readers.d.ts +2 -2
- package/packages/core/dist/{types-tSix7tfv.d.ts → types-Ap9uMdx_.d.ts} +1 -1
- package/packages/core/dist/{types-DB8jB6Jg.d.cts → types-B2-BU5CS.d.cts} +1 -1
- package/packages/core/dist/writers.d.cts +2 -2
- package/packages/core/dist/writers.d.ts +2 -2
package/docs/API.md
CHANGED
|
@@ -9,14 +9,42 @@ AgentInspect is a **local-first execution-tree debugger**. It is not a SaaS, not
|
|
|
9
9
|
- **Stable**: intended to be compatible across v1.x. Breaking changes require v2.0.
|
|
10
10
|
- **Experimental**: available for adoption, but subject to refinement (including naming/shape changes) before a future stability declaration. Experimental APIs may change in v1.x.
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
Use the root import for stable beginner APIs. Use subpaths for advanced, experimental, or lower-level workflows.
|
|
13
|
+
|
|
14
|
+
```ts
|
|
15
|
+
import {
|
|
16
|
+
observe,
|
|
17
|
+
inspectRun,
|
|
18
|
+
maybeInspectRun,
|
|
19
|
+
step,
|
|
20
|
+
getCurrentCorrelationMetadata,
|
|
21
|
+
} from "agent-inspect";
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
**v1.9 root API direction:** do not add new root exports casually. Existing root imports keep working through v1.x for compatibility, but new advanced examples should use the subpath where the API lives. The intended stable root set for v2 is:
|
|
25
|
+
|
|
26
|
+
```ts
|
|
27
|
+
import {
|
|
28
|
+
createInspector,
|
|
29
|
+
observe,
|
|
30
|
+
inspectRun,
|
|
31
|
+
maybeInspectRun,
|
|
32
|
+
step,
|
|
33
|
+
getCurrentCorrelationMetadata,
|
|
34
|
+
} from "agent-inspect";
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
**1.x subpath exports:** Additive subpaths (`/logs`, `/exporters`, `/persisted`, `/diff`, `/advanced`, `/writers`, `/readers`, `/checks`) narrow the import surface for experimental and advanced APIs. Root `"."` imports remain valid through v1.x. Design: [API-BOUNDARY-V1.5.md](./implementation/API-BOUNDARY-V1.5.md).
|
|
13
38
|
|
|
14
39
|
```ts
|
|
15
|
-
import { inspectRun, step } from "agent-inspect";
|
|
16
40
|
import { parseLogsToTrees } from "agent-inspect/logs";
|
|
17
41
|
import { exportMarkdown } from "agent-inspect/exporters";
|
|
18
42
|
import { memoryWriter } from "agent-inspect/writers";
|
|
19
43
|
import { openTrace } from "agent-inspect/readers";
|
|
44
|
+
import { runTraceChecks } from "agent-inspect/checks";
|
|
45
|
+
import { diffTraceEvents } from "agent-inspect/diff";
|
|
46
|
+
import { traceEventsToPersistedInspectEvents } from "agent-inspect/persisted";
|
|
47
|
+
import { createInspector } from "agent-inspect/advanced";
|
|
20
48
|
```
|
|
21
49
|
|
|
22
50
|
Notes:
|
|
@@ -24,6 +52,7 @@ Notes:
|
|
|
24
52
|
- The core guarantee of v1.x is **stable local debugging**: manual tracing + CLI inspection.
|
|
25
53
|
- Export formats (OpenInference / OTLP JSON) are **local-only** and **compatibility-oriented**. They do **not** upload anywhere.
|
|
26
54
|
- There are **zero production sinks** in v1.x; sink/uploader APIs are not stable.
|
|
55
|
+
- Advanced root exports in v1.x are compatibility aliases. Prefer `agent-inspect/advanced`, `agent-inspect/readers`, `agent-inspect/writers`, `agent-inspect/checks`, `agent-inspect/diff`, `agent-inspect/exporters`, `agent-inspect/logs`, and `agent-inspect/persisted` for new code.
|
|
27
56
|
|
|
28
57
|
## 2. Stable core APIs (manual tracing)
|
|
29
58
|
|
|
@@ -32,7 +61,13 @@ These are the recommended entry points for manual instrumentation. They are desi
|
|
|
32
61
|
Import from `agent-inspect`:
|
|
33
62
|
|
|
34
63
|
```ts
|
|
35
|
-
import {
|
|
64
|
+
import {
|
|
65
|
+
observe,
|
|
66
|
+
inspectRun,
|
|
67
|
+
maybeInspectRun,
|
|
68
|
+
step,
|
|
69
|
+
getCurrentCorrelationMetadata,
|
|
70
|
+
} from "agent-inspect";
|
|
36
71
|
```
|
|
37
72
|
|
|
38
73
|
- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output.
|
|
@@ -90,7 +125,7 @@ These APIs support local workflows like listing traces, extracting metadata/summ
|
|
|
90
125
|
|
|
91
126
|
## 5. Experimental log parsing APIs
|
|
92
127
|
|
|
93
|
-
These are compatibility-oriented utilities for turning structured logs into normalized `InspectEvent` and grouped trees. They remain conservative: **no eval**, **no parsing JS object literals**, JSON logs first-class, log4js best-effort.
|
|
128
|
+
Advanced ingestion: use this when your app already emits structured logs. These are compatibility-oriented utilities for turning structured logs into normalized `InspectEvent` and grouped trees. They remain conservative: **no eval**, **no parsing JS object literals**, JSON logs first-class, log4js best-effort.
|
|
94
129
|
|
|
95
130
|
- **`parseLogsToTrees`**
|
|
96
131
|
- **`JsonLogParser`**, **`Log4jsParser`**
|
|
@@ -152,7 +187,7 @@ Rationale: v1.x includes one official adapter and **zero production sinks**, so
|
|
|
152
187
|
|
|
153
188
|
## 11. Experimental `@agent-inspect/ai-sdk` APIs
|
|
154
189
|
|
|
155
|
-
`@agent-inspect/ai-sdk` is an optional
|
|
190
|
+
`@agent-inspect/ai-sdk` is an optional adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental and published as part of the aligned v1.8.0 package set.
|
|
156
191
|
|
|
157
192
|
Import from `@agent-inspect/ai-sdk`:
|
|
158
193
|
|
|
@@ -164,9 +199,10 @@ import { agentInspect } from "@agent-inspect/ai-sdk";
|
|
|
164
199
|
- **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
|
|
165
200
|
- **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
|
|
166
201
|
- **`runName`**: optional local run name.
|
|
167
|
-
- **`capture`**: `"metadata-only"` (default) or `"preview"`
|
|
168
|
-
- **`redactionProfile`** and **`maxPreviewChars`**:
|
|
169
|
-
- **`getDiagnostics()`**: exposes isolated adapter write failures without throwing into AI SDK callbacks.
|
|
202
|
+
- **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
|
|
203
|
+
- **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
|
|
204
|
+
- **`getDiagnostics()`**: exposes isolated adapter write, lifecycle/configuration, flush, and close failures without throwing into AI SDK callbacks.
|
|
205
|
+
- **`getWriterStats()`**, **`flush()`**, and **`close()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
|
|
170
206
|
|
|
171
207
|
Every AI SDK call using the adapter must keep telemetry local and metadata-only:
|
|
172
208
|
|
|
@@ -179,15 +215,89 @@ experimental_telemetry: {
|
|
|
179
215
|
}
|
|
180
216
|
```
|
|
181
217
|
|
|
182
|
-
The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata.
|
|
218
|
+
The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. It does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`. Unsupported preview capture options are explicit diagnostics and keep this metadata-only behavior.
|
|
183
219
|
|
|
184
220
|
No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-patch is part of this package.
|
|
185
221
|
|
|
186
222
|
Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
|
|
187
223
|
|
|
188
|
-
## 12. Experimental `@agent-inspect/
|
|
224
|
+
## 12. Experimental `@agent-inspect/vitest` APIs
|
|
225
|
+
|
|
226
|
+
`@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
|
|
227
|
+
|
|
228
|
+
Import from `@agent-inspect/vitest`:
|
|
229
|
+
|
|
230
|
+
```ts
|
|
231
|
+
import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
- **`createAgentInspectVitestReporter(options?)`**: returns a structural Vitest reporter facade with `onTestCaseResult`, `onTaskUpdate`, and `onFinished` hooks.
|
|
235
|
+
- **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/vitest-artifacts`.
|
|
236
|
+
- **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
|
|
237
|
+
- **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
|
|
238
|
+
- **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
|
|
239
|
+
- **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
|
|
240
|
+
- **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
|
|
241
|
+
- **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
|
|
242
|
+
- **`agentInspectVitestReporter`**: alias for `createAgentInspectVitestReporter`.
|
|
243
|
+
|
|
244
|
+
Trace association must be explicit. The default resolver reads `meta.agentInspect`, `meta["agent-inspect"]`, `meta.trace`, `context.meta.agentInspect`, or result metadata when present:
|
|
245
|
+
|
|
246
|
+
```ts
|
|
247
|
+
ctx.task.meta.agentInspect = {
|
|
248
|
+
runId: "support-agent",
|
|
249
|
+
tracePath: ".agent-inspect/support-agent.jsonl",
|
|
250
|
+
artifactLabel: "support-agent",
|
|
251
|
+
};
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
|
|
255
|
+
|
|
256
|
+
## 13. Experimental `@agent-inspect/jest` APIs
|
|
257
|
+
|
|
258
|
+
`@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
|
|
259
|
+
|
|
260
|
+
Import from `@agent-inspect/jest`:
|
|
261
|
+
|
|
262
|
+
```ts
|
|
263
|
+
import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent-inspect/jest";
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
- **`AgentInspectJestReporter`**: default Jest custom reporter class for `reporters: [["@agent-inspect/jest", options]]`.
|
|
267
|
+
- **`createAgentInspectJestReporter(options?)`**: returns a structural reporter facade with `onTestResult` and `onRunComplete` hooks for tests and custom harnesses.
|
|
268
|
+
- **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/jest-artifacts`.
|
|
269
|
+
- **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
|
|
270
|
+
- **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
|
|
271
|
+
- **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
|
|
272
|
+
- **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
|
|
273
|
+
- **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
|
|
274
|
+
- **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
|
|
275
|
+
- **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
|
|
276
|
+
- **`agentInspectJestReporter`**: alias for `createAgentInspectJestReporter`.
|
|
277
|
+
|
|
278
|
+
Jest association is explicit because Jest assertion results do not expose Vitest-style mutable task metadata:
|
|
279
|
+
|
|
280
|
+
```js
|
|
281
|
+
reporters: [
|
|
282
|
+
[
|
|
283
|
+
"@agent-inspect/jest",
|
|
284
|
+
{
|
|
285
|
+
associations: {
|
|
286
|
+
"agent.test.cjs::agent suite agent workflow": {
|
|
287
|
+
runId: "support-agent",
|
|
288
|
+
tracePath: ".agent-inspect/support-agent.jsonl",
|
|
289
|
+
},
|
|
290
|
+
},
|
|
291
|
+
},
|
|
292
|
+
],
|
|
293
|
+
],
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
|
|
297
|
+
|
|
298
|
+
## 14. Experimental `@agent-inspect/openai-agents` APIs
|
|
189
299
|
|
|
190
|
-
`@agent-inspect/openai-agents` is an optional
|
|
300
|
+
`@agent-inspect/openai-agents` is an optional experimental package for OpenAI Agents JS tracing processor integration. It is public in the aligned v1.8.0 package set and records runtime metadata locally.
|
|
191
301
|
|
|
192
302
|
Import from `@agent-inspect/openai-agents`:
|
|
193
303
|
|
|
@@ -195,11 +305,16 @@ Import from `@agent-inspect/openai-agents`:
|
|
|
195
305
|
import { agentInspectProcessor } from "@agent-inspect/openai-agents";
|
|
196
306
|
```
|
|
197
307
|
|
|
198
|
-
- **`agentInspectProcessor(options?)`**: returns a local-only
|
|
308
|
+
- **`agentInspectProcessor(options?)`**: returns a local-only OpenAI Agents `TracingProcessor`.
|
|
199
309
|
- **`installMode`**: always `"setTraceProcessors"` to document the safe replacement install path.
|
|
200
|
-
- **`localOnly`**: always `true`; the
|
|
201
|
-
- **`writer
|
|
202
|
-
- **`
|
|
310
|
+
- **`localOnly`**: always `true`; the processor performs no network I/O and does not install itself globally.
|
|
311
|
+
- **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
|
|
312
|
+
- **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
|
|
313
|
+
- **`workflowName`**: optional local run name overriding the SDK trace name.
|
|
314
|
+
- **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
|
|
315
|
+
- **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
|
|
316
|
+
- **`getDiagnostics()`**: exposes isolated processor write, lifecycle/configuration, flush, and shutdown failures without throwing into OpenAI Agents callbacks.
|
|
317
|
+
- **`getWriterStats()`**, **`forceFlush()`**, and **`shutdown()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
|
|
203
318
|
|
|
204
319
|
Safe future usage must replace processors explicitly:
|
|
205
320
|
|
|
@@ -209,11 +324,13 @@ setTraceProcessors([agentInspectProcessor({ traceDir: "./.agent-inspect" })]);
|
|
|
209
324
|
|
|
210
325
|
Do not use `addTraceProcessor()` as the default AgentInspect path; that leaves existing/default processors in place and can preserve backend export behavior in server runtimes.
|
|
211
326
|
|
|
212
|
-
|
|
327
|
+
The processor records local v0.2 persisted events for trace/run, agent, generation/response, function/tool, handoff, guardrail, MCP tools, custom, transcription, and speech span metadata where safely representable. It does not persist raw prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, or hosted tool payloads by default.
|
|
328
|
+
|
|
329
|
+
## 15. Experimental persisted-event foundation (v1.2.0)
|
|
213
330
|
|
|
214
331
|
These helpers expose the **source-agnostic `PersistedInspectEvent` model** (`schemaVersion: "0.2"`). They are **local-only**, **in-memory**, and **do not change** storage write/read or CLI behavior in v1.2.0.
|
|
215
332
|
|
|
216
|
-
Import from `agent-inspect`:
|
|
333
|
+
Import from `agent-inspect/persisted`:
|
|
217
334
|
|
|
218
335
|
| API | Role |
|
|
219
336
|
| --- | ---- |
|
|
@@ -235,7 +352,7 @@ Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEv
|
|
|
235
352
|
- v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
|
|
236
353
|
- Inspection read paths normalize v0.1 and v0.2 JSONL for local CLI/API use. v0.2 remains experimental as a persisted-event foundation and is not the default writer.
|
|
237
354
|
|
|
238
|
-
##
|
|
355
|
+
## 16. Local observability helpers (v1.4.0+)
|
|
239
356
|
|
|
240
357
|
Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1 manual traces remain the default writer; v0.2 persisted-event files are accepted where the shared dual-format read path is used. Local files only.
|
|
241
358
|
|
|
@@ -245,7 +362,7 @@ Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1
|
|
|
245
362
|
|
|
246
363
|
CLI wrappers: `agent-inspect timeline`, `stats`, `search` — see [CLI.md](./CLI.md).
|
|
247
364
|
|
|
248
|
-
##
|
|
365
|
+
## 17. Report and what helpers (v1.5.0+)
|
|
249
366
|
|
|
250
367
|
Read-only helpers for concise inspection summaries and local reports:
|
|
251
368
|
|
|
@@ -254,7 +371,7 @@ Read-only helpers for concise inspection summaries and local reports:
|
|
|
254
371
|
|
|
255
372
|
Report redaction profiles are key-based safeguards applied to the complete rendered report input, not only to the tree section. Review generated reports before sharing; this is not compliance-grade DLP.
|
|
256
373
|
|
|
257
|
-
##
|
|
374
|
+
## 18. Experimental trace writers (v1.6)
|
|
258
375
|
|
|
259
376
|
Trace writers are the first slice of the v1.6 runtime foundation. They are experimental during v1.x and intended for tests, adapters, and future `createInspector` work.
|
|
260
377
|
|
|
@@ -286,7 +403,7 @@ import type {
|
|
|
286
403
|
|
|
287
404
|
No network writer or vendor sink exists in this package.
|
|
288
405
|
|
|
289
|
-
##
|
|
406
|
+
## 19. Experimental inspector API/runtime (v1.6)
|
|
290
407
|
|
|
291
408
|
`createInspector()` is the experimental public instance API for local-first tracing with explicit writers. It owns an instance-specific runtime context, records v0.2 persisted inspect events, preserves application return values/errors, and exposes diagnostics plus deterministic `flush()`/`close()` lifecycle hooks.
|
|
292
409
|
|
|
@@ -328,7 +445,7 @@ Public methods:
|
|
|
328
445
|
|
|
329
446
|
These APIs are experimental during v1.x. They do not add a default network writer or vendor sink.
|
|
330
447
|
|
|
331
|
-
##
|
|
448
|
+
## 20. Experimental trace readers (v1.6)
|
|
332
449
|
|
|
333
450
|
`agent-inspect/readers` exposes the experimental local trace reader contract and detection pipeline. It includes AgentInspect JSONL for v0.1, v0.2, and mixed local trace files, plus local OpenInference JSON and OTLP JSON compatibility readers.
|
|
334
451
|
|
|
@@ -359,23 +476,94 @@ import type { TraceReader } from "agent-inspect/readers";
|
|
|
359
476
|
|
|
360
477
|
The reader contract does not silently accept arbitrary JSON and does not add OTel SDK, database, hosted ingestion, or network upload dependencies.
|
|
361
478
|
|
|
362
|
-
##
|
|
479
|
+
## 21. Experimental Checks
|
|
480
|
+
|
|
481
|
+
`agent-inspect/checks` exposes the experimental deterministic trace-check engine foundation. It consumes normalized reader output, runs supplied pure rules in stable order, and returns aggregate findings/diagnostics. It does not read files, discover config, call providers, perform network I/O, mutate inputs, or create a new persisted schema.
|
|
482
|
+
|
|
483
|
+
Import from `agent-inspect/checks`:
|
|
484
|
+
|
|
485
|
+
```ts
|
|
486
|
+
import { runTraceChecks } from "agent-inspect/checks";
|
|
487
|
+
import type { TraceCheckRule, TraceCheckResult } from "agent-inspect/checks";
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
- **`runTraceChecks({ read }, { rules?, select?, runId? })`**: executes provided rules against a `TraceReadResult` from `agent-inspect/readers`.
|
|
491
|
+
- **Built-in rule factories**: run, tool, LLM, structure, retrieval, guardrail, decision, safety, and baseline helpers including `createRunStatusRule`, `createToolUsageRule`, `createLlmUsageRule`, `createStructureOrphanRule`, `createStructureCycleRule`, `createStructureRelationshipRule`, `createRetrievalRule`, `createGuardrailRule`, `createDecisionRule`, `createSafetyRawContentRule`, `createSafetySecretPatternRule`, and `createBaselineRegressionRule`.
|
|
492
|
+
- **`TraceCheckRule`**: synchronous pure rule contract.
|
|
493
|
+
- **`TraceCheckResult`**: deterministic aggregate result with findings, evidence, summary counts, and execution diagnostics.
|
|
494
|
+
|
|
495
|
+
The checks API is experimental in v1.x. The `agent-inspect check` CLI uses this API for local reader-backed checks and deterministic JSON output; `agent-inspect artifacts` reuses the same safe findings for local CI artifact bundles and optional step-summary file output. Built-in rules operate on normalized event metadata, tree relationships, bounded summaries, token counts, and normalized baseline facts; safety and baseline findings identify event IDs and field paths rather than emitting raw prompts, outputs, secrets, headers, request/response bodies, or full tool payloads.
|
|
496
|
+
|
|
497
|
+
Recipes: [deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for check/baseline/artifact workflows, and [test-reporter-artifacts](../examples/recipes/test-reporter-artifacts/README.md) for Vitest/Jest reporter configuration patterns.
|
|
498
|
+
|
|
499
|
+
## 22. Experimental local explain APIs (v1.9)
|
|
500
|
+
|
|
501
|
+
`buildLocalExplanation()` creates a deterministic local explanation payload from a reader-selected `InspectRunTree`. It performs no network I/O, does not call model providers, and separates observed facts from deterministic inference labels.
|
|
502
|
+
|
|
503
|
+
Import from `agent-inspect`:
|
|
504
|
+
|
|
505
|
+
```ts
|
|
506
|
+
import { buildLocalExplanation } from "agent-inspect";
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
- **`buildLocalExplanation(run, options?)`**:
|
|
510
|
+
- **`mode: "dry-run"`**: returns redacted observed facts and no inference labels.
|
|
511
|
+
- **`mode: "local"`** (default): returns observed facts plus deterministic local inference labels.
|
|
512
|
+
- **`redactionProfile`**: `local`, `share`, or `strict`; profile keys are redacted before the payload is returned.
|
|
513
|
+
|
|
514
|
+
CLI wrapper: `agent-inspect explain <trace-path-or-run-id> --dry-run --json`.
|
|
515
|
+
|
|
516
|
+
Provider design gate:
|
|
517
|
+
|
|
518
|
+
- No provider payload is submitted in v1.9 implementation chunks; `--provider <provider>` is reserved and rejected with `PROVIDER_NOT_IMPLEMENTED`.
|
|
519
|
+
- The reviewable provider payload contract is the `ExplainResult` object: `mode`, `runId`, optional `name` / `status`, `redactionProfile`, `facts`, `inferences`, and `notes`.
|
|
520
|
+
- Provider implementations must require explicit provider selection and documented environment requirements. The current local API reads no provider credentials.
|
|
521
|
+
- Provider prompts must use redacted facts only, label inferred claims, and must not request raw chain-of-thought.
|
|
522
|
+
- Provider packages or SDKs must not become root/core runtime dependencies.
|
|
523
|
+
|
|
524
|
+
## 23. Experimental `@agent-inspect/harness` APIs
|
|
525
|
+
|
|
526
|
+
`@agent-inspect/harness` is a private experimental workspace package during the v1.9 release train. It provides a no-framework fixture runner for local targets and recipes; first public package publication remains a manual maintainer gate.
|
|
527
|
+
|
|
528
|
+
Import from `@agent-inspect/harness` inside the workspace:
|
|
529
|
+
|
|
530
|
+
```ts
|
|
531
|
+
import { createFixtureRunner, defineTarget } from "@agent-inspect/harness";
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
- **`defineTarget(definition)`**: returns a typed target definition with `resolve(app, context)` and `invoke(target, input, context)` hooks.
|
|
535
|
+
- **`createFixtureRunner(options)`**: returns a local runner with:
|
|
536
|
+
- **`listTargets()`**: deterministic target metadata listing.
|
|
537
|
+
- **`runTarget(name, input, options?)`**: bootstrap, resolve, invoke, and shutdown lifecycle.
|
|
538
|
+
- **`runFromArgv(argv?, io?)`**: CLI-friendly execution with target listing, JSON fixture files, JSON stdin, JSON stdout, stderr summaries, trace flags, and expected-output comparison.
|
|
539
|
+
- **`getDiagnostics()`**: deterministic diagnostics for missing targets, bootstrap failures, resolve failures, invocation failures, and shutdown failures.
|
|
540
|
+
- **`trace`** options use existing AgentInspect local APIs only:
|
|
541
|
+
- **`mode: "run-if-enabled"`** (default): uses `maybeInspectRun()` and writes no trace unless `options.enabled` or `AGENT_INSPECT` enables tracing.
|
|
542
|
+
- **`mode: "run"`**: explicitly wraps the target invocation in `inspectRun()`.
|
|
543
|
+
- **`mode: "observe"`**: proxies the resolved target with `observe()` for `run` / `execute` / `invoke` methods when enabled.
|
|
544
|
+
- **`mode: "off"`**: invokes the target without AgentInspect tracing.
|
|
545
|
+
|
|
546
|
+
The harness package does not add root/core dependencies, does not upload traces, does not call providers, and does not capture raw prompts or outputs by itself. It writes only local AgentInspect traces when explicitly enabled by runner options or environment-gated tracing.
|
|
547
|
+
|
|
548
|
+
Recipes: [harness-basic](../examples/recipes/harness-basic/README.md) and [harness-adapter-local](../examples/recipes/harness-adapter-local/README.md).
|
|
549
|
+
|
|
550
|
+
## 24. Deprecated APIs
|
|
363
551
|
|
|
364
552
|
No deprecated APIs are declared as of 1.4.0.
|
|
365
553
|
|
|
366
|
-
##
|
|
554
|
+
## 25. Removal / deprecation policy
|
|
367
555
|
|
|
368
556
|
- Stable APIs are not removed in v1.x.
|
|
369
557
|
- If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
|
|
370
558
|
|
|
371
|
-
##
|
|
559
|
+
## 26. Backward compatibility policy
|
|
372
560
|
|
|
373
561
|
- Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
|
|
374
562
|
- Additive schema changes are allowed in minor versions.
|
|
375
563
|
- Breaking changes require a major version.
|
|
376
564
|
- Unknown fields should be ignored where safe.
|
|
377
565
|
|
|
378
|
-
##
|
|
566
|
+
## 25. Examples
|
|
379
567
|
|
|
380
568
|
### Minimal manual trace
|
|
381
569
|
|
package/docs/CLI.md
CHANGED
|
@@ -26,12 +26,17 @@ Core commands:
|
|
|
26
26
|
- `tail` — live-tail logs into updating local trees
|
|
27
27
|
- `export` — export manual traces to Markdown/HTML/OpenInference/OTLP JSON (local only)
|
|
28
28
|
- `open` — read supported local trace files, directories, or stdin through the canonical reader pipeline
|
|
29
|
+
- `check` — run deterministic local trace checks with stable JSON and exit codes
|
|
30
|
+
- `scan` — best-effort local safety scan for trace capture risks
|
|
31
|
+
- `verify-safe` — best-effort local trace safety verification
|
|
32
|
+
- `artifacts` — create safe local CI trace artifact bundles and optional step summaries
|
|
29
33
|
- `diff` — compare two manual traces (local, read-only)
|
|
30
34
|
- `timeline` — chronological view of one run (local JSONL)
|
|
31
35
|
- `stats` — local aggregate stats over a trace directory
|
|
32
36
|
- `search` — deterministic local search over traces
|
|
33
37
|
- `what` — concise summary of a single run (local JSONL)
|
|
34
38
|
- `report` — markdown or HTML inspection report for a single run
|
|
39
|
+
- `explain` — deterministic local facts/inferences for a trace, with dry-run payloads
|
|
35
40
|
|
|
36
41
|
## 2. Environment variables
|
|
37
42
|
|
|
@@ -44,6 +49,20 @@ Core commands:
|
|
|
44
49
|
- **0**: command succeeded (even if a diff reports “differences”)
|
|
45
50
|
- **1**: command error (invalid args, missing files, missing runs, parse failures, validation failures, etc.)
|
|
46
51
|
|
|
52
|
+
Exception: `check` uses CI-oriented semantic exit codes:
|
|
53
|
+
|
|
54
|
+
- **0**: all selected checks passed
|
|
55
|
+
- **1**: checks ran and at least one error-severity rule failed
|
|
56
|
+
- **2**: invalid arguments or invalid config
|
|
57
|
+
- **3**: trace input could not be read
|
|
58
|
+
- **4**: unsupported or ambiguous trace format
|
|
59
|
+
|
|
60
|
+
Exception: `scan` and `verify-safe` use local safety status exit codes:
|
|
61
|
+
|
|
62
|
+
- **0**: status is SAFE or SAFE WITH WARNINGS
|
|
63
|
+
- **1**: status is UNSAFE
|
|
64
|
+
- **2**: status is UNKNOWN, including unreadable, unsupported, ambiguous, or invalid inputs
|
|
65
|
+
|
|
47
66
|
AgentInspect favors **human-readable errors without stack traces** for expected user mistakes.
|
|
48
67
|
|
|
49
68
|
## 4. JSON output policy
|
|
@@ -59,6 +78,8 @@ Many commands support `--json` for scripting. JSON output is intended to be:
|
|
|
59
78
|
- Log-derived output includes **confidence** labels and avoids inventing parent-child relationships.
|
|
60
79
|
- Redaction defaults are conservative (e.g. `authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`).
|
|
61
80
|
- Exported payloads are **redacted by default** unless explicitly configured otherwise.
|
|
81
|
+
- `scan` and `verify-safe` are best-effort local checks, not compliance, privacy, security, or regulatory certifications.
|
|
82
|
+
- `artifacts` renders structural summaries and check evidence only; it does not include raw prompt/output bodies, request/response bodies, headers, API keys, secrets, or full tool payloads.
|
|
62
83
|
|
|
63
84
|
## 6. Command reference
|
|
64
85
|
|
|
@@ -118,7 +139,7 @@ Recommendation: run with `--dry-run` first.
|
|
|
118
139
|
|
|
119
140
|
### 6.4 `logs`
|
|
120
141
|
|
|
121
|
-
|
|
142
|
+
Advanced ingestion: use this when your app already emits structured logs. Parse those logs into local execution trees.
|
|
122
143
|
|
|
123
144
|
```bash
|
|
124
145
|
agent-inspect logs <file> [options]
|
|
@@ -222,7 +243,135 @@ cat packages/core/test/fixtures/openinference-basic.json | npx agent-inspect ope
|
|
|
222
243
|
|
|
223
244
|
When a directory or payload contains multiple runs, `open` lists the run ids and exits until you pass `--run <run-id>`.
|
|
224
245
|
|
|
225
|
-
### 6.8 `
|
|
246
|
+
### 6.8 `check`
|
|
247
|
+
|
|
248
|
+
Run deterministic checks against a local trace. This command is local and read-only: it does not rerun agents, call models, upload traces, or mutate input files.
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
agent-inspect check <trace-path-or-run-id> [options]
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
`<trace-path-or-run-id>` may be a trace file, directory, `-` for stdin, or a run id resolved with `--dir`.
|
|
255
|
+
|
|
256
|
+
Options:
|
|
257
|
+
|
|
258
|
+
- `--dir <path>`: trace directory for run-id lookup
|
|
259
|
+
- `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
|
|
260
|
+
- `--run <run-id>`: select a run when input contains multiple runs
|
|
261
|
+
- `--config <path>`: check config (`.json`, `.js`, `.mjs`, or `.cjs`)
|
|
262
|
+
- `--json`: print deterministic `TraceCheckResult` JSON
|
|
263
|
+
- `--rule <id>`: select a rule id; repeatable
|
|
264
|
+
- `--max-duration-ms <number>`: add `run.duration`
|
|
265
|
+
- `--required-tool <name>` / `--forbidden-tool <name>`: add `tool.usage`
|
|
266
|
+
- `--allowed-model <model>` / `--max-total-tokens <number>`: add `llm.usage`
|
|
267
|
+
|
|
268
|
+
By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available.
|
|
269
|
+
|
|
270
|
+
Config files use this shape:
|
|
271
|
+
|
|
272
|
+
```json
|
|
273
|
+
{
|
|
274
|
+
"checks": {
|
|
275
|
+
"select": ["run.status", "run.duration"],
|
|
276
|
+
"run": { "maxDurationMs": 30000 },
|
|
277
|
+
"tool": { "required": ["search_docs"] },
|
|
278
|
+
"llm": { "allowedModels": ["gpt-4.1-mini"], "maxTotalTokens": 12000 }
|
|
279
|
+
}
|
|
280
|
+
}
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
YAML is not supported. TypeScript config files (`.ts`, `.mts`, `.cts`) fail clearly unless a future explicit loader strategy is added; use precompiled JavaScript config instead.
|
|
284
|
+
|
|
285
|
+
Examples:
|
|
286
|
+
|
|
287
|
+
```bash
|
|
288
|
+
npx agent-inspect check fixtures/traces-v0.2/manual-basic.jsonl --json
|
|
289
|
+
npx agent-inspect check minimal-success --dir fixtures/traces --rule run.status
|
|
290
|
+
npx agent-inspect check trace.jsonl --max-duration-ms 30000 --required-tool search_docs --json
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
Recipe: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
|
|
294
|
+
|
|
295
|
+
### 6.9 `scan` and `verify-safe`
|
|
296
|
+
|
|
297
|
+
Run best-effort local safety verification for supported trace inputs. These commands are local and read-only: they do not rerun agents, call models, upload traces, mutate input files, or certify compliance.
|
|
298
|
+
|
|
299
|
+
```bash
|
|
300
|
+
agent-inspect scan <trace-path-or-run-id> [options]
|
|
301
|
+
agent-inspect verify-safe <trace-path-or-run-id> [options]
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
`<trace-path-or-run-id>` may be a trace file, directory, `-` for stdin, or a run id resolved with `--dir`.
|
|
305
|
+
|
|
306
|
+
Statuses:
|
|
307
|
+
|
|
308
|
+
- `SAFE`: no safety findings and no reader warnings.
|
|
309
|
+
- `SAFE WITH WARNINGS`: no safety findings, but the reader reported warnings or unsupported fields.
|
|
310
|
+
- `UNSAFE`: safety findings were detected.
|
|
311
|
+
- `UNKNOWN`: the input could not be read, normalized, or selected conservatively.
|
|
312
|
+
|
|
313
|
+
Options:
|
|
314
|
+
|
|
315
|
+
- `--dir <path>`: trace directory for run-id lookup
|
|
316
|
+
- `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
|
|
317
|
+
- `--run <run-id>`: select a run when input contains multiple runs
|
|
318
|
+
- `--json`: print deterministic JSON safety result
|
|
319
|
+
- `--max-string-length <number>`: unsafe threshold for string values
|
|
320
|
+
- `--max-array-length <number>`: unsafe threshold for array values
|
|
321
|
+
- `--max-object-keys <number>`: unsafe threshold for object key counts
|
|
322
|
+
- `--max-serialized-bytes <number>`: unsafe threshold for serialized values
|
|
323
|
+
|
|
324
|
+
The scan looks for raw prompt/output-like capture paths, unredacted sensitive-looking keys, secret-like string patterns, and oversized values. It reports evidence paths rather than raw prompt, output, request/response, header, API key, secret, or full tool payload values. Secret detection is best-effort and should not be treated as exhaustive.
|
|
325
|
+
|
|
326
|
+
Examples:
|
|
327
|
+
|
|
328
|
+
```bash
|
|
329
|
+
npx agent-inspect scan fixtures/traces-v0.2/manual-basic.jsonl --json
|
|
330
|
+
npx agent-inspect verify-safe minimal-success --dir fixtures/traces
|
|
331
|
+
npx agent-inspect verify-safe trace.jsonl --max-string-length 8192 --json
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
### 6.10 `artifacts`
|
|
335
|
+
|
|
336
|
+
Create deterministic local CI artifacts for supported trace inputs. This command is local and read-only for trace inputs: it does not rerun agents, call models, upload files, use GitHub APIs, or mutate repository state. It writes only to `--output-dir` and, when requested, a local step-summary file.
|
|
337
|
+
|
|
338
|
+
```bash
|
|
339
|
+
agent-inspect artifacts <trace-path-or-run-id> --output-dir <path> [options]
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
Generated files:
|
|
343
|
+
|
|
344
|
+
- `trace.json`: structural trace summary only
|
|
345
|
+
- `check.json`: safety check result
|
|
346
|
+
- `diff.json`: baseline diff result, or `not_requested`
|
|
347
|
+
- `summary.md`: safe Markdown CI summary
|
|
348
|
+
- `report.html`: safe HTML CI summary
|
|
349
|
+
- `manifest.json`: deterministic file/status manifest
|
|
350
|
+
|
|
351
|
+
Options:
|
|
352
|
+
|
|
353
|
+
- `--output-dir <path>`: required local artifact directory
|
|
354
|
+
- `--dir <path>`: trace directory for run-id lookup
|
|
355
|
+
- `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
|
|
356
|
+
- `--run <run-id>`: select a run when input contains multiple runs
|
|
357
|
+
- `--baseline <trace-path-or-run-id>`: optional baseline trace for diff artifacts
|
|
358
|
+
- `--baseline-run <run-id>`: select a run from the baseline trace
|
|
359
|
+
- `--github-summary <path>`: append the safe Markdown summary to this file, such as `$GITHUB_STEP_SUMMARY`
|
|
360
|
+
- `--json`: print deterministic `manifest.json` content
|
|
361
|
+
|
|
362
|
+
The artifact command runs safety checks before rendering and only includes structural counts, statuses, bounded check findings, diagnostics, and evidence paths. Baseline diff artifacts use normalized baseline checks and also avoid raw prompt/output/tool payload values. `--github-summary` is plain local file output; AgentInspect does not call GitHub APIs or upload artifacts.
|
|
363
|
+
|
|
364
|
+
Examples:
|
|
365
|
+
|
|
366
|
+
```bash
|
|
367
|
+
npx agent-inspect artifacts fixtures/traces-v0.2/manual-basic.jsonl --output-dir ./artifacts --json
|
|
368
|
+
npx agent-inspect artifacts minimal-success --dir fixtures/traces --output-dir ./artifacts --github-summary "$GITHUB_STEP_SUMMARY"
|
|
369
|
+
npx agent-inspect artifacts candidate.jsonl --baseline baseline.jsonl --output-dir ./artifacts
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
Recipe and sample workflow: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
|
|
373
|
+
|
|
374
|
+
### 6.11 `diff`
|
|
226
375
|
|
|
227
376
|
Compare two manual trace runs. Diff is **local** and **read-only** (does not rerun agents).
|
|
228
377
|
|
|
@@ -286,7 +435,7 @@ Differences:
|
|
|
286
435
|
|
|
287
436
|
More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
|
|
288
437
|
|
|
289
|
-
### 6.
|
|
438
|
+
### 6.12 `timeline`
|
|
290
439
|
|
|
291
440
|
Chronological step list for one manual trace. Read-only; does not mutate JSONL files.
|
|
292
441
|
|
|
@@ -302,7 +451,7 @@ Options:
|
|
|
302
451
|
|
|
303
452
|

|
|
304
453
|
|
|
305
|
-
### 6.
|
|
454
|
+
### 6.13 `stats`
|
|
306
455
|
|
|
307
456
|
Local aggregate statistics over trace files in a directory. Read-only.
|
|
308
457
|
|
|
@@ -322,7 +471,7 @@ Options:
|
|
|
322
471
|
|
|
323
472
|
Use `--correlation-id` or `--group-id` to filter runs by `run_started` metadata (see [API.md](./API.md)).
|
|
324
473
|
|
|
325
|
-
### 6.
|
|
474
|
+
### 6.14 `search`
|
|
326
475
|
|
|
327
476
|
Deterministic search over local traces (substring / exact filters). No semantic search.
|
|
328
477
|
|
|
@@ -352,7 +501,7 @@ npx agent-inspect search --duration ">100ms" --json
|
|
|
352
501
|
|
|
353
502
|

|
|
354
503
|
|
|
355
|
-
### 6.
|
|
504
|
+
### 6.15 `what`
|
|
356
505
|
|
|
357
506
|
Concise human-readable summary of one local trace run. Read-only; accepts v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path. Vocabulary: [TRACE-VOCABULARY-V1.5.md](./proposals/TRACE-VOCABULARY-V1.5.md).
|
|
358
507
|
|
|
@@ -381,7 +530,7 @@ Outcome: Completed successfully.
|
|
|
381
530
|
Slowest: plan (100ms, logic)
|
|
382
531
|
```
|
|
383
532
|
|
|
384
|
-
### 6.
|
|
533
|
+
### 6.16 `report`
|
|
385
534
|
|
|
386
535
|
Generate a local inspection report combining **what happened**, **timeline**, and **execution tree** sections. The command reads local v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path without mutating them. Distinct from `export` (which targets shareable tree snapshots and standards formats).
|
|
387
536
|
|
|
@@ -406,6 +555,39 @@ Example:
|
|
|
406
555
|
npx agent-inspect report minimal-success --dir fixtures/traces --format html -o report.html
|
|
407
556
|
```
|
|
408
557
|
|
|
558
|
+
### 6.17 `explain`
|
|
559
|
+
|
|
560
|
+
Explain a local trace using deterministic facts and local inference labels. This command reads through the same local reader pipeline as `open` / `check`; it does not call a model provider, upload traces, replay agents, or mutate input files.
|
|
561
|
+
|
|
562
|
+
```bash
|
|
563
|
+
agent-inspect explain <trace-path-or-run-id> [options]
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
Options:
|
|
567
|
+
|
|
568
|
+
- `--dir <path>` — trace directory for run-id lookup
|
|
569
|
+
- `--format <agent-inspect-jsonl|openinference-json|otlp-json>` — explicit input format
|
|
570
|
+
- `--run <run-id>` — select a run when the trace contains multiple runs
|
|
571
|
+
- `--dry-run` — emit only the redacted facts payload, with no local inference labels
|
|
572
|
+
- `--provider <provider>` — reserved for an explicit future provider mode; currently rejected without network calls
|
|
573
|
+
- `--json` — print deterministic JSON output
|
|
574
|
+
- `--redaction-profile <local|share|strict>` — key-based redaction profile for the explanation payload (default `local`)
|
|
575
|
+
|
|
576
|
+
Examples:
|
|
577
|
+
|
|
578
|
+
```bash
|
|
579
|
+
npx agent-inspect explain minimal-success --dir fixtures/traces
|
|
580
|
+
npx agent-inspect explain fixtures/traces/minimal-success.jsonl --dry-run --json --redaction-profile strict
|
|
581
|
+
```
|
|
582
|
+
|
|
583
|
+
Provider design gate:
|
|
584
|
+
|
|
585
|
+
- Current behavior is local only. `--provider <provider>` exits with a user-facing `PROVIDER_NOT_IMPLEMENTED` error and performs no provider call.
|
|
586
|
+
- `--dry-run --json` is the payload review surface. The provider payload contract is the returned `explanation` object: `mode`, `runId`, optional `name` / `status`, `redactionProfile`, `facts`, `inferences`, and `notes`.
|
|
587
|
+
- Provider chunks must require explicit provider selection, document required environment variables, and keep credentials out of trace data and dry-run output.
|
|
588
|
+
- Provider prompts must ask for concise explanations from redacted facts only. They must not request, expose, or preserve raw chain-of-thought.
|
|
589
|
+
- Cloud provider behavior is never selected by default and must be reviewed before implementation. Local provider support must still be explicit and opt-in.
|
|
590
|
+
|
|
409
591
|
## 7. Optional TUI behavior
|
|
410
592
|
|
|
411
593
|
`view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.
|