agent-inspect 1.7.0 → 1.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/CHANGELOG.md +21 -1
  2. package/README.md +145 -21
  3. package/docs/ADAPTER-CONFORMANCE.md +7 -3
  4. package/docs/ADAPTERS.md +155 -5
  5. package/docs/API.md +214 -26
  6. package/docs/CLI.md +189 -7
  7. package/docs/GETTING-STARTED.md +71 -16
  8. package/docs/KNOWN-ISSUES.md +7 -1
  9. package/docs/LIMITATIONS.md +7 -1
  10. package/docs/LOG-TO-TREE-QUICKSTART.md +1 -2
  11. package/docs/MIGRATION.md +67 -0
  12. package/docs/SCHEMA.md +1 -0
  13. package/package.json +12 -2
  14. package/packages/cli/dist/index.cjs +2454 -140
  15. package/packages/cli/dist/index.cjs.map +1 -1
  16. package/packages/cli/dist/index.mjs +2454 -140
  17. package/packages/cli/dist/index.mjs.map +1 -1
  18. package/packages/core/dist/advanced.d.cts +4 -4
  19. package/packages/core/dist/advanced.d.ts +4 -4
  20. package/packages/core/dist/checks.cjs +1535 -0
  21. package/packages/core/dist/checks.cjs.map +1 -0
  22. package/packages/core/dist/checks.d.cts +585 -0
  23. package/packages/core/dist/checks.d.ts +585 -0
  24. package/packages/core/dist/checks.mjs +1512 -0
  25. package/packages/core/dist/checks.mjs.map +1 -0
  26. package/packages/core/dist/diff.d.cts +3 -3
  27. package/packages/core/dist/diff.d.ts +3 -3
  28. package/packages/core/dist/exporters.d.cts +3 -3
  29. package/packages/core/dist/exporters.d.ts +3 -3
  30. package/packages/core/dist/index.cjs +146 -0
  31. package/packages/core/dist/index.cjs.map +1 -1
  32. package/packages/core/dist/index.d.cts +44 -7
  33. package/packages/core/dist/index.d.ts +44 -7
  34. package/packages/core/dist/index.mjs +148 -1
  35. package/packages/core/dist/index.mjs.map +1 -1
  36. package/packages/core/dist/{inspect-event-Des4JDHo.d.cts → inspect-event-CevRYp58.d.cts} +1 -1
  37. package/packages/core/dist/{inspect-event-Des4JDHo.d.ts → inspect-event-CevRYp58.d.ts} +1 -1
  38. package/packages/core/dist/{log-config-C1GcJPIM.d.ts → log-config-BPHS4Sds.d.ts} +1 -1
  39. package/packages/core/dist/{log-config-BnH8Ykcb.d.cts → log-config-DanPV3P9.d.cts} +1 -1
  40. package/packages/core/dist/logs.d.cts +3 -3
  41. package/packages/core/dist/logs.d.ts +3 -3
  42. package/packages/core/dist/{persisted-inspect-event-DiFto0K2.d.ts → persisted-inspect-event-Cw7TeYGr.d.ts} +1 -1
  43. package/packages/core/dist/{persisted-inspect-event-0kaRADsp.d.cts → persisted-inspect-event-DHPfzUd8.d.cts} +1 -1
  44. package/packages/core/dist/persisted.d.cts +5 -5
  45. package/packages/core/dist/persisted.d.ts +5 -5
  46. package/packages/core/dist/readers.d.cts +2 -2
  47. package/packages/core/dist/readers.d.ts +2 -2
  48. package/packages/core/dist/{types-tSix7tfv.d.ts → types-Ap9uMdx_.d.ts} +1 -1
  49. package/packages/core/dist/{types-DB8jB6Jg.d.cts → types-B2-BU5CS.d.cts} +1 -1
  50. package/packages/core/dist/writers.d.cts +2 -2
  51. package/packages/core/dist/writers.d.ts +2 -2
package/docs/API.md CHANGED
@@ -9,14 +9,42 @@ AgentInspect is a **local-first execution-tree debugger**. It is not a SaaS, not
9
9
  - **Stable**: intended to be compatible across v1.x. Breaking changes require v2.0.
10
10
  - **Experimental**: available for adoption, but subject to refinement (including naming/shape changes) before a future stability declaration. Experimental APIs may change in v1.x.
11
11
 
12
- **1.x subpath exports:** Additive subpaths (`/logs`, `/exporters`, `/persisted`, `/diff`, `/advanced`, `/writers`, `/readers`) narrow the import surface for experimental and advanced APIs. Root `"."` imports remain valid through v1.x. Design: [API-BOUNDARY-V1.5.md](./implementation/API-BOUNDARY-V1.5.md).
12
+ Use the root import for stable beginner APIs. Use subpaths for advanced, experimental, or lower-level workflows.
13
+
14
+ ```ts
15
+ import {
16
+ observe,
17
+ inspectRun,
18
+ maybeInspectRun,
19
+ step,
20
+ getCurrentCorrelationMetadata,
21
+ } from "agent-inspect";
22
+ ```
23
+
24
+ **v1.9 root API direction:** do not add new root exports casually. Existing root imports keep working through v1.x for compatibility, but new advanced examples should use the subpath where the API lives. The intended stable root set for v2 is:
25
+
26
+ ```ts
27
+ import {
28
+ createInspector,
29
+ observe,
30
+ inspectRun,
31
+ maybeInspectRun,
32
+ step,
33
+ getCurrentCorrelationMetadata,
34
+ } from "agent-inspect";
35
+ ```
36
+
37
+ **1.x subpath exports:** Additive subpaths (`/logs`, `/exporters`, `/persisted`, `/diff`, `/advanced`, `/writers`, `/readers`, `/checks`) narrow the import surface for experimental and advanced APIs. Root `"."` imports remain valid through v1.x. Design: [API-BOUNDARY-V1.5.md](./implementation/API-BOUNDARY-V1.5.md).
13
38
 
14
39
  ```ts
15
- import { inspectRun, step } from "agent-inspect";
16
40
  import { parseLogsToTrees } from "agent-inspect/logs";
17
41
  import { exportMarkdown } from "agent-inspect/exporters";
18
42
  import { memoryWriter } from "agent-inspect/writers";
19
43
  import { openTrace } from "agent-inspect/readers";
44
+ import { runTraceChecks } from "agent-inspect/checks";
45
+ import { diffTraceEvents } from "agent-inspect/diff";
46
+ import { traceEventsToPersistedInspectEvents } from "agent-inspect/persisted";
47
+ import { createInspector } from "agent-inspect/advanced";
20
48
  ```
21
49
 
22
50
  Notes:
@@ -24,6 +52,7 @@ Notes:
24
52
  - The core guarantee of v1.x is **stable local debugging**: manual tracing + CLI inspection.
25
53
  - Export formats (OpenInference / OTLP JSON) are **local-only** and **compatibility-oriented**. They do **not** upload anywhere.
26
54
  - There are **zero production sinks** in v1.x; sink/uploader APIs are not stable.
55
+ - Advanced root exports in v1.x are compatibility aliases. Prefer `agent-inspect/advanced`, `agent-inspect/readers`, `agent-inspect/writers`, `agent-inspect/checks`, `agent-inspect/diff`, `agent-inspect/exporters`, `agent-inspect/logs`, and `agent-inspect/persisted` for new code.
27
56
 
28
57
  ## 2. Stable core APIs (manual tracing)
29
58
 
@@ -32,7 +61,13 @@ These are the recommended entry points for manual instrumentation. They are desi
32
61
  Import from `agent-inspect`:
33
62
 
34
63
  ```ts
35
- import { inspectRun, maybeInspectRun, step, observe } from "agent-inspect";
64
+ import {
65
+ observe,
66
+ inspectRun,
67
+ maybeInspectRun,
68
+ step,
69
+ getCurrentCorrelationMetadata,
70
+ } from "agent-inspect";
36
71
  ```
37
72
 
38
73
  - **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output.
@@ -90,7 +125,7 @@ These APIs support local workflows like listing traces, extracting metadata/summ
90
125
 
91
126
  ## 5. Experimental log parsing APIs
92
127
 
93
- These are compatibility-oriented utilities for turning structured logs into normalized `InspectEvent` and grouped trees. They remain conservative: **no eval**, **no parsing JS object literals**, JSON logs first-class, log4js best-effort.
128
+ Advanced ingestion: use this when your app already emits structured logs. These are compatibility-oriented utilities for turning structured logs into normalized `InspectEvent` and grouped trees. They remain conservative: **no eval**, **no parsing JS object literals**, JSON logs first-class, log4js best-effort.
94
129
 
95
130
  - **`parseLogsToTrees`**
96
131
  - **`JsonLogParser`**, **`Log4jsParser`**
@@ -152,7 +187,7 @@ Rationale: v1.x includes one official adapter and **zero production sinks**, so
152
187
 
153
188
  ## 11. Experimental `@agent-inspect/ai-sdk` APIs
154
189
 
155
- `@agent-inspect/ai-sdk` is an optional v1.7 adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental during the v1.7 train and remains private until release readiness.
190
+ `@agent-inspect/ai-sdk` is an optional adapter package for Vercel AI SDK v6 telemetry integrations. It is experimental and published as part of the aligned v1.8.0 package set.
156
191
 
157
192
  Import from `@agent-inspect/ai-sdk`:
158
193
 
@@ -164,9 +199,10 @@ import { agentInspect } from "@agent-inspect/ai-sdk";
164
199
  - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
165
200
  - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
166
201
  - **`runName`**: optional local run name.
167
- - **`capture`**: `"metadata-only"` (default) or `"preview"` (reserved for explicit bounded capture).
168
- - **`redactionProfile`** and **`maxPreviewChars`**: reserved for bounded/redacted future preview behavior.
169
- - **`getDiagnostics()`**: exposes isolated adapter write failures without throwing into AI SDK callbacks.
202
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
203
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
204
+ - **`getDiagnostics()`**: exposes isolated adapter write, lifecycle/configuration, flush, and close failures without throwing into AI SDK callbacks.
205
+ - **`getWriterStats()`**, **`flush()`**, and **`close()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
170
206
 
171
207
  Every AI SDK call using the adapter must keep telemetry local and metadata-only:
172
208
 
@@ -179,15 +215,89 @@ experimental_telemetry: {
179
215
  }
180
216
  ```
181
217
 
182
- The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. By default it does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`.
218
+ The adapter records local v0.2 persisted events for run, LLM step, and tool lifecycle metadata. It does not persist raw prompts, messages, generated text, stream chunks, tool inputs, tool outputs, headers, request bodies, response bodies, or user `experimental_context`. Unsupported preview capture options are explicit diagnostics and keep this metadata-only behavior.
183
219
 
184
220
  No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-patch is part of this package.
185
221
 
186
222
  Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
187
223
 
188
- ## 12. Experimental `@agent-inspect/openai-agents` APIs
224
+ ## 12. Experimental `@agent-inspect/vitest` APIs
225
+
226
+ `@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
227
+
228
+ Import from `@agent-inspect/vitest`:
229
+
230
+ ```ts
231
+ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
232
+ ```
233
+
234
+ - **`createAgentInspectVitestReporter(options?)`**: returns a structural Vitest reporter facade with `onTestCaseResult`, `onTaskUpdate`, and `onFinished` hooks.
235
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/vitest-artifacts`.
236
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
237
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
238
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
239
+ - **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
240
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
241
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
242
+ - **`agentInspectVitestReporter`**: alias for `createAgentInspectVitestReporter`.
243
+
244
+ Trace association must be explicit. The default resolver reads `meta.agentInspect`, `meta["agent-inspect"]`, `meta.trace`, `context.meta.agentInspect`, or result metadata when present:
245
+
246
+ ```ts
247
+ ctx.task.meta.agentInspect = {
248
+ runId: "support-agent",
249
+ tracePath: ".agent-inspect/support-agent.jsonl",
250
+ artifactLabel: "support-agent",
251
+ };
252
+ ```
253
+
254
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
255
+
256
+ ## 13. Experimental `@agent-inspect/jest` APIs
257
+
258
+ `@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
259
+
260
+ Import from `@agent-inspect/jest`:
261
+
262
+ ```ts
263
+ import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent-inspect/jest";
264
+ ```
265
+
266
+ - **`AgentInspectJestReporter`**: default Jest custom reporter class for `reporters: [["@agent-inspect/jest", options]]`.
267
+ - **`createAgentInspectJestReporter(options?)`**: returns a structural reporter facade with `onTestResult` and `onRunComplete` hooks for tests and custom harnesses.
268
+ - **`artifactDir`**: local output directory for safe artifacts; defaults to `.agent-inspect/jest-artifacts`.
269
+ - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
270
+ - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
271
+ - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
272
+ - **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
273
+ - **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
274
+ - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
275
+ - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
276
+ - **`agentInspectJestReporter`**: alias for `createAgentInspectJestReporter`.
277
+
278
+ Jest association is explicit because Jest assertion results do not expose Vitest-style mutable task metadata:
279
+
280
+ ```js
281
+ reporters: [
282
+ [
283
+ "@agent-inspect/jest",
284
+ {
285
+ associations: {
286
+ "agent.test.cjs::agent suite agent workflow": {
287
+ runId: "support-agent",
288
+ tracePath: ".agent-inspect/support-agent.jsonl",
289
+ },
290
+ },
291
+ },
292
+ ],
293
+ ],
294
+ ```
295
+
296
+ Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
297
+
298
+ ## 14. Experimental `@agent-inspect/openai-agents` APIs
189
299
 
190
- `@agent-inspect/openai-agents` is an optional v1.7 scaffold for future OpenAI Agents JS tracing processor integration. Runtime span mapping is not implemented yet.
300
+ `@agent-inspect/openai-agents` is an optional experimental package for OpenAI Agents JS tracing processor integration. It is public in the aligned v1.8.0 package set and records runtime metadata locally.
191
301
 
192
302
  Import from `@agent-inspect/openai-agents`:
193
303
 
@@ -195,11 +305,16 @@ Import from `@agent-inspect/openai-agents`:
195
305
  import { agentInspectProcessor } from "@agent-inspect/openai-agents";
196
306
  ```
197
307
 
198
- - **`agentInspectProcessor(options?)`**: returns a local-only scaffold object with diagnostics.
308
+ - **`agentInspectProcessor(options?)`**: returns a local-only OpenAI Agents `TracingProcessor`.
199
309
  - **`installMode`**: always `"setTraceProcessors"` to document the safe replacement install path.
200
- - **`localOnly`**: always `true`; the scaffold performs no network I/O and does not install itself globally.
201
- - **`writer`**, **`traceDir`**, **`workflowName`**, **`capture`**, **`redactionProfile`**, and **`maxPreviewChars`** are reserved for the later runtime mapping chunk.
202
- - **`getDiagnostics()`**: exposes scaffold diagnostics and reports `runtimeMappingImplemented: false`.
310
+ - **`localOnly`**: always `true`; the processor performs no network I/O and does not install itself globally.
311
+ - **`writer`**: optional explicit local `TraceWriter` for tests, recipes, and controlled runtime integration.
312
+ - **`traceDir`**: optional local directory that creates a file writer inside the adapter package.
313
+ - **`workflowName`**: optional local run name overriding the SDK trace name.
314
+ - **`capture`**: `"metadata-only"` (default) or `"preview"`; `preview` currently emits a diagnostic and falls back to metadata-only.
315
+ - **`redactionProfile`** and **`maxPreviewChars`**: preview-only knobs; when preview capture is unsupported or not selected, they emit diagnostics instead of silently doing nothing.
316
+ - **`getDiagnostics()`**: exposes isolated processor write, lifecycle/configuration, flush, and shutdown failures without throwing into OpenAI Agents callbacks.
317
+ - **`getWriterStats()`**, **`forceFlush()`**, and **`shutdown()`**: explicit writer lifecycle helpers. Failures are captured in diagnostics.
203
318
 
204
319
  Safe future usage must replace processors explicitly:
205
320
 
@@ -209,11 +324,13 @@ setTraceProcessors([agentInspectProcessor({ traceDir: "./.agent-inspect" })]);
209
324
 
210
325
  Do not use `addTraceProcessor()` as the default AgentInspect path; that leaves existing/default processors in place and can preserve backend export behavior in server runtimes.
211
326
 
212
- ## 13. Experimental persisted-event foundation (v1.2.0)
327
+ The processor records local v0.2 persisted events for trace/run, agent, generation/response, function/tool, handoff, guardrail, MCP tools, custom, transcription, and speech span metadata where safely representable. It does not persist raw prompts, messages, generated text, function inputs/outputs, arbitrary custom data, trace exporter credentials, headers, request bodies, response bodies, or hosted tool payloads by default.
328
+
329
+ ## 15. Experimental persisted-event foundation (v1.2.0)
213
330
 
214
331
  These helpers expose the **source-agnostic `PersistedInspectEvent` model** (`schemaVersion: "0.2"`). They are **local-only**, **in-memory**, and **do not change** storage write/read or CLI behavior in v1.2.0.
215
332
 
216
- Import from `agent-inspect`:
333
+ Import from `agent-inspect/persisted`:
217
334
 
218
335
  | API | Role |
219
336
  | --- | ---- |
@@ -235,7 +352,7 @@ Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEv
235
352
  - v0.2 is **not written by default**; use converters and `fixtures/traces-v0.2/` samples for validation.
236
353
  - Inspection read paths normalize v0.1 and v0.2 JSONL for local CLI/API use. v0.2 remains experimental as a persisted-event foundation and is not the default writer.
237
354
 
238
- ## 14. Local observability helpers (v1.4.0+)
355
+ ## 16. Local observability helpers (v1.4.0+)
239
356
 
240
357
  Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1 manual traces remain the default writer; v0.2 persisted-event files are accepted where the shared dual-format read path is used. Local files only.
241
358
 
@@ -245,7 +362,7 @@ Read-only helpers for timeline, stats, and search over local JSONL traces. v0.1
245
362
 
246
363
  CLI wrappers: `agent-inspect timeline`, `stats`, `search` — see [CLI.md](./CLI.md).
247
364
 
248
- ## 15. Report and what helpers (v1.5.0+)
365
+ ## 17. Report and what helpers (v1.5.0+)
249
366
 
250
367
  Read-only helpers for concise inspection summaries and local reports:
251
368
 
@@ -254,7 +371,7 @@ Read-only helpers for concise inspection summaries and local reports:
254
371
 
255
372
  Report redaction profiles are key-based safeguards applied to the complete rendered report input, not only to the tree section. Review generated reports before sharing; this is not compliance-grade DLP.
256
373
 
257
- ## 16. Experimental trace writers (v1.6)
374
+ ## 18. Experimental trace writers (v1.6)
258
375
 
259
376
  Trace writers are the first slice of the v1.6 runtime foundation. They are experimental during v1.x and intended for tests, adapters, and future `createInspector` work.
260
377
 
@@ -286,7 +403,7 @@ import type {
286
403
 
287
404
  No network writer or vendor sink exists in this package.
288
405
 
289
- ## 17. Experimental inspector API/runtime (v1.6)
406
+ ## 19. Experimental inspector API/runtime (v1.6)
290
407
 
291
408
  `createInspector()` is the experimental public instance API for local-first tracing with explicit writers. It owns an instance-specific runtime context, records v0.2 persisted inspect events, preserves application return values/errors, and exposes diagnostics plus deterministic `flush()`/`close()` lifecycle hooks.
292
409
 
@@ -328,7 +445,7 @@ Public methods:
328
445
 
329
446
  These APIs are experimental during v1.x. They do not add a default network writer or vendor sink.
330
447
 
331
- ## 18. Experimental trace readers (v1.6)
448
+ ## 20. Experimental trace readers (v1.6)
332
449
 
333
450
  `agent-inspect/readers` exposes the experimental local trace reader contract and detection pipeline. It includes AgentInspect JSONL for v0.1, v0.2, and mixed local trace files, plus local OpenInference JSON and OTLP JSON compatibility readers.
334
451
 
@@ -359,23 +476,94 @@ import type { TraceReader } from "agent-inspect/readers";
359
476
 
360
477
  The reader contract does not silently accept arbitrary JSON and does not add OTel SDK, database, hosted ingestion, or network upload dependencies.
361
478
 
362
- ## 19. Deprecated APIs
479
+ ## 21. Experimental Checks
480
+
481
+ `agent-inspect/checks` exposes the experimental deterministic trace-check engine foundation. It consumes normalized reader output, runs supplied pure rules in stable order, and returns aggregate findings/diagnostics. It does not read files, discover config, call providers, perform network I/O, mutate inputs, or create a new persisted schema.
482
+
483
+ Import from `agent-inspect/checks`:
484
+
485
+ ```ts
486
+ import { runTraceChecks } from "agent-inspect/checks";
487
+ import type { TraceCheckRule, TraceCheckResult } from "agent-inspect/checks";
488
+ ```
489
+
490
+ - **`runTraceChecks({ read }, { rules?, select?, runId? })`**: executes provided rules against a `TraceReadResult` from `agent-inspect/readers`.
491
+ - **Built-in rule factories**: run, tool, LLM, structure, retrieval, guardrail, decision, safety, and baseline helpers including `createRunStatusRule`, `createToolUsageRule`, `createLlmUsageRule`, `createStructureOrphanRule`, `createStructureCycleRule`, `createStructureRelationshipRule`, `createRetrievalRule`, `createGuardrailRule`, `createDecisionRule`, `createSafetyRawContentRule`, `createSafetySecretPatternRule`, and `createBaselineRegressionRule`.
492
+ - **`TraceCheckRule`**: synchronous pure rule contract.
493
+ - **`TraceCheckResult`**: deterministic aggregate result with findings, evidence, summary counts, and execution diagnostics.
494
+
495
+ The checks API is experimental in v1.x. The `agent-inspect check` CLI uses this API for local reader-backed checks and deterministic JSON output; `agent-inspect artifacts` reuses the same safe findings for local CI artifact bundles and optional step-summary file output. Built-in rules operate on normalized event metadata, tree relationships, bounded summaries, token counts, and normalized baseline facts; safety and baseline findings identify event IDs and field paths rather than emitting raw prompts, outputs, secrets, headers, request/response bodies, or full tool payloads.
496
+
497
+ Recipes: [deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for check/baseline/artifact workflows, and [test-reporter-artifacts](../examples/recipes/test-reporter-artifacts/README.md) for Vitest/Jest reporter configuration patterns.
498
+
499
+ ## 22. Experimental local explain APIs (v1.9)
500
+
501
+ `buildLocalExplanation()` creates a deterministic local explanation payload from a reader-selected `InspectRunTree`. It performs no network I/O, does not call model providers, and separates observed facts from deterministic inference labels.
502
+
503
+ Import from `agent-inspect`:
504
+
505
+ ```ts
506
+ import { buildLocalExplanation } from "agent-inspect";
507
+ ```
508
+
509
+ - **`buildLocalExplanation(run, options?)`**:
510
+ - **`mode: "dry-run"`**: returns redacted observed facts and no inference labels.
511
+ - **`mode: "local"`** (default): returns observed facts plus deterministic local inference labels.
512
+ - **`redactionProfile`**: `local`, `share`, or `strict`; profile keys are redacted before the payload is returned.
513
+
514
+ CLI wrapper: `agent-inspect explain <trace-path-or-run-id> --dry-run --json`.
515
+
516
+ Provider design gate:
517
+
518
+ - No provider payload is submitted in v1.9 implementation chunks; `--provider <provider>` is reserved and rejected with `PROVIDER_NOT_IMPLEMENTED`.
519
+ - The reviewable provider payload contract is the `ExplainResult` object: `mode`, `runId`, optional `name` / `status`, `redactionProfile`, `facts`, `inferences`, and `notes`.
520
+ - Provider implementations must require explicit provider selection and documented environment requirements. The current local API reads no provider credentials.
521
+ - Provider prompts must use redacted facts only, label inferred claims, and must not request raw chain-of-thought.
522
+ - Provider packages or SDKs must not become root/core runtime dependencies.
523
+
524
+ ## 23. Experimental `@agent-inspect/harness` APIs
525
+
526
+ `@agent-inspect/harness` is a private experimental workspace package during the v1.9 release train. It provides a no-framework fixture runner for local targets and recipes; first public package publication remains a manual maintainer gate.
527
+
528
+ Import from `@agent-inspect/harness` inside the workspace:
529
+
530
+ ```ts
531
+ import { createFixtureRunner, defineTarget } from "@agent-inspect/harness";
532
+ ```
533
+
534
+ - **`defineTarget(definition)`**: returns a typed target definition with `resolve(app, context)` and `invoke(target, input, context)` hooks.
535
+ - **`createFixtureRunner(options)`**: returns a local runner with:
536
+ - **`listTargets()`**: deterministic target metadata listing.
537
+ - **`runTarget(name, input, options?)`**: bootstrap, resolve, invoke, and shutdown lifecycle.
538
+ - **`runFromArgv(argv?, io?)`**: CLI-friendly execution with target listing, JSON fixture files, JSON stdin, JSON stdout, stderr summaries, trace flags, and expected-output comparison.
539
+ - **`getDiagnostics()`**: deterministic diagnostics for missing targets, bootstrap failures, resolve failures, invocation failures, and shutdown failures.
540
+ - **`trace`** options use existing AgentInspect local APIs only:
541
+ - **`mode: "run-if-enabled"`** (default): uses `maybeInspectRun()` and writes no trace unless `options.enabled` or `AGENT_INSPECT` enables tracing.
542
+ - **`mode: "run"`**: explicitly wraps the target invocation in `inspectRun()`.
543
+ - **`mode: "observe"`**: proxies the resolved target with `observe()` for `run` / `execute` / `invoke` methods when enabled.
544
+ - **`mode: "off"`**: invokes the target without AgentInspect tracing.
545
+
546
+ The harness package does not add root/core dependencies, does not upload traces, does not call providers, and does not capture raw prompts or outputs by itself. It writes only local AgentInspect traces when explicitly enabled by runner options or environment-gated tracing.
547
+
548
+ Recipes: [harness-basic](../examples/recipes/harness-basic/README.md) and [harness-adapter-local](../examples/recipes/harness-adapter-local/README.md).
549
+
550
+ ## 24. Deprecated APIs
363
551
 
364
552
  No deprecated APIs are declared as of 1.4.0.
365
553
 
366
- ## 20. Removal / deprecation policy
554
+ ## 25. Removal / deprecation policy
367
555
 
368
556
  - Stable APIs are not removed in v1.x.
369
557
  - If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
370
558
 
371
- ## 21. Backward compatibility policy
559
+ ## 26. Backward compatibility policy
372
560
 
373
561
  - Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
374
562
  - Additive schema changes are allowed in minor versions.
375
563
  - Breaking changes require a major version.
376
564
  - Unknown fields should be ignored where safe.
377
565
 
378
- ## 22. Examples
566
+ ## 25. Examples
379
567
 
380
568
  ### Minimal manual trace
381
569
 
package/docs/CLI.md CHANGED
@@ -26,12 +26,17 @@ Core commands:
26
26
  - `tail` — live-tail logs into updating local trees
27
27
  - `export` — export manual traces to Markdown/HTML/OpenInference/OTLP JSON (local only)
28
28
  - `open` — read supported local trace files, directories, or stdin through the canonical reader pipeline
29
+ - `check` — run deterministic local trace checks with stable JSON and exit codes
30
+ - `scan` — best-effort local safety scan for trace capture risks
31
+ - `verify-safe` — best-effort local trace safety verification
32
+ - `artifacts` — create safe local CI trace artifact bundles and optional step summaries
29
33
  - `diff` — compare two manual traces (local, read-only)
30
34
  - `timeline` — chronological view of one run (local JSONL)
31
35
  - `stats` — local aggregate stats over a trace directory
32
36
  - `search` — deterministic local search over traces
33
37
  - `what` — concise summary of a single run (local JSONL)
34
38
  - `report` — markdown or HTML inspection report for a single run
39
+ - `explain` — deterministic local facts/inferences for a trace, with dry-run payloads
35
40
 
36
41
  ## 2. Environment variables
37
42
 
@@ -44,6 +49,20 @@ Core commands:
44
49
  - **0**: command succeeded (even if a diff reports “differences”)
45
50
  - **1**: command error (invalid args, missing files, missing runs, parse failures, validation failures, etc.)
46
51
 
52
+ Exception: `check` uses CI-oriented semantic exit codes:
53
+
54
+ - **0**: all selected checks passed
55
+ - **1**: checks ran and at least one error-severity rule failed
56
+ - **2**: invalid arguments or invalid config
57
+ - **3**: trace input could not be read
58
+ - **4**: unsupported or ambiguous trace format
59
+
60
+ Exception: `scan` and `verify-safe` use local safety status exit codes:
61
+
62
+ - **0**: status is SAFE or SAFE WITH WARNINGS
63
+ - **1**: status is UNSAFE
64
+ - **2**: status is UNKNOWN, including unreadable, unsupported, ambiguous, or invalid inputs
65
+
47
66
  AgentInspect favors **human-readable errors without stack traces** for expected user mistakes.
48
67
 
49
68
  ## 4. JSON output policy
@@ -59,6 +78,8 @@ Many commands support `--json` for scripting. JSON output is intended to be:
59
78
  - Log-derived output includes **confidence** labels and avoids inventing parent-child relationships.
60
79
  - Redaction defaults are conservative (e.g. `authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`).
61
80
  - Exported payloads are **redacted by default** unless explicitly configured otherwise.
81
+ - `scan` and `verify-safe` are best-effort local checks, not compliance, privacy, security, or regulatory certifications.
82
+ - `artifacts` renders structural summaries and check evidence only; it does not include raw prompt/output bodies, request/response bodies, headers, API keys, secrets, or full tool payloads.
62
83
 
63
84
  ## 6. Command reference
64
85
 
@@ -118,7 +139,7 @@ Recommendation: run with `--dry-run` first.
118
139
 
119
140
  ### 6.4 `logs`
120
141
 
121
- Parse structured logs into local execution trees.
142
+ Advanced ingestion: use this when your app already emits structured logs. Parse those logs into local execution trees.
122
143
 
123
144
  ```bash
124
145
  agent-inspect logs <file> [options]
@@ -222,7 +243,135 @@ cat packages/core/test/fixtures/openinference-basic.json | npx agent-inspect ope
222
243
 
223
244
  When a directory or payload contains multiple runs, `open` lists the run ids and exits until you pass `--run <run-id>`.
224
245
 
225
- ### 6.8 `diff`
246
+ ### 6.8 `check`
247
+
248
+ Run deterministic checks against a local trace. This command is local and read-only: it does not rerun agents, call models, upload traces, or mutate input files.
249
+
250
+ ```bash
251
+ agent-inspect check <trace-path-or-run-id> [options]
252
+ ```
253
+
254
+ `<trace-path-or-run-id>` may be a trace file, directory, `-` for stdin, or a run id resolved with `--dir`.
255
+
256
+ Options:
257
+
258
+ - `--dir <path>`: trace directory for run-id lookup
259
+ - `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
260
+ - `--run <run-id>`: select a run when input contains multiple runs
261
+ - `--config <path>`: check config (`.json`, `.js`, `.mjs`, or `.cjs`)
262
+ - `--json`: print deterministic `TraceCheckResult` JSON
263
+ - `--rule <id>`: select a rule id; repeatable
264
+ - `--max-duration-ms <number>`: add `run.duration`
265
+ - `--required-tool <name>` / `--forbidden-tool <name>`: add `tool.usage`
266
+ - `--allowed-model <model>` / `--max-total-tokens <number>`: add `llm.usage`
267
+
268
+ By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available.
269
+
270
+ Config files use this shape:
271
+
272
+ ```json
273
+ {
274
+ "checks": {
275
+ "select": ["run.status", "run.duration"],
276
+ "run": { "maxDurationMs": 30000 },
277
+ "tool": { "required": ["search_docs"] },
278
+ "llm": { "allowedModels": ["gpt-4.1-mini"], "maxTotalTokens": 12000 }
279
+ }
280
+ }
281
+ ```
282
+
283
+ YAML is not supported. TypeScript config files (`.ts`, `.mts`, `.cts`) fail clearly unless a future explicit loader strategy is added; use precompiled JavaScript config instead.
284
+
285
+ Examples:
286
+
287
+ ```bash
288
+ npx agent-inspect check fixtures/traces-v0.2/manual-basic.jsonl --json
289
+ npx agent-inspect check minimal-success --dir fixtures/traces --rule run.status
290
+ npx agent-inspect check trace.jsonl --max-duration-ms 30000 --required-tool search_docs --json
291
+ ```
292
+
293
+ Recipe: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
294
+
295
+ ### 6.9 `scan` and `verify-safe`
296
+
297
+ Run best-effort local safety verification for supported trace inputs. These commands are local and read-only: they do not rerun agents, call models, upload traces, mutate input files, or certify compliance.
298
+
299
+ ```bash
300
+ agent-inspect scan <trace-path-or-run-id> [options]
301
+ agent-inspect verify-safe <trace-path-or-run-id> [options]
302
+ ```
303
+
304
+ `<trace-path-or-run-id>` may be a trace file, directory, `-` for stdin, or a run id resolved with `--dir`.
305
+
306
+ Statuses:
307
+
308
+ - `SAFE`: no safety findings and no reader warnings.
309
+ - `SAFE WITH WARNINGS`: no safety findings, but the reader reported warnings or unsupported fields.
310
+ - `UNSAFE`: safety findings were detected.
311
+ - `UNKNOWN`: the input could not be read, normalized, or selected conservatively.
312
+
313
+ Options:
314
+
315
+ - `--dir <path>`: trace directory for run-id lookup
316
+ - `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
317
+ - `--run <run-id>`: select a run when input contains multiple runs
318
+ - `--json`: print deterministic JSON safety result
319
+ - `--max-string-length <number>`: unsafe threshold for string values
320
+ - `--max-array-length <number>`: unsafe threshold for array values
321
+ - `--max-object-keys <number>`: unsafe threshold for object key counts
322
+ - `--max-serialized-bytes <number>`: unsafe threshold for serialized values
323
+
324
+ The scan looks for raw prompt/output-like capture paths, unredacted sensitive-looking keys, secret-like string patterns, and oversized values. It reports evidence paths rather than raw prompt, output, request/response, header, API key, secret, or full tool payload values. Secret detection is best-effort and should not be treated as exhaustive.
325
+
326
+ Examples:
327
+
328
+ ```bash
329
+ npx agent-inspect scan fixtures/traces-v0.2/manual-basic.jsonl --json
330
+ npx agent-inspect verify-safe minimal-success --dir fixtures/traces
331
+ npx agent-inspect verify-safe trace.jsonl --max-string-length 8192 --json
332
+ ```
333
+
334
+ ### 6.10 `artifacts`
335
+
336
+ Create deterministic local CI artifacts for supported trace inputs. This command is local and read-only for trace inputs: it does not rerun agents, call models, upload files, use GitHub APIs, or mutate repository state. It writes only to `--output-dir` and, when requested, a local step-summary file.
337
+
338
+ ```bash
339
+ agent-inspect artifacts <trace-path-or-run-id> --output-dir <path> [options]
340
+ ```
341
+
342
+ Generated files:
343
+
344
+ - `trace.json`: structural trace summary only
345
+ - `check.json`: safety check result
346
+ - `diff.json`: baseline diff result, or `not_requested`
347
+ - `summary.md`: safe Markdown CI summary
348
+ - `report.html`: safe HTML CI summary
349
+ - `manifest.json`: deterministic file/status manifest
350
+
351
+ Options:
352
+
353
+ - `--output-dir <path>`: required local artifact directory
354
+ - `--dir <path>`: trace directory for run-id lookup
355
+ - `--format <agent-inspect-jsonl|openinference-json|otlp-json>`: explicit reader format override
356
+ - `--run <run-id>`: select a run when input contains multiple runs
357
+ - `--baseline <trace-path-or-run-id>`: optional baseline trace for diff artifacts
358
+ - `--baseline-run <run-id>`: select a run from the baseline trace
359
+ - `--github-summary <path>`: append the safe Markdown summary to this file, such as `$GITHUB_STEP_SUMMARY`
360
+ - `--json`: print deterministic `manifest.json` content
361
+
362
+ The artifact command runs safety checks before rendering and only includes structural counts, statuses, bounded check findings, diagnostics, and evidence paths. Baseline diff artifacts use normalized baseline checks and also avoid raw prompt/output/tool payload values. `--github-summary` is plain local file output; AgentInspect does not call GitHub APIs or upload artifacts.
363
+
364
+ Examples:
365
+
366
+ ```bash
367
+ npx agent-inspect artifacts fixtures/traces-v0.2/manual-basic.jsonl --output-dir ./artifacts --json
368
+ npx agent-inspect artifacts minimal-success --dir fixtures/traces --output-dir ./artifacts --github-summary "$GITHUB_STEP_SUMMARY"
369
+ npx agent-inspect artifacts candidate.jsonl --baseline baseline.jsonl --output-dir ./artifacts
370
+ ```
371
+
372
+ Recipe and sample workflow: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
373
+
374
+ ### 6.11 `diff`
226
375
 
227
376
  Compare two manual trace runs. Diff is **local** and **read-only** (does not rerun agents).
228
377
 
@@ -286,7 +435,7 @@ Differences:
286
435
 
287
436
  More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
288
437
 
289
- ### 6.9 `timeline`
438
+ ### 6.12 `timeline`
290
439
 
291
440
  Chronological step list for one manual trace. Read-only; does not mutate JSONL files.
292
441
 
@@ -302,7 +451,7 @@ Options:
302
451
 
303
452
  ![Timeline with slow-step focus](../assets/demos/timeline.gif)
304
453
 
305
- ### 6.10 `stats`
454
+ ### 6.13 `stats`
306
455
 
307
456
  Local aggregate statistics over trace files in a directory. Read-only.
308
457
 
@@ -322,7 +471,7 @@ Options:
322
471
 
323
472
  Use `--correlation-id` or `--group-id` to filter runs by `run_started` metadata (see [API.md](./API.md)).
324
473
 
325
- ### 6.11 `search`
474
+ ### 6.14 `search`
326
475
 
327
476
  Deterministic search over local traces (substring / exact filters). No semantic search.
328
477
 
@@ -352,7 +501,7 @@ npx agent-inspect search --duration ">100ms" --json
352
501
 
353
502
  ![Search traces by status error](../assets/demos/search.gif)
354
503
 
355
- ### 6.12 `what`
504
+ ### 6.15 `what`
356
505
 
357
506
  Concise human-readable summary of one local trace run. Read-only; accepts v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path. Vocabulary: [TRACE-VOCABULARY-V1.5.md](./proposals/TRACE-VOCABULARY-V1.5.md).
358
507
 
@@ -381,7 +530,7 @@ Outcome: Completed successfully.
381
530
  Slowest: plan (100ms, logic)
382
531
  ```
383
532
 
384
- ### 6.13 `report`
533
+ ### 6.16 `report`
385
534
 
386
535
  Generate a local inspection report combining **what happened**, **timeline**, and **execution tree** sections. The command reads local v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path without mutating them. Distinct from `export` (which targets shareable tree snapshots and standards formats).
387
536
 
@@ -406,6 +555,39 @@ Example:
406
555
  npx agent-inspect report minimal-success --dir fixtures/traces --format html -o report.html
407
556
  ```
408
557
 
558
+ ### 6.17 `explain`
559
+
560
+ Explain a local trace using deterministic facts and local inference labels. This command reads through the same local reader pipeline as `open` / `check`; it does not call a model provider, upload traces, replay agents, or mutate input files.
561
+
562
+ ```bash
563
+ agent-inspect explain <trace-path-or-run-id> [options]
564
+ ```
565
+
566
+ Options:
567
+
568
+ - `--dir <path>` — trace directory for run-id lookup
569
+ - `--format <agent-inspect-jsonl|openinference-json|otlp-json>` — explicit input format
570
+ - `--run <run-id>` — select a run when the trace contains multiple runs
571
+ - `--dry-run` — emit only the redacted facts payload, with no local inference labels
572
+ - `--provider <provider>` — reserved for an explicit future provider mode; currently rejected without network calls
573
+ - `--json` — print deterministic JSON output
574
+ - `--redaction-profile <local|share|strict>` — key-based redaction profile for the explanation payload (default `local`)
575
+
576
+ Examples:
577
+
578
+ ```bash
579
+ npx agent-inspect explain minimal-success --dir fixtures/traces
580
+ npx agent-inspect explain fixtures/traces/minimal-success.jsonl --dry-run --json --redaction-profile strict
581
+ ```
582
+
583
+ Provider design gate:
584
+
585
+ - Current behavior is local only. `--provider <provider>` exits with a user-facing `PROVIDER_NOT_IMPLEMENTED` error and performs no provider call.
586
+ - `--dry-run --json` is the payload review surface. The provider payload contract is the returned `explanation` object: `mode`, `runId`, optional `name` / `status`, `redactionProfile`, `facts`, `inferences`, and `notes`.
587
+ - Provider chunks must require explicit provider selection, document required environment variables, and keep credentials out of trace data and dry-run output.
588
+ - Provider prompts must ask for concise explanations from redacted facts only. They must not request, expose, or preserve raw chain-of-thought.
589
+ - Cloud provider behavior is never selected by default and must be reviewed before implementation. Local provider support must still be explicit and opt-in.
590
+
409
591
  ## 7. Optional TUI behavior
410
592
 
411
593
  `view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.