agent-inspect 6.19.0 → 6.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. package/CHANGELOG.md +12 -0
  2. package/README.md +3 -1
  3. package/SECURITY.md +2 -0
  4. package/docs/ARCHITECTURE.md +28 -16
  5. package/docs/KNOWN-ISSUES.md +3 -3
  6. package/docs/LIMITATIONS.md +5 -4
  7. package/docs/PERFORMANCE.md +1 -1
  8. package/docs/SUPPORT-LEVELS.md +2 -1
  9. package/docs/TRACE-CONTRACTS.md +64 -52
  10. package/package.json +3 -2
  11. package/packages/cli/dist/{chunk-R7Y5SGH5.mjs → chunk-QPRJUVLH.mjs} +57 -17
  12. package/packages/cli/dist/chunk-QPRJUVLH.mjs.map +1 -0
  13. package/packages/cli/dist/index.cjs +69 -18
  14. package/packages/cli/dist/index.cjs.map +1 -1
  15. package/packages/cli/dist/index.mjs +9 -6
  16. package/packages/cli/dist/index.mjs.map +1 -1
  17. package/packages/cli/dist/{src-T4EZERCX.mjs → src-6W5ZR3I7.mjs} +3 -3
  18. package/packages/cli/dist/{src-T4EZERCX.mjs.map → src-6W5ZR3I7.mjs.map} +1 -1
  19. package/packages/core/dist/advanced.cjs +118 -18
  20. package/packages/core/dist/advanced.cjs.map +1 -1
  21. package/packages/core/dist/advanced.d.cts +2 -2
  22. package/packages/core/dist/advanced.d.ts +2 -2
  23. package/packages/core/dist/advanced.mjs +8 -8
  24. package/packages/core/dist/checks.cjs +443 -9
  25. package/packages/core/dist/checks.cjs.map +1 -1
  26. package/packages/core/dist/checks.d.cts +78 -16
  27. package/packages/core/dist/checks.d.ts +78 -16
  28. package/packages/core/dist/checks.mjs +3 -3
  29. package/packages/core/dist/{chunk-VFO76UH3.mjs → chunk-3XS5O4JI.mjs} +3 -3
  30. package/packages/core/dist/{chunk-VFO76UH3.mjs.map → chunk-3XS5O4JI.mjs.map} +1 -1
  31. package/packages/core/dist/{chunk-ZWA5RWMV.mjs → chunk-5YFJJFXU.mjs} +4 -4
  32. package/packages/core/dist/{chunk-ZWA5RWMV.mjs.map → chunk-5YFJJFXU.mjs.map} +1 -1
  33. package/packages/core/dist/{chunk-NABMW5DX.mjs → chunk-G47KQB42.mjs} +445 -13
  34. package/packages/core/dist/chunk-G47KQB42.mjs.map +1 -0
  35. package/packages/core/dist/{chunk-SGL4JGCD.mjs → chunk-JBNRJLIV.mjs} +58 -18
  36. package/packages/core/dist/chunk-JBNRJLIV.mjs.map +1 -0
  37. package/packages/core/dist/{chunk-ULCGIRTC.mjs → chunk-U25RHRSU.mjs} +65 -5
  38. package/packages/core/dist/chunk-U25RHRSU.mjs.map +1 -0
  39. package/packages/core/dist/{context-Ca5yK8Cs.d.cts → context-C5ye_xcS.d.cts} +0 -4
  40. package/packages/core/dist/{context-D6cS4XIL.d.ts → context-CNsNSgE2.d.ts} +0 -4
  41. package/packages/core/dist/exporters.cjs +55 -15
  42. package/packages/core/dist/exporters.cjs.map +1 -1
  43. package/packages/core/dist/exporters.mjs +3 -3
  44. package/packages/core/dist/{index-DKwuGSe-.d.ts → index-Bg9ZdsdS.d.ts} +10 -0
  45. package/packages/core/dist/{index-DCR816Yt.d.cts → index-rVpX5PPh.d.cts} +10 -0
  46. package/packages/core/dist/index.cjs +63 -3
  47. package/packages/core/dist/index.cjs.map +1 -1
  48. package/packages/core/dist/index.d.cts +1 -1
  49. package/packages/core/dist/index.d.ts +1 -1
  50. package/packages/core/dist/index.mjs +4 -4
  51. package/packages/core/dist/writers.cjs +33 -2
  52. package/packages/core/dist/writers.cjs.map +1 -1
  53. package/packages/core/dist/writers.mjs +1 -1
  54. package/packages/cli/dist/chunk-R7Y5SGH5.mjs.map +0 -1
  55. package/packages/core/dist/chunk-NABMW5DX.mjs.map +0 -1
  56. package/packages/core/dist/chunk-SGL4JGCD.mjs.map +0 -1
  57. package/packages/core/dist/chunk-ULCGIRTC.mjs.map +0 -1
package/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # Changelog
2
2
 
3
+ ## 6.20.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 43a4481: Flexible deterministic contracts: selectable `requiredOrderMode` (`first-occurrence` | `happens-before` | `all-occurrences`), one-level `alternatives.anyOf`, and `lintTraceContract` / `explainTraceContract` helpers. Includes MCP expected-rejection and local Promptfoo use-together recipes.
8
+
9
+ ## 6.19.1
10
+
11
+ ### Patch Changes
12
+
13
+ - e23b94b: 6.19.1 trust restoration: redact free-text credentials in persisted error messages before disk, refresh shipped docs to match 6.19.0 reality, and enforce all 18 fixed-group package READMEs (including root `agent-inspect`) via `package-readmes:check`.
14
+
3
15
  ## 6.19.0
4
16
 
5
17
  ### Minor Changes
package/README.md CHANGED
@@ -33,6 +33,8 @@
33
33
  <a href="package.json"><img src="https://img.shields.io/badge/node-%3E%3D20-brightgreen" alt="Node.js 20 or newer"></a>
34
34
  </p>
35
35
 
36
+ **Support level:** Stable — see [SUPPORT-LEVELS.md](docs/SUPPORT-LEVELS.md). Network behavior: [NETWORK-BEHAVIOR.md](docs/NETWORK-BEHAVIOR.md).
37
+
36
38
  ```bash
37
39
  npm install agent-inspect
38
40
  ```
@@ -210,7 +212,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
210
212
 
211
213
  ## Status and documentation
212
214
 
213
- **Current published baseline:** **6.19.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
215
+ **Current published baseline:** **6.20.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
214
216
 
215
217
  Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
216
218
 
package/SECURITY.md CHANGED
@@ -82,6 +82,8 @@ For a practical pre-share workflow, see `docs/SAFE-TRACE-SHARING.md`. For schema
82
82
 
83
83
  Pull requests run GitHub **Dependency review** (`.github/workflows/dependency-review.yml`) and fail on newly introduced **high** or **critical** advisory severity. Routine unit/typecheck/size CI also exercises redaction, path containment, and Evidence safety tests. These scanners do **not** replace manual review of share profiles or Evidence packages before you disclose traces.
84
84
 
85
+ For synthetic fixtures, dynamic-import boundaries, and an explicit “what scanners verify vs not” table, see [docs/SECURITY-SCANNERS.md](docs/SECURITY-SCANNERS.md).
86
+
85
87
  ## Optional surfaces (v2.6)
86
88
 
87
89
  ### Local viewer (`agent-inspect serve`)
@@ -1,27 +1,41 @@
1
1
  ## Architecture
2
2
 
3
- > **Adoption-focused overview:** see [TECHNICAL-GUIDE.md](./TECHNICAL-GUIDE.md) for a comprehensive technical guide (v3.5.5) suitable for blogs and outreach.
3
+ > **Adoption-focused overview:** see [TECHNICAL-GUIDE.md](./TECHNICAL-GUIDE.md) for a comprehensive technical guide suitable for blogs and outreach.
4
4
 
5
- AgentInspect is a **local-first execution-tree debugger**: manual traces and log ingest produce inspectable step trees stored as JSONL on disk (default `.agent-inspect-runs/`), with a CLI for list/view/clean/logs/tail/export/diff workflows.
5
+ AgentInspect is a **local-first execution-tree debugger and trajectory-test toolkit** for TypeScript AI agents: framework-native capture and manual traces produce inspectable step trees as local JSONL (default `.agent-inspect`), with CLI, TraceContract / TraceFacts checks, Evidence v2 packaging, and optional read-only MCP.
6
6
 
7
7
  ### Package layout
8
8
 
9
9
  | Package | Published? | Role |
10
10
  | -------- | ---------- | ---- |
11
11
  | `agent-inspect` | Yes | Public tarball: core tracing APIs + CLI (`agent-inspect` binary) |
12
- | `@agent-inspect/core` | No (private) | Tracing, storage, log parsing, export, diff |
12
+ | `@agent-inspect/core` | No (private) | Tracing, storage, readers/writers, checks, export, diff |
13
13
  | `@agent-inspect/cli` | No (private) | Commander CLI implementation |
14
- | `@agent-inspect/langchain` | Yes (optional) | LangChain.js callback adapter (experimental) |
15
- | `@agent-inspect/tui` | Yes (optional) | Ink/React terminal viewer (experimental) |
16
-
17
- Root `agent-inspect` uses **conditional exports** for ESM/CJS TypeScript consumers (`import.types` / `require.types`). Heavy dependencies (LangChain, Ink/React) stay in optional packages.
14
+ | `@agent-inspect/redact` | Yes | Standalone redaction engine (also used by CLI `redact`) |
15
+ | `@agent-inspect/ai-sdk` | Yes (optional) | Vercel AI SDK telemetry integration |
16
+ | `@agent-inspect/openai-agents` | Yes (optional) | OpenAI Agents JS `setTraceProcessors()` adapter |
17
+ | `@agent-inspect/langchain` | Yes (optional) | LangChain.js / LangGraph callback adapter |
18
+ | `@agent-inspect/mcp` | Yes (optional) | MCP client `tools/list` / `tools/call` tracing |
19
+ | `@agent-inspect/adapter-sdk` | Yes (optional) | Shared adapter helpers / plugin surface |
20
+ | `@agent-inspect/vitest` / `@agent-inspect/jest` | Yes (optional) | Test reporters and TraceContract matchers |
21
+ | `@agent-inspect/eval` | Yes (optional) | Deterministic local eval heuristics |
22
+ | `@agent-inspect/guardrails` / `@agent-inspect/circuit` | Yes (optional) | Local guardrail and circuit utilities |
23
+ | `@agent-inspect/harness` | Yes (optional) | Lightweight local harness helpers |
24
+ | `@agent-inspect/viewer` | Yes (optional) | Local HTML/viewer surface |
25
+ | `@agent-inspect/tui` | Yes (optional) | Ink/React terminal viewer |
26
+ | `@agent-inspect/mcp-server` | Yes (optional) | Read-only local MCP server over traces |
27
+ | `@agent-inspect/index-sqlite` | Yes (optional) | Optional local SQLite metadata index |
28
+ | `@agent-inspect/studio` | Yes (optional) | Local studio CLI / ingest surfaces |
29
+
30
+ Root `agent-inspect` uses **conditional exports** for ESM/CJS TypeScript consumers (`import.types` / `require.types`). Heavy framework dependencies stay in optional packages.
18
31
 
19
32
  ### Event model and schema
20
33
 
21
- - Manual traces use **`schemaVersion: "0.1"`** JSONL events (`run_started`, `step_started`, `step_completed`, `run_completed`).
22
- - Failures use `step_completed` with `status: "error"` — there is no `step_failed` event.
34
+ - Manual global helpers (`inspectRun()` / `step()`) still write **`schemaVersion: "0.1"`** JSONL for compatibility (`run_started`, `step_started`, `step_completed`, `run_completed`).
35
+ - Failures use `step_completed` / `run_completed` with `status: "error"` — there is no `step_failed` event.
36
+ - **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema **1.0** persisted rows; v0.1 and v0.2 remain readable.
23
37
  - Log-derived runs use confidence labels (`explicit`, `correlated`, `heuristic`, `unknown`) and conservative tree-building rules.
24
- - **v1.2.0 persisted-event foundation (released):** source-agnostic `PersistedInspectEvent` helpers (`schemaVersion: "0.2"`) types, validators, converters, and in-memory tree bridge. Existing **`TreeBuilder`** remains the canonical tree builder. v0.2 bridge works **in memory only**; storage dual-read and CLI integration are v1.3+ / post-foundation work. See [proposals/UNIFIED-PERSISTED-INSPECT-EVENT.md](./proposals/UNIFIED-PERSISTED-INSPECT-EVENT.md) and [implementation/V1.3.0-RELEASE-TRAIN.md](./implementation/V1.3.0-RELEASE-TRAIN.md).
38
+ - Migration is explicit (`agent-inspect migrate …`); AgentInspect does not rewrite old traces in place.
25
39
 
26
40
  See [SCHEMA.md](./SCHEMA.md) for field reference and [API.md](./API.md) for public surfaces (stable vs experimental).
27
41
 
@@ -29,21 +43,19 @@ See [SCHEMA.md](./SCHEMA.md) for field reference and [API.md](./API.md) for publ
29
43
 
30
44
  - Instrumentation must **not throw into user code**; trace safety failures degrade gracefully.
31
45
  - **Manual metadata** is redacted before disk by default; `redact: false` opts out.
46
+ - **Error messages** also run high-confidence free-text credential detectors before disk (provider keys, bearer tokens, JWTs, and related patterns), unless `redact: false`.
32
47
  - **Size bounds** cap persisted event and metadata size.
33
48
  - Log ingest: JSON first-class; log4js best-effort; no `eval` or JS object-literal parsing.
34
49
 
35
50
  See [SECURITY.md](../SECURITY.md) and [LIMITATIONS.md](./LIMITATIONS.md).
36
51
 
37
- ### Optional adapters
38
-
39
- - **`@agent-inspect/langchain`**: in-memory by default; `persist: true` writes local JSONL. Experimental.
40
- - **`@agent-inspect/tui`**: interactive viewer; isolated from root deps. Experimental.
52
+ ### Optional adapters and surfaces
41
53
 
42
- See [ADAPTERS.md](./ADAPTERS.md).
54
+ Official adapters (`@agent-inspect/ai-sdk`, `@agent-inspect/openai-agents`, `@agent-inspect/langchain`) share a bounded preview-capture contract. MCP client tracing, reporters, eval, redact, viewer/TUI, studio, and read-only MCP are separate optional packages — see [ADAPTERS.md](./ADAPTERS.md) and [SUPPORT-LEVELS.md](./SUPPORT-LEVELS.md).
43
55
 
44
56
  ### Architecture proposals
45
57
 
46
- Maintainer-owned RFCs and planning proposals are indexed in [proposals/README.md](./proposals/README.md). The active post-v1.5 roadmap is [implementation/ROADMAP-V1.6-TO-V3.md](./implementation/ROADMAP-V1.6-TO-V3.md).
58
+ Maintainer-owned RFCs and planning proposals are indexed in [proposals/README.md](./proposals/README.md). The active roadmap is [implementation/ROADMAP.md](./implementation/ROADMAP.md).
47
59
 
48
60
  ### Where to read next
49
61
 
@@ -28,10 +28,10 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
28
28
  ## Integrations
29
29
 
30
30
  - **Vendor sinks** (hosted dashboards, Langfuse/Braintrust/New Relic/Datadog native uploads, OTLP gRPC streaming, etc.) are **not implemented** in the core packages described here.
31
- - **AI SDK adapter** (`@agent-inspect/ai-sdk`) is experimental and metadata-first. It depends on explicit AI SDK telemetry configuration and requires `recordInputs: false` / `recordOutputs: false` for the documented safe path.
32
- - **OpenAI Agents JS adapter** (`@agent-inspect/openai-agents`) is experimental and metadata-first. Runtime metadata mapping is local-only; the safe install path is `setTraceProcessors()` rather than `addTraceProcessor()`. The v1.9 package publication retry is pending maintainer-side npm auto-publish setup and is separate from v2 contract work.
31
+ - **AI SDK adapter** (`@agent-inspect/ai-sdk`) is Supported and metadata-first by default. It depends on explicit AI SDK telemetry configuration and requires `recordInputs: false` / `recordOutputs: false` for the documented safe path. Opt-in `capture: "preview"` uses the shared bounded preview helper.
32
+ - **OpenAI Agents JS adapter** (`@agent-inspect/openai-agents`) is Supported and metadata-first. Runtime metadata mapping is local-only; the safe install path is `setTraceProcessors()` rather than `addTraceProcessor()`. Bounded `capture: "preview"` parity ships with the other official adapters.
33
33
  - **LangGraph support** is currently a documented boundary through `@agent-inspect/langchain`, not a dedicated package.
34
- - **LangChain adapter** captures **metadata-oriented** signals by default; it does not replace full framework observability.
34
+ - **LangChain adapter** (`@agent-inspect/langchain`) is Supported; it captures **metadata-oriented** signals by default and shares the same preview contract when `capture: "preview"` is set.
35
35
  - **LangChain `stream: true`** records chunk counts and timing only — not a full token replay. Per-token JSONL events are not emitted.
36
36
  - **Deep swarm / nested RunnableSequence (6.14.2+):** capture resolves parents before registering child indexes and rejects `parentId === stepId`. Legacy traces that still contain self-parent edges are normalized in logical projection (`AI_LOGICAL_SELF_PARENT_REMOVED`) and rendered with visibility-first, cycle-safe trees so nested LLM/tool steps remain visible.
37
37
  - **Correlation metadata** (`correlationId`, `requestId`, `decisionId`, `groupId`) is written on `run_started` but **CLI list/view does not filter by correlation fields** yet.
@@ -13,10 +13,10 @@ This document states what AgentInspect **does not** provide today. It complement
13
13
 
14
14
  - **`list` / `view` / `export`** do not filter by correlation fields yet — `stats --correlation-id` / `--group-id` and `search` provide targeted read paths; full CLI filtering remains incremental.
15
15
 
16
- ## Persisted event model (v1.2.0 foundation)
16
+ ## Persisted event model
17
17
 
18
18
  - **Manual global tracing remains v0.1.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL for compatibility.
19
- - **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema 1.0 persisted rows; v0.2 remains a readable compatibility foundation.
19
+ - **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema 1.0 persisted rows; v0.1 and v0.2 remain readable.
20
20
  - **Migration is explicit, not automatic.** `agent-inspect migrate <input> --to 1.0 --dry-run` reports what would change, and `--output <file>` writes a separate file. AgentInspect does not rewrite old traces in place.
21
21
 
22
22
  ## Runtime writers and universal readers (v1.6)
@@ -32,7 +32,7 @@ This document states what AgentInspect **does not** provide today. It complement
32
32
 
33
33
  - **AI SDK integration is explicit telemetry wiring.** Use `@agent-inspect/ai-sdk` through AI SDK `experimental_telemetry.integrations`; AgentInspect does not wrap providers, patch fetch, or enable telemetry globally.
34
34
  - **AI SDK privacy settings are caller-owned.** Examples set `recordInputs: false` and `recordOutputs: false`; leaving those enabled in user code can cause the AI SDK telemetry layer to include richer data before AgentInspect receives events.
35
- - **OpenAI Agents JS support is experimental.** `@agent-inspect/openai-agents` maps metadata-only runtime spans through the safe `setTraceProcessors()` boundary and does not capture raw payloads by default. The v1.9 package publication retry is a separate maintainer npm automation task, not part of the v2 contract work.
35
+ - **OpenAI Agents JS support is Supported (metadata-first).** `@agent-inspect/openai-agents` maps runtime spans through the safe `setTraceProcessors()` boundary and does not capture raw payloads by default. Bounded `capture: "preview"` is available through the shared adapter capture contract.
36
36
  - **LangGraph support is a boundary decision, not a separate package.** Initial support is expected through `@agent-inspect/langchain` callbacks unless no-network fixtures prove a separate package is needed.
37
37
  - **No root/core adapter dependencies.** AI SDK, OpenAI Agents, LangGraph, OpenTelemetry, and LangChain remain outside the root/core runtime dependency graph.
38
38
  - **Preview capture is bounded, not sanitized.** `capture: "preview"` persists truncated, key-redacted previews of framework-provided input/output fields. Key-based redaction cannot detect a secret embedded in free text, and there is no full-content capture mode. Fields the framework never exposes are reported as `AI_CAPTURE_FIELD_UNAVAILABLE` rather than reconstructed.
@@ -53,7 +53,8 @@ This document states what AgentInspect **does not** provide today. It complement
53
53
 
54
54
  - **Redaction profiles** (`local`, `share`, `strict`) are key-based presets — not compliance-grade PII detection. Review exports before sharing even with `--redaction-profile strict`.
55
55
  - **`@agent-inspect/redact` and `agent-inspect redact` create redacted copies.** They do not encrypt source traces, mutate originals, certify compliance, or guarantee every sensitive value is detected.
56
- - **Default metadata redaction** covers common sensitive keys only (exact key match, case-insensitive). Custom secret field names are not redacted unless you add rules via `redact: { rules: [...] }`.
56
+ - **Default metadata redaction** covers common sensitive keys (exact key match, case-insensitive). Custom secret field names are not redacted unless you add rules via `redact: { rules: [...] }`.
57
+ - **Error-message free text** also runs high-confidence credential detectors before disk (provider API keys, bearer tokens, JWTs, GitHub tokens, PEM private keys, and high-confidence `key=value` secrets). This is still not compliance-grade detection; `redact: false` opts out.
57
58
  - **Metadata truncation** applies to string values and nested structures; very large metadata may be replaced with a truncation marker when `maxEventBytes` is exceeded (default 64 KiB per JSONL line).
58
59
  - **Redaction is not encryption.** Local trace files remain readable on disk; treat `.agent-inspect-runs/` like any developer artifact that may contain operational data.
59
60
 
@@ -30,7 +30,7 @@ npx agent-inspect stats --dir fixtures/performance
30
30
 
31
31
  All three are validated by `pnpm fixtures:check` and small enough for git (the largest is well under 1 MB).
32
32
 
33
- ## Comfortable ranges (v3.4)
33
+ ## Comfortable ranges
34
34
 
35
35
  | Workload | Comfortable | Warning | Not designed for |
36
36
  | -------- | ----------- | ------- | ---------------- |
@@ -49,7 +49,8 @@ shows). `pnpm package-readmes:check` enforces the agreement and runs as part of
49
49
  To promote or demote a surface:
50
50
 
51
51
  1. Edit the row in the matrix above.
52
- 2. Edit the `**Support level:**` line in each affected `packages/*/README.md`.
52
+ 2. Edit the `**Support level:**` line in the affected public README: `README.md`
53
+ for the root `agent-inspect` package, or `packages/*/README.md` for scoped packages.
53
54
  3. Run `pnpm package-readmes:check`.
54
55
 
55
56
  The check also reports packages whose level is **unenforced** — those the matrix
@@ -10,6 +10,9 @@ Contracts compile to deterministic check rules for common cases:
10
10
 
11
11
  - run status / completion / max duration
12
12
  - tool required / forbidden / allowed / maxCalls / order (`requiredTools` / `forbiddenTools` aliases)
13
+ - selectable `requiredOrderMode` (`first-occurrence` | `happens-before` | `all-occurrences`)
14
+ - `alternatives.anyOf` for one level of legitimate alternate paths
15
+ - `lintTraceContract` / `explainTraceContract` for brittle-contract diagnostics
13
16
  - LLM maxCalls / maxTotalTokens / allowedModels
14
17
  - evidence-bearing findings on failures
15
18
  - evaluation over **logical** TraceFacts (raw events remain available)
@@ -24,13 +27,14 @@ Contracts compile to deterministic check rules for common cases:
24
27
  → contract.tool.order.1: B before C
25
28
  ```
26
29
 
27
- Each pair compares the **first occurrence** (start/encounter order in the evaluated event stream):
30
+ `requiredOrderMode` selects one ordering relation for every generated pair:
28
31
 
29
32
  - unlisted intermediate tools are allowed;
30
- - later repetitions do not invalidate an earlier valid first-occurrence order;
31
33
  - TraceContract `requiredOrder` **implies presence** — every listed name is added to the effective required-tool set;
32
- - this is **not** causal happens-before; overlapping intervals emit a non-failing `tool.order.overlap` warning;
33
- - combine ordering with `maxCalls` or custom rules when repeated calls matter.
34
+ - `first-occurrence` (default when omitted) compares first occurrences in start/encounter order; later repetitions do not invalidate an earlier valid order, and interval overlap emits a non-failing `tool.order.overlap` warning;
35
+ - `happens-before` requires the first `before` occurrence to finish before the first `after` occurrence starts;
36
+ - `all-occurrences` requires every `before` occurrence to finish before every `after` occurrence starts (`max(before.end) <= min(after.start)`);
37
+ - causal modes fail when a required interval boundary cannot be resolved instead of falling back to encounter order.
34
38
 
35
39
  Examples for `requiredOrder: ["retrieve", "generate"]`:
36
40
 
@@ -38,12 +42,16 @@ Examples for `requiredOrder: ["retrieve", "generate"]`:
38
42
  | --- | --- |
39
43
  | `retrieve → generate` | PASS |
40
44
  | `retrieve → rerank → generate` | PASS |
41
- | `retrieve → generate → retrieve` | PASS (first-occurrence) |
45
+ | `retrieve → generate → retrieve` | PASS under omitted / `first-occurrence`; FAIL under `all-occurrences` |
42
46
  | `generate → retrieve` | FAIL (order) |
43
47
  | `cache_lookup → generate` | FAIL (missing `retrieve` via implied presence) |
44
48
 
45
49
  Low-level `createToolOrderingRule({ before, after })` alone may still pass when an endpoint is missing (compositional). TraceContract `requiredOrder` does not.
46
50
 
51
+ For overlapping first calls, omitted / `first-occurrence` warns while `happens-before` fails.
52
+
53
+ Immediate or positional `all-pairs` matching is not implemented.
54
+
47
55
  ### Experimental Vitest / Jest matchers (shipped)
48
56
 
49
57
  | Package | Export | Matchers |
@@ -55,7 +63,7 @@ These are **Experimental** — API names may evolve. There is no `expectTrace(..
55
63
 
56
64
  See [API.md](./API.md), [TRACE-FACTS.md](./TRACE-FACTS.md), and `packages/core/src/checks/contract.ts`.
57
65
 
58
- ## Rule kinds (shipped vs planned)
66
+ ## Rule kinds
59
67
 
60
68
  TraceContract rules fall into distinct categories. Mixing them incorrectly is a common source of false failures (see GitHub #308 and #309).
61
69
 
@@ -65,65 +73,68 @@ Unconditional path invariant: every named tool must appear **at least once** in
65
73
 
66
74
  - Use when the tool is always part of a valid execution path.
67
75
  - **Do not** use for steps that legitimate shortcuts may skip (for example cache hits that bypass `retrieve`).
68
- - When a shortcut is valid but you still need evidence of the outcome, prefer `observations.required` until `alternatives.anyOf` ships (6.20.0).
76
+ - Prefer `alternatives.anyOf` or `observations.required` when a shortcut is valid.
69
77
 
70
- ### `tools.requiredOrder` (shipped — first-occurrence / start-encounter)
78
+ ### `tools.requiredOrder` (shipped — selectable ordering modes)
71
79
 
72
- Legacy first-start / encounter ordering. The evaluator walks the trace and checks that each listed tool's **first occurrence** appears after the previous tool's first occurrence.
80
+ The evaluator expands each list into adjacent pairs and applies one `requiredOrderMode` to every pair.
73
81
 
74
82
  - TraceContract `requiredOrder` **implies presence** of every listed tool (unioned into `tools.required`).
75
- - Default mode is **first-occurrence** start/encounter order **not** causal happens-before.
76
- - Overlapping intervals that still satisfy start order emit a non-failing overlap warning.
77
- - **Planned (6.20.0, GitHub #308):**
78
- - `requiredOrderMode: "happens-before"` first before must **end** before first after **starts**
79
- - `requiredOrderMode: "all-occurrences"` — every before must end before every after starts
80
-
81
- ### `observations.required` (shipped)
82
-
83
- Requires externally observed or effect evidence (for example HTTP status, file write, cache key) rather than a specific tool call. Prefer this when the invariant is about **outcome** rather than **which tool ran**.
84
-
85
- ### Planned (6.20.0 — not shipped)
86
-
87
- Document only; **do not** use these fields in contracts today:
83
+ - `requiredOrderMode: "first-occurrence"` is the default first-occurrence start/encounter relation; overlapping intervals emit a non-failing warning.
84
+ - `requiredOrderMode: "happens-before"` requires the first before to **end** before the first after **starts**; overlap fails.
85
+ - `requiredOrderMode: "all-occurrences"` requires every before to end before every after starts; any cross-boundary overlap or later before fails.
86
+ - Missing interval boundaries fail closed in the two causal modes.
87
+
88
+ ### `alternatives.anyOf` (shipped)
89
+
90
+ One level of named deterministic branches. Base rules always apply. At least one complete branch must pass.
91
+
92
+ ```ts
93
+ defineTraceContract({
94
+ run: { requireCompleted: true },
95
+ tools: { required: ["generate"] },
96
+ alternatives: {
97
+ anyOf: [
98
+ {
99
+ id: "cache-hit",
100
+ contract: {
101
+ tools: { required: ["cache_lookup"], forbidden: ["retrieve"] },
102
+ observations: { required: ["cache-hit-valid"] },
103
+ },
104
+ },
105
+ {
106
+ id: "retrieve",
107
+ contract: {
108
+ tools: { required: ["retrieve"], requiredOrder: ["retrieve", "generate"] },
109
+ observations: { required: ["retrieval-context-valid"] },
110
+ },
111
+ },
112
+ ],
113
+ },
114
+ });
115
+ ```
88
116
 
89
- | Planned field | Purpose | GitHub |
90
- |---------------|---------|--------|
91
- | `alternatives.anyOf` | One of several deterministic valid paths (one level, no nested groups, no predicates) | #309 |
92
- | `requiredOrderMode: "happens-before"` | Causal completion-before-start ordering | #308 |
93
- | `requiredOrderMode: "all-occurrences"` | Strict ordering across all tool occurrences | #308 |
117
+ Constraints:
94
118
 
95
- API shape for both requires maintainer approval before external PR lands. @HsienW volunteered on #308 for `requiredOrderMode` implementation.
119
+ - unique branch ids
120
+ - no nested `alternatives`
121
+ - no predicates / runtime DSL
122
+ - unused failed branches do not fail the contract when another branch passes
123
+ - if none pass → `contract.alternatives.none-satisfied`
96
124
 
97
- ## Workaround until 6.20.0
125
+ ### `observations.required` (shipped)
98
126
 
99
- When a legitimate shortcut skips a tool you would otherwise require:
127
+ Requires externally observed or effect evidence (for example HTTP status, file write, cache key) rather than a specific tool call. Prefer this when the invariant is about **outcome** rather than **which tool ran**.
100
128
 
101
- 1. **Remove** unconditional `tools.required` for that step.
102
- 2. **Express** the verified outcome via `observations.required` when possible.
103
- 3. **Document** the cache-hit or alternate path in contract comments for reviewers.
129
+ ### Lint and explain (shipped)
104
130
 
105
- Example matching GitHub #309 (cache hit skips second `retrieve`):
131
+ ```ts
132
+ import { lintTraceContract, explainTraceContract } from "agent-inspect/checks";
106
133
 
107
- ```yaml
108
- contract:
109
- tools:
110
- required: [generate] # not retrieve — cache may skip it
111
- requiredOrder: [generate] # ordering only among tools that ran
112
- observations:
113
- required: [cache_hit_or_retrieve_evidence]
134
+ lintTraceContract(contract); // brittle / invalid shape diagnostics
135
+ explainTraceContract(contract); // human-readable intent lines
114
136
  ```
115
137
 
116
- With first-occurrence ordering, `retrieve → generate → retrieve` still **passes** when both retrieves are present (see worked example below).
117
-
118
- ## What is not shipped (yet)
119
-
120
- Do **not** document these as available:
121
-
122
- - `expectTrace(...).toSatisfyTraceContract` (different API shape than the shipped matchers)
123
- - Full workflow handoff / approval / MCP protocol contract rules
124
- - Per-tool argument schema / regex validators on the contract surface
125
- - Every structure rule (orphan/cycle/depth) exposed on the contract API (many exist as standalone check rules)
126
-
127
138
  ## CLI relationship
128
139
 
129
140
  ```bash
@@ -137,3 +148,4 @@ Suites and gates can consume check results; see [SUITES-COHORTS-GATES.md](./SUIT
137
148
  - Experimental/Beta API — may evolve in minors
138
149
  - Contract tests are smoke-level; prefer check-engine tests for deep rule coverage
139
150
  - Always review findings before treating a green check as product proof
151
+ - No nested alternatives, all-pairs matching, or general temporal DSL
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-inspect",
3
- "version": "6.19.0",
3
+ "version": "6.20.0",
4
4
  "license": "MIT",
5
5
  "type": "module",
6
6
  "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
@@ -243,7 +243,8 @@
243
243
  "public-truth:check": "node scripts/validate-public-truth.mjs",
244
244
  "ai-assets:check": "node scripts/validate-ai-assets.mjs",
245
245
  "package-readmes:check": "node scripts/validate-package-readmes.mjs",
246
- "docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check && pnpm run package-readmes:check && pnpm run repo:health && pnpm run build:cli && pnpm run demo:verify",
246
+ "actions:check": "node scripts/check-github-actions-pins.mjs",
247
+ "docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check && pnpm run package-readmes:check && pnpm run repo:health && pnpm run actions:check && pnpm run build:cli && pnpm run demo:verify",
247
248
  "build:cli": "pnpm exec tsup --config tsup.cli.config.ts",
248
249
  "repo:health": "node scripts/validate-repo-health.mjs",
249
250
  "demo:generate": "node scripts/demo-generate.mjs",
@@ -1714,9 +1714,63 @@ async function extractMetadata(filePath, _quickScan) {
1714
1714
  createdAt: stats.birthtime
1715
1715
  };
1716
1716
  }
1717
+ var ROOT_STEP_DEPTH = 0;
1718
+ var MAX_RUN_SUMMARY_DEPTH = 1e3;
1717
1719
  function isNonNegativeFiniteNumber(value) {
1718
1720
  return typeof value === "number" && Number.isFinite(value) && value >= 0;
1719
1721
  }
1722
+ function resolveParentStepId(step, steps) {
1723
+ const parentId = step.parentId;
1724
+ if (typeof parentId !== "string" || parentId.trim() === "") {
1725
+ return void 0;
1726
+ }
1727
+ return steps.has(parentId) ? parentId : void 0;
1728
+ }
1729
+ function computeStepDepth(stepId, steps, depthCache) {
1730
+ const cached = depthCache.get(stepId);
1731
+ if (cached !== void 0) return cached;
1732
+ const ancestry = [];
1733
+ const ancestryIndexes = /* @__PURE__ */ new Map();
1734
+ let currentStepId = stepId;
1735
+ let resolvedDepth;
1736
+ while (resolvedDepth === void 0) {
1737
+ const cachedDepth = depthCache.get(currentStepId);
1738
+ if (cachedDepth !== void 0) {
1739
+ resolvedDepth = cachedDepth;
1740
+ continue;
1741
+ }
1742
+ const cycleStart = ancestryIndexes.get(currentStepId);
1743
+ if (cycleStart !== void 0) {
1744
+ const cycleStepIds = ancestry.splice(cycleStart);
1745
+ for (const cycleStepId of cycleStepIds) {
1746
+ depthCache.set(cycleStepId, ROOT_STEP_DEPTH);
1747
+ }
1748
+ resolvedDepth = ROOT_STEP_DEPTH;
1749
+ continue;
1750
+ }
1751
+ const currentStep = steps.get(currentStepId);
1752
+ if (!currentStep) {
1753
+ resolvedDepth = ROOT_STEP_DEPTH;
1754
+ continue;
1755
+ }
1756
+ ancestryIndexes.set(currentStepId, ancestry.length);
1757
+ ancestry.push(currentStepId);
1758
+ const parentStepId = resolveParentStepId(currentStep, steps);
1759
+ if (parentStepId === void 0) {
1760
+ ancestry.pop();
1761
+ depthCache.set(currentStepId, ROOT_STEP_DEPTH);
1762
+ resolvedDepth = ROOT_STEP_DEPTH;
1763
+ continue;
1764
+ }
1765
+ currentStepId = parentStepId;
1766
+ }
1767
+ ancestry.reverse();
1768
+ for (const ancestorStepId of ancestry) {
1769
+ resolvedDepth = Math.min(MAX_RUN_SUMMARY_DEPTH, resolvedDepth + 1);
1770
+ depthCache.set(ancestorStepId, resolvedDepth);
1771
+ }
1772
+ return depthCache.get(stepId) ?? ROOT_STEP_DEPTH;
1773
+ }
1720
1774
  function buildRunSummary(events) {
1721
1775
  const started = events.find(
1722
1776
  (e) => e.event === "run_started"
@@ -1770,27 +1824,13 @@ function buildRunSummary(events) {
1770
1824
  let stepsWithKnownTotal = 0;
1771
1825
  let hasCachedTokens = false;
1772
1826
  const depthCache = /* @__PURE__ */ new Map();
1773
- const computeDepth = (stepId) => {
1774
- const cached = depthCache.get(stepId);
1775
- if (cached !== void 0) return cached;
1776
- const node = steps.get(stepId);
1777
- if (!node) return 0;
1778
- const parent = node.parentId;
1779
- if (typeof parent !== "string" || parent.trim() === "" || !steps.has(parent)) {
1780
- depthCache.set(stepId, 0);
1781
- return 0;
1782
- }
1783
- const d = Math.min(1e3, computeDepth(parent) + 1);
1784
- depthCache.set(stepId, d);
1785
- return d;
1786
- };
1787
1827
  for (const [id, s] of steps.entries()) {
1788
1828
  totalSteps += 1;
1789
1829
  if (s.type === "llm") llmSteps += 1;
1790
1830
  else if (s.type === "tool") toolSteps2 += 1;
1791
1831
  else logicSteps += 1;
1792
1832
  if (s.status === "error") errorSteps += 1;
1793
- const depth = computeDepth(id);
1833
+ const depth = computeStepDepth(id, steps, depthCache);
1794
1834
  if (depth > maxDepth) maxDepth = depth;
1795
1835
  if (typeof s.durationMs === "number" && Number.isFinite(s.durationMs)) {
1796
1836
  if (!longestStep || s.durationMs > longestStep.durationMs) {
@@ -11741,5 +11781,5 @@ function renderGateReport(result, options = {}) {
11741
11781
  }
11742
11782
 
11743
11783
  export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
11744
- //# sourceMappingURL=chunk-R7Y5SGH5.mjs.map
11745
- //# sourceMappingURL=chunk-R7Y5SGH5.mjs.map
11784
+ //# sourceMappingURL=chunk-QPRJUVLH.mjs.map
11785
+ //# sourceMappingURL=chunk-QPRJUVLH.mjs.map