agent-inspect 6.29.5 → 6.30.0

package/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 6.30.0
+
+### Minor Changes
+
+- 094854f: Strict CLI TraceContract JSON for `agent-inspect check --config`:
+
+  - Top-level `contract` evaluates through `defineTraceContract` / `evaluateTraceContract`.
+  - Strict nested validation for run/tools/llm/observations/scope/alternatives/controls/retry (unknown keys fail closed).
+  - Mutually exclusive with an effective top-level `checks` block.
+  - `--evidence-on` binds the resolved contract (`contract.resolved.json` + check-results digests).
+
+## 6.29.6
+
+### Patch Changes
+
+- 129f45a: Safe sharing and integration clarity for 6.29.6:
+
+  - Treat fully redacted raw-content fields as share-safe while retaining other safety checks (N-8).
+  - Clarify trajectory vs outcome CI gates and TOOL-only `requiredOrder` diagnostics in CLI help/docs.
+  - Recipe corrections: pairwise measured stage commitments for comparable-cohort-runs; browser observed-outcome identity/precondition/postcondition matrix.
+  - Jest reporter association remains explicit (`associations` / `resolveTrace`); private hotel capture (N-9) stays unverified without application checkouts.
+
 ## 6.29.5
 
 ### Patch Changes

package/README.md

@@ -212,7 +212,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
 
 ## Status and documentation
 
-**Current published baseline:** **6.29.5** · persisted schema `1.0` · Node.js `>=20` · MIT.
+**Current published baseline:** **6.30.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
 
 Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
 

package/docs/CLI.md

@@ -58,9 +58,9 @@ No new CLI commands were added for branching paths.
 | ---- | ------ |
 | One tool must always run | CLI `--required-tool` / contract `tools.required` |
 | Tool must never run | CLI `--forbidden-tool` / contract `tools.forbidden` |
-| Legitimate alternate paths (OR) | TraceContract `alternatives.anyOf` (not CLI flags) |
-| Causal order between tools | TraceContract `requiredOrder` + `requiredOrderMode` |
-| Read recovery vs write fail-closed | TraceContract `retry.operations` + `sideEffectClass` |
+| Legitimate alternate paths (OR) | TraceContract `alternatives.anyOf` via `--config` `contract` (or TS API) |
+| Causal order between tools | TraceContract `requiredOrder` + `requiredOrderMode` via `--config` `contract` |
+| Read recovery vs write fail-closed | TraceContract `retry.operations` + `sideEffectClass` via `--config` `contract` |
 | Sensitive expected literals in Evidence | Contract binding safety (`unavailable` on credential-like expected); never silent complete packaging |
 | Share-safe offline review package | CLI `bundle` / Evidence CI helpers |
 
@@ -337,7 +337,7 @@ Options:
 - `--correlate-group`: when using `--session`, also match synthetic `group:` session keys
 - `--guardrails <rule>`: optional deterministic guardrail rules (`banned-phrase`, `pii-leak`, `prompt-injection`, …); repeatable
 - `--circuit <rule>`: optional circuit analyzers (`same-tool-repetition`, `max-retries`, …); repeatable
-- `--fail-on-observation <status>`: add `outcome.status` rule; repeatable (`failed`, `passed`, `unknown`, `skipped`; default when flag present without value: `failed`)
+- `--fail-on-observation <status>`: **outcome gate** — adds `outcome.status` with `requireAny=true` (needs ≥1 OUTCOME event). Comma-separated statuses: `failed`, `passed`, `unknown`, `skipped`. Does **not** invent a passed outcome from tool/LLM/run success. For adapter structure without outcomes, use `--preset trajectory` (plus `--required-tool` etc.); a trajectory pass is not share safety (`verify-safe`).
 - `--preset <trajectory|safety|comprehensive|behavioral-session>`: additive check preset (does not change the default when omitted)
   - `behavioral-session` (6.26): require harness completion + score OUTCOME events (`--fail-on-observation failed` by default); does **not** treat every TOOL `error` as a failed run
   - `trajectory`: completion/structure/relationship focus; excludes share-safety findings
@@ -351,7 +351,8 @@ Options:
 
 By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available. Prefer `--preset trajectory` in CI, then `verify-safe` before sharing.
 
-Config files use this shape:
+Config files use either `checks` (flat rule options) or top-level `contract`
+(TraceContract vocabulary). Do not combine both in one file.
 
 ```json
 {
@@ -364,6 +365,27 @@ Config files use this shape:
 }
 ```
 
+Rich TraceContract JSON (6.30+):
+
+```json
+{
+  "contract": {
+    "tools": {
+      "required": ["retrieve_policy"],
+      "forbidden": ["send_email"],
+      "requiredOrder": ["retrieve_policy", "generate_answer"],
+      "requiredOrderMode": "first-occurrence"
+    },
+    "observations": { "failOn": ["failed"] },
+    "scope": { "runId": "optional-run-id" }
+  }
+}
+```
+
+Supported `contract` fields: `run`, `tools` (including `arguments`, `orderRules`),
+`llm`, `observations` (including `requireProvenance`), `scope`, `alternatives.anyOf`,
+`controls`, and `retry`. Unknown keys fail closed (exit 2). With `--evidence-on`,
+contract mode writes `contract.resolved.json` and binds digests into check-results.
 YAML is not supported. TypeScript config files (`.ts`, `.mts`, `.cts`) fail clearly unless a future explicit loader strategy is added; use precompiled JavaScript config instead.
 
 Examples:

package/docs/TRACE-CONTRACTS.md

@@ -25,6 +25,8 @@ Contracts compile to deterministic check rules for common cases:
 
 ## `tools.requiredOrder` semantics
 
+`requiredOrder` / `orderRules` match **TOOL** events only (via canonical tool names). LLM / LOGIC / other kinds with the same display name do not satisfy the order and are not relabeled as tools. When a required name exists only under another kind, `tool.usage` reports that kind in the finding message.
+
 `requiredOrder` is expanded into **adjacent pair** ordering rules with unique ids:
 
 ```text
@@ -42,15 +44,16 @@ Contracts compile to deterministic check rules for common cases:
 - `all-occurrences` requires every `before` occurrence to finish before every `after` occurrence starts (`max(before.end) <= min(after.start)`);
 - causal modes fail when a required interval boundary cannot be resolved instead of falling back to encounter order.
 
-Examples for `requiredOrder: ["retrieve", "generate"]`:
+Examples for `requiredOrder: ["retrieve_policy", "send_email"]` (both TOOL-typed):
 
 | Trajectory | Result |
 | --- | --- |
-| `retrieve → generate` | PASS |
-| `retrieve → rerank → generate` | PASS |
-| `retrieve → generate → retrieve` | PASS under omitted / `first-occurrence`; FAIL under `all-occurrences` |
-| `generate → retrieve` | FAIL (order) |
-| `cache_lookup → generate` | FAIL (missing `retrieve` via implied presence) |
+| `retrieve_policy → send_email` | PASS |
+| `retrieve_policy → rerank_docs → send_email` | PASS |
+| `retrieve_policy → send_email → retrieve_policy` | PASS under omitted / `first-occurrence`; FAIL under `all-occurrences` |
+| `send_email → retrieve_policy` | FAIL (order) |
+| `cache_lookup → send_email` | FAIL (missing `retrieve_policy` via implied presence) |
+| LLM named `generate` present, no TOOL `generate` | FAIL presence with non-TOOL kind diagnostic; not treated as a tool |
 
 Low-level `createToolOrderingRule({ before, after })` alone may still pass when an endpoint is missing (compositional). TraceContract `requiredOrder` does not.
 
@@ -111,7 +114,7 @@ defineTraceContract({
       {
         id: "retrieve",
         contract: {
-          tools: { required: ["retrieve"], requiredOrder: ["retrieve", "generate"] },
+          tools: { required: ["retrieve_policy"], requiredOrder: ["retrieve_policy", "send_email"] },
           observations: { required: ["retrieval-context-valid"] },
         },
       },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "6.29.5",
+  "version": "6.30.0",
   "license": "MIT",
   "type": "module",
   "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",