agent-inspect 6.19.1 → 6.21.0

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 6.21.0
+
+### Minor Changes
+
+- 9d6a24f: Actor-scoped TraceContracts (`scope` selectors) and structural observation provenance (`observations.requireProvenance`) for multi-agent precision (#320/#321). Includes planner/verifier recipe.
+
+## 6.20.0
+
+### Minor Changes
+
+- 43a4481: Flexible deterministic contracts: selectable `requiredOrderMode` (`first-occurrence` | `happens-before` | `all-occurrences`), one-level `alternatives.anyOf`, and `lintTraceContract` / `explainTraceContract` helpers. Includes MCP expected-rejection and local Promptfoo use-together recipes.
+
 ## 6.19.1
 
 ### Patch Changes

package/README.md

@@ -212,7 +212,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
 
 ## Status and documentation
 
-**Current published baseline:** **6.19.1** · persisted schema `1.0` · Node.js `>=20` · MIT.
+**Current published baseline:** **6.21.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
 
 Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
 

package/docs/TRACE-CONTRACTS.md

@@ -10,6 +10,11 @@ Contracts compile to deterministic check rules for common cases:
 
 - run status / completion / max duration
 - tool required / forbidden / allowed / maxCalls / order (`requiredTools` / `forbiddenTools` aliases)
+- selectable `requiredOrderMode` (`first-occurrence` | `happens-before` | `all-occurrences`)
+- `alternatives.anyOf` for one level of legitimate alternate paths
+- actor `scope` selectors (`runId`, `subAgentId`, `groupId`, `workflowStep`, `rootEventId`)
+- observation `requireProvenance` (structural method / evidence / same-run event references)
+- `lintTraceContract` / `explainTraceContract` for brittle-contract diagnostics
 - LLM maxCalls / maxTotalTokens / allowedModels
 - evidence-bearing findings on failures
 - evaluation over **logical** TraceFacts (raw events remain available)
@@ -24,13 +29,14 @@ Contracts compile to deterministic check rules for common cases:
 → contract.tool.order.1: B before C
 ```
 
-Each pair compares the **first occurrence** (start/encounter order in the evaluated event stream):
+`requiredOrderMode` selects one ordering relation for every generated pair:
 
 - unlisted intermediate tools are allowed;
-- later repetitions do not invalidate an earlier valid first-occurrence order;
 - TraceContract `requiredOrder` **implies presence** — every listed name is added to the effective required-tool set;
-- this is **not** causal happens-before; overlapping intervals emit a non-failing `tool.order.overlap` warning;
-- combine ordering with `maxCalls` or custom rules when repeated calls matter.
+- `first-occurrence` (default when omitted) compares first occurrences in start/encounter order; later repetitions do not invalidate an earlier valid order, and interval overlap emits a non-failing `tool.order.overlap` warning;
+- `happens-before` requires the first `before` occurrence to finish before the first `after` occurrence starts;
+- `all-occurrences` requires every `before` occurrence to finish before every `after` occurrence starts (`max(before.end) <= min(after.start)`);
+- causal modes fail when a required interval boundary cannot be resolved instead of falling back to encounter order.
 
 Examples for `requiredOrder: ["retrieve", "generate"]`:
 
@@ -38,12 +44,16 @@ Examples for `requiredOrder: ["retrieve", "generate"]`:
 | --- | --- |
 | `retrieve → generate` | PASS |
 | `retrieve → rerank → generate` | PASS |
-| `retrieve → generate → retrieve` | PASS (first-occurrence) |
+| `retrieve → generate → retrieve` | PASS under omitted / `first-occurrence`; FAIL under `all-occurrences` |
 | `generate → retrieve` | FAIL (order) |
 | `cache_lookup → generate` | FAIL (missing `retrieve` via implied presence) |
 
 Low-level `createToolOrderingRule({ before, after })` alone may still pass when an endpoint is missing (compositional). TraceContract `requiredOrder` does not.
 
+For overlapping first calls, omitted / `first-occurrence` warns while `happens-before` fails.
+
+Immediate or positional `all-pairs` matching is not implemented.
+
 ### Experimental Vitest / Jest matchers (shipped)
 
 | Package | Export | Matchers |
@@ -55,7 +65,7 @@ These are **Experimental** — API names may evolve. There is no `expectTrace(..
 
 See [API.md](./API.md), [TRACE-FACTS.md](./TRACE-FACTS.md), and `packages/core/src/checks/contract.ts`.
 
-## Rule kinds (shipped vs planned)
+## Rule kinds
 
 TraceContract rules fall into distinct categories. Mixing them incorrectly is a common source of false failures (see GitHub #308 and #309).
 
@@ -65,64 +75,104 @@ Unconditional path invariant: every named tool must appear **at least once** in
 
 - Use when the tool is always part of a valid execution path.
 - **Do not** use for steps that legitimate shortcuts may skip (for example cache hits that bypass `retrieve`).
-- When a shortcut is valid but you still need evidence of the outcome, prefer `observations.required` until `alternatives.anyOf` ships (6.20.0).
+- Prefer `alternatives.anyOf` or `observations.required` when a shortcut is valid.
 
-### `tools.requiredOrder` (shipped — first-occurrence / start-encounter)
+### `tools.requiredOrder` (shipped — selectable ordering modes)
 
-Legacy first-start / encounter ordering. The evaluator walks the trace and checks that each listed tool's **first occurrence** appears after the previous tool's first occurrence.
+The evaluator expands each list into adjacent pairs and applies one `requiredOrderMode` to every pair.
 
 - TraceContract `requiredOrder` **implies presence** of every listed tool (unioned into `tools.required`).
-- Default mode is **first-occurrence** start/encounter order — **not** causal happens-before.
-- Overlapping intervals that still satisfy start order emit a non-failing overlap warning.
-- **Planned (6.20.0, GitHub #308):**
-  - `requiredOrderMode: "happens-before"` — first before must **end** before first after **starts**
-  - `requiredOrderMode: "all-occurrences"` — every before must end before every after starts
+- `requiredOrderMode: "first-occurrence"` is the default first-occurrence start/encounter relation; overlapping intervals emit a non-failing warning.
+- `requiredOrderMode: "happens-before"` requires the first before to **end** before the first after **starts**; overlap fails.
+- `requiredOrderMode: "all-occurrences"` requires every before to end before every after starts; any cross-boundary overlap or later before fails.
+- Missing interval boundaries fail closed in the two causal modes.
+
+### `alternatives.anyOf` (shipped)
+
+One level of named deterministic branches. Base rules always apply. At least one complete branch must pass.
+
+```ts
+defineTraceContract({
+  run: { requireCompleted: true },
+  tools: { required: ["generate"] },
+  alternatives: {
+    anyOf: [
+      {
+        id: "cache-hit",
+        contract: {
+          tools: { required: ["cache_lookup"], forbidden: ["retrieve"] },
+          observations: { required: ["cache-hit-valid"] },
+        },
+      },
+      {
+        id: "retrieve",
+        contract: {
+          tools: { required: ["retrieve"], requiredOrder: ["retrieve", "generate"] },
+          observations: { required: ["retrieval-context-valid"] },
+        },
+      },
+    ],
+  },
+});
+```
+
+Constraints:
+
+- unique branch ids
+- no nested `alternatives`
+- no predicates / runtime DSL
+- unused failed branches do not fail the contract when another branch passes
+- if none pass → `contract.alternatives.none-satisfied`
 
 ### `observations.required` (shipped)
 
 Requires externally observed or effect evidence (for example HTTP status, file write, cache key) rather than a specific tool call. Prefer this when the invariant is about **outcome** rather than **which tool ran**.
 
-### Planned (6.20.0 — not shipped)
-
-Document only; **do not** use these fields in contracts today:
+### `scope` (shipped — experimental)
 
-| Planned field | Purpose | GitHub |
-|---------------|---------|--------|
-| `alternatives.anyOf` | One of several deterministic valid paths (one level, no nested groups, no predicates) | #309 |
-| `requiredOrderMode: "happens-before"` | Causal completion-before-start ordering | #308 |
-| `requiredOrderMode: "all-occurrences"` | Strict ordering across all tool occurrences | #308 |
+Select one actor before evaluation using **explicit** metadata only:
 
-API shape for both requires maintainer approval before external PR lands. @HsienW volunteered on #308 for `requiredOrderMode` implementation.
+```ts
+defineTraceContract({
+  scope: { subAgentId: "verifier-agent" },
+  tools: { required: ["run_tests"] },
+});
+```
 
-## Workaround until 6.20.0
+Supported selectors: `runId`, `subAgentId`, `groupId`, `workflowStep`, `rootEventId` (subtree projection).
 
-When a legitimate shortcut skips a tool you would otherwise require:
+- zero matches → error (no whole-session fallback)
+- singular selector matching multiple runs → error
+- no timestamp, prose, or display-name inference
+- successful selection reports the actor and evidence event count
 
-1. **Remove** unconditional `tools.required` for that step.
-2. **Express** the verified outcome via `observations.required` when possible.
-3. **Document** the cache-hit or alternate path in contract comments for reviewers.
+### `observations.requireProvenance` (shipped — experimental)
 
-Example matching GitHub #309 (cache hit skips second `retrieve`):
+Structural provenance for named outcomes. These checks prove method/evidence linkage was recorded; they do **not** prove the claim is semantically true, authorized, complete, or externally trusted.
 
-```yaml
-contract:
-  tools:
-    required: [generate] # not retrieve — cache may skip it
-    requiredOrder: [generate] # ordering only among tools that ran
-  observations:
-    required: [cache_hit_or_retrieve_evidence]
+```ts
+defineTraceContract({
+  observations: {
+    required: ["refund-confirmed"],
+    requireProvenance: {
+      method: true,
+      evidence: true,
+      sameRunEventReference: true,
+    },
+  },
+});
 ```
 
-With first-occurrence ordering, `retrieve → generate → retrieve` still **passes** when both retrieves are present (see worked example below).
+Bounded evidence shapes: string event id, `{ eventId }`, or `{ eventIds }` (max 16). Method must be in the `ObservedOutcomeMethod` vocabulary. Omitting `requireProvenance` leaves prior observation behavior unchanged.
 
-## What is not shipped (yet)
+### Lint and explain (shipped)
 
-Do **not** document these as available:
+```ts
+import { lintTraceContract, explainTraceContract } from "agent-inspect/checks";
 
-- `expectTrace(...).toSatisfyTraceContract` (different API shape than the shipped matchers)
-- Full workflow handoff / approval / MCP protocol contract rules
-- Per-tool argument schema / regex validators on the contract surface
-- Every structure rule (orphan/cycle/depth) exposed on the contract API (many exist as standalone check rules)
+lintTraceContract(contract);   // brittle / invalid shape diagnostics
+explainTraceContract(contract); // human-readable intent lines
+```
 
 ## CLI relationship
 
@@ -137,3 +187,4 @@ Suites and gates can consume check results; see [SUITES-COHORTS-GATES.md](./SUIT
 - Experimental/Beta API — may evolve in minors
 - Contract tests are smoke-level; prefer check-engine tests for deep rule coverage
 - Always review findings before treating a green check as product proof
+- No nested alternatives, all-pairs matching, or general temporal DSL

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "6.19.1",
+  "version": "6.21.0",
   "license": "MIT",
   "type": "module",
   "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",

package/packages/cli/dist/{chunk-QDYVK4MF.mjs → chunk-CRUHIL5J.mjs}

@@ -2579,6 +2579,17 @@ var OBSERVED_OUTCOME_STATUSES = [
   "unknown",
   "skipped"
 ];
+var OBSERVED_OUTCOME_METHODS = [
+  "dom",
+  "accessibility",
+  "snapshot",
+  "network",
+  "storage",
+  "filesystem",
+  "database",
+  "queue",
+  "custom"
+];
 var OUTCOME_ATTRIBUTE_STATUS_KEY = "outcomeStatus";
 var OUTCOME_ATTRIBUTE_EXPECTATION_KEY = "expectation";
 var OUTCOME_ATTRIBUTE_METHOD_KEY = "method";
@@ -6681,6 +6692,9 @@ function isCredentialSensitiveKey(key, sensitiveKeys = DEFAULT_CREDENTIAL_SENSIT
   return false;
 }
 
+// packages/core/src/checks/contract.ts
+new Set(OBSERVED_OUTCOME_METHODS);
+
 // packages/core/src/checks/index.ts
 var SEVERITY_RANK = {
   error: 0,
@@ -11781,5 +11795,5 @@ function renderGateReport(result, options = {}) {
 }
 
 export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
-//# sourceMappingURL=chunk-QDYVK4MF.mjs.map
-//# sourceMappingURL=chunk-QDYVK4MF.mjs.map
+//# sourceMappingURL=chunk-CRUHIL5J.mjs.map
+//# sourceMappingURL=chunk-CRUHIL5J.mjs.map