agent-inspect 6.29.3 → 6.29.4

package/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 6.29.4
+
+### Patch Changes
+
+- d7a3801: Trust and package integrity for 6.29.4:
+
+  - Advertise honest MCP `tools/list` annotations (readOnlyHint / openWorldHint / destructiveHint) without treating hints as authorization.
+  - Ship an exact LICENSE copy in every public fixed-group tarball and enforce it in pack-smoke / docs gates.
+  - Keep Evidence demo generator versions synced through `version:packages` (`demo:generate` + verify).
+  - Document run comparability and transport retry (429) evaluation guidance; AgentInspect evaluates retries, it does not execute them.
+
 ## 6.29.3
 
 ### Patch Changes

package/README.md

@@ -212,7 +212,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
 
 ## Status and documentation
 
-**Current published baseline:** **6.29.3** · persisted schema `1.0` · Node.js `>=20` · MIT.
+**Current published baseline:** **6.29.4** · persisted schema `1.0` · Node.js `>=20` · MIT.
 
 Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
 

package/docs/LIMITATIONS.md

@@ -10,6 +10,7 @@ This document states what AgentInspect **does not** provide today. It complement
 - **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
 - **No retry execution engine:** TraceContract `retry` rules (including additive `retry.operations[]` recovery oracles in 6.27) evaluate attempt identity and evidence only. AgentInspect does not retry tools, remediates nothing, and does not treat a client `idempotencyKey` as proof of exactly-once writes. Write-tool timeout/unknown completion is unevaluable without authoritative idempotency evidence; write-retry is not safe by default.
 - **Transport facts are source-boundary only:** HTTP status, server-advertised retry delay, selected delay, and their sources must be captured at the transport/fetch boundary. An MCP wrapper cannot recover facts already collapsed into an SDK error message.
+- **Run comparability is optional guidance:** Ordinary trajectory checks do not fail when cohort/prompt/tool commitments are absent. See [RUN-COMPARABILITY.md](./RUN-COMPARABILITY.md). A machine-readable comparable-run profile is not a default gate in this line.
 - **Omitted-payload digests** (`createOmittedPayloadCommitment`) are bounded (1 MiB preflight). Digests prove omitted bytes existed; they are not redaction or authorization.
 
 ## Correlation metadata (v1.3.0)

package/docs/RUN-COMPARABILITY.md

@@ -0,0 +1,54 @@
+# Run comparability (guidance)
+
+AgentInspect trajectory checks are deterministic **given the evidence they receive**. That does not mean two agent runs are automatically comparable. A changed path may reflect nondeterministic tools, sampling, prompt drift, or missing capture—not only an application regression.
+
+Ordinary `check` / TraceContract evaluation does **not** fail when comparability provenance is absent.
+
+## Classification
+
+| Class | Meaning |
+| --- | --- |
+| `comparable` | Enough shared identity and commitments to interpret path differences as meaningful |
+| `partially comparable` | Shared cohort/test case, but missing prompt/tools/model commitments |
+| `not comparable` | Identity conflicts or insufficient linkage |
+| `unknown` | No comparability metadata |
+
+## Optional provenance fields
+
+Attach as run or step metadata when you need honest before/after interpretation:
+
+- `cohortId`, `testCaseId`, `attempt`
+- `promptTemplateVersion` (or a content hash commitment)
+- per-LLM-step **input commitment** (hash; prefer not to store raw prompts)
+- upstream **tool-output commitment** (hash)
+- `availableToolsCommitment`
+- `requestedModel` / `resolvedModel`
+- SDK/framework version
+- sampling parameters (temperature, top_p, …)
+- system fingerprint / finish reason when the provider exposes them
+- contract digest
+- outcome evaluator method/version
+
+## Privacy
+
+- Hashes show equality, not content.
+- Low-entropy values may still be guessed.
+- Caller-owned HMAC is allowed; AgentInspect does not manage keys.
+- Cohort labels alone are insufficient.
+- Temperature `0` is not proof of determinism.
+- A digest does not prove complete capture.
+
+## Retry and transport identity
+
+Do not infer retries by matching tool names across steps. Prefer explicit:
+
+- `operationId`, `attemptId`, `attemptNumber`, `retryOf`
+- HTTP status, server-advertised delay, selected delay, delay source
+
+The MCP client wrapper records each call as a separate operation unless the application supplies that identity. AgentInspect **evaluates** retry evidence; it does not execute retries.
+
+See recipe `examples/recipes/comparable-cohort-runs/` and `examples/recipes/mcp-transport-retry-429/`.
+
+## Future profile
+
+A machine-readable comparable-run profile remains gated for `6.30.0` and requires external validation before it can become a default gate.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "6.29.3",
+  "version": "6.29.4",
   "license": "MIT",
   "type": "module",
   "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
@@ -152,6 +152,7 @@
     "docs/ADAPTERS.md",
     "docs/ADAPTER-CONFORMANCE.md",
     "docs/COMPARE.md",
+    "docs/RUN-COMPARABILITY.md",
     "docs/LOG-TO-TREE-QUICKSTART.md",
     "docs/SCREENSHOTS.md",
     "docs/SAFE-TRACE-SHARING.md",
@@ -235,7 +236,7 @@
     "test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run build && pnpm run test && pnpm run size",
     "prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run repo:health && pnpm run pack:smoke",
     "pack:dry-run": "pnpm run build && npm pack --dry-run",
-    "pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-openai-agents-e2e.mjs && node scripts/packed-ai-sdk-e2e.mjs && node scripts/packed-mcp-e2e.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs && node scripts/packed-swarm-loop-e2e.mjs && node scripts/evidence-ci-golden-paths.mjs",
+    "pack:smoke": "pnpm run package-licenses:check && pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-openai-agents-e2e.mjs && node scripts/packed-ai-sdk-e2e.mjs && node scripts/packed-mcp-e2e.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs && node scripts/packed-swarm-loop-e2e.mjs && node scripts/evidence-ci-golden-paths.mjs",
     "linked-versions:check": "node scripts/check-linked-versions.mjs",
     "docs:commands": "node scripts/validate-doc-commands.mjs",
     "docs:links": "node scripts/validate-doc-links.mjs",
@@ -244,7 +245,7 @@
     "ai-assets:check": "node scripts/validate-ai-assets.mjs",
     "package-readmes:check": "node scripts/validate-package-readmes.mjs",
     "actions:check": "node scripts/check-github-actions-pins.mjs",
-    "docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check && pnpm run package-readmes:check && pnpm run repo:health && pnpm run actions:check && pnpm run build:cli && pnpm run demo:verify",
+    "docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check && pnpm run package-readmes:check && pnpm run package-licenses:check && pnpm run build:cli && pnpm run demo-version-sync:check && pnpm run repo:health && pnpm run actions:check && pnpm run demo:verify",
     "build:cli": "pnpm exec tsup --config tsup.cli.config.ts",
     "repo:health": "node scripts/validate-repo-health.mjs",
     "demo:generate": "node scripts/demo-generate.mjs",
@@ -256,10 +257,13 @@
     "perf:baseline": "node scripts/performance-baseline.mjs",
     "examples:check": "pnpm install && pnpm --filter agent-inspect-example-01-basic run start",
     "changeset": "changeset",
-    "version:packages": "changeset version && pnpm public-truth:sync && pnpm public-truth:check",
+    "version:packages": "changeset version && pnpm public-truth:sync && pnpm build && pnpm demo:generate && pnpm public-truth:check && pnpm demo:verify && pnpm package-licenses:check",
     "release": "changeset publish",
     "website:dev": "pnpm --filter @agent-inspect/website dev",
     "website:build": "pnpm --filter @agent-inspect/website build",
-    "website:typecheck": "pnpm --filter @agent-inspect/website typecheck"
+    "website:typecheck": "pnpm --filter @agent-inspect/website typecheck",
+    "package-licenses:check": "node scripts/validate-package-licenses.mjs",
+    "package-licenses:sync": "node scripts/sync-package-licenses.mjs",
+    "demo-version-sync:check": "node scripts/check-demo-version-sync.mjs"
   }
 }

package/packages/cli/dist/{chunk-TBTRH7KY.mjs → chunk-FJRUJSST.mjs}

@@ -7206,8 +7206,9 @@ ${nav}
     <p>Artifact status: <span class="badge ${statusClass(input.assessmentStatus)}">${escapeHtml(input.assessmentStatus)}</span>
     ${input.sourceStatus !== void 0 ? ` \xB7 Source status: <span class="badge ${statusClass(input.sourceStatus)}">${escapeHtml(input.sourceStatus)}</span>` : ""}</p>
     <p>Profile: <code>${escapeHtml(input.redactionProfile)}</code> \xB7 Verification: <code>${escapeHtml(input.verificationPolicy)}</code></p>
-    ${input.contractBinding !== void 0 ? `<p>Contract binding: <code>${escapeHtml(input.contractBinding.status)}</code>${input.contractBinding.ruleCount !== void 0 ? ` \xB7 Rules: <code>${escapeHtml(String(input.contractBinding.ruleCount))}</code>` : ""}${input.contractBinding.sha256 !== void 0 ? ` \xB7 Digest: <code>${escapeHtml(input.contractBinding.sha256.slice(0, 16))}\u2026</code>` : ""}</p>${input.contractBinding.note !== void 0 && input.contractBinding.note.trim() !== "" ? `<p class="muted">${escapeHtml(input.contractBinding.note)}</p>` : ""}` : ""}
-    <p>Generator: <code>${escapeHtml(input.generatorName)}@${escapeHtml(input.generatorVersion)}</code>
+${input.contractBinding !== void 0 ? `    <p>Contract binding: <code>${escapeHtml(input.contractBinding.status)}</code>${input.contractBinding.ruleCount !== void 0 ? ` \xB7 Rules: <code>${escapeHtml(String(input.contractBinding.ruleCount))}</code>` : ""}${input.contractBinding.sha256 !== void 0 ? ` \xB7 Digest: <code>${escapeHtml(input.contractBinding.sha256.slice(0, 16))}\u2026</code>` : ""}</p>${input.contractBinding.note !== void 0 && input.contractBinding.note.trim() !== "" ? `
+    <p class="muted">${escapeHtml(input.contractBinding.note)}</p>` : ""}
+` : ""}    <p>Generator: <code>${escapeHtml(input.generatorName)}@${escapeHtml(input.generatorVersion)}</code>
     ${input.createdAt ? ` \xB7 Created: <code>${escapeHtml(input.createdAt)}</code>` : ""}</p>
     <h3>Runs</h3>
     <ul class="runs">${runList}</ul>
@@ -12488,5 +12489,5 @@ function renderGateReport(result, options = {}) {
 }
 
 export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
-//# sourceMappingURL=chunk-TBTRH7KY.mjs.map
-//# sourceMappingURL=chunk-TBTRH7KY.mjs.map
+//# sourceMappingURL=chunk-FJRUJSST.mjs.map
+//# sourceMappingURL=chunk-FJRUJSST.mjs.map