agent-inspect 6.31.34 → 6.31.36

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 6.31.36
+
+### Patch Changes
+
+- 4e4d8dc: Fail closed when a token budget is set but completed LLM calls record no usage (#534/#535; thanks Rodrigo). Recipe INV-08 now cross-checks Bezalel remediation relationship endpoints (#537). Docs clarify manual `inspectRun`/`step` still omit `Error.code` (#541), standalone starter install on 6.31.35, and dated Windows packed MCP evidence (#526). Publish workflow splits verify (read-only) from OIDC publish (#536/#539).
+
+## 6.31.35
+
+### Patch Changes
+
+- 3cb8284: Bound and order tool-name diagnostics: cap codepoint annotations, skip pure non-Latin noise, keep bidi/control visible, and sort allowlist causes ahead of missing-tool consequences across rules (#522 follow-up). Add recipe-local refund ceiling/entity/approval (#527) and illustrative RETRIEVER→LLM OpenInference order examples via TraceCheckRule extension.
+
 ## 6.31.34
 
 ### Patch Changes

package/README.md

@@ -213,7 +213,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
 
 ## Status and documentation
 
-**Current published baseline:** **6.31.34** · persisted schema `1.0` · Node.js `>=20` · MIT.
+**Current published baseline:** **6.31.36** · persisted schema `1.0` · Node.js `>=20` · MIT.
 
 Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
 

package/docs/API.md

@@ -71,7 +71,7 @@ import {
 } from "agent-inspect";
 ```
 
-- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output.
+- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output. On failure, persisted `error` is `{ message, stack? }` only — thrown `Error.code` / numeric `status` are **not** retained on this manual v0.1 path (use `createInspector()` when bounded `error.code` persistence matters; see §19 and [SCHEMA.md](./SCHEMA.md) §4).
   - **`redact`**: default `true` — redacts sensitive metadata keys before disk (`authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`). Pass `false` to persist metadata as-is. Pass `{ rules?: RedactionRule[] }` for custom rules (defaults still apply). **`redact: false` wins** over `redactionProfile` for trace writing.
   - **`redactionProfile`**: optional preset — `local` (default), `share`, or `strict`. Adds extra key-based redaction and tighter metadata string bounds for trace writing. Key-based only — not compliance-grade DLP.
   - **`maxMetadataValueLength`**: max string length for metadata values (default `2000`).
@@ -474,7 +474,7 @@ Public methods:
 
 `capture` is explicit and metadata-only. `onSuccess: "metadata-only"` records safe type/length/key-count summaries in `outputSummary`; `onError: "metadata-only"` records thrown-value type/name summaries. It does not store raw return values, prompts, outputs, or thrown objects. Use `"none"` to disable a capture side.
 
-Persisted step/run errors keep `name` and `message`. When present, a **bounded safe** `code` is also retained: finite numeric `code`/`status` values (for example HTTP `404`) or short strings matching letters, digits, underscore, hyphen, dot, and colon (max 128 characters). Response bodies, headers, multiline values, and arbitrary enumerable error properties are not persisted.
+Persisted step/run errors keep `name` and `message`. When present, a **bounded safe** `code` is also retained: finite numeric `code`/`status` values (for example HTTP `404` / `429`) or short strings matching letters, digits, underscore, hyphen, dot, and colon (max 128 characters, for example `ECONNRESET`). Response bodies, headers, multiline values, and arbitrary enumerable error properties are not persisted. This `code` retention applies to **`createInspector()`** schema 1.0 writers only — not to global `inspectRun` / `step` (those still omit codes; see [SCHEMA.md](./SCHEMA.md) §4).
 
 Pass structured tool arguments for contracts via step `metadata.arguments` / `metadata.input` / `metadata.toolArguments` (or top-level attributes with the same keys). TraceContract tool-argument checks read those locations; they do not enable default raw capture.
 

package/docs/CLI.md

@@ -332,6 +332,7 @@ Options:
 - `--max-duration-ms <number>`: add `run.duration`
 - `--required-tool <name>` / `--forbidden-tool <name>` / `--forbid-tool <name>` (alias): add `tool.usage`
 - `--allowed-model <model>` / `--max-total-tokens <number>`: add `llm.usage`
+  - A token limit fails when no token usage is read for a completed LLM call, because the limit cannot be verified. A failed LLM call does not need to record usage.
 - `--session <id>`: check all runs in a workflow session (uses `--dir`; target may be `.`)
 - `--group <id>`: check all runs sharing a `groupId` metadata value
 - `--correlate-group`: when using `--session`, also match synthetic `group:` session keys

package/docs/KNOWN-ISSUES.md

@@ -4,6 +4,7 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
 
 ## Contracts and safety
 
+- **Manual traces omit `Error.code`:** Global `inspectRun` / `step` still persist `{ message, stack? }` only. Bounded `error.code` retention shipped for `createInspector()` in 6.29.2 (#415); the issue title said “manual instrumentation,” but the manual helpers were not changed. Prefer `createInspector()` or explicit metadata `errorCode` when codes matter. Documented in [SCHEMA.md](./SCHEMA.md) §4 / [LIMITATIONS.md](./LIMITATIONS.md).
 - **Retry / side-effect rules (6.23+; chronology fixed in 6.25.1):** `requireIdempotencyEvidenceForRetry` applies to genuine retries including `error → success`, not only retries after a prior `ok`. Ambiguous attempt identity fails closed under that rule. Write timeouts remain **unknown completion** — safe write retry is not claimed.
 - **Omitted-payload digests:** inputs larger than 1 MiB are rejected after a length preflight; oversized rejection must not be treated as a free full-buffer copy path.
 

package/docs/LIMITATIONS.md

@@ -20,6 +20,7 @@ This document states what AgentInspect **does not** provide today. It complement
 ## Persisted event model
 
 - **Manual global tracing remains v0.1.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL for compatibility.
+- **Manual `error` omits codes.** Thrown `Error.code` values (for example `429`, `ECONNRESET`) and numeric `status` are **not** written on the manual path (`formatError` → `{ message, stack? }` only). `createInspector()` retains bounded safe codes on schema 1.0 rows. Issue #415 shipped the inspector path; the manual helpers are unchanged. See [SCHEMA.md](./SCHEMA.md) §4.
 - **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema 1.0 persisted rows; v0.1 and v0.2 remain readable.
 - **Migration is explicit, not automatic.** `agent-inspect migrate <input> --to 1.0 --dry-run` reports what would change, and `--output <file>` writes a separate file. AgentInspect does not rewrite old traces in place.
 
@@ -98,4 +99,26 @@ Visual demos: [SCREENSHOTS.md](./SCREENSHOTS.md) · [CLI.md](./CLI.md)
 
 - Designed for **developer machines** and **inner-loop debugging**, not petabyte log warehouses.
 
+## Trust boundary (four independent questions)
+
+AgentInspect answers local evidence questions. Treat these as **separate**:
+
+1. **Capture / coverage** — What events were recorded under the declared instrumentation boundary? Missing spans, uninstrumented SDK calls, and effects outside that boundary are not inventable from silence.
+2. **Trajectory contract** — Did a TraceContract / check preset pass on the selected run(s)? Pass/fail is rule evaluation over captured evidence, not proof of business correctness.
+3. **Residual artifact safety** — Do written share/strict artifacts pass the residual safety policy (`verify-safe` / Evidence assessment)? This is best-effort local policy, not a compliance certification or whole-org DLP guarantee.
+4. **Independent authorization / effect / origin** — Is there separate evidence of lawful authority, successful side effect, and trusted producer origin? AgentInspect does not supply those by default.
+
+Implications:
+
+- A recomputed **unsigned** Evidence manifest can verify **internal consistency** (listed files + hashes). It does **not** prove authenticity or producer identity.
+- A **signature** (if supplied externally) can authenticate bytes to a trusted signer. It still does **not** prove completeness, lawful authority, successful business effect, or absence of an uninstrumented side effect.
+- A recorded **approval-before-action** event is evidence that an approval was *recorded*. It is **not** proof that the approver had authority.
+- Signed receipts or durable journals (parked designs) do **not** create exactly-once side effects.
+
+Hypothetical approval evidence fields for a future recipe (not a shipped schema): distinct `decisionId`, `policyVersion`, `actorAuthorityRef`, `actionDigest`, `targetScope`, `decisionTime`, `executionId`, and a downstream `receiptId`. These are a proposed evidence set for discussion, not a banking integration.
+
+Parked larger designs (coverage registry, cross-run linkage, durable intent journal, signing/sealing, general value comparisons, data-flow causal explanations) stay out of incidental patches — see [implementation/active/PARKED-DESIGNS.md](./implementation/active/PARKED-DESIGNS.md).
+
+Related: [EVIDENCE-FORMAT.md](./EVIDENCE-FORMAT.md), [ADR-0012](./decisions/ADR-0012-evidence-contract-binding.md), [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md), [NETWORK-BEHAVIOR.md](./NETWORK-BEHAVIOR.md), [NO-EGRESS-POLICY.md](./NO-EGRESS-POLICY.md).
+
 For roadmap intent, see [ROADMAP.md](../ROADMAP.md) (direction only — not a delivery guarantee).

package/docs/SAFE-TRACE-SHARING.md

@@ -4,7 +4,7 @@
 
 AgentInspect traces, log-ingest outputs, and exports are local files. They may still contain sensitive metadata that you attached manually, collected from logs, or included through optional preview settings. Use this checklist before sharing an artifact in a GitHub issue, Discussion, PR, support thread, or public post.
 
-This guide is practical sharing guidance, not a guarantee that any artifact is safe to publish. Redaction profiles are **best-effort transformation**, not compliance-grade DLP or a safety certification. Always finish with `verify-safe` before sharing.
+This guide is practical sharing guidance, not a guarantee that any artifact is safe to publish. Redaction profiles are **best-effort transformation**, not compliance-grade DLP or a safety certification. Always finish with `verify-safe` before sharing. Passing residual safety is still only one of the [four trust-boundary questions](./LIMITATIONS.md#trust-boundary-four-independent-questions) — it does not prove capture completeness, contract truth, or independent authorization.
 
 Built-in profiles redact high-confidence credential forms (provider keys, JWTs, bearer tokens, and bounded `token=` / `api_key=` / `internal_token=` style key/value secrets). Broad or context-sensitive findings—such as private filesystem paths—may still appear under `verify-safe` and as residual status after `redact`. Org-specific patterns can use a local CLI `--policy` JSON file (`extraKeys` plus bounded literal/prefix patterns; no user regex) or programmatic custom detectors.
 

package/docs/SCHEMA.md

@@ -139,8 +139,17 @@ Records whether an external side effect matched expectations. Distinct from step
 
 ## 4. Error representation
 
-- Errors are structured as `{ message, stack? }`.
+Writer paths differ; this is intentional compatibility, **not** a claim that manual tracing preserves `Error.code`:
+
+| Writer path | Schema | Persisted `error` shape | `Error.code` / numeric `status` |
+| ----------- | ------ | ----------------------- | -------------------------------- |
+| Manual global helpers (`inspectRun` / `step` / `maybeInspectRun`) | `"0.1"` | `{ message, stack? }` (`ErrorInfo`) | **Omitted.** `formatError()` keeps message and optional stack only. |
+| `createInspector()` + built-in persisted writers | `"1.0"` (also readable as v0.2 shape) | `{ name?, message, code? }` (`PersistedInspectError`) | **Bounded safe codes retained** when present (finite numeric `code`/`status`, or short alphanumeric strings). See [API.md](./API.md) §19. |
+
+- Manual JSONL examples above use `{ message, stack? }` because that is what the v0.1 writer still emits.
+- Persisted schema 1.0 rows may include `error.code` (for example `"429"` or `"ECONNRESET"`). Migrating a manual v0.1 file to 1.0 does **not** invent codes that were never written.
 - AgentInspect prints short human errors by default; verbose modes may include stack strings.
+- Recovery / retry checks that need codes should prefer `createInspector()` traces, or place an explicit `errorCode` / `code` in start metadata when using the manual path (see recovery-operations notes in core).
 
 ## 5. Status representation
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "6.31.34",
+  "version": "6.31.36",
   "license": "MIT",
   "type": "module",
   "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",

package/packages/cli/dist/{chunk-6TXW53UR.mjs → chunk-Z3WXP44D.mjs}

@@ -8011,11 +8011,27 @@ function eventTimestamp(finding, eventById) {
   const eventId = finding.evidence[0]?.eventId;
   return eventId ? eventById.get(eventId)?.timestamp ?? "" : "";
 }
+function isObservedToolPolicyFinding(finding) {
+  return finding.ruleId === "tool.usage" && finding.evidence[0]?.eventId !== void 0;
+}
+function isMissingToolConsequenceFinding(finding) {
+  if (finding.evidence[0]?.eventId !== void 0) return false;
+  if (finding.ruleId === "tool.usage" && /did not appear/.test(finding.message)) {
+    return true;
+  }
+  return finding.ruleId.startsWith("contract.tool.arguments") && /^No finished tool named /.test(finding.message);
+}
 function compareFindings(eventById) {
   return (a, b) => {
     if (SEVERITY_RANK[a.severity] !== SEVERITY_RANK[b.severity]) {
       return SEVERITY_RANK[a.severity] - SEVERITY_RANK[b.severity];
     }
+    const aPolicy = isObservedToolPolicyFinding(a);
+    const bPolicy = isObservedToolPolicyFinding(b);
+    const aMissing = isMissingToolConsequenceFinding(a);
+    const bMissing = isMissingToolConsequenceFinding(b);
+    if (aPolicy && bMissing) return -1;
+    if (bPolicy && aMissing) return 1;
     const byRule = a.ruleId.localeCompare(b.ruleId);
     if (byRule !== 0) return byRule;
     if (STATUS_RANK[a.status] !== STATUS_RANK[b.status]) {
@@ -8151,20 +8167,43 @@ function failFinding(ruleId, message, evidence, expected, actual, meta) {
 function toolName(event) {
   return resolveCanonicalToolName(event);
 }
+var MAX_TOOL_NAME_CODEPOINT_DETAILS = 3;
+function isAsciiLetterCodePoint(codePoint) {
+  return codePoint >= 65 && codePoint <= 90 || codePoint >= 97 && codePoint <= 122;
+}
+function isRiskyToolNameCodePoint(codePoint) {
+  if (codePoint <= 31 || codePoint >= 127 && codePoint <= 159) return true;
+  return codePoint === 1564 || codePoint === 8203 || codePoint === 8204 || codePoint === 8205 || codePoint === 8206 || codePoint === 8207 || codePoint >= 8234 && codePoint <= 8238 || codePoint >= 8294 && codePoint <= 8297 || codePoint === 65279;
+}
+function formatCodePointDetail(codePoint, index) {
+  return `U+${codePoint.toString(16).toUpperCase().padStart(4, "0")} at index ${index}`;
+}
 function formatToolNameForMessage(name) {
+  let hasAsciiLetter = false;
+  let hasNonAscii = false;
+  let hasRisky = false;
   const details = [];
   for (let index = 0; index < name.length; ) {
     const codePoint = name.codePointAt(index);
     const width = codePoint > 65535 ? 2 : 1;
+    if (isAsciiLetterCodePoint(codePoint)) hasAsciiLetter = true;
+    const risky = isRiskyToolNameCodePoint(codePoint);
+    if (risky) hasRisky = true;
     if (codePoint > 127) {
-      details.push(
-        `U+${codePoint.toString(16).toUpperCase().padStart(4, "0")} at index ${index}`
-      );
+      hasNonAscii = true;
+      details.push(formatCodePointDetail(codePoint, index));
+    } else if (risky) {
+      details.push(formatCodePointDetail(codePoint, index));
     }
     index += width;
   }
-  if (details.length === 0) return name;
-  return `"${name}" (contains ${details.join("; ")})`;
+  if (!hasRisky && !(hasAsciiLetter && hasNonAscii) || details.length === 0) {
+    return name;
+  }
+  const shown = details.slice(0, MAX_TOOL_NAME_CODEPOINT_DETAILS);
+  const remaining = details.length - shown.length;
+  const detailText = remaining > 0 ? `${shown.join("; ")}; and ${remaining} more` : shown.join("; ");
+  return `"${name}" (contains ${detailText})`;
 }
 function describeRunningEvent(event) {
   if (event.kind === "TOOL") {
@@ -9109,6 +9148,9 @@ function createToolFailureRule(options) {
     }
   };
 }
+function recordsTokenUsage(event) {
+  return Object.values(event.tokenUsage ?? {}).some((count) => typeof count === "number");
+}
 function createLlmUsageRule(options) {
   return {
     id: "llm.usage",
@@ -9166,6 +9208,21 @@ function createLlmUsageRule(options) {
         ["total", options.maxTotalTokens],
         ["cached", options.maxCachedTokens]
       ];
+      const hasTokenLimit = tokenLimits.some(([, limit]) => limit !== void 0);
+      const callsWithoutUsage = llms.filter(
+        (event) => event.status !== "error" && !recordsTokenUsage(event)
+      );
+      if (hasTokenLimit && callsWithoutUsage.length > 0) {
+        findings.push(
+          failFinding(
+            "llm.usage",
+            `No token usage was read for ${callsWithoutUsage.length} of ${llms.length} LLM call(s), so the token limit cannot be verified.`,
+            callsWithoutUsage.map((event) => eventEvidence3(event, "tokenUsage")),
+            "token usage on every completed LLM call",
+            { callsWithoutUsage: callsWithoutUsage.length, calls: llms.length }
+          )
+        );
+      }
       for (const [key, limit] of tokenLimits) {
         if (limit !== void 0 && tokenTotals[key] > limit) {
           findings.push(
@@ -16186,5 +16243,5 @@ function renderGateReport(result, options = {}) {
 }
 
 export { ATTRIBUTION_CONFIDENCES, COHORT_METRIC_IDS, DEFAULT_REDACT_KEYS, DEFAULT_SUITE_ARTIFACTS_DIR, DELIVERY_CONFLICT_ATTR, DELIVERY_EVENT_BASE_ATTR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, IDENTITY_CONFLICT_ATTR, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, defineTraceContract, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, evaluateTraceContract, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isAttributionConfidence, isCredentialSensitiveKey, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToRunTrees, persistedInspectEventsToTraceEvents, projectLogicalEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, stringContainsHighConfidenceCredential, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
-//# sourceMappingURL=chunk-6TXW53UR.mjs.map
-//# sourceMappingURL=chunk-6TXW53UR.mjs.map
+//# sourceMappingURL=chunk-Z3WXP44D.mjs.map
+//# sourceMappingURL=chunk-Z3WXP44D.mjs.map