agent-inspect 6.31.35 → 6.31.36

package/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 6.31.36
+
+### Patch Changes
+
+- 4e4d8dc: Fail closed when a token budget is set but completed LLM calls record no usage (#534/#535; thanks Rodrigo). Recipe INV-08 now cross-checks Bezalel remediation relationship endpoints (#537). Docs clarify manual `inspectRun`/`step` still omit `Error.code` (#541), standalone starter install on 6.31.35, and dated Windows packed MCP evidence (#526). Publish workflow splits verify (read-only) from OIDC publish (#536/#539).
+
 ## 6.31.35
 
 ### Patch Changes

package/README.md

@@ -213,7 +213,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
 
 ## Status and documentation
 
-**Current published baseline:** **6.31.35** · persisted schema `1.0` · Node.js `>=20` · MIT.
+**Current published baseline:** **6.31.36** · persisted schema `1.0` · Node.js `>=20` · MIT.
 
 Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
 

package/docs/API.md

@@ -71,7 +71,7 @@ import {
 } from "agent-inspect";
 ```
 
-- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output.
+- **`inspectRun(name, fn, options?)`**: wraps a workflow in a local JSONL trace (`run_started` / `run_completed`), prints terminal progress, and swallows instrumentation failures (user errors are re-thrown). **Traces by default** when `enabled` is omitted or `true`. Pass **`enabled: false`** to run `fn` with no trace file, no execution context, and no terminal output. On failure, persisted `error` is `{ message, stack? }` only — thrown `Error.code` / numeric `status` are **not** retained on this manual v0.1 path (use `createInspector()` when bounded `error.code` persistence matters; see §19 and [SCHEMA.md](./SCHEMA.md) §4).
   - **`redact`**: default `true` — redacts sensitive metadata keys before disk (`authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`). Pass `false` to persist metadata as-is. Pass `{ rules?: RedactionRule[] }` for custom rules (defaults still apply). **`redact: false` wins** over `redactionProfile` for trace writing.
   - **`redactionProfile`**: optional preset — `local` (default), `share`, or `strict`. Adds extra key-based redaction and tighter metadata string bounds for trace writing. Key-based only — not compliance-grade DLP.
   - **`maxMetadataValueLength`**: max string length for metadata values (default `2000`).
@@ -474,7 +474,7 @@ Public methods:
 
 `capture` is explicit and metadata-only. `onSuccess: "metadata-only"` records safe type/length/key-count summaries in `outputSummary`; `onError: "metadata-only"` records thrown-value type/name summaries. It does not store raw return values, prompts, outputs, or thrown objects. Use `"none"` to disable a capture side.
 
-Persisted step/run errors keep `name` and `message`. When present, a **bounded safe** `code` is also retained: finite numeric `code`/`status` values (for example HTTP `404`) or short strings matching letters, digits, underscore, hyphen, dot, and colon (max 128 characters). Response bodies, headers, multiline values, and arbitrary enumerable error properties are not persisted.
+Persisted step/run errors keep `name` and `message`. When present, a **bounded safe** `code` is also retained: finite numeric `code`/`status` values (for example HTTP `404` / `429`) or short strings matching letters, digits, underscore, hyphen, dot, and colon (max 128 characters, for example `ECONNRESET`). Response bodies, headers, multiline values, and arbitrary enumerable error properties are not persisted. This `code` retention applies to **`createInspector()`** schema 1.0 writers only — not to global `inspectRun` / `step` (those still omit codes; see [SCHEMA.md](./SCHEMA.md) §4).
 
 Pass structured tool arguments for contracts via step `metadata.arguments` / `metadata.input` / `metadata.toolArguments` (or top-level attributes with the same keys). TraceContract tool-argument checks read those locations; they do not enable default raw capture.
 

package/docs/CLI.md

@@ -332,6 +332,7 @@ Options:
 - `--max-duration-ms <number>`: add `run.duration`
 - `--required-tool <name>` / `--forbidden-tool <name>` / `--forbid-tool <name>` (alias): add `tool.usage`
 - `--allowed-model <model>` / `--max-total-tokens <number>`: add `llm.usage`
+  - A token limit fails when no token usage is read for a completed LLM call, because the limit cannot be verified. A failed LLM call does not need to record usage.
 - `--session <id>`: check all runs in a workflow session (uses `--dir`; target may be `.`)
 - `--group <id>`: check all runs sharing a `groupId` metadata value
 - `--correlate-group`: when using `--session`, also match synthetic `group:` session keys

package/docs/KNOWN-ISSUES.md

@@ -4,6 +4,7 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
 
 ## Contracts and safety
 
+- **Manual traces omit `Error.code`:** Global `inspectRun` / `step` still persist `{ message, stack? }` only. Bounded `error.code` retention shipped for `createInspector()` in 6.29.2 (#415); the issue title said “manual instrumentation,” but the manual helpers were not changed. Prefer `createInspector()` or explicit metadata `errorCode` when codes matter. Documented in [SCHEMA.md](./SCHEMA.md) §4 / [LIMITATIONS.md](./LIMITATIONS.md).
 - **Retry / side-effect rules (6.23+; chronology fixed in 6.25.1):** `requireIdempotencyEvidenceForRetry` applies to genuine retries including `error → success`, not only retries after a prior `ok`. Ambiguous attempt identity fails closed under that rule. Write timeouts remain **unknown completion** — safe write retry is not claimed.
 - **Omitted-payload digests:** inputs larger than 1 MiB are rejected after a length preflight; oversized rejection must not be treated as a free full-buffer copy path.
 

package/docs/LIMITATIONS.md

@@ -20,6 +20,7 @@ This document states what AgentInspect **does not** provide today. It complement
 ## Persisted event model
 
 - **Manual global tracing remains v0.1.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL for compatibility.
+- **Manual `error` omits codes.** Thrown `Error.code` values (for example `429`, `ECONNRESET`) and numeric `status` are **not** written on the manual path (`formatError` → `{ message, stack? }` only). `createInspector()` retains bounded safe codes on schema 1.0 rows. Issue #415 shipped the inspector path; the manual helpers are unchanged. See [SCHEMA.md](./SCHEMA.md) §4.
 - **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema 1.0 persisted rows; v0.1 and v0.2 remain readable.
 - **Migration is explicit, not automatic.** `agent-inspect migrate <input> --to 1.0 --dry-run` reports what would change, and `--output <file>` writes a separate file. AgentInspect does not rewrite old traces in place.
 

package/docs/SCHEMA.md

@@ -139,8 +139,17 @@ Records whether an external side effect matched expectations. Distinct from step
 
 ## 4. Error representation
 
-- Errors are structured as `{ message, stack? }`.
+Writer paths differ; this is intentional compatibility, **not** a claim that manual tracing preserves `Error.code`:
+
+| Writer path | Schema | Persisted `error` shape | `Error.code` / numeric `status` |
+| ----------- | ------ | ----------------------- | -------------------------------- |
+| Manual global helpers (`inspectRun` / `step` / `maybeInspectRun`) | `"0.1"` | `{ message, stack? }` (`ErrorInfo`) | **Omitted.** `formatError()` keeps message and optional stack only. |
+| `createInspector()` + built-in persisted writers | `"1.0"` (also readable as v0.2 shape) | `{ name?, message, code? }` (`PersistedInspectError`) | **Bounded safe codes retained** when present (finite numeric `code`/`status`, or short alphanumeric strings). See [API.md](./API.md) §19. |
+
+- Manual JSONL examples above use `{ message, stack? }` because that is what the v0.1 writer still emits.
+- Persisted schema 1.0 rows may include `error.code` (for example `"429"` or `"ECONNRESET"`). Migrating a manual v0.1 file to 1.0 does **not** invent codes that were never written.
 - AgentInspect prints short human errors by default; verbose modes may include stack strings.
+- Recovery / retry checks that need codes should prefer `createInspector()` traces, or place an explicit `errorCode` / `code` in start metadata when using the manual path (see recovery-operations notes in core).
 
 ## 5. Status representation
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "6.31.35",
+  "version": "6.31.36",
   "license": "MIT",
   "type": "module",
   "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",

package/packages/cli/dist/{chunk-RLFYFWIZ.mjs → chunk-Z3WXP44D.mjs}

@@ -9148,6 +9148,9 @@ function createToolFailureRule(options) {
     }
   };
 }
+function recordsTokenUsage(event) {
+  return Object.values(event.tokenUsage ?? {}).some((count) => typeof count === "number");
+}
 function createLlmUsageRule(options) {
   return {
     id: "llm.usage",
@@ -9205,6 +9208,21 @@ function createLlmUsageRule(options) {
         ["total", options.maxTotalTokens],
         ["cached", options.maxCachedTokens]
       ];
+      const hasTokenLimit = tokenLimits.some(([, limit]) => limit !== void 0);
+      const callsWithoutUsage = llms.filter(
+        (event) => event.status !== "error" && !recordsTokenUsage(event)
+      );
+      if (hasTokenLimit && callsWithoutUsage.length > 0) {
+        findings.push(
+          failFinding(
+            "llm.usage",
+            `No token usage was read for ${callsWithoutUsage.length} of ${llms.length} LLM call(s), so the token limit cannot be verified.`,
+            callsWithoutUsage.map((event) => eventEvidence3(event, "tokenUsage")),
+            "token usage on every completed LLM call",
+            { callsWithoutUsage: callsWithoutUsage.length, calls: llms.length }
+          )
+        );
+      }
       for (const [key, limit] of tokenLimits) {
         if (limit !== void 0 && tokenTotals[key] > limit) {
           findings.push(
@@ -16225,5 +16243,5 @@ function renderGateReport(result, options = {}) {
 }
 
 export { ATTRIBUTION_CONFIDENCES, COHORT_METRIC_IDS, DEFAULT_REDACT_KEYS, DEFAULT_SUITE_ARTIFACTS_DIR, DELIVERY_CONFLICT_ATTR, DELIVERY_EVENT_BASE_ATTR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, IDENTITY_CONFLICT_ATTR, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, defineTraceContract, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, evaluateTraceContract, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isAttributionConfidence, isCredentialSensitiveKey, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToRunTrees, persistedInspectEventsToTraceEvents, projectLogicalEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, stringContainsHighConfidenceCredential, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
-//# sourceMappingURL=chunk-RLFYFWIZ.mjs.map
-//# sourceMappingURL=chunk-RLFYFWIZ.mjs.map
+//# sourceMappingURL=chunk-Z3WXP44D.mjs.map
+//# sourceMappingURL=chunk-Z3WXP44D.mjs.map