agent-inspect 6.31.31 → 6.31.33
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +1 -1
- package/docs/CLI.md +3 -1
- package/docs/TRACE-CONTRACTS.md +4 -0
- package/package.json +1 -1
- package/packages/cli/dist/{chunk-4GTOJ3G2.mjs → chunk-AILSHDEF.mjs} +191 -31
- package/packages/cli/dist/chunk-AILSHDEF.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +209 -33
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +23 -7
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-BF74ZVDF.mjs → src-SP36M4VB.mjs} +3 -3
- package/packages/cli/dist/{src-BF74ZVDF.mjs.map → src-SP36M4VB.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +80 -18
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +2 -2
- package/packages/core/dist/advanced.d.ts +2 -2
- package/packages/core/dist/advanced.mjs +9 -9
- package/packages/core/dist/checks.cjs +185 -29
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.d.cts +2 -2
- package/packages/core/dist/checks.d.ts +2 -2
- package/packages/core/dist/checks.mjs +3 -3
- package/packages/core/dist/{chunk-76QIWWJ5.mjs → chunk-6S666BF4.mjs} +3 -3
- package/packages/core/dist/{chunk-76QIWWJ5.mjs.map → chunk-6S666BF4.mjs.map} +1 -1
- package/packages/core/dist/{chunk-BYNAJPNH.mjs → chunk-EMDJLFRK.mjs} +188 -32
- package/packages/core/dist/chunk-EMDJLFRK.mjs.map +1 -0
- package/packages/core/dist/{chunk-QMO3ITYF.mjs → chunk-FBPDF7HO.mjs} +3 -3
- package/packages/core/dist/{chunk-QMO3ITYF.mjs.map → chunk-FBPDF7HO.mjs.map} +1 -1
- package/packages/core/dist/{chunk-XM27TZLD.mjs → chunk-RB6WANSW.mjs} +6 -2
- package/packages/core/dist/chunk-RB6WANSW.mjs.map +1 -0
- package/packages/core/dist/{chunk-JTJN4KBJ.mjs → chunk-VIEJJ5JB.mjs} +4 -4
- package/packages/core/dist/{chunk-JTJN4KBJ.mjs.map → chunk-VIEJJ5JB.mjs.map} +1 -1
- package/packages/core/dist/exporters.cjs +4 -0
- package/packages/core/dist/exporters.cjs.map +1 -1
- package/packages/core/dist/exporters.mjs +3 -3
- package/packages/core/dist/{index-DACU4Dy1.d.cts → index-Ceyw5x7b.d.cts} +4 -2
- package/packages/core/dist/{index-b4tIGrzq.d.ts → index-K5Cinf1r.d.ts} +4 -2
- package/packages/core/dist/index.cjs +4 -0
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.mjs +4 -4
- package/packages/core/dist/writers.cjs +4 -0
- package/packages/core/dist/writers.cjs.map +1 -1
- package/packages/core/dist/writers.mjs +1 -1
- package/packages/cli/dist/chunk-4GTOJ3G2.mjs.map +0 -1
- package/packages/core/dist/chunk-BYNAJPNH.mjs.map +0 -1
- package/packages/core/dist/chunk-XM27TZLD.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.31.33
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- b2adf03: Fail requiredOrder when a before tool starts but never finishes while after runs (#521). Escape mixed-script lookalike tool names in tool.usage findings and sort allowlist causes first (#522).
|
|
8
|
+
|
|
9
|
+
## 6.31.32
|
|
10
|
+
|
|
11
|
+
### Patch Changes
|
|
12
|
+
|
|
13
|
+
- d9ab3e9: Break causal ordering timestamp ties by recorded row order (#518 / #517). Detect Stripe `sk_live_`/`rk_live_`/`sk_test_` and Slack `xox…` values in redact, pre-disk free-text, and verify-safe (#520). Set `check --evidence-on` Evidence `assessment.status` from residual verify-safe, not check pass/fail (#519). Unlink starter control traces so prove-same-output-wrong-path is idempotent (#523).
|
|
14
|
+
|
|
3
15
|
## 6.31.31
|
|
4
16
|
|
|
5
17
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -213,7 +213,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
|
|
|
213
213
|
|
|
214
214
|
## Status and documentation
|
|
215
215
|
|
|
216
|
-
**Current published baseline:** **6.31.
|
|
216
|
+
**Current published baseline:** **6.31.33** · persisted schema `1.0` · Node.js `>=20` · MIT.
|
|
217
217
|
|
|
218
218
|
Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
|
|
219
219
|
|
package/docs/CLI.md
CHANGED
|
@@ -349,6 +349,8 @@ Options:
|
|
|
349
349
|
- `--evidence-profile <local|share|strict>`: redaction profile for Evidence (default `share`)
|
|
350
350
|
- `--evidence-format <directory|html|zip>`: Evidence layout (default `directory`)
|
|
351
351
|
|
|
352
|
+
Evidence `assessment.status` is residual `verify-safe` of the redacted artifact (same sense as `bundle`), not the trajectory check pass/fail. Check results stay in `check-results.json` / the summary text.
|
|
353
|
+
|
|
352
354
|
By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available. Prefer `--preset trajectory` in CI, then `verify-safe` before sharing.
|
|
353
355
|
|
|
354
356
|
Config files use either `checks` (flat rule options) or top-level `contract`
|
|
@@ -1097,7 +1099,7 @@ Options:
|
|
|
1097
1099
|
- `--format <format>` — `markdown`, `json`, `json-compact`, `html`, `junit`, `github`, or `github-annotations` (default: `markdown`)
|
|
1098
1100
|
- `-o, --output <dir>` — write `gate-results.json`, `gate-summary.md`, `gate-report.html`, `junit.xml`, `github-step-summary.md`, `github-annotations.txt`
|
|
1099
1101
|
- `--json` — print deterministic JSON result (pretty); use `--format json-compact` for CI logs
|
|
1100
|
-
- `--evidence-on <fail|always|never>` — write local Evidence v2 (no upload); omitted = never
|
|
1102
|
+
- `--evidence-on <fail|always|never>` — write local Evidence v2 (no upload); omitted = never. `assessment.status` is residual verify-safe of the redacted artifact, not gate pass/fail.
|
|
1101
1103
|
- `--evidence-dir <path>` — Evidence output directory or base path
|
|
1102
1104
|
- `--evidence-profile <local|share|strict>` — redaction profile (default `share`)
|
|
1103
1105
|
- `--evidence-format <directory|html|zip>` — Evidence layout (default `directory`)
|
package/docs/TRACE-CONTRACTS.md
CHANGED
|
@@ -44,6 +44,8 @@ Contracts compile to deterministic check rules for common cases:
|
|
|
44
44
|
- `happens-before` requires the first `before` occurrence to finish before the first `after` occurrence starts;
|
|
45
45
|
- `all-occurrences` requires every `before` occurrence to finish before every `after` occurrence starts (`max(before.end) <= min(after.start)`);
|
|
46
46
|
- causal modes fail when a required interval boundary cannot be resolved instead of falling back to encounter order.
|
|
47
|
+
- when the boundary timestamps tie at millisecond resolution (for example scripted or mocked tools that start and finish inside one millisecond), causal modes decide by recorded event order: the before event must have been recorded as finished before the after event was recorded as started. Such findings carry `resolvedBy: "recorded-order"`.
|
|
48
|
+
- a `before` tool that starts but never finishes (`status: "running"`) while a later `after` tool finishes fails the order rule in every mode with `tool.order.interval-unresolved` (unfinished before), naming the abandoned tool — not only the generic incomplete-running `run.status` finding.
|
|
47
49
|
|
|
48
50
|
Examples for `requiredOrder: ["retrieve_policy", "send_email"]` (both TOOL-typed):
|
|
49
51
|
|
|
@@ -118,6 +120,8 @@ The evaluator expands each list into adjacent pairs and applies one `requiredOrd
|
|
|
118
120
|
- `requiredOrderMode: "happens-before"` requires the first before to **end** before the first after **starts**; overlap fails.
|
|
119
121
|
- `requiredOrderMode: "all-occurrences"` requires every before to end before every after starts; any cross-boundary overlap or later before fails.
|
|
120
122
|
- Missing interval boundaries fail closed in the two causal modes.
|
|
123
|
+
- Millisecond timestamp ties are decided by recorded event order in the two causal modes.
|
|
124
|
+
- A started-but-never-finished `before` tool (running, no completion) fails order in every mode when `after` has finished, with `tool.order.interval-unresolved`.
|
|
121
125
|
|
|
122
126
|
### `alternatives.anyOf` (shipped)
|
|
123
127
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.31.
|
|
3
|
+
"version": "6.31.33",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
|
|
@@ -937,6 +937,10 @@ var HIGH_CONFIDENCE_CREDENTIAL_PATTERNS = [
|
|
|
937
937
|
/\b[a-z0-9_.-]+=[^;\s]+(?:;\s*[a-z0-9_.-]+=[^;\s]+)+/i,
|
|
938
938
|
/\beyJ[a-zA-Z0-9_-]{8,}\.[a-zA-Z0-9_-]{8,}\.[a-zA-Z0-9_-]{8,}\b/,
|
|
939
939
|
/\b(?:sk-(?:proj-)?[a-zA-Z0-9_-]{16,}|sk-ant-[a-zA-Z0-9_-]{16,}|AIza[0-9A-Za-z_-]{20,})\b/,
|
|
940
|
+
// Stripe secret / restricted keys use underscores (not sk-). Keep in sync with
|
|
941
|
+
// packages/redact/src/index.ts + packages/core/src/checks DEFAULT_SECRET_PATTERNS (#520).
|
|
942
|
+
/\b(?:sk|rk)_(?:live|test)_[A-Za-z0-9]{16,}\b/,
|
|
943
|
+
/\bxox[abposr]-[A-Za-z0-9-]{10,}\b/,
|
|
940
944
|
/\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{20,}\b/,
|
|
941
945
|
/\b(?:AKIA|ASIA)[A-Z0-9]{16}\b/,
|
|
942
946
|
/-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*-----END [A-Z ]*PRIVATE KEY-----/,
|
|
@@ -7047,10 +7051,17 @@ function contractToRules(contract) {
|
|
|
7047
7051
|
(event) => event.status === "running"
|
|
7048
7052
|
);
|
|
7049
7053
|
if (running.length > 0) {
|
|
7054
|
+
const labels = running.map((event) => {
|
|
7055
|
+
if (event.kind === "TOOL") {
|
|
7056
|
+
return `tool ${resolveCanonicalToolName(event)}`;
|
|
7057
|
+
}
|
|
7058
|
+
return `${event.kind} ${event.name}`;
|
|
7059
|
+
});
|
|
7060
|
+
const message = labels.length === 1 ? `Run contains incomplete running events: ${labels[0]} never completed.` : `Run contains incomplete running events: ${labels.join("; ")}.`;
|
|
7050
7061
|
findings.push(
|
|
7051
7062
|
contractFailFinding(
|
|
7052
7063
|
"contract.run.allowedStatuses",
|
|
7053
|
-
|
|
7064
|
+
message,
|
|
7054
7065
|
running.map((event) => ({
|
|
7055
7066
|
runId: event.runId,
|
|
7056
7067
|
eventId: event.eventId,
|
|
@@ -7059,7 +7070,10 @@ function contractToRules(contract) {
|
|
|
7059
7070
|
status: event.status
|
|
7060
7071
|
})),
|
|
7061
7072
|
"no running events",
|
|
7062
|
-
|
|
7073
|
+
{
|
|
7074
|
+
count: running.length,
|
|
7075
|
+
unfinished: labels
|
|
7076
|
+
}
|
|
7063
7077
|
)
|
|
7064
7078
|
);
|
|
7065
7079
|
}
|
|
@@ -7818,6 +7832,12 @@ var SAFE_USAGE_LEAF_KEYS = /* @__PURE__ */ new Set([
|
|
|
7818
7832
|
var DEFAULT_SECRET_PATTERNS = [
|
|
7819
7833
|
{ id: "bearer-token", pattern: /Bearer\s+[A-Za-z0-9._~+/-]{12,}=*/ },
|
|
7820
7834
|
{ id: "openai-key", pattern: /sk-[A-Za-z0-9_-]{16,}/ },
|
|
7835
|
+
// Stripe + Slack: keep in sync with packages/redact + credential-value-patterns (#520).
|
|
7836
|
+
{
|
|
7837
|
+
id: "stripe-key",
|
|
7838
|
+
pattern: /\b(?:sk|rk)_(?:live|test)_[A-Za-z0-9]{16,}\b/
|
|
7839
|
+
},
|
|
7840
|
+
{ id: "slack-token", pattern: /\bxox[abposr]-[A-Za-z0-9-]{10,}\b/ },
|
|
7821
7841
|
{ id: "aws-access-key", pattern: /AKIA[0-9A-Z]{16}/ },
|
|
7822
7842
|
{ id: "github-token", pattern: /gh[opsu]_[A-Za-z0-9_]{20,}/ },
|
|
7823
7843
|
// Keep in sync with packages/redact/src/key-value-secret.ts (KEY_VALUE_SECRET_PATTERN_SOURCE).
|
|
@@ -7999,6 +8019,11 @@ function compareFindings(eventById) {
|
|
|
7999
8019
|
if (STATUS_RANK[a.status] !== STATUS_RANK[b.status]) {
|
|
8000
8020
|
return STATUS_RANK[a.status] - STATUS_RANK[b.status];
|
|
8001
8021
|
}
|
|
8022
|
+
if (a.ruleId === "tool.usage") {
|
|
8023
|
+
const aHasEvent = a.evidence[0]?.eventId !== void 0;
|
|
8024
|
+
const bHasEvent = b.evidence[0]?.eventId !== void 0;
|
|
8025
|
+
if (aHasEvent !== bHasEvent) return aHasEvent ? -1 : 1;
|
|
8026
|
+
}
|
|
8002
8027
|
const byRun = compareStrings(a.evidence[0]?.runId, b.evidence[0]?.runId);
|
|
8003
8028
|
if (byRun !== 0) return byRun;
|
|
8004
8029
|
const byTime = eventTimestamp(a, eventById).localeCompare(eventTimestamp(b, eventById));
|
|
@@ -8124,6 +8149,37 @@ function failFinding(ruleId, message, evidence, expected, actual, meta) {
|
|
|
8124
8149
|
function toolName(event) {
|
|
8125
8150
|
return resolveCanonicalToolName(event);
|
|
8126
8151
|
}
|
|
8152
|
+
function formatToolNameForMessage(name) {
|
|
8153
|
+
const details = [];
|
|
8154
|
+
for (let index = 0; index < name.length; ) {
|
|
8155
|
+
const codePoint = name.codePointAt(index);
|
|
8156
|
+
const width = codePoint > 65535 ? 2 : 1;
|
|
8157
|
+
if (codePoint > 127) {
|
|
8158
|
+
details.push(
|
|
8159
|
+
`U+${codePoint.toString(16).toUpperCase().padStart(4, "0")} at index ${index}`
|
|
8160
|
+
);
|
|
8161
|
+
}
|
|
8162
|
+
index += width;
|
|
8163
|
+
}
|
|
8164
|
+
if (details.length === 0) return name;
|
|
8165
|
+
return `"${name}" (contains ${details.join("; ")})`;
|
|
8166
|
+
}
|
|
8167
|
+
function describeRunningEvent(event) {
|
|
8168
|
+
if (event.kind === "TOOL") {
|
|
8169
|
+
return `tool ${formatToolNameForMessage(resolveCanonicalToolName(event))}`;
|
|
8170
|
+
}
|
|
8171
|
+
if (event.kind === "LLM") {
|
|
8172
|
+
return `llm ${stepEndpointName(event, "LLM")}`;
|
|
8173
|
+
}
|
|
8174
|
+
return `${event.kind} ${event.name}`;
|
|
8175
|
+
}
|
|
8176
|
+
function incompleteRunningMessage(prefix, running) {
|
|
8177
|
+
const labels = running.map(describeRunningEvent);
|
|
8178
|
+
if (labels.length === 1) {
|
|
8179
|
+
return `${prefix}: ${labels[0]} never completed.`;
|
|
8180
|
+
}
|
|
8181
|
+
return `${prefix}: ${labels.join("; ")}.`;
|
|
8182
|
+
}
|
|
8127
8183
|
function stepEndpointName(event, kind) {
|
|
8128
8184
|
if (kind === "TOOL") return resolveCanonicalToolName(event);
|
|
8129
8185
|
for (const prefix of ["llm:", "generation:", "transcription:", "speech:"]) {
|
|
@@ -8198,6 +8254,56 @@ function eventEndMs(event) {
|
|
|
8198
8254
|
}
|
|
8199
8255
|
return void 0;
|
|
8200
8256
|
}
|
|
8257
|
+
function rowIndexByEventId(events) {
|
|
8258
|
+
const index = /* @__PURE__ */ new Map();
|
|
8259
|
+
for (const [position, event] of events.entries()) {
|
|
8260
|
+
if (!index.has(event.eventId)) index.set(event.eventId, position);
|
|
8261
|
+
}
|
|
8262
|
+
return index;
|
|
8263
|
+
}
|
|
8264
|
+
function recordedRowSpan(rowIndex, event) {
|
|
8265
|
+
const sourceIds = "sourceEventIds" in event && Array.isArray(event.sourceEventIds) ? event.sourceEventIds : [event.eventId];
|
|
8266
|
+
const rows = [];
|
|
8267
|
+
for (const id of sourceIds) {
|
|
8268
|
+
const row = typeof id === "string" ? rowIndex.get(id) : void 0;
|
|
8269
|
+
if (row === void 0) return void 0;
|
|
8270
|
+
rows.push(row);
|
|
8271
|
+
}
|
|
8272
|
+
if (rows.length === 0) return void 0;
|
|
8273
|
+
return { first: Math.min(...rows), last: Math.max(...rows) };
|
|
8274
|
+
}
|
|
8275
|
+
function recordedTieOrder(context, tiedBefore, tiedAfter) {
|
|
8276
|
+
const rowIndex = rowIndexByEventId(context.events);
|
|
8277
|
+
let lastFinished;
|
|
8278
|
+
let firstStarted;
|
|
8279
|
+
for (const event of tiedBefore) {
|
|
8280
|
+
const span = recordedRowSpan(rowIndex, event);
|
|
8281
|
+
if (span === void 0) return { inOrder: void 0 };
|
|
8282
|
+
if (lastFinished === void 0 || span.last > lastFinished.row) {
|
|
8283
|
+
lastFinished = { event, row: span.last };
|
|
8284
|
+
}
|
|
8285
|
+
}
|
|
8286
|
+
for (const event of tiedAfter) {
|
|
8287
|
+
const span = recordedRowSpan(rowIndex, event);
|
|
8288
|
+
if (span === void 0) return { inOrder: void 0 };
|
|
8289
|
+
if (firstStarted === void 0 || span.first < firstStarted.row) {
|
|
8290
|
+
firstStarted = { event, row: span.first };
|
|
8291
|
+
}
|
|
8292
|
+
}
|
|
8293
|
+
if (lastFinished === void 0 || firstStarted === void 0) return { inOrder: void 0 };
|
|
8294
|
+
return {
|
|
8295
|
+
inOrder: lastFinished.row < firstStarted.row,
|
|
8296
|
+
lastFinishedBefore: lastFinished.event,
|
|
8297
|
+
firstStartedAfter: firstStarted.event
|
|
8298
|
+
};
|
|
8299
|
+
}
|
|
8300
|
+
function tieResolution(order) {
|
|
8301
|
+
return order.inOrder === void 0 ? "recorded-order-unavailable" : "recorded-order";
|
|
8302
|
+
}
|
|
8303
|
+
function tieMessage(order, beforeLabel, afterLabel) {
|
|
8304
|
+
if (order === void 0 || order.inOrder === true) return "";
|
|
8305
|
+
return order.inOrder === void 0 ? " Their timestamps tie at the same millisecond and the recorded event order could not be resolved." : ` Their timestamps tie at the same millisecond; the recorded event order shows ${afterLabel} started before ${beforeLabel} finished.`;
|
|
8306
|
+
}
|
|
8201
8307
|
function normalizedKey(value) {
|
|
8202
8308
|
return value.toLowerCase().replace(/[^a-z0-9_]/g, "");
|
|
8203
8309
|
}
|
|
@@ -8415,10 +8521,13 @@ function createRunStatusRule(options = {}) {
|
|
|
8415
8521
|
findings.push(
|
|
8416
8522
|
failFinding(
|
|
8417
8523
|
"run.status",
|
|
8418
|
-
"Run contains incomplete running events
|
|
8524
|
+
incompleteRunningMessage("Run contains incomplete running events", running),
|
|
8419
8525
|
running.map((event) => eventEvidence3(event)),
|
|
8420
8526
|
"no running events",
|
|
8421
|
-
|
|
8527
|
+
{
|
|
8528
|
+
count: running.length,
|
|
8529
|
+
unfinished: running.map(describeRunningEvent)
|
|
8530
|
+
}
|
|
8422
8531
|
)
|
|
8423
8532
|
);
|
|
8424
8533
|
}
|
|
@@ -8588,6 +8697,34 @@ function createToolUsageRule(options) {
|
|
|
8588
8697
|
const names = tools.map(toolName);
|
|
8589
8698
|
const nameSet = new Set(names);
|
|
8590
8699
|
const findings = [];
|
|
8700
|
+
const forbidden = new Set(options.forbidden ?? []);
|
|
8701
|
+
const allowed = options.allowed?.length ? new Set(options.allowed) : void 0;
|
|
8702
|
+
for (const event of tools) {
|
|
8703
|
+
const name = toolName(event);
|
|
8704
|
+
const display = formatToolNameForMessage(name);
|
|
8705
|
+
if (forbidden.has(name)) {
|
|
8706
|
+
findings.push(
|
|
8707
|
+
failFinding(
|
|
8708
|
+
"tool.usage",
|
|
8709
|
+
`Forbidden tool ${display} appeared.`,
|
|
8710
|
+
[eventEvidence3(event)],
|
|
8711
|
+
"tool absent",
|
|
8712
|
+
name
|
|
8713
|
+
)
|
|
8714
|
+
);
|
|
8715
|
+
}
|
|
8716
|
+
if (allowed && !allowed.has(name)) {
|
|
8717
|
+
findings.push(
|
|
8718
|
+
failFinding(
|
|
8719
|
+
"tool.usage",
|
|
8720
|
+
`Tool ${display} is not in the allowed tool set.`,
|
|
8721
|
+
[eventEvidence3(event)],
|
|
8722
|
+
[...allowed].sort(),
|
|
8723
|
+
name
|
|
8724
|
+
)
|
|
8725
|
+
);
|
|
8726
|
+
}
|
|
8727
|
+
}
|
|
8591
8728
|
for (const required of options.required ?? []) {
|
|
8592
8729
|
if (!nameSet.has(required)) {
|
|
8593
8730
|
const otherKinds = [
|
|
@@ -8605,7 +8742,7 @@ function createToolUsageRule(options) {
|
|
|
8605
8742
|
findings.push(
|
|
8606
8743
|
failFinding(
|
|
8607
8744
|
"tool.usage",
|
|
8608
|
-
`Required tool ${required} did not appear.${hint}`,
|
|
8745
|
+
`Required tool ${formatToolNameForMessage(required)} did not appear.${hint}`,
|
|
8609
8746
|
runEvidence(context.selectedRun),
|
|
8610
8747
|
required,
|
|
8611
8748
|
names
|
|
@@ -8613,21 +8750,6 @@ function createToolUsageRule(options) {
|
|
|
8613
8750
|
);
|
|
8614
8751
|
}
|
|
8615
8752
|
}
|
|
8616
|
-
const forbidden = new Set(options.forbidden ?? []);
|
|
8617
|
-
const allowed = options.allowed?.length ? new Set(options.allowed) : void 0;
|
|
8618
|
-
for (const event of tools) {
|
|
8619
|
-
const name = toolName(event);
|
|
8620
|
-
if (forbidden.has(name)) {
|
|
8621
|
-
findings.push(
|
|
8622
|
-
failFinding("tool.usage", `Forbidden tool ${name} appeared.`, [eventEvidence3(event)], "tool absent", name)
|
|
8623
|
-
);
|
|
8624
|
-
}
|
|
8625
|
-
if (allowed && !allowed.has(name)) {
|
|
8626
|
-
findings.push(
|
|
8627
|
-
failFinding("tool.usage", `Tool ${name} is not in the allowed tool set.`, [eventEvidence3(event)], [...allowed].sort(), name)
|
|
8628
|
-
);
|
|
8629
|
-
}
|
|
8630
|
-
}
|
|
8631
8753
|
if (options.minCount !== void 0 && tools.length < options.minCount) {
|
|
8632
8754
|
findings.push(
|
|
8633
8755
|
failFinding("tool.usage", `Tool count ${tools.length} was below minimum ${options.minCount}.`, runEvidence(context.selectedRun), { minCount: options.minCount }, tools.length)
|
|
@@ -8661,6 +8783,28 @@ function createStepOrderingRule(options) {
|
|
|
8661
8783
|
const afterEvents = finishedEvents(context, options.after.kind).filter(
|
|
8662
8784
|
(event) => stepEndpointName(event, options.after.kind) === options.after.name
|
|
8663
8785
|
);
|
|
8786
|
+
const beforeRunning = semanticEvents(context).filter(
|
|
8787
|
+
(event) => event.kind === options.before.kind && event.status === "running" && stepEndpointName(event, options.before.kind) === options.before.name
|
|
8788
|
+
);
|
|
8789
|
+
if (beforeEvents.length === 0 && afterEvents.length > 0 && beforeRunning.length > 0) {
|
|
8790
|
+
const endpointExpected2 = bothTools ? { before: options.before.name, after: options.after.name } : { before: options.before, after: options.after };
|
|
8791
|
+
return [
|
|
8792
|
+
failFinding(
|
|
8793
|
+
ruleId,
|
|
8794
|
+
`${subject} ${beforeLabel} started but never finished before ${afterLabel} ran.`,
|
|
8795
|
+
[
|
|
8796
|
+
...beforeRunning.map((event) => eventEvidence3(event)),
|
|
8797
|
+
eventEvidence3(afterEvents[0])
|
|
8798
|
+
],
|
|
8799
|
+
{ ...endpointExpected2, mode },
|
|
8800
|
+
{
|
|
8801
|
+
code: bothTools ? "tool.order.interval-unresolved" : "step.order.interval-unresolved",
|
|
8802
|
+
unfinishedBefore: true,
|
|
8803
|
+
beforeStatus: "running"
|
|
8804
|
+
}
|
|
8805
|
+
)
|
|
8806
|
+
];
|
|
8807
|
+
}
|
|
8664
8808
|
if (beforeEvents.length === 0 || afterEvents.length === 0) {
|
|
8665
8809
|
if (!requireEndpoints) return [];
|
|
8666
8810
|
const findings = [];
|
|
@@ -8777,16 +8921,18 @@ function createStepOrderingRule(options) {
|
|
|
8777
8921
|
)
|
|
8778
8922
|
];
|
|
8779
8923
|
}
|
|
8780
|
-
|
|
8924
|
+
const tie = beforeEnd === afterStart ? recordedTieOrder(context, [beforeEvent], [afterEvent]) : void 0;
|
|
8925
|
+
if (beforeEnd > afterStart || tie !== void 0 && tie.inOrder !== true) {
|
|
8781
8926
|
return [
|
|
8782
8927
|
failFinding(
|
|
8783
8928
|
ruleId,
|
|
8784
|
-
`${subject} ${beforeLabel} must finish before ${afterLabel} starts
|
|
8929
|
+
`${subject} ${beforeLabel} must finish before ${afterLabel} starts.${tieMessage(tie, beforeLabel, afterLabel)}`,
|
|
8785
8930
|
[eventEvidence3(beforeEvent), eventEvidence3(afterEvent)],
|
|
8786
8931
|
expected,
|
|
8787
8932
|
{
|
|
8788
8933
|
beforeEndedAt: new Date(beforeEnd).toISOString(),
|
|
8789
|
-
afterStartedAt: new Date(afterStart).toISOString()
|
|
8934
|
+
afterStartedAt: new Date(afterStart).toISOString(),
|
|
8935
|
+
...tie !== void 0 ? { resolvedBy: tieResolution(tie) } : {}
|
|
8790
8936
|
}
|
|
8791
8937
|
)
|
|
8792
8938
|
];
|
|
@@ -8849,16 +8995,27 @@ function createStepOrderingRule(options) {
|
|
|
8849
8995
|
)
|
|
8850
8996
|
];
|
|
8851
8997
|
}
|
|
8852
|
-
|
|
8998
|
+
const tiedEnd = latestBefore.end;
|
|
8999
|
+
const tiedStart = earliestAfter.start;
|
|
9000
|
+
const tie = tiedEnd === tiedStart ? recordedTieOrder(
|
|
9001
|
+
context,
|
|
9002
|
+
beforeEvents.filter((event) => eventEndMs(event) === tiedEnd),
|
|
9003
|
+
afterEvents.filter((event) => eventStartMs(event) === tiedStart)
|
|
9004
|
+
) : void 0;
|
|
9005
|
+
if (latestBefore.end > earliestAfter.start || tie !== void 0 && tie.inOrder !== true) {
|
|
8853
9006
|
return [
|
|
8854
9007
|
failFinding(
|
|
8855
9008
|
ruleId,
|
|
8856
|
-
bothTools ? `Every ${options.before.name} tool call must finish before any ${options.after.name} tool call starts.` : `Every ${beforeLabel} must finish before any ${afterLabel} starts
|
|
8857
|
-
[
|
|
9009
|
+
(bothTools ? `Every ${options.before.name} tool call must finish before any ${options.after.name} tool call starts.` : `Every ${beforeLabel} must finish before any ${afterLabel} starts.`) + tieMessage(tie, beforeLabel, afterLabel),
|
|
9010
|
+
[
|
|
9011
|
+
eventEvidence3(tie?.lastFinishedBefore ?? latestBefore.event),
|
|
9012
|
+
eventEvidence3(tie?.firstStartedAfter ?? earliestAfter.event)
|
|
9013
|
+
],
|
|
8858
9014
|
expected,
|
|
8859
9015
|
{
|
|
8860
9016
|
latestBeforeEndedAt: new Date(latestBefore.end).toISOString(),
|
|
8861
|
-
earliestAfterStartedAt: new Date(earliestAfter.start).toISOString()
|
|
9017
|
+
earliestAfterStartedAt: new Date(earliestAfter.start).toISOString(),
|
|
9018
|
+
...tie !== void 0 ? { resolvedBy: tieResolution(tie) } : {}
|
|
8862
9019
|
}
|
|
8863
9020
|
)
|
|
8864
9021
|
];
|
|
@@ -9024,10 +9181,13 @@ function createStructureIncompleteRule(options = {}) {
|
|
|
9024
9181
|
findings.push(
|
|
9025
9182
|
failFinding(
|
|
9026
9183
|
"structure.incomplete",
|
|
9027
|
-
"Trace contains incomplete running events
|
|
9184
|
+
incompleteRunningMessage("Trace contains incomplete running events", running),
|
|
9028
9185
|
running.map((event) => eventEvidence3(event, "status")),
|
|
9029
9186
|
"no running events",
|
|
9030
|
-
|
|
9187
|
+
{
|
|
9188
|
+
count: running.length,
|
|
9189
|
+
unfinished: running.map(describeRunningEvent)
|
|
9190
|
+
}
|
|
9031
9191
|
)
|
|
9032
9192
|
);
|
|
9033
9193
|
}
|
|
@@ -16011,5 +16171,5 @@ function renderGateReport(result, options = {}) {
|
|
|
16011
16171
|
}
|
|
16012
16172
|
|
|
16013
16173
|
export { ATTRIBUTION_CONFIDENCES, COHORT_METRIC_IDS, DEFAULT_REDACT_KEYS, DEFAULT_SUITE_ARTIFACTS_DIR, DELIVERY_CONFLICT_ATTR, DELIVERY_EVENT_BASE_ATTR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, IDENTITY_CONFLICT_ATTR, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, defineTraceContract, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, evaluateTraceContract, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isAttributionConfidence, isCredentialSensitiveKey, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToRunTrees, persistedInspectEventsToTraceEvents, projectLogicalEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, stringContainsHighConfidenceCredential, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
|
|
16014
|
-
//# sourceMappingURL=chunk-
|
|
16015
|
-
//# sourceMappingURL=chunk-
|
|
16174
|
+
//# sourceMappingURL=chunk-AILSHDEF.mjs.map
|
|
16175
|
+
//# sourceMappingURL=chunk-AILSHDEF.mjs.map
|