agent-inspect 6.18.0 → 6.19.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +1 -1
- package/docs/API.md +2 -1
- package/package.json +1 -1
- package/packages/cli/dist/{chunk-S4TLTLXN.mjs → chunk-R7Y5SGH5.mjs} +348 -27
- package/packages/cli/dist/chunk-R7Y5SGH5.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +375 -46
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +6 -5
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-R6V5EZZH.mjs → src-T4EZERCX.mjs} +3 -3
- package/packages/cli/dist/{src-R6V5EZZH.mjs.map → src-T4EZERCX.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +13 -2
- package/packages/core/dist/advanced.d.ts +13 -2
- package/packages/core/dist/advanced.mjs +1 -1
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/checks.cjs +334 -5
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.d.cts +81 -4
- package/packages/core/dist/checks.d.ts +81 -4
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-LQC7IED3.mjs → chunk-NABMW5DX.mjs} +336 -8
- package/packages/core/dist/chunk-NABMW5DX.mjs.map +1 -0
- package/packages/core/dist/{index-B9ZGvUVL.d.ts → index-AMAGuO_i.d.ts} +1 -1
- package/packages/core/dist/{index-DWu54Y28.d.cts → index-DCR816Yt.d.cts} +2 -2
- package/packages/core/dist/{index-DlwbVqEs.d.ts → index-DKwuGSe-.d.ts} +2 -2
- package/packages/core/dist/{index-B259NKkH.d.cts → index-vqr-g6ev.d.cts} +1 -1
- package/packages/core/dist/readers.d.cts +1 -1
- package/packages/core/dist/readers.d.ts +1 -1
- package/packages/cli/dist/chunk-S4TLTLXN.mjs.map +0 -1
- package/packages/core/dist/chunk-LQC7IED3.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.19.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 07358f2: Add custom TraceReader authoring guidance with a synthetic foreign-session recipe, additive derived failure roles on TraceFacts (MCP/Evidence counts), architectural-intent interop docs/recipe, and bounded priorContextReferences convention.
|
|
8
|
+
|
|
3
9
|
## 6.18.0
|
|
4
10
|
|
|
5
11
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -210,7 +210,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
|
|
|
210
210
|
|
|
211
211
|
## Status and documentation
|
|
212
212
|
|
|
213
|
-
**Current published baseline:** **6.
|
|
213
|
+
**Current published baseline:** **6.19.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
|
|
214
214
|
|
|
215
215
|
Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
|
|
216
216
|
|
package/docs/API.md
CHANGED
|
@@ -514,6 +514,7 @@ import type { TraceReader } from "agent-inspect/readers";
|
|
|
514
514
|
- **`openInferenceJsonReader`**: local OpenInference JSON compatibility reader. Prompt/output-like attributes are summarized and bounded rather than stored as raw content.
|
|
515
515
|
- **`otlpJsonReader`**: local OTLP/HTTP JSON trace payload reader. Resource, scope, span, status, event, and parent metadata are preserved where possible with warnings for unsupported fields.
|
|
516
516
|
- **`DEFAULT_TRACE_READERS`**: ordered built-in reader registry used when no custom `readers` array is supplied.
|
|
517
|
+
- **Custom readers**: implement `TraceReader` and pass `{ readers: [myReader, ...DEFAULT_TRACE_READERS] }` — see [CUSTOM-TRACE-READER.md](./CUSTOM-TRACE-READER.md).
|
|
517
518
|
|
|
518
519
|
The reader contract does not silently accept arbitrary JSON and does not add OTel SDK, database, hosted ingestion, or network upload dependencies. See also [PROGRAMMATIC-TRACE-ANALYSIS.md](./PROGRAMMATIC-TRACE-ANALYSIS.md).
|
|
519
520
|
|
|
@@ -536,7 +537,7 @@ import type { TraceCheckRule, TraceCheckResult, TraceFacts } from "agent-inspect
|
|
|
536
537
|
```
|
|
537
538
|
|
|
538
539
|
- **`runTraceChecks({ read }, { rules?, select?, runId? })`**: executes provided rules against a `TraceReadResult` from `agent-inspect/readers`.
|
|
539
|
-
- **`buildTraceFacts(read | events)`**: experimental TraceFacts over logical projection; accepts `TraceReadResult` or normalized `PersistedInspectEvent[]`.
|
|
540
|
+
- **`buildTraceFacts(read | events)`**: experimental TraceFacts over logical projection; accepts `TraceReadResult` or normalized `PersistedInspectEvent[]`. Additive 6.19 fields: `failureFacts`, `failuresByRole`, `summary.failureRoleCounts`.
|
|
540
541
|
- **`defineTraceContract` / `evaluateTraceContract` / `evaluateTraceContractRead`**: experimental contract helpers; `evaluateTraceContractRead(read, contract)` wraps `{ read }`.
|
|
541
542
|
- **`formatProgrammaticDiagnostic` / `PROGRAMMATIC_DIAGNOSTIC_SPECS`**: stable `AI_*` codes with remediation hints.
|
|
542
543
|
- **Built-in rule factories**: run, tool, LLM, structure, retrieval, guardrail, decision, safety, and baseline helpers including `createRunStatusRule`, `createToolUsageRule`, `createLlmUsageRule`, `createStructureOrphanRule`, `createStructureCycleRule`, `createStructureRelationshipRule`, `createRetrievalRule`, `createGuardrailRule`, `createDecisionRule`, `createSafetyRawContentRule`, `createSafetySecretPatternRule`, and `createBaselineRegressionRule`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.
|
|
3
|
+
"version": "6.19.0",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
|
|
@@ -6221,6 +6221,325 @@ function pickString(record, keys) {
|
|
|
6221
6221
|
return void 0;
|
|
6222
6222
|
}
|
|
6223
6223
|
|
|
6224
|
+
// packages/core/src/checks/derived-failure.ts
|
|
6225
|
+
function isRecord8(value) {
|
|
6226
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
6227
|
+
}
|
|
6228
|
+
function pickString2(record, keys) {
|
|
6229
|
+
if (!record) return void 0;
|
|
6230
|
+
for (const key of keys) {
|
|
6231
|
+
const value = record[key];
|
|
6232
|
+
if (typeof value === "string" && value.trim() !== "") return value;
|
|
6233
|
+
}
|
|
6234
|
+
return void 0;
|
|
6235
|
+
}
|
|
6236
|
+
function pickNumber(record, keys) {
|
|
6237
|
+
if (!record) return void 0;
|
|
6238
|
+
for (const key of keys) {
|
|
6239
|
+
const value = record[key];
|
|
6240
|
+
if (typeof value === "number" && Number.isFinite(value) && value > 0) {
|
|
6241
|
+
return value;
|
|
6242
|
+
}
|
|
6243
|
+
}
|
|
6244
|
+
return void 0;
|
|
6245
|
+
}
|
|
6246
|
+
function eventMetadata(event) {
|
|
6247
|
+
const attrs = isRecord8(event.attributes) ? event.attributes : void 0;
|
|
6248
|
+
const nested = attrs !== void 0 && isRecord8(attrs.metadata) ? attrs.metadata : void 0;
|
|
6249
|
+
return {
|
|
6250
|
+
...attrs ?? {},
|
|
6251
|
+
...nested ?? {}
|
|
6252
|
+
};
|
|
6253
|
+
}
|
|
6254
|
+
function canonicalName(event) {
|
|
6255
|
+
if (event.kind === "TOOL") return resolveCanonicalToolName(event);
|
|
6256
|
+
return event.name;
|
|
6257
|
+
}
|
|
6258
|
+
function linkKeys(event) {
|
|
6259
|
+
const meta = eventMetadata(event);
|
|
6260
|
+
const keys = [];
|
|
6261
|
+
for (const key of ["linkedStepId", "toolCallId", "mcpToolCallId"]) {
|
|
6262
|
+
const value = pickString2(meta, [key]);
|
|
6263
|
+
if (value !== void 0) keys.push(`${key}:${value}`);
|
|
6264
|
+
}
|
|
6265
|
+
const stepId = pickString2(meta, ["stepId"]);
|
|
6266
|
+
if (stepId !== void 0) keys.push(`stepId:${stepId}`);
|
|
6267
|
+
return keys;
|
|
6268
|
+
}
|
|
6269
|
+
function buildRunContexts(logicalEvents) {
|
|
6270
|
+
const byRun = /* @__PURE__ */ new Map();
|
|
6271
|
+
for (const event of logicalEvents) {
|
|
6272
|
+
const existing = byRun.get(event.runId) ?? {
|
|
6273
|
+
runId: event.runId,
|
|
6274
|
+
name: event.name
|
|
6275
|
+
};
|
|
6276
|
+
const meta = eventMetadata(event);
|
|
6277
|
+
if (event.kind === "RUN" || existing.name === event.runId) {
|
|
6278
|
+
existing.name = event.name || existing.name;
|
|
6279
|
+
}
|
|
6280
|
+
if (event.kind === "RUN" && event.status !== void 0 && event.status !== "running") {
|
|
6281
|
+
existing.status = event.status;
|
|
6282
|
+
}
|
|
6283
|
+
existing.retryOf ??= pickString2(meta, ["retryOf"]);
|
|
6284
|
+
existing.attempt ??= pickNumber(meta, ["attempt", "retryAttempt", "retryCount"]);
|
|
6285
|
+
existing.sessionId ??= pickString2(meta, ["sessionId", "conversationId"]);
|
|
6286
|
+
existing.groupId ??= pickString2(meta, ["groupId"]);
|
|
6287
|
+
existing.parentGroupId ??= pickString2(meta, ["parentGroupId"]);
|
|
6288
|
+
existing.fallbackOf ??= pickString2(meta, ["fallbackOf", "fallbackFrom"]);
|
|
6289
|
+
byRun.set(event.runId, existing);
|
|
6290
|
+
}
|
|
6291
|
+
return byRun;
|
|
6292
|
+
}
|
|
6293
|
+
function sameCorrelationScope(a, b) {
|
|
6294
|
+
if (a.sessionId && b.sessionId && a.sessionId === b.sessionId) return true;
|
|
6295
|
+
if (a.groupId && b.groupId && a.groupId === b.groupId) return true;
|
|
6296
|
+
if (a.parentGroupId && b.parentGroupId && a.parentGroupId === b.parentGroupId) {
|
|
6297
|
+
return true;
|
|
6298
|
+
}
|
|
6299
|
+
return false;
|
|
6300
|
+
}
|
|
6301
|
+
function isSuccessful(event) {
|
|
6302
|
+
return event.status === "ok";
|
|
6303
|
+
}
|
|
6304
|
+
function isFailure(event) {
|
|
6305
|
+
return event.status === "error";
|
|
6306
|
+
}
|
|
6307
|
+
function compareEventOrder(a, b) {
|
|
6308
|
+
const byTime = a.timestamp.localeCompare(b.timestamp);
|
|
6309
|
+
if (byTime !== 0) return byTime;
|
|
6310
|
+
return a.eventId.localeCompare(b.eventId);
|
|
6311
|
+
}
|
|
6312
|
+
function collectCandidates(failure, logicalEvents, runs) {
|
|
6313
|
+
const failureRun = runs.get(failure.runId);
|
|
6314
|
+
const failureLinks = new Set(linkKeys(failure));
|
|
6315
|
+
const failureName = canonicalName(failure);
|
|
6316
|
+
const failureAttempt = pickNumber(eventMetadata(failure), ["attempt", "retryAttempt", "retryCount"]) ?? failureRun?.attempt;
|
|
6317
|
+
const candidates = [];
|
|
6318
|
+
for (const event of logicalEvents) {
|
|
6319
|
+
if (event.eventId === failure.eventId) continue;
|
|
6320
|
+
if (event.status === "running") continue;
|
|
6321
|
+
const eventRun = runs.get(event.runId);
|
|
6322
|
+
const eventMeta = eventMetadata(event);
|
|
6323
|
+
const sameRun = event.runId === failure.runId;
|
|
6324
|
+
if (eventRun?.retryOf === failure.runId) {
|
|
6325
|
+
candidates.push({
|
|
6326
|
+
event,
|
|
6327
|
+
basis: "retryOf",
|
|
6328
|
+
confidence: "explicit",
|
|
6329
|
+
viaRunId: event.runId
|
|
6330
|
+
});
|
|
6331
|
+
continue;
|
|
6332
|
+
}
|
|
6333
|
+
if (eventRun?.fallbackOf === failure.runId || pickString2(eventMeta, ["fallbackOf", "fallbackFrom"]) === failure.runId) {
|
|
6334
|
+
candidates.push({
|
|
6335
|
+
event,
|
|
6336
|
+
basis: "fallbackOf",
|
|
6337
|
+
confidence: "explicit",
|
|
6338
|
+
viaRunId: event.runId
|
|
6339
|
+
});
|
|
6340
|
+
continue;
|
|
6341
|
+
}
|
|
6342
|
+
if (sameRun && compareEventOrder(failure, event) >= 0) continue;
|
|
6343
|
+
const eventLinks = linkKeys(event);
|
|
6344
|
+
const sharedLink = eventLinks.find((key) => failureLinks.has(key));
|
|
6345
|
+
if (sharedLink !== void 0) {
|
|
6346
|
+
candidates.push({
|
|
6347
|
+
event,
|
|
6348
|
+
basis: sharedLink.split(":")[0] ?? "linkedId",
|
|
6349
|
+
confidence: "explicit"
|
|
6350
|
+
});
|
|
6351
|
+
continue;
|
|
6352
|
+
}
|
|
6353
|
+
const eventAttempt = pickNumber(eventMeta, ["attempt", "retryAttempt", "retryCount"]) ?? eventRun?.attempt;
|
|
6354
|
+
const sameParent = failure.parentId !== void 0 && event.parentId !== void 0 && failure.parentId === event.parentId;
|
|
6355
|
+
const differentParent = failure.parentId !== void 0 && event.parentId !== void 0 && failure.parentId !== event.parentId;
|
|
6356
|
+
const sessionScoped = failureRun !== void 0 && eventRun !== void 0 && sameCorrelationScope(failureRun, eventRun);
|
|
6357
|
+
if (canonicalName(event) === failureName && failureAttempt !== void 0 && eventAttempt !== void 0 && eventAttempt > failureAttempt && !differentParent && (sameParent || sessionScoped || sameRun)) {
|
|
6358
|
+
candidates.push({
|
|
6359
|
+
event,
|
|
6360
|
+
basis: "attempt-progression",
|
|
6361
|
+
confidence: "correlated",
|
|
6362
|
+
...event.runId !== failure.runId ? { viaRunId: event.runId } : {}
|
|
6363
|
+
});
|
|
6364
|
+
}
|
|
6365
|
+
}
|
|
6366
|
+
const byId = /* @__PURE__ */ new Map();
|
|
6367
|
+
for (const candidate of candidates) {
|
|
6368
|
+
const prev = byId.get(candidate.event.eventId);
|
|
6369
|
+
if (!prev || prev.confidence !== "explicit" && candidate.confidence === "explicit") {
|
|
6370
|
+
byId.set(candidate.event.eventId, candidate);
|
|
6371
|
+
}
|
|
6372
|
+
}
|
|
6373
|
+
return [...byId.values()].sort((a, b) => compareEventOrder(a.event, b.event));
|
|
6374
|
+
}
|
|
6375
|
+
function classifyFailure(failure, candidates, runs, logicalEvents) {
|
|
6376
|
+
const successful = candidates.filter((c) => isSuccessful(c.event));
|
|
6377
|
+
const unsuccessful = candidates.filter((c) => !isSuccessful(c.event));
|
|
6378
|
+
const retryRunIds = Object.freeze(
|
|
6379
|
+
[...new Set(candidates.map((c) => c.viaRunId).filter((id) => id !== void 0))].sort(
|
|
6380
|
+
(a, b) => a.localeCompare(b)
|
|
6381
|
+
)
|
|
6382
|
+
);
|
|
6383
|
+
if (successful.length > 1) {
|
|
6384
|
+
const distinctRuns = new Set(successful.map((c) => c.event.runId));
|
|
6385
|
+
const distinctParents = new Set(
|
|
6386
|
+
successful.map((c) => c.event.parentId ?? "").filter((id) => id !== "")
|
|
6387
|
+
);
|
|
6388
|
+
if (distinctRuns.size > 1 || distinctParents.size > 1) {
|
|
6389
|
+
return {
|
|
6390
|
+
eventId: failure.eventId,
|
|
6391
|
+
runId: failure.runId,
|
|
6392
|
+
name: failure.name,
|
|
6393
|
+
kind: failure.kind,
|
|
6394
|
+
role: "unknown",
|
|
6395
|
+
confidence: "unknown",
|
|
6396
|
+
basis: Object.freeze(["ambiguous-recovery-candidates"]),
|
|
6397
|
+
recoveryEventIds: Object.freeze(
|
|
6398
|
+
successful.map((c) => c.event.eventId).sort((a, b) => a.localeCompare(b))
|
|
6399
|
+
),
|
|
6400
|
+
retryRunIds
|
|
6401
|
+
};
|
|
6402
|
+
}
|
|
6403
|
+
}
|
|
6404
|
+
if (successful.length >= 1) {
|
|
6405
|
+
const best = successful[0];
|
|
6406
|
+
return {
|
|
6407
|
+
eventId: failure.eventId,
|
|
6408
|
+
runId: failure.runId,
|
|
6409
|
+
name: failure.name,
|
|
6410
|
+
kind: failure.kind,
|
|
6411
|
+
role: "recovered",
|
|
6412
|
+
confidence: best.confidence,
|
|
6413
|
+
basis: Object.freeze([best.basis]),
|
|
6414
|
+
recoveryEventIds: Object.freeze(
|
|
6415
|
+
successful.map((c) => c.event.eventId).sort((a, b) => a.localeCompare(b))
|
|
6416
|
+
),
|
|
6417
|
+
retryRunIds
|
|
6418
|
+
};
|
|
6419
|
+
}
|
|
6420
|
+
if (candidates.length > 0) {
|
|
6421
|
+
const best = candidates[0];
|
|
6422
|
+
return {
|
|
6423
|
+
eventId: failure.eventId,
|
|
6424
|
+
runId: failure.runId,
|
|
6425
|
+
name: failure.name,
|
|
6426
|
+
kind: failure.kind,
|
|
6427
|
+
role: "transient",
|
|
6428
|
+
confidence: best.confidence,
|
|
6429
|
+
basis: Object.freeze([
|
|
6430
|
+
best.basis,
|
|
6431
|
+
unsuccessful.some((c) => c.event.status === void 0) ? "retry-incomplete" : "retry-without-success"
|
|
6432
|
+
]),
|
|
6433
|
+
recoveryEventIds: Object.freeze([]),
|
|
6434
|
+
retryRunIds
|
|
6435
|
+
};
|
|
6436
|
+
}
|
|
6437
|
+
const failureMeta = eventMetadata(failure);
|
|
6438
|
+
const declaredSuccessor = pickString2(failureMeta, [
|
|
6439
|
+
"retriedBy",
|
|
6440
|
+
"nextRetryRunId",
|
|
6441
|
+
"retryRunId"
|
|
6442
|
+
]);
|
|
6443
|
+
if (declaredSuccessor !== void 0 && !runs.has(declaredSuccessor)) {
|
|
6444
|
+
return {
|
|
6445
|
+
eventId: failure.eventId,
|
|
6446
|
+
runId: failure.runId,
|
|
6447
|
+
name: failure.name,
|
|
6448
|
+
kind: failure.kind,
|
|
6449
|
+
role: "transient",
|
|
6450
|
+
confidence: "explicit",
|
|
6451
|
+
basis: Object.freeze(["retry-declared", "retry-run-missing"]),
|
|
6452
|
+
recoveryEventIds: Object.freeze([]),
|
|
6453
|
+
retryRunIds: Object.freeze([declaredSuccessor])
|
|
6454
|
+
};
|
|
6455
|
+
}
|
|
6456
|
+
for (const run of runs.values()) {
|
|
6457
|
+
if (run.retryOf === failure.runId) {
|
|
6458
|
+
return {
|
|
6459
|
+
eventId: failure.eventId,
|
|
6460
|
+
runId: failure.runId,
|
|
6461
|
+
name: failure.name,
|
|
6462
|
+
kind: failure.kind,
|
|
6463
|
+
role: "transient",
|
|
6464
|
+
confidence: "explicit",
|
|
6465
|
+
basis: Object.freeze(["retryOf", "retry-run-missing-or-empty"]),
|
|
6466
|
+
recoveryEventIds: Object.freeze([]),
|
|
6467
|
+
retryRunIds: Object.freeze([run.runId])
|
|
6468
|
+
};
|
|
6469
|
+
}
|
|
6470
|
+
}
|
|
6471
|
+
const failureRun = runs.get(failure.runId);
|
|
6472
|
+
const hasSuccessorDeclared = [...runs.values()].some((run) => run.retryOf === failure.runId);
|
|
6473
|
+
const isFinalInChain = failureRun !== void 0 && !hasSuccessorDeclared && (failureRun.retryOf !== void 0 || failureRun.attempt !== void 0 && failureRun.attempt > 1 || pickNumber(eventMetadata(failure), ["attempt"]) !== void 0);
|
|
6474
|
+
if (isFinalInChain && failureRun?.status === "error" && !logicalEvents.some(
|
|
6475
|
+
(event) => event.runId === failure.runId && event.eventId !== failure.eventId && isSuccessful(event) && canonicalName(event) === canonicalName(failure)
|
|
6476
|
+
)) {
|
|
6477
|
+
return {
|
|
6478
|
+
eventId: failure.eventId,
|
|
6479
|
+
runId: failure.runId,
|
|
6480
|
+
name: failure.name,
|
|
6481
|
+
kind: failure.kind,
|
|
6482
|
+
role: "terminal",
|
|
6483
|
+
confidence: failureRun.retryOf !== void 0 ? "explicit" : "correlated",
|
|
6484
|
+
basis: Object.freeze(["final-retry-chain-member", "enclosing-run-error"]),
|
|
6485
|
+
recoveryEventIds: Object.freeze([]),
|
|
6486
|
+
retryRunIds: Object.freeze(
|
|
6487
|
+
failureRun.retryOf !== void 0 ? [failureRun.retryOf] : []
|
|
6488
|
+
)
|
|
6489
|
+
};
|
|
6490
|
+
}
|
|
6491
|
+
return {
|
|
6492
|
+
eventId: failure.eventId,
|
|
6493
|
+
runId: failure.runId,
|
|
6494
|
+
name: failure.name,
|
|
6495
|
+
kind: failure.kind,
|
|
6496
|
+
role: "unknown",
|
|
6497
|
+
confidence: "unknown",
|
|
6498
|
+
basis: Object.freeze(["no-explicit-or-correlated-recovery"]),
|
|
6499
|
+
recoveryEventIds: Object.freeze([]),
|
|
6500
|
+
retryRunIds: Object.freeze([])
|
|
6501
|
+
};
|
|
6502
|
+
}
|
|
6503
|
+
function deriveFailureFacts(logicalEvents) {
|
|
6504
|
+
const runs = buildRunContexts(logicalEvents);
|
|
6505
|
+
const failures = logicalEvents.filter((event) => isFailure(event)).sort(compareEventOrder);
|
|
6506
|
+
const failureFacts = failures.map(
|
|
6507
|
+
(failure) => classifyFailure(failure, collectCandidates(failure, logicalEvents, runs), runs, logicalEvents)
|
|
6508
|
+
);
|
|
6509
|
+
const byRole = /* @__PURE__ */ new Map([
|
|
6510
|
+
["transient", []],
|
|
6511
|
+
["recovered", []],
|
|
6512
|
+
["terminal", []],
|
|
6513
|
+
["unknown", []]
|
|
6514
|
+
]);
|
|
6515
|
+
for (const fact2 of failureFacts) {
|
|
6516
|
+
byRole.get(fact2.role).push(fact2);
|
|
6517
|
+
}
|
|
6518
|
+
for (const [role, list] of byRole) {
|
|
6519
|
+
byRole.set(
|
|
6520
|
+
role,
|
|
6521
|
+
Object.freeze(
|
|
6522
|
+
[...list].sort((a, b) => {
|
|
6523
|
+
const byRun = a.runId.localeCompare(b.runId);
|
|
6524
|
+
if (byRun !== 0) return byRun;
|
|
6525
|
+
return a.eventId.localeCompare(b.eventId);
|
|
6526
|
+
})
|
|
6527
|
+
)
|
|
6528
|
+
);
|
|
6529
|
+
}
|
|
6530
|
+
const failureRoleCounts = {
|
|
6531
|
+
transient: byRole.get("transient").length,
|
|
6532
|
+
recovered: byRole.get("recovered").length,
|
|
6533
|
+
terminal: byRole.get("terminal").length,
|
|
6534
|
+
unknown: byRole.get("unknown").length
|
|
6535
|
+
};
|
|
6536
|
+
return {
|
|
6537
|
+
failureFacts: Object.freeze(failureFacts),
|
|
6538
|
+
failuresByRole: byRole,
|
|
6539
|
+
failureRoleCounts
|
|
6540
|
+
};
|
|
6541
|
+
}
|
|
6542
|
+
|
|
6224
6543
|
// packages/core/src/checks/trace-facts.ts
|
|
6225
6544
|
function summarizeSemanticParity(events) {
|
|
6226
6545
|
const projection = projectLogicalEvents(events);
|
|
@@ -6231,6 +6550,7 @@ function summarizeSemanticParity(events) {
|
|
|
6231
6550
|
const finishedToolNames = Object.freeze(
|
|
6232
6551
|
finishedTools.map((event) => resolveCanonicalToolName(event)).sort((a, b) => a.localeCompare(b))
|
|
6233
6552
|
);
|
|
6553
|
+
const derived = deriveFailureFacts(logical);
|
|
6234
6554
|
return {
|
|
6235
6555
|
rawEventCount: events.length,
|
|
6236
6556
|
logicalEventCount: logical.length,
|
|
@@ -6241,7 +6561,8 @@ function summarizeSemanticParity(events) {
|
|
|
6241
6561
|
parentRemapCount: projection.diagnostics.filter(
|
|
6242
6562
|
(item) => item.code === "AI_LOGICAL_PARENT_REMAPPED"
|
|
6243
6563
|
).length,
|
|
6244
|
-
diagnostics: projection.diagnostics
|
|
6564
|
+
diagnostics: projection.diagnostics,
|
|
6565
|
+
failureRoleCounts: derived.failureRoleCounts
|
|
6245
6566
|
};
|
|
6246
6567
|
}
|
|
6247
6568
|
formatProgrammaticDiagnostic(
|
|
@@ -6713,7 +7034,7 @@ function finishedEvents(context, kind) {
|
|
|
6713
7034
|
function toolInvocationEvents(context) {
|
|
6714
7035
|
return semanticEvents(context).filter((event) => event.kind === "TOOL");
|
|
6715
7036
|
}
|
|
6716
|
-
function
|
|
7037
|
+
function isRecord9(value) {
|
|
6717
7038
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
6718
7039
|
}
|
|
6719
7040
|
function eventMap(events) {
|
|
@@ -6764,7 +7085,7 @@ function pushValueEntries(entries, event, value, path14, key, depth = 0) {
|
|
|
6764
7085
|
}
|
|
6765
7086
|
return;
|
|
6766
7087
|
}
|
|
6767
|
-
if (!
|
|
7088
|
+
if (!isRecord9(value)) return;
|
|
6768
7089
|
for (const nestedKey of Object.keys(value).sort((a, b) => a.localeCompare(b))) {
|
|
6769
7090
|
pushValueEntries(
|
|
6770
7091
|
entries,
|
|
@@ -7590,7 +7911,7 @@ function createSafetyOversizedAttributeRule(options) {
|
|
|
7590
7911
|
)
|
|
7591
7912
|
);
|
|
7592
7913
|
}
|
|
7593
|
-
if (
|
|
7914
|
+
if (isRecord9(entry.value) && options.maxObjectKeys !== void 0 && Object.keys(entry.value).length > options.maxObjectKeys) {
|
|
7594
7915
|
findings.push(
|
|
7595
7916
|
failFinding(
|
|
7596
7917
|
"safety.oversizedAttribute",
|
|
@@ -7882,14 +8203,14 @@ function runTraceChecks(input, options = {}) {
|
|
|
7882
8203
|
}
|
|
7883
8204
|
|
|
7884
8205
|
// packages/core/src/persisted/token-usage.ts
|
|
7885
|
-
function
|
|
8206
|
+
function isRecord10(value) {
|
|
7886
8207
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
7887
8208
|
}
|
|
7888
8209
|
function nonNegativeFinite(value) {
|
|
7889
8210
|
return typeof value === "number" && Number.isFinite(value) && value >= 0 ? value : void 0;
|
|
7890
8211
|
}
|
|
7891
8212
|
function normalizeTokenUsage(value) {
|
|
7892
|
-
if (!
|
|
8213
|
+
if (!isRecord10(value)) return void 0;
|
|
7893
8214
|
const input = nonNegativeFinite(value.input);
|
|
7894
8215
|
const output = nonNegativeFinite(value.output);
|
|
7895
8216
|
const suppliedTotal = nonNegativeFinite(value.total);
|
|
@@ -8770,7 +9091,7 @@ function persistedEventsForParsedTrace(parsed) {
|
|
|
8770
9091
|
sourceName: "agent-inspect-jsonl-reader"
|
|
8771
9092
|
});
|
|
8772
9093
|
}
|
|
8773
|
-
function
|
|
9094
|
+
function isRecord11(value) {
|
|
8774
9095
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
8775
9096
|
}
|
|
8776
9097
|
function isNonEmptyString4(value) {
|
|
@@ -8785,13 +9106,13 @@ function readStringField(record, keys) {
|
|
|
8785
9106
|
}
|
|
8786
9107
|
function readRecordField(record, key) {
|
|
8787
9108
|
const value = record[key];
|
|
8788
|
-
return
|
|
9109
|
+
return isRecord11(value) ? value : void 0;
|
|
8789
9110
|
}
|
|
8790
9111
|
function parseJsonDocument(content) {
|
|
8791
9112
|
return JSON.parse(content);
|
|
8792
9113
|
}
|
|
8793
9114
|
function looksLikeOpenInferenceSpan(value) {
|
|
8794
|
-
if (!
|
|
9115
|
+
if (!isRecord11(value)) return false;
|
|
8795
9116
|
const attributes = readRecordField(value, "attributes");
|
|
8796
9117
|
return readStringField(value, ["trace_id", "traceId"]) !== void 0 && readStringField(value, ["span_id", "spanId"]) !== void 0 && (readStringField(value, ["name"]) !== void 0 || attributes?.["openinference.span.kind"] !== void 0);
|
|
8797
9118
|
}
|
|
@@ -8816,7 +9137,7 @@ function extractOpenInferenceDocument(root) {
|
|
|
8816
9137
|
unsupportedFields
|
|
8817
9138
|
};
|
|
8818
9139
|
}
|
|
8819
|
-
if (!
|
|
9140
|
+
if (!isRecord11(root)) return void 0;
|
|
8820
9141
|
const rootFormat = root.format;
|
|
8821
9142
|
const rootCompatibility = root.compatibility;
|
|
8822
9143
|
const version = typeof root.version === "string" && root.version.trim() !== "" ? root.version : void 0;
|
|
@@ -8956,7 +9277,7 @@ function summarizeAttributeValue(value) {
|
|
|
8956
9277
|
if (Array.isArray(value)) {
|
|
8957
9278
|
return { type: "array", length: value.length };
|
|
8958
9279
|
}
|
|
8959
|
-
if (
|
|
9280
|
+
if (isRecord11(value)) {
|
|
8960
9281
|
return { type: "object", keyCount: Object.keys(value).length };
|
|
8961
9282
|
}
|
|
8962
9283
|
if (value === null) {
|
|
@@ -9043,7 +9364,7 @@ function mapOpenInferenceKind(span, attributes, pathPrefix) {
|
|
|
9043
9364
|
}
|
|
9044
9365
|
}
|
|
9045
9366
|
function mapOpenInferenceStatus(status) {
|
|
9046
|
-
if (!
|
|
9367
|
+
if (!isRecord11(status)) return void 0;
|
|
9047
9368
|
const rawCode = status.code;
|
|
9048
9369
|
if (typeof rawCode !== "string") return void 0;
|
|
9049
9370
|
switch (rawCode.toUpperCase()) {
|
|
@@ -9143,7 +9464,7 @@ function mapOpenInferenceSpan(span, index, version) {
|
|
|
9143
9464
|
warnings.push(...kindWarnings);
|
|
9144
9465
|
const status = mapOpenInferenceStatus(span.status);
|
|
9145
9466
|
const tokenUsage = readOpenInferenceTokenUsage(rawAttributes);
|
|
9146
|
-
const errorMessage =
|
|
9467
|
+
const errorMessage = isRecord11(span.status) && typeof span.status.message === "string" ? span.status.message : void 0;
|
|
9147
9468
|
const event = {
|
|
9148
9469
|
schemaVersion: "0.2",
|
|
9149
9470
|
eventId: typeof rawAttributes["agent_inspect.event_id"] === "string" ? rawAttributes["agent_inspect.event_id"] : spanId,
|
|
@@ -9289,7 +9610,7 @@ var openInferenceJsonReader = {
|
|
|
9289
9610
|
}
|
|
9290
9611
|
};
|
|
9291
9612
|
function parseOtlpAnyValue(value, field, warnings, unsupportedFields) {
|
|
9292
|
-
if (!
|
|
9613
|
+
if (!isRecord11(value)) {
|
|
9293
9614
|
unsupportedFields.push(field);
|
|
9294
9615
|
warnings.push({
|
|
9295
9616
|
code: "otlp_attribute_value_invalid",
|
|
@@ -9311,15 +9632,15 @@ function parseOtlpAnyValue(value, field, warnings, unsupportedFields) {
|
|
|
9311
9632
|
if (typeof value.doubleValue === "number" && Number.isFinite(value.doubleValue)) {
|
|
9312
9633
|
return value.doubleValue;
|
|
9313
9634
|
}
|
|
9314
|
-
if (
|
|
9635
|
+
if (isRecord11(value.arrayValue) && Array.isArray(value.arrayValue.values)) {
|
|
9315
9636
|
return value.arrayValue.values.map(
|
|
9316
9637
|
(item, index) => parseOtlpAnyValue(item, `${field}.arrayValue.values[${index}]`, warnings, unsupportedFields)
|
|
9317
9638
|
);
|
|
9318
9639
|
}
|
|
9319
|
-
if (
|
|
9640
|
+
if (isRecord11(value.kvlistValue) && Array.isArray(value.kvlistValue.values)) {
|
|
9320
9641
|
const out = {};
|
|
9321
9642
|
for (const [index, item] of value.kvlistValue.values.entries()) {
|
|
9322
|
-
if (!
|
|
9643
|
+
if (!isRecord11(item) || typeof item.key !== "string") {
|
|
9323
9644
|
unsupportedFields.push(`${field}.kvlistValue.values[${index}]`);
|
|
9324
9645
|
continue;
|
|
9325
9646
|
}
|
|
@@ -9370,7 +9691,7 @@ function parseOtlpAttributes(value, pathPrefix) {
|
|
|
9370
9691
|
}
|
|
9371
9692
|
for (const [index, item] of value.entries()) {
|
|
9372
9693
|
const field = `${pathPrefix}[${index}]`;
|
|
9373
|
-
if (!
|
|
9694
|
+
if (!isRecord11(item) || typeof item.key !== "string") {
|
|
9374
9695
|
unsupportedFields.push(field);
|
|
9375
9696
|
warnings.push({
|
|
9376
9697
|
code: "otlp_attribute_invalid",
|
|
@@ -9393,16 +9714,16 @@ function parseOtlpAttributes(value, pathPrefix) {
|
|
|
9393
9714
|
return { attributes, warnings, unsupportedFields };
|
|
9394
9715
|
}
|
|
9395
9716
|
function looksLikeOtlpSpan(value) {
|
|
9396
|
-
return
|
|
9717
|
+
return isRecord11(value) && readStringField(value, ["traceId"]) !== void 0 && readStringField(value, ["spanId"]) !== void 0 && readStringField(value, ["name"]) !== void 0;
|
|
9397
9718
|
}
|
|
9398
9719
|
function extractOtlpDocument(root) {
|
|
9399
|
-
if (!
|
|
9720
|
+
if (!isRecord11(root) || !Array.isArray(root.resourceSpans)) return void 0;
|
|
9400
9721
|
const spans = [];
|
|
9401
9722
|
const warnings = [];
|
|
9402
9723
|
const unsupportedFields = [];
|
|
9403
9724
|
for (const [resourceIndex, resourceSpan] of root.resourceSpans.entries()) {
|
|
9404
9725
|
const resourcePath = `resourceSpans[${resourceIndex}]`;
|
|
9405
|
-
if (!
|
|
9726
|
+
if (!isRecord11(resourceSpan)) {
|
|
9406
9727
|
unsupportedFields.push(resourcePath);
|
|
9407
9728
|
continue;
|
|
9408
9729
|
}
|
|
@@ -9425,7 +9746,7 @@ function extractOtlpDocument(root) {
|
|
|
9425
9746
|
}
|
|
9426
9747
|
for (const [scopeIndex, scopeSpan] of resourceSpan.scopeSpans.entries()) {
|
|
9427
9748
|
const scopePath = `${resourcePath}.scopeSpans[${scopeIndex}]`;
|
|
9428
|
-
if (!
|
|
9749
|
+
if (!isRecord11(scopeSpan)) {
|
|
9429
9750
|
unsupportedFields.push(scopePath);
|
|
9430
9751
|
continue;
|
|
9431
9752
|
}
|
|
@@ -9492,7 +9813,7 @@ function extractOtlpDocument(root) {
|
|
|
9492
9813
|
};
|
|
9493
9814
|
}
|
|
9494
9815
|
function mapOtlpStatus(status) {
|
|
9495
|
-
if (!
|
|
9816
|
+
if (!isRecord11(status)) return void 0;
|
|
9496
9817
|
const rawCode = status.code;
|
|
9497
9818
|
if (typeof rawCode !== "string") return void 0;
|
|
9498
9819
|
switch (rawCode.toUpperCase()) {
|
|
@@ -9592,7 +9913,7 @@ function mapOtlpEvents(value, pathPrefix) {
|
|
|
9592
9913
|
const events = [];
|
|
9593
9914
|
for (const [index, event] of value.entries()) {
|
|
9594
9915
|
const eventPath = `${pathPrefix}[${index}]`;
|
|
9595
|
-
if (!
|
|
9916
|
+
if (!isRecord11(event)) {
|
|
9596
9917
|
unsupportedFields.push(eventPath);
|
|
9597
9918
|
continue;
|
|
9598
9919
|
}
|
|
@@ -9730,7 +10051,7 @@ function mapOtlpSpan(context) {
|
|
|
9730
10051
|
warnings.push(...kindWarnings);
|
|
9731
10052
|
const status = mapOtlpStatus(span.status);
|
|
9732
10053
|
const tokenUsage = readOtlpTokenUsage(parsedSpanAttributes.attributes);
|
|
9733
|
-
const errorMessage =
|
|
10054
|
+
const errorMessage = isRecord11(span.status) && typeof span.status.message === "string" ? span.status.message : void 0;
|
|
9734
10055
|
const event = {
|
|
9735
10056
|
schemaVersion: "0.2",
|
|
9736
10057
|
eventId: typeof parsedSpanAttributes.attributes["agent_inspect.event_id"] === "string" ? parsedSpanAttributes.attributes["agent_inspect.event_id"] : spanId,
|
|
@@ -11420,5 +11741,5 @@ function renderGateReport(result, options = {}) {
|
|
|
11420
11741
|
}
|
|
11421
11742
|
|
|
11422
11743
|
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
|
|
11423
|
-
//# sourceMappingURL=chunk-
|
|
11424
|
-
//# sourceMappingURL=chunk-
|
|
11744
|
+
//# sourceMappingURL=chunk-R7Y5SGH5.mjs.map
|
|
11745
|
+
//# sourceMappingURL=chunk-R7Y5SGH5.mjs.map
|