agent-inspect 6.7.3 → 6.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +1 -1
- package/docs/CLI.md +4 -4
- package/docs/NESTJS.md +10 -0
- package/package.json +2 -2
- package/packages/cli/dist/{chunk-EMYDJREA.mjs → chunk-TO4VENHV.mjs} +228 -234
- package/packages/cli/dist/chunk-TO4VENHV.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +523 -1176
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +126 -44
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/src-Z5W27YCW.mjs +419 -0
- package/packages/cli/dist/src-Z5W27YCW.mjs.map +1 -0
- package/packages/core/dist/advanced.cjs +225 -199
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.mjs +26 -241
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/{chunk-X2FLDF7M.mjs → chunk-CTA6XNDP.mjs} +3 -3
- package/packages/core/dist/{chunk-X2FLDF7M.mjs.map → chunk-CTA6XNDP.mjs.map} +1 -1
- package/packages/core/dist/{chunk-3USJVBLA.mjs → chunk-KDDU2R5P.mjs} +236 -5
- package/packages/core/dist/chunk-KDDU2R5P.mjs.map +1 -0
- package/packages/core/dist/{chunk-E5F2LQCX.mjs → chunk-T5MASTIW.mjs} +15 -4
- package/packages/core/dist/chunk-T5MASTIW.mjs.map +1 -0
- package/packages/core/dist/{chunk-KNYL56KZ.mjs → chunk-UOPVGCOB.mjs} +3 -3
- package/packages/core/dist/{chunk-KNYL56KZ.mjs.map → chunk-UOPVGCOB.mjs.map} +1 -1
- package/packages/core/dist/exporters.cjs +136 -129
- package/packages/core/dist/exporters.cjs.map +1 -1
- package/packages/core/dist/exporters.mjs +1 -1
- package/packages/core/dist/logs.cjs +13 -2
- package/packages/core/dist/logs.cjs.map +1 -1
- package/packages/core/dist/logs.mjs +2 -2
- package/packages/core/dist/persisted.cjs +13 -2
- package/packages/core/dist/persisted.cjs.map +1 -1
- package/packages/core/dist/persisted.mjs +3 -3
- package/packages/core/dist/readers.cjs +13 -2
- package/packages/core/dist/readers.cjs.map +1 -1
- package/packages/core/dist/readers.mjs +3 -3
- package/packages/cli/dist/chunk-EMYDJREA.mjs.map +0 -1
- package/packages/cli/dist/src-IPJNFV37.mjs +0 -1160
- package/packages/cli/dist/src-IPJNFV37.mjs.map +0 -1
- package/packages/core/dist/chunk-3USJVBLA.mjs.map +0 -1
- package/packages/core/dist/chunk-E5F2LQCX.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.7.5
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 5c4197f: Consumer and DX reliability: doctor resolves packages via entry (not package.json exports); Studio/index bump better-sqlite3 to 12.11.1 with lazy native load; LangChain omits absolute traceDir from attrs; Jest diagnoses missing trace associations; CLI output/profile aliases; NestJS/LangGraph env-gated recipe.
|
|
8
|
+
|
|
9
|
+
## 6.7.4
|
|
10
|
+
|
|
11
|
+
### Patch Changes
|
|
12
|
+
|
|
13
|
+
- ab2ad83: Real-integration blocker patch: standalone LangGraph-shaped callback runs complete via active lifecycle; CLI shorthand check flags auto-select their rules; human tool display names; shared step labels and newest-first search; cross-command run-status golden; synthetic LangGraph fixtures; publish prior RUN-lifecycle and stats label fixes.
|
|
14
|
+
|
|
3
15
|
## 6.7.3
|
|
4
16
|
|
|
5
17
|
### Patch Changes
|
|
@@ -8,6 +20,15 @@
|
|
|
8
20
|
|
|
9
21
|
## Unreleased
|
|
10
22
|
|
|
23
|
+
### Documentation
|
|
24
|
+
|
|
25
|
+
- Activate the Stability and Focus roadmap (v6.7.3 → v6.12 → conditional v7): operational source-of-truth reconciliation, baseline audit, and release-train plans. No runtime or schema change in this docs activation.
|
|
26
|
+
|
|
27
|
+
### Patch candidates (unpublished on main; intended for 6.7.4)
|
|
28
|
+
|
|
29
|
+
- Completed-run status derived from terminal RUN lifecycle; explain uses shared user-facing status vocabulary (`8e525f1`).
|
|
30
|
+
- Stats no longer double-prefixes already-typed step names (`tool:tool:` / `llm:llm:`) (`ee49d4c`).
|
|
31
|
+
|
|
11
32
|
## 6.7.2
|
|
12
33
|
|
|
13
34
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -131,7 +131,7 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
131
131
|
|
|
132
132
|
## Project status
|
|
133
133
|
|
|
134
|
-
**Current release:** **6.7.
|
|
134
|
+
**Current release:** **6.7.4** (eighteen linked npm packages). Technical launch candidate; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
135
135
|
|
|
136
136
|
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
137
137
|
|
package/docs/CLI.md
CHANGED
|
@@ -218,13 +218,13 @@ Options:
|
|
|
218
218
|
|
|
219
219
|
- `--dir <path>`
|
|
220
220
|
- `--format <format>`
|
|
221
|
-
- `-o, --output <path>`: write file
|
|
221
|
+
- `-o, --output <path>`: write file (canonical; `--out` is a legacy alias)
|
|
222
222
|
- `--json`: JSON wrapper output (includes content if not writing to file)
|
|
223
223
|
- `--validate`: validate exported payload shape
|
|
224
224
|
- `--include-attributes`: include bounded attributes (review before sharing)
|
|
225
225
|
- `--no-metadata`: omit summary/metadata sections
|
|
226
226
|
- `--no-errors`: omit error sections
|
|
227
|
-
- `--redaction-profile <profile>`: redaction profile for exported copies — `local` (default), `share`, or `strict`. Key-based safety only; review exports before sharing.
|
|
227
|
+
- `--redaction-profile <profile>`: redaction profile for exported copies — `local` (default), `share`, or `strict`. Key-based safety only; review exports before sharing. (`--profile` is a legacy alias.)
|
|
228
228
|
|
|
229
229
|
Examples:
|
|
230
230
|
|
|
@@ -879,8 +879,8 @@ Options:
|
|
|
879
879
|
- `--dir <path>` — trace directory
|
|
880
880
|
- `--session <session-id>` — bundle all runs in a session
|
|
881
881
|
- `--since <duration>` — bundle runs with activity since a window (e.g. `24h`, `7d`)
|
|
882
|
-
- `--profile <profile>` — `local`, `share` (default), or `strict` redaction for exported copies
|
|
883
|
-
- `--out <path>` — output directory; `.zip` suffix is stripped (folder-first)
|
|
882
|
+
- `--profile <profile>` — `local`, `share` (default), or `strict` redaction for exported copies (`--redaction-profile` is the canonical alias)
|
|
883
|
+
- `--out <path>` — output directory; `.zip` suffix is stripped (folder-first; `--output` is the canonical alias)
|
|
884
884
|
- `--allow-unsafe` — write bundle even when verify-safe reports UNSAFE
|
|
885
885
|
- `--json` — print deterministic JSON manifest
|
|
886
886
|
|
package/docs/NESTJS.md
CHANGED
|
@@ -40,7 +40,17 @@ await createFixtureRunner({
|
|
|
40
40
|
|
|
41
41
|
If you only need log ingestion, see [examples/recipes/nestjs-json-logging](../examples/recipes/nestjs-json-logging/).
|
|
42
42
|
|
|
43
|
+
## LangGraph agent callbacks (env-gated)
|
|
44
|
+
|
|
45
|
+
For Nest services that invoke LangGraph/LangChain with a `callbacks` array, prefer the
|
|
46
|
+
blessed recipe [examples/recipes/nestjs-langgraph-local](../examples/recipes/nestjs-langgraph-local/):
|
|
47
|
+
|
|
48
|
+
- `AGENT_INSPECT` gate + lazy dynamic import (no adapter load when disabled)
|
|
49
|
+
- metadata-only capture and workspace-relative `traceDir`
|
|
50
|
+
- no production-path change when the helper returns `[]`
|
|
51
|
+
|
|
43
52
|
## Related
|
|
44
53
|
|
|
45
54
|
- [`@agent-inspect/harness`](../packages/harness/README.md)
|
|
46
55
|
- [examples/starters/harness-nestjs](../examples/starters/harness-nestjs/)
|
|
56
|
+
- [examples/recipes/nestjs-langgraph-local](../examples/recipes/nestjs-langgraph-local/)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.7.
|
|
3
|
+
"version": "6.7.5",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Debug, regression-test, and safely share TypeScript AI-agent behavior locally — no account, no default upload, metadata-only by default",
|
|
@@ -223,7 +223,7 @@
|
|
|
223
223
|
"test:watch": "vitest",
|
|
224
224
|
"test:coverage": "vitest run --coverage",
|
|
225
225
|
"size": "size-limit --config size-limit.config.mjs",
|
|
226
|
-
"test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run
|
|
226
|
+
"test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run build && pnpm run test && pnpm run size",
|
|
227
227
|
"prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run pack:smoke",
|
|
228
228
|
"pack:dry-run": "pnpm run build && npm pack --dry-run",
|
|
229
229
|
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-quickstart-e2e.mjs",
|
|
@@ -9,38 +9,6 @@ import { createReadStream } from 'fs';
|
|
|
9
9
|
import { createInterface } from 'readline';
|
|
10
10
|
import { pathToFileURL } from 'url';
|
|
11
11
|
|
|
12
|
-
var __create = Object.create;
|
|
13
|
-
var __defProp = Object.defineProperty;
|
|
14
|
-
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
|
|
15
|
-
var __getOwnPropNames = Object.getOwnPropertyNames;
|
|
16
|
-
var __getProtoOf = Object.getPrototypeOf;
|
|
17
|
-
var __hasOwnProp = Object.prototype.hasOwnProperty;
|
|
18
|
-
var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
|
|
19
|
-
get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
|
|
20
|
-
}) : x)(function(x) {
|
|
21
|
-
if (typeof require !== "undefined") return require.apply(this, arguments);
|
|
22
|
-
throw Error('Dynamic require of "' + x + '" is not supported');
|
|
23
|
-
});
|
|
24
|
-
var __commonJS = (cb, mod) => function __require2() {
|
|
25
|
-
return mod || (0, cb[__getOwnPropNames(cb)[0]])((mod = { exports: {} }).exports, mod), mod.exports;
|
|
26
|
-
};
|
|
27
|
-
var __copyProps = (to, from, except, desc) => {
|
|
28
|
-
if (from && typeof from === "object" || typeof from === "function") {
|
|
29
|
-
for (let key of __getOwnPropNames(from))
|
|
30
|
-
if (!__hasOwnProp.call(to, key) && key !== except)
|
|
31
|
-
__defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
|
|
32
|
-
}
|
|
33
|
-
return to;
|
|
34
|
-
};
|
|
35
|
-
var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__getProtoOf(mod)) : {}, __copyProps(
|
|
36
|
-
// If the importer is in node compatibility mode or this is not an ESM
|
|
37
|
-
// file that has been converted to a CommonJS file using a Babel-
|
|
38
|
-
// compatible transform (i.e. "__esModule" has not been set), then set
|
|
39
|
-
// "default" to the CommonJS "module.exports" for node compatibility.
|
|
40
|
-
isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target,
|
|
41
|
-
mod
|
|
42
|
-
));
|
|
43
|
-
|
|
44
12
|
// packages/core/src/types/persisted-inspect-event.ts
|
|
45
13
|
var INSPECT_KINDS = [
|
|
46
14
|
"RUN",
|
|
@@ -1904,6 +1872,202 @@ function filterTraces(traces, options) {
|
|
|
1904
1872
|
return out;
|
|
1905
1873
|
}
|
|
1906
1874
|
|
|
1875
|
+
// packages/core/src/stats.ts
|
|
1876
|
+
function percentile(sorted, p) {
|
|
1877
|
+
if (sorted.length === 0) return void 0;
|
|
1878
|
+
const idx = Math.min(
|
|
1879
|
+
sorted.length - 1,
|
|
1880
|
+
Math.max(0, Math.ceil(p / 100 * sorted.length) - 1)
|
|
1881
|
+
);
|
|
1882
|
+
return sorted[idx];
|
|
1883
|
+
}
|
|
1884
|
+
async function readRunStartedMetadata(filePath) {
|
|
1885
|
+
try {
|
|
1886
|
+
const events = await readTraceEventsFromFile(filePath);
|
|
1887
|
+
for (const event of events) {
|
|
1888
|
+
if (event.event !== "run_started") continue;
|
|
1889
|
+
const rs = event;
|
|
1890
|
+
if (rs.metadata && typeof rs.metadata === "object") {
|
|
1891
|
+
return rs.metadata;
|
|
1892
|
+
}
|
|
1893
|
+
return void 0;
|
|
1894
|
+
}
|
|
1895
|
+
} catch {
|
|
1896
|
+
}
|
|
1897
|
+
return void 0;
|
|
1898
|
+
}
|
|
1899
|
+
function metaMatchesCorrelation(metadata, correlationId, groupId) {
|
|
1900
|
+
if (correlationId) {
|
|
1901
|
+
const v = metadata?.correlationId;
|
|
1902
|
+
if (typeof v !== "string" || v !== correlationId) return false;
|
|
1903
|
+
}
|
|
1904
|
+
if (groupId) {
|
|
1905
|
+
const v = metadata?.groupId;
|
|
1906
|
+
if (typeof v !== "string" || v !== groupId) return false;
|
|
1907
|
+
}
|
|
1908
|
+
return true;
|
|
1909
|
+
}
|
|
1910
|
+
async function buildTraceStats(metas, options) {
|
|
1911
|
+
let filtered = filterTraces(metas, { since: options.since });
|
|
1912
|
+
if (options.correlationId || options.groupId) {
|
|
1913
|
+
const next = [];
|
|
1914
|
+
for (const m of filtered) {
|
|
1915
|
+
const md = await readRunStartedMetadata(m.filePath);
|
|
1916
|
+
if (metaMatchesCorrelation(md, options.correlationId, options.groupId)) {
|
|
1917
|
+
next.push(m);
|
|
1918
|
+
}
|
|
1919
|
+
}
|
|
1920
|
+
filtered = next;
|
|
1921
|
+
}
|
|
1922
|
+
let successCount = 0;
|
|
1923
|
+
let errorCount = 0;
|
|
1924
|
+
let runningCount = 0;
|
|
1925
|
+
let unknownCount = 0;
|
|
1926
|
+
const durations = [];
|
|
1927
|
+
let totalSteps = 0;
|
|
1928
|
+
let totalLlmSteps = 0;
|
|
1929
|
+
let totalToolSteps = 0;
|
|
1930
|
+
let totalErrorSteps = 0;
|
|
1931
|
+
const slowestRuns = [];
|
|
1932
|
+
const slowestSteps = [];
|
|
1933
|
+
for (const m of filtered) {
|
|
1934
|
+
if (m.status === "success") successCount += 1;
|
|
1935
|
+
else if (m.status === "error") errorCount += 1;
|
|
1936
|
+
else if (m.status === "running") runningCount += 1;
|
|
1937
|
+
else unknownCount += 1;
|
|
1938
|
+
if (typeof m.durationMs === "number" && Number.isFinite(m.durationMs) && m.durationMs >= 0) {
|
|
1939
|
+
durations.push(m.durationMs);
|
|
1940
|
+
slowestRuns.push({
|
|
1941
|
+
runId: m.runId,
|
|
1942
|
+
name: m.name,
|
|
1943
|
+
durationMs: m.durationMs,
|
|
1944
|
+
status: m.status
|
|
1945
|
+
});
|
|
1946
|
+
}
|
|
1947
|
+
try {
|
|
1948
|
+
const events = await readTraceEventsFromFile(m.filePath);
|
|
1949
|
+
if (events.length === 0) continue;
|
|
1950
|
+
const summary = buildRunSummary(events);
|
|
1951
|
+
totalSteps += summary.totalSteps;
|
|
1952
|
+
totalLlmSteps += summary.llmSteps;
|
|
1953
|
+
totalToolSteps += summary.toolSteps;
|
|
1954
|
+
totalErrorSteps += summary.errorSteps;
|
|
1955
|
+
const steps = collectCompletedSteps(events, m.runId);
|
|
1956
|
+
for (const s of steps) {
|
|
1957
|
+
slowestSteps.push(s);
|
|
1958
|
+
}
|
|
1959
|
+
} catch {
|
|
1960
|
+
}
|
|
1961
|
+
}
|
|
1962
|
+
slowestRuns.sort((a, b) => (b.durationMs ?? 0) - (a.durationMs ?? 0));
|
|
1963
|
+
slowestSteps.sort((a, b) => b.durationMs - a.durationMs);
|
|
1964
|
+
const runLimit = options.slowRunLimit ?? 5;
|
|
1965
|
+
const stepLimit = options.slowStepLimit ?? 5;
|
|
1966
|
+
const sortedDur = [...durations].sort((a, b) => a - b);
|
|
1967
|
+
const totalRuns = filtered.length;
|
|
1968
|
+
const errorRate = totalRuns > 0 ? errorCount / totalRuns : 0;
|
|
1969
|
+
const sumDur = durations.reduce((a, b) => a + b, 0);
|
|
1970
|
+
return {
|
|
1971
|
+
traceDir: options.traceDir,
|
|
1972
|
+
...options.since ? { since: options.since } : {},
|
|
1973
|
+
...options.correlationId ? { correlationId: options.correlationId } : {},
|
|
1974
|
+
...options.groupId ? { groupId: options.groupId } : {},
|
|
1975
|
+
totalRuns,
|
|
1976
|
+
successCount,
|
|
1977
|
+
errorCount,
|
|
1978
|
+
runningCount,
|
|
1979
|
+
unknownCount,
|
|
1980
|
+
errorRate,
|
|
1981
|
+
duration: {
|
|
1982
|
+
...sortedDur.length > 0 ? {
|
|
1983
|
+
minMs: sortedDur[0],
|
|
1984
|
+
maxMs: sortedDur[sortedDur.length - 1],
|
|
1985
|
+
avgMs: sumDur / sortedDur.length,
|
|
1986
|
+
p50Ms: percentile(sortedDur, 50),
|
|
1987
|
+
p95Ms: percentile(sortedDur, 95)
|
|
1988
|
+
} : {}
|
|
1989
|
+
},
|
|
1990
|
+
totalSteps,
|
|
1991
|
+
avgStepsPerRun: totalRuns > 0 ? totalSteps / totalRuns : 0,
|
|
1992
|
+
totalLlmSteps,
|
|
1993
|
+
totalToolSteps,
|
|
1994
|
+
totalErrorSteps,
|
|
1995
|
+
slowestRuns: slowestRuns.slice(0, runLimit),
|
|
1996
|
+
slowestSteps: slowestSteps.slice(0, stepLimit)
|
|
1997
|
+
};
|
|
1998
|
+
}
|
|
1999
|
+
function collectCompletedSteps(events, runId) {
|
|
2000
|
+
const started = /* @__PURE__ */ new Map();
|
|
2001
|
+
const out = [];
|
|
2002
|
+
for (const e of events) {
|
|
2003
|
+
if (e.event === "step_started") {
|
|
2004
|
+
const s = e;
|
|
2005
|
+
started.set(s.stepId, { name: s.name, type: s.type });
|
|
2006
|
+
}
|
|
2007
|
+
if (e.event === "step_completed") {
|
|
2008
|
+
const c = e;
|
|
2009
|
+
if (c.status !== "success" && c.status !== "error") continue;
|
|
2010
|
+
if (typeof c.durationMs !== "number" || !Number.isFinite(c.durationMs)) {
|
|
2011
|
+
continue;
|
|
2012
|
+
}
|
|
2013
|
+
const meta = started.get(c.stepId);
|
|
2014
|
+
out.push({
|
|
2015
|
+
runId,
|
|
2016
|
+
stepName: meta?.name ?? c.stepId,
|
|
2017
|
+
stepType: meta?.type ?? "logic",
|
|
2018
|
+
durationMs: c.durationMs
|
|
2019
|
+
});
|
|
2020
|
+
}
|
|
2021
|
+
}
|
|
2022
|
+
return out;
|
|
2023
|
+
}
|
|
2024
|
+
function formatStepLabel(stepType, stepName) {
|
|
2025
|
+
return stepName.startsWith(`${stepType}:`) ? stepName : `${stepType}:${stepName}`;
|
|
2026
|
+
}
|
|
2027
|
+
function renderTraceStats(stats) {
|
|
2028
|
+
const lines = [];
|
|
2029
|
+
lines.push("Trace stats (local)");
|
|
2030
|
+
lines.push(`Directory: ${stats.traceDir}`);
|
|
2031
|
+
if (stats.since) lines.push(`Since: ${stats.since}`);
|
|
2032
|
+
if (stats.correlationId) lines.push(`Correlation ID: ${stats.correlationId}`);
|
|
2033
|
+
if (stats.groupId) lines.push(`Group ID: ${stats.groupId}`);
|
|
2034
|
+
lines.push("");
|
|
2035
|
+
lines.push(`Runs: ${stats.totalRuns}`);
|
|
2036
|
+
lines.push(
|
|
2037
|
+
` success: ${stats.successCount} error: ${stats.errorCount} running: ${stats.runningCount} unknown: ${stats.unknownCount}`
|
|
2038
|
+
);
|
|
2039
|
+
lines.push(`Error rate: ${(stats.errorRate * 100).toFixed(1)}%`);
|
|
2040
|
+
if (stats.duration.avgMs !== void 0) {
|
|
2041
|
+
lines.push(
|
|
2042
|
+
`Duration: min ${formatDuration2(stats.duration.minMs ?? 0)} | avg ${formatDuration2(stats.duration.avgMs)} | p50 ${formatDuration2(stats.duration.p50Ms ?? 0)} | p95 ${formatDuration2(stats.duration.p95Ms ?? 0)} | max ${formatDuration2(stats.duration.maxMs ?? 0)}`
|
|
2043
|
+
);
|
|
2044
|
+
}
|
|
2045
|
+
lines.push("");
|
|
2046
|
+
lines.push(`Steps: ${stats.totalSteps} (avg ${stats.avgStepsPerRun.toFixed(1)} per run)`);
|
|
2047
|
+
lines.push(
|
|
2048
|
+
` LLM: ${stats.totalLlmSteps} tool: ${stats.totalToolSteps} errors: ${stats.totalErrorSteps}`
|
|
2049
|
+
);
|
|
2050
|
+
if (stats.slowestRuns.length > 0) {
|
|
2051
|
+
lines.push("");
|
|
2052
|
+
lines.push("Slowest runs:");
|
|
2053
|
+
for (const r of stats.slowestRuns) {
|
|
2054
|
+
lines.push(
|
|
2055
|
+
` ${r.runId} | ${r.name ?? "-"} | ${formatDuration2(r.durationMs ?? 0)} | ${r.status}`
|
|
2056
|
+
);
|
|
2057
|
+
}
|
|
2058
|
+
}
|
|
2059
|
+
if (stats.slowestSteps.length > 0) {
|
|
2060
|
+
lines.push("");
|
|
2061
|
+
lines.push("Slowest steps:");
|
|
2062
|
+
for (const s of stats.slowestSteps) {
|
|
2063
|
+
lines.push(
|
|
2064
|
+
` ${s.runId} | ${formatStepLabel(s.stepType, s.stepName)} | ${formatDuration2(s.durationMs)}`
|
|
2065
|
+
);
|
|
2066
|
+
}
|
|
2067
|
+
}
|
|
2068
|
+
return lines.join("\n");
|
|
2069
|
+
}
|
|
2070
|
+
|
|
1907
2071
|
// packages/core/src/timeline.ts
|
|
1908
2072
|
function finite(n) {
|
|
1909
2073
|
return typeof n === "number" && Number.isFinite(n);
|
|
@@ -2061,7 +2225,7 @@ function renderTimeline(timeline, options = {}) {
|
|
|
2061
2225
|
const dur = e.durationMs !== void 0 ? formatDuration2(e.durationMs) : "-";
|
|
2062
2226
|
const err = e.isError ? " error" : "";
|
|
2063
2227
|
const off = formatDuration2(e.offsetMs);
|
|
2064
|
-
let line = `${prefix}+${off} ${typeTag
|
|
2228
|
+
let line = `${prefix}+${off} ${formatStepLabel(typeTag, e.name)} (${dur})${err}`;
|
|
2065
2229
|
if (e.streaming?.chunkCount !== void 0) {
|
|
2066
2230
|
line += ` chunks=${e.streaming.chunkCount}`;
|
|
2067
2231
|
}
|
|
@@ -2213,6 +2377,12 @@ function renderRunWhat(summary, options = {}) {
|
|
|
2213
2377
|
}
|
|
2214
2378
|
|
|
2215
2379
|
// packages/core/src/explain.ts
|
|
2380
|
+
function toDisplayStatus(status) {
|
|
2381
|
+
if (status === "ok") return "success";
|
|
2382
|
+
if (status === "error") return "error";
|
|
2383
|
+
if (status === "running") return "running";
|
|
2384
|
+
return "unknown";
|
|
2385
|
+
}
|
|
2216
2386
|
function flatten(nodes, out = []) {
|
|
2217
2387
|
for (const node of nodes) {
|
|
2218
2388
|
out.push({ node, index: out.length + 1 });
|
|
@@ -2270,7 +2440,7 @@ function buildFacts(run, redactor) {
|
|
|
2270
2440
|
const facts = [
|
|
2271
2441
|
fact("run.id", "Run id", run.runId, redactor),
|
|
2272
2442
|
fact("run.name", "Run name", run.name ?? run.runId, redactor),
|
|
2273
|
-
fact("run.status", "Run status", run.status
|
|
2443
|
+
fact("run.status", "Run status", toDisplayStatus(run.status), redactor),
|
|
2274
2444
|
fact("run.totalEvents", "Total events", run.metadata.totalEvents, redactor),
|
|
2275
2445
|
fact("run.stepCount", "Top-level step count", run.children.length, redactor),
|
|
2276
2446
|
fact("run.nodeCount", "Total node count", nodes.length, redactor),
|
|
@@ -2346,7 +2516,7 @@ function buildLocalExplanation(run, options = {}) {
|
|
|
2346
2516
|
mode,
|
|
2347
2517
|
runId: String(redactValue(redactor, "runId", run.runId)),
|
|
2348
2518
|
...run.name !== void 0 ? { name: String(redactValue(redactor, "name", run.name)) } : {},
|
|
2349
|
-
...run.status !== void 0 ? { status: run.status } : {},
|
|
2519
|
+
...run.status !== void 0 ? { status: toDisplayStatus(run.status) } : {},
|
|
2350
2520
|
redactionProfile,
|
|
2351
2521
|
facts,
|
|
2352
2522
|
inferences: mode === "dry-run" ? [] : buildInferences(run, facts),
|
|
@@ -2357,199 +2527,6 @@ function buildLocalExplanation(run, options = {}) {
|
|
|
2357
2527
|
};
|
|
2358
2528
|
}
|
|
2359
2529
|
|
|
2360
|
-
// packages/core/src/stats.ts
|
|
2361
|
-
function percentile(sorted, p) {
|
|
2362
|
-
if (sorted.length === 0) return void 0;
|
|
2363
|
-
const idx = Math.min(
|
|
2364
|
-
sorted.length - 1,
|
|
2365
|
-
Math.max(0, Math.ceil(p / 100 * sorted.length) - 1)
|
|
2366
|
-
);
|
|
2367
|
-
return sorted[idx];
|
|
2368
|
-
}
|
|
2369
|
-
async function readRunStartedMetadata(filePath) {
|
|
2370
|
-
try {
|
|
2371
|
-
const events = await readTraceEventsFromFile(filePath);
|
|
2372
|
-
for (const event of events) {
|
|
2373
|
-
if (event.event !== "run_started") continue;
|
|
2374
|
-
const rs = event;
|
|
2375
|
-
if (rs.metadata && typeof rs.metadata === "object") {
|
|
2376
|
-
return rs.metadata;
|
|
2377
|
-
}
|
|
2378
|
-
return void 0;
|
|
2379
|
-
}
|
|
2380
|
-
} catch {
|
|
2381
|
-
}
|
|
2382
|
-
return void 0;
|
|
2383
|
-
}
|
|
2384
|
-
function metaMatchesCorrelation(metadata, correlationId, groupId) {
|
|
2385
|
-
if (correlationId) {
|
|
2386
|
-
const v = metadata?.correlationId;
|
|
2387
|
-
if (typeof v !== "string" || v !== correlationId) return false;
|
|
2388
|
-
}
|
|
2389
|
-
if (groupId) {
|
|
2390
|
-
const v = metadata?.groupId;
|
|
2391
|
-
if (typeof v !== "string" || v !== groupId) return false;
|
|
2392
|
-
}
|
|
2393
|
-
return true;
|
|
2394
|
-
}
|
|
2395
|
-
async function buildTraceStats(metas, options) {
|
|
2396
|
-
let filtered = filterTraces(metas, { since: options.since });
|
|
2397
|
-
if (options.correlationId || options.groupId) {
|
|
2398
|
-
const next = [];
|
|
2399
|
-
for (const m of filtered) {
|
|
2400
|
-
const md = await readRunStartedMetadata(m.filePath);
|
|
2401
|
-
if (metaMatchesCorrelation(md, options.correlationId, options.groupId)) {
|
|
2402
|
-
next.push(m);
|
|
2403
|
-
}
|
|
2404
|
-
}
|
|
2405
|
-
filtered = next;
|
|
2406
|
-
}
|
|
2407
|
-
let successCount = 0;
|
|
2408
|
-
let errorCount = 0;
|
|
2409
|
-
let runningCount = 0;
|
|
2410
|
-
let unknownCount = 0;
|
|
2411
|
-
const durations = [];
|
|
2412
|
-
let totalSteps = 0;
|
|
2413
|
-
let totalLlmSteps = 0;
|
|
2414
|
-
let totalToolSteps = 0;
|
|
2415
|
-
let totalErrorSteps = 0;
|
|
2416
|
-
const slowestRuns = [];
|
|
2417
|
-
const slowestSteps = [];
|
|
2418
|
-
for (const m of filtered) {
|
|
2419
|
-
if (m.status === "success") successCount += 1;
|
|
2420
|
-
else if (m.status === "error") errorCount += 1;
|
|
2421
|
-
else if (m.status === "running") runningCount += 1;
|
|
2422
|
-
else unknownCount += 1;
|
|
2423
|
-
if (typeof m.durationMs === "number" && Number.isFinite(m.durationMs) && m.durationMs >= 0) {
|
|
2424
|
-
durations.push(m.durationMs);
|
|
2425
|
-
slowestRuns.push({
|
|
2426
|
-
runId: m.runId,
|
|
2427
|
-
name: m.name,
|
|
2428
|
-
durationMs: m.durationMs,
|
|
2429
|
-
status: m.status
|
|
2430
|
-
});
|
|
2431
|
-
}
|
|
2432
|
-
try {
|
|
2433
|
-
const events = await readTraceEventsFromFile(m.filePath);
|
|
2434
|
-
if (events.length === 0) continue;
|
|
2435
|
-
const summary = buildRunSummary(events);
|
|
2436
|
-
totalSteps += summary.totalSteps;
|
|
2437
|
-
totalLlmSteps += summary.llmSteps;
|
|
2438
|
-
totalToolSteps += summary.toolSteps;
|
|
2439
|
-
totalErrorSteps += summary.errorSteps;
|
|
2440
|
-
const steps = collectCompletedSteps(events, m.runId);
|
|
2441
|
-
for (const s of steps) {
|
|
2442
|
-
slowestSteps.push(s);
|
|
2443
|
-
}
|
|
2444
|
-
} catch {
|
|
2445
|
-
}
|
|
2446
|
-
}
|
|
2447
|
-
slowestRuns.sort((a, b) => (b.durationMs ?? 0) - (a.durationMs ?? 0));
|
|
2448
|
-
slowestSteps.sort((a, b) => b.durationMs - a.durationMs);
|
|
2449
|
-
const runLimit = options.slowRunLimit ?? 5;
|
|
2450
|
-
const stepLimit = options.slowStepLimit ?? 5;
|
|
2451
|
-
const sortedDur = [...durations].sort((a, b) => a - b);
|
|
2452
|
-
const totalRuns = filtered.length;
|
|
2453
|
-
const errorRate = totalRuns > 0 ? errorCount / totalRuns : 0;
|
|
2454
|
-
const sumDur = durations.reduce((a, b) => a + b, 0);
|
|
2455
|
-
return {
|
|
2456
|
-
traceDir: options.traceDir,
|
|
2457
|
-
...options.since ? { since: options.since } : {},
|
|
2458
|
-
...options.correlationId ? { correlationId: options.correlationId } : {},
|
|
2459
|
-
...options.groupId ? { groupId: options.groupId } : {},
|
|
2460
|
-
totalRuns,
|
|
2461
|
-
successCount,
|
|
2462
|
-
errorCount,
|
|
2463
|
-
runningCount,
|
|
2464
|
-
unknownCount,
|
|
2465
|
-
errorRate,
|
|
2466
|
-
duration: {
|
|
2467
|
-
...sortedDur.length > 0 ? {
|
|
2468
|
-
minMs: sortedDur[0],
|
|
2469
|
-
maxMs: sortedDur[sortedDur.length - 1],
|
|
2470
|
-
avgMs: sumDur / sortedDur.length,
|
|
2471
|
-
p50Ms: percentile(sortedDur, 50),
|
|
2472
|
-
p95Ms: percentile(sortedDur, 95)
|
|
2473
|
-
} : {}
|
|
2474
|
-
},
|
|
2475
|
-
totalSteps,
|
|
2476
|
-
avgStepsPerRun: totalRuns > 0 ? totalSteps / totalRuns : 0,
|
|
2477
|
-
totalLlmSteps,
|
|
2478
|
-
totalToolSteps,
|
|
2479
|
-
totalErrorSteps,
|
|
2480
|
-
slowestRuns: slowestRuns.slice(0, runLimit),
|
|
2481
|
-
slowestSteps: slowestSteps.slice(0, stepLimit)
|
|
2482
|
-
};
|
|
2483
|
-
}
|
|
2484
|
-
function collectCompletedSteps(events, runId) {
|
|
2485
|
-
const started = /* @__PURE__ */ new Map();
|
|
2486
|
-
const out = [];
|
|
2487
|
-
for (const e of events) {
|
|
2488
|
-
if (e.event === "step_started") {
|
|
2489
|
-
const s = e;
|
|
2490
|
-
started.set(s.stepId, { name: s.name, type: s.type });
|
|
2491
|
-
}
|
|
2492
|
-
if (e.event === "step_completed") {
|
|
2493
|
-
const c = e;
|
|
2494
|
-
if (c.status !== "success" && c.status !== "error") continue;
|
|
2495
|
-
if (typeof c.durationMs !== "number" || !Number.isFinite(c.durationMs)) {
|
|
2496
|
-
continue;
|
|
2497
|
-
}
|
|
2498
|
-
const meta = started.get(c.stepId);
|
|
2499
|
-
out.push({
|
|
2500
|
-
runId,
|
|
2501
|
-
stepName: meta?.name ?? c.stepId,
|
|
2502
|
-
stepType: meta?.type ?? "logic",
|
|
2503
|
-
durationMs: c.durationMs
|
|
2504
|
-
});
|
|
2505
|
-
}
|
|
2506
|
-
}
|
|
2507
|
-
return out;
|
|
2508
|
-
}
|
|
2509
|
-
function renderTraceStats(stats) {
|
|
2510
|
-
const lines = [];
|
|
2511
|
-
lines.push("Trace stats (local)");
|
|
2512
|
-
lines.push(`Directory: ${stats.traceDir}`);
|
|
2513
|
-
if (stats.since) lines.push(`Since: ${stats.since}`);
|
|
2514
|
-
if (stats.correlationId) lines.push(`Correlation ID: ${stats.correlationId}`);
|
|
2515
|
-
if (stats.groupId) lines.push(`Group ID: ${stats.groupId}`);
|
|
2516
|
-
lines.push("");
|
|
2517
|
-
lines.push(`Runs: ${stats.totalRuns}`);
|
|
2518
|
-
lines.push(
|
|
2519
|
-
` success: ${stats.successCount} error: ${stats.errorCount} running: ${stats.runningCount} unknown: ${stats.unknownCount}`
|
|
2520
|
-
);
|
|
2521
|
-
lines.push(`Error rate: ${(stats.errorRate * 100).toFixed(1)}%`);
|
|
2522
|
-
if (stats.duration.avgMs !== void 0) {
|
|
2523
|
-
lines.push(
|
|
2524
|
-
`Duration: min ${formatDuration2(stats.duration.minMs ?? 0)} | avg ${formatDuration2(stats.duration.avgMs)} | p50 ${formatDuration2(stats.duration.p50Ms ?? 0)} | p95 ${formatDuration2(stats.duration.p95Ms ?? 0)} | max ${formatDuration2(stats.duration.maxMs ?? 0)}`
|
|
2525
|
-
);
|
|
2526
|
-
}
|
|
2527
|
-
lines.push("");
|
|
2528
|
-
lines.push(`Steps: ${stats.totalSteps} (avg ${stats.avgStepsPerRun.toFixed(1)} per run)`);
|
|
2529
|
-
lines.push(
|
|
2530
|
-
` LLM: ${stats.totalLlmSteps} tool: ${stats.totalToolSteps} errors: ${stats.totalErrorSteps}`
|
|
2531
|
-
);
|
|
2532
|
-
if (stats.slowestRuns.length > 0) {
|
|
2533
|
-
lines.push("");
|
|
2534
|
-
lines.push("Slowest runs:");
|
|
2535
|
-
for (const r of stats.slowestRuns) {
|
|
2536
|
-
lines.push(
|
|
2537
|
-
` ${r.runId} | ${r.name ?? "-"} | ${formatDuration2(r.durationMs ?? 0)} | ${r.status}`
|
|
2538
|
-
);
|
|
2539
|
-
}
|
|
2540
|
-
}
|
|
2541
|
-
if (stats.slowestSteps.length > 0) {
|
|
2542
|
-
lines.push("");
|
|
2543
|
-
lines.push("Slowest steps:");
|
|
2544
|
-
for (const s of stats.slowestSteps) {
|
|
2545
|
-
lines.push(
|
|
2546
|
-
` ${s.runId} | ${s.stepType}:${s.stepName} | ${formatDuration2(s.durationMs)}`
|
|
2547
|
-
);
|
|
2548
|
-
}
|
|
2549
|
-
}
|
|
2550
|
-
return lines.join("\n");
|
|
2551
|
-
}
|
|
2552
|
-
|
|
2553
2530
|
// packages/core/src/outcomes/types.ts
|
|
2554
2531
|
var OBSERVED_OUTCOME_STATUSES = [
|
|
2555
2532
|
"passed",
|
|
@@ -2752,6 +2729,12 @@ async function searchTraces(metas, options) {
|
|
|
2752
2729
|
...sessionLabel ? { sessionId: sessionLabel } : {}
|
|
2753
2730
|
});
|
|
2754
2731
|
}
|
|
2732
|
+
results.sort((a, b) => {
|
|
2733
|
+
const ta = a.timestamp ?? 0;
|
|
2734
|
+
const tb = b.timestamp ?? 0;
|
|
2735
|
+
if (ta !== tb) return tb - ta;
|
|
2736
|
+
return a.runId.localeCompare(b.runId);
|
|
2737
|
+
});
|
|
2755
2738
|
return results.slice(0, limit);
|
|
2756
2739
|
}
|
|
2757
2740
|
for (const m of filtered) {
|
|
@@ -2799,7 +2782,7 @@ async function searchTraces(metas, options) {
|
|
|
2799
2782
|
results.sort((a, b) => {
|
|
2800
2783
|
const ta = a.timestamp ?? 0;
|
|
2801
2784
|
const tb = b.timestamp ?? 0;
|
|
2802
|
-
if (ta !== tb) return
|
|
2785
|
+
if (ta !== tb) return tb - ta;
|
|
2803
2786
|
const runCmp = a.runId.localeCompare(b.runId);
|
|
2804
2787
|
if (runCmp !== 0) return runCmp;
|
|
2805
2788
|
return (a.stepName ?? "").localeCompare(b.stepName ?? "");
|
|
@@ -5814,13 +5797,24 @@ function inc(map, key) {
|
|
|
5814
5797
|
map[key] = (map[key] ?? 0) + 1;
|
|
5815
5798
|
}
|
|
5816
5799
|
function computeRunStatus(events) {
|
|
5800
|
+
let runTerminal;
|
|
5801
|
+
let sawRunEvent = false;
|
|
5802
|
+
for (const e of events) {
|
|
5803
|
+
if (e.kind !== "RUN") continue;
|
|
5804
|
+
sawRunEvent = true;
|
|
5805
|
+
if (e.status === "ok" || e.status === "error") {
|
|
5806
|
+
runTerminal = e.status;
|
|
5807
|
+
}
|
|
5808
|
+
}
|
|
5809
|
+
if (sawRunEvent) {
|
|
5810
|
+
return runTerminal ?? "running";
|
|
5811
|
+
}
|
|
5817
5812
|
let hasRunning = false;
|
|
5818
5813
|
for (const e of events) {
|
|
5819
5814
|
if (e.status === "error") return "error";
|
|
5820
5815
|
if (e.status === "running") hasRunning = true;
|
|
5821
5816
|
}
|
|
5822
|
-
|
|
5823
|
-
return "ok";
|
|
5817
|
+
return hasRunning ? "running" : "ok";
|
|
5824
5818
|
}
|
|
5825
5819
|
var TreeBuilder = class {
|
|
5826
5820
|
constructor(options) {
|
|
@@ -9060,6 +9054,6 @@ function renderGateReport(result, options = {}) {
|
|
|
9060
9054
|
return renderGateSummaryMarkdown(result);
|
|
9061
9055
|
}
|
|
9062
9056
|
|
|
9063
|
-
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, Redactor, TraceDirectory, TraceReadError, TreeBuilder,
|
|
9064
|
-
//# sourceMappingURL=chunk-
|
|
9065
|
-
//# sourceMappingURL=chunk-
|
|
9057
|
+
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, bundleFailsOnSafety, bundleRunAssetRelativePath, compactAttributes3 as compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, source_default, stableJson, summarizeObservedOutcomes, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, zeroKinds };
|
|
9058
|
+
//# sourceMappingURL=chunk-TO4VENHV.mjs.map
|
|
9059
|
+
//# sourceMappingURL=chunk-TO4VENHV.mjs.map
|