agent-inspect 6.7.2 → 6.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +1 -1
- package/docs/EXPORTS.md +9 -0
- package/docs/PERFORMANCE.md +18 -0
- package/docs/STANDARDS.md +4 -0
- package/package.json +2 -2
- package/packages/cli/dist/{chunk-M6PC5KUT.mjs → chunk-EMYDJREA.mjs} +26 -10
- package/packages/cli/dist/chunk-EMYDJREA.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +46 -21
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +23 -14
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-27LC2BB3.mjs → src-IPJNFV37.mjs} +3 -3
- package/packages/cli/dist/{src-27LC2BB3.mjs.map → src-IPJNFV37.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +24 -8
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.mjs +25 -9
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/checks.cjs +56 -4
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-EWHN6NA4.mjs → chunk-EQVXDGGU.mjs} +58 -6
- package/packages/core/dist/chunk-EQVXDGGU.mjs.map +1 -0
- package/packages/core/dist/diff.cjs +4 -0
- package/packages/core/dist/diff.cjs.map +1 -1
- package/packages/core/dist/diff.mjs +4 -0
- package/packages/core/dist/diff.mjs.map +1 -1
- package/packages/core/dist/exporters.cjs +9 -4
- package/packages/core/dist/exporters.cjs.map +1 -1
- package/packages/core/dist/exporters.d.cts +3 -2
- package/packages/core/dist/exporters.d.ts +3 -2
- package/packages/core/dist/exporters.mjs +9 -4
- package/packages/core/dist/exporters.mjs.map +1 -1
- package/packages/core/dist/logs.cjs +3 -2
- package/packages/core/dist/logs.cjs.map +1 -1
- package/packages/core/dist/logs.mjs +3 -2
- package/packages/core/dist/logs.mjs.map +1 -1
- package/packages/cli/dist/chunk-M6PC5KUT.mjs.map +0 -1
- package/packages/core/dist/chunk-EWHN6NA4.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.7.3
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- ac6747d: Corrective patch after 6.7.2: TraceContract allowed-status handling, OpenInference nano timestamp strings, cohort maxRelativeDelta for all numeric metrics, Windows path portability, CLI search --session exit codes, diff/normalizer correctness, Studio auth timing / ingest / bundle dedup, and related test hardening. No schema break; no new product surface.
|
|
8
|
+
|
|
3
9
|
## Unreleased
|
|
4
10
|
|
|
5
11
|
## 6.7.2
|
package/README.md
CHANGED
|
@@ -131,7 +131,7 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
131
131
|
|
|
132
132
|
## Project status
|
|
133
133
|
|
|
134
|
-
**Current release:** **6.7.
|
|
134
|
+
**Current release:** **6.7.3** (eighteen linked npm packages). Technical launch candidate; external pilot evidence pending. Persisted schema **1.0**. Node.js **≥ 20**. **v7 not scheduled.**
|
|
135
135
|
|
|
136
136
|
[Roadmap](ROADMAP.md) · [Pilot kit](https://github.com/rajudandigam/agent-inspect/blob/main/docs/PRE-V7-PILOT-KIT.md) · [Changelog](CHANGELOG.md)
|
|
137
137
|
|
package/docs/EXPORTS.md
CHANGED
|
@@ -36,6 +36,14 @@ agent-inspect export minimal-success --dir fixtures/traces --format markdown
|
|
|
36
36
|
- **HTML:** export writes a local file; a rendered-report visual is pending re-record ([RECORDING.md](./assets/demos/RECORDING.md)). Use `agent-inspect export <run-id> --format html -o report.html` and open the file locally.
|
|
37
37
|
- **OpenInference / OTLP JSON:** compatibility-oriented shapes for local handoff — validate with `--validate` before sharing. No dedicated GIF; see format tables below.
|
|
38
38
|
|
|
39
|
+
### Fixture-backed export evidence
|
|
40
|
+
|
|
41
|
+
The canonical OpenInference export of a persisted schema **1.0** trace is committed at
|
|
42
|
+
[`fixtures/standards/openinference-export-golden.json`](../fixtures/standards/openinference-export-golden.json)
|
|
43
|
+
(generated from `fixtures/traces-v1.0/manual-basic.jsonl`; regeneration is asserted byte-stable in
|
|
44
|
+
`packages/core/test/exporters/openinference-export-golden.test.ts`). Nanosecond timestamps are decimal strings,
|
|
45
|
+
matching the import fixtures and the OTLP exporter.
|
|
46
|
+
|
|
39
47
|
## Common options
|
|
40
48
|
|
|
41
49
|
| Flag | Notes |
|
|
@@ -90,6 +98,7 @@ Prefer the smallest useful artifact — a summary or synthetic reproduction beat
|
|
|
90
98
|
|
|
91
99
|
## Related docs
|
|
92
100
|
|
|
101
|
+
- [STANDARDS-GRADUATION.md](./STANDARDS-GRADUATION.md) — export → review → optional import path
|
|
93
102
|
- [CLI.md](./CLI.md) — `export` command reference
|
|
94
103
|
- [SCHEMA.md](./SCHEMA.md) — trace event model and redaction notes
|
|
95
104
|
- [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) — checklist before posting
|
package/docs/PERFORMANCE.md
CHANGED
|
@@ -12,6 +12,24 @@ pnpm perf:baseline
|
|
|
12
12
|
|
|
13
13
|
The script prints deterministic timings for parse, normalize, tree build, diff, export, and checks on synthetic data. Warnings appear when a step exceeds internal soft thresholds.
|
|
14
14
|
|
|
15
|
+
## Performance fixtures
|
|
16
|
+
|
|
17
|
+
[`fixtures/performance/`](../fixtures/performance/README.md) ships three deterministic size tiers with the same event shape as the baseline's synthetic traces, for exercising CLI paths against stable on-disk inputs:
|
|
18
|
+
|
|
19
|
+
| File | Steps | Events |
|
|
20
|
+
| ---- | ----- | ------ |
|
|
21
|
+
| `perf-small.jsonl` | 10 | 22 |
|
|
22
|
+
| `perf-medium.jsonl` | 100 | 202 |
|
|
23
|
+
| `perf-large.jsonl` | 500 | 1,002 |
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
npx agent-inspect view perf-large --dir fixtures/performance
|
|
27
|
+
npx agent-inspect report perf-large --dir fixtures/performance
|
|
28
|
+
npx agent-inspect stats --dir fixtures/performance
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
All three are validated by `pnpm fixtures:check` and small enough for git (the largest is well under 1 MB).
|
|
32
|
+
|
|
15
33
|
## Comfortable ranges (v3.4)
|
|
16
34
|
|
|
17
35
|
| Workload | Comfortable | Warning | Not designed for |
|
package/docs/STANDARDS.md
CHANGED
|
@@ -31,6 +31,10 @@ GenAI attribute mapping follows `OTEL_GEN_AI_SEMCONV_PIN` (see exporters API). N
|
|
|
31
31
|
|
|
32
32
|
Fixture: [fixtures/standards/otlp-basic.json](../fixtures/standards/otlp-basic.json)
|
|
33
33
|
|
|
34
|
+
## Graduation guide
|
|
35
|
+
|
|
36
|
+
Full path from local export through review to optional customer-owned import: [STANDARDS-GRADUATION.md](./STANDARDS-GRADUATION.md).
|
|
37
|
+
|
|
34
38
|
## Import recipes
|
|
35
39
|
|
|
36
40
|
- [Phoenix / OpenInference](../examples/recipes/phoenix-openinference-import/)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.7.
|
|
3
|
+
"version": "6.7.3",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Debug, regression-test, and safely share TypeScript AI-agent behavior locally — no account, no default upload, metadata-only by default",
|
|
@@ -216,7 +216,7 @@
|
|
|
216
216
|
"vitest": "^2.1.8"
|
|
217
217
|
},
|
|
218
218
|
"scripts": {
|
|
219
|
-
"clean": "pnpm -r exec --
|
|
219
|
+
"clean": "pnpm -r exec -- node -e \"require('node:fs').rmSync('dist',{recursive:true,force:true})\"",
|
|
220
220
|
"build": "pnpm exec tsup --config tsup.core.config.ts && pnpm exec tsup --config tsup.cli.config.ts && pnpm exec tsup --config tsup.langchain.config.ts && pnpm exec tsup --config tsup.tui.config.ts && pnpm exec tsup --config tsup.ai-sdk.config.ts && pnpm exec tsup --config tsup.vitest.config.ts && pnpm exec tsup --config tsup.jest.config.ts && pnpm exec tsup --config tsup.openai-agents.config.ts && pnpm exec tsup --config tsup.harness.config.ts && pnpm exec tsup --config tsup.redact.config.ts && pnpm exec tsup --config tsup.eval.config.ts && pnpm exec tsup --config tsup.mcp.config.ts && pnpm exec tsup --config tsup.guardrails.config.ts && pnpm exec tsup --config tsup.circuit.config.ts && pnpm exec tsup --config tsup.viewer.config.ts && pnpm exec tsup --config tsup.mcp-server.config.ts && pnpm exec tsup --config tsup.adapter-sdk.config.ts && pnpm exec tsup --config tsup.vscode.config.ts && pnpm exec tsup --config tsup.index-sqlite.config.ts && pnpm exec tsup --config tsup.studio.config.ts",
|
|
221
221
|
"typecheck": "tsc --noEmit",
|
|
222
222
|
"test": "vitest run",
|
|
@@ -8137,7 +8137,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8137
8137
|
metric,
|
|
8138
8138
|
"Average duration (ms)",
|
|
8139
8139
|
baselineAgg?.avgDurationMs,
|
|
8140
|
-
candidateAgg?.avgDurationMs
|
|
8140
|
+
candidateAgg?.avgDurationMs,
|
|
8141
|
+
true,
|
|
8142
|
+
tolerance
|
|
8141
8143
|
);
|
|
8142
8144
|
if (item) comparisons.push(item);
|
|
8143
8145
|
break;
|
|
@@ -8147,7 +8149,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8147
8149
|
metric,
|
|
8148
8150
|
"Average LLM calls",
|
|
8149
8151
|
baselineAgg?.avgLlmCallCount,
|
|
8150
|
-
candidateAgg?.avgLlmCallCount
|
|
8152
|
+
candidateAgg?.avgLlmCallCount,
|
|
8153
|
+
true,
|
|
8154
|
+
tolerance
|
|
8151
8155
|
);
|
|
8152
8156
|
if (item) comparisons.push(item);
|
|
8153
8157
|
break;
|
|
@@ -8157,7 +8161,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8157
8161
|
metric,
|
|
8158
8162
|
"Average token usage",
|
|
8159
8163
|
baselineAgg?.avgTokenUsage,
|
|
8160
|
-
candidateAgg?.avgTokenUsage
|
|
8164
|
+
candidateAgg?.avgTokenUsage,
|
|
8165
|
+
true,
|
|
8166
|
+
tolerance
|
|
8161
8167
|
);
|
|
8162
8168
|
if (item) comparisons.push(item);
|
|
8163
8169
|
break;
|
|
@@ -8167,7 +8173,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8167
8173
|
metric,
|
|
8168
8174
|
"Average retries",
|
|
8169
8175
|
baselineAgg?.avgRetryCount,
|
|
8170
|
-
candidateAgg?.avgRetryCount
|
|
8176
|
+
candidateAgg?.avgRetryCount,
|
|
8177
|
+
true,
|
|
8178
|
+
tolerance
|
|
8171
8179
|
);
|
|
8172
8180
|
if (item) comparisons.push(item);
|
|
8173
8181
|
break;
|
|
@@ -8177,7 +8185,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8177
8185
|
metric,
|
|
8178
8186
|
"Observation failure rate",
|
|
8179
8187
|
baselineAgg?.observationFailureRate,
|
|
8180
|
-
candidateAgg?.observationFailureRate
|
|
8188
|
+
candidateAgg?.observationFailureRate,
|
|
8189
|
+
true,
|
|
8190
|
+
tolerance
|
|
8181
8191
|
);
|
|
8182
8192
|
if (item) comparisons.push(item);
|
|
8183
8193
|
break;
|
|
@@ -8213,7 +8223,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8213
8223
|
metric,
|
|
8214
8224
|
"Guardrail failures",
|
|
8215
8225
|
baselineAgg?.avgGuardrailFailures,
|
|
8216
|
-
candidateAgg?.avgGuardrailFailures
|
|
8226
|
+
candidateAgg?.avgGuardrailFailures,
|
|
8227
|
+
true,
|
|
8228
|
+
tolerance
|
|
8217
8229
|
);
|
|
8218
8230
|
if (item) comparisons.push(item);
|
|
8219
8231
|
break;
|
|
@@ -8223,7 +8235,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8223
8235
|
metric,
|
|
8224
8236
|
"Circuit violations",
|
|
8225
8237
|
baselineAgg?.avgCircuitViolations,
|
|
8226
|
-
candidateAgg?.avgCircuitViolations
|
|
8238
|
+
candidateAgg?.avgCircuitViolations,
|
|
8239
|
+
true,
|
|
8240
|
+
tolerance
|
|
8227
8241
|
);
|
|
8228
8242
|
if (item) comparisons.push(item);
|
|
8229
8243
|
break;
|
|
@@ -8233,7 +8247,9 @@ function compareCohortAggregates(groups, options) {
|
|
|
8233
8247
|
metric,
|
|
8234
8248
|
"Redaction warnings",
|
|
8235
8249
|
baselineAgg?.avgRedactionWarnings,
|
|
8236
|
-
candidateAgg?.avgRedactionWarnings
|
|
8250
|
+
candidateAgg?.avgRedactionWarnings,
|
|
8251
|
+
true,
|
|
8252
|
+
tolerance
|
|
8237
8253
|
);
|
|
8238
8254
|
if (item) comparisons.push(item);
|
|
8239
8255
|
break;
|
|
@@ -9045,5 +9061,5 @@ function renderGateReport(result, options = {}) {
|
|
|
9045
9061
|
}
|
|
9046
9062
|
|
|
9047
9063
|
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, Redactor, TraceDirectory, TraceReadError, TreeBuilder, __commonJS, __require, __toESM, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, bundleFailsOnSafety, bundleRunAssetRelativePath, compactAttributes3 as compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, source_default, stableJson, summarizeObservedOutcomes, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, zeroKinds };
|
|
9048
|
-
//# sourceMappingURL=chunk-
|
|
9049
|
-
//# sourceMappingURL=chunk-
|
|
9064
|
+
//# sourceMappingURL=chunk-EMYDJREA.mjs.map
|
|
9065
|
+
//# sourceMappingURL=chunk-EMYDJREA.mjs.map
|