agent-inspect 6.17.3 → 6.17.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +1 -1
- package/docs/ADAPTER-CONFORMANCE.md +24 -0
- package/docs/CI-ARTIFACTS.md +2 -1
- package/docs/OPENAI-AGENTS-LOCAL.md +14 -0
- package/docs/SAFE-TRACE-SHARING.md +1 -0
- package/docs/SELF-HOSTING.md +2 -0
- package/docs/STANDARDS.md +15 -0
- package/docs/SUPPORT-LEVELS.md +23 -0
- package/package.json +4 -3
- package/packages/cli/dist/{chunk-5GGYDIZD.mjs → chunk-YX6RFLV5.mjs} +9 -4
- package/packages/cli/dist/chunk-YX6RFLV5.mjs.map +1 -0
- package/packages/cli/dist/index.cjs +26 -9
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +22 -10
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/cli/dist/{src-IDMCWKRH.mjs → src-X6SLMBNH.mjs} +3 -3
- package/packages/cli/dist/{src-IDMCWKRH.mjs.map → src-X6SLMBNH.mjs.map} +1 -1
- package/packages/core/dist/advanced.cjs +1 -1
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.mjs +1 -1
- package/packages/core/dist/checks.cjs +7 -2
- package/packages/core/dist/checks.cjs.map +1 -1
- package/packages/core/dist/checks.mjs +1 -1
- package/packages/core/dist/{chunk-UFP54T7F.mjs → chunk-HN377P23.mjs} +9 -4
- package/packages/core/dist/chunk-HN377P23.mjs.map +1 -0
- package/packages/cli/dist/chunk-5GGYDIZD.mjs.map +0 -1
- package/packages/core/dist/chunk-UFP54T7F.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.17.4
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- c4b0f03: Fix camelCase / kebab / dot compound credential key redaction (`userPassword`, `clientSecret`) while keeping token-config keys and camelCase topic fields (`emailNote`) un-key-redacted.
|
|
8
|
+
|
|
3
9
|
## 6.17.3
|
|
4
10
|
|
|
5
11
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -210,7 +210,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
|
|
|
210
210
|
|
|
211
211
|
## Status and documentation
|
|
212
212
|
|
|
213
|
-
**Current published baseline:** **6.17.
|
|
213
|
+
**Current published baseline:** **6.17.3** · persisted schema `1.0` · Node.js `>=20` · MIT.
|
|
214
214
|
|
|
215
215
|
Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
|
|
216
216
|
|
|
@@ -58,6 +58,30 @@ This is an internal official-adapter gate, not a third-party certification progr
|
|
|
58
58
|
|
|
59
59
|
An adapter can be documented as supported only after no-network fixtures cover run, step, tool, LLM, error, streaming, and metadata-bound expectations for that framework path.
|
|
60
60
|
|
|
61
|
+
## Third-party adapter conformance CI
|
|
62
|
+
|
|
63
|
+
The gate above is the internal official-adapter gate, not a third-party certification program. Third-party adapter authors run the **public** conformance helper from [`@agent-inspect/adapter-sdk`](https://www.npmjs.com/package/@agent-inspect/adapter-sdk) in their own CI:
|
|
64
|
+
|
|
65
|
+
```ts
|
|
66
|
+
import { runAdapterConformance } from "@agent-inspect/adapter-sdk";
|
|
67
|
+
|
|
68
|
+
const result = await runAdapterConformance({
|
|
69
|
+
adapterId: "my-adapter",
|
|
70
|
+
events, // PersistedInspectEvent[] your adapter emits
|
|
71
|
+
expectedKinds: ["RUN", "LLM"],
|
|
72
|
+
forbiddenRawStrings: ["super-secret-key"], // must not leak into events
|
|
73
|
+
});
|
|
74
|
+
// assert result.ok in your test runner
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
A copyable GitHub Actions template lives at [`examples/adapter-sdk/third-party-conformance-ci.yml`](../examples/adapter-sdk/third-party-conformance-ci.yml). Copy it to `.github/workflows/agent-inspect-conformance.yml` in your adapter repo and adjust the install/build/test commands. The template:
|
|
78
|
+
|
|
79
|
+
- runs on a Node LTS matrix (18 / 20 / 22),
|
|
80
|
+
- requires **no provider keys** and makes **no network calls** (conformance is metadata-only),
|
|
81
|
+
- keeps the framework SDK as a peer dependency of your adapter, never of AgentInspect core.
|
|
82
|
+
|
|
83
|
+
See the runnable examples under [`examples/adapter-sdk/`](../examples/adapter-sdk/) for adapters that call `runAdapterConformance` end to end.
|
|
84
|
+
|
|
61
85
|
## v3.2 adoption evidence refresh
|
|
62
86
|
|
|
63
87
|
- [AI-SDK-ADOPTION.md](./AI-SDK-ADOPTION.md) — blessed AI SDK path (`generateText`, `streamText`, Next.js recipe, troubleshooting)
|
package/docs/CI-ARTIFACTS.md
CHANGED
|
@@ -86,8 +86,9 @@ Recipes:
|
|
|
86
86
|
|
|
87
87
|
- [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for v1.8 `check`, baseline, safe artifact, and step-summary workflows.
|
|
88
88
|
- [examples/recipes/github-actions-artifact](../examples/recipes/github-actions-artifact/README.md) for share-safe trace exports and reporter manifest summaries.
|
|
89
|
+
- [examples/recipes/github-actions-gate](../examples/recipes/github-actions-gate/README.md) for a retained broken-to-fixed suite/gate pilot with separate Evidence v2 artifacts.
|
|
89
90
|
|
|
90
|
-
Sample workflows: [deterministic checks workflow](../examples/recipes/deterministic-ci-checks/workflow-example.yml), [share-safe export workflow](../examples/recipes/github-actions-artifact/workflow-example.yml)
|
|
91
|
+
Sample workflows: [deterministic checks workflow](../examples/recipes/deterministic-ci-checks/workflow-example.yml), [share-safe export workflow](../examples/recipes/github-actions-artifact/workflow-example.yml), [retained gate workflow](../examples/recipes/github-actions-gate/workflow-example.yml)
|
|
91
92
|
|
|
92
93
|
```yaml
|
|
93
94
|
- uses: actions/upload-artifact@v4
|
|
@@ -41,6 +41,20 @@ setTraceProcessors([
|
|
|
41
41
|
|
|
42
42
|
- [openai-agents-local-processor](../../examples/recipes/openai-agents-local-processor/)
|
|
43
43
|
|
|
44
|
+
## No-key packed consumer check
|
|
45
|
+
|
|
46
|
+
The repository includes a packed-consumer check that installs the root and
|
|
47
|
+
OpenAI Agents adapter tarballs into a temporary project. It drives deterministic
|
|
48
|
+
tracing fixtures without `OPENAI_API_KEY` or a live provider call, writes a local
|
|
49
|
+
trace, and inspects it through the packed AgentInspect CLI.
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
pnpm build
|
|
53
|
+
node scripts/packed-openai-agents-e2e.mjs
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The check is also included in `pnpm pack:smoke`.
|
|
57
|
+
|
|
44
58
|
## Troubleshooting
|
|
45
59
|
|
|
46
60
|
| Symptom | Check |
|
|
@@ -36,6 +36,7 @@ When a maintainer or support responder needs reproducible evidence, follow the [
|
|
|
36
36
|
- Inspect manual metadata passed to `inspectRun()`, `step()`, `step.tool()`, `step.llm()`, or `observe()`.
|
|
37
37
|
- Inspect log-derived fields from `logs` / `tail` ingest configs, including custom `run-id`, `event`, `parent`, timestamp, and attribute mappings.
|
|
38
38
|
- Avoid posting raw prompts, completions, tool inputs, or tool outputs in public threads unless the content is approved for public disclosure.
|
|
39
|
+
- For cross-system correlation, retain a bounded identifier instead of copying the external record or payload. Follow [External reference metadata](./EXTERNAL-REFERENCES.md), and expect `share` / `strict` profiles to redact the named correlation fields.
|
|
39
40
|
- Prefer Markdown export for issue or PR sharing when a summarized tree is enough.
|
|
40
41
|
|
|
41
42
|
## Remove or replace sensitive values
|
package/docs/SELF-HOSTING.md
CHANGED
|
@@ -125,6 +125,8 @@ Postgres URLs are reserved for team deployments and are **not required** for loc
|
|
|
125
125
|
|
|
126
126
|
## Related docs
|
|
127
127
|
|
|
128
|
+
- [STUDIO-IMPORT-BUNDLE.md](./STUDIO-IMPORT-BUNDLE.md) — walkthrough: share-safe bundle → local Studio
|
|
129
|
+
- [STUDIO-IMPORT-GITHUB-ARTIFACT.md](./STUDIO-IMPORT-GITHUB-ARTIFACT.md) — walkthrough: GitHub Actions artifact → local Studio
|
|
128
130
|
- [SELF-HOSTED-STUDIO-V6.0.md](./proposals/SELF-HOSTED-STUDIO-V6.0.md)
|
|
129
131
|
- [CLIENT-HOSTED-INGESTION-V6.1.md](./proposals/CLIENT-HOSTED-INGESTION-V6.1.md)
|
|
130
132
|
- [LOCAL-TRACE-WORKSPACE.md](./proposals/LOCAL-TRACE-WORKSPACE.md)
|
package/docs/STANDARDS.md
CHANGED
|
@@ -21,6 +21,8 @@ Shape validation is **compatible**; semantic checks add field-level warnings for
|
|
|
21
21
|
|
|
22
22
|
Fixture: [fixtures/standards/openinference-basic.json](../fixtures/standards/openinference-basic.json)
|
|
23
23
|
|
|
24
|
+
The fixture's top-level `version` is an **AgentInspect reference fixture revision**, not an upstream OpenInference version.
|
|
25
|
+
|
|
24
26
|
## OTLP JSON (experimental)
|
|
25
27
|
|
|
26
28
|
```bash
|
|
@@ -31,10 +33,23 @@ GenAI attribute mapping follows `OTEL_GEN_AI_SEMCONV_PIN` (see exporters API). N
|
|
|
31
33
|
|
|
32
34
|
Fixture: [fixtures/standards/otlp-basic.json](../fixtures/standards/otlp-basic.json)
|
|
33
35
|
|
|
36
|
+
The fixture's `scope.version` is an **AgentInspect test-scope fixture revision**, not the `OTEL_GEN_AI_SEMCONV_PIN` and not an upstream OpenTelemetry version.
|
|
37
|
+
|
|
34
38
|
## Graduation guide
|
|
35
39
|
|
|
36
40
|
Full path from local export through review to optional customer-owned import: [STANDARDS-GRADUATION.md](./STANDARDS-GRADUATION.md).
|
|
37
41
|
|
|
42
|
+
That guide is the canonical source for standards known-loss boundaries, including kind degradation, bounded metadata, no chain-of-thought capture, and snapshot limitations.
|
|
43
|
+
|
|
44
|
+
## Maintaining tested provenance
|
|
45
|
+
|
|
46
|
+
- When the OTLP mapping changes, update `OTEL_GEN_AI_SEMCONV_PIN` in [`packages/core/src/exporters/semconv.ts`](../packages/core/src/exporters/semconv.ts) and any explicit tested-version claims together.
|
|
47
|
+
- When the OTLP reference shape changes, update its test-scope fixture revision in [`fixtures/standards/otlp-basic.json`](../fixtures/standards/otlp-basic.json) and any repeated fixture revision together.
|
|
48
|
+
- When the OpenInference reference shape changes, update the top-level fixture revision in [`fixtures/standards/openinference-basic.json`](../fixtures/standards/openinference-basic.json) and any repeated fixture revision together. The value remains an AgentInspect fixture revision, not an upstream version.
|
|
49
|
+
- Keep known-loss behavior canonical in [`STANDARDS-GRADUATION.md`](./STANDARDS-GRADUATION.md) and update its validation alongside any intentional exporter behavior change.
|
|
50
|
+
|
|
51
|
+
Run `pnpm public-truth:check` and `pnpm docs:check` after changing these sources.
|
|
52
|
+
|
|
38
53
|
## Import recipes
|
|
39
54
|
|
|
40
55
|
- [Phoenix / OpenInference](../examples/recipes/phoenix-openinference-import/)
|
package/docs/SUPPORT-LEVELS.md
CHANGED
|
@@ -37,6 +37,29 @@ Canonical maturity labels for AgentInspect public packages and major surfaces (6
|
|
|
37
37
|
|
|
38
38
|
Part of the fixed AgentInspect release line — see the npm badge for the current version.
|
|
39
39
|
|
|
40
|
+
> Every level in this matrix must be one of the Definitions levels above, and `docs/product/PUBLIC-PRODUCT-FACTS.json` `matchers.status` must match the matchers row here. `pnpm public-truth:check` enforces both; update the doc and the facts file together.
|
|
41
|
+
|
|
42
|
+
## Changing a level
|
|
43
|
+
|
|
44
|
+
A level is stated in two places, and they must not disagree: the row above, and
|
|
45
|
+
the `**Support level:**` line in the package's own README (which is what npm
|
|
46
|
+
shows). `pnpm package-readmes:check` enforces the agreement and runs as part of
|
|
47
|
+
`pnpm docs:check`.
|
|
48
|
+
|
|
49
|
+
To promote or demote a surface:
|
|
50
|
+
|
|
51
|
+
1. Edit the row in the matrix above.
|
|
52
|
+
2. Edit the `**Support level:**` line in each affected `packages/*/README.md`.
|
|
53
|
+
3. Run `pnpm package-readmes:check`.
|
|
54
|
+
|
|
55
|
+
The check also reports packages whose level is **unenforced** — those the matrix
|
|
56
|
+
above does not name, so the README is their only source. Adding a row that names
|
|
57
|
+
the package (in backticks) puts it under enforcement.
|
|
58
|
+
|
|
59
|
+
It rejects an absolute "no network" claim from any surface
|
|
60
|
+
[NETWORK-BEHAVIOR.md](./NETWORK-BEHAVIOR.md) records as making network calls, so
|
|
61
|
+
a promotion that changes network behavior cannot leave a stale guarantee on npm.
|
|
62
|
+
|
|
40
63
|
## Public package groups (presentation only)
|
|
41
64
|
|
|
42
65
|
Physical packages stay the fixed group of 18. Outreach/install kits group them as:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.17.
|
|
3
|
+
"version": "6.17.4",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
|
|
@@ -235,13 +235,14 @@
|
|
|
235
235
|
"test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run build && pnpm run test && pnpm run size",
|
|
236
236
|
"prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run repo:health && pnpm run pack:smoke",
|
|
237
237
|
"pack:dry-run": "pnpm run build && npm pack --dry-run",
|
|
238
|
-
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs && node scripts/packed-swarm-loop-e2e.mjs && node scripts/evidence-ci-golden-paths.mjs",
|
|
238
|
+
"pack:smoke": "pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-openai-agents-e2e.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs && node scripts/packed-swarm-loop-e2e.mjs && node scripts/evidence-ci-golden-paths.mjs",
|
|
239
239
|
"linked-versions:check": "node scripts/check-linked-versions.mjs",
|
|
240
240
|
"docs:commands": "node scripts/validate-doc-commands.mjs",
|
|
241
241
|
"docs:links": "node scripts/validate-doc-links.mjs",
|
|
242
242
|
"public-truth:check": "node scripts/validate-public-truth.mjs",
|
|
243
243
|
"ai-assets:check": "node scripts/validate-ai-assets.mjs",
|
|
244
|
-
"
|
|
244
|
+
"package-readmes:check": "node scripts/validate-package-readmes.mjs",
|
|
245
|
+
"docs:check": "pnpm run docs:commands && pnpm run docs:links && pnpm run public-truth:check && pnpm run ai-assets:check && pnpm run package-readmes:check && pnpm run repo:health && pnpm run demo:verify",
|
|
245
246
|
"repo:health": "node scripts/validate-repo-health.mjs",
|
|
246
247
|
"demo:generate": "node scripts/demo-generate.mjs",
|
|
247
248
|
"demo:verify": "node scripts/demo-verify.mjs",
|
|
@@ -6237,8 +6237,11 @@ formatProgrammaticDiagnostic(
|
|
|
6237
6237
|
);
|
|
6238
6238
|
|
|
6239
6239
|
// packages/core/src/safety/sensitive-key.ts
|
|
6240
|
+
function keyHasExplicitSeparator(value) {
|
|
6241
|
+
return /[_\-.]/.test(value);
|
|
6242
|
+
}
|
|
6240
6243
|
function normalizeSensitiveKey(value) {
|
|
6241
|
-
return value.toLowerCase().replace(/[^a-z0-
|
|
6244
|
+
return value.replace(/([a-z0-9])([A-Z])/g, "$1_$2").toLowerCase().replace(/[^a-z0-9]+/g, "_").replace(/^_+|_+$/g, "");
|
|
6242
6245
|
}
|
|
6243
6246
|
var NON_CREDENTIAL_TOKEN_CONFIG_KEYS = new Set(
|
|
6244
6247
|
[
|
|
@@ -6290,6 +6293,7 @@ function isCredentialSensitiveKey(key, sensitiveKeys = DEFAULT_CREDENTIAL_SENSIT
|
|
|
6290
6293
|
const normalized = normalizeSensitiveKey(key);
|
|
6291
6294
|
if (!normalized) return false;
|
|
6292
6295
|
if (NON_CREDENTIAL_TOKEN_CONFIG_KEYS.has(normalized)) return false;
|
|
6296
|
+
const allowPrefixCompound = keyHasExplicitSeparator(key);
|
|
6293
6297
|
for (const sensitive of sensitiveKeys) {
|
|
6294
6298
|
const s = normalizeSensitiveKey(sensitive);
|
|
6295
6299
|
if (!s) continue;
|
|
@@ -6298,7 +6302,8 @@ function isCredentialSensitiveKey(key, sensitiveKeys = DEFAULT_CREDENTIAL_SENSIT
|
|
|
6298
6302
|
continue;
|
|
6299
6303
|
}
|
|
6300
6304
|
if (normalized === s) return true;
|
|
6301
|
-
if (normalized.endsWith(`_${s}`)
|
|
6305
|
+
if (normalized.endsWith(`_${s}`)) return true;
|
|
6306
|
+
if (allowPrefixCompound && normalized.startsWith(`${s}_`)) return true;
|
|
6302
6307
|
}
|
|
6303
6308
|
return false;
|
|
6304
6309
|
}
|
|
@@ -11341,5 +11346,5 @@ function renderGateReport(result, options = {}) {
|
|
|
11341
11346
|
}
|
|
11342
11347
|
|
|
11343
11348
|
export { COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
|
|
11344
|
-
//# sourceMappingURL=chunk-
|
|
11345
|
-
//# sourceMappingURL=chunk-
|
|
11349
|
+
//# sourceMappingURL=chunk-YX6RFLV5.mjs.map
|
|
11350
|
+
//# sourceMappingURL=chunk-YX6RFLV5.mjs.map
|