agent-inspect 6.16.2 → 6.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +1 -1
- package/docs/CI-ARTIFACTS.md +31 -0
- package/docs/CLI.md +24 -2
- package/package.json +1 -1
- package/packages/cli/dist/index.cjs +910 -320
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +832 -248
- package/packages/cli/dist/index.mjs.map +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 6.17.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 3294db3: Add check presets (`trajectory` / `safety` / `comprehensive`), local Evidence-on-failure flags for `check`/`gate`, and `bundle open` for verified local Evidence HTML.
|
|
8
|
+
|
|
3
9
|
## 6.16.2
|
|
4
10
|
|
|
5
11
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -157,7 +157,7 @@ Details: [Safe sharing](https://github.com/rajudandigam/agent-inspect/blob/main/
|
|
|
157
157
|
|
|
158
158
|
## Project status
|
|
159
159
|
|
|
160
|
-
**Current release:** **6.
|
|
160
|
+
**Current release:** **6.17.0** · schema **1.0** · Node.js **≥ 20** · **MIT** · **actively maintained** (eighteen linked npm packages).
|
|
161
161
|
|
|
162
162
|
The 6.16 line is actively maintained for correctness, compatibility, documentation, security, and framework evolution.
|
|
163
163
|
|
package/docs/CI-ARTIFACTS.md
CHANGED
|
@@ -98,6 +98,37 @@ Sample workflows: [deterministic checks workflow](../examples/recipes/determinis
|
|
|
98
98
|
./artifacts
|
|
99
99
|
```
|
|
100
100
|
|
|
101
|
+
## Trajectory gate + Evidence on failure (recommended)
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
npx agent-inspect check --dir .agent-inspect --preset trajectory \
|
|
105
|
+
--evidence-on fail --evidence-profile share --evidence-format directory
|
|
106
|
+
npx agent-inspect verify-safe . --dir .agent-inspect
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`init --ci github` scaffolds this pattern. Evidence stays local; upload with your CI provider’s artifact action.
|
|
110
|
+
|
|
111
|
+
## GitLab CI / generic CI
|
|
112
|
+
|
|
113
|
+
Same local commands work outside GitHub. Example GitLab job fragment:
|
|
114
|
+
|
|
115
|
+
```yaml
|
|
116
|
+
trajectory_gate:
|
|
117
|
+
image: node:22
|
|
118
|
+
script:
|
|
119
|
+
- npm ci
|
|
120
|
+
- node examples/agent-inspect-demo.mjs
|
|
121
|
+
- npx --yes agent-inspect check --dir .agent-inspect --preset trajectory --evidence-on fail
|
|
122
|
+
- npx --yes agent-inspect verify-safe . --dir .agent-inspect
|
|
123
|
+
artifacts:
|
|
124
|
+
when: always
|
|
125
|
+
paths:
|
|
126
|
+
- .agent-inspect/
|
|
127
|
+
# No provider API keys required for the deterministic fixture.
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
AgentInspect performs no network upload; CI platforms attach local paths only.
|
|
131
|
+
|
|
101
132
|
## Inspect artifacts locally after download
|
|
102
133
|
|
|
103
134
|
```bash
|
package/docs/CLI.md
CHANGED
|
@@ -318,8 +318,16 @@ Options:
|
|
|
318
318
|
- `--guardrails <rule>`: optional deterministic guardrail rules (`banned-phrase`, `pii-leak`, `prompt-injection`, …); repeatable
|
|
319
319
|
- `--circuit <rule>`: optional circuit analyzers (`same-tool-repetition`, `max-retries`, …); repeatable
|
|
320
320
|
- `--fail-on-observation <status>`: add `outcome.status` rule; repeatable (`failed`, `passed`, `unknown`, `skipped`; default when flag present without value: `failed`)
|
|
321
|
+
- `--preset <trajectory|safety|comprehensive>`: additive check preset (does not change the default when omitted)
|
|
322
|
+
- `trajectory`: completion/structure/relationship focus; excludes share-safety findings
|
|
323
|
+
- `safety`: raw-content / secret / redaction focus
|
|
324
|
+
- `comprehensive`: union of trajectory and safety
|
|
325
|
+
- `--evidence-on <fail|always|never>`: write local Evidence v2 (no upload); omitted = never
|
|
326
|
+
- `--evidence-dir <path>`: Evidence output directory or base path
|
|
327
|
+
- `--evidence-profile <local|share|strict>`: redaction profile for Evidence (default `share`)
|
|
328
|
+
- `--evidence-format <directory|html|zip>`: Evidence layout (default `directory`)
|
|
321
329
|
|
|
322
|
-
By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available.
|
|
330
|
+
By default, `check` runs `run.status`. Additional built-in rules can be selected with `--rule` or config when their options are available. Prefer `--preset trajectory` in CI, then `verify-safe` before sharing.
|
|
323
331
|
|
|
324
332
|
Config files use this shape:
|
|
325
333
|
|
|
@@ -341,6 +349,7 @@ Examples:
|
|
|
341
349
|
```bash
|
|
342
350
|
npx agent-inspect check fixtures/traces-v0.2/manual-basic.jsonl --json
|
|
343
351
|
npx agent-inspect check minimal-success --dir fixtures/traces --rule run.status
|
|
352
|
+
npx agent-inspect check trace.jsonl --preset trajectory --evidence-on fail
|
|
344
353
|
npx agent-inspect check trace.jsonl --max-duration-ms 30000 --required-tool search_docs --json
|
|
345
354
|
npx agent-inspect check trace.jsonl --guardrails pii-leak --guardrails prompt-injection --json
|
|
346
355
|
npx agent-inspect check trace.jsonl --circuit same-tool-repetition --circuit max-retries --json
|
|
@@ -900,6 +909,14 @@ agent-inspect bundle verify <path> [--unexpected fail|warn|ignore] [--json]
|
|
|
900
909
|
|
|
901
910
|
Checks manifest schema, listed-file presence, SHA-256 hashes, unexpected files (default fail), assessment, and generator provenance. Unpack ZIP archives before verify.
|
|
902
911
|
|
|
912
|
+
Open a verified local Evidence HTML file in the platform browser (no network):
|
|
913
|
+
|
|
914
|
+
```bash
|
|
915
|
+
agent-inspect bundle open <path> [--skip-verify] [--json]
|
|
916
|
+
```
|
|
917
|
+
|
|
918
|
+
`<path>` may be an Evidence directory or an `evidence.html` file. Verify runs first unless `--skip-verify` is set. If the OS cannot open a browser, the command prints the local file path as a fallback and does not upload anything.
|
|
919
|
+
|
|
903
920
|
Output includes `trace.html`, `trace.jsonl`, `summary.md`, `metadata.json`, `check-results.json`, `redaction-report.json`, and `assets/runs/` mirrors for multi-run bundles.
|
|
904
921
|
|
|
905
922
|
Examples:
|
|
@@ -1003,14 +1020,19 @@ Options:
|
|
|
1003
1020
|
- `--format <format>` — `markdown`, `json`, `html`, `junit`, or `github` (default: `markdown`)
|
|
1004
1021
|
- `-o, --output <dir>` — write `gate-results.json`, `gate-summary.md`, `gate-report.html`, `junit.xml`, `github-step-summary.md`
|
|
1005
1022
|
- `--json` — print deterministic JSON result
|
|
1023
|
+
- `--evidence-on <fail|always|never>` — write local Evidence v2 (no upload); omitted = never
|
|
1024
|
+
- `--evidence-dir <path>` — Evidence output directory or base path
|
|
1025
|
+
- `--evidence-profile <local|share|strict>` — redaction profile (default `share`)
|
|
1026
|
+
- `--evidence-format <directory|html|zip>` — Evidence layout (default `directory`)
|
|
1006
1027
|
|
|
1007
|
-
Exit codes: **0** pass, **1** gate failed, **2** invalid config, **3** trace read failure, **4** unsupported format.
|
|
1028
|
+
Exit codes: **0** pass, **1** gate failed, **2** invalid config, **3** trace read failure, **4** unsupported format. Evidence emission never suppresses the original gate exit code.
|
|
1008
1029
|
|
|
1009
1030
|
Example:
|
|
1010
1031
|
|
|
1011
1032
|
```bash
|
|
1012
1033
|
npx agent-inspect gate --suite fixtures/configs/outcome-suite.suite.json --output ./gate-artifacts
|
|
1013
1034
|
npx agent-inspect gate --dir fixtures/cohorts/before-after --max-error-rate 5 --forbid-tool deleteAccount
|
|
1035
|
+
npx agent-inspect gate --suite agent-inspect.suite.ts --evidence-on fail --evidence-profile share
|
|
1014
1036
|
```
|
|
1015
1037
|
|
|
1016
1038
|
Recipe: [github-actions-gate](../examples/recipes/github-actions-gate/README.md).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-inspect",
|
|
3
|
-
"version": "6.
|
|
3
|
+
"version": "6.17.0",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
|