agentverify 1.2.0 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,28 @@
1
+ # Changelog
2
+
3
+ All notable changes to the `agentverify` CLI/SDK package are documented here.
4
+
5
+ ## [1.4.0] - Unreleased
6
+
7
+ ### Changed — CI exit codes now distinguish a security failure from an execution error (behavior change)
8
+
9
+ Previously, `--ci` mode used exit code `1` for **both** a real `NOT_VERIFIED` verdict **and** any execution error (missing/invalid API key, network failure, missing file). That made it impossible for a pipeline to tell "the agent failed the security check" apart from "the scan itself couldn't run."
10
+
11
+ New exit codes:
12
+
13
+ | Code | Meaning | Previously |
14
+ | --- | --- | --- |
15
+ | `0` | All scanned files verified | `0` (unchanged) |
16
+ | `1` | `NOT_VERIFIED` — a real security/verification failure | `1` (unchanged) |
17
+ | `2` | `NOT_ASSESSED` — insufficient evidence for a verdict | `2` (unchanged) |
18
+ | `3` | **New.** Execution error — bad/missing API key, network failure, missing file, unknown command | was `1` |
19
+
20
+ **If your pipeline only checks `exit code != 0` to block a deployment, this is not a breaking change** — every previously-non-zero case is still non-zero. **If your pipeline branches on the exact code `1`**, an execution error will now surface as `3` instead — update any such logic to treat both `1` and `3` as failures, or to react differently to `3` (an infrastructure problem, not a security finding).
21
+
22
+ ### Fixed
23
+
24
+ - Scanner: findings that reference a detected credential no longer include the live secret value in `evidence` — only a redacted preview (e.g. `api_key: "sk_l****" (redacted)`). Affects report evidence produced by any scan, including those made public via report sharing.
25
+
26
+ ## [1.3.0] - 2026-07-13
27
+
28
+ Initial public release. See git history for prior changes.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  **Execution trust analysis for AI agents — CLI and SDK**
4
4
 
5
- Scan AI agent configurations for security issues, credential exposure, and governance gaps. Results save automatically to your Agent Verify dashboard.
5
+ Scan AI agent configurations for security issues, credential exposure, and governance gaps. Saved CLI reports appear in your Agent Verify dashboard when API report save succeeds.
6
6
 
7
7
  ## Install
8
8
 
@@ -37,6 +37,31 @@ agentverify scan ./agents
37
37
  agentverify scan . --key av_your_key --json
38
38
  ```
39
39
 
40
+ ### Markdown summary
41
+
42
+ ```bash
43
+ agentverify scan ./agents --markdown
44
+ ```
45
+
46
+ ### CI mode
47
+
48
+ ```bash
49
+ agentverify scan ./agents --ci
50
+ ```
51
+
52
+ Normal scans print findings without failing the shell just because issues exist. CI mode is stricter: it exits non-zero when an agent is `NOT_VERIFIED`, when a scan errors, or when submitted content cannot be assessed.
53
+
54
+ Exit codes distinguish a security/verification result from an application error, so a pipeline can tell "the agent failed the check" apart from "the scan itself couldn't run":
55
+
56
+ | Code | Meaning |
57
+ | --- | --- |
58
+ | `0` | All scanned files verified |
59
+ | `1` | `NOT_VERIFIED` — a security/verification check failed |
60
+ | `2` | `NOT_ASSESSED` — insufficient evidence for a verdict (see `--allow-not-assessed`) |
61
+ | `3` | Execution error — the scan itself could not run (bad key, network failure, missing file) |
62
+
63
+ All non-zero codes still fail a pipeline step by default, so an existing `if [ $? -ne 0 ]`-style check keeps working unchanged.
64
+
40
65
  ## SDK Usage
41
66
 
42
67
  ```ts
@@ -51,7 +76,7 @@ const result = await scan(
51
76
  { apiKey: process.env.AGENTVERIFY_API_KEY }
52
77
  )
53
78
 
54
- result.verdict // 'VERIFIED' | 'NOT VERIFIED'
79
+ result.verdict // 'VERIFIED' | 'NOT_VERIFIED' | 'NOT_ASSESSED'
55
80
  result.riskScore // 0-100
56
81
  result.findings // Array of security findings
57
82
  ```
@@ -73,10 +98,28 @@ const results = await scanMany(
73
98
  ### GitHub Actions
74
99
 
75
100
  ```yaml
76
- - name: Scan agents
77
- run: npx agentverify scan ./agents --key ${{ secrets.AGENTVERIFY_API_KEY }} --json
101
+ - name: Agent Verify
102
+ run: npx agentverify scan ./agents --ci
103
+ env:
104
+ AGENTVERIFY_API_KEY: ${{ secrets.AGENTVERIFY_API_KEY }}
105
+ ```
106
+
107
+ Use this in pull requests to block merges when an agent is `NOT_VERIFIED`. The CLI prints score, verdict, risk level, top blocker, fix-first guidance, threat categories, A2SPA docs, and the canonical report URL:
108
+
109
+ ```text
110
+ https://aimodularity.com/agentverify/report/?id=REPORT_ID
78
111
  ```
79
112
 
113
+ For PR comments, run with `--markdown` and post the output with your preferred GitHub Action. For machine-readable pipelines, run with `--json`.
114
+
115
+ The workflow requires a valid `AGENTVERIFY_API_KEY`. The local Worker code path fails closed for missing, malformed, invalid, disabled, or revoked keys before findings are returned; validate the deployed API path before relying on this as production enforcement. Wire the workflow into branch protection so the non-zero `--ci` exit code blocks merges when an agent is `NOT_VERIFIED`, not assessed, or the API key is unauthorized.
116
+
117
+ Saved CLI reports sync to the dashboard through the API `cliReports` save path. If the API cannot save the report, the CLI still prints local scan output and tells you dashboard sync did not complete.
118
+
119
+ Verified reports can expose a README badge from the report page. A packaged Agent Verify GitHub Action (`AI-Blockchain-Ventures/agentverify@v1`) already exists and produces text/Markdown/JSON output plus an optional workflow artifact — see below. Automatically posting that output as a PR comment (rather than just producing it for you to post) is still a planned enhancement, not yet built.
120
+
121
+ See the full GitHub Action guide at `docs/github-action.md` for composite action inputs, JSON/Markdown examples, branch protection, optional PR comments, and badge guidance.
122
+
80
123
  ## Get your API key
81
124
 
82
125
  Sign up at https://aimodularity.com/agentverify/dashboard and go to Dashboard -> API
@@ -0,0 +1,131 @@
1
+ // src/sdk.ts
2
+ var DEFAULT_API_URL = "https://agentverify-api.agentverify.workers.dev/v1/scan";
3
+ var DEFAULT_TIMEOUT = 3e4;
4
+ function normalizeVerdict(value) {
5
+ if (value === "VERIFIED") return "VERIFIED";
6
+ if (value === "NOT_ASSESSED" || value === "NOT ASSESSED") return "NOT_ASSESSED";
7
+ return "NOT_VERIFIED";
8
+ }
9
+ function normalizeScanResult(data) {
10
+ const findings = Array.isArray(data.findings) ? data.findings : [];
11
+ const riskScore = typeof data.riskScore === "number" ? data.riskScore : 0;
12
+ return {
13
+ ...data,
14
+ reportId: typeof data.reportId === "string" ? data.reportId : "",
15
+ verdict: normalizeVerdict(data.verdict),
16
+ riskScore,
17
+ riskLevel: data.riskLevel ?? "High Risk",
18
+ confidence: typeof data.confidence === "number" ? data.confidence : riskScore > 0 ? 80 : 0,
19
+ optimizationScore: typeof data.optimizationScore === "number" ? data.optimizationScore : 0,
20
+ reportInsights: data.reportInsights,
21
+ threatCategories: Array.isArray(data.threatCategories) ? data.threatCategories : [],
22
+ findings,
23
+ categoryScores: Array.isArray(data.categoryScores) ? data.categoryScores : [],
24
+ bom: data.bom ?? {
25
+ detectedLanguage: "Unknown",
26
+ detectedFramework: null,
27
+ detectedPlatform: null,
28
+ agentName: null,
29
+ toolAccessLevel: "Unknown",
30
+ credentialExposure: "Not Detected",
31
+ memoryPersistence: "Unknown",
32
+ auditLogging: "Unknown",
33
+ humanGates: "Unknown",
34
+ rateLimiting: "Unknown",
35
+ promptInjectionSurface: "Unknown",
36
+ delegationScope: "Unknown"
37
+ },
38
+ metadata: data.metadata ?? {
39
+ schemaVersion: "1.3.0",
40
+ scannerVersion: "unknown",
41
+ fileName: "unknown",
42
+ fileSize: 0,
43
+ scannedAt: (/* @__PURE__ */ new Date()).toISOString(),
44
+ detectedLanguage: "Unknown",
45
+ detectedFramework: null,
46
+ selectedPlatform: null,
47
+ agentName: null,
48
+ scanDuration: 0
49
+ },
50
+ saved: data.saved === true,
51
+ reportUrl: data.reportUrl ?? null,
52
+ artifactFingerprint: data.artifactFingerprint,
53
+ reportIntegrity: data.reportIntegrity,
54
+ policyProfile: data.policyProfile ?? null,
55
+ policyResult: data.policyResult ?? null,
56
+ attestation: data.attestation ?? null
57
+ };
58
+ }
59
+ async function scan(input, options) {
60
+ const { apiKey, apiUrl = DEFAULT_API_URL, timeout = DEFAULT_TIMEOUT } = options;
61
+ if (!apiKey) {
62
+ throw new Error(
63
+ "API key required. Generate one at https://aimodularity.com/agentverify/dashboard"
64
+ );
65
+ }
66
+ if (!input.content || input.content.trim().length === 0) {
67
+ throw new Error("content is required and cannot be empty");
68
+ }
69
+ const controller = new AbortController();
70
+ const timer = setTimeout(() => controller.abort(), timeout);
71
+ try {
72
+ const response = await fetch(apiUrl, {
73
+ method: "POST",
74
+ headers: {
75
+ "Authorization": `Bearer ${apiKey}`,
76
+ "Content-Type": "application/json",
77
+ "User-Agent": "agentverify-sdk/1.0.0"
78
+ },
79
+ body: JSON.stringify({
80
+ content: input.content,
81
+ fileName: input.fileName ?? "agent",
82
+ platform: input.platform,
83
+ policyId: input.policyId
84
+ }),
85
+ signal: controller.signal
86
+ });
87
+ if (response.status === 401) {
88
+ throw new Error(
89
+ "Invalid or unauthorized Agent Verify API key. Generate a valid key at https://aimodularity.com/agentverify/dashboard/"
90
+ );
91
+ }
92
+ if (response.status === 400) {
93
+ const err = await response.json().catch(() => ({}));
94
+ throw new Error(err.error ?? "Invalid request");
95
+ }
96
+ if (!response.ok) {
97
+ throw new Error(`API error: ${response.status} ${response.statusText}`);
98
+ }
99
+ const data = await response.json();
100
+ return normalizeScanResult(data);
101
+ } catch (err) {
102
+ if (err instanceof Error && err.name === "AbortError") {
103
+ throw new Error(`Request timed out after ${timeout}ms`);
104
+ }
105
+ throw err;
106
+ } finally {
107
+ clearTimeout(timer);
108
+ }
109
+ }
110
+ async function scanMany(inputs, options, concurrency = 3) {
111
+ const results = [];
112
+ for (let i = 0; i < inputs.length; i += concurrency) {
113
+ const batch = inputs.slice(i, i + concurrency);
114
+ const batchResults = await Promise.allSettled(
115
+ batch.map((input) => scan(input, options))
116
+ );
117
+ batchResults.forEach((res, j) => {
118
+ if (res.status === "fulfilled") {
119
+ results.push({ input: batch[j], result: res.value, error: null });
120
+ } else {
121
+ results.push({ input: batch[j], result: null, error: res.reason instanceof Error ? res.reason.message : String(res.reason) });
122
+ }
123
+ });
124
+ }
125
+ return results;
126
+ }
127
+
128
+ export {
129
+ scan,
130
+ scanMany
131
+ };
package/dist/cli.js CHANGED
@@ -8,6 +8,61 @@ var import_path = require("path");
8
8
  // src/sdk.ts
9
9
  var DEFAULT_API_URL = "https://agentverify-api.agentverify.workers.dev/v1/scan";
10
10
  var DEFAULT_TIMEOUT = 3e4;
11
+ function normalizeVerdict(value) {
12
+ if (value === "VERIFIED") return "VERIFIED";
13
+ if (value === "NOT_ASSESSED" || value === "NOT ASSESSED") return "NOT_ASSESSED";
14
+ return "NOT_VERIFIED";
15
+ }
16
+ function normalizeScanResult(data) {
17
+ const findings = Array.isArray(data.findings) ? data.findings : [];
18
+ const riskScore = typeof data.riskScore === "number" ? data.riskScore : 0;
19
+ return {
20
+ ...data,
21
+ reportId: typeof data.reportId === "string" ? data.reportId : "",
22
+ verdict: normalizeVerdict(data.verdict),
23
+ riskScore,
24
+ riskLevel: data.riskLevel ?? "High Risk",
25
+ confidence: typeof data.confidence === "number" ? data.confidence : riskScore > 0 ? 80 : 0,
26
+ optimizationScore: typeof data.optimizationScore === "number" ? data.optimizationScore : 0,
27
+ reportInsights: data.reportInsights,
28
+ threatCategories: Array.isArray(data.threatCategories) ? data.threatCategories : [],
29
+ findings,
30
+ categoryScores: Array.isArray(data.categoryScores) ? data.categoryScores : [],
31
+ bom: data.bom ?? {
32
+ detectedLanguage: "Unknown",
33
+ detectedFramework: null,
34
+ detectedPlatform: null,
35
+ agentName: null,
36
+ toolAccessLevel: "Unknown",
37
+ credentialExposure: "Not Detected",
38
+ memoryPersistence: "Unknown",
39
+ auditLogging: "Unknown",
40
+ humanGates: "Unknown",
41
+ rateLimiting: "Unknown",
42
+ promptInjectionSurface: "Unknown",
43
+ delegationScope: "Unknown"
44
+ },
45
+ metadata: data.metadata ?? {
46
+ schemaVersion: "1.3.0",
47
+ scannerVersion: "unknown",
48
+ fileName: "unknown",
49
+ fileSize: 0,
50
+ scannedAt: (/* @__PURE__ */ new Date()).toISOString(),
51
+ detectedLanguage: "Unknown",
52
+ detectedFramework: null,
53
+ selectedPlatform: null,
54
+ agentName: null,
55
+ scanDuration: 0
56
+ },
57
+ saved: data.saved === true,
58
+ reportUrl: data.reportUrl ?? null,
59
+ artifactFingerprint: data.artifactFingerprint,
60
+ reportIntegrity: data.reportIntegrity,
61
+ policyProfile: data.policyProfile ?? null,
62
+ policyResult: data.policyResult ?? null,
63
+ attestation: data.attestation ?? null
64
+ };
65
+ }
11
66
  async function scan(input, options) {
12
67
  const { apiKey, apiUrl = DEFAULT_API_URL, timeout = DEFAULT_TIMEOUT } = options;
13
68
  if (!apiKey) {
@@ -31,13 +86,14 @@ async function scan(input, options) {
31
86
  body: JSON.stringify({
32
87
  content: input.content,
33
88
  fileName: input.fileName ?? "agent",
34
- platform: input.platform
89
+ platform: input.platform,
90
+ policyId: input.policyId
35
91
  }),
36
92
  signal: controller.signal
37
93
  });
38
94
  if (response.status === 401) {
39
95
  throw new Error(
40
- "Invalid API key. Generate one at https://aimodularity.com/agentverify/dashboard"
96
+ "Invalid or unauthorized Agent Verify API key. Generate a valid key at https://aimodularity.com/agentverify/dashboard/"
41
97
  );
42
98
  }
43
99
  if (response.status === 400) {
@@ -48,7 +104,7 @@ async function scan(input, options) {
48
104
  throw new Error(`API error: ${response.status} ${response.statusText}`);
49
105
  }
50
106
  const data = await response.json();
51
- return data;
107
+ return normalizeScanResult(data);
52
108
  } catch (err) {
53
109
  if (err instanceof Error && err.name === "AbortError") {
54
110
  throw new Error(`Request timed out after ${timeout}ms`);
@@ -60,8 +116,13 @@ async function scan(input, options) {
60
116
  }
61
117
 
62
118
  // src/cli.ts
63
- var VERSION = "1.2.0";
64
- var API_URL = "https://agentverify-api.agentverify.workers.dev/v1/scan";
119
+ var VERSION = "1.4.0";
120
+ var API_URL = process.env.AGENTVERIFY_API_URL ?? "https://agentverify-api.agentverify.workers.dev/v1/scan";
121
+ var A2SPA_DOCS_URL = "https://aimodularity.com/A2SPA/docs";
122
+ var EXIT_OK = 0;
123
+ var EXIT_NOT_VERIFIED = 1;
124
+ var EXIT_NOT_ASSESSED = 2;
125
+ var EXIT_EXECUTION_ERROR = 3;
65
126
  var c = {
66
127
  reset: "\x1B[0m",
67
128
  bold: "\x1B[1m",
@@ -76,6 +137,7 @@ var c = {
76
137
  var write = (value = "") => process.stdout.write(`${value}
77
138
  `);
78
139
  var SUPPORTED_EXTENSIONS = /* @__PURE__ */ new Set([".js", ".ts", ".py", ".json", ".yaml", ".yml", ".md", ".mjs", ".cjs"]);
140
+ var MAX_SCAN_FILE_BYTES = 5 * 1024 * 1024;
79
141
  var SKIP_DIRS = /* @__PURE__ */ new Set(["node_modules", ".git", "dist", "build", ".next", "vendor", "coverage", ".cache"]);
80
142
  function collectFiles(dir) {
81
143
  const files = [];
@@ -103,8 +165,13 @@ function parseArgs(argv) {
103
165
  key: "",
104
166
  file: "",
105
167
  json: false,
168
+ markdown: false,
169
+ ci: false,
170
+ allowNotAssessed: false,
106
171
  help: false,
107
- version: false
172
+ version: false,
173
+ policy: "",
174
+ summaryFile: ""
108
175
  };
109
176
  if (result.command === "--help" || result.command === "-h") {
110
177
  result.command = "";
@@ -119,6 +186,11 @@ function parseArgs(argv) {
119
186
  if (arg === "--key" || arg === "-k") result.key = args[++i] ?? "";
120
187
  else if (arg === "--file" || arg === "-f") result.file = args[++i] ?? "";
121
188
  else if (arg === "--json") result.json = true;
189
+ else if (arg === "--markdown") result.markdown = true;
190
+ else if (arg === "--ci") result.ci = true;
191
+ else if (arg === "--allow-not-assessed") result.allowNotAssessed = true;
192
+ else if (arg === "--policy") result.policy = args[++i] ?? "";
193
+ else if (arg === "--summary-file") result.summaryFile = args[++i] ?? "";
122
194
  else if (arg === "--help" || arg === "-h") result.help = true;
123
195
  else if (arg === "--version" || arg === "-v") result.version = true;
124
196
  else if (!arg.startsWith("-")) result.dir = arg;
@@ -142,6 +214,14 @@ ${c.bold}Options:${c.reset}
142
214
  --key, -k API key (or set AGENTVERIFY_API_KEY env var)
143
215
  --file, -f Scan a single file instead of a directory
144
216
  --json Output results as JSON
217
+ --markdown Output a Markdown summary
218
+ --ci CI mode: concise output and non-zero exit for NOT VERIFIED
219
+ --allow-not-assessed
220
+ In CI mode, do not fail only because content is NOT_ASSESSED
221
+ --policy <id> Evaluate a built-in policy profile against this scan's evidence
222
+ in addition to the scanner's own verdict (e.g. financial-agent)
223
+ --summary-file <path>
224
+ Write results as JSON to this path (in addition to any other output)
145
225
  --version, -v Print version
146
226
  --help, -h Show this help message
147
227
 
@@ -158,17 +238,123 @@ ${c.bold}Examples:${c.reset}
158
238
  ${c.gray}# Output as JSON (useful for CI/CD)${c.reset}
159
239
  agentverify scan . --key av_your_key --json
160
240
 
241
+ ${c.gray}# Markdown summary for pull requests${c.reset}
242
+ agentverify scan . --key av_your_key --markdown
243
+
244
+ ${c.bold}Exit codes (--ci):${c.reset}
245
+ 0 All scanned files verified
246
+ 1 NOT VERIFIED \u2014 a security/verification check failed
247
+ 2 NOT ASSESSED \u2014 insufficient evidence for a verdict (see --allow-not-assessed)
248
+ 3 Execution error \u2014 the scan itself could not run (bad key, network, missing file)
249
+
161
250
  ${c.bold}Get your API key:${c.reset}
162
251
  https://aimodularity.com/agentverify/dashboard
163
252
 
164
253
  ${c.bold}Docs:${c.reset}
165
254
  https://github.com/AI-Blockchain-Ventures/agentverify
255
+
256
+ ${c.bold}A2SPA docs:${c.reset}
257
+ ${A2SPA_DOCS_URL}
258
+ `);
259
+ }
260
+ function getTopBlocker(result) {
261
+ return result.reportInsights?.topBlocker ?? result.findings[0]?.title ?? "No blocking finding detected";
262
+ }
263
+ function getFixFirst(result) {
264
+ const fromInsights = result.reportInsights?.fixPriority?.filter((item) => item.priority === "fix_first").map((item) => item.title) ?? [];
265
+ const fallback = result.findings.slice(0, 3).map((item) => item.title);
266
+ return (fromInsights.length ? fromInsights : fallback).slice(0, 3);
267
+ }
268
+ function getRelevantThreatCategories(result) {
269
+ return (result.threatCategories ?? []).filter((item) => item.status !== "not_assessed").slice(0, 6).map((item) => `${item.label} (${item.status.replace("_", " ")})`);
270
+ }
271
+ function criticalCount(result) {
272
+ return result.findings.filter((f) => f.severity === "critical").length;
273
+ }
274
+ function highCount(result) {
275
+ return result.findings.filter((f) => f.severity === "high").length;
276
+ }
277
+ function fingerprintPreview(result) {
278
+ const hash = result.artifactFingerprint?.artifactHash;
279
+ return hash ? `${hash.slice(0, 12)}\u2026` : null;
280
+ }
281
+ function printDetailedSummary(result, fileName) {
282
+ const verified = result.verdict === "VERIFIED";
283
+ const verdictColor = verified ? c.green : result.verdict === "NOT_ASSESSED" ? c.yellow : c.red;
284
+ write(`
285
+ ${c.bold}${c.cyan}Agent Verify${c.reset}`);
286
+ write(`${c.gray}${fileName}${c.reset}`);
287
+ write(`${c.bold}Score:${c.reset} ${result.riskScore}/100`);
288
+ write(`${c.bold}Verdict:${c.reset} ${verdictColor}${result.verdict === "NOT_VERIFIED" ? "NOT VERIFIED" : result.verdict.replace("_", " ")}${c.reset}`);
289
+ write(`${c.bold}Risk level:${c.reset} ${result.riskLevel}`);
290
+ write(`${c.bold}Confidence:${c.reset} ${result.confidence}/100`);
291
+ write(`${c.bold}Critical / High findings:${c.reset} ${criticalCount(result)} / ${highCount(result)}`);
292
+ if (result.metadata?.scannerVersion) write(`${c.bold}Ruleset version:${c.reset} ${result.metadata.scannerVersion}`);
293
+ if (fingerprintPreview(result)) write(`${c.bold}Artifact fingerprint:${c.reset} ${fingerprintPreview(result)}`);
294
+ if (result.policyProfile) write(`${c.bold}Policy (${result.policyProfile}):${c.reset} ${result.policyResult === "PASS" ? `${c.green}PASS${c.reset}` : `${c.red}FAIL${c.reset}`}`);
295
+ write(`${c.bold}Top blocker:${c.reset} ${getTopBlocker(result)}`);
296
+ write(`
297
+ ${c.bold}Fix first:${c.reset}`);
298
+ const fixes = getFixFirst(result);
299
+ if (fixes.length) fixes.forEach((item, index) => write(` ${index + 1}. ${item}`));
300
+ else write(` No blocking fixes detected. Re-scan after any material change.`);
301
+ if (result.reportInsights?.nextAction) write(`
302
+ ${c.bold}Next action:${c.reset} ${result.reportInsights.nextAction}`);
303
+ const threats = getRelevantThreatCategories(result);
304
+ if (threats.length) {
305
+ write(`
306
+ ${c.bold}Threat categories:${c.reset}`);
307
+ threats.forEach((item) => write(` - ${item}`));
308
+ }
309
+ if (result.saved && result.reportId) write(`${c.bold}View report:${c.reset} https://aimodularity.com/agentverify/report/?id=${result.reportId}`);
310
+ write(`${c.bold}A2SPA docs:${c.reset} ${A2SPA_DOCS_URL}`);
311
+ write(`${c.gray}Re-scan:${c.reset} agentverify scan --file ${fileName} --key $AGENTVERIFY_API_KEY
166
312
  `);
167
313
  }
314
+ function markdownSummary(results, summary) {
315
+ const lines = ["# Agent Verify", "", `Files scanned: ${summary.total}`, `Verified: ${summary.verified}`, `Not verified: ${summary.notVerified}`, `Not assessed: ${summary.notAssessed}`, `Errors: ${summary.errors}`, ""];
316
+ for (const item of results) {
317
+ if (!item.result) {
318
+ lines.push(`## ${item.file}`, "", `Error: ${item.error ?? "Unknown error"}`, "");
319
+ continue;
320
+ }
321
+ lines.push(`## ${item.file}`, "", `Score: ${item.result.riskScore}/100`, `Verdict: ${item.result.verdict === "NOT_VERIFIED" ? "NOT VERIFIED" : item.result.verdict.replace("_", " ")}`, `Risk level: ${item.result.riskLevel}`, `Critical / High findings: ${criticalCount(item.result)} / ${highCount(item.result)}`);
322
+ if (item.result.metadata?.scannerVersion) lines.push(`Ruleset version: ${item.result.metadata.scannerVersion}`);
323
+ if (fingerprintPreview(item.result)) lines.push(`Artifact fingerprint: \`${fingerprintPreview(item.result)}\``);
324
+ if (item.result.policyProfile) lines.push(`Policy (${item.result.policyProfile}): ${item.result.policyResult}`);
325
+ lines.push(`Top blocker: ${getTopBlocker(item.result)}`, "");
326
+ const fixes = getFixFirst(item.result);
327
+ if (fixes.length) lines.push("Fix first:", ...fixes.map((fix, index) => `${index + 1}. ${fix}`), "");
328
+ const threats = getRelevantThreatCategories(item.result);
329
+ if (threats.length) lines.push("Threat categories:", ...threats.map((threat) => `- ${threat}`), "");
330
+ if (item.result.reportId) lines.push(`Report: https://aimodularity.com/agentverify/report/?id=${item.result.reportId}`, "");
331
+ }
332
+ lines.push(`A2SPA docs: ${A2SPA_DOCS_URL}`);
333
+ return lines.join("\n");
334
+ }
335
+ function writeGithubJobSummary(results, summary) {
336
+ const summaryPath = process.env.GITHUB_STEP_SUMMARY;
337
+ if (!summaryPath) return;
338
+ try {
339
+ (0, import_fs.appendFileSync)(summaryPath, `${markdownSummary(results, summary)}
340
+ `);
341
+ } catch (e) {
342
+ console.error(`Warning: could not write GitHub job summary: ${e instanceof Error ? e.message : e}`);
343
+ }
344
+ }
345
+ function writeSummaryFile(path, results, summary) {
346
+ if (!path) return;
347
+ try {
348
+ (0, import_fs.writeFileSync)(path, JSON.stringify({ results, summary }, null, 2));
349
+ } catch (e) {
350
+ console.error(`Warning: could not write summary file: ${e instanceof Error ? e.message : e}`);
351
+ }
352
+ }
168
353
  function printVerdict(result, fileName) {
169
354
  const verified = result.verdict === "VERIFIED";
170
- const icon = verified ? `${c.green}\u2713${c.reset}` : `${c.red}\u2717${c.reset}`;
171
- const verdict = verified ? `${c.green}VERIFIED${c.reset} ` : `${c.red}NOT VERIFIED${c.reset}`;
355
+ const notAssessed = result.verdict === "NOT_ASSESSED";
356
+ const icon = verified ? `${c.green}\u2713${c.reset}` : notAssessed ? `${c.yellow}\u26A0${c.reset}` : `${c.red}\u2717${c.reset}`;
357
+ const verdict = verified ? `${c.green}VERIFIED${c.reset} ` : notAssessed ? `${c.yellow}NOT ASSESSED${c.reset}` : `${c.red}NOT VERIFIED${c.reset}`;
172
358
  const score = verified ? `${c.green}${result.riskScore}/100${c.reset}` : result.riskScore >= 50 ? `${c.yellow}${result.riskScore}/100${c.reset}` : `${c.red}${result.riskScore}/100${c.reset}`;
173
359
  const findings = result.findings.length > 0 ? `${c.gray}\u2014 ${result.findings.length} finding${result.findings.length === 1 ? "" : "s"}${c.reset}` : "";
174
360
  const name = fileName.padEnd(30);
@@ -182,20 +368,23 @@ async function runScan(args) {
182
368
  console.error(`
183
369
  ${c.red}Error:${c.reset} API key required
184
370
  `);
185
- console.error(` Set with --key av_your_key or AGENTVERIFY_API_KEY env var`);
186
- console.error(` Get your key at https://aimodularity.com/agentverify/dashboard
371
+ console.error(` Set with --key av_your_key or AGENTVERIFY_API_KEY env var.`);
372
+ console.error(` Get your key at https://aimodularity.com/agentverify/dashboard/`);
373
+ console.error(` Example: AGENTVERIFY_API_KEY=av_your_key agentverify scan .
187
374
  `);
188
- process.exit(1);
375
+ process.exit(EXIT_EXECUTION_ERROR);
189
376
  }
190
377
  let files = [];
191
378
  if (args.file) {
192
379
  files = [args.file];
193
380
  } else {
194
381
  const dir = args.dir;
195
- write(`
382
+ if (!args.markdown && !args.json) {
383
+ write(`
196
384
  ${c.bold}Agent Verify${c.reset} ${c.gray}v${VERSION}${c.reset}`);
197
- write(`${c.gray}Scanning ${dir}...${c.reset}
385
+ write(`${c.gray}Scanning ${dir}...${c.reset}
198
386
  `);
387
+ }
199
388
  files = collectFiles(dir);
200
389
  if (files.length === 0) {
201
390
  write(`${c.yellow}No agent files found in ${dir}${c.reset}`);
@@ -207,16 +396,21 @@ ${c.bold}Agent Verify${c.reset} ${c.gray}v${VERSION}${c.reset}`);
207
396
  const jsonResults = [];
208
397
  let verified = 0;
209
398
  let notVerified = 0;
399
+ let notAssessed = 0;
210
400
  let errors = 0;
211
401
  const CONCURRENCY = 3;
212
402
  for (let i = 0; i < files.length; i += CONCURRENCY) {
213
403
  const batch = files.slice(i, i + CONCURRENCY);
214
404
  const batchResults = await Promise.allSettled(
215
405
  batch.map(async (filePath) => {
406
+ const fileStat = (0, import_fs.statSync)(filePath);
407
+ if (fileStat.size > MAX_SCAN_FILE_BYTES) {
408
+ throw new Error(`File is ${(fileStat.size / 1024 / 1024).toFixed(1)}MB, over the ${MAX_SCAN_FILE_BYTES / 1024 / 1024}MB scan limit \u2014 skipped`);
409
+ }
216
410
  const content = (0, import_fs.readFileSync)(filePath, "utf-8");
217
411
  const relPath = (0, import_path.relative)(args.dir === "." ? process.cwd() : args.dir, filePath);
218
412
  const result = await scan(
219
- { content, fileName: relPath },
413
+ { content, fileName: relPath, policyId: args.policy || void 0 },
220
414
  { apiKey: args.key, apiUrl: API_URL }
221
415
  );
222
416
  return { filePath, relPath, result };
@@ -226,9 +420,13 @@ ${c.bold}Agent Verify${c.reset} ${c.gray}v${VERSION}${c.reset}`);
226
420
  const outcome = batchResults[j];
227
421
  if (outcome.status === "fulfilled") {
228
422
  const { relPath, result } = outcome.value;
229
- if (!args.json) printVerdict(result, relPath);
423
+ if (!args.json && !args.markdown) {
424
+ if (args.file && !args.ci) printDetailedSummary(result, relPath);
425
+ else printVerdict(result, relPath);
426
+ }
230
427
  jsonResults.push({ file: relPath, result, error: null });
231
428
  if (result.verdict === "VERIFIED") verified++;
429
+ else if (result.verdict === "NOT_ASSESSED") notAssessed++;
232
430
  else notVerified++;
233
431
  } else {
234
432
  const filePath = batch[j];
@@ -242,32 +440,58 @@ ${c.bold}Agent Verify${c.reset} ${c.gray}v${VERSION}${c.reset}`);
242
440
  }
243
441
  }
244
442
  }
443
+ const summary = { total: files.length, verified, notVerified, notAssessed, errors };
444
+ writeGithubJobSummary(jsonResults, summary);
445
+ writeSummaryFile(args.summaryFile, jsonResults, summary);
245
446
  if (args.json) {
246
- write(JSON.stringify({ results: jsonResults, summary: { total: files.length, verified, notVerified, errors } }, null, 2));
447
+ write(JSON.stringify({ results: jsonResults, summary }, null, 2));
448
+ if (errors > 0) process.exit(EXIT_EXECUTION_ERROR);
449
+ if (args.ci) {
450
+ if (notVerified > 0) process.exit(EXIT_NOT_VERIFIED);
451
+ if (notAssessed > 0 && !args.allowNotAssessed) process.exit(EXIT_NOT_ASSESSED);
452
+ }
247
453
  return;
248
454
  }
455
+ if (args.markdown) {
456
+ write(markdownSummary(jsonResults, summary));
457
+ if (errors > 0) process.exit(EXIT_EXECUTION_ERROR);
458
+ if (args.ci) {
459
+ if (notVerified > 0) process.exit(EXIT_NOT_VERIFIED);
460
+ if (notAssessed > 0 && !args.allowNotAssessed) process.exit(EXIT_NOT_ASSESSED);
461
+ }
462
+ return;
463
+ }
464
+ const savedReports = jsonResults.filter((item) => item.result?.saved && item.result.reportId).length;
249
465
  write(`
250
466
  ${c.gray}${"\u2500".repeat(60)}${c.reset}`);
251
467
  write(`${c.bold}Agent Verify Scan Complete${c.reset}`);
252
468
  write(`${c.gray}Files scanned:${c.reset} ${files.length}`);
253
469
  write(`${c.green}Verified:${c.reset} ${verified}`);
254
470
  if (notVerified > 0) write(`${c.red}Not Verified:${c.reset} ${notVerified}`);
471
+ if (notAssessed > 0) write(`${c.yellow}Not Assessed:${c.reset} ${notAssessed}`);
255
472
  if (errors > 0) write(`${c.yellow}Errors:${c.reset} ${errors}`);
256
473
  write(`${c.gray}${"\u2500".repeat(60)}${c.reset}`);
257
- write(`
258
- ${c.gray}View reports: https://aimodularity.com/agentverify/dashboard${c.reset}
474
+ if (savedReports > 0) write(`
475
+ ${c.gray}Saved CLI reports appear in your dashboard: https://aimodularity.com/agentverify/dashboard/${c.reset}`);
476
+ else write(`
477
+ ${c.gray}No dashboard report was saved. Check API key access and retry if you need dashboard sync.${c.reset}`);
478
+ write(`${c.gray}A2SPA docs: ${A2SPA_DOCS_URL}${c.reset}
259
479
  `);
260
- if (notVerified > 0) process.exit(1);
480
+ if (errors > 0) process.exit(EXIT_EXECUTION_ERROR);
481
+ if (args.ci) {
482
+ if (notVerified > 0) process.exit(EXIT_NOT_VERIFIED);
483
+ if (notAssessed > 0 && !args.allowNotAssessed) process.exit(EXIT_NOT_ASSESSED);
484
+ }
261
485
  }
262
486
  async function main() {
263
487
  const args = parseArgs(process.argv);
264
488
  if (args.version) {
265
489
  write(`agentverify v${VERSION}`);
266
- process.exit(0);
490
+ process.exit(EXIT_OK);
267
491
  }
268
492
  if (args.help || !args.command) {
269
493
  printHelp();
270
- process.exit(0);
494
+ process.exit(EXIT_OK);
271
495
  }
272
496
  if (args.command === "scan") {
273
497
  await runScan(args);
@@ -276,11 +500,11 @@ async function main() {
276
500
  ${c.red}Unknown command: ${args.command}${c.reset}`);
277
501
  console.error(`Run ${c.cyan}agentverify --help${c.reset} for usage
278
502
  `);
279
- process.exit(1);
503
+ process.exit(EXIT_EXECUTION_ERROR);
280
504
  }
281
505
  }
282
506
  main().catch((err) => {
283
507
  console.error(`
284
508
  ${c.red}Fatal error:${c.reset}`, err instanceof Error ? err.message : err);
285
- process.exit(1);
509
+ process.exit(EXIT_EXECUTION_ERROR);
286
510
  });