@blamejs/exceptd-skills 0.18.16 → 0.18.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,31 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.18.17 — 2026-06-24
4
+
5
+ A correctness pass across the upstream-refresh pipeline, the credential and MCP collectors, scoring and framework analysis, attestation hashing, and the release gates.
6
+
7
+ The scheduled data refresh no longer fails on transient upstream throttling. The cache-warm step now separates retryable errors the upstream is asking us to back off on — a rate-limit (HTTP 429), a timeout, a 5xx — from hard data errors like a 404 or a parse failure. Only hard errors count toward the failure budget; transient throttling is surfaced in the run summary and retried on the next run. Previously a burst of NVD rate-limits failed the entire refresh and skipped the automated update PR. A feed that returns nothing at all still fails. The exported `prefetch()` API also honors `EXCEPTD_AIR_GAP=1` now, so a programmatic call from an air-gapped host no longer reaches the network.
8
+
9
+ Cleartext-credential detection covers more of what it claims. An Azure, Google, or Cohere API key found in a shell dotfile now surfaces as a confirmed finding instead of being downgraded to inconclusive. An AWS profile that carries a static access key alongside a `role_arn` — the assume-role bootstrap shape — is flagged as a static-key exposure rather than treated as fully federated. On Windows, the POSIX-permission posture checks (world-writable env files, SSH key modes) report inconclusive rather than a false "clean" for a check that cannot run there.
10
+
11
+ MCP integrity detection catches single-token launch commands. A server started with a whole pip invocation in one argument — `sh -c "pip install pkg==1.2.3"` — is now flagged as pinned-without-integrity; previously only the split-argument form was caught.
12
+
13
+ "Rejected or disputed CVE" citations no longer false-positive. The check fired on any note that mentioned a dispute — including a CVSS *scoring* dispute, a disclosure-coordination dispute, or a different CVE the note merely referenced. It now fires only when the cited CVE's own record is rejected or disputed, and no longer attaches a false attestation that suppressed the downgrade for a valid, often actively-exploited citation.
14
+
15
+ The runtime collector's `collect | run` pipe reaches analysis. The collector now attests the read-only-inventory precondition the runner cannot resolve mechanically, so `collect runtime | run runtime` no longer halts at preflight despite valid evidence.
16
+
17
+ The RWEP-vs-CVSS explanation lists every factor it counts. Suspected and unknown active-exploitation (contributing +10 and +5) and the blast-radius contribution were omitted from the "driving factors" sentence, so a divergence driven by them showed a higher score with an understated — sometimes empty — reason. Both the driving and mitigating sides now enumerate the real contributions and name the structural cause when no single factor dominates.
18
+
19
+ Framework gap analysis resolves frameworks whose catalog label differs from their formal name. The Australian Government ISM — and any framework in the same situation — reported zero control gaps and null metadata because its catalog labels did not match its registered name; a data-driven alias map now resolves both, and a framework that resolves to zero gaps is flagged instead of silently empty.
20
+
21
+ EPSS and NVD scores are matched to the requested CVE by id. The cache-diff and KEV auto-import paths fell back to the first row or record in a payload, which could attribute one CVE's EPSS or CVSS to another when a cache entry held a different CVE's response. They now match by id and skip on a mismatch.
22
+
23
+ Attestation and reattestation no longer report false drift from a relabeled observation. The evidence hash, and the deterministic session id derived from it, re-key evidence by the stable indicator it targets, so identical evidence submitted under a different observation label produces the same hash; a changed evidence value still changes it.
24
+
25
+ The GHSA and OSV regression watch stops flagging editorial fields. Curated-only fields — AI-discovery, PoC availability, active-exploitation — were treated as upstream-sourced and flagged as "dropped" on every re-import; the watch now covers only fields the feed actually populates, and the diff summary counts new entries and field regressions separately rather than labeling both as new.
26
+
27
+ Release gates fail closed on absent input. The version-cadence, test-subject, and code-pattern checks each had a path that passed without checking anything — an unreadable changelog, an empty playbook or module set, an empty scan universe. Each now fails rather than reporting clean when it has nothing to check.
28
+
3
29
  ## 0.18.16 — 2026-06-22
4
30
 
5
31
  A correctness pass on the evidence-output and attestation surfaces.
@@ -1,10 +1,10 @@
1
1
  {
2
2
  "schema_version": "1.1.0",
3
- "generated_at": "2026-06-22T17:11:04.498Z",
3
+ "generated_at": "2026-06-25T03:42:18.191Z",
4
4
  "generator": "scripts/build-indexes.js",
5
5
  "source_count": 64,
6
6
  "source_hashes": {
7
- "manifest.json": "c8376cbda42a9657b6a5d50f75d8c08a0bcab7c8456340042be7998133922ecb",
7
+ "manifest.json": "20836959eb7562901f9fb4445fa930f3e6b9890439eb97edefa00de23ff40d93",
8
8
  "README.md": "e7b854e7db9a364a1b368b5084b4f0c2a8282f0459ce39800ac1d1dabdc06074",
9
9
  "data/atlas-ttps.json": "5bc59e23d6c2defa54168de161a0825299b9cc4a49c6b26df2dae70b4f42eedf",
10
10
  "data/attack-techniques.json": "53c6f248760eecb11a0354f74ab467a5814e95075a686b9b3bf18c34e2f7435e",
@@ -14,7 +14,7 @@
14
14
  "data/dlp-controls.json": "d2406c482dddd30e49203879999dc4b3a7fd4d0494d6a61d86b91ee76415df19",
15
15
  "data/exploit-availability.json": "ec2656f0d9a893610e27b43eb6035fe9b18e057c9f6dfaac7e7d4959bbcbb795",
16
16
  "data/framework-control-gaps.json": "760c2275803c6da3665ce538c5176bde6f041b68cc3d4808b8de961dfcdee6b8",
17
- "data/global-frameworks.json": "9ba563a85f7f8d6c3c957de64945e20925a89d0ed6ea6fc561cf093811acf558",
17
+ "data/global-frameworks.json": "20b18a851b3ab82e57cd159b49476d69fe544f02523398b2015189ab81c7c3f8",
18
18
  "data/rfc-references.json": "5db2f7006a1d7f2b8642a3e393213394e92798604576df973d0a0b8550f4b5ac",
19
19
  "data/zeroday-lessons.json": "8c69eec9103eeea236ceb1a157d62b35bafcf8a5de86502e25e4587cc5931247",
20
20
  "skills/kernel-lpe-triage/skill.md": "0f79c641cef6e5f4a942eb94f43c460562bf83dfb67ae112d146c39c6b320fb0",
@@ -256,6 +256,7 @@
256
256
  "frameworks": {
257
257
  "ASD_ISM": {
258
258
  "full_name": "Australian Signals Directorate Information Security Manual",
259
+ "catalog_aliases": ["au-ism", "AU ISM", "ACSC ISM", "Australian Government Information Security Manual"],
259
260
  "authority": "Australian Signals Directorate (ASD)",
260
261
  "source": "https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/ism",
261
262
  "effective_date": "Monthly updates",
@@ -185,8 +185,15 @@ function readCachedJson(cacheDir, source, id) {
185
185
  return parsed;
186
186
  }
187
187
 
188
- function extractNvdMetrics(payload) {
189
- const vuln = payload?.vulnerabilities?.[0]?.cve;
188
+ function extractNvdMetrics(payload, id) {
189
+ // Resolve the NVD vuln by id, not by position. vulnerabilities[0] blindly
190
+ // took the first record, so a cache entry keyed under `id` that held another
191
+ // CVE's response would attribute that CVE's CVSS/CWE/description to this id.
192
+ // When no id is supplied (legacy callers), fall back to the first record.
193
+ const cves = (payload?.vulnerabilities || []).map((v) => v?.cve).filter(Boolean);
194
+ const vuln = id
195
+ ? cves.find((c) => c.id && String(c.id).toUpperCase() === String(id).toUpperCase())
196
+ : cves[0];
190
197
  if (!vuln) return null;
191
198
  // Prefer the newest CVSS version (Primary within it) and normalize a bare
192
199
  // v2 vector to its canonical prefix so an auto-imported draft never carries
@@ -238,7 +245,7 @@ function extractEpss(payload, id) {
238
245
  */
239
246
  function buildKevDraftEntry(kevEntry, nvdPayload, epssPayload) {
240
247
  const id = String(kevEntry.cveID);
241
- const nvd = nvdPayload ? extractNvdMetrics(nvdPayload) : null;
248
+ const nvd = nvdPayload ? extractNvdMetrics(nvdPayload, id) : null;
242
249
  const epss = epssPayload ? extractEpss(epssPayload, id) : null;
243
250
 
244
251
  const knownRansomware =
@@ -63,6 +63,16 @@ const AI_KEY_VALUE_RE = {
63
63
  openai: /OPENAI_API_KEY\s*[= ]\s*['"]?(sk-[A-Za-z0-9_-]+)/,
64
64
  anthropic: /ANTHROPIC_API_KEY\s*[= ]\s*['"]?(sk-ant-[A-Za-z0-9_-]+)/,
65
65
  huggingface: /(?:HUGGINGFACE_TOKEN|HF_TOKEN)\s*[= ]\s*['"]?(hf_[A-Za-z0-9]+)/,
66
+ // Azure/Google/Cohere keys carry no vendor prefix, so the captured value IS
67
+ // the entropy body. Without these, cleartextFpIndices found no value for an
68
+ // azure/google/cohere-only dotfile, returned an empty attestation, and the
69
+ // runner downgraded a real cleartext-key hit to inconclusive — the indicator
70
+ // fired then vanished for half the supported vendors. The 30-char entropy
71
+ // floor (the `: 30` else-branch below) and PLACEHOLDER_RE both apply, since
72
+ // the prefix-strip leaves these values unchanged.
73
+ azure: /AZURE_OPENAI(?:_API)?_KEY\s*[= ]\s*['"]?([A-Za-z0-9]{20,})/,
74
+ google: /(?:GOOGLE_API_KEY|GOOGLE_GENAI_API_KEY|GEMINI_API_KEY)\s*[= ]\s*['"]?([A-Za-z0-9_-]{20,})/,
75
+ cohere: /COHERE_API_KEY\s*[= ]\s*['"]?([A-Za-z0-9-]{30,})/,
66
76
  };
67
77
  const PLACEHOLDER_RE = /placeholder|example|redacted|dummy|x{4,}|0{6,}|test-/i;
68
78
 
@@ -110,8 +110,50 @@ const CVE_CANONICAL_RE = /^CVE-\d{4}-\d{4,}$/;
110
110
  // RFC citation: `RFC 9404`, `RFC9404`, `RFC-9404`. Capture the number.
111
111
  const RFC_CITATION_RE = /RFC[\s-]?(\d{1,5})\b/gi;
112
112
 
113
- // Words that mark a catalog note as recording a rejected / disputed status.
114
- const REJECT_DISPUTE_RE = /\b(reject(?:ed|s|ion)?|disputed?|withdrawn)\b/i;
113
+ // A catalog note records a rejected/disputed RECORD for THIS CVE only when a
114
+ // reject/dispute/withdraw word refers to the citation's own record — not to a
115
+ // CVSS *scoring* disagreement, a disclosure-coordination dispute, or a
116
+ // DIFFERENT CVE the note merely mentions. A bare word-anywhere scan matched all
117
+ // three, producing a false "citation to a rejected record" hit AND a false
118
+ // __fp_checks[1] attestation (telling the runner not to downgrade), so a valid,
119
+ // often actively-exploited CVE surfaced as a confirmed rejected/disputed
120
+ // citation. `selfId` is the citation's own CVE id.
121
+ function recordRejectedOrDisputed(note, selfId) {
122
+ if (!note) return false;
123
+ const self = String(selfId || "").toUpperCase();
124
+ const re = /\b(reject(?:ed|s|ion)?|disputed?|withdrawn)\b/gi;
125
+ // Qualifier nouns that make a "dispute" a disagreement about something OTHER
126
+ // than the record's validity (the score, the severity, the disclosure
127
+ // process, …). 'rejected'/'withdrawn' are record-level words and bypass this.
128
+ const QUALIFIER = /\b(cvss|scoring|score|severity|coordination|disclosure|methodolog\w*|attribution|naming|assignment|priorit\w*)\b/i;
129
+ // A "duplicate of / superseded by / replaced by / merged into / in favour of"
130
+ // construction names the REPLACEMENT cve — THIS record is still the rejected
131
+ // one, so a different cve appearing as that replacement must NOT suppress the
132
+ // flag (e.g. "this record was rejected as a duplicate of CVE-Y").
133
+ const REPLACEMENT_OF = /\b(?:duplicate|dup)\b[\s\w-]*\bof\b|\b(?:supersed\w+|replaced|merged)\b[\s\w-]*\b(?:by|into)\b|\bin\s+favou?r\s+of\b/i;
134
+ const otherCve = (s) => (s.match(/CVE-\d{4}-\d{4,}/gi) || []).some((c) => c.toUpperCase() !== self);
135
+ let m;
136
+ while ((m = re.exec(note)) !== null) {
137
+ const word = m[1].toLowerCase();
138
+ const before = note.slice(Math.max(0, m.index - 60), m.index);
139
+ const after = note.slice(re.lastIndex, re.lastIndex + 60);
140
+ // (a) A different cve BEFORE the word is the subject ("CVE-Y was rejected")
141
+ // — the status is about that record, not this citation.
142
+ if (otherCve(before)) continue;
143
+ // (b) A different cve AFTER the word suppresses too, UNLESS it is the
144
+ // replacement target of a duplicate-of/superseded-by construction, in
145
+ // which case THIS record is the rejected one — keep it flagged.
146
+ if (otherCve(after) && !REPLACEMENT_OF.test(after)) continue;
147
+ // (c) A 'dispute(d)' qualified by a non-record noun is a disagreement about
148
+ // that noun, not a record rejection.
149
+ if (word.startsWith("disput")) {
150
+ const lastTokens = before.trim().split(/[\s-]+/).slice(-3).join(" ");
151
+ if (QUALIFIER.test(lastTokens)) continue;
152
+ }
153
+ return true;
154
+ }
155
+ return false;
156
+ }
115
157
 
116
158
  // Draft-language proximity for the (unflipped) draft-as-RFC heuristic.
117
159
  const DRAFT_LANGUAGE_RE = /\b(draft-[a-z0-9-]+|internet[- ]draft|work[- ]in[- ]progress|i-d\b)\b/i;
@@ -356,7 +398,7 @@ function collect({ cwd = process.cwd() } = {}) {
356
398
  // Well-formed. Cross-reference the catalog.
357
399
  if (cveKeys.has(full)) {
358
400
  const note = cveNotes.get(full) || "";
359
- if (REJECT_DISPUTE_RE.test(note) && !illustrative) {
401
+ if (recordRejectedOrDisputed(note, full) && !illustrative) {
360
402
  hits["rejected-or-disputed-cve"].push({ file: f.rel, citation: full, line: cveLine });
361
403
  }
362
404
  } else if (catalogsLoaded && !illustrative) {
@@ -603,4 +645,4 @@ async function applyResolution(submission, opts = {}) {
603
645
  return out;
604
646
  }
605
647
 
606
- module.exports = { playbook_id: COLLECTOR_ID, collect, applyResolution };
648
+ module.exports = { playbook_id: COLLECTOR_ID, collect, applyResolution, recordRejectedOrDisputed };
@@ -84,7 +84,16 @@ function parseAwsCredentials(content) {
84
84
  const staticKeys = {};
85
85
  for (const [name, kv] of Object.entries(profiles)) {
86
86
  const hasKey = !!kv["aws_access_key_id"];
87
- const hasFederation = !!(kv["sso_session"] || kv["credential_process"] || kv["role_arn"]);
87
+ // role_arn is NOT a federation marker for this predicate. A profile that
88
+ // carries a static aws_access_key_id (AKIA*) alongside a role_arn is the
89
+ // canonical source_profile/assume-role setup where the static key IS the
90
+ // long-lived credential that bootstraps the assumed role — a real static-key
91
+ // exposure. The playbook's aws-static-key-present predicate lists only
92
+ // sso_session / credential_process as federation; treating role_arn as
93
+ // federation here suppressed a genuinely-present IAM user key. A pure
94
+ // assume-role profile (role_arn, no own access key) is still excluded by the
95
+ // hasKey gate below.
96
+ const hasFederation = !!(kv["sso_session"] || kv["credential_process"]);
88
97
  if (hasKey && !hasFederation) {
89
98
  staticProfiles.push(name);
90
99
  staticKeys[name] = kv["aws_access_key_id"];
@@ -120,7 +120,13 @@ function isPinnedNoIntegrity(server) {
120
120
  if (typeof tok !== "string") continue;
121
121
  // npm package@version (excluding scope name@version)
122
122
  if (/@[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+@\d+\.\d+\.\d+\b/.test(tok)) return true;
123
- if (/\bpip\s+install\b/.test(tok)) continue;
123
+ // A `==X.Y.Z` pin (pip/uvx/poetry) with no integrity is the python analog
124
+ // of the npm pin above. A single-token launch command like
125
+ // `sh -c "pip install some-mcp==1.2.3"` carries the whole pip invocation in
126
+ // one token, so testing the token for `pip install` and `continue`-ing here
127
+ // skipped the `==version` check below for exactly the shape the comment
128
+ // says operators care about. A bare `pip install requests` (no `==`) can't
129
+ // match the pin regex, so checking every token is safe.
124
130
  if (/==\d+\.\d+\.\d+/.test(tok)) return true;
125
131
  }
126
132
  return false;
@@ -341,6 +341,17 @@ function collect({ cwd = process.cwd(), env = process.env, args = {} } = {}) {
341
341
  return {
342
342
  precondition_checks: {
343
343
  "linux-platform": true,
344
+ // The collector performs a read-only inventory directly via fs (reads
345
+ // /etc/sudoers, /etc/passwd, /proc/<pid>/status) and execs no commands,
346
+ // so the `exec-allowed` precondition ('permitted to execute read-only
347
+ // inventory commands') is satisfied by construction once the collection
348
+ // reaches at least one inventory source. The runner cannot mechanically
349
+ // resolve `agent_has_command_exec == true`, so without this attestation
350
+ // the on_fail=halt preflight blocked the canonical `collect runtime | run
351
+ // runtime` pipe even with valid evidence — the sibling fs-only cred-stores
352
+ // collector attests its own halt precondition the same way. Gated on
353
+ // readability so a fully-masked /proc+/etc scope reports it false.
354
+ "exec-allowed": Boolean(sudoersReadable || passwdContent != null || procWalkable || anyTpReadable),
344
355
  },
345
356
  artifacts,
346
357
  signal_overrides,
@@ -488,7 +488,14 @@ function collect({ cwd = process.cwd(), env = process.env, args = {} } = {}) {
488
488
  // any .env / .env.* / .envrc with mode 0666 or 0664 (group/world writable)
489
489
  // i.e. group-write OR world-write bit set (mode & 0o022).
490
490
  const envFilePostures = process.platform === "win32" ? [] : envFiles.map(f => ({ file: f.rel, ...statPosture(f.full) }));
491
- signal_overrides["world-writable-env-file"] = envFilePostures.some(p => p.error == null && (p.mode & 0o022) !== 0) ? "hit" : "miss";
491
+ // POSIX mode bits are unreadable on win32, so this posture cannot be checked
492
+ // there. OMIT the signal on Windows (mirroring cred-stores' credentials-file-
493
+ // bad-perms) so the runner returns `inconclusive` rather than a deterministic
494
+ // false `miss` for a check that physically cannot run. (The posture array is
495
+ // still declared — empty on win32 — for the SARIF per-indicator locations.)
496
+ if (process.platform !== "win32") {
497
+ signal_overrides["world-writable-env-file"] = envFilePostures.some(p => p.error == null && (p.mode & 0o022) !== 0) ? "hit" : "miss";
498
+ }
492
499
 
493
500
  // ssh-key-bad-perms predicate (per playbook):
494
501
  // restricted to ssh-private-keys artifact + ~/.ssh/id_* paths
@@ -499,7 +506,11 @@ function collect({ cwd = process.cwd(), env = process.env, args = {} } = {}) {
499
506
  // (prodSshPrivateKeys) — matching ssh-private-key-block — so a fixture
500
507
  // key checked in under a test/ path doesn't raise a bad-perms posture.
501
508
  const sshKeyPostures = process.platform === "win32" ? [] : prodSshPrivateKeys.map(f => ({ file: f.rel, ...statPosture(f.full) }));
502
- signal_overrides["ssh-key-bad-perms"] = sshKeyPostures.some(p => p.error == null && p.mode !== 0o600) ? "hit" : "miss";
509
+ // Same as world-writable-env-file: omit on win32 (mode bits unreadable) so the
510
+ // runner returns inconclusive instead of a forced false `miss`.
511
+ if (process.platform !== "win32") {
512
+ signal_overrides["ssh-key-bad-perms"] = sshKeyPostures.some(p => p.error == null && p.mode !== 0o600) ? "hit" : "miss";
513
+ }
503
514
 
504
515
  // Per-indicator file locations for every indicator flipped to "hit", so
505
516
  // a SARIF result points at the file carrying the secret / bad posture.
@@ -303,13 +303,67 @@ function bySkill(skillName) {
303
303
  return { skill: skillName, summary_card: card, cve_refs: cveRefs, ttp_refs: ttpRefs };
304
304
  }
305
305
 
306
+ // global-frameworks.json is keyed by REGION (EU/UK/AU/...), each with a nested
307
+ // `frameworks: { SHORTKEY: {full_name, catalog_aliases?, ...} }` map. A flat
308
+ // `global[frameworkId]` lookup therefore ALWAYS returned null — frameworkId is a
309
+ // short key or a catalog display name, never a region — so framework_meta was
310
+ // universally null. Walk the nested structure and match the requested id against
311
+ // the short key, the full_name, or any catalog_alias (normalized), returning the
312
+ // matched framework object annotated with its region + jurisdiction.
313
+ function resolveFrameworkMeta(global, frameworkId) {
314
+ if (!global || frameworkId == null) return null;
315
+ const norm = (s) => String(s == null ? '' : s).toLowerCase().replace(/\([^)]*\)/g, '').replace(/[\s_-]/g, '');
316
+ const want = norm(frameworkId);
317
+ if (!want) return null;
318
+ for (const [region, rv] of Object.entries(global)) {
319
+ if (region === '_meta' || !rv || typeof rv !== 'object') continue;
320
+ const fws = rv.frameworks || {};
321
+ for (const [shortKey, fv] of Object.entries(fws)) {
322
+ if (!fv || typeof fv !== 'object') continue;
323
+ const aliases = Array.isArray(fv.catalog_aliases) ? fv.catalog_aliases.map(norm) : [];
324
+ if (shortKey === frameworkId ||
325
+ norm(shortKey) === want ||
326
+ (fv.full_name && norm(fv.full_name) === want) ||
327
+ aliases.some((a) => a && (a === want || a.includes(want) || want.includes(a)))) {
328
+ return { ...fv, _framework_key: shortKey, _region: region, _jurisdiction: rv.jurisdiction || null };
329
+ }
330
+ }
331
+ }
332
+ return null;
333
+ }
334
+
306
335
  function byFramework(frameworkId) {
307
336
  const gaps = loadCatalog('framework-control-gaps.json');
308
337
  const global = loadCatalog('global-frameworks.json');
338
+ const fwMeta = resolveFrameworkMeta(global, frameworkId);
339
+ // Match gap rows by the framework's full LABEL SET, not just the literal id.
340
+ // The catalog stores a framework's gaps under labels (au-ism / AU ISM / ACSC
341
+ // ISM …) that diverge from its short key and full_name, so an exact
342
+ // `g.framework === frameworkId` match returned a gap set inconsistent with the
343
+ // (now alias-resolved) framework_meta — non-null metadata but a partial,
344
+ // sometimes empty, gap list. Resolve the same alias set used for the metadata
345
+ // and match any gap label (string or array element) against it.
346
+ const norm = (s) => String(s == null ? '' : s).toLowerCase().replace(/\([^)]*\)/g, '').replace(/[\s_-]/g, '');
347
+ const labels = new Set([norm(frameworkId)]);
348
+ if (fwMeta) {
349
+ if (fwMeta._framework_key) labels.add(norm(fwMeta._framework_key));
350
+ if (fwMeta.full_name) labels.add(norm(fwMeta.full_name));
351
+ for (const a of (Array.isArray(fwMeta.catalog_aliases) ? fwMeta.catalog_aliases : [])) labels.add(norm(a));
352
+ }
353
+ labels.delete('');
354
+ const labelMatch = (fw) => {
355
+ const n = norm(fw);
356
+ if (!n) return false;
357
+ for (const l of labels) if (n === l || n.includes(l) || l.includes(n)) return true;
358
+ return false;
359
+ };
309
360
  const matching = entries(gaps)
310
- .filter(([, g]) => g.framework === frameworkId || g.framework === 'ALL')
361
+ .filter(([, g]) => {
362
+ if (g.framework === 'ALL') return true;
363
+ const fwList = Array.isArray(g.framework) ? g.framework : [g.framework];
364
+ return fwList.some(labelMatch);
365
+ })
311
366
  .map(([id, g]) => ({ id, ...g }));
312
- const fwMeta = global[frameworkId] || null;
313
367
  return { framework: frameworkId, framework_meta: fwMeta, gaps: matching, gap_count: matching.length };
314
368
  }
315
369
 
@@ -108,6 +108,17 @@ function lagScore(frameworkId, controlGaps, globalFrameworks) {
108
108
  const normalize = (s) => String(s).toLowerCase().replace(/[\s_-]/g, '');
109
109
  const idNorm = normalize(frameworkId);
110
110
  const nameNorm = frameworkData?.full_name ? normalize(frameworkData.full_name) : null;
111
+ // Data-driven aliases close the naming divergence between a framework's
112
+ // global-frameworks full_name and the (often different) labels the control-gap
113
+ // catalog uses for it — e.g. ASD_ISM's full_name is "Australian Signals
114
+ // Directorate Information Security Manual" but the catalog labels its 5 gaps
115
+ // "au-ism" / "ACSC ISM" / "Australian Government Information Security Manual",
116
+ // so neither nameNorm nor idNorm matched and lagScore reported 0 gaps. Aliases
117
+ // live in data/global-frameworks.json (catalog_aliases) so this generalizes to
118
+ // any future framework whose catalog label diverges from its name.
119
+ const aliasNorms = Array.isArray(frameworkData?.catalog_aliases)
120
+ ? frameworkData.catalog_aliases.map((a) => normalize(a)).filter(Boolean)
121
+ : [];
111
122
  const gaps = Object.entries(controlGaps).filter(([key, g]) => {
112
123
  if (key.startsWith('_')) return false;
113
124
  if (g.status !== 'open' || !g.framework) return false;
@@ -116,10 +127,17 @@ function lagScore(frameworkId, controlGaps, globalFrameworks) {
116
127
  const fwNorm = normalize(fw);
117
128
  if (nameNorm && fwNorm.includes(nameNorm)) return true; // display-name match
118
129
  if (fwNorm.includes(idNorm)) return true; // short-key substring
130
+ if (aliasNorms.some((a) => a && (fwNorm.includes(a) || a.includes(fwNorm)))) return true; // catalog-label alias
119
131
  }
120
132
  if (normalize(key).startsWith(idNorm)) return true; // gap-key prefix
121
133
  return false;
122
134
  });
135
+ // Observable backstop: a framework that EXISTS in global-frameworks resolving
136
+ // to zero catalog gaps is almost always a fresh naming divergence (a new
137
+ // catalog label not yet in catalog_aliases), not a genuinely gap-free
138
+ // framework. Surface it as a structured flag rather than a silent 0 so a
139
+ // regression is greppable in the breakdown instead of invisible.
140
+ const resolvedButZeroGaps = Boolean(frameworkData && gaps.length === 0);
123
141
 
124
142
  const universalGaps = Object.values(controlGaps).filter(g =>
125
143
  g.framework === 'ALL' && g.status === 'open'
@@ -149,7 +167,10 @@ function lagScore(frameworkId, controlGaps, globalFrameworks) {
149
167
  ai_coverage: { coverage: frameworkData?.ai_coverage ?? 'unknown', score: aiCoverageScore },
150
168
  pqc_coverage: { coverage: frameworkData?.pqc_coverage ?? 'unknown', score: pqcScore },
151
169
  universal_gaps: { count: universalGaps.length, score: universalGapScore },
152
- framework_specific_gaps: gaps.length
170
+ framework_specific_gaps: gaps.length,
171
+ // True only when the framework resolves in global-frameworks yet matched
172
+ // zero catalog gaps — a likely naming divergence worth investigating.
173
+ framework_resolved_but_zero_gaps: resolvedButZeroGaps
153
174
  }
154
175
  };
155
176
  }
@@ -4137,6 +4137,36 @@ function canonicalStringify(v, _depth = 0) {
4137
4137
  return '{' + keys.map(k => JSON.stringify(k) + ':' + canonicalStringify(v[k], _depth + 1)).join(',') + '}';
4138
4138
  }
4139
4139
 
4140
+ // Re-key an artifacts map by the stable indicator id each artifact was bound to
4141
+ // (recovered by inverting _signal_origins: indicator-id -> observation-key), so
4142
+ // the evidence_hash reflects the evidence VALUE + its stable binding rather than
4143
+ // the operator's free-text observation label. Without this, two submissions with
4144
+ // identical (indicator, value) evidence under different observation keys
4145
+ // (obs-kver vs x1) hashed differently and attest/reattest reported false drift —
4146
+ // the attest diff re-keys the COMPARISON (bin/exceptd.js); the hash itself was
4147
+ // not covered. Collision-safe (mirrors that helper): re-key only when the stable
4148
+ // id is not already a DISTINCT original key and has not been claimed by an
4149
+ // earlier entry, else keep the original key so no artifact is silently dropped.
4150
+ function _rekeyArtifactsByStableId(artifacts, signalOrigins) {
4151
+ // The sole caller (extractSubmissionForHash) only invokes this inside
4152
+ // `if (sub.artifacts && typeof sub.artifacts === 'object')`, so `artifacts`
4153
+ // is always a non-null object — guard only on the re-key map being usable.
4154
+ if (!signalOrigins || typeof signalOrigins !== 'object') return artifacts;
4155
+ const obsKeyToIndicator = {};
4156
+ for (const [indicatorId, obsKey] of Object.entries(signalOrigins)) {
4157
+ if (typeof obsKey === 'string') obsKeyToIndicator[obsKey] = indicatorId;
4158
+ }
4159
+ const originalKeys = new Set(Object.keys(artifacts));
4160
+ const out = {};
4161
+ for (const [k, v] of Object.entries(artifacts)) {
4162
+ const mapped = obsKeyToIndicator[k];
4163
+ const stable = (mapped && mapped !== k && !originalKeys.has(mapped)
4164
+ && !Object.prototype.hasOwnProperty.call(out, mapped)) ? mapped : k;
4165
+ out[stable] = v;
4166
+ }
4167
+ return out;
4168
+ }
4169
+
4140
4170
  /**
4141
4171
  * Pick the operator-meaningful fields out of the normalized submission
4142
4172
  * for hashing. captured_at, _signal_origins, _signal_origins_collisions,
@@ -4152,15 +4182,20 @@ function extractSubmissionForHash(sub) {
4152
4182
  // optional indicator binding) is what matters for "did the operator
4153
4183
  // submit the same evidence?".
4154
4184
  if (sub.artifacts && typeof sub.artifacts === 'object') {
4155
- pick.artifacts = {};
4185
+ const stripped = {};
4156
4186
  for (const [k, v] of Object.entries(sub.artifacts)) {
4157
4187
  if (v && typeof v === 'object') {
4158
4188
  const { captured_at, _captured_at, ...rest } = v;
4159
- pick.artifacts[k] = rest;
4189
+ stripped[k] = rest;
4160
4190
  } else {
4161
- pick.artifacts[k] = v;
4191
+ stripped[k] = v;
4162
4192
  }
4163
4193
  }
4194
+ // Re-key by the stable indicator id so the digest reflects the evidence
4195
+ // VALUE + its binding, never the operator's free-text observation label.
4196
+ // Identical (indicator, value) evidence under different observation keys
4197
+ // must produce the SAME evidence_hash / submission_digest / session_id.
4198
+ pick.artifacts = _rekeyArtifactsByStableId(stripped, sub._signal_origins);
4164
4199
  }
4165
4200
  if (sub.signal_overrides && typeof sub.signal_overrides === 'object') {
4166
4201
  pick.signal_overrides = sub.signal_overrides;
package/lib/prefetch.js CHANGED
@@ -39,7 +39,7 @@
39
39
  const fs = require("fs");
40
40
  const path = require("path");
41
41
  const crypto = require("crypto");
42
- const { JobQueue } = require("./job-queue");
42
+ const { JobQueue, isRetryable } = require("./job-queue");
43
43
 
44
44
  const ROOT = path.join(__dirname, "..");
45
45
  const DEFAULT_CACHE = path.join(ROOT, ".cache", "upstream");
@@ -220,11 +220,21 @@ function errorBudget(maxErrors, planned) {
220
220
  }
221
221
 
222
222
  // Decide prefetch's 0-vs-1 exit code from a completed run. Per-entry fetch
223
- // errors are counted only after the job queue exhausts its retries, so they
224
- // are genuine failures — but a best-effort cache warm should tolerate a few
225
- // transient upstream misses. `opts.maxErrors` is the budget (default 0, so any
226
- // error exits 1 — the strict contract a manual operator expects). Fatal
227
- // errors (bad flags, an unhandled throw) are handled in main() and exit 2.
223
+ // errors are counted only after the job queue exhausts its retries. They split
224
+ // into two classes that mean very different things for a best-effort cache
225
+ // warm:
226
+ // - HARD errors (404/410/parse failure/4xx-not-429) are real data faults.
227
+ // These count toward `opts.maxErrors` (default 0, so any hard error exits
228
+ // 1 — the strict contract a manual operator expects).
229
+ // - TRANSIENT errors (HTTP 408/425/429/5xx + ETIMEDOUT/ECONNRESET et al that
230
+ // exhausted their retry budget) are the upstream throttling us, not a data
231
+ // fault. They are surfaced in the summary and deferred to the next run
232
+ // (where the entries that DID land this run are fresh-skipped, freeing rate
233
+ // budget for the throttled ones) — they never fail the run on their own.
234
+ // Without the split, a daily NVD rate-limit on a subset of CVEs hard-failed the
235
+ // whole scheduled refresh and skipped the auto-PR every single run, since the
236
+ // ephemeral runner cache restarts cold each time and re-hits the same throttle.
237
+ // Fatal errors (bad flags, an unhandled throw) are handled in main() and exit 2.
228
238
  function exitCodeForResult(result, opts = {}) {
229
239
  const errors = (result && result.errors) || 0;
230
240
  if (errors === 0) return 0;
@@ -232,16 +242,22 @@ function exitCodeForResult(result, opts = {}) {
232
242
  // and nothing already fresh in the cache, yet errors recorded — is entirely
233
243
  // unreachable, and the refresh would silently skip it. A dead KEV feed is
234
244
  // only one error (well under any global budget) but means the run missed
235
- // every new KEV flag. Fail regardless of the budget so a single fully-dead
236
- // feed can't pass quietly.
245
+ // every new KEV flag. Fail regardless of the budget OR error class so a
246
+ // single fully-dead feed (incl. an NVD that 429s/503s every single request)
247
+ // can't pass quietly.
237
248
  const bySource = (result && result.by_source) || {};
238
249
  for (const s of Object.values(bySource)) {
239
250
  if (s && (s.errors || 0) > 0 && (s.fetched || 0) === 0 && (s.skipped_fresh || 0) === 0) {
240
251
  return 1;
241
252
  }
242
253
  }
254
+ // Only HARD errors gate the exit code against the budget. Back-compat: a
255
+ // result built without the split (errors_hard undefined) treats every error
256
+ // as hard, preserving the prior strict behavior for callers/tests that
257
+ // construct a bare { errors } result.
258
+ const hardErrors = (result && result.errors_hard != null) ? result.errors_hard : errors;
243
259
  const budget = errorBudget(opts.maxErrors, plannedCount(result));
244
- return errors > budget ? 1 : 0;
260
+ return hardErrors > budget ? 1 : 0;
245
261
  }
246
262
 
247
263
  // One-line run summary. When a run has errors, names the per-source counts so
@@ -254,6 +270,12 @@ function formatSummary(result, opts = {}) {
254
270
  .map(([name, s]) => `${name}=${s.errors}`);
255
271
  if (parts.length) line += ` [${parts.join(", ")}]`;
256
272
  }
273
+ // Name the transient vs hard split when present so a large error count that
274
+ // is purely upstream throttling reads as "throttled — retried, deferred to
275
+ // next run" rather than a silent data failure. Only hard errors gate exit 1.
276
+ if (result.errors > 0 && result.errors_transient != null && result.errors_hard != null) {
277
+ line += ` (${result.errors_transient} transient/throttled, ${result.errors_hard} hard)`;
278
+ }
257
279
  if (opts.noNetwork) line += " (dry-run)";
258
280
  return line;
259
281
  }
@@ -275,8 +297,11 @@ Options:
275
297
  --no-network report-only; list what would be fetched.
276
298
  --cache-dir <path> override cache root (default .cache/upstream).
277
299
  --quiet suppress per-entry log lines.
278
- --max-errors <n|n%> tolerate up to n (or n% of planned) per-entry fetch
279
- errors before exit 1. Default: 0 (any error exits 1).
300
+ --max-errors <n|n%> tolerate up to n (or n% of planned) HARD per-entry fetch
301
+ errors before exit 1. Default: 0 (any hard error exits 1).
302
+ Transient errors (rate-limit / timeout / 5xx that
303
+ exhausted retries) never fail the run on their own — they
304
+ are surfaced in the summary and retried on the next run.
280
305
  A fully-dead source still exits 1 regardless of budget.
281
306
 
282
307
  Use NVD_API_KEY / GITHUB_TOKEN env vars to lift rate limits.
@@ -599,6 +624,12 @@ function authHeadersForSource(source) {
599
624
 
600
625
  async function prefetch(options = {}) {
601
626
  const opts = { maxAgeMs: 24 * 3600 * 1000, source: null, force: false, noNetwork: false, cacheDir: DEFAULT_CACHE, quiet: false, ...options };
627
+ // Honor the global air-gap switch for programmatic callers too. parseArgs
628
+ // applies EXCEPTD_AIR_GAP for the CLI path, but a direct prefetch({...}) call
629
+ // bypasses parseArgs — so without this guard an air-gapped host that imports
630
+ // and calls prefetch() would egress live. Bind it here, at the function that
631
+ // actually issues the fetches, covering both the CLI and exported-API callers.
632
+ if (process.env.EXCEPTD_AIR_GAP === "1" || opts.airGap) opts.noNetwork = true;
602
633
  const ctx = loadCtx();
603
634
  // Distinguish "operator omitted --source" (resolve to all sources, the
604
635
  // documented default) from "operator passed --source but it resolved to
@@ -659,8 +690,8 @@ async function prefetch(options = {}) {
659
690
  log(`Cache dir: ${path.relative(ROOT, opts.cacheDir)}`);
660
691
  log(`Max age: ${(opts.maxAgeMs / 3_600_000).toFixed(1)}h${opts.force ? " (forced)" : ""}`);
661
692
 
662
- const result = { fetched: 0, skipped_fresh: 0, errors: 0, by_source: {} };
663
- for (const s of chosen) result.by_source[s] = { fetched: 0, skipped_fresh: 0, errors: 0 };
693
+ const result = { fetched: 0, skipped_fresh: 0, errors: 0, errors_transient: 0, errors_hard: 0, by_source: {} };
694
+ for (const s of chosen) result.by_source[s] = { fetched: 0, skipped_fresh: 0, errors: 0, errors_transient: 0, errors_hard: 0 };
664
695
 
665
696
  if (opts.noNetwork) {
666
697
  for (const item of plan) {
@@ -756,10 +787,25 @@ async function prefetch(options = {}) {
756
787
  .catch((err) => {
757
788
  result.errors++;
758
789
  result.by_source[item.source].errors++;
790
+ // Classify the post-retry error. Transient iff the job queue would
791
+ // have retried it (the same isRetryable classifier the queue used):
792
+ // HTTP 408/425/429/5xx + ETIMEDOUT/ECONNRESET et al — the upstream
793
+ // throttling/timing-out, not a data fault. Anything else (404/410/
794
+ // parse failure) is hard. Only hard errors gate the exit code; a
795
+ // best-effort warm tolerates transient throttling and retries it on
796
+ // the next run. The split is surfaced in the summary so nothing hides.
797
+ const transient = isRetryable(err);
798
+ if (transient) {
799
+ result.errors_transient++;
800
+ result.by_source[item.source].errors_transient++;
801
+ } else {
802
+ result.errors_hard++;
803
+ result.by_source[item.source].errors_hard++;
804
+ }
759
805
  // Errors go to stderr unconditionally — they are diagnostics, not the
760
806
  // per-entry success chatter --quiet suppresses. A CI run with --quiet
761
- // still surfaces which source/id failed.
762
- console.error(` [${item.source}] ${item.id} — error: ${err.message}`);
807
+ // still surfaces which source/id failed and whether it was transient.
808
+ console.error(` [${item.source}] ${item.id} — ${transient ? "transient" : "hard"} error: ${err.message}`);
763
809
  });
764
810
  });
765
811
 
@@ -1023,7 +1023,13 @@ function epssDiffFromCache(ctx) {
1023
1023
  for (const id of cves) {
1024
1024
  const payload = readCachedJson(ctx.cacheDir, "epss", id, { forceStale: ctx.forceStale });
1025
1025
  if (!payload) { errors++; continue; }
1026
- const row = (payload.data || []).find((r) => r?.cve === id) || (payload.data || [])[0];
1026
+ // Match the EPSS row by id. The old `|| data[0]` blanket fallback
1027
+ // attributed a DIFFERENT CVE's score to this id whenever the cache entry
1028
+ // keyed under `id` actually held another CVE's payload. Accept the
1029
+ // single-row fallback ONLY when that row carries no cve key (a keyless
1030
+ // legacy payload), never a row naming a different CVE.
1031
+ let row = (payload.data || []).find((r) => r?.cve === id);
1032
+ if (!row && (payload.data || []).length === 1 && (payload.data || [])[0]?.cve == null) row = (payload.data || [])[0];
1027
1033
  if (!row) continue;
1028
1034
  const score = row.epss != null ? Number(row.epss) : null;
1029
1035
  const pct = row.percentile != null ? Number(row.percentile) : null;
@@ -1051,7 +1057,15 @@ function nvdDiffFromCache(ctx) {
1051
1057
  for (const id of cves) {
1052
1058
  const payload = readCachedJson(ctx.cacheDir, "nvd", id, { forceStale: ctx.forceStale });
1053
1059
  if (!payload) { errors++; continue; }
1054
- const vuln = payload.vulnerabilities?.[0]?.cve;
1060
+ // Resolve the NVD vuln by id, not by position. vulnerabilities[0] blindly
1061
+ // took the first record, attributing a DIFFERENT CVE's CVSS to this id when
1062
+ // the cache entry keyed under `id` held another CVE's response.
1063
+ const vulnCves = (payload.vulnerabilities || []).map((v) => v?.cve).filter(Boolean);
1064
+ let vuln = vulnCves.find((c) => c.id && String(c.id).toUpperCase() === id.toUpperCase());
1065
+ // Keyless single-record fallback: a single vulnerability whose cve carries
1066
+ // no id can't be a mismatch — the id-keyed cache file IS the binding — so
1067
+ // use it. A record whose id names a DIFFERENT cve is still rejected.
1068
+ if (!vuln && vulnCves.length === 1 && vulnCves[0].id == null) vuln = vulnCves[0];
1055
1069
  if (!vuln) continue;
1056
1070
  // Prefer the newest CVSS version NVD publishes (Primary within that
1057
1071
  // version), and normalize a bare v2 vector to its canonical prefix.