@clear-capabilities/agentic-security-scanner 0.142.0 → 0.143.0

package/CHANGELOG.md

@@ -11,6 +11,101 @@
 
 
 
+## 0.143.0 — OSCAL output, and the finding an OSCAL document must refuse to make
+
+`--format oscal` was documented in `commands/compliance.md` long before anything
+implemented it. In 0.139.0 that was corrected to an explicit refusal rather than
+left aspirational. This release makes it real, in both places the request asked
+for: any scan, and any framework assessment.
+
+```bash
+agentic-security scan . --format oscal
+agentic-security compliance --report <framework> --format oscal
+agentic-security compliance --format oscal          # NIST Privacy Framework 1.1
+```
+
+Both emit NIST [OSCAL](https://pages.nist.gov/OSCAL-Reference/models/) 1.1.2
+`assessment-results`. `--report <fw>` is also now a real CLI synonym for
+`--walkthrough <fw>`; the slash command had always spelled it that way, and it
+previously reached the frameworks only through an inlined script in `commands/`.
+
+**The interesting part is what these documents refuse to say.** An OSCAL
+`finding` is a statement about a control, and its `status.state` is binary:
+satisfied or not-satisfied. There is no "unknown" and no "we did not look". So:
+
+- **A raw scan emits no findings at all.** Observations (what the scanner saw)
+  and risks (what it would mean), and a `reviewed-controls` block that says
+  plainly that no catalog was in scope. A SQL-injection hit is not an opinion
+  about a control; emitting one would publish a CWE→control mapping nobody
+  wrote. `include-all` is likewise absent — it would assert this scan reviewed
+  every control of an unnamed catalog.
+- **A control the engine could not decide carries no finding.** `manual`
+  controls, and on the privacy path `engine-gap` controls — where NIST rates the
+  control code-testable and *this scanner has no check for it* — become
+  observations with method `EXAMINE`. Calling them satisfied would be a false
+  compliance claim; calling them not-satisfied would blame the assessed system
+  for a hole in the tool.
+
+The distinction OSCAL cannot express rides along as an `assessment-status`
+property, so nothing is lost by the conversion. Full mapping table in
+`docs/OSCAL.md`.
+
+**A bug this found in itself.** The first adapter mapped `present` to satisfied,
+`partial` to not-satisfied, and everything else to unassessed. `evaluateFramework`
+also returns `absent` — signals exist and not one cleared, the strongest failure
+it can express — and the catch-all silently relabelled it "requires human
+judgement", deleting real control failures from the document and attaching a
+remark that was false. It was caught by running the exporter against a bundled
+framework and reading the output. There is now no catch-all: the mapping is
+exhaustive, and an unrecognised upstream status is reported *as* unrecognised,
+naming itself as an exporter defect rather than making a claim about the control.
+
+**Two more defects the checking found, both invisible at emit time.** OSCAL's
+`assessment-assets` — the block that identifies what performed the assessment —
+is scoped to the *result*, not the document (`assessment-results/local-definitions`
+carries objectives-and-methods and activities, and nothing else), and it requires
+at least one `assessment-platforms` entry. The first draft had it at document
+level with no platform. Both produce a document that emits cleanly and fails
+validation, which is precisely the failure mode of a format claim nobody ran
+through a validator.
+
+**Control identifiers are rewritten, and the originals kept.** OSCAL's `token`
+datatype is an NCName. The CCPA catalog bundled with this engine uses ids like
+`§1798.100`, which is not one — emitting it raw produces a document a validator
+rejects at the first control, the usual failure mode of an OSCAL export that was
+never run through one. Ids are sanitised to legal tokens and the publisher's
+original is carried on every observation and finding as `source-control-id`.
+
+**Checked, and the checks were checked.** `test/oscal-conformance.test.js` (15
+tests) validates required fields, the constrained datatypes (`uuid`, `token`,
+`dateTime-with-timezone`), the closed value sets, and referential integrity —
+every `*-uuid` and `#fragment` must resolve inside the document. It also pins the
+doctrine above, which is not a schema property and would otherwise be one
+refactor from reversing. Three deliberate regressions (neutering the token
+sanitiser, the deterministic uuid shaping, and the no-findings rule) were each
+confirmed to fail the suite before the source was restored. Scope is stated in
+the file: structural validation, not full JSON-Schema validation against NIST's
+published schema — fetching it at test time breaks the no-network rule and
+vendoring it adds a file that rots silently. Same call, same reasoning, as
+`test/sbom-conformance.test.js`.
+
+`oscal` joins the `format-determinism` gate, so two emits of one scan are
+byte-identical and an attestation over the document still verifies. That gate
+also exposed a real gap: every conformance test ran the `crypto.randomUUID()`
+branch, leaving the `--deterministic` branch — the one an attestation is
+actually taken over — untested. A digest slice is not a legal uuid; roughly 15
+in 16 fail on the variant nibble alone. It is now covered explicitly.
+
+**Also:** the two load-bearing caveats (ordinal scores are not probabilities;
+benchmark-tuned F1 does not generalize) now come from one exported constant
+rather than being spelled out inside `toSARIF`. SARIF carries them as run
+notifications, OSCAL as back-matter resources, from the same source — two copies
+of a caveat is one copy that goes stale, and the stale one is always the one
+somebody reads. The three inline "is this a machine format" lists in the CLI
+became one set, for the same reason: each new format previously had to be added
+to all three or it got human chatter interleaved into its output.
+
+
 ## 0.142.0 — The last five PRD items, and three criteria that now fail on evidence
 
 0.141.0 built instruments for the surfaces that had none. This closes the

package/bin/agentic-security.js

@@ -39,6 +39,17 @@ function writeStdout(s) {
   }
 }
 import { toJSON, toMarkdown, toSARIF, toSTIX, toCSV, toJUnit, toCLI, toCLIByProfile, toShipVerdict, toProTable, toHTML, toSummary, toVex, exitCodeFor, normalizeFindings } from '../src/report/index.js';
+import { toOSCAL } from '../src/report/oscal.js';
+
+// Formats whose output is a machine artifact, not something a human reads in a
+// terminal. Three separate copies of this list used to be spelled out inline —
+// the MTTR line, the fix-duration line and the streak line — and each new
+// format had to be added to all three or it silently got human chatter
+// interleaved into its stdout/stderr. One list, one place to update.
+const MACHINE_FORMATS = new Set([
+  'json', 'sarif', 'oscal', 'cyclonedx', 'sbom', 'spdx', 'vex', 'openvex', 'pbom', 'aibom',
+]);
+function isMachineFormat(fmt) { return MACHINE_FORMATS.has(String(fmt)); }
 import { toCycloneDX, toSPDX } from '../src/posture/sbom.js';
 import { toPBOM } from '../src/sast/pipeline.js';
 import { buildAIBOM, aibomToMarkdown } from '../src/posture/aibom.js';
@@ -96,8 +107,13 @@ Commands:
   compliance [--privacy]       Assess the last scan against NIST Privacy Framework 1.1
                                --list                 show bundled + BYO frameworks
                                --walkthrough <id>     auditor narrative for any framework
+                               --report <id>          synonym for --walkthrough
                                --gap                  only the failing controls
-                               --format cli|json|md   (default cli)
+                               --format cli|json|md|oscal   (default cli)
+                               --format oscal         NIST OSCAL assessment-results.
+                                                      Controls the engine could not
+                                                      decide carry NO finding — see
+                                                      docs/OSCAL.md.
                                --fail-on gap          exit 1 when a control is failing
                                Reads .agentic-security/last-scan.json — run a scan first.
   version                      Print version
@@ -113,7 +129,7 @@ Commands:
 Options:
   --profile vibecoder|pro      Override profile for this run
   --only sast|sca|secrets      Limit scan to one pillar
-  --format <fmt>               cli | json | md | sarif | stix | junit | csv | html | cyclonedx | spdx | pbom | aibom | aibom-md
+  --format <fmt>               cli | json | md | sarif | oscal | stix | junit | csv | html | cyclonedx | spdx | pbom | aibom | aibom-md
   --pack <name>                Focus on a curated rule pack (repeatable): owasp-top-10 | cwe-top-25 | llm-security | supply-chain
   --baseline <ref>             Diff against a git ref; only findings new vs. that ref count (ci subcommand)
   --fail-on critical|high|medium|low|none  ci-mode exit policy (default: critical)
@@ -678,6 +694,10 @@ async function cmdScan(args) {
   if (format === 'json') body = JSON.stringify(toJSON(scan, meta, { includeSuppressed }), null, 2);
   else if (format === 'md' || format === 'markdown') body = toMarkdown(scan, meta);
   else if (format === 'sarif') body = JSON.stringify(toSARIF(scan, meta), null, 2);
+  // OSCAL assessment-results. Observations and risks only, never findings — an
+  // OSCAL finding is a statement about a control, and a source scan reviews no
+  // control catalog. See src/report/oscal.js.
+  else if (format === 'oscal') body = JSON.stringify(toOSCAL(scan, meta), null, 2);
   else if (format === 'stix') body = JSON.stringify(toSTIX(scan, meta), null, 2);
   else if (format === 'junit') body = toJUnit(scan, meta);
   else if (format === 'csv')   body = toCSV(scan);
@@ -759,7 +779,7 @@ async function cmdScan(args) {
         const removed = prevAll.filter(f => !currentFps.has(fingerprintFinding(f)));
         persistedScan.mttr = computeMTTR(removed);
         // Surface the SLA-breach line on human-readable formats (not JSON/CI pipes).
-        const isJson = format === 'json' || format === 'sarif' || format === 'cyclonedx' || format === 'sbom' || format === 'spdx' || format === 'vex' || format === 'openvex' || format === 'pbom' || format === 'aibom';
+        const isJson = isMachineFormat(format);
         if (!isJson) {
           const sla = renderSlaSummary(persistedScan.findings || []);
           if (sla) process.stderr.write(`⏰ agentic-security: ${sla}\n`);
@@ -789,7 +809,7 @@ async function cmdScan(args) {
         const fixMetrics = fixDurationReport(path.resolve(target));
         if (fixMetrics.attempts > 0) {
           persistedScan.fixMetrics = fixMetrics;
-          const isJsonFmt = format === 'json' || format === 'sarif' || format === 'cyclonedx' || format === 'sbom' || format === 'spdx' || format === 'vex' || format === 'openvex' || format === 'pbom' || format === 'aibom';
+          const isJsonFmt = isMachineFormat(format);
           const line = renderFixDurationSummary(fixMetrics);
           if (!isJsonFmt && line) process.stderr.write(`🔧 agentic-security: ${line}\n`);
         }
@@ -832,7 +852,7 @@ async function cmdScan(args) {
   try {
     const streak = persistedScan !== null ? recordScan(stateDirPath, persistedScan) : null;
     // Print celebration / streak line to stderr so it doesn't pollute --format json
-    if (streak && process.stderr.isTTY && format !== 'json' && format !== 'sarif') {
+    if (streak && process.stderr.isTTY && !isMachineFormat(format)) {
       const delta = formatGradeDelta(streak);
       const line = formatStreakLine(streak);
       if (delta) process.stderr.write('\n' + delta + '\n');
@@ -1791,14 +1811,27 @@ async function cmdCompliance(args) {
   // The engine records this two ways depending on the emit path; take either.
   scan.filesScanned = scan._scanMeta?.filesScanned ?? scan.scanned?.files ?? 0;
 
-  const wt = args.flags.walkthrough;
+  // `--report <fw>` is accepted as a synonym for `--walkthrough <fw>`: the
+  // slash-command surface has always spelled it that way, and it previously
+  // reached the frameworks only through an inlined node call in commands/
+  // rather than through this CLI. One code path, two spellings.
+  const wt = args.flags.walkthrough || args.flags.report;
   if (wt && wt !== true) {
     const fw = loadFramework(scanRoot, String(wt));
     if (!fw) {
       console.error(`Unknown framework "${wt}". Try --list.`);
       return 2;
     }
-    console.log(renderWalkthrough(fw, evaluateFramework(scanRoot, fw, scan), {}));
+    const evaluation = evaluateFramework(scanRoot, fw, scan);
+    if (fmt === 'oscal') {
+      const { toOSCALCompliance, complianceRowsFromEvaluation } = await import('../src/report/oscal.js');
+      writeStdout(JSON.stringify(
+        toOSCALCompliance(fw, complianceRowsFromEvaluation(evaluation), { startedAt: scan._scanMeta?.startedAt }),
+        null, 2) + '\n');
+      return 0;
+    }
+    if (fmt === 'json') { writeStdout(JSON.stringify(evaluation, null, 2) + '\n'); return 0; }
+    console.log(renderWalkthrough(fw, evaluation, {}));
     return 0;
   }
 
@@ -1808,6 +1841,19 @@ async function cmdCompliance(args) {
   if (!r) { console.error(`Framework ${PRIVACY_FRAMEWORK_ID} could not be loaded.`); return 2; }
 
   const gapsOnly = !!args.flags.gap;
+  if (fmt === 'oscal') {
+    // The privacy assessment's own bucket model, not evaluateFramework's — see
+    // complianceRowsFromPrivacy for why `engine-gap` must not become a control
+    // failure. `--gap` is deliberately NOT applied here: an OSCAL document that
+    // silently omitted the satisfied controls would understate what was
+    // reviewed, and reviewed-controls would then disagree with the findings.
+    const { toOSCALCompliance, complianceRowsFromPrivacy } = await import('../src/report/oscal.js');
+    const fwMeta = loadFramework(scanRoot, PRIVACY_FRAMEWORK_ID) || { id: PRIVACY_FRAMEWORK_ID, name: r.frameworkName };
+    writeStdout(JSON.stringify(
+      toOSCALCompliance(fwMeta, complianceRowsFromPrivacy(r), { startedAt: scan._scanMeta?.startedAt }),
+      null, 2) + '\n');
+    return args.flags['fail-on'] === 'gap' && r.summary.gap > 0 ? 1 : 0;
+  }
   if (fmt === 'json') {
     writeStdout(JSON.stringify(gapsOnly ? { ...r, controls: r.controls.filter(c => c.bucket === 'gap') } : r, null, 2) + '\n');
   } else if (fmt === 'md') {