@clear-capabilities/agentic-security-scanner 0.130.0 → 0.132.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +122 -0
- package/bin/agentic-security.js +19 -2
- package/dist/113.index.js +292 -3
- package/dist/207.index.js +7 -4
- package/dist/238.index.js +218 -0
- package/dist/259.index.js +975 -0
- package/dist/435.index.js +2 -2
- package/dist/526.index.js +292 -3
- package/dist/agentic-security.mjs +23 -62
- package/dist/agentic-security.mjs.sha256 +1 -1
- package/package.json +17 -9
- package/src/engine.js +16 -0
- package/src/ir/parser-js.js +8 -0
- package/src/mcp/tools.js +2 -2
- package/src/posture/CLAUDE.md +83 -6
- package/src/posture/attestation.js +7 -4
- package/src/posture/corpus-enroll.js +303 -0
- package/src/posture/corpus-match.js +52 -0
- package/src/posture/custom-rules.js +2 -2
- package/src/posture/execution-proof.js +44 -4
- package/src/posture/fix-metrics.js +197 -0
- package/src/posture/fix-verify.js +76 -2
- package/src/posture/root-cause-sweep.js +0 -0
- package/src/runScan.js +2 -6
- package/src/sandbox/CLAUDE.md +168 -46
- package/src/sandbox/backend-namespace.js +292 -40
- package/src/sandbox/backend-userspace.js +2 -19
- package/src/sandbox/capabilities.js +132 -4
- package/src/sandbox/limits.js +21 -0
- package/src/sandbox/result.js +1 -1
- package/src/util/glob.js +173 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,127 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.132.0 — a proven exploit becomes a permanent regression test
|
|
4
|
+
|
|
5
|
+
The corpus stops being only a regression net and starts being fed by the engine
|
|
6
|
+
itself. Every claim below was verified by a command in the session that made it.
|
|
7
|
+
|
|
8
|
+
- **Execution-proven findings auto-enrol as corpus entries.** This was R2's
|
|
9
|
+
differentiator and the last missing piece of it. A finding whose
|
|
10
|
+
proof-of-concept RAN in the sandbox and produced the predicted effect can now
|
|
11
|
+
be turned into a permanent `pre:TP post:TN` corpus entry, so every exploit
|
|
12
|
+
proved once is defended against its own regression forever. Demonstrated end
|
|
13
|
+
to end rather than in a unit test: a real command-injection finding was proved
|
|
14
|
+
in the sandbox, its fix supplied the `post/` tree, and the entry was enrolled
|
|
15
|
+
and scored by the real gate — **the baseline moved 199 → 200**. Proven in the
|
|
16
|
+
other direction too: with the entry's `pre/` neutered the gate reports
|
|
17
|
+
`REGRESSED (1)` and exits non-zero.
|
|
18
|
+
- **Nothing reaches the corpus unscored.** The entry is built in a temporary
|
|
19
|
+
directory, `pre/` and `post/` are scanned, and it is moved into the corpus
|
|
20
|
+
only on `pre:TP post:TN`. There is no force flag, and the scoring function is
|
|
21
|
+
unexported so no caller can score by one route and write by another — the
|
|
22
|
+
v0.106.0 mistake (fixtures committed without verifying they score) is the one
|
|
23
|
+
thing an automated writer must never industrialise. Scoring itself now lives
|
|
24
|
+
in one module shared by the enroller and the corpus runner, so the two cannot
|
|
25
|
+
drift apart; the refactor was proven behaviour-preserving at 199/199, no drift.
|
|
26
|
+
New entries land in `capability/`, never the CI-gated `regression/` tier — a
|
|
27
|
+
machine must not decide what blocks everyone's build.
|
|
28
|
+
- **`verify_fix` now runs the proof-of-concept against the candidate patch.** A
|
|
29
|
+
re-scan only proves the DETECTOR stopped firing, which a cosmetic edit
|
|
30
|
+
achieves; re-running the exploit proves the hole is shut. Still-exploitable
|
|
31
|
+
after the patch is a hard failure. Deliberately asymmetric: a PoC that could
|
|
32
|
+
not run is recorded `inconclusive` and excluded from the verdict, because
|
|
33
|
+
reading "could not prove it" as "fixed" is exactly the false confidence this
|
|
34
|
+
leg exists to prevent. Both directions were executed against the real sandbox.
|
|
35
|
+
- **Time-to-validated-fix is now measured and reported**, closing R5. Every
|
|
36
|
+
verification stage is timed and appended to a per-project log, and the
|
|
37
|
+
reported distribution is deliberately hard to flatter: failed attempts never
|
|
38
|
+
enter the validated median (they short-circuit, so blending them makes a worse
|
|
39
|
+
pipeline look faster), "no test suite to run" is bucketed apart from "tests
|
|
40
|
+
passed", and per-stage timings come from validated runs only. Percentiles are
|
|
41
|
+
nearest-rank — every figure shown is a duration some run actually took — and
|
|
42
|
+
are flagged unreliable below n=10 rather than quoted as settled.
|
|
43
|
+
- **Cross-machine determinism now has a gate behind it.** A dependency-free
|
|
44
|
+
fixture is scanned on two operating systems and the run-attestation digests
|
|
45
|
+
must match. The comparator refuses every route to a meaningless pass: fewer
|
|
46
|
+
than two attestations, two runs from the same platform, a zero-finding digest,
|
|
47
|
+
mismatched canonicalisations, unparseable input. Each refusal was fired
|
|
48
|
+
deliberately and confirmed to exit non-zero.
|
|
49
|
+
|
|
50
|
+
Honest limits, stated rather than implied:
|
|
51
|
+
|
|
52
|
+
- The enrolment loop is **not yet automatic end to end**. Nothing in the scan
|
|
53
|
+
pipeline attaches a proof-of-concept to a finding or promotes proof tiers, so
|
|
54
|
+
a scan never produces an `execution-proven` finding on its own; PoCs come from
|
|
55
|
+
the generator and are proved at enrol time. Automatic attachment during a scan
|
|
56
|
+
is the remaining work.
|
|
57
|
+
- The cross-machine determinism jobs have **not yet run in CI**, so no second
|
|
58
|
+
machine has actually been compared. Same-machine repeatability is verified.
|
|
59
|
+
The attestation's own "does not prove" statement is unchanged and stays
|
|
60
|
+
correct either way: one attestation is one run on one machine.
|
|
61
|
+
- The sandbox wall-clock timeout stops the **direct child, not the process
|
|
62
|
+
tree**, on both backends. No test asserts a backgrounded grandchild dies.
|
|
63
|
+
|
|
64
|
+
Also in this release:
|
|
65
|
+
|
|
66
|
+
- **Documentation corrected against evidence.** The roadmap and the sandbox and
|
|
67
|
+
posture guides still described the kernel-namespace backend as "implemented,
|
|
68
|
+
unverified" after CI had already proved otherwise; they now cite the run
|
|
69
|
+
(Ubuntu 24.04, kernel 6.17.0-1020-azure, 41 assertions, 0 failures, all eight
|
|
70
|
+
escape cases). The corpus entry count was three tiers and 14 entries out of
|
|
71
|
+
date.
|
|
72
|
+
- **Dependency advisories cleared** in the extension tree and `@types/node`
|
|
73
|
+
brought current in both trees.
|
|
74
|
+
|
|
75
|
+
## 0.131.0 — decorated files stop vanishing; the Linux sandbox is verified
|
|
76
|
+
|
|
77
|
+
Two correctness fixes, both found by pushing measurement further than the last
|
|
78
|
+
release did.
|
|
79
|
+
|
|
80
|
+
- **Decorator syntax is now accepted by the IR frontend.** The parser was told
|
|
81
|
+
about TypeScript and JSX but not decorators, so it rejected the ENTIRE file on
|
|
82
|
+
the first `@tracked` it met — no error, no warning, just findings that never
|
|
83
|
+
existed. Measured on a live third-party target: **4,023 of 4,271 JavaScript
|
|
84
|
+
files parsed before, 4,262 after** — 239 files, roughly 6% of that project,
|
|
85
|
+
were invisible to the scanner and are now analysed. `decorators-legacy` plus
|
|
86
|
+
`decoratorAutoAccessors` was chosen over the modern `decorators` variant
|
|
87
|
+
because the modern one cannot parse TypeScript parameter decorators; it would
|
|
88
|
+
have swapped one blind spot for another. No new dependency. Guarded by a
|
|
89
|
+
regression test proven in both directions — 5/5 pass with the fix, 4 fail
|
|
90
|
+
without it.
|
|
91
|
+
- **The kernel-namespace sandbox now confines writes, and is verified.** It
|
|
92
|
+
previously confined network only and had never been executed. It now enters a
|
|
93
|
+
private mount namespace, rebinds every mount read-only except the sandbox
|
|
94
|
+
root, and drops the entire capability set before exec so a payload cannot
|
|
95
|
+
rebind the tree writable. A CI job relaxes the host restriction on
|
|
96
|
+
unprivileged user namespaces and runs the real escape suite: a write outside
|
|
97
|
+
the root is blocked with no file created, and outbound egress is blocked. The
|
|
98
|
+
verifier exits non-zero unless that suite actually RAN, so a skip can never be
|
|
99
|
+
read as a pass. Execution-proof on Linux no longer rests on an unexercised
|
|
100
|
+
backend.
|
|
101
|
+
|
|
102
|
+
Also in this release:
|
|
103
|
+
|
|
104
|
+
- **A quadratic blowup in the root-cause sweep**, which ran on every full scan:
|
|
105
|
+
it re-split the whole corpus into lines once per finding and kept a record per
|
|
106
|
+
finding x matching line. The largest benchmark corpus went from dying after
|
|
107
|
+
55+ minutes to **exit 0, 933 MB peak, 330s** — smaller and roughly ten times
|
|
108
|
+
faster, with metrics identical before and after.
|
|
109
|
+
- **Benchmark corpora are fetched by pinned commit** instead of guessing a clone
|
|
110
|
+
depth; six corpora had never been scoreable because their pin sat more than
|
|
111
|
+
100 commits back.
|
|
112
|
+
- **A pre-push gate** runs bundle integrity, the full suite, the corpus baseline
|
|
113
|
+
and the precision baseline before anything leaves the machine, plus branch
|
|
114
|
+
protection requiring green checks to merge.
|
|
115
|
+
- **A dependency-currency release gate**: any advisory at moderate or above
|
|
116
|
+
fails with no opt-out; anything behind latest fails unless explicitly held
|
|
117
|
+
with a stated reason and a review date that expires.
|
|
118
|
+
- **The glob dependency was replaced by the platform built-in** — 92 to 73
|
|
119
|
+
production packages — after a differential over 320 trees and 1,419,229 paths
|
|
120
|
+
showed zero differences.
|
|
121
|
+
|
|
122
|
+
`npm test` 2072/0; cve-replay 199/199; self-scan no drift; proof corpus
|
|
123
|
+
ghost/superset/godot all 100% parse coverage.
|
|
124
|
+
|
|
3
125
|
## 0.130.0 — the roadmap's first ten: provable security over orchestration parity
|
|
4
126
|
|
|
5
127
|
A capability roadmap (`docs/ROADMAP.md`) plus its first ten items, derived from a
|
package/bin/agentic-security.js
CHANGED
|
@@ -28,7 +28,7 @@ import { syncTickets } from '../src/integrations/tickets.js';
|
|
|
28
28
|
import { decide as decideNextAction, explain as explainDecision } from '../src/posture/router.js';
|
|
29
29
|
import * as triage from '../src/posture/triage.js';
|
|
30
30
|
import { buildSlackDigest, buildDiscordDigest, postWebhook, buildJiraIssue, buildPrComment, buildSiemEvent, loadIntegrationConfig } from '../src/integrations/index.js';
|
|
31
|
-
import
|
|
31
|
+
import { globFiles } from '../src/util/glob.js';
|
|
32
32
|
|
|
33
33
|
// last-scan.json integrity helpers — implementation in posture/integrity.js
|
|
34
34
|
// so the MCP server tools can share verification.
|
|
@@ -483,7 +483,7 @@ async function cmdScan(args) {
|
|
|
483
483
|
// fingerprint and tracking provenance via sources[].
|
|
484
484
|
if (args.flags['ingest-sarif']) {
|
|
485
485
|
const glob = args.flags['ingest-sarif'];
|
|
486
|
-
const paths = await
|
|
486
|
+
const paths = await globFiles(glob);
|
|
487
487
|
if (paths.length) {
|
|
488
488
|
const r = ingestAndMerge(scan, paths);
|
|
489
489
|
if (process.stderr.isTTY) process.stderr.write(`[ingest] merged ${r.merged} / added ${r.added} findings from ${paths.length} SARIF file(s)\n`);
|
|
@@ -655,6 +655,23 @@ async function cmdScan(args) {
|
|
|
655
655
|
if (sla) process.stderr.write(`⏰ agentic-security: ${sla}\n`);
|
|
656
656
|
}
|
|
657
657
|
} catch { /* MTTR is best-effort — never block a scan write */ }
|
|
658
|
+
|
|
659
|
+
// R5 — report the observed time-to-validated-fix distribution from the
|
|
660
|
+
// fix attempts recorded by `verifyFix`. This is measurement, not
|
|
661
|
+
// estimation: it says nothing until fixes have actually been verified in
|
|
662
|
+
// this project, and it prints nothing when there is nothing measured.
|
|
663
|
+
// Skipped under --deterministic for the same reason MTTR is: the
|
|
664
|
+
// durations are wall-clock and would break byte-identical state.
|
|
665
|
+
try {
|
|
666
|
+
const { fixDurationReport, renderFixDurationSummary } = await import('../src/posture/fix-metrics.js');
|
|
667
|
+
const fixMetrics = fixDurationReport(path.resolve(target));
|
|
668
|
+
if (fixMetrics.attempts > 0) {
|
|
669
|
+
persistedScan.fixMetrics = fixMetrics;
|
|
670
|
+
const isJsonFmt = format === 'json' || format === 'sarif' || format === 'cyclonedx' || format === 'sbom' || format === 'spdx' || format === 'vex' || format === 'openvex' || format === 'pbom' || format === 'aibom';
|
|
671
|
+
const line = renderFixDurationSummary(fixMetrics);
|
|
672
|
+
if (!isJsonFmt && line) process.stderr.write(`🔧 agentic-security: ${line}\n`);
|
|
673
|
+
}
|
|
674
|
+
} catch { /* fix metrics are best-effort — never block a scan write */ }
|
|
658
675
|
}
|
|
659
676
|
// #22 — live-secret validation (opt-in, offline-degrading). Label each
|
|
660
677
|
// detected secret live | dead | unknown via a read-only provider "whoami".
|
package/dist/113.index.js
CHANGED
|
@@ -1,7 +1,220 @@
|
|
|
1
1
|
export const id = 113;
|
|
2
|
-
export const ids = [113,526];
|
|
2
|
+
export const ids = [113,238,526];
|
|
3
3
|
export const modules = {
|
|
4
4
|
|
|
5
|
+
/***/ 2238:
|
|
6
|
+
/***/ ((__unused_webpack___webpack_module__, __webpack_exports__, __webpack_require__) => {
|
|
7
|
+
|
|
8
|
+
/* harmony export */ __webpack_require__.d(__webpack_exports__, {
|
|
9
|
+
/* harmony export */ I3: () => (/* binding */ recordFixAttempt),
|
|
10
|
+
/* harmony export */ fixDurationReport: () => (/* binding */ fixDurationReport),
|
|
11
|
+
/* harmony export */ renderFixDurationSummary: () => (/* binding */ renderFixDurationSummary)
|
|
12
|
+
/* harmony export */ });
|
|
13
|
+
/* unused harmony exports FIX_STAGES, loadFixAttempts, bucketOf, summarizeFixDurations, _internals */
|
|
14
|
+
/* harmony import */ var node_fs__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(3024);
|
|
15
|
+
/* harmony import */ var node_path__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(6760);
|
|
16
|
+
/* harmony import */ var _state_dir_js__WEBPACK_IMPORTED_MODULE_2__ = __webpack_require__(1174);
|
|
17
|
+
// Time-to-validated-fix (R5, the reporting half).
|
|
18
|
+
//
|
|
19
|
+
// `verifyFix` already RUNS the stages and `test-runner.js` already times the
|
|
20
|
+
// slowest one. What did not exist was anything durable to read afterwards, so
|
|
21
|
+
// "how long does a fix actually take to validate" had no answer from real runs
|
|
22
|
+
// — only an estimate (`time-to-fix.js` guesses engineering hours from family
|
|
23
|
+
// and patch shape, before anything runs). This module is the opposite: it
|
|
24
|
+
// records what the pipeline observed and reports the distribution.
|
|
25
|
+
//
|
|
26
|
+
// THE HONESTY RULES, which are most of why this file is longer than a mean:
|
|
27
|
+
//
|
|
28
|
+
// 1. A failed attempt is NOT a data point about how long a fix takes. Fixes
|
|
29
|
+
// that fail verification fail fast (a re-scan that still sees the finding
|
|
30
|
+
// never reaches the test suite), so blending them into one average makes
|
|
31
|
+
// the pipeline look faster the worse it performs. Validated and failed
|
|
32
|
+
// attempts are summarised separately and never merged.
|
|
33
|
+
//
|
|
34
|
+
// 2. "Tests skipped" is not "tests passed". A project with no detectable
|
|
35
|
+
// suite can reach `ok:true` having run only the re-scan and the linter.
|
|
36
|
+
// That is a weaker claim than a fix whose suite executed, and it is also
|
|
37
|
+
// much faster, so counting the two together would quietly deflate the
|
|
38
|
+
// headline. They get their own bucket: `validated` means the suite ran and
|
|
39
|
+
// passed, `validatedWithoutTests` means there was no suite to run.
|
|
40
|
+
//
|
|
41
|
+
// 3. Every figure carries its `n`, and a percentile computed from too few
|
|
42
|
+
// samples is labelled unreliable rather than omitted or silently reported.
|
|
43
|
+
// Same precedent as the accuracy scorecard's `{n, d}` rates: a number
|
|
44
|
+
// without its denominator is not a measurement.
|
|
45
|
+
//
|
|
46
|
+
// Storage is append-only JSONL at `<scanRoot>/.agentic-security/fix-metrics.jsonl`,
|
|
47
|
+
// one record per verification attempt. Nothing here throws (posture
|
|
48
|
+
// convention) — an unwritable or corrupt log degrades to "no metrics", never
|
|
49
|
+
// to a failed verification.
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
const STATE_DIR = '.agentic-security';
|
|
56
|
+
const LOG_FILE = 'fix-metrics.jsonl';
|
|
57
|
+
|
|
58
|
+
// Below this many samples a percentile is an artifact of the sample, not a
|
|
59
|
+
// property of the pipeline. Reported anyway (hiding it invites re-deriving it
|
|
60
|
+
// wrong downstream) but flagged, so a caller cannot quote it as settled.
|
|
61
|
+
const RELIABLE_N = 10;
|
|
62
|
+
|
|
63
|
+
// The stages verifyFix runs, in execution order. Kept here so the recorder and
|
|
64
|
+
// the summariser cannot drift apart on stage naming.
|
|
65
|
+
const FIX_STAGES = Object.freeze(['rescan', 'lint', 'tests', 'honesty', 'poc']);
|
|
66
|
+
|
|
67
|
+
function _logPath(scanRoot) {
|
|
68
|
+
return node_path__WEBPACK_IMPORTED_MODULE_1__.join(scanRoot, STATE_DIR, LOG_FILE);
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
/**
|
|
72
|
+
* Append one verification attempt. Best-effort and silent on failure: metrics
|
|
73
|
+
* must never be able to fail a fix that otherwise verified.
|
|
74
|
+
*
|
|
75
|
+
* @returns {boolean} whether the record was written (for tests, not callers).
|
|
76
|
+
*/
|
|
77
|
+
function recordFixAttempt(scanRoot, record) {
|
|
78
|
+
if (!scanRoot || !record || typeof record !== 'object') return false;
|
|
79
|
+
try {
|
|
80
|
+
const dir = node_path__WEBPACK_IMPORTED_MODULE_1__.join(scanRoot, STATE_DIR);
|
|
81
|
+
if (!(0,_state_dir_js__WEBPACK_IMPORTED_MODULE_2__.isSafeStateDir)(dir)) return false;
|
|
82
|
+
node_fs__WEBPACK_IMPORTED_MODULE_0__.mkdirSync(dir, { recursive: true });
|
|
83
|
+
// One writeSync of one newline-terminated line: a concurrent reader sees
|
|
84
|
+
// whole records or nothing, and a torn tail is dropped on read.
|
|
85
|
+
node_fs__WEBPACK_IMPORTED_MODULE_0__.appendFileSync(_logPath(scanRoot), JSON.stringify(record) + '\n', 'utf8');
|
|
86
|
+
return true;
|
|
87
|
+
} catch { return false; }
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* Read every well-formed attempt. A line that does not parse is skipped, not
|
|
92
|
+
* fatal — the last line of an interrupted write is the expected case.
|
|
93
|
+
*/
|
|
94
|
+
function loadFixAttempts(scanRoot) {
|
|
95
|
+
try {
|
|
96
|
+
const raw = node_fs__WEBPACK_IMPORTED_MODULE_0__.readFileSync(_logPath(scanRoot), 'utf8');
|
|
97
|
+
const out = [];
|
|
98
|
+
for (const line of raw.split('\n')) {
|
|
99
|
+
if (!line.trim()) continue;
|
|
100
|
+
try {
|
|
101
|
+
const rec = JSON.parse(line);
|
|
102
|
+
if (rec && typeof rec === 'object' && typeof rec.totalMs === 'number') out.push(rec);
|
|
103
|
+
} catch { /* torn or hand-edited line — drop it, keep the rest */ }
|
|
104
|
+
}
|
|
105
|
+
return out;
|
|
106
|
+
} catch { return []; }
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
// Nearest-rank percentile over an ascending array. Nearest-rank rather than
|
|
110
|
+
// interpolated because these are observed durations, and an interpolated p50
|
|
111
|
+
// reports a duration that no run actually took.
|
|
112
|
+
function _pct(sorted, p) {
|
|
113
|
+
if (!sorted.length) return null;
|
|
114
|
+
const rank = Math.ceil((p / 100) * sorted.length);
|
|
115
|
+
return sorted[Math.min(sorted.length - 1, Math.max(0, rank - 1))];
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function _dist(values) {
|
|
119
|
+
const v = values.filter(x => typeof x === 'number' && Number.isFinite(x) && x >= 0).sort((a, b) => a - b);
|
|
120
|
+
if (!v.length) return { n: 0, minMs: null, p50Ms: null, p90Ms: null, maxMs: null, meanMs: null, reliable: false };
|
|
121
|
+
const sum = v.reduce((a, b) => a + b, 0);
|
|
122
|
+
return {
|
|
123
|
+
n: v.length,
|
|
124
|
+
minMs: v[0],
|
|
125
|
+
p50Ms: _pct(v, 50),
|
|
126
|
+
p90Ms: _pct(v, 90),
|
|
127
|
+
maxMs: v[v.length - 1],
|
|
128
|
+
meanMs: Math.round(sum / v.length),
|
|
129
|
+
// Says whether the percentiles above may be quoted, not whether the count
|
|
130
|
+
// is real. n and min/max/mean are exact at any sample size.
|
|
131
|
+
reliable: v.length >= RELIABLE_N,
|
|
132
|
+
};
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
// Which bucket an attempt belongs to. Deliberately total: every attempt lands
|
|
136
|
+
// in exactly one, so the bucket counts always sum to the attempt count and a
|
|
137
|
+
// mis-shaped record cannot silently vanish from the denominator.
|
|
138
|
+
function bucketOf(a) {
|
|
139
|
+
if (!a?.ok) return 'failed';
|
|
140
|
+
return a.testsRan ? 'validated' : 'validatedWithoutTests';
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
/**
|
|
144
|
+
* Summarise a set of attempts into the reported distribution.
|
|
145
|
+
*
|
|
146
|
+
* `validated` is the headline: attempts that verified AND whose test suite
|
|
147
|
+
* actually ran and passed. The other two buckets exist so that headline cannot
|
|
148
|
+
* be inflated by counting weaker or faster outcomes inside it.
|
|
149
|
+
*/
|
|
150
|
+
function summarizeFixDurations(attempts) {
|
|
151
|
+
const all = Array.isArray(attempts) ? attempts : [];
|
|
152
|
+
const buckets = { validated: [], validatedWithoutTests: [], failed: [] };
|
|
153
|
+
for (const a of all) buckets[bucketOf(a)].push(a);
|
|
154
|
+
|
|
155
|
+
const byStage = {};
|
|
156
|
+
for (const stage of FIX_STAGES) {
|
|
157
|
+
// Per-stage timings come from validated runs only. A stage's duration in a
|
|
158
|
+
// failed run is truncated by the failure (the pipeline stops), so mixing
|
|
159
|
+
// them in would understate every stage after the first failure point.
|
|
160
|
+
byStage[stage] = _dist(buckets.validated.map(a => a?.stages?.[stage]));
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
return {
|
|
164
|
+
attempts: all.length,
|
|
165
|
+
counts: {
|
|
166
|
+
validated: buckets.validated.length,
|
|
167
|
+
validatedWithoutTests: buckets.validatedWithoutTests.length,
|
|
168
|
+
failed: buckets.failed.length,
|
|
169
|
+
},
|
|
170
|
+
timeToValidatedFix: _dist(buckets.validated.map(a => a.totalMs)),
|
|
171
|
+
timeToValidatedFixWithoutTests: _dist(buckets.validatedWithoutTests.map(a => a.totalMs)),
|
|
172
|
+
timeToFailure: _dist(buckets.failed.map(a => a.totalMs)),
|
|
173
|
+
byStage,
|
|
174
|
+
reliableAtOrAbove: RELIABLE_N,
|
|
175
|
+
};
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
/** Read + summarise in one step. */
|
|
179
|
+
function fixDurationReport(scanRoot) {
|
|
180
|
+
return summarizeFixDurations(loadFixAttempts(scanRoot));
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
function _ms(v) {
|
|
184
|
+
if (v == null) return '—';
|
|
185
|
+
return v >= 1000 ? `${(v / 1000).toFixed(1)}s` : `${v}ms`;
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
/**
|
|
189
|
+
* One-paragraph human summary. Returns null when there is nothing measured —
|
|
190
|
+
* callers print nothing rather than printing an empty table.
|
|
191
|
+
*/
|
|
192
|
+
function renderFixDurationSummary(sum) {
|
|
193
|
+
if (!sum || !sum.attempts) return null;
|
|
194
|
+
const d = sum.timeToValidatedFix;
|
|
195
|
+
const parts = [];
|
|
196
|
+
if (d.n) {
|
|
197
|
+
parts.push(
|
|
198
|
+
`time-to-validated-fix: median ${_ms(d.p50Ms)}, p90 ${_ms(d.p90Ms)} `
|
|
199
|
+
+ `(n=${d.n}${d.reliable ? '' : `, below ${sum.reliableAtOrAbove} — percentiles not yet reliable`})`,
|
|
200
|
+
);
|
|
201
|
+
} else {
|
|
202
|
+
parts.push('time-to-validated-fix: no fix has both verified and had its test suite run yet');
|
|
203
|
+
}
|
|
204
|
+
if (sum.counts.validatedWithoutTests) {
|
|
205
|
+
parts.push(`${sum.counts.validatedWithoutTests} verified with no detectable test suite (excluded from the median above)`);
|
|
206
|
+
}
|
|
207
|
+
if (sum.counts.failed) {
|
|
208
|
+
parts.push(`${sum.counts.failed} failed verification, median ${_ms(sum.timeToFailure.p50Ms)} (counted separately)`);
|
|
209
|
+
}
|
|
210
|
+
return parts.join('; ') + '.';
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
const _internals = { _dist, _pct, RELIABLE_N };
|
|
214
|
+
|
|
215
|
+
|
|
216
|
+
/***/ }),
|
|
217
|
+
|
|
5
218
|
/***/ 4113:
|
|
6
219
|
/***/ ((__unused_webpack___webpack_module__, __webpack_exports__, __webpack_require__) => {
|
|
7
220
|
|
|
@@ -521,6 +734,8 @@ function runProjectTests(scanRoot, { timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
|
|
521
734
|
};
|
|
522
735
|
}
|
|
523
736
|
|
|
737
|
+
// EXTERNAL MODULE: ./src/posture/fix-metrics.js
|
|
738
|
+
var fix_metrics = __webpack_require__(2238);
|
|
524
739
|
;// CONCATENATED MODULE: ./src/posture/fix-verify.js
|
|
525
740
|
// Closed-loop /fix verification (Sentinel-parity FR-L4-4, FR-L4-5).
|
|
526
741
|
//
|
|
@@ -546,6 +761,7 @@ function runProjectTests(scanRoot, { timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
|
|
546
761
|
|
|
547
762
|
|
|
548
763
|
|
|
764
|
+
|
|
549
765
|
const SEVERITY_RANK = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
|
|
550
766
|
|
|
551
767
|
// Run a focused re-scan over just the patched file(s) using the in-memory
|
|
@@ -688,10 +904,23 @@ async function verifyFix({
|
|
|
688
904
|
depFileContents,
|
|
689
905
|
fixMeta,
|
|
690
906
|
testTimeoutMs,
|
|
907
|
+
recordMetrics = true,
|
|
908
|
+
poc,
|
|
691
909
|
} = {}) {
|
|
910
|
+
// R5 (reporting half) — time each stage as it runs. Measured here rather
|
|
911
|
+
// than inside each stage because only this function knows the boundaries of
|
|
912
|
+
// one verification ATTEMPT, which is the unit the distribution is over.
|
|
913
|
+
const stages = {};
|
|
914
|
+
const t0 = Date.now();
|
|
915
|
+
let mark = t0;
|
|
916
|
+
const _lap = (name) => { const now = Date.now(); stages[name] = now - mark; mark = now; };
|
|
917
|
+
|
|
692
918
|
const rescan = await verifyPatch({ scanRoot, originalFindingStableId, files, depFileContents });
|
|
919
|
+
_lap('rescan');
|
|
693
920
|
const lint = runProjectLinter(scanRoot, Object.keys(files || {}));
|
|
921
|
+
_lap('lint');
|
|
694
922
|
const tests = runProjectTests(scanRoot, testTimeoutMs != null ? { timeoutMs: testTimeoutMs } : {});
|
|
923
|
+
_lap('tests');
|
|
695
924
|
// True when a candidate patch was supplied but has not been written, so the
|
|
696
925
|
// suite necessarily ran against the pre-patch tree. Surfaced in the summary
|
|
697
926
|
// and on the result so a caller cannot mistake it for a verified patch.
|
|
@@ -701,7 +930,39 @@ async function verifyFix({
|
|
|
701
930
|
if (fixMeta && typeof fixMeta === 'object') {
|
|
702
931
|
try { honesty = gateFixOutput(fixMeta); } catch { honesty = null; }
|
|
703
932
|
}
|
|
704
|
-
|
|
933
|
+
_lap('honesty');
|
|
934
|
+
|
|
935
|
+
// R5 — the PoC leg. Re-run the finding's proof-of-concept against the
|
|
936
|
+
// CANDIDATE patch inside R1's sandbox. A patch that still lets the PoC
|
|
937
|
+
// demonstrate the predicted effect has not fixed anything, however green the
|
|
938
|
+
// re-scan looks: the re-scan only proves the DETECTOR stopped firing, which
|
|
939
|
+
// a cosmetic edit can achieve. Execution is the stronger claim.
|
|
940
|
+
//
|
|
941
|
+
// Direction matters and is asymmetric on purpose. `execution-proven` after
|
|
942
|
+
// the patch is a hard FAIL. Anything else is NOT a pass — a PoC that failed
|
|
943
|
+
// to run, or a sandbox that could not start, is recorded as `inconclusive`
|
|
944
|
+
// and left out of the verdict entirely. Treating "could not prove it" as
|
|
945
|
+
// "fixed" is exactly the false confidence this leg exists to prevent.
|
|
946
|
+
let pocLeg = { status: 'not-requested', reason: null, tier: null };
|
|
947
|
+
if (poc?.code) {
|
|
948
|
+
try {
|
|
949
|
+
const { proveFinding } = await __webpack_require__.e(/* import() */ 259).then(__webpack_require__.bind(__webpack_require__, 8259));
|
|
950
|
+
const proved = await proveFinding({ ...(poc.finding || {}), poc }, { files });
|
|
951
|
+
const tier = proved.proofTier;
|
|
952
|
+
pocLeg = tier === 'execution-proven'
|
|
953
|
+
? { status: 'still-exploitable', tier, reason: proved.proofEvidence?.observed || null }
|
|
954
|
+
: proved.proofEvidence?.ran
|
|
955
|
+
? { status: 'no-longer-proven', tier, reason: proved.proofEvidence?.reason || null }
|
|
956
|
+
: { status: 'inconclusive', tier, reason: proved.proofEvidence?.reason || null };
|
|
957
|
+
} catch (e) {
|
|
958
|
+
pocLeg = { status: 'inconclusive', tier: null, reason: `proof harness error: ${e.message}` };
|
|
959
|
+
}
|
|
960
|
+
}
|
|
961
|
+
_lap('poc');
|
|
962
|
+
const pocOk = pocLeg.status !== 'still-exploitable';
|
|
963
|
+
|
|
964
|
+
const ok = rescan.ok && (lint.ok || lint.skipped) && testsOk && pocOk && (honesty ? honesty.ok : true);
|
|
965
|
+
const durations = { ...stages, totalMs: Date.now() - t0 };
|
|
705
966
|
const summary = [
|
|
706
967
|
`re-scan: ${rescan.ok ? 'PASS' : 'FAIL — ' + rescan.reason}`,
|
|
707
968
|
`linter: ${lint.runner === 'none' ? 'skipped (no linter config)'
|
|
@@ -717,8 +978,36 @@ async function verifyFix({
|
|
|
717
978
|
: tests.passed ? `PASS${_testedPrePatch ? ' — on the CURRENT on-disk tree, NOT the candidate patch' : ''}`
|
|
718
979
|
: `FAIL (exit ${tests.exitCode})`}`,
|
|
719
980
|
honesty ? `honesty: ${honesty.ok ? `PASS (${honesty.tier})` : 'FAIL — ' + honesty.violations.join('; ')}` : null,
|
|
981
|
+
// Never render `inconclusive` as a pass — say plainly that nothing was proven.
|
|
982
|
+
pocLeg.status === 'not-requested' ? null
|
|
983
|
+
: pocLeg.status === 'still-exploitable' ? `poc: FAIL — the proof-of-concept still demonstrates the vulnerability against the patch`
|
|
984
|
+
: pocLeg.status === 'no-longer-proven' ? 'poc: PASS (ran against the patch and no longer demonstrates the vulnerability)'
|
|
985
|
+
: `poc: inconclusive — not counted either way (${pocLeg.reason || 'no detail reported'})`,
|
|
720
986
|
].filter(Boolean).join('\n');
|
|
721
|
-
|
|
987
|
+
// Persist the attempt so the distribution can be reported from real runs.
|
|
988
|
+
// `testsRan` is the load-bearing field: it is what keeps "verified with no
|
|
989
|
+
// test suite to run" out of the headline time-to-validated-fix bucket.
|
|
990
|
+
// A patch that was never written to disk is recorded too, but flagged — its
|
|
991
|
+
// suite ran against the pre-patch tree, so its timing is real while its
|
|
992
|
+
// verdict is about a different tree.
|
|
993
|
+
if (recordMetrics && scanRoot) {
|
|
994
|
+
(0,fix_metrics/* recordFixAttempt */.I3)(scanRoot, {
|
|
995
|
+
at: new Date().toISOString(),
|
|
996
|
+
stableId: originalFindingStableId || null,
|
|
997
|
+
ok,
|
|
998
|
+
testsRan: !tests.skipped,
|
|
999
|
+
testsPassed: tests.skipped ? null : tests.passed === true,
|
|
1000
|
+
testedPrePatch: _testedPrePatch,
|
|
1001
|
+
lintRan: !(lint.skipped || lint.runner === 'none'),
|
|
1002
|
+
honestyGated: honesty != null,
|
|
1003
|
+
pocStatus: pocLeg.status,
|
|
1004
|
+
files: Object.keys(files || {}).length,
|
|
1005
|
+
stages,
|
|
1006
|
+
totalMs: durations.totalMs,
|
|
1007
|
+
});
|
|
1008
|
+
}
|
|
1009
|
+
|
|
1010
|
+
return { ok, rescan, lint, tests, testedPrePatch: _testedPrePatch, honesty, poc: pocLeg, durations, summary };
|
|
722
1011
|
}
|
|
723
1012
|
|
|
724
1013
|
|
package/dist/207.index.js
CHANGED
|
@@ -73,10 +73,13 @@ const PROVES =
|
|
|
73
73
|
'(same rule id, severity, file, line, cwe, vuln, and multiplicity) produced by the same ' +
|
|
74
74
|
'engine version, ruleset version, and bundle — regardless of emission order.';
|
|
75
75
|
const DOES_NOT_PROVE =
|
|
76
|
-
'It does not prove cross-machine reproducibility:
|
|
77
|
-
'
|
|
78
|
-
'
|
|
79
|
-
'
|
|
76
|
+
'It does not prove cross-machine reproducibility: this attestation is one run on one machine, ' +
|
|
77
|
+
'nothing here compares a second machine, OS, or Node version, and some detectors are ' +
|
|
78
|
+
'environment-sensitive. That property is tested separately by the determinism-attest / ' +
|
|
79
|
+
'determinism-compare CI jobs, which run the same commit on two operating systems and fail ' +
|
|
80
|
+
'unless the digests match — evidence about the ENGINE, not about this attestation. ' +
|
|
81
|
+
'A signature, when present, is a symmetric per-install HMAC — tamper-evidence for this ' +
|
|
82
|
+
'install, not third-party non-repudiation.';
|
|
80
83
|
|
|
81
84
|
function _str(v) { return v === undefined || v === null ? '' : String(v); }
|
|
82
85
|
|