jules-orchestrator-kit 0.60.0 → 0.64.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/package.json +4 -2
- package/scripts/guard-reach-check.mjs +226 -0
- package/scripts/package-integrity-check.mjs +251 -0
- package/scripts/release.mjs +33 -0
- package/src/assertions.mjs +16 -0
- package/src/config.mjs +68 -0
- package/src/coverage.mjs +17 -2
- package/src/engine.mjs +33 -2
- package/src/evidence.mjs +38 -1
- package/src/guard-policy.mjs +481 -0
- package/src/ops/test-collection.mjs +149 -0
- package/src/security.mjs +216 -10
- package/src/stack-detector.mjs +113 -10
- package/src/task-optimizer.mjs +7 -3
package/README.md
CHANGED
|
@@ -204,7 +204,7 @@ To maximize PR merge rates, dispatch tasks according to deterministic boundaries
|
|
|
204
204
|
* **Fail-Closed Security & Secret Redaction:** Evaluates explicit Deny rules before Allow rules against canonicalized, case-folded paths. Redacts high-entropy keys and base64-encoded credentials (such as Kubernetes `Secret` manifests).
|
|
205
205
|
* **Complexity & Cost Router:** Zero-dependency heuristic classifier (`src/router.mjs`) routing mechanical tasks to lightweight models while reserving primary models for complex refactors, with a `node --check` syntax-verification gate that transparently escalates a FAST-tier result to the primary provider if it left broken JS on disk.
|
|
206
206
|
* **Terminal UI & Diagnostic Matrix (`agentctl doctor`):** Interactive terminal dashboard, task sidecar manager, and automated transactional self-repair.
|
|
207
|
-
* **Verified Test Suite:** Tested with **
|
|
207
|
+
* **Verified Test Suite:** Tested with **1085 unit tests across 150 suites passing in < 15.0s**.
|
|
208
208
|
|
|
209
209
|
<br/>
|
|
210
210
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "jules-orchestrator-kit",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.64.0",
|
|
4
4
|
"description": "Zero-dependency safety gatekeeper, test oracle generator, and multi-agent coordination protocol for autonomous coding agents — Google Jules, Claude Code, Codex and Gemini CLI.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -55,7 +55,9 @@
|
|
|
55
55
|
"jules:rules-lint": "node scripts/rules-lint.mjs",
|
|
56
56
|
"jules:doc-sync": "node scripts/doc-sync-check.mjs",
|
|
57
57
|
"release": "node scripts/release.mjs",
|
|
58
|
-
"
|
|
58
|
+
"guard-reach": "node scripts/guard-reach-check.mjs",
|
|
59
|
+
"lint": "eslint .",
|
|
60
|
+
"package-integrity": "node scripts/package-integrity-check.mjs"
|
|
59
61
|
},
|
|
60
62
|
"engines": {
|
|
61
63
|
"node": ">=20.0.0"
|
|
@@ -0,0 +1,226 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Activation coverage: proof that every blocking check can still be made red.
|
|
5
|
+
*
|
|
6
|
+
* A defect that turns a check off cannot be found by the check it turns off.
|
|
7
|
+
* That is not a hypothetical — `isTestFile` matched the substring `/test/`,
|
|
8
|
+
* which does not occur in `tests/test_calc.py`, so the entire tamper guard was
|
|
9
|
+
* silent for the standard pytest, Rust and RSpec layouts. Every mechanism that
|
|
10
|
+
* should have caught it was working exactly as designed:
|
|
11
|
+
*
|
|
12
|
+
* - the unit suite sampled the same distribution the implementation was
|
|
13
|
+
* written from, so its fixtures re-confirmed the dialect it already knew;
|
|
14
|
+
* - the doc-sync gate compares counts and versions, and a guard that guards
|
|
15
|
+
* nothing still contributes passing tests;
|
|
16
|
+
* - the nine-way CI matrix varies OS and Node version — dimensions
|
|
17
|
+
* orthogonal to the defect. Nine runs of `test/foo.test.js` never explore
|
|
18
|
+
* `tests/test_calc.py`;
|
|
19
|
+
* - cold review reads the code against its stated intent, and here the code
|
|
20
|
+
* and the intent agreed. The eye supplies the leading slash;
|
|
21
|
+
* - the release gate is a conjunction over those four, and a signal that
|
|
22
|
+
* silently goes absent contributes `true`.
|
|
23
|
+
*
|
|
24
|
+
* The common property: `ok: true` from a check that examined nothing is
|
|
25
|
+
* byte-identical to `ok: true` from a check that examined everything. There is
|
|
26
|
+
* no denominator. This script supplies one.
|
|
27
|
+
*
|
|
28
|
+
* Three steps, all in-process, no dependencies, well under a second:
|
|
29
|
+
*
|
|
30
|
+
* 1. POLICY — the hand-written witness table in src/guard-policy.mjs
|
|
31
|
+
* must hold. It is derived from what the tool advertises, never
|
|
32
|
+
* from the regexes that implement it.
|
|
33
|
+
* 2. CANARIES — every known-bad input must produce the finding it names. A
|
|
34
|
+
* canary that comes back clean is not a pass; it is proof that
|
|
35
|
+
* the rule stopped being reachable.
|
|
36
|
+
* 3. MUTANTS — each hand-written mutant of the applicability predicate must
|
|
37
|
+
* kill at least one canary. A surviving mutant means no canary
|
|
38
|
+
* ever required the guard to activate, so the suite would stay
|
|
39
|
+
* green if it silently stopped looking.
|
|
40
|
+
*
|
|
41
|
+
* Usage: node scripts/guard-reach-check.mjs [--json]
|
|
42
|
+
* Exit codes: 0 = every guard reachable, 1 = a guard has gone silent.
|
|
43
|
+
*/
|
|
44
|
+
|
|
45
|
+
import { checkTestTampering, checkScope } from "../src/security.mjs";
|
|
46
|
+
import { isTestPath } from "../src/test-paths.mjs";
|
|
47
|
+
import { normalizeScope } from "../src/config.mjs";
|
|
48
|
+
import { parseCollectedTests } from "../src/ops/test-collection.mjs";
|
|
49
|
+
import {
|
|
50
|
+
TEST_PATH_CASES,
|
|
51
|
+
TAMPER_CANARIES,
|
|
52
|
+
PREDICATE_MUTANTS,
|
|
53
|
+
EMPTY_RUN_CANARIES,
|
|
54
|
+
SCOPE_CANARIES,
|
|
55
|
+
INNOCENT_EDITS,
|
|
56
|
+
UNREADABLE_DIALECTS,
|
|
57
|
+
} from "../src/guard-policy.mjs";
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* Build a unified diff for one canary.
|
|
61
|
+
*
|
|
62
|
+
* The context line carries the comment syntax of the file's own language.
|
|
63
|
+
* A `//` line in a `.py` fixture is not a comment, and a fixture that lies
|
|
64
|
+
* about the language under test measures the fixture rather than the guard —
|
|
65
|
+
* which is how two false results were once read as two defects.
|
|
66
|
+
*/
|
|
67
|
+
function canaryDiff(c) {
|
|
68
|
+
const ctx = c.context || "// context";
|
|
69
|
+
const lines = [`--- a/${c.file}`, `+++ b/${c.file}`, "@@ -1,20 +1,20 @@", ` ${ctx}`];
|
|
70
|
+
for (const l of c.removed) lines.push(`-${l}`);
|
|
71
|
+
for (const l of c.added) lines.push(`+${l}`);
|
|
72
|
+
lines.push(` ${ctx}`);
|
|
73
|
+
return lines.join("\n");
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
const failures = [];
|
|
77
|
+
const checks = [];
|
|
78
|
+
const add = (name, ok, detail) => {
|
|
79
|
+
checks.push({ name, ok, detail });
|
|
80
|
+
if (!ok) failures.push(`${name}: ${detail}`);
|
|
81
|
+
};
|
|
82
|
+
|
|
83
|
+
// --- 1. Policy contract -----------------------------------------------------
|
|
84
|
+
{
|
|
85
|
+
const wrong = TEST_PATH_CASES.filter((c) => isTestPath(c.path) !== c.expected);
|
|
86
|
+
add(
|
|
87
|
+
"policy: test-path domain",
|
|
88
|
+
wrong.length === 0,
|
|
89
|
+
wrong.length
|
|
90
|
+
? wrong.map((c) => `${c.path} → ${isTestPath(c.path)}, policy says ${c.expected} (${c.why})`).join("; ")
|
|
91
|
+
: `${TEST_PATH_CASES.length} witnesses hold`
|
|
92
|
+
);
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
{
|
|
96
|
+
const scope = normalizeScope({ deny: [], allow: [], protect: [] });
|
|
97
|
+
const wrong = [];
|
|
98
|
+
for (const c of SCOPE_CANARIES) {
|
|
99
|
+
const res = checkScope([c.path], scope);
|
|
100
|
+
const rule = res.ok ? "none" : res.violations[0].rule;
|
|
101
|
+
if (rule !== c.rule) wrong.push(`${c.path} → ${rule}, policy says ${c.rule} (${c.why})`);
|
|
102
|
+
}
|
|
103
|
+
add("policy: scope tiers", wrong.length === 0, wrong.length ? wrong.join("; ") : `${SCOPE_CANARIES.length} paths tiered as declared`);
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
{
|
|
107
|
+
const missed = EMPTY_RUN_CANARIES.filter((c) => parseCollectedTests(c.output, "").count !== 0);
|
|
108
|
+
add(
|
|
109
|
+
"policy: empty-run detection",
|
|
110
|
+
missed.length === 0,
|
|
111
|
+
missed.length ? `${missed.map((m) => m.id).join(", ")} report zero tests in a spelling the floor cannot read` : `${EMPTY_RUN_CANARIES.length} runners recognised`
|
|
112
|
+
);
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
// --- 2. Canaries ------------------------------------------------------------
|
|
116
|
+
const canaryResults = new Map();
|
|
117
|
+
{
|
|
118
|
+
const silent = [];
|
|
119
|
+
const noDenominator = [];
|
|
120
|
+
const noAssertions = [];
|
|
121
|
+
for (const c of TAMPER_CANARIES) {
|
|
122
|
+
const res = checkTestTampering(canaryDiff(c));
|
|
123
|
+
const hit = (res.violations || []).some((v) => v.type === c.expect);
|
|
124
|
+
canaryResults.set(c.id, hit);
|
|
125
|
+
if (!hit) silent.push(`${c.id} expected ${c.expect}, got ${JSON.stringify((res.violations || []).map((v) => v.type))}`);
|
|
126
|
+
// A finding with no denominator is the shape this script exists to reject.
|
|
127
|
+
if (hit && !(res.inputsSeen > 0)) noDenominator.push(c.id);
|
|
128
|
+
// Counting lines was not enough: a JUnit diff reported one input examined
|
|
129
|
+
// and a clean PASS while every assertion in it went unrecognised. A rule
|
|
130
|
+
// about assertions has to say how many assertions it actually read.
|
|
131
|
+
if (hit && c.expect !== "TEST_SKIP_INJECTION" && !(res.assertionsSeen > 0)) {
|
|
132
|
+
noAssertions.push(`${c.id} (${res.assertionsSeen} assertions parsed)`);
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
add("canaries: every tamper rule still fires", silent.length === 0, silent.length ? silent.join("; ") : `${TAMPER_CANARIES.length} canaries red as required`);
|
|
136
|
+
add("canaries: every finding carries a denominator", noDenominator.length === 0, noDenominator.length ? noDenominator.join(", ") : "inputsSeen > 0 on every hit");
|
|
137
|
+
add("canaries: assertion rules parsed an assertion", noAssertions.length === 0, noAssertions.length ? noAssertions.join(", ") : "assertionsSeen > 0 on every assertion finding");
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
// --- 3. The opposite failure: flagging what is innocent ---------------------
|
|
141
|
+
{
|
|
142
|
+
const noisy = [];
|
|
143
|
+
for (const e of INNOCENT_EDITS) {
|
|
144
|
+
const res = checkTestTampering(canaryDiff(e));
|
|
145
|
+
const types = (res.violations || []).map((v) => v.type);
|
|
146
|
+
if (types.length > 0) noisy.push(`${e.id} → ${JSON.stringify(types)} (${e.why})`);
|
|
147
|
+
}
|
|
148
|
+
add(
|
|
149
|
+
"innocent edits stay silent",
|
|
150
|
+
noisy.length === 0,
|
|
151
|
+
noisy.length
|
|
152
|
+
? noisy.join("; ")
|
|
153
|
+
: `${INNOCENT_EDITS.length} ordinary edits produce no finding`
|
|
154
|
+
);
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
{
|
|
158
|
+
const quiet = [];
|
|
159
|
+
for (const d of UNREADABLE_DIALECTS) {
|
|
160
|
+
const res = checkTestTampering(canaryDiff(d));
|
|
161
|
+
if (res.status !== "UNREADABLE") quiet.push(`${d.id} → ${res.status} (${d.why})`);
|
|
162
|
+
}
|
|
163
|
+
add(
|
|
164
|
+
"an unparsable dialect says so",
|
|
165
|
+
quiet.length === 0,
|
|
166
|
+
quiet.length
|
|
167
|
+
? `${quiet.join("; ")} — coverage ending is fine, ending silently is not`
|
|
168
|
+
: `${UNREADABLE_DIALECTS.length} unsupported dialects reported, not passed`
|
|
169
|
+
);
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
// --- 4. Predicate mutants ---------------------------------------------------
|
|
173
|
+
{
|
|
174
|
+
const survivors = [];
|
|
175
|
+
for (const mutant of PREDICATE_MUTANTS) {
|
|
176
|
+
let killed = false;
|
|
177
|
+
for (const c of TAMPER_CANARIES) {
|
|
178
|
+
// Only canaries the healthy predicate catches can kill a mutant.
|
|
179
|
+
if (!canaryResults.get(c.id)) continue;
|
|
180
|
+
const res = checkTestTampering(canaryDiff(c), { isTestPath: mutant.fn });
|
|
181
|
+
if (!(res.violations || []).some((v) => v.type === c.expect)) {
|
|
182
|
+
killed = true;
|
|
183
|
+
break;
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
if (!killed) survivors.push(`${mutant.id} (${mutant.why})`);
|
|
187
|
+
}
|
|
188
|
+
add(
|
|
189
|
+
"mutants: blinding the predicate breaks a canary",
|
|
190
|
+
survivors.length === 0,
|
|
191
|
+
survivors.length
|
|
192
|
+
? `survived: ${survivors.join(", ")} — no canary required the guard to activate`
|
|
193
|
+
: `${PREDICATE_MUTANTS.length} mutants killed`
|
|
194
|
+
);
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
// --- Report -----------------------------------------------------------------
|
|
198
|
+
const json = process.argv.includes("--json");
|
|
199
|
+
const activated = [...canaryResults.values()].filter(Boolean).length;
|
|
200
|
+
|
|
201
|
+
if (json) {
|
|
202
|
+
console.log(
|
|
203
|
+
JSON.stringify(
|
|
204
|
+
{
|
|
205
|
+
ok: failures.length === 0,
|
|
206
|
+
checks,
|
|
207
|
+
activationCoverage: { canaries: canaryResults.size, activated },
|
|
208
|
+
},
|
|
209
|
+
null,
|
|
210
|
+
2
|
|
211
|
+
)
|
|
212
|
+
);
|
|
213
|
+
} else {
|
|
214
|
+
console.log("\n🎯 Guard Reach Check (activation coverage)");
|
|
215
|
+
console.log("-------------------------------------------------------");
|
|
216
|
+
for (const c of checks) console.log(` ${c.ok ? "✅" : "❌"} ${c.name.padEnd(46)} ${c.detail}`);
|
|
217
|
+
console.log("-------------------------------------------------------");
|
|
218
|
+
console.log(` canaries activated: ${activated}/${canaryResults.size}`);
|
|
219
|
+
console.log(
|
|
220
|
+
failures.length === 0
|
|
221
|
+
? "✅ Every blocking guard can still be made red.\n"
|
|
222
|
+
: `\n❌ ${failures.length} guard(s) may have gone silent. A check that cannot be made red is not a check.\n`
|
|
223
|
+
);
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
process.exit(failures.length === 0 ? 0 : 1);
|
|
@@ -0,0 +1,251 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Does the package we publish actually work?
|
|
5
|
+
*
|
|
6
|
+
* Every test in this repository runs against the source tree, where every
|
|
7
|
+
* file is present by definition. What users install is a tarball built from
|
|
8
|
+
* the `files` list in package.json, and nothing checked that the two agreed.
|
|
9
|
+
*
|
|
10
|
+
* They did not. v0.63.0 shipped `scripts/guard-reach-check.mjs` — the check
|
|
11
|
+
* whose entire purpose is to prove no guard has silently gone missing —
|
|
12
|
+
* while leaving behind the policy contract it imports. Unpacked and run, it
|
|
13
|
+
* threw ERR_MODULE_NOT_FOUND. The CLI was fine, 1015 tests were green, nine
|
|
14
|
+
* CI cells passed, and the published artefact still had a hole in it,
|
|
15
|
+
* because every one of those signals was measured somewhere the file existed.
|
|
16
|
+
*
|
|
17
|
+
* This asks the packer what it would ship, and then resolves the import
|
|
18
|
+
* graph inside that answer.
|
|
19
|
+
*
|
|
20
|
+
* Usage: node scripts/package-integrity-check.mjs [--json]
|
|
21
|
+
* Exit codes: 0 = the tarball is self-contained, 1 = it is not.
|
|
22
|
+
*/
|
|
23
|
+
|
|
24
|
+
import { execFileSync } from "node:child_process";
|
|
25
|
+
import { readFileSync, existsSync } from "node:fs";
|
|
26
|
+
import { resolve, dirname, relative, sep } from "node:path";
|
|
27
|
+
import { fileURLToPath } from "node:url";
|
|
28
|
+
|
|
29
|
+
import { IMPORT_EXTRACTION_CASES } from "../src/guard-policy.mjs";
|
|
30
|
+
|
|
31
|
+
const root = fileURLToPath(new URL("..", import.meta.url));
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* Mark every character that sits inside a string literal or a comment.
|
|
35
|
+
*
|
|
36
|
+
* Without this, an import quoted *inside* a fixture string reads as an
|
|
37
|
+
* import of the file itself — this check's first run reported six broken
|
|
38
|
+
* imports in the policy contract, all of them example text. A guard that
|
|
39
|
+
* cries wolf about its own fixtures gets switched off in a week.
|
|
40
|
+
*
|
|
41
|
+
* Approximate on purpose, and biased on purpose: a regex literal holding a
|
|
42
|
+
* quote can open a phantom string, so the counts reported below exist to
|
|
43
|
+
* make a mask that swallowed the file visible rather than silent.
|
|
44
|
+
*/
|
|
45
|
+
function stringMask(src) {
|
|
46
|
+
const mask = new Uint8Array(src.length);
|
|
47
|
+
let i = 0;
|
|
48
|
+
let quote = null;
|
|
49
|
+
let comment = null;
|
|
50
|
+
while (i < src.length) {
|
|
51
|
+
const c = src[i];
|
|
52
|
+
const d = src[i + 1];
|
|
53
|
+
if (comment === "line") {
|
|
54
|
+
if (c === "\n") comment = null;
|
|
55
|
+
else mask[i] = 1;
|
|
56
|
+
i++;
|
|
57
|
+
continue;
|
|
58
|
+
}
|
|
59
|
+
if (comment === "block") {
|
|
60
|
+
mask[i] = 1;
|
|
61
|
+
if (c === "*" && d === "/") {
|
|
62
|
+
mask[i + 1] = 1;
|
|
63
|
+
comment = null;
|
|
64
|
+
i += 2;
|
|
65
|
+
continue;
|
|
66
|
+
}
|
|
67
|
+
i++;
|
|
68
|
+
continue;
|
|
69
|
+
}
|
|
70
|
+
if (quote !== null) {
|
|
71
|
+
mask[i] = 1;
|
|
72
|
+
if (c === "\\") {
|
|
73
|
+
if (i + 1 < src.length) mask[i + 1] = 1;
|
|
74
|
+
i += 2;
|
|
75
|
+
continue;
|
|
76
|
+
}
|
|
77
|
+
if (c === quote) quote = null;
|
|
78
|
+
i++;
|
|
79
|
+
continue;
|
|
80
|
+
}
|
|
81
|
+
if (c === "/" && d === "/") { comment = "line"; mask[i] = 1; i++; continue; }
|
|
82
|
+
if (c === "/" && d === "*") { comment = "block"; mask[i] = 1; i++; continue; }
|
|
83
|
+
// A regex literal holding a quote — `/["']/` — opens a string that never
|
|
84
|
+
// closes, and everything after it is misread. This file's own subject
|
|
85
|
+
// matter is regexes full of quote characters, so the mask desynchronised
|
|
86
|
+
// and a `require("./calc")` written inside a comment was reported as a
|
|
87
|
+
// missing module. Deciding regex-versus-division on the previous token is
|
|
88
|
+
// the same approximation the diff scanner already makes.
|
|
89
|
+
if (c === "/") {
|
|
90
|
+
let k = i - 1;
|
|
91
|
+
while (k >= 0 && /\s/.test(src[k])) k--;
|
|
92
|
+
const prev = k >= 0 ? src[k] : "";
|
|
93
|
+
if (prev === "" || "(,=:[!&|?{};+-*%~^<>".includes(prev)) {
|
|
94
|
+
mask[i] = 1;
|
|
95
|
+
let j = i + 1;
|
|
96
|
+
let cls = false;
|
|
97
|
+
while (j < src.length) {
|
|
98
|
+
const e = src[j];
|
|
99
|
+
mask[j] = 1;
|
|
100
|
+
if (e === "\\") { if (j + 1 < src.length) mask[j + 1] = 1; j += 2; continue; }
|
|
101
|
+
if (e === "[") cls = true;
|
|
102
|
+
else if (e === "]") cls = false;
|
|
103
|
+
else if (e === "/" && !cls) { j++; break; }
|
|
104
|
+
else if (e === "\n") break;
|
|
105
|
+
j++;
|
|
106
|
+
}
|
|
107
|
+
i = j;
|
|
108
|
+
continue;
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
if (c === '"' || c === "'" || c === "`") { quote = c; mask[i] = 1; i++; continue; }
|
|
112
|
+
i++;
|
|
113
|
+
}
|
|
114
|
+
return mask;
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
/**
|
|
118
|
+
* Every module specifier in a source file.
|
|
119
|
+
*
|
|
120
|
+
* Deliberately newline-tolerant. A matcher bounded by `[^;\n]*?` cannot see
|
|
121
|
+
* a multi-line named import, which is exactly the import that was missing
|
|
122
|
+
* from the tarball, so the first version of this check certified the broken
|
|
123
|
+
* package as sound. The extraction cases in the policy contract exist to
|
|
124
|
+
* keep that from being a private mistake twice.
|
|
125
|
+
*/
|
|
126
|
+
export function extractSpecifiers(src, { skipQuoted = true } = {}) {
|
|
127
|
+
const mask = skipQuoted ? stringMask(src) : null;
|
|
128
|
+
const found = new Set();
|
|
129
|
+
const collect = (re) => {
|
|
130
|
+
for (const m of src.matchAll(re)) {
|
|
131
|
+
// What decides is the token immediately before the specifier — the
|
|
132
|
+
// `from`, or the `(` of a call. In a real import it is code; in a
|
|
133
|
+
// fixture it is the middle of a string. Testing the *keyword* instead
|
|
134
|
+
// was not enough: `export const CASES = [` at the top of a file
|
|
135
|
+
// matched lazily forward into the first `from "…"` inside an example,
|
|
136
|
+
// so a genuine keyword lent its authority to quoted text.
|
|
137
|
+
if (mask) {
|
|
138
|
+
let j = m.indices[1][0] - 2;
|
|
139
|
+
while (j >= 0 && /\s/.test(src[j])) j--;
|
|
140
|
+
if (j >= 0 && mask[j]) continue;
|
|
141
|
+
}
|
|
142
|
+
found.add(m[1]);
|
|
143
|
+
}
|
|
144
|
+
};
|
|
145
|
+
collect(/(?:^|[\s;}])(?:import|export)\s[\s\S]{0,500}?from\s*["']([^"']+)["']/gd);
|
|
146
|
+
collect(/(?:^|[\s;}])import\s*["']([^"']+)["']/gd);
|
|
147
|
+
collect(/\bimport\s*\(\s*["']([^"']+)["']/gd);
|
|
148
|
+
collect(/\brequire\s*\(\s*["']([^"']+)["']/gd);
|
|
149
|
+
return [...found];
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
/**
|
|
153
|
+
* Run every integrity check and return the result.
|
|
154
|
+
*
|
|
155
|
+
* Exported as a function rather than run on import: a module that checks the
|
|
156
|
+
* package must not exit the process of anything that merely imports it.
|
|
157
|
+
*/
|
|
158
|
+
export function checkPackageIntegrity() {
|
|
159
|
+
const failures = [];
|
|
160
|
+
const checks = [];
|
|
161
|
+
const add = (name, ok, detail) => {
|
|
162
|
+
checks.push({ name, ok, detail });
|
|
163
|
+
if (!ok) failures.push(`${name}: ${detail}`);
|
|
164
|
+
};
|
|
165
|
+
|
|
166
|
+
// --- 1. The extractor must be able to see what it claims to look for -------
|
|
167
|
+
{
|
|
168
|
+
const wrong = [];
|
|
169
|
+
for (const c of IMPORT_EXTRACTION_CASES) {
|
|
170
|
+
const got = extractSpecifiers(c.src).sort();
|
|
171
|
+
const want = [...c.expect].sort();
|
|
172
|
+
if (JSON.stringify(got) !== JSON.stringify(want)) {
|
|
173
|
+
wrong.push(`${c.id}: found ${JSON.stringify(got)}, contract says ${JSON.stringify(want)}`);
|
|
174
|
+
}
|
|
175
|
+
}
|
|
176
|
+
add("extractor: every import form is visible", wrong.length === 0, wrong.length ? wrong.join("; ") : `${IMPORT_EXTRACTION_CASES.length} forms found`);
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
// --- 2. Ask the packer what it would actually ship -------------------------
|
|
180
|
+
let shipped = new Set();
|
|
181
|
+
try {
|
|
182
|
+
const out = execFileSync("npm", ["pack", "--dry-run", "--json"], { cwd: root, encoding: "utf-8", stdio: ["ignore", "pipe", "ignore"] });
|
|
183
|
+
shipped = new Set(JSON.parse(out)[0].files.map((f) => f.path.split(/[\\/]/).join("/")));
|
|
184
|
+
} catch (err) {
|
|
185
|
+
add("packer: `npm pack --dry-run` answers", false, err.message);
|
|
186
|
+
}
|
|
187
|
+
add("packer: the tarball is not empty", shipped.size > 0, `${shipped.size} files`);
|
|
188
|
+
|
|
189
|
+
// --- 3. Resolve the import graph inside the tarball ------------------------
|
|
190
|
+
{
|
|
191
|
+
const broken = [];
|
|
192
|
+
let resolved = 0;
|
|
193
|
+
let scanned = 0;
|
|
194
|
+
|
|
195
|
+
for (const file of shipped) {
|
|
196
|
+
if (!/\.(mjs|cjs|js)$/.test(file)) continue;
|
|
197
|
+
let src;
|
|
198
|
+
try {
|
|
199
|
+
src = readFileSync(resolve(root, file), "utf-8");
|
|
200
|
+
} catch {
|
|
201
|
+
continue;
|
|
202
|
+
}
|
|
203
|
+
scanned++;
|
|
204
|
+
for (const spec of extractSpecifiers(src)) {
|
|
205
|
+
if (!spec.startsWith(".")) continue; // bare and node: specifiers are not ours to resolve
|
|
206
|
+
resolved++;
|
|
207
|
+
const target = relative(root, resolve(dirname(resolve(root, file)), spec)).split(sep).join("/");
|
|
208
|
+
if (!shipped.has(target)) {
|
|
209
|
+
broken.push(`${file} imports ${spec} — ${target} is not in the tarball (on disk: ${existsSync(resolve(root, target)) ? "yes" : "no"})`);
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
add(
|
|
215
|
+
"tarball: every relative import resolves",
|
|
216
|
+
broken.length === 0,
|
|
217
|
+
broken.length ? broken.join("; ") : `${resolved} relative imports across ${scanned} shipped modules`
|
|
218
|
+
);
|
|
219
|
+
// A resolver that resolved nothing would report the same clean line.
|
|
220
|
+
add("tarball: the graph was actually walked", resolved > 0 && scanned > 0, `${scanned} modules, ${resolved} relative imports`);
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
// --- 4. Every advertised entry point has to be in the box ------------------
|
|
224
|
+
{
|
|
225
|
+
const pkg = JSON.parse(readFileSync(resolve(root, "package.json"), "utf-8"));
|
|
226
|
+
const entries = [...Object.values(pkg.bin || {}), pkg.main, pkg.module].filter(Boolean);
|
|
227
|
+
const missing = entries.map((e) => e.replace(/^\.\//, "")).filter((e) => !shipped.has(e));
|
|
228
|
+
add("entry points: every bin and main ships", missing.length === 0, missing.length ? missing.join(", ") : `${entries.length} entry points present`);
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
return { ok: failures.length === 0, checks, failures };
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
const isMain = process.argv[1] && process.argv[1].endsWith("package-integrity-check.mjs");
|
|
235
|
+
if (isMain) {
|
|
236
|
+
const { ok, checks: rows, failures: bad } = checkPackageIntegrity();
|
|
237
|
+
if (process.argv.includes("--json")) {
|
|
238
|
+
console.log(JSON.stringify({ ok, checks: rows }, null, 2));
|
|
239
|
+
} else {
|
|
240
|
+
console.log("\n📦 Package Integrity Check (what we actually publish)");
|
|
241
|
+
console.log("-------------------------------------------------------");
|
|
242
|
+
for (const c of rows) console.log(` ${c.ok ? "✅" : "❌"} ${c.name.padEnd(46)} ${c.detail}`);
|
|
243
|
+
console.log("-------------------------------------------------------");
|
|
244
|
+
console.log(
|
|
245
|
+
ok
|
|
246
|
+
? "✅ The published package is self-contained.\n"
|
|
247
|
+
: `\n❌ ${bad.length} problem(s). The tarball is not what the source tree looks like.\n`
|
|
248
|
+
);
|
|
249
|
+
}
|
|
250
|
+
process.exit(ok ? 0 : 1);
|
|
251
|
+
}
|
package/scripts/release.mjs
CHANGED
|
@@ -41,6 +41,39 @@ try {
|
|
|
41
41
|
process.exit(1);
|
|
42
42
|
}
|
|
43
43
|
|
|
44
|
+
// 1a. Activation coverage (blocking).
|
|
45
|
+
//
|
|
46
|
+
// Step 1 proved the suite is green. Green is only evidence if the guards were
|
|
47
|
+
// switched on: a check that silently stopped applying contributes passing
|
|
48
|
+
// tests and a zero exit code exactly like one that ran. This asks the question
|
|
49
|
+
// the suite cannot — can every blocking guard still be made red? — and it runs
|
|
50
|
+
// before the doc-sync gate because a silent guard makes every later signal
|
|
51
|
+
// meaningless.
|
|
52
|
+
console.log("1a. Verifying every blocking guard can still be made red...");
|
|
53
|
+
try {
|
|
54
|
+
execSync("node scripts/guard-reach-check.mjs", { cwd: root, stdio: "inherit" });
|
|
55
|
+
console.log("");
|
|
56
|
+
} catch (_) {
|
|
57
|
+
console.error("\n❌ Release Aborted: a guard has gone silent. See the failing rows above.");
|
|
58
|
+
process.exit(1);
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// Step 1a proved the guards work here. Here is not where users run them.
|
|
62
|
+
//
|
|
63
|
+
// Every signal so far was measured in the source tree, where every file
|
|
64
|
+
// exists by construction. What ships is a tarball built from the `files`
|
|
65
|
+
// list, and v0.63.0 published the guard-reach check without the policy
|
|
66
|
+
// contract it imports — green suite, green matrix, green release, and a
|
|
67
|
+
// module that threw ERR_MODULE_NOT_FOUND the moment anyone installed it.
|
|
68
|
+
console.log("1a-2. Verifying the package we would publish is self-contained...");
|
|
69
|
+
try {
|
|
70
|
+
execSync("node scripts/package-integrity-check.mjs", { cwd: root, stdio: "inherit" });
|
|
71
|
+
console.log("");
|
|
72
|
+
} catch (_) {
|
|
73
|
+
console.error("\n❌ Release Aborted: the tarball is not what the source tree looks like.");
|
|
74
|
+
process.exit(1);
|
|
75
|
+
}
|
|
76
|
+
|
|
44
77
|
// 1b. Documentation / version consistency gate (blocking).
|
|
45
78
|
console.log("1b. Verifying documentation is in sync with package.json & test suite...");
|
|
46
79
|
{
|
package/src/assertions.mjs
CHANGED
|
@@ -561,12 +561,28 @@ export function assertTestIntegrity(config = {}, root = process.cwd()) {
|
|
|
561
561
|
const res = checkTestTampering(diffStr, config);
|
|
562
562
|
const diagnostics = (res.violations || []).map((v) => v.reason);
|
|
563
563
|
|
|
564
|
+
// A clean result from a guard that could not read the dialect is not a
|
|
565
|
+
// clean result, and it must not be reported as one. This does not fail the
|
|
566
|
+
// check — an unlisted assertion library is the user's normal, not their
|
|
567
|
+
// fault — but they get to know the guard is not covering them.
|
|
568
|
+
for (const u of res.unreadable || []) {
|
|
569
|
+
diagnostics.push(
|
|
570
|
+
`Test Tamper Guard: ${u.count} assertion-shaped line(s) in ${u.file} matched no known dialect, ` +
|
|
571
|
+
`so this file was not checked for tampering (e.g. ${JSON.stringify(u.samples[0])}). ` +
|
|
572
|
+
`The guard reports what it could not read rather than passing silently.`
|
|
573
|
+
);
|
|
574
|
+
}
|
|
575
|
+
|
|
564
576
|
return {
|
|
565
577
|
ok: res.ok,
|
|
566
578
|
violations: res.violations || [],
|
|
567
579
|
diagnostics,
|
|
568
580
|
metrics: {
|
|
569
581
|
violationCount: res.violations?.length || 0,
|
|
582
|
+
assertionsSeen: res.assertionsSeen ?? 0,
|
|
583
|
+
filesSeen: res.filesSeen ?? 0,
|
|
584
|
+
unreadableFiles: (res.unreadable || []).length,
|
|
585
|
+
status: res.status,
|
|
570
586
|
},
|
|
571
587
|
};
|
|
572
588
|
}
|
package/src/config.mjs
CHANGED
|
@@ -54,6 +54,9 @@ const CI_DEFINITIONS = [
|
|
|
54
54
|
"appveyor.yml",
|
|
55
55
|
".teamcity/**",
|
|
56
56
|
".githooks/**",
|
|
57
|
+
"buildspec.yml",
|
|
58
|
+
"**/buildspec.yml",
|
|
59
|
+
".buildspec/**",
|
|
57
60
|
];
|
|
58
61
|
|
|
59
62
|
export const BUILTIN_DENY = [
|
|
@@ -64,6 +67,24 @@ export const BUILTIN_DENY = [
|
|
|
64
67
|
"**/*.key",
|
|
65
68
|
"**/id_rsa*",
|
|
66
69
|
".agent/jules-queue/**",
|
|
70
|
+
|
|
71
|
+
// Shell that runs on `cd`, and credentials in plaintext. Same class as a CI
|
|
72
|
+
// definition: code or secrets that take effect before anyone reviews them.
|
|
73
|
+
"**/.envrc",
|
|
74
|
+
"**/.git-credentials",
|
|
75
|
+
"**/.aws/**",
|
|
76
|
+
"**/.ssh/**",
|
|
77
|
+
"**/.kube/**",
|
|
78
|
+
"**/kubeconfig*",
|
|
79
|
+
"**/.docker/config.json",
|
|
80
|
+
"**/*.p12",
|
|
81
|
+
"**/*.pfx",
|
|
82
|
+
"**/*.p8",
|
|
83
|
+
"**/id_ed25519*",
|
|
84
|
+
"**/credentials.json",
|
|
85
|
+
"**/service-account*.json",
|
|
86
|
+
"**/*.tfstate",
|
|
87
|
+
"**/*.tfstate.*",
|
|
67
88
|
...CI_DEFINITIONS,
|
|
68
89
|
];
|
|
69
90
|
|
|
@@ -110,6 +131,53 @@ export const BUILTIN_PROTECT = [
|
|
|
110
131
|
"Dockerfile",
|
|
111
132
|
"**/Dockerfile",
|
|
112
133
|
|
|
134
|
+
// Lockfiles decide which code actually *runs*. `package.json` was protected
|
|
135
|
+
// and `package-lock.json` was not, so an agent could change a resolved URL
|
|
136
|
+
// or an integrity hash — swapping the code that gets installed — without
|
|
137
|
+
// touching a single declared dependency, and the gate said nothing. The
|
|
138
|
+
// entropy scanner is deliberately blind to lockfiles too (they are full of
|
|
139
|
+
// hashes), so the change was invisible twice over. `BUILTIN_RESTRICTED` in
|
|
140
|
+
// risk.mjs already knew these mattered; only the risk tier consumed it,
|
|
141
|
+
// never checkScope.
|
|
142
|
+
"package-lock.json",
|
|
143
|
+
"**/package-lock.json",
|
|
144
|
+
"pnpm-lock.yaml",
|
|
145
|
+
"**/pnpm-lock.yaml",
|
|
146
|
+
"yarn.lock",
|
|
147
|
+
"**/yarn.lock",
|
|
148
|
+
"bun.lockb",
|
|
149
|
+
"bun.lock",
|
|
150
|
+
"Cargo.lock",
|
|
151
|
+
"**/Cargo.lock",
|
|
152
|
+
"go.sum",
|
|
153
|
+
"**/go.sum",
|
|
154
|
+
"poetry.lock",
|
|
155
|
+
"uv.lock",
|
|
156
|
+
"Pipfile.lock",
|
|
157
|
+
"Gemfile.lock",
|
|
158
|
+
"composer.lock",
|
|
159
|
+
"gradle.lockfile",
|
|
160
|
+
"npm-shrinkwrap.json",
|
|
161
|
+
"mix.lock",
|
|
162
|
+
"pubspec.lock",
|
|
163
|
+
"Podfile.lock",
|
|
164
|
+
"Package.resolved",
|
|
165
|
+
".terraform.lock.hcl",
|
|
166
|
+
|
|
167
|
+
// Toolchain pins choose the compiler that runs the whole suite.
|
|
168
|
+
".nvmrc",
|
|
169
|
+
".tool-versions",
|
|
170
|
+
".mise.toml",
|
|
171
|
+
"rust-toolchain",
|
|
172
|
+
"rust-toolchain.toml",
|
|
173
|
+
"**/gradle/wrapper/gradle-wrapper.properties",
|
|
174
|
+
"**/gradle/wrapper/gradle-wrapper.jar",
|
|
175
|
+
|
|
176
|
+
// Who has to approve a change, and what runs on every developer's machine.
|
|
177
|
+
"CODEOWNERS",
|
|
178
|
+
"docs/CODEOWNERS",
|
|
179
|
+
".pre-commit-config.yaml",
|
|
180
|
+
|
|
113
181
|
// Test-runner configuration decides which tests run and what counts as a
|
|
114
182
|
// pass. Rewriting it is the cheapest way to make a suite green without
|
|
115
183
|
// touching a single assertion — `--passWithNoTests`, an added ignore
|