@smoothbricks/nx-plugin 0.5.4 → 0.5.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +31 -5
- package/dist/runtime-input-policy.d.ts +60 -0
- package/dist/runtime-input-policy.d.ts.map +1 -0
- package/dist/runtime-input-policy.js +379 -0
- package/package.json +6 -1
- package/src/runtime-input-policy.ts +395 -0
- package/src/runtime-input.test.ts +160 -3
package/README.md
CHANGED
|
@@ -315,6 +315,10 @@ The reliable diagnostic after a daemon or headless gate's decoder error is:
|
|
|
315
315
|
cat .runtime-input-failures/*/report
|
|
316
316
|
```
|
|
317
317
|
|
|
318
|
+
`smoo github-ci nx-smart`, `nx-run-many` and `nx-deploy` print every report retained during a failed Nx run
|
|
319
|
+
(`runtimeInputFailureReports(root, sinceMs)`, `@smoothbricks/nx-plugin/runtime-input-policy`), so a CI log names the
|
|
320
|
+
command and its cause beside Nx's decoder error, which names neither.
|
|
321
|
+
|
|
318
322
|
Each readable `report` names the command, its indexed arguments, working directory, exit status and original stderr.
|
|
319
323
|
`argv` retains the exact NUL-delimited argument vector (including empty arguments), `stderr` retains the original bytes
|
|
320
324
|
(including trailing newlines), and `exit-status` retains the decimal status. The wrapper also attempts a stderr summary
|
|
@@ -329,6 +333,26 @@ evidence path, timestamp or diagnostic enters a successful key. `runtime-input.t
|
|
|
329
333
|
its hash with the previous success path, and plants a failure through real Nx (daemon and daemonless) to prove both
|
|
330
334
|
refusal and retained evidence.
|
|
331
335
|
|
|
336
|
+
A runtime input runs only programs the workspace declares: devenv's Nix profile, the checkout's `node_modules/.bin` and
|
|
337
|
+
its own scripts. Nx hashes on a developer's Mac, on a NixOS CI runner and inside cowshed's sandbox, and only those are
|
|
338
|
+
the same in all three. A program the host's operating system ships is not: `shasum` is a Perl script that macOS ships in
|
|
339
|
+
`/usr/bin`, not coreutils, and no devenv profile provides it, so an input that named it hashed on every Mac and exited
|
|
340
|
+
127 on a NixOS runner. Hash a file with Bun, as the plugin's own file inputs do (`Bun.CryptoHasher`); devenv's coreutils
|
|
341
|
+
provide `sha256sum` where a pipeline needs one. The only host paths admitted are `/bin/sh` and `/usr/bin/env`, which
|
|
342
|
+
NixOS keeps too. `@smoothbricks/nx-plugin/runtime-input-policy` exports `hostPrograms(command, { path, root, home })`:
|
|
343
|
+
each program the command runs (through the wrapper, `env`, `exec`, `command`, `sh -c` and command substitutions; a
|
|
344
|
+
script file it runs is that script's own concern) that PATH finds outside `/nix`, the checkout and the home, that
|
|
345
|
+
nothing on PATH provides, or that only a link in the checkout's `node_modules/.bin` to a build output of one of its own
|
|
346
|
+
packages provides. Bun links a workspace package's bin only when its target exists at install time, and a fresh checkout
|
|
347
|
+
installs before it builds, so such a name is absent on a fresh runner and present in every checkout that built first;
|
|
348
|
+
run the file instead (`node node_modules/@smoothbricks/nx-plugin/dist/bin/smoo-nx-cargo-hash.js`, as the plugin's own
|
|
349
|
+
Cargo input does). A consumer's installed package ships its build, so its bins are not refused. Each refusal is a tagged
|
|
350
|
+
`HostProgram`, and `hostProgramReason` says it. `runtime-input.test.ts` judges this repository's declared inputs on
|
|
351
|
+
`tooling/direnv/repo-path --print`, the PATH entries the devenv shell and CI give Nx (which hands a runtime input its
|
|
352
|
+
own environment): the devenv profile and the checkout's directories, with no `/usr/bin` or `/bin`, so a program only the
|
|
353
|
+
host provides fails on macOS as it does on the NixOS runner. A consumer judges its own declarations with the same
|
|
354
|
+
export.
|
|
355
|
+
|
|
332
356
|
An input that runs inside a cowshed workspace's sandboxed daemon gets the client's environment, a host shell's `HOME`
|
|
333
357
|
included, but not the client's filesystem view (cowshed `04_sandbox.md`). An input that reads the operator's home fails
|
|
334
358
|
there, so a runtime input reads only the checkout and its declared environment. A `git` command in one sets
|
|
@@ -583,11 +607,13 @@ A test pays its setup once and asserts after each step; two cases that test one
|
|
|
583
607
|
repositories enforce this in their root Nx lint with the `smoo-nx-test-walks` bin. The bin judges only pairs with a case
|
|
584
608
|
changed since the upstream merge base. Among those pairs it reports each case's nearest refusing partner and its nearest
|
|
585
609
|
advisory partner separately, so an advisory pair never hides a refusal. Existing pairs therefore need no allowance
|
|
586
|
-
table.
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
`
|
|
590
|
-
|
|
610
|
+
table. The verdict depends on that base and no file hash names it, so the plugin resolves it as a run precondition
|
|
611
|
+
before Nx hashes any task (`assertUpstreamBasePrecondition`, from its pre-tasks hook): a checkout without a main ref
|
|
612
|
+
refuses the run with the check's own reason rather than carrying the base through a runtime input. `upstreamBase` picks
|
|
613
|
+
the upstream in this order: `NX_BASE` (refused when it names no commit), `refs/remotes/main/main`, the local main's
|
|
614
|
+
upstream branch, `refs/remotes/origin/main`, then `main`. `findDuplicateScenarios(root, files, changes)`,
|
|
615
|
+
`formatDuplicateScenario`, `diffScope` and `upstreamBase` are exported from `@smoothbricks/nx-plugin/test-walk-policy`,
|
|
616
|
+
and git failures come back as `GitAnswer` values.
|
|
591
617
|
|
|
592
618
|
The policy reads bun:test `test`/`it` cases in test files and `#[test]`-family fns in any Rust file. A body that is one
|
|
593
619
|
wrapper call, `await withHarness(memoryBackend, async (h) => { … })` or `runtime().block_on(async { … })`, walks the
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The programs `script` runs by name or path: the program of each simple command, through `env`, `exec`,
|
|
3
|
+
* `command`, `sh -c` and command substitutions, and the command a `runtime-input.sh` input runs. Shell builtins are
|
|
4
|
+
* not programs. A script file a command runs (`sh tooling/x.sh`) is its own source's concern, not read here.
|
|
5
|
+
*/
|
|
6
|
+
export declare function commandPrograms(script: string): string[];
|
|
7
|
+
/** Where a runtime input's programs are looked up, and the locations that count as declared rather than the host's. */
|
|
8
|
+
export interface ProgramSearch {
|
|
9
|
+
/** The PATH the input runs with: Nx passes its own environment through. */
|
|
10
|
+
readonly path: string;
|
|
11
|
+
/** The workspace root: its `node_modules/.bin` and tooling are declared by the checkout. */
|
|
12
|
+
readonly root: string;
|
|
13
|
+
/**
|
|
14
|
+
* The user's home. A developer's devenv shell keeps its PATH entries in the checkout direnv loaded (a main
|
|
15
|
+
* checkout's `node_modules/.bin` while a shed is judged) and the user's own tools there; neither is the OS image.
|
|
16
|
+
*/
|
|
17
|
+
readonly home: string;
|
|
18
|
+
}
|
|
19
|
+
/** A program a runtime input runs that the workspace does not declare, and why it is refused. */
|
|
20
|
+
export type HostProgram =
|
|
21
|
+
/** PATH finds it outside /nix, the checkout, the home and /bin/sh or /usr/bin/env. */
|
|
22
|
+
{
|
|
23
|
+
readonly program: string;
|
|
24
|
+
readonly refusal: 'host';
|
|
25
|
+
readonly found: string;
|
|
26
|
+
}
|
|
27
|
+
/** Nothing on PATH provides it. */
|
|
28
|
+
| {
|
|
29
|
+
readonly program: string;
|
|
30
|
+
readonly refusal: 'absent';
|
|
31
|
+
}
|
|
32
|
+
/**
|
|
33
|
+
* PATH finds it in the checkout's `node_modules/.bin` as a link to a build output of a package in this checkout. Bun
|
|
34
|
+
* links a workspace package's bin only when its target exists at install time, and a fresh checkout installs before
|
|
35
|
+
* it builds, so the name is absent there while every checkout that built first has it.
|
|
36
|
+
*/
|
|
37
|
+
| {
|
|
38
|
+
readonly program: string;
|
|
39
|
+
readonly refusal: 'workspace-bin';
|
|
40
|
+
readonly found: string;
|
|
41
|
+
readonly target: string;
|
|
42
|
+
};
|
|
43
|
+
/**
|
|
44
|
+
* Every program `command` runs that the workspace does not declare: one PATH finds outside the Nix store, the
|
|
45
|
+
* checkout, the home and the guaranteed host paths, one nothing on PATH provides, or one only a workspace package's
|
|
46
|
+
* build output provides. A relative path names a file in the checkout.
|
|
47
|
+
*/
|
|
48
|
+
export declare function hostPrograms(command: string, search: ProgramSearch): HostProgram[];
|
|
49
|
+
/** Why `host` is refused, naming where PATH found it and what to run instead. */
|
|
50
|
+
export declare function hostProgramReason(host: HostProgram): string;
|
|
51
|
+
/** The directory under the workspace root where runtime-input.sh retains each failed input. */
|
|
52
|
+
export declare const RUNTIME_INPUT_FAILURES = ".runtime-input-failures";
|
|
53
|
+
/**
|
|
54
|
+
* The report of every failure runtime-input.sh retained under `root` at or after `sinceMs`, oldest first, each headed
|
|
55
|
+
* by its path. Nx 23.2.1 names a failed runtime input only as `invalid utf-8 sequence of 1 bytes from index N` and
|
|
56
|
+
* drops the input's stderr, so a headless run prints these after Nx fails: they name the command, its exit status and
|
|
57
|
+
* its stderr. A record without a report (its evidence could not be written) is named as such.
|
|
58
|
+
*/
|
|
59
|
+
export declare function runtimeInputFailureReports(root: string, sinceMs: number): string[];
|
|
60
|
+
//# sourceMappingURL=runtime-input-policy.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"runtime-input-policy.d.ts","sourceRoot":"","sources":["../src/runtime-input-policy.ts"],"names":[],"mappings":"AAsOA;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,EAAE,CAiBxD;AAED,uHAAuH;AACvH,MAAM,WAAW,aAAa;IAC5B,2EAA2E;IAC3E,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,4FAA4F;IAC5F,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB;;;OAGG;IACH,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;CACvB;AAED,iGAAiG;AACjG,MAAM,MAAM,WAAW;AACrB,sFAAsF;AACpF;IAAE,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;CAAE;AAChF,mCAAmC;GACjC;IAAE,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,QAAQ,CAAA;CAAE;AAC1D;;;;GAIG;GACD;IAAE,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,eAAe,CAAC;IAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;CAAE,CAAC;AA+CrH;;;;GAIG;AACH,wBAAgB,YAAY,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,aAAa,GAAG,WAAW,EAAE,CAiBlF;AAED,iFAAiF;AACjF,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,WAAW,GAAG,MAAM,CAY3D;AAED,+FAA+F;AAC/F,eAAO,MAAM,sBAAsB,4BAA4B,CAAC;AAWhE;;;;;GAKG;AACH,wBAAgB,0BAA0B,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAYlF"}
|
|
@@ -0,0 +1,379 @@
|
|
|
1
|
+
import { accessSync, constants, readdirSync, readFileSync, realpathSync, statSync } from 'node:fs';
|
|
2
|
+
import { delimiter, isAbsolute, join, relative, resolve, sep } from 'node:path';
|
|
3
|
+
import { RUNTIME_INPUT_SCRIPT } from './runtime-input.js';
|
|
4
|
+
/**
|
|
5
|
+
* What a runtime input may run, and how its failures are read back after Nx refuses it.
|
|
6
|
+
*
|
|
7
|
+
* A runtime input runs wherever Nx hashes: a developer's Mac, a NixOS CI runner, a cowshed sandbox. Only the programs a
|
|
8
|
+
* workspace declares are the same in all of them: devenv's Nix profile and the checkout's own `node_modules`. A
|
|
9
|
+
* program the host's operating system ships is not. macOS ships `shasum` in /usr/bin; it is a Perl script, not
|
|
10
|
+
* coreutils, and no devenv profile provides it (devenv's carries coreutils, `sha256sum` included). An input that named
|
|
11
|
+
* it hashed on every Mac and exited 127 on the NixOS runner, which Nx reported only as
|
|
12
|
+
* `invalid utf-8 sequence of 1 bytes from index 0` (runtime-input.sh says why).
|
|
13
|
+
*/
|
|
14
|
+
/** Words a POSIX `sh` runs itself, without a PATH lookup. */
|
|
15
|
+
const SHELL_BUILTINS = new Set([
|
|
16
|
+
'.',
|
|
17
|
+
':',
|
|
18
|
+
'[',
|
|
19
|
+
'alias',
|
|
20
|
+
'break',
|
|
21
|
+
'cd',
|
|
22
|
+
'continue',
|
|
23
|
+
'echo',
|
|
24
|
+
'eval',
|
|
25
|
+
'exit',
|
|
26
|
+
'export',
|
|
27
|
+
'false',
|
|
28
|
+
'getopts',
|
|
29
|
+
'hash',
|
|
30
|
+
'local',
|
|
31
|
+
'printf',
|
|
32
|
+
'pwd',
|
|
33
|
+
'read',
|
|
34
|
+
'readonly',
|
|
35
|
+
'return',
|
|
36
|
+
'set',
|
|
37
|
+
'shift',
|
|
38
|
+
'test',
|
|
39
|
+
'times',
|
|
40
|
+
'trap',
|
|
41
|
+
'true',
|
|
42
|
+
'type',
|
|
43
|
+
'ulimit',
|
|
44
|
+
'umask',
|
|
45
|
+
'unalias',
|
|
46
|
+
'unset',
|
|
47
|
+
'wait',
|
|
48
|
+
]);
|
|
49
|
+
/** Reserved words that may stand before a command's program. */
|
|
50
|
+
const SHELL_KEYWORDS = new Set([
|
|
51
|
+
'!',
|
|
52
|
+
'{',
|
|
53
|
+
'}',
|
|
54
|
+
'if',
|
|
55
|
+
'then',
|
|
56
|
+
'else',
|
|
57
|
+
'elif',
|
|
58
|
+
'fi',
|
|
59
|
+
'do',
|
|
60
|
+
'done',
|
|
61
|
+
'while',
|
|
62
|
+
'until',
|
|
63
|
+
]);
|
|
64
|
+
/** Reserved words whose clause names no program (`for name in words`, `case word in`). */
|
|
65
|
+
const CLAUSE_KEYWORDS = new Set(['for', 'case', 'esac', 'in']);
|
|
66
|
+
/** Shells whose `-c` operand is a script of further commands. */
|
|
67
|
+
const SHELLS = new Set(['sh', 'bash']);
|
|
68
|
+
const ASSIGNMENT = /^[A-Za-z_][A-Za-z0-9_]*=/;
|
|
69
|
+
/** The index of the character that closes the quote opened before `from`, or the end of `text` when none does. */
|
|
70
|
+
function closingQuote(text, from, quote) {
|
|
71
|
+
for (let at = from; at < text.length; at += 1) {
|
|
72
|
+
if (text[at] === '\\' && quote !== "'")
|
|
73
|
+
at += 1;
|
|
74
|
+
else if (text[at] === quote)
|
|
75
|
+
return at;
|
|
76
|
+
}
|
|
77
|
+
return text.length;
|
|
78
|
+
}
|
|
79
|
+
/** The index of the `close` that balances the `open` before `from`, skipping quoted text, or the end of `text`. */
|
|
80
|
+
function closingBracket(text, from, open, close) {
|
|
81
|
+
let depth = 1;
|
|
82
|
+
for (let at = from; at < text.length; at += 1) {
|
|
83
|
+
const character = text[at];
|
|
84
|
+
if (character === '\\')
|
|
85
|
+
at += 1;
|
|
86
|
+
else if (character === "'" || character === '"' || character === '`')
|
|
87
|
+
at = closingQuote(text, at + 1, character);
|
|
88
|
+
else if (character === open)
|
|
89
|
+
depth += 1;
|
|
90
|
+
else if (character === close) {
|
|
91
|
+
depth -= 1;
|
|
92
|
+
if (depth === 0)
|
|
93
|
+
return at;
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
return text.length;
|
|
97
|
+
}
|
|
98
|
+
/**
|
|
99
|
+
* The shell tokens of `script`. A command substitution is part of its word, and the programs it runs are appended to
|
|
100
|
+
* `nested`; quoting is removed from the words, which are only inspected, never run.
|
|
101
|
+
*/
|
|
102
|
+
function shellTokens(script, nested) {
|
|
103
|
+
const tokens = [];
|
|
104
|
+
let word;
|
|
105
|
+
const flush = () => {
|
|
106
|
+
if (word !== undefined)
|
|
107
|
+
tokens.push({ kind: 'word', text: word });
|
|
108
|
+
word = undefined;
|
|
109
|
+
};
|
|
110
|
+
const append = (text) => {
|
|
111
|
+
word = (word ?? '') + text;
|
|
112
|
+
};
|
|
113
|
+
/** Consumes the expansion starting with `$` or a backquote at `at`; returns the index after it. */
|
|
114
|
+
const expansion = (at) => {
|
|
115
|
+
if (script[at] === '`') {
|
|
116
|
+
const end = closingQuote(script, at + 1, '`');
|
|
117
|
+
nested.push(...commandPrograms(script.slice(at + 1, end)));
|
|
118
|
+
append('`…`');
|
|
119
|
+
return end + 1;
|
|
120
|
+
}
|
|
121
|
+
if (script.startsWith('$((', at)) {
|
|
122
|
+
const end = closingBracket(script, at + 2, '(', ')');
|
|
123
|
+
append(script.slice(at, end + 1));
|
|
124
|
+
return end + 1;
|
|
125
|
+
}
|
|
126
|
+
if (script.startsWith('$(', at)) {
|
|
127
|
+
const end = closingBracket(script, at + 2, '(', ')');
|
|
128
|
+
nested.push(...commandPrograms(script.slice(at + 2, end)));
|
|
129
|
+
append('$(…)');
|
|
130
|
+
return end + 1;
|
|
131
|
+
}
|
|
132
|
+
if (script.startsWith('${', at)) {
|
|
133
|
+
const end = closingBracket(script, at + 2, '{', '}');
|
|
134
|
+
append(script.slice(at, end + 1));
|
|
135
|
+
return end + 1;
|
|
136
|
+
}
|
|
137
|
+
append('$');
|
|
138
|
+
return at + 1;
|
|
139
|
+
};
|
|
140
|
+
let at = 0;
|
|
141
|
+
while (at < script.length) {
|
|
142
|
+
const character = script[at];
|
|
143
|
+
if (character === "'") {
|
|
144
|
+
const end = closingQuote(script, at + 1, "'");
|
|
145
|
+
append(script.slice(at + 1, end));
|
|
146
|
+
at = end + 1;
|
|
147
|
+
}
|
|
148
|
+
else if (character === '"') {
|
|
149
|
+
append('');
|
|
150
|
+
at += 1;
|
|
151
|
+
while (at < script.length && script[at] !== '"') {
|
|
152
|
+
if (script[at] === '\\' && at + 1 < script.length) {
|
|
153
|
+
append('$`"\\\n'.includes(script[at + 1]) ? script[at + 1] : script.slice(at, at + 2));
|
|
154
|
+
at += 2;
|
|
155
|
+
}
|
|
156
|
+
else if (script[at] === '$' || script[at] === '`') {
|
|
157
|
+
at = expansion(at);
|
|
158
|
+
}
|
|
159
|
+
else {
|
|
160
|
+
append(script[at]);
|
|
161
|
+
at += 1;
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
at += 1;
|
|
165
|
+
}
|
|
166
|
+
else if (character === '\\') {
|
|
167
|
+
append(script.slice(at + 1, at + 2));
|
|
168
|
+
at += 2;
|
|
169
|
+
}
|
|
170
|
+
else if (character === '$' || character === '`') {
|
|
171
|
+
at = expansion(at);
|
|
172
|
+
}
|
|
173
|
+
else if (character === '#' && word === undefined) {
|
|
174
|
+
at = script.indexOf('\n', at);
|
|
175
|
+
if (at < 0)
|
|
176
|
+
at = script.length;
|
|
177
|
+
}
|
|
178
|
+
else if (character === ' ' || character === '\t') {
|
|
179
|
+
flush();
|
|
180
|
+
at += 1;
|
|
181
|
+
}
|
|
182
|
+
else if (';&|()\n'.includes(character)) {
|
|
183
|
+
flush();
|
|
184
|
+
tokens.push({ kind: 'separator' });
|
|
185
|
+
at += 1;
|
|
186
|
+
}
|
|
187
|
+
else if (character === '<' || character === '>') {
|
|
188
|
+
// A file-descriptor number written against the operator (`2>&1`) is part of it, not a word.
|
|
189
|
+
if (word !== undefined && /^\d+$/.test(word))
|
|
190
|
+
word = undefined;
|
|
191
|
+
flush();
|
|
192
|
+
at += 1;
|
|
193
|
+
while (at < script.length && '<>&|'.includes(script[at]))
|
|
194
|
+
at += 1;
|
|
195
|
+
tokens.push({ kind: 'redirect' });
|
|
196
|
+
}
|
|
197
|
+
else {
|
|
198
|
+
append(character);
|
|
199
|
+
at += 1;
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
flush();
|
|
203
|
+
return tokens;
|
|
204
|
+
}
|
|
205
|
+
/** The programs one simple command runs, given its words in order. */
|
|
206
|
+
function simpleCommandPrograms(words) {
|
|
207
|
+
let at = 0;
|
|
208
|
+
while (at < words.length && (SHELL_KEYWORDS.has(words[at]) || ASSIGNMENT.test(words[at])))
|
|
209
|
+
at += 1;
|
|
210
|
+
const program = words[at];
|
|
211
|
+
if (program === undefined || CLAUSE_KEYWORDS.has(program))
|
|
212
|
+
return [];
|
|
213
|
+
const operands = words.slice(at + 1);
|
|
214
|
+
if (program === 'exec')
|
|
215
|
+
return simpleCommandPrograms(operands);
|
|
216
|
+
// `command -v name` looks a name up; `command name` runs it.
|
|
217
|
+
if (program === 'command')
|
|
218
|
+
return operands[0]?.startsWith('-') ? [] : simpleCommandPrograms(operands);
|
|
219
|
+
if (program === 'env') {
|
|
220
|
+
let operand = 0;
|
|
221
|
+
while (operand < operands.length && operands[operand].startsWith('-') && operands[operand] !== '--') {
|
|
222
|
+
operand += operands[operand] === '-u' || operands[operand] === '-C' ? 2 : 1;
|
|
223
|
+
}
|
|
224
|
+
if (operands[operand] === '--')
|
|
225
|
+
operand += 1;
|
|
226
|
+
return [program, ...simpleCommandPrograms(operands.slice(operand))];
|
|
227
|
+
}
|
|
228
|
+
if (SHELLS.has(program)) {
|
|
229
|
+
// The wrapper runs its operands as one command.
|
|
230
|
+
if (operands[0] === RUNTIME_INPUT_SCRIPT)
|
|
231
|
+
return [program, ...simpleCommandPrograms(operands.slice(1))];
|
|
232
|
+
const options = operands.findIndex((operand) => !operand.startsWith('-'));
|
|
233
|
+
const flags = operands.slice(0, options < 0 ? operands.length : options);
|
|
234
|
+
const script = options < 0 ? undefined : operands[options];
|
|
235
|
+
if (script !== undefined && flags.some((flag) => flag !== '--' && !flag.startsWith('--') && flag.includes('c'))) {
|
|
236
|
+
return [program, ...commandPrograms(script)];
|
|
237
|
+
}
|
|
238
|
+
}
|
|
239
|
+
return [program];
|
|
240
|
+
}
|
|
241
|
+
/**
|
|
242
|
+
* The programs `script` runs by name or path: the program of each simple command, through `env`, `exec`,
|
|
243
|
+
* `command`, `sh -c` and command substitutions, and the command a `runtime-input.sh` input runs. Shell builtins are
|
|
244
|
+
* not programs. A script file a command runs (`sh tooling/x.sh`) is its own source's concern, not read here.
|
|
245
|
+
*/
|
|
246
|
+
export function commandPrograms(script) {
|
|
247
|
+
const programs = [];
|
|
248
|
+
const nested = [];
|
|
249
|
+
let words = [];
|
|
250
|
+
let redirectTarget = false;
|
|
251
|
+
const end = () => {
|
|
252
|
+
programs.push(...simpleCommandPrograms(words));
|
|
253
|
+
words = [];
|
|
254
|
+
};
|
|
255
|
+
for (const token of shellTokens(script, nested)) {
|
|
256
|
+
if (token.kind === 'separator')
|
|
257
|
+
end();
|
|
258
|
+
else if (token.kind === 'redirect')
|
|
259
|
+
redirectTarget = true;
|
|
260
|
+
else if (redirectTarget)
|
|
261
|
+
redirectTarget = false;
|
|
262
|
+
else
|
|
263
|
+
words.push(token.text);
|
|
264
|
+
}
|
|
265
|
+
end();
|
|
266
|
+
return [...programs, ...nested].filter((program) => !SHELL_BUILTINS.has(program));
|
|
267
|
+
}
|
|
268
|
+
function within(directory, file) {
|
|
269
|
+
const path = relative(directory, file);
|
|
270
|
+
return path !== '' && !path.startsWith('..') && !isAbsolute(path);
|
|
271
|
+
}
|
|
272
|
+
function isExecutableFile(file) {
|
|
273
|
+
try {
|
|
274
|
+
accessSync(file, constants.X_OK);
|
|
275
|
+
return statSync(file).isFile();
|
|
276
|
+
}
|
|
277
|
+
catch (error) {
|
|
278
|
+
if (error instanceof Error && 'code' in error && ['ENOENT', 'ENOTDIR', 'EACCES'].includes(String(error.code))) {
|
|
279
|
+
return false;
|
|
280
|
+
}
|
|
281
|
+
throw error;
|
|
282
|
+
}
|
|
283
|
+
}
|
|
284
|
+
function onPath(program, path) {
|
|
285
|
+
for (const directory of path.split(delimiter)) {
|
|
286
|
+
// An empty entry means the working directory; Nx runs inputs from the root, which a name never resolves into.
|
|
287
|
+
if (directory === '')
|
|
288
|
+
continue;
|
|
289
|
+
const candidate = join(directory, program);
|
|
290
|
+
if (isExecutableFile(candidate))
|
|
291
|
+
return candidate;
|
|
292
|
+
}
|
|
293
|
+
return undefined;
|
|
294
|
+
}
|
|
295
|
+
/**
|
|
296
|
+
* The only host paths every platform guarantees: NixOS keeps exactly these two outside the store, for `#!` lines and
|
|
297
|
+
* for Nx, which runs every runtime input as `sh -c`.
|
|
298
|
+
*/
|
|
299
|
+
const GUARANTEED_HOST_PROGRAMS = new Set(['/bin/sh', '/usr/bin/env']);
|
|
300
|
+
/**
|
|
301
|
+
* The build output of a package in this checkout that `found`, a link in the checkout's `node_modules/.bin`, names;
|
|
302
|
+
* undefined for any other file, an installed package's bin included.
|
|
303
|
+
*/
|
|
304
|
+
function workspaceBuildOutput(found, root) {
|
|
305
|
+
if (!within(join(root, 'node_modules', '.bin'), found))
|
|
306
|
+
return undefined;
|
|
307
|
+
const physicalRoot = realpathSync(root);
|
|
308
|
+
const target = realpathSync(found);
|
|
309
|
+
return within(physicalRoot, target) && !relative(physicalRoot, target).split(sep).includes('node_modules')
|
|
310
|
+
? target
|
|
311
|
+
: undefined;
|
|
312
|
+
}
|
|
313
|
+
/**
|
|
314
|
+
* Every program `command` runs that the workspace does not declare: one PATH finds outside the Nix store, the
|
|
315
|
+
* checkout, the home and the guaranteed host paths, one nothing on PATH provides, or one only a workspace package's
|
|
316
|
+
* build output provides. A relative path names a file in the checkout.
|
|
317
|
+
*/
|
|
318
|
+
export function hostPrograms(command, search) {
|
|
319
|
+
const declared = (file) => file.startsWith('/nix/') ||
|
|
320
|
+
GUARANTEED_HOST_PROGRAMS.has(file) ||
|
|
321
|
+
within(search.root, file) ||
|
|
322
|
+
within(search.home, file);
|
|
323
|
+
return [...new Set(commandPrograms(command))].flatMap((program) => {
|
|
324
|
+
if (program.includes('/')) {
|
|
325
|
+
const found = resolve(search.root, program);
|
|
326
|
+
return declared(found) ? [] : [{ program, refusal: 'host', found }];
|
|
327
|
+
}
|
|
328
|
+
const found = onPath(program, search.path);
|
|
329
|
+
if (found === undefined)
|
|
330
|
+
return [{ program, refusal: 'absent' }];
|
|
331
|
+
if (!declared(found))
|
|
332
|
+
return [{ program, refusal: 'host', found }];
|
|
333
|
+
const target = workspaceBuildOutput(found, search.root);
|
|
334
|
+
return target === undefined ? [] : [{ program, refusal: 'workspace-bin', found, target }];
|
|
335
|
+
});
|
|
336
|
+
}
|
|
337
|
+
/** Why `host` is refused, naming where PATH found it and what to run instead. */
|
|
338
|
+
export function hostProgramReason(host) {
|
|
339
|
+
switch (host.refusal) {
|
|
340
|
+
case 'host':
|
|
341
|
+
return `PATH finds ${host.program} at ${host.found}, a host program no devenv profile declares`;
|
|
342
|
+
case 'absent':
|
|
343
|
+
return `nothing on PATH provides ${host.program}`;
|
|
344
|
+
case 'workspace-bin':
|
|
345
|
+
return (`PATH finds ${host.program} at ${host.found}, a link to this checkout's build output ${host.target}, ` +
|
|
346
|
+
'which an install that runs before the build does not create; run that file with node or bun instead');
|
|
347
|
+
}
|
|
348
|
+
}
|
|
349
|
+
/** The directory under the workspace root where runtime-input.sh retains each failed input. */
|
|
350
|
+
export const RUNTIME_INPUT_FAILURES = '.runtime-input-failures';
|
|
351
|
+
function absentAsUndefined(read) {
|
|
352
|
+
try {
|
|
353
|
+
return read();
|
|
354
|
+
}
|
|
355
|
+
catch (error) {
|
|
356
|
+
if (error instanceof Error && 'code' in error && error.code === 'ENOENT')
|
|
357
|
+
return undefined;
|
|
358
|
+
throw error;
|
|
359
|
+
}
|
|
360
|
+
}
|
|
361
|
+
/**
|
|
362
|
+
* The report of every failure runtime-input.sh retained under `root` at or after `sinceMs`, oldest first, each headed
|
|
363
|
+
* by its path. Nx 23.2.1 names a failed runtime input only as `invalid utf-8 sequence of 1 bytes from index N` and
|
|
364
|
+
* drops the input's stderr, so a headless run prints these after Nx fails: they name the command, its exit status and
|
|
365
|
+
* its stderr. A record without a report (its evidence could not be written) is named as such.
|
|
366
|
+
*/
|
|
367
|
+
export function runtimeInputFailureReports(root, sinceMs) {
|
|
368
|
+
const directory = join(root, RUNTIME_INPUT_FAILURES);
|
|
369
|
+
return (absentAsUndefined(() => readdirSync(directory)) ?? [])
|
|
370
|
+
.map((name) => join(directory, name))
|
|
371
|
+
.map((record) => ({ record, modifiedMs: statSync(record).mtimeMs }))
|
|
372
|
+
.filter(({ modifiedMs }) => modifiedMs >= sinceMs)
|
|
373
|
+
.sort((left, right) => left.modifiedMs - right.modifiedMs)
|
|
374
|
+
.map(({ record }) => {
|
|
375
|
+
const report = join(record, 'report');
|
|
376
|
+
const text = absentAsUndefined(() => readFileSync(report, 'utf8'));
|
|
377
|
+
return text === undefined ? `${record}: no report; its evidence is incomplete\n` : `${report}:\n${text}`;
|
|
378
|
+
});
|
|
379
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@smoothbricks/nx-plugin",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.5",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"sideEffects": false,
|
|
6
6
|
"main": "./loader.js",
|
|
@@ -103,6 +103,11 @@
|
|
|
103
103
|
"import": "./dist/cargo-toolchain-policy.js",
|
|
104
104
|
"default": "./dist/cargo-toolchain-policy.js"
|
|
105
105
|
},
|
|
106
|
+
"./runtime-input-policy": {
|
|
107
|
+
"types": "./dist/runtime-input-policy.d.ts",
|
|
108
|
+
"import": "./dist/runtime-input-policy.js",
|
|
109
|
+
"default": "./dist/runtime-input-policy.js"
|
|
110
|
+
},
|
|
106
111
|
"./cargo-dev-profile": {
|
|
107
112
|
"types": "./dist/cargo-dev-profile.d.ts",
|
|
108
113
|
"import": "./dist/cargo-dev-profile.js",
|
|
@@ -0,0 +1,395 @@
|
|
|
1
|
+
import { accessSync, constants, readdirSync, readFileSync, realpathSync, statSync } from 'node:fs';
|
|
2
|
+
import { delimiter, isAbsolute, join, relative, resolve, sep } from 'node:path';
|
|
3
|
+
import { RUNTIME_INPUT_SCRIPT } from './runtime-input.js';
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* What a runtime input may run, and how its failures are read back after Nx refuses it.
|
|
7
|
+
*
|
|
8
|
+
* A runtime input runs wherever Nx hashes: a developer's Mac, a NixOS CI runner, a cowshed sandbox. Only the programs a
|
|
9
|
+
* workspace declares are the same in all of them: devenv's Nix profile and the checkout's own `node_modules`. A
|
|
10
|
+
* program the host's operating system ships is not. macOS ships `shasum` in /usr/bin; it is a Perl script, not
|
|
11
|
+
* coreutils, and no devenv profile provides it (devenv's carries coreutils, `sha256sum` included). An input that named
|
|
12
|
+
* it hashed on every Mac and exited 127 on the NixOS runner, which Nx reported only as
|
|
13
|
+
* `invalid utf-8 sequence of 1 bytes from index 0` (runtime-input.sh says why).
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
/** Words a POSIX `sh` runs itself, without a PATH lookup. */
|
|
17
|
+
const SHELL_BUILTINS: ReadonlySet<string> = new Set([
|
|
18
|
+
'.',
|
|
19
|
+
':',
|
|
20
|
+
'[',
|
|
21
|
+
'alias',
|
|
22
|
+
'break',
|
|
23
|
+
'cd',
|
|
24
|
+
'continue',
|
|
25
|
+
'echo',
|
|
26
|
+
'eval',
|
|
27
|
+
'exit',
|
|
28
|
+
'export',
|
|
29
|
+
'false',
|
|
30
|
+
'getopts',
|
|
31
|
+
'hash',
|
|
32
|
+
'local',
|
|
33
|
+
'printf',
|
|
34
|
+
'pwd',
|
|
35
|
+
'read',
|
|
36
|
+
'readonly',
|
|
37
|
+
'return',
|
|
38
|
+
'set',
|
|
39
|
+
'shift',
|
|
40
|
+
'test',
|
|
41
|
+
'times',
|
|
42
|
+
'trap',
|
|
43
|
+
'true',
|
|
44
|
+
'type',
|
|
45
|
+
'ulimit',
|
|
46
|
+
'umask',
|
|
47
|
+
'unalias',
|
|
48
|
+
'unset',
|
|
49
|
+
'wait',
|
|
50
|
+
]);
|
|
51
|
+
|
|
52
|
+
/** Reserved words that may stand before a command's program. */
|
|
53
|
+
const SHELL_KEYWORDS: ReadonlySet<string> = new Set([
|
|
54
|
+
'!',
|
|
55
|
+
'{',
|
|
56
|
+
'}',
|
|
57
|
+
'if',
|
|
58
|
+
'then',
|
|
59
|
+
'else',
|
|
60
|
+
'elif',
|
|
61
|
+
'fi',
|
|
62
|
+
'do',
|
|
63
|
+
'done',
|
|
64
|
+
'while',
|
|
65
|
+
'until',
|
|
66
|
+
]);
|
|
67
|
+
|
|
68
|
+
/** Reserved words whose clause names no program (`for name in words`, `case word in`). */
|
|
69
|
+
const CLAUSE_KEYWORDS: ReadonlySet<string> = new Set(['for', 'case', 'esac', 'in']);
|
|
70
|
+
|
|
71
|
+
/** Shells whose `-c` operand is a script of further commands. */
|
|
72
|
+
const SHELLS: ReadonlySet<string> = new Set(['sh', 'bash']);
|
|
73
|
+
|
|
74
|
+
const ASSIGNMENT = /^[A-Za-z_][A-Za-z0-9_]*=/;
|
|
75
|
+
|
|
76
|
+
type Token =
|
|
77
|
+
| { readonly kind: 'word'; readonly text: string }
|
|
78
|
+
| { readonly kind: 'separator' }
|
|
79
|
+
| { readonly kind: 'redirect' };
|
|
80
|
+
|
|
81
|
+
/** The index of the character that closes the quote opened before `from`, or the end of `text` when none does. */
|
|
82
|
+
function closingQuote(text: string, from: number, quote: string): number {
|
|
83
|
+
for (let at = from; at < text.length; at += 1) {
|
|
84
|
+
if (text[at] === '\\' && quote !== "'") at += 1;
|
|
85
|
+
else if (text[at] === quote) return at;
|
|
86
|
+
}
|
|
87
|
+
return text.length;
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/** The index of the `close` that balances the `open` before `from`, skipping quoted text, or the end of `text`. */
|
|
91
|
+
function closingBracket(text: string, from: number, open: string, close: string): number {
|
|
92
|
+
let depth = 1;
|
|
93
|
+
for (let at = from; at < text.length; at += 1) {
|
|
94
|
+
const character = text[at];
|
|
95
|
+
if (character === '\\') at += 1;
|
|
96
|
+
else if (character === "'" || character === '"' || character === '`') at = closingQuote(text, at + 1, character);
|
|
97
|
+
else if (character === open) depth += 1;
|
|
98
|
+
else if (character === close) {
|
|
99
|
+
depth -= 1;
|
|
100
|
+
if (depth === 0) return at;
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
return text.length;
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
/**
|
|
107
|
+
* The shell tokens of `script`. A command substitution is part of its word, and the programs it runs are appended to
|
|
108
|
+
* `nested`; quoting is removed from the words, which are only inspected, never run.
|
|
109
|
+
*/
|
|
110
|
+
function shellTokens(script: string, nested: string[]): Token[] {
|
|
111
|
+
const tokens: Token[] = [];
|
|
112
|
+
let word: string | undefined;
|
|
113
|
+
const flush = (): void => {
|
|
114
|
+
if (word !== undefined) tokens.push({ kind: 'word', text: word });
|
|
115
|
+
word = undefined;
|
|
116
|
+
};
|
|
117
|
+
const append = (text: string): void => {
|
|
118
|
+
word = (word ?? '') + text;
|
|
119
|
+
};
|
|
120
|
+
/** Consumes the expansion starting with `$` or a backquote at `at`; returns the index after it. */
|
|
121
|
+
const expansion = (at: number): number => {
|
|
122
|
+
if (script[at] === '`') {
|
|
123
|
+
const end = closingQuote(script, at + 1, '`');
|
|
124
|
+
nested.push(...commandPrograms(script.slice(at + 1, end)));
|
|
125
|
+
append('`…`');
|
|
126
|
+
return end + 1;
|
|
127
|
+
}
|
|
128
|
+
if (script.startsWith('$((', at)) {
|
|
129
|
+
const end = closingBracket(script, at + 2, '(', ')');
|
|
130
|
+
append(script.slice(at, end + 1));
|
|
131
|
+
return end + 1;
|
|
132
|
+
}
|
|
133
|
+
if (script.startsWith('$(', at)) {
|
|
134
|
+
const end = closingBracket(script, at + 2, '(', ')');
|
|
135
|
+
nested.push(...commandPrograms(script.slice(at + 2, end)));
|
|
136
|
+
append('$(…)');
|
|
137
|
+
return end + 1;
|
|
138
|
+
}
|
|
139
|
+
if (script.startsWith('${', at)) {
|
|
140
|
+
const end = closingBracket(script, at + 2, '{', '}');
|
|
141
|
+
append(script.slice(at, end + 1));
|
|
142
|
+
return end + 1;
|
|
143
|
+
}
|
|
144
|
+
append('$');
|
|
145
|
+
return at + 1;
|
|
146
|
+
};
|
|
147
|
+
let at = 0;
|
|
148
|
+
while (at < script.length) {
|
|
149
|
+
const character = script[at];
|
|
150
|
+
if (character === "'") {
|
|
151
|
+
const end = closingQuote(script, at + 1, "'");
|
|
152
|
+
append(script.slice(at + 1, end));
|
|
153
|
+
at = end + 1;
|
|
154
|
+
} else if (character === '"') {
|
|
155
|
+
append('');
|
|
156
|
+
at += 1;
|
|
157
|
+
while (at < script.length && script[at] !== '"') {
|
|
158
|
+
if (script[at] === '\\' && at + 1 < script.length) {
|
|
159
|
+
append('$`"\\\n'.includes(script[at + 1]) ? script[at + 1] : script.slice(at, at + 2));
|
|
160
|
+
at += 2;
|
|
161
|
+
} else if (script[at] === '$' || script[at] === '`') {
|
|
162
|
+
at = expansion(at);
|
|
163
|
+
} else {
|
|
164
|
+
append(script[at]);
|
|
165
|
+
at += 1;
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
at += 1;
|
|
169
|
+
} else if (character === '\\') {
|
|
170
|
+
append(script.slice(at + 1, at + 2));
|
|
171
|
+
at += 2;
|
|
172
|
+
} else if (character === '$' || character === '`') {
|
|
173
|
+
at = expansion(at);
|
|
174
|
+
} else if (character === '#' && word === undefined) {
|
|
175
|
+
at = script.indexOf('\n', at);
|
|
176
|
+
if (at < 0) at = script.length;
|
|
177
|
+
} else if (character === ' ' || character === '\t') {
|
|
178
|
+
flush();
|
|
179
|
+
at += 1;
|
|
180
|
+
} else if (';&|()\n'.includes(character)) {
|
|
181
|
+
flush();
|
|
182
|
+
tokens.push({ kind: 'separator' });
|
|
183
|
+
at += 1;
|
|
184
|
+
} else if (character === '<' || character === '>') {
|
|
185
|
+
// A file-descriptor number written against the operator (`2>&1`) is part of it, not a word.
|
|
186
|
+
if (word !== undefined && /^\d+$/.test(word)) word = undefined;
|
|
187
|
+
flush();
|
|
188
|
+
at += 1;
|
|
189
|
+
while (at < script.length && '<>&|'.includes(script[at])) at += 1;
|
|
190
|
+
tokens.push({ kind: 'redirect' });
|
|
191
|
+
} else {
|
|
192
|
+
append(character);
|
|
193
|
+
at += 1;
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
flush();
|
|
197
|
+
return tokens;
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
/** The programs one simple command runs, given its words in order. */
|
|
201
|
+
function simpleCommandPrograms(words: readonly string[]): string[] {
|
|
202
|
+
let at = 0;
|
|
203
|
+
while (at < words.length && (SHELL_KEYWORDS.has(words[at]) || ASSIGNMENT.test(words[at]))) at += 1;
|
|
204
|
+
const program = words[at];
|
|
205
|
+
if (program === undefined || CLAUSE_KEYWORDS.has(program)) return [];
|
|
206
|
+
const operands = words.slice(at + 1);
|
|
207
|
+
if (program === 'exec') return simpleCommandPrograms(operands);
|
|
208
|
+
// `command -v name` looks a name up; `command name` runs it.
|
|
209
|
+
if (program === 'command') return operands[0]?.startsWith('-') ? [] : simpleCommandPrograms(operands);
|
|
210
|
+
if (program === 'env') {
|
|
211
|
+
let operand = 0;
|
|
212
|
+
while (operand < operands.length && operands[operand].startsWith('-') && operands[operand] !== '--') {
|
|
213
|
+
operand += operands[operand] === '-u' || operands[operand] === '-C' ? 2 : 1;
|
|
214
|
+
}
|
|
215
|
+
if (operands[operand] === '--') operand += 1;
|
|
216
|
+
return [program, ...simpleCommandPrograms(operands.slice(operand))];
|
|
217
|
+
}
|
|
218
|
+
if (SHELLS.has(program)) {
|
|
219
|
+
// The wrapper runs its operands as one command.
|
|
220
|
+
if (operands[0] === RUNTIME_INPUT_SCRIPT) return [program, ...simpleCommandPrograms(operands.slice(1))];
|
|
221
|
+
const options = operands.findIndex((operand) => !operand.startsWith('-'));
|
|
222
|
+
const flags = operands.slice(0, options < 0 ? operands.length : options);
|
|
223
|
+
const script = options < 0 ? undefined : operands[options];
|
|
224
|
+
if (script !== undefined && flags.some((flag) => flag !== '--' && !flag.startsWith('--') && flag.includes('c'))) {
|
|
225
|
+
return [program, ...commandPrograms(script)];
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
return [program];
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
/**
|
|
232
|
+
* The programs `script` runs by name or path: the program of each simple command, through `env`, `exec`,
|
|
233
|
+
* `command`, `sh -c` and command substitutions, and the command a `runtime-input.sh` input runs. Shell builtins are
|
|
234
|
+
* not programs. A script file a command runs (`sh tooling/x.sh`) is its own source's concern, not read here.
|
|
235
|
+
*/
|
|
236
|
+
export function commandPrograms(script: string): string[] {
|
|
237
|
+
const programs: string[] = [];
|
|
238
|
+
const nested: string[] = [];
|
|
239
|
+
let words: string[] = [];
|
|
240
|
+
let redirectTarget = false;
|
|
241
|
+
const end = (): void => {
|
|
242
|
+
programs.push(...simpleCommandPrograms(words));
|
|
243
|
+
words = [];
|
|
244
|
+
};
|
|
245
|
+
for (const token of shellTokens(script, nested)) {
|
|
246
|
+
if (token.kind === 'separator') end();
|
|
247
|
+
else if (token.kind === 'redirect') redirectTarget = true;
|
|
248
|
+
else if (redirectTarget) redirectTarget = false;
|
|
249
|
+
else words.push(token.text);
|
|
250
|
+
}
|
|
251
|
+
end();
|
|
252
|
+
return [...programs, ...nested].filter((program) => !SHELL_BUILTINS.has(program));
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
/** Where a runtime input's programs are looked up, and the locations that count as declared rather than the host's. */
|
|
256
|
+
export interface ProgramSearch {
|
|
257
|
+
/** The PATH the input runs with: Nx passes its own environment through. */
|
|
258
|
+
readonly path: string;
|
|
259
|
+
/** The workspace root: its `node_modules/.bin` and tooling are declared by the checkout. */
|
|
260
|
+
readonly root: string;
|
|
261
|
+
/**
|
|
262
|
+
* The user's home. A developer's devenv shell keeps its PATH entries in the checkout direnv loaded (a main
|
|
263
|
+
* checkout's `node_modules/.bin` while a shed is judged) and the user's own tools there; neither is the OS image.
|
|
264
|
+
*/
|
|
265
|
+
readonly home: string;
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
/** A program a runtime input runs that the workspace does not declare, and why it is refused. */
|
|
269
|
+
export type HostProgram =
|
|
270
|
+
/** PATH finds it outside /nix, the checkout, the home and /bin/sh or /usr/bin/env. */
|
|
271
|
+
| { readonly program: string; readonly refusal: 'host'; readonly found: string }
|
|
272
|
+
/** Nothing on PATH provides it. */
|
|
273
|
+
| { readonly program: string; readonly refusal: 'absent' }
|
|
274
|
+
/**
|
|
275
|
+
* PATH finds it in the checkout's `node_modules/.bin` as a link to a build output of a package in this checkout. Bun
|
|
276
|
+
* links a workspace package's bin only when its target exists at install time, and a fresh checkout installs before
|
|
277
|
+
* it builds, so the name is absent there while every checkout that built first has it.
|
|
278
|
+
*/
|
|
279
|
+
| { readonly program: string; readonly refusal: 'workspace-bin'; readonly found: string; readonly target: string };
|
|
280
|
+
|
|
281
|
+
function within(directory: string, file: string): boolean {
|
|
282
|
+
const path = relative(directory, file);
|
|
283
|
+
return path !== '' && !path.startsWith('..') && !isAbsolute(path);
|
|
284
|
+
}
|
|
285
|
+
|
|
286
|
+
function isExecutableFile(file: string): boolean {
|
|
287
|
+
try {
|
|
288
|
+
accessSync(file, constants.X_OK);
|
|
289
|
+
return statSync(file).isFile();
|
|
290
|
+
} catch (error) {
|
|
291
|
+
if (error instanceof Error && 'code' in error && ['ENOENT', 'ENOTDIR', 'EACCES'].includes(String(error.code))) {
|
|
292
|
+
return false;
|
|
293
|
+
}
|
|
294
|
+
throw error;
|
|
295
|
+
}
|
|
296
|
+
}
|
|
297
|
+
|
|
298
|
+
function onPath(program: string, path: string): string | undefined {
|
|
299
|
+
for (const directory of path.split(delimiter)) {
|
|
300
|
+
// An empty entry means the working directory; Nx runs inputs from the root, which a name never resolves into.
|
|
301
|
+
if (directory === '') continue;
|
|
302
|
+
const candidate = join(directory, program);
|
|
303
|
+
if (isExecutableFile(candidate)) return candidate;
|
|
304
|
+
}
|
|
305
|
+
return undefined;
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
/**
|
|
309
|
+
* The only host paths every platform guarantees: NixOS keeps exactly these two outside the store, for `#!` lines and
|
|
310
|
+
* for Nx, which runs every runtime input as `sh -c`.
|
|
311
|
+
*/
|
|
312
|
+
const GUARANTEED_HOST_PROGRAMS: ReadonlySet<string> = new Set(['/bin/sh', '/usr/bin/env']);
|
|
313
|
+
/**
|
|
314
|
+
* The build output of a package in this checkout that `found`, a link in the checkout's `node_modules/.bin`, names;
|
|
315
|
+
* undefined for any other file, an installed package's bin included.
|
|
316
|
+
*/
|
|
317
|
+
function workspaceBuildOutput(found: string, root: string): string | undefined {
|
|
318
|
+
if (!within(join(root, 'node_modules', '.bin'), found)) return undefined;
|
|
319
|
+
const physicalRoot = realpathSync(root);
|
|
320
|
+
const target = realpathSync(found);
|
|
321
|
+
return within(physicalRoot, target) && !relative(physicalRoot, target).split(sep).includes('node_modules')
|
|
322
|
+
? target
|
|
323
|
+
: undefined;
|
|
324
|
+
}
|
|
325
|
+
|
|
326
|
+
/**
|
|
327
|
+
* Every program `command` runs that the workspace does not declare: one PATH finds outside the Nix store, the
|
|
328
|
+
* checkout, the home and the guaranteed host paths, one nothing on PATH provides, or one only a workspace package's
|
|
329
|
+
* build output provides. A relative path names a file in the checkout.
|
|
330
|
+
*/
|
|
331
|
+
export function hostPrograms(command: string, search: ProgramSearch): HostProgram[] {
|
|
332
|
+
const declared = (file: string): boolean =>
|
|
333
|
+
file.startsWith('/nix/') ||
|
|
334
|
+
GUARANTEED_HOST_PROGRAMS.has(file) ||
|
|
335
|
+
within(search.root, file) ||
|
|
336
|
+
within(search.home, file);
|
|
337
|
+
return [...new Set(commandPrograms(command))].flatMap((program): HostProgram[] => {
|
|
338
|
+
if (program.includes('/')) {
|
|
339
|
+
const found = resolve(search.root, program);
|
|
340
|
+
return declared(found) ? [] : [{ program, refusal: 'host', found }];
|
|
341
|
+
}
|
|
342
|
+
const found = onPath(program, search.path);
|
|
343
|
+
if (found === undefined) return [{ program, refusal: 'absent' }];
|
|
344
|
+
if (!declared(found)) return [{ program, refusal: 'host', found }];
|
|
345
|
+
const target = workspaceBuildOutput(found, search.root);
|
|
346
|
+
return target === undefined ? [] : [{ program, refusal: 'workspace-bin', found, target }];
|
|
347
|
+
});
|
|
348
|
+
}
|
|
349
|
+
|
|
350
|
+
/** Why `host` is refused, naming where PATH found it and what to run instead. */
|
|
351
|
+
export function hostProgramReason(host: HostProgram): string {
|
|
352
|
+
switch (host.refusal) {
|
|
353
|
+
case 'host':
|
|
354
|
+
return `PATH finds ${host.program} at ${host.found}, a host program no devenv profile declares`;
|
|
355
|
+
case 'absent':
|
|
356
|
+
return `nothing on PATH provides ${host.program}`;
|
|
357
|
+
case 'workspace-bin':
|
|
358
|
+
return (
|
|
359
|
+
`PATH finds ${host.program} at ${host.found}, a link to this checkout's build output ${host.target}, ` +
|
|
360
|
+
'which an install that runs before the build does not create; run that file with node or bun instead'
|
|
361
|
+
);
|
|
362
|
+
}
|
|
363
|
+
}
|
|
364
|
+
|
|
365
|
+
/** The directory under the workspace root where runtime-input.sh retains each failed input. */
|
|
366
|
+
export const RUNTIME_INPUT_FAILURES = '.runtime-input-failures';
|
|
367
|
+
|
|
368
|
+
function absentAsUndefined<T>(read: () => T): T | undefined {
|
|
369
|
+
try {
|
|
370
|
+
return read();
|
|
371
|
+
} catch (error) {
|
|
372
|
+
if (error instanceof Error && 'code' in error && error.code === 'ENOENT') return undefined;
|
|
373
|
+
throw error;
|
|
374
|
+
}
|
|
375
|
+
}
|
|
376
|
+
|
|
377
|
+
/**
|
|
378
|
+
* The report of every failure runtime-input.sh retained under `root` at or after `sinceMs`, oldest first, each headed
|
|
379
|
+
* by its path. Nx 23.2.1 names a failed runtime input only as `invalid utf-8 sequence of 1 bytes from index N` and
|
|
380
|
+
* drops the input's stderr, so a headless run prints these after Nx fails: they name the command, its exit status and
|
|
381
|
+
* its stderr. A record without a report (its evidence could not be written) is named as such.
|
|
382
|
+
*/
|
|
383
|
+
export function runtimeInputFailureReports(root: string, sinceMs: number): string[] {
|
|
384
|
+
const directory = join(root, RUNTIME_INPUT_FAILURES);
|
|
385
|
+
return (absentAsUndefined(() => readdirSync(directory)) ?? [])
|
|
386
|
+
.map((name) => join(directory, name))
|
|
387
|
+
.map((record) => ({ record, modifiedMs: statSync(record).mtimeMs }))
|
|
388
|
+
.filter(({ modifiedMs }) => modifiedMs >= sinceMs)
|
|
389
|
+
.sort((left, right) => left.modifiedMs - right.modifiedMs)
|
|
390
|
+
.map(({ record }) => {
|
|
391
|
+
const report = join(record, 'report');
|
|
392
|
+
const text = absentAsUndefined(() => readFileSync(report, 'utf8'));
|
|
393
|
+
return text === undefined ? `${record}: no report; its evidence is incomplete\n` : `${report}:\n${text}`;
|
|
394
|
+
});
|
|
395
|
+
}
|
|
@@ -2,12 +2,31 @@ import { describe, expect, it } from 'bun:test';
|
|
|
2
2
|
import { spawnSync } from 'node:child_process';
|
|
3
3
|
import { createHash } from 'node:crypto';
|
|
4
4
|
import { existsSync } from 'node:fs';
|
|
5
|
-
import {
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
import {
|
|
6
|
+
chmod,
|
|
7
|
+
mkdir,
|
|
8
|
+
mkdtemp,
|
|
9
|
+
readdir,
|
|
10
|
+
readFile,
|
|
11
|
+
realpath,
|
|
12
|
+
rm,
|
|
13
|
+
stat,
|
|
14
|
+
symlink,
|
|
15
|
+
utimes,
|
|
16
|
+
writeFile,
|
|
17
|
+
} from 'node:fs/promises';
|
|
18
|
+
import { homedir, tmpdir } from 'node:os';
|
|
19
|
+
import { delimiter, dirname, join } from 'node:path';
|
|
8
20
|
import { fileURLToPath } from 'node:url';
|
|
9
21
|
import { fixtureNxEnv, withNxFixture } from './__tests__/fixture-nx-env.js';
|
|
10
22
|
import { RUNTIME_INPUT_SCRIPT, runtimeInput, shellWord } from './runtime-input.js';
|
|
23
|
+
import {
|
|
24
|
+
commandPrograms,
|
|
25
|
+
hostProgramReason,
|
|
26
|
+
hostPrograms,
|
|
27
|
+
RUNTIME_INPUT_FAILURES,
|
|
28
|
+
runtimeInputFailureReports,
|
|
29
|
+
} from './runtime-input-policy.js';
|
|
11
30
|
|
|
12
31
|
const packageRoot = dirname(dirname(fileURLToPath(import.meta.url)));
|
|
13
32
|
const repoRoot = dirname(dirname(packageRoot));
|
|
@@ -130,11 +149,18 @@ describe('a planted runtime input failure through real Nx', () => {
|
|
|
130
149
|
expect(good.status, good.output).toBe(0);
|
|
131
150
|
expect(existsSync(join(workspace, 'good.ran'))).toBe(true);
|
|
132
151
|
|
|
152
|
+
const started = Date.now();
|
|
133
153
|
const planted = nx(workspace, env, 'probe:planted');
|
|
134
154
|
expect(planted.status, planted.output).toBe(1);
|
|
135
155
|
expect(planted.output).toContain('invalid utf-8 sequence');
|
|
156
|
+
// Nx's message names neither the command nor its cause; the retained report names both.
|
|
157
|
+
expect(planted.output).not.toContain('planted-cause');
|
|
136
158
|
expect(existsSync(join(workspace, 'planted.ran'))).toBe(false);
|
|
137
159
|
const record = await failureRecord(workspace);
|
|
160
|
+
const reports = runtimeInputFailureReports(workspace, started);
|
|
161
|
+
expect(reports).toHaveLength(1);
|
|
162
|
+
expect(reports[0]).toContain(`${join(record, 'report')}:\nruntime input exited 3: sh -c`);
|
|
163
|
+
expect(reports[0]).toContain('planted-cause');
|
|
138
164
|
expect(await readFile(join(record, 'argv'))).toEqual(Buffer.from('sh\0-c\0echo planted-cause >&2; exit 3\0'));
|
|
139
165
|
expect(await readFile(join(record, 'stderr'), 'utf8')).toBe('planted-cause\n');
|
|
140
166
|
expect(await readFile(join(record, 'exit-status'), 'utf8')).toBe('3\n');
|
|
@@ -200,6 +226,137 @@ describe('runtime inputs of this repository', () => {
|
|
|
200
226
|
}
|
|
201
227
|
expect(offenders).toEqual([]);
|
|
202
228
|
});
|
|
229
|
+
|
|
230
|
+
it('name only programs the workspace declares, on the PATH it declares', async () => {
|
|
231
|
+
// Nx hands a runtime input its own process environment unchanged (nx task-env.js getTaskSpecificEnv), and this
|
|
232
|
+
// repository composes that PATH with tooling/direnv/repo-path, in its devenv shell and on CI (`--github-path`): the
|
|
233
|
+
// devenv profile and the checkout's own directories, no /usr/bin or /bin. Judged on that PATH alone, an input that
|
|
234
|
+
// needs a program the host's directories supply fails here on every OS, as it fails on the NixOS runner.
|
|
235
|
+
const composed = spawnSync('bash', [join(repoRoot, 'tooling', 'direnv', 'repo-path'), '--print'], {
|
|
236
|
+
cwd: repoRoot,
|
|
237
|
+
encoding: 'utf8',
|
|
238
|
+
});
|
|
239
|
+
expect(composed.status, composed.stderr).toBe(0);
|
|
240
|
+
const path = composed.stdout.trim().split('\n').join(delimiter);
|
|
241
|
+
const venv = process.env.VIRTUAL_ENV === undefined ? undefined : join(process.env.VIRTUAL_ENV, 'bin');
|
|
242
|
+
expect(
|
|
243
|
+
path
|
|
244
|
+
.split(delimiter)
|
|
245
|
+
.filter((entry) => !entry.startsWith('/nix/') && !entry.startsWith(`${repoRoot}/`) && entry !== venv),
|
|
246
|
+
).toEqual([]);
|
|
247
|
+
const search = { path, root: repoRoot, home: homedir() };
|
|
248
|
+
const offenders: string[] = [];
|
|
249
|
+
const declared = (file: string, value: unknown): void => {
|
|
250
|
+
if (Array.isArray(value)) {
|
|
251
|
+
for (const item of value) declared(file, item);
|
|
252
|
+
} else if (value !== null && typeof value === 'object') {
|
|
253
|
+
for (const [key, item] of Object.entries(value)) {
|
|
254
|
+
if (key !== 'runtime' || typeof item !== 'string') declared(file, item);
|
|
255
|
+
else for (const host of hostPrograms(item, search)) offenders.push(`${file}: ${hostProgramReason(host)}`);
|
|
256
|
+
}
|
|
257
|
+
}
|
|
258
|
+
};
|
|
259
|
+
const tracked = spawnSync(
|
|
260
|
+
'git',
|
|
261
|
+
['ls-files', '-z', 'nx.json', 'package.json', '**/package.json', '**/project.json'],
|
|
262
|
+
{
|
|
263
|
+
cwd: repoRoot,
|
|
264
|
+
encoding: 'utf8',
|
|
265
|
+
},
|
|
266
|
+
);
|
|
267
|
+
expect(tracked.status, tracked.stderr).toBe(0);
|
|
268
|
+
for (const file of tracked.stdout.split('\0').filter(Boolean)) {
|
|
269
|
+
declared(file, JSON.parse(await readFile(join(repoRoot, file), 'utf8')));
|
|
270
|
+
}
|
|
271
|
+
expect(offenders).toEqual([]);
|
|
272
|
+
});
|
|
273
|
+
});
|
|
274
|
+
|
|
275
|
+
describe('runtime input programs', () => {
|
|
276
|
+
it("are each simple command's program, through the wrapper, env, sh -c and substitutions, without builtins", () => {
|
|
277
|
+
expect(commandPrograms(`sh ${RUNTIME_INPUT_SCRIPT} shasum -a 256 packages/x/y.wasm`)).toEqual(['sh', 'shasum']);
|
|
278
|
+
expect(
|
|
279
|
+
commandPrograms(
|
|
280
|
+
`sh ${RUNTIME_INPUT_SCRIPT} sh -c 'if [ -n "\${SRC:-}" ]; then cd "$SRC" && export A=/dev/null && git rev-parse HEAD && changes=$(git status --porcelain) && printf "%s\\n" "$changes" | shasum -a 256; else echo installed; fi'`,
|
|
281
|
+
),
|
|
282
|
+
).toEqual(['sh', 'sh', 'git', 'shasum', 'git']);
|
|
283
|
+
expect(
|
|
284
|
+
commandPrograms(
|
|
285
|
+
'env GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git -c core.excludesFile=/dev/null ls-files -z',
|
|
286
|
+
),
|
|
287
|
+
).toEqual(['env', 'git']);
|
|
288
|
+
expect(commandPrograms("bun -e 'console.log(1)' >/dev/null 2>&1 </dev/null")).toEqual(['bun']);
|
|
289
|
+
expect(commandPrograms('x=`cut -d: -f1 list` && command -v tr; exec date')).toEqual(['date', 'cut']);
|
|
290
|
+
expect(commandPrograms('for f in a b; do stat "$f"; done # head')).toEqual(['stat']);
|
|
291
|
+
expect(commandPrograms("sh tooling/identity.sh rustc 'sh -c awk'")).toEqual(['sh']);
|
|
292
|
+
});
|
|
293
|
+
|
|
294
|
+
it('that the host supplies, nothing supplies, or only a workspace build supplies, are refused by name', async () => {
|
|
295
|
+
const scratch = await mkdtemp(join(tmpdir(), 'runtime-input-programs-'));
|
|
296
|
+
try {
|
|
297
|
+
const host = join(scratch, 'usr-bin');
|
|
298
|
+
const root = join(scratch, 'checkout');
|
|
299
|
+
const bin = join(root, 'node_modules', '.bin');
|
|
300
|
+
await mkdir(host, { recursive: true });
|
|
301
|
+
await mkdir(bin, { recursive: true });
|
|
302
|
+
await mkdir(join(root, 'packages', 'tool', 'dist'), { recursive: true });
|
|
303
|
+
await mkdir(join(root, 'node_modules', '.bun', 'ext', 'bin'), { recursive: true });
|
|
304
|
+
for (const file of [
|
|
305
|
+
join(host, 'shasum'),
|
|
306
|
+
join(bin, 'tool'),
|
|
307
|
+
join(root, 'packages', 'tool', 'dist', 'built.js'),
|
|
308
|
+
join(root, 'node_modules', '.bun', 'ext', 'bin', 'ext.js'),
|
|
309
|
+
]) {
|
|
310
|
+
await writeFile(file, '#!/bin/sh\n');
|
|
311
|
+
await chmod(file, 0o755);
|
|
312
|
+
}
|
|
313
|
+
// A workspace package's bin links into its build output; an installed package's into node_modules.
|
|
314
|
+
await symlink('../../packages/tool/dist/built.js', join(bin, 'built'));
|
|
315
|
+
await symlink('../.bun/ext/bin/ext.js', join(bin, 'ext'));
|
|
316
|
+
// A file that is not executable provides nothing.
|
|
317
|
+
await writeFile(join(host, 'absent'), '');
|
|
318
|
+
const search = { path: `${host}:${bin}`, root, home: join(scratch, 'home') };
|
|
319
|
+
const command = 'shasum -a 256 f && tool && ext && tooling/x.sh && /nix/store/0-a/bin/a && absent; built';
|
|
320
|
+
const refused = hostPrograms(`${command}; /usr/bin/env true; /usr/bin/shasum`, search);
|
|
321
|
+
const built = await realpath(join(root, 'packages', 'tool', 'dist', 'built.js'));
|
|
322
|
+
expect(refused).toEqual([
|
|
323
|
+
{ program: 'shasum', refusal: 'host', found: join(host, 'shasum') },
|
|
324
|
+
{ program: 'absent', refusal: 'absent' },
|
|
325
|
+
{ program: 'built', refusal: 'workspace-bin', found: join(bin, 'built'), target: built },
|
|
326
|
+
{ program: '/usr/bin/shasum', refusal: 'host', found: '/usr/bin/shasum' },
|
|
327
|
+
]);
|
|
328
|
+
expect(refused.map(hostProgramReason)).toEqual([
|
|
329
|
+
`PATH finds shasum at ${join(host, 'shasum')}, a host program no devenv profile declares`,
|
|
330
|
+
'nothing on PATH provides absent',
|
|
331
|
+
`PATH finds built at ${join(bin, 'built')}, a link to this checkout's build output ${built}, which an ` +
|
|
332
|
+
'install that runs before the build does not create; run that file with node or bun instead',
|
|
333
|
+
'PATH finds /usr/bin/shasum at /usr/bin/shasum, a host program no devenv profile declares',
|
|
334
|
+
]);
|
|
335
|
+
} finally {
|
|
336
|
+
await rm(scratch, { recursive: true, force: true });
|
|
337
|
+
}
|
|
338
|
+
});
|
|
339
|
+
});
|
|
340
|
+
|
|
341
|
+
describe('retained runtime input failure reports', () => {
|
|
342
|
+
it('are read back from a time on, oldest first, naming a record whose report is missing', async () => {
|
|
343
|
+
await withWrapperWorkspace(async (workspace) => {
|
|
344
|
+
const failures = join(workspace, RUNTIME_INPUT_FAILURES);
|
|
345
|
+
const record = async (name: string, at: number, report?: string): Promise<void> => {
|
|
346
|
+
await mkdir(join(failures, name), { recursive: true });
|
|
347
|
+
if (report !== undefined) await writeFile(join(failures, name, 'report'), report);
|
|
348
|
+
await utimes(join(failures, name), at / 1000, at / 1000);
|
|
349
|
+
};
|
|
350
|
+
expect(runtimeInputFailureReports(workspace, 0)).toEqual([]);
|
|
351
|
+
await record('failure.old', 1_000_000, 'old\n');
|
|
352
|
+
await record('failure.second', 3_000_000, 'second\n');
|
|
353
|
+
await record('failure.first', 2_000_000);
|
|
354
|
+
expect(runtimeInputFailureReports(workspace, 2_000_000)).toEqual([
|
|
355
|
+
`${join(failures, 'failure.first')}: no report; its evidence is incomplete\n`,
|
|
356
|
+
`${join(failures, 'failure.second', 'report')}:\nsecond\n`,
|
|
357
|
+
]);
|
|
358
|
+
});
|
|
359
|
+
});
|
|
203
360
|
});
|
|
204
361
|
|
|
205
362
|
async function writeProbeWorkspace(workspace: string): Promise<void> {
|