@redue/cli 0.1.0-alpha.26 → 0.1.0-alpha.27

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,45 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.0-alpha.27 — 2026-11-01
4
+
5
+ A build or test run no longer makes a sandboxed typecheck or lint STALE,
6
+ and `redue stats` shows what REDUE saved you.
7
+
8
+ - **Files git ignores are hidden from sandboxed runs.** A sandboxed
9
+ typecheck or lint now runs without the files git ignores (`dist/`,
10
+ `.next/`, coverage files, `.env`), except `node_modules`, `.venv` and
11
+ runner caches. Those files are not its inputs, so a build or test run that
12
+ writes them leaves the result CURRENT. The claim: the result holds for a
13
+ run that sees only the project's files that git doesn't ignore, its
14
+ installed packages and the toolchain. Only the project's `.gitignore`
15
+ files and `.git/info/exclude` count, not your global excludes file.
16
+ - **Checks that need ignored files keep working.** `init` tries the
17
+ sandbox with ignored files hidden first (`sandbox-v2`), then with the
18
+ whole checkout (`sandbox-v1`), then the general recipe. A hidden run that
19
+ fails on a hidden file runs again in the sandbox without hiding, and that
20
+ result is kept. Checks set up by alpha.26 stay `sandbox-v1` until you run
21
+ `redue init` again.
22
+ - **Measured.** In the product census, after each project's own build and
23
+ tests ran, checks that stayed CURRENT (alpha.26 in brackets, from the
24
+ measurement census): TypeScript on Linux 31 of 37 (5 of 43), on macOS 24
25
+ of 34 (5 of 38); Python on Linux 31 of 38 (24 of 37), on macOS 28 of 34
26
+ (20 of 33). Where a check qualifies with both, the sandbox stayed CURRENT
27
+ 16 times and the general recipe 9 of 23 (Linux). The strace cross-check
28
+ found no read outside the recorded inputs.
29
+ - **`redue stats`.** Per check: reruns the hook skipped, the time that
30
+ saved, recorded runs, and the top reasons a check was UNVERIFIED, from a
31
+ log in REDUE's local state.
32
+ - `redue remove-state` no longer fails on Windows while the observer is
33
+ still closing its files.
34
+ - Known limits: a sandboxed run can still see that a hidden file exists
35
+ (its name on macOS; listings, sizes and times on Linux), so a tool that
36
+ acts on existence alone is not covered; tests do not use hidden files for
37
+ that reason. On Linux, a file a run creates in a folder that holds hidden
38
+ files cannot be read back in the same run.
39
+ - Contributors: eval entry points that run third-party code refuse to start
40
+ unless `GITHUB_ACTIONS=true`, and importing any eval entry point throws
41
+ before it does anything.
42
+
3
43
  ## 0.1.0-alpha.26 — 2026-10-31
4
44
 
5
45
  Typecheck and lint checks run inside an OS sandbox, so REDUE can reuse them
package/README.md CHANGED
@@ -59,25 +59,32 @@ what the agent asks. Plain `redue status` is a fast cached answer and can say
59
59
  UNVERIFIED ("cannot confirm your execution context") where `--sync` would say
60
60
  CURRENT.
61
61
 
62
+ `redue stats` shows, per check, the reruns the hook skipped, the time saved,
63
+ the recorded runs and the top reasons a check was UNVERIFIED, from a log kept
64
+ on your machine.
65
+
62
66
  To undo everything setup did: `redue remove`.
63
67
 
64
68
  ## What it covers today
65
69
 
66
- In a census of 120 active public TypeScript repositories (October 2026, REDUE
67
- at commit 046cd0e, before 0.1.0-alpha.26), each installed as its users would,
68
- with dependencies' install scripts on, 86 installed cleanly on a macOS runner.
70
+ In a census of 120 active public TypeScript repositories (November 2026, REDUE
71
+ at commit e264a67, before 0.1.0-alpha.27), each installed as its users would,
72
+ with dependencies' install scripts on, 90 installed cleanly on a macOS runner.
69
73
  A check counts when two recorded runs on the same code both came out CURRENT
70
74
  on the same inputs. 35 repositories had at least one such check:
71
75
 
72
76
  | Check | Repositories | How |
73
77
  |---|---|---|
74
- | Typecheck | 22 | 15 by running the project's script in an OS sandbox ([sandboxed recipe](docs/user/sandbox.md)); 5 by running the local `tsc` directly ([direct recipe](docs/user/direct-typescript.md)); 2 by running the script with every Node process recorded ([general recipe](docs/user/node-script.md)) |
75
- | Lint | 25 | 21 sandboxed recipe, 4 general recipe |
78
+ | Typecheck | 22 | 16 by running the project's script in an OS sandbox ([sandboxed recipe](docs/user/sandbox.md)), 15 of them with the files git ignores hidden; 5 by running the local `tsc` directly ([direct recipe](docs/user/direct-typescript.md)); 1 by running the script with every Node process recorded ([general recipe](docs/user/node-script.md)) |
79
+ | Lint | 24 | 21 sandboxed recipe (19 with ignored files hidden), 3 general recipe |
76
80
  | Tests | 2 | Vitest run directly ([direct test recipe](docs/user/tests-direct.md)) |
77
81
 
78
- In a census of 120 Python repositories, 75 installed on a Linux runner
82
+ In a census of 120 Python repositories, 74 installed on a Linux runner
79
83
  and 23 had a qualifying check: typecheck (mypy, pyright, ty) 15, lint (ruff)
80
- 22, all through the sandboxed recipe. On Linux, every sandboxed run was traced
84
+ 22, all through the sandboxed recipe with ignored files hidden. A check whose
85
+ git-ignored files are hidden stays CURRENT when a build or test run rewrites
86
+ them: after each project's build and tests ran, 31 of 37 such TypeScript
87
+ checks and 31 of 38 Python checks were still CURRENT on Linux. On Linux, every sandboxed run was traced
81
88
  with strace: each file it read in the checkout was a recorded input, a cache
82
89
  or its own output.
83
90
 
package/bin/redue.mjs CHANGED
@@ -7,7 +7,7 @@ import {directCompiler,directQualification,directEnvironment,directInstalledInpu
7
7
  import {runArgv,tscRun} from '../src/typescript-invocation.mjs';
8
8
  import {testEnvironment,testRunnerAt,testsInterpretation,testsQualification} from '../src/test-direct.mjs';
9
9
  import {scriptEnvironment,scriptInterpretation,scriptKinds,scriptQualification} from '../src/node-script.mjs';
10
- import {pythonTool,sandboxEnvironment,sandboxInterpretation,sandboxKinds,sandboxQualification} from '../src/sandbox.mjs';
10
+ import {hiddenListIssue,hiddenQualification,isSandboxed,pythonTool,sandboxEnvironment,sandboxInterpretationOf,sandboxKinds,sandboxQualification} from '../src/sandbox.mjs';
11
11
  import os from 'node:os';
12
12
  import path from 'node:path';
13
13
  import {createHash} from 'node:crypto';
@@ -46,6 +46,7 @@ Commands:
46
46
  detail [CHECK] [--json] Alias for explain
47
47
  run CHECK [--if-due] Execute one configured check and record its outcome; with --if-due,
48
48
  only when its last pass does not apply to the files as they are now
49
+ stats [--json] Per check: skips, time saved, recorded runs, why checks were UNVERIFIED
49
50
  remove-state Stop and remove only this configuration's owned local state
50
51
 
51
52
  No checks run in the background. Direct commands outside redue create no receipts.`;
@@ -95,7 +96,7 @@ if(command==='agent'){
95
96
  catch(e){error(e.message);}
96
97
  await exit(0);
97
98
  }
98
- if(!['init','start','stop','status','detail','explain','run','remove-state'].includes(command))
99
+ if(!['init','start','stop','status','detail','explain','run','stats','remove-state'].includes(command))
99
100
  error(`unknown command ${command||'<none>'}; use --help`);
100
101
  const json=args.includes('--json');
101
102
  // run --if-due: run the check only when its last pass does not apply now.
@@ -150,21 +151,30 @@ if(command==='init'){
150
151
  if(!chosen.length)error('no discovered check matches the selected workspace and kind');
151
152
  discovery.checks=chosen;discovery.config.checks=chosen.map(row=>row.config);
152
153
  }
153
- // A sandboxed check is proposed only when it gives the same exit status
154
- // inside the sandbox as outside it. Otherwise init writes what it would
154
+ // A sandboxed check is proposed only when it gives the same exit status and
155
+ // output inside the sandbox as outside it: first with the checkout's
156
+ // git-ignored files hidden (sandbox-v2), else with the whole checkout
157
+ // (sandbox-v1), reusing the plain run. Otherwise init writes what it would
155
158
  // have without the sandboxed recipe, with the reason.
159
+ const trialIssue=trial=>trial.issue?`the sandbox cannot run here (${trial.issue})`:
160
+ // A failure both ways that shows a refusal cannot be told from a real one.
161
+ trial.same?null:trial.plain!==trial.sandboxed?`exit ${trial.sandboxed} inside the sandbox, ${trial.plain} outside it${trial.line?` (${trial.line})`:''}`:
162
+ trial.line?`it fails both ways (exit ${trial.plain}), with a sign of a refusal inside the sandbox (${trial.line})`:
163
+ `the output differs inside the sandbox: ${JSON.stringify(trial.output.sandboxed)} instead of ${JSON.stringify(trial.output.plain)}`;
156
164
  let trials=0;
157
165
  for(const [at,row] of [...discovery.checks.entries()]){
158
166
  if(row.qualification!==sandboxQualification||row.level!=='ready')continue;
159
167
  if(!json)console.error(`Running ${row.config.name} inside and outside the sandbox to confirm it...`);
160
- const trial=await sandboxTrial(discovery.root,row.config);
161
- if(!trial.issue)trials++;
162
- const issue=trial.issue?`the sandbox cannot run here (${trial.issue})`:
163
- // A failure both ways that shows a refusal cannot be told from a real one.
164
- trial.same?null:trial.plain!==trial.sandboxed?`exit ${trial.sandboxed} inside the sandbox, ${trial.plain} outside it${trial.line?` (${trial.line})`:''}`:
165
- trial.line?`it fails both ways (exit ${trial.plain}), with a sign of a refusal inside the sandbox (${trial.line})`:
166
- `the output differs inside the sandbox: ${JSON.stringify(trial.output.sandboxed)} instead of ${JSON.stringify(trial.output.plain)}`;
167
- if(!issue){row.reason+=`; ran both ways at init, exit ${trial.plain}`;continue;}
168
+ const hidden=await sandboxTrial(discovery.root,row.config,{hide:true});
169
+ if(!hidden.issue)trials++;
170
+ if(!hidden.issue&&hidden.same){
171
+ row.config={...row.config,qualification:hiddenQualification};row.qualification=hiddenQualification;
172
+ row.reason=row.reason.replace('the whole checkout is its input',"the checkout's files git doesn't ignore are its input")+`; ran both ways at init, exit ${hidden.plain}`;continue;}
173
+ // Without git, nothing is hidden: the whole checkout is tried on its own.
174
+ const trial=hidden.plainRun?await sandboxTrial(discovery.root,row.config,{plain:hidden.plainRun}):
175
+ hidden.issue===hiddenListIssue?await sandboxTrial(discovery.root,row.config):hidden;
176
+ const issue=trialIssue(trial);
177
+ if(!issue){row.reason+=`; ran both ways at init, exit ${trial.plain}; it needs files git ignores (${hidden.issue??trialIssue(hidden)})`;continue;}
168
178
  if(row.fallback)discovery.checks[at]={...row.fallback,reason:`Sandboxed recipe unavailable: ${issue}; ${row.fallback.reason}`};
169
179
  else{discovery.checks[at]=null;discovery.skipped.push({kind:row.config.name,reason:`${row.config.name}: sandboxed recipe unavailable: ${issue}`});}
170
180
  }
@@ -310,7 +320,7 @@ const checks=publicConfig.checks.map(check=>{
310
320
  const pnpmTypecheck=check.qualification==='pnpm-tsc-v1';
311
321
  const testsDirect=check.qualification===testsQualification;
312
322
  const generalScript=check.qualification===scriptQualification;
313
- const sandboxed=check.qualification===sandboxQualification;
323
+ const sandboxed=isSandboxed(check.qualification);
314
324
  if(check.script&&check.command&&!pnpmTypecheck&&!npmDirect&&!testsDirect)
315
325
  error(`${check.name}: choose script or command, not both`);
316
326
  const manager=publicConfig.packageManager;
@@ -398,12 +408,18 @@ const checks=publicConfig.checks.map(check=>{
398
408
  ...(npmDirect?{qualificationInterpretation:directInterpretation}:{}),
399
409
  ...(testsDirect?{qualificationInterpretation:testsInterpretation}:{}),
400
410
  ...(generalScript?{qualificationInterpretation:scriptInterpretation}:{}),
401
- ...(sandboxed?{qualificationInterpretation:sandboxInterpretation}:{}),
411
+ ...(sandboxed?{qualificationInterpretation:sandboxInterpretationOf(check.qualification)}:{}),
402
412
  workspace:check.workspace||null,kind:check.kind||null};
403
413
  });
404
414
  const runtime={schema:1,provider:'declared-project@1',root,state,checks};
405
415
  try{assertOwnedStatePlacement(runtime);}catch(e){error(e.message);}
406
416
  const serialized=JSON.stringify(runtime,null,2)+'\n';
417
+ if(command==='stats'){
418
+ const {readStats,renderStats}=await import('../src/stats.mjs');
419
+ const value=readStats(state,publicConfig.checks.map(item=>item.name));
420
+ console.log(json?JSON.stringify(value):renderStats(value));
421
+ await exit(0);
422
+ }
407
423
  const maintenance=command==='stop'||command==='remove-state';
408
424
  if(maintenance&&!fs.existsSync(runtimeFile)){
409
425
  console.log(`No REDUE state exists for ${configFile}.`);
@@ -443,11 +459,14 @@ const internal=command==='remove-state'?'uninstall':
443
459
  // Asked when this part of a command runs, after earlier parts may have
444
460
  // changed files. REDUE_FORCE=1 runs it anyway.
445
461
  if(ifDue&&process.env.REDUE_FORCE!=='1'){
446
- const {verdict}=await import('../src/claude-hook.mjs');
462
+ const [{verdict},{note,skipNote,dueNote}]=await Promise.all([import('../src/claude-hook.mjs'),import('../src/stats.mjs')]);
447
463
  const asked=spawnSync(process.execPath,[fileURLToPath(import.meta.url),'--config',configFile,...(stateOverride?['--state-dir',stateOverride]:[]),
448
464
  'status','--sync','--json'],{encoding:'utf8',stdio:['ignore','pipe','ignore'],env:process.env,timeout:600_000});
449
465
  let status=null;try{status=JSON.parse(asked.stdout);}catch{}
466
+ const due=dueNote(status,check);
467
+ if(due)note(state,due);
450
468
  if(verdict(status,check)){
469
+ note(state,skipNote(status,check,'if-due'));
451
470
  const row=status.checks.find(item=>item.name===check);
452
471
  if(json)console.log(JSON.stringify({check,skipped:true,result:'PASS',run_id:row.invocation.runId}));
453
472
  else console.error(`REDUE: ${check} passed on these exact inputs (run ${row.invocation.runId}); not run again. REDUE_FORCE=1 runs it anyway.`);
@@ -7,10 +7,15 @@ covers programs REDUE cannot record: TypeScript 7's native compiler, esbuild,
7
7
  oxlint and tsgolint, biome, turbo, and Python tools such as mypy, pyright, ty
8
8
  and ruff.
9
9
 
10
- **The claim.** A sandboxed result holds for a run that sees only the project,
11
- its installed packages and the toolchain, the way CI does. It does not say
12
- that a run with your own configuration files, a service on your machine or the
13
- network would give the same result.
10
+ **The claim.** A sandboxed result holds for a run that sees only the project's
11
+ files that git doesn't ignore, its installed packages and the toolchain, the
12
+ way CI does. It does not say that a run with your own configuration files, a
13
+ service on your machine or the network would give the same result.
14
+
15
+ A check that needs files git ignores (generated code, another package's
16
+ built declarations) gets the whole checkout instead: its result holds for a
17
+ run that sees the project, git-ignored files included, its installed packages
18
+ and the toolchain. See [files git ignores](#files-git-ignores).
14
19
 
15
20
  ## Where it applies
16
21
 
@@ -19,9 +24,9 @@ network would give the same result.
19
24
  [direct TypeScript recipe](direct-typescript.md) and the pnpm and Yarn
20
25
  TypeScript recipes first; they stay first where they apply. Then the
21
26
  sandboxed recipe, before the [general recipe](node-script.md), since it
22
- also covers the native programs a script starts. Where init's trial shows a
23
- difference, or the sandbox cannot run, `init` writes the general recipe as
24
- before. Where the sandbox cannot start at run time (inside Claude Code's
27
+ also covers the native programs a script starts. Where init's trials show
28
+ a difference, or the sandbox cannot run, `init` writes the general recipe
29
+ as before. Where the sandbox cannot start at run time (inside Claude Code's
25
30
  macOS sandbox), a script the general recipe takes is recorded as it records
26
31
  it.
27
32
  - **Python projects:** mypy, pyright, ty and ruff installed in the project's
@@ -44,12 +49,14 @@ redue status --sync
44
49
  A Python check in the config:
45
50
 
46
51
  ```json
47
- {"name": "typecheck", "kind": "typecheck", "qualification": "sandbox-v1",
52
+ {"name": "typecheck", "kind": "typecheck", "qualification": "sandbox-v2",
48
53
  "command": [".venv/bin/mypy", "src"], "inputs": ["**", "..."], "installedInputs": [".venv/**", "..."]}
49
54
  ```
50
55
 
51
56
  A Node check names its script, as the general recipe's do:
52
- `{"name": "lint", "script": "lint", "kind": "lint", "qualification": "sandbox-v1", ...}`.
57
+ `{"name": "lint", "script": "lint", "kind": "lint", "qualification": "sandbox-v2", ...}`.
58
+ `sandbox-v2` hides the files git ignores from the run; `sandbox-v1` runs it on
59
+ the whole checkout.
53
60
 
54
61
  With the Claude Code hook, `uv run mypy src`, `poetry run mypy src`,
55
62
  `python -m mypy src`, `.venv/bin/mypy src` and `mypy src` are recognized as
@@ -57,11 +64,32 @@ that check. The last two count only when PATH finds the tool or Python in the
57
64
  project's `.venv/bin` first. When the hook runs the check through REDUE, it
58
65
  keeps `uv run` or `poetry run` in front, so the environment still syncs first.
59
66
 
67
+ ## Files git ignores
68
+
69
+ `init` runs each check three ways at most: plainly, sandboxed with the files
70
+ git ignores hidden, and, when those two differ, sandboxed with the whole
71
+ checkout (reusing the plain run). The first sandboxed way that gives the same
72
+ result is written: `sandbox-v2` or `sandbox-v1`.
73
+
74
+ With `sandbox-v2`, each run hides what `git ls-files --others --ignored
75
+ --exclude-standard --directory` lists, except `node_modules`, `.venv` and
76
+ runner caches: build output (`dist/`, `.next/`), coverage files, `.env` files.
77
+ Only the project's rules count: its `.gitignore` files and
78
+ `.git/info/exclude`, not your global excludes file. Those files are not
79
+ inputs, so a build or test run that writes them leaves the result CURRENT.
80
+
81
+ When a `sandbox-v2` run fails with a refusal that names a hidden file (a
82
+ missing file counts only when it names one, since Linux tools report an
83
+ unreadable file as missing), REDUE runs the check again in the sandbox
84
+ without hiding anything. That result is kept and compared on every file,
85
+ git-ignored ones included. Run `redue init` again to write `sandbox-v1` for a
86
+ check that keeps needing them.
87
+
60
88
  ## What the run sees
61
89
 
62
90
  It may read and execute only beneath:
63
91
 
64
- - the checkout;
92
+ - the checkout, without the files git ignores for `sandbox-v2`;
65
93
  - the toolchain: Node's installation and the package managers' (with
66
94
  Corepack's downloads), or the Python installation the `.venv` was made from;
67
95
  - system directories (`/usr`, `/bin`, `/etc` and the like on Linux; `/usr`,
@@ -100,10 +128,11 @@ It gets a controlled environment, not yours:
100
128
 
101
129
  CURRENT means the last sandboxed run's inputs are unchanged:
102
130
 
103
- - every file in the checkout, git-ignored files and git-ignored links to
104
- files in it included, except `.git`, `node_modules` and `.venv` (observed
105
- as installed files, by identity) and runner caches: the general recipe's,
106
- plus `__pycache__`, `.mypy_cache`, `.ruff_cache` and `.pytest_cache`;
131
+ - every file in the checkout git doesn't ignore (`sandbox-v2`), or every file
132
+ in it, git-ignored files and git-ignored links to files in it included
133
+ (`sandbox-v1`), except `.git` and runner caches (the general recipe's, plus
134
+ `__pycache__`, `.mypy_cache`, `.ruff_cache` and `.pytest_cache`);
135
+ - `node_modules` and `.venv`, observed as installed files, by identity;
107
136
  - the environment the run gets (without display variables such as `TERM`
108
137
  and `COLUMNS`), and the places it may read;
109
138
  - Node's and the package managers' executables, or the interpreter the
@@ -111,7 +140,8 @@ CURRENT means the last sandboxed run's inputs are unchanged:
111
140
  - every directory on PATH, by identity: installing, removing or upgrading a
112
141
  program in one makes the result STALE;
113
142
  - `.git/info/exclude` and `.git/config`, by content: ruff and biome read the
114
- ignore rules there.
143
+ ignore rules there, and they decide what `sandbox-v2` hides. For a project
144
+ in a subdirectory of its repository, also the `.gitignore` files above it.
115
145
 
116
146
  ## Telling a refusal from a real failure
117
147
 
@@ -124,6 +154,9 @@ every failure twice to find out:
124
154
  timings, the temporary and home directories, and lines about caches.
125
155
  Otherwise `init` writes what it would have without this recipe and says why.
126
156
  These are the only runs `init` makes.
157
+ - When a `sandbox-v2` run fails with a refusal that names a hidden file, it
158
+ runs again in the sandbox without hiding (see
159
+ [files git ignores](#files-git-ignores)).
127
160
  - When a sandboxed run fails and its output has a line that looks like a
128
161
  refusal ("Operation not permitted", "Permission denied", "Read-only file
129
162
  system", a name lookup or a connection that failed), REDUE runs the check
@@ -167,3 +200,22 @@ it with `redue remove-state`.
167
200
  nested example project's).
168
201
  - A script that changes files (`--fix`, `--write`) does not get this recipe:
169
202
  `init` would run it twice.
203
+ - With `sandbox-v2`, a run cannot read a hidden file, but it can still see
204
+ that one exists. On macOS it sees the names of hidden files and folders at
205
+ the level where they start (`dist` in the checkout's root). On Linux it can
206
+ also list a hidden folder and see its files' sizes and times: Landlock
207
+ restricts neither beneath a folder the run may list. A tool that acts on
208
+ whether a git-ignored file exists, without reading it, is not covered: a
209
+ new or removed one does not make the result STALE. Typecheckers and
210
+ linters read what they act on; a test that skips itself when a built file
211
+ is missing does not, which is why hidden files are not used for tests yet.
212
+ - On Linux, a file a `sandbox-v2` run creates in a folder that holds hidden
213
+ files (the checkout's root, when anything in it is hidden) cannot be read
214
+ back in the same run. `init`'s trial and the rerun without hiding catch a
215
+ check that needs to.
216
+ - A file whose git status changes without a change to its content (`git add
217
+ -f` of an ignored file, `git rm --cached` of a tracked one) is noticed only
218
+ when its content next changes. A change to `.gitignore` or
219
+ `.git/info/exclude` makes the result STALE.
220
+ - A run that moves a hidden folder into a visible place can read it there.
221
+ That changes the checkout during the run, so the run is not reused.
@@ -20,8 +20,9 @@ build scripts. It prefers familiar script names (for typecheck: `typecheck`,
20
20
  matches are reported as ambiguous rather than guessed. It never creates a new
21
21
  project script. Without `--recipe`, `init` tries the strict recipes below first, then,
22
22
  for a root typecheck or lint script, the [sandboxed recipe](sandbox.md) on
23
- macOS and Linux (the script run inside an OS sandbox, the whole checkout as
24
- its input), then the [general recipe](node-script.md): the script run
23
+ macOS and Linux (the script run inside an OS sandbox, with the files git
24
+ ignores hidden when it gives the same result that way, else on the whole
25
+ checkout), then the [general recipe](node-script.md): the script run
25
26
  through the package manager with every Node process recorded and the whole
26
27
  checkout as its input. Python projects get the sandboxed recipe for mypy,
27
28
  pyright, ty and ruff in their `.venv`. `--recipe typescript-direct`,
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@redue/cli",
3
3
  "private": false,
4
- "version": "0.1.0-alpha.26",
4
+ "version": "0.1.0-alpha.27",
5
5
  "publishConfig": {
6
6
  "access": "public",
7
7
  "tag": "alpha"
@@ -13,6 +13,7 @@ import {processAlive} from './process-liveness.mjs';
13
13
  import {scriptRunChecks} from './typescript-invocation.mjs';
14
14
  import {recipeLabel} from './presentation.mjs';
15
15
  import {findExecutable} from './executable-lookup.mjs';
16
+ import {note,skipNote,dueNote} from './stats.mjs';
16
17
 
17
18
  export const hookCommand='redue agent hook claude';
18
19
  // No shell metacharacters; `@` starts scoped package names.
@@ -25,7 +26,8 @@ const tests=check=>check?.qualification==='npm-tests-direct-v1'&&Array.isArray(c
25
26
  const general=check=>(check?.qualification==='node-script-v1'&&['typecheck','test','lint'].includes(check.kind)||
26
27
  sandboxed(check))&&typeof check.script==='string';
27
28
  // A sandboxed check: a root package script, or a tool from the project's .venv.
28
- const sandboxed=check=>check?.qualification==='sandbox-v1'&&['typecheck','lint'].includes(check.kind);
29
+ // sandbox.mjs's isSandboxed, written out: the hook does not load the sandbox module.
30
+ const sandboxed=check=>['sandbox-v1','sandbox-v2'].includes(check?.qualification)&&['typecheck','lint'].includes(check.kind);
29
31
  const python=check=>sandboxed(check)&&Array.isArray(check.command)&&/^\.venv\/bin\/[A-Za-z0-9_.-]+$/.test(check.command[0]||'')&&
30
32
  check.command.every(word=>typeof word==='string'&&safe.test(word));
31
33
  const eligibleCheck=check=>typecheck(check)||tests(check)||general(check)||python(check);
@@ -505,6 +507,9 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
505
507
  const project=hookProject(root);
506
508
  if(!project)return null;
507
509
  const {record,config,parsed}=project;
510
+ // For `redue stats`: the checks a decision skipped, or the ones that were due.
511
+ const tally=(status,names,skipped)=>{let state;try{state=projectState(project.projectRoot,config);}catch{return;}
512
+ for(const name of names){const entry=skipped?skipNote(status,name,'hook'):dueNote(status,name);if(entry)note(state,entry);}};
508
513
  const entered=enteredDir(command,root,project.projectRoot);
509
514
  if(!entered)return null;
510
515
  let scripts={};
@@ -537,6 +542,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
537
542
  if(!names)return null;
538
543
  const status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,...names)),
539
544
  reason=fanoutVerdict(status,names,match.rest);
545
+ tally(status,names,Boolean(reason));
540
546
  return reason?{hookSpecificOutput:{hookEventName:'PreToolUse',permissionDecision:'deny',permissionDecisionReason:reason}}:null;
541
547
  }
542
548
  // A compiler script: its checks in the order of its runs, or nothing when
@@ -550,6 +556,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
550
556
  else{
551
557
  const status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,...names)),
552
558
  reason=fanoutVerdict(status,names,match.rest);
559
+ tally(status,names,Boolean(reason));
553
560
  if(reason)return {hookSpecificOutput:{hookEventName:'PreToolUse',permissionDecision:'deny',permissionDecisionReason:reason}};
554
561
  const pending=names.filter(name=>!verdict(status,name));
555
562
  const rewritten=status?.schema===1&&status.observation?.healthy===true&&Array.isArray(status.checks)&&
@@ -561,6 +568,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
561
568
  }
562
569
  const name=match.check,status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,name)),
563
570
  reason=verdict(status,name,match.rest);
571
+ tally(status,[name],Boolean(reason));
564
572
  if(reason)return {hookSpecificOutput:{hookEventName:'PreToolUse',
565
573
  permissionDecision:'deny',permissionDecisionReason:reason}};
566
574
  // Rewrite only on a healthy answer for this check: without an observer,
package/src/cli.mjs CHANGED
@@ -8,8 +8,8 @@ import {patternMatcher} from './glob.mjs';
8
8
  import {observedBy} from './installed-observation.mjs';
9
9
  import {changedFiles,coverageReadable,gitIgnored,recordedEnvironment,testEnvironmentIssue,testGuardConfig,testRunGuard,testRunFacts,sharedInstalledFile,testsInterpretation,testsQualification} from './test-direct.mjs';
10
10
  import {childProgram,generalIssue,lookup,pnpmConfigFiles,scriptCaches,scriptInstalledPaths,scriptInterpretation,scriptQualification,scriptRunFacts,pnpmNameless,toolOutput,toolchainDirs} from './node-script.mjs';
11
- import {denialLine,endWithChild,runToEnd,sandboxArgv,sandboxContext,sandboxDirs,sandboxEnv,sandboxInterpretation,sandboxPlaces,
12
- sandboxQualification,sandboxSupport,systemPlaces} from './sandbox.mjs';
11
+ import {denialLine,endWithChild,hiddenEntries,hiddenQualification,hiddenRefusal,isSandboxed,runToEnd,sandboxArgv,sandboxContext,sandboxDirs,sandboxEnv,
12
+ sandboxInterpretation,sandboxInterpretationOf,sandboxPlaces,sandboxQualification,sandboxSupport,systemPlaces} from './sandbox.mjs';
13
13
  import path from 'node:path';
14
14
  import net from 'node:net';
15
15
  import os from 'node:os';
@@ -29,6 +29,7 @@ import {atomicJson,commitReceipt,readReceipts,receiptRevision} from './state-sto
29
29
  import {timing,profiling,mark,traceContext} from './decision-profile.mjs';
30
30
  import {readCachedStatus,unavailableCached} from './cached-state.mjs';
31
31
  import {renderRun} from './presentation.mjs';
32
+ import {note} from './stats.mjs';
32
33
 
33
34
  mark('cli.loaded');
34
35
  const here=path.dirname(fileURLToPath(import.meta.url));
@@ -86,7 +87,7 @@ function environmentReads() {
86
87
  }
87
88
  const recordedRecipes=[testsQualification,scriptQualification];
88
89
  // A sandboxed check's run is recorded where its sandbox cannot start (execute).
89
- function contextHashes() {const reads=config.checks.some(check=>[...recordedRecipes,sandboxQualification].includes(check.qualification))?environmentReads():{};
90
+ function contextHashes() {const reads=config.checks.some(check=>recordedRecipes.includes(check.qualification)||isSandboxed(check.qualification))?environmentReads():{};
90
91
  return Object.fromEntries(config.checks.map(check=>[
91
92
  check.name,Object.fromEntries([...(check.environment?.variables||[])
92
93
  .map(key=>[key,digest(process.env[key]??'<unset>')]),
@@ -99,7 +100,7 @@ function contextHashes() {const reads=config.checks.some(check=>[...recordedReci
99
100
  `__vstate_path_executable_${name}__`,pathExecutableIdentity(name)]),
100
101
  ...(check.environment?.executableIdentity?[['__vstate_node_identity__',nodeIdentity()]]:[]),
101
102
  // A sandboxed run's environment, toolchain and places (sandbox.mjs).
102
- ...(check.qualification===sandboxQualification?Object.entries(sandboxContext(check,process.env,root)):[]),
103
+ ...(isSandboxed(check.qualification)?Object.entries(sandboxContext(check,process.env,root)):[]),
103
104
  // The variables the last test run read directly, compared here: which
104
105
  // list was compared, and the first variable whose value now differs.
105
106
  // Control requests are small, so the values stay out of them.
@@ -299,7 +300,7 @@ async function execute() {
299
300
  // A sandboxed check runs inside the OS sandbox. Where the sandbox cannot
300
301
  // start, a script the general recipe takes is recorded as that recipe
301
302
  // records it (`fallback`); any other runs as it is, and is not reused.
302
- const sandboxed=selected.qualification===sandboxQualification,support=sandboxed?sandboxSupport():null;
303
+ const sandboxed=isSandboxed(selected.qualification),support=sandboxed?sandboxSupport():null;
303
304
  const fallback=Boolean(support?.issue&&typeof selected.script==='string'&&!generalIssue(root,selected.script));
304
305
  // A test run's processes record what the guard checks (test-recorder.cjs),
305
306
  // and so do a general recipe run's, through the package manager.
@@ -331,7 +332,11 @@ async function execute() {
331
332
  else{
332
333
  const dirs=sandboxDirs(state,name);
333
334
  const reads=[root,dirs.home,dirs.tmp,...sandboxPlaces(selected,process.env,root),...systemPlaces];
334
- sandbox={mode:support.mode,dirs,argv:sandboxArgv(support.mode,command,{reads,tmp:dirs.tmp,python:support.python}),
335
+ // sandbox-v2 hides the checkout's git-ignored entries. Where git
336
+ // cannot list them, nothing is hidden, and the receipt compares every file.
337
+ const hidden=selected.qualification===hiddenQualification?hiddenEntries(root):null;
338
+ sandbox={mode:support.mode,dirs,reads,hidden,argv:sandboxArgv(support.mode,command,{reads,tmp:dirs.tmp,python:support.python,
339
+ hidden:(hidden||[]).map(rel=>path.join(root,rel.replace(/\/$/,'')))}),
335
340
  env:sandboxEnv(selected,process.env,{root,...dirs})};
336
341
  }
337
342
  }
@@ -516,6 +521,45 @@ async function execute() {
516
521
  // A recorded certificate changes only the probe part of the fingerprint.
517
522
  if(!outputs&&(!snap||!end||snap.planId!==end.planId||(recording?snap.inputKey!==end.inputKey:snap.fingerprint!==end.fingerprint)||
518
523
  snap.revision!==end.revision))captureIssues.push('declared inputs changed or were not observed');
524
+ // A run that hid git-ignored files and failed with a refusal that names
525
+ // one runs again in the sandbox without hiding them: the check needs
526
+ // them. That run's result is kept, compared on every file, when the
527
+ // inputs held through it too. Its own refusal is handled as below.
528
+ let hiddenRerun=null;
529
+ if(sandbox?.hidden?.length&&result==='FAIL'&&!requestedSignal){
530
+ const line=hiddenRefusal(output.head+'\n'+output.tail,sandbox.hidden);
531
+ if(line){
532
+ console.error(`redue: the sandboxed run failed with what looks like a refusal of a file git ignores (${line}); running it again in the sandbox without hiding them`);
533
+ const again=await runToEnd(sandboxArgv(sandbox.mode,command,{reads:sandbox.reads,tmp:sandbox.dirs.tmp,python:support.python}),
534
+ {cwd:plan.cwd,env:sandbox.env,echo:true,started:child=>{activeChild=child;}});
535
+ activeChild=null;
536
+ hiddenRerun={line,exitCode:again.status,signal:again.signal||null,...again.error?{errorCode:again.error.code||again.error.name}:{}};
537
+ if(requestedSignal){
538
+ captureIssues.push('interrupted during the run without hidden files');
539
+ Object.assign(invocation,{status:'interrupted',exitCode:null,signal:requestedSignal});
540
+ }
541
+ else if(again.error||again.status===null)
542
+ captureIssues.push(`the run without hidden files did not finish (${again.error?.code||again.signal}), so a refusal of a file git ignores is not ruled out (${line})`);
543
+ else{
544
+ const wait=decisionWait(process.env.VSTATE_CAPTURE_TIMEOUT_MS);
545
+ // Inputs, plan, observer and environment held through it as through the first run.
546
+ let confirmed=null;const environmentNow=contextHashes()[name];
547
+ try{confirmed=await call({action:'snapshot',name,contextHashes:contextHashes(),
548
+ ...Number.isSafeInteger(snap?.inputEventSerial)?{sinceSerial:snap.inputEventSerial}:{}},wait.timeout,wait.deadline);}catch{}
549
+ const last=confirmed?.snapshots?.[name];
550
+ if(!last?.observationHealthy||!snap||last.planId!==snap.planId||last.fingerprint!==snap.fingerprint||last.revision!==snap.revision||
551
+ last.inputEventSerial!==end?.inputEventSerial||last.planGeneration!==snap.planGeneration||
552
+ confirmed.observation?.generation!==before?.observation?.generation)
553
+ captureIssues.push('declared inputs changed or were not observed during the run without hidden files');
554
+ if(JSON.stringify(environmentNow)!==JSON.stringify(environmentBefore))
555
+ captureIssues.push('declared environment changed during the run without hidden files');
556
+ invocation.hiddenExitCode=invocation.exitCode;invocation.exitCode=again.status;
557
+ result=again.status===0?'PASS':'FAIL';
558
+ output.head=`${again.stdout}\n${again.stderr}`.slice(0,65536);output.tail='';
559
+ sandbox.hidden=null;
560
+ }
561
+ }
562
+ }
519
563
  // A sandboxed run that failed with what looks like a refusal by the
520
564
  // sandbox runs again outside it. When that run exits differently, the
521
565
  // sandbox changed the result: the plain run's result is kept, not reused.
@@ -562,7 +606,9 @@ async function execute() {
562
606
  // A recorded run's receipt carries its run facts, even none; observers that
563
607
  // predate run facts do not reuse version 2.
564
608
  // The recipe that made the receipt, which status and the hook name.
565
- const receipt={result,invocation,target,stable,fingerprintVersion:recorded||testRun?2:1,recipe:(fallback?scriptQualification:selected.qualification)||null,
609
+ // A sandboxed receipt names what the run did: hid the git-ignored files or not.
610
+ const receipt={result,invocation,target,stable,fingerprintVersion:recorded||testRun?2:1,
611
+ recipe:(fallback?scriptQualification:own?(sandbox?.hidden?hiddenQualification:sandboxQualification):selected.qualification)||null,
566
612
  ...(recorded||testRun?{runFacts}:{}),
567
613
  ...(testRun?{verificationRecipe:general?scriptInterpretation:testsInterpretation,guardIssues:guard?.issues||['the run record is unavailable'],
568
614
  environmentReads:guard?.environmentReads||{},...(nondeterministic?{nondeterministic}:{})}:{}),
@@ -576,12 +622,15 @@ async function execute() {
576
622
  planId:snap?.planId??plan.id,fingerprint:(recording||outputs?end:snap)?.fingerprint??null,
577
623
  provider:plan.provider,environmentHashes:environmentBefore,
578
624
  ...(observedContext?{observedContext,verificationRecipe:directInterpretation}:{}),
579
- ...(own?{verificationRecipe:sandboxInterpretation,sandbox:{mode:sandbox?.mode??null,...sandboxRerun?{rerun:sandboxRerun}:{}}}:{}),
625
+ // `hidden`: the git-ignored entries the run did not see, which the observer does not compare.
626
+ ...(own?{verificationRecipe:sandbox?.hidden?sandboxInterpretationOf(hiddenQualification):sandboxInterpretation,
627
+ sandbox:{mode:sandbox?.mode??null,hidden:sandbox?.hidden??null,...hiddenRerun?{hiddenRerun}:{},...sandboxRerun?{rerun:sandboxRerun}:{}}}:{}),
580
628
  ...(fallback?{sandbox:{mode:null,fallback:'node-script',issue:support.issue}}:{}),
581
629
  files:(outputs?end:snap)?.files??null,observerGeneration:before?.observation?.generation??null,
582
630
  startRevision:snap?.revision??null,endRevision:end?.revision??null};
583
631
  const persistenceStarted=performance.now();
584
632
  commitReceipt(state,name,receipt);
633
+ note(state,{kind:'run',check:name,ms:invocation.durationMs??null,result,stable});
585
634
  if(testRun){const reads=environmentReads();reads[name]=receipt.environmentReads;atomicJson(readsFile,{schema:1,checks:reads});}
586
635
  timing('cli.receipt_persistence',persistenceStarted);
587
636
  const reloadStarted=performance.now();
@@ -671,7 +720,8 @@ async function main(){
671
720
  await new Promise(r=>setTimeout(r,100));
672
721
  if(fs.existsSync(path.join(state,'observer.lock')))throw Error('observer did not stop');}
673
722
  }
674
- fs.rmSync(state,{recursive:true,force:true});
723
+ // A stopping observer can still be closing its files (Windows: ENOTEMPTY, EBUSY).
724
+ fs.rmSync(state,{recursive:true,force:true,maxRetries:10,retryDelay:200});
675
725
  console.log(JSON.stringify({removed:state}));return;
676
726
  }
677
727
  if(action==='status'){console.log(JSON.stringify(cached()));return;}
package/src/daemon.mjs CHANGED
@@ -25,7 +25,7 @@ import {npmCacheDir} from './npm-integrity.mjs';
25
25
  import {runFromArgv} from './typescript-invocation.mjs';
26
26
  import {buildInfoPath,recordedCertificate} from './direct-run-certificate.mjs';
27
27
  import {timing,mark,traceBase,traceContext,traceScope,traceEnvironment,traceId} from './decision-profile.mjs';
28
- import {sandboxChange} from './sandbox.mjs';
28
+ import {ignoredNow,isSandboxed,sandboxChange,trackedBeneath} from './sandbox.mjs';
29
29
 
30
30
  const configFile=path.resolve(process.argv[2]);
31
31
  const config=JSON.parse(fs.readFileSync(configFile));
@@ -348,6 +348,39 @@ function describeChangedInput(plan,file) {
348
348
  .map(mapping=>mapping.logical).slice(0,2);
349
349
  return file+' changed'+(matches.length?` (installed via ${matches.join(', ')})`:'');
350
350
  }
351
+ // A run that hid the checkout's git-ignored entries (sandbox@2) did not see
352
+ // them. A changed file the run saw always counts. A file beneath a hidden
353
+ // entry counts only once git tracks it, and a new file only when git does
354
+ // not list it as ignored now, as the next run's hidden entries would not.
355
+ // Installed trees count whatever git says. The first changed input that
356
+ // counts, or null; once per receipt and pair of fingerprints. When git
357
+ // cannot tell, every change counts.
358
+ const hiddenRun=r=>Array.isArray(r?.sandbox?.hidden)&&Boolean(r.files);
359
+ function visibleChange(row,r,name) {
360
+ const key=(r.invocation?.runId||'')+'\0'+r.fingerprint+'\0'+combinedSummary(name).fingerprint;
361
+ if(row.visibleKey===key)return row.visibleChange;
362
+ const hidden=r.sandbox.hidden.map(entry=>entry.replace(/\/$/,''));
363
+ const beneath=file=>hidden.some(entry=>file===entry||file.startsWith(entry+'/'));
364
+ const current=combinedFiles(name);
365
+ const changed=[...new Set([...Object.keys(r.files),...Object.keys(current)])].filter(file=>r.files[file]!==current[file]).sort();
366
+ const seen=changed.filter(file=>file in r.files&&!beneath(file));
367
+ const covered=changed.filter(file=>beneath(file)),fresh=changed.filter(file=>!(file in r.files)&&!beneath(file));
368
+ let found=seen[0]??null;
369
+ if(!found&&covered.length){
370
+ const tracked=trackedBeneath(root,hidden.filter(entry=>covered.some(file=>file===entry||file.startsWith(entry+'/'))));
371
+ found=covered.find(file=>!tracked||tracked.has(file))??null;
372
+ }
373
+ if(!found&&fresh.length){
374
+ // Ask once per directory the run had no file in, then the files outside one.
375
+ const before=new Set(Object.keys(r.files).flatMap(file=>file.split('/').slice(0,-1).map((_,i,parts)=>parts.slice(0,i+1).join('/'))));
376
+ const newDir=file=>{const parts=file.split('/');for(let i=1;i<parts.length;i++){const dir=parts.slice(0,i).join('/');if(!before.has(dir))return dir+'/';}return null;};
377
+ const asked=[...new Set(fresh.filter(file=>!file.startsWith('external:')).map(file=>newDir(file)??file))];
378
+ const ignored=ignoredNow(root,asked);
379
+ found=fresh.find(file=>file.startsWith('external:')||!ignored||!ignored.has(file))??null;
380
+ }
381
+ row.visibleKey=key;row.visibleChange=found;
382
+ return found;
383
+ }
351
384
  function rows(contextHashes,synchronizedInstalled=false,synchronizedFilesystem=false) {
352
385
  if(!index)return [];
353
386
  return [...index.checks].map(([name,row])=>{
@@ -368,7 +401,7 @@ function rows(contextHashes,synchronizedInstalled=false,synchronizedFilesystem=f
368
401
  return item;}
369
402
  if(pendingChecks.has(name)){item.reason='filesystem change pending content inspection';return item;}
370
403
  if(r&&r.planId) item.declared_inputs_match=r.planId===row.plan.id&&
371
- r.fingerprint===combinedSummary(name).fingerprint;
404
+ (r.fingerprint===combinedSummary(name).fingerprint||hiddenRun(r)&&visibleChange(row,r,name)===null);
372
405
  if(r&&!(r.fingerprintVersion===1||r.fingerprintVersion===2&&Array.isArray(r.runFacts))){
373
406
  item.reason='receipt uses earlier input-resolution format';return item;}
374
407
  if(row.plan.unresolved.length){
@@ -410,7 +443,7 @@ function rows(contextHashes,synchronizedInstalled=false,synchronizedFilesystem=f
410
443
  ...(row.plan.environment.executableIdentity?['__vstate_node_identity__']:[])];
411
444
  // A sandboxed run's variables, toolchain, PATH directories and places
412
445
  // (sandbox.mjs): every entry either side has, and the first change named.
413
- const sandboxChanged=row.plan.qualification==='sandbox-v1'&&contextHashes?.[name]?
446
+ const sandboxChanged=isSandboxed(row.plan.qualification)&&contextHashes?.[name]?
414
447
  sandboxChange(r?.environmentHashes,contextHashes[name]):null;
415
448
  if(names.length&&!contextHashes?.[name])unconfirmed??='execution environment context not supplied';
416
449
  item.freshness='STALE';
@@ -431,8 +464,10 @@ function rows(contextHashes,synchronizedInstalled=false,synchronizedFilesystem=f
431
464
  item.freshness='UNVERIFIED';item.reason='environment variables the last run read were not compared';}
432
465
  else item.reason=`environment variable ${contextHashes[name].__redue_env_changed__} differs from the last run`;
433
466
  }
434
- else if(r.fingerprint!==combinedSummary(name).fingerprint) {
467
+ else if(r.fingerprint!==combinedSummary(name).fingerprint&&!(hiddenRun(r)&&visibleChange(row,r,name)===null)) {
435
468
  const key=r.fingerprint+'\0'+combinedSummary(name).fingerprint;
469
+ if(row.staleReasonKey!==key&&hiddenRun(r)){
470
+ row.staleInputs=[visibleChange(row,r,name)];row.staleReason=describeChangedInput(row.plan,row.staleInputs[0]);row.staleReasonKey=key;}
436
471
  if(row.staleReasonKey!==key){
437
472
  let changed=row.lastReason;row.staleInputs=[];
438
473
  const candidate=changed?.endsWith(' changed')?changed.slice(0,-8):null;
@@ -807,7 +842,7 @@ function markInputEvent(name,rel,type,replay) {
807
842
  const serial=(inputEventSerial.get(name)||0)+1;
808
843
  inputEventSerial.set(name,serial);
809
844
  // Kept only for recorded test and script checks and sandboxed ones; trimmed in batches.
810
- if(['npm-tests-direct-v1','node-script-v1','sandbox-v1'].includes(index?.checks.get(name)?.plan.qualification)){
845
+ if(['npm-tests-direct-v1','node-script-v1'].includes(index?.checks.get(name)?.plan.qualification)||isSandboxed(index?.checks.get(name)?.plan.qualification)){
811
846
  const list=inputEventPaths.get(name)||[];list.push([serial,rel]);
812
847
  if(list.length>2*eventPathLimit)list.splice(0,list.length-eventPathLimit);inputEventPaths.set(name,list);
813
848
  }
@@ -8,7 +8,7 @@ import {qualifyTypeScript} from './typescript-qualification.mjs';
8
8
  import {directBuildInfo} from './direct-typescript.mjs';
9
9
  import {qualifyTests,testWorkspacePatterns,testsQualification} from './test-direct.mjs';
10
10
  import {qualifyNodeScript,scriptQualification} from './node-script.mjs';
11
- import {qualifySandbox,sandboxInstalledExclusions,sandboxQualification} from './sandbox.mjs';
11
+ import {isSandboxed,qualifySandbox,sandboxInstalledExclusions} from './sandbox.mjs';
12
12
  import {yarnWorkspaceTypecheckInputs} from './yarn-workspace-inputs.mjs';
13
13
  import {pnpmTypecheckInputs} from './pnpm-inputs.mjs';
14
14
  import {checkRun} from './typescript-invocation.mjs';
@@ -81,7 +81,7 @@ export function discoverDeclared(config) {
81
81
  const excluded=buildInfo?['!'+buildInfo]:[];
82
82
  // A sandboxed check leaves out the .venv's links that lead out of the
83
83
  // checkout; its qualification names them.
84
- const sandboxed=selected.qualification===sandboxQualification;
84
+ const sandboxed=isSandboxed(selected.qualification);
85
85
  if(sandboxed)excluded.push(...sandboxInstalledExclusions(root));
86
86
  const installedStarted=performance.now();
87
87
  const installed=resolveDeclaredInstalled(root,
package/src/index.mjs CHANGED
@@ -6,6 +6,7 @@ import micromatch from './glob.mjs';
6
6
  import {sha} from './plan.mjs';
7
7
  import {resolveLinks} from './installed-inputs.mjs';
8
8
  import {validInputRelative,identityFields,racyUntil} from './path-identity.mjs';
9
+ import {isSandboxed} from './sandbox.mjs';
9
10
 
10
11
  const tracked = root => [...new Set(execFileSync('git',
11
12
  ['ls-files','--cached','--others','--exclude-standard','-z'],
@@ -204,7 +205,7 @@ export class InputIndex {
204
205
  // (its package.json stands for them).
205
206
  // The second form is for recorded checks only: their recorder
206
207
  // classifies a read through the link where it leads.
207
- const recorded=name=>['node-script-v1','npm-tests-direct-v1','sandbox-v1'].includes(this.checks.get(name).plan.qualification);
208
+ const recorded=name=>['node-script-v1','npm-tests-direct-v1'].includes(this.checks.get(name).plan.qualification)||isSandboxed(this.checks.get(name).plan.qualification);
208
209
  const observedDir=installed?physical=>!this.includeInstalled&&
209
210
  ([...this.installedRootSet].some(root=>physical.startsWith(root+'/'))||
210
211
  !physical.split('/').includes('node_modules')&&!this.installedPath(physical+'/package.json')&&matched.every(recorded))&&
package/src/landlock.py CHANGED
@@ -6,6 +6,14 @@
6
6
  # connection to an abstract Unix socket outside the run. Writes are not
7
7
  # handled, so they are not restricted.
8
8
  #
9
+ # --hide PATH keeps the run from reading or executing beneath PATH, inside a
10
+ # --read place. Landlock only grants, beneath a whole directory, so such a
11
+ # place gets listing rights (beneath it all, hidden entries included) and
12
+ # read and execute rights on each of its entries that is not hidden and
13
+ # holds nothing hidden, descending into the ones that do. A hidden path can
14
+ # still be listed and stat'ed. --stats FILE writes the rule count and the
15
+ # time the rules took, as JSON.
16
+ #
9
17
  # The launcher stays as the command's parent and a child subreaper: when the
10
18
  # command exits, it stops every process the command left running, daemons
11
19
  # that started their own session included (they are reparented to it), as a
@@ -13,7 +21,7 @@
13
21
  # launcher gets is passed to that group once. The launcher then ends as the
14
22
  # command did: with its exit status, or by its signal.
15
23
  #
16
- # landlock.py --read /usr --read /tmp ... [--no-tcp] -- cmd args...
24
+ # landlock.py --read /usr --read /tmp ... [--hide P ...] [--stats F] [--no-tcp] -- cmd args...
17
25
  # landlock.py --probe # prints {"abi": N} or {"abi": null, "errno": E}
18
26
  import ctypes
19
27
  import json
@@ -53,10 +61,14 @@ def main(argv):
53
61
  v, err = abi()
54
62
  print(json.dumps({'abi': v, 'errno': err}))
55
63
  return 0
56
- reads, no_tcp = [], False
64
+ reads, hidden, stats, no_tcp = [], set(), None, False
57
65
  while argv and argv[0] != '--':
58
66
  if argv[0] == '--read':
59
67
  reads.append(argv[1]); argv = argv[2:]
68
+ elif argv[0] == '--hide':
69
+ hidden.add(os.path.normpath(argv[1])); argv = argv[2:]
70
+ elif argv[0] == '--stats':
71
+ stats = argv[1]; argv = argv[2:]
60
72
  elif argv[0] == '--no-tcp':
61
73
  no_tcp = True; argv = argv[1:]
62
74
  else:
@@ -72,17 +84,50 @@ def main(argv):
72
84
  ruleset = libc.syscall(SYS_CREATE, ctypes.byref(attr), ctypes.c_size_t(size), ctypes.c_uint32(0))
73
85
  if ruleset < 0:
74
86
  sys.exit(f'landlock.py: create_ruleset errno {ctypes.get_errno()}')
75
- for place in reads:
87
+ started, count = time.monotonic(), 0
88
+ def add(place, rights=None):
89
+ nonlocal count
76
90
  try:
77
91
  fd = os.open(place, os.O_PATH | os.O_CLOEXEC)
78
92
  except OSError:
79
- continue
93
+ return
80
94
  # A file takes file rights only.
81
- rights = handled if os.path.isdir(place) else EXECUTE | READ_FILE
95
+ if rights is None:
96
+ rights = handled if os.path.isdir(place) else EXECUTE | READ_FILE
82
97
  rule = PathBeneath(rights, fd)
83
98
  if libc.syscall(SYS_ADD, ctypes.c_int(ruleset), ctypes.c_int(1), ctypes.byref(rule), ctypes.c_uint32(0)) < 0:
84
99
  sys.exit(f'landlock.py: add_rule {place} errno {ctypes.get_errno()}')
85
100
  os.close(fd)
101
+ count += 1
102
+ # The directories that hold a hidden path, up to the root.
103
+ holders = set()
104
+ for path in hidden:
105
+ at = os.path.dirname(path)
106
+ while at not in holders and at != os.path.dirname(at):
107
+ holders.add(at); at = os.path.dirname(at)
108
+ def grant(directory):
109
+ # Links are not followed: what they lead to is granted on its own or not at all.
110
+ try:
111
+ entries = list(os.scandir(directory))
112
+ except OSError:
113
+ return
114
+ for entry in entries:
115
+ if entry.path in hidden or entry.is_symlink():
116
+ continue
117
+ if entry.path in holders:
118
+ grant(entry.path)
119
+ else:
120
+ add(entry.path)
121
+ for place in reads:
122
+ place = os.path.normpath(place)
123
+ if place in holders and os.path.isdir(place):
124
+ add(place, READ_DIR)
125
+ grant(place)
126
+ else:
127
+ add(place)
128
+ if stats:
129
+ with open(stats, 'w') as f:
130
+ json.dump({'rules': count, 'ms': round((time.monotonic() - started) * 1000, 1)}, f)
86
131
  if libc.prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0:
87
132
  sys.exit(f'landlock.py: no_new_privs errno {ctypes.get_errno()}')
88
133
  if libc.syscall(SYS_RESTRICT, ctypes.c_int(ruleset), ctypes.c_uint32(0)) < 0:
@@ -2,7 +2,8 @@
2
2
  // The recipe that made a receipt, by the qualification it names.
3
3
  const recipes={'npm-typescript-direct-v1':'direct TypeScript recipe','npm-tests-direct-v1':'direct test recipe',
4
4
  'node-script-v1':'general recipe','pnpm-tsc-v1':'pnpm TypeScript recipe','yarn-workspace-tsc-v1':'Yarn workspace TypeScript recipe',
5
- 'typescript-noemit-v1':'npm TypeScript recipe','sandbox-v1':'sandboxed recipe'};
5
+ 'typescript-noemit-v1':'npm TypeScript recipe','sandbox-v1':'sandboxed recipe',
6
+ 'sandbox-v2':'sandboxed recipe, git-ignored files hidden'};
6
7
  export const recipeLabel=recipe=>recipes[recipe]??null;
7
8
  export function checkLabel(row) {
8
9
  return row.result==='FAIL'?'FAILED':row.freshness;
package/src/sandbox.mjs CHANGED
@@ -19,9 +19,18 @@ import {spawn,spawnSync} from 'node:child_process';
19
19
  import {fileURLToPath} from 'node:url';
20
20
  import {lookup,scriptCaches,scriptCheckInputs,scriptProject,toolchainDirs} from './node-script.mjs';
21
21
  import {displayVariables} from './test-direct.mjs';
22
+ import picomatch from 'picomatch';
22
23
 
23
24
  export const sandboxQualification='sandbox-v1';
24
25
  export const sandboxInterpretation='sandbox@1';
26
+ // The same recipe with the checkout's git-ignored files hidden from the run
27
+ // (hiddenEntries): they are not its inputs, so build and test outputs do not
28
+ // make it STALE. init gives it when the run gives the same result that way.
29
+ export const hiddenQualification='sandbox-v2';
30
+ export const hiddenInterpretation='sandbox@2';
31
+ export const hiddenListIssue='git cannot list the ignored files';
32
+ export const isSandboxed=qualification=>qualification===sandboxQualification||qualification===hiddenQualification;
33
+ export const sandboxInterpretationOf=qualification=>qualification===hiddenQualification?hiddenInterpretation:sandboxInterpretation;
25
34
  export const sandboxKinds=['typecheck','lint'];
26
35
  const fail=code=>{throw Object.assign(Error(code),{code});};
27
36
  const digest=value=>createHash('sha256').update(value).digest('hex');
@@ -157,16 +166,26 @@ export function sandboxEnv(check,env,{root,home,tmp}){
157
166
  }
158
167
  return out;
159
168
  }
160
- // The files in the checkout's .git that tools read to choose files (ruff's
161
- // and biome's ignore rules): the repository's exclude file and config. The
162
- // observer does not watch .git, so they are compared as context.
169
+ // The files in the repository's .git that tools read to choose files (ruff's
170
+ // and biome's ignore rules), and that decide what sandbox-v2 hides: its
171
+ // exclude file and config. For a project in a subdirectory of the
172
+ // repository, also the .gitignore files above it. The observer watches
173
+ // neither, so they are compared as context.
163
174
  function gitIgnoreSources(root){
164
- let dir=path.join(root,'.git');
175
+ let top=root;
176
+ while(!fs.existsSync(path.join(top,'.git'))){const up=path.dirname(top);if(up===top)return [];top=up;}
177
+ let dir=path.join(top,'.git');
165
178
  try{if(fs.statSync(dir).isFile()){
166
- const gitdir=path.resolve(root,/^gitdir:\s*(.+)$/m.exec(fs.readFileSync(dir,'utf8'))?.[1].trim()??'');
179
+ const gitdir=path.resolve(top,/^gitdir:\s*(.+)$/m.exec(fs.readFileSync(dir,'utf8'))?.[1].trim()??'');
167
180
  let common=gitdir;try{common=path.resolve(gitdir,fs.readFileSync(path.join(gitdir,'commondir'),'utf8').trim());}catch{}
168
181
  dir=common;}}catch{return [];}
169
- return ['info/exclude','config'].map(rel=>{try{return [rel,digest(fs.readFileSync(path.join(dir,rel)))];}catch{return [rel,null];}});
182
+ const read=file=>{try{return digest(fs.readFileSync(file));}catch{return null;}};
183
+ const above=[];
184
+ for(let at=path.dirname(root);top!==root&&(at===top||at.startsWith(top+path.sep));at=path.dirname(at)){
185
+ above.push([path.relative(top,path.join(at,'.gitignore')),read(path.join(at,'.gitignore'))]);
186
+ if(at===top)break;
187
+ }
188
+ return [...['info/exclude','config'].map(rel=>[rel,read(path.join(dir,rel))]),...above];
170
189
  }
171
190
  // What a sandboxed result depends on outside the observed files, compared
172
191
  // before reuse, one entry each so a change can be named (sandboxChange): every
@@ -211,33 +230,102 @@ export function sandboxDirs(state,name){
211
230
  return {home,tmp:fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(),'redue-run-')))};
212
231
  }
213
232
 
233
+ // The checkout's git-ignored entries a run may be kept from reading: what
234
+ // `git ls-files --others --ignored --exclude-standard --directory` lists (a
235
+ // wholly ignored directory as one entry, ending in /), but the installed
236
+ // trees (node_modules, .venv) and runner caches. Relative to root. null when
237
+ // git cannot list them. Only the project's rules count (.gitignore files and
238
+ // .git/info/exclude), not a user's global excludes file, as on CI.
239
+ const installedTree=/(?:^|\/)(?:node_modules|\.venv)(?:\/|$)/;
240
+ const runnerCache=picomatch(sandboxCaches,{dot:true});
241
+ const projectGit=['-c','core.excludesFile=/dev/null'],gitEnv=()=>({...process.env,GIT_OPTIONAL_LOCKS:'0'});
242
+ export function hiddenEntries(root){
243
+ const r=spawnSync('git',[...projectGit,'ls-files','-z','--others','--ignored','--exclude-standard','--directory'],
244
+ {cwd:root,encoding:'utf8',maxBuffer:256*1024*1024,timeout:60000,stdio:['ignore','pipe','ignore'],env:gitEnv()});
245
+ if(r.status!==0)return null;
246
+ return r.stdout.split('\0').filter(Boolean).filter(rel=>!installedTree.test(rel)&&
247
+ !runnerCache(rel)&&!runnerCache(rel.replace(/\/$/,'')+'/x'));
248
+ }
249
+
250
+ // What git lists now as ignored at or beneath `paths` (relative files, or
251
+ // directories ending in /), as hiddenEntries would: nested repositories are
252
+ // not entered, and an entry that is an installed tree is not ignored (one
253
+ // inside an ignored build directory is). {has(rel)} for a file, or null when
254
+ // git cannot tell. A directory git ignores holds no file it does not, so
255
+ // asking a new directory once covers its files.
256
+ export function ignoredNow(root,paths){
257
+ const asked=paths;
258
+ if(!asked.length)return {has:()=>false};
259
+ const listed=new Set();
260
+ for(let at=0;at<asked.length;at+=500){
261
+ const r=spawnSync('git',[...projectGit,'ls-files','-z','--others','--ignored','--exclude-standard','--directory','--',...asked.slice(at,at+500)],
262
+ {cwd:root,encoding:'utf8',maxBuffer:256*1024*1024,timeout:10000,stdio:['ignore','pipe','ignore'],env:gitEnv()});
263
+ if(r.status!==0)return null;
264
+ for(const entry of r.stdout.split('\0'))if(entry)listed.add(entry);
265
+ }
266
+ const entries=[...listed].filter(entry=>!installedTree.test(entry));
267
+ return {has:rel=>entries.some(entry=>rel===entry||entry.endsWith('/')&&rel.startsWith(entry))};
268
+ }
269
+ // The files git tracks now beneath `entries` (relative). null when git cannot tell.
270
+ export function trackedBeneath(root,entries){
271
+ if(!entries.length)return new Set();
272
+ const r=spawnSync('git',['ls-files','-z','--cached','--',...entries],
273
+ {cwd:root,encoding:'utf8',maxBuffer:256*1024*1024,timeout:10000,stdio:['ignore','pipe','ignore'],env:gitEnv()});
274
+ return r.status===0?new Set(r.stdout.split('\0').filter(Boolean)):null;
275
+ }
276
+ // A refusal line that names a hidden entry (relative): a refusal by the
277
+ // sandbox, or a missing file, which Linux tools report for an unreadable one.
278
+ export function hiddenRefusal(text,hidden){
279
+ const names=hidden.map(file=>file.replace(/\/$/,'')).filter(Boolean)
280
+ .map(name=>new RegExp(`(?:^|[\\s'"\`(=/])${escape(name)}(?:[/\\s'"\`):,]|$)`));
281
+ for(const line of String(text).split(/\r?\n/))
282
+ if((denial.test(line)||missing.test(line))&&names.some(name=>name.test(line)))return line.trim().slice(0,200);
283
+ return null;
284
+ }
285
+
214
286
  const ancestors=list=>[...new Set(list.flatMap(dir=>{const out=[];for(let at=path.dirname(dir);at!=='/'&&at!=='.';at=path.dirname(at))out.push(at);return out;}))];
215
287
  // The Seatbelt profile: reads and executes only beneath the places, the
216
288
  // metadata of their parent directories, and no network but Unix sockets in
217
289
  // the run's own temporary directory (`own`), which is new for each run: a
218
- // daemon an earlier run left behind cannot answer a later one.
219
- export function seatbeltProfile(reads,own=null){
290
+ // daemon an earlier run left behind cannot answer a later one. `hidden`:
291
+ // absolute paths it may not read at all, not even their metadata.
292
+ export function seatbeltProfile(reads,own=null,hidden=[]){
220
293
  const quote=value=>JSON.stringify(value);
221
294
  const all=[...new Set(reads.flatMap(dir=>[dir,real(dir)]).filter(Boolean))];
222
295
  return ['(version 1)','(allow default)','(deny file-read* process-exec*)',
223
296
  `(allow file-read* process-exec* ${all.map(dir=>`(subpath ${quote(dir)})`).join(' ')} (literal "/") (literal "/etc") (literal "/var") (literal "/tmp"))`,
224
297
  `(allow file-read-metadata ${ancestors(all).map(dir=>`(literal ${quote(dir)})`).join(' ')})`,
298
+ // The entry's own path, by its directory's real path: a hidden link's target is not hidden.
299
+ ...hidden.length?[`(deny file-read* process-exec* ${[...new Set(hidden.flatMap(file=>[file,path.join(real(path.dirname(file))??path.dirname(file),path.basename(file))]))]
300
+ .map(file=>`(subpath ${quote(file)})`).join(' ')})`]:[],
225
301
  '(deny network*)',...own?[`(allow network* ${[own,real(own)].filter(Boolean).map(dir=>`(subpath ${quote(dir)})`).join(' ')})`]:[]].join('\n');
226
302
  }
227
303
  // The argv that runs `argv` sandboxed: reads beneath `reads`, no network but
228
- // Unix sockets beneath `tmp` on macOS.
229
- export function sandboxArgv(mode,argv,{reads,tmp=null,python}){
230
- if(mode==='seatbelt')return ['/usr/bin/sandbox-exec','-p',seatbeltProfile(reads,tmp),...argv];
304
+ // Unix sockets beneath `tmp` on macOS, nothing beneath `hidden` (absolute).
305
+ // On Linux a hidden directory can still be listed and its entries stat'ed:
306
+ // Landlock restricts neither beneath a listable parent.
307
+ export function sandboxArgv(mode,argv,{reads,tmp=null,python,hidden=[],stats=null}){
308
+ if(mode==='seatbelt')return ['/usr/bin/sandbox-exec','-p',seatbeltProfile(reads,tmp,hidden),...argv];
231
309
  // The launcher's own warnings would become the check's output.
232
- if(mode==='landlock')return [python,'-I','-W','ignore',landlockScript,...reads.flatMap(dir=>['--read',dir]),'--no-tcp','--',...argv];
310
+ if(mode==='landlock')return [python,'-I','-W','ignore',landlockScript,...reads.flatMap(dir=>['--read',dir]),
311
+ ...hidden.flatMap(file=>['--hide',file]),...stats?['--stats',stats]:[],'--no-tcp','--',...argv];
233
312
  throw Error(`unknown sandbox ${mode}`);
234
313
  }
235
314
  // A line of a failed run's output that looks like the sandbox refused
236
315
  // something: a refused read or write, or the network unreachable. Such a
237
316
  // failure is run again outside the sandbox to tell a denial from a real one.
238
317
  const denial=/operation not permitted|permission denied|read-only file system|\bE(?:PERM|ACCES|ROFS|NOTFOUND|AI_AGAIN|CONNREFUSED|NETUNREACH|HOSTUNREACH)\b|sandbox-exec:|landlock\.py:|network is unreachable|could not resolve host|name or service not known|nodename nor servname|temporary failure in name resolution|getaddrinfo|connection refused/i;
239
- export function denialLine(text){
240
- for(const line of String(text).split(/\r?\n/))if(denial.test(line))return line.trim().slice(0,200);
318
+ // A missing file counts only when the line names a path the run was kept
319
+ // from (`hidden`, relative to the checkout or absolute): Linux tools report
320
+ // an unreadable file as not found.
321
+ const missing=/not found|no such file|cannot find|can't find|could not find|cannot read|\bENOENT\b/i;
322
+ const escape=text=>text.replace(/[.*+?^${}()|[\]\\]/g,'\\$&');
323
+ export function denialLine(text,hidden=[]){
324
+ // A name as a whole path or path component, not inside a longer word.
325
+ const names=hidden.map(file=>file.replace(/\/$/,'')).filter(Boolean)
326
+ .map(name=>new RegExp(`(?:^|[\\s'"\`(=/])${escape(name)}(?:[/\\s'"\`):,]|$)`));
327
+ for(const line of String(text).split(/\r?\n/))
328
+ if(denial.test(line)||missing.test(line)&&names.some(name=>name.test(line)))return line.trim().slice(0,200);
241
329
  return null;
242
330
  }
243
331
 
@@ -366,7 +454,11 @@ export function runToEnd(argv,{cwd,env,timeout=0,echo=false,started=null}){
366
454
  });
367
455
  }
368
456
  // {same, plain, sandboxed, line} or {issue} when the sandbox cannot run here.
369
- export async function sandboxTrial(root,check,{timeout=300000}={}){
457
+ // With `hide`, the git-ignored entries hiddenEntries lists are kept from the
458
+ // sandboxed run, and it also returns them, their count, the run times and, on
459
+ // Linux, the launcher's rule count and setup time.
460
+ // `plain`: the plain run of an earlier trial of the same check, reused.
461
+ export async function sandboxTrial(root,check,{timeout=300000,hide=false,plain:earlier=null}={}){
370
462
  const supported=sandboxSupport();
371
463
  if(supported.issue)return {issue:supported.issue};
372
464
  const python=Boolean(pythonTool(check));
@@ -375,13 +467,16 @@ export async function sandboxTrial(root,check,{timeout=300000}={}){
375
467
  const scratch=fs.mkdtempSync(path.join(os.tmpdir(),'redue-trial-')),home=path.join(scratch,'home'),tmp=path.join(scratch,'tmp');
376
468
  fs.mkdirSync(home);fs.mkdirSync(tmp);
377
469
  try{
378
- const run=async(argv,env)=>{const r=await runToEnd(argv,{cwd:root,env,timeout});
379
- return {status:r.timedOut?'timeout':r.status??r.signal??r.error?.code,output:`${r.stdout}\n${r.stderr}`};};
380
- const plain=await run(command,{...process.env,...python?{}:{npm_config_update_notifier:'false'}});
470
+ const entries=hide?hiddenEntries(root):[];
471
+ if(!entries)return {issue:hiddenListIssue};
472
+ const hidden=entries.map(rel=>path.join(root,rel.replace(/\/$/,""))),stats=hide&&supported.mode==='landlock'?path.join(scratch,'stats.json'):null;
473
+ const run=async(argv,env)=>{const started=Date.now(),r=await runToEnd(argv,{cwd:root,env,timeout});
474
+ return {status:r.timedOut?'timeout':r.status??r.signal??r.error?.code,output:`${r.stdout}\n${r.stderr}`,ms:Date.now()-started};};
475
+ const plain=earlier??await run(command,{...process.env,...python?{}:{npm_config_update_notifier:'false'}});
381
476
  const reads=[root,home,tmp,...sandboxPlaces(selected,process.env,root),...systemPlaces];
382
- const sandboxed=await run(sandboxArgv(supported.mode,command,{reads,tmp,python:supported.python}),sandboxEnv(selected,process.env,{root,home,tmp}));
477
+ const sandboxed=await run(sandboxArgv(supported.mode,command,{reads,tmp,python:supported.python,hidden,stats}),sandboxEnv(selected,process.env,{root,home,tmp}));
383
478
  // A failure both ways counts as the same only without a sign of a refusal.
384
- const denied=sandboxed.status===0?null:denialLine(sandboxed.output);
479
+ const denied=sandboxed.status===0?null:denialLine(sandboxed.output,entries);
385
480
  const realOf=dir=>{try{return fs.realpathSync(dir);}catch{return null;}},plainTmp=process.env.TMPDIR||os.tmpdir();
386
481
  const paths=[[home,'<home>'],[realOf(home),'<home>'],[tmp,'<tmp>'],[realOf(tmp),'<tmp>'],[plainTmp,'<tmp>'],[realOf(plainTmp),'<tmp>'],
387
482
  [os.homedir(),'<home>'],[realOf(os.homedir()),'<home>']];
@@ -390,7 +485,9 @@ export async function sandboxTrial(root,check,{timeout=300000}={}){
390
485
  let at=a.findIndex((line,i)=>line!==b[i]);
391
486
  if(at<0&&b.length>a.length)at=a.length;
392
487
  const output=at<0?null:{plain:(a[at]??'(no more output)').slice(0,160),sandboxed:(b[at]??'(no more output)').slice(0,160)};
393
- return {same:plain.status===sandboxed.status&&!denied&&!output,plain:plain.status,sandboxed:sandboxed.status,output,
394
- line:denied||(plain.status===sandboxed.status?null:sandboxed.output.trim().split('\n').at(-1)?.slice(0,200)||null)};
488
+ let launcher=null;if(stats)try{launcher=JSON.parse(fs.readFileSync(stats,'utf8'));}catch{}
489
+ return {same:plain.status===sandboxed.status&&!denied&&!output,plain:plain.status,sandboxed:sandboxed.status,output,plainRun:plain,
490
+ line:denied||(plain.status===sandboxed.status?null:sandboxed.output.trim().split('\n').at(-1)?.slice(0,200)||null),
491
+ ...hide?{hidden:entries.length,entries,plainMs:plain.ms,sandboxedMs:sandboxed.ms,...launcher?{rules:launcher.rules,setupMs:launcher.ms}:{}}:{}};
395
492
  }finally{fs.rmSync(scratch,{recursive:true,force:true});}
396
493
  }
package/src/stats.mjs ADDED
@@ -0,0 +1,72 @@
1
+ // `redue stats`: what REDUE did for each check, from a local log. The hook,
2
+ // `run --if-due` and recorded runs append one line each to stats-v1.jsonl in
3
+ // the project's state: a skip (with the skipped run's duration), a recorded
4
+ // run, or a check that was due (STALE or UNVERIFIED, with the reason). The log
5
+ // stays on this machine and goes away with `redue remove-state`.
6
+ import fs from 'node:fs';
7
+ import path from 'node:path';
8
+
9
+ export const statsFile=state=>path.join(state,'stats-v1.jsonl');
10
+
11
+ // Never fails the command it is called from.
12
+ export function note(state,entry){
13
+ try{fs.appendFileSync(statsFile(state),JSON.stringify({at:Date.now(),...entry})+'\n',{mode:0o600});}catch{}
14
+ }
15
+
16
+ // A skip of `name`, from a status row that allowed it.
17
+ export const skipNote=(status,name,via)=>{
18
+ const ms=status?.checks?.find(row=>row?.name===name)?.invocation?.durationMs;
19
+ return {kind:'skip',check:name,via,savedMs:Number.isFinite(ms)&&ms>=0?ms:null};
20
+ };
21
+ // A check about to run because its last result does not apply, or null.
22
+ export const dueNote=(status,name)=>{
23
+ const row=status?.observation?.healthy===true?status.checks?.find(item=>item?.name===name):null;
24
+ return row&&row.freshness!=='CURRENT'?{kind:'due',check:name,freshness:row.freshness,reason:row.reason??null}:null;
25
+ };
26
+
27
+ // Run ids, hashes and digits in a reason say nothing about its kind.
28
+ const reasonKind=reason=>String(reason??'no reason').replace(/[0-9a-f]{8}(?:-?[0-9a-f]{4,})+|[0-9a-f]{12,}/g,'…').slice(0,160);
29
+
30
+ export function readStats(state,names=[]){
31
+ let text='';try{text=fs.readFileSync(statsFile(state),'utf8');}catch{}
32
+ // Configured checks first, in config order, then any the log still names.
33
+ const checks=new Map();
34
+ const row=name=>{let value=checks.get(name);
35
+ if(!value)checks.set(name,value={name,skips:0,savedMs:0,runs:0,passed:0,failed:0,runMs:0,stale:0,unverified:0,reasons:{}});
36
+ return value;};
37
+ for(const name of names)row(name);
38
+ let since=null;
39
+ for(const line of text.split('\n')){
40
+ let entry;try{entry=JSON.parse(line);}catch{continue;}
41
+ if(typeof entry?.check!=='string')continue;
42
+ since??=entry.at;
43
+ const value=row(entry.check);
44
+ if(entry.kind==='skip'){value.skips++;value.savedMs+=entry.savedMs??0;}
45
+ else if(entry.kind==='run'){value.runs++;value.runMs+=entry.ms??0;
46
+ if(entry.result==='PASS')value.passed++;else if(entry.result==='FAIL')value.failed++;}
47
+ else if(entry.kind==='due'&&entry.freshness==='STALE')value.stale++;
48
+ else if(entry.kind==='due'&&entry.freshness==='UNVERIFIED'){value.unverified++;
49
+ const kind=reasonKind(entry.reason);value.reasons[kind]=(value.reasons[kind]??0)+1;}
50
+ }
51
+ const list=[...checks.values()].map(({reasons,...value})=>({...value,
52
+ topUnverified:Object.entries(reasons).sort((a,b)=>b[1]-a[1]).slice(0,3).map(([reason,count])=>({reason,count}))}));
53
+ return {schema:1,since:since===null?null:new Date(since).toISOString(),checks:list};
54
+ }
55
+
56
+ export const duration=ms=>!ms?'0s':ms<60_000?`${(ms/1000).toFixed(ms<10_000?1:0)}s`:
57
+ ms<3_600_000?`${Math.floor(ms/60_000)}m ${Math.round(ms%60_000/1000)}s`:`${Math.floor(ms/3_600_000)}h ${Math.round(ms%3_600_000/60_000)}m`;
58
+
59
+ export function renderStats(value){
60
+ if(!value.since)return 'No skips or recorded runs yet.';
61
+ const head=['check','skips','saved','runs','pass','fail','stale','unverified'];
62
+ const rows=value.checks.map(c=>[c.name,c.skips,duration(c.savedMs),c.runs,c.passed,c.failed,c.stale,c.unverified].map(String));
63
+ const total=value.checks.reduce((sum,c)=>({skips:sum.skips+c.skips,savedMs:sum.savedMs+c.savedMs}),{skips:0,savedMs:0});
64
+ const widths=head.map((h,i)=>Math.max(h.length,...rows.map(r=>r[i].length)));
65
+ const line=r=>r.map((cell,i)=>i?cell.padStart(widths[i]):cell.padEnd(widths[i])).join(' ');
66
+ const lines=[`Since ${value.since.slice(0,10)}: ${total.skips} skips saved ${duration(total.savedMs)}.`,'',line(head),...rows.map(line)];
67
+ const reasons=value.checks.filter(c=>c.topUnverified.length);
68
+ if(reasons.length){lines.push('','Why checks were UNVERIFIED when they were due:');
69
+ for(const c of reasons)for(const r of c.topUnverified)lines.push(` ${c.name} ${r.count}× ${r.reason}`);}
70
+ lines.push('','Saved is the duration of the run each skip reused.');
71
+ return lines.join('\n');
72
+ }