@redue/cli 0.1.0-alpha.26 → 0.1.0-alpha.28

package/CHANGELOG.md

@@ -1,5 +1,85 @@
 # Changelog
 
+## 0.1.0-alpha.28 — 2026-11-02
+
+Test suites can run in the sandbox, so REDUE can reuse a passing test run
+for more projects, Python included.
+
+- **Sandboxed tests.** A root test script that the direct test recipe does
+  not take, and pytest in a Python project's `.venv`, can now use the
+  sandboxed recipe. `init` keeps it only when the run in the sandbox gives the
+  same exit status and test counts as a plain run. On top of what a
+  typecheck or lint gets, a test run gets:
+  - **Loopback only.** It may listen on any address and connect to the local
+    host and Unix sockets, never to a remote host. On macOS, Seatbelt
+    enforces that. On Linux, a seccomp filter refuses UDP and other non-TCP
+    sockets, TCP Fast Open and `io_uring`, and hands each `connect` to
+    REDUE's launcher. The launcher allows a local port nothing listens on or
+    one the run's own processes listen on, and refuses everything else,
+    including a service you started outside the run (a local database). If
+    the launcher cannot start or stops early, the result is UNVERIFIED.
+  - **A private `/tmp`.** Its `TMPDIR` is new for each run, and it may not
+    write elsewhere in `/tmp` or `/var/tmp`.
+  - **A fresh home** for each run, as on CI.
+  - **Ignored files hidden** when the trial agrees, else the whole checkout.
+- **Measured.** In the product census, test suites with a qualifying check
+  (alpha.27 in brackets): TypeScript on Linux 11 (2), on macOS 12 (2);
+  Python on Linux 17 (0), on macOS 14 (0). The strace cross-check found no
+  read outside the recorded inputs. The Linux connect supervisor added at
+  most 1.5% over the sandbox without it on suites with worker processes
+  (pytest-xdist, Jest, Vitest): apprise 233 s to 237 s, run-to-run spread
+  218-243 s. In a pilot with Claude Code on two pytest projects, the hook
+  skipped both "do the tests still pass?" reruns (19 s to 6 s and 28 s to
+  7 s), and plain reruns agreed with both skips.
+- **macOS hides contents only.** On macOS, a hidden file's contents and
+  execution are refused, but its metadata is not, as on Linux. A test that
+  writes into an ignored folder works.
+- Known limits: a suite that reads back ignored files it writes in the same
+  run (a coverage database, a build it starts) gets the whole checkout, and
+  one that writes tracked files or coverage data during the run ends
+  UNVERIFIED. On Linux, a test that needs a service started outside the run
+  fails in the sandbox, so `init` leaves it to a plain run.
+
+## 0.1.0-alpha.27 — 2026-11-01
+
+A build or test run no longer makes a sandboxed typecheck or lint STALE,
+and `redue stats` shows what REDUE saved you.
+
+- **Files git ignores are hidden from sandboxed runs.** A sandboxed
+  typecheck or lint now runs without the files git ignores (`dist/`,
+  `.next/`, coverage files, `.env`), except `node_modules`, `.venv` and
+  runner caches. Those files are not its inputs, so a build or test run that
+  writes them leaves the result CURRENT. The claim: the result holds for a
+  run that sees only the project's files that git doesn't ignore, its
+  installed packages and the toolchain. Only the project's `.gitignore`
+  files and `.git/info/exclude` count, not your global excludes file.
+- **Checks that need ignored files keep working.** `init` tries the
+  sandbox with ignored files hidden first (`sandbox-v2`), then with the
+  whole checkout (`sandbox-v1`), then the general recipe. A hidden run that
+  fails on a hidden file runs again in the sandbox without hiding, and that
+  result is kept. Checks set up by alpha.26 stay `sandbox-v1` until you run
+  `redue init` again.
+- **Measured.** In the product census, after each project's own build and
+  tests ran, checks that stayed CURRENT (alpha.26 in brackets, from the
+  measurement census): TypeScript on Linux 31 of 37 (5 of 43), on macOS 24
+  of 34 (5 of 38); Python on Linux 31 of 38 (24 of 37), on macOS 28 of 34
+  (20 of 33). Where a check qualifies with both, the sandbox stayed CURRENT
+  16 times and the general recipe 9 of 23 (Linux). The strace cross-check
+  found no read outside the recorded inputs.
+- **`redue stats`.** Per check: reruns the hook skipped, the time that
+  saved, recorded runs, and the top reasons a check was UNVERIFIED, from a
+  log in REDUE's local state.
+- `redue remove-state` no longer fails on Windows while the observer is
+  still closing its files.
+- Known limits: a sandboxed run can still see that a hidden file exists
+  (its name on macOS; listings, sizes and times on Linux), so a tool that
+  acts on existence alone is not covered; tests do not use hidden files for
+  that reason. On Linux, a file a run creates in a folder that holds hidden
+  files cannot be read back in the same run.
+- Contributors: eval entry points that run third-party code refuse to start
+  unless `GITHUB_ACTIONS=true`, and importing any eval entry point throws
+  before it does anything.
+
 ## 0.1.0-alpha.26 — 2026-10-31
 
 Typecheck and lint checks run inside an OS sandbox, so REDUE can reuse them

package/README.md

@@ -59,25 +59,53 @@ what the agent asks. Plain `redue status` is a fast cached answer and can say
 UNVERIFIED ("cannot confirm your execution context") where `--sync` would say
 CURRENT.
 
+`redue stats` shows, per check, the reruns the hook skipped, the time saved,
+the recorded runs and the top reasons a check was UNVERIFIED, from a log kept
+on your machine.
+
 To undo everything setup did: `redue remove`.
 
+## Still CURRENT after your build and tests
+
+Running the project's build or tests writes files (`dist/`, `.next/`,
+coverage). REDUE leaves those out when the files are git-ignored, so a
+typecheck or lint that passed stays CURRENT. In the census, after each
+project's own build, tests, typecheck and lint ran, the share of sandboxed
+checks still CURRENT:
+
+| | alpha.26 | alpha.27 |
+|---|---|---|
+| TypeScript, Linux | 5 of 43 | 31 of 37 |
+| TypeScript, macOS | 5 of 38 | 24 of 34 |
+| Python, Linux | 24 of 37 | 31 of 38 |
+| Python, macOS | 20 of 33 | 28 of 34 |
+
+The rest changed for real: the build rewrote a tracked file (a sitemap, a
+feed, a generated source), added a file git doesn't ignore, or installed
+packages.
+
 ## What it covers today
 
-In a census of 120 active public TypeScript repositories (October 2026, REDUE
-at commit 046cd0e, before 0.1.0-alpha.26), each installed as its users would,
-with dependencies' install scripts on, 86 installed cleanly on a macOS runner.
+In a census of 120 active public TypeScript repositories (November 2026, REDUE
+at commit a2401cc, before 0.1.0-alpha.28), each installed as its users would,
+with dependencies' install scripts on, 89 installed cleanly on a macOS runner.
 A check counts when two recorded runs on the same code both came out CURRENT
-on the same inputs. 35 repositories had at least one such check:
+on the same inputs. 38 repositories had at least one such check:
 
 | Check | Repositories | How |
 |---|---|---|
-| Typecheck | 22 | 15 by running the project's script in an OS sandbox ([sandboxed recipe](docs/user/sandbox.md)); 5 by running the local `tsc` directly ([direct recipe](docs/user/direct-typescript.md)); 2 by running the script with every Node process recorded ([general recipe](docs/user/node-script.md)) |
-| Lint | 25 | 21 sandboxed recipe, 4 general recipe |
-| Tests | 2 | Vitest run directly ([direct test recipe](docs/user/tests-direct.md)) |
-
-In a census of 120 Python repositories, 75 installed on a Linux runner
-and 23 had a qualifying check: typecheck (mypy, pyright, ty) 15, lint (ruff)
-22, all through the sandboxed recipe. On Linux, every sandboxed run was traced
+| Typecheck | 20 | 14 by running the project's script in an OS sandbox ([sandboxed recipe](docs/user/sandbox.md)), 13 of them with the files git ignores hidden; 5 by running the local `tsc` directly ([direct recipe](docs/user/direct-typescript.md)); 1 by running the script with every Node process recorded ([general recipe](docs/user/node-script.md)) |
+| Lint | 22 | 19 sandboxed recipe (18 with ignored files hidden), 3 general recipe |
+| Tests | 12 | 10 sandboxed recipe, with ignored files hidden and loopback-only network; 2 Vitest run directly ([direct test recipe](docs/user/tests-direct.md)) |
+
+In alpha.27 the same census found 2 repositories with a qualifying test check
+on macOS and on Linux; now 12 on macOS and 11 on Linux.
+
+In a census of 120 Python repositories, 74 installed on a Linux runner
+and 31 had a qualifying check: typecheck (mypy, pyright, ty) 15, lint (ruff)
+22, tests (pytest) 17, all through the sandboxed recipe (14 of the test
+suites with ignored files hidden, 3 with the whole checkout). In alpha.27, no
+Python test suite qualified. On Linux, every sandboxed run was traced
 with strace: each file it read in the checkout was a recorded input, a cache
 or its own output.
 

package/bin/redue.mjs

@@ -7,7 +7,7 @@ import {directCompiler,directQualification,directEnvironment,directInstalledInpu
 import {runArgv,tscRun} from '../src/typescript-invocation.mjs';
 import {testEnvironment,testRunnerAt,testsInterpretation,testsQualification} from '../src/test-direct.mjs';
 import {scriptEnvironment,scriptInterpretation,scriptKinds,scriptQualification} from '../src/node-script.mjs';
-import {pythonTool,sandboxEnvironment,sandboxInterpretation,sandboxKinds,sandboxQualification} from '../src/sandbox.mjs';
+import {hiddenListIssue,hiddenQualification,isSandboxed,pythonTool,sandboxEnvironment,sandboxInterpretationOf,sandboxKinds,sandboxQualification} from '../src/sandbox.mjs';
 import os from 'node:os';
 import path from 'node:path';
 import {createHash} from 'node:crypto';
@@ -46,6 +46,7 @@ Commands:
   detail [CHECK] [--json]  Alias for explain
   run CHECK [--if-due] Execute one configured check and record its outcome; with --if-due,
                        only when its last pass does not apply to the files as they are now
+  stats [--json]       Per check: skips, time saved, recorded runs, why checks were UNVERIFIED
   remove-state         Stop and remove only this configuration's owned local state
 
 No checks run in the background. Direct commands outside redue create no receipts.`;
@@ -95,7 +96,7 @@ if(command==='agent'){
   catch(e){error(e.message);}
   await exit(0);
 }
-if(!['init','start','stop','status','detail','explain','run','remove-state'].includes(command))
+if(!['init','start','stop','status','detail','explain','run','stats','remove-state'].includes(command))
   error(`unknown command ${command||'<none>'}; use --help`);
 const json=args.includes('--json');
 // run --if-due: run the check only when its last pass does not apply now.
@@ -150,21 +151,30 @@ if(command==='init'){
     if(!chosen.length)error('no discovered check matches the selected workspace and kind');
     discovery.checks=chosen;discovery.config.checks=chosen.map(row=>row.config);
   }
-  // A sandboxed check is proposed only when it gives the same exit status
-  // inside the sandbox as outside it. Otherwise init writes what it would
+  // A sandboxed check is proposed only when it gives the same exit status and
+  // output inside the sandbox as outside it: first with the checkout's
+  // git-ignored files hidden (sandbox-v2), else with the whole checkout
+  // (sandbox-v1), reusing the plain run. Otherwise init writes what it would
   // have without the sandboxed recipe, with the reason.
+  const trialIssue=trial=>trial.issue?`the sandbox cannot run here (${trial.issue})`:
+    // A failure both ways that shows a refusal cannot be told from a real one.
+    trial.same?null:trial.plain!==trial.sandboxed?`exit ${trial.sandboxed} inside the sandbox, ${trial.plain} outside it${trial.line?` (${trial.line})`:''}`:
+    trial.line?`it fails both ways (exit ${trial.plain}), with a sign of a refusal inside the sandbox (${trial.line})`:
+    `the output differs inside the sandbox: ${JSON.stringify(trial.output.sandboxed)} instead of ${JSON.stringify(trial.output.plain)}`;
   let trials=0;
   for(const [at,row] of [...discovery.checks.entries()]){
     if(row.qualification!==sandboxQualification||row.level!=='ready')continue;
     if(!json)console.error(`Running ${row.config.name} inside and outside the sandbox to confirm it...`);
-    const trial=await sandboxTrial(discovery.root,row.config);
-    if(!trial.issue)trials++;
-    const issue=trial.issue?`the sandbox cannot run here (${trial.issue})`:
-      // A failure both ways that shows a refusal cannot be told from a real one.
-      trial.same?null:trial.plain!==trial.sandboxed?`exit ${trial.sandboxed} inside the sandbox, ${trial.plain} outside it${trial.line?` (${trial.line})`:''}`:
-      trial.line?`it fails both ways (exit ${trial.plain}), with a sign of a refusal inside the sandbox (${trial.line})`:
-      `the output differs inside the sandbox: ${JSON.stringify(trial.output.sandboxed)} instead of ${JSON.stringify(trial.output.plain)}`;
-    if(!issue){row.reason+=`; ran both ways at init, exit ${trial.plain}`;continue;}
+    const hidden=await sandboxTrial(discovery.root,row.config,{hide:true});
+    if(!hidden.issue)trials++;
+    if(!hidden.issue&&hidden.same){
+      row.config={...row.config,qualification:hiddenQualification};row.qualification=hiddenQualification;
+      row.reason=row.reason.replace('the whole checkout is its input',"the checkout's files git doesn't ignore are its input")+`; ran both ways at init, exit ${hidden.plain}`;continue;}
+    // Without git, nothing is hidden: the whole checkout is tried on its own.
+    const trial=hidden.plainRun?await sandboxTrial(discovery.root,row.config,{plain:hidden.plainRun}):
+      hidden.issue===hiddenListIssue?await sandboxTrial(discovery.root,row.config):hidden;
+    const issue=trialIssue(trial);
+    if(!issue){row.reason+=`; ran both ways at init, exit ${trial.plain}; it needs files git ignores (${hidden.issue??trialIssue(hidden)})`;continue;}
     if(row.fallback)discovery.checks[at]={...row.fallback,reason:`Sandboxed recipe unavailable: ${issue}; ${row.fallback.reason}`};
     else{discovery.checks[at]=null;discovery.skipped.push({kind:row.config.name,reason:`${row.config.name}: sandboxed recipe unavailable: ${issue}`});}
   }
@@ -310,7 +320,7 @@ const checks=publicConfig.checks.map(check=>{
   const pnpmTypecheck=check.qualification==='pnpm-tsc-v1';
   const testsDirect=check.qualification===testsQualification;
   const generalScript=check.qualification===scriptQualification;
-  const sandboxed=check.qualification===sandboxQualification;
+  const sandboxed=isSandboxed(check.qualification);
   if(check.script&&check.command&&!pnpmTypecheck&&!npmDirect&&!testsDirect)
     error(`${check.name}: choose script or command, not both`);
   const manager=publicConfig.packageManager;
@@ -371,7 +381,7 @@ const checks=publicConfig.checks.map(check=>{
   const windowsContext=process.platform==='win32'?
     ['ComSpec','COMSPEC','PATHEXT','USERPROFILE']:[];
   const autoEnvironment=npmDirect?directEnvironment:testsDirect?testEnvironment:generalScript?scriptEnvironment(manager):
-    sandboxed?sandboxEnvironment(Boolean(pythonTool(check)),check.script?manager:null):npmAutomatic?{variables:['NODE_OPTIONS','NODE_PATH','CI','HOME',
+    sandboxed?sandboxEnvironment(Boolean(pythonTool(check)),check.script?manager:null,check.kind):npmAutomatic?{variables:['NODE_OPTIONS','NODE_PATH','CI','HOME',
     'NODE_ENV','BASH_ENV','ENV',...windowsContext],prefixes:['npm_config_','DYLD_','TSGO_'],
     pathExecutables:['node','npm'],executableIdentity:true}:yarnWorkspace?
     {variables:['NODE_OPTIONS','NODE_PATH','CI','HOME','NODE_ENV','BASH_ENV','ENV',...windowsContext],
@@ -398,12 +408,18 @@ const checks=publicConfig.checks.map(check=>{
     ...(npmDirect?{qualificationInterpretation:directInterpretation}:{}),
     ...(testsDirect?{qualificationInterpretation:testsInterpretation}:{}),
     ...(generalScript?{qualificationInterpretation:scriptInterpretation}:{}),
-    ...(sandboxed?{qualificationInterpretation:sandboxInterpretation}:{}),
+    ...(sandboxed?{qualificationInterpretation:sandboxInterpretationOf(check.qualification)}:{}),
     workspace:check.workspace||null,kind:check.kind||null};
 });
 const runtime={schema:1,provider:'declared-project@1',root,state,checks};
 try{assertOwnedStatePlacement(runtime);}catch(e){error(e.message);}
 const serialized=JSON.stringify(runtime,null,2)+'\n';
+if(command==='stats'){
+  const {readStats,renderStats}=await import('../src/stats.mjs');
+  const value=readStats(state,publicConfig.checks.map(item=>item.name));
+  console.log(json?JSON.stringify(value):renderStats(value));
+  await exit(0);
+}
 const maintenance=command==='stop'||command==='remove-state';
 if(maintenance&&!fs.existsSync(runtimeFile)){
   console.log(`No REDUE state exists for ${configFile}.`);
@@ -443,11 +459,14 @@ const internal=command==='remove-state'?'uninstall':
 // Asked when this part of a command runs, after earlier parts may have
 // changed files. REDUE_FORCE=1 runs it anyway.
 if(ifDue&&process.env.REDUE_FORCE!=='1'){
-  const {verdict}=await import('../src/claude-hook.mjs');
+  const [{verdict},{note,skipNote,dueNote}]=await Promise.all([import('../src/claude-hook.mjs'),import('../src/stats.mjs')]);
   const asked=spawnSync(process.execPath,[fileURLToPath(import.meta.url),'--config',configFile,...(stateOverride?['--state-dir',stateOverride]:[]),
     'status','--sync','--json'],{encoding:'utf8',stdio:['ignore','pipe','ignore'],env:process.env,timeout:600_000});
   let status=null;try{status=JSON.parse(asked.stdout);}catch{}
+  const due=dueNote(status,check);
+  if(due)note(state,due);
   if(verdict(status,check)){
+    note(state,skipNote(status,check,'if-due'));
     const row=status.checks.find(item=>item.name===check);
     if(json)console.log(JSON.stringify({check,skipped:true,result:'PASS',run_id:row.invocation.runId}));
     else console.error(`REDUE: ${check} passed on these exact inputs (run ${row.invocation.runId}); not run again. REDUE_FORCE=1 runs it anyway.`);

package/docs/user/node-script.md

@@ -8,8 +8,8 @@ pnpm and Yarn TypeScript recipes, the [direct test recipe](tests-direct.md)
 and, on macOS and Linux, the [sandboxed recipe](sandbox.md) first. It uses
 the general recipe for a root typecheck or lint script none of them takes,
 and where the sandbox cannot run (at init, or at run time inside Claude
-Code's macOS sandbox). Test scripts use the direct test recipe; `init` does
-not propose the general recipe for them.
+Code's macOS sandbox). Test scripts use the direct test recipe or the
+sandboxed recipe; `init` does not propose the general recipe for them.
 
 It supports a project installed by npm, pnpm 9-12 (isolated or hoisted
 `node_modules`) or Yarn with the `node-modules` linker, with one lockfile kind

package/docs/user/sandbox.md

@@ -1,27 +1,34 @@
-# Sandboxed recipe (typecheck and lint, any language)
+# Sandboxed recipe (typecheck, lint and tests, any language)
 
-The sandboxed recipe runs a typecheck or lint command inside an operating
+The sandboxed recipe runs a typecheck, lint or test command inside an operating
 system sandbox and reports later whether that result still applies to the
 files you have. The sandbox holds for every process the command starts, so it
 covers programs REDUE cannot record: TypeScript 7's native compiler, esbuild,
 oxlint and tsgolint, biome, turbo, and Python tools such as mypy, pyright, ty
-and ruff.
+and ruff, and test runners such as pytest.
 
-**The claim.** A sandboxed result holds for a run that sees only the project,
-its installed packages and the toolchain, the way CI does. It does not say
-that a run with your own configuration files, a service on your machine or the
-network would give the same result.
+**The claim.** A sandboxed result holds for a run that sees only the project's
+files that git doesn't ignore, its installed packages and the toolchain, the
+way CI does. It does not say that a run with your own configuration files, a
+service on your machine or the network would give the same result.
+
+A check that needs files git ignores (generated code, another package's
+built declarations) gets the whole checkout instead: its result holds for a
+run that sees the project, git-ignored files included, its installed packages
+and the toolchain. See [files git ignores](#files-git-ignores).
 
 ## Where it applies
 
+- **Tests:** a root `test` script the [direct test recipe](tests-direct.md)
+  does not take, and pytest in the project's `.venv`. See [tests](#tests).
 - **Node projects:** a root `typecheck` or `lint` script (npm, pnpm, or Yarn
   with the `node-modules` linker). `init` tries the
   [direct TypeScript recipe](direct-typescript.md) and the pnpm and Yarn
   TypeScript recipes first; they stay first where they apply. Then the
   sandboxed recipe, before the [general recipe](node-script.md), since it
-  also covers the native programs a script starts. Where init's trial shows a
-  difference, or the sandbox cannot run, `init` writes the general recipe as
-  before. Where the sandbox cannot start at run time (inside Claude Code's
+  also covers the native programs a script starts. Where init's trials show
+  a difference, or the sandbox cannot run, `init` writes the general recipe
+  as before. Where the sandbox cannot start at run time (inside Claude Code's
   macOS sandbox), a script the general recipe takes is recorded as it records
   it.
 - **Python projects:** mypy, pyright, ty and ruff installed in the project's
@@ -44,12 +51,14 @@ redue status --sync
 A Python check in the config:
 
 ```json
-{"name": "typecheck", "kind": "typecheck", "qualification": "sandbox-v1",
+{"name": "typecheck", "kind": "typecheck", "qualification": "sandbox-v2",
  "command": [".venv/bin/mypy", "src"], "inputs": ["**", "..."], "installedInputs": [".venv/**", "..."]}
 ```
 
 A Node check names its script, as the general recipe's do:
-`{"name": "lint", "script": "lint", "kind": "lint", "qualification": "sandbox-v1", ...}`.
+`{"name": "lint", "script": "lint", "kind": "lint", "qualification": "sandbox-v2", ...}`.
+`sandbox-v2` hides the files git ignores from the run; `sandbox-v1` runs it on
+the whole checkout.
 
 With the Claude Code hook, `uv run mypy src`, `poetry run mypy src`,
 `python -m mypy src`, `.venv/bin/mypy src` and `mypy src` are recognized as
@@ -57,18 +66,40 @@ that check. The last two count only when PATH finds the tool or Python in the
 project's `.venv/bin` first. When the hook runs the check through REDUE, it
 keeps `uv run` or `poetry run` in front, so the environment still syncs first.
 
+## Files git ignores
+
+`init` runs each check three ways at most: plainly, sandboxed with the files
+git ignores hidden, and, when those two differ, sandboxed with the whole
+checkout (reusing the plain run). The first sandboxed way that gives the same
+result is written: `sandbox-v2` or `sandbox-v1`.
+
+With `sandbox-v2`, each run hides what `git ls-files --others --ignored
+--exclude-standard --directory` lists, except `node_modules`, `.venv` and
+runner caches: build output (`dist/`, `.next/`), coverage files, `.env` files.
+Only the project's rules count: its `.gitignore` files and
+`.git/info/exclude`, not your global excludes file. Those files are not
+inputs, so a build or test run that writes them leaves the result CURRENT.
+
+When a `sandbox-v2` run fails with a refusal that names a hidden file (a
+missing file counts only when it names one, since Linux tools report an
+unreadable file as missing), REDUE runs the check again in the sandbox
+without hiding anything. That result is kept and compared on every file,
+git-ignored ones included. Run `redue init` again to write `sandbox-v1` for a
+check that keeps needing them.
+
 ## What the run sees
 
 It may read and execute only beneath:
 
-- the checkout;
+- the checkout, without the files git ignores for `sandbox-v2`;
 - the toolchain: Node's installation and the package managers' (with
   Corepack's downloads), or the Python installation the `.venv` was made from;
 - system directories (`/usr`, `/bin`, `/etc` and the like on Linux; `/usr`,
   `/System`, `/Library`, `/opt/homebrew` and the like on macOS);
 - a private home directory and a fresh temporary directory, in REDUE's state.
 
-It has no network: no TCP connection, loopback included. On macOS its only
+A typecheck or lint has no network: no TCP connection, loopback included
+(a test run gets loopback; see [tests](#tests)). On macOS its only
 Unix sockets are in its own temporary directory, which is new for each run.
 On Linux 6.12 and later it cannot reach an abstract Unix socket outside the
 run.
@@ -96,14 +127,50 @@ It gets a controlled environment, not yours:
 - `HOME`, `TMPDIR` and the XDG directories in its private directories;
   `VIRTUAL_ENV` for a Python check; Corepack's home where Corepack keeps it.
 
+## Tests
+
+A test run gets what a typecheck or lint gets, and also:
+
+- **Loopback, and no remote host.** It may listen on any address and accept
+  connections, and connect only to the local host (`127.0.0.0/8`, `::1`,
+  `0.0.0.0`, `::`) and Unix sockets.
+  - On macOS, Seatbelt enforces that.
+  - On Linux, Landlock cannot tell a loopback address from a remote one, so a
+    seccomp filter in the test's processes does instead. It refuses sockets
+    other than Unix, netlink and TCP when they are created (UDP included),
+    TCP Fast Open (`MSG_FASTOPEN`) and `io_uring`. It hands each `connect` to
+    REDUE's launcher. The launcher allows a local port no process listens on
+    (the test gets "connection refused") or one the run's own processes
+    listen on, and a Unix socket file in the checkout, the run's home or its
+    temporary directory. It refuses a service started outside the run (your
+    local database) and every remote host. Nothing else goes through the
+    launcher, so worker processes talk to each other as usual. If the
+    launcher's supervisor cannot start, or stops before the run ends, the
+    result is UNVERIFIED, never CURRENT.
+- **A fresh home.** Its home directory is new and empty for each run, as on
+  CI: what a test leaves there is not observed, so it must not reach a later
+  run.
+- **A private `/tmp`.** Its `TMPDIR` is its own, new for each run, and it may
+  not write in `/tmp` or `/var/tmp` outside it. A test that writes to a fixed
+  path there is refused.
+
+`init` compares a test run's exit status and the counts its summary prints
+(passed, failed, skipped), not every line: test runners print in the order
+their workers finish. Git-ignored files are hidden from a test run when the
+trial agrees, as for typecheck and lint. A suite that reads back git-ignored
+files it writes in the same run (a coverage database, a build it starts
+itself) gets the whole checkout. Files a hidden run writes beneath hidden
+folders, or new files git ignores, are not changes to its inputs.
+
 ## What REDUE claims
 
 CURRENT means the last sandboxed run's inputs are unchanged:
 
-- every file in the checkout, git-ignored files and git-ignored links to
-  files in it included, except `.git`, `node_modules` and `.venv` (observed
-  as installed files, by identity) and runner caches: the general recipe's,
-  plus `__pycache__`, `.mypy_cache`, `.ruff_cache` and `.pytest_cache`;
+- every file in the checkout git doesn't ignore (`sandbox-v2`), or every file
+  in it, git-ignored files and git-ignored links to files in it included
+  (`sandbox-v1`), except `.git` and runner caches (the general recipe's, plus
+  `__pycache__`, `.mypy_cache`, `.ruff_cache` and `.pytest_cache`);
+- `node_modules` and `.venv`, observed as installed files, by identity;
 - the environment the run gets (without display variables such as `TERM`
   and `COLUMNS`), and the places it may read;
 - Node's and the package managers' executables, or the interpreter the
@@ -111,7 +178,8 @@ CURRENT means the last sandboxed run's inputs are unchanged:
 - every directory on PATH, by identity: installing, removing or upgrading a
   program in one makes the result STALE;
 - `.git/info/exclude` and `.git/config`, by content: ruff and biome read the
-  ignore rules there.
+  ignore rules there, and they decide what `sandbox-v2` hides. For a project
+  in a subdirectory of its repository, also the `.gitignore` files above it.
 
 ## Telling a refusal from a real failure
 
@@ -124,6 +192,9 @@ every failure twice to find out:
   timings, the temporary and home directories, and lines about caches.
   Otherwise `init` writes what it would have without this recipe and says why.
   These are the only runs `init` makes.
+- When a `sandbox-v2` run fails with a refusal that names a hidden file, it
+  runs again in the sandbox without hiding (see
+  [files git ignores](#files-git-ignores)).
 - When a sandboxed run fails and its output has a line that looks like a
   refusal ("Operation not permitted", "Permission denied", "Read-only file
   system", a name lookup or a connection that failed), REDUE runs the check
@@ -133,7 +204,7 @@ every failure twice to find out:
 
 ## The private home
 
-The run's home directory is kept between runs, per check, so caches tools
+A typecheck or lint's home directory is kept between runs, per check, so caches tools
 build there (font caches, a type checker's cache) are not rebuilt every run.
 It starts empty: your configuration files are not in it. What a run leaves
 there can change a later run, and REDUE trusts it like a runner cache. Remove
@@ -167,3 +238,33 @@ it with `redue remove-state`.
   nested example project's).
 - A script that changes files (`--fix`, `--write`) does not get this recipe:
   `init` would run it twice.
+- With `sandbox-v2`, a run cannot read a hidden file, but it can still see
+  that one exists, its size and its times, and write beneath it. On Linux it
+  can also list a hidden folder; on macOS it cannot. A tool that acts on
+  whether a git-ignored file exists, without reading it, is not covered: a
+  new or removed one does not make the result STALE. In a census of 136
+  test suites with ignored files after their build, none did.
+- On Linux, a test run's UDP is refused, loopback included: the filter
+  decides when the socket is made, before its address is known. On macOS UDP
+  to loopback works. A test that needs it fails its trial and keeps another
+  recipe.
+- On macOS, a test run can connect to a service on loopback that it did not
+  start (a local database on `127.0.0.1:5432`), and that service's state is
+  not observed: a change to it does not make the result STALE. On Linux the
+  launcher refuses it.
+- On Linux, the launcher decides a `connect` from the address it reads in
+  the process's memory, and the kernel reads it again after. A process that
+  changes the address in between, from another thread, could reach another
+  host. Honest test code does not.
+- On Linux, a typecheck or lint can still connect to a Unix socket file
+  outside it (a local database's).
+- On Linux, a file a `sandbox-v2` run creates in a folder that holds hidden
+  files (the checkout's root, when anything in it is hidden) cannot be read
+  back in the same run. `init`'s trial and the rerun without hiding catch a
+  check that needs to.
+- A file whose git status changes without a change to its content (`git add
+  -f` of an ignored file, `git rm --cached` of a tracked one) is noticed only
+  when its content next changes. A change to `.gitignore` or
+  `.git/info/exclude` makes the result STALE.
+- A run that moves a hidden folder into a visible place can read it there.
+  That changes the checkout during the run, so the run is not reused.

package/docs/user/supported-projects.md

@@ -19,12 +19,14 @@ build scripts. It prefers familiar script names (for typecheck: `typecheck`,
 `tsc`, Vitest/Jest, ESLint, or Vite commands under other names. Multiple
 matches are reported as ambiguous rather than guessed. It never creates a new
 project script. Without `--recipe`, `init` tries the strict recipes below first, then,
-for a root typecheck or lint script, the [sandboxed recipe](sandbox.md) on
-macOS and Linux (the script run inside an OS sandbox, the whole checkout as
-its input), then the [general recipe](node-script.md): the script run
+for a root typecheck, lint or test script, the [sandboxed recipe](sandbox.md) on
+macOS and Linux (the script run inside an OS sandbox, with the files git
+ignores hidden when it gives the same result that way, else on the whole
+checkout; a test run also gets loopback-only network and a private `/tmp`),
+then, for typecheck and lint, the [general recipe](node-script.md): the script run
 through the package manager with every Node process recorded and the whole
 checkout as its input. Python projects get the sandboxed recipe for mypy,
-pyright, ty and ruff in their `.venv`. `--recipe typescript-direct`,
+pyright, ty, ruff and pytest in their `.venv`. `--recipe typescript-direct`,
 `tests-direct`, `node-script` or `sandbox` selects one recipe alone. The direct TypeScript recipe supports
 a typecheck script of `tsc --noEmit` runs in any argument order, chained runs,
 or no typecheck script at all (a proposed `tsc --noEmit`; see
@@ -55,7 +57,9 @@ typechecks, Vitest/Jest, ESLint, and builds in workspace packages start
 recording-only unless a separately justified explicit contract is provided.
 For a single-package project whose test script runs Vitest 4/5 or Jest 29
 directly, `init` uses the direct test recipe, which records the run and guards
-its inputs; see [direct tests](tests-direct.md).
+its inputs; see [direct tests](tests-direct.md). Other root test scripts get
+the sandboxed recipe when its trial gives the same exit status and test counts
+as a plain run.
 In particular, test caches and build-generated inputs are not excluded to
 make evidence green.
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@redue/cli",
   "private": false,
-  "version": "0.1.0-alpha.26",
+  "version": "0.1.0-alpha.28",
   "publishConfig": {
     "access": "public",
     "tag": "alpha"

package/src/claude-hook.mjs

@@ -13,6 +13,7 @@ import {processAlive} from './process-liveness.mjs';
 import {scriptRunChecks} from './typescript-invocation.mjs';
 import {recipeLabel} from './presentation.mjs';
 import {findExecutable} from './executable-lookup.mjs';
+import {note,skipNote,dueNote} from './stats.mjs';
 
 export const hookCommand='redue agent hook claude';
 // No shell metacharacters; `@` starts scoped package names.
@@ -25,7 +26,8 @@ const tests=check=>check?.qualification==='npm-tests-direct-v1'&&Array.isArray(c
 const general=check=>(check?.qualification==='node-script-v1'&&['typecheck','test','lint'].includes(check.kind)||
   sandboxed(check))&&typeof check.script==='string';
 // A sandboxed check: a root package script, or a tool from the project's .venv.
-const sandboxed=check=>check?.qualification==='sandbox-v1'&&['typecheck','lint'].includes(check.kind);
+// sandbox.mjs's isSandboxed, written out: the hook does not load the sandbox module.
+const sandboxed=check=>['sandbox-v1','sandbox-v2'].includes(check?.qualification)&&['typecheck','lint','test'].includes(check.kind);
 const python=check=>sandboxed(check)&&Array.isArray(check.command)&&/^\.venv\/bin\/[A-Za-z0-9_.-]+$/.test(check.command[0]||'')&&
   check.command.every(word=>typeof word==='string'&&safe.test(word));
 const eligibleCheck=check=>typecheck(check)||tests(check)||general(check)||python(check);
@@ -505,6 +507,9 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
   const project=hookProject(root);
   if(!project)return null;
   const {record,config,parsed}=project;
+  // For `redue stats`: the checks a decision skipped, or the ones that were due.
+  const tally=(status,names,skipped)=>{let state;try{state=projectState(project.projectRoot,config);}catch{return;}
+    for(const name of names){const entry=skipped?skipNote(status,name,'hook'):dueNote(status,name);if(entry)note(state,entry);}};
   const entered=enteredDir(command,root,project.projectRoot);
   if(!entered)return null;
   let scripts={};
@@ -537,6 +542,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
     if(!names)return null;
     const status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,...names)),
       reason=fanoutVerdict(status,names,match.rest);
+    tally(status,names,Boolean(reason));
     return reason?{hookSpecificOutput:{hookEventName:'PreToolUse',permissionDecision:'deny',permissionDecisionReason:reason}}:null;
   }
   // A compiler script: its checks in the order of its runs, or nothing when
@@ -550,6 +556,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
     else{
       const status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,...names)),
         reason=fanoutVerdict(status,names,match.rest);
+      tally(status,names,Boolean(reason));
       if(reason)return {hookSpecificOutput:{hookEventName:'PreToolUse',permissionDecision:'deny',permissionDecisionReason:reason}};
       const pending=names.filter(name=>!verdict(status,name));
       const rewritten=status?.schema===1&&status.observation?.healthy===true&&Array.isArray(status.checks)&&
@@ -561,6 +568,7 @@ export async function claudeHook(input,{entry,timeout,invoke=null}){
   }
   const name=match.check,status=await synchronizedStatus(entry,config,root,timeout??lastRunTimeout(config,parsed,name)),
     reason=verdict(status,name,match.rest);
+  tally(status,[name],Boolean(reason));
   if(reason)return {hookSpecificOutput:{hookEventName:'PreToolUse',
     permissionDecision:'deny',permissionDecisionReason:reason}};
   // Rewrite only on a healthy answer for this check: without an observer,