@redue/cli 0.1.0-alpha.21 → 0.1.0-alpha.22

package/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## 0.1.0-alpha.22 — 2026-10-25
+
+A correctness fix for recorded runs. Earlier alphas could report CURRENT for
+a script that replaced itself with another program.
+
+- **Fixed: a script that replaced itself with another program could be
+  CURRENT.** Node's `process.execve` (Node 22.15 and later) swaps a process
+  for another program under the same PID. The recorder did not see it, so
+  that program ran unseen. TypeScript 7's `tsc` does this to start its
+  native compiler, so alphas 19 to 21 could report a TypeScript 7 typecheck
+  as CURRENT while REDUE saw none of what the compiler did. The general
+  recipe and the direct test recipe now judge the new program like any
+  program a run starts: TypeScript 7's `tsc` makes the run UNVERIFIED and
+  names it. Support for TypeScript 7 is planned.
+- Runs that use Node's bindings with file, network or program access
+  (`process.binding('fs')`, `'spawn_sync'` and others), or run Node with
+  `--expose-internals`, are UNVERIFIED: the recorder cannot follow them.
+- Direct test recipe: a shell a test starts is read by its command, as in the
+  general recipe. One that runs another program before it becomes Node makes
+  the run UNVERIFIED.
+- General recipe, on Linux: pnpm 12 writes a file with no name in
+  `node_modules` when it rewrites its workspace state, which left the first
+  run after an install UNVERIFIED. That file now counts as the run's output
+  when the pnpm executable ran and no file has its name. Known limit: another
+  program's file with no name there, during such a run, is taken as pnpm's.
+- UNVERIFIED reasons for an input that changed during a run name up to five
+  files and a count of the rest.
+
+- Contributors: the census traces each run on Linux and flags programs the
+  recorder missed.
+
 ## 0.1.0-alpha.21 — 2026-10-25
 
 `init` finds more typecheck scripts and no longer stops in large monorepos.

package/docs/user/node-script.md

@@ -69,12 +69,16 @@ anywhere in it.
 
 Two installed files are rewritten by tools during a run. pnpm rewrites
 `node_modules/.pnpm-workspace-state-v1.json` at the first run after an
-install, through a temporary `node_modules/.tmpXXXXXX` file. vue-tsc writes
+install, through a temporary `node_modules/.tmpXXXXXX` file, and on Linux
+writes a file with no name there (`O_TMPFILE`), whose writes the file
+watcher reports as `node_modules/#NUMBER`. vue-tsc writes
 `node_modules/.vue-global-types/*.d.ts` before it reads them. These files stay
 inputs. When they are the only files a run changed, and the run wrote them,
 the result is kept with the files as the run left them. A recorded process
 counts as writing a file it wrote; for pnpm's files, the pnpm executable must
-have run. A later change to them makes the result STALE.
+have run. A `#NUMBER` event counts only directly in the root `node_modules`,
+on Linux, when no file has that name. A later change to them makes the
+result STALE.
 
 It does not mean a rerun would give the same result: a test that depends on
 the time or on randomness can pass now and fail later on the same inputs.
@@ -107,6 +111,10 @@ one REDUE can account for:
   content;
 - `ps -A -o ppid,pid`, which pnpm loaded into Corepack runs.
 
+A process that replaces itself with another program through
+`process.execve` (TypeScript 7's `tsc` does this to start its native
+compiler) is judged the same way as a program it starts.
+
 Anything else makes the result UNVERIFIED and names the program: `git`,
 `sleep`, `find`, and native programs from installed packages such as esbuild,
 turbo, tsgo, tsgolint and biome, whose reads, writes and network use REDUE
@@ -142,11 +150,19 @@ inputs.
   run (pnpm 12 does this) can be reported by the file watcher as an event on
   the root itself, an observation gap that makes the run UNVERIFIED. Linux
   reports the file, which is not an input.
+- On Linux, any program's file with no name in the root `node_modules`,
+  written during a run that started the pnpm executable, is taken as pnpm's.
+  If that program gives the file a real name (any name, `node_modules`
+  included) and then writes through the open file, REDUE misses those
+  writes. A real file named `node_modules/#NUMBER` that is created and
+  removed within such a run is treated the same way. Outside a pnpm run,
+  these writes leave the run UNVERIFIED.
 - The pnpm executable and the pnpm launchers' helpers are trusted as
   toolchain: what they read is not checked.
 - Code that copies the environment and reads the copy, and native code that
-  reads files or variables (native addons such as Rust-based resolvers), is
-  not seen.
+  reads files or variables or starts programs (native addons such as
+  Rust-based resolvers), is not seen. Neither are files opened by
+  `node:sqlite` or preopened for `node:wasi`.
 - A project whose `.npmrc` sets `node-options` stays UNVERIFIED: npm replaces
   `NODE_OPTIONS` with it, which drops REDUE's recorder.
 - A script that runs a build first (`npm run build && tsc --noEmit`) usually

package/docs/user/tests-direct.md

@@ -80,7 +80,14 @@ kept but marked UNVERIFIED, with the reason, when the run:
 - connected to anything other than loopback or a local socket, or asked DNS
   about a name other than localhost;
 - started a process that did not load the recorder: a shell, a binary, or a
-  Node process with its own environment;
+  Node process with its own environment. A process that replaces itself with
+  another program (`process.execve`) counts as starting it, and it must
+  become a recorded Node process. A shell is read by its command, as the
+  [general recipe](node-script.md#programs) reads it: one that runs another
+  program before it becomes Node counts too;
+- used one of Node's bindings with file, network or program access
+  (`process.binding('fs')`, `'spawn_sync'` and others), or ran Node with
+  `--expose-internals`: the recorder cannot follow those;
 - had an installed file with another hard link, other than a link into the
   pnpm store the install recorded in `node_modules/.modules.yaml` (pnpm's
   default on Linux). A change through a link elsewhere, such as Yarn's

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@redue/cli",
   "private": false,
-  "version": "0.1.0-alpha.21",
+  "version": "0.1.0-alpha.22",
   "publishConfig": {
     "access": "public",
     "tag": "alpha"

package/src/cli.mjs

@@ -5,7 +5,7 @@ import {directQualification,directInterpretation,observedNpmContext} from './dir
 import {directRunCertificateAllowed,directRunCertificate,directRunRecordable,directRunRecording,probeQuestions,validateRunRecord} from './direct-run-certificate.mjs';
 import {queryAnswer,runFactChange} from './typescript-list.mjs';
 import {changedFiles,coverageReadable,gitIgnored,recordedEnvironment,testEnvironmentIssue,testGuardConfig,testRunGuard,testRunFacts,sharedInstalledFile,testsInterpretation,testsQualification} from './test-direct.mjs';
-import {childProgram,lookup,pnpmConfigFiles,scriptCaches,scriptInstalledPaths,scriptInterpretation,scriptQualification,scriptRunFacts,toolOutput,toolchainDirs} from './node-script.mjs';
+import {childProgram,lookup,pnpmConfigFiles,scriptCaches,scriptInstalledPaths,scriptInterpretation,scriptQualification,scriptRunFacts,pnpmNameless,toolOutput,toolchainDirs} from './node-script.mjs';
 import path from 'node:path';
 import net from 'node:net';
 import os from 'node:os';
@@ -400,7 +400,7 @@ async function execute() {
     // what was there (test/tmp). When they are the only observed change, the
     // run is kept with the end checkpoint, which includes them.
     let outputs=null;
-    let eventsExplained=false;
+    let eventsExplained=false,unexplained=end?.inputEventPaths||null;
     if(guard&&snap&&end&&snap.planId===end.planId&&snap.probeHash===end.probeHash&&snap.observationHealthy&&end.observationHealthy&&
       (snap.inputEventSerial===end.inputEventSerial||end.inputEventPathsComplete===true)){
       // A directory event is not an input change; a change to its files shows
@@ -408,9 +408,10 @@ async function execute() {
       const directory=rel=>{try{return fs.lstatSync(path.join(root,rel)).isDirectory();}catch{return false;}};
       const changed=changedFiles(snap.files,end.files),events=(end.inputEventPaths||[]).filter(rel=>!directory(rel));
       const touched=[...new Set([...changed,...events])].sort();
-      const output=rel=>guard.isOutput(rel)||general&&toolOutput(rel,guard);
+      const output=rel=>guard.isOutput(rel)||general&&(toolOutput(rel,guard)||pnpmNameless(rel,guard,root,[snap.files,end.files]));
       const ignored=touched.length&&touched.every(output)?gitIgnored(root,touched):new Set();
-      if(touched.every(file=>ignored.has(file))){eventsExplained=true;if(changed.length)outputs=changed;}
+      unexplained=touched.filter(file=>!ignored.has(file));
+      if(!unexplained.length){eventsExplained=true;if(changed.length)outputs=changed;}
     }
     // The end certificate, recorded or probed, must cover what this compile received.
     // With no input event, gap or content change in between, it also holds at the start.
@@ -442,8 +443,13 @@ async function execute() {
       captureIssues.push('observer generation changed during execution');
     if(snap?.planGeneration!==end?.planGeneration)
       captureIssues.push('input plan rebuilt during execution');
-    if(snap?.inputEventSerial!==end?.inputEventSerial&&!eventsExplained)
-      captureIssues.push('declared input event occurred during execution');
+    // The reason names up to five files the run did not explain, and a count of the rest.
+    if(snap?.inputEventSerial!==end?.inputEventSerial&&!eventsExplained){
+      const named=[...new Set(unexplained||[])],more=named.length-5,
+        unrecorded=end?.inputEventPathsComplete===false?' (not every event path was kept)':'';
+      captureIssues.push('declared input event occurred during execution'+(named.length?
+        `: ${named.slice(0,5).join(', ')}${more>0?` and ${more} more`:''}`:'')+unrecorded);
+    }
     // A recorded certificate changes only the probe part of the fingerprint.
     if(!outputs&&(!snap||!end||snap.planId!==end.planId||(recording?snap.inputKey!==end.inputKey:snap.fingerprint!==end.fingerprint)||
       snap.revision!==end.revision))captureIssues.push('declared inputs changed or were not observed');

package/src/node-script.mjs

@@ -57,6 +57,15 @@ export const toolOutputPath=rel=>pnpmState(rel)||pnpmTemp(rel)||vueTypes(rel);
 // The run wrote it: a recorded process did, or, for pnpm's files, the pnpm
 // executable ran, which writes them natively.
 export const toolOutput=(rel,guard)=>toolOutputPath(rel)&&(guard.wroteInstalled(rel)||(pnpmState(rel)||pnpmTemp(rel))&&guard.opaque);
+// On Linux, pnpm 12 also writes a file with no name (O_TMPFILE) in the root
+// node_modules, whose writes inotify reports as '#<inode>'. Such an event is
+// the run's output when the pnpm executable ran and no file has that name, at
+// either checkpoint or now. Limit: another program's file with no name there,
+// during a run that started the pnpm executable, is taken as pnpm's too; once
+// linked in under any name, its writes through the open file are missed.
+export const pnpmNameless=(rel,guard,root,checkpoints,platform=process.platform)=>platform==='linux'&&/^node_modules\/#\d+$/.test(rel)&&
+  guard.opaque&&checkpoints.every(files=>files&&!Object.hasOwn(files,rel))&&
+  (()=>{try{fs.lstatSync(path.join(root,rel));return false;}catch(error){return error.code==='ENOENT';}})();
 // The installed paths the recorder treats as observed by identity.
 export const scriptInstalledPaths=(root,dirs=[])=>[...modulesDirs(dirs),...installState].map(at=>path.join(root,at));
 // Variables a recorded run cannot be seen reading: Node's and libc's (as in

package/src/test-direct.mjs

@@ -16,6 +16,8 @@ import {queryAnswer} from './typescript-list.mjs';
 import {pnpmInstallInfo,pnpmProjectInfo,yaml} from './pnpm-inputs.mjs';
 import {workspaceManifests} from './workspace-manifests.mjs';
 import {atomicJson} from './state-store.mjs';
+// node-script.mjs imports this module too; childProgram is only called at run time.
+import {childProgram} from './node-script.mjs';
 import {racyUntil} from './path-identity.mjs';
 
 const digest=value=>createHash('sha256').update(value).digest('hex');
@@ -73,7 +75,7 @@ export function testRunGuard(dir,{root,sourcePatterns,files,runner,environment=t
   const issues=[],add=issue=>{if(!issues.includes(issue))issues.push(issue);};
   const included=patternMatcher(sourcePatterns);
   const observed=rel=>Object.hasOwn(files||{},rel);
-  const headers=new Map(),records=[];
+  const headers=new Map(),records=[],headerCount=new Map(),threadsStarted=new Map();
   let names=[];
   try{names=fs.readdirSync(dir).filter(name=>name.endsWith('.jsonl'));}
   catch{return {issues:['the run record is unavailable'],runFacts:[],environmentReads:{},isOutput:()=>false};}
@@ -84,6 +86,9 @@ export function testRunGuard(dir,{root,sourcePatterns,files,runner,environment=t
     const header=rows[0]?.[0]==='H'?rows[0][1]:null;
     if(!header||!Number.isSafeInteger(header.pid)||!Number.isSafeInteger(header.threadId)){add('a run record is unreadable');continue;}
     headers.set(`${header.pid}-${header.threadId}`,header);
+    // After process.execve, the new program's threads append to the same
+    // records: each thread started needs its own header.
+    headerCount.set(`${header.pid}-${header.threadId}`,rows.filter(row=>row?.[0]==='H'&&!row[1]?.internal).length);
     records.push({header,rows:rows.slice(1)});
   }
   let opaque=false;
@@ -101,7 +106,24 @@ export function testRunGuard(dir,{root,sourcePatterns,files,runner,environment=t
   // PIDs is that child, not an execSync child (which reports no PID).
   const installedWrites=new Set();
   const reads=new Map(),produced=new Set(),created=new Set(),environmentReads={},spawned=new Map(),childPids=new Map();
+  // Whether a started program is accounted for: {issue} or {needsHeader}.
+  // The tests recipe wants every program to become a recorded process; a
+  // shell, which can run other programs first, is read by its command as
+  // the general recipe reads it.
+  const judge=(child,main)=>{
+    if(!programs){
+      const judged=/^(sh|bash|dash|zsh|ksh|fish|csh|tcsh)$/.test(path.basename(String(child.file)))?childProgram(child,{parentMain:main}):{};
+      return judged.issue?judged:{needsHeader:true};}
+    const judged=programs.judge(child,main);
+    if(!judged.issue)opaque||=Boolean(judged.opaque);
+    return judged;
+  };
   for(const {header,rows} of records){
+    // A program this process replaced itself with (process.execve) that must
+    // load the recorder: its header follows in this record, from the same
+    // parent. Later rows come from the program now running. (A reused PID
+    // also appends a header to an old record.)
+    let replaced=null,main=header.main??null;
     for(const row of rows){
       const [kind,...value]=Array.isArray(row)?row:[];
       if(kind==='U')add(`recorder: ${value[0]}`);
@@ -121,16 +143,26 @@ export function testRunGuard(dir,{root,sourcePatterns,files,runner,environment=t
         if(kind==='C')childPids.set(header.pid,[...childPids.get(header.pid)||[],value[0]]);
         // The general recipe accounts for each program; one that needs no
         // recorder (a shell, the pnpm executable) is not matched to a header.
-        if(programs){const judged=programs.judge({file,args,cwd,searchPath,env,script},header.main??null);
-          if(judged.issue){add(judged.issue);continue;}
-          opaque||=Boolean(judged.opaque);
-          if(!judged.needsHeader)continue;}
+        const judged=judge({file,args,cwd,searchPath,env,script},main);
+        if(judged.issue){add(judged.issue);continue;}
+        if(!judged.needsHeader)continue;
         const list=spawned.get(header.pid)||[];list.push({pid:kind==='C'?value[0]:null,file});
         spawned.set(header.pid,list);}
-      else if(kind==='T'){if(!headers.has(`${header.pid}-${value[0]}`))add('a worker thread did not load the REDUE recorder');}
-      else if(kind!=='H')add('a run record is unreadable');
+      else if(kind==='X'){
+        const [file,args,cwd,searchPath,env,script]=value;
+        const judged=judge({file,args,cwd,searchPath,env,script},main);
+        if(judged.issue){add(judged.issue);replaced=null;continue;}
+        replaced=judged.needsHeader?file:null;}
+      else if(kind==='Y')replaced=null;
+      else if(kind==='T'){const key=`${header.pid}-${value[0]}`;threadsStarted.set(key,(threadsStarted.get(key)||0)+1);}
+      else if(kind==='H'){
+        if(replaced!==null&&value[0]?.ppid!==header.ppid)add(`replaced itself with ${path.basename(String(replaced))}, which REDUE does not record`);
+        replaced=null;main=value[0]?.main??null;}
+      else add('a run record is unreadable');
     }
+    if(replaced!==null)add(`replaced itself with ${path.basename(String(replaced))}, which REDUE does not record`);
   }
+  for(const [key,count] of threadsStarted)if((headerCount.get(key)||0)<count)add('a worker thread did not load the REDUE recorder');
   // Every child must be a recorded Node process: by PID when known, otherwise
   // by the number of recorded processes whose parent started it.
   for(const [parent,children] of spawned){

package/src/test-recorder.cjs

@@ -15,7 +15,7 @@ if(configFile)install();
 function install(){
   const original={openSync:fs.openSync,writeSync:fs.writeSync,readFileSync:fs.readFileSync,
     statSync:fs.statSync,readdirSync:fs.readdirSync,real:fs.realpathSync.native};
-  const {threadId}=require('node:worker_threads');
+  const {threadId,isInternalThread}=require('node:worker_threads');
   let config,fd;
   try{config=JSON.parse(original.readFileSync(configFile,'utf8'));
     fd=original.openSync(path.join(config.dir,`${process.pid}-${threadId}.jsonl`),'a',0o600);}
@@ -25,7 +25,10 @@ function install(){
   const failed=reason=>emit(['U',reason]);
   // The main script tells the guard which program a process runs (a package
   // manager may start helpers of its own).
-  emit(['H',{pid:process.pid,ppid:process.ppid,threadId,main:typeof process.argv[1]==='string'?process.argv[1]:null}]);
+  // Node's own threads (the ESM hooks thread) load it too; the guard does not
+  // count them as threads the run started.
+  emit(['H',{pid:process.pid,ppid:process.ppid,threadId,main:typeof process.argv[1]==='string'?process.argv[1]:null,
+    ...(isInternalThread===true?{internal:true}:{})}]);
   // Every node_modules observed as installed: the root's and each workspace package's.
   const sep=path.sep,root=config.root,installed=Array.isArray(config.installed)?config.installed:[path.join(root,'node_modules')],
     tmp=config.tmp;
@@ -395,6 +398,32 @@ function install(){
         catch{failed('recorder failure');}}
     };
   }
+  // process.execve replaces this process with another program, under the
+  // same PID: the row comes first, since a successful call does not return
+  // (a failed one aborts the process). Node refuses it in a worker and for
+  // bad arguments by throwing; the row is then withdrawn. The kernel does not
+  // search PATH, so the file is taken from the working directory. A Node
+  // program that replaces this one loads the recorder again and appends a
+  // header to the same record.
+  if(typeof process.execve==='function'){const execve=process.execve;
+    process.execve=function(file,args,env,...rest){
+      let row=false;
+      if(threadId===0)try{const options={env:env??process.env},full=path.resolve(String(file)),where_=where(options);
+        emit(['X',full,argsOf(Array.isArray(args)?args.slice(1):[]),...where_,scriptOf(full,where_[0],null)]);row=true;}
+      catch{failed('recorder failure');}
+      try{return execve.call(this,file,args,env,...rest);}
+      catch(error){if(row)emit(['Y']);throw error;}};}
+  // Bindings that read files, use the network or start programs out of the
+  // recorder's sight. Only bindings without such access are allowed.
+  const harmlessBindings=new Set(['async_wrap','buffer','config','constants','contextify','crypto','http_parser','icu',
+    'js_stream','natives','os','signal_wrap','stream_wrap','tty_wrap','url','util','uv','v8','zlib']);
+  if(typeof process.binding==='function'){const binding=process.binding;
+    process.binding=function(name,...rest){
+      const result=binding.call(this,name,...rest);
+      if(!harmlessBindings.has(name))failed(`process.binding('${String(name)}') reads files, uses the network or starts programs out of REDUE's sight`);
+      return result;};}
+  // Node's internal modules include child_process's unhooked internals.
+  if(process.execArgv.some(arg=>/^--expose[-_]internals(=|$)/.test(arg)))failed('a Node process exposed its internal modules (--expose-internals)');
   const workers=require('node:worker_threads'),Worker=workers.Worker;
   workers.Worker=class extends Worker{
     constructor(...args){super(...args);try{emit(['T',this.threadId]);}catch{failed('recorder failure');}}