@decohere/flux 0.0.1-dev.112 → 0.0.1-dev.114

package/README.md

@@ -109,19 +109,19 @@ export default {
 };
 ```
 
-These are three ways to start the same serve (flags combine; `flux serve --project my-api --reboot-debounce-ms 2000` is valid):
+These are three ways to start the same serve (flags combine; `flux serve --project my-api --reboot-debounce-ms 500` is valid):
 
 ```bash
 flux serve
 flux serve --project my-api
-flux serve --reboot-debounce-ms 2000
+flux serve --reboot-debounce-ms 500
 ```
 
 | | What it does |
 |---|---|
 | `flux serve` | Boots the first discovered project (Mesh lists packages by `package.json` `name`, A→Z). |
 | `--project my-api` | Boots that package by name. |
-| `--reboot-debounce-ms 2000` | Same serve, but waits 2s after the last save before rebooting (default 250ms). |
+| `--reboot-debounce-ms 500` | Same serve, but waits 500ms after the last save before rebooting (default 250ms). |
 
 Use `--project` or [directory slices](#directory-slice-flux-serve-token) when the first package is not the app you want.
 
@@ -132,10 +132,25 @@ Ctrl+C (SIGINT) or SIGTERM (Docker, CI, process managers) disposes the entry, th
 | `[slices…]` | — | One or more module tokens among discovered packages (union; see **Directory slice**) |
 | `--configuration <name>` | `debug` | Build configuration name (not a file path) |
 | `--project <name>` | first discovered package | Single package name to serve (not with slices) |
-| `--reboot-debounce-ms <ms>` | `250` | Quiet window after the last affecting save before a reboot. Use ~2000 when an agent writes many files in sequence. |
+| `--reboot-debounce-ms <ms>` | `250` | Quiet window after the last affecting save before a reboot. See [Reboot debounce](#reboot-debounce) before raising it. |
 
 On a watched change that reaches a served entry, Flux invalidates that subgraph, calls `asyncDispose` on the previous default export, and re-imports. Rapid saves are trailing-debounced and serialized (overlapping dispose cannot run). A change that never reaches an entry does nothing. Each reboot logs `flux reboot` / `flux ready` at info (project names, triggering files, elapsed ms) so the event stays visible when the app boot is chatty. File-watch detail stays at trace.
 
+### Reboot debounce
+
+Saves less than the window apart (250ms by default) become one reboot. Saves further apart each reboot, and a save during a reboot folds into one follow-up reboot. Whatever the timing, the app ends up on the last save; a reboot in between may fail (an import not yet updated) and logs the error.
+
+The window adds its length to every reload, including a single save. Measured on a rename across six files (#484):
+
+| Gap between saves | Reboots at 250ms | Reboots at 2000ms |
+|---|---|---|
+| 0–100ms: parallel edits by an agent, a formatter, a checkout | 1 | 1 |
+| 400ms–1.5s | 6 | 1 |
+| 4s: an agent's edits made one tool call at a time | 6 | 6 |
+| Time from the last save to the app ready | ~0.3s | ~2s |
+
+So keep the default. An agent's saves in one go land inside it, and its one-at-a-time saves are seconds apart, beyond any window short enough for a person's save. Raise it only for a tool you know saves at a steady pace inside your chosen window.
+
 ### Dispose on reboot
 
 `serve` loads the entry’s **default export**. If that object implements `Symbol.asyncDispose`, Flux calls it:
@@ -200,6 +215,17 @@ Foreign scripts win over `main.ts` so Vite (and similar) apps are never loaded i
 
 Logs name skipped libraries and deferred packages (script + cwd). Child processes are stopped when the CLI exits (SIGINT or SIGTERM → dispose).
 
+## Compile cache
+
+Flux keeps the code it compiles on disk, so starting again compiles only what changed (#514). On a cold start with nothing changed, it reads every file from the cache instead of compiling it (`pnpm bench flux-load`, `flux.load.cold-cached`).
+
+- **Where:** `<workspace>/node_modules/.cache/decohere/flux`. When the workspace has no `node_modules`, it's under `$XDG_CACHE_HOME/decohere`, then the OS temp directory.
+- **Settings:** `DECOHERE_CACHE_DIR=<dir>` moves the cache; `DECOHERE_CACHE=0` turns it off.
+- **When it's used:** entries are keyed by the file's path and source, the SWC options it compiles with and SWC's version. Any change to those is a miss, never stale code.
+- **Concurrency:** entries are written to a temp file and renamed into place, so processes starting at once share one cache safely.
+- **Housekeeping:** entries untouched for 30 days are pruned, at most once a day. To clear the cache, delete the folder. `forge clean` leaves it alone, because cleaning runs before every build.
+- **If it can't be used:** when flux can't read or write the folder, it warns once and runs without the cache.
+
 ## Limits
 
 Flux does not support `createRequire`, worker threads, or browser HMR. Reloads always dispose and re-import the entry — there is no `import.meta.hot.accept()` boundary yet.
@@ -214,7 +240,7 @@ Only source files (`.ts`) that a served entry imports trigger a reboot.
 
 **Memory grows over a long session.** Each reboot imports changed modules under a new `?version=` URL, and Node's ES module cache never releases the old versions. That's fine for a working day; if a session runs for days with thousands of reboots, restart it.
 
-Flux uses **Forge’s compiler**, with one SWC config per project built once per run. SWC leaves imports as written; Flux's resolve hook maps each one to its `.ts` source: `@scope/pkg` and `@scope/pkg/sub` through the Mesh aliases (and tsconfig `paths`), `./file.js` and `./file` to `file.ts`, and `./dir` to `dir/index.ts`. Doing it in one place, instead of in SWC (`jsc.paths`, `resolveFully`) and again in the hook, made loading about 20% faster once Node and SWC are warm (`pnpm bench flux-load`, #513). Before Node imports an entry, Flux compiles it and the files it imports, following their imports out from the entry, in parallel on SWC's thread pool (#661). The load hook is synchronous and would otherwise compile one file at a time; files the app never imports, such as tests, are not compiled. A file that fails to compile there is compiled again when Node loads it, and that load fails with the file's name, as before. That is a debug rule so you can edit a library and hit HMR without a pre-build or `node_modules` workspace symlink. Release still resolves through real `node_modules` (published `exports` / `main`).
+Flux uses **Forge’s compiler**, with one SWC config per project built once per run. SWC leaves imports as written; Flux's resolve hook maps each one to its TypeScript source: `@scope/pkg` and `@scope/pkg/sub` through the Mesh aliases (and tsconfig `paths`), `./file.js` and `./file` to `file.ts` or `file.tsx`, `./file.mjs` to `file.mts`, `./file.cjs` to `file.cts` (which runs as CommonJS), and `./dir` to `dir/index.ts`. Doing it in one place, instead of in SWC (`jsc.paths`, `resolveFully`) and again in the hook, made loading about 20% faster once Node and SWC are warm (`pnpm bench flux-load`, #513). Before Node imports an entry, Flux compiles it and the files it imports, following their imports out from the entry, in parallel on SWC's thread pool (#661). The load hook is synchronous and would otherwise compile one file at a time; files the app never imports, such as tests, are not compiled. A file that fails to compile there is compiled again when Node loads it, and that load fails with the file's name, as before. That is a debug rule so you can edit a library and hit HMR without a pre-build or `node_modules` workspace symlink. Release still resolves through real `node_modules` (published `exports` / `main`).
 
 Why aliases, and a tree of where a symlink misses `src/index.ts`: [Forge — debug path aliases](../forge/README.md#debug-path-aliases-not-node_modules-links).
 
@@ -234,7 +260,7 @@ const server = await flux.serve({
   configuration: "debug",
   project: "my-api", // optional single package
   // projects: ["api", "svc"], // optional multi-entry
-  rebootDebounceMs: 2000, // optional; default 250
+  rebootDebounceMs: 500, // optional; default 250
 });
 server.on("reboot", () => {}); // after dispose + boot attempt for affected entries
 await server.dispose();

package/commonjs/src/rewrite-serve-specifier.js

@@ -30,16 +30,18 @@ function rewriteServeSpecifier(options) {
     return null;
 }
 /**
- * The watched `.ts` file a path names: itself (with `.js` read as `.ts`, or `.ts` added), else
- * the `index.ts` of the directory it names. A file wins over a directory of the same name.
+ * The watched source file a path names: itself, or the source its extension stands for (`.js` is
+ * `.ts` or `.tsx`, `.mjs` is `.mts`, `.cjs` is `.cts`, none adds `.ts` or `.tsx`), else the
+ * `index` source of the directory it names. A file wins over a directory of the same name.
  */ function watchedTs(filePath, isWatchedTs) {
     const withoutSlash = filePath.replace(/[/\\]+$/, "");
-    const stem = withoutSlash.replace(/\.(js|ts)$/, "");
-    // From the stem: Mesh's alias lookup has already added `.ts` to a subpath that names a
-    // directory (`@lib/models` arrives as `…/models.ts`).
+    // Mesh's alias lookup may have added `.ts` to a subpath that names a directory (`@lib/models`
+    // arrives as `…/models.ts`), so the directory is the path without any extension.
+    const directory = withoutSlash.replace(/\.([mc]?[jt]sx?)$/, "");
     const candidates = [
-        `${stem}.ts`,
-        _nodepath.default.join(stem, "index.ts")
+        ..._mesh.sourceExtensions.candidates(withoutSlash),
+        ..._mesh.sourceExtensions.candidates(directory),
+        ..._mesh.sourceExtensions.candidates(_nodepath.default.join(directory, "index"))
     ];
     return candidates.find((candidate)=>isWatchedTs(candidate)) ?? null;
 }

package/commonjs/src/source-module-loader.js

@@ -8,6 +8,7 @@ Object.defineProperty(exports, "SourceModuleLoader", {
         return SourceModuleLoader;
     }
 });
+const _nodefs = require("node:fs");
 const _nodemodule = require("node:module");
 const _nodepath = /*#__PURE__*/ _interop_require_default(require("node:path"));
 const _nodeurl = require("node:url");
@@ -18,6 +19,7 @@ const _io = require("@decohere/nexus/io");
 const _scribe = require("@decohere/scribe");
 const _rewriteservespecifier = require("./rewrite-serve-specifier.js");
 const _staticimports = require("./static-imports.js");
+const _transpilediskcache = require("./transpile-disk-cache.js");
 function _interop_require_default(obj) {
     return obj && obj.__esModule ? obj : {
         default: obj
@@ -81,8 +83,10 @@ class SourceModuleLoader {
     transpileCache = new Map();
     /** Bumped when a file's cached code is dropped, so an up-front compile already running can't put it back. */ transpileVersions = new Map();
     /** The source files each entry's walk reached, so a reboot only walks out from what changed. */ reachedFrom = new Map();
+    /** Compiled code on disk across runs (#514); null when caching is off. */ diskCache = null;
     /** Up-front compiles in flight, shared by entries booting at the same time. */ compilingAhead = new Map();
     /** SWC options per project, built on first use and kept until the next `configure()`. */ swcConfigs = new Map();
+    commonJsConfigs = new Map();
     aliasPaths = {};
     aliasBaseUrl = _nodepath.default.resolve(".");
     versionFileUrl = (fileUrl)=>fileUrl;
@@ -90,9 +94,6 @@ class SourceModuleLoader {
     getWorkspace() {
         return this.workspace;
     }
-    getBuildConfig() {
-        return this.buildConfig;
-    }
     hasSourceFile(absolutePath) {
         return this.fileInfoMap.has(absolutePath);
     }
@@ -110,11 +111,15 @@ class SourceModuleLoader {
         this.transpileCache = new Map();
         this.reachedFrom = new Map();
         this.swcConfigs.clear();
+        this.commonJsConfigs.clear();
         this.buildConfig = options.buildConfig;
         this.workspace = options.workspace ?? await (0, _forge.createWorkspace)();
         this.versionFileUrl = options.versionFileUrl ?? ((fileUrl)=>fileUrl);
         this.onResolved = options.onResolved;
         this.fileInfoMap = await this.buildFileInfoMap(this.buildConfig);
+        this.diskCache = _transpilediskcache.TranspileDiskCache.forWorkspace(this.workspace.rootDir);
+        // In the background; it turns the cache off, with a warning, rather than throw.
+        void this.diskCache?.pruneIfDue();
         const aliases = this.workspace.getPackageAliases(this.buildConfig);
         this.aliasBaseUrl = aliases.baseUrl;
         this.aliasPaths = aliases.paths;
@@ -145,7 +150,7 @@ class SourceModuleLoader {
                 const result = this.loadTs(url);
                 if (result != null) {
                     return {
-                        format: "module",
+                        format: result.format,
                         source: result.source,
                         shortCircuit: true
                     };
@@ -163,9 +168,6 @@ class SourceModuleLoader {
         for (const fullPath of absolutePaths){
             this.transpileCache.delete(fullPath);
             this.transpileVersions.set(fullPath, (this.transpileVersions.get(fullPath) ?? 0) + 1);
-            this.transpiler.invalidateContentCache(new _io.FileEx({
-                fullPath
-            }));
         }
     }
     dispose() {
@@ -173,6 +175,7 @@ class SourceModuleLoader {
         this.transpileCache = new Map();
         this.reachedFrom = new Map();
         this.swcConfigs.clear();
+        this.commonJsConfigs.clear();
         this.fileInfoMap.clear();
         this.workspace?.[Symbol.dispose]();
         this.workspace = undefined;
@@ -233,14 +236,12 @@ class SourceModuleLoader {
         if (inFlight != null) return inFlight;
         const version = this.transpileVersions.get(fullPath);
         const cache = this.transpileCache;
-        const compiling = this.transpiler.transformFileAsync(new _io.FileEx({
-            fullPath
-        }), _forge.BuildTargets.es6, {}, this.swcConfigFor(fileInfo.project)).then((response)=>{
-            if (response.outcome !== "success") return undefined;
+        const compiling = this.compileAsync(fullPath, fileInfo.project).then((code)=>{
+            if (code == null) return undefined;
             // Saved, reconfigured or disposed while compiling: don't cache stale code.
             const current = !this.disposed && cache === this.transpileCache && version === this.transpileVersions.get(fullPath);
-            if (current) cache.set(fullPath, response.code);
-            return response.code;
+            if (current) cache.set(fullPath, code);
+            return code;
         }).finally(()=>this.compilingAhead.delete(fullPath));
         this.compilingAhead.set(fullPath, compiling);
         return compiling;
@@ -262,29 +263,95 @@ class SourceModuleLoader {
         const cleanPath = (0, _nodeurl.fileURLToPath)(url);
         const fileInfo = this.fileInfoMap.get(cleanPath);
         if (fileInfo == null) return null;
+        const format = _mesh.sourceExtensions.isCommonJs(cleanPath) ? "commonjs" : "module";
         const cached = this.transpileCache.get(cleanPath);
         if (cached != null) return {
-            source: cached
+            source: cached,
+            format
         };
+        const code = this.compileSync(cleanPath, fileInfo.project);
+        this.transpileCache.set(cleanPath, code);
+        return {
+            source: code,
+            format
+        };
+    }
+    /**
+	 * Compiles a file for the load hook, through the disk cache (#514). Throws, naming the file,
+	 * when it can't be read or doesn't compile.
+	 */ compileSync(fullPath, project) {
         const file = new _io.FileEx({
-            fullPath: cleanPath
+            fullPath
         });
-        const response = this.transpiler.transformFileSync(file, _forge.BuildTargets.es6, {}, this.swcConfigFor(fileInfo.project));
+        const baseConfig = this.swcConfigFor(project, fullPath);
+        let source;
+        try {
+            source = (0, _nodefs.readFileSync)(fullPath, "utf8");
+        } catch (error) {
+            this.logger.error(`Failed to transpile ${fullPath}`, error);
+            throw new Error(`Failed to transpile ${fullPath}`, {
+                cause: error
+            });
+        }
+        const key = this.diskCache?.keyFor(fullPath, source, baseConfig);
+        const hit = key == null ? undefined : this.diskCache?.readSync(key);
+        if (hit != null) return hit;
+        const response = this.transpiler.transformCodeSync(source, file, _forge.BuildTargets.es6, baseConfig);
         if (response.outcome !== "success") {
-            this.logger.error(`Failed to transpile ${cleanPath}`, response);
-            throw new Error(`Failed to transpile ${cleanPath}`);
+            this.logger.error(`Failed to transpile ${fullPath}`, response);
+            throw new Error(`Failed to transpile ${fullPath}`);
         }
-        this.transpileCache.set(cleanPath, response.code);
-        return {
-            source: response.code
-        };
+        if (key != null) this.diskCache?.writeSync(key, response.code);
+        return response.code;
+    }
+    /**
+	 * Compiles a file ahead, through the disk cache; undefined when it doesn't compile, which the
+	 * load hook then reports (see {@link compileAhead}). The source and the cache entry are read
+	 * synchronously: for small files that is far cheaper than a trip through libuv's thread pool,
+	 * which async SWC needs for the compiles that miss (#514).
+	 */ async compileAsync(fullPath, project) {
+        const file = new _io.FileEx({
+            fullPath
+        });
+        const baseConfig = this.swcConfigFor(project, fullPath);
+        let source;
+        try {
+            source = (0, _nodefs.readFileSync)(fullPath, "utf8");
+        } catch  {
+            // Left to the load hook, which reads it again and fails naming it, as for a file
+            // that doesn't compile.
+            return undefined;
+        }
+        const key = this.diskCache?.keyFor(fullPath, source, baseConfig);
+        const hit = key == null ? undefined : this.diskCache?.readSync(key);
+        if (hit != null) return hit;
+        const response = await this.transpiler.transformCode(source, file, _forge.BuildTargets.es6, baseConfig);
+        if (response.outcome !== "success") return undefined;
+        if (key != null) await this.diskCache?.write(key, response.code);
+        return response.code;
     }
     /**
 	 * The project's SWC options, built once: rebuilding them for every file was about a quarter
 	 * of load time (#513). Import specifiers are left as written, without `jsc.paths` or
 	 * `resolveFully`: the resolve hook maps them to source files (aliases, `.js` to `.ts`,
 	 * directory indexes), so SWC does not resolve them a second time against the disk.
-	 */ swcConfigFor(project) {
+	 */ swcConfigFor(project, fullPath) {
+        const config = this.esModuleConfigFor(project);
+        if (!_mesh.sourceExtensions.isCommonJs(fullPath)) return config;
+        // `.cts` is CommonJS: Node loads it with `require` semantics, so SWC emits CommonJS (#518).
+        // Kept per project, like the ES module options, so the disk cache hashes them once.
+        const cached = this.commonJsConfigs.get(project);
+        if (cached != null) return cached;
+        const commonJs = {
+            ...config,
+            module: {
+                type: "commonjs"
+            }
+        };
+        this.commonJsConfigs.set(project, commonJs);
+        return commonJs;
+    }
+    esModuleConfigFor(project) {
         const cached = this.swcConfigs.get(project);
         if (cached != null) return cached;
         const base = this.transpiler.createBaseConfig(_forge.BuildTargets.es6, {

package/commonjs/src/transpile-disk-cache.js

@@ -0,0 +1,148 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", {
+    value: true
+});
+Object.defineProperty(exports, "TranspileDiskCache", {
+    enumerable: true,
+    get: function() {
+        return TranspileDiskCache;
+    }
+});
+const _nodecrypto = require("node:crypto");
+const _nodefs = /*#__PURE__*/ _interop_require_default(require("node:fs"));
+const _nodepath = /*#__PURE__*/ _interop_require_default(require("node:path"));
+const _forge = require("@decohere/forge");
+const _scribe = require("@decohere/scribe");
+function _interop_require_default(obj) {
+    return obj && obj.__esModule ? obj : {
+        default: obj
+    };
+}
+/** Bump when what flux stores changes, so older entries stop matching. */ const FORMAT = 1;
+/** Entries untouched for this long are pruned. */ const MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000;
+/** Pruning runs at most this often, recorded by the mtime of a marker file. */ const PRUNE_EVERY_MS = 24 * 60 * 60 * 1000;
+const PRUNE_MARKER = ".last-prune";
+class TranspileDiskCache {
+    dir;
+    logger = _scribe.scribe.logger(TranspileDiskCache.name);
+    disabled = false;
+    constructor(dir){
+        this.dir = dir;
+    }
+    /** The cache for a workspace, or null when caching is off (`DECOHERE_CACHE=0`). */ static forWorkspace(rootDir) {
+        const base = _forge.transpileCache.directory(rootDir);
+        return base == null ? null : new TranspileDiskCache(_nodepath.default.join(base, "flux"));
+    }
+    get directory() {
+        return this.dir;
+    }
+    /**
+	 * The key for a file compiled with `baseOptions`: the per-file options are those plus the
+	 * file's path (forge's `createFileConfig`), so hashing the base once per project and adding
+	 * the path covers them without serialising options for every file.
+	 */ keyFor(fullPath, source, baseOptions) {
+        return (0, _nodecrypto.createHash)("sha256").update(`${FORMAT}\0${_forge.transpileCache.swcVersion}\0${this.optionsHash(baseOptions)}\0`).update(`${fullPath}\0`).update(source).digest("hex");
+    }
+    optionsHashes = new WeakMap();
+    optionsHash(options) {
+        const cached = this.optionsHashes.get(options);
+        if (cached != null) return cached;
+        const hash = (0, _nodecrypto.createHash)("sha256").update(JSON.stringify(options)).digest("hex");
+        this.optionsHashes.set(options, hash);
+        return hash;
+    }
+    readSync(key) {
+        if (this.disabled) return undefined;
+        try {
+            return _nodefs.default.readFileSync(this.entryPath(key), "utf8");
+        } catch (error) {
+            return this.missOrDisable(error);
+        }
+    }
+    async read(key) {
+        if (this.disabled) return undefined;
+        try {
+            return await _nodefs.default.promises.readFile(this.entryPath(key), "utf8");
+        } catch (error) {
+            return this.missOrDisable(error);
+        }
+    }
+    writeSync(key, code) {
+        if (this.disabled) return;
+        const file = this.entryPath(key);
+        const temp = this.tempPath(file);
+        try {
+            _nodefs.default.mkdirSync(_nodepath.default.dirname(file), {
+                recursive: true
+            });
+            _nodefs.default.writeFileSync(temp, code);
+            _nodefs.default.renameSync(temp, file);
+        } catch (error) {
+            this.disable("write", error);
+        }
+    }
+    async write(key, code) {
+        if (this.disabled) return;
+        const file = this.entryPath(key);
+        const temp = this.tempPath(file);
+        try {
+            await _nodefs.default.promises.mkdir(_nodepath.default.dirname(file), {
+                recursive: true
+            });
+            await _nodefs.default.promises.writeFile(temp, code);
+            await _nodefs.default.promises.rename(temp, file);
+        } catch (error) {
+            this.disable("write", error);
+        }
+    }
+    /** Deletes entries older than 30 days, if it hasn't done so in the last day. */ async pruneIfDue(now = Date.now()) {
+        if (this.disabled) return;
+        const marker = _nodepath.default.join(this.dir, PRUNE_MARKER);
+        try {
+            const last = await _nodefs.default.promises.stat(marker).then((stat)=>stat.mtimeMs, (error)=>{
+                if (error.code === "ENOENT") return 0;
+                throw error;
+            });
+            if (now - last < PRUNE_EVERY_MS) return;
+            await _nodefs.default.promises.mkdir(this.dir, {
+                recursive: true
+            });
+            await _nodefs.default.promises.writeFile(marker, "");
+            for (const shard of (await _nodefs.default.promises.readdir(this.dir, {
+                withFileTypes: true
+            }))){
+                if (!shard.isDirectory()) continue;
+                const shardDir = _nodepath.default.join(this.dir, shard.name);
+                for (const entry of (await _nodefs.default.promises.readdir(shardDir))){
+                    // Old entries, and temp files a crashed process left mid-write.
+                    const file = _nodepath.default.join(shardDir, entry);
+                    const mtime = await _nodefs.default.promises.stat(file).then((stat)=>stat.mtimeMs, (error)=>{
+                        // Renamed or removed by another process since the listing: nothing to prune.
+                        if (error.code === "ENOENT") return undefined;
+                        throw error;
+                    });
+                    if (mtime != null && now - mtime > MAX_AGE_MS) await _nodefs.default.promises.rm(file, {
+                        force: true
+                    });
+                }
+            }
+        } catch (error) {
+            this.disable("prune", error);
+        }
+    }
+    entryPath(key) {
+        return _nodepath.default.join(this.dir, key.slice(0, 2), `${key}.js`);
+    }
+    tempPath(file) {
+        return `${file}.${process.pid}.${(0, _nodecrypto.randomBytes)(4).toString("hex")}.tmp`;
+    }
+    /** A missing entry is a miss; any other read error turns the cache off. */ missOrDisable(error) {
+        if (error.code !== "ENOENT") this.disable("read", error);
+        return undefined;
+    }
+    disable(operation, error) {
+        if (this.disabled) return;
+        this.disabled = true;
+        this.logger.warn(`Transpile cache turned off for this run: could not ${operation} ${this.dir} (${String(error)}). ` + "Flux compiles without it. Set DECOHERE_CACHE_DIR to a writable directory, or DECOHERE_CACHE=0 to silence this.");
+    }
+}

package/es6/src/rewrite-serve-specifier.js

@@ -1,6 +1,6 @@
 import path from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
-import { packageAliases } from "@decohere/mesh";
+import { packageAliases, sourceExtensions } from "@decohere/mesh";
 /**
  * Map a resolve-hook specifier to a versioned watched `.ts` file URL.
  * Covers relative `.js` / `.ts` / extensionless imports, directory imports (`./models` is
@@ -20,16 +20,18 @@ import { packageAliases } from "@decohere/mesh";
     return null;
 }
 /**
- * The watched `.ts` file a path names: itself (with `.js` read as `.ts`, or `.ts` added), else
- * the `index.ts` of the directory it names. A file wins over a directory of the same name.
+ * The watched source file a path names: itself, or the source its extension stands for (`.js` is
+ * `.ts` or `.tsx`, `.mjs` is `.mts`, `.cjs` is `.cts`, none adds `.ts` or `.tsx`), else the
+ * `index` source of the directory it names. A file wins over a directory of the same name.
  */ function watchedTs(filePath, isWatchedTs) {
     const withoutSlash = filePath.replace(/[/\\]+$/, "");
-    const stem = withoutSlash.replace(/\.(js|ts)$/, "");
-    // From the stem: Mesh's alias lookup has already added `.ts` to a subpath that names a
-    // directory (`@lib/models` arrives as `…/models.ts`).
+    // Mesh's alias lookup may have added `.ts` to a subpath that names a directory (`@lib/models`
+    // arrives as `…/models.ts`), so the directory is the path without any extension.
+    const directory = withoutSlash.replace(/\.([mc]?[jt]sx?)$/, "");
     const candidates = [
-        `${stem}.ts`,
-        path.join(stem, "index.ts")
+        ...sourceExtensions.candidates(withoutSlash),
+        ...sourceExtensions.candidates(directory),
+        ...sourceExtensions.candidates(path.join(directory, "index"))
     ];
     return candidates.find((candidate)=>isWatchedTs(candidate)) ?? null;
 }

package/es6/src/source-module-loader.js

@@ -4,16 +4,18 @@ function _ts_decorate(decorators, target, key, desc) {
     else for(var i = decorators.length - 1; i >= 0; i--)if (d = decorators[i]) r = (c < 3 ? d(r) : c > 3 ? d(target, key, r) : d(target, key)) || r;
     return c > 3 && r && Object.defineProperty(target, key, r), r;
 }
+import { readFileSync } from "node:fs";
 import { registerHooks } from "node:module";
 import path from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 import { injectable } from "@decohere/crucible/decorators";
 import { BuildTargets, createWorkspace, SwcTranspiler } from "@decohere/forge";
-import { packageAliases } from "@decohere/mesh";
+import { packageAliases, sourceExtensions } from "@decohere/mesh";
 import { FileEx, io } from "@decohere/nexus/io";
 import { scribe } from "@decohere/scribe";
 import { rewriteServeSpecifier } from "./rewrite-serve-specifier.js";
 import { staticImports } from "./static-imports.js";
+import { TranspileDiskCache } from "./transpile-disk-cache.js";
 export class SourceModuleLoader {
     logger = scribe.logger(SourceModuleLoader.name);
     transpiler = new SwcTranspiler();
@@ -25,8 +27,10 @@ export class SourceModuleLoader {
     transpileCache = new Map();
     /** Bumped when a file's cached code is dropped, so an up-front compile already running can't put it back. */ transpileVersions = new Map();
     /** The source files each entry's walk reached, so a reboot only walks out from what changed. */ reachedFrom = new Map();
+    /** Compiled code on disk across runs (#514); null when caching is off. */ diskCache = null;
     /** Up-front compiles in flight, shared by entries booting at the same time. */ compilingAhead = new Map();
     /** SWC options per project, built on first use and kept until the next `configure()`. */ swcConfigs = new Map();
+    commonJsConfigs = new Map();
     aliasPaths = {};
     aliasBaseUrl = path.resolve(".");
     versionFileUrl = (fileUrl)=>fileUrl;
@@ -34,9 +38,6 @@ export class SourceModuleLoader {
     getWorkspace() {
         return this.workspace;
     }
-    getBuildConfig() {
-        return this.buildConfig;
-    }
     hasSourceFile(absolutePath) {
         return this.fileInfoMap.has(absolutePath);
     }
@@ -54,11 +55,15 @@ export class SourceModuleLoader {
         this.transpileCache = new Map();
         this.reachedFrom = new Map();
         this.swcConfigs.clear();
+        this.commonJsConfigs.clear();
         this.buildConfig = options.buildConfig;
         this.workspace = options.workspace ?? await createWorkspace();
         this.versionFileUrl = options.versionFileUrl ?? ((fileUrl)=>fileUrl);
         this.onResolved = options.onResolved;
         this.fileInfoMap = await this.buildFileInfoMap(this.buildConfig);
+        this.diskCache = TranspileDiskCache.forWorkspace(this.workspace.rootDir);
+        // In the background; it turns the cache off, with a warning, rather than throw.
+        void this.diskCache?.pruneIfDue();
         const aliases = this.workspace.getPackageAliases(this.buildConfig);
         this.aliasBaseUrl = aliases.baseUrl;
         this.aliasPaths = aliases.paths;
@@ -89,7 +94,7 @@ export class SourceModuleLoader {
                 const result = this.loadTs(url);
                 if (result != null) {
                     return {
-                        format: "module",
+                        format: result.format,
                         source: result.source,
                         shortCircuit: true
                     };
@@ -107,9 +112,6 @@ export class SourceModuleLoader {
         for (const fullPath of absolutePaths){
             this.transpileCache.delete(fullPath);
             this.transpileVersions.set(fullPath, (this.transpileVersions.get(fullPath) ?? 0) + 1);
-            this.transpiler.invalidateContentCache(new FileEx({
-                fullPath
-            }));
         }
     }
     dispose() {
@@ -117,6 +119,7 @@ export class SourceModuleLoader {
         this.transpileCache = new Map();
         this.reachedFrom = new Map();
         this.swcConfigs.clear();
+        this.commonJsConfigs.clear();
         this.fileInfoMap.clear();
         this.workspace?.[Symbol.dispose]();
         this.workspace = undefined;
@@ -177,14 +180,12 @@ export class SourceModuleLoader {
         if (inFlight != null) return inFlight;
         const version = this.transpileVersions.get(fullPath);
         const cache = this.transpileCache;
-        const compiling = this.transpiler.transformFileAsync(new FileEx({
-            fullPath
-        }), BuildTargets.es6, {}, this.swcConfigFor(fileInfo.project)).then((response)=>{
-            if (response.outcome !== "success") return undefined;
+        const compiling = this.compileAsync(fullPath, fileInfo.project).then((code)=>{
+            if (code == null) return undefined;
             // Saved, reconfigured or disposed while compiling: don't cache stale code.
             const current = !this.disposed && cache === this.transpileCache && version === this.transpileVersions.get(fullPath);
-            if (current) cache.set(fullPath, response.code);
-            return response.code;
+            if (current) cache.set(fullPath, code);
+            return code;
         }).finally(()=>this.compilingAhead.delete(fullPath));
         this.compilingAhead.set(fullPath, compiling);
         return compiling;
@@ -206,29 +207,95 @@ export class SourceModuleLoader {
         const cleanPath = fileURLToPath(url);
         const fileInfo = this.fileInfoMap.get(cleanPath);
         if (fileInfo == null) return null;
+        const format = sourceExtensions.isCommonJs(cleanPath) ? "commonjs" : "module";
         const cached = this.transpileCache.get(cleanPath);
         if (cached != null) return {
-            source: cached
+            source: cached,
+            format
         };
+        const code = this.compileSync(cleanPath, fileInfo.project);
+        this.transpileCache.set(cleanPath, code);
+        return {
+            source: code,
+            format
+        };
+    }
+    /**
+	 * Compiles a file for the load hook, through the disk cache (#514). Throws, naming the file,
+	 * when it can't be read or doesn't compile.
+	 */ compileSync(fullPath, project) {
         const file = new FileEx({
-            fullPath: cleanPath
+            fullPath
         });
-        const response = this.transpiler.transformFileSync(file, BuildTargets.es6, {}, this.swcConfigFor(fileInfo.project));
+        const baseConfig = this.swcConfigFor(project, fullPath);
+        let source;
+        try {
+            source = readFileSync(fullPath, "utf8");
+        } catch (error) {
+            this.logger.error(`Failed to transpile ${fullPath}`, error);
+            throw new Error(`Failed to transpile ${fullPath}`, {
+                cause: error
+            });
+        }
+        const key = this.diskCache?.keyFor(fullPath, source, baseConfig);
+        const hit = key == null ? undefined : this.diskCache?.readSync(key);
+        if (hit != null) return hit;
+        const response = this.transpiler.transformCodeSync(source, file, BuildTargets.es6, baseConfig);
         if (response.outcome !== "success") {
-            this.logger.error(`Failed to transpile ${cleanPath}`, response);
-            throw new Error(`Failed to transpile ${cleanPath}`);
+            this.logger.error(`Failed to transpile ${fullPath}`, response);
+            throw new Error(`Failed to transpile ${fullPath}`);
         }
-        this.transpileCache.set(cleanPath, response.code);
-        return {
-            source: response.code
-        };
+        if (key != null) this.diskCache?.writeSync(key, response.code);
+        return response.code;
+    }
+    /**
+	 * Compiles a file ahead, through the disk cache; undefined when it doesn't compile, which the
+	 * load hook then reports (see {@link compileAhead}). The source and the cache entry are read
+	 * synchronously: for small files that is far cheaper than a trip through libuv's thread pool,
+	 * which async SWC needs for the compiles that miss (#514).
+	 */ async compileAsync(fullPath, project) {
+        const file = new FileEx({
+            fullPath
+        });
+        const baseConfig = this.swcConfigFor(project, fullPath);
+        let source;
+        try {
+            source = readFileSync(fullPath, "utf8");
+        } catch  {
+            // Left to the load hook, which reads it again and fails naming it, as for a file
+            // that doesn't compile.
+            return undefined;
+        }
+        const key = this.diskCache?.keyFor(fullPath, source, baseConfig);
+        const hit = key == null ? undefined : this.diskCache?.readSync(key);
+        if (hit != null) return hit;
+        const response = await this.transpiler.transformCode(source, file, BuildTargets.es6, baseConfig);
+        if (response.outcome !== "success") return undefined;
+        if (key != null) await this.diskCache?.write(key, response.code);
+        return response.code;
     }
     /**
 	 * The project's SWC options, built once: rebuilding them for every file was about a quarter
 	 * of load time (#513). Import specifiers are left as written, without `jsc.paths` or
 	 * `resolveFully`: the resolve hook maps them to source files (aliases, `.js` to `.ts`,
 	 * directory indexes), so SWC does not resolve them a second time against the disk.
-	 */ swcConfigFor(project) {
+	 */ swcConfigFor(project, fullPath) {
+        const config = this.esModuleConfigFor(project);
+        if (!sourceExtensions.isCommonJs(fullPath)) return config;
+        // `.cts` is CommonJS: Node loads it with `require` semantics, so SWC emits CommonJS (#518).
+        // Kept per project, like the ES module options, so the disk cache hashes them once.
+        const cached = this.commonJsConfigs.get(project);
+        if (cached != null) return cached;
+        const commonJs = {
+            ...config,
+            module: {
+                type: "commonjs"
+            }
+        };
+        this.commonJsConfigs.set(project, commonJs);
+        return commonJs;
+    }
+    esModuleConfigFor(project) {
         const cached = this.swcConfigs.get(project);
         if (cached != null) return cached;
         const base = this.transpiler.createBaseConfig(BuildTargets.es6, {

package/es6/src/transpile-disk-cache.js

@@ -0,0 +1,142 @@
+import { createHash, randomBytes } from "node:crypto";
+import fs from "node:fs";
+import path from "node:path";
+import { transpileCache } from "@decohere/forge";
+import { scribe } from "@decohere/scribe";
+/** Bump when what flux stores changes, so older entries stop matching. */ const FORMAT = 1;
+/** Entries untouched for this long are pruned. */ const MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000;
+/** Pruning runs at most this often, recorded by the mtime of a marker file. */ const PRUNE_EVERY_MS = 24 * 60 * 60 * 1000;
+const PRUNE_MARKER = ".last-prune";
+/**
+ * Compiled code on disk, so a cold start doesn't compile files that haven't changed (#514).
+ * Content-addressed: the key hashes the file's path and source, the SWC options it compiles
+ * with, SWC's version and this cache's format, so any change to those is a miss, never a stale
+ * hit. Entries are written to a temp file and renamed into place, so a process reading an entry
+ * another is writing sees all of it or none.
+ *
+ * The cache is an optimisation: if it can't read or write its directory, it says so once and
+ * turns itself off for the process, and flux compiles as it would without it.
+ */ export class TranspileDiskCache {
+    dir;
+    logger = scribe.logger(TranspileDiskCache.name);
+    disabled = false;
+    constructor(dir){
+        this.dir = dir;
+    }
+    /** The cache for a workspace, or null when caching is off (`DECOHERE_CACHE=0`). */ static forWorkspace(rootDir) {
+        const base = transpileCache.directory(rootDir);
+        return base == null ? null : new TranspileDiskCache(path.join(base, "flux"));
+    }
+    get directory() {
+        return this.dir;
+    }
+    /**
+	 * The key for a file compiled with `baseOptions`: the per-file options are those plus the
+	 * file's path (forge's `createFileConfig`), so hashing the base once per project and adding
+	 * the path covers them without serialising options for every file.
+	 */ keyFor(fullPath, source, baseOptions) {
+        return createHash("sha256").update(`${FORMAT}\0${transpileCache.swcVersion}\0${this.optionsHash(baseOptions)}\0`).update(`${fullPath}\0`).update(source).digest("hex");
+    }
+    optionsHashes = new WeakMap();
+    optionsHash(options) {
+        const cached = this.optionsHashes.get(options);
+        if (cached != null) return cached;
+        const hash = createHash("sha256").update(JSON.stringify(options)).digest("hex");
+        this.optionsHashes.set(options, hash);
+        return hash;
+    }
+    readSync(key) {
+        if (this.disabled) return undefined;
+        try {
+            return fs.readFileSync(this.entryPath(key), "utf8");
+        } catch (error) {
+            return this.missOrDisable(error);
+        }
+    }
+    async read(key) {
+        if (this.disabled) return undefined;
+        try {
+            return await fs.promises.readFile(this.entryPath(key), "utf8");
+        } catch (error) {
+            return this.missOrDisable(error);
+        }
+    }
+    writeSync(key, code) {
+        if (this.disabled) return;
+        const file = this.entryPath(key);
+        const temp = this.tempPath(file);
+        try {
+            fs.mkdirSync(path.dirname(file), {
+                recursive: true
+            });
+            fs.writeFileSync(temp, code);
+            fs.renameSync(temp, file);
+        } catch (error) {
+            this.disable("write", error);
+        }
+    }
+    async write(key, code) {
+        if (this.disabled) return;
+        const file = this.entryPath(key);
+        const temp = this.tempPath(file);
+        try {
+            await fs.promises.mkdir(path.dirname(file), {
+                recursive: true
+            });
+            await fs.promises.writeFile(temp, code);
+            await fs.promises.rename(temp, file);
+        } catch (error) {
+            this.disable("write", error);
+        }
+    }
+    /** Deletes entries older than 30 days, if it hasn't done so in the last day. */ async pruneIfDue(now = Date.now()) {
+        if (this.disabled) return;
+        const marker = path.join(this.dir, PRUNE_MARKER);
+        try {
+            const last = await fs.promises.stat(marker).then((stat)=>stat.mtimeMs, (error)=>{
+                if (error.code === "ENOENT") return 0;
+                throw error;
+            });
+            if (now - last < PRUNE_EVERY_MS) return;
+            await fs.promises.mkdir(this.dir, {
+                recursive: true
+            });
+            await fs.promises.writeFile(marker, "");
+            for (const shard of (await fs.promises.readdir(this.dir, {
+                withFileTypes: true
+            }))){
+                if (!shard.isDirectory()) continue;
+                const shardDir = path.join(this.dir, shard.name);
+                for (const entry of (await fs.promises.readdir(shardDir))){
+                    // Old entries, and temp files a crashed process left mid-write.
+                    const file = path.join(shardDir, entry);
+                    const mtime = await fs.promises.stat(file).then((stat)=>stat.mtimeMs, (error)=>{
+                        // Renamed or removed by another process since the listing: nothing to prune.
+                        if (error.code === "ENOENT") return undefined;
+                        throw error;
+                    });
+                    if (mtime != null && now - mtime > MAX_AGE_MS) await fs.promises.rm(file, {
+                        force: true
+                    });
+                }
+            }
+        } catch (error) {
+            this.disable("prune", error);
+        }
+    }
+    entryPath(key) {
+        return path.join(this.dir, key.slice(0, 2), `${key}.js`);
+    }
+    tempPath(file) {
+        return `${file}.${process.pid}.${randomBytes(4).toString("hex")}.tmp`;
+    }
+    /** A missing entry is a miss; any other read error turns the cache off. */ missOrDisable(error) {
+        if (error.code !== "ENOENT") this.disable("read", error);
+        return undefined;
+    }
+    disable(operation, error) {
+        if (this.disabled) return;
+        this.disabled = true;
+        this.logger.warn(`Transpile cache turned off for this run: could not ${operation} ${this.dir} (${String(error)}). ` + "Flux compiles without it. Set DECOHERE_CACHE_DIR to a writable directory, or DECOHERE_CACHE=0 to silence this.");
+    }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@decohere/flux",
-  "version": "0.0.1-dev.112",
+  "version": "0.0.1-dev.114",
   "description": "Hot module reload server for Node.js ESM applications",
   "keywords": [
     "typescript",
@@ -25,11 +25,11 @@
     "test": "echo \"Error: no test specified\" && exit 1"
   },
   "dependencies": {
-    "@decohere/crucible": "0.0.1-dev.112",
-    "@decohere/forge": "0.0.1-dev.112",
-    "@decohere/mesh": "0.0.1-dev.112",
-    "@decohere/nexus": "0.0.1-dev.112",
-    "@decohere/scribe": "0.0.1-dev.112",
+    "@decohere/crucible": "0.0.1-dev.114",
+    "@decohere/forge": "0.0.1-dev.114",
+    "@decohere/mesh": "0.0.1-dev.114",
+    "@decohere/nexus": "0.0.1-dev.114",
+    "@decohere/scribe": "0.0.1-dev.114",
     "commander": "^13.1.0",
     "rxjs": "^7.8.2"
   },

package/types/src/hot-reload-server.d.ts

@@ -28,7 +28,8 @@ export interface HotReloadServerOptions {
     workspace?: Workspace;
     /**
      * Quiet window in milliseconds after the last affecting save before a reboot.
-     * Defaults to {@link DEFAULT_REBOOT_DEBOUNCE_MS}. Raise this for agent save trains.
+     * Defaults to {@link DEFAULT_REBOOT_DEBOUNCE_MS}. Every reload waits this long, so a larger
+     * window only pays off for saves known to come at a steady pace inside it (#484).
      */
     rebootDebounceMs?: number;
 }

package/types/src/source-module-loader.d.ts

@@ -34,16 +34,18 @@ export declare class SourceModuleLoader {
     private transpileVersions;
     /** The source files each entry's walk reached, so a reboot only walks out from what changed. */
     private reachedFrom;
+    /** Compiled code on disk across runs (#514); null when caching is off. */
+    private diskCache;
     /** Up-front compiles in flight, shared by entries booting at the same time. */
     private compilingAhead;
     /** SWC options per project, built on first use and kept until the next `configure()`. */
     private swcConfigs;
+    private commonJsConfigs;
     private aliasPaths;
     private aliasBaseUrl;
     private versionFileUrl;
     private onResolved?;
     getWorkspace(): Workspace | undefined;
-    getBuildConfig(): string;
     hasSourceFile(absolutePath: string): boolean;
     sourceFilePaths(): IterableIterator<string>;
     setSourceFile(absolutePath: string, project: Project): void;
@@ -71,6 +73,18 @@ export declare class SourceModuleLoader {
     /** The source file a specifier names, mapped the way the resolve hook maps it. */
     private sourcePathOf;
     private loadTs;
+    /**
+     * Compiles a file for the load hook, through the disk cache (#514). Throws, naming the file,
+     * when it can't be read or doesn't compile.
+     */
+    private compileSync;
+    /**
+     * Compiles a file ahead, through the disk cache; undefined when it doesn't compile, which the
+     * load hook then reports (see {@link compileAhead}). The source and the cache entry are read
+     * synchronously: for small files that is far cheaper than a trip through libuv's thread pool,
+     * which async SWC needs for the compiles that miss (#514).
+     */
+    private compileAsync;
     /**
      * The project's SWC options, built once: rebuilding them for every file was about a quarter
      * of load time (#513). Import specifiers are left as written, without `jsc.paths` or
@@ -78,4 +92,5 @@ export declare class SourceModuleLoader {
      * directory indexes), so SWC does not resolve them a second time against the disk.
      */
     private swcConfigFor;
+    private esModuleConfigFor;
 }

package/types/src/transpile-disk-cache.d.ts

@@ -0,0 +1,38 @@
+/**
+ * Compiled code on disk, so a cold start doesn't compile files that haven't changed (#514).
+ * Content-addressed: the key hashes the file's path and source, the SWC options it compiles
+ * with, SWC's version and this cache's format, so any change to those is a miss, never a stale
+ * hit. Entries are written to a temp file and renamed into place, so a process reading an entry
+ * another is writing sees all of it or none.
+ *
+ * The cache is an optimisation: if it can't read or write its directory, it says so once and
+ * turns itself off for the process, and flux compiles as it would without it.
+ */
+export declare class TranspileDiskCache {
+    private readonly dir;
+    private readonly logger;
+    private disabled;
+    private constructor();
+    /** The cache for a workspace, or null when caching is off (`DECOHERE_CACHE=0`). */
+    static forWorkspace(rootDir: string): TranspileDiskCache | null;
+    get directory(): string;
+    /**
+     * The key for a file compiled with `baseOptions`: the per-file options are those plus the
+     * file's path (forge's `createFileConfig`), so hashing the base once per project and adding
+     * the path covers them without serialising options for every file.
+     */
+    keyFor(fullPath: string, source: string, baseOptions: object): string;
+    private readonly optionsHashes;
+    private optionsHash;
+    readSync(key: string): string | undefined;
+    read(key: string): Promise<string | undefined>;
+    writeSync(key: string, code: string): void;
+    write(key: string, code: string): Promise<void>;
+    /** Deletes entries older than 30 days, if it hasn't done so in the last day. */
+    pruneIfDue(now?: number): Promise<void>;
+    private entryPath;
+    private tempPath;
+    /** A missing entry is a miss; any other read error turns the cache off. */
+    private missOrDisable;
+    private disable;
+}