fulmine.js 5.5.2 → 5.6.0

package/COMPRESSION_LICENSE

@@ -0,0 +1,25 @@
+compression License (only code derived from the compression module).
+
+(The MIT License)
+
+Copyright (c) 2014 Jonathan Ong <me@jongleberry.com>
+Copyright (c) 2014-2015 Douglas Christopher Wilson <doug@somethingdoug.com>
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/NOTICE

@@ -43,6 +43,15 @@ every copy:
   IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
   DEALINGS IN THE SOFTWARE.
 
+This product includes test code derived from the compression middleware
+(https://github.com/expressjs/compression), ported in
+tests/unit/compression.test.js and licensed under the MIT License. Its terms
+travel with this repository in COMPRESSION_LICENSE, and the copyright it
+requires to be carried is:
+
+  Copyright (c) 2014 Jonathan Ong <me@jongleberry.com>
+  Copyright (c) 2014-2015 Douglas Christopher Wilson <doug@somethingdoug.com>
+
 As required by section 4(b) of the Apache License, the following are the
 significant changes made to the original work:
 

package/README.md

@@ -1,6 +1,6 @@
 <img src="./assets/logo-mark.svg" alt="" width="88" align="right">
 
-# Fulmine
+# Fulmine.js
 
 A drop-in replacement for Express 5, running on [µWebSockets.js](https://github.com/uNetworking/uWebSockets.js) instead of `node:http`. Your existing middleware keeps working.
 
@@ -59,7 +59,6 @@ See [Migrating](#migrating) for what it handles and what it deliberately does no
     - [Router](#router)
 - [Tested middlewares](#tested-middlewares)
 - [Tested view engines](#tested-view-engines)
-- [The demo](https://fulmine-demo.fly.dev)
 - [Working on Fulmine](./CONTRIBUTING.md)
 
 ## Why this exists
@@ -90,7 +89,7 @@ to run it yourself.
 
 Numbers produced by a project about itself deserve suspicion, so Fulmine also stands in public arenas, run by their own rigs under their own rules:
 
-- **[HttpArena](https://www.http-arena.com/#sort=rps:-1&q=Js)** (the link lands filtered on the JavaScript entries): first among the JavaScript entries, across fifteen subscribed profiles. The saved runs measure 24.3 million WebSocket echoes per second pipelined and 3.77 million one-at-a-time (past Bun's own dedicated WebSocket entry), 7.3 million pipelined HTTP requests per second, 1.12 million on the json profile with 1.04 million of that surviving TLS, 457 thousand on compressed json, and 359 thousand on the Postgres CRUD profile, within ten percent of the leading Rust and C# entries there.
+- **[HttpArena](https://www.http-arena.com/#sort=rps:-1&q=Js)**: thirty profiles on 64-core dedicated hardware, same conditions for every entry, rerun whenever one of them changes. The link lands filtered on the JavaScript entries. No figures are copied here on purpose: the board is the current one and this page would not be.
 - **[web-frameworks](https://web-frameworks-benchmark.netlify.app/result?l=javascript)**: entry merged, numbers arrive with their next published round.
 
 More to come as their maintainers take the entries in.
@@ -125,6 +124,10 @@ npx fulmine differences         # print the list below and change nothing
 The command is installed under both `fulmine` and `fulmine.js`. Use `fulmine`: `npx` cannot run a
 command whose name ends in `.js` on Windows, where it exits without a word.
 
+It also names the middlewares it found that have a faster one built in here, `compression`,
+`body-parser` and `serve-static`, and leaves them to you: the replacement is reached through the
+`express` import, and no rewrite can know that it is in scope where they are required.
+
 ### Angular SSR
 
 The `server.ts` that `ng add @angular/ssr` generates is an ordinary Express application, so the same
@@ -196,12 +199,17 @@ Bun is not an option: µWebSockets.js is a native Node addon, and Bun does not l
 
 ## Docker
 
-Two things about µWebSockets.js make a Dockerfile that works for Express fail here, and both have easy answers:
+Three things about µWebSockets.js make a Dockerfile that works for Express fail here, and all three have easy answers:
 
 - **No Alpine, and no Debian bookworm either.** µWebSockets.js ships prebuilt binaries linked against glibc 2.38 or newer. Alpine images use musl, so the binary does not load at all; `node:26` and `node:26-slim` are Debian bookworm, whose glibc 2.36 fails at startup with `GLIBC_2.38' not found`. Use the trixie variants: `node:26-trixie-slim` and up.
 - **`git` must be there when `npm install` runs.** µWebSockets.js is not on npm; it is installed straight from GitHub (`github:uNetworking/uWebSockets.js`), and npm uses git to fetch it. Full images like `node:26-trixie` have git; `-slim` ones do not.
+- **git must be allowed to speak https.** Where the build environment rewrites GitHub URLs to ssh, which some CI images and company-wide git configs do, the fetch asks for a key the image does not have and the install dies on a permission denied that never names µWebSockets.js. One line before `npm ci` puts it back:
+
+    ```dockerfile
+    RUN git config --global url."https://github.com/".insteadOf "ssh://git@github.com/"
+    ```
 
-The clean way to satisfy both is a multi-stage build: install with the full image, run with the slim one.
+The clean way to satisfy the first two is a multi-stage build: install with the full image, run with the slim one.
 
 ```dockerfile
 FROM node:26-trixie AS build
@@ -366,17 +374,24 @@ Worth changing, if these are routes that carry traffic
 
 It loads the application with `listen()` replaced by the half that compiles the routes, so nothing binds a port and the listen callback does not run: profiling a running service does not start a second copy of it. There is no score, on purpose. A percentage of routes is not a percentage of traffic, and an application with a thousand cold routes and one hot one that fell back would score well and serve badly.
 
-2. Do not use external `serve-static` module. Instead use built-in `express.static()` middleware, which is optimized for Fulmine.
+2. Do not use external `serve-static` module. Instead use built-in `express.static()` middleware, which is optimized for Fulmine. If your build already writes `.br` and `.gz` files next to the originals, `express.static(dir, { preCompressed: true })` serves those to the clients that accept them, so nothing is compressed at request time. It costs one more `stat` per request and sends a fraction of the bytes: on a 4KB script with a brotli twin, 12 times fewer. `Vary: Accept-Encoding` is sent whether or not a variant is found, the content type stays the one the requested name implies, and each variant carries its own ETag.
 
 3. Do not use `body-parser` module. Instead use built-in `express.text()`, `express.json()` etc.
 
-4. If a route answers with a JSON shape you know in advance, [express-fast-json-stringify](https://www.npmjs.com/package/express-fast-json-stringify) compiles that shape into a serializer and `res.fastJson()` replaces `res.json()`. `JSON.stringify()` has to walk an object it knows nothing about; a compiled serializer does not.
+4. Do not use the `compression` module. `express.compression()` takes the same options and decides the same way, and it served about 50% more requests per second on an 8KB JSON body here, gzip and brotli alike. A response that arrives whole, which is every `res.send()` and `res.json()`, is compressed in one call rather than through a transform stream and goes out with a `Content-Length` instead of chunked; a response written in pieces still streams. The bytes are the same bytes either way.
+
+```js
+// the compression module's options, unchanged: threshold, filter, level, brotli, enforceEncoding
+app.use(express.compression({ threshold: 1024 }));
+```
+
+5. If a route answers with a JSON shape you know in advance, [express-fast-json-stringify](https://www.npmjs.com/package/express-fast-json-stringify) compiles that shape into a serializer and `res.fastJson()` replaces `res.json()`. `JSON.stringify()` has to walk an object it knows nothing about; a compiled serializer does not.
 
-5. Do not set `body methods` to read body of requests with GET method or other methods that don't need a body. Reading body makes endpoint about 15% slower.
+6. Do not set `body methods` to read body of requests with GET method or other methods that don't need a body. Reading body makes endpoint about 15% slower.
 
-6. `app.set("etag", false)` is worth about 8% on small responses, measured on both Fulmine and Express, which pay it almost identically. Know what you are trading: without an ETag a client cannot make a conditional request, so there are no `304 Not Modified` replies and every response is downloaded in full. On anything cacheable the bandwidth a 304 saves is usually worth far more than the 8%. It is left on by default for that reason. Turn it off for an API whose responses are never revalidated.
+7. `app.set("etag", false)` is worth about 8% on small responses, measured on both Fulmine and Express, which pay it almost identically. Know what you are trading: without an ETag a client cannot make a conditional request, so there are no `304 Not Modified` replies and every response is downloaded in full. On anything cacheable the bandwidth a 304 saves is usually worth far more than the 8%. It is left on by default for that reason. Turn it off for an API whose responses are never revalidated.
 
-7. By default, Fulmine creates 1 (or 0 if your CPU has only 1 core) child thread to improve performance of reading files. You can change this number by setting `threads` to a different number in `express()`, or set to 0 to disable thread pool (`express({ threads: 0 })`). Threads are shared between all express() instances, with largest `threads` number being used. Using more threads will not necessarily improve performance. Sometimes not using threads at all is faster, so measure both.
+8. By default, Fulmine creates 1 (or 0 if your CPU has only 1 core) child thread to improve performance of reading files. You can change this number by setting `threads` to a different number in `express()`, or set to 0 to disable thread pool (`express({ threads: 0 })`). Threads are shared between all express() instances, with largest `threads` number being used. Using more threads will not necessarily improve performance. Sometimes not using threads at all is faster, so measure both.
 
 ## WebSockets
 
@@ -506,6 +521,7 @@ In general, basically all features and options are supported. Use the [Express 5
 - ✅ express.static()
 - ✅ express.text()
 - ✅ express.raw()
+- ✅ express.compression(). Fulmine's own, since Express has none: it is the [compression](https://npmjs.com/package/compression) module's options and behaviour built in, described under [Performance tips](#performance-tips).
 - 🚧 express.request (this is not a constructor but a prototype for replacing methods)
 - 🚧 express.response (this is not a constructor but a prototype for replacing methods)
 - 🚧 express.application (likewise: a method added here is on every app)
@@ -564,7 +580,7 @@ Two of these keep a compiled form alongside the value, which you can also set di
 Fulmine adds three of its own:
 
 - `declarative responses`, on by default. Lets a simple enough handler be compiled into a native uWS response, described under [Performance tips](#performance-tips).
-- `file cache`, on by default. Small files served by `res.sendFile` come from a bounded in-process cache, checked against the file's `stat` on every request, so an edited file is never served stale.
+- `file cache`, on by default. Small files served by `res.sendFile` come from a bounded in-process cache, checked against the file's `stat` on every request, so an edited file is never served stale. Turn it off where every request has to reach the disk, which is what a public benchmark asks of a standard entry: it was worth about 4% on a 4KB file here, so the cost of turning it off is small.
 - `trust proxy protocol`, off by default. Takes `req.ip` from a PROXY protocol preamble, described under [Behind a proxy](#behind-a-proxy). Read the warning there before turning it on.
 
 ### Request
@@ -672,7 +688,7 @@ Almost all middlewares that are compatible with Express are compatible with Fulm
 - ✅ [body-parser](https://npmjs.com/package/body-parser) (use `express.text()` etc instead for better performance)
 - ✅ [cookie-parser](https://npmjs.com/package/cookie-parser)
 - ✅ [cookie-session](https://npmjs.com/package/cookie-session)
-- ✅ [compression](https://npmjs.com/package/compression)
+- ✅ [compression](https://npmjs.com/package/compression) (use `express.compression()` instead for better performance)
 - ✅ [serve-static](https://npmjs.com/package/serve-static) (use `express.static()` instead for better performance)
 - ✅ [serve-index](https://npmjs.com/package/serve-index)
 - ✅ [cors](https://npmjs.com/package/cors)

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "fulmine.js",
-    "version": "5.5.2",
+    "version": "5.6.0",
     "description": "Drop-in Express 5 replacement on uWebSockets.js. Your existing middleware keeps working.",
     "main": "src/index.js",
     "bin": {
@@ -40,7 +40,8 @@
         "src",
         "LICENSE",
         "NOTICE",
-        "EXPRESS_LICENSE"
+        "EXPRESS_LICENSE",
+        "COMPRESSION_LICENSE"
     ],
     "repository": {
         "type": "git",
@@ -72,6 +73,7 @@
         "accepts": "^2.0.0",
         "acorn": "^8.16.0",
         "bytes": "^3.1.2",
+        "compressible": "^2.0.18",
         "content-disposition": "^1.1.0",
         "cookie": "^1.1.1",
         "cookie-signature": "^1.2.2",

package/src/cli.js

@@ -34,6 +34,16 @@ const acorn = require("acorn");
 const FROM = "express";
 const TO = "fulmine.js";
 
+// Modules this has a faster version of, spotted while the files are being read anyway. They are
+// reported and not rewritten: the replacement is reached through the express import, which this
+// command cannot know is in scope in the file that requires them, and body-parser is four
+// functions rather than one.
+const BUILT_IN_INSTEAD = {
+    compression: "express.compression(), which takes the same options",
+    "serve-static": "express.static()",
+    "body-parser": "express.json(), express.urlencoded(), express.text(), express.raw()"
+};
+
 const SKIP_DIRS = new Set(["node_modules", ".git", "dist", "build", "coverage", ".nyc_output", ".next"]);
 const EXTENSIONS = new Set([".js", ".mjs", ".cjs", ".ts", ".mts", ".cts", ".tsx"]);
 const TYPESCRIPT_EXTENSIONS = new Set([".ts", ".mts", ".cts", ".tsx"]);
@@ -139,9 +149,10 @@ function loadTypeScript(target) {
  * @param {string} source
  * @param {string} fileName decides whether JSX is allowed, so a .tsx angle bracket is not a cast
  * @param {any} ts the compiler
+ * @param {Set<string>} [seen] as in findSpecifiers
  * @returns {{start: number, end: number}[]}
  */
-function findSpecifiersTypeScript(source, fileName, ts) {
+function findSpecifiersTypeScript(source, fileName, ts, seen) {
     const sourceFile = ts.createSourceFile(
         fileName,
         source,
@@ -152,7 +163,14 @@ function findSpecifiersTypeScript(source, fileName, ts) {
 
     /** @type {{start: number, end: number}[]} */
     const found = [];
-    const take = (node) => found.push({ start: node.getStart(sourceFile), end: node.getEnd() });
+    /** @param {any} node a string literal naming a module */
+    const take = (node) => {
+        if (node.text === FROM) {
+            found.push({ start: node.getStart(sourceFile), end: node.getEnd() });
+        } else if (seen && BUILT_IN_INSTEAD[node.text]) {
+            seen.add(node.text);
+        }
+    };
 
     const visit = (node) => {
         // import express from "express", import type { Request } from "express", export * from it.
@@ -160,23 +178,21 @@ function findSpecifiersTypeScript(source, fileName, ts) {
         if (
             (ts.isImportDeclaration(node) || ts.isExportDeclaration(node)) &&
             node.moduleSpecifier &&
-            ts.isStringLiteral(node.moduleSpecifier) &&
-            node.moduleSpecifier.text === FROM
+            ts.isStringLiteral(node.moduleSpecifier)
         ) {
             take(node.moduleSpecifier);
         } else if (
             // import express = require("express"), which is TypeScript's own spelling
             ts.isImportEqualsDeclaration(node) &&
             ts.isExternalModuleReference(node.moduleReference) &&
-            ts.isStringLiteral(node.moduleReference.expression) &&
-            node.moduleReference.expression.text === FROM
+            ts.isStringLiteral(node.moduleReference.expression)
         ) {
             take(node.moduleReference.expression);
         } else if (ts.isCallExpression(node)) {
             const isRequire = ts.isIdentifier(node.expression) && node.expression.text === "require";
             const isDynamicImport = node.expression.kind === ts.SyntaxKind.ImportKeyword;
             const arg = node.arguments[0];
-            if ((isRequire || isDynamicImport) && arg && ts.isStringLiteral(arg) && arg.text === FROM) {
+            if ((isRequire || isDynamicImport) && arg && ts.isStringLiteral(arg)) {
                 take(arg);
             }
         }
@@ -193,9 +209,11 @@ function findSpecifiersTypeScript(source, fileName, ts) {
  * imports at all, and none of those may be rewritten.
  *
  * @param {string} source
+ * @param {Set<string>} [seen] collects the names of the modules with something built in here,
+ *   which are recognised on the same walk rather than on one of their own
  * @returns {{start: number, end: number}[]|null} null when the file does not parse
  */
-function findSpecifiers(source) {
+function findSpecifiers(source, seen) {
     /** @type {any} */
     let tree;
     // A file is either a module or a script and the parser has to be told which. Try module first,
@@ -220,15 +238,23 @@ function findSpecifiers(source) {
 
     /** @type {{start: number, end: number}[]} */
     const found = [];
+    /** @param {any} node a string literal naming a module */
+    const record = (node) => {
+        if (node.value === FROM) {
+            found.push({ start: node.start, end: node.end });
+        } else if (seen && BUILT_IN_INSTEAD[node.value]) {
+            seen.add(node.value);
+        }
+    };
     walk(tree, (node) => {
         // import express from "express", export * from "express"
         if (
             (node.type === "ImportDeclaration" ||
                 node.type === "ExportNamedDeclaration" ||
                 node.type === "ExportAllDeclaration") &&
-            node.source?.value === FROM
+            typeof node.source?.value === "string"
         ) {
-            found.push({ start: node.source.start, end: node.source.end });
+            record(node.source);
             return;
         }
         // require("express") and import("express"), the second being a node of its own
@@ -237,8 +263,8 @@ function findSpecifiers(source) {
         const isDynamicImport = node.type === "ImportExpression";
         if (isRequire || isDynamicImport) {
             const arg = isDynamicImport ? node.source : node.arguments?.[0];
-            if (arg?.type === "Literal" && arg.value === FROM) {
-                found.push({ start: arg.start, end: arg.end });
+            if (arg?.type === "Literal" && typeof arg.value === "string") {
+                record(arg);
             }
         }
     });
@@ -582,6 +608,8 @@ Options:
     const unparsed = [];
     /** @type {string[]} */
     const needTypeScript = [];
+    /** @type {Set<string>} */
+    const builtInInstead = new Set();
 
     // resolved once, and only if there is anything to use it on
     const hasTypeScriptFiles = files.some((file) => TYPESCRIPT_EXTENSIONS.has(path.extname(file)));
@@ -590,8 +618,8 @@ Options:
     for (const file of files) {
         const source = fs.readFileSync(file, "utf8");
         // reading every file's AST to find nothing is the common case, so skip the ones that
-        // cannot contain the specifier at all
-        if (!source.includes(FROM)) continue;
+        // cannot contain any of the names being looked for
+        if (!source.includes(FROM) && !Object.keys(BUILT_IN_INSTEAD).some((name) => source.includes(name))) continue;
 
         const isTypeScript = TYPESCRIPT_EXTENSIONS.has(path.extname(file));
         if (isTypeScript && !ts) {
@@ -599,7 +627,9 @@ Options:
             continue;
         }
 
-        const specifiers = isTypeScript ? findSpecifiersTypeScript(source, file, ts) : findSpecifiers(source);
+        const specifiers = isTypeScript
+            ? findSpecifiersTypeScript(source, file, ts, builtInInstead)
+            : findSpecifiers(source, builtInInstead);
         if (specifiers === null) {
             unparsed.push(path.relative(target, file));
             continue;
@@ -635,6 +665,18 @@ Options:
     console.log(
         `\n${dryRun ? "would rewrite" : "rewrote"} ${changedImports} import(s) in ${changedFiles} file(s) of ${files.length} scanned`
     );
+
+    // said whether or not anything was rewritten: an application migrated last month still has
+    // these, and they are the difference between running on µWS and running through a middleware
+    // that was written for node streams
+    if (builtInInstead.size) {
+        console.log(`\n${builtInInstead.size} module(s) with a faster one built in here, worth replacing by hand:`);
+        for (const name of builtInInstead) {
+            console.log(`  ${name} -> ${BUILT_IN_INSTEAD[name]}`);
+        }
+        console.log("");
+    }
+
     if (changedFiles) {
         console.log(`Remember to install it: npm install ${TO}`);
         printDifferences();

package/src/compression.js

@@ -0,0 +1,400 @@
+/*
+Copyright 2026 Nigro Simone
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// express.compression(), which answers with a compressed body when the client asked for one.
+//
+// The options, the defaults and the order the decision is taken in are the compression module's,
+// so a front that already uses it can drop the require and change nothing else. Two things are
+// different, and both only ever turn a worse answer into a better one:
+//
+//   - a response that arrives whole, which is every res.send() and res.json(), is compressed in
+//     one call instead of through a transform stream, and goes out with a Content-Length. The
+//     bytes are the same bytes: zlib.gzipSync and a createGzip that receives the same body in one
+//     write produce the same deflate output.
+//   - partial content is left alone. The compression module compresses a 206 as well, and the
+//     result is a byte range of the file described as gzip, which no client can decode.
+//
+// The streaming half is the module's own design, because it is the right one: a transform stream,
+// its output written as it comes, and the drain listeners moved onto it so a pipe that fills up
+// hears from the compressor rather than from a socket that is no longer what it is waiting for.
+
+"use strict";
+
+const zlib = require("zlib");
+const bytes = require("bytes");
+const compressible = require("compressible");
+const { negotiateEncoding, ENCODING_ANY } = require("./utils.js");
+
+// Cache-Control: no-transform forbids recoding the body, which is what this does
+const NO_TRANSFORM = /(?:^|,)\s*?no-transform\s*?(?:,|$)/;
+
+// what enforceEncoding is allowed to name, the compression module's list
+const ENFORCEABLE = new Set(["gzip", "deflate", "identity", "br"]);
+
+// Up to this many bytes a whole body is compressed on this thread, and above it on the libuv pool.
+// One call either way; what changes is who waits. A small body pays more for the hop onto the pool
+// than the compression costs, and a large one is worth handing over, since the pool has four
+// threads and the loop has everyone else to serve: measured with gzip at the default level, sync
+// wins by 43% at 1.4KB and by 22% at 16KB, and loses by 32% at 32KB and by 90% at 78KB.
+const SYNC_LIMIT = 24 * 1024;
+
+/**
+ * The default filter: whether the content type is worth compressing at all. A response with no
+ * type is left alone, since nothing says what its bytes are.
+ *
+ * @param {any} req
+ * @param {any} res
+ * @returns {boolean}
+ */
+function shouldCompress(req, res) {
+    const type = res.getHeader("Content-Type");
+    if (type === undefined || !compressible(String(type))) {
+        return false;
+    }
+    return true;
+}
+
+/**
+ * How many bytes a chunk is, which is what the threshold is compared against.
+ *
+ * @param {any} chunk
+ * @param {BufferEncoding} [encoding]
+ * @returns {number}
+ */
+function chunkLength(chunk, encoding) {
+    if (chunk === undefined || chunk === null) {
+        return 0;
+    }
+    return Buffer.isBuffer(chunk) ? chunk.length : Buffer.byteLength(chunk, encoding);
+}
+
+/**
+ * The bytes of a chunk, whatever it arrived as.
+ *
+ * @param {any} chunk
+ * @param {BufferEncoding} [encoding]
+ * @returns {Buffer}
+ */
+function toBuffer(chunk, encoding) {
+    if (Buffer.isBuffer(chunk)) {
+        return chunk;
+    }
+    // end() with nothing to send still has to hand the compressor something, and a threshold of 0
+    // lets an empty body reach it: Buffer.from(undefined) throws where this sends the empty answer
+    if (chunk === undefined || chunk === null) {
+        return Buffer.alloc(0);
+    }
+    return Buffer.from(chunk, encoding);
+}
+
+/**
+ * Compresses a response body as the client asked for it.
+ *
+ * @param {object} [options]
+ * @param {number|string} [options.threshold] the smallest body worth compressing, bytes or "1kb".
+ *   Default 1024. A response whose size is not known in advance is compressed whatever its size.
+ * @param {(req: any, res: any) => boolean} [options.filter] whether this response should be
+ *   compressed at all. The default says yes to any compressible content type.
+ * @param {string} [options.enforceEncoding] what to use when the request carries no
+ *   Accept-Encoding at all. Default "identity", which is to say nothing is compressed.
+ * @param {object} [options.brotli] brotli options, `params` included. The default quality is 4.
+ * @param {number} [options.level] zlib compression level, for gzip and deflate.
+ * @param {number} [options.chunkSize] zlib chunk size.
+ * @param {number} [options.memLevel] zlib memory level.
+ * @param {number} [options.strategy] zlib strategy.
+ * @param {number} [options.windowBits] zlib window size.
+ * @returns {(req: any, res: any, next: (err?: any) => void) => void} the middleware
+ */
+function compression(options) {
+    const opts = options || {};
+    // the whole bag goes to zlib, as the compression module does: level, memLevel, strategy,
+    // windowBits and chunkSize arrive under their own names and zlib ignores the rest
+    const zlibOptions = /** @type {any} */ (opts);
+    const brotliOptions = { ...opts.brotli };
+    brotliOptions.params = {
+        [zlib.constants.BROTLI_PARAM_QUALITY]: 4,
+        ...(opts.brotli && /** @type {any} */ (opts.brotli).params)
+    };
+    const filter = opts.filter || shouldCompress;
+    const enforceEncoding = opts.enforceEncoding || "identity";
+    // bytes.parse reads "1kb" and hands back null for anything it cannot, an absent option
+    // included, which is where the default comes in
+    const threshold = bytes.parse(/** @type {any} */ (opts.threshold)) ?? 1024;
+
+    /**
+     * A whole body, compressed on this thread. Blocks the event loop for as long as it takes,
+     * which is why only a small one comes here, see SYNC_LIMIT.
+     *
+     * @param {string} method
+     * @param {Buffer} body
+     * @returns {Buffer}
+     */
+    function compressWhole(method, body) {
+        if (method === "gzip") {
+            return zlib.gzipSync(body, zlibOptions);
+        }
+        if (method === "br") {
+            return zlib.brotliCompressSync(body, brotliOptions);
+        }
+        return zlib.deflateSync(body, zlibOptions);
+    }
+
+    /**
+     * The same, on the libuv thread pool.
+     *
+     * @param {string} method
+     * @param {Buffer} body
+     * @param {(err: Error|null, out: Buffer) => void} done
+     */
+    function compressWholeAsync(method, body, done) {
+        if (method === "gzip") {
+            zlib.gzip(body, zlibOptions, done);
+        } else if (method === "br") {
+            zlib.brotliCompress(body, brotliOptions, done);
+        } else {
+            zlib.deflate(body, zlibOptions, done);
+        }
+    }
+
+    /**
+     * @param {string} method
+     * @returns {any} the transform stream for a body that arrives in pieces
+     */
+    function compressStream(method) {
+        if (method === "gzip") {
+            return zlib.createGzip(zlibOptions);
+        }
+        if (method === "br") {
+            return zlib.createBrotliCompress(brotliOptions);
+        }
+        return zlib.createDeflate(zlibOptions);
+    }
+
+    return function compression(req, res, next) {
+        const _write = res.write;
+        const _end = res.end;
+        const _on = res.on;
+
+        /** drain listeners parked until there is a compressor to hang them on, see res.on below */
+        let listeners = /** @type {any[][]|null} */ ([]);
+        /** @type {any} */
+        let stream = null;
+        let decided = false;
+        let ended = false;
+        /** what end() was given to call back, held until the compressor has finished */
+        let endCallback = /** @type {any} */ (undefined);
+
+        // the compression module adds this, and code written against it calls it: an SSE feed
+        // pushes its event out with res.flush(). Nothing to flush before there is a compressor
+        res.flush = function flush() {
+            if (stream) {
+                stream.flush();
+            }
+        };
+
+        /**
+         * Hands back the parked drain listeners: this response is not being compressed, so the
+         * response itself is what a pipe should hear from.
+         * @returns {string} the empty method, so the callers can `return noCompress()`
+         */
+        function noCompress() {
+            if (listeners) {
+                for (const listener of listeners) {
+                    _on.call(res, listener[0], listener[1]);
+                }
+                listeners = null;
+            }
+            return "";
+        }
+
+        /**
+         * Whether this response is compressed, and how. Taken once, when the first byte of the
+         * body arrives, which is also when the headers are decided: everything read here is set
+         * by then. The order is the compression module's, and so is the Vary, which is added even
+         * when the answer goes out uncompressed because the answer still depends on the header.
+         *
+         * @param {number} [length] the size of the body, when end() already has all of it
+         * @returns {string} the encoding chosen, "" to send the body as it is
+         */
+        function decide(length) {
+            decided = true;
+            // res.flushHeaders() commits the head here rather than holding it until the body, so a
+            // response that used it has no room left for a Content-Encoding
+            if (res.headersSent) {
+                return noCompress();
+            }
+            if (!filter(req, res)) {
+                return noCompress();
+            }
+            const cacheControl = res.getHeader("Cache-Control");
+            if (cacheControl && NO_TRANSFORM.test(String(cacheControl))) {
+                return noCompress();
+            }
+            res.vary("Accept-Encoding");
+            // NaN when there is no Content-Length, and a comparison against NaN is false: a body
+            // whose size is not known yet is compressed whatever the threshold says
+            if (Number(res.getHeader("Content-Length")) < threshold || Number(length) < threshold) {
+                return noCompress();
+            }
+            const already = res.getHeader("Content-Encoding");
+            if (already && already !== "identity") {
+                return noCompress();
+            }
+            if (req.method === "HEAD") {
+                return noCompress();
+            }
+            // a range is a window into the bytes on disk, and a client that asked for one cannot
+            // decode a compressed answer to it
+            if (res.statusCode === 206 || res.getHeader("Content-Range") !== undefined) {
+                return noCompress();
+            }
+            const accept = req.headers["accept-encoding"];
+            let method = negotiateEncoding(accept === undefined ? "" : accept, ENCODING_ANY);
+            if (accept === undefined && ENFORCEABLE.has(enforceEncoding)) {
+                method = enforceEncoding;
+            }
+            if (!method || method === "identity") {
+                return noCompress();
+            }
+            res.setHeader("Content-Encoding", method);
+            // what it says is the size of the body before this middleware saw it. The whole-body
+            // path below puts the right one back; the streaming one cannot know it in advance
+            res.removeHeader("Content-Length");
+            return method;
+        }
+
+        /**
+         * Starts the compressor for a body that arrives in pieces, and wires it to the response.
+         * @param {string} method
+         */
+        function startStream(method) {
+            stream = compressStream(method);
+            // The parked listeners, and the list itself stays rather than being emptied: res.on
+            // reads it to know that a drain listener belongs on the compressor from here on. That
+            // matters because a pipe registers its own the first time write() tells it to slow
+            // down, which is after this, and from here on the compressor is what fills up.
+            for (const listener of /** @type {any[][]} */ (listeners)) {
+                stream.on(listener[0], listener[1]);
+            }
+            stream.on("data", (chunk) => {
+                if (_write.call(res, chunk) === false) {
+                    stream.pause();
+                }
+            });
+            stream.on("end", () => {
+                _end.call(res, endCallback);
+            });
+            _on.call(res, "drain", () => stream.resume());
+            // an aborted response never reaches the end of the stream, and the zlib context behind
+            // it is native memory that a garbage collector is in no hurry to reach
+            _on.call(res, "close", () => stream.destroy());
+        }
+
+        res.write = function write(chunk, encoding, callback) {
+            if (typeof encoding === "function") {
+                callback = encoding;
+                encoding = undefined;
+            }
+            if (ended) {
+                return false;
+            }
+            if (!decided) {
+                const method = decide();
+                if (method) {
+                    startStream(method);
+                }
+            }
+            if (stream) {
+                return stream.write(toBuffer(chunk, encoding), callback);
+            }
+            return _write.call(this, chunk, encoding, callback);
+        };
+
+        res.end = function end(chunk, encoding, callback) {
+            // node's shapes, of which this project's own end() takes (data, cb): the third
+            // argument only arrives from code written against node's ServerResponse
+            if (typeof chunk === "function") {
+                callback = chunk;
+                chunk = undefined;
+                encoding = undefined;
+            } else if (typeof encoding === "function") {
+                callback = encoding;
+                encoding = undefined;
+            }
+            if (ended) {
+                return this;
+            }
+            if (stream) {
+                ended = true;
+                endCallback = callback;
+                if (chunk === undefined || chunk === null || chunk === "") {
+                    stream.end();
+                } else {
+                    stream.end(toBuffer(chunk, encoding));
+                }
+                return this;
+            }
+            if (!decided) {
+                const method = decide(chunkLength(chunk, encoding));
+                if (method) {
+                    // the whole answer is here, so it is compressed in one call rather than
+                    // through a stream, and goes out with the length it ended up being
+                    ended = true;
+                    const input = toBuffer(chunk, encoding);
+                    if (input.length <= SYNC_LIMIT) {
+                        const body = compressWhole(method, input);
+                        res.setHeader("Content-Length", String(body.length));
+                        return _end.call(this, body, callback);
+                    }
+                    compressWholeAsync(method, input, (err, body) => {
+                        // the client can leave while the pool is working, and writing to a
+                        // response that is already gone is not something uWS survives
+                        if (res.aborted || res.finished) {
+                            return;
+                        }
+                        if (err) {
+                            return res.destroy(err);
+                        }
+                        res.setHeader("Content-Length", String(body.length));
+                        _end.call(res, body, callback);
+                    });
+                    return this;
+                }
+            }
+            ended = true;
+            return _end.call(this, chunk, callback);
+        };
+
+        res.on = function on(type, listener) {
+            if (!listeners || type !== "drain") {
+                return _on.call(this, type, listener);
+            }
+            if (stream) {
+                return stream.on(type, listener);
+            }
+            // there is nothing to listen to yet: a compressor that does not exist has not filled up
+            listeners.push([type, listener]);
+            return this;
+        };
+
+        next();
+    };
+}
+
+module.exports = compression;
+// the compression module exports its default filter, and a front that wants to compress one more
+// type than the default calls it and adds to what it says
+module.exports.filter = shouldCompress;

package/src/index.js

@@ -49,6 +49,7 @@ try {
  *   response: object,
  *   application: object,
  *   static: Function,
+ *   compression: Function,
  *   json: Function,
  *   urlencoded: Function,
  *   text: Function,
@@ -72,6 +73,9 @@ module.exports.response = Response.prototype;
 module.exports.application = Application.Application.prototype;
 
 module.exports.static = middlewares.static;
+// not one of express's, since express has none: the compression module is what everyone installs
+// instead, and this is that middleware's options and behaviour without the install
+module.exports.compression = require("./compression.js");
 module.exports.json = middlewares.json;
 module.exports.urlencoded = middlewares.urlencoded;
 module.exports.text = middlewares.text;

package/src/middlewares.js

@@ -22,11 +22,22 @@ const path = require("path");
 const bytes = require("bytes");
 const zlib = require("fast-zlib");
 const typeis = require("type-is");
+const mime = require("mime-types");
 const qs = require("qs");
 const parseQuery = require("./parse-query.js");
 const { kGetSafe } = require("./usage.js");
 const { AsyncResource } = require("async_hooks");
-const { fastQueryParse, NullObject, asStatError, httpError, memoizeByString, containsDotFile } = require("./utils.js");
+const {
+    fastQueryParse,
+    NullObject,
+    asStatError,
+    httpError,
+    memoizeByString,
+    containsDotFile,
+    negotiateEncoding,
+    ENCODING_BR,
+    ENCODING_GZIP
+} = require("./utils.js");
 
 // largest content-length we will allocate a body buffer for up front. above this the body is
 // collected chunk by chunk instead, so a declared-but-unsent body cannot pin more memory than a
@@ -36,6 +47,14 @@ const MAX_PREALLOCATED_BODY = 1024 * 1024;
 // what the finish pass feeds zlib: no bytes, only the flush flag
 const EMPTY_BUFFER = Buffer.alloc(0);
 
+// What express.static serves instead of the file itself when preCompressed is on and the client
+// takes it: the suffix nginx, brotli_static and every build tool that writes these agree on.
+// Ordered by what is worth having, and negotiation decides between them.
+const PRECOMPRESSED = [
+    { encoding: "br", suffix: ".br", flag: ENCODING_BR },
+    { encoding: "gzip", suffix: ".gz", flag: ENCODING_GZIP }
+];
+
 // The failures express.static answers by moving on to the next handler rather than by reporting
 // them, when fallthrough is on. They all mean the same thing: the request is not a file here.
 //
@@ -248,6 +267,46 @@ function bodyError(message, status, type, extra) {
     return Object.assign(err, extra);
 }
 
+/**
+ * The compressed twin of a file to serve in its place, or undefined when the client would rather
+ * have the file itself or the twin is not there.
+ *
+ * The stat comes back with it, and is what sendFile then answers from: the ETag and the
+ * Last-Modified of a variant are its own, which is the whole point. Two bodies sharing one ETag is
+ * how a shared cache ends up handing brotli to a client that cannot read it.
+ *
+ * At most two stats, and usually one: negotiation picks the best of the two encodings first, and
+ * only looks at the other when the client takes it too and the first file is missing.
+ *
+ * @param {string} filePath absolute path of the file that was asked for
+ * @param {string|undefined} accept the request's Accept-Encoding
+ * @returns {{suffix: string, encoding: string, stat: import("fs").Stats}|undefined}
+ */
+function pickPrecompressed(filePath, accept) {
+    if (!accept) {
+        return undefined;
+    }
+    let allowed = ENCODING_BR | ENCODING_GZIP;
+    // twice at most: the second pass is the case where brotli won and there is no .br on disk
+    for (let attempt = 0; attempt < 2; attempt++) {
+        const chosen = negotiateEncoding(accept, allowed);
+        const variant = PRECOMPRESSED.find((candidate) => candidate.encoding === chosen);
+        if (!variant) {
+            return undefined;
+        }
+        try {
+            const stat = fs.statSync(filePath + variant.suffix);
+            if (!stat.isDirectory()) {
+                return { suffix: variant.suffix, encoding: variant.encoding, stat };
+            }
+        } catch {
+            // not on disk, which is the ordinary case for a file nobody precompressed
+        }
+        allowed &= ~variant.flag;
+    }
+    return undefined;
+}
+
 /**
  * express.static, which is a thin front for res.sendFile: it resolves the path, refuses anything
  * that climbs out of the root, applies the dotfiles and index rules, and hands the rest over.
@@ -337,6 +396,9 @@ function serveStatic(root, options) {
         }
         let _path = url;
         const fullpath = path.resolve(path.join(root, url));
+        // the same file as _path, absolute: the two move together through the index and extension
+        // rules below, and only the precompressed lookup needs the absolute one
+        let filePath = fullpath;
         // What serve-static hands send is this path, except that a bare "/" under a mount the
         // request did not write with one becomes "": without that rule a mount whose root is a file
         // would ask the disk for a directory and could never answer at all.
@@ -400,6 +462,7 @@ function serveStatic(root, options) {
                     try {
                         stat = fs.statSync(fullpath + "." + options.extensions[i]);
                         _path = url + "." + options.extensions[i];
+                        filePath = fullpath + "." + options.extensions[i];
                         break;
                     } catch (extensionError) {
                         statError = extensionError;
@@ -453,6 +516,7 @@ function serveStatic(root, options) {
                 try {
                     stat = fs.statSync(path.join(fullpath, options.index));
                     _path = path.join(url, options.index);
+                    filePath = path.join(fullpath, options.index);
                 } catch (err) {
                     if (!options.fallthrough) {
                         res.status(404);
@@ -473,6 +537,23 @@ function serveStatic(root, options) {
             }
         }
 
+        if (options.preCompressed) {
+            // whatever is served, the answer depended on the header, so a shared cache has to be
+            // told. Said before the lookup, because it is true even when there is no variant
+            res.vary("Accept-Encoding");
+            const variant = pickPrecompressed(filePath, req.headers["accept-encoding"]);
+            if (variant) {
+                _path += variant.suffix;
+                stat = variant.stat;
+                res.setHeader("Content-Encoding", variant.encoding);
+                // from the name of the file that was asked for, since the one being sent ends in
+                // .br and nothing would call that javascript. sendFile leaves a content-type that
+                // is already there alone, which is what makes this the deciding one
+                const type = mime.lookup(filePath);
+                res.type(type || "application/octet-stream");
+            }
+        }
+
         options._stat = stat;
 
         return res.sendFile(

package/src/options.d.ts

@@ -68,6 +68,12 @@ export interface StaticOptions extends SendFileOptions {
     fallthrough?: boolean;
     /** Extensions tried when the path names no file, or false to try none. */
     extensions?: string[] | false;
+    /**
+     * Serve `file.br` or `file.gz` in place of `file` when one is on disk and the client takes it.
+     * Off by default. Vary: Accept-Encoding is sent whether or not a variant is found, and the
+     * content type stays the one the requested name implies.
+     */
+    preCompressed?: boolean;
 }
 
 /** A body parser's options once its factory has filled in every default it needs. */

package/src/response.js

@@ -790,7 +790,10 @@ module.exports = class Response extends LazyWritable {
                 // remembered rather than measured: only a caller that asks for content-length pays
                 // for it, and uWS is measuring the same bytes for the wire anyway
                 this._sentBody = data ?? "";
-                this._res.end(data);
+                // and null is sent as the empty body it means. uWS answers end(null) with a
+                // response the client never sees the end of, where node and express send an empty
+                // 200: res.end(null) is what the compression module's own test suite does
+                this._res.end(data ?? "");
             }
         }
 

package/src/types.d.ts

@@ -17,6 +17,7 @@ limitations under the License.
 declare module "fulmine.js" {
     import e from "express";
     import uWS from "uWebSockets.js";
+    import { ZlibOptions, BrotliOptions } from "zlib";
 
     type Settings = {
         uwsOptions?: uWS.AppOptions;
@@ -37,6 +38,24 @@ declare module "fulmine.js" {
         export import static = e.static;
         // export import query = e.query;
 
+        // express has no compression middleware, so there is nothing to re-export: these are the
+        // compression module's options, which this one takes as they are
+        interface CompressionOptions extends ZlibOptions {
+            /** The smallest body worth compressing, in bytes or as "1kb". Default 1024. */
+            threshold?: number | string;
+            /** Whether this response should be compressed at all. */
+            filter?: (req: e.Request, res: e.Response) => boolean;
+            /** What to use when the request carries no Accept-Encoding. Default "identity". */
+            enforceEncoding?: string;
+            /** Brotli options. The default quality is 4. */
+            brotli?: BrotliOptions;
+        }
+        export function compression(options?: CompressionOptions): e.RequestHandler;
+        export namespace compression {
+            /** The default filter: any compressible content type. */
+            function filter(req: e.Request, res: e.Response): boolean;
+        }
+
         export import urlencoded = e.urlencoded;
 
         export import RouterOptions = e.RouterOptions;

package/src/utils.js

@@ -788,6 +788,113 @@ function stringify(value, replacer, spaces, escape) {
     return json;
 }
 
+// What negotiateEncoding may answer with, since a caller can only offer what it can produce
+const ENCODING_BR = 1;
+const ENCODING_GZIP = 2;
+const ENCODING_DEFLATE = 4;
+const ENCODING_ANY = ENCODING_BR | ENCODING_GZIP | ENCODING_DEFLATE;
+
+/**
+ * The encoding to answer with, read straight off Accept-Encoding rather than through negotiator:
+ * the header is a short list of names with an optional q, and building a Negotiator per response
+ * to read it costs more than the scan does.
+ *
+ * The tie-break is negotiator's, for the list the compression module hands it: brotli first, then
+ * gzip, then deflate, and identity last.
+ *
+ * Only the encodings named in `allowed` are on offer, since the caller may not be able to
+ * produce all three: express.static offers the two it can have lying on disk. An uncompressed
+ * answer is always on offer, and is what an empty header ends up choosing.
+ *
+ * @param {string} accept the header, or "" when the request carried none
+ * @param {number} allowed ENCODING_BR, ENCODING_GZIP and ENCODING_DEFLATE, or'd together
+ * @returns {string} "br", "gzip", "deflate", "identity", or "" when nothing is acceptable
+ */
+function negotiateEncoding(accept, allowed) {
+    // -1 while a name has not appeared: q=0 is a refusal and has to be told apart from silence
+    let br = -1;
+    let gzip = -1;
+    let deflate = -1;
+    let identity = -1;
+    let star = -1;
+    // the lowest q anything was named with, which is what an unnamed identity is worth, see below
+    let minQuality = 1;
+    let index = 0;
+    while (index < accept.length) {
+        let end = accept.indexOf(",", index);
+        if (end === -1) {
+            end = accept.length;
+        }
+        let semi = accept.indexOf(";", index);
+        if (semi === -1 || semi > end) {
+            semi = end;
+        }
+        const name = accept.slice(index, semi).trim().toLowerCase();
+        let q = 1;
+        if (semi < end) {
+            const params = accept.slice(semi + 1, end);
+            const at = params.indexOf("q=");
+            if (at !== -1) {
+                const parsed = parseFloat(params.slice(at + 2));
+                // a q nobody can read is a refusal, which is how negotiator reads it too
+                q = parsed === parsed ? parsed : 0;
+            }
+        }
+        if (q < minQuality) {
+            minQuality = q;
+        }
+        switch (name) {
+            case "br":
+                br = q;
+                break;
+            case "gzip":
+                gzip = q;
+                break;
+            case "deflate":
+                deflate = q;
+                break;
+            case "identity":
+                identity = q;
+                break;
+            case "*":
+                star = q;
+                break;
+        }
+        index = end + 1;
+    }
+    if (br < 0) br = star;
+    if (gzip < 0) gzip = star;
+    if (deflate < 0) deflate = star;
+    // An uncompressed answer that the request did not name is worth the lowest q it named
+    // anything with, which is negotiator's rule and not the obvious one: "br;q=0.5, gzip;q=0.9"
+    // means gzip, because identity comes in at 0.5 rather than at 1 and does not win the list.
+    // A "*" names identity as much as it names anything else, so its q is identity's.
+    if (identity < 0) identity = star < 0 ? minQuality : star;
+
+    if (!(allowed & ENCODING_BR)) br = -1;
+    if (!(allowed & ENCODING_GZIP)) gzip = -1;
+    if (!(allowed & ENCODING_DEFLATE)) deflate = -1;
+
+    let best = "";
+    let bestQ = 0;
+    if (br > bestQ) {
+        best = "br";
+        bestQ = br;
+    }
+    if (gzip > bestQ) {
+        best = "gzip";
+        bestQ = gzip;
+    }
+    if (deflate > bestQ) {
+        best = "deflate";
+        bestQ = deflate;
+    }
+    if (identity > bestQ) {
+        best = "identity";
+    }
+    return best;
+}
+
 const defaultSettings = {
     "jsonp callback name": "callback",
     env: () => process.env.NODE_ENV ?? "development",
@@ -1305,6 +1412,10 @@ module.exports = {
     entityTag,
     statTag,
     contentTypeFor,
+    negotiateEncoding,
+    ENCODING_BR,
+    ENCODING_GZIP,
+    ENCODING_ANY,
     memoizeByString,
     isRangeFresh,
     findIndexStartingFrom,