fulmine.js 5.3.0 → 5.4.0

package/README.md

@@ -53,6 +53,7 @@ shared machine, so the number would describe the machine rather than the framewo
 - [WebSockets](#websockets)
     - [socket.io](#socketio)
 - [HTTP/3](#http3)
+- [Behind a proxy](#behind-a-proxy)
 - [Versioning](#versioning)
 - [Compatibility](#compatibility)
     - [express](#express)
@@ -246,6 +247,56 @@ app.listen(3000, () => {
 
 ## Performance tips
 
+Where the speed comes from, before the rules that govern it. Express finds a route by walking its
+stack and testing each layer against the path. Fulmine hands every route it can to µWS's own router,
+which matches in C++, and works out at `listen()` which layers stand in front of each one, so
+arriving at a handler costs no matching at all:
+
+```text
+   Express                              Fulmine
+   GET /users/42                        GET /users/42
+        |                                    |
+        v                                    v
+   +--------------+                    +------------------+
+   | layer 1      | path? no           |   µWS router     |  one match, in C++,
+   | layer 2      | path? no           |   /users/:id     |  against every path
+   | ...          |                    +--------+---------+  registered
+   | layer 214    | path? yes -+                |
+   +--------------+            |                v
+     a test per layer,         |       +------------------+
+     every request             |       | the chain, known |  the layers in front,
+                               |       | since listen()   |  in order, no matching
+                               v       +--------+---------+
+                            handler             |
+                                                v
+                                             handler
+```
+
+That is the whole difference on a large route table: the scan grows with the table and the match
+does not, which is why a thousand routes measure 10x and a handful measure 3x.
+
+Two more things happen on the way in, and `npx fulmine profile` will tell you which of them your
+routes get:
+
+```text
+   a request arriving at a compiled route
+
+   µWS match ──► the chain ──────────────────────────► handler ──► response
+                    |                                     |
+                    |  a body parser is stepped over      |  the Readable is not
+                    |  when the request declared no       |  built unless something
+                    |  body and the verb reads none       |  asks the body for one
+                    |                                     |
+                    |  the headers are not copied out     |  the two internal
+                    |  of µWS when the analysis proved    |  listeners are written
+                    |  nothing in the chain reads one     |  into the event map
+                    v                                     v
+              work that does not happen           work that is not prepared
+
+   and when the handler is simple enough to be read at registration time, none of the
+   above happens either: µWS answers from a response written once, at startup
+```
+
 1. Fulmine tries to optimize routing as much as possible, but it's only possible if:
 
 - the path is a plain string, or its parameters are whole segments: `/users/:id` and `/a/:b/c/:d` qualify, `/flights/:from-:to` does not, and neither does a `*splat` or a `{}` group. Routing is case-insensitive by default, as in Express; a request in the registered case is still served natively, any other case takes the ordinary path, and a route whose overlap with an earlier one leans on a cased literal goes the ordinary way for every request.
@@ -378,6 +429,32 @@ const app = express({
 });
 ```
 
+## Behind a proxy
+
+`trust proxy` works as it does in Express: set it and `req.ip`, `req.ips`, `req.protocol` and
+`req.hostname` are read from `X-Forwarded-*` when the connection comes from a peer you trust.
+
+Fulmine adds the other way of being told, the one that does not use headers at all. HAProxy, AWS
+NLB, nginx with `proxy_protocol` and Envoy can prepend a **PROXY protocol** preamble to the
+connection, and µWebSockets.js parses it. Off by default, and one line turns it on:
+
+```js
+app.set("trust proxy protocol", true);
+// req.ip, req.socket.remoteAddress and everything reading them are now the address the proxy
+// declared, and fall back to the socket's own on a connection that sent no preamble
+```
+
+> [!WARNING]
+> **Only turn this on when nothing but the proxy can reach the server.** µWS reads the preamble
+> from whoever sends it. There is no way to say which peers may use it, so on a port open to the
+> internet the first sixteen bytes of any connection are enough for a client to become `10.0.0.1`
+> for your rate limiter, your allow list and your audit log. Bind to the private interface, or
+> keep this off.
+
+`trust proxy` and this can both be on. The preamble decides what the connection's address is, and
+`trust proxy` then peels `X-Forwarded-For` off that, so a proxy that sends both is read the way it
+meant.
+
 ## Versioning
 
 **The major number tracks Express, not semver.** Fulmine 5.x follows Express 5. If Express 6
@@ -461,9 +538,11 @@ Two of these keep a compiled form alongside the value, which you can also set di
 - `etag fn`, the function that produces an ETag. Setting `etag` compiles one; setting this replaces it.
 - `query parser fn`, likewise for `query parser`.
 
-Fulmine adds one of its own:
+Fulmine adds three of its own:
 
 - `declarative responses`, on by default. Lets a simple enough handler be compiled into a native uWS response, described under [Performance tips](#performance-tips).
+- `file cache`, on by default. Small files served by `res.sendFile` come from a bounded in-process cache, checked against the file's `stat` on every request, so an edited file is never served stale.
+- `trust proxy protocol`, off by default. Takes `req.ip` from a PROXY protocol preamble, described under [Behind a proxy](#behind-a-proxy). Read the warning there before turning it on.
 
 ### Request
 
@@ -582,7 +661,8 @@ Almost all middlewares that are compatible with Express are compatible with Fulm
 - ✅ [express-rate-limit](https://npmjs.com/package/express-rate-limit)
 - ✅ [express-subdomain](https://npmjs.com/package/express-subdomain)
 - ✅ [vhost](https://npmjs.com/package/vhost)
-- ✅ [tsoa](https://github.com/lukeautry/tsoa)
+- ✅ [http-proxy-middleware](https://www.npmjs.com/package/http-proxy-middleware)
+- ✅ [express-http-proxy](https://www.npmjs.com/package/express-http-proxy)
 - ✅ [express-mongo-sanitize](https://www.npmjs.com/package/express-mongo-sanitize)
 - ✅ [helmet](https://www.npmjs.com/package/helmet)
 - ✅ [passport](https://www.npmjs.com/package/passport)
@@ -592,6 +672,10 @@ Almost all middlewares that are compatible with Express are compatible with Fulm
 - ✅ [better-sse](https://www.npmjs.com/package/better-sse)
 - ✅ [supertest](https://www.npmjs.com/package/supertest)
 
+[tsoa](https://github.com/lukeautry/tsoa) works too, but it is not in the suite above: it resolves
+`express` itself, so testing it here needs a dependency override rather than the one-line swap
+everything else takes.
+
 ## Tested view engines
 
 Any Express view engine should work. Here's list of engines we include in our test suite:

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "fulmine.js",
-    "version": "5.3.0",
+    "version": "5.4.0",
     "description": "Drop-in Express 5 replacement on uWebSockets.js. Your existing middleware keeps working.",
     "main": "src/index.js",
     "bin": {
@@ -30,7 +30,7 @@
         "cover:full": "nyc --silent npm run test && nyc --silent --no-clean npm run test:unit && nyc --silent --no-clean npm run test:express && nyc report",
         "cover:check": "nyc check-coverage --statements 93 --branches 88 --functions 92 --lines 93",
         "demo:start": "npm --prefix demo install && npm --prefix demo start",
-        "demo:deploy": "fly deploy ./demo --config ./demo/fly.toml",
+        "demo:deploy": "cd demo && fly deploy",
         "demo:logs": "fly logs --app fulmine-demo"
     },
     "engines": {
@@ -128,6 +128,7 @@
         "exit-hook": "^2.2.1",
         "express": "^5",
         "express-art-template": "^1.0.1",
+        "express-basic-auth": "^1.2.1",
         "express-dot-engine": "^1.0.8",
         "express-fast-json-stringify": "^1.3.0",
         "express-fileupload": "^1.5.2",
@@ -137,6 +138,7 @@
         "express-rate-limit": "^8.5.2",
         "express-session": "^1.19.0",
         "express-subdomain": "^1.0.6",
+        "express-validator": "^7.3.2",
         "fast-querystring": "^1.1.2",
         "globals": "^17.8.0",
         "graphql-http": "^1.22.4",
@@ -149,6 +151,8 @@
         "multer": "^2.1.1",
         "mustache-express": "^1.3.2",
         "nyc": "^17.1.0",
+        "on-finished": "^2.4.1",
+        "on-headers": "^1.1.0",
         "pako": "^2.1.0",
         "passport": "^0.7.0",
         "passport-local": "^1.0.0",
@@ -157,6 +161,7 @@
         "pug": "^3.0.4",
         "release-it": "^21.0.1",
         "response-time": "^2.3.4",
+        "serve-favicon": "^2.5.1",
         "serve-index": "^1.9.2",
         "serve-static": "^2.2.1",
         "socket.io": "^4.8.3",

package/src/declarative.js

@@ -166,6 +166,15 @@ module.exports = function compileDeclarative(cb, app) {
         // out as `return false`, which is the fallback to ordinary routing. That is the design, and
         // it is why the tree is walked loosely: acorn types every shape JavaScript can take, and
         // enumerating them here would be a second, worse copy of that catch.
+        //
+        // The list below looks like the thing to widen, and it is not. Counted over the 1113
+        // handlers in this repository's tests, demo and benchmark scenarios: 42.6% call something
+        // that is not res, which no syntax admitted here can reach, and admitting `const` would
+        // unlock 7 handlers, 0.6%, one conditional a further 0.1%. On the demo and the benchmark
+        // alone, where the handlers look more like an application's, the share that calls something
+        // rises to 59% and the two would unlock nothing at all. What keeps a route off this path is
+        // that its answer is not knowable until the request arrives, and a variable is only another
+        // way to spell an answer that already was.
         /** @type {any[]} */
         const tokens = [...acorn.tokenizer(code, { ecmaVersion: "latest" })];
 

package/src/node-shim.js

@@ -24,6 +24,9 @@ limitations under the License.
 
 const { IncomingMessage } = require("http");
 
+/** What µWS returns for an address nobody declared. */
+const emptyAddress = new ArrayBuffer(0);
+
 /**
  * An IP address as the four or sixteen bytes uWS hands over, since that is what req.ip parses.
  * Anything unreadable comes back empty, which req.ip reports as undefined, the same answer it gives
@@ -370,6 +373,14 @@ class NodeHttpResponse {
         return Buffer.from(this._nodeReq.socket?.remoteAddress || "");
     }
 
+    /**
+     * Always empty: node's server does not read the PROXY protocol, so a request that arrived
+     * through it never carries an address a proxy declared, whatever "trust proxy protocol" says.
+     */
+    getProxiedRemoteAddress() {
+        return emptyAddress;
+    }
+
     /** The client port, or 0 when the socket has already gone. */
     getRemotePort() {
         return this._nodeReq.socket?.remotePort ?? 0;

package/src/request.js

@@ -125,6 +125,9 @@ function mapsIPv4Peer(app) {
     return !(host && isIP(host) === 4);
 }
 
+/** What µWS returns for a proxied address when no PROXY protocol preamble arrived. */
+const emptyAddress = new ArrayBuffer(0);
+
 const discardedDuplicates = new Set([
     "age",
     "authorization",
@@ -146,8 +149,6 @@ const discardedDuplicates = new Set([
     "user-agent"
 ]);
 
-let key = 0;
-
 // 128 KB of body buffered before uWS is asked to pause
 const READABLE_OPTIONS = { highWaterMark: 128 * 1024 };
 
@@ -418,6 +419,13 @@ module.exports = class Request extends LazyReadable {
      */
     rawIp;
 
+    /**
+     * Whether rawIp came from a PROXY protocol preamble rather than from the socket. Only the
+     * IPv4 mapping reads it, see parsedIp. Declared for the same reason as rawIp.
+     * @type {boolean}
+     */
+    _ipFromProxy = false;
+
     /**
      * Whether the request declared a body, content-length or transfer-encoding, spotted during
      * the header copy. Declared for the same reason as rawIp.
@@ -489,6 +497,14 @@ module.exports = class Request extends LazyReadable {
             // framework itself: body framing, keep-alive, and accept for the error page a
             // throw could still need. A GET that does declare a body is the rare case, and
             // the parsers and the stream want the whole picture, so it takes the full copy.
+            //
+            // Seven named reads against one forEach looks like it should lose, and does not: the
+            // seven are flat at 0.75us however many headers are on the wire, since each one is a
+            // napi crossing and the scan behind it is nothing, while the copy pays a hop back into
+            // JS per header and grows, 1.16us at four headers, 1.61 at eight, 2.90 at sixteen. They
+            // do not cross, and the gap widens exactly where real traffic lives, since a browser
+            // sends a dozen or more. The body case pays two of the seven and then copies anyway,
+            // which is 0.2us on a request that is about to read a body.
             const length = req.getHeader("content-length");
             const transferEncoding = req.getHeader("transfer-encoding");
             if (length !== "" || transferEncoding !== "") {
@@ -533,10 +549,6 @@ module.exports = class Request extends LazyReadable {
             currentRequest = null;
         }
         this.routeCount = 1;
-        this.key = key++;
-        if (key > 100000) {
-            key = 0;
-        }
         this.app = app;
         // both forms are kept, because both are asked for: the query with its "?" goes into
         // req.url, and req.query parses the raw one. Keeping only the first meant slicing the "?"
@@ -608,10 +620,15 @@ module.exports = class Request extends LazyReadable {
         this._appStack = undefined;
         this.receivedData = false;
         // reading ip is very slow in UWS, so its better to not do it unless truly needed
-        if (this.app.needsIpAfterResponse || this.key < 100) {
-            // if app needs ip after response, read it now because after response its not accessible
-            // also read it for first 100 requests to not error
-            this.rawIp = this._res.getRemoteAddress();
+        if (app.needsIpAfterResponse) {
+            // an app that has been seen asking after the response reads it now, because by then
+            // µWS has freed it
+            this.rawIp = this._readRawIp();
+        } else if (app._ipProbes < 100) {
+            // and until this app has been seen either way, the first hundred requests read it, so
+            // one of them can be the one that finds out
+            app._ipProbes++;
+            this.rawIp = this._readRawIp();
         }
 
         // A body exists on the wire only when the request declares one, content-length or
@@ -884,27 +901,39 @@ module.exports = class Request extends LazyReadable {
     }
 
     /**
-     * The query string parsed by whichever parser the "query parser" setting names, cached for the
-     * life of the request. A null-prototype object, so a key like "__proto__" cannot reach
-     * Object.prototype. No setter, so assigning to req.query throws as it does on Express.
+     * The query string parsed by whichever parser the "query parser" setting names. A null-prototype
+     * object, so a key like "__proto__" cannot reach Object.prototype. No setter, so assigning to
+     * req.query throws as it does on Express.
+     *
+     * Every read answers a new object, because express's getter re-parses on every read and so hands
+     * one back too. Two consequences an application can see, and both of them bite: `req.query` is
+     * never the object another reader holds, and a write to a key of it is gone by the next read.
+     * That second one is how express-validator's sanitisers behave: `.trim()` on a query parameter
+     * changes nothing an ordinary handler will see, which is why it also offers matchedData(). With
+     * the parse cached and handed out as itself, the sanitised value leaked into req.query here and
+     * a handler written against express read a trimmed value where express gives it the raw one.
+     *
+     * The parse itself is still done once. What is copied per read is the shallow result, which is
+     * cheaper than express's re-parse and answers the same for everything but a write to a nested
+     * key, which only the extended parser can produce.
      *
      * @returns {Record<string, any>}
      */
     get query() {
-        if (this.#cachedQuery) {
-            return this.#cachedQuery;
+        let parsed = this.#cachedQuery;
+        if (parsed === null) {
+            const qp = this.app.get("query parser fn");
+            // the vendored default already answers on a bare null prototype, so it goes out as is;
+            // any other parser is copied onto one, which is what kept fast-querystring's result from
+            // inspecting as "Empty <[Object: null prototype] {}>" where Express shows the bare form
+            parsed = qp
+                ? qp === parseQuery
+                    ? parseQuery(this._rawQuery)
+                    : Object.assign(Object.create(null), qp(this._rawQuery))
+                : Object.create(null);
+            this.#cachedQuery = parsed;
         }
-        const qp = this.app.get("query parser fn");
-        // the vendored default already answers on a bare null prototype, so it goes out as is;
-        // any other parser is copied onto one, which is what kept fast-querystring's result from
-        // inspecting as "Empty <[Object: null prototype] {}>" where Express shows the bare form
-        const parsed = qp
-            ? qp === parseQuery
-                ? parseQuery(this._rawQuery)
-                : Object.assign(Object.create(null), qp(this._rawQuery))
-            : Object.create(null);
-        this.#cachedQuery = parsed;
-        return parsed;
+        return Object.assign(Object.create(null), parsed);
     }
 
     /**
@@ -949,6 +978,32 @@ module.exports = class Request extends LazyReadable {
         return typeof val === "string" && val.toLowerCase() === "xmlhttprequest";
     }
 
+    /**
+     * The peer address bytes, from the socket or, when the application asked for it, from a PROXY
+     * protocol preamble the load balancer in front of this server sent ahead of the request.
+     *
+     * The setting is off by default and has to stay that way. µWS parses the preamble from whoever
+     * sends it, with nothing to ask for it at listen time and no way to restrict who may, so an
+     * application that took the address unconditionally would let any client claim any address:
+     * the first sixteen bytes of a connection are enough to become 10.0.0.1 for a rate limiter, an
+     * allow list or an audit log. Turn it on only when nothing can reach this server except the
+     * proxy in front of it.
+     *
+     * @returns {ArrayBuffer} the socket's own address when no preamble arrived
+     */
+    _readRawIp() {
+        const uwsRes = this._res;
+        if (this.app.get("trust proxy protocol")) {
+            const proxied = uwsRes.getProxiedRemoteAddress();
+            // empty unless a preamble arrived, which is the only thing that tells the two apart
+            if (proxied.byteLength !== 0) {
+                this._ipFromProxy = true;
+                return proxied;
+            }
+        }
+        return uwsRes.getRemoteAddress();
+    }
+
     /**
      * The peer address as text, read from uWS and cached. Reading it is expensive and it is gone
      * once the response has finished, so it is read up front for the first hundred requests, and
@@ -971,7 +1026,7 @@ module.exports = class Request extends LazyReadable {
                 // fallback once
                 return mapsIPv4Peer(this.app) ? "::ffff:127.0.0.1" : "127.0.0.1";
             }
-            this.rawIp = this._res.getRemoteAddress();
+            this.rawIp = this._readRawIp();
         }
         // read once: the branch above settled it, and every use below wants the bytes
         const rawIp = /** @type {ArrayBuffer} */ (this.rawIp);
@@ -980,7 +1035,10 @@ module.exports = class Request extends LazyReadable {
         if (rawIp.byteLength === 4) {
             // ipv4
             ip = new Uint8Array(rawIp).join(".");
-            if (mapsIPv4Peer(this.app)) {
+            // the mapped form belongs to a dual stack listener, which is what makes an IPv4 peer
+            // arrive as ::ffff:a.b.c.d. An address a proxy declared never came through that socket,
+            // so it is left as the four numbers the proxy sent
+            if (!this._ipFromProxy && mapsIPv4Peer(this.app)) {
                 ip = "::ffff:" + ip;
             }
         } else if (rawIp.byteLength === 16) {
@@ -1053,12 +1111,15 @@ module.exports = class Request extends LazyReadable {
     _detachFromResponse() {
         const uwsRes = this._res;
         if (!this.rawIp) {
-            this.rawIp = uwsRes.getRemoteAddress();
+            this.rawIp = this._readRawIp();
         }
         const remotePort = uwsRes.getRemotePort();
         const rawIp = this.rawIp;
         this._res = {
             getRemoteAddress: () => rawIp,
+            // whatever a preamble said is already in rawIp, and asking again is the use after free
+            // this method exists to avoid
+            getProxiedRemoteAddress: () => emptyAddress,
             getRemotePort: () => remotePort,
             // a body cannot arrive on an upgraded socket, and a stray reader must not reach µWS
             onData() {},

package/src/response.js

@@ -121,7 +121,84 @@ function statusLine(code, text) {
     return `${code} ${text ?? statuses.message[code] ?? "unknown"}`.trim();
 }
 
-module.exports = class Response extends Writable {
+/**
+ * A Writable that has not built its state yet, the mirror of LazyReadable in request.js and there
+ * for the same reason: a response is a Writable because middleware expects one, and the ordinary
+ * one never uses it. `send()` reaches `end()`, which is overridden here and goes straight to
+ * _finish, so the WritableState is allocated for every response and read by nobody. It is needed
+ * only by res.write(), by a stream piped into the response, by cork and by the writableX getters.
+ *
+ * `Response extends LazyWritable`, whose prototype is Writable's, so `res instanceof Writable`
+ * stays true and every Writable method is reachable; what is missing is `_writableState`, built on
+ * the first touch. Measured at 45 nanoseconds a response on the machine this was written on.
+ *
+ * As on the Readable side the wrapping is generated rather than written out: every own member of
+ * Writable's prototype gets a version that materialises first, so there is no list to keep in step.
+ * Missing one would not be a slow path, it would be a TypeError on `undefined._writableState`.
+ */
+class LazyWritableBase {}
+Object.setPrototypeOf(LazyWritableBase.prototype, Writable.prototype);
+Object.setPrototypeOf(LazyWritableBase, Writable);
+
+// what the chain says at runtime, said again for the type checker, which cannot see a prototype
+// being reassigned
+const LazyWritable = /** @type {typeof Writable} */ (/** @type {unknown} */ (LazyWritableBase));
+
+/**
+ * Builds the stream this object has been pretending to be. Idempotent: everything reachable from
+ * outside goes through it, so it is called far more often than it does anything.
+ *
+ * EventEmitter's init keeps an _events that is already there, so both the shape the constructor
+ * wrote and any listener added before this survive it.
+ *
+ * @param {any} stream
+ */
+function materialiseWritable(stream) {
+    if (stream._writableState === undefined) {
+        Writable.call(stream);
+    }
+}
+
+for (const member of [
+    ...Object.getOwnPropertyNames(Writable.prototype),
+    ...Object.getOwnPropertySymbols(Writable.prototype)
+]) {
+    if (member === "constructor") {
+        continue;
+    }
+    const descriptor = /** @type {PropertyDescriptor} */ (Object.getOwnPropertyDescriptor(Writable.prototype, member));
+    if (typeof descriptor.value === "function") {
+        const inner = descriptor.value;
+        Object.defineProperty(LazyWritableBase.prototype, member, {
+            ...descriptor,
+            /** @this {any} @param {...any} args */
+            value: function (...args) {
+                materialiseWritable(this);
+                return inner.apply(this, args);
+            }
+        });
+    } else if (descriptor.get || descriptor.set) {
+        const innerGet = descriptor.get;
+        const innerSet = descriptor.set;
+        Object.defineProperty(LazyWritableBase.prototype, member, {
+            ...descriptor,
+            get: innerGet
+                ? /** @this {any} */ function () {
+                      materialiseWritable(this);
+                      return innerGet.call(this);
+                  }
+                : undefined,
+            set: innerSet
+                ? /** @this {any} @param {any} value */ function (value) {
+                      materialiseWritable(this);
+                      innerSet.call(this, value);
+                  }
+                : undefined
+        });
+    }
+}
+
+module.exports = class Response extends LazyWritable {
     /** @type {Socket|null} */
     #socket = null;
 
@@ -151,6 +228,19 @@ module.exports = class Response extends Writable {
      */
     constructor(res, req, app) {
         super();
+        // the EventEmitter half stays eager, since the stream half is what LazyWritable defers and
+        // the two listeners below are written straight into this map. These are the five keys and
+        // the order node's own Writable constructor lays down, so the hidden class is the one every
+        // other stream in the process has, and node's init keeps this object when the state is
+        // finally built
+        this._events = {
+            close: undefined,
+            error: undefined,
+            prefinish: undefined,
+            finish: undefined,
+            drain: undefined
+        };
+        this._eventsCount = 0;
         this._req = req;
         // linked here rather than by the caller: the pair is built together, and a field the
         // constructor leaves unset is a shape change on whoever assigns it first
@@ -185,6 +275,10 @@ module.exports = class Response extends Writable {
         }
 
         this.body = undefined;
+        // what was handed to uWS, kept so a caller asking for content-length after the fact can be
+        // answered, see get(). Undefined until the response ends, and for one that sends no body
+        /** @type {string|Buffer|undefined} */
+        this._sentBody = undefined;
         // false while the uWS route handler is still in its synchronous window, where uWS holds
         // the socket corked itself; the two uWS entry points flip it once that window closes
         this._corkNeeded = false;
@@ -448,6 +542,12 @@ module.exports = class Response extends Writable {
         // the value here is nearly always the 10-char "keep-alive", which paid a scan per response
         const closing =
             typeof connection === "string" && connection.length === 5 && connection.toLowerCase() === "close";
+        // for..in over an object some responses delete from, which is the shape the request side
+        // was taken off for #rawHeadersEntries. It stays here, and the difference is where the
+        // deletes are: a 200 with a body performs none. Only 204, 304, 205, the freshness branch of
+        // sendFile and removeHeader do, this object is built fresh per response, so a dictionary one
+        // of them made costs that response and nothing after it. The request side deleted on every
+        // request, which is what made it worth a different structure
         for (const header in headers) {
             if (closing && header === "keep-alive") {
                 continue;
@@ -584,8 +684,12 @@ module.exports = class Response extends Writable {
             // an allocation per body, and uWS reads the view's own offset and length
             if (this.req.method === "HEAD") {
                 const length = Buffer.byteLength(data ?? "");
+                this.headers["content-length"] = String(length);
                 this._res.endWithoutBody(length.toString());
             } else {
+                // remembered rather than measured: only a caller that asks for content-length pays
+                // for it, and uWS is measuring the same bytes for the wire anyway
+                this._sentBody = data ?? "";
                 this._res.end(data);
             }
         }
@@ -1185,7 +1289,19 @@ module.exports = class Response extends Writable {
      * @returns {string|string[]|undefined}
      */
     get(field) {
-        return this.headers[field.toLowerCase()];
+        const name = field.toLowerCase();
+        const value = this.headers[name];
+        // Content-Length is on the wire but not in here: uWS measures the body it is handed and
+        // writes the header itself, which saves measuring it twice. Express sets it in send(), so
+        // anything reading it back finds it there, and morgan's common and combined formats do
+        // exactly that on every line they write. Worked out here rather than in send() so a
+        // response nobody asks pays nothing, and kept once worked out.
+        if (value === undefined && name === "content-length" && this._sentBody !== undefined) {
+            const length = Buffer.byteLength(this._sentBody);
+            this.headers["content-length"] = String(length);
+            return String(length);
+        }
+        return value;
     }
 
     /**
@@ -1362,23 +1478,22 @@ module.exports = class Response extends Writable {
 
         this.vary("Accept");
 
-        // req.next, not the router next sendFile and render use. Express's is the router's here
-        // too, so inside a route with a four argument handler of its own this hands the error to
-        // that handler where express would have left the route. Passing the router next instead
-        // fails express's own res.format test, which asserts the handler is given the very same
-        // function the surrounding middleware received: outside a route the two are equivalent but
-        // not identical. The whole thing is the open req.next question at Walk's constructor
+        // the router next, as express hands over: inside a route with a four argument handler of
+        // its own, a 406 leaves the route rather than being caught by that handler. Where the two
+        // are the same step, _leaveRoute is the very object the surrounding layer received, which
+        // is what express's own test asserts, see Walk#runRoute
+        const next = this.req._leaveRoute ?? this.req.next;
         if (key) {
             this.set("Content-Type", normalizeType(key).value);
-            object[key](this.req, this, this.req.next);
+            object[key](this.req, this, next);
         } else if (object.default) {
-            object.default(this.req, this, this.req.next);
+            object.default(this.req, this, next);
         } else {
             // an error and not an answer: express hands the error handler the types it could have
             // sent, which is how an application says what it supports
             const err = httpError(406);
             err.types = keys.map((type) => normalizeType(type).value);
-            this.req.next(err);
+            next(err);
         }
 
         return this;

package/src/router.js

@@ -333,7 +333,11 @@ class Walk {
             }
         }
         req.next = this.next;
-        req._leaveRoute = this.leaveRoute;
+        // the same step when the route has one callback, and then it has to be the same object:
+        // express hands res.format's handlers the next its own layer received, and its test asserts
+        // that identity. With more than one callback the two differ for real, and what express
+        // hands over is the one that leaves the route
+        req._leaveRoute = route.callbacks.length > 1 ? this.leaveRoute : this.next;
         if (continueRoute === "route") {
             this.step("route");
         } else if (continueRoute) {
@@ -920,6 +924,16 @@ const BODY_METHODS = new Set(["POST", "PUT", "PATCH", "QUERY"]);
  * written on, and the parser prologue it reaches measured 38. Ten to one, for a layer that had
  * nothing to do.
  *
+ * That number is also why fusing consecutive layers into one generated function keeps coming up,
+ * and why it is not here. Counted over a real front, morgan, helmet, compression, cors, the two body
+ * parsers, express-session, a middleware of one's own and express.static: three of the nine can be
+ * fused, and the longest run of fusable ones in a row is one. Fusing needs two. The rule was relaxed
+ * from "calls next once, unconditionally" to merely "calls next synchronously" and the answer did
+ * not move, because the six that fail all call next from inside a callback: they are asynchronous by
+ * nature, reading a body, stat-ing a file, loading a session. A layer that has not decided by the
+ * time it returns cannot be fused by any design that keeps the semantics. What fuses is a run of
+ * trivial middlewares, which is a benchmark shape rather than an application's.
+ *
  * @param {any} route
  * @param {any} req
  * @returns {boolean}
@@ -956,6 +970,68 @@ function couldAnswer(route, path) {
     return route.pattern === path;
 }
 
+/**
+ * Whether a layer written before a mount could answer a request for one of the paths inside it.
+ *
+ * A mount covers everything under its path, so this is a question about a subtree rather than about
+ * the mount point, and the two answers differ: `/a` and `/:p0/:p1/:p2` match none of each other's
+ * text, and both answer `/a/x/y`. µWS jumps straight to whichever leaf it registered, so a leaf a
+ * layer like this could have answered has to stay on the generic path, which is the only place
+ * express's registration order decides.
+ *
+ * Only layers with more segments than the mount path reach this: one with as few already matches
+ * the mount point itself, and _optimizeRoute has refused the mount before the walk gets here.
+ *
+ * Compared folded whichever way the routers are set. A wrong yes costs a leaf its native
+ * registration and nothing else.
+ *
+ * @param {{path: string, use: boolean, method: string, all: boolean}} guard
+ * @param {string} leafPath the leaf's absolute path, parameters and all
+ * @param {any} leaf
+ * @returns {boolean}
+ */
+function shadowsLeaf(guard, leafPath, leaf) {
+    if (!guard.all && guard.method !== leaf.method && !(guard.method === "HEAD" && leaf.method === "GET")) {
+        return false;
+    }
+    return pathsCanOverlap(guard.path.toLowerCase(), leafPath.toLowerCase(), guard.use);
+}
+
+/**
+ * The layers before a mount that answer some of what is inside it and not all of it, which is the
+ * one thing neither the chain nor µWS's own choice can say: the chain runs what is in it without
+ * matching again, and µWS picks by specificity. They are carried down the walk instead and asked
+ * about every leaf, see shadowsLeaf.
+ *
+ * @param {any} router the router the mount belongs to
+ * @param {any} mount
+ * @param {string} pathPrefix what the mounts above this one consumed
+ * @param {any[]} chain the layers that always run before the mount, which need no guard
+ * @param {any[]} inherited the guards from further out, since a mount two levels down is under
+ *   everything written before either of them
+ * @returns {any[]|null} null when a path cannot be read segment by segment, which leaves the mount
+ *   to ordinary dispatch rather than guessing about it
+ */
+function guardsInside(router, mount, pathPrefix, chain, inherited) {
+    let guards = inherited;
+    for (const r of router._routes) {
+        if (r.routeKey > mount.routeKey) {
+            break;
+        }
+        if (r === mount || chain.includes(r)) {
+            continue;
+        }
+        if (typeof r.path !== "string") {
+            return null;
+        }
+        if (guards === inherited) {
+            guards = [...inherited];
+        }
+        guards.push({ path: pathPrefix + r.path, use: r.use === true, method: r.method, all: r.all === true });
+    }
+    return guards;
+}
+
 /**
  * Notes which application is current before a mounted one is entered, so that exact one comes back
  * when it hands over.
@@ -1146,6 +1222,26 @@ module.exports = class Router extends EventEmitter {
      */
     _isApplication = false;
 
+    /**
+     * Whether anything served from here has been seen reading req.ip after the response, by which
+     * point µWS has freed the address. Set once, from Request#parsedIp, and read on every request
+     * after that. Here rather than on Application because a plain Router serves requests of its own
+     * through the node shim.
+     *
+     * @type {boolean}
+     */
+    needsIpAfterResponse = false;
+
+    /**
+     * How many requests still read the peer address up front whether or not anyone asks, so that
+     * one of them can be the one that finds out. Counts to a hundred and stops: it used to be read
+     * off a module-wide counter that wrapped at 100000, so the window reopened every time it did
+     * and a hundred requests paid again for a discovery made long before.
+     *
+     * @type {number}
+     */
+    _ipProbes = 0;
+
     /**
      * The two routing flags once read, undefined until then. Express passes caseSensitive and
      * strict in when it builds a router and never looks at them again, so they are frozen here at
@@ -1701,8 +1797,9 @@ module.exports = class Router extends EventEmitter {
             return;
         }
 
-        // pathPrefix/chainPrefix accumulate across nested sole-callback mounts
-        const walk = (router, pathPrefix, chainPrefix) => {
+        // pathPrefix/chainPrefix accumulate across nested sole-callback mounts, and outerGuards
+        // carries what was written before them and answers only part of what is under them
+        const walk = (router, pathPrefix, chainPrefix, outerGuards) => {
             for (const route of router._routes) {
                 if (route.use) {
                     // only sole-callback mounts. Case rules do not gate the walk: each level's
@@ -1721,23 +1818,33 @@ module.exports = class Router extends EventEmitter {
                             route._whyGeneric = "something before it in the same router overlaps its paths";
                             continue;
                         }
-                        route._walkedInto = true;
                         pathToMount = pathToMount.slice(0, -1);
-                        walk(route.callbacks[0], pathPrefix + route.path, [
-                            ...chainPrefix,
-                            ...pathToMount,
-                            {
-                                ...route,
-                                callbacks: [],
-                                callbackKinds: [],
-                                keepMount: true,
-                                // mounted sub-apps become req.app during their dispatch, like express
-                                mountApp:
-                                    route.callbacks[0].constructor.name === "Application"
-                                        ? route.callbacks[0]
-                                        : undefined
-                            }
-                        ]);
+                        const guards = guardsInside(router, route, pathPrefix, pathToMount, outerGuards);
+                        if (guards === null) {
+                            route._whyGeneric = "a path written before it cannot be read segment by segment";
+                            continue;
+                        }
+                        route._walkedInto = true;
+                        walk(
+                            route.callbacks[0],
+                            pathPrefix + route.path,
+                            [
+                                ...chainPrefix,
+                                ...pathToMount,
+                                {
+                                    ...route,
+                                    callbacks: [],
+                                    callbackKinds: [],
+                                    keepMount: true,
+                                    // mounted sub-apps become req.app during their dispatch, like express
+                                    mountApp:
+                                        route.callbacks[0].constructor.name === "Application"
+                                            ? route.callbacks[0]
+                                            : undefined
+                                }
+                            ],
+                            guards
+                        );
                     } else {
                         // said once here rather than at each condition above: a mount is walked into
                         // only when µWS can match its path on its own and it carries exactly one
@@ -1757,6 +1864,16 @@ module.exports = class Router extends EventEmitter {
                             (!pathPrefix || !router._isFollowedByAnOverlap(route, router._routes)))) &&
                     supportedUwsMethods.has(route.method)
                 ) {
+                    // something outside this router, written before the mount it is in, that could
+                    // answer this exact path. µWS would jump here and never give it its turn
+                    if (outerGuards.length > 0 && typeof route.path === "string") {
+                        const absolute = pathPrefix + route.path;
+                        const guard = outerGuards.find((g) => shadowsLeaf(g, absolute, route));
+                        if (guard) {
+                            route._whyGeneric = `${guard.path} is written before the mount it is in and answers the same paths`;
+                            continue;
+                        }
+                    }
                     const leafPath = router._optimizeRoute(route, router._routes);
                     if (!leafPath) {
                         route._whyGeneric = "something before it in the same router overlaps its paths";
@@ -1816,7 +1933,7 @@ module.exports = class Router extends EventEmitter {
             }
         };
 
-        walk(this, "", []);
+        walk(this, "", [], []);
     }
 
     /**

package/src/utils.js

@@ -809,7 +809,11 @@ const defaultSettings = {
     // "case sensitive routing" is deliberately absent: unset means insensitive, as in Express 5.
     // The native µWS router matches bytes, so the compiler in _compileOptimizedRoutes only hands
     // it routes whose earlier siblings it can prove agree under either case rule.
-    "declarative responses": true
+    "declarative responses": true,
+    // off, and it is a security setting rather than a compatibility one: with it on, req.ip is the
+    // address a PROXY protocol preamble declared. µWS reads that preamble from any client, so this
+    // belongs only to a server nothing can reach except the proxy in front of it. See Request#_readRawIp
+    "trust proxy protocol": false
 };
 
 /**