@jsenv/server 17.6.7 → 17.6.8

package/docs/server_sent_events.md

@@ -0,0 +1,64 @@
+# Server-Sent Events
+
+`ServerEvents` keeps track of the connected clients and broadcasts events to them. A client connects with an `EventSource` (`accept: text/event-stream`) or with a `WebSocket`: `serverEvents.fetch` handles both.
+
+_server.js_
+
+```js
+import { ServerEvents, startServer } from "@jsenv/server";
+
+const serverEvents = new ServerEvents();
+
+setInterval(() => {
+  serverEvents.sendEventToAllClients({
+    type: "ping",
+    data: JSON.stringify({ ts: Date.now() }),
+  });
+}, 1000);
+
+await startServer({
+  port: 3456,
+  routes: [
+    {
+      endpoint: "GET /events",
+      fetch: serverEvents.fetch,
+    },
+  ],
+});
+```
+
+_client.js_
+
+```js
+const eventSource = new EventSource("http://localhost:3456/events");
+eventSource.addEventListener("ping", (event) => {
+  console.log("ping from server", event.lastEventId, JSON.parse(event.data));
+});
+```
+
+![Screencast of server sent events execution in a terminal](./screenshots/sse-screencast.gif)
+
+An event is `{ type, data, id, retry }`; `data` is sent as is (stringify objects yourself) and must hold no line break: the event stream is line based, a line break ends the event. `JSON.stringify` without indentation is safe. Every event gets an incrementing `id` and is kept (`historyLength`, 1000 by default): a client reconnecting with a `last-event-id` header, or a `?last-event-id=` search param, receives what it missed. A comment is sent every `keepaliveDuration` (30s) so that proxies keep the connection open.
+
+A websocket client receives each event as one message, the whole event as json: `JSON.parse(message.data)` gives `{ type, data, id }`.
+
+Past `maxClientAllowed` (100) a new client is refused with 503, or the oldest one is disconnected with `actionOnClientLimitReached: "kick-oldest"`. `close()` disconnects everyone and answers 204 until `open()`. `getClientCount()` and `getAllEventSince(id)` tell where things stand.
+
+## Producing events only when someone listens
+
+`LazyServerEvents` runs a producer when the first client connects and its cleanup when the last one leaves — a file watcher, a database subscription, a timer. Only `fetch` is exposed: events can only come from the producer.
+
+```js
+import { LazyServerEvents } from "@jsenv/server";
+
+const ticks = new LazyServerEvents(({ sendEvent }) => {
+  const interval = setInterval(() => {
+    sendEvent({ type: "tick", data: new Date().toISOString() });
+  }, 1000);
+  return () => {
+    clearInterval(interval);
+  };
+});
+
+// route: { endpoint: "GET /ticks", fetch: ticks.fetch }
+```

package/docs/server_timing.md

@@ -0,0 +1,70 @@
+# Server timing
+
+`serverTiming: true` sends a `server-timing` response header telling what the server spent its time on. Devtools show it next to the TTFB of each request.
+
+_Chrome devtools, TTFB:_
+
+![screenshot of chrome devtools TTFB](./screenshots/devtools_TTFB.png)
+
+_Chrome devtools, server timing:_
+
+![screenshot of chrome devtools server timing](./screenshots/devtools_server_timing.png)
+
+```js
+import { startServer } from "@jsenv/server";
+
+await startServer({
+  serverTiming: true,
+  routes: [
+    {
+      endpoint: "GET *",
+      fetch: async (request, { timing }) => {
+        const waitTiming = timing("waiting 50ms");
+        await new Promise((resolve) => {
+          setTimeout(resolve, 50);
+        });
+        waitTiming.end();
+        return new Response("Hello");
+      },
+    },
+  ],
+});
+```
+
+```console
+server-timing: a;desc="time to start responding";dur=51.2, b;desc="routing";dur=0.1, c;desc="waiting 50ms";dur=50.7
+```
+
+The header holds:
+
+- `time to start responding`, from the request to the response headers;
+- the routing of each group of routes (`routing`, or `<plugin>.routing` for the routes a plugin brought);
+- what the route measured with `helpers.timing(name)`; `createFileSystemFetch` measures its stat and etag;
+- the `timing` property a plain response object hands back: `{ "db query": 12.3, "served from cache": null }` — a `null` duration is a marker, something to say with nothing to measure.
+
+Entries are named `a`, `b`, `c`… because devtools sort them alphabetically; the `desc` carries the name.
+
+`serverTiming: { minDuration: 0.5 }` drops the entries that took less than that many milliseconds: a request crosses every route, and a wall of sub-millisecond `routing` entries buries the measures worth reading. `time to start responding` and the markers always stay.
+
+## Measuring without telling everyone
+
+The header describes what the server does internally; an api usually wants it for itself, a bench or a probe, and for nobody else. Give a function instead of `true` and it is asked once per request, on what the client sent:
+
+```js
+const PERF_TIMING_TOKEN = process.env.PERF_TIMING_TOKEN;
+const isPerfTimingClient = (request) =>
+  request.headers["x-perf-timing"] === PERF_TIMING_TOKEN;
+
+await startServer({
+  serverTiming: isPerfTimingClient,
+  plugins: [
+    serverPluginCORS({
+      accessControlAllowedOrigins: ["https://app.example.com"],
+      // without it a page cannot read the timings of a cross-origin response
+      timingAllowOrigin: isPerfTimingClient,
+    }),
+  ],
+});
+```
+
+`{ enabled: isPerfTimingClient, minDuration: 1 }` says the same thing with a `minDuration`. When the predicate says no nothing is measured for that request and no header is composed; routes still call `helpers.timing(name)` unconditionally, it costs them nothing.

package/docs/serving_files.md

@@ -0,0 +1,64 @@
+# Serving files
+
+```js
+import { createFileSystemFetch, startServer } from "@jsenv/server";
+
+await startServer({
+  routes: [
+    {
+      endpoint: "GET /assets/*",
+      fetch: createFileSystemFetch(import.meta.resolve("./assets/")),
+    },
+    {
+      endpoint: "GET *",
+      fetch: createFileSystemFetch(import.meta.resolve("./public/"), {
+        mainFileRelativeUrl: "./index.html",
+      }),
+    },
+  ],
+});
+```
+
+The part of the url captured by `*` (the whole resource when the endpoint has none) is resolved inside the directory. A url leaving the directory (`..`) is answered 403. Any method other than GET and HEAD is declined: the next route is tried. The `content-type` comes from the file extension.
+
+The options are in the JSDoc of `createFileSystemFetch`; what follows is how they combine.
+
+## Client cache
+
+By default every request gets 200 with the content. Two ways to unlock 304:
+
+- `etagEnabled` — an `etag` computed from the content, remembered per file until its stats change (`etagMemory`, `etagMemoryMaxSize`); a matching `if-none-match` gets 304. Robust.
+- `mtimeEnabled` — a `last-modified` header from the filesystem date, second precision; a recent enough `if-modified-since` gets 304. Relies on filesystem dates being meaningful.
+
+When both are set, etag wins.
+
+`cacheControl` becomes the `cache-control` header. By default a versioned url (`isVersioned`: the url has a `v` search param) is `private,max-age=2592000,immutable` and anything else `private,max-age=0,must-revalidate`. It can be a function of the request:
+
+```js
+createFileSystemFetch(import.meta.resolve("./"), {
+  etagEnabled: true,
+  cacheControl: (request) =>
+    request.resource === "/"
+      ? `private,max-age=0,must-revalidate`
+      : `private,max-age=3600,immutable`,
+});
+```
+
+`"no-store"` disables etag and mtime, they would be pointless.
+
+## Compression
+
+`compressionEnabled` compresses textual files bigger than `compressionSizeThreshold` (1024 bytes) with brotli, gzip or deflate, whichever `accept-encoding` prefers. The compression happens for every request (brotli at quality 4 to keep it cheap) and the response has no `content-length`: for production, compress the files once at build time instead.
+
+## Directories
+
+A url pointing to a directory gets 403, unless:
+
+- `canReadDirectory: true` — the listing is sent, as json or as an html page of links (see `fetchDirectory`),
+- `mainFileRelativeUrl` — that file is sent for the root of the served directory, and for any extension-less url that does not exist (client side routing needs the same page for `/users/42`). A subdirectory still gets 403.
+
+`ENOENTFallback: () => fileUrl` serves another file when the requested one does not exist.
+
+## What a 404 says
+
+The status text is `ENOENT: File not found`; the file path is added only when the server runs with `canExposeSensitiveData` (see [security](./security.md)). Same for the other filesystem errors (EACCES → 403, EBUSY → 503 with `retry-after`, …).

package/docs/websocket.md

@@ -0,0 +1,64 @@
+# WebSocket
+
+A route accepts a websocket by returning a `WebSocketResponse`:
+
+_server.js_
+
+```js
+import {
+  createFileSystemFetch,
+  startServer,
+  WebSocketResponse,
+} from "@jsenv/server";
+
+await startServer({
+  port: 3000,
+  routes: [
+    {
+      endpoint: "GET /chat.websocket",
+      fetch: () => {
+        return new WebSocketResponse((websocket) => {
+          websocket.send("Hello world");
+          websocket.on("message", (data) => {
+            websocket.send(data);
+          });
+          return () => {
+            // the client is gone
+          };
+        });
+      },
+    },
+    {
+      endpoint: "GET *",
+      fetch: createFileSystemFetch(import.meta.resolve("./")),
+    },
+  ],
+});
+```
+
+_client.html_
+
+```html
+<!doctype html>
+<html lang="en">
+  <head>
+    <title>Title</title>
+    <link rel="icon" href="data:," />
+  </head>
+
+  <body>
+    <script>
+      const websocket = new WebSocket("ws://localhost:3000/chat.websocket");
+      websocket.onmessage = (message) => {
+        document.body.appendChild(document.createTextNode(message.data));
+      };
+    </script>
+  </body>
+</html>
+```
+
+A route is a websocket route when its endpoint ends with `.websocket` or its `headers` pattern has `upgrade: "websocket"`; a plain request to it is answered 426. The handler receives the [ws](https://github.com/websockets/ws/blob/master/doc/ws.md) socket once the upgrade is done; if it returns a function, that function runs when the socket closes.
+
+A websocket route can still refuse the upgrade by returning a regular response (a 401 for instance). A 101 response that is not a `WebSocketResponse` is an error, and so is a `WebSocketResponse` returned to a request that did not ask for an upgrade.
+
+`server.webSocketOrigin` is the `ws://` (or `wss://`) origin of the server. To broadcast to many clients, see [server sent events](./server_sent_events.md): `ServerEvents` accepts websocket clients too.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jsenv/server",
-  "version": "17.6.7",
+  "version": "17.6.8",
   "type": "module",
   "description": "A modern Node.js HTTP server with declarative routing, content negotiation, and WebSocket support",
   "repository": {
@@ -28,6 +28,7 @@
   },
   "files": [
     "/dist/",
+    "/docs/",
     "/index.js",
     "/src/"
   ],
@@ -54,10 +55,10 @@
     "@jsenv/url-pattern": "../../tooling/url-pattern",
     "@jsenv/urls": "../../tooling/urls",
     "@jsenv/utils": "../../tooling/utils",
-    "eventsource": "5.1.1",
+    "eventsource": "5.1.2",
     "formidable": "3.5.4",
     "playwright": "1.63.0",
-    "undici": "8.10.2",
+    "undici": "8.11.2",
     "ws": "8.21.3"
   },
   "publishConfig": {

package/src/internal/track_server_pending_requests.js

@@ -21,14 +21,6 @@ export const trackServerPendingRequests = (nodeServer) => {
     pendingClients.clear();
     await Promise.all(
       pendingClientsArray.map(({ nodeRequest, nodeResponse }) => {
-        if (nodeResponse.headersSent === false) {
-          if (nodeRequest.stream) {
-            // http2 has no reason phrase
-            nodeResponse.writeHead(status);
-          } else {
-            nodeResponse.writeHead(status, asReasonPhrase(reason));
-          }
-        }
         return new Promise((resolve) => {
           if (nodeResponse.destroyed) {
             resolve();
@@ -37,7 +29,20 @@ export const trackServerPendingRequests = (nodeServer) => {
           nodeResponse.once("close", () => {
             resolve();
           });
-          nodeResponse.destroy();
+          if (nodeResponse.headersSent) {
+            // a status already sent cannot be replaced: the response is cut
+            nodeResponse.destroy();
+            return;
+          }
+          if (nodeRequest.stream) {
+            // http2 has no reason phrase, nor a connection header
+            nodeResponse.writeHead(status);
+          } else {
+            nodeResponse.writeHead(status, asReasonPhrase(reason), {
+              connection: "close",
+            });
+          }
+          nodeResponse.end();
         });
       }),
     );

package/src/plugins/cors/server_plugin_cors.js

@@ -115,9 +115,10 @@ const createAllowedOriginChecker = (allowedOrigins) => {
   }
 
   return {
-    // when the request origin cannot be reflected back we must still send a
-    // single valid origin, never a pattern
-    defaultOrigin: literalOrigins[0] ?? "*",
+    // sent when the request origin cannot be reflected back: a single valid
+    // origin, never a pattern, and never "*" which would allow every origin.
+    // With only patterns there is nothing to send.
+    defaultOrigin: literalOrigins[0] ?? null,
     isAllowed: (origin) => {
       if (literalOrigins.includes(origin)) {
         return true;
@@ -161,7 +162,6 @@ const generateAccessControlHeaders = ({
 
   // Access-Control-Allow-Origin must be a single value (not a list).
   // We reflect back the request's origin if it is in the allowed list.
-  // If no origin matches we fall back to "*" (only when not using credentials).
   let allowOrigin = null;
 
   const requestOrigin = readRequestOrigin(headers);
@@ -213,14 +213,16 @@ const generateAccessControlHeaders = ({
   }
 
   return {
-    "access-control-allow-origin": allowOrigin,
+    ...(allowOrigin === null
+      ? {}
+      : { "access-control-allow-origin": allowOrigin }),
     "access-control-allow-methods": allowedMethodArray.join(", "),
     "access-control-allow-headers": allowedHeaderArray.join(", "),
     ...(accessControlAllowCredentials
       ? { "access-control-allow-credentials": true }
       : {}),
     "access-control-max-age": accessControlMaxAge,
-    ...(timingAllowOriginEnabled(request)
+    ...(allowOrigin !== null && timingAllowOriginEnabled(request)
       ? { "timing-allow-origin": allowOrigin }
       : {}),
     ...(vary.length ? { vary: vary.join(", ") } : {}),

package/src/plugins/filesystem/fetch_file.js

@@ -50,8 +50,9 @@ export const createFileSystemFetch = (directoryUrl, options) => {
  * @param {Object} [helpers] - The helpers given to a route `fetch` (used for `timing` and `canExposeSensitiveData`).
  * @param {string|URL} directoryUrl - `file://` url (or filesystem path) of the directory to serve.
  * @param {Object} [options]
- * @param {string} [options.mainFileRelativeUrl] - File served for the directory itself and,
- *   as a fallback, for extension-less urls that do not exist (client side routing).
+ * @param {string} [options.mainFileRelativeUrl] - File served for the root of `directoryUrl`
+ *   and, as a fallback, for extension-less urls that do not exist (client side routing).
+ *   A subdirectory still gets 403 unless `canReadDirectory`.
  * @param {boolean} [options.etagEnabled=false] - Send an `etag` (hash of the content) and
  *   answer 304 to a matching `if-none-match`.
  * @param {boolean} [options.etagMemory=true] - Remember etags per file (invalidated when the

package/src/plugins/server_plugins_controller.js

@@ -14,10 +14,10 @@ const SERVER_PLUGIN_PROPERTIES = {
   grantPermissions: { type: "hook" },
   // async (error, { request }) => response | null
   handleError: { type: "hook" },
-  // (request, { response, warn }) to look at the response before it is sent
-  inspectResponse: { type: "hook" },
   // (request, response) => responseToCompose | null
   injectResponseProperties: { type: "hook" },
+  // (request, { response, warn }) to look at the final response before it is sent
+  inspectResponse: { type: "hook" },
   // ({ reason }) once the server is stopped
   serverStopped: { type: "hook" },
   // route descriptors, appended after the routes given to startServer

package/src/progressive_response.js

@@ -8,7 +8,7 @@ import { createObservable } from "./interfacing_with_node/observable.js";
  *
  * @param {(body: { write: (chunk: string|Uint8Array) => void, end: () => void }) => void | (() => void)} responseBodyHandler
  *   Receives `write` and `end`. If it returns a function, that function runs
- *   when the client disconnects before `end` was called (cleanup).
+ *   once the response is over: after `end()`, or when the client disconnects.
  * @param {Object} [init]
  * @param {number} [init.status=200]
  * @param {string} [init.statusText]

package/src/router/router.js

@@ -659,7 +659,8 @@ const createRoute = ({
   if (!endpoint || typeof endpoint !== "string") {
     throw new TypeError(`endpoint must be a string, received ${endpoint}`);
   }
-  const [method, resource] = endpoint === "*" ? ["* *"] : endpoint.split(" ");
+  const [method, resource] =
+    endpoint === "*" ? ["*", "*"] : endpoint.split(" ");
   if (method !== "*" && !HTTP_METHODS.includes(method)) {
     throw new TypeError(`"${method}" is not an HTTP method`);
   }
@@ -878,8 +879,9 @@ const createResourceOptionsResponse = (request, resourceOptions) => {
  * @param {Object} params - Content negotiation parameters
  * @param {Array<string>} params.availableMediaTypes - Content types the server can produce
  * @param {Array<string>} params.availableLanguages - Languages the server can respond with
+ * @param {Array<string|number|Function>} params.availableVersions - Versions the server can respond with
  * @param {Array<string>} params.availableEncodings - Encodings the server supports
- * @returns {Response} A 406 Not Acceptable response
+ * @returns {Object} 406 response properties
  */
 const createNotAcceptableResponse = (
   request,

package/src/server_events.js

@@ -14,7 +14,7 @@ import { WebSocketResponse } from "./web_socket_response.js";
  * @class
  * @param {Object} options - Configuration options for the SSE controller
  * @param {String} [options.logLevel] - Controls logging verbosity ('debug', 'info', 'warn', 'error', etc.)
- * @param {Boolean} [options.keepProcessAlive=false] - If true, prevents Node.js from exiting while SSE connections are active
+ * @param {Boolean} [options.keepProcessAlive=false] - If true, the keepalive timer keeps the process alive until `close()`
  * @param {Number} [options.keepaliveDuration=30000] - Milliseconds between keepalive messages to prevent connection timeout
  * @param {Number} [options.retryDuration=1000] - Suggested client reconnection delay in milliseconds
  * @param {Number} [options.historyLength=1000] - Maximum number of events to keep in history for reconnecting clients
@@ -70,7 +70,7 @@ export class ServerEvents {
  * @param {Function} producer - Function called when first client connects
  * @param {Object} [options] - Configuration options for the SSE controller
  * @param {String} [options.logLevel] - Controls logging verbosity ('debug', 'info', 'warn', 'error', etc.)
- * @param {Boolean} [options.keepProcessAlive=false] - If true, prevents Node.js from exiting while SSE connections are active
+ * @param {Boolean} [options.keepProcessAlive=false] - If true, the keepalive timer keeps the process alive until `close()`
  * @param {Number} [options.keepaliveDuration=30000] - Milliseconds between keepalive messages to prevent connection timeout
  * @param {Number} [options.retryDuration=1000] - Suggested client reconnection delay in milliseconds
  * @param {Number} [options.historyLength=1000] - Maximum number of events to keep in history for reconnecting clients
@@ -139,8 +139,9 @@ const createServerEvents = ({
 } = {}) => {
   const logger = createLogger({ logLevel });
 
+  // starts closed: open() below installs the keepalive interval
   const serverEventSource = {
-    closed: false,
+    closed: true,
   };
   const clientArray = [];
   const eventHistory = createEventHistory(historyLength);
@@ -382,6 +383,7 @@ const createServerEvents = ({
     close,
     open,
   });
+  open();
   return serverEventSource;
 };
 

package/src/start_server.js

@@ -117,8 +117,9 @@ const TIMING_NOOP = () => {
  * @param {boolean} [params.stopOnSIGINT] - Stop on SIGINT (ctrl+c). Defaults to true, except
  *   inside a cluster worker where the primary process is in charge.
  * @param {boolean} [params.stopOnExit=true] - Stop on SIGHUP, SIGTERM, beforeExit and exit.
- * @param {boolean} [params.stopOnInternalError=false] - Stop when a route throws (after the
- *   "handleError" plugins answered).
+ * @param {boolean} [params.stopOnInternalError=false] - Stop as soon as a route throws. The
+ *   request that threw and every other pending request are answered 500 with no body, the
+ *   "handleError" plugins' response is not sent.
  * @param {boolean} [params.keepProcessAlive=true] - When false the server alone does not keep
  *   the process alive.
  * @param {boolean} [params.canExposeSensitiveData=false] - Lets the server hand out what
@@ -150,8 +151,9 @@ const TIMING_NOOP = () => {
  * @returns {Promise<Object>} The server: `{ origin, origins, port, hostname, nodeServer,
  *   webSocketOrigin, stop, stoppedPromise, getStatus, addEffect }`.
  *   - `origins`: `{ local, localip, externalip }`, `origin` being `origins.local`.
- *   - `stop(reason)`: resolves once every connection is closed; `reason` can be anything
- *     and defaults to `STOP_REASON_NOT_SPECIFIED`.
+ *   - `stop(reason)`: answers the pending requests with 503, closes every connection and
+ *     resolves once done; `reason` can be anything, becomes the 503 status text and
+ *     defaults to `STOP_REASON_NOT_SPECIFIED`.
  *   - `stoppedPromise`: resolves with the reason the server stopped for (one of the
  *     `STOP_REASON_*` exports or what was given to `stop`).
  *   - `getStatus()`: `"starting"`, `"opened"`, `"stopping"` or `"stopped"`.
@@ -442,16 +444,16 @@ export const startServer = async ({
   stopCallbackSet.add(removeConnectionErrorListener);
 
   const connectionsTracker = trackServerPendingConnections(nodeServer);
-  // opened connection must be shutdown before the close event is emitted
-  stopCallbackSet.add(connectionsTracker.stop);
-
   const pendingRequestsTracker = trackServerPendingRequests(nodeServer);
-  // ensure pending requests got a response from the server
-  stopCallbackSet.add((reason) => {
-    pendingRequestsTracker.stop({
+  // The pending requests are answered before any connection is destroyed:
+  // destroyed first, their clients would only see a socket error. The
+  // connections must still be shut down for the close event to be emitted.
+  stopCallbackSet.add(async ({ reason }) => {
+    await pendingRequestsTracker.stop({
       status: reason === STOP_REASON_INTERNAL_ERROR ? 500 : 503,
       reason,
     });
+    await connectionsTracker.stop(reason);
   });
 
   const applyRequestInternalRedirection = (request) => {