@amritk/nish-x86_64-linux 0.29.0 → 0.30.0

package/INSTALL.md

@@ -256,7 +256,7 @@ routes are
     runtime/runtime-cross.c runtime/runtime-host.c runtime/runtime-net.c \
     runtime/runtime-simd.c runtime/runtime-aes.c runtime/runtime-sha256.c runtime/runtime-term.c \
     runtime/runtime-bytes.c runtime/runtime-checksum.c runtime/runtime-sandbox.c \
-    runtime/runtime-shm.c -lm -o app
+    runtime/runtime-shm.c runtime/runtime-region.c -lm -o app
   ```
 
   `--link`ing inside the container is the mistake to avoid: it shells out to the
@@ -392,7 +392,7 @@ it into `build/seed/`; `NISH_BOOTSTRAP=<path>` names one you already have and
 skips the download. §2a has what the build does with it.
 
 The published package ships `bin/` (the `nish` command, which is a launcher),
-`runtime/` (the C runtime — fourteen translation units, the wasm subset and their
+`runtime/` (the C runtime — fifteen translation units, the wasm subset and their
 header — and `runtime/sqlite/`, SQLite's vendored source),
 `scripts/build.sh` (the link pipeline), `std/` (the standard library) and
 `docs/AI.md` (the rules card). The
@@ -405,7 +405,7 @@ working directory — and the binary runs from inside its own platform package
 for exactly that reason.
 
 Building the IR yourself rather than through `--link` means naming the runtime
-on the `clang` line, and it is fourteen files:
+on the `clang` line, and it is fifteen files:
 
 ```bash
 clang app.ll runtime/runtime.c runtime/runtime-os.c runtime/runtime-parallel.c \

package/docs/AI.md

@@ -2743,6 +2743,21 @@ request. It reads no clock: when to take is the loop's choice, or a `PgBatchCloc
 guide's [PostgreSQL page](guide/using-nish/postgres.md#batching-keyed-reads)
 its walk-through.)
 
+**Call another service from a route handler** — there is no `await fetch()`
+in a handler and no event-loop callback: `nish/net/upstream`'s `Upstream(h,
+origin, limits)` is a pool of keep-alive connections to one `http:` origin on
+the `HttpHandler`'s own loop. `users.get(req, path)` returns at once — test it
+for `UPSTREAM_E_FULL`, which holds nothing — and keeps `req` held; after each
+`h.wait(ms)`, `while (users.ready())` `take()`s every finished call, answers
+`r.req` (`r.ok`, `r.status` and `r.body[0 .. r.length)`, or `r.error`: the
+codec's `HTTP1_E_` refusal or an `UPSTREAM_E_` code such as `TIMEOUT`) and
+`release`s it once. The origin is looked up once at start-up; set
+`UpstreamLimits.outbound` when it comes from input. Each connection takes a
+handler watch id from `firstId` (8) up, so keep your own `h.watch` ids below
+it. (No block, as for `nish/text`; `tests/link/net_upstream` is its program,
+and the user guide's [calling-services page](guide/using-nish/calling-services.md)
+its walk-through.)
+
 **Draw to the whole terminal** (a game, a dashboard) — there is no curses and
 no `process.stdout.on("resize")`: `runTerminal(options, state, update, render)`
 from `nish/term-io` opens the terminal, probes what it can draw, steps

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@amritk/nish-x86_64-linux",
-  "version": "0.29.0",
+  "version": "0.30.0",
   "description": "The nish native compiler for Linux on x86_64",
   "license": "MIT",
   "os": [

package/runtime/nish.h

@@ -104,8 +104,9 @@ extern NISH_TLS struct nish_arena nish_arena;
 
 /* Bump allocation: 8-byte rounded and aligned, uninitialised. */
 void *nish_alloc_struct(uint64_t size);
-/* Slow path: push a new chunk (at least 64 KB) and bump from it. Called by
- * the inlined fast path when the current chunk is full; rarely by hosts. */
+/* Slow path: push a new chunk (at least 64 KB, or a region's next size) and
+ * bump from it. Called by the inlined fast path when the current chunk is
+ * full; rarely by hosts. */
 void *nish_arena_grow(uint64_t size);
 /* Recycle everything in O(1): keeps the newest chunk, frees the rest. Every
  * arena string and object becomes invalid. Call it between batches. */
@@ -132,6 +133,25 @@ void *nish_arena_keep(uint64_t mark, void *p);
 /* Bytes bumped in the current chunk (`Arena.used()`); a steady-state loop keeps it flat. */
 uint64_t nish_arena_used(void);
 
+/* ---- Regions (runtime-region.c) -----------------------------------------
+ * Arenas made, entered, left and freed one at a time
+ * (docs/wp46-long-running-memory.md). A handle is a nonzero `uint64_t`: a
+ * 24-bit slot and its 40-bit generation. Every call below panics on a zero
+ * handle, a freed (stale) one and one made on another thread.
+ * `nish_region_new` makes an empty region, owned by the calling thread.
+ * `nish_region_enter` makes the region the thread's arena until the matching
+ * `nish_region_leave`, so every allocation in between lands in it, and
+ * answers the root `leave` last recorded (null at first); entering a region
+ * already entered panics. `nish_region_leave` must name the region entered
+ * last on this thread, puts back the arena its enter displaced and records
+ * `root`. `nish_region_free` frees every chunk of a region that is not
+ * entered and makes its handle stale. The wasm profile has none of these
+ * (scripts/build.sh refuses IR that names them); wasi has. */
+uint64_t nish_region_new(void);
+void *nish_region_enter(uint64_t r);
+void nish_region_leave(uint64_t r, void *root);
+void nish_region_free(uint64_t r);
+
 /* ---- String operations ------------------------------------------------- */
 /* Copy `len` bytes into a new arena string (`bytes` need not be terminated). */
 nish_str *nish_str_new(const char *bytes, uint64_t len);

package/runtime/runtime-region.c

@@ -0,0 +1,231 @@
+/* Nish runtime, the region half: arenas a program makes, enters, leaves and
+ * frees one at a time, so a long-running program can give back the memory of
+ * one session, one cache entry or one connection without a garbage collector
+ * (docs/wp46-long-running-memory.md, section 2.1, "Runtime and ABI").
+ *
+ * A translation unit of its own, paired with runtime.c in scripts/build.sh's
+ * `halves` and src/run-cache.ts's `runtimeUnits()` like the others, so it has
+ * its own measured `.text*` ceiling in tests/runtime-budgets.js and a program
+ * that never makes a region links none of it. The wasm profile has no malloc
+ * to give a region its chunks, so `scripts/build.sh --profile wasm` refuses IR
+ * that names `@nish_region_`; the wasi profile links this file.
+ *
+ * **A handle is a number.** `nish_region_new` answers an `i64`: the slot's
+ * index in the low 24 bits and the slot's generation in the 40 above them.
+ * Generations start at 1, so 0 -- what a fresh `Region<T>[]` holds -- is never
+ * a handle. `nish_region_free` moves the slot's generation on, so a copy of a
+ * freed handle is stale; a slot whose generation would pass 40 bits is
+ * retired rather than reused, so no handle ever names a slot twice. Every
+ * call that takes a handle checks it first and panics on a zero handle, a
+ * stale one, and one made on another thread. The 24-bit index caps a process
+ * at 16,777,216 live regions; `nish_region_new` past that panics.
+ *
+ * **The table** is one per process: a directory of pages, each page 1,024
+ * slots allocated (zeroed) the first time a slot in it is handed out, so a
+ * program that makes ten regions holds one page. Taking a slot and giving one
+ * back take the table's lock, once per `new` and once per `free`; `enter` and
+ * `leave` take none, because a slot is only ever entered by the thread that
+ * made it and the check reads the two fields another thread's `free` could
+ * write (generation and thread) with atomic loads. Only a threads build
+ * (`-DNISH_THREADS`) has a second thread to lock against, so only it pays for
+ * the lock.
+ *
+ * **Entering swaps arenas.** A slot holds a whole `struct nish_arena`.
+ * `nish_region_enter` swaps it with the thread's `nish_arena`, so every
+ * allocation the program makes until `nish_region_leave` -- the inlined bump
+ * as well as the runtime's own -- lands in the region; `leave` swaps them
+ * back. While a region is entered its slot holds the arena it displaced, so
+ * entering a second region inside the first nests: each leave restores what
+ * its enter took. A slot that is entered cannot be entered again (it would
+ * swap its own arena with itself) or freed, and `leave` must name the region
+ * entered last on this thread.
+ *
+ * **Chunk size.** A thread's own arena grows by 64 KiB chunks, which is right
+ * for one arena a thread and wrong for a hundred thousand of them. A region's
+ * first chunk is the size of its first allocation, and each later chunk
+ * doubles, or takes the allocation's own size when that is more. The size of
+ * the next chunk is runtime.c's thread-local `nish_region_chunk`, which
+ * `nish_arena_grow` reads on its slow path and which `enter` and `leave` swap
+ * with the slot's as they swap the arena; 0 there means the thread's own
+ * arena and the 64 KiB floor. A region starts it at 8, the smallest block.
+ *
+ * **The root.** `Region<T>` is only the number, so the slot keeps the root
+ * object `region`'s function answered: `nish_region_leave` records it and
+ * `nish_region_enter` answers it.
+ *
+ * Freeing a region frees its chunks through runtime.c's `nish_free_arena`,
+ * with the region's arena swapped in for the call, so an `-DNISH_ASAN` build
+ * forgets the freed blocks as it does for the thread's arena. */
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "nish.h"
+
+#define NISH_COLD __attribute__((noreturn, cold, noinline))
+
+/* runtime.c's: the size of the next chunk `nish_arena_grow` makes for the
+   arena in `nish_arena`, or 0 for a thread's own, which takes 64 KiB. */
+extern NISH_TLS uint64_t nish_region_chunk;
+
+#define NISH_REGION_SLOT_BITS 24
+#define NISH_REGION_SLOTS ((uint64_t)1 << NISH_REGION_SLOT_BITS)
+#define NISH_REGION_GEN_END ((uint64_t)1 << 40)
+#define NISH_REGION_PAGE 1024
+
+typedef struct nish_region_slot {
+  struct nish_arena arena; /* the region's arena, or, while entered, the one it displaced */
+  uint64_t gen;            /* the live handle's generation; NISH_REGION_GEN_END once retired */
+  uint64_t thread;         /* the thread that made it, 0 while the slot is free */
+  uint64_t chunk;          /* the region's next chunk size, or, while entered, the displaced one */
+  uint64_t outer;          /* while entered: the handle entered before it; while free: the next free slot + 1 */
+  void *root;              /* what `leave` recorded, answered by `enter` */
+  uint64_t entered;
+} nish_region_slot;
+
+static nish_region_slot *nish_region_pages[NISH_REGION_SLOTS / NISH_REGION_PAGE];
+static uint64_t nish_region_used; /* slots ever handed out: the next never-used index */
+static uint64_t nish_region_free_list; /* a free slot's index + 1, or 0 */
+/* The handle entered last on this thread, or 0. */
+static NISH_TLS uint64_t nish_region_current;
+
+#ifdef NISH_THREADS
+static int nish_region_lock_word;
+static uint64_t nish_region_threads;
+static NISH_TLS uint64_t nish_region_thread_id;
+static void nish_region_lock(void) {
+  while (__atomic_exchange_n(&nish_region_lock_word, 1, __ATOMIC_ACQUIRE))
+    while (__atomic_load_n(&nish_region_lock_word, __ATOMIC_RELAXED)) {}
+}
+static void nish_region_unlock(void) { __atomic_store_n(&nish_region_lock_word, 0, __ATOMIC_RELEASE); }
+/* A number no other thread of the process has: 64 bits, so it never wraps
+   back onto a live thread's. */
+static uint64_t nish_region_me(void) {
+  if (!nish_region_thread_id) nish_region_thread_id = __atomic_add_fetch(&nish_region_threads, 1, __ATOMIC_RELAXED);
+  return nish_region_thread_id;
+}
+#else
+static void nish_region_lock(void) {}
+static void nish_region_unlock(void) {}
+static uint64_t nish_region_me(void) { return 1; }
+#endif
+
+/* The panics, by number: one cold function and one table, rather than a
+   message's address loaded at each call site. Each ends in `nish_panic`, so
+   only the first panic of a process prints and atexit handlers run. */
+enum {
+  NISH_REGION_ZERO,
+  NISH_REGION_STALE,
+  NISH_REGION_THREAD,
+  NISH_REGION_ENTERED,
+  NISH_REGION_FREE_ENTERED,
+  NISH_REGION_LEAVE,
+  NISH_REGION_FULL,
+  NISH_REGION_OOM,
+};
+#define NISH_REGION_MSG(text) { sizeof text - 1, text }
+static const struct { uint64_t len; char data[72]; } nish_region_messages[] = {
+  NISH_REGION_MSG("nish: a zero region handle: no region was made for it"),
+  NISH_REGION_MSG("nish: a stale region handle: the region was freed"),
+  NISH_REGION_MSG("nish: a region handle made on another thread"),
+  NISH_REGION_MSG("nish: enter of a region that is already entered"),
+  NISH_REGION_MSG("nish: free of a region that is entered"),
+  NISH_REGION_MSG("nish: leave of a region that is not the one entered last"),
+  NISH_REGION_MSG("nish: more than 16777216 live regions"),
+  NISH_REGION_MSG("nish: out of memory"),
+};
+static NISH_COLD void nish_region_die(int why) {
+  nish_panic((const nish_str *)&nish_region_messages[why]);
+}
+
+/* The slot `r` names, or a panic: zero, stale, another thread's. */
+static nish_region_slot *nish_region_slot_of(uint64_t r) {
+  if (!r) nish_region_die(NISH_REGION_ZERO);
+  uint64_t index = r & (NISH_REGION_SLOTS - 1);
+  nish_region_slot *page = __atomic_load_n(&nish_region_pages[index / NISH_REGION_PAGE], __ATOMIC_ACQUIRE);
+  if (!page) nish_region_die(NISH_REGION_STALE);
+  nish_region_slot *s = &page[index % NISH_REGION_PAGE];
+  uint64_t thread = __atomic_load_n(&s->thread, __ATOMIC_RELAXED);
+  if (!thread || __atomic_load_n(&s->gen, __ATOMIC_RELAXED) != r >> NISH_REGION_SLOT_BITS)
+    nish_region_die(NISH_REGION_STALE);
+  if (thread != nish_region_me()) nish_region_die(NISH_REGION_THREAD);
+  return s;
+}
+
+/* Swap the slot's arena and chunk size with the thread's: `enter` and `leave`
+   both, and `free` around its `nish_free_arena`. */
+static void nish_region_swap(nish_region_slot *s) {
+  struct nish_arena a = nish_arena;
+  nish_arena = s->arena;
+  s->arena = a;
+  uint64_t c = nish_region_chunk;
+  nish_region_chunk = s->chunk;
+  s->chunk = c;
+}
+
+uint64_t nish_region_new(void) {
+  uint64_t me = nish_region_me();
+  nish_region_lock();
+  uint64_t index;
+  nish_region_slot *s;
+  if (nish_region_free_list) {
+    index = nish_region_free_list - 1;
+    s = &nish_region_pages[index / NISH_REGION_PAGE][index % NISH_REGION_PAGE];
+    nish_region_free_list = s->outer;
+  } else {
+    if (nish_region_used == NISH_REGION_SLOTS) nish_region_die(NISH_REGION_FULL);
+    index = nish_region_used++;
+    nish_region_slot **page = &nish_region_pages[index / NISH_REGION_PAGE];
+    if (!*page) {
+      nish_region_slot *p = calloc(NISH_REGION_PAGE, sizeof *p);
+      if (!p) nish_region_die(NISH_REGION_OOM);
+      __atomic_store_n(page, p, __ATOMIC_RELEASE);
+    }
+    s = &(*page)[index % NISH_REGION_PAGE];
+    __atomic_store_n(&s->gen, 1, __ATOMIC_RELAXED);
+  }
+  s->chunk = 8;
+  s->outer = 0;
+  __atomic_store_n(&s->thread, me, __ATOMIC_RELAXED);
+  uint64_t r = s->gen << NISH_REGION_SLOT_BITS | index;
+  nish_region_unlock();
+  return r;
+}
+
+void *nish_region_enter(uint64_t r) {
+  nish_region_slot *s = nish_region_slot_of(r);
+  if (s->entered) nish_region_die(NISH_REGION_ENTERED);
+  s->entered = 1;
+  s->outer = nish_region_current;
+  nish_region_current = r;
+  nish_region_swap(s);
+  return s->root;
+}
+
+void nish_region_leave(uint64_t r, void *root) {
+  nish_region_slot *s = nish_region_slot_of(r);
+  if (nish_region_current != r) nish_region_die(NISH_REGION_LEAVE);
+  nish_region_swap(s);
+  nish_region_current = s->outer;
+  s->entered = 0;
+  s->root = root;
+}
+
+void nish_region_free(uint64_t r) {
+  nish_region_slot *s = nish_region_slot_of(r);
+  if (s->entered) nish_region_die(NISH_REGION_FREE_ENTERED);
+  nish_region_swap(s);
+  nish_free_arena();
+  nish_region_swap(s);
+  s->root = 0;
+  uint64_t index = r & (NISH_REGION_SLOTS - 1);
+  nish_region_lock();
+  __atomic_store_n(&s->thread, 0, __ATOMIC_RELAXED);
+  uint64_t gen = s->gen + 1;
+  __atomic_store_n(&s->gen, gen, __ATOMIC_RELAXED);
+  if (gen < NISH_REGION_GEN_END) {
+    s->outer = nish_region_free_list;
+    nish_region_free_list = index + 1;
+  }
+  nish_region_unlock();
+}

package/runtime/runtime.c

@@ -174,14 +174,24 @@ static NISH_COLD void nish_oom(void) { nish_die("nish: out of memory\n"); }
    `nish_array_grow` and `nish_str_concat`. */
 #define NISH_ARRAY_MAX 2147483647u
 
-/* Slow path (size 8-byte rounded): push a chunk (>= 64 KB), bump from it. */
+/* The size of the next chunk while a region is entered (runtime-region.c
+   swaps it in and out with the region's arena), or 0 for the thread's own
+   arena, whose chunks are at least 64 KB. Defined here, not there, so that
+   this file still links alone. */
+NISH_TLS uint64_t nish_region_chunk;
+
+/* Slow path (size 8-byte rounded): push a chunk (>= 64 KB, or a region's
+   next size), bump from it. A region's chunks double from its first. */
 void *nish_arena_grow(uint64_t size) {
 #ifdef NISH_ASAN
   /* An exact chunk leaves off == cap, so the inline bump never fits again and
      every block comes through here: one malloc, one redzone, per block. */
   uint64_t cap = size;
 #else
-  uint64_t cap = size > 65536 ? size : 65536;
+  uint64_t next = nish_region_chunk;
+  uint64_t floor = next ? next : 65536;
+  uint64_t cap = size > floor ? size : floor;
+  if (next) nish_region_chunk = cap * 2;
 #endif
   /* A 64-bit request cannot always be asked for: wasm32's `size_t` is 32 bits,
      and malloc would silently take the low half of a chunk nobody can address. */

package/scripts/build.sh

@@ -3,7 +3,7 @@
 #
 #   scripts/build.sh <module.ll> [more .ll/.c files...] -o <out> [--profile debug|speed|size|wasm|wasi|napi]
 #
-# The C runtime is fourteen translation units and is named as one: an input
+# The C runtime is fifteen translation units and is named as one: an input
 # <dir>/runtime.c also compiles <dir>/runtime-os.c, the half that wraps the
 # system calls (files, directories, subprocesses, the environment, the clock),
 # <dir>/runtime-parallel.c, the half that divides a range of work across
@@ -17,8 +17,9 @@
 # <dir>/runtime-checksum.c, CRC-32 on its carry-less multiply or CRC instructions,
 # <dir>/runtime-term.c, stdin and the terminal of `nish:term`,
 # <dir>/runtime-bytes.c, the byte-class runs of `nish/bytes`,
-# <dir>/runtime-sandbox.c, the confinement `--sandbox` applies, and
-# <dir>/runtime-shm.c, the shared-memory regions of `nish/sync/shm`. Each of those
+# <dir>/runtime-sandbox.c, the confinement `--sandbox` applies,
+# <dir>/runtime-shm.c, the shared-memory regions of `nish/sync/shm`, and
+# <dir>/runtime-region.c, the arenas a program makes and frees one at a time. Each of those
 # files says why they are compiled and measured apart.
 #
 # Profiles:
@@ -222,11 +223,11 @@ if [ -n "$sqlite_to" ] || [ -n "$sqlite_from" ]; then
 fi
 [ -n "$out" ] || [ -n "$objects_to" ] || [ -n "$sqlite_to" ] || { echo "error: -o <out> is required" >&2; exit 2; }
 
-# The runtime is fourteen translation units, and a caller names one: whoever passes
+# The runtime is fifteen translation units, and a caller names one: whoever passes
 # <dir>/runtime.c gets <dir>/runtime-os.c, <dir>/runtime-parallel.c,
 # <dir>/runtime-cross.c, <dir>/runtime-host.c, <dir>/runtime-net.c, <dir>/runtime-simd.c,
 # <dir>/runtime-aes.c, <dir>/runtime-sha256.c, <dir>/runtime-checksum.c, <dir>/runtime-term.c,
-# <dir>/runtime-bytes.c, <dir>/runtime-sandbox.c and <dir>/runtime-shm.c compiled beside it, on every profile,
+# <dir>/runtime-bytes.c, <dir>/runtime-sandbox.c, <dir>/runtime-shm.c and <dir>/runtime-region.c compiled beside it, on every profile,
 # wasi included: runtime-simd.c and runtime-bytes.c take their scalar paths there
 # unless the build adds -msimd128,
 # runtime-aes.c, runtime-sha256.c and runtime-checksum.c compile to an answer of 0 and
@@ -243,7 +244,7 @@ fi
 # here keeps every one of those correct, and keeps "the runtime" one thing to
 # name from the outside. A caller that names one itself is left alone, because
 # naming one object twice is a duplicate-symbol error.
-halves="runtime-os.c runtime-parallel.c runtime-cross.c runtime-host.c runtime-net.c runtime-simd.c runtime-aes.c runtime-sha256.c runtime-checksum.c runtime-term.c runtime-bytes.c runtime-sandbox.c runtime-shm.c"
+halves="runtime-os.c runtime-parallel.c runtime-cross.c runtime-host.c runtime-net.c runtime-simd.c runtime-aes.c runtime-sha256.c runtime-checksum.c runtime-term.c runtime-bytes.c runtime-sandbox.c runtime-shm.c runtime-region.c"
 for i in ${inputs[@]+"${inputs[@]}"}; do
   case "$i" in
     */runtime.c|runtime.c)
@@ -671,6 +672,18 @@ case "$profile" in
     "$CC" "${common[@]}" "${optimise[@]}" ${lto_parts[@]+"${lto_parts[@]}"} \
       "${gc[@]}" ${strip_flag[@]+"${strip_flag[@]}"} "${inputs[@]}" -lm -o "$out" ;;
   wasm)
+    # A region's chunks come from malloc, and the freestanding wasm runtime
+    # (runtime-wasm.c) has none, so IR that makes or enters one is refused here
+    # rather than failing at the link on an undefined `nish_region_*`. The wasi
+    # profile links runtime-region.c with the rest of the runtime.
+    for i in ${inputs[@]+"${inputs[@]}"}; do
+      case "$i" in
+        *.ll) if grep -q "@nish_region_" "$i"; then
+          echo "error: $i uses regions, which the wasm profile cannot link (it has no malloc); build it with --profile wasi" >&2
+          exit 2
+        fi ;;
+      esac
+    done
     # clang resolves wasm-ld next to its own binary first, then on PATH; ask it
     # rather than probing PATH so a Homebrew llvm without a PATH entry still works.
     wasm_ld=$("$CC" -print-prog-name=wasm-ld)

package/std/README.md

@@ -250,7 +250,9 @@ reads one, and it still takes its randomness as an argument.
 | [`net/http-compress.ts`](./net/http-compress.ts) | Content coding for HTTP responses: `httpAcceptEncoding(buf, start, end, offered)` reads an `Accept-Encoding` value (q-values, `*`, `identity`, `x-gzip`) into `HTTP_CODING_GZIP`, `HTTP_CODING_DEFLATE`, `HTTP_CODING_IDENTITY` or `HTTP_CODING_NONE`, allocating nothing; `HttpCompressor(level, minLength, maxLength)` gzips a body into its own buffer when it is worth it (-1 to send it as it is) and builds the response fields (`addFields`: `Content-Encoding: gzip` and, while `vary`, `Vary: Accept-Encoding`). It sends gzip only when it pays: `minLength`, `minSaving` bytes or `minSavingPercent`, and `segmentMode` (fewer TCP segments of `mss`, head included, than as it is); `decision` says why (`HTTP_COMPRESS_SENT`, `_NOT_ACCEPTED`, `_TOO_SMALL`, `_TOO_LARGE`, `_TOO_LITTLE_SAVED`, `_NO_FEWER_SEGMENTS`). Bodies up to `fixedLimit` use the fixed-Huffman encoder (`encoder`: `HTTP_ENCODER_AUTO`, `_FIXED`, `_LEVEL`). On HTTP/1.1, `Http1Connection.acceptEncoding()` and `respondEncoded(status, names, values, body, off, len, enc, coding)` answer a request in one call with the compressed `Content-Length`, or nothing at all (`H1_AGAIN`, `H1_INVALID`); `identity;q=0` is honoured. Opt-in; BREACH is the program's to avoid (`docs/security/compress.md`) | RFC 9110 §8.4 and §12.5.3: the negotiation table, and sans-IO responses gunzipped back (`tests/link/std_http_compress`) |
 | [`net/router.ts`](./net/router.ts) | `Router`, a method and path router that knows no carrier and allocates nothing per match: `add(method, pattern)` answers the route's index (0, 1, 2 in the order added) or a refusal — `ROUTER_BAD_PATTERN`, `ROUTER_BAD_METHOD`, `ROUTER_TOO_LONG`, `ROUTER_AMBIGUOUS` (`*` not last), `ROUTER_DUPLICATE` (the same shape and method, whatever the parameter names) or `ROUTER_FULL`, against `RouterLimits`; `match(method, mStart, mEnd, target, tStart, tEnd, params)` answers the index, `ROUTER_NOT_FOUND` or `ROUTER_METHOD_NOT_ALLOWED`, writing each `:name` and a final `*` as a `[start, end)` pair into the caller's `params` (`ROUTER_TOO_MANY_PARAMS` when it is short). Literal before parameter before `*`, segment by segment, backtracking; a lookup enters each node at most once. The query is left unparsed and nothing is decoded: `routerDecode` decodes one parameter (`ROUTER_BAD_ESCAPE`, `ROUTER_NO_ROOM`) (`docs/security/router.md`) | Every refusal code, precedence and backtracking, 405 against 404, `%2F` kept in one segment, the query, `*` and authority-form targets, a target deeper than any route, the lookup bound, params past storage, and the arena flat over matches (`tests/link/net_router`, `net_router_f64`, five `net_router_negative_*`) |
 | [`net/handler.ts`](./net/handler.ts) | `HttpHandler`, complete requests and their answers over the HTTP/1.1 carrier: it owns the readiness loop, routes each request through a `Router` at its head, gathers its body up to `HandlerLimits.http.maxBody` (1 MiB; past it the carrier answers 413), and hands it over whole. The program pulls — `wait(ms)`, `ready()`, `take()` — and switches on `req.route`; `HttpRequest` reads parameters, form-decoded query values and fields as ranges into its own buffers. Each request is answered once, now or in a later turn, by `send`, `sendFields`, `empty`, `json()` + `sendJson`, `problem` (RFC 9457), `refuse` (404, or 405 with `Allow`) or `start`/`write`/`end`; an answer whose connection closed first is `HANDLER_DROPPED` and reaches no later peer. `HEAD` takes a `GET` route. `watch`/`fired` add the program's own descriptors (`signalFd()`), and `drain` stops gracefully. Nothing is allocated per request once warm (`docs/security/handler.md`) | Routing, parameters and form-decoded queries, `HEAD`, 404 and 405 problems, chunked bodies in and out, the body cap at and one past it, pipelining order, late answers dropped across a reused slot, the arena and pool flat over two thousand requests, and the drain (`tests/link/net_handler`, and `net_http_app` over `examples/http-app`) |
+| [`net/upstream.ts`](./net/upstream.ts) | `Upstream`, outbound HTTP/1.1 calls from a route handler that never block it: a pool of keep-alive connections to one `http:` origin on the `HttpHandler`'s own loop (watch ids `UpstreamLimits.firstId` and up). `get(req, path)` returns at once, or `UPSTREAM_E_FULL` with nothing held; each call comes back once through `ready()`/`take()` tagged with `r.req`, `ok` with its status and `body[0 .. length)`, or failed with the codec's `HTTP1_E_` refusal or an `UPSTREAM_E_` code (timeout, closed, connect, origin, lookup, blocked, request), and goes back with `release`. The origin is looked up once at start-up and pinned; `UpstreamLimits.outbound` checks it against an `OutboundPolicy`. A failed call's connection is closed. Each connection keeps one codec in spans mode, so a warm pool allocates nothing per call | A slow upstream while other clients and a second upstream are answered, the timeout, keep-alive and tagging, five codec refusals as failures, the policy refusing loopback, origin and request-target refusals, three thousand warm calls on one connection with `Arena.used()` flat (`tests/link/net_upstream`), and a second `release` panicking (`net_upstream_release`) |
 | [`net/postgres.ts`](./net/postgres.ts) | `PgConnection`, a PostgreSQL client that reads no socket: the program fills `inputBuffer()` and sends `outputBuffer()`, and `next()` answers `PG_NEED_MORE`, `PG_READY`, `PG_DESCRIBED`, `PG_ROW`, `PG_COMPLETE`, `PG_ERROR`, `PG_NOTICE`, `PG_GROUP_DONE` or the final `PG_FAILED` (`failure`, one of the `PG_ERR_*` codes). Protocol 3.0: startup, `trust` and SCRAM-SHA-256 with the server's signature checked (cleartext and MD5 refused), the extended query protocol — `prepare`, `bindStart`, `paramInt` / `paramText` / `paramBytes` / `paramTextArray` / `paramIntArray` / `paramNull`, an array built element by element into the Bind (`arrayStart`, `arrayElement`, `arrayInt`, `arrayNull`, `arrayEnd`), `bindEnd`, `bindStartBinary` for a Bind whose result columns come in binary (read in place by `binaryInt`, `binaryBool`, `binaryFloat` and `PgBinary`'s `timestamp` and `iso`, and a one-dimensional `int2[]`, `int4[]` or `int8[]` by `PgBinaryArray`), `bindCancel` to drop a Bind not yet ended, `closeStatement`, `sync` — with up to `maxInFlight` groups pipelined. `PgBatcher` queues keyed reads of one statement shape (`addInt`, `addText`) and `take(conn)` binds up to `maxRequests` of them, each key once, as one execution over `unnest($1) with ordinality`; `row`, `end` and `fail` route its rows by their server-supplied ordinal, checked first (read with `binaryInt` when `PgBatchConfig.binary` is set, so the batch's rows come in binary too), to `next(run)`'s per-request `PG_BATCH_ROW` and `PG_BATCH_DONE` / `PG_BATCH_MISSING` / `PG_BATCH_FAILED`; `take` answers a fresh handle each time, so a server's late or repeated event on a spent one is counted in `stray`, never routed or panicked on. It reads no clock and allocates nothing per request or row. `PgBatchClock` says when to take, from times the program passes in: once max(`minCount`, the moving average of batch sizes × `factor`) wait, the oldest has waited `maxWaitUs`, or, with the idle bypass, nothing is in flight and the load is light (its defaults are the fixed window); `PgBatcher.takeDue` takes only then, and `PgBatchTurns` lets one free connection collect while the others execute. A row's columns are windows onto the receive buffer (`colStart`, `colLength`), read in place by `int`, `bool`, `isNull` and `PgText`'s decoders (`timestamptz`, `PgArrayWalker` for `text[]`); no per-row or per-column allocation. The password is a `Secret<u8[]>`, wiped once the proof is made. Caps on message size, columns, groups in flight and SCRAM iterations (`docs/security/postgres.md`) | PostgreSQL 18's frontend/backend protocol chapter, RFC 5802 and RFC 7677: every builder, event and refusal (`tests/link/std_postgres`), and exchanges recorded from PostgreSQL 18 by `tests/postgres/record.mjs` replayed byte for byte (`tests/link/std_postgres_replay`); the batcher's queueing, caps and ordinal refusals (`tests/link/std_postgres_batch`); the clock's reasons, backlog and turns (`tests/link/std_postgres_batch_clock`) |
+| [`net/postgres-carrier.ts`](./net/postgres-carrier.ts) | `PgCarrier`, a `PgPool`'s queries on an `HttpHandler`'s loop: its sockets sit in the handler's epoll set, a route handler hands a request to `run` (or `runInt` / `runText` for one of up to 8 attached `PgBatcher`s) and gets its number at once, and `next()` answers `PG_CARRIER_BUILD` to bind a plain query on `connection()`, `PG_CARRIER_ROW`, `PG_CARRIER_COMPLETE`, `PG_CARRIER_ERROR` and one final `PG_CARRIER_DONE`, `PG_CARRIER_MISSING` or `PG_CARRIER_FAILED` (`failure`, `socketError`) with `request()` the request to answer and `tag` the caller's. It connects the pool's slots itself (TCP or a Unix socket), times out a connect after `connectTimeoutMs`, sheds batched keys when no slot can connect, and sizes every table from `PgCarrierLimits` at construction: `run` answers `PG_CARRIER_FULL` rather than grow one, and once connections are up it allocates nothing per query. Kept apart from `net/postgres.ts` so that module reaches no capability (`docs/security/postgres.md`) | A loopback backend in the test itself: routing, a slow query not stalling another client, lost and refused connections, a connect timeout, and `Arena.used()` flat over thousands of queries (`tests/link/std_postgres_carrier`); a live server by `tests/postgres/live/run.mjs` |
 | [`net/prefork.ts`](./net/prefork.ts) | A server on every core: `preforkWorkers(process.argv)` or `preforkWith(process.argv, config)` answers a worker's index in each worker it starts again from the same program, and in the supervisor runs them until SIGTERM or SIGINT, restarting one that exits after a backoff. `PreforkConfig.supervisorChannels` gives each worker a `unixSocketPair` to the supervisor, which `preforkSupervise(args, config, state, onChannel)` hands to a callback; `serviceWorker` gives every other worker a pair to that one, and makes the workers one group that restarts together on fresh pairs. The ends arrive from descriptor 3 through `spawnWithFds`, found with `preforkSupervisorChannel`, `preforkServiceChannel` and `preforkServiceChannelOf`. Native only | `tests/link/net_prefork` (the spread, a dead worker, SIGTERM, the backoff); `tests/link/net_prefork_channels` (each worker holds exactly its own ends, a restart, the supervisor's descriptors flat) |
 
 **How a carrier drives `TlsServer`.** The server speaks in handshake