@amritk/nish-x86_64-linux 0.30.0 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/nish +0 -0
- package/docs/AI.md +56 -4
- package/package.json +1 -1
- package/runtime/core/host.d.ts +2 -0
- package/runtime/core/host.mjs +9 -0
- package/runtime/nish.d.ts +48 -2
- package/runtime/nish.h +9 -1
- package/runtime/nish.mjs +36 -0
- package/runtime/runtime-net.c +32 -3
- package/runtime/runtime-os.c +33 -1
- package/runtime/runtime-region.c +15 -3
- package/runtime/runtime.c +9 -2
- package/std/README.md +8 -7
- package/std/net/{http-client.ts → fetch.ts} +17 -17
- package/std/net/handler.ts +1 -1
- package/std/net/{http1-client.ts → http1-client-codec.ts} +6 -6
- package/std/net/{http1.ts → http1-codec.ts} +2 -2
- package/std/net/http1-server.ts +4 -4
- package/std/net/outbound.ts +2 -2
- package/std/net/postgres.ts +383 -49
- package/std/net/prefork.ts +149 -13
- package/std/net/router.ts +1 -1
- package/std/net/upstream.ts +3 -3
- package/std/net/websocket.ts +5 -5
- package/std/region.ts +95 -0
package/bin/nish
CHANGED
|
Binary file
|
package/docs/AI.md
CHANGED
|
@@ -2222,6 +2222,13 @@ system call either way. `setIdlePriority()` moves the program to Linux's `SCHED_
|
|
|
2222
2222
|
`main`, never in a task. An unprivileged process can never leave `SCHED_IDLE`
|
|
2223
2223
|
and an idle one can starve on a saturated machine, so keep a normal-priority
|
|
2224
2224
|
fallback. Neither exists on wasm; both throw under Node.
|
|
2225
|
+
`setCpuAffinity(cpu)` pins the process to one CPU, narrow-only: a CPU outside
|
|
2226
|
+
the mask it already has is `-22`; `-95` on Darwin and under Node, refused on
|
|
2227
|
+
wasm. `nish/net/prefork`'s `PreforkConfig` takes `pin = true` to pin worker
|
|
2228
|
+
`k` to CPU `k` (more workers than CPUs panics), and `listenPort` (with
|
|
2229
|
+
`listenHost`, `listenBacklog`, `listenFlags` 0 or 2) to have the supervisor
|
|
2230
|
+
bind one `SO_REUSEPORT` listener per worker, steered on Linux by the receiving
|
|
2231
|
+
CPU when pinned; a worker takes its own with `preforkListener(process.argv)`.
|
|
2225
2232
|
|
|
2226
2233
|
**Sockets** (`nish:net`, or the same names as globals). `tcpListen(host, port,
|
|
2227
2234
|
backlog)`, `tcpAccept(fd, peer)`, `netRead(fd, buf, off, len)`,
|
|
@@ -2236,10 +2243,10 @@ wait for it to be writable, then `connectResult(fd)` is 0 or the failure. Buffer
|
|
|
2236
2243
|
are `u8[]` only; an address is 18 bytes of one (IPv4 as `::ffff:a.b.c.d`, then
|
|
2237
2244
|
the port big-endian); hosts are numeric literals; to reach a name, resolve it first with
|
|
2238
2245
|
`nish/net/dns` (`DnsClient` on your loop, or the blocking `resolveHost`).
|
|
2239
|
-
To fetch a URL from a CLI, skip all of this: `nish/net/
|
|
2240
|
-
`
|
|
2246
|
+
To fetch a URL from a CLI, skip all of this: `nish/net/fetch`'s
|
|
2247
|
+
`fetchSync(new HttpRequest("GET", url), new HttpOptions())` blocks until the
|
|
2241
2248
|
response's head is in and answers it with `body.readAll(limit)`, or null on a
|
|
2242
|
-
failure (
|
|
2249
|
+
failure (a `FetchSyncClient` keeps the connection and says why in `error`).
|
|
2243
2250
|
`netRead` / `netWrite` panic on a range outside the buffer.
|
|
2244
2251
|
`tcpListenFlags(host, port, backlog, flags)` is `tcpListen` with flag 1
|
|
2245
2252
|
`SO_REUSEPORT`, so several listeners, in one process or many, share a port (Linux
|
|
@@ -2596,6 +2603,46 @@ returns. To give memory back sooner, scope it with `using a = arena()`;
|
|
|
2596
2603
|
`Arena.release` and `Arena.reset` are gone, and a call to either is an unknown
|
|
2597
2604
|
builtin (NL2244).
|
|
2598
2605
|
|
|
2606
|
+
### Regions: `nish:region`
|
|
2607
|
+
|
|
2608
|
+
**For memory a long-running program frees itself — per connection, per cache
|
|
2609
|
+
entry — use a region.** `region(f)` / `regionWith(arg, f)` make one and run
|
|
2610
|
+
`f` in it, and what `f` answers is the root; `enter(r, f)` /
|
|
2611
|
+
`enterWith(r, arg, f)` run `f(root)` / `f(root, arg)` in it again; `free(r)`
|
|
2612
|
+
gives all of it back. `f` is a named function or an arrow that captures
|
|
2613
|
+
nothing. `Region<T>` is a number, not a pointer.
|
|
2614
|
+
|
|
2615
|
+
```ts nish:ok
|
|
2616
|
+
import { Region, enter, free, region } from "nish:region";
|
|
2617
|
+
|
|
2618
|
+
class Session {
|
|
2619
|
+
hits: i32 = 0;
|
|
2620
|
+
}
|
|
2621
|
+
|
|
2622
|
+
const open = (): Session => new Session();
|
|
2623
|
+
const touch = (s: Session): i32 => {
|
|
2624
|
+
s.hits = s.hits + 1;
|
|
2625
|
+
return s.hits;
|
|
2626
|
+
};
|
|
2627
|
+
|
|
2628
|
+
export const main = (): i32 => {
|
|
2629
|
+
const r: Region<Session> = region(open);
|
|
2630
|
+
console.log(enter(r, touch));
|
|
2631
|
+
free(r);
|
|
2632
|
+
return 0;
|
|
2633
|
+
};
|
|
2634
|
+
```
|
|
2635
|
+
|
|
2636
|
+
The checker refuses every way region memory could outlive `free`, and every way
|
|
2637
|
+
outer memory could be pointed at from inside: only numbers, booleans, enums and
|
|
2638
|
+
`Region` handles cross. `enter` answers a scalar or a handle (NL2611), `f` may
|
|
2639
|
+
not store region memory into `arg` or other outer memory (NL2609) nor outer
|
|
2640
|
+
memory into the region (NL2610), nor `push`/`set`/`add` to an outer collection
|
|
2641
|
+
(NL2615). A handle stays on its thread (NL2612, NL2613, NL2351), is never a
|
|
2642
|
+
`using` (NL2387) and is never `new Region` (NL2618). A bad handle, a double
|
|
2643
|
+
`free` or a nested `enter` of one region stops the program. `--emit-ts` refuses
|
|
2644
|
+
the module (NL4015).
|
|
2645
|
+
|
|
2599
2646
|
## Recipes for what is missing
|
|
2600
2647
|
|
|
2601
2648
|
**Iterate and transform** — there is no `map`, and no function to pass it:
|
|
@@ -2705,7 +2752,12 @@ parameter. `paramIntArray(values, start, count)` binds an `i64[]` window;
|
|
|
2705
2752
|
element, then `arrayEnd()`, builds a `text[]` straight into the Bind from
|
|
2706
2753
|
windows of a buffer. An element refused for room answers `false` and leaves
|
|
2707
2754
|
the array open: end it and send the rest in the next statement. Every other
|
|
2708
|
-
`param...` call and `bindEnd()` panic until `arrayEnd()`.
|
|
2755
|
+
`param...` call and `bindEnd()` panic until `arrayEnd()`. For a long
|
|
2756
|
+
`int4[]` or `int8[]`, `paramInt4Array` / `paramInt8Array` (same window) or
|
|
2757
|
+
`binaryArrayStart(23 or 20)`, `binaryArrayInt` / `binaryArrayNull`,
|
|
2758
|
+
`arrayEnd()` send it in binary, at a tenth of the client CPU; the element type
|
|
2759
|
+
must be the statement's (`int4[]` to an `int8[]` parameter is SQLSTATE
|
|
2760
|
+
`42804`), and an `int4` element out of range panics. (No block, as for
|
|
2709
2761
|
`nish/text`; `tests/postgres/live` runs it against a server, and the user
|
|
2710
2762
|
guide's [PostgreSQL page](guide/using-nish/postgres.md#array-parameters) its
|
|
2711
2763
|
walk-through.)
|
package/package.json
CHANGED
package/runtime/core/host.d.ts
CHANGED
|
@@ -82,6 +82,8 @@ export declare function readSignal(fd: i32): i32;
|
|
|
82
82
|
/** These two have no reading under Node: each throws, naming itself. */
|
|
83
83
|
export declare function setNice(n: i32): i32;
|
|
84
84
|
export declare function setIdlePriority(): i32;
|
|
85
|
+
/** -95: there is no affinity call under Node. */
|
|
86
|
+
export declare function setCpuAffinity(cpu: i32): i32;
|
|
85
87
|
/** What each `nish:net` function is under the core: a throw naming `name`. */
|
|
86
88
|
export declare function noNetReading(name: string): never;
|
|
87
89
|
/** What `readStdin`, `waitStdin`, `setRawMode` and `setTerminalRestore` are under the core: a throw naming `name`. */
|
package/runtime/core/host.mjs
CHANGED
|
@@ -335,6 +335,15 @@ export function setIdlePriority() {
|
|
|
335
335
|
return noSchedulingReading("setIdlePriority");
|
|
336
336
|
}
|
|
337
337
|
|
|
338
|
+
/**
|
|
339
|
+
* `setCpuAffinity(cpu)` answers -95, the native build's answer on a host with
|
|
340
|
+
* no affinity call: unlike the two above it has a defined "not supported"
|
|
341
|
+
* value, so a program that pins where it can runs unpinned here.
|
|
342
|
+
*/
|
|
343
|
+
export function setCpuAffinity() {
|
|
344
|
+
return -95;
|
|
345
|
+
}
|
|
346
|
+
|
|
338
347
|
/**
|
|
339
348
|
* `signalFd()` and `readSignal(fd)` have no faithful reading under Node, and
|
|
340
349
|
* these say so rather than answer something else. Node delivers a signal to
|
package/runtime/nish.d.ts
CHANGED
|
@@ -378,6 +378,14 @@ declare function setNice(n: i32): i32;
|
|
|
378
378
|
* normal-priority fallback for work that must finish. Throws under Node.
|
|
379
379
|
*/
|
|
380
380
|
declare function setIdlePriority(): i32;
|
|
381
|
+
/**
|
|
382
|
+
* Pin this process to CPU `cpu` alone (`sched_setaffinity`): 0, or a negative
|
|
383
|
+
* errno. Narrow-only: a CPU outside the mask the process already has (an
|
|
384
|
+
* operator's `taskset` or cpuset), a negative one, or one past the kernel's
|
|
385
|
+
* set size is -22 with no system call. -95 where the host has no affinity
|
|
386
|
+
* call (Darwin, and under Node).
|
|
387
|
+
*/
|
|
388
|
+
declare function setCpuAffinity(cpu: i32): i32;
|
|
381
389
|
/** One environment variable, or `null` when it is unset (an empty value is a set variable). */
|
|
382
390
|
declare function getenv(name: string): string | null;
|
|
383
391
|
/**
|
|
@@ -422,7 +430,13 @@ declare function netAddress(out: u8[], host: string, port: i32): i32;
|
|
|
422
430
|
declare function netLocalPort(fd: i32): i32;
|
|
423
431
|
/** A listening TCP socket with `SO_REUSEADDR`; `"::"` hears IPv4 and IPv6. */
|
|
424
432
|
declare function tcpListen(host: string, port: i32, backlog: i32): i32;
|
|
425
|
-
/**
|
|
433
|
+
/**
|
|
434
|
+
* `tcpListen`'s socket, with `SO_REUSEPORT` for flag 1 and `TCP_NODELAY` on
|
|
435
|
+
* each accepted socket for flag 2. Flag 4, with flag 1, steers by CPU: a new
|
|
436
|
+
* connection goes to the group's listener whose bind order is the index of the
|
|
437
|
+
* CPU that received it (Linux; -95 elsewhere). Any other bit, or 4 without 1,
|
|
438
|
+
* is -22.
|
|
439
|
+
*/
|
|
426
440
|
declare function tcpListenFlags(host: string, port: i32, backlog: i32, flags: i32): i32;
|
|
427
441
|
/** The next connection's descriptor, its address in `peer`; -11 when none waits. */
|
|
428
442
|
declare function tcpAccept(fd: i32, peer: u8[]): i32;
|
|
@@ -667,6 +681,8 @@ declare module "nish:process" {
|
|
|
667
681
|
* on a saturated machine, so a caller keeps a normal-priority fallback.
|
|
668
682
|
*/
|
|
669
683
|
export function setIdlePriority(): i32;
|
|
684
|
+
/** Pin this process to CPU `cpu` alone: 0, -22 for a CPU outside the current mask (narrow-only), or -95 off Linux. */
|
|
685
|
+
export function setCpuAffinity(cpu: i32): i32;
|
|
670
686
|
/**
|
|
671
687
|
* A monotonic clock in nanoseconds. The origin is arbitrary, so only the
|
|
672
688
|
* difference between two reads is meaningful.
|
|
@@ -693,7 +709,13 @@ declare module "nish:net" {
|
|
|
693
709
|
export function netLocalPort(fd: i32): i32;
|
|
694
710
|
/** A listening TCP socket with `SO_REUSEADDR`; `"::"` hears IPv4 and IPv6. */
|
|
695
711
|
export function tcpListen(host: string, port: i32, backlog: i32): i32;
|
|
696
|
-
/**
|
|
712
|
+
/**
|
|
713
|
+
* `tcpListen`'s socket, with `SO_REUSEPORT` for flag 1 and `TCP_NODELAY` on
|
|
714
|
+
* each accepted socket for flag 2. Flag 4, with flag 1, steers by CPU: a new
|
|
715
|
+
* connection goes to the group's listener whose bind order is the index of the
|
|
716
|
+
* CPU that received it (Linux; -95 elsewhere). Any other bit, or 4 without 1,
|
|
717
|
+
* is -22.
|
|
718
|
+
*/
|
|
697
719
|
export function tcpListenFlags(host: string, port: i32, backlog: i32, flags: i32): i32;
|
|
698
720
|
/** The next connection's descriptor, its address in `peer`; -11 when none waits. */
|
|
699
721
|
export function tcpAccept(fd: i32, peer: u8[]): i32;
|
|
@@ -936,6 +958,30 @@ declare module "nish:secret" {
|
|
|
936
958
|
export function wipe(target: object): void;
|
|
937
959
|
}
|
|
938
960
|
|
|
961
|
+
// `nish:region` has source behind it, `std/region.ts`, but a program imports it
|
|
962
|
+
// by this name. The class has a private constructor, so `tsc` refuses
|
|
963
|
+
// `new Region()` as `nish` does (NL2618). What `nish` refuses about the
|
|
964
|
+
// functions a region runs — keeping its memory outside it, growing outer
|
|
965
|
+
// storage, reaching threads — is flow, which `tsc` cannot see
|
|
966
|
+
// (docs/LANGUAGE.md -> Regions).
|
|
967
|
+
declare module "nish:region" {
|
|
968
|
+
/** A handle to a region whose root is a `T`: 64 bits, not a pointer. */
|
|
969
|
+
export class Region<T> {
|
|
970
|
+
private constructor();
|
|
971
|
+
private readonly root: T;
|
|
972
|
+
}
|
|
973
|
+
/** A new region whose root is what `f` answers; everything `f` allocates is in it. */
|
|
974
|
+
export function region<T>(f: () => T): Region<T>;
|
|
975
|
+
/** `region` with one outside argument for `f`, which `f` may read but not keep. */
|
|
976
|
+
export function regionWith<A, T>(arg: A, f: (arg: A) => T): Region<T>;
|
|
977
|
+
/** `f(root)`, allocating in the region. What `f` answers is a scalar or a handle. */
|
|
978
|
+
export function enter<T, R>(r: Region<T>, f: (root: T) => R): R;
|
|
979
|
+
/** `f(root, arg)`, under `enter`'s rules; `f` may not keep or grow `arg`. */
|
|
980
|
+
export function enterWith<T, A, R>(r: Region<T>, arg: A, f: (root: T, arg: A) => R): R;
|
|
981
|
+
/** Give the region's memory back. A freed handle used again stops the program. */
|
|
982
|
+
export function free<T>(r: Region<T>): void;
|
|
983
|
+
}
|
|
984
|
+
|
|
939
985
|
// ---- Arena (docs/LANGUAGE.md -> Arena) ---------------------------------------
|
|
940
986
|
|
|
941
987
|
declare const Arena: {
|
package/runtime/nish.h
CHANGED
|
@@ -577,6 +577,11 @@ int32_t nish_set_nice(int32_t n);
|
|
|
577
577
|
* negative errno; -95 (Linux's EOPNOTSUPP) elsewhere, -ENOSYS under WASI. An
|
|
578
578
|
* unprivileged process cannot leave the policy again. */
|
|
579
579
|
int32_t nish_set_idle_priority(void);
|
|
580
|
+
/* `setCpuAffinity(cpu)`: this process pinned to CPU `cpu` alone
|
|
581
|
+
* (`sched_setaffinity`, Linux), 0 or a negative errno. Narrow-only: a CPU not
|
|
582
|
+
* in the current mask, negative, or past `CPU_SETSIZE` is -22 (-EINVAL) with
|
|
583
|
+
* no system call. -95 (Linux's EOPNOTSUPP) elsewhere, -ENOSYS under WASI. */
|
|
584
|
+
int32_t nish_set_cpu_affinity(int32_t cpu);
|
|
580
585
|
|
|
581
586
|
/* ---- The environment (WP19 R1) ------------------------------------------
|
|
582
587
|
* `getenv(name)`: the value of environment variable `name`, copied into the
|
|
@@ -679,7 +684,10 @@ bool nish_is_executable(const nish_str *path);
|
|
|
679
684
|
* `nish_tcp_listen(host, port, backlog)`: `SO_REUSEADDR`, and `::` hears
|
|
680
685
|
* both families, or IPv4 alone on a kernel without IPv6.
|
|
681
686
|
* `nish_tcp_listen_flags(host, port, backlog, flags)`: the same listener,
|
|
682
|
-
* with `SO_REUSEPORT` for flag 1
|
|
687
|
+
* with `SO_REUSEPORT` for flag 1, `TCP_NODELAY` on each accepted socket for
|
|
688
|
+
* flag 2, and, for flag 4 with flag 1, a reuseport program steering each new
|
|
689
|
+
* connection to the listener bound at the index of the CPU that received it
|
|
690
|
+
* (Linux; -95 elsewhere). Any other bit, or 4 without 1, is -22.
|
|
683
691
|
* `nish_tcp_accept(fd, peer)`: a new descriptor and the peer's address.
|
|
684
692
|
* `nish_net_read` / `nish_net_write(fd, buf, off, len)`: bytes `[off, off +
|
|
685
693
|
* len)` of `buf`, which the compiled call has range-checked; a write to a
|
package/runtime/nish.mjs
CHANGED
|
@@ -121,6 +121,9 @@ for (const name of ["setNice", "setIdlePriority"]) {
|
|
|
121
121
|
throw new Error(`\`${name}\` has no reading under Node: it changes how the kernel schedules the native program`);
|
|
122
122
|
});
|
|
123
123
|
}
|
|
124
|
+
// `setCpuAffinity` answers -95, as on Darwin: Node has no affinity call, and
|
|
125
|
+
// "not supported" is a value a caller already branches on.
|
|
126
|
+
provide("setCpuAffinity", shim.setCpuAffinity);
|
|
124
127
|
provide("getenv", shim.getenv);
|
|
125
128
|
provide("statMtimeSync", shim.statMtimeSync);
|
|
126
129
|
provide("signalFd", shim.signalFd);
|
|
@@ -328,11 +331,41 @@ provide("arena", shim.arena);
|
|
|
328
331
|
// zero-fills it. `nish:unsafe` and `nish:term` are each answered by a module
|
|
329
332
|
// that hands back the globals installed above: its functions are globals of this module either
|
|
330
333
|
// way, and the import is what the compiler requires a program to write.
|
|
334
|
+
// `nish:region` is answered by a module of its own, below.
|
|
331
335
|
// `nish:simd` is `runtime/core.mjs`, whose vectors are typed arrays made once
|
|
332
336
|
// and never written after, so `--emit-ts` output and a program run as it was
|
|
333
337
|
// written share one implementation.
|
|
334
338
|
const UNSAFE_EXPORTS = ["uncheckedGet", "uncheckedSet", "wrappingAdd", "wrappingSub", "wrappingMul"];
|
|
335
339
|
const unsafeSource = UNSAFE_EXPORTS.map((name) => `export const ${name} = globalThis.${name};`).join("\n");
|
|
340
|
+
// `nish:region` has no allocator to swap under Node: a region is an entry in a
|
|
341
|
+
// table, keyed by its handle (a BigInt, as every `i64` is here), `enter` runs
|
|
342
|
+
// its function on the root, and `free` drops the entry. The handle checks the
|
|
343
|
+
// native runtime makes are made here too, so a freed handle or a second
|
|
344
|
+
// `enter` throws where the native program stops.
|
|
345
|
+
const regionSource = `
|
|
346
|
+
const roots = new Map();
|
|
347
|
+
const entered = new Set();
|
|
348
|
+
let next = 0n;
|
|
349
|
+
const rootOf = (r) => {
|
|
350
|
+
if (!roots.has(r)) throw new Error("nish:region: the region is freed or was never made");
|
|
351
|
+
if (entered.has(r)) throw new Error("nish:region: the region is entered already");
|
|
352
|
+
return roots.get(r);
|
|
353
|
+
};
|
|
354
|
+
const run = (r, f, arg, withArg) => {
|
|
355
|
+
const root = rootOf(r);
|
|
356
|
+
entered.add(r);
|
|
357
|
+
try { return withArg ? f(root, arg) : f(root); } finally { entered.delete(r); }
|
|
358
|
+
};
|
|
359
|
+
export class Region {}
|
|
360
|
+
export const region = (f) => { next += 1n; roots.set(next, f()); return next; };
|
|
361
|
+
export const regionWith = (arg, f) => { next += 1n; roots.set(next, f(arg)); return next; };
|
|
362
|
+
export const enter = (r, f) => run(r, f, undefined, false);
|
|
363
|
+
export const enterWith = (r, arg, f) => run(r, f, arg, true);
|
|
364
|
+
export const free = (r) => {
|
|
365
|
+
if (entered.has(r)) throw new Error("nish:region: the region is freed while it is entered");
|
|
366
|
+
if (!roots.delete(r)) throw new Error("nish:region: the region is freed or was never made");
|
|
367
|
+
};
|
|
368
|
+
`;
|
|
336
369
|
const TERM_EXPORTS = ["readStdin", "waitStdin", "isTerminal", "terminalSize", "setRawMode", "setTerminalRestore"];
|
|
337
370
|
const termSource = TERM_EXPORTS.map((name) => `export const ${name} = globalThis.${name};`).join("\n");
|
|
338
371
|
registerHooks({
|
|
@@ -346,6 +379,9 @@ registerHooks({
|
|
|
346
379
|
if (specifier === "nish:simd") {
|
|
347
380
|
return next(new URL("./core.mjs", import.meta.url).href, context);
|
|
348
381
|
}
|
|
382
|
+
if (specifier === "nish:region") {
|
|
383
|
+
return { url: `data:text/javascript,${encodeURIComponent(regionSource)}`, shortCircuit: true };
|
|
384
|
+
}
|
|
349
385
|
if (specifier === "nish:term") {
|
|
350
386
|
return { url: `data:text/javascript,${encodeURIComponent(termSource)}`, shortCircuit: true };
|
|
351
387
|
}
|
package/runtime/runtime-net.c
CHANGED
|
@@ -73,6 +73,7 @@ typedef int nish_net_unused;
|
|
|
73
73
|
#include <sys/un.h>
|
|
74
74
|
#include <unistd.h>
|
|
75
75
|
#if defined(__linux__)
|
|
76
|
+
#include <linux/filter.h>
|
|
76
77
|
#include <netinet/udp.h>
|
|
77
78
|
#include <sys/epoll.h>
|
|
78
79
|
/* glibc before 2.29 names neither; the numbers are the kernel's ABI. */
|
|
@@ -82,6 +83,10 @@ typedef int nish_net_unused;
|
|
|
82
83
|
#ifndef UDP_GRO
|
|
83
84
|
#define UDP_GRO 104
|
|
84
85
|
#endif
|
|
86
|
+
/* Linux 4.5's; a header older than the kernel may not name it. */
|
|
87
|
+
#ifndef SO_ATTACH_REUSEPORT_CBPF
|
|
88
|
+
#define SO_ATTACH_REUSEPORT_CBPF 51
|
|
89
|
+
#endif
|
|
85
90
|
#else
|
|
86
91
|
#include <sys/event.h>
|
|
87
92
|
#include <sys/time.h>
|
|
@@ -243,7 +248,23 @@ static int32_t nish_net_bound(const nish_str *host, int32_t port, int type, int3
|
|
|
243
248
|
}
|
|
244
249
|
nish_sockaddr s;
|
|
245
250
|
socklen_t n = nish_net_sockaddr(&s, a, port, v4);
|
|
246
|
-
|
|
251
|
+
#if defined(__linux__)
|
|
252
|
+
/* `tcpListenFlags` flag 4's program, two instructions, the kernel
|
|
253
|
+
selftest's: load the receiving CPU, return it. It is the reuseport
|
|
254
|
+
group's, so attaching it from every listener installs the same program
|
|
255
|
+
each time, and it goes on after `listen`, once the socket is in its
|
|
256
|
+
group. Only a stream listener's `opts` can hold 4. */
|
|
257
|
+
static struct sock_filter cpu[] = {
|
|
258
|
+
{BPF_LD | BPF_W | BPF_ABS, 0, 0, (uint32_t)(SKF_AD_OFF + SKF_AD_CPU)},
|
|
259
|
+
{BPF_RET | BPF_A, 0, 0, 0},
|
|
260
|
+
};
|
|
261
|
+
static const struct sock_fprog steer = {2, cpu};
|
|
262
|
+
#endif
|
|
263
|
+
if (bad != 0 || bind(fd, &s.sa, n) != 0 || (type == SOCK_STREAM && listen(fd, arg) != 0)
|
|
264
|
+
#if defined(__linux__)
|
|
265
|
+
|| ((opts & 4) && setsockopt(fd, SOL_SOCKET, SO_ATTACH_REUSEPORT_CBPF, &steer, sizeof steer) != 0)
|
|
266
|
+
#endif
|
|
267
|
+
) {
|
|
247
268
|
int e = errno;
|
|
248
269
|
close(fd);
|
|
249
270
|
return nish_net_err(e);
|
|
@@ -262,9 +283,17 @@ int32_t nish_tcp_listen(const nish_str *host, int32_t port, int32_t backlog) {
|
|
|
262
283
|
`SO_REUSEPORT` for flag 1, so several listeners of one uid share the port
|
|
263
284
|
(Linux spreads new connections across them; Darwin hands them all to one),
|
|
264
285
|
and `TCP_NODELAY` for flag 2, which the listener carries so that
|
|
265
|
-
`tcpAccept` sets it on each connection it takes.
|
|
286
|
+
`tcpAccept` sets it on each connection it takes. Flag 4, which needs flag
|
|
287
|
+
1, steers by CPU: a reuseport program (`SO_ATTACH_REUSEPORT_CBPF`) hands a
|
|
288
|
+
new connection to the group's listener whose index is the CPU that received
|
|
289
|
+
it, the index being the order the group's sockets were bound in, and the
|
|
290
|
+
kernel falls back to its hash when there is no such listener. Linux alone
|
|
291
|
+
has it; elsewhere flag 4 is -95. Any other bit, or 4 without 1, is -22. */
|
|
266
292
|
int32_t nish_tcp_listen_flags(const nish_str *host, int32_t port, int32_t backlog, int32_t flags) {
|
|
267
|
-
if ((uint32_t)flags >
|
|
293
|
+
if ((uint32_t)flags > 7 || (flags & 5) == 4) return -22;
|
|
294
|
+
#if !defined(__linux__)
|
|
295
|
+
if (flags & 4) return -95;
|
|
296
|
+
#endif
|
|
268
297
|
return nish_net_bound(host, port, SOCK_STREAM, backlog, flags);
|
|
269
298
|
}
|
|
270
299
|
|
package/runtime/runtime-os.c
CHANGED
|
@@ -26,7 +26,8 @@
|
|
|
26
26
|
* from drifting from the contract it is published under.
|
|
27
27
|
*/
|
|
28
28
|
#if defined(__linux__) && !defined(_GNU_SOURCE)
|
|
29
|
-
#define _GNU_SOURCE /* `sched_getaffinity` and `CPU_COUNT`, for
|
|
29
|
+
#define _GNU_SOURCE /* `sched_getaffinity`, `sched_setaffinity` and `CPU_COUNT`, for
|
|
30
|
+
`nish_cpu_online` and `nish_set_cpu_affinity` */
|
|
30
31
|
#endif
|
|
31
32
|
#define _POSIX_C_SOURCE 200809L
|
|
32
33
|
/* `realpath` is XSI rather than POSIX base, so _POSIX_C_SOURCE alone does not
|
|
@@ -860,6 +861,37 @@ int32_t nish_set_idle_priority(void) {
|
|
|
860
861
|
#endif
|
|
861
862
|
}
|
|
862
863
|
|
|
864
|
+
/* `setCpuAffinity(cpu)`: this process pinned to CPU `cpu` alone, 0 or a
|
|
865
|
+
negative errno. Narrow-only: `cpu` must be in the mask the process already
|
|
866
|
+
has (`sched_getaffinity`), so a pin never escapes an operator's `taskset` or
|
|
867
|
+
cpuset; a CPU outside it, a negative one, or one past `CPU_SETSIZE` is
|
|
868
|
+
-EINVAL with no `sched_setaffinity` call. Linux alone has the call; Darwin
|
|
869
|
+
answers -95 as `setIdlePriority` does, WASI -ENOSYS. Like the nice value,
|
|
870
|
+
the mask is the calling thread's and every thread started after it. */
|
|
871
|
+
int32_t nish_set_cpu_affinity(int32_t cpu) {
|
|
872
|
+
#if defined(__linux__)
|
|
873
|
+
/* The mask as words: the set to pin is the word holding `cpu` with only its
|
|
874
|
+
bit, the words below it zero, and the kernel zero-fills a shorter mask.
|
|
875
|
+
`sched_getaffinity` fails only on a bad pid or size, neither of which
|
|
876
|
+
this call can pass, so its failure shares the range check's answer. */
|
|
877
|
+
unsigned long m[CPU_SETSIZE / (8 * sizeof(unsigned long))];
|
|
878
|
+
uint32_t w = (uint32_t)cpu / (8 * sizeof m[0]);
|
|
879
|
+
unsigned long bit = 1UL << ((uint32_t)cpu % (8 * sizeof m[0]));
|
|
880
|
+
if ((uint32_t)cpu >= CPU_SETSIZE || sched_getaffinity(0, sizeof m, (cpu_set_t *)m) != 0 ||
|
|
881
|
+
!(m[w] & bit))
|
|
882
|
+
return -EINVAL;
|
|
883
|
+
for (uint32_t i = 0; i < w; i++) m[i] = 0;
|
|
884
|
+
m[w] = bit;
|
|
885
|
+
return sched_setaffinity(0, (w + 1) * sizeof m[0], (cpu_set_t *)m) == 0 ? 0 : -errno;
|
|
886
|
+
#elif defined(__wasi__)
|
|
887
|
+
(void)cpu;
|
|
888
|
+
return -ENOSYS;
|
|
889
|
+
#else
|
|
890
|
+
(void)cpu;
|
|
891
|
+
return -95;
|
|
892
|
+
#endif
|
|
893
|
+
}
|
|
894
|
+
|
|
863
895
|
/* `readdirSync(path)`: the listing a driver needs to discover its own inputs.
|
|
864
896
|
|
|
865
897
|
**Sorted by bytes**, which `readdir(3)` and Node are not, for two reasons
|
package/runtime/runtime-region.c
CHANGED
|
@@ -173,14 +173,26 @@ uint64_t nish_region_new(void) {
|
|
|
173
173
|
s = &nish_region_pages[index / NISH_REGION_PAGE][index % NISH_REGION_PAGE];
|
|
174
174
|
nish_region_free_list = s->outer;
|
|
175
175
|
} else {
|
|
176
|
-
|
|
177
|
-
|
|
176
|
+
/* Each panic lets go of the lock first: an atexit handler, or another
|
|
177
|
+
thread while the exit is on its way, may still make or free a region
|
|
178
|
+
(docs/security/runtime.md, RT-46). */
|
|
179
|
+
if (nish_region_used == NISH_REGION_SLOTS) {
|
|
180
|
+
nish_region_unlock();
|
|
181
|
+
nish_region_die(NISH_REGION_FULL);
|
|
182
|
+
}
|
|
183
|
+
/* The slot is counted once its page is there: a failed page leaves the
|
|
184
|
+
count as it was (RT-46). */
|
|
185
|
+
index = nish_region_used;
|
|
178
186
|
nish_region_slot **page = &nish_region_pages[index / NISH_REGION_PAGE];
|
|
179
187
|
if (!*page) {
|
|
180
188
|
nish_region_slot *p = calloc(NISH_REGION_PAGE, sizeof *p);
|
|
181
|
-
if (!p)
|
|
189
|
+
if (!p) {
|
|
190
|
+
nish_region_unlock();
|
|
191
|
+
nish_region_die(NISH_REGION_OOM);
|
|
192
|
+
}
|
|
182
193
|
__atomic_store_n(page, p, __ATOMIC_RELEASE);
|
|
183
194
|
}
|
|
195
|
+
nish_region_used = index + 1;
|
|
184
196
|
s = &(*page)[index % NISH_REGION_PAGE];
|
|
185
197
|
__atomic_store_n(&s->gen, 1, __ATOMIC_RELAXED);
|
|
186
198
|
}
|
package/runtime/runtime.c
CHANGED
|
@@ -180,8 +180,15 @@ static NISH_COLD void nish_oom(void) { nish_die("nish: out of memory\n"); }
|
|
|
180
180
|
this file still links alone. */
|
|
181
181
|
NISH_TLS uint64_t nish_region_chunk;
|
|
182
182
|
|
|
183
|
+
/* The most a region's next chunk grows to: past it the chunks stop doubling,
|
|
184
|
+
so a region that keeps allocating never asks malloc for one block of
|
|
185
|
+
gigabytes when smaller ones would do (docs/security/runtime.md, RT-47). A
|
|
186
|
+
single allocation larger than this still gets a chunk of its own size. */
|
|
187
|
+
#define NISH_REGION_CHUNK_MAX ((uint64_t)1 << 20)
|
|
188
|
+
|
|
183
189
|
/* Slow path (size 8-byte rounded): push a chunk (>= 64 KB, or a region's
|
|
184
|
-
next size), bump from it. A region's chunks double from its first
|
|
190
|
+
next size), bump from it. A region's chunks double from its first, up to
|
|
191
|
+
NISH_REGION_CHUNK_MAX. */
|
|
185
192
|
void *nish_arena_grow(uint64_t size) {
|
|
186
193
|
#ifdef NISH_ASAN
|
|
187
194
|
/* An exact chunk leaves off == cap, so the inline bump never fits again and
|
|
@@ -191,7 +198,7 @@ void *nish_arena_grow(uint64_t size) {
|
|
|
191
198
|
uint64_t next = nish_region_chunk;
|
|
192
199
|
uint64_t floor = next ? next : 65536;
|
|
193
200
|
uint64_t cap = size > floor ? size : floor;
|
|
194
|
-
if (next) nish_region_chunk = cap * 2;
|
|
201
|
+
if (next) nish_region_chunk = cap < NISH_REGION_CHUNK_MAX / 2 ? cap * 2 : NISH_REGION_CHUNK_MAX;
|
|
195
202
|
#endif
|
|
196
203
|
/* A 64-bit request cannot always be asked for: wasm32's `size_t` is 32 bits,
|
|
197
204
|
and malloc would silently take the low half of a chunk nobody can address. */
|
package/std/README.md
CHANGED
|
@@ -41,6 +41,7 @@ whatever program imports it, and subject to the same rules as `examples/` or
|
|
|
41
41
|
| [`term-rate.ts`](./term-rate.ts) | how fast the link to the terminal is, and the frame rate and detail a program keeps to under it. A `RateController` is told each frame sent (`frameSent(bytes, costNs, nowMs)`) and each acknowledgement — the answer to the DSR 5 query sent after a frame — (`ackReceived(nowMs)`), keeps running averages of the round trip, the bytes a second delivered and the cost of a frame, and answers `mayDraw(nowMs)` and a `RateDecision`: the frame interval, a resolution divisor, a backend cap and a palette, stepped down a tier when frames back up and up again when the link has room; `stats()` is the `TermStats` a program can show. Pure: no capability and no clock (`tests/link/std_term_rate`, `std_term_rate_f64`; [guide](../docs/guide/using-nish/terminal.md#how-the-frame-rate-adapts)) |
|
|
42
42
|
| [`term-pixels.ts`](./term-pixels.ts) | a `Framebuffer` of `0xRRGGBB` pixels and the encoders that draw it with the backend the terminal has: `kittyFrame` (kitty graphics, one image id retransmitted each frame), `sixelFrame` (a fixed palette, optionally dithered), `drawHalfBlocks` onto a `Canvas` and `drawAscii` onto a `Screen`, each into the `PixelRect` `fitFramebuffer` letterboxes in the cells; `scaleFramebuffer` shrinks a frame by the controller's divisor. `PixelBytes` keeps the encoders' output and working arrays, so frame after frame allocates nothing. Pure (`tests/link/std_term_pixels`, `std_term_pixels_f64`; [guide](../docs/guide/using-nish/standard-library.md#nishterm-pixels)) |
|
|
43
43
|
| [`term-io.ts`](./term-io.ts) | the term module that does the I/O. `runTerminal(options, state, update, render)` is a whole program loop: it opens the terminal, steps `update` at a fixed rate, calls `render` with a `Frame` (the `Screen`, the `Framebuffer`, the step `alpha`, `TermCaps` and `TermStats`) whenever the controller lets a frame go, and closes when `update` answers false or the user presses Ctrl-C. `openTerminal(options)` answers a `Terminal` for a program's own loop (`readEvents`, `present`, `stats`, `close`). Opening refuses a pipe, enters raw mode and the alternate screen, probes the terminal and turns on the kitty keyboard, mouse, paste and focus reports the options ask for, with the runtime's restore string set before each so a panic or signal leaves the terminal as it was; `present` sends the frame in the best backend the controller allows as one write and acknowledges it with DSR 5. Reaches `tty` (`tests/term-io/relay.mjs` drives it in a pseudo-terminal from `tests/run.js`'s `term-io` section; `tests/link/std_term_io_deny`; [guide](../docs/guide/using-nish/terminal.md)) |
|
|
44
|
+
| [`region.ts`](./region.ts) | The source behind the builtin module `nish:region`, and imported only by that name: `Region<T>`, `region`, `regionWith`, `enter`, `enterWith` and `free`. A `Region<T>` is a handle, a 64-bit integer, to memory the program frees when it says so; `region(f)` allocates what `f` builds into a new region and keeps its answer as the root, `enter(r, f)` runs `f(root)` there again and answers its scalar, the `*With` forms pass one argument through, and `free(r)` gives all of it back. A handle used after `free`, entered twice or freed while entered stops the program. The bodies are the meaning; the allocator swap is the emitter's (`src/emit-region.ts`) and what may cross a region's edge is the checker's (`src/region-check.ts`), keyed on this file ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#regions-nishregion); [guide](../docs/guide/language/regions.md)) |
|
|
44
45
|
| [`secret.ts`](./secret.ts) | The source behind the builtin module `nish:secret`, and imported only by that name: `Secret<T>`, `secret`, `expose`, `exposeWith` and `wipe`. The bodies are the meaning; what makes a `Secret` one is the checker's, keyed on this file ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#secrets-nishsecret)). `wipe`'s body is written by the emitter, one volatile `llvm.memset`, and like `collections.ts` the module writes no `.ll` of its own |
|
|
45
46
|
| [`threads.ts`](./threads.ts) | `parallelMapInto(src, dst, f)` and `parallelReduce(src, f, identity)`: a function over every element of an array, on as many threads as the length is worth. Its bodies are the sequential meaning, which is what runs under Node; the compiler recognises the two templates by module and name, lowers the one loop in each onto `nish_parallel_range`, holds the function to the rules that make that safe, and compiles an importing program with `--threads` ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#data-parallelism-nishthreads)). `tests/link/par_*` are its programs. `scope()`, bound by `using s = scope()`, runs unlike tasks, each on a thread of its own, and joins them at every exit of the block; `s.spawn(entry, arg, dst, at)` stores a task's answer, and `s.spawn(entry, arg)` and `s.spawn(() => ...)`, whose arrow reads the function's `const` locals, run one that answers `void` (`spawnWith` and `spawnThunk` are the methods the compiler reads them as, and under Node `spawn` itself tells them apart) ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#scoped-tasks-using-s--scope)). `scopeWithStack(bytes)` is a scope whose tasks each run on a new thread with at least `bytes` of stack, for a task that recurses deeper than a thread's default allows; under Node and without threads the size is checked and otherwise ignored (`tests/link/thread_stack_deep`). `Mutex<T>` is a lock that owns its data: the value is reachable only through the guard `using g = m.lock()` binds, which releases it at every exit of the block, so a scope's tasks can share and update one value without a data race or a deadlock ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#a-lock-that-owns-its-data-mutext); `tests/link/thread_mutex_*`, and `examples/histogram.ts` measured). `Channel<T>` carries scalars to one receiver, a task of the scope or the parent after the join, from the scope's sender tasks and from the parent before the block that spawns them, and closes when every sending task has returned ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#channels-channelt); [`docs/wp29-thread-surface.md`](../docs/wp29-thread-surface.md#44-stage-p3--channels-of-scalars) §4.4). Every body in the module is its meaning under Node, where a task runs at its `spawn` |
|
|
46
47
|
| [`sync/shm.ts`](./sync/shm.ts) | `createRegion(name, bytes)`, `openRegion(name)` and `unlinkRegion(name)`: a named region of memory that several processes of one user map at once, through `runtime/runtime-shm.c`. A `SharedRegion` is read and written through checked slots, `loadI32` to `compareExchangeI64` at an ordering the call names (`RELAXED` to `SEQ_CST`), and byte windows (`read`, `write`); an offset outside the region or not a multiple of the slot's width stops the program. `waitI32` and `wake` sleep and wake on a slot across processes (Linux futex; `SHM_UNSUPPORTED` elsewhere). Native only, and it reaches `ffi` ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#shared-memory-between-processes-nishsyncshm-and-nishsyncring); [`docs/wp47-shared-memory.md`](../docs/wp47-shared-memory.md); [`docs/security/shm.md`](../docs/security/shm.md)) |
|
|
@@ -231,13 +232,13 @@ reads one, and it still takes its randomness as an argument.
|
|
|
231
232
|
| [`net/dns.ts`](./net/dns.ts) | `nish/net/dns-resolver` on `nish:net` sockets, on the caller's loop: `DnsClient(resolver, loop, tokenBase)` opens a new UDP socket on a kernel-chosen port for each try, or a TCP connection for a TCP one, under a token of its range (`owns`), draws each query ID from `crypto.getRandomValues`, and never waits: the program's `pollWait` hands it `ready(token, events, now)`, calls `tick(now)` and waits no longer than `timeoutMs(now, own)`. `resolveHost(name, timeoutMs, port, out, ttls)` reads `/etc/resolv.conf` and `/etc/hosts` and blocks on a loop of its own, for a CLI or start-up; `dnsLookupWith` does the same with a configuration given. | A Nish DNS server on loopback: UDP, TCP, TC, CNAME, the search list, a dead server, `attempts`, the hosts file and spoofed replies (`tests/link/std_dns`); the capability report (`std_dns_caps`); a flat arena over 2,000 lookups (`std_dns_soak`) |
|
|
232
233
|
| [`net/hpack.ts`](./net/hpack.ts) | HPACK, HTTP/2's header compression: `HpackEncoder` and `HpackDecoder`, one per direction of a connection, each with its `HpackTable`; and beneath them the §5.1 integer (`hpackEncodeInteger` / `hpackDecodeInteger`), the §5.2 string (`hpackEncodeString` / `hpackDecodeString`) and Appendix B's Huffman code (`HpackHuffman`, `hpackHuffmanEncode`, `hpackHuffmanDecode`, `hpackHuffmanLength`). Names and values are `u8[]`. The encoder takes the representation and the Huffman choice per field, and sends a never-indexed field as a literal even when a table holds it. The decoder answers a negative `HPACK_ERR_*` code, and stays spent, for a truncated block, an integer past 2^31 - 1, an index of zero or out of range, bad Huffman padding, EOS, a size update above the SETTINGS limit, after a field or missing when a lowered limit requires one, and for a block that would copy more than `HPACK_COPY_FACTOR` times the larger of the table's size and the block's length into and out of the table (`HPACK_ERR_COPY_BUDGET`; `tableCopied` counts those octets); a header list past the caller's limit is decoded to the end, its excess dropped, and answered with `HPACK_LIST_TOO_LARGE`, which is not fatal. A window outside its buffer, a negative size or limit, and an integer prefix outside 1 to 8 bits are the program's mistakes and panic | RFC 7541 Appendix C (C.1–C.6, with the dynamic table after every step) and every code of Appendix B |
|
|
233
234
|
| [`net/qpack.ts`](./net/qpack.ts) | QPACK, HTTP/3's field compression, with a dynamic table of capacity 0: `QpackEncoder` and `QpackDecoder`, one per connection. The encoder writes a field section that starts `00 00` (Required Insert Count 0, Base 0) with `beginSection`, then one field line per `encodeField(out, name, off, len, value, off, len, neverIndexed, huffman)`: an Indexed Field Line when the 99-entry static table holds the field, else a literal with the name by static index when it can, each string Huffman-coded when that is shorter and `huffman` is set, and a never-indexed field always a literal with the N bit. The decoder answers a section's fields as windows into its own `bytes` (`nameStart`, `nameLength`, `valueStart`, `valueLength`, `neverIndexed`, `count`), its arrays reused from section to section. It answers QPACK_DECOMPRESSION_FAILED, and stays spent, for a truncated section, an integer past 2^62 − 1, a non-zero Required Insert Count, a negative Base, any dynamic-table reference, a static index past 98 and bad Huffman; a section past `maxFieldSectionSize` answers `QPACK_SECTION_TOO_LARGE`, which is not fatal. `receiveEncoderStream` accepts Set Dynamic Table Capacity 0 and answers QPACK_ENCODER_STREAM_ERROR for any other capacity, Insert or Duplicate; the encoder's `receiveDecoderStream` accepts Stream Cancellations, split anywhere, and answers QPACK_DECODER_STREAM_ERROR for a Section Acknowledgment or Insert Count Increment; `reason` names the rule. `qpackPushStreamCancellation` is the one instruction it may send. The Huffman code is `nish/net/hpack`'s. A window outside its buffer, a negative limit and a stream ID outside 0 to 2^62 − 1 panic | RFC 9204 Appendix B.1, every Appendix A entry by its index, B.2 to B.5's instructions refused, and a request matched byte for byte with pylsqpack's encoding |
|
|
234
|
-
| [`net/http1.ts`](./net/http1.ts) | `Http1Parser`, an incremental HTTP/1.1 request parser: `feed(buf, off, len)` any slice, then `next()` answers `HTTP1_NEED_MORE`, `HTTP1_HEAD` (`method`, `target`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength` or `chunked`, `keepAlive`, `upgrade`), `HTTP1_BODY` (a window onto its buffer, de-chunked), `HTTP1_END`, or one of three final events: `HTTP1_UPGRADE` (`upgradeBytes()` and `declineUpgrade()`), `HTTP1_CLOSED` and `HTTP1_ERROR` with the `status` to answer. It refuses with 400 every request-smuggling shape — `Content-Length` with `Transfer-Encoding`, a repeated or non-digit `Content-Length`, an obs-fold, a bare LF, whitespace before a colon, a malformed chunk size or terminator — and with 413, 414, 431, 501 and 505 what those mean. The head is also kept as spans in a buffer reused from request to request (`head`, `spans`, `methodIs`, `targetIs`, `headerIndex`, `headerIs`, `upgradeIs`), so a parser with `keepText` off allocates nothing per request; `maxChunk` caps each `HTTP1_BODY`, and `restart()` readies it for the next connection. The writer is `http1ResponseHead(status, reason, names, values, bodyLength)`, which adds the framing field itself and refuses (`null`) a CR, LF or other control character anywhere a caller's string lands, and `http1Chunk` and `http1LastChunk` for a chunked body; `http1WriteResponseHead` and `http1WriteChunk` write the same bytes into a caller's buffer at an offset, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED`, and the first takes a date's bytes to send as a `Date` field on a final response, unless the caller's fields hold one | RFC 9112 §2–§7 and §9.3, RFC 9110 §5, §6.2, §7.2 and §7.8; a corpus fed at every split point (`tests/link/net_http1`) |
|
|
235
|
-
| [`net/http1-client.ts`](./net/http1-client.ts) | The client side of HTTP/1.1, sans-IO. The writer is `http1RequestHead(method, target, names, values, bodyLength)`, `Host` first and the framing field added for `bodyLength` (`Content-Length`, `HTTP1_CHUNKED` or `HTTP1_NO_BODY`), and `http1WriteRequestHead` into a caller's buffer, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED` with nothing written; it refuses a method that is not a token, a target that is not a form the method may send, a CR, LF or control byte in a value, not exactly one `Host`, the caller's own framing fields and a `Connection: upgrade`; `http1UpgradeRequestHead(method, target, names, values, upgrade)` writes `Upgrade` and `Connection: Upgrade` itself, with no body. `Http1ResponseParser(maxStatusLine, maxHeaderBytes, maxHeaders, maxBody, maxTrailerBytes)` is told what was asked (`expect(head, connect)`, then `expectUpgrade()` for a request that asked to switch protocols), fed any slice, and `next()` answers `HTTP1_INFORMATIONAL`, `HTTP1_HEAD` (`status`, `reason`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength`), `HTTP1_BODY` (a window, de-chunked), `HTTP1_END` with `keepAlive()`, or the final `HTTP1_UPGRADE` (a 2xx to CONNECT or a 101 asked for, `tunnelBytes()`), `HTTP1_CLOSED` and `HTTP1_ERROR`; `finish()` reports the close that ends a body read until close, and anywhere else is `HTTP1_E_TRUNCATED`. Every smuggling shape is refused with its own `HTTP1_E_*` code, as is a 101 not asked for, more than `HTTP1_MAX_INFORMATIONAL` interim responses and a chunk-size line over `HTTP1_MAX_CHUNK_LINE`. Record: [`docs/security/http1-client.md`](../docs/security/http1-client.md) | RFC 9112 §4–§7 and §9.3, RFC 9110 §9.3.6 and §15.2; a corpus fed whole, cut at every byte and a byte at a time, every refusal and limit at its edge, and the server parser reading what the writer writes (`tests/link/net_http1_client`, `_f64`, `_feed_window`) |
|
|
236
|
-
| [`net/
|
|
235
|
+
| [`net/http1-codec.ts`](./net/http1-codec.ts) | `Http1Parser`, an incremental HTTP/1.1 request parser: `feed(buf, off, len)` any slice, then `next()` answers `HTTP1_NEED_MORE`, `HTTP1_HEAD` (`method`, `target`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength` or `chunked`, `keepAlive`, `upgrade`), `HTTP1_BODY` (a window onto its buffer, de-chunked), `HTTP1_END`, or one of three final events: `HTTP1_UPGRADE` (`upgradeBytes()` and `declineUpgrade()`), `HTTP1_CLOSED` and `HTTP1_ERROR` with the `status` to answer. It refuses with 400 every request-smuggling shape — `Content-Length` with `Transfer-Encoding`, a repeated or non-digit `Content-Length`, an obs-fold, a bare LF, whitespace before a colon, a malformed chunk size or terminator — and with 413, 414, 431, 501 and 505 what those mean. The head is also kept as spans in a buffer reused from request to request (`head`, `spans`, `methodIs`, `targetIs`, `headerIndex`, `headerIs`, `upgradeIs`), so a parser with `keepText` off allocates nothing per request; `maxChunk` caps each `HTTP1_BODY`, and `restart()` readies it for the next connection. The writer is `http1ResponseHead(status, reason, names, values, bodyLength)`, which adds the framing field itself and refuses (`null`) a CR, LF or other control character anywhere a caller's string lands, and `http1Chunk` and `http1LastChunk` for a chunked body; `http1WriteResponseHead` and `http1WriteChunk` write the same bytes into a caller's buffer at an offset, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED`, and the first takes a date's bytes to send as a `Date` field on a final response, unless the caller's fields hold one | RFC 9112 §2–§7 and §9.3, RFC 9110 §5, §6.2, §7.2 and §7.8; a corpus fed at every split point (`tests/link/net_http1`) |
|
|
236
|
+
| [`net/http1-client-codec.ts`](./net/http1-client-codec.ts) | The client side of HTTP/1.1, sans-IO. The writer is `http1RequestHead(method, target, names, values, bodyLength)`, `Host` first and the framing field added for `bodyLength` (`Content-Length`, `HTTP1_CHUNKED` or `HTTP1_NO_BODY`), and `http1WriteRequestHead` into a caller's buffer, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED` with nothing written; it refuses a method that is not a token, a target that is not a form the method may send, a CR, LF or control byte in a value, not exactly one `Host`, the caller's own framing fields and a `Connection: upgrade`; `http1UpgradeRequestHead(method, target, names, values, upgrade)` writes `Upgrade` and `Connection: Upgrade` itself, with no body. `Http1ResponseParser(maxStatusLine, maxHeaderBytes, maxHeaders, maxBody, maxTrailerBytes)` is told what was asked (`expect(head, connect)`, then `expectUpgrade()` for a request that asked to switch protocols), fed any slice, and `next()` answers `HTTP1_INFORMATIONAL`, `HTTP1_HEAD` (`status`, `reason`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength`), `HTTP1_BODY` (a window, de-chunked), `HTTP1_END` with `keepAlive()`, or the final `HTTP1_UPGRADE` (a 2xx to CONNECT or a 101 asked for, `tunnelBytes()`), `HTTP1_CLOSED` and `HTTP1_ERROR`; `finish()` reports the close that ends a body read until close, and anywhere else is `HTTP1_E_TRUNCATED`. Every smuggling shape is refused with its own `HTTP1_E_*` code, as is a 101 not asked for, more than `HTTP1_MAX_INFORMATIONAL` interim responses and a chunk-size line over `HTTP1_MAX_CHUNK_LINE`. Record: [`docs/security/http1-client-codec.md`](../docs/security/http1-client-codec.md) | RFC 9112 §4–§7 and §9.3, RFC 9110 §9.3.6 and §15.2; a corpus fed whole, cut at every byte and a byte at a time, every refusal and limit at its edge, and the server parser reading what the writer writes (`tests/link/net_http1_client`, `_f64`, `_feed_window`) |
|
|
237
|
+
| [`net/fetch.ts`](./net/fetch.ts) | A blocking HTTP/1.1 client for command-line programs: `fetchSync(request, options)` looks the host up, connects, sends an `HttpRequest` (`method`, `url`, `header(name, value)`, `body`) and answers an `HttpResponse` (`status`, `reason`, `url`, `header(name)`) once its head is in, its body read through `body.read` or `body.readAll(limit)`; null on a failure. `FetchSyncClient(options)` does the same and keeps one connection alive between fetches, sending an idempotent request once more when a kept connection turns out to be gone, and says why a fetch failed in `error` (an `HTTP_E_*` code, `httpErrorName`) and `errorText`. `HttpOptions` bounds every wait (`connectTimeoutMs`, `readTimeoutMs`, `totalTimeoutMs`) and every body (`maxBodyBytes`), follows redirects only when asked (`followRedirects`, `maxRedirects`, `crossOriginRedirects`, never https to http, credentials dropped across origins), and reads `http_proxy`, `https_proxy` and `no_proxy` when `useEnvProxy` is set (`HTTP_PROXY` ignored under CGI; https through `CONNECT`). Every byte goes through `nish/net/http1-client-codec`. Native only. Record: [`docs/security/fetch.md`](../docs/security/fetch.md) | RFC 9110 §9.2.2 and §15.4, RFC 9112; a loopback server and proxy in the test itself: keep-alive, retries, redirects, proxies, timeouts and body caps (`tests/link/net_http_fetch`) |
|
|
237
238
|
| [`net/outbound.ts`](./net/outbound.ts) | The policy for fetching URLs your program did not choose, set as `HttpOptions.outbound`: `OutboundPolicy` reaches only public addresses (no loopback, private, link-local, CGNAT, metadata or other special-purpose range, IPv4-mapped, NAT64 and 6to4 forms judged as the IPv4 address they carry), checked on the resolved address the client then connects to, so DNS rebinding cannot move it; every redirect hop is checked again, the proxy is not used, and redirects, total time and body size are capped. `permit(cidr)` lets a trusted prefix through, `allowHost(name)` limits the names a URL or hop may carry (`HTTP_E_BLOCKED` before the lookup). Native only. Record: [`docs/security/outbound.md`](../docs/security/outbound.md) | RFC 6890, RFC 4291, RFC 6052, RFC 3056 and the IANA special-purpose address registries; every range at its edges and a loopback server fetched with and without a policy (`tests/link/net_outbound`) |
|
|
238
239
|
| [`net/tls/ticket.ts`](./net/tls/ticket.ts) | Session tickets a server seals for itself (RFC 8446 §4.6.1): `tlsTicketSeal` and `tlsTicketOpen` seal a `TlsTicketState` (suite, issue time, lifetime, chain start, server name, resumption PSK) under ChaCha20-Poly1305 with a borrowed `Secret<u8[]>` key, as key id ‖ nonce ‖ ciphertext ‖ tag, and wipe the plaintext on every path. `TlsTicketKeyInfo`, `tlsTicketKeyDraw` and `tlsTicketKeyDue` are a key's public record, its draw and when to replace it: a key seals for 12 h of awake time and 2^30 tickets, and opens for 14 h. `TlsTicketStrikes` is the bounded strike register (§8.1) that makes a ticket single-use, failing closed when full | `tests/link/net_tls_ticket`: every bound at its instant and 1 ns past it, a flipped bit in each region, three key rotations, replay and a full register |
|
|
239
240
|
| [`net/websocket.ts`](./net/websocket.ts) | `WsDecoder(expectMasked, maxMessage)`, the same `feed` and `next()` over WebSocket frames: `WS_MESSAGE` (fragments joined, text checked as UTF-8 fragment by fragment), `WS_PING`, `WS_PONG`, and the final `WS_CLOSE` and `WS_ERROR`, whose `closeCode` is the 1002, 1007 or 1009 to close with. `websocketFrame` encodes a frame with the mask key the caller supplies (`websocketWriteFrame` into a caller's buffer, `websocketFrameSize` its length), `reset()` readies a decoder for the next connection, `websocketClosePayload` a close frame's code and reason, and `websocketIsUtf8` checks a window. The handshake is `websocketRequestKey(parser)` over an `Http1Parser`'s head, `websocketAcceptKey`, `websocketKeyIsValid`, `websocketOfferedProtocols` and `websocketUpgradeResponse`, the 101. The client's side is `websocketClientKey(nonce)` (a 16-byte nonce the caller draws; the module draws no randomness), `websocketUpgradeRequest(host, target, key, protocols)` and `websocketCheckResponse(parser, key, offered)` over an `Http1ResponseParser` told `expectUpgrade()`, answering a `WsHandshake` (`error`, a `WS_HS_*` code, and `protocol`) | RFC 6455 §1.3's accept key, the §5.7 example frames byte for byte, §5.5's control-frame rules, §7.4's close codes |
|
|
240
|
-
| [`net/http1-server.ts`](./net/http1-server.ts) | The server side of HTTP/1.1 (WP34 H1): `Http1Connection`, sans-IO, drives an `Http1Parser` — `feed`, then `next()` answers `H1_REQUEST` (the parser's head), `H1_BODY` (at most `chunkSize` bytes, either framing), `H1_END`, `H1_WRITE` (a held-back write may go again), `H1_WS_MESSAGE`, `H1_WS_CLOSE` or the final `H1_ERROR`; the program answers with `respond(status, names, values, bodyLength)`, `write` and `end`, or `acceptWebSocket(protocol)` and then `sendFrame` and `closeWebSocket`. Keep-alive and pipelining one exchange at a time, a response with a length or chunked (unframed to an HTTP/1.0 client), `Connection` and the framing decided by the server, every parser refusal answered with its status and a close, pings and closes answered on a WebSocket. Two slot pools for a program that owns its loop: `Http1Server` on plain TCP and `Http1TlsServer` on `nish/net/tls-tcp` with ALPN `http/1.1` or none (`accept`, `readable` / `writable`, `next(slot)`, `flush(slot)`, `close`, `expire(now)` for the idle timeout and the deadline for a whole head, `headTimeout`, against a client that trickles its head). Both build a slot's connection on the first accept into it rather than at start-up. `Http1Server` reads a socket until a short read rather than until it would block, takes a free slot off a stack of released slots or the next never-used one rather than scanning for one, and answers `rearm(slot, wants)`, whether the interest changed and needs a `pollModify` at all: a keep-alive request is one `recv` and one `send`. The slots borrow their output from one `Http1OutputPool` while they write and give it back once sent, so a pool's memory follows the connections the socket is holding back, not its size: an idle slot holds no output buffer. Every final response carries a `Date` from `HttpDate`, which reads `Date.now()` and formats at most once a second, shared by a pool's slots; `pin(unixMs)` fixes it for a test, and a program's own `date` field replaces it. Every cap is an `Http1Config` number fixed at start-up, and a warmed slot allocates nothing per request (`docs/security/http1.md`). Native only, as `nish:net` is | RFC 9112 §6, §7, §9; RFC 9110 §9.3.2, §15; RFC 6455 §4.2, §5.5. Nish clients over loopback against both carriers, requests cut at every byte, 2 MiB bodies each way, pipelining, a WebSocket echo, the arena flat over two hundred requests, the date formatted once a second, and every refusal (`tests/link/net_http1_server`); `curl` against its `serve` mode |
|
|
241
|
+
| [`net/http1-server.ts`](./net/http1-server.ts) | The server side of HTTP/1.1 (WP34 H1): `Http1Connection`, sans-IO, drives an `Http1Parser` — `feed`, then `next()` answers `H1_REQUEST` (the parser's head), `H1_BODY` (at most `chunkSize` bytes, either framing), `H1_END`, `H1_WRITE` (a held-back write may go again), `H1_WS_MESSAGE`, `H1_WS_CLOSE` or the final `H1_ERROR`; the program answers with `respond(status, names, values, bodyLength)`, `write` and `end`, or `acceptWebSocket(protocol)` and then `sendFrame` and `closeWebSocket`. Keep-alive and pipelining one exchange at a time, a response with a length or chunked (unframed to an HTTP/1.0 client), `Connection` and the framing decided by the server, every parser refusal answered with its status and a close, pings and closes answered on a WebSocket. Two slot pools for a program that owns its loop: `Http1Server` on plain TCP and `Http1TlsServer` on `nish/net/tls-tcp` with ALPN `http/1.1` or none (`accept`, `readable` / `writable`, `next(slot)`, `flush(slot)`, `close`, `expire(now)` for the idle timeout and the deadline for a whole head, `headTimeout`, against a client that trickles its head). Both build a slot's connection on the first accept into it rather than at start-up. `Http1Server` reads a socket until a short read rather than until it would block, takes a free slot off a stack of released slots or the next never-used one rather than scanning for one, and answers `rearm(slot, wants)`, whether the interest changed and needs a `pollModify` at all: a keep-alive request is one `recv` and one `send`. The slots borrow their output from one `Http1OutputPool` while they write and give it back once sent, so a pool's memory follows the connections the socket is holding back, not its size: an idle slot holds no output buffer. Every final response carries a `Date` from `HttpDate`, which reads `Date.now()` and formats at most once a second, shared by a pool's slots; `pin(unixMs)` fixes it for a test, and a program's own `date` field replaces it. Every cap is an `Http1Config` number fixed at start-up, and a warmed slot allocates nothing per request (`docs/security/http1-codec.md`). Native only, as `nish:net` is | RFC 9112 §6, §7, §9; RFC 9110 §9.3.2, §15; RFC 6455 §4.2, §5.5. Nish clients over loopback against both carriers, requests cut at every byte, 2 MiB bodies each way, pipelining, a WebSocket echo, the arena flat over two hundred requests, the date formatted once a second, and every refusal (`tests/link/net_http1_server`); `curl` against its `serve` mode |
|
|
241
242
|
| [`net/url.ts`](./net/url.ts) | The request target's path and query, read in place: `urlQueryStart` finds the `?`, `UrlPathSegments` splits a path on the raw `/` and `UrlQuery` yields a query's pairs as offsets, both reusable by `reset`; `percentDecode` and `percentDecodePathSegment` decode a window into the caller's array and answer the length or a `URL_DECODE_ERR_*` code, `percentDecodeErrorAt` answers the offset an error is at, and `urlQueryFind` finds a key. Decoding is strict by default (a malformed escape and `%00` are refused); `URL_DECODE_FORM`, `URL_DECODE_LENIENT` and `URL_DECODE_ALLOW_NUL` together decode valid UTF-8 as `URLSearchParams` does. A query yields at most `UrlQueryLimits.maxPairs` pairs (100 by default) and then answers `URL_QUERY_ERR_PAIRS`. The byte API allocates nothing; `urlDecode` and `urlQueryGet` answer strings and refuse what is not UTF-8. Whole URLs: `urlParse` splits an absolute URL into a `Url` (`scheme`, `hasAuthority`, `username`, `password`, `host`, `port`, `path`, `query`, `fragment`, each still percent-encoded, the scheme and host lower-cased), `urlResolve(base, ref)` resolves a reference against one, `urlSerialize` writes one back, and `urlDefaultPort` and `urlOrigin` answer the port and the origin a client connects to. Both refuse with null what two readers could take for different hosts: a byte outside printable ASCII, a backslash, a malformed `%`, a second `@`, a non-unreserved host byte, a numeric host that is not strict dotted-decimal IPv4, an IPv6 zone, a port past 65535. Records: [`docs/security/url.md`](../docs/security/url.md) | RFC 3986 §2.1 and §3.3, the WHATWG URL Standard's percent-decode and `application/x-www-form-urlencoded` parser; `decodeURIComponent` and `URLSearchParams` on 3,000 seeded inputs; RFC 3986 §3 and §5.2, every §5.4 example, 57 refusals and 4,000 seeded round trips in both number modes (`tests/link/net_url_parse`) |
|
|
242
243
|
| [`net/http-fields.ts`](./net/http-fields.ts) | The header model HTTP/2 and HTTP/3 share, version-neutral: `HttpFields` reads a decoded section with `readRequest(names, values, extendedConnect)`, `readResponse` or `readTrailers` into `method`, `scheme`, `authority`, `path`, `protocol`, `status`, the regular fields in order (`get(name)`) and `contentLength`, answering `HTTP_FIELDS_OK` or the first refusal: a name that is empty, not a token or not lowercase; a value with NUL, CR or LF or whitespace at an end; a connection-specific field (`te` but `trailers` included); a pseudo-header unknown, repeated, after a regular field, forbidden by the shape or missing; a pseudo-header value of the wrong syntax; a `host` that disagrees with `:authority`; a bad or disagreeing `content-length`. `:protocol` (RFC 8441) is read only where the caller enabled extended CONNECT. `httpFieldNameValid`, `httpFieldValueValid`, `httpFieldConnectionSpecific` and `httpFieldsCheckOutgoing` check one name, one value or a section a program sends. `httpDateWrite(out, at, seconds)` writes a second since 1970 as the IMF-fixdate a `Date` field holds (`HTTP_DATE_SIZE` bytes, to 9999), reading no clock, so it compiles for wasm32 | RFC 9110 §5, RFC 9113 §8.2–§8.5, RFC 9114 §4.2–§4.3, RFC 8441 §4; every refusal in `tests/link/net_http_fields`; RFC 9110 §5.6.7, every day from 1970 to 2399 (`tests/link/net_http_date`) |
|
|
243
244
|
| [`net/http2-frame.ts`](./net/http2-frame.ts) | HTTP/2's frames, sans-IO and allocation-free: `http2ReadHeader` and `http2ParseFrame` read any frame of RFC 9113 §6 into a reused `Http2Frame` (its content left in the buffer as a window), answering 0 or the connection error — FRAME_SIZE_ERROR past the reader's SETTINGS_MAX_FRAME_SIZE or for a frame of the wrong length, PROTOCOL_ERROR for a frame on the wrong stream, padding as long as its frame, or a WINDOW_UPDATE of zero on the connection — with a stream error (`streamError`) for a PRIORITY of the wrong length, a self-dependency or a WINDOW_UPDATE of zero on a stream. `http2SettingCount`, `http2SettingId`, `http2SettingValue` and `http2SettingError` read and check SETTINGS. The writers, one per type (`http2WriteData` … `http2WriteContinuation`, `http2WriteSettingsAck`), write into the caller's buffer at an offset and answer -1 when a frame does not fit | Every type against frames built by hand from §4.1 and §6, both directions, and every refusal (`tests/link/net_http2_frame`) |
|
|
@@ -251,7 +252,7 @@ reads one, and it still takes its randomness as an argument.
|
|
|
251
252
|
| [`net/router.ts`](./net/router.ts) | `Router`, a method and path router that knows no carrier and allocates nothing per match: `add(method, pattern)` answers the route's index (0, 1, 2 in the order added) or a refusal — `ROUTER_BAD_PATTERN`, `ROUTER_BAD_METHOD`, `ROUTER_TOO_LONG`, `ROUTER_AMBIGUOUS` (`*` not last), `ROUTER_DUPLICATE` (the same shape and method, whatever the parameter names) or `ROUTER_FULL`, against `RouterLimits`; `match(method, mStart, mEnd, target, tStart, tEnd, params)` answers the index, `ROUTER_NOT_FOUND` or `ROUTER_METHOD_NOT_ALLOWED`, writing each `:name` and a final `*` as a `[start, end)` pair into the caller's `params` (`ROUTER_TOO_MANY_PARAMS` when it is short). Literal before parameter before `*`, segment by segment, backtracking; a lookup enters each node at most once. The query is left unparsed and nothing is decoded: `routerDecode` decodes one parameter (`ROUTER_BAD_ESCAPE`, `ROUTER_NO_ROOM`) (`docs/security/router.md`) | Every refusal code, precedence and backtracking, 405 against 404, `%2F` kept in one segment, the query, `*` and authority-form targets, a target deeper than any route, the lookup bound, params past storage, and the arena flat over matches (`tests/link/net_router`, `net_router_f64`, five `net_router_negative_*`) |
|
|
252
253
|
| [`net/handler.ts`](./net/handler.ts) | `HttpHandler`, complete requests and their answers over the HTTP/1.1 carrier: it owns the readiness loop, routes each request through a `Router` at its head, gathers its body up to `HandlerLimits.http.maxBody` (1 MiB; past it the carrier answers 413), and hands it over whole. The program pulls — `wait(ms)`, `ready()`, `take()` — and switches on `req.route`; `HttpRequest` reads parameters, form-decoded query values and fields as ranges into its own buffers. Each request is answered once, now or in a later turn, by `send`, `sendFields`, `empty`, `json()` + `sendJson`, `problem` (RFC 9457), `refuse` (404, or 405 with `Allow`) or `start`/`write`/`end`; an answer whose connection closed first is `HANDLER_DROPPED` and reaches no later peer. `HEAD` takes a `GET` route. `watch`/`fired` add the program's own descriptors (`signalFd()`), and `drain` stops gracefully. Nothing is allocated per request once warm (`docs/security/handler.md`) | Routing, parameters and form-decoded queries, `HEAD`, 404 and 405 problems, chunked bodies in and out, the body cap at and one past it, pipelining order, late answers dropped across a reused slot, the arena and pool flat over two thousand requests, and the drain (`tests/link/net_handler`, and `net_http_app` over `examples/http-app`) |
|
|
253
254
|
| [`net/upstream.ts`](./net/upstream.ts) | `Upstream`, outbound HTTP/1.1 calls from a route handler that never block it: a pool of keep-alive connections to one `http:` origin on the `HttpHandler`'s own loop (watch ids `UpstreamLimits.firstId` and up). `get(req, path)` returns at once, or `UPSTREAM_E_FULL` with nothing held; each call comes back once through `ready()`/`take()` tagged with `r.req`, `ok` with its status and `body[0 .. length)`, or failed with the codec's `HTTP1_E_` refusal or an `UPSTREAM_E_` code (timeout, closed, connect, origin, lookup, blocked, request), and goes back with `release`. The origin is looked up once at start-up and pinned; `UpstreamLimits.outbound` checks it against an `OutboundPolicy`. A failed call's connection is closed. Each connection keeps one codec in spans mode, so a warm pool allocates nothing per call | A slow upstream while other clients and a second upstream are answered, the timeout, keep-alive and tagging, five codec refusals as failures, the policy refusing loopback, origin and request-target refusals, three thousand warm calls on one connection with `Arena.used()` flat (`tests/link/net_upstream`), and a second `release` panicking (`net_upstream_release`) |
|
|
254
|
-
| [`net/postgres.ts`](./net/postgres.ts) | `PgConnection`, a PostgreSQL client that reads no socket: the program fills `inputBuffer()` and sends `outputBuffer()`, and `next()` answers `PG_NEED_MORE`, `PG_READY`, `PG_DESCRIBED`, `PG_ROW`, `PG_COMPLETE`, `PG_ERROR`, `PG_NOTICE`, `PG_GROUP_DONE` or the final `PG_FAILED` (`failure`, one of the `PG_ERR_*` codes). Protocol 3.0: startup, `trust` and SCRAM-SHA-256 with the server's signature checked (cleartext and MD5 refused), the extended query protocol — `prepare`, `bindStart`, `paramInt` / `paramText` / `paramBytes` / `paramTextArray` / `paramIntArray` / `paramNull`, an array built element by element into the Bind (`arrayStart`, `arrayElement`, `arrayInt`, `arrayNull`, `arrayEnd`), `bindEnd`, `bindStartBinary` for a Bind whose result columns come in binary (read in place by `binaryInt`, `binaryBool`, `binaryFloat` and `PgBinary`'s `timestamp` and `iso`, and a one-dimensional `int2[]`, `int4[]` or `int8[]` by `PgBinaryArray`), `bindCancel` to drop a Bind not yet ended, `closeStatement`, `sync` — with up to `maxInFlight` groups pipelined. `PgBatcher` queues keyed reads of one statement shape (`addInt`, `addText`) and `take(conn)` binds up to `maxRequests` of them, each key once, as one execution over `unnest($1) with ordinality`; `row`, `end` and `fail` route its rows by their server-supplied ordinal, checked first (read with `binaryInt` when `PgBatchConfig.binary` is set, so the batch's rows come in binary too), to `next(run)`'s per-request `PG_BATCH_ROW` and `PG_BATCH_DONE` / `PG_BATCH_MISSING` / `PG_BATCH_FAILED`; `take` answers a fresh handle each time, so a server's late or repeated event on a spent one is counted in `stray`, never routed or panicked on. It reads no clock and allocates nothing per request or row. `PgBatchClock` says when to take, from times the program passes in: once max(`minCount`, the moving average of batch sizes × `factor`) wait, the oldest has waited `maxWaitUs`, or, with the idle bypass, nothing is in flight and the load is light (its defaults are the fixed window); `PgBatcher.takeDue` takes only then, and `PgBatchTurns` lets one free connection collect while the others execute. A row's columns are windows onto the receive buffer (`colStart`, `colLength`), read in place by `int`, `bool`, `isNull` and `PgText`'s decoders (`timestamptz`, `PgArrayWalker` for `text[]`); no per-row or per-column allocation. The password is a `Secret<u8[]>`, wiped once the proof is made. Caps on message size, columns, groups in flight and SCRAM iterations (`docs/security/postgres.md`) | PostgreSQL 18's frontend/backend protocol chapter, RFC 5802 and RFC 7677: every builder, event and refusal (`tests/link/std_postgres`), and exchanges recorded from PostgreSQL 18 by `tests/postgres/record.mjs` replayed byte for byte (`tests/link/std_postgres_replay`); the batcher's queueing, caps and ordinal refusals (`tests/link/std_postgres_batch`); the clock's reasons, backlog and turns (`tests/link/std_postgres_batch_clock`) |
|
|
255
|
+
| [`net/postgres.ts`](./net/postgres.ts) | `PgConnection`, a PostgreSQL client that reads no socket: the program fills `inputBuffer()` and sends `outputBuffer()`, and `next()` answers `PG_NEED_MORE`, `PG_READY`, `PG_DESCRIBED`, `PG_ROW`, `PG_COMPLETE`, `PG_ERROR`, `PG_NOTICE`, `PG_GROUP_DONE` or the final `PG_FAILED` (`failure`, one of the `PG_ERR_*` codes). Protocol 3.0: startup, `trust` and SCRAM-SHA-256 with the server's signature checked (cleartext and MD5 refused), the extended query protocol — `prepare`, `bindStart`, `paramInt` / `paramText` / `paramBytes` / `paramTextArray` / `paramIntArray` / `paramNull`, an array built element by element into the Bind (`arrayStart`, `arrayElement`, `arrayInt`, `arrayNull`, `arrayEnd`), binary `int4[]` / `int8[]` parameters (`paramInt4Array` / `paramInt8Array` over an `i64[]` window, or `binaryArrayStart`, `binaryArrayInt`, `binaryArrayNull` and the same `arrayEnd`; per-parameter format codes only once a binary one is bound; an `int4` element out of range panics), `bindEnd`, `bindStartBinary` for a Bind whose result columns come in binary (read in place by `binaryInt`, `binaryBool`, `binaryFloat` and `PgBinary`'s `timestamp` and `iso`, and a one-dimensional `int2[]`, `int4[]` or `int8[]` by `PgBinaryArray`), `bindCancel` to drop a Bind not yet ended, `closeStatement`, `sync` — with up to `maxInFlight` groups pipelined. `PgBatcher` queues keyed reads of one statement shape (`addInt`, `addText`) and `take(conn)` binds up to `maxRequests` of them, each key once, as one execution over `unnest($1) with ordinality`; `row`, `end` and `fail` route its rows by their server-supplied ordinal, checked first (read with `binaryInt` when `PgBatchConfig.binary` is set, so the batch's rows come in binary too), to `next(run)`'s per-request `PG_BATCH_ROW` and `PG_BATCH_DONE` / `PG_BATCH_MISSING` / `PG_BATCH_FAILED`; `take` answers a fresh handle each time, so a server's late or repeated event on a spent one is counted in `stray`, never routed or panicked on. It reads no clock and allocates nothing per request or row. `PgBatchClock` says when to take, from times the program passes in: once max(`minCount`, the moving average of batch sizes × `factor`) wait, the oldest has waited `maxWaitUs`, or, with the idle bypass, nothing is in flight and the load is light (its defaults are the fixed window); `PgBatcher.takeDue` takes only then, and `PgBatchTurns` lets one free connection collect while the others execute. A row's columns are windows onto the receive buffer (`colStart`, `colLength`), read in place by `int`, `bool`, `isNull` and `PgText`'s decoders (`timestamptz`, `PgArrayWalker` for `text[]`); no per-row or per-column allocation. The password is a `Secret<u8[]>`, wiped once the proof is made. Caps on message size, columns, groups in flight and SCRAM iterations (`docs/security/postgres.md`) | PostgreSQL 18's frontend/backend protocol chapter, RFC 5802 and RFC 7677: every builder, event and refusal (`tests/link/std_postgres`), and exchanges recorded from PostgreSQL 18 by `tests/postgres/record.mjs` replayed byte for byte (`tests/link/std_postgres_replay`); the batcher's queueing, caps and ordinal refusals (`tests/link/std_postgres_batch`); the clock's reasons, backlog and turns (`tests/link/std_postgres_batch_clock`) |
|
|
255
256
|
| [`net/postgres-carrier.ts`](./net/postgres-carrier.ts) | `PgCarrier`, a `PgPool`'s queries on an `HttpHandler`'s loop: its sockets sit in the handler's epoll set, a route handler hands a request to `run` (or `runInt` / `runText` for one of up to 8 attached `PgBatcher`s) and gets its number at once, and `next()` answers `PG_CARRIER_BUILD` to bind a plain query on `connection()`, `PG_CARRIER_ROW`, `PG_CARRIER_COMPLETE`, `PG_CARRIER_ERROR` and one final `PG_CARRIER_DONE`, `PG_CARRIER_MISSING` or `PG_CARRIER_FAILED` (`failure`, `socketError`) with `request()` the request to answer and `tag` the caller's. It connects the pool's slots itself (TCP or a Unix socket), times out a connect after `connectTimeoutMs`, sheds batched keys when no slot can connect, and sizes every table from `PgCarrierLimits` at construction: `run` answers `PG_CARRIER_FULL` rather than grow one, and once connections are up it allocates nothing per query. Kept apart from `net/postgres.ts` so that module reaches no capability (`docs/security/postgres.md`) | A loopback backend in the test itself: routing, a slow query not stalling another client, lost and refused connections, a connect timeout, and `Arena.used()` flat over thousands of queries (`tests/link/std_postgres_carrier`); a live server by `tests/postgres/live/run.mjs` |
|
|
256
257
|
| [`net/prefork.ts`](./net/prefork.ts) | A server on every core: `preforkWorkers(process.argv)` or `preforkWith(process.argv, config)` answers a worker's index in each worker it starts again from the same program, and in the supervisor runs them until SIGTERM or SIGINT, restarting one that exits after a backoff. `PreforkConfig.supervisorChannels` gives each worker a `unixSocketPair` to the supervisor, which `preforkSupervise(args, config, state, onChannel)` hands to a callback; `serviceWorker` gives every other worker a pair to that one, and makes the workers one group that restarts together on fresh pairs. The ends arrive from descriptor 3 through `spawnWithFds`, found with `preforkSupervisorChannel`, `preforkServiceChannel` and `preforkServiceChannelOf`. Native only | `tests/link/net_prefork` (the spread, a dead worker, SIGTERM, the backoff); `tests/link/net_prefork_channels` (each worker holds exactly its own ends, a restart, the supervisor's descriptors flat) |
|
|
257
258
|
|
|
@@ -347,12 +348,12 @@ send, over loopback from a Nish UDP client, exchanges recorded against
|
|
|
347
348
|
aioquic by their `record.sh`: a handshake and echo, and Version Negotiation,
|
|
348
349
|
Retry, key update both ways, the idle timeout and a stateless reset.
|
|
349
350
|
|
|
350
|
-
`net/http1.ts`, `net/http1-server.ts` and `net/websocket.ts` run their checks
|
|
351
|
+
`net/http1-codec.ts`, `net/http1-server.ts` and `net/websocket.ts` run their checks
|
|
351
352
|
again under `--number-mode f64` (`net_http1_f64`, `net_http1_server_f64`,
|
|
352
353
|
`net_websocket_f64`). So does the HTTP date (`net_http_date_f64`). None
|
|
353
354
|
handles a secret but through `nish/net/tls-tcp`, so none is in the
|
|
354
355
|
constant-time check; their record is
|
|
355
|
-
[`docs/security/http1.md`](../docs/security/http1.md), written with the code
|
|
356
|
+
[`docs/security/http1-codec.md`](../docs/security/http1-codec.md), written with the code
|
|
356
357
|
rather than after an audit.
|
|
357
358
|
|
|
358
359
|
## How a program imports it
|