@amritk/nish-x86_64-linux 0.17.0 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/bin/nish CHANGED
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@amritk/nish-x86_64-linux",
3
- "version": "0.17.0",
3
+ "version": "0.18.0",
4
4
  "description": "The @amritk/nish native compiler for Linux on x86_64",
5
5
  "license": "MIT",
6
6
  "os": [
package/runtime/nish.h CHANGED
@@ -3,15 +3,17 @@
3
3
  * Include this from C drivers, N-API shims, or any other host that links the C
4
4
  * runtime next to compiled Nish modules. Everything here is a contract shared
5
5
  * with `src/runtime.ts` (the IR side) and the implementation, which is
6
- * two translation units: `runtime/runtime.c` holds the core every program
7
- * touches — the arena, strings, arrays, number formatting, the panics — and
6
+ * several translation units: `runtime/runtime.c` holds the core every program
7
+ * touches — the arena, strings, arrays, number formatting, the panics —
8
8
  * `runtime/runtime-os.c` holds everything that wraps a system call: the file
9
9
  * functions, the directory and subprocess calls, `nish_getenv`, the monotonic
10
- * clock, `nish_platform` / `nish_arch`. They are separate so that each carries
10
+ * clock, `nish_platform` / `nish_arch` — and runtime-parallel.c, runtime-host.c,
11
+ * runtime-net.c and runtime-simd.c (the byte searches that read a vector at a
12
+ * time) split off the same way since. They are separate so that each carries
11
13
  * its own measured code-size ceiling (docs/wp7-runtime.md, "Runtime additions
12
- * and budget"); a host links both, which `scripts/build.sh` does for you when
13
- * handed either. `tests/run.js` checks that every runtime function declared in
14
- * runtime.ts has a prototype in this file.
14
+ * and budget"); a host links all of them, which `scripts/build.sh` does for
15
+ * you when handed runtime.c. `tests/run.js` checks that every runtime function
16
+ * declared in runtime.ts has a prototype in this file.
15
17
  *
16
18
  * Headers generated by `nish --emit-header` include this one, so keep
17
19
  * it C11-clean under -Wall -Wextra -Werror and usable from C++.
@@ -137,6 +139,18 @@ bool nish_str_at(const nish_str *s, int64_t at, const nish_str *sub);
137
139
  /* `s.indexOf(sub)`: the first byte offset where `sub` occurs, or -1. An empty
138
140
  needle answers 0 and one longer than `s` answers -1, as in JavaScript. */
139
141
  int64_t nish_str_index_of(const nish_str *s, const nish_str *sub);
142
+ /* `s.indexOf(sub, from)`: the same search from byte `from`, which must already
143
+ be clamped into [0, s->len]; an empty needle answers `from`. Outside that
144
+ range nothing outside `s` is read: a non-empty needle answers -1 and an
145
+ empty one answers `from` unchanged. */
146
+ int64_t nish_str_index_of_from(const nish_str *s, const nish_str *sub, int64_t from);
147
+ /* First index i >= from with s[i] in set, or -1.
148
+ from is already clamped to [0, len] by the caller; set holds 1..16 bytes.
149
+ It lives in runtime/runtime-simd.c, which picks a scalar, a baseline vector
150
+ or an AVX2 path on the first call; `NISH_SIMD=scalar|base|avx2` pins it. A
151
+ set outside 1..16 bytes is answered by the scalar path rather than refused,
152
+ but only 1..16 is the contract. Variable-time: never call it on a secret. */
153
+ int64_t nish_str_index_of_any(const nish_str *s, const nish_str *set, int64_t from);
140
154
  /* `console.log(s)`: one write(2) of the bytes plus a newline to stdout. */
141
155
  void nish_print(const nish_str *s);
142
156
  /* `console.error` and the newline-free `write` / `writeError`: fd 1 or 2. */
@@ -0,0 +1,203 @@
1
+ /* Nish runtime, the vector half: byte searches that read sixteen or thirty-two
2
+ * bytes at a time (WP38 S1, docs/wp38-simd.md §3.1).
3
+ *
4
+ * A translation unit of its own for the reason runtime-host.c is one: each file
5
+ * carries its own measured `.text*` ceiling in tests/run.js, and runtime.c had
6
+ * no room left when this arrived. Section GC means a program that never calls a
7
+ * kernel links none of this file, and scripts/build.sh pairs it with runtime.c
8
+ * like the other halves, so a link line still names one runtime.
9
+ *
10
+ * Every kernel has up to three paths. The scalar one runs anywhere. The base
11
+ * one uses the vectors every machine of the target has -- SSE2 on x86-64, NEON
12
+ * on AArch64, simd128 on wasm only when built with `-msimd128`, which is not
13
+ * the default (wp38 §5) -- so it needs no question asked of the CPU. The AVX2
14
+ * one is compiled for that extension alone with `target("avx2")`, so the binary
15
+ * stays a baseline binary and only a machine that has it ever runs it.
16
+ *
17
+ * Which path runs is decided on a kernel's first call, never at program start.
18
+ * The kernel is called through a pointer that starts at its resolver; the
19
+ * resolver asks the CPU, stores the path it picked and calls it, and every
20
+ * later call goes straight to the stored path. `NISH_SIMD` pins the choice:
21
+ * `scalar` and `base` name their paths, and `avx2`, any other value, or no
22
+ * value at all mean the best this machine has. A machine without AVX2 that is
23
+ * asked for it gets the base path, because running it would fault. The
24
+ * instruction gate sets `NISH_SIMD=base` (bench/run.mjs) so that a count does
25
+ * not depend on the host it was measured on.
26
+ */
27
+ #include <stdint.h>
28
+ #include <stdlib.h>
29
+ #include <string.h>
30
+
31
+ #include "nish.h"
32
+
33
+ #if defined(__x86_64__) || defined(__i386__)
34
+ #if defined(__SSE2__)
35
+ #include <cpuid.h>
36
+ #include <immintrin.h>
37
+ #define NISH_SIMD_SSE2 1
38
+ #endif
39
+ #elif defined(__ARM_NEON)
40
+ #include <arm_neon.h>
41
+ #define NISH_SIMD_NEON 1
42
+ #elif defined(__wasm_simd128__)
43
+ #include <wasm_simd128.h>
44
+ #define NISH_SIMD_WASM 1
45
+ #endif
46
+
47
+ /* One path of `nish_str_index_of_any`: the contract in nish.h. */
48
+ typedef int64_t (*nish_any_path)(const nish_str *s, const nish_str *set, int64_t from);
49
+
50
+ /* The scalar path, and the tail of the vector ones. A 256-bit membership
51
+ table answers each byte with one load and one shift whatever the size of
52
+ the set, so this path is also total: an empty set finds nothing and a set
53
+ longer than sixteen bytes is answered correctly, which is why the vector
54
+ paths hand any set outside 1..16 bytes to it rather than overrun their
55
+ sixteen splatted registers. */
56
+ static int64_t nish_any_scalar(const nish_str *s, const nish_str *set, int64_t from) {
57
+ uint32_t in[8] = {0};
58
+ for (uint64_t k = 0; k < set->len; k++) {
59
+ uint8_t b = (uint8_t)set->data[k];
60
+ in[b >> 5] |= 1u << (b & 31);
61
+ }
62
+ for (uint64_t i = (uint64_t)from; i < s->len; i++) {
63
+ uint8_t b = (uint8_t)s->data[i];
64
+ if (in[b >> 5] >> (b & 31) & 1) return (int64_t)i;
65
+ }
66
+ return -1;
67
+ }
68
+
69
+ /* The vector paths compare each block against every byte of the set and OR
70
+ the answers, rather than classify each byte by its two nibbles through a
71
+ shuffle table as simdjson does. The shuffle is `pshufb`, which is SSSE3 and
72
+ not in the x86-64 baseline, and a two-table classify is only exact for a
73
+ set whose bytes fit eight nibble classes; an arbitrary set of sixteen bytes
74
+ does not. A compare per set byte is exact for every set, and the sets this
75
+ was built for -- a quote and a backslash, JSON's six structural bytes -- are
76
+ small enough that the compares cost less than the shuffles would. */
77
+ #if defined(NISH_SIMD_SSE2)
78
+ static int64_t nish_any_base(const nish_str *s, const nish_str *set, int64_t from) {
79
+ uint64_t k = set->len, n = s->len, i = (uint64_t)from;
80
+ if (k - 1 >= 16) return nish_any_scalar(s, set, from);
81
+ __m128i want[16];
82
+ for (uint64_t j = 0; j < k; j++) want[j] = _mm_set1_epi8(set->data[j]);
83
+ for (; i + 16 <= n; i += 16) {
84
+ __m128i v = _mm_loadu_si128((const __m128i *)(s->data + i));
85
+ __m128i hit = _mm_cmpeq_epi8(v, want[0]);
86
+ for (uint64_t j = 1; j < k; j++) hit = _mm_or_si128(hit, _mm_cmpeq_epi8(v, want[j]));
87
+ unsigned m = (unsigned)_mm_movemask_epi8(hit);
88
+ if (m) return (int64_t)(i + (uint64_t)__builtin_ctz(m));
89
+ }
90
+ return nish_any_scalar(s, set, (int64_t)i);
91
+ }
92
+
93
+ /* Thirty-two bytes a block, then the base path for what is left: a tail of
94
+ up to thirty-one bytes is at most one sixteen-byte block and a scalar run. */
95
+ __attribute__((target("avx2"))) static int64_t nish_any_avx2(const nish_str *s, const nish_str *set,
96
+ int64_t from) {
97
+ uint64_t k = set->len, n = s->len, i = (uint64_t)from;
98
+ if (k - 1 >= 16) return nish_any_scalar(s, set, from);
99
+ __m256i want[16];
100
+ for (uint64_t j = 0; j < k; j++) want[j] = _mm256_set1_epi8(set->data[j]);
101
+ for (; i + 32 <= n; i += 32) {
102
+ __m256i v = _mm256_loadu_si256((const __m256i *)(s->data + i));
103
+ __m256i hit = _mm256_cmpeq_epi8(v, want[0]);
104
+ for (uint64_t j = 1; j < k; j++) hit = _mm256_or_si256(hit, _mm256_cmpeq_epi8(v, want[j]));
105
+ unsigned m = (unsigned)_mm256_movemask_epi8(hit);
106
+ if (m) return (int64_t)(i + (uint64_t)__builtin_ctz(m));
107
+ }
108
+ return nish_any_base(s, set, (int64_t)i);
109
+ }
110
+ #elif defined(NISH_SIMD_NEON)
111
+ static int64_t nish_any_base(const nish_str *s, const nish_str *set, int64_t from) {
112
+ uint64_t k = set->len, n = s->len, i = (uint64_t)from;
113
+ if (k - 1 >= 16) return nish_any_scalar(s, set, from);
114
+ uint8x16_t want[16];
115
+ for (uint64_t j = 0; j < k; j++) want[j] = vdupq_n_u8((uint8_t)set->data[j]);
116
+ for (; i + 16 <= n; i += 16) {
117
+ uint8x16_t v = vld1q_u8((const uint8_t *)s->data + i);
118
+ uint8x16_t hit = vceqq_u8(v, want[0]);
119
+ for (uint64_t j = 1; j < k; j++) hit = vorrq_u8(hit, vceqq_u8(v, want[j]));
120
+ /* NEON has no movemask. Shifting each 16-bit pair of answers right by four
121
+ and narrowing keeps four bits of every byte's answer, in order, in one
122
+ 64-bit lane, so the first hit is the lowest set bit over four. The
123
+ idea is Danila Kutenin's ("Porting x86 vector bitmask optimizations to
124
+ Arm NEON", Arm Community blog, 2022). */
125
+ uint64_t m = vget_lane_u64(vreinterpret_u64_u8(vshrn_n_u16(vreinterpretq_u16_u8(hit), 4)), 0);
126
+ if (m) return (int64_t)(i + ((uint64_t)__builtin_ctzll(m) >> 2));
127
+ }
128
+ return nish_any_scalar(s, set, (int64_t)i);
129
+ }
130
+ #elif defined(NISH_SIMD_WASM)
131
+ static int64_t nish_any_base(const nish_str *s, const nish_str *set, int64_t from) {
132
+ uint64_t k = set->len, n = s->len, i = (uint64_t)from;
133
+ if (k - 1 >= 16) return nish_any_scalar(s, set, from);
134
+ v128_t want[16];
135
+ for (uint64_t j = 0; j < k; j++) want[j] = wasm_i8x16_splat(set->data[j]);
136
+ for (; i + 16 <= n; i += 16) {
137
+ v128_t v = wasm_v128_load(s->data + i);
138
+ v128_t hit = wasm_i8x16_eq(v, want[0]);
139
+ for (uint64_t j = 1; j < k; j++) hit = wasm_v128_or(hit, wasm_i8x16_eq(v, want[j]));
140
+ uint32_t m = wasm_i8x16_bitmask(hit);
141
+ if (m) return (int64_t)(i + (uint64_t)__builtin_ctz(m));
142
+ }
143
+ return nish_any_scalar(s, set, (int64_t)i);
144
+ }
145
+ #else
146
+ /* A target with no vectors in its baseline: wasm without `-msimd128`, the
147
+ default there, among them. Its base path is the scalar one. */
148
+ #define nish_any_base nish_any_scalar
149
+ #endif
150
+
151
+ #if defined(NISH_SIMD_SSE2)
152
+ /* Whether this machine runs AVX2 code: the CPU has the extension (leaf 7,
153
+ EBX bit 5) and the operating system saves the 256-bit registers across a
154
+ context switch (OSXSAVE, then XCR0's SSE and AVX bits): the procedure the
155
+ Intel SDM gives for detecting AVX2, and the three facts
156
+ `__builtin_cpu_supports("avx2")` reads. That builtin is not used,
157
+ because its answer lives in libgcc's `__cpu_model`, and the object that
158
+ defines it carries a constructor that fills it in at start-up: once a link
159
+ names it, every program pays, a kernel called or not. Measured on
160
+ `examples/hello.ts` at `--profile speed`, that object took the binary from
161
+ 4,712 bytes to 10,000 and the run 374 instructions longer. `cpuid.h` is inline
162
+ assembly and nothing else, so here the question is asked on the first call
163
+ and never before it. */
164
+ static int nish_cpu_has_avx2(void) {
165
+ unsigned a, b, c, d;
166
+ if (!__get_cpuid(1, &a, &b, &c, &d) || !(c & bit_OSXSAVE) || !(c & bit_AVX)) return 0;
167
+ unsigned lo, hi;
168
+ __asm__("xgetbv" : "=a"(lo), "=d"(hi) : "c"(0));
169
+ if ((lo & 6) != 6) return 0;
170
+ return __get_cpuid_count(7, 0, &a, &b, &c, &d) && (b & bit_AVX2);
171
+ }
172
+ #endif
173
+
174
+ /* Which path `NISH_SIMD` and this machine pick. AVX2 is asked of the CPU only
175
+ when the answer could matter, so a pinned `scalar` or `base` never reads
176
+ the CPU's feature bits at all. */
177
+ static nish_any_path nish_any_choose(void) {
178
+ const char *pin = getenv("NISH_SIMD");
179
+ if (pin != NULL && strcmp(pin, "scalar") == 0) return nish_any_scalar;
180
+ if (pin != NULL && strcmp(pin, "base") == 0) return nish_any_base;
181
+ #if defined(NISH_SIMD_SSE2)
182
+ if (nish_cpu_has_avx2()) return nish_any_avx2;
183
+ #endif
184
+ return nish_any_base;
185
+ }
186
+
187
+ static int64_t nish_any_resolve(const nish_str *s, const nish_str *set, int64_t from);
188
+
189
+ /* The path every call goes through, the resolver until the first call has
190
+ run. The loads and the store are relaxed atomics, which are plain moves on
191
+ every target, because under `--threads` two workers may make the first call
192
+ at once; both pick the same path, so whichever store lands is the answer. */
193
+ static nish_any_path nish_any = nish_any_resolve;
194
+
195
+ static int64_t nish_any_resolve(const nish_str *s, const nish_str *set, int64_t from) {
196
+ nish_any_path path = nish_any_choose();
197
+ __atomic_store_n(&nish_any, path, __ATOMIC_RELAXED);
198
+ return path(s, set, from);
199
+ }
200
+
201
+ int64_t nish_str_index_of_any(const nish_str *s, const nish_str *set, int64_t from) {
202
+ return __atomic_load_n(&nish_any, __ATOMIC_RELAXED)(s, set, from);
203
+ }
@@ -115,6 +115,17 @@ void nish_array_grow(nish_array *a, uint64_t elem_size) {
115
115
  a->cap = cap;
116
116
  }
117
117
 
118
+ /* ---- Math.min and Math.max of a float: llvm.minnum / llvm.maxnum, which wasm32
119
+ * has no instruction for. Its `f64.min` answers NaN when either operand is NaN,
120
+ * and minnum answers the other operand, so the backend lowers minnum to a libm
121
+ * call, and these are that libm. They answer what minnum answers natively: a
122
+ * NaN operand is dropped, and two NaNs give NaN (tests/cases/math_minmax_float,
123
+ * held to these by tests/run.js). Between `0` and `-0` minnum may answer either. */
124
+ double fmin(double a, double b) { return a != a ? b : b != b ? a : a < b ? a : b; }
125
+ double fmax(double a, double b) { return a != a ? b : b != b ? a : a > b ? a : b; }
126
+ float fminf(float a, float b) { return a != a ? b : b != b ? a : a < b ? a : b; }
127
+ float fmaxf(float a, float b) { return a != a ? b : b != b ? a : a > b ? a : b; }
128
+
118
129
  /* ---- Panics: `unreachable`, which the host sees as a RuntimeError */
119
130
  void nish_panic_index(uint64_t idx, uint64_t len) { (void)idx; (void)len; __builtin_trap(); }
120
131
  void nish_panic_slice(int64_t s, int64_t e, int64_t len) { (void)s; (void)e; (void)len; __builtin_trap(); }
package/runtime/runtime.c CHANGED
@@ -251,19 +251,29 @@ _Bool nish_str_at(const nish_str *s, int64_t at, const nish_str *sub) {
251
251
  exactly such a header from `examples/strings.ts`, and the interop tests
252
252
  caught runtime.c picking it up and inheriting `nish.h` through it. A C host
253
253
  would hit the same. A fifth of the time is not worth making the runtime
254
- sensitive to its includer's `-I` paths. */
255
- int64_t nish_str_index_of(const nish_str *s, const nish_str *sub) {
256
- if (sub->len == 0) return 0;
257
- if (sub->len > s->len) return -1;
258
- const char *p = s->data, *end = s->data + s->len - sub->len + 1;
259
- while (p < end) {
260
- const char *c = memchr(p, sub->data[0], (size_t)(end - p));
261
- if (!c) return -1;
262
- if (memcmp(c + 1, sub->data + 1, sub->len - 1) == 0) return (int64_t)(c - s->data);
263
- p = c + 1;
254
+ sensitive to its includer's `-I` paths.
255
+
256
+ `s.indexOf(sub, from)` is the same search from byte `from`, and
257
+ `s.indexOf(sub)` is it from 0: one loop, not two. The compiler clamps `from`
258
+ into [0, s.length] first, as JavaScript does, with `llvm.smin`/`smax` where
259
+ LLVM can fold them, so the clamp costs this file nothing; an empty needle
260
+ answers that clamped `from`. The clamp is a precondition: a caller that
261
+ skips it never reads outside the string, but gets -1 for a non-empty needle
262
+ (a negative `from` reads as one past the end) and `from` itself, unchanged,
263
+ for an empty one. */
264
+ int64_t nish_str_index_of_from(const nish_str *s, const nish_str *sub, int64_t from) {
265
+ uint64_t at = (uint64_t)from;
266
+ if (sub->len == 0) return from;
267
+ while (at <= s->len && sub->len <= s->len - at) {
268
+ const char *c = memchr(s->data + at, sub->data[0], s->len - at - sub->len + 1);
269
+ if (!c) break;
270
+ at = (uint64_t)(c - s->data);
271
+ if (memcmp(c + 1, sub->data + 1, sub->len - 1) == 0) return (int64_t)at;
272
+ at++;
264
273
  }
265
274
  return -1;
266
275
  }
276
+ int64_t nish_str_index_of(const nish_str *s, const nish_str *sub) { return nish_str_index_of_from(s, sub, 0); }
267
277
 
268
278
  /* `console.log` / `console.error` / `write` / `writeError` and the message of
269
279
  `panic`: fd 1 or 2, with or without the trailing newline.
package/scripts/build.sh CHANGED
@@ -3,13 +3,14 @@
3
3
  #
4
4
  # scripts/build.sh <module.ll> [more .ll/.c files...] -o <out> [--profile debug|speed|size|wasm|wasi|napi]
5
5
  #
6
- # The C runtime is five translation units and is named as one: an input
6
+ # The C runtime is six translation units and is named as one: an input
7
7
  # <dir>/runtime.c also compiles <dir>/runtime-os.c, the half that wraps the
8
8
  # system calls (files, directories, subprocesses, the environment, the clock),
9
9
  # <dir>/runtime-parallel.c, the half that divides a range of work across
10
10
  # threads, <dir>/runtime-host.c, the wall clock, entropy, file times and
11
- # signals, and <dir>/runtime-net.c, the sockets of `nish:net`. Each of those
12
- # files says why they are compiled and measured apart.
11
+ # signals, <dir>/runtime-net.c, the sockets of `nish:net`, and
12
+ # <dir>/runtime-simd.c, the byte searches that read a vector at a time. Each of
13
+ # those files says why they are compiled and measured apart.
13
14
  #
14
15
  # Profiles:
15
16
  # debug clang defaults: no optimisation, symbols kept. The "before" number.
@@ -76,18 +77,21 @@ while [ $# -gt 0 ]; do
76
77
  --pgo-use)
77
78
  [ -f "$2" ] || { echo "error: --pgo-use: profile '$2' not found (run the instrumented binary, then llvm-profdata merge)" >&2; exit 2; }
78
79
  pgo=("-fprofile-use=$2"); shift 2 ;;
79
- -h|--help) sed -n '2,41p' "$0"; exit 0 ;;
80
+ -h|--help) sed -n '2,42p' "$0"; exit 0 ;;
80
81
  *) inputs+=("$1"); shift ;;
81
82
  esac
82
83
  done
83
84
  [ ${#inputs[@]} -gt 0 ] || { echo "error: no input files" >&2; exit 2; }
84
85
  [ -n "$out" ] || { echo "error: -o <out> is required" >&2; exit 2; }
85
86
 
86
- # The runtime is five translation units, and a caller names one: whoever passes
87
+ # The runtime is six translation units, and a caller names one: whoever passes
87
88
  # <dir>/runtime.c gets <dir>/runtime-os.c, <dir>/runtime-parallel.c,
88
- # <dir>/runtime-host.c and <dir>/runtime-net.c compiled beside it. They were one file until the operating-system half was split out
89
- # for its own size budget, and the parallel half followed for the same reason
90
- # (each file's header comment says why), and a link line is where those splits
89
+ # <dir>/runtime-host.c, <dir>/runtime-net.c and <dir>/runtime-simd.c compiled
90
+ # beside it, on every profile, wasi included: runtime-simd.c takes its scalar
91
+ # path there unless the build adds -msimd128. They were one file until the
92
+ # operating-system half was split out for its own size budget, and the parallel
93
+ # half followed for the same reason (each file's header comment says why), and
94
+ # a link line is where those splits
91
95
  # would otherwise leak: `nish --link` builds its command line in
92
96
  # src/compile.ts, the published package's recipe in every document and
93
97
  # README names runtime.c, and a user's own clang line does too. Pairing them
@@ -97,7 +101,7 @@ done
97
101
  for i in ${inputs[@]+"${inputs[@]}"}; do
98
102
  case "$i" in
99
103
  */runtime.c|runtime.c)
100
- for half in runtime-os.c runtime-parallel.c runtime-host.c runtime-net.c; do
104
+ for half in runtime-os.c runtime-parallel.c runtime-host.c runtime-net.c runtime-simd.c; do
101
105
  side="${i%runtime.c}$half"
102
106
  have=0
103
107
  for j in "${inputs[@]}"; do
package/std/README.md CHANGED
@@ -2,14 +2,15 @@
2
2
 
3
3
  Nish modules written in Nish, for Nish programs to import. There is no magic
4
4
  here and — with four exceptions, `threads.ts`, `collections.ts`, `map.ts` and
5
- `secret.ts` — nothing the compiler knows about: a module in this directory is an ordinary Nish source file, compiled as part of
5
+ `secret.ts`, and one function, `text.ts`'s `indexOfAny` — nothing the compiler
6
+ knows about: a module in this directory is an ordinary Nish source file, compiled as part of
6
7
  whatever program imports it, and subject to the same rules as `examples/` or
7
8
  `src/` ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md) is the style guide).
8
9
 
9
10
  | Module | What it is |
10
11
  | --- | --- |
11
12
  | [`testing.ts`](./testing.ts) | a test runner: a `Suite` a program drives with straight-line assertions, printing the `PASS` / `FAIL` / `SKIP` lines the repository's own harness prints, and answering the exit code |
12
- | [`text.ts`](./text.ts) | the string operations a program would otherwise write inline: `splitLines`, `splitWhitespace`, `trim` and its halves, `contains`, `replaceAll`, and `firstDifference` over two arrays of lines |
13
+ | [`text.ts`](./text.ts) | the string operations a program would otherwise write inline: `splitLines`, `splitWhitespace`, `trim` and its halves, `contains`, `replaceAll`, `firstDifference` over two arrays of lines, and `indexOfAny(text, bytes, from)`, the first index at or after `from` whose byte is one of 1 to 16 ASCII `bytes`. Its walker is the meaning and what runs under Node; natively the compiler replaces the one call to it with the runtime kernel `nish_str_index_of_any`, which reads 16 or 32 bytes at a time and is variable-time ([`docs/LANGUAGE.md`](../docs/LANGUAGE.md#byte-search-indexofany-in-nishtext)). `tests/link/std_text_index_of_any*` are its programs |
13
14
  | [`json.ts`](./json.ts) | `jsonField(object, name)`: the value of one field of one flat JSON object, which is the shape the compiler's own `--json` diagnostics have. A reader and not a parser — it answers text, answers `null` for a field that is not there, and does not validate |
14
15
  | [`pair.ts`](./pair.ts) | `Pair<A, B>`: an interface with `first` and `second`, for a function that answers two values from one call. A type and nothing else — the caller writes an object literal at the return — and for returning two values rather than storing them side by side |
15
16
  | [`collections.ts`](./collections.ts) | the global `Map<K, V>` and `Set<T>`: insertion-ordered tables whose buckets carry a hash fingerprint beside the entry index and whose entries keep their full hash, so every `get`, `set`, `add`, `has` and `delete` is one probe. `get` is not a method here: its `V | undefined` never crosses a call, so the compiler lowers it to `probe` and, where the key was found, `valueAt`. Nor are `keys()` and `values()`: an iterator is not a value, so a `for...of` over one is lowered to `walkOpen`, `walkNext`, `keyAt` or `valueAt`, and `walkClose`, and a count of live walks defers compaction until no loop is walking the table. A program never imports it: naming `Map` or `Set` loads it, and the compiler emits what a module uses of it into that module ([`docs/wp32-map.md`](../docs/wp32-map.md), [`docs/LANGUAGE.md`](../docs/LANGUAGE.md#map-and-set)). Its `hashKey`, `sameKey` and `storedKey` are lowered by the compiler per key type |
@@ -61,7 +62,7 @@ take their keys as plain arrays and do not wipe them yet.
61
62
  | [`crypto/x25519.ts`](./crypto/x25519.ts) | `x25519(scalar, u)`, the shared secret as a `Secret<u8[]>`, and `x25519Base(scalar)`, the public key, each from a `Secret<u8[]>` scalar, on ten 25.5-bit limbs in `i64`. Either answers `null` unless its arguments are `X25519_SIZE` (32) bytes; the scalar is clamped on a copy, which is wiped | RFC 7748 §5.2, §6.1, Wycheproof `x25519_test` |
62
63
  | [`crypto/chacha20poly1305.ts`](./crypto/chacha20poly1305.ts) | `chacha20Poly1305Seal(key, nonce, aad, plaintext)`, which answers the ciphertext followed by the 16-byte tag, and `chacha20Poly1305Open`, which checks the whole tag before it decrypts anything and answers `null` for a message that does not authenticate. Beneath them `chacha20Block`, `chacha20`, `chacha20QuarterRound`, `poly1305` and `poly1305KeyGen`, and `chacha20HeaderMask`, QUIC's 5-byte header-protection mask. Every input of the wrong length answers `null` | RFC 8439 §2, RFC 9001 §5.4.4, Wycheproof `chacha20_poly1305` (all 325 cases) |
63
64
  | [`crypto/aes.ts`](./crypto/aes.ts) | AES-128 and AES-256, bitsliced four blocks at a time with the Boyar–Peralta S-box circuit, so no table is read: `aesKey(key)` expands a 16- or 32-byte key into an `AesKey` (or `aesKeyInto(key, out)` into one the caller holds, allocating nothing that outlives the call), then `aesEncryptBlock`, `aesGcmSeal` / `aesGcmOpen` (tag last, checked in full before anything is decrypted; any non-empty IV) and `aesHeaderMask`. AES-192 is out of scope, and a wrong length answers `null`, as does an `AesKey` not made by `aesKey`; one whose H is zero opens nothing | FIPS 197, SP 800-38D, RFC 9001 §5.4.3, Wycheproof `aes_gcm` (all 316 cases; the 103 AES-192 ones check that the key is refused) |
64
- | [`crypto/p256.ts`](./crypto/p256.ts) | ECDSA over P-256 with RFC 6979's deterministic nonces: `p256PublicKey(priv)` (65-byte uncompressed SEC1) from a `Secret<u8[]>` private key, `p256Sign` / `p256Verify` over a digest and `p256SignSha256` / `p256VerifySha256` over a message; a signature is `r` and `s`, 32 big-endian bytes each, not DER (`crypto/x509.ts`'s `x509DerSignatureRS` converts one). A malformed key, a point off the curve or an `r` or `s` out of range answers `null` or `false`, never a panic. ECDSA is malleable, and `p256Verify` accepts a high `s`: when `(r, s)` verifies, so does `(r, n − s)` | RFC 6979 A.2.5, Wycheproof `ecdsa_secp256r1_sha256`, `_sha256_p1363`, `_sha512` and `_sha512_p1363` |
65
+ | [`crypto/p256.ts`](./crypto/p256.ts) | ECDSA over P-256 with RFC 6979's deterministic nonces: `p256PublicKey(priv)` (65-byte uncompressed SEC1) from a `Secret<u8[]>` private key, `p256Sign` / `p256Verify` over a digest and `p256SignSha256` / `p256VerifySha256` over a message, and `p256SignSha256Into(scratch, priv, msg, off, len, out, at)`, the same signature written into the caller's window in a `P256SignScratch` made once, keeping nothing in the arena; a signature is `r` and `s`, 32 big-endian bytes each, not DER (`crypto/x509.ts`'s `x509DerSignatureRS` converts one). A malformed key, a point off the curve or an `r` or `s` out of range answers `null` or `false`, never a panic. ECDSA is malleable, and `p256Verify` accepts a high `s`: when `(r, s)` verifies, so does `(r, n − s)` | RFC 6979 A.2.5, Wycheproof `ecdsa_secp256r1_sha256`, `_sha256_p1363`, `_sha512` and `_sha512_p1363` |
65
66
  | [`crypto/x509.ts`](./crypto/x509.ts) | DER, PEM and X.509 over P-256: `pemToDer` / `derToPem` (RFC 7468, padded standard base64, strict labels), `x509ParseP256PrivateKey` (SEC1 or PKCS#8, answering a `Secret<u8[]>`), `x509ParseCertificate` / `x509ParseChain` into an `X509Certificate`, `x509VerifySignature` (ecdsa-with-SHA256 under an issuer's key), `x509DerSignatureRS` (a DER ECDSA-Sig-Value as the `r || s` `crypto/p256.ts` takes), `x509MintSelfSigned`, the 1-to-14-day self-signed P-256 certificate WebTransport's `serverCertificateHashes` accepts, and `x509CertificateHash`, its SHA-256. DER is read strictly — non-minimal lengths and integers, indefinite lengths and trailing bytes answer `null`. `x509ParseCertificate` and `x509ParseChain` parse and do not validate: there is no path validation, no extension is read or exposed, and no clock is consulted. `pemToDer` refuses a file in which any block, of any label, is malformed. `x509MintSelfSigned` refuses a common name that is not UTF-8 or holds a NUL, and its key and serial must come from `crypto.getRandomValues`, which the module cannot call itself without refusing wasm32 (X509-6) | X.690, RFC 5280, RFC 7468, RFC 5915, RFC 5208 / 5958; certificates checked against OpenSSL |
66
67
 
67
68
  Three rules hold across the modules:
@@ -136,19 +137,19 @@ reads one, and it still takes its randomness as an argument.
136
137
 
137
138
  | Module | What it is | Reproduces |
138
139
  | --- | --- | --- |
139
- | [`net/tls.ts`](./net/tls.ts) | The server side of a TLS 1.3 handshake, ClientHello through the client's Finished. `new TlsServer(config, serverRandom, x25519Private)`, then `receive(level, buf, off, len)` with what the client sent, `takeOutput(level)` for what to send, and `signatureInput()` / `sign(signature)` for the CertificateVerify hand-off (`tlsSignEcdsaP256` signs for a P-256 key, DER-encoded). A `TlsServerConfig` names the certificate chain, the signature scheme, the ALPN protocols, whether the carrier is QUIC, the server's transport parameters, and extensions to pass through in EncryptedExtensions. The suites are `TLS_AES_128_GCM_SHA256`, `TLS_CHACHA20_POLY1305_SHA256` and `TLS_AES_256_GCM_SHA384` (in the client's order), the group x25519 (one HelloRetryRequest when the client offers it without a share), and `server_name`, ALPN and `quic_transport_parameters` are read; PSKs, 0-RTT, NewSessionTicket and client authentication are not. Every refusal is the alert to send — `receive` and `sign` answer 0 or a `TLS_ALERT_*` — never a panic, and a low-order x25519 share is refused. `restart(serverRandom, x25519Private)` hands a server's buffers, made once, to the slot's next connection, so a handshake allocates nothing that outlives it under any suite (TLS-3) | RFC 8446 §4, §7; RFC 8448 §3 byte for byte, and §5's transcript |
140
+ | [`net/tls.ts`](./net/tls.ts) | The server side of a TLS 1.3 handshake, ClientHello through the client's Finished. `new TlsServer(config, serverRandom, x25519Private)`, then `receive(level, buf, off, len)` with what the client sent, `takeOutput(level)` for what to send, and `signatureInput()` / `sign(signature)` for the CertificateVerify hand-off (`tlsSignEcdsaP256` signs for a P-256 key, DER-encoded; a `TlsP256Signer`, made once, gives the same DER from its `sign(priv, content)` and keeps nothing per signature). A `TlsServerConfig` names the certificate chain, the signature scheme, the ALPN protocols, whether the carrier is QUIC, the server's transport parameters, and extensions to pass through in EncryptedExtensions. The suites are `TLS_AES_128_GCM_SHA256`, `TLS_CHACHA20_POLY1305_SHA256` and `TLS_AES_256_GCM_SHA384` (in the client's order), the group x25519 (one HelloRetryRequest when the client offers it without a share), and `server_name`, ALPN and `quic_transport_parameters` are read; PSKs, 0-RTT, NewSessionTicket and client authentication are not. Every refusal is the alert to send — `receive` and `sign` answer 0 or a `TLS_ALERT_*` — never a panic, and a low-order x25519 share is refused. `restart(serverRandom, x25519Private)` hands a server's buffers, made once, to the slot's next connection, so a handshake allocates nothing that outlives it under any suite (TLS-3) | RFC 8446 §4, §7; RFC 8448 §3 byte for byte, and §5's transcript |
140
141
  | [`net/tls/codec.ts`](./net/tls/codec.ts) | The handshake messages as bytes: `tlsReadClientHello` into a reusable `TlsClientHelloView` of windows into the message, and `tlsParseClientHello` into a `TlsClientHello` of copies (structure checked, an alert on a malformed or duplicated field); each message's `tlsWrite*` writer into the caller's array at an offset, and `tlsEncodeServerHello`, `tlsEncodeHelloRetryRequest`, `tlsEncodeEncryptedExtensions`, `tlsEncodeCertificate`, `tlsEncodeCertificateVerify`, `tlsEncodeFinished` and `tlsCertificateVerifyContent`. The wire's constants: handshake and extension types, versions, the x25519 group, the two signature schemes, and the `TLS_ALERT_*` descriptions | RFC 8446 §4, RFC 6066 §3, RFC 7301 §3.1, RFC 9001 §8.2 |
141
142
  | [`net/tls/schedule.ts`](./net/tls/schedule.ts) | The key schedule over SHA-256 or SHA-384: `tlsEarlySecret`, `tlsHandshakeSecret`, `tlsMasterSecret`, `tlsDeriveSecret`, `tlsExpandLabel`, `tlsFinishedVerifyData`, and `tlsTrafficKey` / `tlsTrafficIv` for a record layer, with in-place forms over an `HkdfScratch` (`tlsDeriveSecretInto`, `tlsMasterSecretInto`, `tlsFinishedVerifyDataInto`, `tlsTrafficKeysInto`). `TlsTranscript` is the running transcript hash, with HelloRetryRequest's `message_hash` rule, `reset` for the next connection and `hashInto` the caller's array. The suite constants and `tlsSuiteHashLength` / `tlsSuiteKeyLength` | RFC 8446 §4.4.1, §7.1, §7.3 |
142
143
  | [`net/tls/record.ts`](./net/tls/record.ts) | The record layer (RFC 8446 §5): `TlsRecordReader` cuts a stream into records in a buffer allocated once, refusing a header of unknown type or a body past 2^14 + 256 bytes before the body is waited for; `TlsRecordProtection` is one direction, cleartext until `install(suite, secret)`, then `seal` and `open` a record into the caller's buffer — the nonce from the sequence number, the header as additional data, the inner content type and zero padding found without branching on the padding. `tlsNextTrafficSecret` is KeyUpdate's next secret. Every refusal is the alert, negated | RFC 8448 §3's nine records byte for byte; ChaCha20-Poly1305, AES-256-GCM-SHA384 and padding against Python's `cryptography` |
143
144
  | [`net/tls/record-server.ts`](./net/tls/record-server.ts) | `TlsRecordServer`: a TLS 1.3 server over a byte stream, sans-IO, wrapping a `TlsServer` in records. `receive` the client's bytes, send `output[outputStart .. outputEnd)` and `consume` what was sent, `read` and `write` application data, `sign` when asked, `keyUpdate`, `close`; `interest()` answers what it wants next as `TLS_RECORD_*` bits. It adds the compatibility `change_cipher_spec`, alerts both ways, KeyUpdate both ways and on a schedule, a limit of `TLS_RECORD_IDLE_LIMIT` (16) records in a row that carry nothing, a cap of `TLS_RECORD_MAX_KEY_UPDATES` (64) KeyUpdates from the client, and fixed buffers per connection; the handshake is sealed straight out of `TlsServer`'s buffers | RFC 8448 §3 replayed at every cut of the client's stream |
144
145
  | [`net/tls-tcp.ts`](./net/tls-tcp.ts) | `TlsTcpServer`: TLS 1.3 over `nish:net` TCP for a program that owns its loop — a pool of `TlsRecordServer` slots sized at start-up, `accept(serverRandom, ephemeralPrivate)` into a free one, `readable` / `writable` when the loop says so, `signP256` with a `Secret` key, and `read`, `write`, `shutdown` and `close` answering as `nish:net`'s calls do. `accept` copies the server random and the ephemeral key into the slot, wipes the caller's key array, and refuses either with -22 when it is not 32 bytes. Every call answers the slot's interest, whose low two bits are `pollModify`'s events. Each slot keeps one `TlsServer`, restarted by `accept`, so a thousand connections through the pool leave the arena where the first did, under any suite (TLS-3). Native only, as `nish:net` is | openssl s_client under all three suites and curl, from `tests/run.js`; RFC 8448 §3 over loopback from a Nish client |
145
- | [`net/quic-packet.ts`](./net/quic-packet.ts) | QUIC version 1's packets (WP34 Q1). The variable-length integer (`quicVarintPush`, `quicVarintPushSized`, `quicVarintRead`, `quicVarintLength`, `quicVarintSize`); packet-number length choice and recovery (`quicPacketNumberLength`, `quicPacketNumberDecode`); the long and short headers (`quicLongHeader`, `quicShortHeader`, and `quicParseHeader` into a `QuicHeader`, whose `end` is where the next coalesced packet starts); the Initial secrets from the client's first DCID (`quicInitialSecrets`); the packet keys from a traffic secret (`quicKeys` into a `QuicKeys`, for AES-128-GCM, AES-256-GCM and ChaCha20-Poly1305) and a key update, its secret (`quicKeyUpdateSecret`) and the next generation's keys with the header-protection key kept (`quicKeysUpdate`); packet and header protection (`quicSealPacket`, and `quicOpenPacket` or its two halves `quicRemoveHeaderProtection` and `quicDecryptPacket` into a `QuicPacket`); and, for a connection that allocates nothing per packet (WP34 Q4), `quicParseHeaderInto` (windows, no copy), `quicPutLongHeader` / `quicPutShortHeader` / `quicVarintPut` in place, `quicSealInPlace`, and `quicUnprotectHeader` / `quicDecryptPayload`, which store only numbers into the `QuicPacket`; and the Retry integrity tag (`quicRetryIntegrityTag`, `quicRetryPacket`, `quicRetryVerify`). Every parse is bounds-checked against the datagram and answers a `QUIC_ERR_*` code rather than panicking: a truncated header, a clear fixed bit, another version (with its connection IDs read, for Version Negotiation), a connection ID over 20 bytes, a Length past the datagram, a packet too short to sample, a payload that does not authenticate, and reserved bits set in one that does; keys `quicKeys` did not make are `QUIC_ERR_KEYS`, a caller's error rather than a forgery. Long headers always carry a two-byte Length, so a header's size does not move with its payload, and `quicSealPacket` refuses a header whose Length or packet number disagrees with what it seals. The builders answer `null` for an argument out of range. Its Handshake and 1-RTT keys are not wiped yet ([QUIC-1](../docs/security/quic.md)) | RFC 9000 §16, §17 and Appendix A.1–A.3; RFC 9001 §5 and Appendix A.1–A.5 (client and server Initial, Retry, ChaCha20 short header) |
146
+ | [`net/quic-packet.ts`](./net/quic-packet.ts) | QUIC version 1's packets (WP34 Q1). The variable-length integer (`quicVarintPush`, `quicVarintPushSized`, `quicVarintRead`, `quicVarintLength`, `quicVarintSize`); packet-number length choice and recovery (`quicPacketNumberLength`, `quicPacketNumberDecode`); the long and short headers (`quicLongHeader`, `quicShortHeader`, and `quicParseHeader` into a `QuicHeader`, whose `end` is where the next coalesced packet starts); the Initial secrets from the client's first DCID (`quicInitialSecrets`); the packet keys from a traffic secret (`quicKeys` into a `QuicKeys`, for AES-128-GCM, AES-256-GCM and ChaCha20-Poly1305) and a key update, its secret (`quicKeyUpdateSecret`) and the next generation's keys with the header-protection key kept (`quicKeysUpdate`); the same four in place for a connection slot, into a `QuicKeysSlot` it makes once and over its `HkdfScratch` (`quicInitialSecretsInto`, `quicKeysInto`, `quicKeyUpdateSecretInto`, `quicKeysUpdateInto`; `QuicKeysSlot.wipe` zeroes the keys and AES schedules); packet and header protection (`quicSealPacket`, and `quicOpenPacket` or its two halves `quicRemoveHeaderProtection` and `quicDecryptPacket` into a `QuicPacket`); and, for a connection that allocates nothing per packet (WP34 Q4), `quicParseHeaderInto` (windows, no copy), `quicPutLongHeader` / `quicPutShortHeader` / `quicVarintPut` in place, `quicSealInPlace`, and `quicUnprotectHeader` / `quicDecryptPayload`, which store only numbers into the `QuicPacket`; and the Retry integrity tag (`quicRetryIntegrityTag`, `quicRetryPacket`, `quicRetryVerify`). Every parse is bounds-checked against the datagram and answers a `QUIC_ERR_*` code rather than panicking: a truncated header, a clear fixed bit, another version (with its connection IDs read, for Version Negotiation), a connection ID over 20 bytes, a Length past the datagram, a packet too short to sample, a payload that does not authenticate, and reserved bits set in one that does; keys `quicKeys` did not make are `QUIC_ERR_KEYS`, a caller's error rather than a forgery. Long headers always carry a two-byte Length, so a header's size does not move with its payload, and `quicSealPacket` refuses a header whose Length or packet number disagrees with what it seals. The builders answer `null` for an argument out of range. Its Handshake and 1-RTT keys are not wiped yet ([QUIC-1](../docs/security/quic.md)) | RFC 9000 §16, §17 and Appendix A.1–A.3; RFC 9001 §5 and Appendix A.1–A.5 (client and server Initial, Retry, ChaCha20 short header) |
146
147
  | [`net/quic-frame.ts`](./net/quic-frame.ts) | QUIC version 1's frames (WP34 Q2). `quicParseFrame` reads any frame of RFC 9000 §19, 0x00 to 0x1e, into a reused `QuicFrame`, leaving a CRYPTO or STREAM frame's data in the packet (`dataStart`, `dataLength`) and answering 0 or the transport error the RFC names: FRAME_ENCODING_ERROR for a truncated field, a length past the packet, an ACK range below zero, an offset past 2^62 − 1, a stream count past 2^60, an empty NEW_TOKEN or a bad NEW_CONNECTION_ID, and PROTOCOL_VIOLATION for a frame type spelled in more bytes than it needs. The writers (`quicPushAck`, `quicPushCrypto`, `quicPushStream`, `quicPushValue`, `quicPushStreamValue`, `quicPushStreamError`, `quicPushNewConnectionId`, `quicPushPathData`, `quicPushConnectionClose`, `quicPushNewToken`, `quicPushTypeOnly`, `quicPushPadding`) answer `false` rather than write a frame that would not parse; `quicCryptoOverhead` and `quicStreamOverhead` size a frame's header before it is written. `quicFrameAllowed` is §12.4's table of which frames each packet type carries, `quicFrameAckEliciting` §13.2's rule, and `QUIC_ERROR_*` the transport error codes of §20.1. RFC 9221's DATAGRAM frames (0x30, 0x31) are read too (WP34 Q4); every writer has a `put` form (`quicPutAck`, `quicPutStream`, …, `quicPutDatagram`) that writes in place into a caller's buffer before an end and answers the offset past it, which the `push` form calls, and NEW_CONNECTION_ID's ID and token are left in the packet as windows | RFC 9000 §12.4, §19, §20; RFC 9221 §4; RFC 9001 A.2's and A.3's frames |
147
- | [`net/quic-conn-params.ts`](./net/quic-conn-params.ts) | Transport parameters (RFC 9000 §18): a `QuicTransportParameters` that starts at every default, `quicEncodeTransportParameters` (a parameter only when it is not its default) and `quicParseTransportParameters`, which refuses with TRANSPORT_PARAMETER_ERROR, naming the parameter, a parameter that does not frame, a known one sent twice, a varint-valued one that is not exactly one varint, a server-only one from a client, and every value §18.2 bounds. Unknown parameters are skipped. RFC 9221's `max_datagram_frame_size` is read and written too (WP34 Q4) | RFC 9000 §7.4, §18; RFC 9221 §3; RFC 9001 A.2's client parameters |
148
+ | [`net/quic-conn-params.ts`](./net/quic-conn-params.ts) | Transport parameters (RFC 9000 §18): a `QuicTransportParameters` that starts at every default, `quicEncodeTransportParameters` (a parameter only when it is not its default) and `quicParseTransportParameters`, with in-place forms that refill a slot's own array and object (`quicEncodeTransportParametersInto`, `quicParseTransportParametersInto`), which refuses with TRANSPORT_PARAMETER_ERROR, naming the parameter, a parameter that does not frame, a known one sent twice, a varint-valued one that is not exactly one varint, a server-only one from a client, and every value §18.2 bounds. Unknown parameters are skipped. RFC 9221's `max_datagram_frame_size` is read and written too (WP34 Q4) | RFC 9000 §7.4, §18; RFC 9221 §3; RFC 9001 A.2's client parameters |
148
149
  | [`net/quic-conn-ack.ts`](./net/quic-conn-ack.ts) | `QuicAckRanges`: the packet numbers one space received, as at most 32 disjoint ranges, for duplicate detection (`record`, `contains`) and for the ACK frame written from them (`pushAck`, or `putAck` in place). Past 32 ranges the lowest is dropped and a floor raised over it, below which every packet number reads as already received | RFC 9000 §12.3, §13.2, §19.3 |
149
150
  | [`net/quic-conn-cid.ts`](./net/quic-conn-cid.ts) | `QuicCidTable`: the connection IDs a connection issued (routed by `ownsLocal`, retired by the peer with `retireLocal`) and the ones its peer issued (`addPeer`, with Retire Prior To retiring older ones into `retirePending`, and `currentPeer` the one sent to). A reused sequence number, an ID under two sequence numbers and retiring the ID a packet was sent to are PROTOCOL_VIOLATION; more active IDs than advertised, or more owed retirements than twice that, CONNECTION_ID_LIMIT_ERROR. Its slots are fixed when it is made (WP34 Q4): an ID is copied into one, `addPeerAt` / `ownsLocalAt` / `retireLocal` read it from the packet in place, and `reset` empties the table for the next connection | RFC 9000 §5.1, §19.15, §19.16 |
150
- | [`net/quic.ts`](./net/quic.ts) | The server side of a QUIC connection (WP34 Q2): `new QuicConnection(config, entropy)`, `receive(datagram, now)`, `signatureInput()` / `sign(signature)`, `takeDatagram(now)` until `null`, `deadline()` / `handleTimer(now)` for its timers, and `readStream()` / `writeStream(id, data, fin)` once connected; `updateKeys()`, `close(code)` and `release()`; `acceptRetry(originalDcid, retryScid)` for a connection a Retry token opened; `quicStatelessResetToken(key, cid)`. TLS 1.3 runs over CRYPTO frames at the Initial, Handshake and 1-RTT levels (`TlsServer` with `quic` set), the keys of each level come from `nish/net/quic-packet` as the secrets appear and are discarded and wiped as RFC 9001 §4.9 says, transport parameters go both ways and the client's are checked, every ack-eliciting packet is acknowledged in its space, the client gets new connection IDs once the handshake is confirmed, PATH_CHALLENGE is answered, and an ack-eliciting Initial is padded to 1200 bytes and held to three times what the unvalidated client sent. A protocol violation closes the connection with the RFC's transport error, a TLS alert with CRYPTO_ERROR; a packet that cannot be used is dropped and counted. The idle timeout is the smaller of both sides' `max_idle_timeout`, at least three probe timeouts, and closes silently; key update runs both ways, the client's followed (at most `QUIC_CONN_MAX_KEY_UPDATES`, QUIC-4) and the server's started by `updateKeys()`, with the next read keys derived in advance and the previous kept a probe timeout for reordered packets; every connection ID carries the stateless reset token the configuration's static key gives it. Loss recovery is `nish/net/quic-recovery`'s (WP34 Q3): what a lost packet carried (CRYPTO, STREAM, HANDSHAKE_DONE and the connection-ID frames) is sent again, a probe timeout sends a probe, and NewReno's window holds back everything but ACKs and probes. Streams (WP34 Q4) are `nish/net/quic-stream`'s, both ways and both kinds, through `nextStreamEvent()`, `openStream(bidirectional)`, `streamRead(id, buf, at, length)`, `streamWrite(id, buf, from, length, fin)`, `streamReset(id, code)` and `streamStopSending(id, code)`; RFC 9221's datagrams through `sendDatagram(buf, from, length)`, `readDatagram(buf, at, cap)` and `maxDatagramPayload()` when `config.maxDatagramFrameSize` offers them. Nothing is allocated per packet (QUIC-3): `receiveWindow(buf, off, len, now)` and `takeDatagramInto(out, at, now)` open and build packets in place inside an arena block, and `reset(entropy)` makes the connection a reusable slot for the next client. Not here yet: the AEAD limits of RFC 9001 §6.6 (QUIC-6) | RFC 9000 §10.1, §10.3.2; RFC 9001 §4, §5.7, §6, §8; RFC 9221; a handshake and stream echo, and an update each way, against aioquic, recorded and replayed byte for byte |
151
- | [`net/quic-listener.ts`](./net/quic-listener.ts) | What a QUIC server answers to a datagram no connection owns (WP34 Q2, second part): `new QuicListener(config, entropy)` and `handle(datagram, address, now)`, which answers a `QuicListenerAnswer` whose `kind` is `QUIC_LISTEN_DROP`, `_ACCEPT` (make a `QuicConnection`; after a Retry, `acceptRetry` it with the answer's IDs), or a `reply` to send: `_VERSION_NEGOTIATION` for another version in a full-sized datagram, `_RETRY` with an address-validation token when `config.retry` is set, `_INVALID_TOKEN` for a Retry token that is forged, moved, expired or for another ID, and `_STATELESS_RESET` for a short header to an ID the server has no connection for. A token is 128 bits of HMAC-SHA256 under `config.retryTokenKey` over its issue time, the original DCID, the new ID and the client's address and port, good for `config.retryTokenLifetime`. A reset ends in `quicStatelessResetToken` of the packet's ID, is always shorter than the datagram it answers and never under 21 bytes, and is rate limited. Its own unpredictable bytes come from an HMAC generator seeded once by the caller. For the carrier's send loop, `quicListenerTakePaced(conn, now)` takes a connection's next datagram only when its pacer has the credit, and `quicListenerPaceTime(conn, now)` says when it will (WP34 Q3); `quicListenerTakeFlight(conn, now, buf, at, cap, flight)` writes a paced flight back to back for one GSO send, with the `UDP_SEGMENT` size in its `QuicFlight`, and `quicListenerReceiveSegments` hands a connection a GRO receive segment by segment, in place (WP34 Q4) | RFC 9002 §7.7; RFC 9000 §5.2.2, §6, §8.1, §10.3, §17.2.1, §17.2.5; Version Negotiation, Retry, a stateless reset and the idle timeout against aioquic, recorded and replayed byte for byte |
151
+ | [`net/quic.ts`](./net/quic.ts) | The server side of a QUIC connection (WP34 Q2): `new QuicConnection(config, entropy)`, `receive(datagram, now)`, `signatureInput()` / `sign(signature)`, `takeDatagram(now)` until `null`, `deadline()` / `handleTimer(now)` for its timers, and `readStream()` / `writeStream(id, data, fin)` once connected; `updateKeys()`, `close(code)` and `release()`; `acceptRetry(originalDcid, retryScid)` for a connection a Retry token opened; `quicStatelessResetToken(key, cid)`. TLS 1.3 runs over CRYPTO frames at the Initial, Handshake and 1-RTT levels (`TlsServer` with `quic` set), the keys of each level come from `nish/net/quic-packet` as the secrets appear and are discarded and wiped as RFC 9001 §4.9 says, transport parameters go both ways and the client's are checked, every ack-eliciting packet is acknowledged in its space, the client gets new connection IDs once the handshake is confirmed, PATH_CHALLENGE is answered, and an ack-eliciting Initial is padded to 1200 bytes and held to three times what the unvalidated client sent. A protocol violation closes the connection with the RFC's transport error, a TLS alert with CRYPTO_ERROR; a packet that cannot be used is dropped and counted. The idle timeout is the smaller of both sides' `max_idle_timeout`, at least three probe timeouts, and closes silently; key update runs both ways, the client's followed (at most `QUIC_CONN_MAX_KEY_UPDATES`, QUIC-4) and the server's started by `updateKeys()`, with the next read keys derived in advance and the previous kept a probe timeout for reordered packets; every connection ID carries the stateless reset token the configuration's static key gives it. Loss recovery is `nish/net/quic-recovery`'s (WP34 Q3): what a lost packet carried (CRYPTO, STREAM, HANDSHAKE_DONE and the connection-ID frames) is sent again, a probe timeout sends a probe (two datagrams in the Initial and Handshake spaces, `QUIC_CONN_HANDSHAKE_PROBES`, each with the oldest CRYPTO data not acknowledged), a client that shows it lacks the handshake flight (an Initial repeating CRYPTO data, an ack-eliciting Handshake packet) gets it again at once, up to `QUIC_CONN_EARLY_RESENDS` times a connection (QUIC-8), and NewReno's window holds back everything but ACKs and probes. Streams (WP34 Q4) are `nish/net/quic-stream`'s, both ways and both kinds, through `nextStreamEvent()`, `openStream(bidirectional)`, `streamRead(id, buf, at, length)`, `streamWrite(id, buf, from, length, fin)`, `streamReset(id, code)` and `streamStopSending(id, code)`; RFC 9221's datagrams through `sendDatagram(buf, from, length)`, `readDatagram(buf, at, cap)` and `maxDatagramPayload()` when `config.maxDatagramFrameSize` offers them. Nothing is allocated per packet (QUIC-3): `receiveWindow(buf, off, len, now)` and `takeDatagramInto(out, at, now)` open and build packets in place inside an arena block, and `reset(entropy)` makes the connection a reusable slot for the next client. Not here yet: the AEAD limits of RFC 9001 §6.6 (QUIC-6) | RFC 9000 §10.1, §10.3.2; RFC 9001 §4, §5.7, §6, §8; RFC 9221; a handshake and stream echo, and an update each way, against aioquic, recorded and replayed byte for byte |
152
+ | [`net/quic-listener.ts`](./net/quic-listener.ts) | What a QUIC server answers to a datagram no connection owns (WP34 Q2, second part): `new QuicListener(config, entropy)` and `handleWindow(buf, at, len, address, now, answer)`, which reads the datagram `buf[at .. at + len)` in place and fills `answer`, a `QuicListenerAnswer` made once with `QUIC_LISTENER_ANSWER_SIZE` of room and refilled in it, so a datagram costs no arena memory whatever it is answered with (H3-3); `handle(datagram, address, now)` does the same over a whole array into an answer of its own. An answer's `kind` is `QUIC_LISTEN_DROP`, `_ACCEPT` (make a `QuicConnection`; after a Retry, `acceptRetry` it with the answer's IDs), or a `reply` to send: `_VERSION_NEGOTIATION` for another version in a full-sized datagram, `_RETRY` with an address-validation token when `config.retry` is set, `_INVALID_TOKEN` for a Retry token that is forged, moved, expired or for another ID, and `_STATELESS_RESET` for a short header to an ID the server has no connection for. A token is 128 bits of HMAC-SHA256 under `config.retryTokenKey` over its issue time, the original DCID, the new ID and the client's address and port, good for `config.retryTokenLifetime`. A reset ends in `quicStatelessResetToken` of the packet's ID, is always shorter than the datagram it answers and never under 21 bytes, and is rate limited. Its own unpredictable bytes come from an HMAC generator seeded once by the caller. For the carrier's send loop, `quicListenerTakePaced(conn, now)` takes a connection's next datagram only when its pacer has the credit, and `quicListenerPaceTime(conn, now)` says when it will (WP34 Q3); `quicListenerTakeFlight(conn, now, buf, at, cap, flight)` writes a paced flight back to back for one GSO send, with the `UDP_SEGMENT` size in its `QuicFlight`, and `quicListenerReceiveSegments` hands a connection a GRO receive segment by segment, in place (WP34 Q4) | RFC 9002 §7.7; RFC 9000 §5.2.2, §6, §8.1, §10.3, §17.2.1, §17.2.5; Version Negotiation, Retry, a stateless reset and the idle timeout against aioquic, recorded and replayed byte for byte |
152
153
  | [`net/quic-recovery.ts`](./net/quic-recovery.ts) | QUIC loss detection and congestion control (WP34 Q3), sans-IO, for the server side: `QuicRecovery` holds one `QuicSentPackets` ring per packet number space (16 packets for Initial and Handshake, 128 for Application Data, fixed when it is made); `onPacketSent(space, pn, size, now)` records an ack-eliciting packet and answers its slot, `onAck(space, ranges, count, ackDelay, now)` reads an ACK frame's ranges and lists the slots it acknowledged and showed lost (`acked`, `lost`), `deadline(amplificationBlocked)` and `onTimeout(now, amplificationBlocked)` run the time-threshold loss timer and the probe timeout with its backoff, `discardSpace` drops a space whose keys are gone, `evictOldest` frees a slot for a probe. The RTT estimate (`latestRtt`, `smoothedRtt`, `rttVar`, `minRtt`, `probeTimeout()`), loss by packet threshold 3 and time threshold 9/8, NewReno (slow start, one halving per recovery period, congestion avoidance, the 2400-byte floor, persistent congestion) and a pacer at 5/4 of the window per smoothed RTT (`pacerDelay`, `onPaced`), all in integer milliseconds and bytes | RFC 9002 §5–7, Appendices A and B, worked by hand in `net_quic_recovery` |
153
154
  | [`net/quic-stream.ts`](./net/quic-stream.ts) | QUIC streams (WP34 Q4), sans-IO, which `nish/net/quic` drives: `QuicStreams` holds one `QuicStream` slot per stream that may be open at once (the client's bidirectional and unidirectional limits and the server's own), each with a fixed receive and send buffer, found by ID through a hash index. The send and receive state machines of §3 (`QUIC_SEND_*`, `QUIC_RECV_*`), reassembly into the receive buffer, flow control per stream and for the connection both ways (MAX_STREAM_DATA and MAX_DATA raised as the application reads, STREAM_DATA_BLOCKED and DATA_BLOCKED once per limit), the stream limits (MAX_STREAMS raised as the client's streams finish, STREAMS_BLOCKED at the client's), STOP_SENDING answered with RESET_STREAM, and the refusals STREAM_STATE_ERROR, STREAM_LIMIT_ERROR, FLOW_CONTROL_ERROR and FINAL_SIZE_ERROR. The application's answers are `QUIC_STREAM_OK`, `QUIC_STREAM_END` and `QUIC_STREAM_ERR_*` | RFC 9000 §2–§4, §19.4–§19.14 |
154
155
  | [`net/quic-datagram.ts`](./net/quic-datagram.ts) | `QuicDatagramQueue` (WP34 Q4): a fixed ring of DATAGRAM payloads, one each way per connection, copied in (`push`) and out (`pop`, or `putFrame` as a DATAGRAM frame), refusing past a full ring and counting what it drops | RFC 9221 §5 |