@amritk/nish-aarch64-darwin 0.34.0 → 0.35.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/INSTALL.md +16 -12
- package/bin/nish +0 -0
- package/docs/AI.md +77 -14
- package/package.json +1 -1
- package/runtime/async.d.mts +49 -0
- package/runtime/async.mjs +27 -0
- package/runtime/nish.d.ts +72 -0
- package/runtime/nish.h +6 -2
- package/runtime/nish.mjs +4 -0
- package/runtime/runtime-net.c +22 -0
- package/runtime/runtime-shm.c +93 -2
- package/std/README.md +131 -131
- package/std/bytes.ts +28 -5
- package/std/crypto/base64url.ts +42 -21
- package/std/crypto/bcrypt.ts +222 -183
- package/std/net/http1-codec.ts +262 -92
- package/std/net/http1-server.ts +87 -9
- package/std/net/http3-server.ts +30 -6
- package/std/net/postgres.ts +129 -40
- package/std/net/quic-recovery.ts +36 -8
- package/std/net/quic.ts +664 -59
- package/std/sync/ring.ts +236 -16
- package/std/sync/shm.ts +29 -0
package/std/README.md
CHANGED
|
@@ -5,52 +5,52 @@ here and — with five exceptions, `threads.ts`, `collections.ts`, `map.ts`,
|
|
|
5
5
|
`secret.ts` and `math.ts`, and one function, `text.ts`'s `indexOfAny` — nothing the compiler
|
|
6
6
|
knows about: a module in this directory is an ordinary Nish source file, compiled as part of
|
|
7
7
|
whatever program imports it, and subject to the same rules as `examples/` or
|
|
8
|
-
`src/` ([`docs/LANGUAGE.md`](
|
|
8
|
+
`src/` ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md) is the style guide).
|
|
9
9
|
|
|
10
10
|
| Module | What it is |
|
|
11
11
|
| --- | --- |
|
|
12
|
-
| [`testing.ts`](
|
|
13
|
-
| [`text.ts`](
|
|
14
|
-
| [`math.ts`](
|
|
15
|
-
| [`sort.ts`](
|
|
16
|
-
| [`regex.ts`](
|
|
17
|
-
| [`json.ts`](
|
|
18
|
-
| [`json-tokenizer.ts`](
|
|
19
|
-
| [`json-typed.ts`](
|
|
20
|
-
| [`json-value.ts`](
|
|
21
|
-
| [`json-pointer.ts`](
|
|
22
|
-
| [`json-equal.ts`](
|
|
23
|
-
| [`json-number.ts`](
|
|
24
|
-
| [`json-format.ts`](
|
|
25
|
-
| [`yaml.ts`](
|
|
26
|
-
| [`yaml-json.ts`](
|
|
27
|
-
| [`yaml-lines.ts`](
|
|
28
|
-
| [`yaml-write.ts`](
|
|
29
|
-
| [`pair.ts`](
|
|
30
|
-
| [`collections.ts`](
|
|
31
|
-
| [`map.ts`](
|
|
32
|
-
| [`bytes.ts`](
|
|
33
|
-
| [`base64.ts`](
|
|
34
|
-
| [`random.ts`](
|
|
35
|
-
| [`cli.ts`](
|
|
36
|
-
| [`cli-args.ts`](
|
|
37
|
-
| [`cli-flags.ts`](
|
|
38
|
-
| [`sqlite.ts`](
|
|
39
|
-
| [`base64.ts`](
|
|
40
|
-
| [`term.ts`](
|
|
41
|
-
| [`term-probe.ts`](
|
|
42
|
-
| [`term-rate.ts`](
|
|
43
|
-
| [`term-pixels.ts`](
|
|
44
|
-
| [`term-io.ts`](
|
|
45
|
-
| [`region.ts`](
|
|
46
|
-
| [`secret.ts`](
|
|
47
|
-
| [`threads.ts`](
|
|
48
|
-
| [`sync/shm.ts`](
|
|
49
|
-
| [`sync/ring.ts`](
|
|
12
|
+
| [`testing.ts`](https://github.com/amritk/nish/blob/main/std/testing.ts) | a test runner: a `Suite` a program drives with straight-line assertions, printing the `PASS` / `FAIL` / `SKIP` lines the repository's own harness prints, and answering the exit code |
|
|
13
|
+
| [`text.ts`](https://github.com/amritk/nish/blob/main/std/text.ts) | the string operations a program would otherwise write inline: `splitLines`, `splitWhitespace`, `trim` and its halves, `contains`, `replaceAll`, `firstDifference` over two arrays of lines, `codePointLength`, the number of Unicode code points (what JSON Schema's `minLength` counts, where `.length` counts UTF-8 bytes natively and UTF-16 units under Node; it answers the same on both by asking which it is reading), and `indexOfAny(text, bytes, from)`, the first index at or after `from` whose byte is one of 1 to 16 ASCII `bytes`. Its walker is the meaning and what runs under Node; natively the compiler replaces the one call to it with the runtime kernel `nish_str_index_of_any`, which reads 16 or 32 bytes at a time and is variable-time ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#byte-search-indexofany-in-nishtext)). `tests/link/std_text_index_of_any*` are its programs |
|
|
14
|
+
| [`math.ts`](https://github.com/amritk/nish/blob/main/std/math.ts) | `Math.tan`, `Math.atan2` and `Math.hypot` on `f64`, ported from fdlibm (`LICENSE-fdlibm`) so that every target, wasm included, answers alike without libm. A program never imports it: a call to one of the three loads it, and the compiler copies the functions the call reaches into the calling module as `internal` ones. JavaScript's answer for every NaN, signed zero and infinity, and within one ulp elsewhere (0.83, 0.84 and 0.93 ulp measured against mpmath). `tests/cases/math_std_edges` pins the special values |
|
|
15
|
+
| [`sort.ts`](https://github.com/amritk/nish/blob/main/std/sort.ts) | `xs.sort(cmp)` on any array: `sortWith`, a stable merge sort over runs of 32 sorted by insertion — one scratch array released before it returns, a pair of runs already in order copied without merging, no bounds check left in it — so its order is element for element Node's `Array.prototype.sort`. Like `math.ts` it is one the compiler knows about: a program never imports it, a `.sort(cmp)` call loads it, the checker rewrites the call to `sortWith`, and each comparator's instantiation — the comparator called directly, no function pointer — is copied into the module that sorts. Under Node the call stays the platform's own. `tests/cases/arr_sort_cmp*` are its programs, a 10,000-element sort with ties among them |
|
|
16
|
+
| [`regex.ts`](https://github.com/amritk/nish/blob/main/std/regex.ts) | Regular expressions in JavaScript's syntax with the `u` flag's strictness, matched by a Pike VM in time linear in the input whatever the pattern; finding every match is one search per match, so `findAll` and its kin can be quadratic when a long branch is preferred to a short one that wins ([`docs/security/regex.md`](../docs/security/regex.md)). `regexCompile(pattern, flags)` answers a `Regex` or the error text; `test`, `exec` into a reused `RegexMatch`, `find`, `findAll`, `count`, `replace` and `replaceAll` with `$&`, `$1` and `$<name>`, and `split`. Flags `i`, `m`, `s` and `u`; `i` folds by Unicode's simple case folding, ASCII letters only to ASCII. `\p{…}` and `\P{…}` take Unicode 16.0's General_Category, Script, Script_Extensions and binary properties by JavaScript's names, from tables `scripts/gen-regex-unicode.mjs` generates into the module and checks against Node's `RegExp`; backreferences and lookaround are refused at compile time. Offsets are UTF-8 bytes. A search allocates nothing beyond its answer: the thread lists and capture tables are the `Regex`'s, so one `Regex` serves one thread at a time. A leading literal is found with `indexOf`, a small leading set with `indexOfAny`. `tests/link/std_regex` and `tests/link/std_regex_unicode` check it against JavaScript's own answers |
|
|
17
|
+
| [`json.ts`](https://github.com/amritk/nish/blob/main/std/json.ts) | `jsonField(object, name)`: the value of one field of one flat JSON object, which is the shape the compiler's own `--json` diagnostics have. A reader and not a parser — it answers text, answers `null` for a field that is not there, and does not validate. `JsonWriter` is the other direction: any JSON value written into a caller-owned `u8[]`, commas and colons included, byte for byte what `JSON.stringify` writes, and allocating nothing per call except an `f64`'s digits, a temporary string the call gives back before it returns (`text()` and `jsonStringifyString` build strings by design); misuse and a full buffer set a sticky negative `JSON_WRITE_*` code in `failed` rather than panicking. `setIndent(n)` and `setIndentTab()` write what `JSON.stringify(v, null, n)` or `(v, null, "\t")` writes, with no allocation for the indent, and `setStrict(true)` makes NaN and the infinities the sticky `JSON_WRITE_NON_FINITE` instead of `null`; both modes are off by default and kept by `reset`. `jsonStringifyString(s)` is the string rule answered as a string. `stringBytes(bytes, off, len)` and `keyBytes(bytes, off, len)` write a window of a byte array as `string` and `key` write a string of those bytes, allocating nothing, and refuse a window outside the array with `JSON_WRITE_BAD_WINDOW` |
|
|
18
|
+
| [`json-tokenizer.ts`](https://github.com/amritk/nish/blob/main/std/json-tokenizer.ts) | `JsonTokenizer`: a validating, streaming tokenizer for RFC 8259 JSON held in a window of a `u8[]`. `next()` answers one `JSON_TOK_*` token at a time — object and array openers and closers, keys, strings, numbers and the three literals — then `JSON_TOK_END`, or a negative `JSON_TOK_ERR_*` code that sticks in `failed`, with the byte in `errorAt`. Strings are checked as UTF-8 and a lone surrogate escape is refused; `stringInto` decodes the current string into the caller's array and `keyEquals` compares it with a name, neither allocating. `numberF64` is correctly rounded; `isInteger` and `numberI64` read an integer that fits an `i64`, and `isU64` and `numberU64` one that fits a `u64`. `isIntegral` asks whether a number's value is whole however it is written (`1.0`, `1e2`, `1e400`), JSON Schema's "integer", from the text and at any exponent. `skipValue` steps over a value. `JsonLimits` caps nesting depth, string, number and total bytes, each with its own code; `reset` reuses the tokenizer, so a reused body keeps the arena flat. Duplicate keys are reported as tokens. No input panics and no byte outside the window is read; [`docs/security/json.md`](../docs/security/json.md) is its record, and `tests/link/std_json_tokenizer` runs JSONTestSuite and a seeded fuzz |
|
|
19
|
+
| [`json-typed.ts`](https://github.com/amritk/nish/blob/main/std/json-typed.ts) | `jsonDecode(d, out)` and `jsonEncode(w, value)`: one JSON object read into a class and written back, with code the compiler writes at each call from the class's fields — scalars (`u64` exact to 2^64 - 1), strings, numeric enums, nested classes, arrays of those, `Map<string, V>` of those and `\| null` — through `JsonDecoder` (a `JsonTokenizer` and a sticky error) and `JsonWriter`. A class's one `static readonly json` literal renames a field's key and makes a key optional, a missing one setting its field to the initializer; otherwise every field is required, each key once, an unknown key refused unless `ignoreUnknown`, and every outcome a sticky `JSON_DECODE_*` or `JSON_TOK_ERR_*` code with its byte in `errorAt`; decoding into a reused object reuses its objects, arrays and equal strings, so the arena stays flat (a map's entries are made anew, at most `maxMapKeys` over all the maps of one decode, 4,096 by default); a key is sent to its field by its length, then one byte. A `JsonNumber` field keeps a number's text byte for byte. Under Node the types are erased and both calls panic ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#typed-json-nishjson-typed), [`docs/security/json.md`](../docs/security/json.md)) |
|
|
20
|
+
| [`json-value.ts`](https://github.com/amritk/nish/blob/main/std/json-value.ts) | `JsonDocument`: any JSON document parsed into a tree of `JsonValue` handles (`i32`s into the document), read with `kind`, `boolean`, `string`, `stringEquals`, `length`, `at`, `keyAt` and `get`, built with the `new*` methods, `push` and `set`, and written back through `JsonWriter` with `write`, or as text with `stringify`. A number keeps its source text and is read on request as an `i64` or `u64` that fits (`isI64`, `isU64`), the nearest `f64`, or the text, and `isIntegral` asks whether its value is whole however it is written (`1.0`, `1e2`), so integers and decimals wider than any type round-trip byte for byte. Objects keep insertion order; a repeated key is `JSON_VALUE_ERR_DUPLICATE` unless `duplicates` is `JSON_VALUE_DUPLICATE_LAST`, `JSON.parse`'s rule. Every parse error and limit is the tokenizer's, with the depth held to the writer's 64 so that whatever parses writes back, nothing panics, and a value is in at most one container, so a tree built through `push` and `set` cannot hold itself. A handle carries the generation of the document it came from, so one kept across a `parse` or `clear` reads as nothing; `maxValues` (262,144 by default, `JSON_VALUE_ERR_VALUES` past it) bounds the memory a document keeps. `parseRanged` and `parseTextRanged` also record each value's byte range (`start`, `end`) and each key's (`keyStart`, `keyEnd`), 20 bytes more per value, and `setRange` and `setKeyRange` give built values theirs; after `parse` or `parseText` every range reads `-1`. Each `parse` reuses the document's arrays, so a reused document keeps the arena flat. `tests/link/std_json_value` runs JSONTestSuite through it ([`docs/security/json.md`](../docs/security/json.md)) |
|
|
21
|
+
| [`json-pointer.ts`](https://github.com/amritk/nish/blob/main/std/json-pointer.ts) | RFC 6901 JSON Pointers over a `JsonDocument`: `jsonPointerParse` (the decoded tokens, or `null`), `jsonPointerEscape` (one token, `~0` and `~1`), and `jsonPointerResolve` (the value, `JSON_POINTER_NONE` when the pointer names nothing, `JSON_POINTER_ERR_SYNTAX` when it is not one); `isJsonPointer` and `isRelativeJsonPointer` check JSON Schema's `json-pointer` and `relative-json-pointer` forms, reading the text once and allocating nothing. A resolve costs the pointer's length plus each object step's size. `tests/link/std_json_pointer` runs the suite's two format files through it |
|
|
22
|
+
| [`json-equal.ts`](https://github.com/amritk/nish/blob/main/std/json-equal.ts) | JSON Schema's value equality for `const`, `enum` and `uniqueItems`: `jsonEqual` across two documents (numbers by value, `1` is `1.0`; member order ignored; `true` is not `1`), `jsonEqualCanonical` (a text equal for exactly the equal values), `jsonEqualHash` (64-bit FNV-1a of it, unkeyed) and `jsonEqualFirstDuplicate` (`-1` when the items are unique). Nothing recurses, so any depth a document holds compares; a call costs the values' sizes plus sorting object keys. `tests/link/std_json_equal` runs the suite's `const`, `enum` and `uniqueItems` |
|
|
23
|
+
| [`json-number.ts`](https://github.com/amritk/nish/blob/main/std/json-number.ts) | Exact arithmetic on JSON number texts, never through `f64`: `jsonNumberCompare` (`-1`, `0`, `1`), `jsonNumberCanonical` (digits and exponent, equal for equal values) and `jsonNumberMultipleOf` (`1` or `0`). It refuses rather than guesses: `JSON_NUMBER_ERR_SYNTAX`, `JSON_NUMBER_ERR_EXPONENT` past `JSON_NUMBER_MAX_EXPONENT` (10^15), `JSON_NUMBER_ERR_DIVISOR` for zero, and `JSON_NUMBER_ERR_GAP` when `multipleOf` would take more than `JSON_NUMBER_MAX_STEPS` (2^20) digit steps (`1e1000000000` by `7`). A compare costs the texts' digits. `tests/link/std_json_number` runs the suite's `multipleOf`, bounds, `bignum` and `float-overflow` |
|
|
24
|
+
| [`json-format.ts`](https://github.com/amritk/nish/blob/main/std/json-format.ts) | `jsonFormatCheck(id, value)` and `jsonFormatValidate(name, value)`: JSON Schema draft 2020-12's `format` checks, answering `JSON_FORMAT_VALID`, `JSON_FORMAT_INVALID`, or `JSON_FORMAT_UNKNOWN` for `idn-email`, `idn-hostname`, `json-pointer`, `relative-json-pointer` and any other name; `jsonFormatId(name)` resolves a name once per schema. Each format is a predicate too: `isDateTime`, `isDate`, `isTime`, `isDuration` (RFC 3339, the duration ABNF as written, so `P1Y1D` is invalid), `isEmail` (RFC 5321), `isHostname` (RFC 1123), `isIpv4`, `isIpv6`, `isUuid`, `isUri`, `isUriReference`, `isIri`, `isIriReference` (RFC 3986 and 3987, walked here because `nish/net/url` refuses valid URIs a client should not follow), `isUriTemplate` (RFC 6570) and `isRegex` (what `nish/regex` compiles, so lookaround and backreferences are false negatives). All but `isRegex` are linear and allocate nothing. `tests/link/std_json_format` runs JSON-Schema-Test-Suite's `optional/format` through it |
|
|
25
|
+
| [`yaml.ts`](https://github.com/amritk/nish/blob/main/std/yaml.ts) | `parseYaml(text)` and `parseYamlWith(text, options)`: a YAML 1.2 stream parsed into a `YamlStream` of documents, nodes and diagnostics, struct of arrays with `i32` handles and `-1` for none. Every node and diagnostic keeps its byte range; a scalar is decoded (`text`) and resolved against the core schema (`type`) only when read. `keyText`, `pathNode`, `pathPair` and `pathMatch` find a key by path through aliases and `<<` merge keys (the map's own keys first, then each merged map in order). A parse never fails or panics on its input: errors and warnings carry the `YAML_ERR_*` codes other YAML tooling reports (`yamlErrorName`), with `maxDepth`, `uniqueKeys`, `merge`, `keepComments` and `firstDocumentOnly` in `YamlOptions`. A merge key's value must name maps (`BAD_MERGE`); tabs are not indentation. |
|
|
26
|
+
| [`yaml-json.ts`](https://github.com/amritk/nish/blob/main/std/yaml-json.ts) | `YamlJsonLoader.load(stream, d, doc)`: document `d` of a `YamlStream` loaded into a `json-value` `JsonDocument`, with `setRange` and `setKeyRange` from the nodes' byte ranges. Core-schema scalars; hex and octal ints as exact decimal; `.inf` and `.nan` refused (`NOT_FINITE`); aliases copied and `<<` merges applied, own keys winning; a non-string key as its source text. The expansion is counted before anything is built, so an alias bomb is `EXPANSION` at the alias, past `maxValues` or `maxTotalBytes`. A failed load keeps nothing and answers a `YAML_JSON_ERR_*` code with its range. |
|
|
27
|
+
| [`yaml-lines.ts`](https://github.com/amritk/nish/blob/main/std/yaml-lines.ts) | `YamlLineIndex`: byte offsets turned into 1-based `line` and `column` (characters) or `byteColumn`, by binary search over line starts found once with `indexOf`. LF, CR LF and a lone CR each break a line; an offset is clamped to the text first. |
|
|
28
|
+
| [`yaml-write.ts`](https://github.com/amritk/nish/blob/main/std/yaml-write.ts) | `stringifyYaml(doc, v)`: a `json-value` value written as block-style YAML that `parseYaml` reads back as the same tree. A string YAML would read as something else (a null, bool or number word, an indicator, a `: ` or ` #`, an edge space, a control character) is double-quoted and escaped; a key over 1024 bytes is an explicit `? key`. `""` for a no-value handle or a tree deeper than `JSON_WRITER_MAX_DEPTH`. |
|
|
29
|
+
| [`pair.ts`](https://github.com/amritk/nish/blob/main/std/pair.ts) | `Pair<A, B>`: an interface with `first` and `second`, for a function that answers two values from one call. A type and nothing else — the caller writes an object literal at the return — and for returning two values rather than storing them side by side |
|
|
30
|
+
| [`collections.ts`](https://github.com/amritk/nish/blob/main/std/collections.ts) | the global `Map<K, V>` and `Set<T>`: insertion-ordered tables whose buckets carry a hash fingerprint beside the entry index and whose entries keep their full hash, so every `get`, `set`, `add`, `has` and `delete` is one probe. `get` is not a method here: its `V | undefined` never crosses a call, so the compiler lowers it to `probe` and, where the key was found, `valueAt`. Nor are `keys()` and `values()`: an iterator is not a value, so a `for...of` over one is lowered to `walkOpen`, `walkNext`, `keyAt` or `valueAt`, and `walkClose`, and a count of live walks defers compaction until no loop is walking the table. A program never imports it: naming `Map` or `Set` loads it, and the compiler emits what a module uses of it into that module ([`docs/wp32-map.md`](https://github.com/amritk/nish/blob/main/docs/wp32-map.md), [`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#map-and-set)). Its `hashKey`, `sameKey` and `storedKey` are lowered by the compiler per key type |
|
|
31
|
+
| [`map.ts`](https://github.com/amritk/nish/blob/main/std/map.ts) | `reserve(m, n)` and `getOrInsert(m, k, v)` for the global `Map`. Their bodies are the meaning, and what runs under Node: `reserve` does nothing, and `getOrInsert` is a `get`, and a `set` of `v` when the key was missing. Natively the compiler lowers every call in place — `reserve` to the table's `reserveSlots`, which grows the buckets once so that `n` entries fit without a rebuild, and `getOrInsert` to one `probe` and a `valueAt` or an `insertAt` through its answer — so, like `collections.ts`, it writes no `.ll` of its own ([`docs/wp32-map.md`](https://github.com/amritk/nish/blob/main/docs/wp32-map.md) §9.2, [`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#map-and-set)) |
|
|
32
|
+
| [`bytes.ts`](https://github.com/amritk/nish/blob/main/std/bytes.ts) | `getU16LE` / `getU16BE` to `getF64LE` / `getF64BE` and the `set` twins: `u16`, `u32`, `u64`, `i32`, `i64` and `f64` read from and written into a `u8[]` at an offset, little- or big-endian. Each access is an ordinary index, so out of range is an `index` panic naming the first missing byte. In range the answers are those of the private copies `std/` keeps (`sha256LoadWord`, `chacha20Word`, `poly1305Le64`, sha512's `loadWord` / `storeWord`, `http2FramePut32`, `tlsPutU16`), none of which it replaces. A guard in the caller does not yet remove the checks after inlining. Under Node only the `u16` and `u32` functions answer as natively. Beside them are the runs a parser walks — `skipBlank`, `skipDigits`, `skipJsonText` and `skipJsonPlain`, the first index in a clamped range whose byte is not a JSON blank, a digit, a byte a JSON string holds as itself or a byte a JSON writer copies without an escape — and `copyBytes`, a window copied as one move that panics on a range outside its array. Their bodies call private walkers, which are what runs under Node; natively the compiler replaces each walker's call with a kernel of `runtime/runtime-bytes.c`, or `llvm.memmove` for the copy ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#byte-runs-skipblank-skipdigits-skipjsontext-skipjsonplain-and-copybytes-in-nishbytes)). The runs are variable-time. `tests/link/std_bytes*` are its programs |
|
|
33
|
+
| [`base64.ts`](https://github.com/amritk/nish/blob/main/std/base64.ts) | `nish/base64`: `base64Encode(bytes)`, `base64EncodeInto(src, off, len, dst, at)` and `base64EncodedLength(n)`, padded standard base64 (RFC 4648 §4) for kitty graphics frames. It only encodes, and it indexes its alphabet by the sextet, so it is not constant-time: it is for pixels, not secrets. A strict decoder, and an encoder for key material, are `crypto/base64url.ts`'s `base64StdPadDecode` and `base64StdPadEncode` (`tests/link/std_base64`; the two modules in one program: `tests/link/std_base64_two_modules`) |
|
|
34
|
+
| [`random.ts`](https://github.com/amritk/nish/blob/main/std/random.ts) | `Rng`, a seeded generator of pseudo-random numbers: `new Rng(seed)`, `nextI32()`, `nextF64()` and `nextInt(lo, hi)`, an unbiased integer in [lo, hi] by rejection sampling. A Weyl sequence through all 2^32 states with MurmurHash3's finaliser on each, written with `nish:unsafe`'s `wrappingAdd`, `wrappingSub` and `wrappingMul` so that every answer is the same bits natively and under Node (`tests/link/std_random`; [`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#random-numbers-rng-in-nishrandom)). A parallel body or a task that makes its own steps it without a shared write. Not for cryptography |
|
|
35
|
+
| [`cli.ts`](https://github.com/amritk/nish/blob/main/std/cli.ts) | a command-line tool's `--help`. `HelpMenu` builds it a line at a time — `header`, `usage`, `section`, `row(flag, placeholder, description)`, `note`, `value` — with descriptions lined up at the caller's columns, counted on the plain text, and answers it with `text()`. `Style` paints it in the basic ANSI colours, bold and faint, which the terminal's theme defines, or in bold and underline alone when `TERM` declares no colours; each role it paints (`heading`, `name`, `flag`, `meta`, `dim`) is a pair of SGR lists, `<role>Themed` and `<role>Mono`, a program may set. `Style` also paints any text: the sixteen colours (`red`, `brightBlack`, ...), their backgrounds (`bgRed`, `bgBrightRed`, ...), the attributes (`bold`, `faint`, `italic`, `underline`, `inverse`, `strikethrough`, which the failsafe keeps) and `sgr(text, codes)` for any parameter list, with the `SGR_` constants spelling the codes. `styleFor(fd, agent)` is the policy `nish --help` follows: plain unless the descriptor is a terminal, `NO_COLOR`, `FORCE_COLOR`, `CLICOLOR_FORCE`, `CLICOLOR` and `TERM=dumb` honoured, plain for an agent (`agentDriven`), forced colour always the theme's; it reads the environment and, for the terminal check, forks `/bin/sh` once, which runs `tput` by absolute path. `HelpMenu` and `Style` reach no capability. The compiler's own help is built with it through a relative import, so the module may use only what the last release compiles (`tests/link/std_cli`; [guide](../docs/guide/using-nish/standard-library.md#nishcli)) |
|
|
36
|
+
| [`cli-args.ts`](https://github.com/amritk/nish/blob/main/std/cli-args.ts) | a command tree for a large command-line tool. `Cli` declares it with a builder — `group`, `command(parent, name, id, description)`, `alias`, `flag`, `globalFlag`, `argument`, then `defaultValue`, `required`, `repeatable`, `negatable` and `epilog` on what they answer — every declaration an `i32` handle and every mistake a panic at its call. `parse(argv, start)` walks the arguments once and answers a `ParseResult`: the reached command's id and path, each flag's value with its `source` (the command line, its default or unset), the positionals and what follows `--`, or the first error with a did-you-mean. `finish` prints help (laid out by `HelpMenu`), the version or the error and exits 0 or 2. `parse` reaches no capability (`tests/link/std_cli_args`, `examples/api-cli`; [guide](../docs/guide/using-nish/standard-library.md#nishcli-args)) |
|
|
37
|
+
| [`cli-flags.ts`](https://github.com/amritk/nish/blob/main/std/cli-flags.ts) | `nish/cli-flags`: a command line read against a declared table. `FlagTable` declares `string`, `number` (an `f64`), `boolean` (`--name` / `--no-name`) and `repeatable` flags and `command` paths of one or more words; `parse(args)` (or `parseFrom(process.argv, 1)`) answers a `ParsedFlags` with the longest matched `command`, the `positionals` and each flag by name (`str`, `num`, `bool`, `all`, `has`). `--name value`, `--name=value`, `--` and a lone `-` follow the GNU long-option grammar; there are no short options. A usage error (unknown flag, missing value, not a number, given twice, value on a boolean, unknown command) is carried as `code`, `message` and `argument`, and `exitUsage()` writes it to stderr and exits 2. `writeHelp(menu)` adds the table's rows to a `nish/cli` `HelpMenu`. Reaches `exit` only through `exitUsage` (`tests/link/std_cli_flags`, `std_cli_flags_bad_*`; [guide](../docs/guide/using-nish/standard-library.md#parsing-flags)) |
|
|
38
|
+
| [`sqlite.ts`](https://github.com/amritk/nish/blob/main/std/sqlite.ts) | SQLite, vendored in `runtime/sqlite/` and linked into a program that imports the module, through the C adapter `runtime/sqlite/nish-sqlite.c`. `openDatabase(path, flags)` answers a `Database`: `exec`, `begin` / `commit` / `rollback`, `prepare` into a `Statement`, `lastInsertRowid`, `changes` and `close`, which finalizes the connection's statements. A `Statement` binds (`bindI64`, `bindF64`, `bindNull`, `bindText`, `bindTextBytes`, `bindBlob`), steps, resets and reads columns (`columnType`, `columnI64`, `columnF64`, `columnBytes`, `columnText`, and `columnInto`, a copy into the caller's `u8[]`), so a row allocates nothing. Every failure is a `Result` of SQLite's extended code, and `error(code)` builds the `SqliteError` with its message. A handle is an `i32` into the adapter's thread-local tables, and one used after `close` or `finalize` answers `SQLITE_MISUSE`. Reaches `ffi`; links natively and under `--profile wasi` (`tests/link/std_sqlite*`; [`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#sqlite-nishsqlite)) |
|
|
39
|
+
| [`base64.ts`](https://github.com/amritk/nish/blob/main/std/base64.ts) | standard base64 (RFC 4648 §4) with `=` padding, encode only: `base64Encode(bytes)` answers a string, `base64EncodeInto(src, off, len, dst, at)` writes into a byte buffer allocating nothing, `base64EncodedLength(n)` sizes it; a range outside its array panics before a byte is written. What kitty graphics frames are sent in; not the constant-time `crypto/base64url` (`tests/link/std_base64`, `std_base64_f64`, `std_base64_negative_*`; [guide](../docs/guide/using-nish/standard-library.md#nishbase64)) |
|
|
40
|
+
| [`term.ts`](https://github.com/amritk/nish/blob/main/std/term.ts) | a terminal UI with no I/O of its own. `Screen` holds `width × height` cells (code point, 24-bit foreground and background, bold and inverse) for the frame being drawn and the frame last sent; `flush()` answers the string that turns one into the other — cursor moves and only the cells that changed, a style sent only where it differs — for the caller to `write`. A `Screen` made with `palette256` sends the nearest 256-colour index instead. `Canvas` draws pixels on a screen, two to a cell with the half block "▀" (`setPixel`, `fillRect`, `line`, `text`). The escapes (`enterAltScreen`, `hideCursor`, `clearScreen`, `beginSync`, …) are functions answering strings, and a `Screen` with `sync` set wraps each flush that changes anything in synchronized output (mode 2026). `FixedStep` is a fixed-step clock: `advance(nowMs)` answers the whole steps of `1000 / stepsPerSecond` ms since the last call, at most `maxSteps`, and `alpha()` the remainder, for logic at 240 Hz and drawing at 120. `KeyDecoder` turns raw-mode bytes into `KeyEvent`s (arrows as `ESC [` or `ESC O`, Enter, Space, Escape, Ctrl-C, characters), carries a sequence a read cut short into the next `feed`, and decides a lone ESC on a timeout the caller's clock measures. `feedEvents` and `pollEvents` read the same bytes as `TermEvent`s, whose `type` is an `EVENT_` constant: keys with `MOD_` bits and an `ACTION_` (legacy, modifyOtherKeys and the kitty keyboard protocol), SGR mouse reports in cells or pixels, bracketed paste, focus, and the terminal's replies to queries, which never come out as keys. `KeyState` tracks which keys are held, making up the releases a terminal without the kitty protocol never sends and turning Ctrl-C into `EVENT_QUIT`. The size, raw mode and the clock are the caller's, so the module reaches no capability and prints the same bytes under Node (`tests/link/std_term`, `std_term_f64`; [guide](../docs/guide/using-nish/standard-library.md#nishterm)) |
|
|
41
|
+
| [`term-probe.ts`](https://github.com/amritk/nish/blob/main/std/term-probe.ts) | what the terminal can do, asked once at start. `probeQueries(env)` is every query as one string — XTVERSION, DECRQM for synchronized output and pixel mouse, kitty's keyboard flags and a kitty graphics query, XTSMGRAPHICS, the cell and text-area sizes, a true-colour read-back, and DA1 last, wrapped for tmux's or screen's passthrough where only the outer terminal can answer. `TermProbe` takes the replies a read at a time (`feed`), holds back any key typed meanwhile (`heldEvents`), is `done()` once DA1 is answered or `timedOut(now)` after 250 ms, and `result()` reads the replies and a `TermEnv` (the `TERM`, `COLORTERM`, `TMUX`, `SSH_*`, `NO_COLOR` and `NISH_TERM_*` variables) into a `TermCaps`: colours, graphics backend, sync, kitty keyboard, cell size, pixel mouse, multiplexer. Pure: the environment, the bytes and the clock are the caller's (`tests/link/std_term_probe`; [guide](../docs/guide/using-nish/terminal.md#what-it-detects-and-what-it-falls-back-to)) |
|
|
42
|
+
| [`term-rate.ts`](https://github.com/amritk/nish/blob/main/std/term-rate.ts) | how fast the link to the terminal is, and the frame rate and detail a program keeps to under it. A `RateController` is told each frame sent (`frameSent(bytes, costNs, nowMs)`) and each acknowledgement — the answer to the DSR 5 query sent after a frame — (`ackReceived(nowMs)`), keeps running averages of the round trip, the bytes a second delivered and the cost of a frame, and answers `mayDraw(nowMs)` and a `RateDecision`: the frame interval, a resolution divisor, a backend cap and a palette, stepped down a tier when frames back up and up again when the link has room; `stats()` is the `TermStats` a program can show. Pure: no capability and no clock (`tests/link/std_term_rate`, `std_term_rate_f64`; [guide](../docs/guide/using-nish/terminal.md#how-the-frame-rate-adapts)) |
|
|
43
|
+
| [`term-pixels.ts`](https://github.com/amritk/nish/blob/main/std/term-pixels.ts) | a `Framebuffer` of `0xRRGGBB` pixels and the encoders that draw it with the backend the terminal has: `kittyFrame` (kitty graphics, one image id retransmitted each frame), `sixelFrame` (a fixed palette, optionally dithered), `drawHalfBlocks` onto a `Canvas` and `drawAscii` onto a `Screen`, each into the `PixelRect` `fitFramebuffer` letterboxes in the cells; `scaleFramebuffer` shrinks a frame by the controller's divisor. `PixelBytes` keeps the encoders' output and working arrays, so frame after frame allocates nothing. Pure (`tests/link/std_term_pixels`, `std_term_pixels_f64`; [guide](../docs/guide/using-nish/standard-library.md#nishterm-pixels)) |
|
|
44
|
+
| [`term-io.ts`](https://github.com/amritk/nish/blob/main/std/term-io.ts) | the term module that does the I/O. `runTerminal(options, state, update, render)` is a whole program loop: it opens the terminal, steps `update` at a fixed rate, calls `render` with a `Frame` (the `Screen`, the `Framebuffer`, the step `alpha`, `TermCaps` and `TermStats`) whenever the controller lets a frame go, and closes when `update` answers false or the user presses Ctrl-C. `openTerminal(options)` answers a `Terminal` for a program's own loop (`readEvents`, `present`, `stats`, `close`). Opening refuses a pipe, enters raw mode and the alternate screen, probes the terminal and turns on the kitty keyboard, mouse, paste and focus reports the options ask for, with the runtime's restore string set before each so a panic or signal leaves the terminal as it was; `present` sends the frame in the best backend the controller allows as one write and acknowledges it with DSR 5. Reaches `tty` (`tests/term-io/relay.mjs` drives it in a pseudo-terminal from `tests/run.js`'s `term-io` section; `tests/link/std_term_io_deny`; [guide](../docs/guide/using-nish/terminal.md)) |
|
|
45
|
+
| [`region.ts`](https://github.com/amritk/nish/blob/main/std/region.ts) | The source behind the builtin module `nish:region`, and imported only by that name: `Region<T>`, `region`, `regionWith`, `enter`, `enterWith` and `free`. A `Region<T>` is a handle, a 64-bit integer, to memory the program frees when it says so; `region(f)` allocates what `f` builds into a new region and keeps its answer as the root, `enter(r, f)` runs `f(root)` there again and answers its scalar, the `*With` forms pass one argument through, and `free(r)` gives all of it back. A handle used after `free`, entered twice or freed while entered stops the program. The bodies are the meaning; the allocator swap is the emitter's (`src/emit-region.ts`) and what may cross a region's edge is the checker's (`src/region-check.ts`), keyed on this file ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#regions-nishregion); [guide](../docs/guide/language/regions.md)) |
|
|
46
|
+
| [`secret.ts`](https://github.com/amritk/nish/blob/main/std/secret.ts) | The source behind the builtin module `nish:secret`, and imported only by that name: `Secret<T>`, `secret`, `expose`, `exposeWith` and `wipe`. The bodies are the meaning; what makes a `Secret` one is the checker's, keyed on this file ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#secrets-nishsecret)). `wipe`'s body is written by the emitter, one volatile `llvm.memset`, and like `collections.ts` the module writes no `.ll` of its own |
|
|
47
|
+
| [`threads.ts`](https://github.com/amritk/nish/blob/main/std/threads.ts) | `parallelMapInto(src, dst, f)` and `parallelReduce(src, f, identity)`: a function over every element of an array, on as many threads as the length is worth. Its bodies are the sequential meaning, which is what runs under Node; the compiler recognises the two templates by module and name, lowers the one loop in each onto `nish_parallel_range`, holds the function to the rules that make that safe, and compiles an importing program with `--threads` ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#data-parallelism-nishthreads)). `tests/link/par_*` are its programs. `scope()`, bound by `using s = scope()`, runs unlike tasks, each on a thread of its own, and joins them at every exit of the block; `s.spawn(entry, arg, dst, at)` stores a task's answer, and `s.spawn(entry, arg)` and `s.spawn(() => ...)`, whose arrow reads the function's `const` locals, run one that answers `void` (`spawnWith` and `spawnThunk` are the methods the compiler reads them as, and under Node `spawn` itself tells them apart) ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#scoped-tasks-using-s--scope)). `scopeWithStack(bytes)` is a scope whose tasks each run on a new thread with at least `bytes` of stack, for a task that recurses deeper than a thread's default allows; under Node and without threads the size is checked and otherwise ignored (`tests/link/thread_stack_deep`). `Mutex<T>` is a lock that owns its data: the value is reachable only through the guard `using g = m.lock()` binds, which releases it at every exit of the block, so a scope's tasks can share and update one value without a data race or a deadlock ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#a-lock-that-owns-its-data-mutext); `tests/link/thread_mutex_*`, and `examples/histogram.ts` measured). `Channel<T>` carries scalars to one receiver, a task of the scope or the parent after the join, from the scope's sender tasks and from the parent before the block that spawns them, and closes when every sending task has returned ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#channels-channelt); [`docs/wp29-thread-surface.md`](https://github.com/amritk/nish/blob/main/docs/wp29-thread-surface.md#44-stage-p3--channels-of-scalars) §4.4). Every body in the module is its meaning under Node, where a task runs at its `spawn` |
|
|
48
|
+
| [`sync/shm.ts`](https://github.com/amritk/nish/blob/main/std/sync/shm.ts) | `createRegion(name, bytes)`, `openRegion(name)` and `unlinkRegion(name)`: a named region of memory that several processes of one user map at once, through `runtime/runtime-shm.c`. A `SharedRegion` is read and written through checked slots, `loadI32` to `compareExchangeI64` at an ordering the call names (`RELAXED` to `SEQ_CST`), and byte windows (`read`, `write`); an offset outside the region or not a multiple of the slot's width stops the program. `waitI32` and `wake` sleep and wake on a slot across processes (Linux futex; `SHM_UNSUPPORTED` elsewhere). Native only, and it reaches `ffi` ([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#shared-memory-between-processes-nishsyncshm-and-nishsyncring); [`docs/wp47-shared-memory.md`](https://github.com/amritk/nish/blob/main/docs/wp47-shared-memory.md); [`docs/security/shm.md`](../docs/security/shm.md)) |
|
|
49
|
+
| [`sync/ring.ts`](https://github.com/amritk/nish/blob/main/std/sync/ring.ts) | Lock-free rings of byte records in a `SharedRegion`: `SpscRing` (one producer, one consumer, records of any length) and `MpscRing` (many producers, one consumer, fixed slots), made with `initSpscRing` / `initMpscRing` and found from another process with `attachSpscRing` / `attachMpscRing`. `tryPush`, `tryPop` and `popBatch` never block; a ring made with a doorbell adds `popWait`, which sleeps on a futex that a push rings only when the consumer is asleep. A length a peer wrote that the ring cannot hold stops the program as corrupt. `tests/link/sync_ring`, `examples/prefork-jobs.ts` and `bench/ring-latency/` are its programs |
|
|
50
50
|
|
|
51
51
|
## `nish/crypto` — the primitives under TLS 1.3
|
|
52
52
|
|
|
53
|
-
The first lanes of [WP34](
|
|
53
|
+
The first lanes of [WP34](https://github.com/amritk/nish/blob/main/docs/wp34-hosting-cs.md) §5: K1's hashes, MACs and
|
|
54
54
|
key derivation, K2's and K3's AEADs, K4's key exchange, K5's signatures and
|
|
55
55
|
K6's certificates, in
|
|
56
56
|
pure Nish (decision S1), each module imported by its own specifier. Every one
|
|
@@ -59,7 +59,7 @@ upstream notice: `crypto/aes.ts`'s `ghashMul32` is adapted from BearSSL,
|
|
|
59
59
|
`crypto/p256.ts`'s field and scalar arithmetic is ported from fiat-crypto and
|
|
60
60
|
its nonce's safegcd inversion from libsecp256k1, and `crypto/p384.ts`'s field
|
|
61
61
|
and scalar arithmetic is ported from fiat-crypto
|
|
62
|
-
([`THIRD_PARTY_NOTICES.md`](
|
|
62
|
+
([`THIRD_PARTY_NOTICES.md`](https://github.com/amritk/nish/blob/main/THIRD_PARTY_NOTICES.md)). Each reproduces its
|
|
63
63
|
specification's published vectors in its `tests/link/crypto_*` programs. The
|
|
64
64
|
performance gate compiles every module with no diagnostics under both
|
|
65
65
|
`--number-mode i32` and `f64`, and the hashes, HMAC, HKDF, X25519,
|
|
@@ -69,7 +69,7 @@ which), and does both.
|
|
|
69
69
|
|
|
70
70
|
Every module here was security-audited (#363); the records are in
|
|
71
71
|
[`docs/security/`](../docs/security/README.md), and how to report a
|
|
72
|
-
vulnerability is in [`SECURITY.md`](
|
|
72
|
+
vulnerability is in [`SECURITY.md`](https://github.com/amritk/nish/blob/main/SECURITY.md). Two limits hold across the
|
|
73
73
|
modules. **Lengths stop at 2^31 − 1.** No array built by `push` or `new Array`,
|
|
74
74
|
no array or string the runtime makes (a file read, a concatenation, a
|
|
75
75
|
template) and no string built by `join` can pass 2^31 − 1 elements or bytes
|
|
@@ -85,31 +85,31 @@ too. `crypto/aes.ts` and
|
|
|
85
85
|
`crypto/chacha20poly1305.ts` take their keys as plain arrays, the caller's to
|
|
86
86
|
wipe, and wipe everything they derive from one (`aesKeyWipe` drops an
|
|
87
87
|
`AesKey`). What no store reaches, registers and vector registers among them,
|
|
88
|
-
is listed in [`SECURITY.md`](
|
|
88
|
+
is listed in [`SECURITY.md`](https://github.com/amritk/nish/blob/main/SECURITY.md#threat-model).
|
|
89
89
|
|
|
90
90
|
| Module | What it is | Reproduces |
|
|
91
91
|
| --- | --- | --- |
|
|
92
|
-
| [`crypto/sha256.ts`](
|
|
93
|
-
| [`crypto/sha512.ts`](
|
|
94
|
-
| [`crypto/hmac.ts`](
|
|
95
|
-
| [`crypto/hkdf.ts`](
|
|
96
|
-
| [`crypto/jwt.ts`](
|
|
97
|
-
| [`crypto/ct.ts`](
|
|
98
|
-
| [`crypto/bcrypt.ts`](
|
|
99
|
-
| [`crypto/bcrypt-batch.ts`](
|
|
100
|
-
| [`crypto/base64url.ts`](
|
|
101
|
-
| [`crypto/sha1.ts`](
|
|
102
|
-
| [`crypto/x25519.ts`](
|
|
103
|
-
| [`crypto/ed25519.ts`](
|
|
104
|
-
| [`crypto/chacha20poly1305.ts`](
|
|
105
|
-
| [`crypto/aes.ts`](
|
|
106
|
-
| [`crypto/p256.ts`](
|
|
107
|
-
| [`crypto/p384.ts`](
|
|
108
|
-
| [`crypto/rsa.ts`](
|
|
109
|
-
| [`crypto/x509.ts`](
|
|
110
|
-
| [`crypto/x509-random.ts`](
|
|
111
|
-
| [`crypto/pki-roots.ts`](
|
|
112
|
-
| [`crypto/pki.ts`](
|
|
92
|
+
| [`crypto/sha256.ts`](https://github.com/amritk/nish/blob/main/std/crypto/sha256.ts) | `sha256(data)`, and `Sha256`, a streaming hasher: `update(buf, off, len)` over a window of a `u8[]`, `copy()` for the hash of a prefix while the original keeps going, and `digest()`, a fresh 32-byte array. `SHA256_SIZE` and `SHA256_BLOCK`. `sha256` panics on an array longer than 2^31 − 1 bytes. Natively, on a CPU with SHA-NI (and SSE4.1 and SSSE3) or the ARMv8 SHA-256 instructions, the compiler sends each block's compression to the runtime's kernel instead (`runtime/runtime-sha256.c`, WP38), and `NISH_SHA256=soft` pins the portable code | FIPS 180-4 §6.2 |
|
|
93
|
+
| [`crypto/sha512.ts`](https://github.com/amritk/nish/blob/main/std/crypto/sha512.ts) | SHA-512 and SHA-384 on one compression function: `sha512` and `sha384`, and the streaming `Sha512` and `Sha384` with `Sha256`'s three methods; digests of 64 and 48 bytes. `SHA512_SIZE`, `SHA384_SIZE` and `SHA512_BLOCK`. The one-shot `sha512` and `sha384` panic on an array longer than 2^31 − 1 bytes | FIPS 180-4 §6.4, §6.5 |
|
|
94
|
+
| [`crypto/hmac.ts`](https://github.com/amritk/nish/blob/main/std/crypto/hmac.ts) | `hmacSha256` and `hmacSha384`, the streaming `HmacSha256` and `HmacSha384` (keyed in the constructor, then `update` and `digest`), and `hmacSha256Verify` / `hmacSha384Verify`, which compare a received tag with `timingSafeEqual`. The one-shot functions panic on a message longer than 2^31 − 1 bytes. `HmacSha256Scratch` / `HmacSha384Scratch` are the same tag over hashers the caller allocates once — `begin(key, off, len)`, `update`, `finishInto(out, at)`, `wipe` — so a loop of them inside `using a = arena()` leaves the arena where it was; `hmacCopySha256` / `hmacCopySha384` copy a hasher's state into one the caller holds. `pbkdf2HmacSha256(password, salt, iterations, dkLen)` is PBKDF2 with HMAC-SHA-256, for SCRAM-SHA-256: the password goes in and the key comes out as a `Secret<u8[]>`, every intermediate is wiped, and fewer than one iteration or a `dkLen` below one answers `null`. `HmacKey(hash, key)` keys the inner and outer pads of HMAC-SHA-256, -384 or -512 once, from a `Secret<u8[]>`, and holds them as a `Secret`; `hmacKeyedInto(key, data, off, len, out, at)` copies that keyed state per message and writes the tag, and `hmacKeyedVerify` compares a received tag window in constant time, so a loop of them allocates nothing past its arena block; `HmacKey.wipe()` zeroes the pads | RFC 2104, RFC 4231 §4, RFC 8018 §5.2 |
|
|
95
|
+
| [`crypto/hkdf.ts`](https://github.com/amritk/nish/blob/main/std/crypto/hkdf.ts) | `hkdfExtractSha256` / `hkdfExtractSha384` (an empty salt is HashLen zeros) and `hkdfExpandSha256` / `hkdfExpandSha384`, which answer `null` for a length below zero or above 255 × HashLen, for an `info` longer than 2^31 − 1 bytes, and for a PRK shorter than HashLen. `hkdfExpandLabelSha256` / `hkdfExpandLabelSha384` are TLS 1.3's HKDF-Expand-Label, which QUIC uses too; the label is given without its `"tls13 "` prefix, and a length outside 0 to 255, a label outside 1 to 249 bytes, a context past 255 bytes or a secret shorter than HashLen panics, since each is the program's choice rather than the peer's. `hkdfExtractInto`, `hkdfExpandInto` (`false` where the others answer `null`) and `hkdfExpandLabelInto` compute the same bytes in an `HkdfScratch` the caller owns, over the hash a length of 32 or 48 names, into the caller's array, and wipe the scratch before they return. A key held as a `Secret<u8[]>` (`nish:secret`) goes through the `*Secret` twins, which read it only inside `exposeWith` and answer a `Secret`: `hkdfSha256Secret` / `hkdfSha384Secret`, each step's, and `hkdfExpandLabelSha256Secret(prk, label, context, length)` / `hkdfExpandLabelSha384Secret`, which panic where HKDF-Expand-Label does. A destination already held as a `Secret` is derived into in place by `hkdfExpandLabelSha256Held(dst, prk, label, context)` / `hkdfExpandLabelSha384Held`, as many bytes as `dst` holds, the plain output copied in inside `exposeWith` and wiped; `dst` may be `prk`. `hkdfExpandLabelHeldInto(hashLength, prk, label, context, out, at, length)` writes a plain window from a held PRK, for output that is plain by design, a record key or IV | RFC 5869 §2, Appendix A; RFC 8448 §3 and RFC 9001 A.1 for the label |
|
|
96
|
+
| [`crypto/jwt.ts`](https://github.com/amritk/nish/blob/main/std/crypto/jwt.ts) | JSON Web Tokens in the JWS compact form. `JwtVerifier(maxToken)` holds its buffers, sized once, and `verify` / `verifyString(key, token, now)` answer `JWT_OK` or a `JWT_ERR_*` code: the key decides the algorithm (`alg` must name it exactly, so `none` and an HS header in front of an RSA key are refused), the header holds `alg` once and no `crit` or `b64`, base64url is strict, the signature is checked before the payload is read, and then `exp`, `nbf` and `iat` with a leeway, and `iss`, `aud` and `sub`, are checked. Keys: `jwtHmacKey` (HS256/384/512, keyed once through `HmacKey`), `jwtRsaKey` / `jwtRsaKeyParts` (RS256/384/512, 2048 bits or more), `jwtEcKey` (ES256, ES384), `jwtEd25519Key` (EdDSA), and the signing `jwtEs256SigningKey` / `jwtEd25519SigningKey`, each from a `Secret<u8[]>` it copies; `JwtKey.wipe()` zeroes the copy. `jwtSign` / `jwtSignString` / `jwtSignKid` mint a token under an HS, ES256 or EdDSA key. `peekKid` reads the unauthenticated `kid`, to choose a key | RFC 7515 Appendix A, RFC 7519, RFC 7518, RFC 8037 A.4, RFC 8725 |
|
|
97
|
+
| [`crypto/ct.ts`](https://github.com/amritk/nish/blob/main/std/crypto/ct.ts) | `timingSafeEqual(a, b)`, which reads every byte whatever it holds, and `timingSafeEqualAt(a, aOff, b, bOff, len)` over two windows, which answers `false` for a window outside its array. Two lengths that differ answer `false` at once, because a length is public, and so does an array longer than 2^31 − 1 bytes | — |
|
|
98
|
+
| [`crypto/bcrypt.ts`](https://github.com/amritk/nish/blob/main/std/crypto/bcrypt.ts) | `bcryptHash(password, salt, cost)`, the 60-character `$2b$` hash of a `Secret<u8[]>` password under a 16-byte salt and a cost of 4 to 31, and `bcryptVerify(password, hash)`, which reads `$2a$`, `$2b$` and `$2y$` hashes, refuses `$2x$`, and compares in constant time. The first 72 bytes of a password count, up to its first NUL, as under libxcrypt's `crypt_r` (where other implementations differ: [the interoperability table](../docs/security/crypto-bcrypt.md#interoperability)); the Blowfish state is wiped before either returns. `bcryptHash4(p0, p1, p2, p3, salts, cost)` and `bcryptVerify4(p0, p1, p2, p3, hashes)` do up to four at once, interleaved (four in the time of about 1.2 single hashes on one core), each lane the single path's answer; verify groups lanes by cost, and a malformed lane makes the whole call an `Err` naming it. `BCRYPT_MAX_PASSWORD`, `BCRYPT_HASH_LENGTH`, `BCRYPT_SALT_SIZE`, `BCRYPT_MIN_COST`, `BCRYPT_MAX_COST`. A `std/threads` task may call all four ([BCRYPT-9](../docs/security/crypto-bcrypt.md#findings)) | OpenBSD `bcrypt(3)`; crypt_blowfish 1.3 for `$2a$` and `$2y$` |
|
|
99
|
+
| [`crypto/bcrypt-batch.ts`](https://github.com/amritk/nish/blob/main/std/crypto/bcrypt-batch.ts) | `bcryptBatcher({ width, holdNanos, cores, cost })`, a login batcher: `verify(password, hash, now)` and `hash(password, salt, now)` queue a request and answer its ticket, `waitNanos(now)` says when a run is due (a full batch, or the oldest request held `holdNanos`), and `run(now)` sends up to `width × cores` requests as `bcryptVerifyN` / `bcryptHashN` batches on one `std/threads` task per core and answers a `BcryptAnswer` (`ticket`, `ok`, `hash`, `error`) for each. It keeps and wipes its own copy of each password; a damaged stored hash answers its `Err` on its own row ([the record](../docs/security/crypto-bcrypt.md#the-login-batcher)) | — |
|
|
100
|
+
| [`crypto/base64url.ts`](https://github.com/amritk/nish/blob/main/std/crypto/base64url.ts) | `base64urlEncode(data)` and `base64urlDecode(text)`, unpadded; `base64StdPadEncode(data)` and `base64StdPadDecode(text)`, the padded standard alphabet that SCRAM and PEM use, constant-time in both directions. These are not `nish/base64`, which encodes only, is not constant-time and is for pixels; the names differ because std modules share the program's one namespace. Decoding is strict, so every byte string has one spelling. `base64urlDecode` answers `null` for any `=`, a character outside the alphabet, a length of 1 mod 4 or nonzero unused low bits; `base64StdPadDecode` requires the padding, answering `null` unless the text is whole four-character groups with `=` only as the last one or two characters, every other character in the alphabet and the unused bits zero. Both refuse a text longer than 2^31 − 1 bytes, and `base64urlEncode` panics on such an array | RFC 4648 §4, §5, §10 (`tests/link/std_base64_padded`) |
|
|
101
|
+
| [`crypto/sha1.ts`](https://github.com/amritk/nish/blob/main/std/crypto/sha1.ts) | `sha1(data)`, a fresh 20-byte digest, and `SHA1_SIZE` and `SHA1_BLOCK`. **Not for security**: SHA-1 is broken for collisions, and it is here only because RFC 6455 §4.2.2's `Sec-WebSocket-Accept` is defined on it (`nish/net/websocket` is its importer). One-shot only; it panics on an array longer than 2^31 − 1 bytes. It came after the #363 audit and has no record in `docs/security/` | FIPS 180-4 §6.1 |
|
|
102
|
+
| [`crypto/x25519.ts`](https://github.com/amritk/nish/blob/main/std/crypto/x25519.ts) | `x25519(scalar, u)`, the shared secret as a `Secret<u8[]>`, and `x25519Base(scalar)`, the public key, each from a `Secret<u8[]>` scalar, on ten 25.5-bit limbs in `i64`. Either answers `null` unless its arguments are `X25519_SIZE` (32) bytes; the scalar is clamped on a copy, which is wiped | RFC 7748 §5.2, §6.1, Wycheproof `x25519_test` |
|
|
103
|
+
| [`crypto/ed25519.ts`](https://github.com/amritk/nish/blob/main/std/crypto/ed25519.ts) | `ed25519PublicKey(seed)`, the 32-byte public key, `ed25519Sign(seed, msg)`, the 64-byte signature `R \|\| S`, each from a `Secret<u8[]>` seed of `ED25519_SEED_SIZE` (32) bytes or `null`, and `ed25519Verify(pub, msg, sig)`. Signing is deterministic and constant time; the seed's hash, the nonce and every intermediate the scalar reaches are wiped. Verify is cofactorless, refuses an S of L or more and a key that does not decode, and is variable time, since its inputs are public. Sign and verify panic on a message longer than 2^31 − 1 bytes. It reuses `crypto/x25519.ts`'s field arithmetic | RFC 8032 §5.1, §7.1 |
|
|
104
|
+
| [`crypto/chacha20poly1305.ts`](https://github.com/amritk/nish/blob/main/std/crypto/chacha20poly1305.ts) | `chacha20Poly1305Seal(key, nonce, aad, plaintext)`, which answers the ciphertext followed by the 16-byte tag, and `chacha20Poly1305Open`, which checks the whole tag before it decrypts anything and answers `null` for a message that does not authenticate. Beneath them `chacha20Block`, `chacha20`, `chacha20QuarterRound`, `poly1305` and `poly1305KeyGen`, and `chacha20HeaderMask`, QUIC's 5-byte header-protection mask. Every input of the wrong length answers `null`. What a call derives from the key — the ChaCha20 state, the keystream, the Poly1305 key, `r` and accumulator, and on open the tag it compared — is wiped with `wipe`'s volatile stores before the call returns, the refusal path included. Natively, on x86-64 with AVX2 or on AArch64, the compiler sends the keystream of a message of 256 bytes or more to the runtime's kernel instead (`runtime/runtime-chacha20.c`, WP38 S10), and `NISH_CHACHA20=soft` pins the Nish code | RFC 8439 §2, RFC 9001 §5.4.4, Wycheproof `chacha20_poly1305` (all 325 cases) |
|
|
105
|
+
| [`crypto/aes.ts`](https://github.com/amritk/nish/blob/main/std/crypto/aes.ts) | AES-128 and AES-256, bitsliced four blocks at a time with the Boyar–Peralta S-box circuit, so no table is read; natively, on a CPU with AES-NI and PCLMULQDQ or the ARMv8 AES and PMULL instructions, the compiler sends the key schedule, the block, GCTR and GHASH to the runtime's kernels instead (`runtime/runtime-aes.c`, WP38 S5), and `NISH_AES=soft` pins the bitsliced code: `aesKey(key)` expands a 16- or 32-byte key into an `AesKey` (or `aesKeyInto(key, out)` into one the caller holds, allocating nothing that outlives the call), then `aesEncryptBlock`, `aesGcmSeal` / `aesGcmOpen` (tag last, checked in full before anything is decrypted; any non-empty IV) and `aesHeaderMask`. AES-192 is out of scope, and a wrong length answers `null`, as does an `AesKey` not made by `aesKey`; one whose H is zero opens nothing. An `AesKey` holds its round keys in `roundKeys` and H in `h`, two `u64` words (high, low); `aesKeyWipe(key)` zeroes both with `wipe`'s volatile stores, the kernels' powers of H after the schedule included, and is how a holder drops a key. The bitsliced schedule's, block's and GCTR's working copies, the tag's E(K, J0) block, the tag open compared and J0 itself are wiped the same way before they are released | FIPS 197, SP 800-38D, RFC 9001 §5.4.3, Wycheproof `aes_gcm` (all 316 cases; the 103 AES-192 ones check that the key is refused) |
|
|
106
|
+
| [`crypto/p256.ts`](https://github.com/amritk/nish/blob/main/std/crypto/p256.ts) | ECDSA over P-256 with RFC 6979's deterministic nonces: `p256PublicKey(priv)` (65-byte uncompressed SEC1) from a `Secret<u8[]>` private key, `p256Sign` / `p256Verify` over a digest and `p256SignSha256` / `p256VerifySha256` over a message, and `p256SignSha256Into(scratch, priv, msg, off, len, out, at)`, the same signature written into the caller's window in a `P256SignScratch` made once, keeping nothing in the arena; a signature is `r` and `s`, 32 big-endian bytes each, not DER (`crypto/x509.ts`'s `x509DerSignatureRS` converts one). A malformed key, a point off the curve or an `r` or `s` out of range answers `null` or `false`, never a panic. ECDSA is malleable, and `p256Verify` accepts a high `s`: when `(r, s)` verifies, so does `(r, n − s)` | RFC 6979 A.2.5, Wycheproof `ecdsa_secp256r1_sha256`, `_sha256_p1363`, `_sha512` and `_sha512_p1363` |
|
|
107
|
+
| [`crypto/p384.ts`](https://github.com/amritk/nish/blob/main/std/crypto/p384.ts) | ECDSA verification over P-384: `p384Verify(pub, digest, sig)` over a digest from any hash (one longer than 48 bytes is truncated to its leftmost 384 bits) and `p384VerifySha384(pub, msg, sig)` over a message; the key is 97-byte uncompressed SEC1 and the signature `r` and `s`, 48 big-endian bytes each, not DER. It only verifies, so it holds no `Secret`. A malformed key, a point off the curve, an `r` or `s` out of range and a sum at the identity answer `false`, never a panic. ECDSA is malleable, and `p384Verify` accepts a high `s` | RFC 6979 A.2.6, Wycheproof `ecdsa_secp384r1_sha384`, `_sha384_p1363`, `_sha256`, `_sha512` and `_sha512_p1363` |
|
|
108
|
+
| [`crypto/rsa.ts`](https://github.com/amritk/nish/blob/main/std/crypto/rsa.ts) | RSA signature verification under a public key, never signing: `rsaParsePublicKey(der)` reads a PKCS#1 `RSAPublicKey` (strict DER, a 1,024-to-8,192-bit modulus, an odd exponent from 3 below 2^32) into an `RsaPublicKey`, and `rsaVerifyPkcs1v15(key, hash, msg, sig)` and `rsaVerifyPss(key, hash, saltLen, msg, sig)` verify RSASSA-PKCS1-v1_5 and RSASSA-PSS (MGF1 with the same hash) over a message, the hash one of `RSA_HASH_SHA256`, `RSA_HASH_SHA384` and `RSA_HASH_SHA512`. A malformed key answers `null` and a bad signature `false`, never a panic. Public inputs only, so nothing is a `Secret` and the arithmetic is not constant-time (RSA-2) | RFC 8017 §8.2.2, §9.1.2, §9.2; Wycheproof `rsa_signature_*` and `rsa_pss_*` |
|
|
109
|
+
| [`crypto/x509.ts`](https://github.com/amritk/nish/blob/main/std/crypto/x509.ts) | DER, PEM and X.509 over P-256: `pemToDer` / `derToPem` (RFC 7468, padded standard base64, strict labels), `x509ParseP256PrivateKey` (SEC1 or PKCS#8, answering a `Secret<u8[]>`), `x509ParseCertificate` / `x509ParseChain` into an `X509Certificate`, `x509VerifySignature` (ecdsa-with-SHA256 under an issuer's key), `x509DerSignatureRS` (a DER ECDSA-Sig-Value as the `r || s` `crypto/p256.ts` takes), `x509MintSelfSigned`, the 1-to-14-day self-signed P-256 certificate WebTransport's `serverCertificateHashes` accepts, and `x509CertificateHash`, its SHA-256. DER is read strictly — non-minimal lengths and integers, indefinite lengths and trailing bytes answer `null`. `x509ParseCertificate` names the signature algorithm (`X509_SIG_*`: ECDSA with SHA-256 or SHA-384, RSA PKCS#1 v1.5 with SHA-1 to SHA-512, RSASSA-PSS with its hash and salt, Ed25519) and key type (`X509_KEY_*`: P-256, P-384, RSA, Ed25519), each in its one strict spelling, and decodes subjectAltName, basicConstraints, keyUsage, extKeyUsage, nameConstraints and the subject and authority key identifiers; a repeated or malformed extension refuses the certificate, and an unknown critical one sets `unknownCritical`. `x509NameEqual` and `x509NameIsPrefix` compare Names byte for byte, not by RFC 5280 §7.1. It parses and does not validate: there is no path validation, nothing applies the extensions to a chain, and no clock is consulted; `x509VerifySignature` checks only ecdsa-with-SHA256 under a P-256 key, and `x509DerSignatureRSWidth` reads a P-384 signature. `pemToDer` refuses a file in which any block, of any label, is malformed. `x509MintSelfSigned` refuses a common name that is not UTF-8 or holds a NUL, and its key and serial must come from `crypto.getRandomValues`, which the module cannot call itself without refusing wasm32: `crypto/x509-random.ts` draws them (X509-6) | X.690, RFC 5280, RFC 7468, RFC 5915, RFC 5208 / 5958, RFC 4055, RFC 8410; certificates checked against OpenSSL |
|
|
110
|
+
| [`crypto/x509-random.ts`](https://github.com/amritk/nish/blob/main/std/crypto/x509-random.ts) | **Native only**: the key and serial `crypto/x509.ts`'s mint needs, drawn from `crypto.getRandomValues`, so a program that imports it does not compile for wasm32 and `crypto/x509.ts` imports nothing from it. `x509DrawP256Key()`, a `Secret<u8[]>` P-256 private key drawn again until it is a scalar in [1, n) (each refused draw wiped); `x509DrawSerial()`, twenty octets whose first is 0x40 to 0x7f, so a positive, non-zero, full-length serial; and `x509MintSelfSignedDrawn(priv, commonName, notBeforeMs, days)`, `x509MintSelfSigned` with a drawn serial | — |
|
|
111
|
+
| [`crypto/pki-roots.ts`](https://github.com/amritk/nish/blob/main/std/crypto/pki-roots.ts) | **Native only**: the trust anchors, read where OpenSSL and Go look. `pkiLoadSystemRoots()` gives a `PkiRoots` — `roots` (parsed `X509Certificate`s, no two with the same DER), `sources` (the files read) and `skipped` — from the file `SSL_CERT_FILE` and the `:`-separated directories `SSL_CERT_DIR` name when either is set and not empty, with no fallback, and otherwise from the first of `pkiSystemBundles()` (Go's six paths, in Go's order) that exists and `/etc/ssl/certs` (`PKI_SYSTEM_DIR`). `pkiLoadRoots(file, dirs, bundles)` and `pkiLoadRootsFor(certFile, certDir, dirs, bundles)` take the paths and the variables' values given. A directory gives its `*.pem`, `*.crt` and `xxxxxxxx.N` entries; only a regular file is read; a PEM block that is not a certificate `x509ParseCertificate` accepts, a file over `PKI_ROOTS_MAX_FILE_BYTES` and an entry past `PKI_ROOTS_MAX_FILES` or `PKI_ROOTS_MAX_ROOTS` count in `skipped` and are passed over. It does not read the macOS Keychain ([PKI-R7](../docs/security/crypto-pki.md#pki-r7-low-the-macos-keychain-is-not-read)) | — |
|
|
112
|
+
| [`crypto/pki.ts`](https://github.com/amritk/nish/blob/main/std/crypto/pki.ts) | Whether a TLS server's chain leads to a trusted root and names the host: RFC 5280 §6 path validation and RFC 6125 §6.4 hostname matching. `pkiTrustStore(roots)` indexes the roots by subject, one per DER; `pkiVerifyServer(chain, hostname, nowMs, store)` takes each certificate's DER as sent, the leaf first and the rest in any order, and gives a `PkiResult` — `code`, `PKI_OK` or one of sixteen `PKI_ERR_` reasons (`pkiReason(code)` is its sentence), and `path`, the leaf to the root. It checks every signature (ECDSA P-256 and P-384 with SHA-256 or SHA-384, Ed25519, RSA PKCS#1 v1.5 and PSS with SHA-256/384/512 under a key of `PKI_MIN_RSA_BITS` or more), validity at `nowMs`, basicConstraints, pathLenConstraint, keyUsage, extendedKeyUsage serverAuth, name constraints and unknown critical extensions, under the bounds `PKI_MAX_CHAIN_CERTS`, `PKI_MAX_CHAIN_BYTES`, `PKI_MAX_PATH`, `PKI_MAX_SIGNATURES`, `PKI_MAX_NAMES`, `PKI_MAX_SUBTREES` and `PKI_MAX_NAME_CHECKS`. `pkiMatchHostname(leaf, hostname)` and `pkiVerifySignature(cert, issuer)` are its two halves alone. Revocation, policies and a root's own constraints are not checked ([crypto-pki.md](../docs/security/crypto-pki.md#open)). Compiles on wasm32 | RFC 5280 §4.2 and §6.1, RFC 6125 §6.4 |
|
|
113
113
|
|
|
114
114
|
Three rules hold across the modules:
|
|
115
115
|
|
|
@@ -192,14 +192,14 @@ libdeflate: `bench/gzip/`; against Go, Rust, Java, Node, Bun and Python:
|
|
|
192
192
|
|
|
193
193
|
| Module | What it is | Reproduces |
|
|
194
194
|
| --- | --- | --- |
|
|
195
|
-
| [`compress/deflate.ts`](
|
|
196
|
-
| [`compress/gzip.ts`](
|
|
197
|
-
| [`compress/zlib.ts`](
|
|
198
|
-
| [`compress/checksum.ts`](
|
|
195
|
+
| [`compress/deflate.ts`](https://github.com/amritk/nish/blob/main/std/compress/deflate.ts) | `Deflater(level)` and `Inflater`, raw DEFLATE. Levels 0 (stored), 1 to 3 (greedy on a four-byte hash, `DEFLATE_FAST` = 1) and 4 to 9 (lazy, `DEFLATE_DEFAULT` = 6); each block is the smallest of dynamic, fixed and stored by its exact size. `compress(src, off, len, out, at)` answers the end or `DEFLATE_NO_ROOM` (-1); `deflateBound(n)` / `compressBound(n)` are the most any level writes. `decompress(src, off, len, out, at, limit)` answers the end or a `DEFLATE_ERR_*` code (truncated, block type, stored length, Huffman, symbol, distance, output limit, trailing) and never panics on a stream; `inflate` is the resumable step under it (`INFLATE_DONE`, `INFLATE_NEED_INPUT`, `INFLATE_NEED_OUTPUT`). `FixedDeflater` is the small-body encoder: one fixed-Huffman block, a 16 KB window, nothing kept between calls, two to four times faster than `DEFLATE_FAST` at about 1.35 times its size; `fixedDeflateBound(n)` is the most it writes | RFC 1951: every level against Node's zlib both ways (`tests/link/std_compress_node`), round trips of empty to 1 MB inputs, a 1,500-input fuzz, every prefix and random corruption refused (`tests/link/std_compress`) |
|
|
196
|
+
| [`compress/gzip.ts`](https://github.com/amritk/nish/blob/main/std/compress/gzip.ts) | `GzipCompressor(level)` / `GzipDecompressor` one-shot (the decompressor reads concatenated members, refuses trailing bytes, checks the header CRC, CRC-32 and length: `GZIP_ERR_HEADER`, `GZIP_ERR_CRC`, `GZIP_ERR_SIZE`), `gzipCompress` / `gzipDecompress`; `GzipFixedCompressor`, the same framing round `FixedDeflater` (`gzipFixedBound(n)`); `GzipWriter` (`write`, `flush`, `finish`, `bound`) and `GzipReader` (`feed` or `inputBuffer` / `commit`, `endInput`, `read` answering bytes, `GZIP_NEED_INPUT`, `GZIP_END` or an error, with a total `limit`) for bodies that arrive in pieces | RFC 1952: Node's gzip at every level, strategy and window, through both readers; the writer's output through Node's gunzip (`std_compress_node`, `std_compress`) |
|
|
197
|
+
| [`compress/zlib.ts`](https://github.com/amritk/nish/blob/main/std/compress/zlib.ts) | `ZlibCompressor(level)` / `ZlibDecompressor`, `zlibCompress` / `zlibDecompress`: the two-byte header and Adler-32 trailer (`ZLIB_ERR_HEADER`, `ZLIB_ERR_ADLER`, `ZLIB_ERR_DICTIONARY` for `FDICT`) | RFC 1950, against Node's `deflateSync` / `inflateSync` (`std_compress_node`) |
|
|
198
|
+
| [`compress/checksum.ts`](https://github.com/amritk/nish/blob/main/std/compress/checksum.ts) | `Crc32` (slicing-by-8 tables built once; `update(crc, buf, off, len)`), `crc32` for a one-off, `adler32`, `adler32Update`. Natively the compiler sends `update`'s walk to `runtime/runtime-checksum.c` where the CPU has PCLMULQDQ or the ARMv8 CRC32 instructions, chosen once per process; `NISH_CRC32=soft` pins the tables | The check values of "123456789", pieces against the whole, Adler-32's deferred reduction against its definition (`std_compress`); every kernel against a bit-at-a-time reference (`tests/simd/crc32.c`); both paths on the same programs (`tests/link/std_compress_crc_paths`, `std_compress_crc_diff`) |
|
|
199
199
|
|
|
200
200
|
## `nish/net` — the protocol stack
|
|
201
201
|
|
|
202
|
-
The protocol lanes of [WP34](
|
|
202
|
+
The protocol lanes of [WP34](https://github.com/amritk/nish/blob/main/docs/wp34-hosting-cs.md) §5, written on
|
|
203
203
|
`nish/crypto` (decision S2). Each module takes bytes and answers bytes:
|
|
204
204
|
randomness is an argument and nothing reads a socket or a clock, so a test
|
|
205
205
|
injects an RFC trace's values and replays it byte for byte. The carriers put
|
|
@@ -208,55 +208,55 @@ reads one, and it still takes its randomness as an argument.
|
|
|
208
208
|
|
|
209
209
|
| Module | What it is | Reproduces |
|
|
210
210
|
| --- | --- | --- |
|
|
211
|
-
| [`net/tls.ts`](
|
|
212
|
-
| [`net/tls/codec.ts`](
|
|
213
|
-
| [`net/tls/client.ts`](
|
|
214
|
-
| [`net/tls/schedule.ts`](
|
|
215
|
-
| [`net/tls/record.ts`](
|
|
216
|
-
| [`net/tls/record-server.ts`](
|
|
217
|
-
| [`net/tls/record-client.ts`](
|
|
218
|
-
| [`net/tls/verify.ts`](
|
|
219
|
-
| [`net/tls-tcp.ts`](
|
|
220
|
-
| [`net/quic-packet.ts`](
|
|
221
|
-
| [`net/quic-frame.ts`](
|
|
222
|
-
| [`net/quic-conn-params.ts`](
|
|
223
|
-
| [`net/quic-conn-ack.ts`](
|
|
224
|
-
| [`net/quic-conn-cid.ts`](
|
|
225
|
-
| [`net/quic.ts`](
|
|
226
|
-
| [`net/quic-listener.ts`](
|
|
227
|
-
| [`net/quic-recovery.ts`](
|
|
228
|
-
| [`net/quic-stream.ts`](
|
|
229
|
-
| [`net/quic-datagram.ts`](
|
|
230
|
-
| [`net/buffer-pool.ts`](
|
|
231
|
-
| [`net/dns-message.ts`](
|
|
232
|
-
| [`net/dns-config.ts`](
|
|
233
|
-
| [`net/dns-resolver.ts`](
|
|
234
|
-
| [`net/dns.ts`](
|
|
235
|
-
| [`net/hpack.ts`](
|
|
236
|
-
| [`net/qpack.ts`](
|
|
237
|
-
| [`net/http1-codec.ts`](
|
|
238
|
-
| [`net/http1-client-codec.ts`](
|
|
239
|
-
| [`net/fetch.ts`](
|
|
240
|
-
| [`net/outbound.ts`](
|
|
241
|
-
| [`net/tls/ticket.ts`](
|
|
242
|
-
| [`net/websocket.ts`](
|
|
243
|
-
| [`net/http1-server.ts`](
|
|
244
|
-
| [`net/url.ts`](
|
|
245
|
-
| [`net/http-fields.ts`](
|
|
246
|
-
| [`net/http2-frame.ts`](
|
|
247
|
-
| [`net/http2.ts`](
|
|
248
|
-
| [`net/http2-tls.ts`](
|
|
249
|
-
| [`net/http3-frame.ts`](
|
|
250
|
-
| [`net/http3.ts`](
|
|
251
|
-
| [`net/http3-server.ts`](
|
|
252
|
-
| [`net/webtransport.ts`](
|
|
253
|
-
| [`net/http-compress.ts`](
|
|
254
|
-
| [`net/router.ts`](
|
|
255
|
-
| [`net/handler.ts`](
|
|
256
|
-
| [`net/upstream.ts`](
|
|
257
|
-
| [`net/postgres.ts`](
|
|
258
|
-
| [`net/postgres-carrier.ts`](
|
|
259
|
-
| [`net/prefork.ts`](
|
|
211
|
+
| [`net/tls.ts`](https://github.com/amritk/nish/blob/main/std/net/tls.ts) | The server side of a TLS 1.3 handshake, ClientHello through the client's Finished. `new TlsServer(config, serverRandom, x25519Private)`, then `receive(level, buf, off, len)` with what the client sent, `takeOutput(level)` for what to send, and `signatureInput()` / `sign(signature)` for the CertificateVerify hand-off (`tlsSignEcdsaP256` signs for a P-256 key, DER-encoded; a `TlsP256Signer`, made once, gives the same DER from its `sign(priv, content)` and keeps nothing per signature). A `TlsServerConfig` names the certificate chain, the signature scheme, the ALPN protocols, whether the carrier is QUIC, the server's transport parameters, and extensions to pass through in EncryptedExtensions. The suites are `TLS_AES_128_GCM_SHA256`, `TLS_CHACHA20_POLY1305_SHA256` and `TLS_AES_256_GCM_SHA384` (in the client's order), the group x25519 (one HelloRetryRequest when the client offers it without a share), and `server_name`, ALPN and `quic_transport_parameters` are read; PSKs, 0-RTT, NewSessionTicket and client authentication are not. Every refusal is the alert to send — `receive` and `sign` answer 0 or a `TLS_ALERT_*` — never a panic, and a low-order x25519 share is refused. `restart(serverRandom, x25519Private)` hands a server's buffers, made once, to the slot's next connection, so a handshake allocates nothing that outlives it under any suite (TLS-3); it copies the key into the server's own `Secret` and wipes the caller's array, and `wipe()` zeroes every secret the server holds | RFC 8446 §4, §7; RFC 8448 §3 byte for byte, and §5's transcript |
|
|
212
|
+
| [`net/tls/codec.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/codec.ts) | The handshake messages as bytes: `tlsReadClientHello` into a reusable `TlsClientHelloView` of windows into the message, and `tlsParseClientHello` into a `TlsClientHello` of copies (structure checked, an alert on a malformed or duplicated field); each message's `tlsWrite*` writer into the caller's array at an offset, and `tlsEncodeServerHello`, `tlsEncodeHelloRetryRequest`, `tlsEncodeEncryptedExtensions`, `tlsEncodeCertificate`, `tlsEncodeCertificateVerify`, `tlsEncodeFinished` and `tlsCertificateVerifyContent`. The wire's constants: handshake and extension types, versions, the x25519 group, the two signature schemes, and the `TLS_ALERT_*` descriptions | RFC 8446 §4, RFC 6066 §3, RFC 7301 §3.1, RFC 9001 §8.2 |
|
|
213
|
+
| [`net/tls/client.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/client.ts) | The client side of a TLS 1.3 handshake, ClientHello through the client's Finished, sans-IO: `new TlsClient(new TlsClientConfig(serverName, alpn), random32, x25519Key, p256Key)`, `start`, `receive(level, data, off, len)` and `takeOutput(level)` by encryption level. It offers the three TLS 1.3 suites and X25519 (and secp256r1 when the config's `groups` names it), answers one HelloRetryRequest, checks the downgrade sentinels, and stops at `TLS_CLIENT_STATE_WAIT_AUTH` with `peerCertificates`, `verifyScheme`, `verifyContent` and `verifySignature` out: the caller checks the chain and signature and calls `authenticate(ok)` or `reject(alert)`. TLS 1.2 and client certificates are refused with an alert | `tests/link/net_tls_client_handshake` against `TlsServer`; `tests/link/net_tls_client_refusals`, each refused server flight with its alert |
|
|
214
|
+
| [`net/tls/schedule.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/schedule.ts) | The key schedule over SHA-256 or SHA-384: `tlsEarlySecret`, `tlsHandshakeSecret`, `tlsMasterSecret`, `tlsDeriveSecret`, `tlsExpandLabel`, `tlsFinishedVerifyData`, and `tlsTrafficKey` / `tlsTrafficIv` for a record layer — these one-shots take every secret as a `Secret<u8[]>` and answer one (`tlsFinishedVerifyData` answers the public verify data as a plain array; transcript hashes, labels and contexts stay plain), and each wipes its derived salts and `finished_key` before it returns — with in-place forms over an `HkdfScratch` (`tlsDeriveSecretInto`, `tlsMasterSecretInto`, `tlsFinishedVerifyDataInto`, `tlsTrafficKeysInto`), and forms into a `Secret<u8[]>` the caller already holds (`tlsDeriveSecretHeld`, `tlsHandshakeSecretHeld`, `tlsMasterSecretHeld`; `dst` may be the source), `tlsFinishedVerifyDataHeld`, the plain verify data from a held base key, and `tlsTrafficKeysFromHeld`, a held secret's key and IV into plain arrays, which answers `false` for a suite or length it does not know. `TlsTranscript` is the running transcript hash, with HelloRetryRequest's `message_hash` rule, `reset` for the next connection and `hashInto` the caller's array. The suite constants and `tlsSuiteHashLength` / `tlsSuiteKeyLength` | RFC 8446 §4.4.1, §7.1, §7.3 |
|
|
215
|
+
| [`net/tls/record.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/record.ts) | The record layer (RFC 8446 §5): `TlsRecordReader` cuts a stream into records in a buffer allocated once, refusing a header of unknown type or a body past 2^14 + 256 bytes before the body is waited for; `TlsRecordProtection` is one direction, cleartext until `install(suite, secret)`, or `installHeld(suite, secret)` from a secret held as a `Secret<u8[]>`, ratcheted in place by `advanceHeld(secret)` — then `seal` and `open` a record into the caller's buffer — the nonce from the sequence number, the header as additional data, the inner content type and zero padding found without branching on the padding. `tlsNextTrafficSecret` is KeyUpdate's next secret, taken and answered as a `Secret<u8[]>`. Every refusal is the alert, negated | RFC 8448 §3's nine records byte for byte; ChaCha20-Poly1305, AES-256-GCM-SHA384 and padding against Python's `cryptography` |
|
|
216
|
+
| [`net/tls/record-server.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/record-server.ts) | `TlsRecordServer`: a TLS 1.3 server over a byte stream, sans-IO, wrapping a `TlsServer` in records. `receive` the client's bytes, send `output[outputStart .. outputEnd)` and `consume` what was sent, `read` and `write` application data, `sign` when asked, `keyUpdate`, `close`; `interest()` answers what it wants next as `TLS_RECORD_*` bits. It adds the compatibility `change_cipher_spec`, alerts both ways, KeyUpdate both ways and on a schedule, a limit of `TLS_RECORD_IDLE_LIMIT` (16) records in a row that carry nothing, a cap of `TLS_RECORD_MAX_KEY_UPDATES` (64) KeyUpdates from the client, and fixed buffers per connection; the handshake is sealed straight out of `TlsServer`'s buffers | RFC 8448 §3 replayed at every cut of the client's stream |
|
|
217
|
+
| [`net/tls/record-client.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/record-client.ts) | `TlsRecordClient`: a TLS 1.3 client over a byte stream, sans-IO, wrapping a `TlsClient` in records — the mirror of `TlsRecordServer`. `receive` the server's bytes, send `output[outputStart .. outputEnd)` and `consume` what was sent, `read` and `write` application data, `keyUpdate`, `close`; `interest()` answers `TLS_RECORD_*` bits, `TLS_RECORD_AUTH` (64) when the server waits on the caller's verdict. The compatibility `change_cipher_spec`, alerts and KeyUpdate both ways, and the server's idle and KeyUpdate limits | `tests/link/net_tls_client_record` against `TlsRecordServer`: each suite, a HelloRetryRequest, data, KeyUpdate and close_notify from each side |
|
|
218
|
+
| [`net/tls/verify.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/verify.ts) | The client's authentication seam: `tlsClientAuthenticate(conn, trust, nowMs)` at `TLS_RECORD_AUTH` checks the CertificateVerify under the leaf's key (ECDSA P-256, ECDSA P-384, RSA-PSS rsae on 2048 bits or more) and then the chain to a root in `trust` (`tlsClientSystemTrust()` reads the system's), and calls `authenticate(true)` or `reject(alert)`: `bad_certificate`, `decrypt_error`, `certificate_expired` or `unknown_ca`. The chain is `nish/crypto/pki`'s `pkiVerifyServer`: a path to a root of `trust`, every certificate valid at `nowMs`, and the name in the leaf's subjectAltName; there is no switch that skips it | `net_tls_client_tcp` |
|
|
219
|
+
| [`net/tls-tcp.ts`](https://github.com/amritk/nish/blob/main/std/net/tls-tcp.ts) | `TlsTcpServer`: TLS 1.3 over `nish:net` TCP for a program that owns its loop — a pool of `TlsRecordServer` slots, each built on the first accept into it, `accept(serverRandom, ephemeralPrivate)` into a free one, `readable` / `writable` when the loop says so, `signP256` with a `Secret` key, and `read`, `write`, `shutdown` and `close` answering as `nish:net`'s calls do. `accept` copies the server random and the ephemeral key into the slot, wipes the caller's key array, and refuses either with -22 when it is not 32 bytes. Every call answers the slot's interest, whose low two bits are `pollModify`'s events. Each slot keeps one `TlsServer`, restarted by `accept`, so a thousand connections through the pool leave the arena where the first did, under any suite (TLS-3). `TlsTcpClient` is the client over one socket: `new TlsTcpClient(addr, config, trust)` connects without blocking, `readable` / `writable` drive it, and at `TLS_RECORD_AUTH` it authenticates the server itself through `nish/net/tls/verify`; `tlsConnect(addr, serverName, alpn, timeoutMs)` runs the handshake to the end in a loop of its own, its last flight sent, and answers the client, open or with its failure (-110 on the timeout); `writeAll` and `readWait` write and read under a timeout of their own, for a program with no loop. The address is numeric and the TLS name always the caller's. Native only, as `nish:net` is | openssl s_client under all three suites and Node's `https`, from `tests/run.js`; RFC 8448 §3 over loopback from a Nish client; the client against `TlsTcpServer` (`net_tls_client_tcp`) and openssl s_server under each suite, group, HRR, ALPN, SNI and leaf key, and through an RSA and a P-384 intermediate, from `tests/run.js` |
|
|
220
|
+
| [`net/quic-packet.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-packet.ts) | QUIC version 1's packets (WP34 Q1). The variable-length integer (`quicVarintPush`, `quicVarintPushSized`, `quicVarintRead`, `quicVarintLength`, `quicVarintSize`); packet-number length choice and recovery (`quicPacketNumberLength`, `quicPacketNumberDecode`); the long and short headers (`quicLongHeader`, `quicShortHeader`, and `quicParseHeader` into a `QuicHeader`, whose `end` is where the next coalesced packet starts); the Initial secrets from the client's first DCID (`quicInitialSecrets`); the packet keys from a traffic secret held as a `Secret` from `nish:secret` (`quicKeys(slot, aead, secret)` into a `QuicKeysSlot` the caller owns, for AES-128-GCM, AES-256-GCM and ChaCha20-Poly1305) and a key update, its secret, answered as a `Secret` (`quicKeyUpdateSecret`), and the next generation's keys with the header-protection key kept (`quicKeysUpdate(slot, keys, next)`, into a slot of its own), neither keeping anything in the arena, and `quicKeysWipe`, which zeroes a `QuicKeys`, AES schedules included; the same four in place for a connection slot, into a `QuicKeysSlot` it makes once and over its `HkdfScratch` (`quicInitialSecretsInto`, `quicKeysInto`, `quicKeyUpdateSecretInto`, `quicKeysUpdateInto`; `QuicKeysSlot.wipe` zeroes the keys and AES schedules), with `quicKeysHeldInto(slot, aead, secret)` and `quicKeyUpdateSecretHeldInto(dst, src)` for a secret held as a `Secret<u8[]>`, the update in place when `dst` is `src`; packet and header protection (`quicSealPacket`, and `quicOpenPacket` or its two halves `quicRemoveHeaderProtection` and `quicDecryptPacket` into a `QuicPacket`); and, for a connection that allocates nothing per packet (WP34 Q4), `quicParseHeaderInto` (windows, no copy), `quicPutLongHeader` / `quicPutShortHeader` / `quicVarintPut` in place, `quicSealInPlace`, and `quicUnprotectHeader` / `quicDecryptPayload`, which store only numbers into the `QuicPacket`; and the Retry integrity tag (`quicRetryIntegrityTag`, `quicRetryPacket`, `quicRetryVerify`). Every parse is bounds-checked against the datagram and answers a `QUIC_ERR_*` code rather than panicking: a truncated header, a clear fixed bit, another version (with its connection IDs read, for Version Negotiation), a connection ID over 20 bytes, a Length past the datagram, a packet too short to sample, a payload that does not authenticate, and reserved bits set in one that does; keys `quicKeys` did not make are `QUIC_ERR_KEYS`, a caller's error rather than a forgery. Long headers always carry a two-byte Length, so a header's size does not move with its payload, and `quicSealPacket` refuses a header whose Length or packet number disagrees with what it seals. The builders answer `null` for an argument out of range. The keys in a slot the caller owns are the caller's to wipe, with `QuicKeysSlot.wipe` or `quicKeysWipe` ([QUIC-1](../docs/security/quic.md)) | RFC 9000 §16, §17 and Appendix A.1–A.3; RFC 9001 §5 and Appendix A.1–A.5 (client and server Initial, Retry, ChaCha20 short header) |
|
|
221
|
+
| [`net/quic-frame.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-frame.ts) | QUIC version 1's frames (WP34 Q2). `quicParseFrame` reads any frame of RFC 9000 §19, 0x00 to 0x1e, into a reused `QuicFrame`, leaving a CRYPTO or STREAM frame's data in the packet (`dataStart`, `dataLength`) and answering 0 or the transport error the RFC names: FRAME_ENCODING_ERROR for a truncated field, a length past the packet, an ACK range below zero, an offset past 2^62 − 1, a stream count past 2^60, an empty NEW_TOKEN or a bad NEW_CONNECTION_ID, and PROTOCOL_VIOLATION for a frame type spelled in more bytes than it needs. The writers (`quicPushAck`, `quicPushCrypto`, `quicPushStream`, `quicPushValue`, `quicPushStreamValue`, `quicPushStreamError`, `quicPushNewConnectionId`, `quicPushPathData`, `quicPushConnectionClose`, `quicPushNewToken`, `quicPushTypeOnly`, `quicPushPadding`) answer `false` rather than write a frame that would not parse; `quicCryptoOverhead` and `quicStreamOverhead` size a frame's header before it is written. `quicFrameAllowed` is §12.4's table of which frames each packet type carries, `quicFrameAckEliciting` §13.2's rule, and `QUIC_ERROR_*` the transport error codes of §20.1. RFC 9221's DATAGRAM frames (0x30, 0x31) are read too (WP34 Q4); every writer has a `put` form (`quicPutAck`, `quicPutStream`, …, `quicPutDatagram`) that writes in place into a caller's buffer before an end and answers the offset past it, which the `push` form calls, and NEW_CONNECTION_ID's ID and token are left in the packet as windows | RFC 9000 §12.4, §19, §20; RFC 9221 §4; RFC 9001 A.2's and A.3's frames |
|
|
222
|
+
| [`net/quic-conn-params.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-conn-params.ts) | Transport parameters (RFC 9000 §18): a `QuicTransportParameters` that starts at every default, `quicEncodeTransportParameters` (a parameter only when it is not its default) and `quicParseTransportParameters`, with in-place forms that refill a slot's own array and object (`quicEncodeTransportParametersInto`, `quicParseTransportParametersInto`), which refuses with TRANSPORT_PARAMETER_ERROR, naming the parameter, a parameter that does not frame, a known one sent twice, a varint-valued one that is not exactly one varint, a server-only one from a client, and every value §18.2 bounds. Unknown parameters are skipped. RFC 9221's `max_datagram_frame_size` is read and written too (WP34 Q4) | RFC 9000 §7.4, §18; RFC 9221 §3; RFC 9001 A.2's client parameters |
|
|
223
|
+
| [`net/quic-conn-ack.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-conn-ack.ts) | `QuicAckRanges`: the packet numbers one space received, as at most 32 disjoint ranges, for duplicate detection (`record`, `contains`) and for the ACK frame written from them (`pushAck`, or `putAck` in place). Past 32 ranges the lowest is dropped and a floor raised over it, below which every packet number reads as already received | RFC 9000 §12.3, §13.2, §19.3 |
|
|
224
|
+
| [`net/quic-conn-cid.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-conn-cid.ts) | `QuicCidTable`: the connection IDs a connection issued (routed by `ownsLocal`, retired by the peer with `retireLocal`) and the ones its peer issued (`addPeer`, with Retire Prior To retiring older ones into `retirePending`, and `currentPeer` the one sent to). A reused sequence number, an ID under two sequence numbers and retiring the ID a packet was sent to are PROTOCOL_VIOLATION; more active IDs than advertised, or more owed retirements than twice that, CONNECTION_ID_LIMIT_ERROR. Its slots are fixed when it is made (WP34 Q4): an ID is copied into one, `addPeerAt` / `ownsLocalAt` / `retireLocal` read it from the packet in place, and `reset` empties the table for the next connection | RFC 9000 §5.1, §19.15, §19.16 |
|
|
225
|
+
| [`net/quic.ts`](https://github.com/amritk/nish/blob/main/std/net/quic.ts) | The server side of a QUIC connection (WP34 Q2): `new QuicConnection(config, entropy)`, `receive(datagram, now)`, `signatureInput()` / `sign(signature)`, `takeDatagram(now)` until `null`, `deadline()` / `handleTimer(now)` for its timers, and `readStream()` / `writeStream(id, data, fin)` once connected; `updateKeys()`, `close(code)` and `release()`; `acceptRetry(originalDcid, retryScid)` for a connection a Retry token opened; `quicStatelessResetToken(key, cid)`. TLS 1.3 runs over CRYPTO frames at the Initial, Handshake and 1-RTT levels (`TlsServer` with `quic` set), the keys of each level come from `nish/net/quic-packet` as the secrets appear and are discarded and wiped as RFC 9001 §4.9 says, transport parameters go both ways and the client's are checked, every ack-eliciting packet is acknowledged in its space, the client gets new connection IDs once the handshake is confirmed, PATH_CHALLENGE is answered, and an ack-eliciting Initial is padded to 1200 bytes and held to three times what the unvalidated client sent. A protocol violation closes the connection with the RFC's transport error, a TLS alert with CRYPTO_ERROR; a packet that cannot be used is dropped and counted. The idle timeout is the smaller of both sides' `max_idle_timeout`, at least three probe timeouts, and closes silently; key update runs both ways, the client's followed (at most `QUIC_CONN_MAX_KEY_UPDATES`, QUIC-4) and the server's started by `updateKeys()`, with the next read keys derived in advance and the previous kept a probe timeout for reordered packets; every connection ID carries the stateless reset token the configuration's static key gives it. Loss recovery is `nish/net/quic-recovery`'s (WP34 Q3): what a lost packet carried (CRYPTO, STREAM, HANDSHAKE_DONE and the connection-ID frames) is sent again, a probe timeout sends a probe (two datagrams in the Initial and Handshake spaces, `QUIC_CONN_HANDSHAKE_PROBES`, each with the oldest CRYPTO data not acknowledged), a client that shows it lacks the handshake flight (an Initial repeating CRYPTO data, an ack-eliciting Handshake packet) gets it again at once, up to `QUIC_CONN_EARLY_RESENDS` times a connection (QUIC-8), and NewReno's window holds back everything but ACKs and probes. Streams (WP34 Q4) are `nish/net/quic-stream`'s, both ways and both kinds, through `nextStreamEvent()`, `openStream(bidirectional)`, `streamRead(id, buf, at, length)`, `streamWrite(id, buf, from, length, fin)`, `streamReset(id, code)` and `streamStopSending(id, code)`; RFC 9221's datagrams through `sendDatagram(buf, from, length)`, `readDatagram(buf, at, cap)` and `maxDatagramPayload()` when `config.maxDatagramFrameSize` offers them. Nothing is allocated per packet (QUIC-3): `receiveWindow(buf, off, len, now)` and `takeDatagramInto(out, at, now)` open and build packets in place inside an arena block, and `reset(entropy)` makes the connection a reusable slot for the next client. The AEAD limits of RFC 9001 §6.6 are kept: a 1-RTT write key under AES-GCM seals at most 2^23 packets, the server starting a key update at three quarters of that and closing with AEAD_LIMIT_REACHED at it if none can be made, and packets that fail to authenticate past the integrity limit (2^52 for AES-GCM, 2^36 for ChaCha20-Poly1305) close the connection the same way (QUIC-6). One Application Data packet number in every 128 to 1024 is skipped, and an ACK of a skipped one is PROTOCOL_VIOLATION, an optimistic ACK (RFC 9000 §21.4, QUIC-7) | RFC 9000 §10.1, §10.3.2, §21.4; RFC 9001 §4, §5.7, §6, §8; RFC 9221; a handshake and stream echo, and an update each way, against aioquic, recorded and replayed byte for byte |
|
|
226
|
+
| [`net/quic-listener.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-listener.ts) | What a QUIC server answers to a datagram no connection owns (WP34 Q2, second part): `new QuicListener(config, entropy)` and `handleWindow(buf, at, len, address, now, answer)`, which reads the datagram `buf[at .. at + len)` in place and fills `answer`, a `QuicListenerAnswer` made once with `QUIC_LISTENER_ANSWER_SIZE` of room and refilled in it, so a datagram costs no arena memory whatever it is answered with (H3-3); `handle(datagram, address, now)` does the same over a whole array into an answer of its own. An answer's `kind` is `QUIC_LISTEN_DROP`, `_ACCEPT` (make a `QuicConnection`; after a Retry, `acceptRetry` it with the answer's IDs), or a `reply` to send: `_VERSION_NEGOTIATION` for another version in a full-sized datagram, `_RETRY` with an address-validation token when `config.retry` is set, `_INVALID_TOKEN` for a Retry token that is forged, moved, expired or for another ID, and `_STATELESS_RESET` for a short header to an ID the server has no connection for. A token is 128 bits of HMAC-SHA256 under `config.retryTokenKey` over its issue time, the original DCID, the new ID and the client's address and port, good for `config.retryTokenLifetime`. A reset ends in `quicStatelessResetToken` of the packet's ID, is always shorter than the datagram it answers and never under 21 bytes, and is rate limited. Its own unpredictable bytes come from an HMAC generator seeded once by the caller. `release()` wipes the seed, the generator, the scratch, the INVALID_TOKEN keys and the spare answer `handle` keeps, and `handle` and `handleWindow` answer nothing after it; its other methods still run, under the zeroed seed, and are the caller's not to call (QUIC-5). For the carrier's send loop, `quicListenerTakePaced(conn, now)` takes a connection's next datagram only when its pacer has the credit, and `quicListenerPaceTime(conn, now)` says when it will (WP34 Q3); `quicListenerTakeFlight(conn, now, buf, at, cap, flight)` writes a paced flight back to back for one GSO send, with the `UDP_SEGMENT` size in its `QuicFlight`, and `quicListenerReceiveSegments` hands a connection a GRO receive segment by segment, in place (WP34 Q4) | RFC 9002 §7.7; RFC 9000 §5.2.2, §6, §8.1, §10.3, §17.2.1, §17.2.5; Version Negotiation, Retry, a stateless reset and the idle timeout against aioquic, recorded and replayed byte for byte |
|
|
227
|
+
| [`net/quic-recovery.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-recovery.ts) | QUIC loss detection and congestion control (WP34 Q3), sans-IO, for the server side: `QuicRecovery` holds one `QuicSentPackets` ring per packet number space (16 packets for Initial and Handshake, 128 for Application Data, fixed when it is made); `onPacketSent(space, pn, size, now)` records an ack-eliciting packet and answers its slot, `onAck(space, ranges, count, ackDelay, now)` reads an ACK frame's ranges and lists the slots it acknowledged and showed lost (`acked`, `lost`), `deadline(amplificationBlocked)` and `onTimeout(now, amplificationBlocked)` run the time-threshold loss timer and the probe timeout with its backoff, `discardSpace` drops a space whose keys are gone, `evictOldest` frees a slot for a probe. The RTT estimate (`latestRtt`, `smoothedRtt`, `rttVar`, `minRtt`, `probeTimeout()`), loss by packet threshold 3 and time threshold 9/8, NewReno (slow start, one halving per recovery period, congestion avoidance, the 2400-byte floor, persistent congestion) and a pacer at 5/4 of the window per smoothed RTT (`pacerDelay`, `onPaced`), all in integer milliseconds and bytes | RFC 9002 §5–7, Appendices A and B, worked by hand in `net_quic_recovery` |
|
|
228
|
+
| [`net/quic-stream.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-stream.ts) | QUIC streams (WP34 Q4), sans-IO, which `nish/net/quic` drives: `QuicStreams` holds one `QuicStream` slot per stream that may be open at once (the client's bidirectional and unidirectional limits and the server's own), each with a fixed receive and send buffer, found by ID through a hash index. The send and receive state machines of §3 (`QUIC_SEND_*`, `QUIC_RECV_*`), reassembly into the receive buffer, flow control per stream and for the connection both ways (MAX_STREAM_DATA and MAX_DATA raised as the application reads, STREAM_DATA_BLOCKED and DATA_BLOCKED once per limit), the stream limits (MAX_STREAMS raised as the client's streams finish, STREAMS_BLOCKED at the client's), STOP_SENDING answered with RESET_STREAM, and the refusals STREAM_STATE_ERROR, STREAM_LIMIT_ERROR, FLOW_CONTROL_ERROR and FINAL_SIZE_ERROR. The application's answers are `QUIC_STREAM_OK`, `QUIC_STREAM_END` and `QUIC_STREAM_ERR_*` | RFC 9000 §2–§4, §19.4–§19.14 |
|
|
229
|
+
| [`net/quic-datagram.ts`](https://github.com/amritk/nish/blob/main/std/net/quic-datagram.ts) | `QuicDatagramQueue` (WP34 Q4): a fixed ring of DATAGRAM payloads, one each way per connection, copied in (`push`) and out (`pop`, or `putFrame` as a DATAGRAM frame), refusing past a full ring and counting what it drops | RFC 9221 §5 |
|
|
230
|
+
| [`net/buffer-pool.ts`](https://github.com/amritk/nish/blob/main/std/net/buffer-pool.ts) | `BufferPool(name, size, ceiling)`: a pool of fixed-size `u8[]` buffers that a server lends across connections. `take()` answers a free buffer, most recently given back first, or a new one while fewer than `ceiling` are lent; `give(buf, used)` zeroes the first `used` bytes and takes it back, and `giveSecret(buf, used)` wipes the whole buffer with `secureZero`; `lent()` counts the buffers out. A `size` not above 0 or a `ceiling` below 1, taking past the ceiling, giving back a buffer of the wrong length, a used extent outside 0 to `size`, and a give-back with none lent panic, naming the pool | Reuse, order, zeroing, the ceiling and a warmed loop that allocates nothing (`tests/link/net_buffer_pool`, and its `_bad_` cases) |
|
|
231
|
+
| [`net/dns-message.ts`](https://github.com/amritk/nish/blob/main/std/net/dns-message.ts) | The DNS wire codec, sans-IO: `dnsBuildQuery(out, id, name, qtype)` writes one A or AAAA question with RD set into the caller's buffer (`DNS_QUERY_MAX_BYTES`, 271, always fits) and answers its length or a negative `DNS_ERR_*` for a name that is not one. `dnsParseReply(msg, len, query, queryLen, port, reply)` reads the reply into a reused `DnsReply`: it is refused unless its ID, QR, opcode and one question match the query's; it answers NOERROR, NXDOMAIN, SERVFAIL or REFUSED, or a `DNS_ERR_*` for a compression pointer that is not backwards, a seventeenth hop, a name past 255 bytes, a count the bytes cannot hold, and RDATA past the end or of the wrong length. The CNAME chain from the question's name is followed and the first `DNS_MAX_ANSWERS` (8) addresses on it are kept as 18-byte addresses with their TTLs, the rest counted; TC set reads no record. A window outside its array and a port or ID that is not 16 bits panic | RFC 1035 §4, RFC 2181 §8, RFC 4343; captured replies for example.com, a CNAME chain, NXDOMAIN and nine answers, and a seeded mutation pass (`tests/link/std_dns_message`) |
|
|
232
|
+
| [`net/dns-config.ts`](https://github.com/amritk/nish/blob/main/std/net/dns-config.ts) | `/etc/resolv.conf` and `/etc/hosts`, read from bytes into fixed tables: `dnsParseResolvConf` fills a `DnsConfig` (up to 3 `nameserver`s on port 53, 6 `search` or `domain` names in 256 bytes, `options ndots timeout attempts` clamped to 15, 30 and 5; no server gives 127.0.0.1 and ::1), `dnsParseHosts` fills a `DnsHosts` (256 addresses, 1024 names in 16 KiB), and `dnsHostsFind` answers the addresses a name has, without case, in file order. What does not fit is dropped and counted, and a line over 1024 bytes is skipped and counted. `dnsParseAddress` reads a strict IPv4 or IPv6 literal (no octal, no zone) into the 18-byte form | glibc's `resolv.conf(5)` and `hosts(5)`; RFC 4291 §2.2 (`tests/link/std_dns_config`) |
|
|
233
|
+
| [`net/dns-resolver.ts`](https://github.com/amritk/nish/blob/main/std/net/dns-resolver.ts) | The DNS stub resolver with no I/O of its own: `DnsResolver(config, hosts, maxLookups)` takes `start(name, port, now)` and answers a lookup's index, or `DNS_RESOLVE_BUSY` past `maxLookups` (at most 64). Each lookup asks A and AAAA at once; the caller sends what `nextOutgoing(out, to, meta, id)` gives (a datagram, a TCP connection or a close, with the caller's random ID), feeds what comes back to `receive` or `receiveStream`, calls `tick(now)` once `nextDeadlineNanos()` passes, and reads `result(lookup, out, ttls)`: the count of 18-byte addresses, A first, each with its TTL, or `DNS_RESOLVE_PENDING`, `NOT_FOUND`, `NO_ANSWER` or `CNAME`. Servers are tried in turn for `attempts` rounds of `timeout`; TC asks again over TCP (a reply over `DNS_TCP_MAX_REPLY`, 16 KiB, fails the server); a CNAME left unanswered is followed by a query of its own, at most 8 hops and never back to the start; the search list and `ndots` run as resolv.conf(5) says. `localhost`, an address literal and a hosts entry answer with no packet. A reply from another address or port, or with another ID or question, is dropped, counted (`droppedSource`, `droppedId`, `droppedQuestion`, `droppedMalformed`) and the query keeps waiting. It never imports `nish:net`, reads a clock or a file, and builds for `wasm32`; every buffer is made in the constructor | RFC 1035 §4.2 and §7.3; RFC 5452 (forgery resilience); RFC 7766 (DNS over TCP); resolv.conf(5); scripted packets for every drop, bound and error path (`tests/link/std_dns_resolver`) |
|
|
234
|
+
| [`net/dns.ts`](https://github.com/amritk/nish/blob/main/std/net/dns.ts) | `nish/net/dns-resolver` on `nish:net` sockets, on the caller's loop: `DnsClient(resolver, loop, tokenBase)` opens a new UDP socket on a kernel-chosen port for each try, or a TCP connection for a TCP one, under a token of its range (`owns`), draws each query ID from `crypto.getRandomValues`, and never waits: the program's `pollWait` hands it `ready(token, events, now)`, calls `tick(now)` and waits no longer than `timeoutMs(now, own)`. `resolveHost(name, timeoutMs, port, out, ttls)` reads `/etc/resolv.conf` and `/etc/hosts` and blocks on a loop of its own, for a CLI or start-up; `dnsLookupWith` does the same with a configuration given. | A Nish DNS server on loopback: UDP, TCP, TC, CNAME, the search list, a dead server, `attempts`, the hosts file and spoofed replies (`tests/link/std_dns`); the capability report (`std_dns_caps`); a flat arena over 2,000 lookups (`std_dns_soak`) |
|
|
235
|
+
| [`net/hpack.ts`](https://github.com/amritk/nish/blob/main/std/net/hpack.ts) | HPACK, HTTP/2's header compression: `HpackEncoder` and `HpackDecoder`, one per direction of a connection, each with its `HpackTable`; and beneath them the §5.1 integer (`hpackEncodeInteger` / `hpackDecodeInteger`), the §5.2 string (`hpackEncodeString` / `hpackDecodeString`) and Appendix B's Huffman code (`HpackHuffman`, `hpackHuffmanEncode`, `hpackHuffmanDecode`, `hpackHuffmanLength`). Names and values are `u8[]`. The encoder takes the representation and the Huffman choice per field, and sends a never-indexed field as a literal even when a table holds it. The decoder answers a negative `HPACK_ERR_*` code, and stays spent, for a truncated block, an integer past 2^31 - 1, an index of zero or out of range, bad Huffman padding, EOS, a size update above the SETTINGS limit, after a field or missing when a lowered limit requires one, and for a block that would copy more than `HPACK_COPY_FACTOR` times the larger of the table's size and the block's length into and out of the table (`HPACK_ERR_COPY_BUDGET`; `tableCopied` counts those octets); a header list past the caller's limit is decoded to the end, its excess dropped, and answered with `HPACK_LIST_TOO_LARGE`, which is not fatal. A window outside its buffer, a negative size or limit, and an integer prefix outside 1 to 8 bits are the program's mistakes and panic | RFC 7541 Appendix C (C.1–C.6, with the dynamic table after every step) and every code of Appendix B |
|
|
236
|
+
| [`net/qpack.ts`](https://github.com/amritk/nish/blob/main/std/net/qpack.ts) | QPACK, HTTP/3's field compression, with a dynamic table of capacity 0: `QpackEncoder` and `QpackDecoder`, one per connection. The encoder writes a field section that starts `00 00` (Required Insert Count 0, Base 0) with `beginSection`, then one field line per `encodeField(out, name, off, len, value, off, len, neverIndexed, huffman)`: an Indexed Field Line when the 99-entry static table holds the field, else a literal with the name by static index when it can, each string Huffman-coded when that is shorter and `huffman` is set, and a never-indexed field always a literal with the N bit. The decoder answers a section's fields as windows into its own `bytes` (`nameStart`, `nameLength`, `valueStart`, `valueLength`, `neverIndexed`, `count`), its arrays reused from section to section. It answers QPACK_DECOMPRESSION_FAILED, and stays spent, for a truncated section, an integer past 2^62 − 1, a non-zero Required Insert Count, a negative Base, any dynamic-table reference, a static index past 98 and bad Huffman; a section past `maxFieldSectionSize` answers `QPACK_SECTION_TOO_LARGE`, which is not fatal. `receiveEncoderStream` accepts Set Dynamic Table Capacity 0 and answers QPACK_ENCODER_STREAM_ERROR for any other capacity, Insert or Duplicate; the encoder's `receiveDecoderStream` accepts Stream Cancellations, split anywhere, and answers QPACK_DECODER_STREAM_ERROR for a Section Acknowledgment or Insert Count Increment; `reason` names the rule. `qpackPushStreamCancellation` is the one instruction it may send. The Huffman code is `nish/net/hpack`'s. A window outside its buffer, a negative limit and a stream ID outside 0 to 2^62 − 1 panic | RFC 9204 Appendix B.1, every Appendix A entry by its index, B.2 to B.5's instructions refused, and a request matched byte for byte with pylsqpack's encoding |
|
|
237
|
+
| [`net/http1-codec.ts`](https://github.com/amritk/nish/blob/main/std/net/http1-codec.ts) | `Http1Parser`, an incremental HTTP/1.1 request parser: `feed(buf, off, len)` any slice, then `next()` answers `HTTP1_NEED_MORE`, `HTTP1_HEAD` (`method`, `target`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength` or `chunked`, `keepAlive`, `upgrade`), `HTTP1_BODY` (a window onto its buffer, de-chunked), `HTTP1_END`, or one of three final events: `HTTP1_UPGRADE` (`upgradeBytes()` and `declineUpgrade()`), `HTTP1_CLOSED` and `HTTP1_ERROR` with the `status` to answer. It refuses with 400 every request-smuggling shape — `Content-Length` with `Transfer-Encoding`, a repeated or non-digit `Content-Length`, an obs-fold, a bare LF, whitespace before a colon, a malformed chunk size or terminator — and with 413, 414, 431, 501 and 505 what those mean. The head is also kept as spans in a buffer reused from request to request (`head`, `spans`, `methodIs`, `targetIs`, `headerIndex`, `headerIs`, `upgradeIs`), so a parser with `keepText` off allocates nothing per request; `maxChunk` caps each `HTTP1_BODY`, and `restart()` readies it for the next connection. The writer is `http1ResponseHead(status, reason, names, values, bodyLength)`, which adds the framing field itself and refuses (`null`) a CR, LF or other control character anywhere a caller's string lands, and `http1Chunk` and `http1LastChunk` for a chunked body; `http1WriteResponseHead` and `http1WriteChunk` write the same bytes into a caller's buffer at an offset, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED`, and the first takes a date's bytes to send as a `Date` field on a final response, unless the caller's fields hold one | RFC 9112 §2–§7 and §9.3, RFC 9110 §5, §6.2, §7.2 and §7.8; a corpus fed at every split point (`tests/link/net_http1`) |
|
|
238
|
+
| [`net/http1-client-codec.ts`](https://github.com/amritk/nish/blob/main/std/net/http1-client-codec.ts) | The client side of HTTP/1.1, sans-IO. The writer is `http1RequestHead(method, target, names, values, bodyLength)`, `Host` first and the framing field added for `bodyLength` (`Content-Length`, `HTTP1_CHUNKED` or `HTTP1_NO_BODY`), and `http1WriteRequestHead` into a caller's buffer, answering `HTTP1_NO_ROOM` or `HTTP1_REFUSED` with nothing written; it refuses a method that is not a token, a target that is not a form the method may send, a CR, LF or control byte in a value, not exactly one `Host`, the caller's own framing fields and a `Connection: upgrade`; `http1UpgradeRequestHead(method, target, names, values, upgrade)` writes `Upgrade` and `Connection: Upgrade` itself, with no body. `Http1ResponseParser(maxStatusLine, maxHeaderBytes, maxHeaders, maxBody, maxTrailerBytes)` is told what was asked (`expect(head, connect)`, then `expectUpgrade()` for a request that asked to switch protocols), fed any slice, and `next()` answers `HTTP1_INFORMATIONAL`, `HTTP1_HEAD` (`status`, `reason`, `minor`, lowercased `names` beside `values`, `header(name)`, `contentLength`), `HTTP1_BODY` (a window, de-chunked), `HTTP1_END` with `keepAlive()`, or the final `HTTP1_UPGRADE` (a 2xx to CONNECT or a 101 asked for, `tunnelBytes()`), `HTTP1_CLOSED` and `HTTP1_ERROR`; `finish()` reports the close that ends a body read until close, and anywhere else is `HTTP1_E_TRUNCATED`. Every smuggling shape is refused with its own `HTTP1_E_*` code, as is a 101 not asked for, more than `HTTP1_MAX_INFORMATIONAL` interim responses and a chunk-size line over `HTTP1_MAX_CHUNK_LINE`. Record: [`docs/security/http1-client-codec.md`](../docs/security/http1-client-codec.md) | RFC 9112 §4–§7 and §9.3, RFC 9110 §9.3.6 and §15.2; a corpus fed whole, cut at every byte and a byte at a time, every refusal and limit at its edge, and the server parser reading what the writer writes (`tests/link/net_http1_client`, `_f64`, `_feed_window`) |
|
|
239
|
+
| [`net/fetch.ts`](https://github.com/amritk/nish/blob/main/std/net/fetch.ts) | A blocking HTTP/1.1 client for command-line programs: `fetchSync(request, options)` looks the host up, connects, sends an `HttpRequest` (`method`, `url`, `header(name, value)`, `body`) and answers an `HttpResponse` (`status`, `reason`, `url`, `header(name)`) once its head is in, its body read through `body.read` or `body.readAll(limit)`; null on a failure. `FetchSyncClient(options)` does the same and keeps one connection alive between fetches, sending an idempotent request once more when a kept connection turns out to be gone, and says why a fetch failed in `error` (an `HTTP_E_*` code, `httpErrorName`) and `errorText`. `HttpOptions` bounds every wait (`connectTimeoutMs`, `readTimeoutMs`, `totalTimeoutMs`) and every body (`maxBodyBytes`), follows redirects only when asked (`followRedirects`, `maxRedirects`, `crossOriginRedirects`, never https to http, credentials dropped across origins), and reads `http_proxy`, `https_proxy` and `no_proxy` when `useEnvProxy` is set (`HTTP_PROXY` ignored under CGI; https through `CONNECT`). Every byte goes through `nish/net/http1-client-codec`. Native only. Record: [`docs/security/fetch.md`](../docs/security/fetch.md) | RFC 9110 §9.2.2 and §15.4, RFC 9112; a loopback server and proxy in the test itself: keep-alive, retries, redirects, proxies, timeouts and body caps (`tests/link/net_http_fetch`) |
|
|
240
|
+
| [`net/outbound.ts`](https://github.com/amritk/nish/blob/main/std/net/outbound.ts) | The policy for fetching URLs your program did not choose, set as `HttpOptions.outbound`: `OutboundPolicy` reaches only public addresses (no loopback, private, link-local, CGNAT, metadata or other special-purpose range, IPv4-mapped, NAT64 and 6to4 forms judged as the IPv4 address they carry), checked on the resolved address the client then connects to, so DNS rebinding cannot move it; every redirect hop is checked again, the proxy is not used, and redirects, total time and body size are capped. `permit(cidr)` lets a trusted prefix through, `allowHost(name)` limits the names a URL or hop may carry (`HTTP_E_BLOCKED` before the lookup). Native only. Record: [`docs/security/outbound.md`](../docs/security/outbound.md) | RFC 6890, RFC 4291, RFC 6052, RFC 3056 and the IANA special-purpose address registries; every range at its edges and a loopback server fetched with and without a policy (`tests/link/net_outbound`) |
|
|
241
|
+
| [`net/tls/ticket.ts`](https://github.com/amritk/nish/blob/main/std/net/tls/ticket.ts) | Session tickets a server seals for itself (RFC 8446 §4.6.1): `tlsTicketSeal` and `tlsTicketOpen` seal a `TlsTicketState` (suite, issue time, lifetime, chain start, server name, resumption PSK) under ChaCha20-Poly1305 with a borrowed `Secret<u8[]>` key, as key id ‖ nonce ‖ ciphertext ‖ tag, and wipe the plaintext on every path. `TlsTicketKeyInfo`, `tlsTicketKeyDraw` and `tlsTicketKeyDue` are a key's public record, its draw and when to replace it: a key seals for 12 h of awake time and 2^30 tickets, and opens for 14 h. `TlsTicketStrikes` is the bounded strike register (§8.1) that makes a ticket single-use, failing closed when full | `tests/link/net_tls_ticket`: every bound at its instant and 1 ns past it, a flipped bit in each region, three key rotations, replay and a full register |
|
|
242
|
+
| [`net/websocket.ts`](https://github.com/amritk/nish/blob/main/std/net/websocket.ts) | `WsDecoder(expectMasked, maxMessage)`, the same `feed` and `next()` over WebSocket frames: `WS_MESSAGE` (fragments joined, text checked as UTF-8 fragment by fragment), `WS_PING`, `WS_PONG`, and the final `WS_CLOSE` and `WS_ERROR`, whose `closeCode` is the 1002, 1007 or 1009 to close with. `websocketFrame` encodes a frame with the mask key the caller supplies (`websocketWriteFrame` into a caller's buffer, `websocketFrameSize` its length), `reset()` readies a decoder for the next connection, `websocketClosePayload` a close frame's code and reason, and `websocketIsUtf8` checks a window. The handshake is `websocketRequestKey(parser)` over an `Http1Parser`'s head, `websocketAcceptKey`, `websocketKeyIsValid`, `websocketOfferedProtocols` and `websocketUpgradeResponse`, the 101. The client's side is `websocketClientKey(nonce)` (a 16-byte nonce the caller draws; the module draws no randomness), `websocketUpgradeRequest(host, target, key, protocols)` and `websocketCheckResponse(parser, key, offered)` over an `Http1ResponseParser` told `expectUpgrade()`, answering a `WsHandshake` (`error`, a `WS_HS_*` code, and `protocol`) | RFC 6455 §1.3's accept key, the §5.7 example frames byte for byte, §5.5's control-frame rules, §7.4's close codes |
|
|
243
|
+
| [`net/http1-server.ts`](https://github.com/amritk/nish/blob/main/std/net/http1-server.ts) | The server side of HTTP/1.1 (WP34 H1): `Http1Connection`, sans-IO, drives an `Http1Parser` — `feed`, then `next()` answers `H1_REQUEST` (the parser's head), `H1_BODY` (at most `chunkSize` bytes, either framing), `H1_END`, `H1_WRITE` (a held-back write may go again), `H1_WS_MESSAGE`, `H1_WS_CLOSE` or the final `H1_ERROR`; the program answers with `respond(status, names, values, bodyLength)`, `write` and `end`, or `acceptWebSocket(protocol)` and then `sendFrame` and `closeWebSocket`. Keep-alive and pipelining one exchange at a time, a response with a length or chunked (unframed to an HTTP/1.0 client), `Connection` and the framing decided by the server, every parser refusal answered with its status and a close, pings and closes answered on a WebSocket. Two slot pools for a program that owns its loop: `Http1Server` on plain TCP and `Http1TlsServer` on `nish/net/tls-tcp` with ALPN `http/1.1` or none (`accept`, `readable` / `writable`, `next(slot)`, `flush(slot)`, `close`, `expire(now)` for the idle timeout and the deadline for a whole head, `headTimeout`, against a client that trickles its head). Both build a slot's connection on the first accept into it rather than at start-up. `Http1Server` reads a socket until a short read rather than until it would block, takes a free slot off a stack of released slots or the next never-used one rather than scanning for one, and answers `rearm(slot, wants)`, whether the interest changed and needs a `pollModify` at all: a keep-alive request is one `recv` and one `send`. The slots borrow their output from one `Http1OutputPool` while they write and give it back once sent, so a pool's memory follows the connections the socket is holding back, not its size: an idle slot holds no output buffer. Every final response carries a `Date` from `HttpDate`, which reads `Date.now()` and formats at most once a second, shared by a pool's slots; `pin(unixMs)` fixes it for a test, and a program's own `date` field replaces it. Every cap is an `Http1Config` number fixed at start-up, and a warmed slot allocates nothing per request (`docs/security/http1-codec.md`). Native only, as `nish:net` is | RFC 9112 §6, §7, §9; RFC 9110 §9.3.2, §15; RFC 6455 §4.2, §5.5. Nish clients over loopback against both carriers, requests cut at every byte, 2 MiB bodies each way, pipelining, a WebSocket echo, the arena flat over two hundred requests, the date formatted once a second, and every refusal (`tests/link/net_http1_server`); `curl` against its `serve` mode |
|
|
244
|
+
| [`net/url.ts`](https://github.com/amritk/nish/blob/main/std/net/url.ts) | The request target's path and query, read in place: `urlQueryStart` finds the `?`, `UrlPathSegments` splits a path on the raw `/` and `UrlQuery` yields a query's pairs as offsets, both reusable by `reset`; `percentDecode` and `percentDecodePathSegment` decode a window into the caller's array and answer the length or a `URL_DECODE_ERR_*` code, `percentDecodeErrorAt` answers the offset an error is at, and `urlQueryFind` finds a key. Decoding is strict by default (a malformed escape and `%00` are refused); `URL_DECODE_FORM`, `URL_DECODE_LENIENT` and `URL_DECODE_ALLOW_NUL` together decode valid UTF-8 as `URLSearchParams` does. A query yields at most `UrlQueryLimits.maxPairs` pairs (100 by default) and then answers `URL_QUERY_ERR_PAIRS`. The byte API allocates nothing; `urlDecode` and `urlQueryGet` answer strings and refuse what is not UTF-8. Whole URLs: `urlParse` splits an absolute URL into a `Url` (`scheme`, `hasAuthority`, `username`, `password`, `host`, `port`, `path`, `query`, `fragment`, each still percent-encoded, the scheme and host lower-cased), `urlResolve(base, ref)` resolves a reference against one, `urlSerialize` writes one back, and `urlDefaultPort` and `urlOrigin` answer the port and the origin a client connects to. Both refuse with null what two readers could take for different hosts: a byte outside printable ASCII, a backslash, a malformed `%`, a second `@`, a non-unreserved host byte, a numeric host that is not strict dotted-decimal IPv4, an IPv6 zone, a port past 65535. Records: [`docs/security/url.md`](../docs/security/url.md) | RFC 3986 §2.1 and §3.3, the WHATWG URL Standard's percent-decode and `application/x-www-form-urlencoded` parser; `decodeURIComponent` and `URLSearchParams` on 3,000 seeded inputs; RFC 3986 §3 and §5.2, every §5.4 example, 57 refusals and 4,000 seeded round trips in both number modes (`tests/link/net_url_parse`) |
|
|
245
|
+
| [`net/http-fields.ts`](https://github.com/amritk/nish/blob/main/std/net/http-fields.ts) | The header model HTTP/2 and HTTP/3 share, version-neutral: `HttpFields` reads a decoded section with `readRequest(names, values, extendedConnect)`, `readResponse` or `readTrailers` into `method`, `scheme`, `authority`, `path`, `protocol`, `status`, the regular fields in order (`get(name)`) and `contentLength`, answering `HTTP_FIELDS_OK` or the first refusal: a name that is empty, not a token or not lowercase; a value with NUL, CR or LF or whitespace at an end; a connection-specific field (`te` but `trailers` included); a pseudo-header unknown, repeated, after a regular field, forbidden by the shape or missing; a pseudo-header value of the wrong syntax; a `host` that disagrees with `:authority`; a bad or disagreeing `content-length`. `:protocol` (RFC 8441) is read only where the caller enabled extended CONNECT. `httpFieldNameValid`, `httpFieldValueValid`, `httpFieldConnectionSpecific` and `httpFieldsCheckOutgoing` check one name, one value or a section a program sends. `httpDateWrite(out, at, seconds)` writes a second since 1970 as the IMF-fixdate a `Date` field holds (`HTTP_DATE_SIZE` bytes, to 9999), reading no clock, so it compiles for wasm32 | RFC 9110 §5, RFC 9113 §8.2–§8.5, RFC 9114 §4.2–§4.3, RFC 8441 §4; every refusal in `tests/link/net_http_fields`; RFC 9110 §5.6.7, every day from 1970 to 2399 (`tests/link/net_http_date`) |
|
|
246
|
+
| [`net/http2-frame.ts`](https://github.com/amritk/nish/blob/main/std/net/http2-frame.ts) | HTTP/2's frames, sans-IO and allocation-free: `http2ReadHeader` and `http2ParseFrame` read any frame of RFC 9113 §6 into a reused `Http2Frame` (its content left in the buffer as a window), answering 0 or the connection error — FRAME_SIZE_ERROR past the reader's SETTINGS_MAX_FRAME_SIZE or for a frame of the wrong length, PROTOCOL_ERROR for a frame on the wrong stream, padding as long as its frame, or a WINDOW_UPDATE of zero on the connection — with a stream error (`streamError`) for a PRIORITY of the wrong length, a self-dependency or a WINDOW_UPDATE of zero on a stream. `http2SettingCount`, `http2SettingId`, `http2SettingValue` and `http2SettingError` read and check SETTINGS. The writers, one per type (`http2WriteData` … `http2WriteContinuation`, `http2WriteSettingsAck`), write into the caller's buffer at an offset and answer -1 when a frame does not fit | Every type against frames built by hand from §4.1 and §6, both directions, and every refusal (`tests/link/net_http2_frame`) |
|
|
247
|
+
| [`net/http2.ts`](https://github.com/amritk/nish/blob/main/std/net/http2.ts) | `Http2Connection`, the server side of HTTP/2, sans-IO: `feed(buf, off, len)`, then `next()` answers `H2_REQUEST` (`stream`, `fields`), `H2_DATA` (a window onto the input), `H2_TRAILERS`, `H2_RESET`, `H2_WINDOW` (a send window opened), `H2_GOAWAY` or the final `H2_ERROR`; send `output[outputStart .. outputEnd)` and `consume`. `respond`, `writeData` (as far as both windows and the output allow), `writeTrailers`, `reset`, `goaway` and `restart`. The §5.1 stream state machine, connection and stream flow control with WINDOW_UPDATE both ways, SETTINGS and the acknowledgement, PING, header blocks across CONTINUATION decoded by `HpackDecoder`, a 431 past the header-list cap, extended CONNECT behind SETTINGS_ENABLE_CONNECT_PROTOCOL, and every connection and stream error of RFC 9113 as GOAWAY or RST_STREAM. Every final head carries one `date` field from the connection's `date` bytes, when it has them, straight after `:status` and never on a 1xx or trailers, unless the program's fields hold one (matched ASCII case-insensitively). Every cap is an `Http2Config` number fixed at start-up; DATA and every control frame allocate nothing ([H2-1](../docs/security/http2.md)) | RFC 9113 §3–§8, RFC 8441; requests, streamed responses, flow control stalling and resuming, concurrent streams and the date on every final head from a scripted client (`tests/link/net_http2`), and a negative for each error class (`tests/link/net_http2_errors`) |
|
|
248
|
+
| [`net/http2-tls.ts`](https://github.com/amritk/nish/blob/main/std/net/http2-tls.ts) | `Http2TlsServer`: HTTP/2 over `nish/net/tls-tcp` for a program that owns its loop — a pool of `Http2Connection`s beside the TLS slots, slot 0's built by the constructor and every other on the first accept into its slot, each reused with `restart`; `accept`, `readable` / `writable`, `signP256`, `next(slot)` for the slot's next event, `flush(slot)`, `close`. A slot whose ALPN did not choose `h2` is shut down with `close_notify` (RFC 9113 §3.2). Every slot's final responses carry the pool's one `HttpDate` (`date`, from `nish/net/http1-server`), refreshed at `connection(slot)` and `next(slot)` and formatted at most once a second; `date.pin(unixMs)` fixes it for a test. Native only, as `nish:net` is | A Nish TLS client over loopback (`tests/link/net_http2_tls`): ALPN, a GET, an echoed POST, a hundred DATA frames with the arena flat, the date with the arena flat while the pinned clock crosses seconds and shared by two slots, GOAWAY, and the pool; `curl --http2` against its `serve` mode |
|
|
249
|
+
| [`net/http3-frame.ts`](https://github.com/amritk/nish/blob/main/std/net/http3-frame.ts) | HTTP/3's frames and stream types, sans-IO and allocation-free: `h3ReadFrameHeader` reads a type and a length (0 until both are in the window), `h3ReadSettings` a SETTINGS payload into a reused `Http3Settings` (the three identifiers it understands, and the first eight it does not, kept for an extension), `h3ReadIdPayload` a GOAWAY, MAX_PUSH_ID or CANCEL_PUSH; `h3PutFrameHeader`, `h3PutSettings`, `h3PutIdFrame` and `h3PutVarint` write in place. `h3ReservedHttp2Frame` names HTTP/2's reserved types, `h3Greased` the `0x1f * N + 0x21` ones, and the `H3_*` constants are RFC 9114's types, settings and error codes | RFC 9114 §6.2, §7, §8.1, against frames built by hand from §7's layouts (`tests/link/net_http3_frame`) |
|
|
250
|
+
| [`net/http3.ts`](https://github.com/amritk/nish/blob/main/std/net/http3.ts) | `Http3Connection`, the server side of HTTP/3 over a `QuicConnection`'s streams: `next()` answers `H3_REQUEST` (`stream`, `fields`), `H3_DATA` (a window onto `data`, at most `bodyChunk`), `H3_TRAILERS`, `H3_END`, `H3_RESET`, `H3_WRITABLE`, `H3_GOAWAY` or the final `H3_ERROR`; `respond`, `writeData` (takes what the send buffer has room for), `writeTrailers`, `reset` and `goaway` write. It opens its control stream with SETTINGS and its QPACK streams (capacity 0, through `nish/net/qpack`), checks every field section with `nish/net/http-fields`, answers a section past `maxFieldSectionSize` with a 431, sends one `date` field from the connection's `date` bytes on every final head (straight after `:status`, counted against the client's field-section limit, never on trailers) unless the program's fields hold one, and closes the connection with RFC 9114's code for each connection error. No server push. A warm connection allocates nothing per request (`docs/security/http3.md`) | RFC 9114 §4–§8 and RFC 9204 §4.2: a Nish client over QUIC, requests with bodies both ways, trailers, the date and its limit, GOAWAY, cancellation, every error code (`tests/link/net_http3`, `net_http3_errors`) |
|
|
251
|
+
| [`net/http3-server.ts`](https://github.com/amritk/nish/blob/main/std/net/http3-server.ts) | `Http3Server`: HTTP/3 over QUIC on one UDP socket for a program that owns its loop — a pool of `QuicConnection` slots with an `Http3Connection` each, a slot's built on the first Initial accepted into it, slots 0 up, and reused, while the connection-ID index, the timer wheel and the rings are sized from the cap at start-up; a `QuicListener` in front; datagrams routed by a hashed index of every slot's connection IDs; `receive(now, key)`, `tick(now)`, `ready()` for the next slot with news, `flush(now)` sending paced GSO flights, `timeout(now)` from a timer wheel. A handshake that did not choose ALPN `h3` is closed with no_application_protocol (RFC 9114 §3.1); a slot done after GOAWAY is closed and freed. Every slot's final responses carry the server's one `HttpDate` (`date`), refreshed at `connection(slot)` and `ready()` and formatted at most once a second. `stop()` releases every connection and the listener and wipes the configuration's stateless reset and Retry token keys, which are the server's once handed in, and its last answer; the keys are zeroed in place, so a `QuicServerConfig` serves one server (QUIC-5). Native only, as `nish:net` is | A Nish client across loopback (`tests/link/net_http3_server`): requests, a paced 300 KB response, GOAWAY and the slot freed, ALPN, a full pool, the arena over connections through one slot while the pinned date moves a second on for each, one date formatted for two slots; every key read back as zero after `stop()` (`tests/link/net_quic_server_wipes`); aioquic and `curl --http3` against its `serve` mode |
|
|
252
|
+
| [`net/webtransport.ts`](https://github.com/amritk/nish/blob/main/std/net/webtransport.ts) | `WebTransport`, the server side of WebTransport over HTTP/3 as draft-ietf-webtrans-http3-02 has it and Chrome and `wtransport` 0.7 negotiate it, a layer over one `Http3Connection` whose `Http3Config.webtransportSessions` turns it on (extended CONNECT, SETTINGS_H3_DATAGRAM and the draft's two WebTransport settings): `next()` answers `WT_SESSION` (the program `accept`s or `refuse`s it), `WT_DATAGRAM`, `WT_STREAM`, `WT_STREAM_DATA`, `WT_STREAM_END`, `WT_STREAM_RESET`, `WT_STREAM_STOPPED`, `WT_WRITABLE`, `WT_DRAIN` and `WT_CLOSED`, and passes a plain request's `H3_*` events through; `sendDatagram` (the quarter stream ID, then the payload), `openStream`, `write`, `resetStream`, `stopSending`, `close` (CLOSE_WEBTRANSPORT_SESSION) and `drain`. Streams that arrive before their session wait, bounded; every cap is fixed at start-up, and a warm session allocates nothing per datagram or stream (`docs/security/webtransport.md`) | RFC 9220, RFC 9297 and draft-02: a Nish client over QUIC — sessions, datagrams at the size limits, streams both ways, close and drain from either side, every refusal — and one exchange recorded from a `wtransport` 0.7 client replayed byte for byte (`tests/link/net_webtransport`) |
|
|
253
|
+
| [`net/http-compress.ts`](https://github.com/amritk/nish/blob/main/std/net/http-compress.ts) | Content coding for HTTP responses: `httpAcceptEncoding(buf, start, end, offered)` reads an `Accept-Encoding` value (q-values, `*`, `identity`, `x-gzip`) into `HTTP_CODING_GZIP`, `HTTP_CODING_DEFLATE`, `HTTP_CODING_IDENTITY` or `HTTP_CODING_NONE`, allocating nothing; `HttpCompressor(level, minLength, maxLength)` gzips a body into its own buffer when it is worth it (-1 to send it as it is) and builds the response fields (`addFields`: `Content-Encoding: gzip` and, while `vary`, `Vary: Accept-Encoding`). It sends gzip only when it pays: `minLength`, `minSaving` bytes or `minSavingPercent`, and `segmentMode` (fewer TCP segments of `mss`, head included, than as it is); `decision` says why (`HTTP_COMPRESS_SENT`, `_NOT_ACCEPTED`, `_TOO_SMALL`, `_TOO_LARGE`, `_TOO_LITTLE_SAVED`, `_NO_FEWER_SEGMENTS`). Bodies up to `fixedLimit` use the fixed-Huffman encoder (`encoder`: `HTTP_ENCODER_AUTO`, `_FIXED`, `_LEVEL`). On HTTP/1.1, `Http1Connection.acceptEncoding()` and `respondEncoded(status, names, values, body, off, len, enc, coding)` answer a request in one call with the compressed `Content-Length`, or nothing at all (`H1_AGAIN`, `H1_INVALID`); `identity;q=0` is honoured. Opt-in; BREACH is the program's to avoid (`docs/security/compress.md`) | RFC 9110 §8.4 and §12.5.3: the negotiation table, and sans-IO responses gunzipped back (`tests/link/std_http_compress`) |
|
|
254
|
+
| [`net/router.ts`](https://github.com/amritk/nish/blob/main/std/net/router.ts) | `Router`, a method and path router that knows no carrier and allocates nothing per match: `add(method, pattern)` answers the route's index (0, 1, 2 in the order added) or a refusal — `ROUTER_BAD_PATTERN`, `ROUTER_BAD_METHOD`, `ROUTER_TOO_LONG`, `ROUTER_AMBIGUOUS` (`*` not last), `ROUTER_DUPLICATE` (the same shape and method, whatever the parameter names) or `ROUTER_FULL`, against `RouterLimits`; `match(method, mStart, mEnd, target, tStart, tEnd, params)` answers the index, `ROUTER_NOT_FOUND` or `ROUTER_METHOD_NOT_ALLOWED`, writing each `:name` and a final `*` as a `[start, end)` pair into the caller's `params` (`ROUTER_TOO_MANY_PARAMS` when it is short). Literal before parameter before `*`, segment by segment, backtracking; a lookup enters each node at most once. The query is left unparsed and nothing is decoded: `routerDecode` decodes one parameter (`ROUTER_BAD_ESCAPE`, `ROUTER_NO_ROOM`) (`docs/security/router.md`) | Every refusal code, precedence and backtracking, 405 against 404, `%2F` kept in one segment, the query, `*` and authority-form targets, a target deeper than any route, the lookup bound, params past storage, and the arena flat over matches (`tests/link/net_router`, `net_router_f64`, five `net_router_negative_*`) |
|
|
255
|
+
| [`net/handler.ts`](https://github.com/amritk/nish/blob/main/std/net/handler.ts) | `HttpHandler`, complete requests and their answers over the HTTP/1.1 carrier: it owns the readiness loop, routes each request through a `Router` at its head, gathers its body up to `HandlerLimits.http.maxBody` (1 MiB; past it the carrier answers 413), and hands it over whole. The program pulls — `wait(ms)`, `ready()`, `take()` — and switches on `req.route`; `HttpRequest` reads parameters, form-decoded query values and fields as ranges into its own buffers. Each request is answered once, now or in a later turn, by `send`, `sendFields`, `empty`, `json()` + `sendJson`, `problem` (RFC 9457), `refuse` (404, or 405 with `Allow`) or `start`/`write`/`end`; an answer whose connection closed first is `HANDLER_DROPPED` and reaches no later peer. `HEAD` takes a `GET` route. `watch`/`fired` add the program's own descriptors (`signalFd()`), and `drain` stops gracefully. Nothing is allocated per request once warm (`docs/security/handler.md`) | Routing, parameters and form-decoded queries, `HEAD`, 404 and 405 problems, chunked bodies in and out, the body cap at and one past it, pipelining order, late answers dropped across a reused slot, the arena and pool flat over two thousand requests, and the drain (`tests/link/net_handler`, and `net_http_app` over `examples/http-app`) |
|
|
256
|
+
| [`net/upstream.ts`](https://github.com/amritk/nish/blob/main/std/net/upstream.ts) | `Upstream`, outbound HTTP/1.1 calls from a route handler that never block it: a pool of keep-alive connections to one `http:` origin on the `HttpHandler`'s own loop (watch ids `UpstreamLimits.firstId` and up). `get(req, path)` returns at once, or `UPSTREAM_E_FULL` with nothing held; each call comes back once through `ready()`/`take()` tagged with `r.req`, `ok` with its status and `body[0 .. length)`, or failed with the codec's `HTTP1_E_` refusal or an `UPSTREAM_E_` code (timeout, closed, connect, origin, lookup, blocked, request), and goes back with `release`. The origin is looked up once at start-up and pinned; `UpstreamLimits.outbound` checks it against an `OutboundPolicy`. A failed call's connection is closed. Each connection keeps one codec in spans mode, so a warm pool allocates nothing per call | A slow upstream while other clients and a second upstream are answered, the timeout, keep-alive and tagging, five codec refusals as failures, the policy refusing loopback, origin and request-target refusals, three thousand warm calls on one connection with `Arena.used()` flat (`tests/link/net_upstream`), and a second `release` panicking (`net_upstream_release`) |
|
|
257
|
+
| [`net/postgres.ts`](https://github.com/amritk/nish/blob/main/std/net/postgres.ts) | `PgConnection`, a PostgreSQL client that reads no socket: the program fills `inputBuffer()` and sends `outputBuffer()`, and `next()` answers `PG_NEED_MORE`, `PG_READY`, `PG_DESCRIBED`, `PG_ROW`, `PG_COMPLETE`, `PG_ERROR`, `PG_NOTICE`, `PG_GROUP_DONE` or the final `PG_FAILED` (`failure`, one of the `PG_ERR_*` codes). Protocol 3.0: startup, `trust` and SCRAM-SHA-256 with the server's signature checked (cleartext and MD5 refused), the extended query protocol — `prepare`, `bindStart`, `paramInt` / `paramText` / `paramBytes` / `paramTextArray` / `paramIntArray` / `paramNull`, an array built element by element into the Bind (`arrayStart`, `arrayElement`, `arrayInt`, `arrayNull`, `arrayEnd`), binary `int4[]` / `int8[]` parameters (`paramInt4Array` / `paramInt8Array` over an `i64[]` window, or `binaryArrayStart`, `binaryArrayInt`, `binaryArrayNull` and the same `arrayEnd`; per-parameter format codes only once a binary one is bound; an `int4` element out of range panics), `bindEnd`, `bindStartBinary` for a Bind whose result columns come in binary (read in place by `binaryInt`, `binaryBool`, `binaryFloat` and `PgBinary`'s `timestamp` and `iso`, and a one-dimensional `int2[]`, `int4[]` or `int8[]` by `PgBinaryArray`), `bindCancel` to drop a Bind not yet ended, `closeStatement`, `sync` — with up to `maxInFlight` groups pipelined. `PgBatcher` queues keyed reads of one statement shape (`addInt`, `addText`) and `take(conn)` binds up to `maxRequests` of them, each key once, as one execution over `unnest($1) with ordinality`; `row`, `end` and `fail` route its rows by their server-supplied ordinal, checked first (read with `binaryInt` when `PgBatchConfig.binary` is set, so the batch's rows come in binary too), to `next(run)`'s per-request `PG_BATCH_ROW` and `PG_BATCH_DONE` / `PG_BATCH_MISSING` / `PG_BATCH_FAILED`; with `PgBatchConfig.sorted` false the statement has no `order by q.ord`, so no Sort node, rows route as they come and each request's last event waits for `end`; `take` answers a fresh handle each time, so a server's late or repeated event on a spent one is counted in `stray`, never routed or panicked on. It reads no clock and allocates nothing per request or row. `PgBatchClock` says when to take, from times the program passes in: once max(`minCount`, the moving average of batch sizes × `factor`) wait, the oldest has waited `maxWaitUs`, or, with the idle bypass, nothing is in flight and the load is light (its defaults are the fixed window); `PgBatcher.takeDue` takes only then, and `PgBatchTurns` lets one free connection collect while the others execute. A row's columns are windows onto the receive buffer (`colStart`, `colLength`), read in place by `int`, `bool`, `isNull` and `PgText`'s decoders (`timestamptz`, `PgArrayWalker` for `text[]`); no per-row or per-column allocation. The password is a `Secret<u8[]>`, wiped once the proof is made. Caps on message size, columns, groups in flight and SCRAM iterations (`docs/security/postgres.md`) | PostgreSQL 18's frontend/backend protocol chapter, RFC 5802 and RFC 7677: every builder, event and refusal (`tests/link/std_postgres`), and exchanges recorded from PostgreSQL 18 by `tests/postgres/record.mjs` replayed byte for byte (`tests/link/std_postgres_replay`); the batcher's queueing, caps and ordinal refusals (`tests/link/std_postgres_batch`), and unsorted (`tests/link/std_postgres_batch_unsorted`); the clock's reasons, backlog and turns (`tests/link/std_postgres_batch_clock`) |
|
|
258
|
+
| [`net/postgres-carrier.ts`](https://github.com/amritk/nish/blob/main/std/net/postgres-carrier.ts) | `PgCarrier`, a `PgPool`'s queries on an `HttpHandler`'s loop: its sockets sit in the handler's epoll set, a route handler hands a request to `run` (or `runInt` / `runText` for one of up to 8 attached `PgBatcher`s) and gets its number at once, and `next()` answers `PG_CARRIER_BUILD` to bind a plain query on `connection()`, `PG_CARRIER_ROW`, `PG_CARRIER_COMPLETE`, `PG_CARRIER_ERROR` and one final `PG_CARRIER_DONE`, `PG_CARRIER_MISSING` or `PG_CARRIER_FAILED` (`failure`, `socketError`) with `request()` the request to answer and `tag` the caller's. It connects the pool's slots itself (TCP or a Unix socket), times out a connect after `connectTimeoutMs`, sheds batched keys when no slot can connect, and sizes every table from `PgCarrierLimits` at construction: `run` answers `PG_CARRIER_FULL` rather than grow one, and once connections are up it allocates nothing per query. Kept apart from `net/postgres.ts` so that module reaches no capability (`docs/security/postgres.md`) | A loopback backend in the test itself: routing, a slow query not stalling another client, lost and refused connections, a connect timeout, and `Arena.used()` flat over thousands of queries (`tests/link/std_postgres_carrier`); a live server by `tests/postgres/live/run.mjs` |
|
|
259
|
+
| [`net/prefork.ts`](https://github.com/amritk/nish/blob/main/std/net/prefork.ts) | A server on every core: `preforkWorkers(process.argv)` or `preforkWith(process.argv, config)` answers a worker's index in each worker it starts again from the same program, and in the supervisor runs them until SIGTERM or SIGINT, restarting one that exits after a backoff. `PreforkConfig.supervisorChannels` gives each worker a `unixSocketPair` to the supervisor, which `preforkSupervise(args, config, state, onChannel)` hands to a callback; `serviceWorker` gives every other worker a pair to that one, and makes the workers one group that restarts together on fresh pairs. The ends arrive from descriptor 3 through `spawnWithFds`, found with `preforkSupervisorChannel`, `preforkServiceChannel` and `preforkServiceChannelOf`. Native only | `tests/link/net_prefork` (the spread, a dead worker, SIGTERM, the backoff); `tests/link/net_prefork_channels` (each worker holds exactly its own ends, a restart, the supervisor's descriptors flat) |
|
|
260
260
|
|
|
261
261
|
**How a carrier drives `TlsServer`.** The server speaks in handshake
|
|
262
262
|
*messages*, tagged by level: `TLS_LEVEL_INITIAL` (cleartext — ClientHello,
|
|
@@ -370,7 +370,7 @@ import { Suite } from "nish/testing";
|
|
|
370
370
|
compiler that is running — not relative to the importing file, so the same
|
|
371
371
|
specifier works at any depth and from outside this repository.
|
|
372
372
|
|
|
373
|
-
**Source is the distribution format** ([`docs/wp21-packages.md`](
|
|
373
|
+
**Source is the distribution format** ([`docs/wp21-packages.md`](https://github.com/amritk/nish/blob/main/docs/wp21-packages.md)
|
|
374
374
|
§2), so an import of a `std/` module is not a link against a built library: the
|
|
375
375
|
module is compiled with the program that imports it, and the whole-program
|
|
376
376
|
attribute pass sees through it exactly as it sees through the program's own
|
|
@@ -436,7 +436,7 @@ that are *not* this package.
|
|
|
436
436
|
per iteration — and the module means one program in both modes. The default
|
|
437
437
|
mode pays nothing for it, because `toI32` on an `i32` is identity.
|
|
438
438
|
`tests/link/std_text_f64` is what keeps this from being prose;
|
|
439
|
-
[`docs/wp26-stdlib.md`](
|
|
439
|
+
[`docs/wp26-stdlib.md`](https://github.com/amritk/nish/blob/main/docs/wp26-stdlib.md) §4 is the reasoning.
|
|
440
440
|
- **Name the private helpers as though they were exported.** A `std/` module
|
|
441
441
|
belongs to package `nish`, so its symbols no longer collide with the
|
|
442
442
|
importing program's (`tests/link/std_package_scope`), but every `std/` module
|
|
@@ -446,22 +446,22 @@ that are *not* this package.
|
|
|
446
446
|
module would stop another `std/` module that declares its own `isBlank` from
|
|
447
447
|
compiling, which is why `text.ts` calls it `isTextBlankByte`. Keep the
|
|
448
448
|
helpers few and their names distinctive
|
|
449
|
-
([`docs/wp26-stdlib.md`](
|
|
449
|
+
([`docs/wp26-stdlib.md`](https://github.com/amritk/nish/blob/main/docs/wp26-stdlib.md) §3e). Module constants are
|
|
450
450
|
exempt — `NEWLINE` and `SPACE` fold at their uses.
|
|
451
451
|
- **It is an Nish program**, so the constraints are the language's: a function is
|
|
452
452
|
an arrow bound to a module-level `const`, a function is never a value, there
|
|
453
|
-
is no `try` / `catch`, and there are no optional parameters (
|
|
454
|
-
constant
|
|
453
|
+
is no `try` / `catch`, and there are no optional parameters (`x?: T`; a
|
|
454
|
+
default is a constant, an earlier parameter or `null` only). Those
|
|
455
455
|
three are what shape an API here more than any style preference — see the
|
|
456
456
|
header of `testing.ts` for what they did to that one, which was written before
|
|
457
457
|
WP18 added generic functions and classes
|
|
458
|
-
([`docs/LANGUAGE.md`](
|
|
459
|
-
one assertion per type ([`docs/wp26-stdlib.md`](
|
|
458
|
+
([`docs/LANGUAGE.md`](https://github.com/amritk/nish/blob/main/docs/LANGUAGE.md#generic-functions)) and still has
|
|
459
|
+
one assertion per type ([`docs/wp26-stdlib.md`](https://github.com/amritk/nish/blob/main/docs/wp26-stdlib.md) §3c).
|
|
460
460
|
`pair.ts` is the first module here to export a generic type.
|
|
461
461
|
- **Ship it with a `tests/link/` case.** `tests/link/<name>/` is the only place a
|
|
462
462
|
multi-module program is exercised end to end, and it is also what puts the
|
|
463
463
|
module into the corpus the stage1 oracles read
|
|
464
|
-
([`.claude/selfhost.md`](
|
|
464
|
+
([`.claude/selfhost.md`](https://github.com/amritk/nish/blob/main/.claude/selfhost.md)): a `std/` module with no
|
|
465
465
|
importer in `tests/link/` is compiled on no run.
|
|
466
466
|
`testing.ts` has two, one per outcome — `tests/link/std_testing` (exit 0) and
|
|
467
467
|
`tests/link/std_testing_fail` (exit 1, and the wording of every failure
|
|
@@ -484,8 +484,8 @@ consumer is a guess.
|
|
|
484
484
|
|
|
485
485
|
| Program | What it does | What it uses |
|
|
486
486
|
| --- | --- | --- |
|
|
487
|
-
| [`tests/nish/run.ts`](
|
|
488
|
-
| [`tests/nish/cli.ts`](
|
|
487
|
+
| [`tests/nish/run.ts`](https://github.com/amritk/nish/blob/main/tests/nish/run.ts) | the golden cases and the `tests/link/` programs, compiled, assembled, linked, run and diffed | `Suite` (including `containsAll` for an `.err` file's fragments and `eqLines` for an IR golden), and all of `std/text` |
|
|
488
|
+
| [`tests/nish/cli.ts`](https://github.com/amritk/nish/blob/main/tests/nish/cli.ts) | the command line's own contract — the streams, the exit-code bands, and one flat `--json` object per diagnostic — read by a program in the language the compiler compiles | `Suite`, `jsonField`, `splitLines` / `trim` / `contains` |
|
|
489
489
|
|
|
490
490
|
Three assertions exist because the first of those two had written them by hand:
|
|
491
491
|
`contains` for a fragment of a captured stream, `containsAll` for an expectation
|
|
@@ -497,11 +497,11 @@ already written.
|
|
|
497
497
|
|
|
498
498
|
`std/json` is the one module admitted with a single importer, and the reason is
|
|
499
499
|
the *format* rather than the count: the object it reads is this project's own
|
|
500
|
-
published surface (`AGENTS.md`, [`docs/wp12-release.md`](
|
|
500
|
+
published surface (`AGENTS.md`, [`docs/wp12-release.md`](https://github.com/amritk/nish/blob/main/docs/wp12-release.md)),
|
|
501
501
|
so every Nish program that ever reads compiler output needs exactly this scan,
|
|
502
502
|
and the next one would copy it out of `tests/nish/cli.ts`. The module is also
|
|
503
503
|
where the format's one ambiguity is written down and tested —
|
|
504
|
-
[`docs/wp26-stdlib.md`](
|
|
504
|
+
[`docs/wp26-stdlib.md`](https://github.com/amritk/nish/blob/main/docs/wp26-stdlib.md) §7 question 3 is where that
|
|
505
505
|
argument is recorded and where its second consumer is expected.
|
|
506
506
|
|
|
507
507
|
## What `std/testing` is not, and what closed
|
|
@@ -518,7 +518,7 @@ have, and all three have since landed as builtins:
|
|
|
518
518
|
| a clock | `monotonicNanos(): i64` |
|
|
519
519
|
|
|
520
520
|
So the driver exists, in the language, and it is
|
|
521
|
-
[`tests/nish/run.ts`](
|
|
521
|
+
[`tests/nish/run.ts`](https://github.com/amritk/nish/blob/main/tests/nish/run.ts): it discovers the golden cases with
|
|
522
522
|
`readdirSync`, compiles each one by spawning the compiler, links and runs the ones
|
|
523
523
|
with a `.out`, diffs the IR against the golden line by line, and reports through a
|
|
524
524
|
`Suite`. It passes over the whole corpus — `npm run test:nish` — and `npm test`
|