@amritk/nish-x86_64-linux 0.21.0 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/INSTALL.md CHANGED
@@ -143,6 +143,36 @@ for — which catches a truncated download, a tarball built for another
143
143
  architecture, and an asset whose name does not match what is inside it — and
144
144
  only then swapped in. If any of that fails the install you had is untouched.
145
145
 
146
+ **Every download is checked before it is unpacked.** install.sh compares the
147
+ tarball's SHA-256 with the release's `SHA256SUMS` (or with the digest pinned
148
+ for releases that predate it) and refuses a mismatch. For releases after
149
+ 0.21.0 it also checks the release's **build provenance attestation** when a
150
+ signed-in GitHub CLI is present whose `gh attestation verify` takes the
151
+ options below. The attestation
152
+ is a Sigstore-signed statement, made by `actions/attest-build-provenance` in
153
+ the release workflow, that every file in `SHA256SUMS` was built by
154
+ `.github/workflows/release.yml` at the release's tag on a GitHub-hosted
155
+ runner. A failed verification refuses the install; with no `gh`, a signed-out
156
+ one or one too old for those options, the check is skipped with a note saying so, and the
157
+ SHA-256 still applies. The same command by hand, on a tarball you downloaded:
158
+
159
+ ```bash
160
+ gh attestation verify nish-<version>-<asset>.tar.gz \
161
+ --repo amritk/nish \
162
+ --signer-workflow amritk/nish/.github/workflows/release.yml \
163
+ --source-ref refs/tags/v<version> \
164
+ --deny-self-hosted-runners
165
+ ```
166
+
167
+ It works on any file the release attaches, the npm tarballs included. Each
168
+ release also attaches `nish-<version>.spdx.json`, an SPDX 2.3 software bill of
169
+ materials: every attached file with its SHA-256, the clang, linker, Node.js
170
+ and seed compiler that built each tarball (recorded by the build row itself),
171
+ and the vendored code from
172
+ [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md) that each of them contains.
173
+ How these fit the rest of the release's defences is
174
+ [docs/security/supply-chain.md](security/supply-chain.md).
175
+
146
176
  The npm route installs the **native** compiler, and it is a download rather than a
147
177
  build: nothing is compiled on your machine. The package declares one
148
178
  `nish-<os>-<arch>` package per supported platform as an `optionalDependencies`
package/bin/nish CHANGED
Binary file
package/docs/AI.md CHANGED
@@ -93,6 +93,24 @@ compiles and the warning would be printed: `NL9007`'s inserts the guard that
93
93
  proves an index in range (not under `--no-warn-performance`), and `NL7002`'s
94
94
  migrates a site off `--unchecked-indexing` (below).
95
95
 
96
+ When it compiles, **check the native build against the program's TypeScript
97
+ reading**. Run the project's tests natively; where it has none, write the
98
+ program back out and run both:
99
+
100
+ ```bash
101
+ nish program.ts --emit-ts out/ # the program as TypeScript, plus out/entry.ts
102
+ node out/entry.ts # Node 22.18 or newer; compare with the binary
103
+ ```
104
+
105
+ The two must print the same and exit the same, a panic included: `--emit-ts`
106
+ keeps the overflow and bounds checks the native build makes, but not a ranged
107
+ integer's range (`integer<0, 255>` is a plain `number`). Where they differ,
108
+ the difference is a row of [the user guide's table](guide/using-nish/to-typescript.md),
109
+ or a bug in `nish` worth reporting. A program on `nish:net`, the signal
110
+ descriptor or a C declaration (NL4010, NL4011), a compound assignment whose
111
+ index would run twice (NL4012) or a generic whose element type decides its
112
+ code (NL4013, NL4014) is refused and has no TypeScript to run.
113
+
96
114
  Two more surfaces worth knowing: `nish --emit-ast f.ts` prints what was parsed
97
115
  and `nish --emit-checked f.ts` prints the side tables the emitter reads. Both
98
116
  are one node per line, which is to say diffable.
@@ -2566,6 +2584,38 @@ most 100 pairs. (No block, as for `nish/text`; `tests/link/std_url` is its
2566
2584
  program, and the user guide's [standard-library
2567
2585
  page](guide/using-nish/standard-library.md#nishneturl) its walk-through.)
2568
2586
 
2587
+ **Pass a list to a PostgreSQL statement** — there is no driver that expands
2588
+ `IN (...)` and no `pg`'s array serialiser: in `nish/net/postgres`, write
2589
+ `= any($1::int8[])` or `unnest($1::text[])` and bind the list as one
2590
+ parameter. `paramIntArray(values, start, count)` binds an `i64[]` window;
2591
+ `arrayStart()`, `arrayElement(bytes, start, length)` or `arrayNull()` per
2592
+ element, then `arrayEnd()`, builds a `text[]` straight into the Bind from
2593
+ windows of a buffer. An element refused for room answers `false` and leaves
2594
+ the array open: end it and send the rest in the next statement. Every other
2595
+ `param...` call and `bindEnd()` panic until `arrayEnd()`. (No block, as for
2596
+ `nish/text`; `tests/postgres/live` runs it against a server, and the user
2597
+ guide's [PostgreSQL page](guide/using-nish/postgres.md#array-parameters) its
2598
+ walk-through.)
2599
+
2600
+ **Batch a server's keyed reads into one PostgreSQL query** — there is no
2601
+ DataLoader and no `setTimeout` to collect a tick's keys: in `nish/net/postgres`,
2602
+ a `PgBatcher` built from a `PgBatchConfig` (`name`, `select` reading the key as
2603
+ `q.key`, `keyType` `PG_BATCH_INT` or `PG_BATCH_TEXT`, caps) queues requests with
2604
+ `addInt(key, tag)` or `addText(bytes, start, length, tag)` — `false` when its
2605
+ queue is full, or for a text key too long for `maxBytes`, with no room left in
2606
+ `queueBytes`, with a NUL or not UTF-8 — and `take(conn)`
2607
+ binds the waiting ones, each key once, as one execution of its `sql` over
2608
+ `unnest($1) with ordinality`, answering a run handle or -1. Hand that
2609
+ execution's `PG_ROW` to `row(run, conn)`, `PG_COMPLETE` to `end(run)`, an error
2610
+ or `PG_LOST` to `fail(run)` (on a spent handle these do nothing and count in
2611
+ `stray`), and after each call `next(run)` until
2612
+ `PG_BATCH_NONE`: `PG_BATCH_ROW` (the connection's row answers request `tag`),
2613
+ then one of `PG_BATCH_DONE`, `PG_BATCH_MISSING` or `PG_BATCH_FAILED` per
2614
+ request. It reads no clock: when to take is the loop's choice. (No block;
2615
+ `tests/postgres/live`'s `batch` scenario runs it against a server, and the user
2616
+ guide's [PostgreSQL page](guide/using-nish/postgres.md#batching-keyed-reads)
2617
+ its walk-through.)
2618
+
2569
2619
  **Draw to the whole terminal** (a game, a dashboard) — there is no curses and
2570
2620
  no `process.stdout.on("resize")`: `runTerminal(options, state, update, render)`
2571
2621
  from `nish/term-io` opens the terminal, probes what it can draw, steps
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@amritk/nish-x86_64-linux",
3
- "version": "0.21.0",
3
+ "version": "0.23.0",
4
4
  "description": "The nish native compiler for Linux on x86_64",
5
5
  "license": "MIT",
6
6
  "os": [
@@ -3,12 +3,4 @@
3
3
  * in `nish/runtime` (docs/guide/using-nish/to-typescript.md).
4
4
  */
5
5
 
6
- /**
7
- * What the program's `exit(code)`, and every panic, throws: a page cannot end
8
- * itself. Tell it apart with `instanceof`, not by its `name`.
9
- */
10
- export class NishExit extends Error {
11
- /** The 8-bit status a native exit would leave: `code & 0xff`. */
12
- readonly code: number;
13
- constructor(code: number);
14
- }
6
+ export { NishExit } from "./core/host.js";
@@ -16,20 +16,9 @@
16
16
  * - Every file-system, process and machine call panics, naming itself and
17
17
  * the browser.
18
18
  */
19
- import { installHost, panic } from "./core/host.mjs";
19
+ import { installHost, NishExit, panic } from "./core/host.mjs";
20
20
 
21
- /**
22
- * What `exit(code)` throws: `code` is the 8-bit status a native exit would
23
- * leave. Exported so a caller tells it apart with `instanceof`, which a thrown
24
- * object that only carries the name does not pass.
25
- */
26
- export class NishExit extends Error {
27
- constructor(code) {
28
- super(`the program exited with ${code}`);
29
- this.name = "NishExit";
30
- this.code = code & 0xff;
31
- }
32
- }
21
+ export { NishExit };
33
22
 
34
23
  /**
35
24
  * One console stream, buffered to the next newline. The `"%s"` keeps a `%d`
@@ -1,18 +1,36 @@
1
1
  /** The array half of the core: bounds-checked access, and the arena stubs. */
2
- import type { i32 } from "./numbers.js";
2
+ import type { i32, i64 } from "./numbers.js";
3
3
  export declare function idx<T>(a: readonly T[], i: number | bigint): T;
4
4
  export declare function pop<T>(a: T[]): T;
5
5
  export declare function setIdx<T>(a: T[], i: number | bigint, v: T): T;
6
6
  export declare function updIdx<T>(a: T[], i: number | bigint, f: (old: T) => T): T;
7
- export declare function uncheckedGet<T extends number>(xs: readonly T[], i: i32): T;
8
- export declare function uncheckedSet<T extends number>(xs: T[], i: i32, v: T): void;
7
+ export declare function uncheckedGet<T extends number | bigint>(xs: readonly T[], i: i32): T;
8
+ export declare function uncheckedSet<T extends number | bigint>(xs: T[], i: i32, v: T): void;
9
9
  export declare function copyIn<T>(v: T): T;
10
10
  export declare function storeIdx<T>(a: T[], i: number | bigint, v: T): T;
11
11
  export declare function arraySet<T>(dst: T[], src: readonly T[], offset?: number): void;
12
- export declare function newArray<T>(n: number | bigint, zero: T): T[];
13
- export declare function arenaUsed(): number;
14
- export declare function arenaMark(): number;
12
+ /** `dst.set(src, offset)` as `--emit-ts` writes it: a 64-bit offset is a bigint, checked exactly. */
13
+ export declare function arraySetAt<T>(dst: T[], src: readonly T[], offset?: number | bigint): void;
14
+ /** `new Array<T>(n)`: `--emit-ts` writes `T` out, so the zero it fills with is not what `T` is read from. */
15
+ export declare function newArray<T>(n: number | bigint, zero: unknown): T[];
16
+ export declare function arenaUsed(): i64;
17
+ export declare function arenaMark(): i64;
15
18
  export declare function arenaRelease(m?: unknown): void;
16
19
  export declare function arenaReset(): void;
17
- /** `using a = arena()`: an object whose `Symbol.dispose` does nothing. */
18
- export declare function arena(): object;
20
+ /**
21
+ * The protocol `using` reads, declared here as `nish.d.ts` does, so `using a =
22
+ * arena()` type-checks with no `"esnext.disposable"` in a program's `lib`.
23
+ */
24
+ declare global {
25
+ interface SymbolConstructor {
26
+ readonly dispose: unique symbol;
27
+ }
28
+ }
29
+
30
+ /** What `arena()` answers: its `Symbol.dispose` does nothing. */
31
+ export interface ArenaHandle {
32
+ [Symbol.dispose](): void;
33
+ }
34
+
35
+ /** `using a = arena()`: the arena is the collector's, so leaving it frees nothing. */
36
+ export declare function arena(): ArenaHandle;
@@ -79,6 +79,22 @@ export function storeIdx(a, i, v) {
79
79
  return v;
80
80
  }
81
81
 
82
+ /**
83
+ * `dst.set(src, offset)` as `nish --emit-ts` writes it: an `i64` or `u64`
84
+ * offset, a bigint here, is compared as one, so a value past 2^53 fails with
85
+ * the native panic rather than being rounded into range, and one in range is
86
+ * the number `arraySet` takes. `runtime/nish.mjs` keeps `arraySet`, whose
87
+ * bigint offset throws as `docs/LANGUAGE.md` says.
88
+ */
89
+ export function arraySetAt(dst, src, offset) {
90
+ if (typeof offset === "bigint") {
91
+ const end = offset + BigInt(src.length);
92
+ if (offset < 0n || end > BigInt(dst.length)) panicSlice(offset, end, dst.length);
93
+ offset = Number(offset);
94
+ }
95
+ arraySet(dst, src, offset);
96
+ }
97
+
82
98
  /**
83
99
  * `dst.set(src, offset)` (WP34 N2): `TypedArray.prototype.set`'s copy on the
84
100
  * plain array a `u8[]` is here. The source is copied first, so a self-copy or
@@ -94,13 +110,25 @@ export function arraySet(dst, src, offset) {
94
110
  // to the nearest double past 2^53 (docs/RUN_UNDER_NODE.md).
95
111
  const at = offset === undefined ? 0 : Math.trunc(offset) || 0;
96
112
  const end = at + src.length;
97
- if (!(at >= 0 && end <= dst.length)) panicSlice(at, end, dst.length);
113
+ if (!(at >= 0 && end <= dst.length)) panicSlice(...saturatedRange(at, src.length), dst.length);
98
114
  // Two plain arrays overlap only when they are one array, which is the one
99
115
  // case that must read the source before writing it.
100
116
  const from = src === dst ? src.slice() : src;
101
117
  for (let i = 0; i < from.length; i++) dst[at + i] = from[i];
102
118
  }
103
119
 
120
+ /**
121
+ * `[at, at + n)` as the native panic spells it: the offset saturated into an
122
+ * `i64`, as `llvm.fptosi.sat` leaves it, and the end wrapped, so `1e300` is
123
+ * `9223372036854775807` and its end `-9223372036854775808`.
124
+ */
125
+ function saturatedRange(at, n) {
126
+ if (Number.isSafeInteger(at + n)) return [at, at + n];
127
+ const max = 2n ** 63n - 1n;
128
+ const start = at === Infinity || at >= 2 ** 63 ? max : at <= -(2 ** 63) ? -max - 1n : BigInt(at);
129
+ return [start, BigInt.asIntN(64, start + BigInt(n))];
130
+ }
131
+
104
132
  /** `new Array<T>(n)`: `n` zero-filled elements (`0`, `0n`, or `false`). */
105
133
  export function newArray(n, zero) {
106
134
  return new Array(toIndex(n)).fill(zero);
@@ -115,8 +143,8 @@ export function newArray(n, zero) {
115
143
  // so "the arena of the calling thread" and "the arena" are the same object, and
116
144
  // these stubs answer for both. If T1 ever lands a spawn the rewrite can reach,
117
145
  // that is when this file grows a second arena to keep count of.
118
- export function arenaUsed() { return 0; }
119
- export function arenaMark() { return 0; }
146
+ export function arenaUsed() { return 0n; }
147
+ export function arenaMark() { return 0n; }
120
148
  export function arenaRelease() {}
121
149
  export function arenaReset() {}
122
150
  /** `using a = arena()`: a disposable whose disposal does nothing, because there is no arena to release. */
@@ -20,6 +20,16 @@ export declare class NishNoHost extends Error {
20
20
  readonly requested: string;
21
21
  }
22
22
 
23
+ /**
24
+ * What the browser host's `exit(code)`, and every panic there, throws: a page
25
+ * cannot end itself. Tell it apart with `instanceof`, not by its `name`.
26
+ */
27
+ export declare class NishExit extends Error {
28
+ /** The 8-bit status a native exit would leave: `code & 0xff`. */
29
+ readonly code: number;
30
+ constructor(code: number);
31
+ }
32
+
23
33
  export declare function installHost(next: Host): void;
24
34
  export declare function log(x: unknown): void;
25
35
  export declare function error(x: unknown): void;
@@ -48,6 +48,20 @@ const NO_HOST = {
48
48
 
49
49
  let host = NO_HOST;
50
50
 
51
+ /**
52
+ * What `exit(code)` throws: `code` is the 8-bit status a native exit would
53
+ * leave. The browser host throws it, since a page cannot end itself; it is
54
+ * exported here and from `nish/runtime/browser` so a caller tells it apart with `instanceof`, which a thrown
55
+ * object that only carries the name does not pass.
56
+ */
57
+ export class NishExit extends Error {
58
+ constructor(code) {
59
+ super(`the program exited with ${code}`);
60
+ this.name = "NishExit";
61
+ this.code = code & 0xff;
62
+ }
63
+ }
64
+
51
65
  /** Make `next` the host every later call goes through. */
52
66
  export function installHost(next) {
53
67
  host = next;
@@ -12,6 +12,16 @@ export type u32 = number;
12
12
  export type u64 = bigint;
13
13
  export type f32 = number;
14
14
  export type f64 = number;
15
+ /**
16
+ * A ranged integer (docs/wp31-ranged-integers.md) is a `number`; its range is
17
+ * the native build's check, which Node does not make (wp33 §3.3, the range row).
18
+ */
19
+ export type integer<Lo extends number, Hi extends number> = number;
20
+ /** The typed-array aliases (docs/LANGUAGE.md): in Nish each is an ordinary array of its width. */
21
+ export type Int32Array = i32[];
22
+ export type Float32Array = f32[];
23
+ export type Float64Array = f64[];
24
+ export type BigInt64Array = i64[];
15
25
 
16
26
  /** A width as the rewriter names it, for `convert`. */
17
27
  export type NumericKind = "i32" | "i64" | "u8" | "u16" | "u32" | "u64" | "f32" | "f64";
@@ -46,7 +56,10 @@ export declare function shlU64(a: u64, b: u64): u64;
46
56
  export declare function lshrU64(a: u64, b: u64): u64;
47
57
  export declare function minU64(a: u64, b: u64): u64;
48
58
  export declare function maxU64(a: u64, b: u64): u64;
49
- export declare function convert(x: number | bigint, from: NumericKind, to: NumericKind): number | bigint;
59
+ /** What `convert` answers for the width `K`: a `bigint` for the 64-bit integers. */
60
+ export type Width<K extends NumericKind> = K extends "i64" | "u64" ? bigint : number;
61
+
62
+ export declare function convert<K extends NumericKind>(x: number | bigint, from: NumericKind, to: K): Width<K>;
50
63
  export declare function parseInt(s: string): i32;
51
64
  export declare function parseFloat(s: string): f64;
52
65
  export declare function number(x: string | number | bigint | boolean): f64;
@@ -248,9 +248,36 @@ export function maxU64(a, b) {
248
248
  * what `sext`/`zext`/`trunc` do natively.
249
249
  */
250
250
  export function convert(x, from, to) {
251
- // A zext of an unsigned 32-bit-or-narrower value to u64 keeps the value as it
252
- // is, and skips the BigInt wrap below: a hot path in u64 arithmetic.
253
- if (to === "u64" && (from === "u32" || from === "u16" || from === "u8")) return BigInt(x);
251
+ // The hot paths of u64 arithmetic written over i64 (`toU64(a) * toU64(b)`)
252
+ // stay in a function small enough for V8 to inline at the call: the two
253
+ // 64-bit kinds reinterpret each other's bits, and a zext of an unsigned
254
+ // 32-bit-or-narrower value to u64 keeps the value as it is.
255
+ if (typeof x === "bigint") {
256
+ if (to === "u64") return BigInt.asUintN(64, x);
257
+ if (to === "i64") return BigInt.asIntN(64, x);
258
+ } else if (to === "u64" && (from === "u32" || from === "u16" || from === "u8")) {
259
+ return BigInt(x);
260
+ }
261
+ return convertOther(x, from, to);
262
+ }
263
+
264
+ /** Every conversion `convert` does not answer inline. */
265
+ function convertOther(x, from, to) {
266
+ if (typeof x === "bigint") {
267
+ // A BigInt source narrowed to 32 bits or less skips the `number` round trip.
268
+ const narrow = INT_KINDS[to];
269
+ if (narrow !== undefined) {
270
+ return Number(narrow.signed ? BigInt.asIntN(narrow.bits, x) : BigInt.asUintN(narrow.bits, x));
271
+ }
272
+ } else if (from === "i32" || from === "u32" || from === "u16" || from === "u8") {
273
+ // An integer `number` source narrowed to a `number` kind: the 32-bit wrap
274
+ // `|0` / `>>>0` and a mask do what trunc/sext/zext do, without a BigInt.
275
+ if (to === "i32") return x | 0;
276
+ if (to === "u32") return x >>> 0;
277
+ if (to === "u16") return x & 0xffff;
278
+ if (to === "u8") return x & 0xff;
279
+ if (to === "i64") return BigInt(x);
280
+ }
254
281
  const num = typeof x === "bigint" ? Number(x) : x;
255
282
  if (to === "f64") return num; // sitofp / uitofp / fpext
256
283
  if (to === "f32") return Math.fround(num); // ... / fptrunc, then rounded to a float
@@ -25,6 +25,10 @@ export declare const Result: { prototype: ResultMethods<unknown, unknown> };
25
25
  export declare function Ok<T, E>(value: T): Result<T, E>;
26
26
  export declare function Ok<E>(): Result<void, E>;
27
27
  export declare function Err<T, E>(error: E): Result<T, E>;
28
+ /** `r.expect(message())`, with `message` called on the `Err` path only. */
29
+ export declare function expectLazy<T, E>(r: Result<T, E>, message: () => string): T;
30
+ /** `r.unwrapOr(fallback())`, with `fallback` called on the `Err` path only. */
31
+ export declare function unwrapOrLazy<T, E>(r: Result<T, E>, fallback: () => T): T;
28
32
  /** The `catch` of a body that calls `orReturn`: the `Err` to return, or `thrown` thrown again. */
29
33
  export declare function caught<T, E>(thrown: unknown): Result<T, E>;
30
34
 
@@ -27,6 +27,19 @@ export function exposeWith(s, arg, f) {
27
27
  return f(s.value, arg);
28
28
  }
29
29
 
30
+ /**
31
+ * `r.expect(message)` and `r.unwrapOr(fallback)` as `--emit-ts` writes them
32
+ * when the argument is more than a name or a literal: natively it is
33
+ * evaluated on the `Err` path only (#182), so here it is a function of none.
34
+ */
35
+ export function expectLazy(r, message) {
36
+ return r.ok ? r.value : r.expect(message());
37
+ }
38
+
39
+ export function unwrapOrLazy(r, fallback) {
40
+ return r.ok ? r.value : fallback();
41
+ }
42
+
30
43
  /** Zero an array of integers, or every integer field of a record: `0` for a number, `0n` for an `i64` or `u64`. */
31
44
  function zeroFill(target) {
32
45
  if (Array.isArray(target) || ArrayBuffer.isView(target)) {
@@ -10,5 +10,5 @@ export declare function indexOf(s: string, sub: string, from?: number | bigint):
10
10
  export declare function startsWith(s: string, sub: string): boolean;
11
11
  export declare function endsWith(s: string, sub: string): boolean;
12
12
  export declare function fromCharCode(code: number | bigint): string;
13
- export declare function fromBytes(bytes: readonly u8[], start: number, end: number): string;
14
- export declare function utf8FromBytes(bytes: readonly u8[], start: number, end: number): string;
13
+ export declare function fromBytes(bytes: readonly u8[], start: number | bigint, end: number | bigint): string;
14
+ export declare function utf8FromBytes(bytes: readonly u8[], start: number | bigint, end: number | bigint): string;
package/runtime/nish.d.ts CHANGED
@@ -353,7 +353,7 @@ declare function netWrite(fd: i32, buf: readonly u8[], off: i32, len: i32): i32;
353
353
  declare function netShutdown(fd: i32, how: i32): i32;
354
354
  /** Close the descriptor. */
355
355
  declare function netClose(fd: i32): i32;
356
- /** A TCP socket connecting to `addr`, answered while it connects: writable once it has, or has failed. */
356
+ /** A TCP socket connecting to `addr`, answered while it connects: writable once it has, or has failed. It holds `TCP_NODELAY`, so a small write is sent at once. */
357
357
  declare function tcpConnect(addr: readonly u8[]): i32;
358
358
  /** Once `fd` is writable: 0 when the connection is made, else its failure (-111 refused). */
359
359
  declare function connectResult(fd: i32): i32;
@@ -510,7 +510,7 @@ declare module "nish:net" {
510
510
  /** Shut the read side (0), the write side (1) or both (2). */
511
511
  export function netShutdown(fd: i32, how: i32): i32;
512
512
  export function netClose(fd: i32): i32;
513
- /** A TCP socket connecting to `addr`, answered while it connects: writable once it has, or has failed. */
513
+ /** A TCP socket connecting to `addr`, answered while it connects: writable once it has, or has failed. It holds `TCP_NODELAY`, so a small write is sent at once. */
514
514
  export function tcpConnect(addr: readonly u8[]): i32;
515
515
  /** Once `fd` is writable: 0 when the connection is made, else its failure (-111 refused). */
516
516
  export function connectResult(fd: i32): i32;
package/runtime/node.mjs CHANGED
@@ -378,6 +378,9 @@ installHost({
378
378
  writeOut: (s) => fs.writeSync(1, s),
379
379
  writeErr: (s) => fs.writeSync(2, s),
380
380
  exit: (code) => process.exit(code),
381
+ // Index 0 is the program on both sides: the executable natively, the entry
382
+ // script here, which is one index later than Node's own `argv`. The global
383
+ // is left as Node set it; the output reads `argv()` (`src/emit-ts-records.ts`).
381
384
  argv: () => process.argv.slice(1),
382
385
  call: (name, args) => CALLS[name](...args),
383
386
  });
@@ -318,13 +318,24 @@ int32_t nish_tcp_accept(int32_t fd, nish_array *peer) {
318
318
  descriptor is answered all the same, and the connection is made once the
319
319
  loop reports it writable, when `connectResult` says how it went. Any other
320
320
  failure closes the socket and is answered instead of it. An `addr` shorter
321
- than 18 bytes is -22 before any socket exists. */
321
+ than 18 bytes is -22 before any socket exists. The socket holds
322
+ `TCP_NODELAY` before it connects, as libpq and Go's net package set it on
323
+ every client socket: a client writes whole requests and waits on the
324
+ answer, and Nagle would hold a small write behind an unacknowledged one
325
+ for a delayed ACK (#942). A refused option closes the socket and is
326
+ answered instead of it. */
322
327
  int32_t nish_tcp_connect(const nish_array *addr) {
323
328
  if (addr->len < NISH_ADDR_BYTES) return -22;
324
329
  const unsigned char *p = (const unsigned char *)addr->data;
325
330
  int v4 = memcmp(p, nish_mapped_prefix, 12) == 0;
326
331
  int fd = nish_net_socket(v4 ? AF_INET : AF_INET6, SOCK_STREAM);
327
332
  if (fd < 0) return nish_net_fail();
333
+ int one = 1;
334
+ if (setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof one) != 0) {
335
+ int e = errno;
336
+ close(fd);
337
+ return nish_net_err(e);
338
+ }
328
339
  nish_sockaddr s;
329
340
  socklen_t n = nish_net_sockaddr(&s, p, (p[16] << 8) | p[17], v4);
330
341
  if (connect(fd, &s.sa, n) != 0 && errno != EINPROGRESS) {
package/scripts/build.sh CHANGED
@@ -334,6 +334,39 @@ if [ "$debug" = 1 ]; then common+=(-g); strip_flag=(); fi
334
334
  # float than the baseline build (docs/LANGUAGE.md, "The CPU level").
335
335
  if [ -n "$cpu" ]; then common+=(-march=x86-64-v3 -ffp-contract=off); fi
336
336
 
337
+ # Hardening, on every native profile (the wasm and wasi ones build their own
338
+ # command line and have none of these to give). docs/security/runtime.md
339
+ # records what each one closes and what it costs.
340
+ #
341
+ # The control-flow flag names the target's own scheme: -fcf-protection is an
342
+ # error on arm64 and -mbranch-protection one on x86, so the triple decides.
343
+ # Nish's IR carries both targets' module flags, and each backend reads its own;
344
+ # these make clang's C units agree, because the linker marks the binary only
345
+ # when every object asks for it. -fstack-clash-protection is ELF's alone: on
346
+ # Mach-O clang ignores it with a warning, and the kernel's guard is the stack
347
+ # itself. -z relro -z now leaves the GOT read-only after relocation, and
348
+ # -z noexecstack says so even when an object without the note slips in.
349
+ #
350
+ # -fstack-protector-strong and _FORTIFY_SOURCE reach the C runtime alone: on a
351
+ # .ll input clang adds no ssp attribute and the macro has no source to change,
352
+ # so one flag list serves the one-command link too. FORTIFY wants an optimiser
353
+ # and is left out of debug, where it would only warn. -U first, because a
354
+ # distribution clang may already define it at another level.
355
+ case "$("$CC" -dumpmachine 2>/dev/null)" in
356
+ x86_64*|amd64*) common+=(-fcf-protection=full); landing=1 ;;
357
+ aarch64*|arm64*) common+=(-mbranch-protection=standard); landing=1 ;;
358
+ *) landing=0 ;;
359
+ esac
360
+ common+=(-fstack-protector-strong)
361
+ if [ "$profile" != debug ]; then common+=(-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3); fi
362
+ case "$(uname -s)" in
363
+ Darwin) ;;
364
+ *)
365
+ common+=(-fstack-clash-protection)
366
+ # shellcheck disable=SC2054 # -Wl,<flag> is one argument: the comma is the linker's
367
+ common+=(-Wl,-z,relro,-z,now -Wl,-z,noexecstack) ;;
368
+ esac
369
+
337
370
  tls=()
338
371
  if [ "$threads" = 1 ]; then
339
372
  common+=(-DNISH_THREADS=1 -ftls-model=initial-exec -pthread)
@@ -353,7 +386,7 @@ case "$profile" in
353
386
  optimise=(-Oz -flto -DNDEBUG ${pgo[@]+"${pgo[@]}"}
354
387
  -ffunction-sections -fdata-sections -fomit-frame-pointer
355
388
  -fno-asynchronous-unwind-tables -fno-unwind-tables ${elf[@]+"${elf[@]}"}
356
- -fno-stack-protector -fvisibility=hidden) ;;
389
+ -fvisibility=hidden) ;;
357
390
  napi)
358
391
  optimise=(-O3 -flto -DNDEBUG ${pgo[@]+"${pgo[@]}"}
359
392
  -ffunction-sections -fdata-sections -fomit-frame-pointer
@@ -362,14 +395,18 @@ esac
362
395
 
363
396
  # The speed link splits full LTO's code generation into one partition per core,
364
397
  # run in parallel: lld's --lto-partitions, after the whole program is optimised
365
- # as one module, so inlining and the instructions a program runs do not change
366
- # (bench/instructions.json agrees to a few instructions in hundreds of
367
- # millions). The bytes depend on the count, so one machine links one input to
368
- # one output, as scripts/bootstrap.sh needs. ld64 and GNU ld have no such flag
369
- # and link as before, and so does every other profile. The count comes
370
- # from nproc (Linux), else getconf (both), else sysctl (macOS).
398
+ # as one module, so inlining does not change. But a partition boundary promotes
399
+ # the internal functions called across it to external ones, and under the
400
+ # control-flow hardening above every external function opens with a landing pad
401
+ # (endbr64, or bti c), which the hot calls then run: fib runs 6.5% more
402
+ # instructions with two partitions than with one. So a target with landing pads
403
+ # links in one partition, and the split is left to the rest. The bytes depend
404
+ # on the count, so one machine links one input to one output, as
405
+ # scripts/bootstrap.sh needs. ld64 and GNU ld have no such flag and link as
406
+ # before, and so does every other profile. The count comes from nproc (Linux),
407
+ # else getconf (both), else sysctl (macOS).
371
408
  lto_parts=()
372
- if [ "$profile" = speed ] && [ "$lld" = 1 ]; then
409
+ if [ "$profile" = speed ] && [ "$lld" = 1 ] && [ "$landing" = 0 ]; then
373
410
  cores=$(nproc 2>/dev/null || getconf _NPROCESSORS_ONLN 2>/dev/null ||
374
411
  sysctl -n hw.ncpu 2>/dev/null || echo 1)
375
412
  # shellcheck disable=SC2054 # -Wl,<flag> is one argument: the comma is the linker's
@@ -379,15 +416,22 @@ fi
379
416
  # --runtime-objects and --runtime-from: the one-command link split in two, with
380
417
  # every compile it held run at once. A `$CC -c` takes `common` and `optimise`,
381
418
  # which is what the one command compiles each input with -- less
382
- # -fuse-ld=lld, which only names the linker -- so a runtime unit compiles to the
383
- # same object either way: native code for debug, -flto bitcode for speed and
384
- # size, which the link still optimises together with the program. The link
385
- # takes the same line the profile always uses, with objects for sources.
419
+ # -fuse-ld=lld and the -Wl, flags, which only speak to the linker -- so a
420
+ # runtime unit compiles to the same object either way: native code for debug,
421
+ # -flto bitcode for speed and size, which the link still optimises together
422
+ # with the program. The link takes the same line the profile always uses, with
423
+ # objects for sources.
386
424
  compile=()
387
425
  for flag in "${common[@]}"; do
388
- case "$flag" in -fuse-ld=*) ;; *) compile+=("$flag") ;; esac
426
+ case "$flag" in -fuse-ld=*|-Wl,*) ;; *) compile+=("$flag") ;; esac
389
427
  done
390
428
  compile+=(${optimise[@]+"${optimise[@]}"})
429
+ # A program's modules are .ll alone, and clang warns that a -U it cannot
430
+ # apply to one went unused, so their compile leaves the FORTIFY pair out.
431
+ module_compile=()
432
+ for flag in "${compile[@]}"; do
433
+ case "$flag" in -U_FORTIFY_SOURCE|-D_FORTIFY_SOURCE=*) ;; *) module_compile+=("$flag") ;; esac
434
+ done
391
435
  # Waits for every job, not just the last, and fails when any of them did.
392
436
  # bash 3.2 has no `wait -n`, so the jobs are waited for in order.
393
437
  pids=()
@@ -542,7 +586,7 @@ if [ -n "$objects_from" ]; then
542
586
  for i in "${inputs[@]}"; do
543
587
  case "$i" in */runtime.c|runtime.c) echo "error: --runtime-from links the runtime in $objects_from, so $i is not an input" >&2; exit 2 ;; esac
544
588
  n=$((n + 1))
545
- "$CC" "${compile[@]}" -c "$i" -o "$work/$n.o" &
589
+ "$CC" "${module_compile[@]}" -c "$i" -o "$work/$n.o" &
546
590
  pids+=("$!")
547
591
  objects+=("$work/$n.o")
548
592
  done