@amritk/nish 0.12.0 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -23
- package/bin/launcher.js +45 -41
- package/bin/packaging.js +39 -33
- package/docs/AI.md +95 -10
- package/docs/INSTALL.md +13 -13
- package/package.json +9 -6
- package/runtime/nish.d.ts +20 -1
- package/runtime/nish.h +31 -11
- package/runtime/nish.mjs +3 -3
- package/runtime/{runtime_os.c → runtime-os.c} +1 -1
- package/runtime/{runtime_parallel.c → runtime-parallel.c} +112 -4
- package/runtime/{runtime_wasm.c → runtime-wasm.c} +6 -1
- package/runtime/runtime.c +5 -5
- package/runtime/shim.mjs +6 -6
- package/scripts/bootstrap.sh +29 -29
- package/scripts/build.sh +14 -9
- package/scripts/changelog-gen.mjs +260 -192
- package/scripts/ci-profile.mjs +80 -68
- package/scripts/codes-registry.js +25 -13
- package/scripts/gen-diagnostic-codes.mjs +86 -69
- package/scripts/nish-compiler.sh +2 -0
- package/scripts/platform-package.mjs +26 -26
- package/scripts/postinstall.mjs +46 -36
- package/scripts/size-report.sh +3 -3
- package/scripts/smoke.sh +1 -1
- package/std/README.md +2 -2
- package/std/collections.ts +194 -188
- package/std/json.ts +136 -136
- package/std/map.ts +9 -7
- package/std/pair.ts +2 -2
- package/std/testing.ts +67 -67
- package/std/text.ts +54 -54
- package/std/threads.ts +126 -38
package/docs/INSTALL.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`nish` compiles a static subset of TypeScript to LLVM IR and, with
|
|
4
4
|
`--link`, to a native binary. The compiler is itself a native binary, written
|
|
5
|
-
in Nish (`
|
|
5
|
+
in Nish (`src/`), and needs nothing to run; the `--link` step (and anything
|
|
6
6
|
else that turns `.ll` into machine code) needs an LLVM toolchain.
|
|
7
7
|
|
|
8
8
|
## 1. Prerequisites
|
|
@@ -142,7 +142,7 @@ The npm route installs the **native** compiler, and it is a download rather than
|
|
|
142
142
|
build: nothing is compiled on your machine. The package declares one
|
|
143
143
|
`nish-<os>-<arch>` package per supported platform as an `optionalDependencies`
|
|
144
144
|
entry with `os` and `cpu` set, so npm fetches exactly the one that matches and
|
|
145
|
-
skips the rest. Each of those carries the self-hosted compiler — `
|
|
145
|
+
skips the rest. Each of those carries the self-hosted compiler — `src/`
|
|
146
146
|
compiled by itself — already built, `--verify`d and smoke-tested on a machine
|
|
147
147
|
of its own architecture by the release workflow.
|
|
148
148
|
|
|
@@ -198,7 +198,7 @@ nish: no prebuilt compiler for freebsd/x64
|
|
|
198
198
|
|
|
199
199
|
Until 0.6.0 it ran the TypeScript compiler that shipped in the same package
|
|
200
200
|
instead — the same compiler by every test here, about eight times slower, and
|
|
201
|
-
no C toolchain needed. That compiler was
|
|
201
|
+
no C toolchain needed. That compiler was stage0, which was deleted in R6
|
|
202
202
|
([wp19](wp19-stage0-retirement.md)), so there is nothing left in the package to
|
|
203
203
|
fall back to. The cost is stated where the rest of that deletion's costs are,
|
|
204
204
|
in [wp19 §6](wp19-stage0-retirement.md#6-what-retirement-costs-stated-plainly),
|
|
@@ -218,7 +218,7 @@ routes are
|
|
|
218
218
|
# in the glibc container, where build/nish runs
|
|
219
219
|
build/nish app.ts -o app.ll
|
|
220
220
|
# on the musl host, with its own clang and the runtime from this repository
|
|
221
|
-
clang app.ll runtime/runtime.c runtime/
|
|
221
|
+
clang app.ll runtime/runtime.c runtime/runtime-os.c -lm -o app
|
|
222
222
|
```
|
|
223
223
|
|
|
224
224
|
`--link`ing inside the container is the mistake to avoid: it shells out to the
|
|
@@ -268,7 +268,7 @@ npm install -g ./amritk-nish-0.4.0.tgz ./amritk-nish-x86_64-linux-0.4.0.tgz
|
|
|
268
268
|
```
|
|
269
269
|
|
|
270
270
|
As a native compiler, which needs no Node at all. A release also attaches the
|
|
271
|
-
self-hosted compiler — the binary `
|
|
271
|
+
self-hosted compiler — the binary `src/` produces by compiling itself — one
|
|
272
272
|
per supported platform, from the version named in the last column:
|
|
273
273
|
|
|
274
274
|
| Asset | For | Attached from |
|
|
@@ -326,7 +326,7 @@ release exists, take its version from
|
|
|
326
326
|
Unpack it and run `bin/nish` from wherever you like; put that on `PATH` if you
|
|
327
327
|
want it there. Keep the directory intact rather than moving the binary out of
|
|
328
328
|
it: `--link` runs `scripts/build.sh` and compiles the C runtime
|
|
329
|
-
(`runtime/runtime.c` and `runtime/
|
|
329
|
+
(`runtime/runtime.c` and `runtime/runtime-os.c`, the system-call half), and the
|
|
330
330
|
compiler finds all of them relative to its own location — `bin/nish` alone in a
|
|
331
331
|
directory can still emit IR with `-o`, but `--link` will tell you it cannot
|
|
332
332
|
find `scripts/build.sh`.
|
|
@@ -367,23 +367,23 @@ Building the IR yourself rather than through `--link` means naming the runtime
|
|
|
367
367
|
on the `clang` line, and it is two files:
|
|
368
368
|
|
|
369
369
|
```bash
|
|
370
|
-
clang app.ll runtime/runtime.c runtime/
|
|
370
|
+
clang app.ll runtime/runtime.c runtime/runtime-os.c -lm -o app
|
|
371
371
|
```
|
|
372
372
|
|
|
373
373
|
`runtime.c` is the half every program touches — the arena, strings, arrays,
|
|
374
|
-
number formatting, the panics — and `
|
|
374
|
+
number formatting, the panics — and `runtime-os.c` is the half that wraps the
|
|
375
375
|
system calls: files, directories, subprocesses, `getenv`, the monotonic clock.
|
|
376
376
|
They are separate so that each carries its own measured size ceiling
|
|
377
377
|
([docs/wp7-runtime.md](wp7-runtime.md)); nothing in the core calls into the
|
|
378
378
|
system-call half, so an older line that names `runtime.c` alone still links a
|
|
379
379
|
program that reads no files and spawns nothing. `scripts/build.sh` compiles
|
|
380
|
-
`
|
|
380
|
+
`runtime-os.c` beside any `runtime.c` it is handed, so a build that goes
|
|
381
381
|
through it — every `--link`, and every `--profile` recipe in these documents —
|
|
382
382
|
needs to name only the one.
|
|
383
383
|
|
|
384
384
|
## 2a. What `npm run build` does
|
|
385
385
|
|
|
386
|
-
`
|
|
386
|
+
`src/` is the compiler, written in Nish, and it compiles itself
|
|
387
387
|
([docs/wp14-selfhost.md](wp14-selfhost.md)). From a checkout, with clang on
|
|
388
388
|
`PATH`, `npm run build` runs `scripts/bootstrap.sh` with the seed from §2:
|
|
389
389
|
|
|
@@ -406,11 +406,11 @@ binary itself. It looks for `scripts/build.sh` and the two `runtime/*.c` files
|
|
|
406
406
|
one level up from wherever it was invoked, then in the working directory, so
|
|
407
407
|
it wants a checkout or an installed package around it the way `nish` does.
|
|
408
408
|
|
|
409
|
-
Because the seed is the last release, `
|
|
409
|
+
Because the seed is the last release, `src/` may only *use* in its own
|
|
410
410
|
source the constructs that release compiles. A new construct is implemented
|
|
411
|
-
in `
|
|
411
|
+
in `src/` and becomes usable inside `src/` from the next release on; CI's
|
|
412
412
|
`bootstrap` job is what checks that the released seed still builds stage1.
|
|
413
|
-
Until R6 the seed was a TypeScript compiler
|
|
413
|
+
Until R6 the seed was a TypeScript compiler, stage0, run under Node; it was
|
|
414
414
|
deleted once the native one answered every flag it did
|
|
415
415
|
([wp19](wp19-stage0-retirement.md)).
|
|
416
416
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@amritk/nish",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.13.0",
|
|
4
4
|
"description": "Nish: an ahead-of-time compiler from a static subset of TypeScript to LLVM IR",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -46,7 +46,9 @@
|
|
|
46
46
|
"test:nish": "build/nish tests/nish/run.ts -o build/nish-runner.ir/ --link build/nish-runner && build/nish-runner",
|
|
47
47
|
"test:cli": "build/nish tests/nish/cli.ts -o build/nish-cli.ir/ --link build/nish-cli && build/nish-cli",
|
|
48
48
|
"test:node": "node tests/differential/unmodified.js",
|
|
49
|
-
"lint": "node scripts/check-filenames.mjs
|
|
49
|
+
"lint": "node scripts/check-filenames.mjs && biome check .",
|
|
50
|
+
"lint:fix": "biome check --write .",
|
|
51
|
+
"lint:dead": "knip --no-progress",
|
|
50
52
|
"format": "biome format --write .",
|
|
51
53
|
"prepublishOnly": "npm run check && npm test",
|
|
52
54
|
"changelog": "node scripts/changelog-gen.mjs"
|
|
@@ -75,14 +77,15 @@
|
|
|
75
77
|
"nish"
|
|
76
78
|
],
|
|
77
79
|
"optionalDependencies": {
|
|
78
|
-
"@amritk/nish-x86_64-linux": "0.
|
|
79
|
-
"@amritk/nish-aarch64-linux": "0.
|
|
80
|
-
"@amritk/nish-aarch64-darwin": "0.
|
|
81
|
-
"@amritk/nish-x86_64-darwin": "0.
|
|
80
|
+
"@amritk/nish-x86_64-linux": "0.13.0",
|
|
81
|
+
"@amritk/nish-aarch64-linux": "0.13.0",
|
|
82
|
+
"@amritk/nish-aarch64-darwin": "0.13.0",
|
|
83
|
+
"@amritk/nish-x86_64-darwin": "0.13.0"
|
|
82
84
|
},
|
|
83
85
|
"devDependencies": {
|
|
84
86
|
"@biomejs/biome": "2.5.12",
|
|
85
87
|
"@types/node": "^22.0.0",
|
|
88
|
+
"knip": "6.38.0",
|
|
86
89
|
"typescript": "^5.6.0"
|
|
87
90
|
}
|
|
88
91
|
}
|
package/runtime/nish.d.ts
CHANGED
|
@@ -54,6 +54,12 @@ type u64 = number;
|
|
|
54
54
|
type f32 = number;
|
|
55
55
|
type f64 = number;
|
|
56
56
|
|
|
57
|
+
// ---- Ranged integers (docs/wp31-ranged-integers.md) --------------------------
|
|
58
|
+
//
|
|
59
|
+
// An `i32` the compiler knows lies in `[Lo, Hi]`. The bounds are numeric
|
|
60
|
+
// literal types, and `tsc` checks only that; the range itself is `nish`'s.
|
|
61
|
+
type integer<Lo extends number, Hi extends number> = number;
|
|
62
|
+
|
|
57
63
|
// ---- Result (docs/LANGUAGE.md -> Result and error handling) ------------------
|
|
58
64
|
//
|
|
59
65
|
// Modelled as the tagged union TypeScript would use anyway, intersected with
|
|
@@ -104,6 +110,19 @@ declare function Err<T, E>(error: E): Result<T, E>;
|
|
|
104
110
|
// `Process` is not that lucky, so a project that needs `@types/node` for other
|
|
105
111
|
// reasons should drop this file's `process` rather than fight it.
|
|
106
112
|
|
|
113
|
+
/**
|
|
114
|
+
* The disposable protocol `using` reads (WP29 P2, docs/wp29-thread-surface.md
|
|
115
|
+
* §5): declared here so that a program using `nish/threads`'s scope needs no
|
|
116
|
+
* `"ESNext.Disposable"` in its `lib`. `nish` itself takes `using` only for a
|
|
117
|
+
* `scope()`, and `[Symbol.dispose]` only in `nish/threads`.
|
|
118
|
+
*/
|
|
119
|
+
interface SymbolConstructor {
|
|
120
|
+
readonly dispose: unique symbol;
|
|
121
|
+
}
|
|
122
|
+
interface Disposable {
|
|
123
|
+
[Symbol.dispose](): void;
|
|
124
|
+
}
|
|
125
|
+
|
|
107
126
|
interface Console {
|
|
108
127
|
/** `x` and a newline to stdout. Statement position; exactly one argument. */
|
|
109
128
|
log(x: string | number | boolean): void;
|
|
@@ -152,7 +171,7 @@ declare function writeError(s: string): void;
|
|
|
152
171
|
* `message` and a newline to stderr, then exit 1. Terminates control flow, so
|
|
153
172
|
* it is `never`: that is what lets `tsc` agree that a function ending in a
|
|
154
173
|
* `panic` returns, and that `x` is not null after `if (x === null) { panic(...); }`
|
|
155
|
-
* — the guard-then-panic shape `
|
|
174
|
+
* — the guard-then-panic shape `src/` uses everywhere in place of an assert.
|
|
156
175
|
*/
|
|
157
176
|
declare function panic(message: string): never;
|
|
158
177
|
/**
|
package/runtime/nish.h
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
*
|
|
3
3
|
* Include this from C drivers, N-API shims, or any other host that links the C
|
|
4
4
|
* runtime next to compiled Nish modules. Everything here is a contract shared
|
|
5
|
-
* with `
|
|
5
|
+
* with `src/runtime.ts` (the IR side) and the implementation, which is
|
|
6
6
|
* two translation units: `runtime/runtime.c` holds the core every program
|
|
7
7
|
* touches — the arena, strings, arrays, number formatting, the panics — and
|
|
8
|
-
* `runtime/
|
|
8
|
+
* `runtime/runtime-os.c` holds everything that wraps a system call: the file
|
|
9
9
|
* functions, the directory and subprocess calls, `nish_getenv`, the monotonic
|
|
10
10
|
* clock, `nish_platform` / `nish_arch`. They are separate so that each carries
|
|
11
11
|
* its own measured code-size ceiling (docs/wp7-runtime.md, "Runtime additions
|
|
@@ -41,7 +41,7 @@ extern "C" {
|
|
|
41
41
|
* ELF refuses to link that against a non-TLS definition. `scripts/build.sh
|
|
42
42
|
* --threads` passes the macro to every input, which is how `nish --threads
|
|
43
43
|
* --link` keeps the two halves in step. The same definition is in
|
|
44
|
-
* runtime/runtime.c and runtime/
|
|
44
|
+
* runtime/runtime.c and runtime/runtime-wasm.c. */
|
|
45
45
|
#ifdef NISH_THREADS
|
|
46
46
|
#define NISH_TLS _Thread_local
|
|
47
47
|
#else
|
|
@@ -73,7 +73,7 @@ typedef struct nish_str {
|
|
|
73
73
|
* `off` and `cap` are `uint64_t` and not `size_t` because that `i64` is what
|
|
74
74
|
* the inlined fast path bumps and compares on every target: under wasm32 a
|
|
75
75
|
* `size_t` pair would put them at bytes 4 and 8 while compiled code reads 8
|
|
76
|
-
* and 16. runtime.c and
|
|
76
|
+
* and 16. runtime.c and runtime-wasm.c static-assert these offsets. */
|
|
77
77
|
struct nish_arena {
|
|
78
78
|
char *buf;
|
|
79
79
|
uint64_t off;
|
|
@@ -140,7 +140,7 @@ nish_str *nish_str_from_u64(uint64_t v);
|
|
|
140
140
|
|
|
141
141
|
/* Process and file I/O (WP7). `nish_exit` never returns; the file functions
|
|
142
142
|
* print a message to stderr and exit(1) on a fatal error. Everything from here
|
|
143
|
-
* to the end of the clock section below is implemented in
|
|
143
|
+
* to the end of the clock section below is implemented in runtime-os.c, with
|
|
144
144
|
* `nish_random` and `nish_argv_init` the exceptions: neither asks the operating
|
|
145
145
|
* system anything (the clock only seeds the first, and the entry point hands the
|
|
146
146
|
* second its arguments), so both stay with the core. `nish_random` keeps one seed
|
|
@@ -181,11 +181,11 @@ void nish_append_file(const nish_str *path, const nish_str *data);
|
|
|
181
181
|
* A returned array lives in the arena (valid until the next reset/release):
|
|
182
182
|
* copy `len` elements out of `data` before recycling.
|
|
183
183
|
*
|
|
184
|
-
* An array field stored inside its object (`
|
|
184
|
+
* An array field stored inside its object (`src/inline-arrays.ts`) is this
|
|
185
185
|
* header followed by its `K` slots, `struct { nish_array h; T slots[K]; }`
|
|
186
|
-
* with `h.data == (char *)slots` and `h.cap == K`; `
|
|
187
|
-
* `ARRAY_TYPE` and `
|
|
188
|
-
* bytes, and `tests/layout/
|
|
186
|
+
* with `h.data == (char *)slots` and `h.cap == K`; `src/runtime.ts`'s
|
|
187
|
+
* `ARRAY_TYPE` and `src/structs.ts`'s `INLINE_HEADER_BYTES` are the same 24
|
|
188
|
+
* bytes, and `tests/layout/inline-array.c` holds the three to it. It only
|
|
189
189
|
* happens where no header, `.d.ts` or N-API shim describes the class, so a
|
|
190
190
|
* host that includes a generated header never meets one. */
|
|
191
191
|
typedef struct nish_array { uint64_t len; uint64_t cap; char *data; } nish_array;
|
|
@@ -260,7 +260,7 @@ int32_t nish_spawn(const nish_array *argv);
|
|
|
260
260
|
* paths must differ: each is opened separately with its own offset, so naming
|
|
261
261
|
* one file twice makes the streams overwrite each other instead of
|
|
262
262
|
* interleaving; capture them apart and concatenate to merge them. One `static`
|
|
263
|
-
* implementation in
|
|
263
|
+
* implementation in runtime-os.c backs both spawn builtins, which is what keeps
|
|
264
264
|
* the argument vector, the wait and the signal convention written once. */
|
|
265
265
|
int32_t nish_spawn_to(const nish_array *argv, const nish_str *out, const nish_str *err);
|
|
266
266
|
|
|
@@ -316,7 +316,7 @@ double nish_parse_number(const nish_str *s, int32_t mode);
|
|
|
316
316
|
/* Checked integer division (Rust semantics): the failed-check path. */
|
|
317
317
|
void nish_panic_div(bool by_zero);
|
|
318
318
|
|
|
319
|
-
/* ---- Parallel work (WP20 T1 / wp29 stage P1), runtime/
|
|
319
|
+
/* ---- Parallel work (WP20 T1 / wp29 stage P1), runtime/runtime-parallel.c ----
|
|
320
320
|
*
|
|
321
321
|
* One region of work, divided. `nish_parallel_range` calls `body(lo, hi, ctx)`
|
|
322
322
|
* once per chunk of a partition of `[0, len)`: contiguous chunks, at most one
|
|
@@ -341,6 +341,26 @@ typedef void (*nish_par_body)(int64_t lo, int64_t hi, void *ctx);
|
|
|
341
341
|
int64_t nish_cpu_count(void);
|
|
342
342
|
void nish_parallel_range(nish_par_body body, void *ctx, int64_t len, int64_t grain);
|
|
343
343
|
|
|
344
|
+
/* ---- A scope's tasks (wp29 stage P2), runtime/runtime-parallel.c ----
|
|
345
|
+
*
|
|
346
|
+
* `nish_scope_spawn` files one task under `scope`, an address the caller keeps
|
|
347
|
+
* alive until it joins: it copies `size` bytes of `payload` and keeps them with
|
|
348
|
+
* `run` and `finish`, and runs nothing yet. `nish_scope_join` takes every task
|
|
349
|
+
* filed under `scope`, runs `run(payload)` for each -- the first on the calling
|
|
350
|
+
* thread and each other on a thread of its own, which frees its arena before it
|
|
351
|
+
* exits -- waits for all of them, and then calls `finish(payload)` for each on
|
|
352
|
+
* the calling thread, in the order they were filed. Without `-DNISH_THREADS`,
|
|
353
|
+
* or when a thread cannot be created, a task runs on the calling thread, so
|
|
354
|
+
* running short of threads costs speed and never an answer.
|
|
355
|
+
*
|
|
356
|
+
* The preconditions are the language's (docs/LANGUAGE.md, "Scoped tasks"):
|
|
357
|
+
* `run` writes nothing another task or the caller can see, its result is a
|
|
358
|
+
* scalar it writes into its own payload, and only `finish`, on the calling
|
|
359
|
+
* thread, stores into memory the caller owns. */
|
|
360
|
+
typedef void (*nish_task_fn)(void *payload);
|
|
361
|
+
void nish_scope_spawn(void *scope, nish_task_fn run, nish_task_fn finish, const void *payload, int64_t size);
|
|
362
|
+
void nish_scope_join(void *scope);
|
|
363
|
+
|
|
344
364
|
#ifdef __cplusplus
|
|
345
365
|
}
|
|
346
366
|
#endif
|
package/runtime/nish.mjs
CHANGED
|
@@ -28,9 +28,9 @@
|
|
|
28
28
|
* so is every string offset. ASCII agrees; nothing else does.
|
|
29
29
|
* - **`a[i]` is unchecked**: out of range is `undefined` here and an exit-1
|
|
30
30
|
* panic natively. Only a program that indexes out of range can tell.
|
|
31
|
-
* - **
|
|
32
|
-
*
|
|
33
|
-
*
|
|
31
|
+
* - **A record put into an array is shared here** and copied natively, and
|
|
32
|
+
* `slice`, `new Array<T>(n)` and the typed-array names each differ too;
|
|
33
|
+
* the document lists them.
|
|
34
34
|
* - **`orReturn()` does not propagate.** It throws a marker the rewriter's
|
|
35
35
|
* `try`/`catch` turns into an early `return`; unmodified there is no
|
|
36
36
|
* `catch`, so it escapes. `Ok`/`Err`/`isOk`/`isErr`/`value`/`error`/
|
|
@@ -319,7 +319,7 @@ nish_str *nish_realpath(const nish_str *path) {
|
|
|
319
319
|
this file is compiled — a cross build compiles the runtime for the target,
|
|
320
320
|
so the answer is the target's — which is why each is a string in constant
|
|
321
321
|
data handed back by address: no allocation and no load, and `readnone` on
|
|
322
|
-
the declaration (
|
|
322
|
+
the declaration (src/runtime.ts) is a fact rather than a hope. The
|
|
323
323
|
spellings are Node's, so a program reads the same answer from this runtime
|
|
324
324
|
and from `runtime/shim.mjs`; anything neither branch names is "unknown",
|
|
325
325
|
which is what `--target host` then refuses. Contracts: nish.h.
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
/* Nish runtime, the parallel half: how one range of work becomes several
|
|
2
2
|
* threads, and nothing else.
|
|
3
3
|
*
|
|
4
|
-
* A third translation unit rather than a third of `
|
|
4
|
+
* A third translation unit rather than a third of `runtime-os.c`, for the
|
|
5
5
|
* reason that file's header gives for being a second one: a budget should mean
|
|
6
|
-
* one thing. `
|
|
6
|
+
* one thing. `runtime-os.c` is the syscall wrappers and it had 29 bytes of its
|
|
7
7
|
* ceiling left; `pthread_create` plus a partitioner does not fit in 29 bytes,
|
|
8
8
|
* and raising the syscall half's ceiling to make room would move the number a
|
|
9
9
|
* reader sees for "the operating-system surface" for a reason that has nothing
|
|
@@ -21,11 +21,16 @@
|
|
|
21
21
|
* function exists. The thread-local arena the divided case needs is WP20 T0 and
|
|
22
22
|
* arrives with the same macro.
|
|
23
23
|
*
|
|
24
|
-
*
|
|
25
|
-
*
|
|
24
|
+
* docs/wp29-thread-surface.md is the surface this is the stage under: P1's
|
|
25
|
+
* `parallelMapInto` and `parallelReduce` divide a range here, and P2's scopes
|
|
26
|
+
* run their tasks here (`nish_scope_spawn`, `nish_scope_join`).
|
|
26
27
|
*/
|
|
27
28
|
#include "nish.h"
|
|
28
29
|
|
|
30
|
+
#include <stdlib.h>
|
|
31
|
+
#include <string.h>
|
|
32
|
+
#include <unistd.h>
|
|
33
|
+
|
|
29
34
|
#if defined(NISH_THREADS) && !defined(__wasi__) && !defined(__wasm__)
|
|
30
35
|
#define NISH_PAR_REAL 1
|
|
31
36
|
#include <pthread.h>
|
|
@@ -154,3 +159,106 @@ void nish_parallel_range(nish_par_body body, void *ctx, int64_t len, int64_t gra
|
|
|
154
159
|
body(0, len, ctx);
|
|
155
160
|
#endif
|
|
156
161
|
}
|
|
162
|
+
|
|
163
|
+
/* ---- A scope's tasks (wp29 P2) ----
|
|
164
|
+
*
|
|
165
|
+
* A task is filed when it is spawned and run when its scope joins, which is
|
|
166
|
+
* what keeps a scope race-free without a borrow checker: while the tasks run,
|
|
167
|
+
* the thread that opened the scope is inside the join and runs nothing else,
|
|
168
|
+
* the tasks write nothing but their own payloads (the language's rule), and
|
|
169
|
+
* every store into the caller's memory is a `finish` made on the caller's
|
|
170
|
+
* thread after the last task has finished.
|
|
171
|
+
*
|
|
172
|
+
* The filed tasks are one list per thread, newest first, and each carries the
|
|
173
|
+
* scope it belongs to. Only the thread that opened a scope files into it or
|
|
174
|
+
* joins it, so the list needs no lock, and scopes may nest: a join takes its
|
|
175
|
+
* own scope's tasks out of the list and leaves the rest. */
|
|
176
|
+
typedef struct nish_task {
|
|
177
|
+
struct nish_task *next;
|
|
178
|
+
void *scope;
|
|
179
|
+
nish_task_fn run;
|
|
180
|
+
nish_task_fn finish;
|
|
181
|
+
#ifdef NISH_PAR_REAL
|
|
182
|
+
pthread_t th;
|
|
183
|
+
int started;
|
|
184
|
+
#endif
|
|
185
|
+
/* The copied payload, aligned for any field it holds. */
|
|
186
|
+
uint64_t payload[];
|
|
187
|
+
} nish_task;
|
|
188
|
+
|
|
189
|
+
static NISH_TLS nish_task *nish_tasks = 0;
|
|
190
|
+
|
|
191
|
+
void nish_scope_spawn(void *scope, nish_task_fn run, nish_task_fn finish, const void *payload, int64_t size) {
|
|
192
|
+
nish_task *t = (nish_task *)malloc(sizeof(nish_task) + (size_t)size);
|
|
193
|
+
/* Out of memory is the end of the program, as it is for every other
|
|
194
|
+
* allocation the runtime makes: running the task here instead would store
|
|
195
|
+
* its answer ahead of tasks filed before it, and a later spawn into the same
|
|
196
|
+
* slot has to be the one that stays. */
|
|
197
|
+
if (!t) {
|
|
198
|
+
(void)!write(2, "nish: out of memory\n", 20);
|
|
199
|
+
_exit(1);
|
|
200
|
+
}
|
|
201
|
+
t->scope = scope;
|
|
202
|
+
t->run = run;
|
|
203
|
+
t->finish = finish;
|
|
204
|
+
memcpy(t->payload, payload, (size_t)size);
|
|
205
|
+
t->next = nish_tasks;
|
|
206
|
+
nish_tasks = t;
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
#ifdef NISH_PAR_REAL
|
|
210
|
+
static void *nish_task_worker(void *p) {
|
|
211
|
+
nish_task *t = (nish_task *)p;
|
|
212
|
+
/* A task is one thread's work, as a chunk is, so a region inside it runs on
|
|
213
|
+
* this thread rather than multiplying the threads. */
|
|
214
|
+
nish_par_depth = 1;
|
|
215
|
+
t->run(t->payload);
|
|
216
|
+
nish_free_arena(); /* as `nish_par_worker`: this thread's arena, and only it */
|
|
217
|
+
return 0;
|
|
218
|
+
}
|
|
219
|
+
#endif
|
|
220
|
+
|
|
221
|
+
void nish_scope_join(void *scope) {
|
|
222
|
+
/* This scope's tasks, oldest first: the list is newest first, and moving each
|
|
223
|
+
* one to the front of `mine` reverses it. */
|
|
224
|
+
nish_task *mine = 0;
|
|
225
|
+
nish_task **link = &nish_tasks;
|
|
226
|
+
while (*link) {
|
|
227
|
+
nish_task *t = *link;
|
|
228
|
+
if (t->scope == scope) {
|
|
229
|
+
*link = t->next;
|
|
230
|
+
t->next = mine;
|
|
231
|
+
mine = t;
|
|
232
|
+
} else {
|
|
233
|
+
link = &t->next;
|
|
234
|
+
}
|
|
235
|
+
}
|
|
236
|
+
if (!mine) return;
|
|
237
|
+
#ifdef NISH_PAR_REAL
|
|
238
|
+
/* Every task but the first on a thread of its own, and the first on this
|
|
239
|
+
* one, so N tasks cost N-1 spawns and this thread is not idle. A task inside
|
|
240
|
+
* a region or another task's thread runs here, as a nested region does. */
|
|
241
|
+
for (nish_task *t = mine->next; t; t = t->next) {
|
|
242
|
+
t->started = nish_par_depth == 0 && pthread_create(&t->th, 0, nish_task_worker, t) == 0;
|
|
243
|
+
}
|
|
244
|
+
nish_par_depth++;
|
|
245
|
+
mine->run(mine->payload);
|
|
246
|
+
for (nish_task *t = mine->next; t; t = t->next) {
|
|
247
|
+
if (!t->started) t->run(t->payload);
|
|
248
|
+
}
|
|
249
|
+
nish_par_depth--;
|
|
250
|
+
for (nish_task *t = mine->next; t; t = t->next) {
|
|
251
|
+
if (t->started) pthread_join(t->th, 0);
|
|
252
|
+
}
|
|
253
|
+
#else
|
|
254
|
+
for (nish_task *t = mine; t; t = t->next) t->run(t->payload);
|
|
255
|
+
#endif
|
|
256
|
+
/* The stores, in the order the tasks were spawned, so two tasks with one
|
|
257
|
+
* destination leave the later one's answer there, as they would in sequence. */
|
|
258
|
+
while (mine) {
|
|
259
|
+
nish_task *t = mine;
|
|
260
|
+
mine = t->next;
|
|
261
|
+
t->finish(t->payload);
|
|
262
|
+
free(t);
|
|
263
|
+
}
|
|
264
|
+
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
/* Nish runtime for the freestanding wasm profile (WP8): the arena and the
|
|
2
2
|
* array cold paths, without libc. Link it next to the module when a function
|
|
3
3
|
* takes or returns an array:
|
|
4
|
-
* scripts/build.sh x.ll runtime/
|
|
4
|
+
* scripts/build.sh x.ll runtime/runtime-wasm.c -o x.wasm --profile wasm
|
|
5
5
|
*
|
|
6
6
|
* Linear memory past `__heap_base` (the linker's end-of-data symbol) is one
|
|
7
7
|
* arena chunk that grows with `memory.grow`; there is no chunk list because
|
|
@@ -97,3 +97,8 @@ void nish_panic_index(uint64_t idx, uint64_t len) { (void)idx; (void)len; __buil
|
|
|
97
97
|
void nish_panic_slice(int64_t s, int64_t e, int64_t len) { (void)s; (void)e; (void)len; __builtin_trap(); }
|
|
98
98
|
void nish_panic_div(_Bool by_zero) { (void)by_zero; __builtin_trap(); }
|
|
99
99
|
void nish_exit(int32_t code) { (void)code; __builtin_trap(); }
|
|
100
|
+
/* A panic's message (`panic`, `expect`, a failed range entry) has no stderr to
|
|
101
|
+
* go to here; the `nish_exit` that follows it is the trap. An exported function
|
|
102
|
+
* with a ranged parameter checks it on entry (WP31 §9), so this is what lets
|
|
103
|
+
* such a module link; the loader throws a RangeError before the call reaches it. */
|
|
104
|
+
void nish_write(const void *s, int32_t fd, _Bool newline) { (void)s; (void)fd; (void)newline; }
|
package/runtime/runtime.c
CHANGED
|
@@ -4,13 +4,13 @@
|
|
|
4
4
|
* it is also the half with the tighter size budget: the arena, strings, number
|
|
5
5
|
* formatting, the array cold paths, `process.argv`, `Math.random`, and the two
|
|
6
6
|
* panics. Files, directories, subprocesses, the environment and the clock are
|
|
7
|
-
* in
|
|
7
|
+
* in runtime-os.c — every one of those wraps a system call, so that surface
|
|
8
8
|
* grows as the language reaches further into the operating system, and a
|
|
9
9
|
* program that reaches nowhere should not pay for it or be measured with it.
|
|
10
|
-
*
|
|
10
|
+
* runtime-os.c's header comment has the reasoning; tests/run.js gates the two
|
|
11
11
|
* `.text*` budgets separately and docs/wp7-runtime.md records both.
|
|
12
12
|
*
|
|
13
|
-
* Nothing here calls into
|
|
13
|
+
* Nothing here calls into runtime-os.c, which is why an old link line that
|
|
14
14
|
* names runtime.c alone still builds a program that uses none of that surface.
|
|
15
15
|
* The other direction does happen: `nish_readdir` allocates through
|
|
16
16
|
* `nish_alloc_struct` and `nish_str_new`, so those calls no longer inline into
|
|
@@ -50,7 +50,7 @@ int __main_argc_argv(int argc, char **argv) { return nish_c_main(argc, argv); }
|
|
|
50
50
|
definition. Off by default so the ordinary build pays nothing.
|
|
51
51
|
|
|
52
52
|
The same definition is in runtime/nish.h (for a host that includes it) and
|
|
53
|
-
runtime/
|
|
53
|
+
runtime/runtime-wasm.c; they are one contract and move together. */
|
|
54
54
|
#ifdef NISH_THREADS
|
|
55
55
|
#define NISH_TLS _Thread_local
|
|
56
56
|
#else
|
|
@@ -59,7 +59,7 @@ int __main_argc_argv(int argc, char **argv) { return nish_c_main(argc, argv); }
|
|
|
59
59
|
|
|
60
60
|
/* ---- Arena: %struct.nish_arena = type { i8*, i64, i64, i8* }
|
|
61
61
|
|
|
62
|
-
The widths are fixed rather than `size_t` for the reason
|
|
62
|
+
The widths are fixed rather than `size_t` for the reason runtime-wasm.c
|
|
63
63
|
gives for its own copy: every compiled function inlines the bump allocator
|
|
64
64
|
and reads these fields directly, so the IR's `i64` is what `off` and `cap`
|
|
65
65
|
have to be on every target, not just the 64-bit ones. With `size_t` they
|
package/runtime/shim.mjs
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
* The differential runner (tests/differential/run.js) rewrites an Nish
|
|
5
5
|
* program into plain JavaScript and runs it under Node with this module as
|
|
6
6
|
* `__nish`. Every helper here reproduces the *runtime* semantics the compiled
|
|
7
|
-
* binary has (runtime/runtime.c and runtime/
|
|
7
|
+
* binary has (runtime/runtime.c and runtime/runtime-os.c, the system-call half,
|
|
8
8
|
* plus the intrinsics in docs/wp7-runtime.md) where JavaScript's own semantics
|
|
9
9
|
* differ:
|
|
10
10
|
*
|
|
@@ -486,10 +486,10 @@ export function isDirectorySync(path) {
|
|
|
486
486
|
* the directory cannot be read. Node throws where the runtime answers a value,
|
|
487
487
|
* so the `catch` is what makes the two agree, and a directory that exists and
|
|
488
488
|
* is empty answers an empty array on both sides. Node's readdir never yields
|
|
489
|
-
* `.` or `..` — the pair `
|
|
489
|
+
* `.` or `..` — the pair `runtime-os.c` skips explicitly — so there is nothing
|
|
490
490
|
* to filter out here.
|
|
491
491
|
*
|
|
492
|
-
* The sort is the semantic point. `
|
|
492
|
+
* The sort is the semantic point. `runtime-os.c` orders the names with
|
|
493
493
|
* `strcmp`, which compares UTF-8 bytes, and `Array#sort` compares UTF-16 code
|
|
494
494
|
* units. The two agree on ASCII names and part company above the BMP, where a
|
|
495
495
|
* surrogate pair sorts below `U+E000`..`U+FFFF` in UTF-16 and above them in
|
|
@@ -526,7 +526,7 @@ export function realpathSync(path) {
|
|
|
526
526
|
|
|
527
527
|
/**
|
|
528
528
|
* `process.platform` / `process.arch` (WP14 §7a). Node's spellings are the
|
|
529
|
-
* ones `
|
|
529
|
+
* ones `runtime-os.c` answers with, so on any machine this compiler has a
|
|
530
530
|
* triple for the two runtimes give the same string; elsewhere the native build
|
|
531
531
|
* says `unknown` where Node names the platform, which is the one place they
|
|
532
532
|
* part.
|
|
@@ -553,7 +553,7 @@ export function getenv(name) {
|
|
|
553
553
|
* `spawnSync(argv)` and `spawnSyncTo(argv, out, err)` (WP14 D4), which are one
|
|
554
554
|
* run with its streams answered differently: the child's exit status, 128 + n
|
|
555
555
|
* when signal n killed it, -1 for an empty vector or a program that would not
|
|
556
|
-
* start. `
|
|
556
|
+
* start. `runtime-os.c` puts one `static nish_spawn_impl` behind both builtins
|
|
557
557
|
* for the same reason this module puts one function behind both helpers — the
|
|
558
558
|
* argument vector, the wait and the signal convention are written once and
|
|
559
559
|
* cannot drift between the two.
|
|
@@ -606,7 +606,7 @@ export function spawnSyncTo(argv, out, err) {
|
|
|
606
606
|
/**
|
|
607
607
|
* `monotonicNanos()`: `process.hrtime.bigint()`, a monotonic clock in
|
|
608
608
|
* nanoseconds (`CLOCK_MONOTONIC` on every platform that has it, which is the
|
|
609
|
-
* one `
|
|
609
|
+
* one `runtime-os.c` reads). The value is a BigInt because that is how an `i64`
|
|
610
610
|
* is held on this side.
|
|
611
611
|
*
|
|
612
612
|
* No rewrite can make a *reading* agree with a native run: both origins are
|