@amritk/nish 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +553 -0
- package/bin/launcher.js +186 -0
- package/bin/nish +23 -0
- package/bin/packaging.js +220 -0
- package/docs/AI.md +1076 -0
- package/docs/INSTALL.md +468 -0
- package/llms.txt +49 -0
- package/package.json +87 -0
- package/runtime/nish.d.ts +290 -0
- package/runtime/nish.h +340 -0
- package/runtime/nish.mjs +143 -0
- package/runtime/runtime.c +1184 -0
- package/runtime/runtime_os.c +351 -0
- package/runtime/runtime_parallel.c +156 -0
- package/runtime/runtime_wasm.c +99 -0
- package/runtime/shim.mjs +672 -0
- package/scripts/bootstrap.sh +357 -0
- package/scripts/build.sh +279 -0
- package/scripts/changelog-gen.mjs +528 -0
- package/scripts/changelog-section.sh +28 -0
- package/scripts/ci-profile.mjs +187 -0
- package/scripts/codes-registry.js +73 -0
- package/scripts/gen-diagnostic-codes.mjs +199 -0
- package/scripts/gen-pow5-tables.py +45 -0
- package/scripts/nish-compiler.sh +17 -0
- package/scripts/platform-package.mjs +91 -0
- package/scripts/postinstall.mjs +133 -0
- package/scripts/size-report.sh +72 -0
- package/scripts/smoke.sh +94 -0
- package/scripts/verify-binaries.sh +213 -0
- package/std/README.md +185 -0
- package/std/json.ts +402 -0
- package/std/pair.ts +28 -0
- package/std/testing.ts +347 -0
- package/std/text.ts +193 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Nish contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,553 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
# Nish
|
|
4
|
+
|
|
5
|
+
**An ahead-of-time compiler for a strictly static subset of TypeScript — LLVM IR in the middle, native binaries at the end. No interpreter, no garbage collector, nothing to ship beside the executable.**
|
|
6
|
+
|
|
7
|
+

|
|
8
|
+

|
|
9
|
+

|
|
10
|
+

|
|
11
|
+

|
|
12
|
+

|
|
13
|
+

|
|
14
|
+

|
|
15
|
+

|
|
16
|
+
|
|
17
|
+
</div>
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
Nish parses source with its own lexer and parser, rejects everything dynamic
|
|
22
|
+
(`any`, prototypes, `eval`, exceptions, a garbage collector), and emits
|
|
23
|
+
textual LLVM IR (`.ll`). LLVM's own toolchain (`clang` / `llc`) then optimises
|
|
24
|
+
and produces native binaries for x86_64, ARM64, or WebAssembly. The compiler
|
|
25
|
+
is itself written in Nish, in `self/`, and ships as a native binary with no
|
|
26
|
+
runtime dependency.
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
TypeScript source ──▶ AST ──▶ validator + checker ──▶ LLVM IR (.ll) ──▶ clang/llc ──▶ native binary
|
|
30
|
+
(self/parser.ts) (self/checker.ts) (self/emit*.ts) + runtime/runtime.c + runtime_os.c
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
If it compiles, every value has one fixed, known memory layout; binaries are
|
|
34
|
+
a few kilobytes; there is no interpreter and no GC anywhere in the pipeline.
|
|
35
|
+
|
|
36
|
+
> [!WARNING]
|
|
37
|
+
> **Nish is pre-alpha.** The language, the CLI flags and the IR the compiler
|
|
38
|
+
> emits all change without notice until 1.0. Every commit compiles,
|
|
39
|
+
> tests and bootstraps itself — see [Project status](#project-status) for what
|
|
40
|
+
> is done and what is next — but nothing here is frozen yet, so pin an exact
|
|
41
|
+
> version rather than a range, expect to fix your source when you move to a
|
|
42
|
+
> newer one, and read [CHANGELOG.md](CHANGELOG.md) before you upgrade.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Quickstart
|
|
47
|
+
|
|
48
|
+
Requirements: Node.js 22.18+ and, to produce binaries, clang (LLVM 18) + lld;
|
|
49
|
+
per-OS install commands are in [docs/INSTALL.md](docs/INSTALL.md).
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
# Once this is published, the whole install is either of:
|
|
53
|
+
# npm install -g @amritk/nish
|
|
54
|
+
# curl -fsSL https://raw.githubusercontent.com/amritk/nish/main/install.sh | sh
|
|
55
|
+
# `nish` on the registry is an unrelated package from 2014, so this one is
|
|
56
|
+
# scoped; the command it installs is still `nish`. Nothing is published yet,
|
|
57
|
+
# so take it from a release -- the npm tarball, plus the prebuilt native
|
|
58
|
+
# compiler for your machine, which is what the scoped install would fetch.
|
|
59
|
+
base=https://github.com/amritk/nish/releases/download/v0.4.0
|
|
60
|
+
curl -LO $base/amritk-nish-0.4.0.tgz
|
|
61
|
+
curl -LO $base/amritk-nish-x86_64-linux-0.4.0.tgz # or the row for your machine
|
|
62
|
+
npm install -g ./amritk-nish-0.4.0.tgz ./amritk-nish-x86_64-linux-0.4.0.tgz
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
> [!NOTE]
|
|
66
|
+
> **Install the pair.** The main package is a launcher and carries no compiler:
|
|
67
|
+
> installed on its own it has nothing to hand over to and says so, exiting 3.
|
|
68
|
+
> The same is true on a platform this project publishes no binary for — musl,
|
|
69
|
+
> FreeBSD, 32-bit anything — which since 0.6.0 gets a diagnostic naming the four
|
|
70
|
+
> that do rather than the TypeScript compiler under Node that used to ship
|
|
71
|
+
> beside it. [docs/INSTALL.md](docs/INSTALL.md) §2 has the table of what is
|
|
72
|
+
> built for what, and what is left for a platform outside it.
|
|
73
|
+
|
|
74
|
+
> [!TIP]
|
|
75
|
+
> Building from source works just as well: `git clone`, `npm ci`,
|
|
76
|
+
> `bash scripts/fetch-seed.sh`, `npm run build`, then `build/nish ...` wherever this README says `nish`.
|
|
77
|
+
> The build is a bootstrap: the previous release's `nish` (the seed, fetched
|
|
78
|
+
> by `scripts/fetch-seed.sh`, or named with `NISH_BOOTSTRAP=<path>`) compiles
|
|
79
|
+
> `self/`, and that compiler compiles `self/` again into `build/nish`.
|
|
80
|
+
|
|
81
|
+
`hello.ts`:
|
|
82
|
+
|
|
83
|
+
```ts
|
|
84
|
+
export const main = (): number => {
|
|
85
|
+
console.log("hello from Nish");
|
|
86
|
+
return 0; // the process exit code
|
|
87
|
+
};
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
nish hello.ts --link hello # writes hello.ll, then builds hello with clang -O3 -flto
|
|
92
|
+
./hello # hello from Nish
|
|
93
|
+
nish hello.ts -o hello.ll # IR only
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The IR is readable as is. `examples/add.ts` compiles to:
|
|
97
|
+
|
|
98
|
+
```llvm
|
|
99
|
+
define noundef i32 @add(i32 noundef %a, i32 noundef %b) #0 {
|
|
100
|
+
entry:
|
|
101
|
+
%0 = add nsw i32 %a, %b
|
|
102
|
+
ret i32 %0
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
attributes #0 = { nounwind willreturn readnone }
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
`--plain` drops the attributes and alignment hints and leaves
|
|
109
|
+
`define i32 @add(i32 %a, i32 %b)` with the same body. Every construct's IR
|
|
110
|
+
is in [docs/IR_COOKBOOK.md](docs/IR_COOKBOOK.md).
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## The language
|
|
115
|
+
|
|
116
|
+
The full reference is [docs/LANGUAGE.md](docs/LANGUAGE.md); every rule
|
|
117
|
+
there cites the test case that proves it. If an **AI** is writing the program —
|
|
118
|
+
or you want the whole language in one pass rather than as a reference to browse
|
|
119
|
+
— read [docs/AI.md](docs/AI.md) instead: the same rules, ordered by which
|
|
120
|
+
TypeScript reflex they reject, with every example compiled by `npm test`. Both
|
|
121
|
+
ship in the npm package, and [llms.txt](llms.txt) indexes them.
|
|
122
|
+
|
|
123
|
+
| Feature | Summary | Reference |
|
|
124
|
+
|:---|:---|:---|
|
|
125
|
+
| Types | `number` (`i32` by default, `double` with `--number-mode f64`), `i32`, `i64`, `f64`, `boolean`, `string`, `T[]`, classes, interfaces, `T \| null`, `void`; 1:1 LLVM mapping, no implicit conversions | [Types](docs/LANGUAGE.md#types) |
|
|
126
|
+
| Functions and modules | annotated signatures, calls in any order, `export`/named relative `import`, whole-program attribute facts, `export const main` as the entry, `internal` linkage for everything not exported | [Declarations](docs/LANGUAGE.md#declarations) |
|
|
127
|
+
| Control flow | `if`/`else`, `while`, `do`, `for`, `for...of`, `break`/`continue`, boolean-only conditions, definite return, unreachable-code errors | [Statements](docs/LANGUAGE.md#statements) |
|
|
128
|
+
| Expressions | `+ - * / %` (integer `/` and `%` checked: zero divisor or `MIN / -1` panics), numeric-only ordering, `=== !==` (strings by content), `&& \|\|`, `?:`, `op=`, `++`/`--`, template literals, contextual numeric literals | [Expressions](docs/LANGUAGE.md#expressions) |
|
|
129
|
+
| Strings | immutable UTF-8 (`.length` is the byte length), literals as constant data, `+`, `===`, templates, `console.log` | [Builtins](docs/LANGUAGE.md#builtins), [Semantics](docs/LANGUAGE.md#semantics-decisions) |
|
|
130
|
+
| Arrays | `T[]` with one element type, literals, `new Array<T>(n)` zero-filled, bounds-checked `a[i]` (panic, or `--unchecked-indexing`), `.length`, `push`, `for...of` | [Arrays](docs/LANGUAGE.md#array-literals) |
|
|
131
|
+
| Classes and interfaces | LLVM structs with clang's layout, constructors, methods, `readonly`, definite assignment, object literals, `implements` by identical layout | [Classes](docs/LANGUAGE.md#classes) |
|
|
132
|
+
| Generics | generic functions, classes, interfaces and methods by monomorphisation: one specialised `define` per instantiation, type arguments inferred at a call and written after `new` or in an annotation, `<T extends Shape>` constraints, exported instantiations callable from C and JavaScript as `nish_gen_identity_i32` | [Generic functions](docs/LANGUAGE.md#generic-functions) |
|
|
133
|
+
| Memory | no GC: objects that provably do not escape their function are stack `alloca`s, functions whose temporaries die with them get an automatic arena scope, `Arena.reset/mark/release/used` for explicit control | [`Arena`](docs/LANGUAGE.md#arena), [Memory](docs/LANGUAGE.md#memory-model) |
|
|
134
|
+
| `T \| null` | for class, interface, array and string types; `=== null`, and narrowing to `T` by `if`, early return, `while`, `&&`, `?:`, enforced by the checker | [Nullable types](docs/LANGUAGE.md#nullable-types) |
|
|
135
|
+
| Errors | Rust-style `Result<T, E>` with `Ok`/`Err`, `isOk()`/`isErr()`, `orReturn()` (the `?`), `unwrapOr`, `expect`; the checker refuses to let a failure be dropped or the success payload be read before the error is handled. No `throw`, no unwinding | [Result and error handling](docs/LANGUAGE.md#result-and-error-handling) |
|
|
136
|
+
| Builtins | `console.log`, `Math.*` as LLVM intrinsics (ECMAScript `pow` corner cases included), `Math.random`, `toI32`/`toI64`/`toF64`, `process.exit`, `readFileSync`/`writeFileSync`/`appendFileSync`; the runtime-backed ones are also importable from `nish:fs` / `nish:process` / `nish:io`, which is the same builtin under a name nothing can shadow | [Builtins](docs/LANGUAGE.md#builtins), [Builtin modules](docs/LANGUAGE.md#builtin-modules-nish) |
|
|
137
|
+
| Rejected | `any`, `unknown`, `var`, `==`, `?.`, `??`, generic type aliases, `async`, `try`, `throw`, `typeof`, `delete`, prototypes, `Object.assign`, string-keyed access, ... with exact messages | [Forbidden constructs](docs/LANGUAGE.md#forbidden-constructs-phase-0-validator) |
|
|
138
|
+
Semantics that differ from JavaScript on purpose: signed integer overflow is
|
|
139
|
+
undefined behaviour (`--wrapping` restores two's-complement wrapping; the
|
|
140
|
+
unsigned widths wrap either way), integer division by zero panics instead of
|
|
141
|
+
yielding `0`, `.length` counts bytes, there is no `throw` and no unwinding,
|
|
142
|
+
`toI32` saturates, `Math.min`/`max` take two arguments. The reasons are in the
|
|
143
|
+
[FAQ](docs/FAQ.md); [docs/wp13-differential.md](docs/wp13-differential.md)
|
|
144
|
+
lists everything the differential test suite found that still differs from
|
|
145
|
+
Node.
|
|
146
|
+
|
|
147
|
+
### The standard library
|
|
148
|
+
|
|
149
|
+
[`std/`](std/README.md) is Nish written in Nish, for Nish programs to import:
|
|
150
|
+
[`std/testing`](std/testing.ts), a test runner, so a compiled program can check
|
|
151
|
+
itself and answer an exit code with no Node in the picture,
|
|
152
|
+
[`std/text`](std/text.ts), the string operations a program would otherwise write
|
|
153
|
+
inline — the language has no `split`, `trim` or regular expression, because each
|
|
154
|
+
of those allocates and some need a character table the runtime has no room for —
|
|
155
|
+
and [`std/json`](std/json.ts), the value of one field of one flat JSON object,
|
|
156
|
+
which is the shape the compiler's own `--json` diagnostics have.
|
|
157
|
+
|
|
158
|
+
```ts
|
|
159
|
+
import { Suite } from "nish/testing";
|
|
160
|
+
|
|
161
|
+
export const main = (): number => {
|
|
162
|
+
const t = new Suite("stats");
|
|
163
|
+
t.eqI32("sumOf", sumOf([3, 9, 4, 9]), 25);
|
|
164
|
+
return t.done(); // prints the report; 0 when nothing failed
|
|
165
|
+
};
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
A library module is source, not a built artifact, so it compiles with the
|
|
169
|
+
program that imports it and the whole-program pass sees straight through it
|
|
170
|
+
([docs/wp21-packages.md](docs/wp21-packages.md)): `nish/<module>` resolves to
|
|
171
|
+
`std/<module>.ts` beside the running compiler, and what you import but never
|
|
172
|
+
call is dropped at the link. What the library does *not* have is callbacks,
|
|
173
|
+
which is what makes a suite a value with methods rather than a
|
|
174
|
+
`test("name", () => ...)`: a function is never a value here.
|
|
175
|
+
|
|
176
|
+
The suite's own golden cases are run by [`tests/nish/run.ts`](tests/nish/run.ts),
|
|
177
|
+
which is this repository's test harness written in the language it tests:
|
|
178
|
+
`readdirSync` finds the cases, `spawnSyncTo` captures each compile and each run,
|
|
179
|
+
and the IR is diffed against the golden line by line. `npm run test:nish` runs it
|
|
180
|
+
over the whole corpus.
|
|
181
|
+
|
|
182
|
+
[`tests/nish/cli.ts`](tests/nish/cli.ts) is the other half of that idea, pointed
|
|
183
|
+
at the compiler's own promises rather than at its output: `--help` on stdout with
|
|
184
|
+
exit 0 against the same text on stderr with exit 2, one flat `--json` object per
|
|
185
|
+
diagnostic with a stable code, and each documented exit-code band. It reads those
|
|
186
|
+
objects with `std/json` and takes the version it expects out of `package.json`, so
|
|
187
|
+
a release cannot leave the expectation behind — and it passes against the
|
|
188
|
+
self-hosted compiler as well as against the one written in TypeScript
|
|
189
|
+
(`npm run test:cli`).
|
|
190
|
+
|
|
191
|
+
---
|
|
192
|
+
|
|
193
|
+
## Memory safety
|
|
194
|
+
|
|
195
|
+
There is no garbage collector and no `free`, so the bugs that need one cannot
|
|
196
|
+
be written. What is left — an index out of range, a null dereference — is
|
|
197
|
+
checked, and every way to give a check up is a flag you pass or a builtin you
|
|
198
|
+
call on purpose.
|
|
199
|
+
|
|
200
|
+
| Bug class | What Nish does | Reference |
|
|
201
|
+
|:---|:---|:---|
|
|
202
|
+
| Use-after-free, double free | Not expressible: nothing is freed individually. Four compile-time mechanisms decide where a value lives — a stack `alloca` when escape analysis proves it dies with the frame, an automatic arena scope when a function's temporaries do, a `nish_arena_keep` reclaim at the call site for a returned string, the bump arena otherwise — and the arena goes back when `main` returns | [Memory model](docs/LANGUAGE.md#memory-model), [wp6-memory.md](docs/wp6-memory.md) |
|
|
203
|
+
| Out-of-bounds read or write | Every `a[i]`, `a[i] op= v`, `s.charCodeAt(i)`, `s.slice(a, b)` and `a.pop()` is bounds-checked, with an unsigned compare, so a negative index fails too; the failure prints `index out of range: <i> >= <len>` (or, for `slice`, `slice out of range: [<a>, <b>) of length <len>`) and exits 1. Where a flow-sensitive proof shows the index is already in range — a loop condition, a length guard, a hoisted `const n = a.length`, an unsigned index — **no check is emitted at all**, and the checks that survive inside a loop say so as a `performance` warning naming the guard that would remove them | [Element access](docs/LANGUAGE.md#element-access), `tests/cases/arr_bounds_panic`, `arr_bounds_proven` |
|
|
204
|
+
| Null dereference | `T \| null` is a separate type, for pointers only; member access needs a narrowing the checker accepts and `?.` is forbidden — which is what lets the emitter put `nonnull dereferenceable` on every pointer that is not one | [Nullable types](docs/LANGUAGE.md#nullable-types) |
|
|
205
|
+
| Uninitialised memory | `new Array<T>(n)` zero-fills and rejects pointer element types, because a zeroed pointer would be a null nobody declared; class fields are definitely assigned | [Classes](docs/LANGUAGE.md#classes), `tests/cases/arr_new_zeroed` |
|
|
206
|
+
| Unwinding past a release | There is none. Every function is `nounwind`; a failure a caller should handle is a `Result<T, E>` and one it should not is `panic(message)` — stderr, exit 1 | [Result](docs/LANGUAGE.md#result-and-error-handling) |
|
|
207
|
+
|
|
208
|
+
### What happens when the proof fails
|
|
209
|
+
|
|
210
|
+
This is where designs actually differ. Asked to place a value it cannot show
|
|
211
|
+
dies with its frame, Rust refuses to compile it, Go falls back to the garbage
|
|
212
|
+
collector, and Zig hands the question back to you and an allocator. Nish
|
|
213
|
+
leaves the value in the arena, where it stays until `main` returns.
|
|
214
|
+
|
|
215
|
+
So [`self/escape.ts`](self/escape.ts) and the whole-program fact
|
|
216
|
+
fixpoint are optimisations and nothing else: a refusal costs memory and never
|
|
217
|
+
correctness, and no program is rejected for a lifetime reason. The compiler
|
|
218
|
+
says so out loud where the cost is real: assigning an allocation to a local
|
|
219
|
+
that already holds one drops the old value where nothing can free it *and*
|
|
220
|
+
costs the whole function its arena scope, and that is a `performance`
|
|
221
|
+
diagnostic naming both rewrites ([Diagnostics](docs/LANGUAGE.md#diagnostics-and-debugging-flags), on by
|
|
222
|
+
default, never fatal). That is the
|
|
223
|
+
trade the whole memory design rests on — the arena discipline a compiler pass
|
|
224
|
+
or a frame loop would otherwise be written around by hand, moved into the
|
|
225
|
+
compiler. `tests/cases/mem_*` pin the placements, and `Arena.used()` either
|
|
226
|
+
side of a 100000-iteration loop is how the suite checks that a scope really
|
|
227
|
+
does recycle.
|
|
228
|
+
|
|
229
|
+
### Which language is this like
|
|
230
|
+
|
|
231
|
+
Not any one of them; it is more useful to say which piece came from where.
|
|
232
|
+
|
|
233
|
+
| Concern | Closest to | Not |
|
|
234
|
+
|:---|:---|:---|
|
|
235
|
+
| Lifetimes | Go's escape analysis, over a Zig-style arena discipline the compiler writes for you | Rust: no ownership, no borrow checker, no lifetime annotations |
|
|
236
|
+
| Bounds and panics | Rust with `panic=abort` | C |
|
|
237
|
+
| Null | Kotlin and C# nullable reference types: flow narrowing, not a wrapper type | Rust's `Option<T>` |
|
|
238
|
+
| Errors | Rust: `Result<T, E>`, and `orReturn()` is `?` | exceptions, or Go's second return value |
|
|
239
|
+
| Signed overflow | C: undefined by default, `--wrapping` to opt out | Rust, where it is defined in both profiles |
|
|
240
|
+
| Syntax | TypeScript | |
|
|
241
|
+
|
|
242
|
+
The older relative is region inference as in Cyclone and MLKit: regions the
|
|
243
|
+
compiler infers, bracketed by a mark and a release, which is exactly what the
|
|
244
|
+
arena scopes and the call-site reclaim are. This version is deliberately
|
|
245
|
+
weaker: one global arena, per-function granularity, no region polymorphism.
|
|
246
|
+
|
|
247
|
+
### Where it is not safe
|
|
248
|
+
|
|
249
|
+
> [!CAUTION]
|
|
250
|
+
> Four holes, every one of them asked for by name.
|
|
251
|
+
|
|
252
|
+
- **`Arena.reset()` / `Arena.release(m)`** release or recycle in O(1), and
|
|
253
|
+
doing either while anything allocated after the mark is still referenced is
|
|
254
|
+
undefined behaviour. The compiler protects its own marks — a function that
|
|
255
|
+
touches either, directly or through a callee, never gets an automatic
|
|
256
|
+
scope — and not yours ([`Arena`](docs/LANGUAGE.md#arena)).
|
|
257
|
+
- **`--unchecked-indexing`** drops the bounds checks, after which an
|
|
258
|
+
out-of-range index is undefined behaviour. It is there for benchmarks
|
|
259
|
+
(`tests/cases/arr_unchecked`), and it is a different thing from the proof
|
|
260
|
+
above: the proof removes a check the compiler showed was never going to
|
|
261
|
+
fire, and changes nothing about what the program means.
|
|
262
|
+
- **Signed integer overflow is undefined** by default, so LLVM may widen
|
|
263
|
+
induction variables and strength-reduce loops; `--wrapping` restores
|
|
264
|
+
two's-complement wrapping for a hash or an LCG that overflows on purpose
|
|
265
|
+
([Semantics](docs/LANGUAGE.md#semantics-decisions)).
|
|
266
|
+
- **Interop** hands a pointer to a C, wasm or N-API host, and what happens to
|
|
267
|
+
it there is the host's business
|
|
268
|
+
([wp8-interop.md](docs/wp8-interop.md)).
|
|
269
|
+
|
|
270
|
+
Memory-safe like Go, allocated like Zig, errors like Rust, nulls like Kotlin,
|
|
271
|
+
overflow like C — with two C-shaped holes you have to ask for by name. What it
|
|
272
|
+
buys is the output: no GC, no runtime, and the sizes under
|
|
273
|
+
[Performance and binary size](#performance-and-binary-size).
|
|
274
|
+
|
|
275
|
+
---
|
|
276
|
+
|
|
277
|
+
## Command line
|
|
278
|
+
|
|
279
|
+
```
|
|
280
|
+
nish <entry.ts> [more.ts ...] [options]
|
|
281
|
+
nish --version | --help
|
|
282
|
+
-o, --output <file.ll> output path for a single module (default: <input>.ll)
|
|
283
|
+
-o, --output <dir>/ output directory: one <dir>/<module>.ll per module
|
|
284
|
+
--link <exe> build a native binary from every module + runtime/runtime.c
|
|
285
|
+
(entry module must declare `export const main`)
|
|
286
|
+
--profile speed|size|debug build profile for --link (default: speed)
|
|
287
|
+
--no-strict-exports every function is an external symbol (default: non-exported
|
|
288
|
+
functions get `internal` linkage)
|
|
289
|
+
--number-mode i32|f64 lowering of `number` (default: i32)
|
|
290
|
+
--plain no performance attributes or alignment hints
|
|
291
|
+
--runtime-decls always emit the runtime ABI prelude (arena + strings)
|
|
292
|
+
--emit-header <file.h> also write a C header for the callable functions
|
|
293
|
+
--emit-dts <file.d.ts> also write TypeScript declarations for the wasm exports, plus
|
|
294
|
+
<file>.mjs, a loader that marshals typed arrays
|
|
295
|
+
--emit-napi <shim.c> also write an N-API shim (build with --profile napi)
|
|
296
|
+
--unchecked-indexing drop array bounds checks (unsafe; for benchmarks)
|
|
297
|
+
--target <triple>|host emit `target datalayout`/`target triple` for that machine
|
|
298
|
+
(x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin,
|
|
299
|
+
aarch64-apple-darwin, wasm32-unknown-unknown, wasm32-wasi); default: target-neutral IR
|
|
300
|
+
--wrapping signed integer add/sub/mul wrap two's-complement (default: they
|
|
301
|
+
carry `nsw`, so signed overflow is undefined, like C)
|
|
302
|
+
--no-stack-alloc keep every allocation in the arena (disables escape-analysed allocas)
|
|
303
|
+
--threads give every thread its own arena and random seed; the runtime is
|
|
304
|
+
built to match by --link (no language surface: nothing in the
|
|
305
|
+
language spawns a thread yet)
|
|
306
|
+
--no-warn-performance do not report the `performance` diagnostics (they are on by default,
|
|
307
|
+
print on stderr, and never change the exit code)
|
|
308
|
+
-g emit DWARF debug info (!dbg locations, variables); kept by --link
|
|
309
|
+
--json print diagnostics as one JSON object per line on stdout (no excerpt)
|
|
310
|
+
--emit-ast print the syntax tree of every module to stdout instead of IR
|
|
311
|
+
--emit-checked print the checker's tables (signatures, locals, structs, facts) instead of IR
|
|
312
|
+
-v, --version print the nish version and exit
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
Exit codes: `0` success, `1` compile error (`file:line:col: error: ...` plus
|
|
316
|
+
a caret excerpt), `2` usage error, `3` toolchain error, `70` internal
|
|
317
|
+
compiler error (please report it; `NISH_DEBUG=1` adds the stack trace).
|
|
318
|
+
A compile that fails reports every error it found (statement by statement,
|
|
319
|
+
declaration by declaration), in source order, up to 20 before `...and N more
|
|
320
|
+
errors`; `--json` gives editors and tools the same list as
|
|
321
|
+
`{"file","line","column","endLine","endColumn","severity","code","message"}`
|
|
322
|
+
objects, one per line, where `code` is a stable identifier for the rule
|
|
323
|
+
(`NL1013`, `NL2231`) and is what to match on rather than the prose. Failures
|
|
324
|
+
with no source position — an unusable C toolchain, an internal error — are JSON
|
|
325
|
+
objects too, so `--json` never leaves a caller with an empty stdout.
|
|
326
|
+
`--help` prints on stdout and exits `0`; only a usage *error* goes to stderr
|
|
327
|
+
with `2`. `-g` adds a DWARF line table and variables to the IR so
|
|
328
|
+
`gdb`/`lldb` step through the `.ts` source of a `--link`ed binary
|
|
329
|
+
([docs/wp10-ci.md](docs/wp10-ci.md)).
|
|
330
|
+
Multi-file programs: `nish examples/multi/main.ts --link build/multi && ./build/multi; echo $?`
|
|
331
|
+
prints `49`.
|
|
332
|
+
|
|
333
|
+
Without `--link`, build the IR yourself: `clang add.ll examples/main.c runtime/runtime.c runtime/runtime_os.c -o app`
|
|
334
|
+
(the `overriding the module target triple` warning is harmless: the IR is
|
|
335
|
+
target-neutral unless you pass `--target`; `-Wno-override-module` silences
|
|
336
|
+
it), or step by step with `llvm-as`, `llc -O2 -filetype=obj`, and
|
|
337
|
+
`opt -S -O2` to watch `mem2reg` turn the `alloca` locals into registers.
|
|
338
|
+
`opt` and `llc` only vectorise when the module carries a data layout, so
|
|
339
|
+
pass `--target host` (or a triple) when you inspect optimised IR by hand;
|
|
340
|
+
`--link` never needs it because clang supplies the layout. Cross-compile
|
|
341
|
+
with `--target aarch64-unknown-linux-gnu` / `wasm32-wasi` plus
|
|
342
|
+
`clang --target=...`, or with `llc -mtriple=...`.
|
|
343
|
+
|
|
344
|
+
---
|
|
345
|
+
|
|
346
|
+
## Performance and binary size
|
|
347
|
+
|
|
348
|
+
Rust-class output is the goal: no GC, no embedded engine, aliasing and
|
|
349
|
+
purity facts handed to LLVM up front, and a link step that strips everything
|
|
350
|
+
unused. `examples/add.ts` + `examples/main.c` + the two runtime translation units
|
|
351
|
+
(`runtime/runtime.c` and `runtime/runtime_os.c`), x86_64 Linux, glibc
|
|
352
|
+
dynamically linked (`npm run size-report`):
|
|
353
|
+
|
|
354
|
+
| Profile | Bytes | What it does |
|
|
355
|
+
|:---|---:|:---|
|
|
356
|
+
| `debug` | 15,072 | `clang` defaults: no optimisation, symbols kept. |
|
|
357
|
+
| `speed` | 4,528 | `-O3 -flto`, section GC, unwind tables off, stripped. Rust `--release`. |
|
|
358
|
+
| `size` | 4,512 | `-Oz -flto`, plus hidden visibility and no stack protector. Rust `opt-level="z"`. |
|
|
359
|
+
| `wasm` | 279 | Freestanding `wasm32` module, every function exported, stripped. |
|
|
360
|
+
| `wasi` | 42,228 (`argv.ts`) | `wasm32-wasi` command module: the runtime linked against wasi-libc, `_start` runs `main`; needs a WASI sysroot ([INSTALL.md](docs/INSTALL.md#wasi-optional-for---profile-wasi)). |
|
|
361
|
+
|
|
362
|
+
`hello.ts` with `--link` is 4,696 bytes. The whole runtime is under 5 KB of
|
|
363
|
+
machine code, in two translation units with a measured ceiling each so that the
|
|
364
|
+
core does not grow every time the language reaches further into the operating
|
|
365
|
+
system: `runtime.c` is the 3,480 bytes every program touches (one chunked bump
|
|
366
|
+
arena with O(1) reset and mark/release, strings, JavaScript-exact number
|
|
367
|
+
formatting, string parsing, `Math.random`, `process.argv`, array growth, the
|
|
368
|
+
panic paths), and `runtime_os.c` the 1,190 bytes that wrap a system call (exit,
|
|
369
|
+
files, directories, subprocesses, the environment, the clock) — see
|
|
370
|
+
[docs/wp7-runtime.md](docs/wp7-runtime.md#runtime-additions-and-budget) for both
|
|
371
|
+
budgets and the reasoning. `Math.*` calls are LLVM intrinsics, so pure functions
|
|
372
|
+
stay `readnone`.
|
|
373
|
+
|
|
374
|
+
Memory is the part that usually costs a compiled-JavaScript design its
|
|
375
|
+
speed, so it is done statically ([docs/wp6-memory.md](docs/wp6-memory.md)):
|
|
376
|
+
a `new`, object literal or array literal that provably never outlives its
|
|
377
|
+
function is an `alloca` (LLVM's SROA then turns its fields into registers);
|
|
378
|
+
what does reach the arena is bumped inline (a load, an add, a compare and a
|
|
379
|
+
store); a function whose arena temporaries all die with it brackets its body
|
|
380
|
+
with `nish_arena_mark` / `nish_arena_release`, so hot loops keep the arena
|
|
381
|
+
flat. Every LLVM attribute the compiler emits (`nounwind`, `willreturn`,
|
|
382
|
+
`readnone`/`readonly`, `noundef`, `zeroext`, `nonnull`, `noalias`,
|
|
383
|
+
`nocapture`, `dereferenceable`) is a proved guarantee, never a hint; the
|
|
384
|
+
rules are in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md#attribute-soundness-rules).
|
|
385
|
+
|
|
386
|
+
The benchmark suite (`bench/`: fib, n-body, spectral norm, sieve, string
|
|
387
|
+
building, a `Vec3` method loop, each in Nish, C and Rust with identical
|
|
388
|
+
algorithms and a shared checksum) is run by `node bench/run.mjs`, which
|
|
389
|
+
writes [docs/BENCHMARKS.md](docs/BENCHMARKS.md): wall time, binary size and
|
|
390
|
+
peak memory per column, plus the exact build commands. The analysis of every
|
|
391
|
+
gap, and what `--nsw` and PGO (`scripts/build.sh --pgo-generate` /
|
|
392
|
+
`--pgo-use`) buy, is in [docs/wp9-optimisation.md](docs/wp9-optimisation.md);
|
|
393
|
+
the rules of the game are in [bench/README.md](bench/README.md).
|
|
394
|
+
|
|
395
|
+
---
|
|
396
|
+
|
|
397
|
+
## Interop: export, do not embed
|
|
398
|
+
|
|
399
|
+
Nish never embeds a JavaScript engine (see the [FAQ](docs/FAQ.md#why-not-embed-a-javascript-engine-for-npm-packages)).
|
|
400
|
+
The supported direction is Node importing Nish:
|
|
401
|
+
|
|
402
|
+
```
|
|
403
|
+
[ Node / Bun process ] imports [ Nish .wasm or .node addon ]
|
|
404
|
+
I/O, HTTP, npm packages math, parsing, data transforms, hot loops
|
|
405
|
+
cross the boundary once per batch, not once per element
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
- `scripts/build.sh --profile wasm` produces a module `WebAssembly.instantiate`
|
|
409
|
+
loads directly (`examples/node-host.mjs`); exports use the plain C ABI.
|
|
410
|
+
Add `runtime/runtime_wasm.c` (arena + arrays, no libc) when a function
|
|
411
|
+
takes or returns an array.
|
|
412
|
+
- `--emit-napi` + `scripts/build.sh --profile napi` build a `.node` addon
|
|
413
|
+
with argument type checks (`examples/node-addon.mjs`).
|
|
414
|
+
- Buffers cross as typed arrays: an Nish `Int32Array` / `Float64Array` /
|
|
415
|
+
`BigInt64Array` parameter (the spellings of `i32[]` / `f64[]` / `i64[]`,
|
|
416
|
+
one layout) is a JS typed array on both paths. The addon borrows it
|
|
417
|
+
(zero-copy; writes are visible in JS), the wasm loader that `--emit-dts`
|
|
418
|
+
writes next to the `.d.ts` copies it into the module's memory and results
|
|
419
|
+
back out; strings cross the addon as copies (`examples/arrays.ts`).
|
|
420
|
+
- `--emit-header` writes C prototypes (`int32_t add(int32_t a, int32_t b);`,
|
|
421
|
+
`double sumF64(const nish_array *xs);`) next to `runtime/nish.h`, the
|
|
422
|
+
public runtime ABI (arena, strings, arrays, `nish_reset_arena`,
|
|
423
|
+
`nish_arena_mark` / `nish_arena_release` for a host that manages batches).
|
|
424
|
+
- An N-API call costs about 30 ns and a wasm call about 2 ns before any work
|
|
425
|
+
is done; one call with a 1M-element `Float64Array` runs at 0.5 ns/element
|
|
426
|
+
(`node bench/ffi.mjs`), so pass whole buffers, not elements.
|
|
427
|
+
|
|
428
|
+
Details: [docs/wp8-interop.md](docs/wp8-interop.md).
|
|
429
|
+
|
|
430
|
+
---
|
|
431
|
+
|
|
432
|
+
## The compiler in a browser
|
|
433
|
+
|
|
434
|
+
`self/` is an Nish program, so the compiler compiles itself to
|
|
435
|
+
WebAssembly like any other one:
|
|
436
|
+
|
|
437
|
+
```bash
|
|
438
|
+
nish self/compile.ts --link web/nish.wasm --profile wasi
|
|
439
|
+
node web/compile.mjs web/nish.wasm examples/add.ts # the IR, from a Web Worker
|
|
440
|
+
```
|
|
441
|
+
|
|
442
|
+
That module is about 480 KB (140 KB gzipped) and lexes, checks and emits IR
|
|
443
|
+
with no server in the loop; `web/index.html` is a playground built on it and
|
|
444
|
+
`web/wasi.mjs` is the in-memory filesystem it runs against. It stops at the
|
|
445
|
+
IR — `clang` and `wasm-ld` are not in a page — so `--link` and `--profile` are
|
|
446
|
+
refused there. [web/README.md](web/README.md) has the rest.
|
|
447
|
+
|
|
448
|
+
---
|
|
449
|
+
|
|
450
|
+
## Self-hosting
|
|
451
|
+
|
|
452
|
+
`self/` is the compiler, and the only one: lexer, parser, checker and emitter,
|
|
453
|
+
written in Nish, with no `typescript` package underneath. It compiles its own
|
|
454
|
+
source to a fixed point:
|
|
455
|
+
|
|
456
|
+
```
|
|
457
|
+
IR(stage1, self/) == IR(stage2, self/) byte for byte, and stage3 == stage2
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
The chain starts from a seed, the previous release's `nish` binary, the way
|
|
461
|
+
Rust and Go build themselves: stage1 is `self/` built by the seed, stage2 is
|
|
462
|
+
`self/` built by stage1 and is what gets installed, and stage3, built by
|
|
463
|
+
stage2, has to reproduce stage2 file for file. `scripts/bootstrap.sh --verify`
|
|
464
|
+
asserts both equalities, and `npm test` re-proves them on every run. Compiling
|
|
465
|
+
the whole compiler costs it **91 ms and 86 MB**; the TypeScript compiler it
|
|
466
|
+
replaced took 786 ms and 178 MB for the same input.
|
|
467
|
+
|
|
468
|
+
```bash
|
|
469
|
+
bash scripts/fetch-seed.sh # the last release, into build/seed/
|
|
470
|
+
npm run build # seed -> stage1 -> stage2 = build/nish
|
|
471
|
+
build/nish hello.ts --link hello # -o, --link, --profile, its own directories
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
Because the seed is the last release, `self/` may only *use* in its own
|
|
475
|
+
source what that release compiles. A new construct is implemented in `self/`
|
|
476
|
+
and becomes usable inside `self/` from the next release on — the rolling
|
|
477
|
+
freeze CI's `bootstrap` job checks. Until R6 there was a second compiler, the
|
|
478
|
+
TypeScript one in `src/`, which seeded every bootstrap and served as the
|
|
479
|
+
oracle each `self/` phase was compared against; it was deleted once the
|
|
480
|
+
self-hosted compiler did everything it did
|
|
481
|
+
([wp19](docs/wp19-stage0-retirement.md)).
|
|
482
|
+
Details, and the subset `self/` is written in, are in
|
|
483
|
+
[docs/wp14-selfhost.md](docs/wp14-selfhost.md).
|
|
484
|
+
|
|
485
|
+
---
|
|
486
|
+
|
|
487
|
+
## Project status
|
|
488
|
+
|
|
489
|
+
Pre-alpha, as above: M4 is the milestone that freezes the language reference
|
|
490
|
+
and tags a release, so until it lands a construct's spelling, a flag's name
|
|
491
|
+
and the IR any of them lowers to are all still free to change.
|
|
492
|
+
|
|
493
|
+
| Milestone | Contents | State |
|
|
494
|
+
|:---|:---|:---|
|
|
495
|
+
| M1 "Programs" | pipeline prep, validator, control flow, strings, modules, CI | done |
|
|
496
|
+
| M2 "Data" | classes and interfaces, arrays, runtime and intrinsics | done |
|
|
497
|
+
| M3 "Rust parity" | interop, memory strategy (stack allocation, arena scopes, `T \| null`), benchmarks with `--target`/`--nsw`/PGO, differential testing against Node | done |
|
|
498
|
+
| M4 "1.0" | frozen language reference, tagged release | next |
|
|
499
|
+
| M5 "Self-hosting" | `self/`: the compiler, written in Nish, compiling itself | done |
|
|
500
|
+
|
|
501
|
+
Not in the language yet, in the order they are likely to land: optional
|
|
502
|
+
reference counting for objects that must outlive an arena reset, and dynamic
|
|
503
|
+
dispatch — which would be a trait object over an interface, since inheritance
|
|
504
|
+
was removed ([docs/wp25-inheritance.md](docs/wp25-inheritance.md)) and every
|
|
505
|
+
method call names one symbol today.
|
|
506
|
+
Generic functions, classes, interfaces and methods compile — each
|
|
507
|
+
instantiation becomes its own specialised function or struct
|
|
508
|
+
([docs/wp18-generics.md](docs/wp18-generics.md)) — while generic type aliases,
|
|
509
|
+
closures, `try`/`catch` and labelled `break`/`continue` are refusals
|
|
510
|
+
rather than gaps, each with the message and the idiom to use instead
|
|
511
|
+
([docs/LANGUAGE.md](docs/LANGUAGE.md#forbidden-constructs-phase-0-validator)).
|
|
512
|
+
Release engineering (`--version`, exit codes, npm packaging, tag-driven
|
|
513
|
+
releases) landed with WP12; see [CHANGELOG.md](CHANGELOG.md) and
|
|
514
|
+
[docs/wp12-release.md](docs/wp12-release.md). The plan itself is
|
|
515
|
+
[docs/MASTER_PLAN.md](docs/MASTER_PLAN.md).
|
|
516
|
+
|
|
517
|
+
---
|
|
518
|
+
|
|
519
|
+
## Contributing
|
|
520
|
+
|
|
521
|
+
```bash
|
|
522
|
+
npm ci
|
|
523
|
+
bash scripts/fetch-seed.sh # the last release into build/seed/, once
|
|
524
|
+
npm run build # seeded bootstrap into build/nish (NISH_BOOTSTRAP=<nish> names the seed)
|
|
525
|
+
npm test # build + goldens, llvm-as, native round trips, runtime, layout, memory, interop, bench checksums, differential
|
|
526
|
+
node tests/run.js locals # only cases whose name contains "locals"
|
|
527
|
+
npm run test:update # write missing .ll goldens for new cases
|
|
528
|
+
npm run test:diff # every whole program natively and under Node (runtime/shim.mjs), compared byte for byte
|
|
529
|
+
node tests/differential/fuzz.js --stage1 --count 200 # random integer programs, the released seed's IR against HEAD's; prints the seed
|
|
530
|
+
npm run check # tsc --noEmit: an ambient type-check of self/, std/ and tests/nish/
|
|
531
|
+
npm run lint # Biome style lint (advisory, never a compile gate)
|
|
532
|
+
npm run smoke # build and run every example with a main
|
|
533
|
+
scripts/bootstrap.sh --verify # the whole chain, with IR(stage1) == IR(stage2) and stage3 == stage2 asserted
|
|
534
|
+
node bench/run.mjs # the benchmark suite; rewrites docs/BENCHMARKS.md (about 3 minutes)
|
|
535
|
+
docs/cookbook/regen.sh # refresh docs/IR_COOKBOOK.md; node docs/check-links.mjs checks the links
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) (the pipeline, side tables,
|
|
539
|
+
the add-a-construct checklist, ABI guard tests) and the conventions in
|
|
540
|
+
[docs/MASTER_PLAN.md §7](docs/MASTER_PLAN.md#7-conventions-for-every-agent):
|
|
541
|
+
every construct ships with a golden `.ll`, an `llvm-as` pass, a native round
|
|
542
|
+
trip, a negative test, and its LANGUAGE.md and cookbook entries; no attribute
|
|
543
|
+
without a proof; layout changes touch `self/runtime.ts` and `runtime/runtime.c` together.
|
|
544
|
+
CI runs the suite on Ubuntu and macOS with LLVM 18
|
|
545
|
+
([docs/wp10-ci.md](docs/wp10-ci.md)). The documentation index is
|
|
546
|
+
[docs/README.md](docs/README.md). Coding guidelines for contributors and
|
|
547
|
+
coding agents are in [AGENTS.md](AGENTS.md) and [`.claude/`](.claude/).
|
|
548
|
+
|
|
549
|
+
---
|
|
550
|
+
|
|
551
|
+
## License
|
|
552
|
+
|
|
553
|
+
MIT, see [LICENSE](LICENSE).
|