xbintsc 0.3.35 → 0.3.49
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +95 -0
- package/README.md +31 -3
- package/README.zh-CN.md +23 -0
- package/dist/src/cli/hints.d.ts +54 -0
- package/dist/src/cli/hints.js +165 -0
- package/dist/src/cli/hints.js.map +1 -0
- package/dist/src/cli/main.js +73 -9
- package/dist/src/cli/main.js.map +1 -1
- package/dist/src/codegen/generator/module.js +10 -0
- package/dist/src/codegen/generator/module.js.map +1 -1
- package/dist/src/codegen/generator/state.js +4 -0
- package/dist/src/codegen/generator/state.js.map +1 -1
- package/dist/src/codegen/generator/tables.d.ts +26 -0
- package/dist/src/codegen/generator/tables.js +64 -12
- package/dist/src/codegen/generator/tables.js.map +1 -1
- package/dist/src/diagnostics/source-text.d.ts +22 -0
- package/dist/src/diagnostics/source-text.js +76 -0
- package/dist/src/diagnostics/source-text.js.map +1 -0
- package/dist/src/driver/bundler/graph.js +2 -1
- package/dist/src/driver/bundler/graph.js.map +1 -1
- package/dist/src/driver/compiler.js +3 -2
- package/dist/src/driver/compiler.js.map +1 -1
- package/dist/src/lexer/scanner/strings.js +16 -3
- package/dist/src/lexer/scanner/strings.js.map +1 -1
- package/dist/tests/cli/hints.test.d.ts +9 -0
- package/dist/tests/cli/hints.test.js +143 -0
- package/dist/tests/cli/hints.test.js.map +1 -0
- package/dist/tests/cli/main.test.js +6 -4
- package/dist/tests/cli/main.test.js.map +1 -1
- package/dist/tests/codegen/llvm.test.js +17 -2
- package/dist/tests/codegen/llvm.test.js.map +1 -1
- package/dist/tests/e2e/gc.test.d.ts +1 -0
- package/dist/tests/e2e/gc.test.js +168 -0
- package/dist/tests/e2e/gc.test.js.map +1 -0
- package/dist/tests/e2e/harness.d.ts +2 -0
- package/dist/tests/e2e/harness.js +1 -0
- package/dist/tests/e2e/harness.js.map +1 -1
- package/dist/tests/helpers.js +3 -2
- package/dist/tests/helpers.js.map +1 -1
- package/dist/tests/lexer/strings.test.js +14 -2
- package/dist/tests/lexer/strings.test.js.map +1 -1
- package/doc/DESIGN.md +117 -0
- package/doc/ai/README.md +63 -0
- package/doc/ai/build-recipe.md +137 -0
- package/doc/ai/cli.md +142 -0
- package/doc/ai/contributing.md +196 -0
- package/doc/ai/extensions.md +148 -0
- package/doc/ai/language-support.md +152 -0
- package/doc/ai/troubleshooting.md +163 -0
- package/doc/ai/zh-CN/README.md +56 -0
- package/doc/ai/zh-CN/build-recipe.md +132 -0
- package/doc/ai/zh-CN/cli.md +127 -0
- package/doc/ai/zh-CN/contributing.md +173 -0
- package/doc/ai/zh-CN/extensions.md +139 -0
- package/doc/ai/zh-CN/language-support.md +147 -0
- package/doc/ai/zh-CN/troubleshooting.md +150 -0
- package/doc/gui-scripts.md +350 -0
- package/doc/gui.md +646 -0
- package/doc/icon.md +265 -0
- package/doc/implemented.md +373 -0
- package/doc/node-implemented.md +588 -0
- package/doc/node-unimplemented.md +167 -0
- package/doc/post/announce.md +43 -0
- package/doc/requirements.md +145 -0
- package/doc/unimplemented.md +286 -0
- package/doc/xbintsc.config.schema.json +67 -0
- package/doc/zh-CN/DESIGN.md +104 -0
- package/doc/zh-CN/gui-scripts.md +329 -0
- package/doc/zh-CN/gui.md +588 -0
- package/doc/zh-CN/icon.md +241 -0
- package/doc/zh-CN/implemented.md +365 -0
- package/doc/zh-CN/node-implemented.md +533 -0
- package/doc/zh-CN/node-unimplemented.md +141 -0
- package/doc/zh-CN/plan-require-node-modules.md +284 -0
- package/doc/zh-CN/post/announce.md +47 -0
- package/doc/zh-CN/requirements.md +134 -0
- package/doc/zh-CN/unimplemented.md +247 -0
- package/llms.txt +45 -0
- package/package.json +6 -2
- package/runtime/ext_gui/dom_api_proto.cpp +5 -0
- package/runtime/ext_gui/gui.cpp +3 -1
- package/runtime/ext_gui/renderer.cpp +13 -11
- package/runtime/ext_gui/renderer_image.cpp +12 -8
- package/runtime/ext_gui/renderer_shaders.h +131 -4
- package/runtime/ext_gui/renderer_shaders_data.h +1809 -0
- package/runtime/ext_gui/renderer_text.cpp +12 -8
- package/runtime/ext_gui/shaders.hlsl +98 -0
- package/runtime/ext_gui/spirv/fill.frag +19 -0
- package/runtime/ext_gui/spirv/fill.vert +42 -0
- package/runtime/ext_gui/spirv/image.frag +16 -0
- package/runtime/ext_gui/spirv/quad.vert +30 -0
- package/runtime/ext_gui/spirv/text.frag +16 -0
- package/runtime/ext_gui/window.cpp +1 -0
- package/runtime/ext_node/buffer/parts/prototype.inc +1 -0
- package/runtime/ext_node/dgram/dgram.c +1 -0
- package/runtime/ext_node/events/events.c +1 -0
- package/runtime/ext_node/fs/fs_ops.c +10 -25
- package/runtime/ext_node/fs/glob.c +13 -30
- package/runtime/ext_node/fs/promises.c +1 -0
- package/runtime/ext_node/http/parts/prototypes.inc +5 -0
- package/runtime/ext_node/net/parts/prototypes.inc +2 -0
- package/runtime/ext_node/process/process.c +9 -7
- package/runtime/ext_node/stream/stream.c +1 -0
- package/runtime/ext_node/util/util.c +6 -6
- package/runtime/rt.h +10 -0
- package/runtime/rt_internal.h +63 -2
- package/runtime/xt_alloc.c +442 -5
- package/runtime/xt_generator.c +95 -1
- package/runtime/xt_loop.c +35 -1
- package/runtime/xt_promise.c +46 -0
- package/runtime/xt_stdlib2/error.inc +1 -0
- package/runtime/xt_stdlib2/regexp-match.inc +8 -7
- package/runtime/xt_symbol.c +2 -0
- package/runtime/xt_typed_array/construction.inc +142 -0
- package/runtime/xt_typed_array/elements.inc +92 -0
- package/runtime/xt_typed_array/methods.inc +329 -0
- package/runtime/xt_typed_array.c +6 -548
- package/runtime/xt_values/number-format.inc +26 -0
- package/scripts/build-gui-shaders.mjs +204 -0
- package/scripts/build-gui.ts +35 -0
- package/scripts/check-file-length.ts +89 -0
- package/src/cli/hints.ts +194 -0
- package/src/cli/main.ts +82 -9
- package/src/codegen/generator/module.ts +10 -0
- package/src/codegen/generator/state.ts +4 -0
- package/src/codegen/generator/tables.ts +60 -14
- package/src/diagnostics/source-text.ts +78 -0
- package/src/driver/bundler/graph.ts +2 -1
- package/src/driver/compiler.ts +3 -2
- package/src/lexer/scanner/strings.ts +16 -3
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"strings.test.js","sourceRoot":"","sources":["../../../tests/lexer/strings.test.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,QAAQ,CAAC;AAC9C,OAAO,EAAE,GAAG,EAAE,MAAM,eAAe,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,0BAA0B,CAAC;AACrD,OAAO,EAAE,cAAc,EAAE,MAAM,qCAAqC,CAAC;AAErE,uDAAuD;AACvD,SAAS,OAAO,CAAC,MAAc;IAC7B,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,MAAM,CAAC,CAAC;IAC5C,MAAM,CAAC,WAAW,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;IACpC,OAAO,MAAM,CAAC,CAAC,CAAE,CAAC,KAAK,CAAC;AAC1B,CAAC;AAED,QAAQ,CAAC,yBAAyB,EAAE,GAAG,EAAE;IACvC,EAAE,CAAC,oCAAoC,EAAE,GAAG,EAAE;QAC5C,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC1C,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,qDAAqD,EAAE,GAAG,EAAE;QAC7D,MAAM,CAAC,OAAO,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI,CAAC,
|
|
1
|
+
{"version":3,"file":"strings.test.js","sourceRoot":"","sources":["../../../tests/lexer/strings.test.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,QAAQ,CAAC;AAC9C,OAAO,EAAE,GAAG,EAAE,MAAM,eAAe,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,0BAA0B,CAAC;AACrD,OAAO,EAAE,cAAc,EAAE,MAAM,qCAAqC,CAAC;AAErE,uDAAuD;AACvD,SAAS,OAAO,CAAC,MAAc;IAC7B,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,MAAM,CAAC,CAAC;IAC5C,MAAM,CAAC,WAAW,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;IACpC,OAAO,MAAM,CAAC,CAAC,CAAE,CAAC,KAAK,CAAC;AAC1B,CAAC;AAED;;;;GAIG;AACH,SAAS,OAAO,CAAC,IAAY;IAC3B,OAAO,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;AAC5E,CAAC;AAED,QAAQ,CAAC,yBAAyB,EAAE,GAAG,EAAE;IACvC,EAAE,CAAC,oCAAoC,EAAE,GAAG,EAAE;QAC5C,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC1C,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,qDAAqD,EAAE,GAAG,EAAE;QAC7D,0EAA0E;QAC1E,wCAAwC;QACxC,MAAM,CAAC,OAAO,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC;QACvD,MAAM,CAAC,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACvC,MAAM,CAAC,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;QACjD,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACrC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;IACjD,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,yCAAyC,EAAE,GAAG,EAAE;QACjD,MAAM,CAAC,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACxC,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxC,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,4DAA4D,EAAE,GAAG,EAAE;QACpE,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACpC,uEAAuE;QACvE,MAAM,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC5C,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,iDAAiD,EAAE,GAAG,EAAE;QACzD,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACrC,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,uDAAuD,EAAE,GAAG,EAAE;QAC/D,MAAM,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC;IACrF,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,6DAA6D,EAAE,GAAG,EAAE;QACrE,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,cAAc,CAAC,CAAC;QAC5C,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC,cAAc,CAAC,kBAAkB,CAAC,CAAC;QACpF,MAAM,CAAC,WAAW,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACtE,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,gDAAgD,EAAE,GAAG,EAAE;QACxD,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,eAAe,CAAC,CAAC;QAC7C,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,cAAc,CAAC,kBAAkB,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3F,CAAC,CAAC,CAAC;AACL,CAAC,CAAC,CAAC;AAEH,QAAQ,CAAC,2BAA2B,EAAE,GAAG,EAAE;IACzC,EAAE,CAAC,+CAA+C,EAAE,GAAG,EAAE;QACvD,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,SAAS,CAAC,CAAC;QAC/C,MAAM,CAAC,WAAW,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;QACpC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,6BAA6B,CAAC,CAAC;QACtE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACxC,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,gCAAgC,EAAE,GAAG,EAAE;QACxC,MAAM,EAAE,MAAM,EAAE,GAAG,GAAG,CAAC,gBAAgB,CAAC,CAAC;QACzC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,6BAA6B,CAAC,CAAC;QACtE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;IAChD,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,oCAAoC,EAAE,GAAG,EAAE;QAC5C,MAAM,EAAE,MAAM,EAAE,GAAG,GAAG,CAAC,YAAY,CAAC,CAAC;QACrC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,YAAY,CAAC,CAAC;QACrD,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACvC,CAAC,CAAC,CAAC;IAEH,EAAE,CAAC,0CAA0C,EAAE,GAAG,EAAE;QAClD,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,GAAG,CAAC,eAAe,CAAC,CAAC;QACrD,MAAM,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,6BAA6B,CAAC,CAAC;QACtE,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,cAAc,CAAC,oBAAoB,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC7F,CAAC,CAAC,CAAC;AACL,CAAC,CAAC,CAAC"}
|
package/doc/DESIGN.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# xbintsc Design
|
|
2
|
+
|
|
3
|
+
Build a binary compiler for TypeScript.
|
|
4
|
+
|
|
5
|
+
> Language: **English** | [简体中文](./zh-CN/DESIGN.md)
|
|
6
|
+
|
|
7
|
+
## Architecture
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
source.ts
|
|
11
|
+
│ lexer src/lexer lexing: full token set, templates, regex, ASI
|
|
12
|
+
▼
|
|
13
|
+
tokens
|
|
14
|
+
│ parser src/parser recursive descent → AST (src/ast)
|
|
15
|
+
▼
|
|
16
|
+
AST
|
|
17
|
+
│ binder src/binder scopes / symbols / hoisting / closure capture
|
|
18
|
+
▼
|
|
19
|
+
bound AST
|
|
20
|
+
│ codegen src/codegen LLVM IR text (llvm.ts); values.ts defines the value model
|
|
21
|
+
▼
|
|
22
|
+
module.ll ──clang──► module.o ──link──► executable
|
|
23
|
+
▲
|
|
24
|
+
runtime/ C runtime (NaN-boxed values)
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
### Value model
|
|
28
|
+
|
|
29
|
+
`xt_value` is a single 64-bit word. Doubles are stored unboxed; everything else
|
|
30
|
+
is a tagged pointer with a 16-bit tag in the high bits plus a 48-bit payload.
|
|
31
|
+
The representation is defined once in `src/codegen/values.ts` and `runtime/rt.h`
|
|
32
|
+
and shared by the compiler and the runtime.
|
|
33
|
+
|
|
34
|
+
### Calling convention
|
|
35
|
+
|
|
36
|
+
Every compiled function uses the same ABI:
|
|
37
|
+
|
|
38
|
+
```c
|
|
39
|
+
xt_value fn(xt_value env, int32_t argc, xt_value *argv);
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
`env` threads captured variables by reference through boxes, so direct calls and
|
|
43
|
+
closure calls share a single code path. JavaScript semantics that are awkward to
|
|
44
|
+
inline (`+` coercion, relational comparison, property access, inspection) are
|
|
45
|
+
delegated to `@xt_*` runtime calls.
|
|
46
|
+
|
|
47
|
+
### Runtime
|
|
48
|
+
|
|
49
|
+
`runtime/xt_runtime.c` implements strings, objects, arrays, closures, arithmetic,
|
|
50
|
+
comparison, exceptions and Node-like `console.log` inspection. It uses a
|
|
51
|
+
non-moving mark-sweep collector behind `xt_alloc` (explicit roots, subsystem
|
|
52
|
+
root providers and a conservative C-stack scan), so the collector's internals
|
|
53
|
+
stay isolated from the compiler.
|
|
54
|
+
|
|
55
|
+
### Replacing llc
|
|
56
|
+
|
|
57
|
+
LLVM/`llc` is not installed on the host, but `clang` can compile LLVM IR text
|
|
58
|
+
directly, so the pipeline is `IR text → clang -c → object file → link C runtime`.
|
|
59
|
+
This keeps the "IR binding + native compilation" goal while remaining
|
|
60
|
+
cross-platform.
|
|
61
|
+
|
|
62
|
+
## Directory structure
|
|
63
|
+
|
|
64
|
+
| Directory | Responsibility |
|
|
65
|
+
| --- | --- |
|
|
66
|
+
| `src/lexer` | Lexing (Scanner, TokenKind) |
|
|
67
|
+
| `src/ast` | AST nodes, factories, visitors |
|
|
68
|
+
| `src/parser` | Recursive descent parser (including type syntax) |
|
|
69
|
+
| `src/binder` | Scope and symbol resolution, closure capture |
|
|
70
|
+
| `src/codegen` | Value model and LLVM IR generation |
|
|
71
|
+
| `src/diagnostics` | Source files, diagnostics, hashing |
|
|
72
|
+
| `src/driver` | Pipeline driver, incremental cache, clang toolchain wrapper |
|
|
73
|
+
| `src/extensions` | Pluggable extension registry and Node extension |
|
|
74
|
+
| `src/cli` | Command-line entry point |
|
|
75
|
+
| `runtime` | C runtime and `rt.h` |
|
|
76
|
+
| `tests` | Tests organised by module (including e2e) |
|
|
77
|
+
| `scripts` | Runtime build and other scripts |
|
|
78
|
+
| `.github/workflows` | Multi-platform, multi-version CI |
|
|
79
|
+
|
|
80
|
+
## Incremental compilation
|
|
81
|
+
|
|
82
|
+
`src/driver/cache.ts` keys the cache on the entry source hash, compiler version,
|
|
83
|
+
compile options, platform and extension set; when all recorded outputs still
|
|
84
|
+
exist the build is skipped. The runtime C sources are cached as object files by
|
|
85
|
+
content hash in the same way.
|
|
86
|
+
|
|
87
|
+
## Extension mechanism
|
|
88
|
+
|
|
89
|
+
An extension is a plain object (see `src/extensions/registry.ts`): it declares
|
|
90
|
+
extra C runtime code, linker flags, and the modules it makes importable, mapping
|
|
91
|
+
exported bindings to runtime symbols with the uniform `(argc, argv)` ABI. Node's
|
|
92
|
+
`import { readFileSync } from "fs"` is wired in through
|
|
93
|
+
`src/extensions/node/fs` + `runtime/ext_node/fs`, and the core compiler never
|
|
94
|
+
needs to know any platform details.
|
|
95
|
+
|
|
96
|
+
`nativeObjects` extends the same shape to code built outside the pipeline: a
|
|
97
|
+
C++ or Rust project compiled to an `extern "C"` object or static archive can be
|
|
98
|
+
linked in and exposed through a JSON manifest (`src/extensions/native.ts`,
|
|
99
|
+
`--ext-native`). The driver links the artifacts verbatim and the generator binds
|
|
100
|
+
their symbols exactly like the C runtime, so the extension language is invisible
|
|
101
|
+
to the core compiler. Windows is covered too: xbintsc links with the toolchain
|
|
102
|
+
found on `PATH` (the MSVC ABI when LLVM + the Visual Studio C++ build tools are
|
|
103
|
+
installed), and each manifest declares any platform-specific linker flags. See
|
|
104
|
+
`examples/extensions/` and `runtime/xt_ext.h` / `runtime/xt_ext.rs`.
|
|
105
|
+
|
|
106
|
+
## Self-hosting roadmap
|
|
107
|
+
|
|
108
|
+
The compiler itself is written in TypeScript and emits IR text, and `runtime` is
|
|
109
|
+
already decoupled from the compiler. Later, the runtime can be rewritten in TS
|
|
110
|
+
and the compiler can compile itself, gradually reaching self-hosting.
|
|
111
|
+
|
|
112
|
+
## Tests
|
|
113
|
+
|
|
114
|
+
Organised by module: `tests/lexer`, `tests/parser`, `tests/binder`,
|
|
115
|
+
`tests/codegen`, `tests/driver`, `tests/extensions`, `tests/cli`, `tests/e2e`.
|
|
116
|
+
When `clang` is present, e2e tests really compile and run binaries; otherwise
|
|
117
|
+
they are skipped automatically.
|
package/doc/ai/README.md
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# xbintsc for AI agents
|
|
2
|
+
|
|
3
|
+
**xbintsc compiles a subset of TypeScript straight to native binaries.** It
|
|
4
|
+
parses TypeScript itself, lowers the program to LLVM IR text, and invokes
|
|
5
|
+
`clang` to produce a standalone executable linked against a small C runtime
|
|
6
|
+
(`runtime/`). The output does not embed Node.js or a TypeScript compiler.
|
|
7
|
+
|
|
8
|
+
```
|
|
9
|
+
source.ts --lexer--> tokens --parser--> AST --binder--> bound AST
|
|
10
|
+
--codegen--> module.ll --clang--> module.o --link--> executable
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
This directory is the **AI-facing, task-oriented** layer of the documentation.
|
|
14
|
+
It is deliberately short and links onward; the canonical detail lives in the
|
|
15
|
+
documents it points at. Read the page for your task, not the whole directory.
|
|
16
|
+
|
|
17
|
+
| Page | Use it when |
|
|
18
|
+
| --- | --- |
|
|
19
|
+
| [build-recipe.md](./build-recipe.md) | You need to compile or run a program, and want the exact command shape |
|
|
20
|
+
| [cli.md](./cli.md) | You need the full flag list, project config, or the programmatic API |
|
|
21
|
+
| [language-support.md](./language-support.md) | You must know whether a syntax feature or API exists, or behaves like Node |
|
|
22
|
+
| [extensions.md](./extensions.md) | You need `fs`/`http`/…, a GUI, or a C++/Rust library |
|
|
23
|
+
| [troubleshooting.md](./troubleshooting.md) | Something failed and you need the cause and the fix |
|
|
24
|
+
|
|
25
|
+
New to this repository (rather than to a program that uses xbintsc)? Read
|
|
26
|
+
[../AGENTS.md](../../AGENTS.md) first — it covers the layout, the build gate and
|
|
27
|
+
the rules for changing the compiler.
|
|
28
|
+
|
|
29
|
+
## The three things that decide everything
|
|
30
|
+
|
|
31
|
+
1. **Only a subset of TypeScript compiles.** Unsupported syntax is rejected, not
|
|
32
|
+
approximated. Check before you write a lot of code.
|
|
33
|
+
2. **Types are erased, never checked.** Annotations, interfaces, generics,
|
|
34
|
+
`as`/`satisfies` and non-null `!` have no runtime effect, and no type checker
|
|
35
|
+
runs. Only real runtime behaviour matters.
|
|
36
|
+
3. **Node compatibility is opt-in and incomplete.** Node modules come from the
|
|
37
|
+
`node` extension (`--ext node`); bare third-party npm imports are not
|
|
38
|
+
supported.
|
|
39
|
+
|
|
40
|
+
## Choose your command
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
# Run a program now (compiles to a temp binary, then executes it)
|
|
44
|
+
xbintsc run app.ts
|
|
45
|
+
|
|
46
|
+
# Produce a standalone binary
|
|
47
|
+
xbintsc build app.ts --out build # -> build/app[.exe]
|
|
48
|
+
|
|
49
|
+
# See the LLVM IR that would be compiled
|
|
50
|
+
xbintsc emit app.ts
|
|
51
|
+
|
|
52
|
+
# Ask what toolchain xbintsc resolved
|
|
53
|
+
xbintsc doctor
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
In a source checkout, replace `xbintsc` with `npx tsx src/cli/main.ts` (or run
|
|
57
|
+
`npm run xbintsc --`). A released archive needs no Node.js.
|
|
58
|
+
|
|
59
|
+
Next: [build-recipe.md](./build-recipe.md) for the working commands,
|
|
60
|
+
[language-support.md](./language-support.md) before writing a nontrivial
|
|
61
|
+
program.
|
|
62
|
+
|
|
63
|
+
> 中文版本:[zh-CN/](./zh-CN/)
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
# Build recipe
|
|
2
|
+
|
|
3
|
+
Copy-paste shapes for the common jobs, plus the reasoning that makes a build
|
|
4
|
+
succeed. For every flag see [cli.md](./cli.md); when a command fails see
|
|
5
|
+
[troubleshooting.md](./troubleshooting.md).
|
|
6
|
+
|
|
7
|
+
## Calling the compiler
|
|
8
|
+
|
|
9
|
+
| Situation | Invocation |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| Released standalone archive | `xbintsc …` (`bin/xbintsc[.exe]` on `PATH`) |
|
|
12
|
+
| Source checkout, compiled | `node dist/src/cli/main.js …` |
|
|
13
|
+
| Source checkout, TypeScript sources | `npx tsx src/cli/main.ts …` |
|
|
14
|
+
| Source checkout, npm script | `npm run xbintsc -- …` |
|
|
15
|
+
| `bin` launcher in a checkout | `node bin/xbintsc.js …` (falls back to `tsx` automatically) |
|
|
16
|
+
|
|
17
|
+
The rest of these pages write `xbintsc` for brevity.
|
|
18
|
+
|
|
19
|
+
## Run a program
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
xbintsc run app.ts
|
|
23
|
+
xbintsc run app.ts -- --flag value # everything after -- goes to the program
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`run` compiles the program to an executable and then spawns it with inherited
|
|
27
|
+
stdio. It requires `--emit exe`; `xbintsc run app.ts --emit ir` is an error.
|
|
28
|
+
The exit status of `run` is the exit status of the program it ran.
|
|
29
|
+
|
|
30
|
+
## Build a standalone binary
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
xbintsc build app.ts # -> build/app (build/app.exe on Windows)
|
|
34
|
+
xbintsc build app.ts --out build/app # choose the output directory
|
|
35
|
+
xbintsc build app.ts -o app.bin # or an explicit output path
|
|
36
|
+
xbintsc build app.ts -O0 # optimization level: -O0 .. -O3 (default -O2)
|
|
37
|
+
xbintsc build app.ts --force # ignore the incremental cache
|
|
38
|
+
xbintsc build app.ts --verbose # print progress
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
`build` prints the artifact it wrote, and `(cached)` when the incremental cache
|
|
42
|
+
made the work unnecessary:
|
|
43
|
+
|
|
44
|
+
```
|
|
45
|
+
xbintsc: wrote /abs/path/build/app
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Other emit kinds are useful for inspection:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
xbintsc build app.ts --emit ir # writes app.ll
|
|
52
|
+
xbintsc build app.ts --emit obj # writes app.o
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Inspect the LLVM IR
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
xbintsc emit app.ts > app.ll # IR on stdout, nothing else written
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
`emit` needs no clang and no runtime library — it is the cheapest way to check
|
|
62
|
+
what the compiler understood. It is the fastest feedback loop when you are
|
|
63
|
+
unsure whether a construct is supported.
|
|
64
|
+
|
|
65
|
+
## Program shape that compiles
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
// app.ts — ESM, no require()
|
|
69
|
+
import { readFileSync } from "fs"; // needs `--ext node`
|
|
70
|
+
|
|
71
|
+
function main(): void {
|
|
72
|
+
const text = readFileSync("package.json", "utf8");
|
|
73
|
+
console.log(text.length);
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
main(); // top-level code runs in order
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
- Use `import`/`export`; `require()` is rejected.
|
|
80
|
+
- Relative imports are bundled (`import { helper } from "./helper.js"` resolves
|
|
81
|
+
to `helper.ts`).
|
|
82
|
+
- Module specifiers for Node built-ins are the bare names (`fs`, `path`,
|
|
83
|
+
`node:fs`), and they need the `node` extension enabled.
|
|
84
|
+
|
|
85
|
+
## Project config: build with no arguments
|
|
86
|
+
|
|
87
|
+
`xbintsc.config.json` (discovered by walking up from the entry file, or selected
|
|
88
|
+
with `--config <path>`; disable with `--no-config`) holds the build options, so
|
|
89
|
+
`xbintsc build` alone works. Paths resolve against the config file's directory,
|
|
90
|
+
and any CLI flag overrides the matching field.
|
|
91
|
+
|
|
92
|
+
```json
|
|
93
|
+
{
|
|
94
|
+
"entry": "src/app.ts",
|
|
95
|
+
"outDir": "build",
|
|
96
|
+
"optimize": "2",
|
|
97
|
+
"extensions": ["node"],
|
|
98
|
+
"app": { "name": "Demo", "icon": "assets/app.png" }
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
The machine-readable schema is [../xbintsc.config.schema.json](../xbintsc.config.schema.json);
|
|
103
|
+
the full field list is in [cli.md](./cli.md).
|
|
104
|
+
|
|
105
|
+
## Compiling with extensions
|
|
106
|
+
|
|
107
|
+
An import of a Node module fails until the extension is enabled. Enable one or
|
|
108
|
+
several, comma-separated:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
xbintsc run app.ts --ext node
|
|
112
|
+
xbintsc run app.ts --ext node,gui
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
A C++/Rust library is added by manifest instead:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
xbintsc run app.ts --ext-native ./mathx.manifest.json
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Details, including how to author both kinds, are in
|
|
122
|
+
[extensions.md](./extensions.md).
|
|
123
|
+
|
|
124
|
+
## Programmatic API
|
|
125
|
+
|
|
126
|
+
When you are writing tooling rather than a program:
|
|
127
|
+
|
|
128
|
+
```ts
|
|
129
|
+
import { build, compileString } from "xbintsc";
|
|
130
|
+
|
|
131
|
+
const { ir } = compileString("console.log(1 + 1);"); // IR text, no clang needed
|
|
132
|
+
const result = build("program.ts", { emit: "exe", outDir: "build" });
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
`build` returns `{ outputPath, irPath?, cached, diagnostics, bundlePath? }` and
|
|
136
|
+
reports recoverable problems through `diagnostics` rather than throwing. The
|
|
137
|
+
subpath export `xbintsc/driver` exposes the driver internals.
|
package/doc/ai/cli.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# CLI and configuration reference
|
|
2
|
+
|
|
3
|
+
The authoritative `--help` text is the `HELP` constant in
|
|
4
|
+
[../../src/cli/main.ts](../../src/cli/main.ts). This page adds the semantics that
|
|
5
|
+
the help text leaves implicit.
|
|
6
|
+
|
|
7
|
+
## Commands
|
|
8
|
+
|
|
9
|
+
| Command | Purpose | Needs clang? |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| `xbintsc build <file.ts>` | Compile to `exe` (default), `obj` or `ir` | yes, except `--emit ir` |
|
|
12
|
+
| `xbintsc run <file.ts> [-- args]` | Compile to an executable and execute it | yes (`--emit exe` only) |
|
|
13
|
+
| `xbintsc emit <file.ts>` | Print LLVM IR to stdout, write nothing | no |
|
|
14
|
+
| `xbintsc doctor` | Report the resolved toolchain, runtime and icon tools | only for the probe |
|
|
15
|
+
| `xbintsc version` | Print the version | no |
|
|
16
|
+
| `xbintsc help` | Print the help text | no |
|
|
17
|
+
|
|
18
|
+
Any command also accepts `--help`, which prints the help text and exits 0. An
|
|
19
|
+
unknown command prints an error plus the help text and exits 1.
|
|
20
|
+
|
|
21
|
+
## Options
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
-o, --output <path> Explicit output path (overrides --out and the config)
|
|
25
|
+
--out <dir> Output directory (default: build/)
|
|
26
|
+
--emit <kind> exe | obj | ir (default: exe)
|
|
27
|
+
-O0 .. -O3 Optimization level passed to clang (default: -O2)
|
|
28
|
+
--ext <names> Enable bundled extensions, comma separated (e.g. node,gui)
|
|
29
|
+
--ext-native <m> Register a C++/Rust extension from a JSON manifest
|
|
30
|
+
(comma separated for several)
|
|
31
|
+
--config <path> Use this project config instead of discovering one
|
|
32
|
+
--no-config Do not read any project config
|
|
33
|
+
--icon <path> Embed an application icon (PNG/ICO/ICNS)
|
|
34
|
+
--bundle macOS: also produce a <name>.app bundle
|
|
35
|
+
--app-name <name> Bundle / display name
|
|
36
|
+
--app-id <id> macOS bundle identifier (e.g. com.example.demo)
|
|
37
|
+
--force Ignore the incremental cache
|
|
38
|
+
--verbose Print progress information
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Parsing details that occasionally surprise:
|
|
42
|
+
|
|
43
|
+
- `--flag=value` and `--flag value` are both accepted.
|
|
44
|
+
- `-O`, `-O1`, `-O2`, `-O3` all work; a bare `-O` means `-O2`.
|
|
45
|
+
- `--` ends option parsing: everything after it belongs to the program under
|
|
46
|
+
`run`, or is a positional for the other commands.
|
|
47
|
+
- `-o` consumes the next argument unconditionally; the other value flags leave
|
|
48
|
+
the value unset when the next argument starts with `-`.
|
|
49
|
+
|
|
50
|
+
## Project config
|
|
51
|
+
|
|
52
|
+
`xbintsc.config.json` is discovered by walking **up** from the entry file's
|
|
53
|
+
directory (or from the current directory when no entry was given). Every path in
|
|
54
|
+
the file resolves relative to the config file, and any CLI flag overrides the
|
|
55
|
+
matching field. `--no-config` skips discovery entirely.
|
|
56
|
+
|
|
57
|
+
| Field | Type | Meaning |
|
|
58
|
+
| --- | --- | --- |
|
|
59
|
+
| `entry` | string | Entry TypeScript file, so `xbintsc build` needs no positional |
|
|
60
|
+
| `outDir` | string | Output directory (default `build/`) |
|
|
61
|
+
| `output` | string | Explicit output path, overriding `outDir` |
|
|
62
|
+
| `optimize` | `"0"|"1"|"2"|"3"` | Optimization level |
|
|
63
|
+
| `extensions` | string[] | Bundled extensions to enable, e.g. `["node"]` |
|
|
64
|
+
| `extNative` | string[] | Native extension manifest paths |
|
|
65
|
+
| `force` | boolean | Ignore the incremental cache |
|
|
66
|
+
| `app.name` / `app.icon` / `app.bundle` / `app.bundleId` | | Application metadata ([icon.md](../icon.md)) |
|
|
67
|
+
|
|
68
|
+
JSON schema: [xbintsc.config.schema.json](../xbintsc.config.schema.json). An
|
|
69
|
+
unreadable or malformed config fails the build with `invalid JSON` or
|
|
70
|
+
`Unable to read` rather than silently falling back to defaults.
|
|
71
|
+
|
|
72
|
+
## Environment variables
|
|
73
|
+
|
|
74
|
+
| Variable | Effect |
|
|
75
|
+
| --- | --- |
|
|
76
|
+
| `xbintsc_CLANG` | Use this clang instead of resolving one from `PATH` |
|
|
77
|
+
| `xbintsc_LINKER_ARGS` | Extra linker flags (e.g. `-fuse-ld=lld`) |
|
|
78
|
+
| `xbintsc_CACHE_DIR` | Object-cache directory (default: `.xbintsc`) |
|
|
79
|
+
| `xbintsc_PREFER_PREBUILT` | `0` forces the runtime to be compiled from source |
|
|
80
|
+
| `xbintsc_BINARY` | Native compiler binary the `bin/` launcher should run |
|
|
81
|
+
|
|
82
|
+
## Incremental compilation
|
|
83
|
+
|
|
84
|
+
Each build is keyed on the entry source hash, the compiler version, the build
|
|
85
|
+
options, the platform and the active extension set
|
|
86
|
+
([../../src/driver/cache.ts](../../src/driver/cache.ts)). If every recorded
|
|
87
|
+
output still exists, the build returns immediately and prints `(cached)`. The C
|
|
88
|
+
runtime and extension sources are cached the same way; objects live in
|
|
89
|
+
`.xbintsc/` (or `xbintsc_CACHE_DIR`).
|
|
90
|
+
|
|
91
|
+
`--force` (or `force: true`) bypasses the freshness check. If a build ever
|
|
92
|
+
returns stale output, that key is the first thing to inspect — and note that the
|
|
93
|
+
object cache key deliberately excludes compiler flags, which is why the runtime
|
|
94
|
+
coverage script uses a separate cache directory.
|
|
95
|
+
|
|
96
|
+
## Exit codes
|
|
97
|
+
|
|
98
|
+
- `0` — success (for `run`: the program exited 0).
|
|
99
|
+
- `1` — diagnostics, a usage error, or a failure in the compiler itself.
|
|
100
|
+
- `run` otherwise propagates the program's own exit status.
|
|
101
|
+
|
|
102
|
+
## Diagnostics
|
|
103
|
+
|
|
104
|
+
Errors are rendered as `file:line:col - error TS<code>: <message>` with a source
|
|
105
|
+
excerpt and caret underline. Codes are grouped by stage
|
|
106
|
+
([../../src/diagnostics/diagnostic.ts](../../src/diagnostics/diagnostic.ts)):
|
|
107
|
+
|
|
108
|
+
| Range | Stage |
|
|
109
|
+
| --- | --- |
|
|
110
|
+
| TS1xxx | lexer |
|
|
111
|
+
| TS2xxx | parser |
|
|
112
|
+
| TS3xxx | binder |
|
|
113
|
+
| TS4xxx | checker (`TS4005` is the `UnsupportedFeature` code) |
|
|
114
|
+
| TS5xxx | codegen |
|
|
115
|
+
| TS6xxx | driver: `TS6001` module not found, `TS6002` IO, `TS6003` toolchain, `TS6004` cache |
|
|
116
|
+
|
|
117
|
+
The CLI appends a `hint:` line pointing at the relevant document (for example
|
|
118
|
+
[language-support.md](./language-support.md) for `TS4005`, or
|
|
119
|
+
[troubleshooting.md](./troubleshooting.md) for `TS6001`/`TS6003`). Toolchain and
|
|
120
|
+
IO failures throw instead of producing a diagnostic, so the CLI reports those as
|
|
121
|
+
`xbintsc: <message>` plus the same style of hint.
|
|
122
|
+
|
|
123
|
+
## Programmatic API
|
|
124
|
+
|
|
125
|
+
```ts
|
|
126
|
+
import { build, compileString } from "xbintsc";
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
- `compileString(source, fileName?, extensions?)` → `{ ir, diagnostics }`.
|
|
130
|
+
Pure IR generation: no filesystem, no clang.
|
|
131
|
+
- `build(entryPath, options?)` → `BuildResult`. `options` accepts `output`,
|
|
132
|
+
`outDir`, `emit`, `optimize`, `force`, `verbose`, `extensions` (a registry),
|
|
133
|
+
`app`, `clang`, `preferPrebuilt`.
|
|
134
|
+
- `canonicalize` of results: `{ outputPath, irPath?, cached, diagnostics, ir?,
|
|
135
|
+
bundlePath? }`.
|
|
136
|
+
|
|
137
|
+
`build` reports source-level problems through `diagnostics` — always check
|
|
138
|
+
`diagnostics.some((d) => d.category === "error")` before using the artifact — but
|
|
139
|
+
**throws** `ToolchainError` when clang itself fails
|
|
140
|
+
([../../src/driver/toolchain.ts](../../src/driver/toolchain.ts)). The
|
|
141
|
+
`xbintsc/driver` subpath additionally exports the cache, toolchain resolution,
|
|
142
|
+
config loader, icon and macOS-bundle helpers.
|
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
# Contributing to the compiler
|
|
2
|
+
|
|
3
|
+
Use this page after [../AGENTS.md](../../AGENTS.md), which covers the layout, the
|
|
4
|
+
build gate and the rules. This one covers the workflow details that are easy to
|
|
5
|
+
get wrong.
|
|
6
|
+
|
|
7
|
+
## Set up
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm install
|
|
11
|
+
npm run typecheck # tsc --noEmit
|
|
12
|
+
npm run lint # eslint + the 600-line file budget
|
|
13
|
+
npm test # unit + end-to-end
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Node.js ≥ 22 is required to run the compiler from source, and a **clang 16+**
|
|
17
|
+
toolchain is required for anything that links a binary (see
|
|
18
|
+
[troubleshooting.md](./troubleshooting.md)). `xbintsc emit` needs neither clang
|
|
19
|
+
nor the runtime library, which makes it the fastest inner loop.
|
|
20
|
+
|
|
21
|
+
Optional but recommended — install the repository hook:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
git config core.hooksPath .githooks
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
`.githooks/pre-commit` **bumps the patch version on every commit** and folds
|
|
28
|
+
`package.json` + `package-lock.json` into that commit (`npm version patch
|
|
29
|
+
--no-git-tag-version`). Expect a version bump in your commits; do not countermand
|
|
30
|
+
it. Release tags are created by `release.yml`, never by the hook.
|
|
31
|
+
|
|
32
|
+
## The test suites
|
|
33
|
+
|
|
34
|
+
| Command | Scope |
|
|
35
|
+
| --- | --- |
|
|
36
|
+
| `npm test` | everything: per-module unit tests plus `tests/e2e` |
|
|
37
|
+
| `npm run test:e2e` | only the compile-and-run tests |
|
|
38
|
+
| `npm run test:watch` | vitest in watch mode |
|
|
39
|
+
| `npm run coverage` | V8 coverage of the TypeScript compiler (`src/`) |
|
|
40
|
+
| `npm run coverage:runtime` | LLVM coverage of the C runtime (`runtime/`) |
|
|
41
|
+
|
|
42
|
+
Unit tests live one directory per module under `tests/` (`lexer`, `parser`,
|
|
43
|
+
`binder`, `codegen`, `driver`, `extensions`, `cli`, `e2e`) and mirror the source
|
|
44
|
+
tree. Edge-case tests belong next to the module they cover.
|
|
45
|
+
|
|
46
|
+
`tests/e2e/` compiles real programs and runs the resulting binaries, including a
|
|
47
|
+
**differential harness** (`tests/e2e/differential-*.test.ts`) that executes the
|
|
48
|
+
same source through both xbintsc and Node and compares output byte-for-byte. That
|
|
49
|
+
harness is the strongest tool in the repository: when you touch semantics, add a
|
|
50
|
+
case there rather than asserting a hand-written expected string. Tests that need
|
|
51
|
+
clang skip themselves when it is unavailable (`hasClang()` in
|
|
52
|
+
[tests/helpers.ts](../../tests/helpers.ts)) — do not turn that into a silent
|
|
53
|
+
pass.
|
|
54
|
+
|
|
55
|
+
## Two invariants you must not break
|
|
56
|
+
|
|
57
|
+
### 1. The self-hosting fixpoint
|
|
58
|
+
|
|
59
|
+
CI (`self-host` job in [.github/workflows/ci.yml](../../.github/workflows/ci.yml))
|
|
60
|
+
compiles the compiler with itself and requires the emitted IR to be identical
|
|
61
|
+
across generations:
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
source --emit--> ref.ll
|
|
65
|
+
source --build--> gen1 binary
|
|
66
|
+
gen1 --emit--> gen2.ll # must equal ref.ll
|
|
67
|
+
gen1 --build--> gen2 binary
|
|
68
|
+
gen2 --emit--> gen3.ll # must equal ref.ll
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
By consequence: **any change that alters emitted IR must still be reproducible by
|
|
72
|
+
the compiler being changed**, and the IR must be byte-for-byte deterministic —
|
|
73
|
+
iteration order, generated symbol names and numbering included. Never introduce
|
|
74
|
+
nondeterminism (a `Map`/`Set` iteration over insertion-ordered data you did not
|
|
75
|
+
control, a timestamp, a filesystem-order dependency) into codegen.
|
|
76
|
+
|
|
77
|
+
A second consequence is the string model: the compiler's own source text has to
|
|
78
|
+
mean the same thing in every generation. A compiled string is UTF-8 bytes, so the
|
|
79
|
+
self-hosted `readFileSync` (`--ext node`) hands the scanner the bytes of a file,
|
|
80
|
+
while Node hands it the same file already decoded — read source files through
|
|
81
|
+
`decodeUtf8` ([src/diagnostics/utf8.ts](../../src/diagnostics/utf8.ts)) and never
|
|
82
|
+
assume a code unit is a byte or a character. Getting this wrong shows up as a
|
|
83
|
+
non-ASCII literal (`—` in a hint string) coming out re-encoded in `gen2.ll`, which
|
|
84
|
+
is why the emitted IR is compared byte-for-byte rather than line-by-line.
|
|
85
|
+
|
|
86
|
+
Verify locally:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
npx tsx src/cli/main.ts emit src/cli/main.ts --ext node > scratch/ref.ll
|
|
90
|
+
npx tsx src/cli/main.ts build src/cli/main.ts --ext node --out scratch/self --force
|
|
91
|
+
./scratch/self/main emit src/cli/main.ts --ext node > scratch/gen2.ll # .exe on Windows
|
|
92
|
+
diff scratch/ref.ll scratch/gen2.ll
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
### 2. The runtime ABI and value model
|
|
96
|
+
|
|
97
|
+
- Every compiled function uses
|
|
98
|
+
`xt_value fn(xt_value env, int32_t argc, xt_value *argv)` — direct calls and
|
|
99
|
+
closure calls share this one path, and `env` threads captured variables
|
|
100
|
+
through boxes.
|
|
101
|
+
- Every value is a single 64-bit word: doubles unboxed, everything else a tagged
|
|
102
|
+
pointer (16-bit tag + 48-bit payload). The representation is defined **once**
|
|
103
|
+
in [../../src/codegen/values.ts](../../src/codegen/values.ts) and
|
|
104
|
+
[../../runtime/rt.h](../../runtime/rt.h); change both together or not at all.
|
|
105
|
+
- Awkward JS semantics (`+` coercion, relational comparison, property access,
|
|
106
|
+
inspection) are delegated to `@xt_*` runtime calls rather than inlined.
|
|
107
|
+
- The GC is a non-moving mark-sweep collector: new heap objects go through
|
|
108
|
+
`xt_alloc`, and a value must be reachable from an explicit root slot, a
|
|
109
|
+
registered root provider (the event loop, the microtask queue) or the
|
|
110
|
+
conservative C-stack scan while it is live. A value held only in a C local that
|
|
111
|
+
is not on the scanned stack is a bug.
|
|
112
|
+
|
|
113
|
+
## Changing the runtime
|
|
114
|
+
|
|
115
|
+
`runtime/` is C, split by function across translation units (`xt_alloc.c`,
|
|
116
|
+
`xt_values.c`, `xt_containers.c`, `xt_stdlib.c`, …) sharing
|
|
117
|
+
`runtime/rt_internal.h`. The runtime must be rebuilt for C changes to take
|
|
118
|
+
effect:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
npm run runtime # tsx scripts/build-runtime.ts -> runtime/lib/<os>-<arch>/
|
|
122
|
+
npm run runtime:clean # drop build/runtime-obj and runtime/lib
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Add a new C file only together with the place that lists the runtime sources
|
|
126
|
+
(`RUNTIME_SOURCES` in [../../src/driver/compiler.ts](../../src/driver/compiler.ts)),
|
|
127
|
+
because that list decides what gets compiled and linked. Extension C sources are
|
|
128
|
+
declared by the extension itself.
|
|
129
|
+
|
|
130
|
+
## Adding or extending an extension
|
|
131
|
+
|
|
132
|
+
An extension is a plain object ([../../src/extensions/registry.ts](../../src/extensions/registry.ts))
|
|
133
|
+
contributing `runtimeSources()`, `modules()` / `builtins()`, and optionally
|
|
134
|
+
`nativeObjects()` or `assetLoaders()`. The Node extension is one folder per
|
|
135
|
+
module, pairing its TypeScript exports with the C sources that implement them:
|
|
136
|
+
|
|
137
|
+
```
|
|
138
|
+
src/extensions/node/fs/index.ts runtime/ext_node/fs/read_file.c
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Adding a Node module therefore means dropping a folder in each place and
|
|
142
|
+
registering it in the module list; the core compiler never changes. If your
|
|
143
|
+
feature is optional or platform-specific, it belongs in an extension rather than
|
|
144
|
+
in `src/codegen`. Bundled extensions are listed in
|
|
145
|
+
[../../src/extensions/catalog.ts](../../src/extensions/catalog.ts) — an extension
|
|
146
|
+
listed there but not enabled produces the actionable `pass --ext <name>` hint, so
|
|
147
|
+
add modules to the list even when the feature is off by default.
|
|
148
|
+
|
|
149
|
+
Extension authoring from C++/Rust without touching the compiler is documented in
|
|
150
|
+
[../../examples/extensions/README.md](../../examples/extensions/README.md).
|
|
151
|
+
|
|
152
|
+
## Adding a diagnostic
|
|
153
|
+
|
|
154
|
+
1. Add a stable code to the enum grouped by pipeline stage in
|
|
155
|
+
[../../src/diagnostics/diagnostic.ts](../../src/diagnostics/diagnostic.ts)
|
|
156
|
+
(1xxx lexer, 2xxx parser, 3xxx binder, 4xxx checker, 5xxx codegen, 6xxx
|
|
157
|
+
driver). Codes are a public interface — never renumber one.
|
|
158
|
+
2. Emit it with the most specific range you have; `formatDiagnostic` renders the
|
|
159
|
+
file, line, column, excerpt and caret from it.
|
|
160
|
+
3. Make the message actionable, in the style of the existing ones: say what is
|
|
161
|
+
wrong *and* what to do (`pass --ext node`, `use an ESM import`, …). If a
|
|
162
|
+
reader needs more, the CLI hint layer maps codes to a document — extend that
|
|
163
|
+
map in [../../src/cli/main.ts](../../src/cli/main.ts) when you add a code.
|
|
164
|
+
4. Cover it in `tests/diagnostics` or the module's own test.
|
|
165
|
+
|
|
166
|
+
## Documentation is part of the change
|
|
167
|
+
|
|
168
|
+
A feature is not done until the docs match:
|
|
169
|
+
|
|
170
|
+
| Document | Update when |
|
|
171
|
+
| --- | --- |
|
|
172
|
+
| [../implemented.md](../implemented.md) | a feature starts working |
|
|
173
|
+
| [../unimplemented.md](../unimplemented.md) | a limitation appears or is lifted |
|
|
174
|
+
| [../node-implemented.md](../node-implemented.md) / [../node-unimplemented.md](../node-unimplemented.md) | Node module coverage changes |
|
|
175
|
+
| [language-support.md](./language-support.md) | the AI-facing summary of the above changes |
|
|
176
|
+
| [../requirements.md](../requirements.md) | the toolchain or platform requirements change |
|
|
177
|
+
| [cli.md](./cli.md) | a flag, config field or environment variable changes |
|
|
178
|
+
|
|
179
|
+
Keep the Chinese translation (`doc/zh-CN/`, `doc/ai/zh-CN/`) in step with the
|
|
180
|
+
English source; a half-translated page is worse than none because it silently
|
|
181
|
+
goes stale.
|
|
182
|
+
|
|
183
|
+
## Repository hygiene
|
|
184
|
+
|
|
185
|
+
- **600 lines per code file**, enforced by ESLint (`max-lines`) for TS/JS and by
|
|
186
|
+
[scripts/check-file-length.ts](../../scripts/check-file-length.ts) for C, C++,
|
|
187
|
+
Rust, `.inc` and shell files. Split by responsibility instead of growing a
|
|
188
|
+
file; vendored sources are exempt.
|
|
189
|
+
- **No `any`**; `prefer-const` and `eqeqeq` are errors. The
|
|
190
|
+
declaration-merging pattern in the parser/generator is deliberately allowed.
|
|
191
|
+
- **Public API changes** go through [../../src/index.ts](../../src/index.ts) and
|
|
192
|
+
must keep the `xbintsc` / `xbintsc/driver` export maps in
|
|
193
|
+
[../../package.json](../../package.json) valid.
|
|
194
|
+
- `npm run package-release` assembles the per-platform archives into
|
|
195
|
+
`dist/release/`; the release workflow owns tagging, so never push a tag by
|
|
196
|
+
hand.
|