typenative 0.5.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/ci.yml +2 -2
- package/.github/workflows/npm-publish-github-packages.yml +2 -2
- package/CHANGELOG.md +339 -0
- package/CONTRIBUTION.md +2 -1
- package/README.md +16 -7
- package/bin/cli.js +1 -0
- package/bin/index.js +62 -3
- package/bin/main.js +1 -0
- package/bin/node/core.js +46 -0
- package/bin/node/events.js +154 -15
- package/bin/node/http.js +1265 -127
- package/bin/node/process.js +13 -0
- package/bin/node/punycode.js +2 -2
- package/bin/node/stream.js +56 -7
- package/bin/node/stream_web.js +2 -2
- package/bin/node/timers.js +4 -2
- package/bin/node/url.js +28 -3
- package/bin/parse-node.js +2 -2
- package/bin/regex_vm.js +2105 -0
- package/bin/transpiler.js +17970 -4419
- package/bin/typecheck-node.js +2 -1
- package/bin/version.js +1 -1
- package/conformance.md +57 -15
- package/examples/README.md +50 -28
- package/examples/angular-templates/README.md +141 -0
- package/examples/angular-templates/bench.mjs +173 -0
- package/examples/angular-templates/dom-type-globals.d.ts +21 -0
- package/examples/angular-templates/dom-type-stubs.ts +21 -0
- package/examples/angular-templates/main.ts +135 -0
- package/examples/angular-templates/parity.mjs +157 -0
- package/examples/angular-templates/run-node.mjs +69 -0
- package/examples/angular-templates/vendor/angular/compiler/LICENSE +21 -0
- package/examples/desktop-notes/README.md +11 -13
- package/examples/dupe-finder/README.md +7 -5
- package/examples/fibonacci/README.md +7 -7
- package/examples/hello-world/README.md +6 -6
- package/examples/json-server/README.md +22 -18
- package/examples/mandelbrot-png/README.md +11 -13
- package/examples/marked-cli/README.md +60 -44
- package/examples/marked-cli/main.ts +122 -4
- package/examples/openai-chat/README.md +148 -0
- package/examples/openai-chat/bench.mjs +175 -0
- package/examples/openai-chat/main.ts +488 -0
- package/examples/react-compiler/README.md +185 -0
- package/examples/react-compiler/bench.mjs +181 -0
- package/examples/react-compiler/main.ts +324 -0
- package/examples/react-compiler/parity.mjs +362 -0
- package/examples/react-compiler/run-node.mjs +90 -0
- package/examples/react-compiler/vendor/babel/LICENSE +22 -0
- package/examples/react-compiler/vendor/babel/helper-globals/data/builtin-lower.json +15 -0
- package/examples/react-compiler/vendor/babel/helper-globals/data/builtin-upper.json +51 -0
- package/examples/react-compiler/vendor/charcodes/package.json +25 -0
- package/examples/react-compiler/vendor/invariant/LICENSE +21 -0
- package/examples/react-compiler/vendor/invariant/invariant.cjs +51 -0
- package/examples/react-compiler/vendor/invariant/invariant.d.cts +11 -0
- package/examples/react-compiler/vendor/invariant/types/LICENSE +21 -0
- package/examples/react-compiler/vendor/jridgewell/gen-mapping/LICENSE +19 -0
- package/examples/react-compiler/vendor/jridgewell/resolve-uri/LICENSE +19 -0
- package/examples/react-compiler/vendor/jridgewell/sourcemap-codec/LICENSE +19 -0
- package/examples/react-compiler/vendor/jridgewell/trace-mapping/LICENSE +19 -0
- package/examples/react-compiler/vendor/js-tokens/LICENSE +21 -0
- package/examples/react-compiler/vendor/js-tokens/index.cjs +23 -0
- package/examples/react-compiler/vendor/js-tokens/index.d.cts +13 -0
- package/examples/react-compiler/vendor/jsesc/LICENSE-MIT.txt +20 -0
- package/examples/react-compiler/vendor/jsesc/jsesc.cjs +337 -0
- package/examples/react-compiler/vendor/jsesc/jsesc.d.cts +384 -0
- package/examples/react-compiler/vendor/jsesc/types/LICENSE +21 -0
- package/examples/react-compiler/vendor/picocolors/LICENSE +15 -0
- package/examples/react-compiler/vendor/picocolors/picocolors.cjs +75 -0
- package/examples/react-compiler/vendor/picocolors/picocolors.d.cts +5 -0
- package/examples/react-compiler/vendor/picocolors/types.d.ts +51 -0
- package/examples/react-compiler/vendor/react-compiler/LICENSE +21 -0
- package/examples/react-compiler/vendor/stubs/babel-core.ts +31 -0
- package/examples/react-compiler/vendor/stubs/debug.ts +18 -0
- package/examples/react-compiler/vendor/stubs/js-tokens.d.ts +13 -0
- package/examples/react-compiler/vendor/stubs/pretty-format.ts +18 -0
- package/examples/react-compiler/vendor/zod/LICENSE +21 -0
- package/examples/react-compiler/vendor/zod-validation-error/LICENSE +9 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/utils/NonEmptyArray.ts +5 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/utils/joinPath.ts +49 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/utils/joinValues.ts +48 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/utils/stringify.ts +50 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/utils/titleCase.ts +6 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/MessageBuilder.ts +126 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/ValidationError.ts +39 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/custom.ts +12 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/errorMap.ts +78 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/index.ts +2 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidElement.ts +12 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidKey.ts +12 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidStringFormat.ts +131 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidType.ts +54 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidUnion.ts +12 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/invalidValue.ts +40 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/notMultipleOf.ts +12 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/tooBig.ts +76 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/tooSmall.ts +76 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/types.ts +27 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/errorMap/unrecognizedKeys.ts +27 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/fromError.ts +13 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/fromZodError.ts +61 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/fromZodIssue.ts +38 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/index.ts +20 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/isValidationError.ts +5 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/isValidationErrorLike.ts +8 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/isZodErrorLike.ts +11 -0
- package/examples/react-compiler/vendor/zod-validation-error/lib/v4/toValidationError.ts +20 -0
- package/examples/react-compiler/vendor.mjs +688 -0
- package/examples/todo-cli/README.md +9 -9
- package/examples/yaml-cli/README.md +50 -34
- package/examples/yaml-cli/main.ts +236 -14
- package/package.json +8 -5
- package/performance/README.md +196 -0
- package/performance/any-dynamic.ts +21 -0
- package/performance/any-record-view.ts +20 -0
- package/performance/arr-hof.ts +21 -0
- package/performance/arr-push-index.ts +15 -0
- package/performance/arr-sort.ts +21 -0
- package/performance/async-await.ts +24 -0
- package/performance/class-dispatch.ts +52 -0
- package/performance/events-emit.ts +27 -0
- package/performance/exc-throw-rare.ts +25 -0
- package/performance/exc-try-catch.ts +26 -0
- package/performance/fn-closures.ts +29 -0
- package/performance/fn-recursion-tree.ts +31 -0
- package/performance/gen-for-of.ts +46 -0
- package/performance/json-roundtrip.ts +27 -0
- package/performance/map-delete.ts +24 -0
- package/performance/map-string-keys.ts +22 -0
- package/performance/num-arith-loop.ts +14 -0
- package/performance/num-bitwise.ts +21 -0
- package/performance/num-mod-int.ts +16 -0
- package/performance/num-to-string.ts +16 -0
- package/performance/obj-alloc.ts +32 -0
- package/performance/regex-exec.ts +29 -0
- package/performance/regex-site.ts +28 -0
- package/performance/set-ops.ts +20 -0
- package/performance/str-char-scan.ts +35 -0
- package/performance/str-concat-plus.ts +18 -0
- package/performance/str-concat.ts +24 -0
- package/performance/str-short-scan.ts +26 -0
- package/performance/str-split-join.ts +23 -0
- package/performance/str-template.ts +16 -0
- package/performance/union-switch.ts +57 -0
- package/tsparser/check.go +2 -1
- package/tsparser/main.go +1 -0
- package/types/typenative-npm.d.ts +3 -2
- package/types/typenative.d.ts +42 -12
- package/ROUND2-HANDOFF.md +0 -200
|
@@ -61,24 +61,24 @@ Reproduce with `node bench.mjs [n=35] [runs=15]`. The script compiles
|
|
|
61
61
|
reports two numbers per run: the program's own in-process compute time
|
|
62
62
|
(parsed from its output) and the total wall time including process startup.
|
|
63
63
|
|
|
64
|
-
**Environment:** Windows 11 Pro, AMD Ryzen 5
|
|
65
|
-
TypeNative 0.
|
|
64
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
65
|
+
TypeNative 0.5.0 (unreleased changes), default (non-`--release`) build.
|
|
66
66
|
|
|
67
67
|
Compute time, `fib(35)` over 15 runs (measured in-program):
|
|
68
68
|
|
|
69
69
|
| Runtime | min | median | mean | max |
|
|
70
70
|
|------------|-----------|-----------|-----------|-----------|
|
|
71
|
-
| Node.js |
|
|
72
|
-
| TypeNative |
|
|
71
|
+
| Node.js | 72.86 ms | 73.81 ms | 73.89 ms | 75.60 ms |
|
|
72
|
+
| TypeNative | 33.94 ms | 36.37 ms | 37.32 ms | 48.15 ms |
|
|
73
73
|
|
|
74
74
|
Total wall time per run, including startup (spawn to exit):
|
|
75
75
|
|
|
76
76
|
| Runtime | min | median | mean | max |
|
|
77
77
|
|------------|-----------|-----------|-----------|-----------|
|
|
78
|
-
| Node.js |
|
|
79
|
-
| TypeNative |
|
|
78
|
+
| Node.js | 129.75 ms | 134.23 ms | 135.53 ms | 144.45 ms |
|
|
79
|
+
| TypeNative | 42.45 ms | 45.12 ms | 53.19 ms | 153.97 ms |
|
|
80
80
|
|
|
81
|
-
**TypeNative computes
|
|
81
|
+
**TypeNative computes 2.0x faster** and finishes the whole run **3.0x
|
|
82
82
|
faster**. The recursive calls, arithmetic, and number boxing all compile
|
|
83
83
|
ahead of time to native machine code via Go, while V8 must tier up through
|
|
84
84
|
its JIT before it reaches comparable speed — and never reaches it here, since
|
|
@@ -47,16 +47,16 @@ the TypeNative CLI, then times **complete process runs** (spawn to exit) of
|
|
|
47
47
|
the same program under both runtimes. Since the program does no work, the
|
|
48
48
|
numbers are almost entirely startup time.
|
|
49
49
|
|
|
50
|
-
**Environment:** Windows 11 Pro, AMD Ryzen 5
|
|
51
|
-
TypeNative 0.
|
|
50
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
51
|
+
TypeNative 0.5.0 (unreleased changes), default (non-`--release`) build.
|
|
52
52
|
|
|
53
53
|
| Runtime | min | median | mean | max |
|
|
54
54
|
|------------|------------|------------|------------|------------|
|
|
55
|
-
| Node.js |
|
|
56
|
-
| TypeNative |
|
|
55
|
+
| Node.js | 54.40 ms | 62.40 ms | 62.86 ms | 78.77 ms |
|
|
56
|
+
| TypeNative | 6.42 ms | 7.38 ms | 10.98 ms | 112.02 ms |
|
|
57
57
|
|
|
58
|
-
**TypeNative starts
|
|
59
|
-
self-contained 2.
|
|
58
|
+
**TypeNative starts 8.5x faster** (median). The compiled binary is a
|
|
59
|
+
self-contained 2.7 MB executable: there is no runtime to boot, no module
|
|
60
60
|
system to initialize, and no JIT to warm up, so process-to-output latency is
|
|
61
61
|
an order of magnitude lower. For CLI tools that are invoked frequently, this
|
|
62
62
|
is the difference users notice.
|
|
@@ -100,28 +100,32 @@ response) and `POST /todos` (10,000 requests).
|
|
|
100
100
|
The load generator is built to stay out of the way, like `wrk` or
|
|
101
101
|
`autocannon`: 8 worker threads × 2 raw keep-alive sockets send prebuilt
|
|
102
102
|
request bytes and read each response just far enough to find its end. A
|
|
103
|
-
plain `node:http` client
|
|
103
|
+
plain `node:http` client would cap out far below either server, so
|
|
104
104
|
with it both servers would simply tie at the client's limit.
|
|
105
105
|
|
|
106
|
-
**Environment:** Windows 11 Pro, AMD Ryzen 5
|
|
107
|
-
TypeNative 0.
|
|
108
|
-
same machine. Re-run variation is about ±
|
|
109
|
-
1.
|
|
106
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
107
|
+
TypeNative 0.6.1, default (non-`--release`) build, client
|
|
108
|
+
and server on the same machine. Re-run variation is about ±20%; across runs
|
|
109
|
+
TypeNative was 1.6–1.8x Node on GET and 1.6–1.8x on POST. The numbers below are
|
|
110
|
+
medians over three runs.
|
|
110
111
|
|
|
111
112
|
| Runtime | Endpoint | req/s | p50 (ms) | p99 (ms) |
|
|
112
113
|
|------------|-------------|-------:|---------:|---------:|
|
|
113
|
-
| Node.js | GET /todos |
|
|
114
|
-
| TypeNative | GET /todos |
|
|
115
|
-
| Node.js | POST /todos |
|
|
116
|
-
| TypeNative | POST /todos |
|
|
117
|
-
|
|
118
|
-
**TypeNative serves ~1.
|
|
119
|
-
latency on both routes.
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
114
|
+
| Node.js | GET /todos | 61957 | 0.23 | 0.63 |
|
|
115
|
+
| TypeNative | GET /todos | 106483 | 0.14 | 0.38 |
|
|
116
|
+
| Node.js | POST /todos | 68097 | 0.21 | 0.46 |
|
|
117
|
+
| TypeNative | POST /todos | 114066 | 0.13 | 0.40 |
|
|
118
|
+
|
|
119
|
+
**TypeNative serves ~1.7x more requests per second here**, with lower median
|
|
120
|
+
latency on both routes. The compiled server reads requests and writes
|
|
121
|
+
responses on each connection's goroutine, and a handler that answers
|
|
122
|
+
synchronously (`writeHead` + `end`, as both routes do) builds the response
|
|
123
|
+
bytes right there, in one write: it never queues an operation for the event
|
|
124
|
+
loop, which keeps the per-request hand-off to JavaScript to a single baton
|
|
125
|
+
pass. JavaScript still runs one thing at a time, but at 105–115k req/s the
|
|
126
|
+
parallel socket I/O and HTTP handling stay ahead of that one thread.
|
|
127
|
+
|
|
128
|
+
On top of that, the binary starts in ~7 ms instead of ~60 ms and is a
|
|
129
|
+
single 11.4 MB executable with zero dependencies — `scp` it to a server and
|
|
126
130
|
run it, no Node install, no `node_modules` — and CPU-heavy work inside a
|
|
127
131
|
handler runs ~2x faster (see the [fibonacci example](../fibonacci/README.md)).
|
|
@@ -4,7 +4,7 @@ Renders the Mandelbrot set and writes it as a PNG. The PNG encoder is about
|
|
|
4
4
|
40 lines of TypeScript on top of `node:zlib` (`deflateSync` for the pixel
|
|
5
5
|
data, `crc32` for the chunk checksums). The program needs no image library,
|
|
6
6
|
no native addon such as `sharp` or `canvas`, and no `node_modules`: the
|
|
7
|
-
compiled binary is a single
|
|
7
|
+
compiled binary is a single 4.2 MB file.
|
|
8
8
|
|
|
9
9
|
Demonstrates: a tight floating-point loop, a function returning a tuple that
|
|
10
10
|
the caller destructures, `Uint8Array` / `DataView` big-endian writes,
|
|
@@ -46,20 +46,18 @@ script compiles `main.ts`, times **complete process runs** (spawn to exit) of
|
|
|
46
46
|
both runtimes rendering the same image, then decodes both PNGs (checking every
|
|
47
47
|
chunk CRC) and compares the pixels.
|
|
48
48
|
|
|
49
|
-
**Environment:** Windows 11 Pro, AMD Ryzen 5
|
|
50
|
-
TypeNative 0.
|
|
49
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
50
|
+
TypeNative 0.5.0 (unreleased changes), default (non-`--release`) build.
|
|
51
51
|
|
|
52
52
|
| Runtime | min | median | mean | max |
|
|
53
53
|
|------------|-----------|-----------|-----------|-----------|
|
|
54
|
-
| Node.js |
|
|
55
|
-
| TypeNative |
|
|
56
|
-
|
|
57
|
-
**TypeNative is 1.
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
iteration into machine code about as good as Go's. The difference comes from
|
|
62
|
-
startup (~100 ms) and slightly faster deflate. For recursion and
|
|
54
|
+
| Node.js | 499.84 ms | 513.64 ms | 508.85 ms | 518.01 ms |
|
|
55
|
+
| TypeNative | 449.83 ms | 452.28 ms | 466.67 ms | 599.75 ms |
|
|
56
|
+
|
|
57
|
+
**TypeNative is 1.1x faster end to end** (median). There's no JIT warm-up.
|
|
58
|
+
The render loop itself takes about the same time in both runtimes. V8 compiles a tight float loop like the escape-time
|
|
59
|
+
iteration into machine code about as good as Go's. The difference is mostly
|
|
60
|
+
Node's startup. For recursion and
|
|
63
61
|
call-heavy code the compute gap is larger (see [fibonacci](../fibonacci/),
|
|
64
|
-
|
|
62
|
+
2.0x). This example's main advantage is distribution: one dependency-free
|
|
65
63
|
binary that writes images.
|
|
@@ -92,68 +92,84 @@ checks that the two HTML outputs are identical:
|
|
|
92
92
|
emphasis, links and code spans, lists, tables, code fences and block quotes,
|
|
93
93
|
from a fixed-seed generator, so every machine gets the same input).
|
|
94
94
|
|
|
95
|
-
**Environment:**
|
|
96
|
-
Node
|
|
97
|
-
(non-`--release`) build. The binary is 6.
|
|
95
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600 (6 cores, 12 threads),
|
|
96
|
+
Node v26.13.0, Go 1.27.4, TypeNative 0.5.0 (unreleased changes), default
|
|
97
|
+
(non-`--release`) build. The binary is 6.5 MB.
|
|
98
98
|
|
|
99
99
|
(a) `marked < README.md` (3.5 KB), 30 runs each:
|
|
100
100
|
|
|
101
101
|
| Runtime | min | median | mean | max |
|
|
102
102
|
| ---------- | --------- | --------- | --------- | --------- |
|
|
103
|
-
| Node.js |
|
|
104
|
-
| TypeNative |
|
|
103
|
+
| Node.js | 101.08 ms | 103.58 ms | 106.68 ms | 119.02 ms |
|
|
104
|
+
| TypeNative | 11.98 ms | 12.67 ms | 20.14 ms | 235.52 ms |
|
|
105
105
|
|
|
106
106
|
(b) `marked < generated.md` (10.0 MB, 114,893 lines), 5 runs each:
|
|
107
107
|
|
|
108
|
-
| Runtime | min
|
|
109
|
-
| ---------- |
|
|
110
|
-
| Node.js |
|
|
111
|
-
| TypeNative |
|
|
108
|
+
| Runtime | min | median | mean | max | MB/s (median) |
|
|
109
|
+
| ---------- | -------- | -------- | -------- | -------- | ------------- |
|
|
110
|
+
| Node.js | 585.94 ms | 595.71 ms | 592.98 ms | 597.79 ms | 16.79 |
|
|
111
|
+
| TypeNative | 500.79 ms | 511.86 ms | 509.63 ms | 513.24 ms | 19.54 |
|
|
112
112
|
|
|
113
|
-
**Per call, TypeNative is
|
|
113
|
+
**Per call, TypeNative is 8.2x faster** (12.7 ms vs 103.6 ms, median). Node's
|
|
114
114
|
cost here is mostly startup: it loads and type-strips the 13 TypeScript files
|
|
115
115
|
and compiles marked's regexes on every run. That's what you feel when a docs
|
|
116
116
|
build or a hook renders one file per process.
|
|
117
117
|
|
|
118
|
-
**On bulk throughput, TypeNative is
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
The ReDoS inputs
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
quadratic in both engines and the backtracker's constant factor shows.
|
|
118
|
+
**On bulk throughput, TypeNative is 1.2x faster** (512 ms vs 596 ms). A
|
|
119
|
+
loaded document is cut before top-level ATX headings that follow a blank line
|
|
120
|
+
and are outside a code fence; up to six worker processes each parse one
|
|
121
|
+
contiguous run of sections, and their HTML is concatenated. Parsing the
|
|
122
|
+
sections separately is byte-for-byte identical to parsing the whole document
|
|
123
|
+
here (bench.mjs checks it), because marked renders top-level blocks
|
|
124
|
+
independently. `main.ts` falls back to one process for inputs under 512 KB, for
|
|
125
|
+
documents with reference link definitions, raw HTML blocks or CRLF line
|
|
126
|
+
endings, and for `--tokens`; `parity.mjs`'s spec cases all take that path, so
|
|
127
|
+
their behavior is unchanged. The workers pass their assignment in `workerData`
|
|
128
|
+
and return gzipped HTML through one temporary file: the worker IPC carries a
|
|
129
|
+
single compact message, not several megabytes.
|
|
130
|
+
|
|
131
|
+
**A single process is still about 1.5x slower than Node on 10 MB.** The
|
|
132
|
+
parallel path is what closes the gap; per section the remaining cost is
|
|
133
|
+
TypeNative's regex VM, which interprets marked's patterns where V8's Irregexp
|
|
134
|
+
compiles them to machine code, plus the object and match-array allocation each
|
|
135
|
+
token costs. The fixes below cut the single-process time from 1,890 ms to
|
|
136
|
+
about 1,620 ms before the parallel path; the parallel run then has six of those
|
|
137
|
+
processes working on independent sections.
|
|
138
|
+
|
|
139
|
+
The ReDoS inputs now favor the native binary: all of them together take 4.7 s
|
|
140
|
+
natively and 13 s under Node, though most of Node's time there is starting up
|
|
141
|
+
once per input. The slowest native case, `quadratic_inline_masking`, takes
|
|
142
|
+
0.7 s against Node's 0.9 s.
|
|
144
143
|
|
|
145
144
|
Getting here fixed performance bugs in TypeNative's runtime and compiler, all
|
|
146
145
|
general:
|
|
147
146
|
|
|
148
|
-
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
147
|
+
- Every regex now runs on one engine, a backtracking VM with JS semantics, in
|
|
148
|
+
place of Go's RE2 for simple patterns and a closure-based matcher for the
|
|
149
|
+
rest. A match allocates only its result. First-byte sets, literal prefixes
|
|
150
|
+
and anchors reject most calls without running anything, which matters here
|
|
151
|
+
because 83% of marked's 1.5 million regex calls per MB miss. RE2 remains as a
|
|
152
|
+
fallback: a pattern it can run that exhausts the VM's step budget switches to
|
|
153
|
+
it, so those patterns keep their linear-time guarantee.
|
|
154
|
+
- Regex literals are compiled once, however many times they're evaluated.
|
|
155
|
+
- `a += b` on a long string held as any is amortized: the string is extended
|
|
156
|
+
in place when it ends where its buffer's written part ends. It used to copy
|
|
157
|
+
on every append, which made the worst ReDoS input take over 10 s.
|
|
156
158
|
- Strings built with `+=` in a loop are `strings.Builder`s, so rendering a
|
|
157
159
|
long document is linear.
|
|
160
|
+
- Short ASCII strings need no code-point index, and slices of a long non-ASCII
|
|
161
|
+
string share its index instead of rebuilding it.
|
|
158
162
|
- Type inference is memoized per query. marked's
|
|
159
163
|
`edit(re).replace(…).replace(…)…` chains took exponential time to compile.
|
|
164
|
+
- `[...s].length`, `[...s][i]` and `[...s].at(i)` are the code point count, the
|
|
165
|
+
i-th code point and the i-th code point (from the end), without building an
|
|
166
|
+
array of every rune; marked uses those for every emphasis delimiter.
|
|
167
|
+
- `re.exec(s)` where only its presence is tested (`!!re.exec(s)`, an `if`
|
|
168
|
+
condition) runs `re.test(s)`: no match array is built.
|
|
169
|
+
- A nullable regex result is boxed only when it matched (`TnSlicePtr`), not by
|
|
170
|
+
a closure and a heap box on every miss.
|
|
171
|
+
- Greedy runs of "everything but one ASCII byte" (`[^\\n]+`, `[^*]+`) use
|
|
172
|
+
`IndexByte` instead of testing each byte.
|
|
173
|
+
- The ASCII cache for long strings is a mutex-guarded entry, not a new atomic
|
|
174
|
+
entry per inline token.
|
|
175
|
+
- Regex replacement builders are grown once to the input's length.
|
|
@@ -21,11 +21,14 @@
|
|
|
21
21
|
// 64 KB).
|
|
22
22
|
import { promises } from 'node:fs';
|
|
23
23
|
import { resolve } from 'node:path';
|
|
24
|
-
import { homedir } from 'node:os';
|
|
24
|
+
import { availableParallelism, homedir, tmpdir } from 'node:os';
|
|
25
|
+
import { Buffer } from 'node:buffer';
|
|
26
|
+
import { gzipSync, gunzipSync } from 'node:zlib';
|
|
27
|
+
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
|
|
25
28
|
import { marked } from './vendor/marked/src/marked.ts';
|
|
26
29
|
import type { MarkedExtension, MarkedOptions } from './vendor/marked/src/marked.ts';
|
|
27
30
|
|
|
28
|
-
const { access, readFile, writeFile } = promises;
|
|
31
|
+
const { access, readFile, unlink, writeFile } = promises;
|
|
29
32
|
|
|
30
33
|
const VERSION = '18.1.2';
|
|
31
34
|
|
|
@@ -55,6 +58,117 @@ interface CliError {
|
|
|
55
58
|
message: string;
|
|
56
59
|
}
|
|
57
60
|
|
|
61
|
+
// Large documents are parsed in a few processes, one contiguous run of
|
|
62
|
+
// sections each: marked renders top-level blocks independently, so the
|
|
63
|
+
// concatenation is the whole document's HTML (checked byte for byte by
|
|
64
|
+
// bench.mjs). Splits happen only before a top-level ATX heading that follows
|
|
65
|
+
// a blank line and is outside a code fence; documents with reference link
|
|
66
|
+
// definitions, raw HTML blocks or CRLF line endings take the sequential path,
|
|
67
|
+
// as the pieces would not stand on their own. Between the processes a section
|
|
68
|
+
// run travels in workerData, and its HTML comes back gzipped through one small
|
|
69
|
+
// temporary file (the worker IPC handles one compact message, not megabytes).
|
|
70
|
+
const PARALLEL_MIN_BYTES = 512 * 1024;
|
|
71
|
+
|
|
72
|
+
function splitForParallel(data: string, options: MarkedOptions): string[] | undefined {
|
|
73
|
+
if (data.length < PARALLEL_MIN_BYTES || data.includes('\r')) return undefined;
|
|
74
|
+
if (options.extensions || options.hooks || options.walkTokens) return undefined;
|
|
75
|
+
// a reference definition is shared by the whole document
|
|
76
|
+
if (/(^|\n) {0,3}\[[^\]\n]+\]:/.test(data)) return undefined;
|
|
77
|
+
// a cut line inside one of these would not be a top-level heading
|
|
78
|
+
if (/<(?:pre|script|style|textarea)\b/i.test(data) || data.includes('<!--') || data.includes('<?') || data.includes('<![CDATA[')) return undefined;
|
|
79
|
+
|
|
80
|
+
const lines = data.split('\n');
|
|
81
|
+
const cuts: number[] = [0];
|
|
82
|
+
let fence = '';
|
|
83
|
+
for (let i = 0; i < lines.length; i++) {
|
|
84
|
+
const line = lines[i];
|
|
85
|
+
const open = /^ {0,3}(`{3,}|~{3,})/.exec(line);
|
|
86
|
+
if (open) {
|
|
87
|
+
if (!fence) fence = open[1][0];
|
|
88
|
+
else if (open[1][0] === fence) fence = '';
|
|
89
|
+
continue;
|
|
90
|
+
}
|
|
91
|
+
if (fence) continue;
|
|
92
|
+
if (i > 0 && lines[i - 1] === '' && /^#{1,6} /.test(line)) cuts.push(i);
|
|
93
|
+
}
|
|
94
|
+
cuts.push(lines.length);
|
|
95
|
+
if (cuts.length < 3) return undefined;
|
|
96
|
+
|
|
97
|
+
// One contiguous run of chunks per process, balanced by size
|
|
98
|
+
const chunks: string[] = [];
|
|
99
|
+
let total = 0;
|
|
100
|
+
for (let i = 0; i + 1 < cuts.length; i++) {
|
|
101
|
+
const chunk = lines.slice(cuts[i], cuts[i + 1]).join('\n');
|
|
102
|
+
chunks.push(chunk);
|
|
103
|
+
total += chunk.length;
|
|
104
|
+
}
|
|
105
|
+
const requested = Number(process.env.MARKED_WORKERS ?? '0') || availableParallelism();
|
|
106
|
+
const count = Math.min(6, Math.max(1, requested));
|
|
107
|
+
if (count < 2) return undefined;
|
|
108
|
+
const parts: string[] = [];
|
|
109
|
+
let at = 0;
|
|
110
|
+
let size = 0;
|
|
111
|
+
for (let m = 0; m < count && at < chunks.length; m++) {
|
|
112
|
+
const target = total * (m + 1) / count;
|
|
113
|
+
const start = at;
|
|
114
|
+
while (at < chunks.length && (at === start || size < target)) {
|
|
115
|
+
size += chunks[at].length;
|
|
116
|
+
at++;
|
|
117
|
+
}
|
|
118
|
+
parts.push(chunks.slice(start, at).join('\n'));
|
|
119
|
+
}
|
|
120
|
+
return parts.length > 1 ? parts : undefined;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
interface ParseJob {
|
|
124
|
+
text: string;
|
|
125
|
+
output: string;
|
|
126
|
+
options: MarkedOptions;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
async function parseHtml(data: string, options: MarkedOptions): Promise<string> {
|
|
130
|
+
const parts = splitForParallel(data, options);
|
|
131
|
+
if (!parts) return await marked.parse(data, options);
|
|
132
|
+
const base = tmpdir() + '/marked-' + process.pid + '-' + Date.now();
|
|
133
|
+
const htmls: string[] = await Promise.all(parts.map((text, index) => new Promise<string>((resolve, reject) => {
|
|
134
|
+
const output = base + '-' + index + '.html';
|
|
135
|
+
const cleanup = () => unlink(output).catch(() => {});
|
|
136
|
+
(async () => {
|
|
137
|
+
try {
|
|
138
|
+
const worker = new Worker(new URL(import.meta.url), { workerData: { text, output, options } as ParseJob });
|
|
139
|
+
worker.on('message', async () => {
|
|
140
|
+
try {
|
|
141
|
+
resolve(gunzipSync(await readFile(output)).toString('utf8'));
|
|
142
|
+
} catch (err) {
|
|
143
|
+
reject(err);
|
|
144
|
+
} finally {
|
|
145
|
+
cleanup();
|
|
146
|
+
}
|
|
147
|
+
worker.terminate();
|
|
148
|
+
});
|
|
149
|
+
worker.on('error', (err) => {
|
|
150
|
+
cleanup();
|
|
151
|
+
reject(err);
|
|
152
|
+
});
|
|
153
|
+
} catch (err) {
|
|
154
|
+
cleanup();
|
|
155
|
+
reject(err);
|
|
156
|
+
}
|
|
157
|
+
})();
|
|
158
|
+
}))) as string[];
|
|
159
|
+
return htmls.join('');
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
async function workerMain() {
|
|
163
|
+
const job = workerData as ParseJob;
|
|
164
|
+
const options = job.options as MarkedOptions;
|
|
165
|
+
const text = job.text as string;
|
|
166
|
+
const output = job.output as string;
|
|
167
|
+
await writeFile(output, gzipSync(Buffer.from(await marked.parse(text, options), 'utf8')));
|
|
168
|
+
parentPort!.postMessage('ok');
|
|
169
|
+
parentPort!.close();
|
|
170
|
+
}
|
|
171
|
+
|
|
58
172
|
export async function main() {
|
|
59
173
|
async function help() {
|
|
60
174
|
console.log(HELP);
|
|
@@ -223,7 +337,7 @@ export async function main() {
|
|
|
223
337
|
|
|
224
338
|
const html = tokens
|
|
225
339
|
? JSON.stringify(marked.lexer(data, options as MarkedOptions), null, 2)
|
|
226
|
-
: await
|
|
340
|
+
: await parseHtml(data, options as MarkedOptions);
|
|
227
341
|
|
|
228
342
|
if (output) {
|
|
229
343
|
try {
|
|
@@ -289,4 +403,8 @@ export async function main() {
|
|
|
289
403
|
}
|
|
290
404
|
}
|
|
291
405
|
|
|
292
|
-
|
|
406
|
+
if (isMainThread) {
|
|
407
|
+
main();
|
|
408
|
+
} else {
|
|
409
|
+
workerMain();
|
|
410
|
+
}
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
# OpenAI-compatible chat CLI
|
|
2
|
+
|
|
3
|
+
A terminal chat client for any OpenAI-compatible API. Answers **stream into
|
|
4
|
+
the terminal token by token** while the model is still writing them. It works
|
|
5
|
+
with the free OpenRouter API out of the box — no payment, just a key from
|
|
6
|
+
[openrouter.ai/keys](https://openrouter.ai/keys) — and with OpenAI, Ollama,
|
|
7
|
+
LM Studio, vLLM, or any other server that speaks `/v1/chat/completions`.
|
|
8
|
+
|
|
9
|
+
Demonstrates: WHATWG `fetch` with a **streaming response body**
|
|
10
|
+
(`response.body.getReader()` read in a loop), hand-parsed server-sent events
|
|
11
|
+
(`data:` lines, comments, `[DONE]`), `TextDecoder` with `{ stream: true }`,
|
|
12
|
+
`AbortController` + `setTimeout` as a request timeout, `node:readline/promises`
|
|
13
|
+
driven by `for await`, `JSON.parse` on each chunk, `process.env` / `argv`
|
|
14
|
+
handling, and writing deltas to `process.stdout` the moment they arrive — all
|
|
15
|
+
in one dependency-free file that runs unchanged under Node and as a compiled
|
|
16
|
+
binary.
|
|
17
|
+
|
|
18
|
+
## Usage
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
openai-chat # interactive multi-turn REPL
|
|
22
|
+
openai-chat "Explain SSE in one sentence" # one question, streamed answer, exit
|
|
23
|
+
openai-chat --model google/gemma-4-31b-it:free
|
|
24
|
+
openai-chat --system "Answer in as few words as possible"
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
In the REPL each message sees the whole conversation. Slash commands:
|
|
28
|
+
`/exit` (or Ctrl+D) quits, `/clear` resets the conversation,
|
|
29
|
+
`/system <text>` sets or replaces the system prompt. Ctrl+C aborts the answer
|
|
30
|
+
in flight without killing the session.
|
|
31
|
+
|
|
32
|
+
| Option | Meaning | Default |
|
|
33
|
+
|--------|---------|---------|
|
|
34
|
+
| `--model <id>` | Model id | provider default (`google/gemma-4-31b-it:free` on OpenRouter, `gpt-4o-mini` on OpenAI) |
|
|
35
|
+
| `--base-url <url>` | API root | `https://openrouter.ai/api/v1`, or `https://api.openai.com/v1` when only `OPENAI_API_KEY` is set |
|
|
36
|
+
| `--api-key <key>` | API key | `OPENROUTER_API_KEY`, else `OPENAI_API_KEY` |
|
|
37
|
+
| `--system <prompt>` | System prompt | none |
|
|
38
|
+
| `--max-tokens <n>` | Cap on generated tokens | provider default |
|
|
39
|
+
|
|
40
|
+
Other OpenAI-compatible servers, no account needed:
|
|
41
|
+
|
|
42
|
+
```sh
|
|
43
|
+
ollama serve # then:
|
|
44
|
+
openai-chat --base-url http://127.0.0.1:11434/v1 --api-key ollama --model llama3.2
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
A one-shot run keeps stdout clean — only the answer goes to stdout, the
|
|
48
|
+
token/time stats go to stderr — so piping works:
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
openai-chat "write a haiku about go channels" | tee haiku.txt
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Run it
|
|
55
|
+
|
|
56
|
+
Get a free OpenRouter key, then:
|
|
57
|
+
|
|
58
|
+
```sh
|
|
59
|
+
# sh
|
|
60
|
+
export OPENROUTER_API_KEY=sk-or-...
|
|
61
|
+
node main.ts "hello!" # under Node.js 24+
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
# with TypeNative, build one binary and run it
|
|
66
|
+
npx typenative --source main.ts --output dist/openai-chat.exe --release
|
|
67
|
+
./dist/openai-chat.exe
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
From a checkout of this repository:
|
|
71
|
+
|
|
72
|
+
```sh
|
|
73
|
+
node bin/cli.js --source examples/openai-chat/main.ts --output dist/openai-chat.exe
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## How the streaming works
|
|
77
|
+
|
|
78
|
+
`streamChat` POSTs the conversation with `stream: true` and
|
|
79
|
+
`stream_options: { include_usage: true }`. The response body is a
|
|
80
|
+
server-sent-event stream, read incrementally:
|
|
81
|
+
|
|
82
|
+
```ts
|
|
83
|
+
const reader = response.body.getReader();
|
|
84
|
+
const decoder = new TextDecoder();
|
|
85
|
+
while (!finished) {
|
|
86
|
+
const { value, done } = await reader.read();
|
|
87
|
+
if (done) break;
|
|
88
|
+
buffer += decoder.decode(value, { stream: true }); // don't split UTF-8 mid-character
|
|
89
|
+
// ... peel off complete lines, collect `data:` lines, dispatch on blank lines
|
|
90
|
+
}
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
One SSE event is one or more `data:` lines ended by a blank line; the line
|
|
94
|
+
`data: [DONE]` ends the stream, and `: OPENROUTER PROCESSING` comment lines
|
|
95
|
+
(the provider's keep-alive) are ignored. Each event's JSON carries a
|
|
96
|
+
`choices[0].delta.content` fragment, which is printed as it arrives. A
|
|
97
|
+
malformed frame is skipped, not fatal. Errors surface the provider's own
|
|
98
|
+
message (`HTTP 401: No auth credentials found`), connection failures get a
|
|
99
|
+
friendly hint, and every request is bounded by an `AbortController` timeout.
|
|
100
|
+
|
|
101
|
+
## Benchmark: Node.js vs TypeNative
|
|
102
|
+
|
|
103
|
+
Reproduce with `node bench.mjs [runs=30]`. A real API call spends its time
|
|
104
|
+
waiting on the model, which would hide any client-side difference, so the
|
|
105
|
+
bench stands up a local mock API (plain `node:http`) that answers
|
|
106
|
+
`POST /v1/chat/completions` with a deterministic 46-chunk event stream sent
|
|
107
|
+
with pauses between chunks — the same shape OpenRouter and OpenAI return.
|
|
108
|
+
First both runtimes answer the same question once and their stdout is
|
|
109
|
+
compared byte for byte (**parity: identical, and equal to the streamed
|
|
110
|
+
answer**). Then it times full process runs (spawn to exit) of the one-shot
|
|
111
|
+
client, which covers startup, the HTTP round trip, and parsing the stream as
|
|
112
|
+
it arrives.
|
|
113
|
+
|
|
114
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
115
|
+
TypeNative 0.5.0 (unreleased changes), default (non-`--release`) build.
|
|
116
|
+
|
|
117
|
+
| Runtime | min | median | mean | max |
|
|
118
|
+
|------------|-----------|-----------|-----------|-----------|
|
|
119
|
+
| Node.js | 775.24 ms | 780.34 ms | 782.11 ms | 796.46 ms |
|
|
120
|
+
| TypeNative | 698.90 ms | 715.82 ms | 712.61 ms | 722.34 ms |
|
|
121
|
+
|
|
122
|
+
**TypeNative is 1.1x faster per answer** (median). The ratio looks small
|
|
123
|
+
because both runtimes spend most of each run waiting for the mock's paced
|
|
124
|
+
chunks — that part is identical by construction. The client-side share
|
|
125
|
+
(startup, connect, first byte) is where they differ: about 65 ms of the
|
|
126
|
+
median run, mostly Node's ~60 ms startup. For an interactive chat session
|
|
127
|
+
the numbers that matter are the first token and memory, and the binary starts
|
|
128
|
+
about 8x faster and holds a fraction of Node's idle memory.
|
|
129
|
+
|
|
130
|
+
## Notes for your own streaming clients
|
|
131
|
+
|
|
132
|
+
Two TypeScript patterns that look natural but hit transpiler sharp edges
|
|
133
|
+
while writing this example (both are avoided above; TypeNative 0.4.1):
|
|
134
|
+
|
|
135
|
+
- `continue`/`break` out of a loop from inside a `catch` block does not
|
|
136
|
+
compile (`continue is not in a loop`) — the catch is emitted as its own Go
|
|
137
|
+
function. Set a flag in the catch and act on it after.
|
|
138
|
+
- `return` inside a `catch` block sets the return value but **falls through**
|
|
139
|
+
to the statements after the try/catch, unlike Node. Assign a result
|
|
140
|
+
variable instead:
|
|
141
|
+
|
|
142
|
+
```ts
|
|
143
|
+
try { throw new Error('boom'); } catch { return 42; }
|
|
144
|
+
console.error('reached in Go, skipped in Node');
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Streaming itself needs no special support: `fetch`, reader loops, async
|
|
148
|
+
iteration over readline, and `AbortController` all behave as under Node.
|