typenative 0.6.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +69 -0
- package/README.md +2 -1
- package/bin/node/core.js +5 -0
- package/bin/node/events.js +139 -5
- package/bin/node/http.js +1265 -127
- package/bin/node/stream.js +56 -7
- package/bin/node/stream_web.js +2 -2
- package/bin/node/url.js +27 -2
- package/bin/regex_vm.js +50 -2
- package/bin/transpiler.js +1340 -184
- package/bin/version.js +1 -1
- package/examples/README.md +28 -29
- package/examples/angular-templates/README.md +22 -20
- package/examples/json-server/README.md +16 -13
- package/examples/marked-cli/README.md +37 -22
- package/examples/marked-cli/main.ts +122 -4
- package/examples/react-compiler/README.md +56 -41
- package/examples/react-compiler/main.ts +173 -7
- package/examples/react-compiler/run-node.mjs +2 -0
- package/examples/yaml-cli/README.md +36 -30
- package/examples/yaml-cli/main.ts +236 -14
- package/package.json +1 -1
- package/performance/README.md +1 -0
package/bin/version.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
1
|
// Generated by scripts/gen-version.mjs — do not edit.
|
|
2
|
-
export const VERSION = '0.6.
|
|
2
|
+
export const VERSION = '0.6.1';
|
package/examples/README.md
CHANGED
|
@@ -8,46 +8,45 @@ Node vs TypeNative on the same workload and a README documenting the results.
|
|
|
8
8
|
|-----------------------------------|----------------------------------------------------------------------------|--------------------------------------------------------------|
|
|
9
9
|
| [hello-world](hello-world/) | `console.log`, `process.argv`, template literals | **8.5x faster startup** (7.4 ms vs 62.4 ms) |
|
|
10
10
|
| [fibonacci](fibonacci/) | Recursion, `perf_hooks` timing, CPU-bound number crunching | **2.0x faster compute** (36.4 ms vs 73.8 ms) |
|
|
11
|
-
| [json-server](json-server/) | `node:http` web server with GET + POST JSON routes, zero dependencies |
|
|
11
|
+
| [json-server](json-server/) | `node:http` web server with GET + POST JSON routes, zero dependencies | **1.7x Node's HTTP throughput** (106k vs 62k GET req/s) |
|
|
12
12
|
| [todo-cli](todo-cli/) | A to-do CLI persisting to a JSON file, run many times a day | **8.2x faster per command** (9.3 ms vs 76.1 ms) |
|
|
13
13
|
| [dupe-finder](dupe-finder/) | Recursive directory walk, SHA-256 hashing, JSON report written to disk | **2.9x faster scan** (224 ms vs 645 ms, 2,000 files) |
|
|
14
14
|
| [mandelbrot-png](mandelbrot-png/) | Fractal render + hand-written PNG encoder on `node:zlib`, no image library | **1.1x faster** (452 ms vs 514 ms) |
|
|
15
15
|
| [desktop-notes](desktop-notes/) | Notes app in a native webview window, UI compiled into one binary | **4.4x faster launch, 6.7x less memory** (11 vs 73 MiB) |
|
|
16
|
-
| [yaml-cli](yaml-cli/) | The `yaml` package's own CLI from its unmodified source, 100% Node parity | **
|
|
17
|
-
| [marked-cli](marked-cli/) | The `marked` markdown CLI from its unmodified source, 100% Node parity | **8.
|
|
16
|
+
| [yaml-cli](yaml-cli/) | The `yaml` package's own CLI from its unmodified source, 100% Node parity | **11.9x faster per call** (9 vs 105 ms); 1.1x faster on 10 MB† |
|
|
17
|
+
| [marked-cli](marked-cli/) | The `marked` markdown CLI from its unmodified source, 100% Node parity | **8.2x faster per call** (13 vs 104 ms); 1.2x faster on 10 MB‡ |
|
|
18
18
|
| [openai-chat](openai-chat/) | Chat client for any OpenAI-compatible API: streaming `fetch` + SSE parsing, readline REPL | **1.1x faster per answer** (716 vs 780 ms streamed) |
|
|
19
|
-
| [angular-templates](angular-templates/) | Angular's template compiler (`@angular/compiler`, 233 files) to Ivy instructions, 100% Node parity | **
|
|
20
|
-
| [react-compiler](react-compiler/) | The React Compiler (`babel-plugin-react-compiler` + Babel 7.29 + zod 4, ~470 files) compiling components to their auto-memoized form |
|
|
19
|
+
| [angular-templates](angular-templates/) | Angular's template compiler (`@angular/compiler`, 233 files) to Ivy instructions, 100% Node parity | **4.4x faster per template** (50 vs 219 ms); 1.2x faster at 64 KB§ |
|
|
20
|
+
| [react-compiler](react-compiler/) | The React Compiler (`babel-plugin-react-compiler` + Babel 7.29 + zod 4, ~470 files) compiling components to their auto-memoized form, 100% Node parity | **3.6x faster per file** (98 vs 352 ms); **1.7x faster at 64 KB** (618 vs 1,037 ms) |
|
|
21
21
|
|
|
22
22
|
\* Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4, TypeNative
|
|
23
|
-
0.
|
|
23
|
+
0.6.1, all examples. See each example's README for full
|
|
24
24
|
methodology and numbers.
|
|
25
25
|
|
|
26
|
-
†
|
|
27
|
-
|
|
28
|
-
Node
|
|
26
|
+
† Large inputs are converted in up to six worker processes, one contiguous
|
|
27
|
+
run of documents each; the single-process path is still about 2.3x slower
|
|
28
|
+
than Node on 10 MB. yaml-cli's [parity.mjs](yaml-cli/parity.mjs) also checks
|
|
29
|
+
every output against Node byte for byte on the yaml-test-suite (1,608 cases).
|
|
29
30
|
|
|
30
|
-
‡
|
|
31
|
-
|
|
32
|
-
|
|
31
|
+
‡ Large inputs are parsed in up to six worker processes, one contiguous run of
|
|
32
|
+
sections each, at top-level heading boundaries; a single process is still about
|
|
33
|
+
1.5x slower than Node on 10 MB. marked-cli's
|
|
34
|
+
[parity.mjs](marked-cli/parity.mjs) checks every output against Node byte for
|
|
35
|
+
byte on marked's spec suites (2,859 cases).
|
|
33
36
|
|
|
34
|
-
§ The native binary is faster
|
|
35
|
-
|
|
37
|
+
§ The native binary is faster at every template size measured (3.1x at 9 KB,
|
|
38
|
+
1.2x at 65 KB); most component templates are a few KB. angular-templates'
|
|
36
39
|
[parity.mjs](angular-templates/parity.mjs) checks every output against Node
|
|
37
40
|
byte for byte on the templates of Angular's compiler compliance suite (612
|
|
38
41
|
templates).
|
|
39
42
|
|
|
40
|
-
¶ The largest codebase TypeNative compiles. The
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
rewriting), tracked in
|
|
48
|
-
[the example's README](react-compiler/README.md#status-what-works-natively);
|
|
49
|
-
[parity.mjs](react-compiler/parity.mjs) reports the current gap and
|
|
50
|
-
[bench.mjs](react-compiler/bench.mjs) is ready.
|
|
43
|
+
¶ The largest codebase TypeNative compiles. The two break even at about
|
|
44
|
+
9 KB of component source per run; most component files are smaller.
|
|
45
|
+
react-compiler's [parity.mjs](react-compiler/parity.mjs) checks every output
|
|
46
|
+
against Node byte for byte (memoized code, `// CompileSuccess` memo slot and
|
|
47
|
+
block counts, and compile errors with their code frames) on 15 fixtures
|
|
48
|
+
covering components, hooks, the compiler's pragmas, JSX outlining, TypeScript
|
|
49
|
+
syntax and the error paths.
|
|
51
50
|
|
|
52
51
|
## Running an example
|
|
53
52
|
|
|
@@ -99,10 +98,10 @@ cd examples/react-compiler && node parity.mjs # and node bench.mjs
|
|
|
99
98
|
TypeNative's clearest wins are process startup and memory. Startup is about
|
|
100
99
|
8x faster, which is what you feel in a CLI like todo-cli. A long-running app
|
|
101
100
|
like desktop-notes idles at about a seventh of Node's memory. Disk- and
|
|
102
|
-
hash-bound work (dupe-finder) runs about 3x faster,
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
101
|
+
hash-bound work (dupe-finder) runs about 3x faster, HTTP serving (json-server)
|
|
102
|
+
handles about 1.7x more requests per second than Node 26, pure floating-point
|
|
103
|
+
loops (mandelbrot-png) run at about the same speed as V8's JIT, and
|
|
104
|
+
recursion-heavy code (fibonacci) runs about 2x faster. Every example
|
|
106
105
|
also ships as one dependency-free binary. The READMEs report what was
|
|
107
106
|
measured, including run-to-run variation and how each load was generated.
|
|
108
107
|
|
|
@@ -94,8 +94,8 @@ and checks that the two outputs are identical:
|
|
|
94
94
|
are.
|
|
95
95
|
|
|
96
96
|
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600 (6 cores, 12 threads),
|
|
97
|
-
Node v26.13.0, Go 1.27.4, TypeNative 0.
|
|
98
|
-
(non-`--release`) build. The binary is
|
|
97
|
+
Node v26.13.0, Go 1.27.4, TypeNative 0.6.1, default
|
|
98
|
+
(non-`--release`) build. The binary is 16.1 MB. Node runs the compiler as
|
|
99
99
|
precompiled JavaScript (see [Run it](#run-it)); that build is done once,
|
|
100
100
|
before timing.
|
|
101
101
|
|
|
@@ -103,34 +103,36 @@ before timing.
|
|
|
103
103
|
|
|
104
104
|
| Runtime | min | median | mean | max |
|
|
105
105
|
| ---------- | --------- | ----------- | ----------- | ----------- |
|
|
106
|
-
| Node.js |
|
|
107
|
-
| TypeNative |
|
|
106
|
+
| Node.js | 212.76 ms | 219.11 ms | 218.77 ms | 223.15 ms |
|
|
107
|
+
| TypeNative | 48.51 ms | 49.96 ms | 51.36 ms | 86.89 ms |
|
|
108
108
|
|
|
109
109
|
(b) `ng-compile generated.component.html`, 5 runs per size (medians):
|
|
110
110
|
|
|
111
111
|
| Template | Node.js | TypeNative | TypeNative vs Node |
|
|
112
112
|
| ------------------ | -------- | ---------- | ------------------ |
|
|
113
|
-
| 9 KB, 186 lines |
|
|
114
|
-
| 16 KB, 362 lines |
|
|
115
|
-
| 32 KB, 706 lines |
|
|
116
|
-
| 65 KB, 1,388 lines |
|
|
113
|
+
| 9 KB, 186 lines | 246 ms | 80 ms | 3.1x faster |
|
|
114
|
+
| 16 KB, 362 lines | 271 ms | 110 ms | 2.5x faster |
|
|
115
|
+
| 32 KB, 706 lines | 305 ms | 176 ms | 1.7x faster |
|
|
116
|
+
| 65 KB, 1,388 lines | 376 ms | 305 ms | 1.2x faster |
|
|
117
117
|
|
|
118
|
-
**Per call, TypeNative is
|
|
118
|
+
**Per call, TypeNative is 4.4x faster** (50 ms vs 219 ms, median). Almost
|
|
119
119
|
all of both times is fixed: a one-line template takes about as long. Node
|
|
120
120
|
spends it loading and evaluating the compiler's 233 modules before it
|
|
121
|
-
compiles anything. The native binary's
|
|
121
|
+
compiles anything. The native binary's 50 ms is mostly the compiler's own
|
|
122
122
|
module setup, such as the DOM schema and entity tables.
|
|
123
123
|
|
|
124
|
-
**
|
|
125
|
-
startup, compiling costs about
|
|
126
|
-
KB under V8's JIT, so the gap
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
124
|
+
**The native binary is faster at every size measured, including 65 KB
|
|
125
|
+
(1.2x).** Past startup, compiling costs about 5 ms per KB natively against
|
|
126
|
+
about 2–3 ms per KB under V8's JIT, so the gap narrows as templates grow. Most
|
|
127
|
+
of the 64 KB run is the compiler's operation traversal: its 74 phases each
|
|
128
|
+
iterate the operation lists (`OpList`), and the garbage collector scans the
|
|
129
|
+
IR they build. Before this was fixed, about a third of the samples were
|
|
130
|
+
goroutine hand-offs: iterating an `OpList` reached through a dynamically
|
|
131
|
+
typed op field (`op.handlerOps`, `op.trackByOps`) went through reflection to
|
|
132
|
+
its `*[Symbol.iterator]()` channel, one goroutine per list. `TnToArray` now
|
|
133
|
+
recognizes the generator's push form directly (`Symbol_iterator__seq`) and
|
|
134
|
+
iterates it in place, with no reflection, channel or goroutine per list; the
|
|
135
|
+
64 KB run fell from about 600 ms to 305 ms.
|
|
134
136
|
|
|
135
137
|
A template whose sections all share the root view is another matter: Angular
|
|
136
138
|
copies every `@let` and `#ref` of a view into each of its child views, so with
|
|
@@ -104,23 +104,26 @@ plain `node:http` client would cap out far below either server, so
|
|
|
104
104
|
with it both servers would simply tie at the client's limit.
|
|
105
105
|
|
|
106
106
|
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600, Node v26.13.0, Go 1.27.4,
|
|
107
|
-
TypeNative 0.
|
|
107
|
+
TypeNative 0.6.1, default (non-`--release`) build, client
|
|
108
108
|
and server on the same machine. Re-run variation is about ±20%; across runs
|
|
109
|
-
TypeNative was
|
|
109
|
+
TypeNative was 1.6–1.8x Node on GET and 1.6–1.8x on POST. The numbers below are
|
|
110
|
+
medians over three runs.
|
|
110
111
|
|
|
111
112
|
| Runtime | Endpoint | req/s | p50 (ms) | p99 (ms) |
|
|
112
113
|
|------------|-------------|-------:|---------:|---------:|
|
|
113
|
-
| Node.js | GET /todos |
|
|
114
|
-
| TypeNative | GET /todos |
|
|
115
|
-
| Node.js | POST /todos |
|
|
116
|
-
| TypeNative | POST /todos |
|
|
117
|
-
|
|
118
|
-
**
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
114
|
+
| Node.js | GET /todos | 61957 | 0.23 | 0.63 |
|
|
115
|
+
| TypeNative | GET /todos | 106483 | 0.14 | 0.38 |
|
|
116
|
+
| Node.js | POST /todos | 68097 | 0.21 | 0.46 |
|
|
117
|
+
| TypeNative | POST /todos | 114066 | 0.13 | 0.40 |
|
|
118
|
+
|
|
119
|
+
**TypeNative serves ~1.7x more requests per second here**, with lower median
|
|
120
|
+
latency on both routes. The compiled server reads requests and writes
|
|
121
|
+
responses on each connection's goroutine, and a handler that answers
|
|
122
|
+
synchronously (`writeHead` + `end`, as both routes do) builds the response
|
|
123
|
+
bytes right there, in one write: it never queues an operation for the event
|
|
124
|
+
loop, which keeps the per-request hand-off to JavaScript to a single baton
|
|
125
|
+
pass. JavaScript still runs one thing at a time, but at 105–115k req/s the
|
|
126
|
+
parallel socket I/O and HTTP handling stay ahead of that one thread.
|
|
124
127
|
|
|
125
128
|
On top of that, the binary starts in ~7 ms instead of ~60 ms and is a
|
|
126
129
|
single 11.4 MB executable with zero dependencies — `scp` it to a server and
|
|
@@ -100,38 +100,41 @@ Node v26.13.0, Go 1.27.4, TypeNative 0.5.0 (unreleased changes), default
|
|
|
100
100
|
|
|
101
101
|
| Runtime | min | median | mean | max |
|
|
102
102
|
| ---------- | --------- | --------- | --------- | --------- |
|
|
103
|
-
| Node.js |
|
|
104
|
-
| TypeNative |
|
|
103
|
+
| Node.js | 101.08 ms | 103.58 ms | 106.68 ms | 119.02 ms |
|
|
104
|
+
| TypeNative | 11.98 ms | 12.67 ms | 20.14 ms | 235.52 ms |
|
|
105
105
|
|
|
106
106
|
(b) `marked < generated.md` (10.0 MB, 114,893 lines), 5 runs each:
|
|
107
107
|
|
|
108
108
|
| Runtime | min | median | mean | max | MB/s (median) |
|
|
109
109
|
| ---------- | -------- | -------- | -------- | -------- | ------------- |
|
|
110
|
-
| Node.js |
|
|
111
|
-
| TypeNative |
|
|
110
|
+
| Node.js | 585.94 ms | 595.71 ms | 592.98 ms | 597.79 ms | 16.79 |
|
|
111
|
+
| TypeNative | 500.79 ms | 511.86 ms | 509.63 ms | 513.24 ms | 19.54 |
|
|
112
112
|
|
|
113
|
-
**Per call, TypeNative is 8.
|
|
113
|
+
**Per call, TypeNative is 8.2x faster** (12.7 ms vs 103.6 ms, median). Node's
|
|
114
114
|
cost here is mostly startup: it loads and type-strips the 13 TypeScript files
|
|
115
115
|
and compiles marked's regexes on every run. That's what you feel when a docs
|
|
116
116
|
build or a hook renders one file per process.
|
|
117
117
|
|
|
118
|
-
**On bulk throughput, TypeNative is 1.
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
118
|
+
**On bulk throughput, TypeNative is 1.2x faster** (512 ms vs 596 ms). A
|
|
119
|
+
loaded document is cut before top-level ATX headings that follow a blank line
|
|
120
|
+
and are outside a code fence; up to six worker processes each parse one
|
|
121
|
+
contiguous run of sections, and their HTML is concatenated. Parsing the
|
|
122
|
+
sections separately is byte-for-byte identical to parsing the whole document
|
|
123
|
+
here (bench.mjs checks it), because marked renders top-level blocks
|
|
124
|
+
independently. `main.ts` falls back to one process for inputs under 512 KB, for
|
|
125
|
+
documents with reference link definitions, raw HTML blocks or CRLF line
|
|
126
|
+
endings, and for `--tokens`; `parity.mjs`'s spec cases all take that path, so
|
|
127
|
+
their behavior is unchanged. The workers pass their assignment in `workerData`
|
|
128
|
+
and return gzipped HTML through one temporary file: the worker IPC carries a
|
|
129
|
+
single compact message, not several megabytes.
|
|
130
|
+
|
|
131
|
+
**A single process is still about 1.5x slower than Node on 10 MB.** The
|
|
132
|
+
parallel path is what closes the gap; per section the remaining cost is
|
|
133
|
+
TypeNative's regex VM, which interprets marked's patterns where V8's Irregexp
|
|
134
|
+
compiles them to machine code, plus the object and match-array allocation each
|
|
135
|
+
token costs. The fixes below cut the single-process time from 1,890 ms to
|
|
136
|
+
about 1,620 ms before the parallel path; the parallel run then has six of those
|
|
137
|
+
processes working on independent sections.
|
|
135
138
|
|
|
136
139
|
The ReDoS inputs now favor the native binary: all of them together take 4.7 s
|
|
137
140
|
natively and 13 s under Node, though most of Node's time there is starting up
|
|
@@ -158,3 +161,15 @@ general:
|
|
|
158
161
|
string share its index instead of rebuilding it.
|
|
159
162
|
- Type inference is memoized per query. marked's
|
|
160
163
|
`edit(re).replace(…).replace(…)…` chains took exponential time to compile.
|
|
164
|
+
- `[...s].length`, `[...s][i]` and `[...s].at(i)` are the code point count, the
|
|
165
|
+
i-th code point and the i-th code point (from the end), without building an
|
|
166
|
+
array of every rune; marked uses those for every emphasis delimiter.
|
|
167
|
+
- `re.exec(s)` where only its presence is tested (`!!re.exec(s)`, an `if`
|
|
168
|
+
condition) runs `re.test(s)`: no match array is built.
|
|
169
|
+
- A nullable regex result is boxed only when it matched (`TnSlicePtr`), not by
|
|
170
|
+
a closure and a heap box on every miss.
|
|
171
|
+
- Greedy runs of "everything but one ASCII byte" (`[^\\n]+`, `[^*]+`) use
|
|
172
|
+
`IndexByte` instead of testing each byte.
|
|
173
|
+
- The ASCII cache for long strings is a mutex-guarded entry, not a new atomic
|
|
174
|
+
entry per inline token.
|
|
175
|
+
- Regex replacement builders are grown once to the input's length.
|
|
@@ -21,11 +21,14 @@
|
|
|
21
21
|
// 64 KB).
|
|
22
22
|
import { promises } from 'node:fs';
|
|
23
23
|
import { resolve } from 'node:path';
|
|
24
|
-
import { homedir } from 'node:os';
|
|
24
|
+
import { availableParallelism, homedir, tmpdir } from 'node:os';
|
|
25
|
+
import { Buffer } from 'node:buffer';
|
|
26
|
+
import { gzipSync, gunzipSync } from 'node:zlib';
|
|
27
|
+
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
|
|
25
28
|
import { marked } from './vendor/marked/src/marked.ts';
|
|
26
29
|
import type { MarkedExtension, MarkedOptions } from './vendor/marked/src/marked.ts';
|
|
27
30
|
|
|
28
|
-
const { access, readFile, writeFile } = promises;
|
|
31
|
+
const { access, readFile, unlink, writeFile } = promises;
|
|
29
32
|
|
|
30
33
|
const VERSION = '18.1.2';
|
|
31
34
|
|
|
@@ -55,6 +58,117 @@ interface CliError {
|
|
|
55
58
|
message: string;
|
|
56
59
|
}
|
|
57
60
|
|
|
61
|
+
// Large documents are parsed in a few processes, one contiguous run of
|
|
62
|
+
// sections each: marked renders top-level blocks independently, so the
|
|
63
|
+
// concatenation is the whole document's HTML (checked byte for byte by
|
|
64
|
+
// bench.mjs). Splits happen only before a top-level ATX heading that follows
|
|
65
|
+
// a blank line and is outside a code fence; documents with reference link
|
|
66
|
+
// definitions, raw HTML blocks or CRLF line endings take the sequential path,
|
|
67
|
+
// as the pieces would not stand on their own. Between the processes a section
|
|
68
|
+
// run travels in workerData, and its HTML comes back gzipped through one small
|
|
69
|
+
// temporary file (the worker IPC handles one compact message, not megabytes).
|
|
70
|
+
const PARALLEL_MIN_BYTES = 512 * 1024;
|
|
71
|
+
|
|
72
|
+
function splitForParallel(data: string, options: MarkedOptions): string[] | undefined {
|
|
73
|
+
if (data.length < PARALLEL_MIN_BYTES || data.includes('\r')) return undefined;
|
|
74
|
+
if (options.extensions || options.hooks || options.walkTokens) return undefined;
|
|
75
|
+
// a reference definition is shared by the whole document
|
|
76
|
+
if (/(^|\n) {0,3}\[[^\]\n]+\]:/.test(data)) return undefined;
|
|
77
|
+
// a cut line inside one of these would not be a top-level heading
|
|
78
|
+
if (/<(?:pre|script|style|textarea)\b/i.test(data) || data.includes('<!--') || data.includes('<?') || data.includes('<![CDATA[')) return undefined;
|
|
79
|
+
|
|
80
|
+
const lines = data.split('\n');
|
|
81
|
+
const cuts: number[] = [0];
|
|
82
|
+
let fence = '';
|
|
83
|
+
for (let i = 0; i < lines.length; i++) {
|
|
84
|
+
const line = lines[i];
|
|
85
|
+
const open = /^ {0,3}(`{3,}|~{3,})/.exec(line);
|
|
86
|
+
if (open) {
|
|
87
|
+
if (!fence) fence = open[1][0];
|
|
88
|
+
else if (open[1][0] === fence) fence = '';
|
|
89
|
+
continue;
|
|
90
|
+
}
|
|
91
|
+
if (fence) continue;
|
|
92
|
+
if (i > 0 && lines[i - 1] === '' && /^#{1,6} /.test(line)) cuts.push(i);
|
|
93
|
+
}
|
|
94
|
+
cuts.push(lines.length);
|
|
95
|
+
if (cuts.length < 3) return undefined;
|
|
96
|
+
|
|
97
|
+
// One contiguous run of chunks per process, balanced by size
|
|
98
|
+
const chunks: string[] = [];
|
|
99
|
+
let total = 0;
|
|
100
|
+
for (let i = 0; i + 1 < cuts.length; i++) {
|
|
101
|
+
const chunk = lines.slice(cuts[i], cuts[i + 1]).join('\n');
|
|
102
|
+
chunks.push(chunk);
|
|
103
|
+
total += chunk.length;
|
|
104
|
+
}
|
|
105
|
+
const requested = Number(process.env.MARKED_WORKERS ?? '0') || availableParallelism();
|
|
106
|
+
const count = Math.min(6, Math.max(1, requested));
|
|
107
|
+
if (count < 2) return undefined;
|
|
108
|
+
const parts: string[] = [];
|
|
109
|
+
let at = 0;
|
|
110
|
+
let size = 0;
|
|
111
|
+
for (let m = 0; m < count && at < chunks.length; m++) {
|
|
112
|
+
const target = total * (m + 1) / count;
|
|
113
|
+
const start = at;
|
|
114
|
+
while (at < chunks.length && (at === start || size < target)) {
|
|
115
|
+
size += chunks[at].length;
|
|
116
|
+
at++;
|
|
117
|
+
}
|
|
118
|
+
parts.push(chunks.slice(start, at).join('\n'));
|
|
119
|
+
}
|
|
120
|
+
return parts.length > 1 ? parts : undefined;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
interface ParseJob {
|
|
124
|
+
text: string;
|
|
125
|
+
output: string;
|
|
126
|
+
options: MarkedOptions;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
async function parseHtml(data: string, options: MarkedOptions): Promise<string> {
|
|
130
|
+
const parts = splitForParallel(data, options);
|
|
131
|
+
if (!parts) return await marked.parse(data, options);
|
|
132
|
+
const base = tmpdir() + '/marked-' + process.pid + '-' + Date.now();
|
|
133
|
+
const htmls: string[] = await Promise.all(parts.map((text, index) => new Promise<string>((resolve, reject) => {
|
|
134
|
+
const output = base + '-' + index + '.html';
|
|
135
|
+
const cleanup = () => unlink(output).catch(() => {});
|
|
136
|
+
(async () => {
|
|
137
|
+
try {
|
|
138
|
+
const worker = new Worker(new URL(import.meta.url), { workerData: { text, output, options } as ParseJob });
|
|
139
|
+
worker.on('message', async () => {
|
|
140
|
+
try {
|
|
141
|
+
resolve(gunzipSync(await readFile(output)).toString('utf8'));
|
|
142
|
+
} catch (err) {
|
|
143
|
+
reject(err);
|
|
144
|
+
} finally {
|
|
145
|
+
cleanup();
|
|
146
|
+
}
|
|
147
|
+
worker.terminate();
|
|
148
|
+
});
|
|
149
|
+
worker.on('error', (err) => {
|
|
150
|
+
cleanup();
|
|
151
|
+
reject(err);
|
|
152
|
+
});
|
|
153
|
+
} catch (err) {
|
|
154
|
+
cleanup();
|
|
155
|
+
reject(err);
|
|
156
|
+
}
|
|
157
|
+
})();
|
|
158
|
+
}))) as string[];
|
|
159
|
+
return htmls.join('');
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
async function workerMain() {
|
|
163
|
+
const job = workerData as ParseJob;
|
|
164
|
+
const options = job.options as MarkedOptions;
|
|
165
|
+
const text = job.text as string;
|
|
166
|
+
const output = job.output as string;
|
|
167
|
+
await writeFile(output, gzipSync(Buffer.from(await marked.parse(text, options), 'utf8')));
|
|
168
|
+
parentPort!.postMessage('ok');
|
|
169
|
+
parentPort!.close();
|
|
170
|
+
}
|
|
171
|
+
|
|
58
172
|
export async function main() {
|
|
59
173
|
async function help() {
|
|
60
174
|
console.log(HELP);
|
|
@@ -223,7 +337,7 @@ export async function main() {
|
|
|
223
337
|
|
|
224
338
|
const html = tokens
|
|
225
339
|
? JSON.stringify(marked.lexer(data, options as MarkedOptions), null, 2)
|
|
226
|
-
: await
|
|
340
|
+
: await parseHtml(data, options as MarkedOptions);
|
|
227
341
|
|
|
228
342
|
if (output) {
|
|
229
343
|
try {
|
|
@@ -289,4 +403,8 @@ export async function main() {
|
|
|
289
403
|
}
|
|
290
404
|
}
|
|
291
405
|
|
|
292
|
-
|
|
406
|
+
if (isMainThread) {
|
|
407
|
+
main();
|
|
408
|
+
} else {
|
|
409
|
+
workerMain();
|
|
410
|
+
}
|
|
@@ -112,7 +112,8 @@ node parity.mjs
|
|
|
112
112
|
Reproduce with `node bench.mjs [runs=30] [sizeRuns=5] [kilobytes=8,16,32,64]`.
|
|
113
113
|
The script compiles `main.ts`, then times **complete process runs** (spawn to
|
|
114
114
|
exit) of `react-compile` under both runtimes, alternating Node and native runs,
|
|
115
|
-
and stops if the two outputs ever differ
|
|
115
|
+
and stops if the two outputs ever differ. `TN_PARALLEL=0` forces the
|
|
116
|
+
single-process path for both runtimes:
|
|
116
117
|
|
|
117
118
|
- **(a) per-call latency:** the built-in demo (a `TodoList` component and a
|
|
118
119
|
`useTotal` hook), the size of a file a build compiles once per process.
|
|
@@ -121,50 +122,64 @@ and stops if the two outputs ever differ:
|
|
|
121
122
|
fixed-seed generator, so every machine gets the same input).
|
|
122
123
|
|
|
123
124
|
**Environment:** Windows 11 Pro, AMD Ryzen 5 7600 (6 cores, 12 threads),
|
|
124
|
-
Node v26.13.0, Go 1.27.4, TypeNative 0.
|
|
125
|
-
|
|
125
|
+
Node v26.13.0, Go 1.27.4, TypeNative 0.6.1, default (non-`--release`) build.
|
|
126
|
+
The binary is 29.9 MB.
|
|
126
127
|
|
|
127
|
-
(a) `react-compile` (built-in demo),
|
|
128
|
+
(a) `react-compile` (built-in demo), 5 runs each:
|
|
128
129
|
|
|
129
130
|
| Runtime | min | median | mean | max |
|
|
130
131
|
| ---------- | --------- | --------- | --------- | --------- |
|
|
131
|
-
| Node.js |
|
|
132
|
-
| TypeNative |
|
|
132
|
+
| Node.js | 347.32 ms | 348.88 ms | 349.49 ms | 353.65 ms |
|
|
133
|
+
| TypeNative | 97.67 ms | 99.61 ms | 107.70 ms | 142.96 ms |
|
|
133
134
|
|
|
134
135
|
(b) `react-compile generated-screen.jsx`, 5 runs per size (medians):
|
|
135
136
|
|
|
136
|
-
| Source | Node.js
|
|
137
|
-
| ----------------- |
|
|
138
|
-
| 9 KB, 238 lines |
|
|
139
|
-
| 17 KB, 460 lines |
|
|
140
|
-
| 32 KB, 892 lines |
|
|
141
|
-
| 64 KB, 1774 lines |
|
|
142
|
-
|
|
143
|
-
**Per call, TypeNative is
|
|
144
|
-
spends most of a short run loading and
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
**
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
137
|
+
| Source | Node.js | TypeNative | TypeNative vs Node |
|
|
138
|
+
| ----------------- | ---------- | ---------- | ------------------ |
|
|
139
|
+
| 9 KB, 238 lines | 704.90 ms | 275.08 ms | 2.6x faster |
|
|
140
|
+
| 17 KB, 460 lines | 761.71 ms | 332.94 ms | 2.3x faster |
|
|
141
|
+
| 32 KB, 892 lines | 874.53 ms | 418.38 ms | 2.1x faster |
|
|
142
|
+
| 64 KB, 1774 lines | 1,030.31 ms| 615.05 ms | 1.7x faster |
|
|
143
|
+
|
|
144
|
+
**Per call, TypeNative is 3.5x faster** (100 ms vs 349 ms, median). Node
|
|
145
|
+
spends most of a short run loading and compiling the ~470 vendored modules;
|
|
146
|
+
the native binary starts with them compiled in.
|
|
147
|
+
|
|
148
|
+
**TypeNative is now faster on large inputs too.** Most of the gap was the
|
|
149
|
+
compiler's passes running through TypeNative's dynamic paths (any-typed
|
|
150
|
+
property reads, prototype-method dispatch, reflection-based struct
|
|
151
|
+
conversions) on every node of every pass, where V8's inline caches make the
|
|
152
|
+
same operations nearly free. Two rounds of profiling narrowed it:
|
|
153
|
+
|
|
154
|
+
- The plain functions Babel assigns to `NodePath.prototype` (`visit`,
|
|
155
|
+
`setScope`, `setContext`, …) declare `this: NodePath`; that now types
|
|
156
|
+
`__this` as `*NodePath_Final`, so `this.opts`, `this.parentPath` and the
|
|
157
|
+
rest are direct field reads and writes instead of plan lookups on every
|
|
158
|
+
AST node.
|
|
159
|
+
- Dynamic calls of the program's own function types (the visitor callbacks,
|
|
160
|
+
prototype methods and Babel's `call`/`apply` shapes) go through a generated
|
|
161
|
+
`tnCallValueTyped` case per Go func type, converting arguments directly
|
|
162
|
+
instead of `reflect.Value.Call`.
|
|
163
|
+
- Getters and setters of a class are recognized while a module is emitting
|
|
164
|
+
even before its class is visited (`classMember`), and optional calls to
|
|
165
|
+
members a class adds to its prototype at run time go through the guarded
|
|
166
|
+
dynamic call.
|
|
167
|
+
- Earlier work still applies: in-place `s += part` on Babel's output buffer
|
|
168
|
+
(printing was quadratic in the output size), the accessor / view-copy /
|
|
169
|
+
added-property registries skipped for types that never had an entry,
|
|
170
|
+
prototype methods called directly, generators as Go range functions, typed
|
|
171
|
+
dispatchers for shared method signatures, and Go integer loop counters.
|
|
172
|
+
|
|
173
|
+
**Large files compile their independent top-level functions in parallel.**
|
|
174
|
+
`main.ts` splits the input at top-level function boundaries when the file has
|
|
175
|
+
enough of them for at least two workers and nothing but imports/comments
|
|
176
|
+
before the first (and only blanks/comments between); each contiguous run is
|
|
177
|
+
padded to keep the
|
|
178
|
+
original line numbers, compiled by a worker process, and the parent joins the
|
|
179
|
+
per-function output. Compiler-generated `_temp` functions are suffixed per
|
|
180
|
+
chunk so the merged program has no name collisions. Under Node the same code
|
|
181
|
+
spawns Node workers, which must re-import the compiler per worker; a
|
|
182
|
+
TypeNative worker starts from the compiled binary, which is why the native
|
|
183
|
+
build now wins on the large sizes instead of only on startup. The merged
|
|
184
|
+
output is byte-identical between the two runtimes (and the small fixtures,
|
|
185
|
+
below eight functions, take the single-process path).
|