typenative 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/ci.yml +2 -0
- package/CHANGELOG.md +372 -0
- package/CONTRIBUTION.md +1 -1
- package/README.md +62 -22
- package/ROUND2-HANDOFF.md +200 -0
- package/bin/cli.js +2 -0
- package/bin/go/bridge.js +74 -14
- package/bin/go/convert.js +48 -0
- package/bin/go/metadata.js +13 -4
- package/bin/go/overrides.js +4 -6
- package/bin/go/stdlib.generated.js +2 -0
- package/bin/index.js +130 -2
- package/bin/main.js +2 -0
- package/bin/node/assert.js +2 -2
- package/bin/node/async_hooks.js +1 -1
- package/bin/node/buffer.js +16 -5
- package/bin/node/child_process.js +57 -9
- package/bin/node/cluster.js +44 -6
- package/bin/node/core.js +1 -1
- package/bin/node/crypto.js +205 -4
- package/bin/node/crypto_keys.js +21 -21
- package/bin/node/dgram.js +231 -7
- package/bin/node/diagnostics_channel.js +3 -3
- package/bin/node/dns.js +472 -9
- package/bin/node/errno_tables.js +445 -0
- package/bin/node/events.js +3 -3
- package/bin/node/fetch.js +2 -2
- package/bin/node/fs.js +24 -17
- package/bin/node/fs_watch.js +5 -5
- package/bin/node/http.js +131 -33
- package/bin/node/http2.js +7 -4
- package/bin/node/module_v8.js +142 -0
- package/bin/node/net.js +17 -14
- package/bin/node/path.js +17 -3
- package/bin/node/process.js +274 -9
- package/bin/node/readline.js +272 -1
- package/bin/node/registry.js +5 -1
- package/bin/node/sqlite.js +65 -17
- package/bin/node/stream.js +220 -22
- package/bin/node/stream_web.js +10 -10
- package/bin/node/system_extra.js +3 -3
- package/bin/node/test.js +834 -234
- package/bin/node/test_coverage.js +187 -0
- package/bin/node/test_report.js +1392 -0
- package/bin/node/test_run.js +575 -0
- package/bin/node/timers.js +4 -4
- package/bin/node/tls.js +5 -5
- package/bin/node/url.js +23 -5
- package/bin/node/util.js +49 -8
- package/bin/node/util_extra.js +13 -37
- package/bin/node/webview.js +654 -0
- package/bin/node/worker_threads.js +80 -20
- package/bin/node/zlib.js +3 -3
- package/bin/parse-node.js +4 -0
- package/bin/transpiler.js +14457 -1723
- package/bin/typecheck-node.js +1 -0
- package/bin/version.js +1 -1
- package/conformance.md +1153 -884
- package/examples/README.md +47 -10
- package/examples/desktop-notes/README.md +108 -0
- package/examples/desktop-notes/bench.mjs +125 -0
- package/examples/desktop-notes/main.ts +131 -0
- package/examples/desktop-notes/package-lock.json +726 -0
- package/examples/desktop-notes/package.json +9 -0
- package/examples/desktop-notes/ui.ts +176 -0
- package/examples/dupe-finder/README.md +76 -0
- package/examples/dupe-finder/bench.mjs +120 -0
- package/examples/dupe-finder/main.ts +87 -0
- package/examples/json-server/README.md +31 -29
- package/examples/json-server/bench.mjs +130 -38
- package/examples/mandelbrot-png/README.md +65 -0
- package/examples/mandelbrot-png/bench.mjs +95 -0
- package/examples/mandelbrot-png/main.ts +98 -0
- package/examples/mandelbrot-png/preview.png +0 -0
- package/examples/marked-cli/README.md +159 -0
- package/examples/marked-cli/bench.mjs +178 -0
- package/examples/marked-cli/main.ts +292 -0
- package/examples/marked-cli/parity.mjs +185 -0
- package/examples/marked-cli/vendor/marked/LICENSE +44 -0
- package/examples/marked-cli/vendor/marked/VENDOR.md +26 -0
- package/examples/todo-cli/README.md +69 -0
- package/examples/todo-cli/bench.mjs +89 -0
- package/examples/todo-cli/main.ts +92 -0
- package/examples/yaml-cli/README.md +136 -0
- package/examples/yaml-cli/bench.mjs +265 -0
- package/examples/yaml-cli/main.ts +29 -0
- package/examples/yaml-cli/parity.mjs +156 -0
- package/examples/yaml-cli/vendor/yaml/LICENSE +13 -0
- package/examples/yaml-cli/vendor/yaml/VENDOR.md +25 -0
- package/govendor/github.com/jchv/go-webview2/LICENSE +22 -0
- package/govendor/github.com/jchv/go-webview2/README.md +21 -0
- package/govendor/github.com/jchv/go-webview2/common.go +84 -0
- package/govendor/github.com/jchv/go-webview2/go.mod +3 -0
- package/govendor/github.com/jchv/go-webview2/internal/w32/w32.go +194 -0
- package/govendor/github.com/jchv/go-webview2/internal/w32/w32_386.go +14 -0
- package/govendor/github.com/jchv/go-webview2/internal/w32/w32_64bit.go +15 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_COLOR.go +8 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_HOST_RESOURCE_ACCESS_KIND.go +9 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_KEY_EVENT_KIND.go +10 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_MOVE_FOCUS_REASON.go +9 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_PHYSICAL_KEY_STATUS.go +10 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/COREWEBVIEW2_WEB_RESOURCE_CONTEXT.go +23 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2AcceleratorKeyPressedEventArgs.go +78 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2AcceleratorKeyPressedEventHandler.go +52 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2Controller.go +132 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2Controller2.go +74 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2CreateCoreWebView2ControllerCompletedHandler.go +48 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2NavigationCompletedEventArgs.go +17 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2NavigationCompletedEventHandler.go +48 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2Settings.go +270 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2WebResourceRequest.go +43 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2WebResourceRequestedEventArgs.go +51 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2WebResourceRequestedEventHandler.go +48 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2WebResourceResponse.go +21 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2_2.go +21 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebView2_3.go +60 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/ICoreWebViewSettings.go +396 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/chromium.go +361 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/chromium_386.go +24 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/chromium_amd64.go +22 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/chromium_arm64.go +26 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/comproc.go +13 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/comproc_go17.go +51 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/comproc_go18.go +16 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/corewebview2.go +411 -0
- package/govendor/github.com/jchv/go-webview2/pkg/edge/guid.go +223 -0
- package/govendor/github.com/jchv/go-webview2/webview.go +482 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/README.md +3 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/module.go +153 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/module_386.go +6 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/module_amd64.go +6 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/module_arm64.go +6 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/sdk/arm64/WebView2Loader.dll +0 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/sdk/x64/WebView2Loader.dll +0 -0
- package/govendor/github.com/jchv/go-webview2/webviewloader/sdk/x86/WebView2Loader.dll +0 -0
- package/govendor/github.com/jchv/go-winloader/LICENSE.md +7 -0
- package/govendor/github.com/jchv/go-winloader/README.md +124 -0
- package/govendor/github.com/jchv/go-winloader/common.go +11 -0
- package/govendor/github.com/jchv/go-winloader/go.mod +3 -0
- package/govendor/github.com/jchv/go-winloader/internal/loader/loader.go +25 -0
- package/govendor/github.com/jchv/go-winloader/internal/loader/machine.go +53 -0
- package/govendor/github.com/jchv/go-winloader/internal/memloader/cache.go +42 -0
- package/govendor/github.com/jchv/go-winloader/internal/memloader/loader.go +253 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/export.go +76 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/format.go +522 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/import.go +106 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/module.go +91 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/reloc.go +177 -0
- package/govendor/github.com/jchv/go-winloader/internal/pe/util.go +40 -0
- package/govendor/github.com/jchv/go-winloader/internal/vmem/common.go +55 -0
- package/govendor/github.com/jchv/go-winloader/internal/vmem/memory_other.go +71 -0
- package/govendor/github.com/jchv/go-winloader/internal/vmem/memory_windows.go +179 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/hacks_other.go +17 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/hacks_windows.go +73 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/loader_windows.go +64 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/machine_windows.go +32 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/nativearch_386.go +9 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/nativearch_amd64.go +9 -0
- package/govendor/github.com/jchv/go-winloader/internal/winloader/nativearch_arm64.go +9 -0
- package/govendor/github.com/jchv/go-winloader/loader_other.go +24 -0
- package/govendor/github.com/jchv/go-winloader/loader_windows.go +33 -0
- package/govendor/manifest.txt +3 -1
- package/package.json +1 -1
- package/tsparser/main.go +4 -0
- package/types/typenative-go.d.ts +23 -0
- package/types/typenative-npm.d.ts +359 -7
- package/types/typenative.d.ts +283 -42
|
@@ -11,20 +11,127 @@
|
|
|
11
11
|
// - GET /todos 20,000 requests (a ~50-item JSON list per response)
|
|
12
12
|
// - POST /todos 10,000 requests (create a todo)
|
|
13
13
|
//
|
|
14
|
+
// The client is built to stay out of the way, like wrk or autocannon: worker
|
|
15
|
+
// threads (8 x 2 connections) send prebuilt request bytes over raw keep-alive
|
|
16
|
+
// sockets and read each response only far enough to find its end. A plain
|
|
17
|
+
// node:http client tops out at ~8-11k req/s on its own here, which would
|
|
18
|
+
// measure the client rather than the servers.
|
|
19
|
+
//
|
|
14
20
|
// Usage: node bench.mjs [gets=20000] [posts=10000]
|
|
15
21
|
|
|
16
22
|
import { spawn, spawnSync } from 'node:child_process';
|
|
17
|
-
import
|
|
23
|
+
import net from 'node:net';
|
|
18
24
|
import fs from 'node:fs';
|
|
19
25
|
import path from 'node:path';
|
|
20
26
|
import { fileURLToPath } from 'node:url';
|
|
27
|
+
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
|
|
28
|
+
|
|
29
|
+
const WORKERS = 8;
|
|
30
|
+
const CONNECTIONS_PER_WORKER = 2;
|
|
31
|
+
const CONCURRENCY = WORKERS * CONNECTIONS_PER_WORKER;
|
|
32
|
+
|
|
33
|
+
// The complete response at the start of buf: { status, length } (null while
|
|
34
|
+
// more bytes are needed). Bodies end by Content-Length or chunked encoding.
|
|
35
|
+
function parseResponse(buf) {
|
|
36
|
+
const headEnd = buf.indexOf('\r\n\r\n');
|
|
37
|
+
if (headEnd < 0) return null;
|
|
38
|
+
const head = buf.toString('latin1', 0, headEnd);
|
|
39
|
+
const status = Number(head.slice(9, 12));
|
|
40
|
+
const bodyStart = headEnd + 4;
|
|
41
|
+
const contentLength = /\r\ncontent-length:\s*(\d+)/i.exec(head);
|
|
42
|
+
if (contentLength) {
|
|
43
|
+
const end = bodyStart + Number(contentLength[1]);
|
|
44
|
+
return buf.length >= end ? { status, length: end } : null;
|
|
45
|
+
}
|
|
46
|
+
if (/\r\ntransfer-encoding:\s*chunked/i.test(head)) {
|
|
47
|
+
for (let at = bodyStart; ; ) {
|
|
48
|
+
const lineEnd = buf.indexOf('\r\n', at);
|
|
49
|
+
if (lineEnd < 0) return null;
|
|
50
|
+
const size = parseInt(buf.toString('latin1', at, lineEnd), 16);
|
|
51
|
+
const next = lineEnd + 2 + size + 2;
|
|
52
|
+
if (buf.length < next) return null;
|
|
53
|
+
if (size === 0) return { status, length: next };
|
|
54
|
+
at = next;
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
return { status, length: bodyStart };
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
// A keep-alive connection with one request in flight at a time;
|
|
61
|
+
// send(bytes) resolves with the response status
|
|
62
|
+
function connect(host, port) {
|
|
63
|
+
return new Promise((resolve, reject) => {
|
|
64
|
+
let buffered = Buffer.alloc(0);
|
|
65
|
+
let pending = null;
|
|
66
|
+
const socket = net.connect(port, host, () => resolve(connection));
|
|
67
|
+
socket.setNoDelay(true);
|
|
68
|
+
const connection = {
|
|
69
|
+
send: (bytes) =>
|
|
70
|
+
new Promise((onStatus, onError) => {
|
|
71
|
+
pending = { onStatus, onError };
|
|
72
|
+
socket.write(bytes);
|
|
73
|
+
}),
|
|
74
|
+
close: () => socket.destroy(),
|
|
75
|
+
};
|
|
76
|
+
socket.on('data', (chunk) => {
|
|
77
|
+
buffered = buffered.length > 0 ? Buffer.concat([buffered, chunk]) : chunk;
|
|
78
|
+
const response = parseResponse(buffered);
|
|
79
|
+
if (response && pending) {
|
|
80
|
+
buffered = buffered.subarray(response.length);
|
|
81
|
+
const { onStatus } = pending;
|
|
82
|
+
pending = null;
|
|
83
|
+
onStatus(response.status);
|
|
84
|
+
}
|
|
85
|
+
});
|
|
86
|
+
socket.on('error', (err) => (pending ? pending.onError(err) : reject(err)));
|
|
87
|
+
});
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
// Worker thread: sends its share of one phase's requests over its own
|
|
91
|
+
// connections, then returns each request's latency
|
|
92
|
+
async function runWorker({ base, method, reqPath, body, total }) {
|
|
93
|
+
const target = new URL(base);
|
|
94
|
+
const request = Buffer.from(
|
|
95
|
+
`${method} ${reqPath} HTTP/1.1\r\nHost: ${target.host}\r\n` +
|
|
96
|
+
(body ? `Content-Type: application/json\r\nContent-Length: ${Buffer.byteLength(body)}\r\n` : '') +
|
|
97
|
+
`\r\n${body ?? ''}`
|
|
98
|
+
);
|
|
99
|
+
const latencies = new Float64Array(total);
|
|
100
|
+
let dispatched = 0;
|
|
101
|
+
const connections = await Promise.all(
|
|
102
|
+
Array.from({ length: CONNECTIONS_PER_WORKER }, () => connect(target.hostname, Number(target.port)))
|
|
103
|
+
);
|
|
104
|
+
|
|
105
|
+
// Wait for the go signal so every worker starts at once
|
|
106
|
+
await new Promise((resolve) => parentPort.once('message', resolve));
|
|
107
|
+
await Promise.all(
|
|
108
|
+
connections.map(async (connection) => {
|
|
109
|
+
for (;;) {
|
|
110
|
+
const index = dispatched;
|
|
111
|
+
dispatched += 1;
|
|
112
|
+
if (index >= total) return;
|
|
113
|
+
const start = performance.now();
|
|
114
|
+
const status = await connection.send(request);
|
|
115
|
+
latencies[index] = performance.now() - start;
|
|
116
|
+
if (status !== 200 && status !== 201) throw new Error(`${method} ${reqPath} -> ${status}`);
|
|
117
|
+
}
|
|
118
|
+
})
|
|
119
|
+
);
|
|
120
|
+
for (const connection of connections) connection.close();
|
|
121
|
+
parentPort.postMessage(latencies, [latencies.buffer]);
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
if (!isMainThread) {
|
|
125
|
+
parentPort.postMessage('ready');
|
|
126
|
+
await runWorker(workerData);
|
|
127
|
+
process.exit(0);
|
|
128
|
+
}
|
|
21
129
|
|
|
22
130
|
const here = path.dirname(fileURLToPath(import.meta.url));
|
|
23
131
|
const repoRoot = path.resolve(here, '..', '..');
|
|
24
132
|
const cli = path.join(repoRoot, 'bin', 'cli.js');
|
|
25
133
|
const GET_REQUESTS = Number(process.argv[2] ?? '20000') || 20000;
|
|
26
134
|
const POST_REQUESTS = Number(process.argv[3] ?? '10000') || 10000;
|
|
27
|
-
const CONCURRENCY = 16;
|
|
28
135
|
const SEEDED = 50;
|
|
29
136
|
const exeName = process.platform === 'win32' ? 'server.exe' : 'server';
|
|
30
137
|
const exePath = path.join(here, 'dist', exeName);
|
|
@@ -74,48 +181,33 @@ async function seed(base) {
|
|
|
74
181
|
}
|
|
75
182
|
}
|
|
76
183
|
|
|
77
|
-
// One load phase over
|
|
184
|
+
// One load phase spread over the worker threads; returns req/s and latencies.
|
|
78
185
|
async function benchPhase(base, { method, reqPath, body, total }) {
|
|
79
|
-
const
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
const headers = body
|
|
84
|
-
? { 'content-type': 'application/json', 'content-length': Buffer.byteLength(body) }
|
|
85
|
-
: {};
|
|
86
|
-
|
|
87
|
-
const request = () =>
|
|
88
|
-
new Promise((resolve, reject) => {
|
|
89
|
-
const req = http.request(
|
|
90
|
-
{ host: target.hostname, port: target.port, method, path: reqPath, agent, headers },
|
|
91
|
-
(res) => {
|
|
92
|
-
res.resume();
|
|
93
|
-
res.on('end', () => {
|
|
94
|
-
if (res.statusCode === 200 || res.statusCode === 201) resolve();
|
|
95
|
-
else reject(new Error(`${method} ${reqPath} -> ${res.statusCode}`));
|
|
96
|
-
});
|
|
97
|
-
}
|
|
98
|
-
);
|
|
99
|
-
req.on('error', reject);
|
|
100
|
-
req.end(body);
|
|
186
|
+
const workers = Array.from({ length: WORKERS }, (_, i) => {
|
|
187
|
+
const share = Math.floor(total / WORKERS) + (i < total % WORKERS ? 1 : 0);
|
|
188
|
+
return new Worker(fileURLToPath(import.meta.url), {
|
|
189
|
+
workerData: { base, method, reqPath, body, total: share },
|
|
101
190
|
});
|
|
191
|
+
});
|
|
192
|
+
const finished = workers.map(
|
|
193
|
+
(worker) =>
|
|
194
|
+
new Promise((resolve, reject) => {
|
|
195
|
+
worker.on('message', (message) => {
|
|
196
|
+
if (message !== 'ready') resolve(message);
|
|
197
|
+
});
|
|
198
|
+
worker.on('error', reject);
|
|
199
|
+
})
|
|
200
|
+
);
|
|
201
|
+
await Promise.all(
|
|
202
|
+
workers.map((worker) => new Promise((resolve) => worker.once('message', resolve)))
|
|
203
|
+
);
|
|
102
204
|
|
|
103
205
|
const t0 = performance.now();
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
const index = dispatched;
|
|
107
|
-
dispatched += 1;
|
|
108
|
-
if (index >= total) return;
|
|
109
|
-
const start = performance.now();
|
|
110
|
-
await request();
|
|
111
|
-
latencies[index] = performance.now() - start;
|
|
112
|
-
}
|
|
113
|
-
});
|
|
114
|
-
await Promise.all(workers);
|
|
206
|
+
for (const worker of workers) worker.postMessage('go');
|
|
207
|
+
const results = await Promise.all(finished);
|
|
115
208
|
const elapsedSec = (performance.now() - t0) / 1000;
|
|
116
|
-
agent.destroy();
|
|
117
209
|
|
|
118
|
-
const sorted = Array.from(latencies).sort((a, b) => a - b);
|
|
210
|
+
const sorted = results.flatMap((latencies) => Array.from(latencies)).sort((a, b) => a - b);
|
|
119
211
|
return {
|
|
120
212
|
rps: total / elapsedSec,
|
|
121
213
|
p50: sorted[Math.floor(total * 0.5)],
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Mandelbrot PNG
|
|
2
|
+
|
|
3
|
+
Renders the Mandelbrot set and writes it as a PNG. The PNG encoder is about
|
|
4
|
+
40 lines of TypeScript on top of `node:zlib` (`deflateSync` for the pixel
|
|
5
|
+
data, `crc32` for the chunk checksums). The program needs no image library,
|
|
6
|
+
no native addon such as `sharp` or `canvas`, and no `node_modules`: the
|
|
7
|
+
compiled binary is a single 3.3 MB file.
|
|
8
|
+
|
|
9
|
+
Demonstrates: a tight floating-point loop, a function returning a tuple that
|
|
10
|
+
the caller destructures, `Uint8Array` / `DataView` big-endian writes,
|
|
11
|
+
`zlib.deflateSync` + `zlib.crc32`, `Math.log2` / `Math.cos`, and
|
|
12
|
+
`fs.writeFileSync` of binary data.
|
|
13
|
+
|
|
14
|
+

|
|
15
|
+
|
|
16
|
+
## Run it
|
|
17
|
+
|
|
18
|
+
With Node.js:
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
node main.ts 1920 1080 mandelbrot.png
|
|
22
|
+
# Rendered 1920x1080 in 533 ms, encoded 416056 bytes in 50 ms -> mandelbrot.png
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
With TypeNative:
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
npx typenative --source main.ts --output dist/mandelbrot.exe
|
|
29
|
+
./dist/mandelbrot.exe 1920 1080 mandelbrot.png
|
|
30
|
+
# Rendered 1920x1080 in 534 ms, encoded 418872 bytes in 46 ms -> mandelbrot.png
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
From a checkout of this repository:
|
|
34
|
+
|
|
35
|
+
```sh
|
|
36
|
+
node bin/cli.js --source examples/mandelbrot-png/main.ts --output dist/mandelbrot.exe
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The two PNGs aren't byte-identical, because Node's deflate is zlib and
|
|
40
|
+
TypeNative's is Go's `compress/flate`. They do decode to **identical pixels**.
|
|
41
|
+
|
|
42
|
+
## Benchmark: Node.js vs TypeNative
|
|
43
|
+
|
|
44
|
+
Reproduce with `node bench.mjs [width=1920] [height=1080] [runs=10]`. The
|
|
45
|
+
script compiles `main.ts`, times **complete process runs** (spawn to exit) of
|
|
46
|
+
both runtimes rendering the same image, then decodes both PNGs (checking every
|
|
47
|
+
chunk CRC) and compares the pixels.
|
|
48
|
+
|
|
49
|
+
**Environment:** Windows 11 Pro, AMD Ryzen 5 5500U, Node v24.19.0, Go 1.26.0,
|
|
50
|
+
TypeNative 0.4.1, default (non-`--release`) build.
|
|
51
|
+
|
|
52
|
+
| Runtime | min | median | mean | max |
|
|
53
|
+
|------------|-----------|-----------|-----------|-----------|
|
|
54
|
+
| Node.js | 678.34 ms | 744.01 ms | 769.12 ms | 918.37 ms |
|
|
55
|
+
| TypeNative | 591.29 ms | 596.63 ms | 599.53 ms | 623.04 ms |
|
|
56
|
+
|
|
57
|
+
**TypeNative is 1.2x faster end to end** (median), and its run-to-run spread
|
|
58
|
+
is much tighter (32 ms vs 240 ms). There's no JIT warm-up and no GC pauses
|
|
59
|
+
from a large heap. The render loop itself takes about the same time in both
|
|
60
|
+
runtimes (~530 ms). V8 compiles a tight float loop like the escape-time
|
|
61
|
+
iteration into machine code about as good as Go's. The difference comes from
|
|
62
|
+
startup (~100 ms) and slightly faster deflate. For recursion and
|
|
63
|
+
call-heavy code the compute gap is larger (see [fibonacci](../fibonacci/),
|
|
64
|
+
1.9x). This example's main advantage is distribution: one dependency-free
|
|
65
|
+
binary that writes images.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
// Render benchmark: Node.js vs TypeNative for the Mandelbrot PNG renderer.
|
|
2
|
+
//
|
|
3
|
+
// Compiles main.ts to a native binary with the TypeNative CLI from this repo
|
|
4
|
+
// checkout (../../bin/cli.js), then times complete process runs (spawn to
|
|
5
|
+
// exit) of both runtimes rendering the same image, and checks that the two
|
|
6
|
+
// PNGs decode to identical pixels. (The compressed bytes differ: Node uses
|
|
7
|
+
// zlib's deflate and TypeNative Go's compress/flate.)
|
|
8
|
+
//
|
|
9
|
+
// Usage: node bench.mjs [width=1920] [height=1080] [runs=10]
|
|
10
|
+
|
|
11
|
+
import { spawnSync } from 'node:child_process';
|
|
12
|
+
import fs from 'node:fs';
|
|
13
|
+
import os from 'node:os';
|
|
14
|
+
import path from 'node:path';
|
|
15
|
+
import zlib from 'node:zlib';
|
|
16
|
+
import { fileURLToPath } from 'node:url';
|
|
17
|
+
|
|
18
|
+
const here = path.dirname(fileURLToPath(import.meta.url));
|
|
19
|
+
const repoRoot = path.resolve(here, '..', '..');
|
|
20
|
+
const cli = path.join(repoRoot, 'bin', 'cli.js');
|
|
21
|
+
const width = process.argv[2] ?? '1920';
|
|
22
|
+
const height = process.argv[3] ?? '1080';
|
|
23
|
+
const runs = Number(process.argv[4] ?? '10') || 10;
|
|
24
|
+
const exeName = process.platform === 'win32' ? 'mandelbrot.exe' : 'mandelbrot';
|
|
25
|
+
const exePath = path.join(here, 'dist', exeName);
|
|
26
|
+
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'mandelbrot-bench-'));
|
|
27
|
+
|
|
28
|
+
function stats(samples) {
|
|
29
|
+
const sorted = [...samples].sort((a, b) => a - b);
|
|
30
|
+
const mean = samples.reduce((sum, x) => sum + x, 0) / samples.length;
|
|
31
|
+
return { min: sorted[0], median: sorted[Math.floor(sorted.length / 2)], mean, max: sorted[sorted.length - 1] };
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
function fmt(ms) {
|
|
35
|
+
return `${ms.toFixed(2)} ms`.padStart(11);
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
// Inflated IDAT data (the raw scanlines), after checking every chunk's CRC
|
|
39
|
+
function decodePixels(file) {
|
|
40
|
+
const png = fs.readFileSync(file);
|
|
41
|
+
const idat = [];
|
|
42
|
+
for (let offset = 8; offset < png.length; ) {
|
|
43
|
+
const length = png.readUInt32BE(offset);
|
|
44
|
+
const typeAndData = png.subarray(offset + 4, offset + 8 + length);
|
|
45
|
+
if (zlib.crc32(typeAndData) !== png.readUInt32BE(offset + 8 + length)) throw new Error(`bad CRC in ${file}`);
|
|
46
|
+
if (typeAndData.toString('latin1', 0, 4) === 'IDAT') idat.push(typeAndData.subarray(4));
|
|
47
|
+
offset += 12 + length;
|
|
48
|
+
}
|
|
49
|
+
return zlib.inflateSync(Buffer.concat(idat));
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
function timeRuns(command, args) {
|
|
53
|
+
const samples = [];
|
|
54
|
+
for (let i = 0; i < runs; i++) {
|
|
55
|
+
const t0 = performance.now();
|
|
56
|
+
const result = spawnSync(command, args, { cwd: here, encoding: 'utf8' });
|
|
57
|
+
const elapsed = performance.now() - t0;
|
|
58
|
+
if (result.status !== 0) {
|
|
59
|
+
console.error(`run failed: ${command} ${args.join(' ')}\n${result.stderr}`);
|
|
60
|
+
process.exit(1);
|
|
61
|
+
}
|
|
62
|
+
samples.push(elapsed);
|
|
63
|
+
}
|
|
64
|
+
return stats(samples);
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
// 1. Compile the renderer to a native binary.
|
|
68
|
+
fs.mkdirSync(path.join(here, 'dist'), { recursive: true });
|
|
69
|
+
console.log('Compiling main.ts to a native binary with TypeNative...');
|
|
70
|
+
const build = spawnSync(process.execPath, [cli, '--source', 'main.ts', '--output', path.join('dist', exeName)], {
|
|
71
|
+
cwd: here,
|
|
72
|
+
stdio: 'inherit',
|
|
73
|
+
});
|
|
74
|
+
if (build.status !== 0) process.exit(build.status ?? 1);
|
|
75
|
+
|
|
76
|
+
// 2. Time full renders under both runtimes.
|
|
77
|
+
const nodePng = path.join(tmpDir, 'node.png');
|
|
78
|
+
const nativePng = path.join(tmpDir, 'native.png');
|
|
79
|
+
console.log(`\nTiming ${runs} full process runs of each runtime at ${width}x${height}...\n`);
|
|
80
|
+
const nodeStats = timeRuns(process.execPath, ['main.ts', width, height, nodePng]);
|
|
81
|
+
const nativeStats = timeRuns(exePath, [width, height, nativePng]);
|
|
82
|
+
|
|
83
|
+
const nodePixels = decodePixels(nodePng);
|
|
84
|
+
const same = nodePixels.equals(decodePixels(nativePng));
|
|
85
|
+
const sizes = [fs.statSync(nodePng).size, fs.statSync(nativePng).size];
|
|
86
|
+
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
87
|
+
|
|
88
|
+
console.log(`Render + PNG encode + write, ${runs} runs (full process: spawn -> exit):\n`);
|
|
89
|
+
console.log('| Runtime | min | median | mean | max |');
|
|
90
|
+
console.log('|------------|-------------|-------------|-------------|-------------|');
|
|
91
|
+
console.log(`| Node.js |${fmt(nodeStats.min)} |${fmt(nodeStats.median)} |${fmt(nodeStats.mean)} |${fmt(nodeStats.max)} |`);
|
|
92
|
+
console.log(`| TypeNative |${fmt(nativeStats.min)} |${fmt(nativeStats.median)} |${fmt(nativeStats.mean)} |${fmt(nativeStats.max)} |`);
|
|
93
|
+
console.log(`\nPNG sizes: Node ${sizes[0]} B, TypeNative ${sizes[1]} B; decoded pixels identical: ${same}`);
|
|
94
|
+
console.log(`TypeNative is ${(nodeStats.median / nativeStats.median).toFixed(1)}x faster (median).`);
|
|
95
|
+
if (!same) process.exit(1);
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
// Renders the Mandelbrot set to a PNG file. The PNG encoder is written here
|
|
2
|
+
// by hand on top of node:zlib (deflate + CRC-32), so the program needs no
|
|
3
|
+
// image library under Node.js or TypeNative.
|
|
4
|
+
//
|
|
5
|
+
// node main.ts [width=1920] [height=1080] [out=mandelbrot.png]
|
|
6
|
+
// ./mandelbrot [width] [height] [out]
|
|
7
|
+
import fs from 'node:fs';
|
|
8
|
+
import zlib from 'node:zlib';
|
|
9
|
+
|
|
10
|
+
const width = Number(process.argv[2] ?? '1920');
|
|
11
|
+
const height = Number(process.argv[3] ?? '1080');
|
|
12
|
+
const out = process.argv[4] ?? 'mandelbrot.png';
|
|
13
|
+
const maxIter = 500;
|
|
14
|
+
|
|
15
|
+
// Smooth escape-time colouring: a cosine palette indexed by the
|
|
16
|
+
// continuous iteration count, black inside the set
|
|
17
|
+
function colorAt(cx: number, cy: number): [number, number, number] {
|
|
18
|
+
let x = 0;
|
|
19
|
+
let y = 0;
|
|
20
|
+
let i = 0;
|
|
21
|
+
while (x * x + y * y <= 256 && i < maxIter) {
|
|
22
|
+
const xt = x * x - y * y + cx;
|
|
23
|
+
y = 2 * x * y + cy;
|
|
24
|
+
x = xt;
|
|
25
|
+
i++;
|
|
26
|
+
}
|
|
27
|
+
if (i === maxIter) return [0, 0, 0];
|
|
28
|
+
const smooth = i + 1 - Math.log2(Math.log2(x * x + y * y) / 2);
|
|
29
|
+
const t = 0.55 + Math.sqrt(smooth) * 0.11;
|
|
30
|
+
const r = Math.round(255 * (0.5 + 0.5 * Math.cos(6.2832 * (t + 0.0))));
|
|
31
|
+
const g = Math.round(255 * (0.5 + 0.5 * Math.cos(6.2832 * (t + 0.1))));
|
|
32
|
+
const b = Math.round(255 * (0.5 + 0.5 * Math.cos(6.2832 * (t + 0.2))));
|
|
33
|
+
return [r, g, b];
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
// Raw PNG scanlines: each row is a filter byte (0 = none) then RGB triples
|
|
37
|
+
function render(): Uint8Array {
|
|
38
|
+
const stride = width * 3 + 1;
|
|
39
|
+
const pixels = new Uint8Array(stride * height);
|
|
40
|
+
const scale = 3 / height;
|
|
41
|
+
for (let py = 0; py < height; py++) {
|
|
42
|
+
const cy = (py - height / 2) * scale;
|
|
43
|
+
let offset = py * stride + 1;
|
|
44
|
+
for (let px = 0; px < width; px++) {
|
|
45
|
+
const cx = (px - width * 0.65) * scale;
|
|
46
|
+
const [r, g, b] = colorAt(cx, cy);
|
|
47
|
+
pixels[offset] = r;
|
|
48
|
+
pixels[offset + 1] = g;
|
|
49
|
+
pixels[offset + 2] = b;
|
|
50
|
+
offset += 3;
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
return pixels;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
// One PNG chunk: length, type, data, CRC-32 of type + data
|
|
57
|
+
function chunk(type: string, data: Uint8Array): Uint8Array {
|
|
58
|
+
const typed = new Uint8Array(4 + data.length);
|
|
59
|
+
for (let i = 0; i < 4; i++) typed[i] = type.charCodeAt(i);
|
|
60
|
+
typed.set(data, 4);
|
|
61
|
+
const result = new Uint8Array(12 + data.length);
|
|
62
|
+
const view = new DataView(result.buffer);
|
|
63
|
+
view.setUint32(0, data.length);
|
|
64
|
+
result.set(typed, 4);
|
|
65
|
+
view.setUint32(8 + data.length, zlib.crc32(typed));
|
|
66
|
+
return result;
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
function encodePng(pixels: Uint8Array): Uint8Array {
|
|
70
|
+
const header = new Uint8Array(13);
|
|
71
|
+
const view = new DataView(header.buffer);
|
|
72
|
+
view.setUint32(0, width);
|
|
73
|
+
view.setUint32(4, height);
|
|
74
|
+
header[8] = 8; // bit depth
|
|
75
|
+
header[9] = 2; // colour type: truecolour RGB
|
|
76
|
+
const parts = [
|
|
77
|
+
new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
|
78
|
+
chunk('IHDR', header),
|
|
79
|
+
chunk('IDAT', new Uint8Array(zlib.deflateSync(pixels))),
|
|
80
|
+
chunk('IEND', new Uint8Array(0))
|
|
81
|
+
];
|
|
82
|
+
const total = parts.reduce((sum, p) => sum + p.length, 0);
|
|
83
|
+
const png = new Uint8Array(total);
|
|
84
|
+
let offset = 0;
|
|
85
|
+
for (const part of parts) {
|
|
86
|
+
png.set(part, offset);
|
|
87
|
+
offset += part.length;
|
|
88
|
+
}
|
|
89
|
+
return png;
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
const t0 = performance.now();
|
|
93
|
+
const pixels = render();
|
|
94
|
+
const t1 = performance.now();
|
|
95
|
+
const png = encodePng(pixels);
|
|
96
|
+
fs.writeFileSync(out, png);
|
|
97
|
+
const t2 = performance.now();
|
|
98
|
+
console.log(`Rendered ${width}x${height} in ${Math.round(t1 - t0)} ms, encoded ${png.length} bytes in ${Math.round(t2 - t1)} ms -> ${out}`);
|
|
Binary file
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# marked CLI
|
|
2
|
+
|
|
3
|
+
The [`marked`](https://github.com/markedjs/marked) package's own command-line
|
|
4
|
+
tool, compiled from its **unmodified** upstream TypeScript source: about 4,000
|
|
5
|
+
lines across 13 files, vendored as is in [vendor/marked/](vendor/marked/) (see
|
|
6
|
+
[VENDOR.md](vendor/marked/VENDOR.md): no patches). [main.ts](main.ts) ports
|
|
7
|
+
upstream's `bin/main.js` to TypeScript. It parses the same options, reads the
|
|
8
|
+
markdown from a string, a file or stdin, and prints the HTML or the lexer's
|
|
9
|
+
token list.
|
|
10
|
+
|
|
11
|
+
This is the second test of TypeNative on real code nobody wrote for it. marked
|
|
12
|
+
is a markdown parser built on regexes: 66 definitions in `rules.ts`, with GFM
|
|
13
|
+
and pedantic variants, many composed at run time by a fluent
|
|
14
|
+
`edit(…).replace(…)` builder (44 `edit` calls). Many use lookarounds or backreferences, which
|
|
15
|
+
Go's RE2 engine doesn't support, so they run on TypeNative's backtracking
|
|
16
|
+
engine. Tokens are structural object types, arrays of tokens are
|
|
17
|
+
filled in later through a queue that holds the same arrays, and the `--tokens`
|
|
18
|
+
JSON must match V8's key order and leave out `undefined` fields, such as the
|
|
19
|
+
title of a link definition that has none.
|
|
20
|
+
|
|
21
|
+
## Usage
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
marked < in.md # render markdown to HTML
|
|
25
|
+
marked -s '*hi*' # render a string
|
|
26
|
+
marked -i in.md -o out.html # read a file, write a file
|
|
27
|
+
marked --tokens < in.md # print the token list as JSON
|
|
28
|
+
marked --no-gfm --pedantic # any boolean option of marked.defaults
|
|
29
|
+
marked --help # all options
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Differences from `bin/main.js` (listed in main.ts's header): `--help` prints an
|
|
33
|
+
option summary instead of opening the man page, `--version` prints the vendored
|
|
34
|
+
version, and a config file (`-c`, `~/.marked.json`) must be JSON. A JavaScript
|
|
35
|
+
config is a module Node would import and run, which a native binary can't. The
|
|
36
|
+
HTML is also flushed before the process exits, under Node too: `bin/main.js`
|
|
37
|
+
exits right after writing, which cuts output to a pipe off at 64 KB.
|
|
38
|
+
|
|
39
|
+
## Run it
|
|
40
|
+
|
|
41
|
+
With Node.js (24 or later runs the TypeScript directly):
|
|
42
|
+
|
|
43
|
+
```sh
|
|
44
|
+
node main.ts < README.md
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
With TypeNative:
|
|
48
|
+
|
|
49
|
+
```sh
|
|
50
|
+
npx typenative --source main.ts --output dist/marked
|
|
51
|
+
./dist/marked < README.md
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
From a checkout of this repository:
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
node bin/cli.js --source examples/marked-cli/main.ts --output dist/marked
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## Parity with Node.js
|
|
61
|
+
|
|
62
|
+
Reproduce with `node parity.mjs [filter]`. The script fetches marked's own spec
|
|
63
|
+
suites (`test/specs` at tag `v18.1.2`) into `.cache/` (it isn't committed) and
|
|
64
|
+
compiles `main.ts`. It then feeds each case's markdown on stdin to both
|
|
65
|
+
runtimes in three modes: default (GFM), `--no-gfm` and `--tokens`. A case's
|
|
66
|
+
front matter (`gfm`, `pedantic`, `breaks`) becomes CLI flags, as marked's test
|
|
67
|
+
runner turns it into options. stdout, stderr and the exit code must match byte
|
|
68
|
+
for byte. The only normalization is on stderr: Node's ` at …` stack frames
|
|
69
|
+
are dropped (a native binary has no JS frames to print). Each run has a 10 s
|
|
70
|
+
timeout. Failures are written to `.cache/failures/`.
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
2859/2859 cases match (100.00%)
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
That's 953 inputs × 3 modes: CommonMark 0.31.2 (652 examples), GFM 0.29 (28),
|
|
77
|
+
marked's own `new` (223) and `original` (22) tests, and its 28 ReDoS
|
|
78
|
+
regression inputs. The ReDoS inputs are pathological on purpose, such as 10,000
|
|
79
|
+
underscores in a row. Every one of them finishes within the timeout natively
|
|
80
|
+
too (see the benchmark notes below for how long they take).
|
|
81
|
+
|
|
82
|
+
## Benchmark: Node.js vs TypeNative
|
|
83
|
+
|
|
84
|
+
Reproduce with `node bench.mjs [runs=30] [throughputRuns=5] [megabytes=10]`.
|
|
85
|
+
The script compiles `main.ts`, then times **complete process runs** (spawn to
|
|
86
|
+
exit) of `marked` under both runtimes, alternating Node and native runs, and
|
|
87
|
+
checks that the two HTML outputs are identical:
|
|
88
|
+
|
|
89
|
+
- **(a) per-call latency:** marked's own README.md (3.5 KB), the kind of file
|
|
90
|
+
a docs build or a git hook renders once per file.
|
|
91
|
+
- **(b) throughput:** 10 MB of generated markdown (headings, paragraphs with
|
|
92
|
+
emphasis, links and code spans, lists, tables, code fences and block quotes,
|
|
93
|
+
from a fixed-seed generator, so every machine gets the same input).
|
|
94
|
+
|
|
95
|
+
**Environment:** Linux 6.18 container, 4 vCPUs of an Intel Xeon @ 2.10GHz,
|
|
96
|
+
Node v24.22.1, Go 1.26.0, TypeNative 0.4.2 (unreleased changes), default
|
|
97
|
+
(non-`--release`) build. The binary is 6.3 MB.
|
|
98
|
+
|
|
99
|
+
(a) `marked < README.md` (3.5 KB), 30 runs each:
|
|
100
|
+
|
|
101
|
+
| Runtime | min | median | mean | max |
|
|
102
|
+
| ---------- | --------- | --------- | --------- | --------- |
|
|
103
|
+
| Node.js | 129.00 ms | 148.15 ms | 148.71 ms | 191.32 ms |
|
|
104
|
+
| TypeNative | 18.04 ms | 19.46 ms | 19.71 ms | 25.27 ms |
|
|
105
|
+
|
|
106
|
+
(b) `marked < generated.md` (10.0 MB, 114,893 lines), 5 runs each:
|
|
107
|
+
|
|
108
|
+
| Runtime | min | median | mean | max | MB/s (median) |
|
|
109
|
+
| ---------- | --------- | --------- | --------- | --------- | ------------- |
|
|
110
|
+
| Node.js | 1,969 ms | 2,053 ms | 2,056 ms | 2,164 ms | 4.87 |
|
|
111
|
+
| TypeNative | 10,378 ms | 10,573 ms | 10,685 ms | 11,006 ms | 0.95 |
|
|
112
|
+
|
|
113
|
+
**Per call, TypeNative is 7.6x faster** (19 ms vs 148 ms, median). Node's
|
|
114
|
+
cost here is mostly startup: it loads and type-strips the 13 TypeScript files
|
|
115
|
+
and compiles marked's regexes on every run. That's what you feel when a docs
|
|
116
|
+
build or a hook renders one file per process.
|
|
117
|
+
|
|
118
|
+
**On bulk throughput, TypeNative is 5.2x slower.** On a long input V8's JIT
|
|
119
|
+
and its regex engine (Irregexp compiles hot patterns to machine code) run
|
|
120
|
+
marked faster than the compiled Go does. In a profile of the native run on
|
|
121
|
+
1 MB, lexing takes half the CPU and TypeNative's backtracking regex engine
|
|
122
|
+
about a third of it: marked's patterns use lookarounds and backreferences, so
|
|
123
|
+
most of them can't run on Go's RE2, and the backtracker interprets them node
|
|
124
|
+
by node. Garbage collection takes about a third more on the other cores (each
|
|
125
|
+
token is an object, each match a few strings). Both costs grow linearly with
|
|
126
|
+
the input, while Node's head start is fixed, so the native binary finishes
|
|
127
|
+
first up to about 150–200 KB of markdown per process:
|
|
128
|
+
|
|
129
|
+
| Input | Node.js | TypeNative |
|
|
130
|
+
| ------ | ------- | ---------- |
|
|
131
|
+
| 16 KB | 153 ms | 37 ms |
|
|
132
|
+
| 64 KB | 167 ms | 96 ms |
|
|
133
|
+
| 128 KB | 182 ms | 149 ms |
|
|
134
|
+
| 256 KB | 215 ms | 287 ms |
|
|
135
|
+
| 1 MB | 377 ms | 1,051 ms |
|
|
136
|
+
|
|
137
|
+
(Medians of 5 full process runs each, same generator.)
|
|
138
|
+
|
|
139
|
+
The ReDoS inputs show the same split. The 28 of them take 3.4 s natively in
|
|
140
|
+
total and 4.1 s under Node, but the worst one, 10,000 underscores
|
|
141
|
+
(`quadratic_emstrong_delim`), takes 1.4 s natively against Node's 0.25 s. Each
|
|
142
|
+
underscore restarts a pattern that scans the rest of the run, so the work is
|
|
143
|
+
quadratic in both engines and the backtracker's constant factor shows.
|
|
144
|
+
|
|
145
|
+
Getting here fixed performance bugs in TypeNative's runtime and compiler, all
|
|
146
|
+
general:
|
|
147
|
+
|
|
148
|
+
- The backtracking engine converted the whole input to runes on every match
|
|
149
|
+
attempt, making a lexer that matches at each position quadratic. It now
|
|
150
|
+
works on byte offsets.
|
|
151
|
+
- A greedy repeat of one character class no longer allocates, and ASCII is
|
|
152
|
+
tested against a table, which took the worst ReDoS input from 6.3 s to 1.4 s.
|
|
153
|
+
- A regex whose match must start with certain bytes skips ahead to them, and
|
|
154
|
+
an anchored one fails at once, instead of RE2 scanning the rest of a long
|
|
155
|
+
input on every call.
|
|
156
|
+
- Strings built with `+=` in a loop are `strings.Builder`s, so rendering a
|
|
157
|
+
long document is linear.
|
|
158
|
+
- Type inference is memoized per query. marked's
|
|
159
|
+
`edit(re).replace(…).replace(…)…` chains took exponential time to compile.
|