@dimina-kit/compiler 0.0.1-dev.20260702173719 → 0.0.1-dev.20260702182435
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -18
- package/dist/pool.browser.js +1 -1
- package/dist/stage-worker.browser.js +9 -2
- package/package.json +3 -2
- package/scripts/test-stage-toolchain.js +263 -0
- package/src/pool.js +3 -1
- package/src/stage-worker.js +18 -2
package/README.md
CHANGED
|
@@ -71,10 +71,12 @@ const { appId, name, files } = await pool.compile({ files: source, workPath: '/p
|
|
|
71
71
|
pool.dispose() // 用完终止 worker
|
|
72
72
|
```
|
|
73
73
|
|
|
74
|
-
> **上面是最小接入代码,但浏览器环境有前置要求**(不满足会在别处报错,不像示例不完整):页面与 worker 开 COOP/COEP;`toolchainSetupURL` 能被 worker 运行时 import;esbuild browser ESM 与 `esbuild.wasm` 可作静态资源访问;npm 用户需显式装 `@oxc-parser/binding-wasm32-wasi`(`cpu:wasm32` 会被跳过);本包经 `file:`/link 引用时要配 Vite `server.fs.allow
|
|
74
|
+
> **上面是最小接入代码,但浏览器环境有前置要求**(不满足会在别处报错,不像示例不完整):页面与 worker 开 COOP/COEP;`toolchainSetupURL` 能被 worker 运行时 import;esbuild browser ESM 与 `esbuild.wasm` 可作静态资源访问;npm 用户需显式装 `@oxc-parser/binding-wasm32-wasi`(`cpu:wasm32` 会被跳过);本包经 `file:`/link 引用时要配 Vite `server.fs.allow`。逐条见本节末尾的「故障排查」。
|
|
75
75
|
|
|
76
76
|
`compile()` 内部:三个常驻 worker **各跑一个完整 stage**(logic / view / style),各自 seed 私有 memfs、各自 `setupCompile` + 编译该 stage,产物取并集回传;跨 `compile()` 复用同一批 worker(每次编译前自动 `resetCompilerState`)。多个 `compile()` 会自动串行(共享常驻 realm 不能并发)。
|
|
77
77
|
|
|
78
|
+
> **连续触发(watch / 保存即编)要自己收敛:** pool 只保证串行,**不合并**——编译进行中再调 N 次 `compile()` 就排队 N 个全量编译,一个不少地跑完(正确性没问题,但白烧算力、垫高最后一次的延迟)。源码快照在**轮到该次编译派发时**才结构化克隆,不是调用时。触发侧应收敛成"至多一个在跑 + 一个待跑"(dirty-flag / debounce)。另外**整个应用共用一个 pool 单例**:多实例之间 realm 全隔离、不会互相污染,但结果没有跨实例顺序保证(旧编译可能晚于新编译返回,按到达顺序应用产物会拿旧盖新),还多付一份工具链常驻内存。
|
|
79
|
+
|
|
78
80
|
```ts
|
|
79
81
|
createCompilerPool(options: {
|
|
80
82
|
createWorker: () => Worker // 必填:起 stage worker 的工厂;pool 按 stage 数(默认 3)调用它多次
|
|
@@ -121,12 +123,30 @@ await installEsbuildFromURL('/esbuild-browser.mjs', '/esbuild.wasm') // 见下
|
|
|
121
123
|
- **`installOxc(oxcModule)`**:把 `await import('oxc-parser')`(你的 bundler 才能解析它 + 取它的 wasm)装成 `__oxcParseSync`。
|
|
122
124
|
- **`installEsbuildFromURL(moduleURL, wasmURL)`**:从**静态资源 URL** 加载 esbuild-wasm 浏览器 ESM 并 `initialize`,装成 `__esbuildTransform`。
|
|
123
125
|
|
|
126
|
+
> **不是每个 worker 都会加载它:** style stage 的编译链路不调用这两个 wasm 钩子(CSS 工具链已内联,见下),所以 **style worker 在 warmup/compile 时会直接跳过 `import(toolchainSetupURL)`**——省掉一份 esbuild.wasm(~13MB)+ oxc 的加载与常驻内存。因此 **setup 模块只应安装这两个钩子**,不要在里面夹带编译所依赖的其他全局(style worker 看不到它们);未知的自定义 stage 会保守照旧加载。
|
|
127
|
+
>
|
|
124
128
|
> **为什么是 URL 而不是 `import`:** esbuild-wasm 的浏览器构建通常只能当**静态资源**托管(把它的 Go 运行时打包会坏),而 bundler(Vite/Webpack/Rollup)**不允许** `import()` 一个静态资源目录里的 JS——`installEsbuildFromURL` 内部用 `fetch + Blob URL` 绕过 bundler 的模块图,替你把这个坑填了。若你的 esbuild-wasm 是 npm 依赖(bundler 能解析),也可以自己 `import * as esbuild from 'esbuild-wasm'` + `esbuild.initialize({ wasmURL })` + 设 `globalThis.__esbuildTransform`,不必用这个助手。
|
|
125
129
|
>
|
|
126
|
-
> **cross-origin isolation:** oxc 的 wasm32-wasi 绑定用到 SharedArrayBuffer,页面与 worker 需开 COOP/COEP(`Cross-Origin-Opener-Policy: same-origin` + `Cross-Origin-Embedder-Policy: require-corp`)。**CSS 工具链不用宿主管**:浏览器 bundle 已内联真实 `cssnano` + `autoprefixer`(autoprefixer pin 到 node 版 `compile-core.node.js` 运行时解析的同一份,当前 10.5.
|
|
130
|
+
> **cross-origin isolation:** oxc 的 wasm32-wasi 绑定用到 SharedArrayBuffer,页面与 worker 需开 COOP/COEP(`Cross-Origin-Opener-Policy: same-origin` + `Cross-Origin-Embedder-Policy: require-corp`)。**CSS 工具链不用宿主管**:浏览器 bundle 已内联真实 `cssnano` + `autoprefixer`(autoprefixer pin 到 node 版 `compile-core.node.js` 运行时解析的同一份,当前 10.5.2),CSS 产物与 node 版逐字节一致。宿主只需提供 esbuild + oxc 两个 wasm 钩子。
|
|
127
131
|
|
|
128
132
|
参考实现与验证:`dimina-web-client` 的 `demo/toolchain-setup.js`(用上面两个助手)+ `demo/pool-test.html`,`npm run test:pool`(三项目产物与单线程逐结构一致,并演示 `onLog`)。
|
|
129
133
|
|
|
134
|
+
### 故障排查(接入环境)
|
|
135
|
+
|
|
136
|
+
pool 的 API 本身很小,但**浏览器/bundler/包管理器环境**有几个已知坑,报错往往指向别处、很难一眼看出是环境问题。按现象对号入座:
|
|
137
|
+
|
|
138
|
+
- **worker 请求 403 `outside of Vite serving allow list`(本包通过 `file:` / `pnpm link` / monorepo workspace 引用时)。** `new Worker(new URL('@dimina-kit/compiler/stage-worker', import.meta.url))` 是**运行时懒解析**,不走 Vite 依赖爬虫的预热白名单,而本包真实目录在你项目 root 之外 → 被 `server.fs.allow` 拦。把本包的**真实目录**(symlink resolve 后的 realpath,不是 node_modules 里的软链)加进 `server.fs.allow`:
|
|
139
|
+
```js
|
|
140
|
+
// vite.config.js
|
|
141
|
+
import fs from 'node:fs'; import path from 'node:path'
|
|
142
|
+
const wcDir = fs.realpathSync(path.dirname(new URL('./node_modules/@dimina-kit/compiler/package.json', import.meta.url).pathname))
|
|
143
|
+
export default defineConfig({ server: { fs: { allow: ['.', wcDir] } }, /* … */ })
|
|
144
|
+
```
|
|
145
|
+
(静态 `import` 的 `pool` 能穿透是因为它被爬虫预热了——这个不对称很反直觉。)
|
|
146
|
+
- **`Failed to resolve import "@oxc-parser/binding-wasm32-wasi"`(用 npm、非 pnpm 时)。** `oxc-parser` 的 wasm binding 的 `package.json` 写了 `"cpu": "wasm32"`,**npm 会把它当成和 arm64/x64 互斥的真实架构而跳过安装**(optionalDependencies 平台过滤)。把它**显式**列进你自己的 `dependencies`(别指望 optional 自动装):`npm i @oxc-parser/binding-wasm32-wasi`。pnpm 一般不会踩这个。
|
|
147
|
+
- **`toolchainSetupURL` 这个文件放哪。** 它被 worker 在运行时 `import(url)`。放 `public/`(静态资源、URL 稳定)最省心,dev/prod 都成立;若放 `src/` 用 bundler 处理,dev 能过但 `vite build` 需要把它声明为独立 entry(`build.rollupOptions.input`),否则生产构建不产出这个文件。
|
|
148
|
+
- **dev 模式下 3 条 `[vite] connecting…` 噪音**:3 个 stage worker 各建一条 HMR 连接,属正常,不是 worker 池重复初始化。
|
|
149
|
+
|
|
130
150
|
## Node 常驻 pool(`./pool-node`,devtools 在用)
|
|
131
151
|
|
|
132
152
|
Node 宿主(Electron devtools、CLI watch 服务等)用这个导出替代直接依赖 dmcc(`@dimina/compiler`)。它复刻 dmcc 的磁盘编译流程——主线程准备(配置/产物目录/npm 包)→ 3 个 stage worker 并发写共享 staging → 发布到 `outputDir/{appId}`——**产物与 dmcc 逐字节等价**(含 sourcemap 与自定义 `fileTypes`),区别只有一个:**worker 常驻**。
|
|
@@ -301,22 +321,6 @@ async function filesFromDir(dir, prefix = '') { // 只读递来的 han
|
|
|
301
321
|
- **`targetPath` 来自环境。** compiler 产物目录取 `process.env.TARGET_PATH`,否则 `os.tmpdir()/dimina-fe-dist-<时间戳>`(浏览器 os shim 下通常 `/tmp/...`)。`setupCompile` 会先 `rmSync` 清空它——别把 `TARGET_PATH` 指到源码目录或共享目录。用 `setupCompile` 返回的 `targetPath` 喂 `collectOutputs`。
|
|
302
322
|
- **不是全 fail-fast。** 缺 fs 方法、坏 appid、坏 `project.config.json`、miniprogram_npm 构建失败会 **reject**;但**样式预处理器失败(如当前浏览器构建暂不支持 `.less`)会被吞掉、降级用原始 CSS**,PostCSS 解析失败返回空串,资源拷贝失败只 `console.log`,logic esbuild 压缩失败回退未压缩代码。**用 pool 时把这些拿出来的办法是 `createCompilerPool({ onLog })`**——它把 worker 内编译器的 `console.*` 诊断(带 stage 标签)转发给你;也可在产物为空/缺失时二次校验。
|
|
303
323
|
|
|
304
|
-
## 故障排查(接入环境)
|
|
305
|
-
|
|
306
|
-
pool 的 API 本身很小,但**浏览器/bundler/包管理器环境**有几个已知坑,报错往往指向别处、很难一眼看出是环境问题。按现象对号入座:
|
|
307
|
-
|
|
308
|
-
- **worker 请求 403 `outside of Vite serving allow list`(本包通过 `file:` / `pnpm link` / monorepo workspace 引用时)。** `new Worker(new URL('@dimina-kit/compiler/stage-worker', import.meta.url))` 是**运行时懒解析**,不走 Vite 依赖爬虫的预热白名单,而本包真实目录在你项目 root 之外 → 被 `server.fs.allow` 拦。把本包的**真实目录**(symlink resolve 后的 realpath,不是 node_modules 里的软链)加进 `server.fs.allow`:
|
|
309
|
-
```js
|
|
310
|
-
// vite.config.js
|
|
311
|
-
import fs from 'node:fs'; import path from 'node:path'
|
|
312
|
-
const wcDir = fs.realpathSync(path.dirname(new URL('./node_modules/@dimina-kit/compiler/package.json', import.meta.url).pathname))
|
|
313
|
-
export default defineConfig({ server: { fs: { allow: ['.', wcDir] } }, /* … */ })
|
|
314
|
-
```
|
|
315
|
-
(静态 `import` 的 `pool` 能穿透是因为它被爬虫预热了——这个不对称很反直觉。)
|
|
316
|
-
- **`Failed to resolve import "@oxc-parser/binding-wasm32-wasi"`(用 npm、非 pnpm 时)。** `oxc-parser` 的 wasm binding 的 `package.json` 写了 `"cpu": "wasm32"`,**npm 会把它当成和 arm64/x64 互斥的真实架构而跳过安装**(optionalDependencies 平台过滤)。把它**显式**列进你自己的 `dependencies`(别指望 optional 自动装):`npm i @oxc-parser/binding-wasm32-wasi`。pnpm 一般不会踩这个。
|
|
317
|
-
- **`toolchainSetupURL` 这个文件放哪。** 它被 worker 在运行时 `import(url)`。放 `public/`(静态资源、URL 稳定)最省心,dev/prod 都成立;若放 `src/` 用 bundler 处理,dev 能过但 `vite build` 需要把它声明为独立 entry(`build.rollupOptions.input`),否则生产构建不产出这个文件。
|
|
318
|
-
- **dev 模式下 3 条 `[vite] connecting…` 噪音**:3 个 stage worker 各建一条 HMR 连接,属正常,不是 worker 池重复初始化。
|
|
319
|
-
|
|
320
324
|
## 依赖前置
|
|
321
325
|
|
|
322
326
|
编译器实体源码在 `dimina` 子模块里,dart-sass 等在其 fe workspace。构建前确保子模块已初始化、依赖已装:
|
package/dist/pool.browser.js
CHANGED
|
@@ -49,7 +49,7 @@ function createCompilerPool(options = {}) {
|
|
|
49
49
|
let warmed = null;
|
|
50
50
|
async function warmup() {
|
|
51
51
|
if (!warmed) {
|
|
52
|
-
warmed = Promise.all(workers.map((x) => x.send({ type: "warmup", toolchainSetupURL }))).then((rs) => rs.forEach((r, i) => {
|
|
52
|
+
warmed = Promise.all(workers.map((x) => x.send({ type: "warmup", toolchainSetupURL, stages: [x.stage] }))).then((rs) => rs.forEach((r, i) => {
|
|
53
53
|
if (r && r.type === "error") throw new Error(r.error || `[compiler] stage '${workers[i].stage}' warmup failed`);
|
|
54
54
|
})).catch((err) => {
|
|
55
55
|
warmed = null;
|
|
@@ -290964,6 +290964,11 @@ function ensureToolchain(url) {
|
|
|
290964
290964
|
}
|
|
290965
290965
|
return toolchainReady;
|
|
290966
290966
|
}
|
|
290967
|
+
var TOOLCHAIN_FREE_STAGES = /* @__PURE__ */ new Set(["style"]);
|
|
290968
|
+
function needsToolchain(stages) {
|
|
290969
|
+
if (!Array.isArray(stages) || stages.length === 0) return true;
|
|
290970
|
+
return stages.some((s) => !TOOLCHAIN_FREE_STAGES.has(s));
|
|
290971
|
+
}
|
|
290967
290972
|
function freshFs(files, workPath) {
|
|
290968
290973
|
return (0, import_memfs.createFsFromVolume)(import_memfs.Volume.fromJSON(files, workPath));
|
|
290969
290974
|
}
|
|
@@ -290984,13 +290989,15 @@ self.onmessage = async (e) => {
|
|
|
290984
290989
|
try {
|
|
290985
290990
|
if (type === "warmup") {
|
|
290986
290991
|
const t0 = performance.now();
|
|
290987
|
-
|
|
290992
|
+
if (e.data.toolchainSetupURL) toolchainURL = e.data.toolchainSetupURL;
|
|
290993
|
+
if (needsToolchain(e.data.stages)) await ensureToolchain();
|
|
290988
290994
|
self.postMessage({ type: "ready", ms: Math.round(performance.now() - t0) });
|
|
290989
290995
|
return;
|
|
290990
290996
|
}
|
|
290991
290997
|
if (type === "compile-subset") {
|
|
290992
290998
|
const { files, workPath = "/work", stages = ["logic", "view", "style"], toolchainSetupURL } = e.data;
|
|
290993
|
-
|
|
290999
|
+
if (toolchainSetupURL) toolchainURL = toolchainSetupURL;
|
|
291000
|
+
if (needsToolchain(stages)) await ensureToolchain();
|
|
290994
291001
|
const warm = !!toolchainReady;
|
|
290995
291002
|
const t = performance.now();
|
|
290996
291003
|
const result2 = await compileSubset(files, workPath, stages);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@dimina-kit/compiler",
|
|
3
|
-
"version": "0.0.1-dev.
|
|
3
|
+
"version": "0.0.1-dev.20260702182435",
|
|
4
4
|
"description": "dmcc compiler bundles (browser + node) that drive @dimina/compiler against a caller-injected node:fs replacement (no bundled fs)",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -82,6 +82,7 @@
|
|
|
82
82
|
"test:realm-reuse": "node scripts/build-compiler.js node && node --import ./scripts/register-kit.js scripts/test-realm-reuse.js",
|
|
83
83
|
"test:pool-node": "node scripts/build-compiler.js node && node --import ./scripts/register-kit.js scripts/test-pool-node.js",
|
|
84
84
|
"test:pool-hardening": "node scripts/build-compiler.js node && node scripts/test-pool-hardening.js",
|
|
85
|
-
"test:npm-scan": "node scripts/build-compiler.js node && node --import ./scripts/register-kit.js scripts/test-npm-scan.js"
|
|
85
|
+
"test:npm-scan": "node scripts/build-compiler.js node && node --import ./scripts/register-kit.js scripts/test-npm-scan.js",
|
|
86
|
+
"test:stage-toolchain": "node scripts/build-compiler.js browser && node scripts/test-stage-toolchain.js"
|
|
86
87
|
}
|
|
87
88
|
}
|
|
@@ -0,0 +1,263 @@
|
|
|
1
|
+
// Per-stage toolchain skip: dist/stage-worker.browser.js currently `import()`s the
|
|
2
|
+
// host's wasm toolchainSetupURL (esbuild-wasm + oxc) on EVERY warmup and
|
|
3
|
+
// compile-subset call, regardless of which stage the worker actually runs. The
|
|
4
|
+
// style stage's compile path (postcss/cssnano/autoprefixer) is bundled in and never
|
|
5
|
+
// touches `__esbuildTransform`/`__oxcParseSync`, so a style-only worker paying that
|
|
6
|
+
// import cost is pure waste — and, observably, it means a style worker can't warm up
|
|
7
|
+
// at all if the host's toolchainSetupURL is broken/unreachable, even though style
|
|
8
|
+
// never needed it.
|
|
9
|
+
//
|
|
10
|
+
// This drives the raw worker message protocol directly (no bundler / real Worker):
|
|
11
|
+
// dist/stage-worker.browser.js is a self-contained ESM (browser platform, memfs +
|
|
12
|
+
// browser shims inlined), so it can be `import()`ed straight into Node once a fake
|
|
13
|
+
// `self` (postMessage/onmessage) is installed on globalThis and the REAL Node
|
|
14
|
+
// `process` global is masked during the import — dart-sass's bundled browser shim
|
|
15
|
+
// checks `process.versions.node` at module-eval time and takes a `require()` path
|
|
16
|
+
// that esbuild's browser platform build cannot satisfy (`Dynamic require of "url"
|
|
17
|
+
// is not supported`) when it sees the real Node process object.
|
|
18
|
+
//
|
|
19
|
+
// Each `import(url + '?n=' + n)` with a distinct query string forces Node to load a
|
|
20
|
+
// FRESH module instance (own `toolchainReady` cache, own `self.onmessage` closure),
|
|
21
|
+
// which is how independent worker instances are simulated here. The worker's console
|
|
22
|
+
// patch forwards console.* to `self.postMessage({ type: 'log' })` after the first
|
|
23
|
+
// import — this test never calls console.* through a live `self`, only through a
|
|
24
|
+
// `rawLog` reference captured before any import.
|
|
25
|
+
import { createRequire } from 'node:module'
|
|
26
|
+
|
|
27
|
+
const require = createRequire(import.meta.url)
|
|
28
|
+
const realProcessExit = process.exit.bind(process)
|
|
29
|
+
const rawLog = console.log.bind(console)
|
|
30
|
+
|
|
31
|
+
// dart-sass's node-vs-browser branch (see stage-worker.browser.js bundle) reads
|
|
32
|
+
// `process.versions.node` once at module-eval time. Masking the global with a
|
|
33
|
+
// browser-shaped process (no `versions`) BEFORE any worker import makes the bundle
|
|
34
|
+
// take the same path it would in a real browser. `realProcessExit` above keeps a
|
|
35
|
+
// working exit for this script's own end regardless of this override.
|
|
36
|
+
globalThis.process = { env: {}, cwd: () => '/' }
|
|
37
|
+
|
|
38
|
+
const WORKER_URL = new URL('../dist/stage-worker.browser.js', import.meta.url).href
|
|
39
|
+
const POOL_MODULE = require.resolve('../dist/pool.browser.js')
|
|
40
|
+
|
|
41
|
+
let failed = 0
|
|
42
|
+
function chk(cond, msg) {
|
|
43
|
+
if (cond) rawLog(`✅ ${msg}`)
|
|
44
|
+
else { rawLog(`❌ ${msg}`); failed++ }
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
// A toolchainSetupURL that always fails to import — stands in for both "points at
|
|
48
|
+
// a URL that doesn't exist" and "points at a module whose import throws" (memfs/oxc
|
|
49
|
+
// setup modules do the latter in practice; the failure shape ensureToolchain()
|
|
50
|
+
// surfaces is identical either way).
|
|
51
|
+
const UNREACHABLE_TOOLCHAIN_URL = 'file:///no/such/path/toolchain-setup-nonexistent.mjs'
|
|
52
|
+
|
|
53
|
+
// A distinct `data:` URL per marker name is a distinct module specifier, so Node
|
|
54
|
+
// gives each a fresh module record — importing it increments a globalThis counter
|
|
55
|
+
// exactly once per actual `import()` call. Reusing the SAME marker name across two
|
|
56
|
+
// sends would hit Node's module cache on the second import and silently read as
|
|
57
|
+
// "imported once" even if the code path runs twice, so every assertion below uses
|
|
58
|
+
// its own marker name.
|
|
59
|
+
const sideEffectURL = (markerName) => `data:text/javascript,globalThis.${markerName}=(globalThis.${markerName}||0)%2B1`
|
|
60
|
+
|
|
61
|
+
// --- fake `self` (Worker global scope) + message round-trip -------------------
|
|
62
|
+
function makeFakeSelf() {
|
|
63
|
+
const inbox = []
|
|
64
|
+
let waiter = null
|
|
65
|
+
const fakeSelf = {
|
|
66
|
+
onmessage: null,
|
|
67
|
+
postMessage(msg) {
|
|
68
|
+
inbox.push(msg)
|
|
69
|
+
if (waiter) { const w = waiter; waiter = null; w() }
|
|
70
|
+
},
|
|
71
|
+
}
|
|
72
|
+
// Diagnostics arrive out-of-band as { type:'log' } (see stage-worker.js's console
|
|
73
|
+
// forwarding) interleaved with the real reply — skip them, same as pool.js does.
|
|
74
|
+
fakeSelf.waitForReply = () => new Promise((resolve) => {
|
|
75
|
+
function tryDrain() {
|
|
76
|
+
while (inbox.length) {
|
|
77
|
+
const m = inbox.shift()
|
|
78
|
+
if (m && m.type === 'log') continue
|
|
79
|
+
resolve(m)
|
|
80
|
+
return
|
|
81
|
+
}
|
|
82
|
+
waiter = tryDrain
|
|
83
|
+
}
|
|
84
|
+
tryDrain()
|
|
85
|
+
})
|
|
86
|
+
return fakeSelf
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
let instanceCounter = 0
|
|
90
|
+
async function loadWorkerInstance() {
|
|
91
|
+
const fakeSelf = makeFakeSelf()
|
|
92
|
+
globalThis.self = fakeSelf
|
|
93
|
+
instanceCounter += 1
|
|
94
|
+
await import(`${WORKER_URL}?n=${instanceCounter}`)
|
|
95
|
+
return {
|
|
96
|
+
// Sends are driven strictly sequentially (never two in-flight sends across
|
|
97
|
+
// different instances) — the worker module resolves the bare `self` identifier
|
|
98
|
+
// against whatever `globalThis.self` is AT CALL TIME, not at import time, so an
|
|
99
|
+
// in-flight send from a different instance would misdirect this one's reply.
|
|
100
|
+
async send(msg) {
|
|
101
|
+
globalThis.self = fakeSelf
|
|
102
|
+
const reply = fakeSelf.waitForReply()
|
|
103
|
+
fakeSelf.onmessage({ data: msg })
|
|
104
|
+
return reply
|
|
105
|
+
},
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
|
|
109
|
+
const FIXTURE_FILES = {
|
|
110
|
+
'app.json': JSON.stringify({ pages: ['pages/index/index'] }),
|
|
111
|
+
'app.js': 'App({})',
|
|
112
|
+
'pages/index/index.js': 'Page({})',
|
|
113
|
+
'pages/index/index.wxml': '<view>hi</view>',
|
|
114
|
+
'pages/index/index.wxss': '.x{color:red}',
|
|
115
|
+
'pages/index/index.json': '{}',
|
|
116
|
+
}
|
|
117
|
+
const WORK_PATH = '/work'
|
|
118
|
+
|
|
119
|
+
function findCompiledCss(files) {
|
|
120
|
+
return Object.entries(files || {}).find(([k, v]) => k.endsWith('.css') && typeof v === 'string' && v.includes('color:red'))
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
// --- A + B: style-only worker never needs the wasm toolchain ------------------
|
|
124
|
+
{
|
|
125
|
+
const styleWorker = await loadWorkerInstance()
|
|
126
|
+
const warmupReply = await styleWorker.send({
|
|
127
|
+
type: 'warmup',
|
|
128
|
+
toolchainSetupURL: UNREACHABLE_TOOLCHAIN_URL,
|
|
129
|
+
stages: ['style'],
|
|
130
|
+
})
|
|
131
|
+
chk(warmupReply && warmupReply.type === 'ready',
|
|
132
|
+
`style-only worker warmup succeeds with an unreachable toolchainSetupURL (skips the wasm import) — got ${JSON.stringify(warmupReply)}`)
|
|
133
|
+
}
|
|
134
|
+
{
|
|
135
|
+
// Fresh instance, no prior warmup at all — isolates compile-subset's OWN skip
|
|
136
|
+
// decision from warmup's memoized toolchainReady state.
|
|
137
|
+
const styleWorker = await loadWorkerInstance()
|
|
138
|
+
const compileReply = await styleWorker.send({
|
|
139
|
+
type: 'compile-subset',
|
|
140
|
+
files: FIXTURE_FILES,
|
|
141
|
+
workPath: WORK_PATH,
|
|
142
|
+
stages: ['style'],
|
|
143
|
+
toolchainSetupURL: UNREACHABLE_TOOLCHAIN_URL,
|
|
144
|
+
})
|
|
145
|
+
chk(compileReply && compileReply.type === 'done',
|
|
146
|
+
`style-only compile-subset succeeds with an unreachable toolchainSetupURL — got ${JSON.stringify(compileReply && compileReply.type === 'error' ? compileReply.error : compileReply)}`)
|
|
147
|
+
const css = compileReply && compileReply.type === 'done' ? findCompiledCss(compileReply.result.files) : null
|
|
148
|
+
chk(!!css, `style-only compile-subset produced a real compiled CSS product (found "${css && css[0]}": ${css && JSON.stringify(css[1])})`)
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
// --- C: logic / view stage worker behavior is unchanged — still imports the
|
|
152
|
+
// toolchain exactly once per warmup ---------------------------------------------
|
|
153
|
+
for (const stage of ['logic', 'view']) {
|
|
154
|
+
const marker = `__stageToolchainMark_${stage}`
|
|
155
|
+
const worker = await loadWorkerInstance()
|
|
156
|
+
const reply = await worker.send({
|
|
157
|
+
type: 'warmup',
|
|
158
|
+
toolchainSetupURL: sideEffectURL(marker),
|
|
159
|
+
stages: [stage],
|
|
160
|
+
})
|
|
161
|
+
chk(reply && reply.type === 'ready', `${stage} worker warmup succeeds`)
|
|
162
|
+
chk(globalThis[marker] === 1, `${stage} worker warmup imports toolchainSetupURL exactly once (count=${globalThis[marker]})`)
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
// --- D: an unrecognized custom stage name loads the toolchain conservatively ---
|
|
166
|
+
{
|
|
167
|
+
const marker = '__stageToolchainMark_custom'
|
|
168
|
+
const worker = await loadWorkerInstance()
|
|
169
|
+
const reply = await worker.send({
|
|
170
|
+
type: 'warmup',
|
|
171
|
+
toolchainSetupURL: sideEffectURL(marker),
|
|
172
|
+
stages: ['my-custom-stage'],
|
|
173
|
+
})
|
|
174
|
+
chk(reply && reply.type === 'ready', 'unknown custom stage worker warmup succeeds')
|
|
175
|
+
chk(globalThis[marker] === 1, `unknown custom stage "my-custom-stage" still imports toolchainSetupURL (conservative default; count=${globalThis[marker]})`)
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
// --- E: warmup with no `stages` field at all (pre-optimization callers) stays
|
|
179
|
+
// conservative too ---------------------------------------------------------------
|
|
180
|
+
{
|
|
181
|
+
const marker = '__stageToolchainMark_legacy'
|
|
182
|
+
const worker = await loadWorkerInstance()
|
|
183
|
+
const reply = await worker.send({
|
|
184
|
+
type: 'warmup',
|
|
185
|
+
toolchainSetupURL: sideEffectURL(marker),
|
|
186
|
+
// no `stages` field — the pre-optimization warmup message shape
|
|
187
|
+
})
|
|
188
|
+
chk(reply && reply.type === 'ready', 'legacy warmup (no stages field) succeeds')
|
|
189
|
+
chk(globalThis[marker] === 1, `legacy warmup without a stages field still imports toolchainSetupURL (count=${globalThis[marker]})`)
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
// --- G: a worker that skipped the import at warmup must still REMEMBER the
|
|
193
|
+
// warmup URL — a later compile-subset that needs the toolchain (logic stage) and
|
|
194
|
+
// carries no toolchainSetupURL of its own must import the remembered URL instead
|
|
195
|
+
// of failing "no toolchainSetupURL / not warmed up" ------------------------------
|
|
196
|
+
{
|
|
197
|
+
const marker = '__stageToolchainMark_deferred'
|
|
198
|
+
// Besides counting the import, install stand-in toolchain hooks so the logic
|
|
199
|
+
// compile has SOMETHING to call if it gets that far. The compile outcome itself
|
|
200
|
+
// is not the guarded contract (stand-in hooks may not satisfy the full logic
|
|
201
|
+
// pipeline) — what must hold is that the remembered URL gets imported and the
|
|
202
|
+
// failure mode is NOT the "no toolchainSetupURL" warmup error.
|
|
203
|
+
const setupURL = 'data:text/javascript,' + encodeURIComponent(
|
|
204
|
+
`globalThis.${marker}=(globalThis.${marker}||0)+1;`
|
|
205
|
+
+ 'globalThis.__esbuildTransform=async(code)=>({code});'
|
|
206
|
+
+ 'globalThis.__oxcParseSync=()=>{throw new Error("stand-in oxc hook")};',
|
|
207
|
+
)
|
|
208
|
+
const worker = await loadWorkerInstance()
|
|
209
|
+
const warmupReply = await worker.send({
|
|
210
|
+
type: 'warmup',
|
|
211
|
+
toolchainSetupURL: setupURL,
|
|
212
|
+
stages: ['style'],
|
|
213
|
+
})
|
|
214
|
+
chk(warmupReply && warmupReply.type === 'ready', 'style-declared worker warmup succeeds with a working setup module')
|
|
215
|
+
chk((globalThis[marker] || 0) === 0,
|
|
216
|
+
`style-declared warmup defers the setup-module import (count=${globalThis[marker] || 0})`)
|
|
217
|
+
const compileReply = await worker.send({
|
|
218
|
+
type: 'compile-subset',
|
|
219
|
+
files: FIXTURE_FILES,
|
|
220
|
+
workPath: WORK_PATH,
|
|
221
|
+
stages: ['logic'],
|
|
222
|
+
// no toolchainSetupURL — the worker must fall back to the URL remembered at warmup
|
|
223
|
+
})
|
|
224
|
+
chk(globalThis[marker] === 1,
|
|
225
|
+
`logic compile-subset without its own toolchainSetupURL imports the URL remembered at warmup (count=${globalThis[marker]})`)
|
|
226
|
+
chk(!!compileReply && !(compileReply.type === 'error' && /no toolchainSetupURL|not warmed up/.test(String(compileReply.error))),
|
|
227
|
+
`logic compile-subset after a deferred warmup does not fail as un-warmed — got ${JSON.stringify(compileReply && (compileReply.type === 'error' ? String(compileReply.error).slice(0, 100) : compileReply.type))}`)
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
// --- F: createCompilerPool tells each resident worker its own stage identity ---
|
|
231
|
+
{
|
|
232
|
+
const { createCompilerPool } = await import(POOL_MODULE)
|
|
233
|
+
const createdWorkers = []
|
|
234
|
+
function createWorker() {
|
|
235
|
+
const messages = []
|
|
236
|
+
const w = {
|
|
237
|
+
onmessage: null,
|
|
238
|
+
postMessage(m) {
|
|
239
|
+
messages.push(m)
|
|
240
|
+
// Reply asynchronously, like a real Worker would, so pool.warmup()'s
|
|
241
|
+
// send()/Promise pairing is exercised the same way it is in production.
|
|
242
|
+
queueMicrotask(() => { if (w.onmessage) w.onmessage({ data: { type: 'ready' } }) })
|
|
243
|
+
},
|
|
244
|
+
terminate() {},
|
|
245
|
+
}
|
|
246
|
+
createdWorkers.push(messages)
|
|
247
|
+
return w
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
const stages = ['logic', 'view', 'style']
|
|
251
|
+
const pool = createCompilerPool({ createWorker, toolchainSetupURL: 'data:text/javascript,export default {}', stages })
|
|
252
|
+
await pool.warmup()
|
|
253
|
+
|
|
254
|
+
for (let i = 0; i < stages.length; i++) {
|
|
255
|
+
const firstMessage = createdWorkers[i] && createdWorkers[i][0]
|
|
256
|
+
chk(firstMessage && firstMessage.type === 'warmup', `pool sent a warmup message to the "${stages[i]}" worker`)
|
|
257
|
+
chk(!!firstMessage && Array.isArray(firstMessage.stages) && firstMessage.stages.length === 1 && firstMessage.stages[0] === stages[i],
|
|
258
|
+
`pool's warmup message to the "${stages[i]}" worker carries its own stage identity (stages:${JSON.stringify(firstMessage && firstMessage.stages)})`)
|
|
259
|
+
}
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
rawLog(failed ? `\n❌ ${failed} stage-toolchain assertion(s) failed.` : '\n✅ style stage skips the wasm toolchain; logic/view/custom/legacy stay conservative; pool announces worker stage identity.')
|
|
263
|
+
realProcessExit(failed ? 1 : 0)
|
package/src/pool.js
CHANGED
|
@@ -67,7 +67,9 @@ export function createCompilerPool(options = {}) {
|
|
|
67
67
|
let warmed = null
|
|
68
68
|
async function warmup() {
|
|
69
69
|
if (!warmed) {
|
|
70
|
-
|
|
70
|
+
// stages tells the worker its stage identity so toolchain-free stages (style)
|
|
71
|
+
// can skip importing toolchainSetupURL at warmup.
|
|
72
|
+
warmed = Promise.all(workers.map((x) => x.send({ type: 'warmup', toolchainSetupURL, stages: [x.stage] })))
|
|
71
73
|
.then((rs) => rs.forEach((r, i) => {
|
|
72
74
|
// The worker's own try/catch reports the REAL cause (e.g. a toolchainSetupURL
|
|
73
75
|
// import failure) as r.error — surface it verbatim, tagged with the stage.
|
package/src/stage-worker.js
CHANGED
|
@@ -43,6 +43,17 @@ function ensureToolchain(url) {
|
|
|
43
43
|
return toolchainReady
|
|
44
44
|
}
|
|
45
45
|
|
|
46
|
+
// Stages whose compile path never calls the wasm hooks (__esbuildTransform /
|
|
47
|
+
// __oxcParseSync). The CSS pipeline (postcss + cssnano + autoprefixer) is inlined
|
|
48
|
+
// in this bundle, so a style-only worker skips importing toolchainSetupURL entirely
|
|
49
|
+
// (~13MB esbuild.wasm + oxc WASI it would never call). Unknown/custom stages and
|
|
50
|
+
// messages without stage identity conservatively load the toolchain.
|
|
51
|
+
const TOOLCHAIN_FREE_STAGES = new Set(['style'])
|
|
52
|
+
function needsToolchain(stages) {
|
|
53
|
+
if (!Array.isArray(stages) || stages.length === 0) return true
|
|
54
|
+
return stages.some((s) => !TOOLCHAIN_FREE_STAGES.has(s))
|
|
55
|
+
}
|
|
56
|
+
|
|
46
57
|
function freshFs(files, workPath) {
|
|
47
58
|
return createFsFromVolume(Volume.fromJSON(files, workPath))
|
|
48
59
|
}
|
|
@@ -69,13 +80,18 @@ self.onmessage = async (e) => {
|
|
|
69
80
|
try {
|
|
70
81
|
if (type === 'warmup') {
|
|
71
82
|
const t0 = performance.now()
|
|
72
|
-
|
|
83
|
+
// Remember the URL even when this worker's stages skip the load, so a later
|
|
84
|
+
// compile-subset that DOES need the toolchain (protocol allows any stages)
|
|
85
|
+
// can still resolve it without re-sending the URL.
|
|
86
|
+
if (e.data.toolchainSetupURL) toolchainURL = e.data.toolchainSetupURL
|
|
87
|
+
if (needsToolchain(e.data.stages)) await ensureToolchain()
|
|
73
88
|
self.postMessage({ type: 'ready', ms: Math.round(performance.now() - t0) })
|
|
74
89
|
return
|
|
75
90
|
}
|
|
76
91
|
if (type === 'compile-subset') {
|
|
77
92
|
const { files, workPath = '/work', stages = ['logic', 'view', 'style'], toolchainSetupURL } = e.data
|
|
78
|
-
|
|
93
|
+
if (toolchainSetupURL) toolchainURL = toolchainSetupURL
|
|
94
|
+
if (needsToolchain(stages)) await ensureToolchain()
|
|
79
95
|
const warm = !!toolchainReady
|
|
80
96
|
const t = performance.now()
|
|
81
97
|
const result = await compileSubset(files, workPath, stages)
|