@gjsify/rolldown-native 0.21.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +55 -1
- package/meson.build +56 -4
- package/package.json +14 -4
- package/prebuilds/linux-x86_64/GjsifyRolldown-1.0.typelib +0 -0
- package/prebuilds/linux-x86_64/libgjsify_rolldown.so +0 -0
- package/prebuilds/linux-x86_64/libgjsifyrolldown.so +0 -0
- package/src/rust/Cargo.toml +5 -0
- package/src/rust/src/lib.rs +4 -2
- package/src/rust/src/plugin_proxy.rs +28 -13
- package/src/rust/src/session.rs +68 -70
- package/src/rust/src/wakeup.rs +180 -0
- package/src/vala/rolldown.vala +55 -31
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @gjsify/rolldown-native
|
|
2
2
|
|
|
3
|
-
A native Rust cdylib + Vala/GObject bridge that wraps the Rust `rolldown` bundler and exposes it to GJS via `gi://`. This is the default bundler engine used by `gjsify build` under GJS — npm's `rolldown` is an N-API addon that cannot load in GJS, so this bridge is how gjsify bundles without a Node runtime. Includes a complete plugin bridge (`bundleWithPlugins`) for load, transform, resolveId, and render-chunk hooks. Ships prebuilt `.so` + `.typelib` for Linux.
|
|
3
|
+
A native Rust cdylib + Vala/GObject bridge that wraps the Rust `rolldown` bundler and exposes it to GJS via `gi://`. This is the default bundler engine used by `gjsify build` under GJS — npm's `rolldown` is an N-API addon that cannot load in GJS, so this bridge is how gjsify bundles without a Node runtime. Includes a complete plugin bridge (`bundleWithPlugins`) for load, transform, resolveId, and render-chunk hooks. Ships prebuilt `.so` + `.typelib` for **Linux**; the source is now platform-neutral and cross-compiles for macOS, but no macOS prebuild is published yet — see [Platform coverage](#platform-coverage).
|
|
4
4
|
|
|
5
5
|
Part of the [gjsify](https://github.com/gjsify/gjsify) project — Node.js and Web APIs for GJS (GNOME JavaScript).
|
|
6
6
|
|
|
@@ -30,6 +30,60 @@ if (hasNativeRolldown()) {
|
|
|
30
30
|
|
|
31
31
|
Under normal usage `@gjsify/rolldown-native` is consumed automatically by the gjsify CLI (`gjsify build`) — direct use is only needed when embedding the bundler in custom build tooling.
|
|
32
32
|
|
|
33
|
+
## Platform coverage
|
|
34
|
+
|
|
35
|
+
| Platform | Prebuild | Built by |
|
|
36
|
+
|---|---|---|
|
|
37
|
+
| `linux-x86_64` | ✅ `.so` + `.gir` + `.typelib` | native runner |
|
|
38
|
+
| `linux-aarch64` | ✅ | native runner |
|
|
39
|
+
| `linux-ppc64`, `linux-s390x`, `linux-riscv64` | ❌ | not built — the rolldown crate graph is too slow under QEMU |
|
|
40
|
+
| macOS (`darwin-arm64`) | ⏳ source-ready, no prebuild yet | pending a green `prebuilds.yml` macOS leg |
|
|
41
|
+
| macOS (`darwin-x64`) | ❌ | — |
|
|
42
|
+
| Windows | ❌ | — no Vala/GI bridge in this repo targets Windows |
|
|
43
|
+
|
|
44
|
+
All prebuilds are produced by [`.github/workflows/prebuilds.yml`](../../../.github/workflows/prebuilds.yml)
|
|
45
|
+
and committed back to the repository.
|
|
46
|
+
|
|
47
|
+
### macOS status
|
|
48
|
+
|
|
49
|
+
The **Rust-level blocker is gone.** The core used to wake the GLib main loop through three
|
|
50
|
+
`libc::eventfd(2)` descriptors — a Linux syscall the `libc` crate does not even expose on
|
|
51
|
+
Apple targets, so `cargo build` could not compile at all there. The wakeup channel is now a
|
|
52
|
+
plain anonymous pipe (`src/rust/src/wakeup.rs`), one portable implementation used on every
|
|
53
|
+
platform, so the crate cross-compiles: `cargo check --target aarch64-apple-darwin` is green.
|
|
54
|
+
`meson.build` was already macOS-ready (`.dylib` naming, `@loader_path` rpath).
|
|
55
|
+
|
|
56
|
+
What is still missing is a **native macOS build**: the `.dylib` + `.gir` + `.typelib` have to
|
|
57
|
+
be produced and load-tested by the `prebuilds.yml` darwin leg (Homebrew `vala`, `gobject-
|
|
58
|
+
introspection`, `json-glib`, `gjs`) before `darwin-arm64` is added to `package.json`'s
|
|
59
|
+
`gjsify.platforms` and a prebuild is committed. Until that leg is green, treat macOS as
|
|
60
|
+
unproven — a cross-`check` does not exercise linking, the GIR/typelib step, or `gi://` load.
|
|
61
|
+
|
|
62
|
+
Because `@gjsify/rolldown-native` is the only bundler engine available to `gjsify build`
|
|
63
|
+
under GJS, it is the last package gating a Node-free gjsify toolchain on macOS —
|
|
64
|
+
`@gjsify/lightningcss-native` and `@gjsify/oxfmt-native` already ship `darwin-arm64`
|
|
65
|
+
prebuilds.
|
|
66
|
+
|
|
67
|
+
### Wakeup channel (implementation note)
|
|
68
|
+
|
|
69
|
+
Three pipes carry "a plugin hook fired" / "the build finished" / "a `this.resolve()` result
|
|
70
|
+
is ready" from the tokio worker threads to the GLib main loop, which watches their read ends
|
|
71
|
+
with `GLib.IOChannel.unix_new()` + `add_watch()`. Two properties are load-bearing and easy to
|
|
72
|
+
break:
|
|
73
|
+
|
|
74
|
+
- **Both ends are `O_NONBLOCK` and a full buffer is not an error.** A pipe (unlike an eventfd
|
|
75
|
+
counter) can fill up. The writer never blocks a tokio worker; an `EAGAIN` write is dropped
|
|
76
|
+
on purpose, because a full buffer means the reader still owes a drain cycle — and the
|
|
77
|
+
reader drains the pipe *before* it re-drains the request queue, so nothing is lost.
|
|
78
|
+
- **The reader consumes everything available**, not one fixed-size chunk. GLib's watch is
|
|
79
|
+
level-triggered, so leftover bytes re-dispatch the callback — re-draining the queue for
|
|
80
|
+
nothing — once per chunk until the pipe empties. Draining to `EAGAIN` collapses a burst of
|
|
81
|
+
N hook wakeups back into a single main-loop iteration, which is the coalescing the eventfd
|
|
82
|
+
counter used to provide for free.
|
|
83
|
+
|
|
84
|
+
Both ends are also `FD_CLOEXEC`, so a subprocess spawned from a plugin hook cannot inherit a
|
|
85
|
+
wakeup fd.
|
|
86
|
+
|
|
33
87
|
## License
|
|
34
88
|
|
|
35
89
|
MIT
|
package/meson.build
CHANGED
|
@@ -21,6 +21,52 @@ vala_deps = [
|
|
|
21
21
|
dependency('json-glib-1.0'),
|
|
22
22
|
]
|
|
23
23
|
|
|
24
|
+
# ----------------------------------------------------------------------------
|
|
25
|
+
# Host-dependent naming + runtime-lookup conventions
|
|
26
|
+
#
|
|
27
|
+
# Three things differ per host and MUST agree, or the prebuild loads on one
|
|
28
|
+
# platform and silently fails on the other (mirrors @gjsify/napi's meson
|
|
29
|
+
# branch):
|
|
30
|
+
#
|
|
31
|
+
# 1. cargo's cdylib leaf — libgjsify_rolldown.{so,dylib}
|
|
32
|
+
# 2. meson's library() leaf — libgjsifyrolldown.{so,dylib}; the typelib
|
|
33
|
+
# records this name and gjs' g_module_open() resolves exactly it at load.
|
|
34
|
+
# 3. how the Vala library finds its sibling Rust cdylib at runtime. Both
|
|
35
|
+
# files ship side by side in prebuilds/<platform>-<arch>/, so the lookup
|
|
36
|
+
# must be relative to the loading binary: $ORIGIN on ELF, @loader_path on
|
|
37
|
+
# Mach-O. On macOS this is load-bearing — unlike Linux (where the CLI
|
|
38
|
+
# exports LD_LIBRARY_PATH for the prebuild dir) nothing sets
|
|
39
|
+
# DYLD_LIBRARY_PATH, so the rpath is the ONLY way the cdylib is found.
|
|
40
|
+
#
|
|
41
|
+
# The Rust core is platform-neutral: it wakes the GLib main loop through three
|
|
42
|
+
# anonymous pipes (src/rust/src/wakeup.rs), not the Linux-only `eventfd(2)` it
|
|
43
|
+
# used to, so `cargo build` compiles for Apple targets too. Verified with
|
|
44
|
+
# `cargo check --target aarch64-apple-darwin`; the first NATIVE macOS build is
|
|
45
|
+
# what `prebuilds.yml`'s darwin leg has to prove before `package.json`'s
|
|
46
|
+
# `gjsify.platforms` gains `darwin-arm64`.
|
|
47
|
+
# The branch below is kept in lockstep with its lightningcss/oxfmt siblings.
|
|
48
|
+
# ----------------------------------------------------------------------------
|
|
49
|
+
host_system = host_machine.system()
|
|
50
|
+
|
|
51
|
+
if host_system == 'darwin'
|
|
52
|
+
rust_cdylib_name = 'libgjsify_rolldown.dylib'
|
|
53
|
+
shared_lib_name = 'libgjsifyrolldown.dylib'
|
|
54
|
+
lib_rpath = '@loader_path'
|
|
55
|
+
# Normalise the cdylib's LC_ID_DYLIB to @rpath/<leaf> BEFORE the Vala
|
|
56
|
+
# library links against it: ld64 copies the dependency's install name
|
|
57
|
+
# verbatim into the dependent's LC_LOAD_DYLIB. rustc's own install name has
|
|
58
|
+
# varied across releases (absolute build path in older rustc, @rpath/<leaf>
|
|
59
|
+
# in current ones); pinning it here makes the recorded dependency
|
|
60
|
+
# rustc-version-independent, and @rpath/ is never longer than an absolute
|
|
61
|
+
# path so install_name_tool can always rewrite in place.
|
|
62
|
+
rust_cdylib_post = ' && install_name_tool -id "@rpath/' + rust_cdylib_name + '" "$3"'
|
|
63
|
+
else
|
|
64
|
+
rust_cdylib_name = 'libgjsify_rolldown.so'
|
|
65
|
+
shared_lib_name = 'libgjsifyrolldown.so'
|
|
66
|
+
lib_rpath = '$ORIGIN'
|
|
67
|
+
rust_cdylib_post = ''
|
|
68
|
+
endif
|
|
69
|
+
|
|
24
70
|
# ----------------------------------------------------------------------------
|
|
25
71
|
# Rust cdylib (cargo)
|
|
26
72
|
# ----------------------------------------------------------------------------
|
|
@@ -31,15 +77,17 @@ rust_sources = files(
|
|
|
31
77
|
'src/rust/src/lib.rs',
|
|
32
78
|
'src/rust/src/plugin_proxy.rs',
|
|
33
79
|
'src/rust/src/session.rs',
|
|
80
|
+
'src/rust/src/wakeup.rs',
|
|
34
81
|
)
|
|
35
82
|
|
|
36
83
|
rust_target_dir = meson.current_build_dir() / 'cargo-target'
|
|
37
84
|
|
|
38
85
|
rust_cdylib = custom_target('gjsify-rolldown-rust',
|
|
39
86
|
input: rust_sources,
|
|
40
|
-
output:
|
|
87
|
+
output: rust_cdylib_name,
|
|
41
88
|
command: ['sh', '-c',
|
|
42
|
-
'cargo build --release --manifest-path "$1" --target-dir "$2" && cp "$2/release/
|
|
89
|
+
'cargo build --release --manifest-path "$1" --target-dir "$2" && cp "$2/release/'
|
|
90
|
+
+ rust_cdylib_name + '" "$3"' + rust_cdylib_post,
|
|
43
91
|
'sh',
|
|
44
92
|
meson.current_source_dir() / 'src/rust/Cargo.toml',
|
|
45
93
|
rust_target_dir,
|
|
@@ -69,7 +117,11 @@ libGjsifyRolldown = library('gjsifyrolldown', sources,
|
|
|
69
117
|
link_depends: rust_cdylib,
|
|
70
118
|
install: true,
|
|
71
119
|
install_dir: [true, true, true, true],
|
|
72
|
-
install_rpath:
|
|
120
|
+
# `build_rpath` (not just `install_rpath`) is what matters for the prebuild:
|
|
121
|
+
# the artifact shipped in prebuilds/ is copied straight out of the build
|
|
122
|
+
# dir, never through `meson install`, so only the build-tree rpath survives.
|
|
123
|
+
build_rpath: lib_rpath,
|
|
124
|
+
install_rpath: lib_rpath,
|
|
73
125
|
)
|
|
74
126
|
|
|
75
127
|
g_ir_compiler = find_program('g-ir-compiler')
|
|
@@ -77,7 +129,7 @@ g_ir_compiler = find_program('g-ir-compiler')
|
|
|
77
129
|
custom_target(meson.project_name() + '-1.0.typelib',
|
|
78
130
|
command: [
|
|
79
131
|
g_ir_compiler,
|
|
80
|
-
'--shared-library',
|
|
132
|
+
'--shared-library', shared_lib_name,
|
|
81
133
|
'--output', '@OUTPUT@',
|
|
82
134
|
meson.current_build_dir() / meson.project_name() + '-1.0.gir',
|
|
83
135
|
],
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@gjsify/rolldown-native",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.23.0",
|
|
4
4
|
"description": "Phase D-2 POC: Vala/GObject + Rust cdylib bridge to rolldown for GJS. Wraps `rolldown::Bundler::generate()` (async, tokio) in a per-call current-thread runtime so JS sees a sync `bundle()` call. JSON-encoded BundlerOptions in, JSON-encoded BundleOutput out — the boundary stays small even though rolldown's option surface is enormous. POC scope: no JS plugins (Phase B), no watch/HMR, no incremental builds. Companion to `@gjsify/lightningcss-native`; together they remove the last two Rust-crate runtime blockers from the gjsify build pipeline (Phase D-3 unblock).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/esm/index.js",
|
|
@@ -20,7 +20,17 @@
|
|
|
20
20
|
"src/vala"
|
|
21
21
|
],
|
|
22
22
|
"gjsify": {
|
|
23
|
+
"platforms": [
|
|
24
|
+
"linux-x86_64",
|
|
25
|
+
"linux-aarch64"
|
|
26
|
+
],
|
|
23
27
|
"prebuilds": "prebuilds",
|
|
28
|
+
"refsLockstep": {
|
|
29
|
+
"refs/rolldown": {
|
|
30
|
+
"npm": "rolldown",
|
|
31
|
+
"tag": "v{version}"
|
|
32
|
+
}
|
|
33
|
+
},
|
|
24
34
|
"tier": 1
|
|
25
35
|
},
|
|
26
36
|
"scripts": {
|
|
@@ -30,9 +40,9 @@
|
|
|
30
40
|
"init:meson:wipe": "gjsify run init:meson --wipe",
|
|
31
41
|
"build": "gjsify run build:gjsify && gjsify run build:types",
|
|
32
42
|
"build:gjsify": "gjsify build --library 'src/ts/**/*.{ts,js}'",
|
|
33
|
-
"build:meson": "gjsify run init:meson && meson compile -C build",
|
|
43
|
+
"build:meson": "node ../../../scripts/check-refs-pin.mjs . && gjsify run init:meson && meson compile -C build",
|
|
34
44
|
"build:types": "gjsify tsc",
|
|
35
|
-
"build:prebuilds": "gjsify run build:meson &&
|
|
45
|
+
"build:prebuilds": "gjsify run build:meson && node ../../../scripts/stage-prebuild.mjs ."
|
|
36
46
|
},
|
|
37
47
|
"keywords": [
|
|
38
48
|
"gjs",
|
|
@@ -48,7 +58,7 @@
|
|
|
48
58
|
"@girs/gobject-2.0": "^4.1.0"
|
|
49
59
|
},
|
|
50
60
|
"devDependencies": {
|
|
51
|
-
"@gjsify/cli": "^0.
|
|
61
|
+
"@gjsify/cli": "^0.23.0",
|
|
52
62
|
"@types/node": "^25.9.2",
|
|
53
63
|
"typescript": "^6.0.3"
|
|
54
64
|
},
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/src/rust/Cargo.toml
CHANGED
|
@@ -2,6 +2,11 @@
|
|
|
2
2
|
name = "gjsify_rolldown"
|
|
3
3
|
version = "0.1.0"
|
|
4
4
|
edition = "2024"
|
|
5
|
+
# `std::io::pipe()` (the portable main-loop wakeup channel in src/wakeup.rs)
|
|
6
|
+
# is stable since 1.87. The rolldown graph already needs >= 1.85 for
|
|
7
|
+
# edition2024, so this only makes the stricter bound fail loudly instead of
|
|
8
|
+
# surfacing as a confusing "no function `pipe` in module `std::io`".
|
|
9
|
+
rust-version = "1.87"
|
|
5
10
|
publish = false
|
|
6
11
|
|
|
7
12
|
[lib]
|
package/src/rust/src/lib.rs
CHANGED
|
@@ -36,11 +36,13 @@ use serde::Serialize;
|
|
|
36
36
|
use tokio::runtime::Builder;
|
|
37
37
|
|
|
38
38
|
// Plugin-callback machinery (Phase B.1). The session module owns the
|
|
39
|
-
// stateful Bundler instance + tokio runtime + Vala-side
|
|
39
|
+
// stateful Bundler instance + tokio runtime + the Vala-side wakeup
|
|
40
40
|
// pumps; plugin_proxy holds the JsPluginProxy that bridges
|
|
41
|
-
// rolldown's Pluginable trait to the channel
|
|
41
|
+
// rolldown's Pluginable trait to the channel; wakeup holds the
|
|
42
|
+
// portable pipe that wakes the GLib main loop.
|
|
42
43
|
pub mod plugin_proxy;
|
|
43
44
|
pub mod session;
|
|
45
|
+
pub mod wakeup;
|
|
44
46
|
|
|
45
47
|
/// Serializable mirror of the parts of `rolldown_common::Output` we expose to JS.
|
|
46
48
|
///
|
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
//! - tokio-task running `Bundler::generate()` on the worker pool
|
|
15
15
|
//! - calls into JsPluginProxy from various worker threads
|
|
16
16
|
//! - JsPluginProxy serializes a `HookRequest`, pushes to channel
|
|
17
|
-
//! -
|
|
17
|
+
//! - a byte on the wakeup pipe wakes the GLib main loop
|
|
18
18
|
//! - Vala source pulls requests, emits GObject signal on main thread
|
|
19
19
|
//! - JS handler runs, calls `session.respond(req_id, json)`
|
|
20
20
|
//! - Vala forwards to `gjsify_rolldown_session_respond` extern
|
|
@@ -35,8 +35,9 @@ use rolldown_plugin::{
|
|
|
35
35
|
HookGenerateBundleArgs, HookInjectionOutputReturn, HookLoadArgs, HookLoadOutput,
|
|
36
36
|
HookLoadReturn, HookNoopReturn, HookRenderChunkArgs, HookRenderChunkOutput,
|
|
37
37
|
HookRenderChunkReturn, HookResolveIdArgs, HookResolveIdOutput, HookResolveIdReturn,
|
|
38
|
-
HookTransformArgs, HookTransformOutput, HookTransformReturn, HookUsage,
|
|
39
|
-
Plugin, PluginContext, SharedLoadPluginContext,
|
|
38
|
+
HookTransformArgs, HookTransformOutput, HookTransformOutputMap, HookTransformReturn, HookUsage,
|
|
39
|
+
HookWriteBundleArgs, Plugin, PluginContext, SharedLoadPluginContext,
|
|
40
|
+
SharedTransformPluginContext,
|
|
40
41
|
};
|
|
41
42
|
use serde::{Deserialize, Serialize};
|
|
42
43
|
use tokio::sync::oneshot;
|
|
@@ -111,7 +112,7 @@ pub enum HookRequestPayload {
|
|
|
111
112
|
}
|
|
112
113
|
|
|
113
114
|
/// Wire-format envelope for one hook invocation. Sent over
|
|
114
|
-
/// crossbeam channel;
|
|
115
|
+
/// crossbeam channel; the wakeup pipe is woken after each push.
|
|
115
116
|
pub struct HookRequest {
|
|
116
117
|
pub req_id: u64,
|
|
117
118
|
pub plugin_index: usize,
|
|
@@ -277,7 +278,13 @@ impl HookResponse {
|
|
|
277
278
|
};
|
|
278
279
|
Ok(Some(HookTransformOutput {
|
|
279
280
|
code,
|
|
280
|
-
map
|
|
281
|
+
// The bridge has no source-map channel at all (JS
|
|
282
|
+
// never gets to return one), so this is the explicit
|
|
283
|
+
// "no sourcemap intended" case rather than rolldown's
|
|
284
|
+
// `Omitted` (= plugin forgot one → SOURCEMAP_BROKEN).
|
|
285
|
+
// Matches how upstream migrated its own `map: None`
|
|
286
|
+
// call sites when the enum replaced the Option.
|
|
287
|
+
map: HookTransformOutputMap::Null,
|
|
281
288
|
side_effects: None,
|
|
282
289
|
module_type,
|
|
283
290
|
}))
|
|
@@ -295,7 +302,8 @@ impl HookResponse {
|
|
|
295
302
|
let module_type = parse_module_type(v.module_type.as_deref())?;
|
|
296
303
|
Ok(Some(HookTransformOutput {
|
|
297
304
|
code: v.code,
|
|
298
|
-
|
|
305
|
+
// See `into_transform_return_with_bytes`.
|
|
306
|
+
map: HookTransformOutputMap::Null,
|
|
299
307
|
side_effects: None,
|
|
300
308
|
module_type,
|
|
301
309
|
}))
|
|
@@ -312,7 +320,8 @@ impl HookResponse {
|
|
|
312
320
|
.map_err(|e| anyhow!("rolldown: malformed renderChunk response: {e}"))?;
|
|
313
321
|
Ok(Some(HookRenderChunkOutput {
|
|
314
322
|
code: v.code,
|
|
315
|
-
|
|
323
|
+
// See `into_transform_return_with_bytes`.
|
|
324
|
+
map: HookTransformOutputMap::Null,
|
|
316
325
|
}))
|
|
317
326
|
}
|
|
318
327
|
HookResponse::Error { message, stack } => Err(combine(message, stack)),
|
|
@@ -367,11 +376,11 @@ pub struct JsPluginProxy {
|
|
|
367
376
|
/// Atomic monotonic ID dispenser; shared across all proxies in
|
|
368
377
|
/// the same session via Arc clone in the session constructor.
|
|
369
378
|
pub next_request_id: std::sync::Arc<AtomicU64>,
|
|
370
|
-
/// Wakeup pipe
|
|
379
|
+
/// Wakeup pipe woken after each request.send() so the Vala
|
|
371
380
|
/// source on the main loop can react. Owned handle (see
|
|
372
|
-
/// `SessionShared`): keeps
|
|
381
|
+
/// `SessionShared`): keeps both fd numbers reserved even when this
|
|
373
382
|
/// proxy outlives the session's runtime shutdown.
|
|
374
|
-
pub
|
|
383
|
+
pub request_wakeup: std::sync::Arc<crate::wakeup::Wakeup>,
|
|
375
384
|
/// Maximum time we wait for a JS response before failing the
|
|
376
385
|
/// build with a timeout error. Defaults to 60s.
|
|
377
386
|
pub response_timeout: Duration,
|
|
@@ -406,7 +415,7 @@ impl JsPluginProxy {
|
|
|
406
415
|
}
|
|
407
416
|
|
|
408
417
|
/// Push a hook request onto the channel + wake the main loop via
|
|
409
|
-
///
|
|
418
|
+
/// the wakeup pipe, then await the JS-side response.
|
|
410
419
|
async fn dispatch(&self, payload: HookRequestPayload) -> anyhow::Result<HookResponse> {
|
|
411
420
|
let (resp, _bytes) = self.dispatch_inner(payload, None, None).await?;
|
|
412
421
|
Ok(resp)
|
|
@@ -469,7 +478,10 @@ impl JsPluginProxy {
|
|
|
469
478
|
return Err(anyhow!("rolldown: plugin channel closed (session aborted)"));
|
|
470
479
|
}
|
|
471
480
|
|
|
472
|
-
|
|
481
|
+
// Wake AFTER the queue push — the reader drains the wakeup pipe
|
|
482
|
+
// before it drains the queue, so this ordering is what makes a
|
|
483
|
+
// coalesced (buffer-full) wake safe. See `crate::wakeup`.
|
|
484
|
+
self.request_wakeup.wake();
|
|
473
485
|
|
|
474
486
|
let result = tokio::time::timeout(self.response_timeout, reply_rx).await;
|
|
475
487
|
// Clear the context registration regardless of outcome; the
|
|
@@ -621,7 +633,10 @@ impl Plugin for JsPluginProxy {
|
|
|
621
633
|
args: &HookRenderChunkArgs<'_>,
|
|
622
634
|
) -> impl std::future::Future<Output = HookRenderChunkReturn> + Send {
|
|
623
635
|
let proxies = self.proxies("renderChunk");
|
|
624
|
-
|
|
636
|
+
// `args.code` is an `Arc<String>` (rolldown keeps the chunk code
|
|
637
|
+
// recoverable after the hook chain); the wire payload owns its
|
|
638
|
+
// own String.
|
|
639
|
+
let code = args.code.as_ref().clone();
|
|
625
640
|
let file_name = args.chunk.filename.to_string();
|
|
626
641
|
let name = args.chunk.name.to_string();
|
|
627
642
|
let is_entry = args.chunk.is_entry;
|
package/src/rust/src/session.rs
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
//! BundleSession — owns the tokio runtime, the plugin-channel pump,
|
|
2
|
-
//! the
|
|
3
|
-
//!
|
|
2
|
+
//! the wakeup pipes, and the BundleHandle the JS layer uses to drive
|
|
3
|
+
//! the build.
|
|
4
4
|
//!
|
|
5
5
|
//! Lifecycle:
|
|
6
6
|
//! 1. `start()` — caller passes options + plugin metadata. We
|
|
@@ -8,16 +8,18 @@
|
|
|
8
8
|
//! Bundler::with_plugins, spawn a tokio task that drives
|
|
9
9
|
//! `Bundler::generate()` to completion, return the session.
|
|
10
10
|
//! 2. While the build runs, the worker pool calls into the proxies,
|
|
11
|
-
//! pushing HookRequests to the channel +
|
|
12
|
-
//!
|
|
11
|
+
//! pushing HookRequests to the channel + waking the request
|
|
12
|
+
//! pipe. JS drains via `next_request()` and replies via
|
|
13
13
|
//! `respond()`.
|
|
14
14
|
//! 3. When the bundle task completes, the result is stored on the
|
|
15
15
|
//! session. The Vala side polls `try_take_result()` after
|
|
16
|
-
//!
|
|
17
|
-
//!
|
|
16
|
+
//! waking on the completion pipe.
|
|
17
|
+
//!
|
|
18
|
+
//! The wakeup channels are plain anonymous pipes on every platform —
|
|
19
|
+
//! see `crate::wakeup` for why (portability) and for the
|
|
20
|
+
//! counter-vs-byte-stream semantics that entails.
|
|
18
21
|
|
|
19
22
|
use std::ffi::CString;
|
|
20
|
-
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};
|
|
21
23
|
use std::os::raw::{c_char, c_int};
|
|
22
24
|
use std::ptr;
|
|
23
25
|
use std::sync::Arc;
|
|
@@ -35,6 +37,7 @@ use tokio::runtime::{Builder, Runtime};
|
|
|
35
37
|
use tokio::sync::oneshot;
|
|
36
38
|
|
|
37
39
|
use crate::plugin_proxy::{HookRequest, HookResponse, JsPluginProxy, PluginMeta};
|
|
40
|
+
use crate::wakeup::Wakeup;
|
|
38
41
|
use crate::{BundleOutputJson, OutputJson, convert_output};
|
|
39
42
|
|
|
40
43
|
#[derive(Debug, Deserialize)]
|
|
@@ -44,7 +47,7 @@ pub struct StartArgs {
|
|
|
44
47
|
}
|
|
45
48
|
|
|
46
49
|
/// One context-resolve sub-result flowing Rust → JS. Surfaces through
|
|
47
|
-
/// the `
|
|
50
|
+
/// the `context_response_wakeup` queue.
|
|
48
51
|
#[derive(Debug, Serialize)]
|
|
49
52
|
#[serde(rename_all = "camelCase")]
|
|
50
53
|
pub struct ContextResolveResponse {
|
|
@@ -64,16 +67,16 @@ pub struct ContextResolveResponse {
|
|
|
64
67
|
/// `Arc` and cloned into each proxy. Holds everything the proxies
|
|
65
68
|
/// need to (a) register/unregister parent `PluginContext` snapshots
|
|
66
69
|
/// for nested-protocol callbacks, (b) deliver context-resolve results
|
|
67
|
-
/// to JS via the dedicated
|
|
70
|
+
/// to JS via the dedicated wakeup pipe, and (c) accumulate `this.warn()`
|
|
68
71
|
/// strings into the final BundleOutput.
|
|
69
72
|
pub struct SessionShared {
|
|
70
73
|
pub contexts: Mutex<std::collections::HashMap<u64, PluginContext>>,
|
|
71
74
|
pub next_child_id: AtomicU64,
|
|
72
75
|
pub context_response_tx: Sender<ContextResolveResponse>,
|
|
73
|
-
/// Owned
|
|
74
|
-
/// (incl. leaked post-shutdown tasks) is alive, so a stale
|
|
76
|
+
/// Owned pipe — both fds stay reserved for as long as any clone
|
|
77
|
+
/// (incl. leaked post-shutdown tasks) is alive, so a stale wake
|
|
75
78
|
/// can never land in a recycled fd number of a later session.
|
|
76
|
-
pub
|
|
79
|
+
pub context_response_wakeup: Arc<Wakeup>,
|
|
77
80
|
pub context_warnings: Mutex<Vec<String>>,
|
|
78
81
|
/// Phase B.4 — parallel bytes-payload slots keyed by req_id.
|
|
79
82
|
/// `request_payloads` holds Rust → JS payloads (e.g. transform
|
|
@@ -96,12 +99,12 @@ pub struct BundleSession {
|
|
|
96
99
|
pub request_rx: Receiver<HookRequest>,
|
|
97
100
|
/// Pending requests waiting on JS reply. Keyed by req_id.
|
|
98
101
|
pub pending: Mutex<std::collections::HashMap<u64, oneshot::Sender<HookResponse>>>,
|
|
99
|
-
///
|
|
100
|
-
pub
|
|
102
|
+
/// "Request available" wakeup → GLib main loop watches its read fd.
|
|
103
|
+
pub request_wakeup: Arc<Wakeup>,
|
|
101
104
|
/// Result of the bundle task once complete. None while still running.
|
|
102
105
|
pub result: Mutex<Option<Result<String, String>>>,
|
|
103
|
-
/// Completion
|
|
104
|
-
pub
|
|
106
|
+
/// Completion wakeup: woken exactly once when `result` is set.
|
|
107
|
+
pub complete_wakeup: Arc<Wakeup>,
|
|
105
108
|
/// Cancel flag — set by the C side via `cancel()`. tokio task
|
|
106
109
|
/// observes via shared atomic and aborts.
|
|
107
110
|
pub cancelled: Arc<std::sync::atomic::AtomicBool>,
|
|
@@ -110,7 +113,7 @@ pub struct BundleSession {
|
|
|
110
113
|
pub shared: Arc<SessionShared>,
|
|
111
114
|
/// Receiver side of the context-response channel. Drained by
|
|
112
115
|
/// `gjsify_rolldown_session_next_context_response` from the GLib
|
|
113
|
-
/// main loop after waking on `
|
|
116
|
+
/// main loop after waking on `context_response_wakeup`.
|
|
114
117
|
pub context_response_rx: Receiver<ContextResolveResponse>,
|
|
115
118
|
}
|
|
116
119
|
|
|
@@ -168,37 +171,39 @@ pub extern "C" fn gjsify_rolldown_session_start(
|
|
|
168
171
|
}
|
|
169
172
|
};
|
|
170
173
|
|
|
171
|
-
// Create the
|
|
172
|
-
//
|
|
173
|
-
//
|
|
174
|
-
// Each fd is wrapped in Arc<OwnedFd> immediately: every holder
|
|
174
|
+
// Create the three wakeup pipes (see `crate::wakeup`): non-blocking
|
|
175
|
+
// + close-on-exec on both ends, one portable implementation on every
|
|
176
|
+
// platform. Each is wrapped in Arc<Wakeup> immediately: every holder
|
|
175
177
|
// (proxies, the bundle task, SessionShared, the session itself)
|
|
176
|
-
// keeps a clone, so
|
|
178
|
+
// keeps a clone, so both fd numbers stay reserved until the LAST
|
|
177
179
|
// user — including tasks leaked past the runtime shutdown
|
|
178
180
|
// timeout — has dropped it. Closing eagerly in session_free while
|
|
179
|
-
// such tasks still hold the raw number let their deferred
|
|
180
|
-
//
|
|
181
|
-
//
|
|
182
|
-
let
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
181
|
+
// such tasks still hold the raw number let their deferred wake land
|
|
182
|
+
// in whatever fd the kernel recycled the number into (issue #501's
|
|
183
|
+
// stale-fd hazard).
|
|
184
|
+
let request_wakeup = match Wakeup::new() {
|
|
185
|
+
Ok(w) => Arc::new(w),
|
|
186
|
+
Err(e) => {
|
|
187
|
+
unsafe { *err_out = err_to_cstr(format!("rolldown: wakeup pipe (request): {e}")) };
|
|
188
|
+
return ptr::null_mut();
|
|
189
|
+
}
|
|
190
|
+
};
|
|
191
|
+
let complete_wakeup = match Wakeup::new() {
|
|
192
|
+
Ok(w) => Arc::new(w),
|
|
193
|
+
Err(e) => {
|
|
194
|
+
unsafe { *err_out = err_to_cstr(format!("rolldown: wakeup pipe (complete): {e}")) };
|
|
195
|
+
return ptr::null_mut();
|
|
196
|
+
}
|
|
197
|
+
};
|
|
198
|
+
let context_response_wakeup = match Wakeup::new() {
|
|
199
|
+
Ok(w) => Arc::new(w),
|
|
200
|
+
Err(e) => {
|
|
201
|
+
unsafe {
|
|
202
|
+
*err_out = err_to_cstr(format!("rolldown: wakeup pipe (context_response): {e}"))
|
|
203
|
+
};
|
|
204
|
+
return ptr::null_mut();
|
|
205
|
+
}
|
|
206
|
+
};
|
|
202
207
|
|
|
203
208
|
let cancelled = Arc::new(std::sync::atomic::AtomicBool::new(false));
|
|
204
209
|
let (request_tx, request_rx) = unbounded::<HookRequest>();
|
|
@@ -209,7 +214,7 @@ pub extern "C" fn gjsify_rolldown_session_start(
|
|
|
209
214
|
contexts: Mutex::new(std::collections::HashMap::new()),
|
|
210
215
|
next_child_id: AtomicU64::new(1),
|
|
211
216
|
context_response_tx,
|
|
212
|
-
|
|
217
|
+
context_response_wakeup: context_response_wakeup.clone(),
|
|
213
218
|
context_warnings: Mutex::new(Vec::new()),
|
|
214
219
|
request_payloads: Mutex::new(std::collections::HashMap::new()),
|
|
215
220
|
response_payloads: Mutex::new(std::collections::HashMap::new()),
|
|
@@ -234,7 +239,7 @@ pub extern "C" fn gjsify_rolldown_session_start(
|
|
|
234
239
|
hooks: meta.hooks.clone(),
|
|
235
240
|
request_tx: request_tx.clone(),
|
|
236
241
|
next_request_id: next_request_id.clone(),
|
|
237
|
-
|
|
242
|
+
request_wakeup: request_wakeup.clone(),
|
|
238
243
|
response_timeout: Duration::from_secs(60),
|
|
239
244
|
load_id_filter,
|
|
240
245
|
transform_id_filter,
|
|
@@ -270,7 +275,7 @@ pub extern "C" fn gjsify_rolldown_session_start(
|
|
|
270
275
|
|
|
271
276
|
let result_slot: Arc<Mutex<Option<Result<String, String>>>> = Arc::new(Mutex::new(None));
|
|
272
277
|
let result_slot_clone = result_slot.clone();
|
|
273
|
-
let
|
|
278
|
+
let complete_wakeup_for_task = complete_wakeup.clone();
|
|
274
279
|
let cancelled_for_task = cancelled.clone();
|
|
275
280
|
let shared_for_task = shared.clone();
|
|
276
281
|
|
|
@@ -282,16 +287,16 @@ pub extern "C" fn gjsify_rolldown_session_start(
|
|
|
282
287
|
};
|
|
283
288
|
*result_slot_clone.lock().unwrap() = Some(final_json);
|
|
284
289
|
// Signal completion to the GLib main loop.
|
|
285
|
-
|
|
290
|
+
complete_wakeup_for_task.wake();
|
|
286
291
|
});
|
|
287
292
|
|
|
288
293
|
let session = Box::new(BundleSession {
|
|
289
294
|
runtime,
|
|
290
295
|
request_rx,
|
|
291
296
|
pending: Mutex::new(std::collections::HashMap::new()),
|
|
292
|
-
|
|
297
|
+
request_wakeup,
|
|
293
298
|
result: Mutex::new(None),
|
|
294
|
-
|
|
299
|
+
complete_wakeup,
|
|
295
300
|
cancelled,
|
|
296
301
|
shared,
|
|
297
302
|
context_response_rx,
|
|
@@ -389,7 +394,7 @@ pub extern "C" fn gjsify_rolldown_session_request_fd(session: *mut BundleSession
|
|
|
389
394
|
if session.is_null() {
|
|
390
395
|
return -1;
|
|
391
396
|
}
|
|
392
|
-
unsafe { (*session).
|
|
397
|
+
unsafe { (*session).request_wakeup.read_fd() }
|
|
393
398
|
}
|
|
394
399
|
|
|
395
400
|
#[unsafe(no_mangle)]
|
|
@@ -397,7 +402,7 @@ pub extern "C" fn gjsify_rolldown_session_complete_fd(session: *mut BundleSessio
|
|
|
397
402
|
if session.is_null() {
|
|
398
403
|
return -1;
|
|
399
404
|
}
|
|
400
|
-
unsafe { (*session).
|
|
405
|
+
unsafe { (*session).complete_wakeup.read_fd() }
|
|
401
406
|
}
|
|
402
407
|
|
|
403
408
|
/// Drain one pending request. Returns NULL when the channel is
|
|
@@ -541,11 +546,11 @@ pub extern "C" fn gjsify_rolldown_session_free(session: *mut BundleSession) {
|
|
|
541
546
|
unregister_result_slot(session);
|
|
542
547
|
let boxed = unsafe { Box::from_raw(session) };
|
|
543
548
|
// Shut the runtime down FIRST (up to 500ms drain; workers that
|
|
544
|
-
// don't finish in time are leaked), THEN let the
|
|
545
|
-
//
|
|
546
|
-
//
|
|
547
|
-
//
|
|
548
|
-
//
|
|
549
|
+
// don't finish in time are leaked), THEN let the wakeup pipes drop.
|
|
550
|
+
// They are Arc<Wakeup>, so even a task leaked past the shutdown
|
|
551
|
+
// timeout keeps its clone alive — the fd numbers stay reserved and
|
|
552
|
+
// a deferred stale wake can never corrupt whatever fd the kernel
|
|
553
|
+
// would otherwise have recycled the number into.
|
|
549
554
|
boxed.runtime.shutdown_timeout(Duration::from_millis(500));
|
|
550
555
|
}
|
|
551
556
|
|
|
@@ -609,7 +614,7 @@ pub extern "C" fn gjsify_rolldown_session_context_resolve(
|
|
|
609
614
|
error: Some(format!("rolldown: malformed context_resolve args JSON: {e}")),
|
|
610
615
|
};
|
|
611
616
|
let _ = session.shared.context_response_tx.send(resp);
|
|
612
|
-
|
|
617
|
+
session.shared.context_response_wakeup.wake();
|
|
613
618
|
return child_id;
|
|
614
619
|
}
|
|
615
620
|
};
|
|
@@ -654,7 +659,7 @@ pub extern "C" fn gjsify_rolldown_session_context_resolve(
|
|
|
654
659
|
},
|
|
655
660
|
};
|
|
656
661
|
let _ = shared.context_response_tx.send(resp);
|
|
657
|
-
|
|
662
|
+
shared.context_response_wakeup.wake();
|
|
658
663
|
});
|
|
659
664
|
|
|
660
665
|
child_id
|
|
@@ -680,8 +685,8 @@ pub extern "C" fn gjsify_rolldown_session_context_warn(
|
|
|
680
685
|
session.shared.context_warnings.lock().unwrap().push(msg);
|
|
681
686
|
}
|
|
682
687
|
|
|
683
|
-
/// FFI: get the
|
|
684
|
-
/// available". GLib main loop watches this with `G_IO_IN`.
|
|
688
|
+
/// FFI: get the wakeup-pipe read fd that signals "a context-resolve
|
|
689
|
+
/// sub-result is available". GLib main loop watches this with `G_IO_IN`.
|
|
685
690
|
#[unsafe(no_mangle)]
|
|
686
691
|
pub extern "C" fn gjsify_rolldown_session_context_response_fd(
|
|
687
692
|
session: *mut BundleSession,
|
|
@@ -690,7 +695,7 @@ pub extern "C" fn gjsify_rolldown_session_context_response_fd(
|
|
|
690
695
|
return -1;
|
|
691
696
|
}
|
|
692
697
|
let session = unsafe { &*session };
|
|
693
|
-
session.shared.
|
|
698
|
+
session.shared.context_response_wakeup.read_fd()
|
|
694
699
|
}
|
|
695
700
|
|
|
696
701
|
/// FFI: drain one pending context-resolve response. Returns NULL when
|
|
@@ -720,13 +725,6 @@ pub extern "C" fn gjsify_rolldown_session_next_context_response(
|
|
|
720
725
|
CString::new(json).ok().map(|c| c.into_raw()).unwrap_or(ptr::null_mut())
|
|
721
726
|
}
|
|
722
727
|
|
|
723
|
-
pub(crate) fn wake_eventfd(fd: &OwnedFd) {
|
|
724
|
-
let one: u64 = 1;
|
|
725
|
-
unsafe {
|
|
726
|
-
libc::write(fd.as_raw_fd(), &one as *const u64 as *const libc::c_void, 8);
|
|
727
|
-
}
|
|
728
|
-
}
|
|
729
|
-
|
|
730
728
|
// ---------------------------------------------------------------------
|
|
731
729
|
// Phase B.4 — bytes-payload side-channel for the transform hook.
|
|
732
730
|
// ---------------------------------------------------------------------
|
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
//! Wakeup — the portable "something is queued for you" channel between
|
|
2
|
+
//! the tokio worker threads and the GLib main loop.
|
|
3
|
+
//!
|
|
4
|
+
//! ONE implementation for every platform: an anonymous pipe. The
|
|
5
|
+
//! previous implementation used `eventfd(2)`, which is Linux-only (the
|
|
6
|
+
//! `libc` crate does not even expose it on Apple targets), so the crate
|
|
7
|
+
//! could not be compiled for macOS at all. A pipe is POSIX, is what the
|
|
8
|
+
//! Vala side already treats the descriptor as (`GLib.IOChannel.unix_new`
|
|
9
|
+
//! + `add_watch`, which only ever needs *a pollable fd*), and therefore
|
|
10
|
+
//! removes the platform split instead of doubling it — a second,
|
|
11
|
+
//! platform-gated wakeup path would only ever be exercised on the
|
|
12
|
+
//! platform nobody develops on.
|
|
13
|
+
//!
|
|
14
|
+
//! ## eventfd counter vs pipe byte stream
|
|
15
|
+
//!
|
|
16
|
+
//! An eventfd is a 64-bit counter: N writes coalesce into one readable
|
|
17
|
+
//! event and a single 8-byte read resets it. A pipe is a byte stream
|
|
18
|
+
//! with a finite buffer (64 KiB on Linux, 16 KiB on macOS). Swapping one
|
|
19
|
+
//! for the other naively introduces two bugs; both are handled
|
|
20
|
+
//! explicitly here and in `rolldown.vala`:
|
|
21
|
+
//!
|
|
22
|
+
//! 1. **Lost wakeup / blocked writer when the buffer is full.** Both
|
|
23
|
+
//! ends are `O_NONBLOCK`, so a full buffer surfaces as `EAGAIN`
|
|
24
|
+
//! (`ErrorKind::WouldBlock`) instead of parking a tokio worker
|
|
25
|
+
//! thread forever. `EAGAIN` is *safe to drop*: a full buffer means
|
|
26
|
+
//! the reader has not consumed the bytes already in it, so a wakeup
|
|
27
|
+
//! is already pending. The reader always drains the pipe *before* it
|
|
28
|
+
//! drains the request queue, and the queue push happens before the
|
|
29
|
+
//! write attempt, so the item is guaranteed to be observed by the
|
|
30
|
+
//! drain cycle that consumes those pending bytes. In other words the
|
|
31
|
+
//! pipe recovers eventfd's coalescing property from its buffer
|
|
32
|
+
//! instead of from a counter.
|
|
33
|
+
//!
|
|
34
|
+
//! 2. **A partial read leaves the fd readable.** The reader must
|
|
35
|
+
//! consume *everything* available, not one fixed-size chunk. GLib's
|
|
36
|
+
//! watch is level-triggered, so leftover bytes re-dispatch the
|
|
37
|
+
//! callback — which re-drains the queue for nothing — once per chunk
|
|
38
|
+
//! until the pipe empties: a burst of N wakes costs N/chunk no-op
|
|
39
|
+
//! main-loop iterations instead of one. `rolldown.vala` reads to
|
|
40
|
+
//! `G_IO_STATUS_AGAIN` so a burst collapses back into a single
|
|
41
|
+
//! wake-up cycle, recovering the coalescing the counter gave for
|
|
42
|
+
//! free.
|
|
43
|
+
//!
|
|
44
|
+
//! ## Ownership
|
|
45
|
+
//!
|
|
46
|
+
//! Both ends live in the same struct, which the session hands out as
|
|
47
|
+
//! `Arc<Wakeup>` to every holder (the proxies, the bundle task,
|
|
48
|
+
//! `SessionShared`, the session itself). Consequences:
|
|
49
|
+
//!
|
|
50
|
+
//! * The fd numbers stay reserved until the LAST holder — including a
|
|
51
|
+
//! task leaked past the runtime shutdown timeout — drops its clone,
|
|
52
|
+
//! so a deferred stale wake can never land in an fd number the kernel
|
|
53
|
+
//! recycled into something else (issue #501's hazard, previously
|
|
54
|
+
//! documented on the `Arc<OwnedFd>` eventfds).
|
|
55
|
+
//! * The read end cannot outlive the write end or vice versa, so
|
|
56
|
+
//! `wake()` can never hit `EPIPE`/`SIGPIPE` — that would require the
|
|
57
|
+
//! read end to be closed while a writer still holds a reference,
|
|
58
|
+
//! which the shared ownership makes unrepresentable. (The Vala watch
|
|
59
|
+
//! borrows the read fd with `set_close_on_unref(false)`, so it never
|
|
60
|
+
//! closes it either.)
|
|
61
|
+
//!
|
|
62
|
+
//! Both ends are close-on-exec: `std::io::pipe()` sets `FD_CLOEXEC`
|
|
63
|
+
//! atomically where the platform supports it (`pipe2`), matching the
|
|
64
|
+
//! `EFD_CLOEXEC` the eventfds were created with. A wakeup fd leaking
|
|
65
|
+
//! into a subprocess spawned by a plugin hook would be a real fd leak.
|
|
66
|
+
|
|
67
|
+
use std::io::{ErrorKind, PipeReader, PipeWriter, Write};
|
|
68
|
+
use std::os::fd::{AsRawFd, RawFd};
|
|
69
|
+
|
|
70
|
+
/// A one-way "wake the main loop" channel.
|
|
71
|
+
#[derive(Debug)]
|
|
72
|
+
pub struct Wakeup {
|
|
73
|
+
reader: PipeReader,
|
|
74
|
+
writer: PipeWriter,
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
impl Wakeup {
|
|
78
|
+
/// Create a non-blocking, close-on-exec wakeup pipe.
|
|
79
|
+
pub fn new() -> std::io::Result<Self> {
|
|
80
|
+
let (reader, writer) = std::io::pipe()?;
|
|
81
|
+
// Non-blocking on BOTH ends: the writer must never park a tokio
|
|
82
|
+
// worker on a full buffer, and the reader must be able to drain
|
|
83
|
+
// to EAGAIN from the GLib main loop without blocking it.
|
|
84
|
+
set_nonblocking(reader.as_raw_fd())?;
|
|
85
|
+
set_nonblocking(writer.as_raw_fd())?;
|
|
86
|
+
Ok(Self { reader, writer })
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
/// Raw read fd for the GLib watch. Borrowed — the `Wakeup` keeps
|
|
90
|
+
/// ownership, so the watch must not close it.
|
|
91
|
+
pub fn read_fd(&self) -> RawFd {
|
|
92
|
+
self.reader.as_raw_fd()
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/// Signal the main loop. Call AFTER pushing onto the queue the
|
|
96
|
+
/// reader will drain.
|
|
97
|
+
///
|
|
98
|
+
/// Never blocks and never fails the build: a full buffer
|
|
99
|
+
/// (`WouldBlock`) means a wakeup is already pending and unread, so
|
|
100
|
+
/// dropping this one loses nothing (see the module docs).
|
|
101
|
+
///
|
|
102
|
+
/// Safe to call concurrently from every tokio worker: a write of
|
|
103
|
+
/// `≤ PIPE_BUF` bytes is atomic per POSIX, so 1-byte wakes from
|
|
104
|
+
/// different threads never interleave or tear.
|
|
105
|
+
pub fn wake(&self) {
|
|
106
|
+
loop {
|
|
107
|
+
match (&self.writer).write(&[1u8]) {
|
|
108
|
+
// A ≤PIPE_BUF write is atomic: either the byte is in or
|
|
109
|
+
// we got EAGAIN, never a short write.
|
|
110
|
+
Ok(_) => return,
|
|
111
|
+
Err(e) if e.kind() == ErrorKind::Interrupted => continue,
|
|
112
|
+
// Buffer full — the reader still owes us a drain cycle,
|
|
113
|
+
// and that cycle re-checks the queue. Coalesced.
|
|
114
|
+
Err(e) if e.kind() == ErrorKind::WouldBlock => return,
|
|
115
|
+
// Nothing actionable during teardown; the build result
|
|
116
|
+
// is delivered through the queue + result slot, not here.
|
|
117
|
+
Err(_) => return,
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
fn set_nonblocking(fd: RawFd) -> std::io::Result<()> {
|
|
124
|
+
// SAFETY: `fd` is owned by the `PipeReader`/`PipeWriter` the caller
|
|
125
|
+
// holds for the duration of the call, so it is a valid open fd.
|
|
126
|
+
let flags = unsafe { libc::fcntl(fd, libc::F_GETFL) };
|
|
127
|
+
if flags < 0 {
|
|
128
|
+
return Err(std::io::Error::last_os_error());
|
|
129
|
+
}
|
|
130
|
+
if unsafe { libc::fcntl(fd, libc::F_SETFL, flags | libc::O_NONBLOCK) } < 0 {
|
|
131
|
+
return Err(std::io::Error::last_os_error());
|
|
132
|
+
}
|
|
133
|
+
Ok(())
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
#[cfg(test)]
|
|
137
|
+
mod tests {
|
|
138
|
+
use super::*;
|
|
139
|
+
use std::io::Read;
|
|
140
|
+
|
|
141
|
+
#[test]
|
|
142
|
+
fn cloexec_and_nonblocking_on_both_ends() {
|
|
143
|
+
let w = Wakeup::new().expect("wakeup");
|
|
144
|
+
for fd in [w.reader.as_raw_fd(), w.writer.as_raw_fd()] {
|
|
145
|
+
let fdflags = unsafe { libc::fcntl(fd, libc::F_GETFD) };
|
|
146
|
+
assert!(fdflags >= 0);
|
|
147
|
+
assert_ne!(fdflags & libc::FD_CLOEXEC, 0, "fd must be close-on-exec");
|
|
148
|
+
let flflags = unsafe { libc::fcntl(fd, libc::F_GETFL) };
|
|
149
|
+
assert!(flflags >= 0);
|
|
150
|
+
assert_ne!(flflags & libc::O_NONBLOCK, 0, "fd must be non-blocking");
|
|
151
|
+
}
|
|
152
|
+
}
|
|
153
|
+
|
|
154
|
+
#[test]
|
|
155
|
+
fn wake_never_blocks_when_the_buffer_is_full() {
|
|
156
|
+
let w = Wakeup::new().expect("wakeup");
|
|
157
|
+
// Far more wakes than any pipe buffer can hold. If `wake()`
|
|
158
|
+
// blocked or panicked on EAGAIN this would hang / fail.
|
|
159
|
+
for _ in 0..200_000 {
|
|
160
|
+
w.wake();
|
|
161
|
+
}
|
|
162
|
+
// A full drain (what the Vala watch does) makes the pipe
|
|
163
|
+
// writable again — i.e. the wakeup channel is self-healing.
|
|
164
|
+
let mut buf = [0u8; 4096];
|
|
165
|
+
loop {
|
|
166
|
+
match (&w.reader).read(&mut buf) {
|
|
167
|
+
Ok(0) => break,
|
|
168
|
+
Ok(_) => continue,
|
|
169
|
+
Err(e) if e.kind() == ErrorKind::WouldBlock => break,
|
|
170
|
+
Err(e) => panic!("unexpected drain error: {e}"),
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
let mut probe = [0u8; 1];
|
|
174
|
+
assert!(
|
|
175
|
+
(&w.writer).write(&probe[..]).is_ok(),
|
|
176
|
+
"a drained wakeup pipe must accept writes again"
|
|
177
|
+
);
|
|
178
|
+
assert_eq!((&w.reader).read(&mut probe).expect("read back"), 1);
|
|
179
|
+
}
|
|
180
|
+
}
|
package/src/vala/rolldown.vala
CHANGED
|
@@ -149,6 +149,14 @@ namespace GjsifyRolldown {
|
|
|
149
149
|
private uint _complete_source_id = 0;
|
|
150
150
|
private uint _ctx_response_source_id = 0;
|
|
151
151
|
|
|
152
|
+
/**
|
|
153
|
+
* Scratch buffer for draining the wakeup pipes. Their content is
|
|
154
|
+
* meaningless — only the readability edge matters — so one buffer
|
|
155
|
+
* is shared by all three watch callbacks; they all dispatch on the
|
|
156
|
+
* same main-loop thread and never run concurrently.
|
|
157
|
+
*/
|
|
158
|
+
private char[] _drain_buf = new char[4096];
|
|
159
|
+
|
|
152
160
|
/**
|
|
153
161
|
* Emitted whenever rolldown invokes a plugin hook.
|
|
154
162
|
*
|
|
@@ -210,9 +218,10 @@ namespace GjsifyRolldown {
|
|
|
210
218
|
int req_fd = _session_request_fd (_handle);
|
|
211
219
|
int comp_fd = _session_complete_fd (_handle);
|
|
212
220
|
|
|
213
|
-
// Watch the
|
|
214
|
-
// the source priority + condition; we own the
|
|
215
|
-
// via Rust
|
|
221
|
+
// Watch the wakeup-pipe read ends on the GLib main loop.
|
|
222
|
+
// add_full hands the source priority + condition; we own the
|
|
223
|
+
// fd lifetime via Rust (both pipe ends live in the session's
|
|
224
|
+
// Arc<Wakeup>), so don't pass close_fd:true.
|
|
216
225
|
//
|
|
217
226
|
// IMPORTANT: each watch source holds a strong GObject ref on
|
|
218
227
|
// the session (the explicit `this.ref ()` below, released 1:1
|
|
@@ -223,8 +232,8 @@ namespace GjsifyRolldown {
|
|
|
223
232
|
// its own signal-handler closures — a self cycle) gets
|
|
224
233
|
// collected by the SpiderMonkey GC: dispose() removes the
|
|
225
234
|
// watch sources while the Rust build is still running, every
|
|
226
|
-
// later
|
|
227
|
-
//
|
|
235
|
+
// later wake lands on an unwatched fd, and the build stalls
|
|
236
|
+
// forever (issue #501). The extra refs also flip GJS's
|
|
228
237
|
// toggle-ref into "keep the JS wrapper rooted" mode, so the
|
|
229
238
|
// signal closures survive until the sources are dropped.
|
|
230
239
|
var req_chan = new GLib.IOChannel.unix_new (req_fd);
|
|
@@ -253,9 +262,9 @@ namespace GjsifyRolldown {
|
|
|
253
262
|
/**
|
|
254
263
|
* close — release the session's resources.
|
|
255
264
|
*
|
|
256
|
-
* Removes the three GLib
|
|
265
|
+
* Removes the three GLib wakeup-pipe watch sources and drops the owned
|
|
257
266
|
* Rust handle (whose `free_function`, gjsify_rolldown_session_free,
|
|
258
|
-
* tears down the bundler + tokio runtime +
|
|
267
|
+
* tears down the bundler + tokio runtime + wakeup pipes). Without this,
|
|
259
268
|
* every bundle leaked its session for the process lifetime — the
|
|
260
269
|
* watch sources hold a ref on the session, so neither the GObject
|
|
261
270
|
* nor the Rust side could ever be collected. Idempotent; safe to
|
|
@@ -268,14 +277,44 @@ namespace GjsifyRolldown {
|
|
|
268
277
|
_handle = null;
|
|
269
278
|
}
|
|
270
279
|
|
|
271
|
-
|
|
272
|
-
|
|
280
|
+
/**
|
|
281
|
+
* Consume EVERY byte currently queued on a wakeup pipe.
|
|
282
|
+
*
|
|
283
|
+
* The wakeup fds are pipes on every platform (see
|
|
284
|
+
* src/rust/src/wakeup.rs) — a byte stream, not an eventfd counter
|
|
285
|
+
* that one 8-byte read resets. Reading a single fixed-size chunk
|
|
286
|
+
* would leave the rest queued, and GLib's watch is
|
|
287
|
+
* level-triggered: the callback is re-dispatched (re-draining the
|
|
288
|
+
* queue for nothing) once per chunk until the pipe finally
|
|
289
|
+
* empties, so a burst of N hook wakeups costs N/chunk no-op main
|
|
290
|
+
* loop iterations instead of one. Draining to `AGAIN` collapses a
|
|
291
|
+
* burst back into a single wake-up cycle — the coalescing the
|
|
292
|
+
* eventfd counter used to give us for free.
|
|
293
|
+
*
|
|
294
|
+
* The order also matters: drain the pipe FIRST, then the queue.
|
|
295
|
+
* The Rust writer drops a wake when the buffer is full, which is
|
|
296
|
+
* only safe because every drain here is followed by a full queue
|
|
297
|
+
* drain (see src/rust/src/wakeup.rs). Every caller below keeps
|
|
298
|
+
* that order.
|
|
299
|
+
*
|
|
300
|
+
* Both ends are O_NONBLOCK, so the final read returns
|
|
301
|
+
* `IOStatus.AGAIN` instead of blocking the main loop.
|
|
302
|
+
*/
|
|
303
|
+
private void drain_wakeup (GLib.IOChannel source) {
|
|
273
304
|
try {
|
|
274
|
-
size_t got;
|
|
275
|
-
source.read_chars (
|
|
305
|
+
size_t got = 0;
|
|
306
|
+
while (source.read_chars (_drain_buf, out got) == GLib.IOStatus.NORMAL && got > 0) {
|
|
307
|
+
// Keep reading until AGAIN/EOF; the bytes themselves
|
|
308
|
+
// carry no information.
|
|
309
|
+
}
|
|
276
310
|
} catch (Error e) {
|
|
277
|
-
//
|
|
311
|
+
// Transient/EAGAIN-equivalent — the queue drain below is
|
|
312
|
+
// what actually matters.
|
|
278
313
|
}
|
|
314
|
+
}
|
|
315
|
+
|
|
316
|
+
private bool on_ctx_response_ready (GLib.IOChannel source, GLib.IOCondition cond) {
|
|
317
|
+
drain_wakeup (source);
|
|
279
318
|
|
|
280
319
|
while (_handle != null) {
|
|
281
320
|
var resp_bytes = _glue_session_next_context_response (_handle);
|
|
@@ -300,18 +339,9 @@ namespace GjsifyRolldown {
|
|
|
300
339
|
}
|
|
301
340
|
|
|
302
341
|
private bool on_request_ready (GLib.IOChannel source, GLib.IOCondition cond) {
|
|
303
|
-
// Drain the
|
|
304
|
-
//
|
|
305
|
-
|
|
306
|
-
// The eventfd content is an opaque 8-byte counter; we
|
|
307
|
-
// discard it (just need the wake-up).
|
|
308
|
-
char[] sink = new char[8];
|
|
309
|
-
try {
|
|
310
|
-
size_t got;
|
|
311
|
-
source.read_chars (sink, out got);
|
|
312
|
-
} catch (Error e) {
|
|
313
|
-
// EAGAIN-equivalent — eventfd had nothing left, fine.
|
|
314
|
-
}
|
|
342
|
+
// Drain the wakeup pipe FIRST, then the queue — see
|
|
343
|
+
// drain_wakeup() for why that order is load-bearing.
|
|
344
|
+
drain_wakeup (source);
|
|
315
345
|
|
|
316
346
|
// Pull all queued requests in this wake-up cycle.
|
|
317
347
|
while (_handle != null) {
|
|
@@ -373,13 +403,7 @@ namespace GjsifyRolldown {
|
|
|
373
403
|
}
|
|
374
404
|
|
|
375
405
|
private bool on_complete_ready (GLib.IOChannel source, GLib.IOCondition cond) {
|
|
376
|
-
|
|
377
|
-
try {
|
|
378
|
-
size_t got;
|
|
379
|
-
source.read_chars (sink, out got);
|
|
380
|
-
} catch (Error e) {
|
|
381
|
-
// ignore
|
|
382
|
-
}
|
|
406
|
+
drain_wakeup (source);
|
|
383
407
|
|
|
384
408
|
if (_handle == null) return false;
|
|
385
409
|
try {
|