xbintsc 0.5.43 → 0.5.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/README.md +5 -4
- package/README.zh-CN.md +4 -3
- package/dist/src/cli/extensions.d.ts +24 -0
- package/dist/src/cli/extensions.js +62 -0
- package/dist/src/cli/extensions.js.map +1 -0
- package/dist/src/cli/help.d.ts +1 -1
- package/dist/src/cli/help.js +3 -1
- package/dist/src/cli/help.js.map +1 -1
- package/dist/src/cli/hints.js +8 -3
- package/dist/src/cli/hints.js.map +1 -1
- package/dist/src/cli/main.js +21 -28
- package/dist/src/cli/main.js.map +1 -1
- package/dist/src/diagnostics/diagnostic.d.ts +3 -1
- package/dist/src/diagnostics/diagnostic.js +2 -0
- package/dist/src/diagnostics/diagnostic.js.map +1 -1
- package/dist/src/driver/build-key.d.ts +2 -0
- package/dist/src/driver/build-key.js +1 -0
- package/dist/src/driver/build-key.js.map +1 -1
- package/dist/src/driver/compiler.js +61 -8
- package/dist/src/driver/compiler.js.map +1 -1
- package/dist/src/driver/config.d.ts +4 -0
- package/dist/src/driver/config.js +2 -0
- package/dist/src/driver/config.js.map +1 -1
- package/dist/src/driver/link-features.d.ts +37 -0
- package/dist/src/driver/link-features.js +123 -0
- package/dist/src/driver/link-features.js.map +1 -0
- package/dist/src/driver/options.d.ts +10 -0
- package/dist/src/driver/runtime-objects.d.ts +5 -0
- package/dist/src/driver/runtime-objects.js +20 -1
- package/dist/src/driver/runtime-objects.js.map +1 -1
- package/dist/src/driver/usage.d.ts +23 -0
- package/dist/src/driver/usage.js +353 -0
- package/dist/src/driver/usage.js.map +1 -0
- package/dist/src/driver/version.d.ts +1 -1
- package/dist/src/driver/version.js +1 -1
- package/dist/src/extensions/catalog.js +1 -2
- package/dist/src/extensions/catalog.js.map +1 -1
- package/dist/src/extensions/gui/features.d.ts +31 -0
- package/dist/src/extensions/gui/features.js +160 -0
- package/dist/src/extensions/gui/features.js.map +1 -0
- package/dist/src/extensions/gui/index.js +10 -4
- package/dist/src/extensions/gui/index.js.map +1 -1
- package/dist/src/extensions/link-feature.d.ts +70 -0
- package/dist/src/extensions/link-feature.js +18 -0
- package/dist/src/extensions/link-feature.js.map +1 -0
- package/dist/src/extensions/registry.d.ts +24 -4
- package/dist/src/extensions/registry.js +26 -3
- package/dist/src/extensions/registry.js.map +1 -1
- package/dist/src/index.d.ts +3 -2
- package/dist/src/index.js +0 -1
- package/dist/src/index.js.map +1 -1
- package/dist/tests/cli/hints.test.js +5 -7
- package/dist/tests/cli/hints.test.js.map +1 -1
- package/dist/tests/driver/link-features.test.js +105 -0
- package/dist/tests/driver/link-features.test.js.map +1 -0
- package/dist/tests/driver/usage.test.d.ts +1 -0
- package/dist/tests/driver/usage.test.js +88 -0
- package/dist/tests/driver/usage.test.js.map +1 -0
- package/dist/tests/e2e/gui-audio.test.js +26 -4
- package/dist/tests/e2e/gui-audio.test.js.map +1 -1
- package/dist/tests/e2e/gui-browser.test.js +3 -0
- package/dist/tests/e2e/gui-browser.test.js.map +1 -1
- package/dist/tests/e2e/gui-events.test.js +67 -0
- package/dist/tests/e2e/gui-events.test.js.map +1 -1
- package/dist/tests/e2e/gui-feature-linking.test.d.ts +9 -0
- package/dist/tests/e2e/gui-feature-linking.test.js +68 -0
- package/dist/tests/e2e/gui-feature-linking.test.js.map +1 -0
- package/dist/tests/e2e/gui-helpers.d.ts +5 -1
- package/dist/tests/e2e/gui-helpers.js +2 -0
- package/dist/tests/e2e/gui-helpers.js.map +1 -1
- package/dist/tests/e2e/gui-layout.test.js +4 -1
- package/dist/tests/e2e/gui-layout.test.js.map +1 -1
- package/dist/tests/e2e/gui-media-disabled.test.d.ts +5 -4
- package/dist/tests/e2e/gui-media-disabled.test.js +7 -6
- package/dist/tests/e2e/gui-media-disabled.test.js.map +1 -1
- package/dist/tests/e2e/gui-media.test.d.ts +5 -4
- package/dist/tests/e2e/gui-media.test.js +9 -9
- package/dist/tests/e2e/gui-media.test.js.map +1 -1
- package/dist/tests/e2e/gui-paint.test.js +5 -2
- package/dist/tests/e2e/gui-paint.test.js.map +1 -1
- package/dist/tests/e2e/gui-shaping.test.d.ts +5 -0
- package/dist/tests/e2e/gui-shaping.test.js +47 -0
- package/dist/tests/e2e/gui-shaping.test.js.map +1 -0
- package/dist/tests/e2e/gui-webgl-pipeline-cache.test.d.ts +18 -0
- package/dist/tests/e2e/gui-webgl-pipeline-cache.test.js +120 -0
- package/dist/tests/e2e/gui-webgl-pipeline-cache.test.js.map +1 -0
- package/dist/tests/extensions/gui-features.test.d.ts +1 -0
- package/dist/tests/extensions/gui-features.test.js +168 -0
- package/dist/tests/extensions/gui-features.test.js.map +1 -0
- package/doc/ai/cli.md +5 -1
- package/doc/ai/extensions.md +52 -25
- package/doc/ai/troubleshooting.md +10 -7
- package/doc/ai/zh-CN/cli.md +5 -1
- package/doc/ai/zh-CN/extensions.md +45 -23
- package/doc/ai/zh-CN/troubleshooting.md +6 -5
- package/doc/gui/api.md +14 -5
- package/doc/gui/build.md +136 -19
- package/doc/gui/canvas.md +1 -1
- package/doc/gui/input.md +7 -0
- package/doc/gui/known-issues.md +32 -3
- package/doc/gui/layout.md +1 -1
- package/doc/gui/media.md +13 -7
- package/doc/gui/rendering.md +32 -15
- package/doc/gui/runtime.md +5 -0
- package/doc/gui/webgl.md +24 -6
- package/doc/gui-scripts.md +2 -1
- package/doc/gui.md +50 -35
- package/doc/implemented/cli.md +2 -0
- package/doc/implemented/extensions.md +9 -3
- package/doc/npm-package.md +4 -4
- package/doc/xbintsc.config.schema.json +10 -0
- package/doc/zh-CN/gui/api.md +13 -5
- package/doc/zh-CN/gui/build.md +110 -16
- package/doc/zh-CN/gui/canvas.md +1 -1
- package/doc/zh-CN/gui/input.md +5 -0
- package/doc/zh-CN/gui/known-issues.md +24 -3
- package/doc/zh-CN/gui/layout.md +1 -1
- package/doc/zh-CN/gui/media.md +11 -6
- package/doc/zh-CN/gui/rendering.md +26 -9
- package/doc/zh-CN/gui/runtime.md +3 -0
- package/doc/zh-CN/gui/webgl.md +3 -3
- package/doc/zh-CN/gui-scripts.md +2 -2
- package/doc/zh-CN/gui.md +42 -29
- package/doc/zh-CN/implemented/cli.md +2 -0
- package/doc/zh-CN/implemented/extensions.md +9 -3
- package/doc/zh-CN/npm-package.md +4 -3
- package/package.json +7 -7
- package/runtime/ext_gui/audio.cpp +18 -0
- package/runtime/ext_gui/audio_device.cpp +12 -2
- package/runtime/ext_gui/browser.cpp +24 -17
- package/runtime/ext_gui/browser_loop.cpp +13 -1
- package/runtime/ext_gui/canvas.h +1 -1
- package/runtime/ext_gui/canvas_raster.cpp +1 -1
- package/runtime/ext_gui/canvas_text.cpp +1 -1
- package/runtime/ext_gui/dom_api_canvas.cpp +15 -8
- package/runtime/ext_gui/dom_api_canvas_extra.cpp +44 -20
- package/runtime/ext_gui/dom_api_canvas_image.cpp +3 -1
- package/runtime/ext_gui/dom_api_events.cpp +2 -1
- package/runtime/ext_gui/dom_api_node_read.cpp +10 -0
- package/runtime/ext_gui/dom_api_offscreen.cpp +3 -1
- package/runtime/ext_gui/dom_api_style.cpp +7 -0
- package/runtime/ext_gui/dom_api_webgl_internal.h +4 -0
- package/runtime/ext_gui/dom_api_webgl_source.cpp +5 -2
- package/runtime/ext_gui/gpu_bridge.c +28 -0
- package/runtime/ext_gui/gpu_feature.cpp +30 -0
- package/runtime/ext_gui/gpu_shader.cpp +22 -0
- package/runtime/ext_gui/gpu_shader_api.h +50 -0
- package/runtime/ext_gui/gui.cpp +13 -37
- package/runtime/ext_gui/gui_canvas.cpp +14 -6
- package/runtime/ext_gui/gui_engine.h +10 -18
- package/runtime/ext_gui/gui_events.cpp +55 -12
- package/runtime/ext_gui/gui_keymap.cpp +254 -0
- package/runtime/ext_gui/gui_keymap.h +31 -0
- package/runtime/ext_gui/gui_webgl.cpp +25 -10
- package/runtime/ext_gui/image_api.cpp +18 -0
- package/runtime/ext_gui/image_api.h +41 -0
- package/runtime/ext_gui/image_bitmap_browser.cpp +20 -11
- package/runtime/ext_gui/image_bridge.c +22 -0
- package/runtime/ext_gui/image_feature.cpp +105 -0
- package/runtime/ext_gui/layout.cpp +13 -8
- package/runtime/ext_gui/layout.h +2 -2
- package/runtime/ext_gui/media.cpp +4 -2
- package/runtime/ext_gui/media.h +3 -3
- package/runtime/ext_gui/media_api.h +7 -6
- package/runtime/ext_gui/renderer.h +1 -1
- package/runtime/ext_gui/renderer_image.cpp +13 -6
- package/runtime/ext_gui/renderer_layer.cpp +10 -5
- package/runtime/ext_gui/stb_truetype.h +5077 -0
- package/runtime/ext_gui/text.cpp +24 -498
- package/runtime/ext_gui/text.h +7 -5
- package/runtime/ext_gui/text_api.h +45 -0
- package/runtime/ext_gui/text_bridge.c +23 -0
- package/runtime/ext_gui/text_cache.cpp +1 -1
- package/runtime/ext_gui/text_glyph_cache.cpp +2 -2
- package/runtime/ext_gui/text_hb.cpp +554 -0
- package/runtime/ext_gui/text_stb.cpp +513 -0
- package/runtime/ext_gui/webgl_api.cpp +19 -0
- package/runtime/ext_gui/webgl_api.h +53 -0
- package/runtime/ext_gui/webgl_bridge.c +25 -0
- package/runtime/ext_gui/webgl_draw_setup.cpp +42 -1
- package/runtime/ext_gui/webgl_feature.cpp +82 -0
- package/runtime/ext_gui/webgl_feature.h +28 -0
- package/runtime/ext_gui/webgl_msl.cpp +41 -0
- package/runtime/ext_gui/webgl_msl.h +6 -0
- package/runtime/ext_gui/webgl_shader.cpp +40 -32
- package/runtime/ext_gui/window.cpp +17 -8
- package/runtime/ext_media/decode_audio.cpp +1 -1
- package/runtime/ext_media/media.cpp +1 -1
- package/runtime/ext_media/media_bridge.c +7 -7
- package/scripts/build-gui.ts +97 -98
- package/scripts/check-file-length.ts +1 -0
- package/scripts/gui-build-options.ts +122 -0
- package/src/cli/extensions.ts +72 -0
- package/src/cli/help.ts +3 -1
- package/src/cli/hints.ts +9 -3
- package/src/cli/main.ts +21 -27
- package/src/diagnostics/diagnostic.ts +2 -0
- package/src/driver/build-key.ts +3 -0
- package/src/driver/compiler.ts +63 -8
- package/src/driver/config.ts +6 -0
- package/src/driver/link-features.ts +163 -0
- package/src/driver/options.ts +10 -0
- package/src/driver/runtime-objects.ts +26 -1
- package/src/driver/usage.ts +376 -0
- package/src/driver/version.ts +1 -1
- package/src/extensions/catalog.ts +1 -2
- package/src/extensions/gui/features.ts +162 -0
- package/src/extensions/gui/index.ts +10 -4
- package/src/extensions/link-feature.ts +72 -0
- package/src/extensions/registry.ts +45 -4
- package/src/index.ts +3 -2
- package/dist/src/extensions/media/index.d.ts +0 -28
- package/dist/src/extensions/media/index.js +0 -59
- package/dist/src/extensions/media/index.js.map +0 -1
- package/dist/tests/extensions/media.test.js +0 -63
- package/dist/tests/extensions/media.test.js.map +0 -1
- package/src/extensions/media/index.ts +0 -65
- /package/dist/tests/{extensions/media.test.d.ts → driver/link-features.test.d.ts} +0 -0
|
@@ -1,8 +1,9 @@
|
|
|
1
|
-
# 扩展:Node 模块、GUI
|
|
1
|
+
# 扩展:Node 模块、GUI、链接特性、C++/Rust 库
|
|
2
2
|
|
|
3
3
|
编译器核心与平台无关。任何与平台相关的东西——Node 的 `fs`、一个 HTML/CSS 渲染器、
|
|
4
4
|
你自己的 C++ 库——都以**扩展**的形式接入:它既贡献模块绑定,也贡献要一起编译/链接的
|
|
5
|
-
C/C++
|
|
5
|
+
C/C++ 源码或目标文件。扩展里的重型可选部分(FFmpeg、glslang)做成**链接特性**,
|
|
6
|
+
由程序用法选择,而不是单独的扩展开关。
|
|
6
7
|
|
|
7
8
|
## 最容易踩的规则
|
|
8
9
|
|
|
@@ -70,8 +71,9 @@ import { createWindow, run } from "gui";
|
|
|
70
71
|
`<style>` 块(运行时解析器只读取内联 `<style>`);并让随包引入的 Three.js 的
|
|
71
72
|
`WebGLRenderer` 先探测 WebGL 上下文:引擎有上下文时使用 Three.js 自己的渲染器,
|
|
72
73
|
直接使用真实的 WebGL 2 后端(只修补其着色器循环展开,不再打包 Canvas2D 回退)。
|
|
73
|
-
`<img>`
|
|
74
|
-
|
|
74
|
+
`<img>` 由引擎自身解码;`<video>`/`<audio>` 解码(FFmpeg)与 GLSL 翻译(glslang)
|
|
75
|
+
是**可选的链接特性**,由程序自动选择——见
|
|
76
|
+
[可选链接特性](#可选链接特性media-gpu-images-shaping-webgl)。也支持 `<canvas>`:
|
|
75
77
|
`getContext("2d")` 返回的上下文,其填充、路径、
|
|
76
78
|
渐变、文本与 `drawImage` 由自研 CPU 光栅器处理(另有线段端点/连接、虚线描边、
|
|
77
79
|
`ellipse` 与 `roundRect`),元素句柄还暴露 `width`/`height`。CSS
|
|
@@ -91,37 +93,57 @@ import { createWindow, run } from "gui";
|
|
|
91
93
|
`getComputedStyle`、`alert`/`prompt`、`escape`、`Event` 家族,以及可用的 Web Audio
|
|
92
94
|
实现(`AudioContext` 提供 gain、oscillator、buffer-source、biquad、compressor、
|
|
93
95
|
stereo-panner、constant-source 与 delay 节点、真实的 48 kHz 混音器、逐采样的
|
|
94
|
-
AudioParam 自动化,`decodeAudioData` 引擎内解码 WAV,压缩格式则通过 `media
|
|
96
|
+
AudioParam 自动化,`decodeAudioData` 引擎内解码 WAV,压缩格式则通过 `media` 特性)—— 另外还有帧泵:
|
|
95
97
|
它在 `loadHTML` 时启动,仅在仍有动画帧回调时重新排程,因此从不调用 `run()` 的页面
|
|
96
98
|
也能绘制并最终退出。**WebGL 1.0 与 2.0 已实现**:`canvas.getContext("webgl")` /
|
|
97
99
|
`"webgl2"` 返回 GPU 上下文,其 GLSL ES 着色器由内置 glslang 编译为 SPIR-V
|
|
98
100
|
(Vulkan 直接消费,Metal 上由内置 SPIRV-Cross 转成 MSL),
|
|
99
101
|
并通过 SDL_GPU 绘制;页面直接合成颜色纹理而无须 CPU 读回,WebGL 2 还增加
|
|
100
102
|
VAO、实例化、整数属性、采样器对象、`texStorage2D` 与缓冲拷贝(见
|
|
101
|
-
[../../zh-CN/gui/webgl.md](../../zh-CN/gui/webgl.md))。WebXR
|
|
102
|
-
`
|
|
103
|
-
`"undefined"
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
`
|
|
110
|
-
|
|
103
|
+
[../../zh-CN/gui/webgl.md](../../zh-CN/gui/webgl.md))。WebXR 与流式接口**尚未实现**:
|
|
104
|
+
`typeof` 会回答
|
|
105
|
+
`"undefined"`,让页面走自己的降级分支。`fetch` 不是 GUI 扩展的功能——它由核心
|
|
106
|
+
运行时提供(`http://`,以及本地 `file://` 与相对路径;不支持 HTTPS 与流式),
|
|
107
|
+
因此 GUI 程序同样可用。见 [../../implemented/fetch.md](../../implemented/fetch.md)。
|
|
108
|
+
|
|
109
|
+
### 可选链接特性(media、gpu、images、shaping、webgl)
|
|
110
|
+
|
|
111
|
+
`gui` 仍然只有一个扩展,但其中的重型部分是可选的**链接特性**。编译器扫描打包后的
|
|
112
|
+
HTML/CSS/TS(导入的 `.html` 与内联 CSS 都属于 bundle),按程序自动选择;不需要额外
|
|
113
|
+
扩展参数:
|
|
114
|
+
|
|
115
|
+
- **`media`** —— `<video>`/`<audio>` 与压缩格式 `decodeAudioData` 的 FFmpeg 解码。
|
|
116
|
+
由 `<video>`/`<audio>` 元素、`createElement("video"/"audio")`、`new Audio()`、
|
|
117
|
+
`decodeAudioData` 或运行时拼出的 markup 触发,链接 FFmpeg 后端
|
|
118
|
+
`media.a`/`media.lib`。
|
|
119
|
+
- **`gpu`** —— WebGL 与 `mix-blend-mode` 共用的 glslang/SPIRV-Cross GLSL 翻译器。
|
|
120
|
+
由 `getContext("webgl"/"webgl2"/…)`(或运行时决定的 kind)、任意 `webgl` 字符串、
|
|
121
|
+
`mix-blend-mode` 触发。
|
|
122
|
+
- **`images`** —— `<img>`、内联 `<svg>`、CSS `background-image`/`url(...)`、
|
|
123
|
+
`drawImage`/`createImageBitmap`/`Image` 与应用图标所需的 stb_image/NanoSVG 解码。
|
|
124
|
+
纯文本页面两个解码器都不链接。
|
|
125
|
+
- **`shaping`** —— HarfBuzz + FreeType 提供的完整 OpenType 整形(连字、复杂文种)
|
|
126
|
+
与 CFF/OTF/可变字体支持。非 ASCII 文本(或转义/实体码点)会自动选中;纯拉丁页面
|
|
127
|
+
保持 stb_truetype 后端,两个库都不链接。
|
|
128
|
+
- **`webgl`** —— WebGL context 与 DOM 绑定。由 WebGL 的 `getContext(...)` 或
|
|
129
|
+
`webgl` 字符串触发;它依赖 `gpu`,两者一起激活。不用 WebGL 的页面两者都不链接。
|
|
130
|
+
|
|
131
|
+
用 `--feature media,gpu,images,shaping,webgl`(或 `all`)覆盖扫描,用 `--no-feature media`
|
|
132
|
+
强制不链接。`--verbose` 会打印选中的集合。只渲染 HTML/CSS/2D canvas 的程序既不链接
|
|
133
|
+
FFmpeg、glslang,也不链接图片解码器、HarfBuzz/FreeType 或 WebGL context;用运行时数据拼 HTML/CSS 的
|
|
134
|
+
页面应传 `--feature all`,因为扫描看不到它。
|
|
135
|
+
自动检测到的特性归档缺失时构建打印 warning,能力在运行时降级(媒体元素触发
|
|
136
|
+
`error`);显式强制的特性则报错。
|
|
111
137
|
|
|
112
138
|
DOM API 与普通 HTML 媒体元素一致:`play()` / `pause()` / `load()`,以及
|
|
113
139
|
`paused` / `ended` / `duration` / `currentTime` / `volume` / `muted` / `loop` /
|
|
114
140
|
`autoplay` / `videoWidth` / `videoHeight` / `readyState` / `src` 属性,还有
|
|
115
141
|
`loadedmetadata`/`play`/`pause`/`ended`/`timeupdate`/`error` 事件。没有内置
|
|
116
142
|
控件——用元素和 CSS 自绘控件(见 `examples/gui/media.ts`)。视频帧与音频 PCM
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
自带的 bash)与原生 GNU make(Chocolatey `make`,或 `xbintsc_MAKE`)。见
|
|
122
|
-
[../../zh-CN/gui/build.md](../../zh-CN/gui/build.md)。
|
|
123
|
-
没有归档、或宿主上还没运行 `npm run media` 时,媒体元素会触发 `error`;见
|
|
124
|
-
[../../zh-CN/gui/known-issues.md](../../zh-CN/gui/known-issues.md)。
|
|
143
|
+
都在主线程解码;视频走图像纹理路径。Windows 上构建归档需要 POSIX shell(Git for
|
|
144
|
+
Windows 自带的 bash)与原生 GNU make(Chocolatey `make`,或 `xbintsc_MAKE`)。见
|
|
145
|
+
[../../zh-CN/gui/build.md](../../zh-CN/gui/build.md) 与
|
|
146
|
+
[../../zh-CN/gui/media.md](../../zh-CN/gui/media.md)。
|
|
125
147
|
|
|
126
148
|
## 第三方 npm 包
|
|
127
149
|
|
|
@@ -113,14 +113,15 @@ npm 包会提供它(去掉 `--omit=optional` 重新安装),但普通源码
|
|
|
113
113
|
|
|
114
114
|
### `media native library not found at …`
|
|
115
115
|
|
|
116
|
-
`<video>`/`<audio>`
|
|
116
|
+
`<video>`/`<audio>` 解码需要可选的 `media` 链接特性及其归档(`media.a`,Windows 上是
|
|
117
117
|
`media.lib`)。发布为其构建时,`@xbintsc/<os>-<arch>` npm 包会提供它;否则用
|
|
118
|
-
`npm run media`
|
|
119
|
-
|
|
120
|
-
|
|
118
|
+
`npm run media` 构建。自动选中该特性而归档缺失时,构建打印 **warning** 后继续,
|
|
119
|
+
媒体元素只会触发 `error`;显式 `--feature media`(或弃用的 `--ext media`)则直接
|
|
120
|
+
报错。可以用 `--no-feature media` 主动关闭。Windows 上 FFmpeg 的
|
|
121
121
|
构建还需要 POSIX shell(Git for Windows 的 bash)与原生 GNU make
|
|
122
122
|
(Chocolatey `make`,或 `xbintsc_MAKE`)。脚本会指出缺失的是哪一个。见
|
|
123
|
-
[../../zh-CN/gui/build.md](../../zh-CN/gui/build.md)
|
|
123
|
+
[../../zh-CN/gui/build.md](../../zh-CN/gui/build.md) 与
|
|
124
|
+
[../../zh-CN/gui/build.md#链接特性](../../zh-CN/gui/build.md#链接特性)。
|
|
124
125
|
|
|
125
126
|
### `Icon file not found` / 图标工具缺失
|
|
126
127
|
|
package/doc/gui/api.md
CHANGED
|
@@ -91,12 +91,13 @@ requestAnimationFrame(frame);
|
|
|
91
91
|
| `navigator` | `userAgent` (a desktop Chrome string ending in `xbintsc`), `platform`, `vendor`, `language(s)`, `hardwareConcurrency`, `maxTouchPoints`, `onLine`, `vibrate()` |
|
|
92
92
|
| `localStorage` | an in-memory `getItem` / `setItem` / `removeItem` / `clear` / `key` / `length` store |
|
|
93
93
|
| `AudioContext`, `webkitAudioContext` | a Web Audio graph with a real mixer and decoder (see below) |
|
|
94
|
+
| `fetch(input, init)` | the core runtime's HTTP/1.1 client, available to GUI programs too: `http://`, plus local `file://` and relative paths; `https://` and streaming are not supported. See [Implemented fetch](../implemented/fetch.md). |
|
|
94
95
|
| `Event`, `CustomEvent`, `ProgressEvent`, `MouseEvent`, `KeyboardEvent`, `PointerEvent`, `WheelEvent`, `FocusEvent`, `InputEvent`, `SubmitEvent` | event constructors; the `init` argument is kept as given (`detail`, `composed`, mouse/keyboard fields, …) |
|
|
95
96
|
| `AbortController`, `AbortSignal` | a real controller/signal pair for `{ signal }` listener removal, with `abort()`, `aborted`, `reason`, `onabort` and `throwIfAborted()` |
|
|
96
97
|
| `HTMLCanvasElement`, `HTMLImageElement`, `WebGLRenderingContext`, `WebGL2RenderingContext`, `WebGLBuffer`/`WebGLTexture`/`WebGLShader`/`WebGLProgram`/`WebGLFramebuffer`/`WebGLRenderbuffer`/`WebGLUniformLocation`/`WebGLActiveInfo`/`WebGLVertexArrayObject`/`WebGLSampler`/`WebGLQuery`/`WebGLSync`/`WebGLTransformFeedback` | real constructor values. `canvas.getContext("webgl")` and `getContext("webgl2")` return GPU contexts (see [WebGL](./webgl.md)). |
|
|
97
98
|
| `ImageBitmap`, `createImageBitmap(source)` | immutable bitmaps backed by decoded pixels (`<img>`, `<canvas>`, `<video>` frames, `OffscreenCanvas` and `ImageData` shapes); the promise resolves immediately and `texImage2D` accepts the result |
|
|
98
99
|
| `OffscreenCanvas` | a real 2D offscreen bitmap with `getContext("2d")`, `width`/`height`, `transferToImageBitmap()` and `convertToBlob()`; `canvas.transferControlToOffscreen()` hands an element's bitmap over (see [Canvas](./canvas.md)). There is no offscreen WebGL context. |
|
|
99
|
-
| `ImageData`, `XRWebGL*`, `Request`, `Headers`, `Response`, `ReadableStream`, `DOMParser` | `typeof` probes answer `"undefined"` (no XR
|
|
100
|
+
| `ImageData`, `XRWebGL*`, `Request`, `Headers`, `Response`, `ReadableStream`, `DOMParser` | `typeof` probes answer `"undefined"` (no XR, DOM parser or streaming backend), which lets a page take its "unsupported" branch instead of calling a no-op stub. `fetch` still works and resolves with a response-shaped object, but there is no `Response` constructor and `response.arrayBuffer()` currently returns a byte array rather than an `ArrayBuffer`. Three.js pages use its real renderer on this backend (see [Known issues](./known-issues.md)). |
|
|
100
101
|
| `requestAnimationFrame` / `cancelAnimationFrame` | queued on the window's frame list; see [Runtime integration](./runtime.md) |
|
|
101
102
|
| `alert` / `confirm` / `prompt`, `matchMedia`, `getComputedStyle`, `escape` | host implementations. `matchMedia` returns a **live** list (cached per query): `matches` is re-evaluated on resize and `change` listeners fire when it flips. `confirm`/`prompt` use a native dialog only with `XT_GUI_DIALOGS=1`; otherwise they report the message on stderr and return the documented default (Annex B `escape` encodes per UTF-16 code unit, matching `decodeURIComponent(escape(bytes))`) |
|
|
102
103
|
|
|
@@ -136,7 +137,8 @@ evaluated per sample, so a ramp keeps moving while the page is idle.
|
|
|
136
137
|
procedurally. `decodeAudioData(bytes)` returns a promise for a decoded
|
|
137
138
|
AudioBuffer: WAV (PCM 8/16/24/32-bit, IEEE float 32/64, A-law and µ-law) is
|
|
138
139
|
decoded by the engine itself; MP3, AAC/M4A, Vorbis, Opus and FLAC need the
|
|
139
|
-
optional `media`
|
|
140
|
+
optional `media` link feature (auto-selected when the program calls
|
|
141
|
+
`decodeAudioData`), whose FFmpeg decoder is used as
|
|
140
142
|
the fallback. `bytes` may be an ArrayBuffer, a typed array, a Node `Buffer`, or
|
|
141
143
|
the raw byte string `fs.readFileSync` returns (this runtime has no dedicated
|
|
142
144
|
binary type, and that string's storage is exactly the file bytes). The result
|
|
@@ -163,17 +165,24 @@ Input handlers receive a single payload object (lifecycle handlers receive none)
|
|
|
163
165
|
```ts
|
|
164
166
|
win.on("click", (e) => console.log(e.x, e.y, e.button, e.target));
|
|
165
167
|
win.on("wheel", (e) => console.log(e.deltaX, e.deltaY));
|
|
166
|
-
win.on("keydown", (e) => console.log(e.key, e.code, e.
|
|
168
|
+
win.on("keydown", (e) => console.log(e.key, e.code, e.ctrlKey, e.shiftKey));
|
|
167
169
|
```
|
|
168
170
|
|
|
171
|
+
`document.addEventListener("keydown", handler)` receives the same event after it
|
|
172
|
+
bubbles from the focused element or `<body>`, before the window listeners run.
|
|
173
|
+
`code` is the physical key and uses the browser spelling (`"KeyW"`,
|
|
174
|
+
`"ArrowUp"`, `"Space"`); `key` is the produced value (`"w"`/`"W"`, `"ArrowUp"`,
|
|
175
|
+
`" "`). See [input.md](./input.md).
|
|
176
|
+
|
|
169
177
|
| Field | Events | Meaning |
|
|
170
178
|
| --- | --- | --- |
|
|
171
179
|
| `x`, `y` / `clientX`, `clientY` | pointer, wheel | viewport-relative logical pixels |
|
|
172
180
|
| `button` | pointer | `0` left, `1` middle, `2` right (`-1` for move) |
|
|
173
181
|
| `clicks` | pointer | click count reported by the OS |
|
|
174
182
|
| `deltaX`, `deltaY` | wheel | scroll amount (`deltaY` positive = down) |
|
|
175
|
-
| `key`, `code` | keyboard |
|
|
176
|
-
| `repeat`, `ctrl`, `shift`, `alt`, `meta` | keyboard | modifiers |
|
|
183
|
+
| `key`, `code` | keyboard | browser `KeyboardEvent` spellings: produced value and physical key |
|
|
184
|
+
| `repeat`, `ctrl`, `shift`, `alt`, `meta` | keyboard | modifiers (the legacy DOM spellings) |
|
|
185
|
+
| `ctrlKey`, `shiftKey`, `altKey`, `metaKey` | keyboard | modifiers (the current DOM spellings) |
|
|
177
186
|
| `target` | pointer, wheel | deepest element under the point as a CSS-like descriptor (`div#main.card`), or absent |
|
|
178
187
|
|
|
179
188
|
## Diagnostics
|
package/doc/gui/build.md
CHANGED
|
@@ -8,13 +8,33 @@
|
|
|
8
8
|
`runtime/ext_media/` and is built separately.
|
|
9
9
|
- `npm run gui` (`scripts/build-gui.ts`) fetches a pinned SDL3 (`SDL3_TAG`,
|
|
10
10
|
default `release-3.2.10`), builds a static SDL3, fetches and builds static
|
|
11
|
-
FreeType (`FREETYPE_VERSION
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
11
|
+
**FreeType** (`FREETYPE_VERSION`, default `2.13.3`) and **HarfBuzz**
|
|
12
|
+
(`HARFBUZZ_VERSION`, default `10.1.0`) for the optional `shaping` feature,
|
|
13
|
+
and fetches and builds static **glslang** (`GLSLANG_VERSION`, default
|
|
14
|
+
`16.6.0`) against the pinned SPIR-V headers (`SPIRV_HEADERS_VERSION`), and —
|
|
15
|
+
**on macOS only** — **SPIRV-Cross** (`SPIRV_CROSS_VERSION`, default
|
|
16
|
+
`vulkan-sdk-1.4.313.0`). glslang is the runtime GLSL ES compiler behind WebGL
|
|
17
|
+
and SPIRV-Cross turns its SPIR-V into MSL for the Metal backend (see
|
|
18
|
+
[WebGL](./webgl.md)); both are merged into the archive like the other
|
|
19
|
+
libraries. Metal is the only SDL_GPU backend that consumes MSL, so on
|
|
20
|
+
Windows/Linux the script neither fetches nor merges SPIRV-Cross:
|
|
21
|
+
`webgl_msl.cpp` compiles to nothing there and the ~2.5 MiB it contributes
|
|
22
|
+
would be unreachable. The default text path uses the bundled
|
|
23
|
+
**stb_truetype** (`runtime/ext_gui/stb_truetype.h`); HarfBuzz + FreeType are
|
|
24
|
+
only extracted when the `shaping` feature is active.
|
|
25
|
+
Every library (SDL3, FreeType, HarfBuzz, glslang, SPIRV-Cross and the engine)
|
|
26
|
+
is compiled with `-ffunction-sections -fdata-sections`, and SDL is configured
|
|
27
|
+
without the subsystems the engine never calls (`SDL_JOYSTICK`, `SDL_HAPTIC`,
|
|
28
|
+
`SDL_SENSOR`, `SDL_CAMERA`, `SDL_RENDER`, `SDL_OPENGL`/`SDL_OPENGLES`,
|
|
29
|
+
`SDL_OFFSCREEN`, `SDL_HIDAPI`). HarfBuzz omits the APIs the shaping backend
|
|
30
|
+
never uses (`HB_NO_DRAW`/`PAINT`/`COLOR`/`NAME`/`STYLE`/`METRICS`/…, while
|
|
31
|
+
keeping `CFF`/`OPEN`/`VAR` for OTF and variable fonts) and is built with
|
|
32
|
+
`HB_NO_PRAGMA_GCC_DIAGNOSTIC_ERROR`, because `src/hb.hh` raises its warning
|
|
33
|
+
set to errors and two lean defines leave macOS/CoreText helpers unreferenced
|
|
34
|
+
(`HB_NO_MMAP` and `HB_NO_OT_FONT_GLYPH_NAMES`). The linker then drops
|
|
35
|
+
the unreachable functions (`--gc-sections`, `-dead_strip`, or link.exe's
|
|
36
|
+
default `/OPT:REF`). The
|
|
37
|
+
script then compiles the engine and
|
|
18
38
|
merges everything into `runtime/lib/<os>-<arch>/gui.a` (or
|
|
19
39
|
`gui.lib` with the MSVC ABI on Windows), same convention as `core.a`,
|
|
20
40
|
statically bundling all three. The merge uses `libtool` on macOS and
|
|
@@ -25,13 +45,17 @@
|
|
|
25
45
|
minimal static build with only the demuxers/parsers/decoders the media
|
|
26
46
|
elements need, compiles `runtime/ext_media/media.cpp` against it and merges
|
|
27
47
|
everything into `runtime/lib/<os>-<arch>/media.a` (`media.lib` with the MSVC
|
|
28
|
-
ABI on Windows). `gui.a` does **not** link FFmpeg:
|
|
29
|
-
|
|
48
|
+
ABI on Windows). `gui.a` does **not** link FFmpeg: the compiler selects the
|
|
49
|
+
`media` link feature for a program that may play media and only then links
|
|
50
|
+
`media.a` (see [Link features](#link-features)).
|
|
30
51
|
- `src/extensions/gui/index.ts` exposes the engine archive through
|
|
31
52
|
`nativeObjects()` (via `findRuntimeLibrary`, so `.a`/`.lib` both work) and the
|
|
32
|
-
OS frameworks through `linkerFlags()`. `
|
|
33
|
-
|
|
34
|
-
|
|
53
|
+
OS frameworks through `linkerFlags()`. Its `linkFeatures()` (in
|
|
54
|
+
`src/extensions/gui/features.ts`) declares `media`, `gpu`, `images`, `shaping` and `webgl`; the driver
|
|
55
|
+
compiles their bridges only when a feature is active.
|
|
56
|
+
- glslang (and SPIRV-Cross on macOS) is merged into `gui.a` like SDL3, but the
|
|
57
|
+
engine reaches it only through the `gpu` feature's dispatch table. A program
|
|
58
|
+
that never compiles GLSL does not extract those archive members.
|
|
35
59
|
- `scripts/build-gui.ts` compiles with the *same* clang the driver resolves
|
|
36
60
|
(`src/driver/toolchain-provider.ts`); on Windows that is the MSVC-targeting
|
|
37
61
|
LLVM bundled with Visual Studio, and its sibling `clang++`/`llvm-ar` are used
|
|
@@ -59,11 +83,12 @@
|
|
|
59
83
|
- CI builds `gui.a`/`gui.lib` and `media.a`/`media.lib` before assembling the
|
|
60
84
|
release archive so they ship inside `runtime/lib/<slug>/` (the release tarball
|
|
61
85
|
copies the whole `runtime/` tree). The same folders are packed into the
|
|
62
|
-
optional `@xbintsc/<slug>` npm package, so `--ext gui
|
|
86
|
+
optional `@xbintsc/<slug>` npm package, so `--ext gui` (with automatic
|
|
87
|
+
`media`/`gpu`/`images`/`shaping`/`webgl` selection) works from
|
|
63
88
|
an npm install on the hosts the release builds for. The archives are built and
|
|
64
89
|
the examples are run on Linux (under Xvfb, with the lavapipe software Vulkan
|
|
65
90
|
driver) and macOS;
|
|
66
|
-
the Windows x64 job builds both archives and links a `gui
|
|
91
|
+
the Windows x64 job builds both archives and links a `gui` program, so
|
|
67
92
|
the MSVC build is validated even though *running* the examples there stays
|
|
68
93
|
provisional (the hosted runner has no GPU/Vulkan driver). The macOS run is
|
|
69
94
|
provisional too: the Intel runner's virtualised GPU lacks
|
|
@@ -82,8 +107,9 @@
|
|
|
82
107
|
- The existing cache fingerprint already hashes `nativeObjects()` contents and
|
|
83
108
|
`runtimeSources()` content, so rebuilding either archive invalidates cached
|
|
84
109
|
binaries automatically.
|
|
85
|
-
- CMake build trees live *next to* their sources (`vendor/sdl-build`,
|
|
86
|
-
`vendor/freetype-build`, `vendor/harfbuzz-build
|
|
110
|
+
- CMake build trees live *next to* their sources (`vendor/sdl-build-lean`,
|
|
111
|
+
`vendor/freetype-build-lean`, `vendor/harfbuzz-build-lean`,
|
|
112
|
+
`vendor/glslang-build-lean`, `vendor/spirv-cross-build-lean`), never inside them: a
|
|
87
113
|
`CMakeCache.txt` records the machine it was configured on, and SDL's video
|
|
88
114
|
backends are `option()`s, so the first configure to run wins. A cached
|
|
89
115
|
`vendor/SDL` tree that also carried `build/` therefore restored one machine's
|
|
@@ -95,6 +121,96 @@
|
|
|
95
121
|
`xbintsc_SKIP_HEADER_CHECK=1` when they live under a prefix that check does
|
|
96
122
|
not know).
|
|
97
123
|
|
|
124
|
+
## Link features
|
|
125
|
+
|
|
126
|
+
`gui` is the only user-facing extension, but two heavy halves are **optional link
|
|
127
|
+
features** selected per program:
|
|
128
|
+
|
|
129
|
+
| Feature | What it links | Selected when the program may use it |
|
|
130
|
+
| --- | --- | --- |
|
|
131
|
+
| `media` | `media_bridge.c` + `media.a` (FFmpeg) | an HTML `<video>`/`<audio>` tag, `createElement("video"/"audio")`, `new Audio()`, `decodeAudioData`, or runtime-built markup |
|
|
132
|
+
| `gpu` | `gpu_bridge.c` (the implementation is already in `gui.a`) | a `getContext("webgl"/"webgl2"/"experimental-webgl")` (or a kind decided at run time), any `webgl` string, `mix-blend-mode`/`mixBlendMode`, or runtime-built markup |
|
|
133
|
+
| `images` | `image_bridge.c` (the implementation is already in `gui.a`) | `<img>`, inline `<svg>`, `background-image`/`url(...)`, a `.svg` path, `drawImage`/`createImageBitmap`/`Image`, or runtime-built markup |
|
|
134
|
+
| `shaping` | `text_bridge.c` (the implementation is already in `gui.a`) | non-ASCII text (CJK/Cyrillic/Greek/…), an escaped/entity codepoint, or runtime-built markup |
|
|
135
|
+
| `webgl` | `webgl_bridge.c` (the implementation is already in `gui.a`; requires `gpu`) | a WebGL `getContext(...)`, a `webgl` string, or runtime-built markup |
|
|
136
|
+
|
|
137
|
+
The driver scans the bundled AST (`src/driver/usage.ts`): every string literal
|
|
138
|
+
(including imported `.html` and the CSS inlined from it), identifiers and
|
|
139
|
+
property names, each `getContext(...)` kind, and whether HTML/CSS can be built
|
|
140
|
+
from data the compiler cannot see. Simple `const` aliases are resolved;
|
|
141
|
+
anything unresolved makes the program “dynamic markup” and enables the optional
|
|
142
|
+
features conservatively.
|
|
143
|
+
|
|
144
|
+
When a feature is active, the driver compiles its C **bridge only for that
|
|
145
|
+
build** and links it before `gui.a`:
|
|
146
|
+
|
|
147
|
+
- `media` → `runtime/ext_media/media_bridge.c`; its constructor installs the
|
|
148
|
+
decoder table and references `xt_media_backend_*`, which is what extracts the
|
|
149
|
+
FFmpeg members from `media.a`.
|
|
150
|
+
- `gpu` → `runtime/ext_gui/gpu_bridge.c`; its constructor calls
|
|
151
|
+
`xt_gpu_feature_register` in `gpu_feature.cpp`, which installs the
|
|
152
|
+
`XtGpuShaderApi` table and references `compile_glsl` / `cross_compile_msl` /
|
|
153
|
+
the reflection helpers, extracting glslang (and SPIRV-Cross on macOS).
|
|
154
|
+
- `images` → `runtime/ext_gui/image_bridge.c`; its constructor installs
|
|
155
|
+
`XtImageApi` via `image_feature.cpp`, which references stb_image and NanoSVG.
|
|
156
|
+
Layout, paint, the canvas `drawImage` path and the app-icon path call through
|
|
157
|
+
that table, so a text-only page links neither decoder.
|
|
158
|
+
- `shaping` → `runtime/ext_gui/text_bridge.c`; its constructor installs
|
|
159
|
+
`TextApi` via `text_hb.cpp`, which references HarfBuzz and FreeType. Without
|
|
160
|
+
it the engine uses the built-in stb_truetype backend, so a Latin-only page
|
|
161
|
+
links neither library.
|
|
162
|
+
- `webgl` → `runtime/ext_gui/webgl_bridge.c`; its constructor installs
|
|
163
|
+
`XtWebglApi` via `webgl_feature.cpp`, which references the WebGL context and
|
|
164
|
+
DOM bindings. The core reaches `getContext`, the canvas width/height setters,
|
|
165
|
+
`createImageBitmap`, the host types and the renderer through that table, so a
|
|
166
|
+
page with no WebGL links none of `webgl_*`/`dom_api_webgl*`.
|
|
167
|
+
|
|
168
|
+
A bridge is the “strong reference” that makes the linker search the archive; with
|
|
169
|
+
no bridge, the engine's table stays null and the implementation members are never
|
|
170
|
+
extracted. That is why the large libraries can stay in the existing archives
|
|
171
|
+
without a new package layout.
|
|
172
|
+
|
|
173
|
+
Overrides: `--feature media,gpu,images,shaping,webgl` (or `all`) forces a
|
|
174
|
+
feature on; `--no-feature media` keeps it out even when detection finds a use.
|
|
175
|
+
`webgl` requires `gpu`, and the resolver activates both together. An app icon
|
|
176
|
+
forces `images` because the icon is decoded through the same pipeline. `--ext media`
|
|
177
|
+
is a deprecated alias for `--ext gui --feature media`. If an auto-detected
|
|
178
|
+
feature's archive is missing, the build warns and continues (the element/context
|
|
179
|
+
degrades as documented); a forced feature makes it an error.
|
|
180
|
+
|
|
181
|
+
> Detection is static by design. A page that builds HTML/CSS from `fetch`,
|
|
182
|
+
> `readFileSync` or other runtime data should pass `--feature all` (or the
|
|
183
|
+
> specific feature). `--verbose` prints the selected set, e.g.
|
|
184
|
+
> `xbintsc: link features: none`.
|
|
185
|
+
|
|
186
|
+
### Binary size
|
|
187
|
+
|
|
188
|
+
The lean build is what makes a DOM-only program small. On Windows x64 a
|
|
189
|
+
`hello`-sized program drops from roughly 6.4 MiB to about 3.1 MiB once SDL is
|
|
190
|
+
configured as above, function/data sections let `/OPT:REF` discard what the page
|
|
191
|
+
never reaches, the WebGL/images link features stay off, and text uses the
|
|
192
|
+
bundled stb_truetype instead of HarfBuzz + FreeType. The remaining weight is
|
|
193
|
+
mostly the static CRT, the lean SDL (~1 MiB), stb_truetype (~0.1 MiB), and the
|
|
194
|
+
DOM/CSS/layout/paint engine itself (~1.2 MiB). Linux and macOS use their own
|
|
195
|
+
linker sections (`--gc-sections`/`-dead_strip`) and differ somewhat, but the
|
|
196
|
+
relative ranking is the same.
|
|
197
|
+
|
|
198
|
+
WebGL and images/SVG are link features (roughly 0.4 MiB of WebGL context/DOM
|
|
199
|
+
code and 0.23 MiB of stb_image/NanoSVG that a DOM-only page does not link).
|
|
200
|
+
Canvas2D, Web Audio and the `mix-blend-mode` compositor are not gated yet; a
|
|
201
|
+
program that uses none of them still pays for their API surface (roughly
|
|
202
|
+
0.1–0.2 MiB each).
|
|
203
|
+
|
|
204
|
+
The text trade-off is deliberate. The default backend is stb_truetype: only
|
|
205
|
+
TrueType outlines and a per-codepoint mapping plus the font's `kern` table, so
|
|
206
|
+
CFF/OTF and variable fonts, OpenType ligatures and complex-script shaping
|
|
207
|
+
(Arabic/Indic/Thai) are not available. The fallback chain still selects another
|
|
208
|
+
face for codepoints the primary lacks. Selecting the optional **`shaping`**
|
|
209
|
+
feature (automatically for non-ASCII text, or with `--feature shaping`) links
|
|
210
|
+
HarfBuzz + FreeType back in (~1.2 MiB) and restores full OpenType shaping and
|
|
211
|
+
CFF/OTF/variable-font support. A Latin-only DOM page therefore stays near
|
|
212
|
+
3.1 MiB; a Chinese/complex-script page is closer to 4.3 MiB.
|
|
213
|
+
|
|
98
214
|
## Running the example
|
|
99
215
|
|
|
100
216
|
```sh
|
|
@@ -133,10 +249,11 @@ its README for controls and the geometry report it prints.
|
|
|
133
249
|
|
|
134
250
|
`media.ts` (with `media.html`) demonstrates `<video>`/`<audio>`: it overlays a
|
|
135
251
|
CSS progress bar on the video and drives playback with `play()` / `pause()` /
|
|
136
|
-
`currentTime` / `volume`.
|
|
137
|
-
|
|
252
|
+
`currentTime` / `volume`. The compiler sees the elements in `media.html` and
|
|
253
|
+
links the `media` feature automatically; build `media.a` once with
|
|
254
|
+
`npm run media`. Media `src` paths resolve relative to the process working
|
|
138
255
|
directory, so run it from the repository root:
|
|
139
|
-
`xbintsc run examples/gui/media.ts --ext gui
|
|
256
|
+
`xbintsc run examples/gui/media.ts --ext gui`.
|
|
140
257
|
|
|
141
258
|
Set `XT_GUI_AUTOCLOSE_MS=<n>` to close all windows after `n` milliseconds,
|
|
142
259
|
which the e2e test (`tests/e2e/gui-*.test.ts`) uses to run headlessly.
|
package/doc/gui/canvas.md
CHANGED
|
@@ -191,7 +191,7 @@ non-finite entries are ignored, an all-zero list means a solid line).
|
|
|
191
191
|
|
|
192
192
|
Text: `measureText(text)` (returns `{ width }`), `fillText(text, x, y,
|
|
193
193
|
maxWidth?)` and `strokeText(text, x, y, maxWidth?)`. Text is shaped with
|
|
194
|
-
|
|
194
|
+
stb_truetype (shaped and rasterised) (the same stack as HTML text) at the
|
|
195
195
|
transform's approximate scale. A positive `maxWidth` condenses advances and
|
|
196
196
|
glyph bitmaps horizontally so the run is at most that wide (as the canvas API
|
|
197
197
|
specifies) instead of being ignored.
|
package/doc/gui/input.md
CHANGED
|
@@ -13,6 +13,13 @@ follows the artwork, not the box flow left behind.
|
|
|
13
13
|
|
|
14
14
|
- SDL pointer/wheel/key events are routed to the owning window, hit tested, and
|
|
15
15
|
delivered to the handlers registered with `on`. `mousedown` also moves focus.
|
|
16
|
+
- Keyboard events also follow the DOM path: the target is the focused element (or
|
|
17
|
+
`<body>` when nothing is focused), the event bubbles through its ancestors to
|
|
18
|
+
`document`, and finally reaches `window` listeners. This is what lets a page
|
|
19
|
+
written for a browser use `document.addEventListener("keydown", …)`. The
|
|
20
|
+
payload carries browser `KeyboardEvent` spellings — `code` is the physical key
|
|
21
|
+
(`"KeyW"`, `"ArrowUp"`, `"Space"`) and `key` the produced value (`"w"`, `"W"`,
|
|
22
|
+
`" "`) — not SDL's scancode/key names.
|
|
16
23
|
- `:hover` matches the hovered element **and its ancestors** (so hovering a child
|
|
17
24
|
lights up its parents); `:focus` matches the focused element. When either
|
|
18
25
|
changes, `XtDocument::setHover`/`setFocus` recompute styles and layout and mark
|
package/doc/gui/known-issues.md
CHANGED
|
@@ -22,6 +22,16 @@
|
|
|
22
22
|
shader unroller; `WebGLRenderer` runs on the real backend and there is no
|
|
23
23
|
Canvas2D fallback. Shadow maps (comparative `sampler2DShadow` targeting depth
|
|
24
24
|
textures), cube textures/environment maps and HDR render targets are supported.
|
|
25
|
+
- **Comparative samplers can miscompile on Intel Macs.** On an Intel GPU (seen
|
|
26
|
+
on Intel Iris Plus Graphics with macOS 26), the Metal shader compiler drops a
|
|
27
|
+
fragment function that calls `sample_compare`: the draw produces no pixels at
|
|
28
|
+
all, so `texture(sampler2DShadow, …)` reads black and shadow maps disappear.
|
|
29
|
+
The MSL is correct (`depth2d<float>` + `sample_compare`), the sampler carries
|
|
30
|
+
the comparison op, and the same program renders correctly once Metal GPU
|
|
31
|
+
shader validation is enabled — running with `MTL_SHADER_VALIDATION=1` makes
|
|
32
|
+
`tests/e2e/gui-webgl2-shadow-sampler.test.ts` pass on such a machine. That
|
|
33
|
+
points at the driver's shader compiler rather than the engine, so the
|
|
34
|
+
workaround is the environment variable; other Apple GPUs may not show it.
|
|
25
35
|
|
|
26
36
|
|
|
27
37
|
|
|
@@ -111,6 +121,22 @@
|
|
|
111
121
|
bottleneck. Inline `style="…"` attributes are re-parsed on each restyle of
|
|
112
122
|
the element that carries them.
|
|
113
123
|
|
|
124
|
+
- **Text shaping is a simple shaper.** stb_truetype handles Latin/CJK/Cyrillic/
|
|
125
|
+
Greek with basic `kern` and per-codepoint fallback, but it does not run
|
|
126
|
+
OpenType ligatures or complex-script shaping (Arabic/Indic/Thai), and it only
|
|
127
|
+
accepts TrueType outlines (no CFF/OTF or variable fonts). Colour emoji are not
|
|
128
|
+
rendered as colour. This is the price of the ~3 MiB DOM-only binary; the
|
|
129
|
+
optional `shaping` feature restores HarfBuzz + FreeType (and full
|
|
130
|
+
complex-script shaping).
|
|
131
|
+
- **Link-feature detection is static.** `media`/`gpu`/`images`/`shaping`/`webgl` are selected from the
|
|
132
|
+
bundled HTML/CSS/TS (including imported `.html` and the CSS inlined from it).
|
|
133
|
+
A page that builds markup or CSS from runtime data (`fetch`, `readFileSync`,
|
|
134
|
+
a value returned by a function) cannot be seen; the scanner falls back to
|
|
135
|
+
enabling both when it can prove the sink is dynamic, but an opaque value that
|
|
136
|
+
it cannot classify needs an explicit `--feature media`/`gpu`/`images`/`shaping`/`webgl` (or `all`).
|
|
137
|
+
`--verbose` prints the selected set. See
|
|
138
|
+
[Link features](./build.md#link-features).
|
|
139
|
+
|
|
114
140
|
## Open questions
|
|
115
141
|
|
|
116
142
|
- Whether Linux ships X11, Wayland, or both in the first cut. (Decision:
|
|
@@ -130,7 +156,8 @@
|
|
|
130
156
|
- Windows: an MSVC-compatible `gui.lib` (COFF objects + `ar -M`/`llvm-ar`) is
|
|
131
157
|
produced by `scripts/build-gui.ts`. The Windows x64 CI job builds it with the
|
|
132
158
|
Visual Studio toolchain (`.github/actions/vs-toolchain` imports `vcvarsall`
|
|
133
|
-
and uses VS's bundled LLVM), links a `gui
|
|
159
|
+
and uses VS's bundled LLVM), links a `gui` program that exercises the `media`
|
|
160
|
+
link feature, and ships it in the
|
|
134
161
|
x64 release. *Running* the examples on the hosted runner stays provisional
|
|
135
162
|
because it has no GPU/Vulkan driver (SDL_GPU selects the **Vulkan** backend
|
|
136
163
|
there, so no DXIL/`dxc` toolchain is needed). Windows arm64 still skips both
|
|
@@ -139,12 +166,14 @@
|
|
|
139
166
|
- **Windows: x64 builds in CI, arm64 and runtime validation stay provisional.**
|
|
140
167
|
`media.lib` is produced by `scripts/build-media.ts` with FFmpeg's MSVC
|
|
141
168
|
toolchain (`--toolchain=msvc --cc=clang-cl`) so its COFF objects and static CRT
|
|
142
|
-
match `gui.lib`, and
|
|
169
|
+
match `gui.lib`, and the auto-selected `media` feature links and plays there.
|
|
170
|
+
Building it needs a
|
|
143
171
|
POSIX shell to run FFmpeg's `configure` and GNU make; the shell comes from Git
|
|
144
172
|
for Windows (already needed to clone SDL3), and make is a native GNU make
|
|
145
173
|
(Chocolatey's `make`, or `xbintsc_MAKE`). CI installs make explicitly,
|
|
146
174
|
builds `gui.lib` + `media.lib` on Windows x64 (using the same Visual Studio
|
|
147
|
-
`vcvarsall` + bundled LLVM as a local build), links a `gui
|
|
175
|
+
`vcvarsall` + bundled LLVM as a local build), links a `gui` program that
|
|
176
|
+
exercises the feature, and
|
|
148
177
|
ships both archives in the Windows x64 release; the example *run* remains
|
|
149
178
|
provisional because hosted runners have no GPU/Vulkan driver. Windows arm64
|
|
150
179
|
still skips both archives, and `<video controls>` is likewise deferred — the
|
package/doc/gui/layout.md
CHANGED
|
@@ -22,7 +22,7 @@ with absolute (viewport-relative) geometry:
|
|
|
22
22
|
Each line shrinks to the band left free by any floats (`float: left|right`)
|
|
23
23
|
that overlap it, so later lines wrap around them.
|
|
24
24
|
Each text fragment remembers the run it covers, so paint can shape it. Text is
|
|
25
|
-
measured with the
|
|
25
|
+
measured with the stb_truetype stack (see [Text stack](./rendering.md#text-stack-m4b)).
|
|
26
26
|
- **Flexbox** — `row`/`column` (and the `-reverse` variants) with `gap`,
|
|
27
27
|
`flex-basis`/`flex-grow`/`flex-shrink`, `justify-content` and `align-items`
|
|
28
28
|
(including `stretch` when the cross size is definite). `flex-wrap: wrap`
|
package/doc/gui/media.md
CHANGED
|
@@ -74,8 +74,9 @@ clip paths are not rendered (see [Known issues](./known-issues.md)).
|
|
|
74
74
|
|
|
75
75
|
## Video & audio (M13)
|
|
76
76
|
|
|
77
|
-
Media playback is split
|
|
78
|
-
a decoder it never uses
|
|
77
|
+
Media playback is split inside the `gui` extension so a GUI program does not pay
|
|
78
|
+
for a decoder it never uses. Decoding is an optional **`media` link feature**,
|
|
79
|
+
not a separate extension:
|
|
79
80
|
|
|
80
81
|
- **`gui`** (`gui.a`/`gui.lib`) owns the `<video>`/`<audio>` elements, the DOM
|
|
81
82
|
API, the renderer integration and the SDL audio sink, but contains **no
|
|
@@ -85,13 +86,18 @@ a decoder it never uses:
|
|
|
85
86
|
- **`media`** (`media.a`, or `media.lib` with the MSVC ABI on Windows;
|
|
86
87
|
`runtime/ext_media/`) is the optional decoder backend:
|
|
87
88
|
`runtime/ext_media/media.cpp` wraps a statically vendored **FFmpeg** build,
|
|
88
|
-
and `runtime/ext_media/media_bridge.c` registers it with the engine.
|
|
89
|
-
|
|
90
|
-
|
|
89
|
+
and `runtime/ext_media/media_bridge.c` registers it with the engine. The
|
|
90
|
+
compiler selects the feature when the program may use it (a `<video>`/`<audio>`
|
|
91
|
+
element in imported HTML, `createElement("video"/"audio")`, `new Audio()`,
|
|
92
|
+
`decodeAudioData`, or runtime-built markup) and only then compiles the bridge
|
|
93
|
+
and links `media.a`. `--feature media` forces it and `--no-feature media`
|
|
94
|
+
keeps it out; see [Link features](./build.md#link-features).
|
|
91
95
|
|
|
92
|
-
Without the
|
|
96
|
+
Without the feature the element still exists and its DOM API is
|
|
93
97
|
callable, but opening a source fails and fires `error`. The same happens on any
|
|
94
|
-
host where the archive has not been built (run `npm run media`)
|
|
98
|
+
host where the archive has not been built (run `npm run media`) — an
|
|
99
|
+
auto-detected feature with no archive produces a build warning and degrades this
|
|
100
|
+
way, while `--feature media` fails the build. Windows is no
|
|
95
101
|
longer special: `media.lib` is FFmpeg compiled with clang-cl for the MSVC ABI,
|
|
96
102
|
so it links against `gui.lib` and plays normally. There is no WebView and no
|
|
97
103
|
platform media framework in the path either way.
|
package/doc/gui/rendering.md
CHANGED
|
@@ -153,33 +153,33 @@ compiler here — keep painting the group with ordinary source-over blending.
|
|
|
153
153
|
|
|
154
154
|
## Text stack (M4b)
|
|
155
155
|
|
|
156
|
-
`runtime/ext_gui/text.{h,cpp}`
|
|
157
|
-
|
|
158
|
-
`
|
|
156
|
+
`runtime/ext_gui/text.{h,cpp}` is the front-end; it dispatches through a small
|
|
157
|
+
backend table. The default backend is the bundled **stb_truetype**
|
|
158
|
+
(`text_stb.cpp`, `runtime/ext_gui/stb_truetype.h`). The optional **`shaping`**
|
|
159
|
+
link feature installs `text_hb.cpp`, a HarfBuzz + FreeType backend, and is
|
|
160
|
+
selected automatically for non-ASCII text.
|
|
159
161
|
|
|
160
162
|
- Fonts are resolved from well-known system paths (Helvetica/Arial on macOS,
|
|
161
163
|
DejaVu/Liberation on Linux, Segoe UI/Arial on Windows), overridable with
|
|
162
164
|
`XT_GUI_FONT` (and `XT_GUI_FONT_MONO`), and cached per `(family class, size)`.
|
|
163
165
|
Only regular upright faces are used for now; weight/italic selection is a
|
|
164
166
|
later refinement.
|
|
165
|
-
- The default candidates prefer a **CJK-capable** face (Microsoft
|
|
166
|
-
Windows, PingFang/STHeiti on macOS, Noto Sans CJK/WenQuanYi on Linux)
|
|
167
|
-
the Latin-only ones.
|
|
168
|
-
|
|
167
|
+
- The default candidates prefer a **CJK-capable TrueType/TTC** face (Microsoft
|
|
168
|
+
YaHei on Windows, PingFang/STHeiti on macOS, Noto Sans CJK/WenQuanYi on Linux)
|
|
169
|
+
before the Latin-only ones. A candidate that stb_truetype cannot parse (for
|
|
170
|
+
example a CFF/OTF collection) is skipped in favour of the next one.
|
|
169
171
|
- **Codepoints the primary face lacks fall back to another face.** The engine
|
|
170
172
|
keeps a small fallback chain — the platform's symbol face (Segoe UI Symbol,
|
|
171
173
|
Apple Symbols, Noto Sans Symbols, DejaVu Sans) first, then the other family
|
|
172
174
|
class — and resolves each codepoint to the first face in it that has the
|
|
173
175
|
glyph. The run is shaped in segments, one per face, so a mixed run (Latin
|
|
174
176
|
plus an arrow, box-drawing or geometric shape, or Latin inside CJK) draws
|
|
175
|
-
every glyph instead of dropping the missing ones as `.notdef`.
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
- `xt_text_measure_width` shapes the run with HarfBuzz (so kerning and
|
|
182
|
-
ligatures are honoured), `xt_text_metrics` returns FreeType's ascent /
|
|
177
|
+
every glyph instead of dropping the missing ones as `.notdef`. Each shaped
|
|
178
|
+
glyph carries its face (`ShapedGlyph::font`) so the renderer rasterises it
|
|
179
|
+
from the right atlas entry; `xt_text_scale` maps a face to the display's scale
|
|
180
|
+
factor.
|
|
181
|
+
- `xt_text_measure_width` maps each codepoint to a glyph and adds its advance
|
|
182
|
+
plus the font's `kern` table; `xt_text_metrics` returns stb_truetype's ascent /
|
|
183
183
|
descent / normal line height. When no font file can be found the module falls
|
|
184
184
|
back to a deterministic per-byte approximation, so layout still works.
|
|
185
185
|
- `xt_text_shape_run` returns positioned glyphs and `xt_text_rasterize` renders
|
|
@@ -189,6 +189,15 @@ compiler here — keep painting the group with ordinary source-over blending.
|
|
|
189
189
|
- Layout uses these real metrics for text widths, line breaking and
|
|
190
190
|
`line-height: normal`; `measureText`/`fontMetrics` expose them to tests.
|
|
191
191
|
|
|
192
|
+
Shape trade-off: the default is a simple shaper, not a full OpenType engine. It
|
|
193
|
+
handles Latin/CJK/Cyrillic/Greek, basic kerning and per-codepoint fallback, but
|
|
194
|
+
it does not run ligatures or complex-script shaping (Arabic/Indic/Thai), and
|
|
195
|
+
stb_truetype only accepts TrueType outlines, not CFF/OTF or variable fonts. The
|
|
196
|
+
choice keeps a DOM-only binary near 3 MiB. The optional **`shaping`** link
|
|
197
|
+
feature installs the HarfBuzz + FreeType backend instead (about 1.2 MiB) and
|
|
198
|
+
restores full OpenType shaping and CFF/OTF/variable-font support; it is selected
|
|
199
|
+
automatically for non-ASCII text and can be forced with `--feature shaping`.
|
|
200
|
+
|
|
192
201
|
## Performance (M20)
|
|
193
202
|
|
|
194
203
|
The engine is a retained-mode pipeline: a DOM mutation restyles the document,
|
|
@@ -211,6 +220,14 @@ is what `tests/e2e/gui-perf.test.ts` asserts on.
|
|
|
211
220
|
and becomes current, so the hot path has no LRU bookkeeping. A word is shaped
|
|
212
221
|
once and reused by every later layout pass and repaint that draws it. Primary
|
|
213
222
|
face lookups are cached by (family class, size) as well.
|
|
223
|
+
- **No-op writes.** `textContent` and `style.<property>` do nothing when the
|
|
224
|
+
node already shows that value: the document is not dirtied, so no restyle,
|
|
225
|
+
relayout, repaint or upload follows. A HUD that assigns the same clock string,
|
|
226
|
+
the same bar width or the same transform every frame therefore costs nothing —
|
|
227
|
+
before this each assignment rebuilt a text node (an allocation, and one the
|
|
228
|
+
collector then had to sweep) or re-serialized the inline style, and dirtied
|
|
229
|
+
the document for a full restyle. `classList` and `removeAttribute` already
|
|
230
|
+
behaved this way; `setAttribute` still dirties unconditionally.
|
|
214
231
|
- **Frame loop.** The media/canvas tables are reconciled with the DOM only when
|
|
215
232
|
the document changed (or on the first frame) instead of walking the tree every
|
|
216
233
|
frame. A frame presents only when something happened — a mutation, a scroll,
|
package/doc/gui/runtime.md
CHANGED
|
@@ -55,6 +55,11 @@ that case with a single 16 ms `xt_set_timeout` tick:
|
|
|
55
55
|
queued animation-frame callbacks before layout, so a page can stop itself),
|
|
56
56
|
- it re-arms **only while callbacks remain**, so an idle page lets the event
|
|
57
57
|
loop drain and the process exit — the browser-tab lifecycle;
|
|
58
|
+
- it schedules the next tick for *frame start + 16 ms*, not 16 ms after the
|
|
59
|
+
frame finished: the frame's own duration is not added to the interval, so a
|
|
60
|
+
10 ms frame runs at a 16 ms period instead of 26 ms, and a page slower than
|
|
61
|
+
16 ms runs as fast as it can instead of falling a frame's worth of time
|
|
62
|
+
further behind on every frame;
|
|
58
63
|
- it stands aside while `xt_gui_run_active()` is set (the program called
|
|
59
64
|
`run()`), so the same window is not rendered twice per tick;
|
|
60
65
|
- `XT_GUI_AUTOCLOSE_MS=n` makes the pump quit after `n` milliseconds, which is
|