xbintsc 0.5.22 → 0.5.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/README.md +15 -5
- package/README.zh-CN.md +11 -4
- package/dist/src/cli/hints.js +11 -0
- package/dist/src/cli/hints.js.map +1 -1
- package/dist/src/cli/main.js +1 -1
- package/dist/src/driver/compiler.js +1 -0
- package/dist/src/driver/compiler.js.map +1 -1
- package/dist/src/driver/version.d.ts +1 -1
- package/dist/src/driver/version.js +1 -1
- package/dist/src/extensions/catalog.js +2 -1
- package/dist/src/extensions/catalog.js.map +1 -1
- package/dist/src/extensions/media/index.d.ts +27 -0
- package/dist/src/extensions/media/index.js +60 -0
- package/dist/src/extensions/media/index.js.map +1 -0
- package/dist/src/extensions/registry.d.ts +12 -0
- package/dist/src/extensions/registry.js +16 -0
- package/dist/src/extensions/registry.js.map +1 -1
- package/dist/src/index.d.ts +1 -0
- package/dist/src/index.js +1 -0
- package/dist/src/index.js.map +1 -1
- package/dist/tests/cli/hints.test.js +15 -0
- package/dist/tests/cli/hints.test.js.map +1 -1
- package/dist/tests/cli/process.test.js +12 -0
- package/dist/tests/cli/process.test.js.map +1 -1
- package/dist/tests/e2e/gui-example-canvas.test.js +3 -1
- package/dist/tests/e2e/gui-example-canvas.test.js.map +1 -1
- package/dist/tests/e2e/gui-helpers.d.ts +3 -0
- package/dist/tests/e2e/gui-helpers.js +2 -0
- package/dist/tests/e2e/gui-helpers.js.map +1 -1
- package/dist/tests/e2e/gui-media-disabled.test.d.ts +9 -0
- package/dist/tests/e2e/gui-media-disabled.test.js +42 -0
- package/dist/tests/e2e/gui-media-disabled.test.js.map +1 -0
- package/dist/tests/e2e/gui-media.test.d.ts +6 -4
- package/dist/tests/e2e/gui-media.test.js +15 -12
- package/dist/tests/e2e/gui-media.test.js.map +1 -1
- package/dist/tests/e2e/unicode-output.test.d.ts +16 -0
- package/dist/tests/e2e/unicode-output.test.js +65 -0
- package/dist/tests/e2e/unicode-output.test.js.map +1 -0
- package/dist/tests/extensions/media.test.d.ts +1 -0
- package/dist/tests/extensions/media.test.js +58 -0
- package/dist/tests/extensions/media.test.js.map +1 -0
- package/dist/tests/extensions/registry.test.js +8 -0
- package/dist/tests/extensions/registry.test.js.map +1 -1
- package/dist/tests/lexer/identifiers.test.js +12 -0
- package/dist/tests/lexer/identifiers.test.js.map +1 -1
- package/dist/tests/lexer/scanner.test.js +8 -0
- package/dist/tests/lexer/scanner.test.js.map +1 -1
- package/dist/tests/lexer/strings.test.js +30 -0
- package/dist/tests/lexer/strings.test.js.map +1 -1
- package/doc/ai/cli.md +1 -1
- package/doc/ai/extensions.md +35 -23
- package/doc/ai/troubleshooting.md +9 -0
- package/doc/ai/zh-CN/cli.md +1 -1
- package/doc/ai/zh-CN/extensions.md +20 -8
- package/doc/ai/zh-CN/troubleshooting.md +8 -0
- package/doc/gui/build.md +37 -21
- package/doc/gui/known-issues.md +7 -6
- package/doc/gui/media.md +26 -11
- package/doc/gui.md +43 -33
- package/doc/zh-CN/gui/build.md +28 -16
- package/doc/zh-CN/gui/known-issues.md +6 -5
- package/doc/zh-CN/gui/media.md +21 -10
- package/doc/zh-CN/gui.md +37 -32
- package/package.json +2 -1
- package/runtime/ext_gui/gui_media.cpp +69 -2
- package/runtime/ext_gui/media.cpp +34 -488
- package/runtime/ext_gui/media.h +54 -19
- package/runtime/ext_gui/media_api.h +65 -0
- package/runtime/ext_media/media.cpp +507 -0
- package/runtime/ext_media/media_bridge.c +35 -0
- package/scripts/build-gui.ts +6 -72
- package/scripts/build-media.ts +245 -0
- package/src/cli/hints.ts +11 -0
- package/src/cli/main.ts +1 -1
- package/src/driver/compiler.ts +1 -0
- package/src/driver/version.ts +1 -1
- package/src/extensions/catalog.ts +2 -1
- package/src/extensions/media/index.ts +67 -0
- package/src/extensions/registry.ts +24 -0
- package/src/index.ts +1 -0
|
@@ -118,6 +118,15 @@ The `gui` extension needs its per-platform prebuilt `gui.a`/`gui.lib`, which is
|
|
|
118
118
|
not built in a plain source checkout. Drop `--ext gui`, or build the archive as
|
|
119
119
|
described in [../gui.md](../gui.md).
|
|
120
120
|
|
|
121
|
+
### `media native library not found at …`
|
|
122
|
+
|
|
123
|
+
`<video>`/`<audio>` playback needs the optional `media` archive (`media.a`),
|
|
124
|
+
built separately with `npm run media`. Build it, or drop `--ext media`; a
|
|
125
|
+
GUI-only build still links and media elements simply fire `error`. The `media`
|
|
126
|
+
extension also requires `gui`, has no Windows archive yet, and `--ext media`
|
|
127
|
+
alone is rejected with the flags to pass. See
|
|
128
|
+
[../gui/build.md](../gui/build.md).
|
|
129
|
+
|
|
121
130
|
### `Icon file not found` / icon tools missing
|
|
122
131
|
|
|
123
132
|
`--icon`/`app.icon` paths resolve against the config file's directory. On
|
package/doc/ai/zh-CN/cli.md
CHANGED
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
--out <dir> 输出目录(默认 build/)
|
|
33
33
|
--emit <kind> exe | obj | ir(默认 exe)
|
|
34
34
|
-O0 .. -O3 传给 clang 的优化级别(默认 -O2)
|
|
35
|
-
--ext <names> 开启内置扩展,逗号分隔(如 node,gui)
|
|
35
|
+
--ext <names> 开启内置扩展,逗号分隔(如 node,gui,media)
|
|
36
36
|
--ext-native <m> 通过 JSON 清单注册 C++/Rust 扩展(多个用逗号分隔)
|
|
37
37
|
--config <path> 使用指定项目配置,不做自动发现
|
|
38
38
|
--no-config 完全不读取项目配置
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# 扩展:Node 模块、GUI
|
|
1
|
+
# 扩展:Node 模块、GUI、媒体、C++/Rust 库
|
|
2
2
|
|
|
3
3
|
编译器核心与平台无关。任何与平台相关的东西——Node 的 `fs`、一个 HTML/CSS 渲染器、
|
|
4
4
|
你自己的 C++ 库——都以**扩展**的形式接入:它既贡献模块绑定,也贡献要一起编译/链接的
|
|
@@ -66,11 +66,9 @@ import { createWindow, run } from "gui";
|
|
|
66
66
|
```
|
|
67
67
|
|
|
68
68
|
它是自带渲染引擎的 GPU 加速实现(自己的 HTML 解析器、CSS 级联、布局与合成器),不是
|
|
69
|
-
系统 WebView。`<img
|
|
70
|
-
|
|
71
|
-
`
|
|
72
|
-
没有内置控件——用元素和 CSS 自绘控件(见 `examples/gui/media.ts`);视频帧与音频 PCM
|
|
73
|
-
都在主线程解码。也支持 `<canvas>`:`getContext("2d")` 返回的上下文,其填充、路径、
|
|
69
|
+
系统 WebView。`<img>` 由引擎自身解码;而 `<video>`/`<audio>` 在只启用 `gui` 时**不可用**
|
|
70
|
+
——它们需要单独的 [`media`](#media--视频音频解码) 扩展。也支持 `<canvas>`:
|
|
71
|
+
`getContext("2d")` 返回的上下文,其填充、路径、
|
|
74
72
|
渐变、文本与 `drawImage` 由自研 CPU 光栅器处理(另有线段端点/连接、虚线描边、
|
|
75
73
|
`ellipse` 与 `roundRect`),元素句柄还暴露 `width`/`height`。CSS
|
|
76
74
|
`transform`/`transform-origin` 作为绘制期效果可用,因此一个盒子(以及其中的文本、
|
|
@@ -78,10 +76,24 @@ import { createWindow, run } from "gui";
|
|
|
78
76
|
`requestAnimationFrame` 逐帧驱动,或写成 `transition: transform`(见
|
|
79
77
|
`examples/gui/pelican-svg`)。因为主体是 C++,它以各平台预编译的 `gui.a`/`gui.lib` 形式分发。如果该
|
|
80
78
|
平台的归档不存在,构建会以明确的提示失败;设计与路线图见 [../../zh-CN/gui.md](../../zh-CN/gui.md)(总览;引擎按功能拆分在 [`doc/zh-CN/gui/`](../../zh-CN/gui/) 下),
|
|
81
|
-
脚本接口见 [../../zh-CN/gui-scripts.md](../../zh-CN/gui-scripts.md)
|
|
82
|
-
Windows 唯一不支持的:Windows 的 `gui.a` 不构建 FFmpeg,媒体槽位为空操作。Canvas API
|
|
79
|
+
脚本接口见 [../../zh-CN/gui-scripts.md](../../zh-CN/gui-scripts.md)。Canvas API
|
|
83
80
|
及其限制见 [../../zh-CN/gui/canvas.md](../../zh-CN/gui/canvas.md)。
|
|
84
81
|
|
|
82
|
+
### `media` —— 视频/音频解码
|
|
83
|
+
|
|
84
|
+
用 `--ext media` **与 `gui` 一起**开启(`--ext gui,media`;注册表会拒绝单独启用
|
|
85
|
+
`media`)。它链接单独的 FFmpeg 后端 `media.a`,因此加入 `<video>`/`<audio>` 播放
|
|
86
|
+
不会让只做界面的二进制变大。
|
|
87
|
+
|
|
88
|
+
DOM API 与普通 HTML 媒体元素一致:`play()` / `pause()` / `load()`,以及
|
|
89
|
+
`paused` / `ended` / `duration` / `currentTime` / `volume` / `muted` / `loop` /
|
|
90
|
+
`autoplay` / `videoWidth` / `videoHeight` / `readyState` / `src` 属性,还有
|
|
91
|
+
`loadedmetadata`/`play`/`pause`/`ended`/`timeupdate`/`error` 事件。没有内置
|
|
92
|
+
控件——用元素和 CSS 自绘控件(见 `examples/gui/media.ts`)。视频帧与音频 PCM
|
|
93
|
+
都在主线程解码;视频走图像纹理路径。没有该扩展时,打开媒体源会失败并触发
|
|
94
|
+
`error`(元素与 API 仍然存在)。该归档目前仅支持 POSIX,因此 Windows 也只会触发
|
|
95
|
+
同样的 `error`;见 [../../zh-CN/gui/known-issues.md](../../zh-CN/gui/known-issues.md)。
|
|
96
|
+
|
|
85
97
|
## 第三方 npm 包
|
|
86
98
|
|
|
87
99
|
没有被已启用扩展认领的裸说明符,会在 `node_modules` 里查找并**作为源码**打包,
|
|
@@ -108,6 +108,14 @@ clang 自身失败。抛出的消息包含完整的 clang 命令与 stderr——
|
|
|
108
108
|
`gui` 扩展需要各平台预编译的 `gui.a`/`gui.lib`,普通源码检出并没有构建它。去掉
|
|
109
109
|
`--ext gui`,或按 [../../zh-CN/gui.md](../../zh-CN/gui.md) 构建该归档。
|
|
110
110
|
|
|
111
|
+
### `media native library not found at …`
|
|
112
|
+
|
|
113
|
+
`<video>`/`<audio>` 播放需要可选的 `media` 归档(`media.a`),用
|
|
114
|
+
`npm run media` 单独构建。构建它,或去掉 `--ext media`;仅启用 `gui` 的构建仍可
|
|
115
|
+
链接,媒体元素只会触发 `error`。`media` 扩展还依赖 `gui`,目前没有 Windows
|
|
116
|
+
归档,单独 `--ext media` 会被拒绝并提示应传的 flag。见
|
|
117
|
+
[../../zh-CN/gui/build.md](../../zh-CN/gui/build.md)。
|
|
118
|
+
|
|
111
119
|
### `Icon file not found` / 图标工具缺失
|
|
112
120
|
|
|
113
121
|
`--icon`/`app.icon` 的路径相对配置文件目录解析。Windows 上嵌入 PE 图标需要
|
package/doc/gui/build.md
CHANGED
|
@@ -4,21 +4,27 @@
|
|
|
4
4
|
> decisions and roadmap live there.
|
|
5
5
|
|
|
6
6
|
- Sources live in `runtime/ext_gui/` (C/C++) plus vendored libs under `vendor/`
|
|
7
|
-
(gitignored; fetched on demand).
|
|
7
|
+
(gitignored; fetched on demand). Media decoding lives in
|
|
8
|
+
`runtime/ext_media/` and is built separately.
|
|
8
9
|
- `npm run gui` (`scripts/build-gui.ts`) fetches a pinned SDL3 (`SDL3_TAG`,
|
|
9
10
|
default `release-3.2.10`), builds a static SDL3, fetches and builds static
|
|
10
11
|
FreeType (`FREETYPE_VERSION`) and HarfBuzz (`HARFBUZZ_VERSION`), compiles the
|
|
11
12
|
engine and merges everything into `runtime/lib/<os>-<arch>/gui.a` (or
|
|
12
13
|
`gui.lib` with the MSVC ABI on Windows), same convention as `core.a`,
|
|
13
|
-
statically bundling all three.
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
14
|
+
statically bundling all three. The merge uses `libtool` on macOS and
|
|
15
|
+
`ar -M` (GNU/LLVM) elsewhere; CMake archives are discovered under the build
|
|
16
|
+
root or `Release/` for either generator style.
|
|
17
|
+
- `npm run media` (`scripts/build-media.ts`) builds the optional **FFmpeg**
|
|
18
|
+
backend. It fetches FFmpeg (`FFMPEG_VERSION`, default `7.1`), configures a
|
|
19
|
+
minimal static build with only the demuxers/parsers/decoders the media
|
|
20
|
+
elements need, compiles `runtime/ext_media/media.cpp` against it and merges
|
|
21
|
+
everything into `runtime/lib/<os>-<arch>/media.a`. `gui.a` does **not** link
|
|
22
|
+
FFmpeg: a program enables playback explicitly with `--ext media`.
|
|
23
|
+
- `src/extensions/gui/index.ts` exposes the engine archive through
|
|
24
|
+
`nativeObjects()` (via `findRuntimeLibrary`, so `.a`/`.lib` both work) and the
|
|
25
|
+
OS frameworks through `linkerFlags()`. `src/extensions/media/index.ts` does
|
|
26
|
+
the same for `media.a` and compiles the registration bridge
|
|
27
|
+
(`runtime/ext_media/media_bridge.c`) as a normal runtime source.
|
|
22
28
|
- `scripts/build-gui.ts` compiles with the *same* clang the driver resolves
|
|
23
29
|
(`src/driver/toolchain-provider.ts`); on Windows that is the MSVC-targeting
|
|
24
30
|
LLVM bundled with Visual Studio, and its sibling `clang++`/`llvm-ar` are used
|
|
@@ -26,14 +32,17 @@
|
|
|
26
32
|
fallbacks, and Windows always uses `llvm-ar`, so the archive cannot mix object
|
|
27
33
|
formats or C++ runtimes. The Visual Studio toolchain is found by absolute
|
|
28
34
|
path, so no developer prompt is required. On Windows SDL_GPU selects the
|
|
29
|
-
Vulkan backend; no DXIL/`dxc` toolchain is needed.
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
35
|
+
Vulkan backend; no DXIL/`dxc` toolchain is needed. `scripts/build-media.ts`
|
|
36
|
+
uses the same resolved toolchain for FFmpeg and the player.
|
|
37
|
+
- CI builds `gui.a` and `media.a` before assembling the release archive so they
|
|
38
|
+
ship inside `runtime/lib/<slug>/` (the release tarball copies the whole
|
|
39
|
+
`runtime/` tree). The archives are built and the examples are run on Linux
|
|
40
|
+
(under Xvfb, with the lavapipe software Vulkan driver) and macOS; Windows is
|
|
41
|
+
provisional because the hosted runners have no GPU/Vulkan driver, and the
|
|
42
|
+
media archive is POSIX-only.
|
|
43
|
+
- The existing cache fingerprint already hashes `nativeObjects()` contents and
|
|
44
|
+
`runtimeSources()` content, so rebuilding either archive invalidates cached
|
|
45
|
+
binaries automatically.
|
|
37
46
|
- CMake build trees live *next to* their sources (`vendor/sdl-build`,
|
|
38
47
|
`vendor/freetype-build`, `vendor/harfbuzz-build`), never inside them: a
|
|
39
48
|
`CMakeCache.txt` records the machine it was configured on, and SDL's video
|
|
@@ -52,6 +61,7 @@
|
|
|
52
61
|
```sh
|
|
53
62
|
npm run runtime # rebuild core.a after a runtime change
|
|
54
63
|
npm run gui # build runtime/lib/<os>-<arch>/gui.a (fetches SDL3 once)
|
|
64
|
+
npm run media # optional: build media.a (fetches and builds FFmpeg once)
|
|
55
65
|
xbintsc run examples/gui/hello.ts --ext gui
|
|
56
66
|
```
|
|
57
67
|
|
|
@@ -62,9 +72,10 @@ its README for controls and the geometry report it prints.
|
|
|
62
72
|
|
|
63
73
|
`media.ts` (with `media.html`) demonstrates `<video>`/`<audio>`: it overlays a
|
|
64
74
|
CSS progress bar on the video and drives playback with `play()` / `pause()` /
|
|
65
|
-
`currentTime` / `volume`.
|
|
66
|
-
|
|
67
|
-
|
|
75
|
+
`currentTime` / `volume`. It needs the separate `media` extension, so build and
|
|
76
|
+
enable both. Media `src` paths resolve relative to the process working
|
|
77
|
+
directory, so run it from the repository root:
|
|
78
|
+
`xbintsc run examples/gui/media.ts --ext gui,media`.
|
|
68
79
|
|
|
69
80
|
Set `XT_GUI_AUTOCLOSE_MS=<n>` to close all windows after `n` milliseconds,
|
|
70
81
|
which the e2e test (`tests/e2e/gui-*.test.ts`) uses to run headlessly.
|
|
@@ -84,10 +95,15 @@ software Vulkan driver:
|
|
|
84
95
|
```sh
|
|
85
96
|
sudo sh scripts/install-gui-deps.sh # clang, cmake, X11/Wayland headers, Vulkan, Xvfb
|
|
86
97
|
npm run runtime && npm run gui
|
|
98
|
+
npm run media # optional: <video>/<audio> playback
|
|
87
99
|
xvfb-run -a --server-args="-screen 0 1280x720x24" \
|
|
88
100
|
npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
|
|
89
101
|
```
|
|
90
102
|
|
|
103
|
+
The media e2e suite (`tests/e2e/gui-media.test.ts`) skips itself unless both
|
|
104
|
+
`gui.a` and `media.a` exist; `tests/e2e/gui-media-disabled.test.ts` covers the
|
|
105
|
+
GUI-only build and runs with `gui.a` alone.
|
|
106
|
+
|
|
91
107
|
`scripts/install-gui-deps.sh` holds the package list (`apt-get install -y clang
|
|
92
108
|
cmake libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev
|
|
93
109
|
libxinerama-dev libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols
|
package/doc/gui/known-issues.md
CHANGED
|
@@ -69,12 +69,13 @@
|
|
|
69
69
|
(`win.driver()` reports `"vulkan"`), so the archive needs no DXIL/`dxc`
|
|
70
70
|
toolchain; the hosted Windows runners have no GPU/Vulkan driver, which is why
|
|
71
71
|
the CI run stays provisional.
|
|
72
|
-
- **Media on Windows:** the media
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
pattern is TS + a CSS
|
|
72
|
+
- **Media on Windows:** the `media` extension is POSIX-only for now. The GUI
|
|
73
|
+
engine itself builds on Windows, but without the extension `<video>`/`<audio>`
|
|
74
|
+
are inert there (an open fails and fires `error`), and `--ext media` reports an
|
|
75
|
+
actionable error instead of linking. Enabling it needs an MSVC-compatible
|
|
76
|
+
FFmpeg build and is deferred until the Windows GUI path itself is validated.
|
|
77
|
+
`<video controls>` is likewise deferred — the intended pattern is TS + a CSS
|
|
78
|
+
overlay, as in `examples/gui/media.ts`.
|
|
78
79
|
- **D3D12 shaders:** the SPIR-V (Vulkan) and DXIL (Direct3D 12) blobs are built
|
|
79
80
|
and embedded, so the non-Metal path no longer skips geometry. DXIL remains
|
|
80
81
|
unusable on **SDL 3.2.10**: its D3D12 backend rejects any graphics pipeline
|
package/doc/gui/media.md
CHANGED
|
@@ -65,17 +65,31 @@ clip paths are not rendered (see [Known issues](./known-issues.md)).
|
|
|
65
65
|
|
|
66
66
|
## Video & audio (M13)
|
|
67
67
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
68
|
+
Media playback is split across two extensions so a GUI program does not pay for
|
|
69
|
+
a decoder it never uses:
|
|
70
|
+
|
|
71
|
+
- **`gui`** (`gui.a`) owns the `<video>`/`<audio>` elements, the DOM API, the
|
|
72
|
+
renderer integration and the SDL audio sink, but contains **no FFmpeg**.
|
|
73
|
+
`runtime/ext_gui/media.{h,cpp}` declares the player interface and dispatches
|
|
74
|
+
through a table the media backend installs at startup
|
|
75
|
+
(`runtime/ext_gui/media_api.h`).
|
|
76
|
+
- **`media`** (`media.a`, `runtime/ext_media/`) is the optional decoder
|
|
77
|
+
backend: `runtime/ext_media/media.cpp` wraps a statically vendored **FFmpeg**
|
|
78
|
+
build, and `runtime/ext_media/media_bridge.c` registers it with the engine.
|
|
79
|
+
Enable it with `--ext media` (or `--ext gui,media`). `media` requires `gui`;
|
|
80
|
+
the registry rejects the combination in the wrong order.
|
|
81
|
+
|
|
82
|
+
Without the `media` extension the element still exists and its DOM API is
|
|
83
|
+
callable, but opening a source fails and fires `error`. That is also how Windows
|
|
84
|
+
behaves today: the archive is POSIX-only. There is no WebView and no platform
|
|
85
|
+
media framework in the path either way.
|
|
72
86
|
|
|
73
87
|
- **Decoding.** `avformat_open_input` opens the `src` (a path — `file://` is
|
|
74
88
|
accepted, but media paths are *not* percent-decoded, unlike images), the
|
|
75
89
|
best stream is selected for each kind, and a software decoder yields
|
|
76
90
|
`yuv/rgb` frames which `sws_scale` converts to RGBA8 at the coded size. The
|
|
77
91
|
supported containers/codecs are the minimal FFmpeg configuration from
|
|
78
|
-
`scripts/build-
|
|
92
|
+
`scripts/build-media.ts`: MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/ASF, MP3/OGG/WAV/FLAC/
|
|
79
93
|
AAC; H.264, HEVC, VP8, VP9, AV1, MPEG-4, MJPEG video; AAC, MP3, Opus, Vorbis,
|
|
80
94
|
FLAC, ALAC, AC3/E-AC3, WMAv2 and the raw PCM formats audio.
|
|
81
95
|
- **Video.** `<video>` is a replaced element like `<img>` (UA default
|
|
@@ -83,13 +97,14 @@ framework in the path.
|
|
|
83
97
|
same `PaintImage` as an image; the renderer uploads each new decoded frame to
|
|
84
98
|
an RGBA texture (`ensureMediaTexture`) and draws a textured quad, so the GPU
|
|
85
99
|
scales it to the CSS box instead of the CPU resizing. Playback timing is the
|
|
86
|
-
|
|
100
|
+
process monotonic clock, not the frame delta.
|
|
87
101
|
- **Audio.** There is no audio callback: the decoder is pushed on the main
|
|
88
|
-
thread as it is ticked and
|
|
89
|
-
|
|
90
|
-
0.25 s ahead at 48 kHz stereo
|
|
91
|
-
`SDL_SetAudioStreamGain`; audio is
|
|
92
|
-
is available (headless CI).
|
|
102
|
+
thread as it is ticked and writes into an `xtgui::AudioSink`. The GUI
|
|
103
|
+
implements that sink in `gui_media.cpp` with `SDL_OpenAudioDeviceStream` in
|
|
104
|
+
push mode (`NULL` callback), buffering about 0.25 s ahead at 48 kHz stereo
|
|
105
|
+
`SDL_AUDIO_F32`. `volume` maps to `SDL_SetAudioStreamGain`; audio is
|
|
106
|
+
best-effort and is skipped when no device is available (headless CI). Keeping
|
|
107
|
+
the sink in `gui.a` means `media.a` depends only on FFmpeg.
|
|
93
108
|
- **Threading.** The engine is single-threaded and not thread-aware, so all
|
|
94
109
|
decoding happens on the main thread inside `xt_gui_media_tick` — no decode
|
|
95
110
|
thread, no cross-thread GC exposure.
|
package/doc/gui.md
CHANGED
|
@@ -21,9 +21,11 @@ what `examples/gui/pelican-bike` — a playable 2D game — is built on. **M12**
|
|
|
21
21
|
DOM node lifetime explicit (what a mutation releases, what it keeps) and gives the
|
|
22
22
|
engine bounded memory: see [Node lifetime](./gui/dom.md#node-lifetime) and
|
|
23
23
|
[Memory & GC](./gui/runtime.md#memory--gc). **M13** plays real
|
|
24
|
-
`<video>`/`<audio>` —
|
|
25
|
-
video frames go through the image texture
|
|
26
|
-
the DOM media API exposing
|
|
24
|
+
`<video>`/`<audio>` — the optional `media` extension decodes them on the main
|
|
25
|
+
thread with a vendored minimal FFmpeg, video frames go through the image texture
|
|
26
|
+
path and audio to an SDL stream, with the DOM media API exposing
|
|
27
|
+
`play`/`pause`/`currentTime`/… and media events. Enabling `gui` alone does not
|
|
28
|
+
link FFmpeg.
|
|
27
29
|
**M14** adds `<canvas>` with a self-written 2D software rasteriser (anti-aliased
|
|
28
30
|
fills, paths, text and `drawImage`) uploaded through the image texture path —
|
|
29
31
|
see [Canvas](./gui/canvas.md). **M15** adds CSS `transform` /
|
|
@@ -84,7 +86,7 @@ pages above.
|
|
|
84
86
|
| 4 | The engine is **not self-hosted** (it is C/C++, not TS), but must not affect the compiler's platform-agnostic design. Delivered as a per-platform prebuilt archive linked via `nativeObjects`. |
|
|
85
87
|
| 5 | The **event loop is generic**: the runtime exposes a generic main-loop hook and a poll primitive; nothing GUI-specific enters `runtime/`. |
|
|
86
88
|
| 6 | **Multiple windows** are supported by the core object model. |
|
|
87
|
-
| 7 | Third-party low-level libs are **statically linked into
|
|
89
|
+
| 7 | Third-party low-level libs are **statically linked into the extension archives** so releases stay self-contained; only OS frameworks are added at link time. `gui.a` carries SDL3/FreeType/HarfBuzz, and the optional `media.a` carries FFmpeg and is linked only with `--ext media`. |
|
|
88
90
|
|
|
89
91
|
## Architecture
|
|
90
92
|
|
|
@@ -106,6 +108,7 @@ pages above.
|
|
|
106
108
|
windowing + input (SDL3) ── multiple windows
|
|
107
109
|
text (HarfBuzz + FreeType) ── glyph atlas / SDF
|
|
108
110
|
images (stb_image; SVG via NanoSVG)
|
|
111
|
+
media DOM + audio sink ── optional media.a (FFmpeg) installs the decoder
|
|
109
112
|
```
|
|
110
113
|
|
|
111
114
|
The engine never talks to the compiler. The compiler only sees an `Extension`
|
|
@@ -120,18 +123,18 @@ with `nativeObjects()`, `linkerFlags()` and `modules()`.
|
|
|
120
123
|
| Text shaping | **HarfBuzz** | correct complex-script shaping |
|
|
121
124
|
| Glyph raster | **FreeType** | glyph outlines → GPU atlas / SDF |
|
|
122
125
|
| Images | **stb_image** (vendored header) for bitmaps, **NanoSVG** (vendored headers) for SVG | single headers to start; SVG files rasterise on demand |
|
|
123
|
-
| Video + audio | **FFmpeg** (statically vendored, minimal build) | decodes the common container/codec set; no WebView or platform media framework |
|
|
126
|
+
| Video + audio | **FFmpeg** (statically vendored, minimal build, optional `media` extension) | decodes the common container/codec set; no WebView or platform media framework |
|
|
124
127
|
|
|
125
128
|
Pinned versions: SDL3 `release-3.2.10`, FreeType `2.13.3`, HarfBuzz `10.1.0`,
|
|
126
|
-
FFmpeg `7.1` (overridable with `SDL3_TAG` /
|
|
127
|
-
`HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
|
|
129
|
+
FFmpeg `7.1` for the optional `media` archive (overridable with `SDL3_TAG` /
|
|
130
|
+
`FREETYPE_VERSION` / `HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
|
|
128
131
|
|
|
129
132
|
Alternative if SDL3 is rejected: **GLFW + OpenGL 3.3** (simpler, but OpenGL is
|
|
130
133
|
deprecated on macOS and gives no modern GPU abstraction). **wgpu-native** if the
|
|
131
134
|
engine were Rust.
|
|
132
135
|
|
|
133
|
-
> These vendored libraries are linked **statically into `gui.a
|
|
134
|
-
> xbintsc stays "download and run"; only OS frameworks are added by
|
|
136
|
+
> These vendored libraries are linked **statically into `gui.a` / `media.a`** so
|
|
137
|
+
> a released xbintsc stays "download and run"; only OS frameworks are added by
|
|
135
138
|
> `linkerFlags()`.
|
|
136
139
|
|
|
137
140
|
## Milestones
|
|
@@ -172,15 +175,16 @@ engine were Rust.
|
|
|
172
175
|
animated `background-color`/`color`/`border-color`/`border-radius`, retargeting
|
|
173
176
|
and `win.advance(ms)`. `@keyframes` remain.
|
|
174
177
|
7. **M7 — CI & releases** ✅ (Linux/macOS build) / 🚧 (run + Windows)
|
|
175
|
-
- `compile-examples` builds `gui.a` on Linux and macOS
|
|
176
|
-
the
|
|
177
|
-
(required). The macOS run is provisional until a
|
|
178
|
-
it is guarded so a failure is logged without
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
178
|
+
- `compile-examples` builds `gui.a` and `media.a` on Linux and macOS
|
|
179
|
+
(required), then runs the examples and `tests/e2e/gui-*.test.ts` under
|
|
180
|
+
Xvfb + lavapipe on Linux (required). The macOS run is provisional until a
|
|
181
|
+
WindowServer is confirmed; it is guarded so a failure is logged without
|
|
182
|
+
annotating the run.
|
|
183
|
+
- The `package` job builds `gui.a` and `media.a` before assembling the
|
|
184
|
+
release, so both ship inside the existing runtime archive
|
|
185
|
+
(`package-release` copies all of `runtime/`).
|
|
186
|
+
- `vendor/` (SDL3/FreeType/HarfBuzz for `gui.a`, FFmpeg for `media.a`) is
|
|
187
|
+
cached per OS/arch, keyed by `scripts/build-gui.ts` / `scripts/build-media.ts`.
|
|
184
188
|
- Windows (build + run) stays provisional in CI: the MSVC-compatible
|
|
185
189
|
`gui.lib` builds and runs locally through SDL_GPU's Vulkan backend, but the
|
|
186
190
|
hosted runners have no GPU/Vulkan driver. A failure is logged without
|
|
@@ -234,13 +238,15 @@ engine were Rust.
|
|
|
234
238
|
`tests/e2e/gui-retention.test.ts` (see [Memory & GC](./gui/runtime.md#memory--gc)).
|
|
235
239
|
13. **M13 — video & audio** ✅
|
|
236
240
|
- `<video>` / `<audio>` decode through a statically vendored minimal FFmpeg
|
|
237
|
-
|
|
238
|
-
existing image pipeline, audio is
|
|
239
|
-
thread
|
|
240
|
-
|
|
241
|
+
in the optional `media` extension (`runtime/ext_media/media.cpp`), frames
|
|
242
|
+
upload to GPU textures through the existing image pipeline, audio is
|
|
243
|
+
pushed to an SDL stream on the main thread by the GUI's audio sink, and
|
|
244
|
+
the DOM media API plus `loadedmetadata`/`play`/`pause`/`ended`/
|
|
245
|
+
`timeupdate`/`error` events are exposed on element handles
|
|
241
246
|
(`dom_api_media.cpp`, `dom_api_proto.cpp`). Controlled from TS + CSS;
|
|
242
247
|
there are no built-in controls. Example in `examples/gui/media.*`, e2e in
|
|
243
|
-
`tests/e2e/gui-media.test.ts
|
|
248
|
+
`tests/e2e/gui-media.test.ts` (with the `media` extension) and
|
|
249
|
+
`tests/e2e/gui-media-disabled.test.ts` (without it).
|
|
244
250
|
14. **M14 — `<canvas>` 2D** ✅
|
|
245
251
|
- `<canvas>` is a replaced element (`xtgui::Canvas2D`, `layout.*`), the
|
|
246
252
|
`getContext("2d")` handle and drawing API live in `dom_api_canvas.cpp`,
|
|
@@ -300,10 +306,11 @@ engine were Rust.
|
|
|
300
306
|
pixel size and cached by that size, so scaling and HiDPI stay crisp.
|
|
301
307
|
- **M6b** ✅ CSS transitions (parsing in `style.*`, an animation clock and
|
|
302
308
|
transition state in `document.*`, `advance`/`advance(ms)` hooks, e2e coverage).
|
|
303
|
-
- **M7** ✅ builds `gui.a` in CI on Linux and macOS
|
|
304
|
-
example plus the GUI e2e suite under Xvfb on Linux;
|
|
305
|
-
`vendor/` is cached, and the `ar -M` merge
|
|
306
|
-
macOS runs and all of Windows remain provisional
|
|
309
|
+
- **M7** ✅ builds `gui.a` (and the optional `media.a`) in CI on Linux and macOS
|
|
310
|
+
(required) and runs the example plus the GUI e2e suite under Xvfb on Linux;
|
|
311
|
+
`package` ships both archives, `vendor/` is cached, and the `ar -M` merge
|
|
312
|
+
works on GNU/Linux and macOS. macOS runs and all of Windows remain provisional
|
|
313
|
+
in CI.
|
|
307
314
|
- **M8** ✅ element/document handles (`dom_api.*`), DOM mutation with lazy
|
|
308
315
|
restyle/relayout (`document.*`, `xt_gui_flush_dom`), element event dispatch
|
|
309
316
|
with capture/bubble and `stopPropagation` (`dom_api.*`, `gui.cpp`), and e2e
|
|
@@ -338,14 +345,17 @@ engine were Rust.
|
|
|
338
345
|
ratcheting upwards. `win.stats()` exposes the engine counters and
|
|
339
346
|
`tests/e2e/gui-retention.test.ts` pins them; a game loop rewriting its sprite
|
|
340
347
|
every frame now runs in constant memory (see [Memory & GC](./gui/runtime.md#memory--gc)).
|
|
341
|
-
- **M13** ✅ `<video>` / `<audio>` support.
|
|
342
|
-
and
|
|
343
|
-
|
|
344
|
-
|
|
348
|
+
- **M13** ✅ `<video>` / `<audio>` support. The optional `media` extension
|
|
349
|
+
vendors and builds a minimal static FFmpeg (`scripts/build-media.ts`) and
|
|
350
|
+
merges it into `media.a`; `runtime/ext_media/media.cpp` decodes video to RGBA
|
|
351
|
+
frames and audio to 48 kHz stereo PCM on the main thread,
|
|
352
|
+
`runtime/ext_gui/gui_media.cpp` owns the players, implements the SDL audio
|
|
353
|
+
sink and pumps the media clock and events each frame,
|
|
345
354
|
`renderer_image.cpp` uploads each new video frame through the image texture
|
|
346
355
|
path, and `dom_api_media.cpp` exposes `play`/`pause`/`load` plus the media
|
|
347
|
-
properties on element handles. `
|
|
348
|
-
|
|
356
|
+
properties on element handles. `gui.a` stays FFmpeg-free; enabling `media`
|
|
357
|
+
without `gui` is rejected by the registry. `examples/gui/media.*` demonstrates
|
|
358
|
+
the TS-plus-CSS-controls pattern and `tests/e2e/gui-media.test.ts` pins it.
|
|
349
359
|
- **M14** ✅ `<canvas>` support. A new replaced element (`layout.*`) carries an
|
|
350
360
|
intrinsic size from its `width`/`height` attributes (default `300x150`),
|
|
351
361
|
`xtgui::Canvas2D` (`canvas.{h,cpp}`) owns the RGBA8 bitmap and drawing state,
|
package/doc/zh-CN/gui/build.md
CHANGED
|
@@ -5,33 +5,39 @@
|
|
|
5
5
|
> 属于 [GUI 扩展](../gui.md) 的一部分;引擎总览、锁定决策与路线图在那里。
|
|
6
6
|
|
|
7
7
|
- 源码位于 `runtime/ext_gui/`(C/C++),外加 `vendor/` 下的 vendored 库
|
|
8
|
-
(被 gitignore
|
|
8
|
+
(被 gitignore;按需获取)。媒体解码位于 `runtime/ext_media/`,单独构建。
|
|
9
9
|
- `npm run gui`(`scripts/build-gui.ts`)获取固定版本的 SDL3(`SDL3_TAG`,
|
|
10
10
|
默认 `release-3.2.10`),构建静态 SDL3,获取并构建静态 FreeType
|
|
11
11
|
(`FREETYPE_VERSION`)与 HarfBuzz(`HARFBUZZ_VERSION`),编译引擎,并把
|
|
12
12
|
所有内容合并到 `runtime/lib/<os>-<arch>/gui.a`(Windows 上 MSVC ABI 则为
|
|
13
|
-
`gui.lib`),与 `core.a`
|
|
14
|
-
一个最小化的静态 **FFmpeg**(`FFMPEG_VERSION`,默认 `7.1`),只启用媒体元素
|
|
15
|
-
所需的 demuxer/parser/decoder,并把那些归档合并进 `gui.a`,同时为引擎定义
|
|
16
|
-
`XT_GUI_WITH_FFMPEG`。合并使用 macOS 上的
|
|
13
|
+
`gui.lib`),与 `core.a` 采用相同约定,静态打包三者。合并使用 macOS 上的
|
|
17
14
|
`libtool` 与其他平台上的 `ar -M`(GNU/LLVM);CMake 归档会在构建根目录或
|
|
18
15
|
`Release/` 下按任一生成器风格被发现。
|
|
19
|
-
- `
|
|
16
|
+
- `npm run media`(`scripts/build-media.ts`)构建可选的 **FFmpeg** 后端。它获取
|
|
17
|
+
FFmpeg(`FFMPEG_VERSION`,默认 `7.1`),配置一个只启用媒体元素所需
|
|
18
|
+
demuxer/parser/decoder 的最小静态构建,用它编译
|
|
19
|
+
`runtime/ext_media/media.cpp`,并把所有内容合并到
|
|
20
|
+
`runtime/lib/<os>-<arch>/media.a`。`gui.a` **不会**链接 FFmpeg:程序通过
|
|
21
|
+
`--ext media` 显式启用播放能力。
|
|
22
|
+
- `src/extensions/gui/index.ts` 通过 `nativeObjects()` 暴露引擎归档(经由
|
|
20
23
|
`findRuntimeLibrary`,因此 `.a`/`.lib` 都可用),并通过 `linkerFlags()`
|
|
21
|
-
|
|
24
|
+
暴露操作系统框架。`src/extensions/media/index.ts` 对 `media.a` 做同样的暴露,
|
|
25
|
+
并把注册桥(`runtime/ext_media/media_bridge.c`)作为普通 runtime 源编译。
|
|
22
26
|
- `scripts/build-gui.ts` 使用与驱动**相同**的 clang
|
|
23
27
|
(`src/driver/toolchain-provider.ts`);Windows 上就是 Visual Studio 自带的、
|
|
24
28
|
面向 MSVC 的 LLVM,并配套使用其同目录下的 `clang++`/`llvm-ar` 来配置 CMake、
|
|
25
29
|
编译引擎和合并归档。脚本不提供 `gcc`/`g++` 回退,Windows 上也始终使用
|
|
26
30
|
`llvm-ar`,因此归档不会混入其他目标文件格式或 C++ 运行时。Visual Studio
|
|
27
31
|
工具链按绝对路径查找,无需开发者命令提示符。Windows 上 SDL_GPU 会选择 Vulkan
|
|
28
|
-
后端,不需要 DXIL/`dxc`
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
32
|
+
后端,不需要 DXIL/`dxc` 工具链。`scripts/build-media.ts` 对 FFmpeg 与播放器
|
|
33
|
+
使用同一套已解析的工具链。
|
|
34
|
+
- CI 在组装发布归档之前构建 `gui.a` 与 `media.a`,因此它们会随
|
|
35
|
+
`runtime/lib/<slug>/` 一起发布(发布 tarball 会复制整个 `runtime/` 树)。
|
|
36
|
+
归档在 Linux(Xvfb 下,使用 lavapipe 软件 Vulkan 驱动)与 macOS 上构建并运行
|
|
37
|
+
示例;Windows 为试验性,因为托管 runner 没有 GPU/Vulkan 驱动,且 media 归档
|
|
38
|
+
目前仅支持 POSIX。
|
|
39
|
+
- 现有的缓存指纹已经会对 `nativeObjects()` 的内容与 `runtimeSources()` 的内容
|
|
40
|
+
做哈希,因此重建任一归档都会自动使缓存二进制失效。
|
|
35
41
|
- CMake 构建树位于其源码**旁边**(`vendor/sdl-build`、`vendor/freetype-build`、
|
|
36
42
|
`vendor/harfbuzz-build`),绝不放在源码内部:`CMakeCache.txt` 记录的是配置它
|
|
37
43
|
的那台机器,而 SDL 的视频后端是 `option()`,因此先运行的那次配置会一直生效。
|
|
@@ -47,6 +53,7 @@
|
|
|
47
53
|
```sh
|
|
48
54
|
npm run runtime # runtime 改动后重建 core.a
|
|
49
55
|
npm run gui # 构建 runtime/lib/<os>-<arch>/gui.a(首次会获取 SDL3)
|
|
56
|
+
npm run media # 可选:构建 media.a(首次会获取并构建 FFmpeg)
|
|
50
57
|
xbintsc run examples/gui/hello.ts --ext gui
|
|
51
58
|
```
|
|
52
59
|
|
|
@@ -59,8 +66,9 @@ xbintsc run examples/gui/hello.ts --ext gui
|
|
|
59
66
|
|
|
60
67
|
`examples/gui/media.ts`(配合 `media.html`)演示 `<video>`/`<audio>`:它在
|
|
61
68
|
`<video>` 上叠一个 CSS 进度条,并用 `play()`/`pause()`/`currentTime`/
|
|
62
|
-
`volume`
|
|
63
|
-
`
|
|
69
|
+
`volume` 驱动播放。它需要单独的可选 `media` 扩展,因此要构建并同时启用两者。
|
|
70
|
+
媒体 `src` 相对于进程工作目录解析,因此从仓库根目录运行:
|
|
71
|
+
`xbintsc run examples/gui/media.ts --ext gui,media`。几何报告见该目录的 README。
|
|
64
72
|
|
|
65
73
|
这些测试套件在 Windows 上默认跳过(归档可以在本机通过 Vulkan 运行,但 CI
|
|
66
74
|
runner 没有可用的 GPU 驱动)。如果本机有 `gui.lib` 与可用的 GPU,可用
|
|
@@ -77,10 +85,14 @@ npx vitest run tests/e2e/gui-layout.test.ts tests/e2e/gui-example.test.ts
|
|
|
77
85
|
```sh
|
|
78
86
|
sudo sh scripts/install-gui-deps.sh # clang、cmake、X11/Wayland 头文件、Vulkan、Xvfb
|
|
79
87
|
npm run runtime && npm run gui
|
|
88
|
+
npm run media # 可选:<video>/<audio> 播放
|
|
80
89
|
xvfb-run -a --server-args="-screen 0 1280x720x24" \
|
|
81
90
|
npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
|
|
82
91
|
```
|
|
83
92
|
|
|
93
|
+
媒体 e2e 套件(`tests/e2e/gui-media.test.ts`)只有在 `gui.a` 与 `media.a` 都
|
|
94
|
+
存在时才会运行;`tests/e2e/gui-media-disabled.test.ts` 覆盖仅 `gui.a` 的构建。
|
|
95
|
+
|
|
84
96
|
包清单集中在 `scripts/install-gui-deps.sh`(`apt-get install -y clang cmake
|
|
85
97
|
libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev libxinerama-dev
|
|
86
98
|
libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols libdecor-0-dev
|
|
@@ -54,11 +54,12 @@
|
|
|
54
54
|
SDL_GPU 会选择 **Vulkan** 后端(`win.driver()` 返回 `"vulkan"`),因此归档
|
|
55
55
|
不需要 DXIL/`dxc` 工具链;托管 Windows runner 没有 GPU/Vulkan 驱动,这才是
|
|
56
56
|
CI 运行仍为试验性的原因。
|
|
57
|
-
- **Windows 上的媒体:**
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
CSS 叠层,如
|
|
57
|
+
- **Windows 上的媒体:** `media` 扩展目前仅支持 POSIX。GUI 引擎本身可在
|
|
58
|
+
Windows 上构建,但没有该扩展时 `<video>`/`<audio>` 为空操作(打开失败并触发
|
|
59
|
+
`error`),`--ext media` 会给出可操作的错误而不是继续链接。启用它需要
|
|
60
|
+
MSVC 兼容的 FFmpeg 构建,推迟到 Windows GUI 路径本身验证之后。
|
|
61
|
+
`<video controls>` 同样推迟——预期模式是 TS + CSS 叠层,如
|
|
62
|
+
`examples/gui/media.ts`。
|
|
62
63
|
- **D3D12 着色器:** SPIR-V(Vulkan)与 DXIL(Direct3D 12)二进制块均已构建并
|
|
63
64
|
内嵌,非 Metal 路径不再跳过几何绘制。但在 **SDL 3.2.10** 上 DXIL 仍不可用:
|
|
64
65
|
其 D3D12 后端会拒绝任何着色器中声明了 uniform buffer 的图形管线,而引擎正是
|
package/doc/zh-CN/gui/media.md
CHANGED
|
@@ -55,25 +55,36 @@ opacity、`display`——与任何其他元素一样。盒子内部的艺术内
|
|
|
55
55
|
|
|
56
56
|
## 视频与音频(M13)
|
|
57
57
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
58
|
+
媒体播放横跨两个扩展,因此只做界面的 GUI 程序不必为用不到的解码器付出体积:
|
|
59
|
+
|
|
60
|
+
- **`gui`**(`gui.a`)负责 `<video>`/`<audio>` 元素、DOM API、渲染器集成与 SDL
|
|
61
|
+
音频 sink,但**不含 FFmpeg**。`runtime/ext_gui/media.{h,cpp}` 声明播放器接口,
|
|
62
|
+
并通过媒体后端在启动时安装的函数表分发(`runtime/ext_gui/media_api.h`)。
|
|
63
|
+
- **`media`**(`media.a`、`runtime/ext_media/`)是可选的解码后端:
|
|
64
|
+
`runtime/ext_media/media.cpp` 封装静态 vendored 的 **FFmpeg** 构建,
|
|
65
|
+
`runtime/ext_media/media_bridge.c` 把它注册进引擎。用 `--ext media`(或
|
|
66
|
+
`--ext gui,media`)启用。`media` 依赖 `gui`;注册表会拒绝错误的组合。
|
|
67
|
+
|
|
68
|
+
没有 `media` 扩展时,元素仍然存在、DOM API 仍可调用,但打开源会失败并触发
|
|
69
|
+
`error`。Windows 目前也是如此:该归档仅支持 POSIX。两条路径都没有 WebView,也
|
|
70
|
+
没有平台媒体框架。
|
|
61
71
|
|
|
62
72
|
- **解码。** `avformat_open_input` 打开 `src`(是路径——接受 `file://`,但与
|
|
63
73
|
图像不同,媒体路径*不做*百分号解码),为每种媒体选取最佳流,软件解码器
|
|
64
74
|
产出 `yuv/rgb` 帧,`sws_scale` 按编码尺寸转换为 RGBA8。支持的容器/编码即
|
|
65
|
-
`scripts/build-
|
|
75
|
+
`scripts/build-media.ts` 的最小 FFmpeg 配置:MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/
|
|
66
76
|
ASF,MP3/OGG/WAV/FLAC/AAC;视频 H.264、HEVC、VP8、VP9、AV1、MPEG-4、MJPEG;
|
|
67
77
|
音频 AAC、MP3、Opus、Vorbis、FLAC、ALAC、AC3/E-AC3、WMAv2 及原始 PCM 格式。
|
|
68
78
|
- **视频。** `<video>` 与 `<img>` 一样是替换元素(UA 默认 `300x150`,解码
|
|
69
79
|
尺寸通过 `setMediaSize` 覆盖)。绘制产出与图像相同的 `PaintImage`;渲染器把
|
|
70
80
|
每个新解码帧上传到 RGBA 纹理(`ensureMediaTexture`)并绘制带纹理四边形,
|
|
71
|
-
因此由 GPU 缩放到 CSS 盒,而非 CPU
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
`
|
|
81
|
+
因此由 GPU 缩放到 CSS 盒,而非 CPU 缩放。播放计时基于进程单调时钟,不是帧间隔。
|
|
82
|
+
- **音频。** 没有音频回调:解码器随主线程 tick 被推送,并写入
|
|
83
|
+
`xtgui::AudioSink`。GUI 在 `gui_media.cpp` 中用 push 模式(`NULL` 回调)的
|
|
84
|
+
`SDL_OpenAudioDeviceStream` 实现该 sink,在前方缓冲约 0.25 秒,格式为
|
|
85
|
+
48 kHz 立体声 `SDL_AUDIO_F32`。`volume` 映射到 `SDL_SetAudioStreamGain`;
|
|
86
|
+
音频是尽力而为,无设备时(无头 CI)跳过。把 sink 留在 `gui.a` 中意味着
|
|
87
|
+
`media.a` 只依赖 FFmpeg。
|
|
77
88
|
- **线程。** 引擎是单线程且非线程感知,因此所有解码都在主线程的
|
|
78
89
|
`xt_gui_media_tick` 内完成——没有解码线程,也没有跨线程 GC 暴露。
|
|
79
90
|
- **DOM API。** 元素句柄直接暴露媒体接口(见
|