xbintsc 0.5.22 → 0.5.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/AGENTS.md +1 -1
  2. package/README.md +15 -5
  3. package/README.zh-CN.md +11 -4
  4. package/dist/src/cli/hints.js +11 -0
  5. package/dist/src/cli/hints.js.map +1 -1
  6. package/dist/src/cli/main.js +1 -1
  7. package/dist/src/driver/compiler.js +1 -0
  8. package/dist/src/driver/compiler.js.map +1 -1
  9. package/dist/src/driver/version.d.ts +1 -1
  10. package/dist/src/driver/version.js +1 -1
  11. package/dist/src/extensions/catalog.js +2 -1
  12. package/dist/src/extensions/catalog.js.map +1 -1
  13. package/dist/src/extensions/media/index.d.ts +27 -0
  14. package/dist/src/extensions/media/index.js +60 -0
  15. package/dist/src/extensions/media/index.js.map +1 -0
  16. package/dist/src/extensions/registry.d.ts +12 -0
  17. package/dist/src/extensions/registry.js +16 -0
  18. package/dist/src/extensions/registry.js.map +1 -1
  19. package/dist/src/index.d.ts +1 -0
  20. package/dist/src/index.js +1 -0
  21. package/dist/src/index.js.map +1 -1
  22. package/dist/tests/cli/hints.test.js +15 -0
  23. package/dist/tests/cli/hints.test.js.map +1 -1
  24. package/dist/tests/cli/process.test.js +12 -0
  25. package/dist/tests/cli/process.test.js.map +1 -1
  26. package/dist/tests/e2e/gui-example-canvas.test.js +3 -1
  27. package/dist/tests/e2e/gui-example-canvas.test.js.map +1 -1
  28. package/dist/tests/e2e/gui-helpers.d.ts +3 -0
  29. package/dist/tests/e2e/gui-helpers.js +2 -0
  30. package/dist/tests/e2e/gui-helpers.js.map +1 -1
  31. package/dist/tests/e2e/gui-media-disabled.test.d.ts +9 -0
  32. package/dist/tests/e2e/gui-media-disabled.test.js +42 -0
  33. package/dist/tests/e2e/gui-media-disabled.test.js.map +1 -0
  34. package/dist/tests/e2e/gui-media.test.d.ts +6 -4
  35. package/dist/tests/e2e/gui-media.test.js +15 -12
  36. package/dist/tests/e2e/gui-media.test.js.map +1 -1
  37. package/dist/tests/e2e/unicode-output.test.d.ts +16 -0
  38. package/dist/tests/e2e/unicode-output.test.js +65 -0
  39. package/dist/tests/e2e/unicode-output.test.js.map +1 -0
  40. package/dist/tests/extensions/media.test.d.ts +1 -0
  41. package/dist/tests/extensions/media.test.js +58 -0
  42. package/dist/tests/extensions/media.test.js.map +1 -0
  43. package/dist/tests/extensions/registry.test.js +8 -0
  44. package/dist/tests/extensions/registry.test.js.map +1 -1
  45. package/dist/tests/lexer/identifiers.test.js +12 -0
  46. package/dist/tests/lexer/identifiers.test.js.map +1 -1
  47. package/dist/tests/lexer/scanner.test.js +8 -0
  48. package/dist/tests/lexer/scanner.test.js.map +1 -1
  49. package/dist/tests/lexer/strings.test.js +30 -0
  50. package/dist/tests/lexer/strings.test.js.map +1 -1
  51. package/doc/ai/cli.md +1 -1
  52. package/doc/ai/extensions.md +35 -23
  53. package/doc/ai/troubleshooting.md +9 -0
  54. package/doc/ai/zh-CN/cli.md +1 -1
  55. package/doc/ai/zh-CN/extensions.md +20 -8
  56. package/doc/ai/zh-CN/troubleshooting.md +8 -0
  57. package/doc/gui/build.md +37 -21
  58. package/doc/gui/known-issues.md +7 -6
  59. package/doc/gui/media.md +26 -11
  60. package/doc/gui.md +43 -33
  61. package/doc/zh-CN/gui/build.md +28 -16
  62. package/doc/zh-CN/gui/known-issues.md +6 -5
  63. package/doc/zh-CN/gui/media.md +21 -10
  64. package/doc/zh-CN/gui.md +37 -32
  65. package/package.json +2 -1
  66. package/runtime/ext_gui/gui_media.cpp +69 -2
  67. package/runtime/ext_gui/media.cpp +34 -488
  68. package/runtime/ext_gui/media.h +54 -19
  69. package/runtime/ext_gui/media_api.h +65 -0
  70. package/runtime/ext_media/media.cpp +507 -0
  71. package/runtime/ext_media/media_bridge.c +35 -0
  72. package/scripts/build-gui.ts +6 -72
  73. package/scripts/build-media.ts +245 -0
  74. package/src/cli/hints.ts +11 -0
  75. package/src/cli/main.ts +1 -1
  76. package/src/driver/compiler.ts +1 -0
  77. package/src/driver/version.ts +1 -1
  78. package/src/extensions/catalog.ts +2 -1
  79. package/src/extensions/media/index.ts +67 -0
  80. package/src/extensions/registry.ts +24 -0
  81. package/src/index.ts +1 -0
@@ -118,6 +118,15 @@ The `gui` extension needs its per-platform prebuilt `gui.a`/`gui.lib`, which is
118
118
  not built in a plain source checkout. Drop `--ext gui`, or build the archive as
119
119
  described in [../gui.md](../gui.md).
120
120
 
121
+ ### `media native library not found at …`
122
+
123
+ `<video>`/`<audio>` playback needs the optional `media` archive (`media.a`),
124
+ built separately with `npm run media`. Build it, or drop `--ext media`; a
125
+ GUI-only build still links and media elements simply fire `error`. The `media`
126
+ extension also requires `gui`, has no Windows archive yet, and `--ext media`
127
+ alone is rejected with the flags to pass. See
128
+ [../gui/build.md](../gui/build.md).
129
+
121
130
  ### `Icon file not found` / icon tools missing
122
131
 
123
132
  `--icon`/`app.icon` paths resolve against the config file's directory. On
@@ -32,7 +32,7 @@
32
32
  --out <dir> 输出目录(默认 build/)
33
33
  --emit <kind> exe | obj | ir(默认 exe)
34
34
  -O0 .. -O3 传给 clang 的优化级别(默认 -O2)
35
- --ext <names> 开启内置扩展,逗号分隔(如 node,gui)
35
+ --ext <names> 开启内置扩展,逗号分隔(如 node,gui,media)
36
36
  --ext-native <m> 通过 JSON 清单注册 C++/Rust 扩展(多个用逗号分隔)
37
37
  --config <path> 使用指定项目配置,不做自动发现
38
38
  --no-config 完全不读取项目配置
@@ -1,4 +1,4 @@
1
- # 扩展:Node 模块、GUI、C++/Rust 库
1
+ # 扩展:Node 模块、GUI、媒体、C++/Rust 库
2
2
 
3
3
  编译器核心与平台无关。任何与平台相关的东西——Node 的 `fs`、一个 HTML/CSS 渲染器、
4
4
  你自己的 C++ 库——都以**扩展**的形式接入:它既贡献模块绑定,也贡献要一起编译/链接的
@@ -66,11 +66,9 @@ import { createWindow, run } from "gui";
66
66
  ```
67
67
 
68
68
  它是自带渲染引擎的 GPU 加速实现(自己的 HTML 解析器、CSS 级联、布局与合成器),不是
69
- 系统 WebView。`<img>`、`<video>`、`<audio>` 都是真正解码:归档内打入一个最小化的
70
- 静态 FFmpeg,DOM 媒体 API(`play`/`pause`/`currentTime`/`volume`/…)与
71
- `loadedmetadata`/`play`/`pause`/`ended`/`timeupdate`/`error` 事件可让程序驱动播放。
72
- 没有内置控件——用元素和 CSS 自绘控件(见 `examples/gui/media.ts`);视频帧与音频 PCM
73
- 都在主线程解码。也支持 `<canvas>`:`getContext("2d")` 返回的上下文,其填充、路径、
69
+ 系统 WebView。`<img>` 由引擎自身解码;而 `<video>`/`<audio>` 在只启用 `gui` 时**不可用**
70
+ ——它们需要单独的 [`media`](#media--视频音频解码) 扩展。也支持 `<canvas>`:
71
+ `getContext("2d")` 返回的上下文,其填充、路径、
74
72
  渐变、文本与 `drawImage` 由自研 CPU 光栅器处理(另有线段端点/连接、虚线描边、
75
73
  `ellipse` 与 `roundRect`),元素句柄还暴露 `width`/`height`。CSS
76
74
  `transform`/`transform-origin` 作为绘制期效果可用,因此一个盒子(以及其中的文本、
@@ -78,10 +76,24 @@ import { createWindow, run } from "gui";
78
76
  `requestAnimationFrame` 逐帧驱动,或写成 `transition: transform`(见
79
77
  `examples/gui/pelican-svg`)。因为主体是 C++,它以各平台预编译的 `gui.a`/`gui.lib` 形式分发。如果该
80
78
  平台的归档不存在,构建会以明确的提示失败;设计与路线图见 [../../zh-CN/gui.md](../../zh-CN/gui.md)(总览;引擎按功能拆分在 [`doc/zh-CN/gui/`](../../zh-CN/gui/) 下),
81
- 脚本接口见 [../../zh-CN/gui-scripts.md](../../zh-CN/gui-scripts.md)。媒体播放是
82
- Windows 唯一不支持的:Windows 的 `gui.a` 不构建 FFmpeg,媒体槽位为空操作。Canvas API
79
+ 脚本接口见 [../../zh-CN/gui-scripts.md](../../zh-CN/gui-scripts.md)。Canvas API
83
80
  及其限制见 [../../zh-CN/gui/canvas.md](../../zh-CN/gui/canvas.md)。
84
81
 
82
+ ### `media` —— 视频/音频解码
83
+
84
+ 用 `--ext media` **与 `gui` 一起**开启(`--ext gui,media`;注册表会拒绝单独启用
85
+ `media`)。它链接单独的 FFmpeg 后端 `media.a`,因此加入 `<video>`/`<audio>` 播放
86
+ 不会让只做界面的二进制变大。
87
+
88
+ DOM API 与普通 HTML 媒体元素一致:`play()` / `pause()` / `load()`,以及
89
+ `paused` / `ended` / `duration` / `currentTime` / `volume` / `muted` / `loop` /
90
+ `autoplay` / `videoWidth` / `videoHeight` / `readyState` / `src` 属性,还有
91
+ `loadedmetadata`/`play`/`pause`/`ended`/`timeupdate`/`error` 事件。没有内置
92
+ 控件——用元素和 CSS 自绘控件(见 `examples/gui/media.ts`)。视频帧与音频 PCM
93
+ 都在主线程解码;视频走图像纹理路径。没有该扩展时,打开媒体源会失败并触发
94
+ `error`(元素与 API 仍然存在)。该归档目前仅支持 POSIX,因此 Windows 也只会触发
95
+ 同样的 `error`;见 [../../zh-CN/gui/known-issues.md](../../zh-CN/gui/known-issues.md)。
96
+
85
97
  ## 第三方 npm 包
86
98
 
87
99
  没有被已启用扩展认领的裸说明符,会在 `node_modules` 里查找并**作为源码**打包,
@@ -108,6 +108,14 @@ clang 自身失败。抛出的消息包含完整的 clang 命令与 stderr——
108
108
  `gui` 扩展需要各平台预编译的 `gui.a`/`gui.lib`,普通源码检出并没有构建它。去掉
109
109
  `--ext gui`,或按 [../../zh-CN/gui.md](../../zh-CN/gui.md) 构建该归档。
110
110
 
111
+ ### `media native library not found at …`
112
+
113
+ `<video>`/`<audio>` 播放需要可选的 `media` 归档(`media.a`),用
114
+ `npm run media` 单独构建。构建它,或去掉 `--ext media`;仅启用 `gui` 的构建仍可
115
+ 链接,媒体元素只会触发 `error`。`media` 扩展还依赖 `gui`,目前没有 Windows
116
+ 归档,单独 `--ext media` 会被拒绝并提示应传的 flag。见
117
+ [../../zh-CN/gui/build.md](../../zh-CN/gui/build.md)。
118
+
111
119
  ### `Icon file not found` / 图标工具缺失
112
120
 
113
121
  `--icon`/`app.icon` 的路径相对配置文件目录解析。Windows 上嵌入 PE 图标需要
package/doc/gui/build.md CHANGED
@@ -4,21 +4,27 @@
4
4
  > decisions and roadmap live there.
5
5
 
6
6
  - Sources live in `runtime/ext_gui/` (C/C++) plus vendored libs under `vendor/`
7
- (gitignored; fetched on demand).
7
+ (gitignored; fetched on demand). Media decoding lives in
8
+ `runtime/ext_media/` and is built separately.
8
9
  - `npm run gui` (`scripts/build-gui.ts`) fetches a pinned SDL3 (`SDL3_TAG`,
9
10
  default `release-3.2.10`), builds a static SDL3, fetches and builds static
10
11
  FreeType (`FREETYPE_VERSION`) and HarfBuzz (`HARFBUZZ_VERSION`), compiles the
11
12
  engine and merges everything into `runtime/lib/<os>-<arch>/gui.a` (or
12
13
  `gui.lib` with the MSVC ABI on Windows), same convention as `core.a`,
13
- statically bundling all three. It also fetches, configures and builds a
14
- minimal static **FFmpeg** (`FFMPEG_VERSION`, default `7.1`) with only the
15
- demuxers/parsers/decoders the media elements need and merges those archives
16
- into `gui.a`, defining `XT_GUI_WITH_FFMPEG` for the engine. The merge uses
17
- `libtool` on macOS and `ar -M` (GNU/LLVM) elsewhere; CMake archives are
18
- discovered under the build root or `Release/` for either generator style.
19
- - `src/extensions/gui/index.ts` exposes the archive through `nativeObjects()`
20
- (via `findRuntimeLibrary`, so `.a`/`.lib` both work) and the OS frameworks
21
- through `linkerFlags()`.
14
+ statically bundling all three. The merge uses `libtool` on macOS and
15
+ `ar -M` (GNU/LLVM) elsewhere; CMake archives are discovered under the build
16
+ root or `Release/` for either generator style.
17
+ - `npm run media` (`scripts/build-media.ts`) builds the optional **FFmpeg**
18
+ backend. It fetches FFmpeg (`FFMPEG_VERSION`, default `7.1`), configures a
19
+ minimal static build with only the demuxers/parsers/decoders the media
20
+ elements need, compiles `runtime/ext_media/media.cpp` against it and merges
21
+ everything into `runtime/lib/<os>-<arch>/media.a`. `gui.a` does **not** link
22
+ FFmpeg: a program enables playback explicitly with `--ext media`.
23
+ - `src/extensions/gui/index.ts` exposes the engine archive through
24
+ `nativeObjects()` (via `findRuntimeLibrary`, so `.a`/`.lib` both work) and the
25
+ OS frameworks through `linkerFlags()`. `src/extensions/media/index.ts` does
26
+ the same for `media.a` and compiles the registration bridge
27
+ (`runtime/ext_media/media_bridge.c`) as a normal runtime source.
22
28
  - `scripts/build-gui.ts` compiles with the *same* clang the driver resolves
23
29
  (`src/driver/toolchain-provider.ts`); on Windows that is the MSVC-targeting
24
30
  LLVM bundled with Visual Studio, and its sibling `clang++`/`llvm-ar` are used
@@ -26,14 +32,17 @@
26
32
  fallbacks, and Windows always uses `llvm-ar`, so the archive cannot mix object
27
33
  formats or C++ runtimes. The Visual Studio toolchain is found by absolute
28
34
  path, so no developer prompt is required. On Windows SDL_GPU selects the
29
- Vulkan backend; no DXIL/`dxc` toolchain is needed.
30
- - CI builds `gui.a` before assembling the release archive so it ships inside
31
- `runtime/lib/<slug>/` (the release tarball copies the whole `runtime/` tree).
32
- The archive is built and the example is run on Linux (under Xvfb, with the
33
- lavapipe software Vulkan driver) and macOS; Windows is provisional because the
34
- hosted runners have no GPU/Vulkan driver.
35
- - The existing cache fingerprint already hashes `nativeObjects()` contents, so
36
- rebuilding `gui.a` invalidates cached binaries automatically.
35
+ Vulkan backend; no DXIL/`dxc` toolchain is needed. `scripts/build-media.ts`
36
+ uses the same resolved toolchain for FFmpeg and the player.
37
+ - CI builds `gui.a` and `media.a` before assembling the release archive so they
38
+ ship inside `runtime/lib/<slug>/` (the release tarball copies the whole
39
+ `runtime/` tree). The archives are built and the examples are run on Linux
40
+ (under Xvfb, with the lavapipe software Vulkan driver) and macOS; Windows is
41
+ provisional because the hosted runners have no GPU/Vulkan driver, and the
42
+ media archive is POSIX-only.
43
+ - The existing cache fingerprint already hashes `nativeObjects()` contents and
44
+ `runtimeSources()` content, so rebuilding either archive invalidates cached
45
+ binaries automatically.
37
46
  - CMake build trees live *next to* their sources (`vendor/sdl-build`,
38
47
  `vendor/freetype-build`, `vendor/harfbuzz-build`), never inside them: a
39
48
  `CMakeCache.txt` records the machine it was configured on, and SDL's video
@@ -52,6 +61,7 @@
52
61
  ```sh
53
62
  npm run runtime # rebuild core.a after a runtime change
54
63
  npm run gui # build runtime/lib/<os>-<arch>/gui.a (fetches SDL3 once)
64
+ npm run media # optional: build media.a (fetches and builds FFmpeg once)
55
65
  xbintsc run examples/gui/hello.ts --ext gui
56
66
  ```
57
67
 
@@ -62,9 +72,10 @@ its README for controls and the geometry report it prints.
62
72
 
63
73
  `media.ts` (with `media.html`) demonstrates `<video>`/`<audio>`: it overlays a
64
74
  CSS progress bar on the video and drives playback with `play()` / `pause()` /
65
- `currentTime` / `volume`. Media `src` paths resolve relative to the process
66
- working directory, so run it from the repository root:
67
- `xbintsc run examples/gui/media.ts --ext gui`.
75
+ `currentTime` / `volume`. It needs the separate `media` extension, so build and
76
+ enable both. Media `src` paths resolve relative to the process working
77
+ directory, so run it from the repository root:
78
+ `xbintsc run examples/gui/media.ts --ext gui,media`.
68
79
 
69
80
  Set `XT_GUI_AUTOCLOSE_MS=<n>` to close all windows after `n` milliseconds,
70
81
  which the e2e test (`tests/e2e/gui-*.test.ts`) uses to run headlessly.
@@ -84,10 +95,15 @@ software Vulkan driver:
84
95
  ```sh
85
96
  sudo sh scripts/install-gui-deps.sh # clang, cmake, X11/Wayland headers, Vulkan, Xvfb
86
97
  npm run runtime && npm run gui
98
+ npm run media # optional: <video>/<audio> playback
87
99
  xvfb-run -a --server-args="-screen 0 1280x720x24" \
88
100
  npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
89
101
  ```
90
102
 
103
+ The media e2e suite (`tests/e2e/gui-media.test.ts`) skips itself unless both
104
+ `gui.a` and `media.a` exist; `tests/e2e/gui-media-disabled.test.ts` covers the
105
+ GUI-only build and runs with `gui.a` alone.
106
+
91
107
  `scripts/install-gui-deps.sh` holds the package list (`apt-get install -y clang
92
108
  cmake libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev
93
109
  libxinerama-dev libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols
@@ -69,12 +69,13 @@
69
69
  (`win.driver()` reports `"vulkan"`), so the archive needs no DXIL/`dxc`
70
70
  toolchain; the hosted Windows runners have no GPU/Vulkan driver, which is why
71
71
  the CI run stays provisional.
72
- - **Media on Windows:** the media decoders (FFmpeg) are only built and merged
73
- into the POSIX `gui.a`; the Windows `gui.lib` is produced without them, so
74
- `<video>`/`<audio>` are inert there (failures fire `error`). Enabling it needs
75
- an MSVC-compatible FFmpeg build and is deferred until the Windows GUI path
76
- itself is validated. `<video controls>` is likewise deferred — the intended
77
- pattern is TS + a CSS overlay, as in `examples/gui/media.ts`.
72
+ - **Media on Windows:** the `media` extension is POSIX-only for now. The GUI
73
+ engine itself builds on Windows, but without the extension `<video>`/`<audio>`
74
+ are inert there (an open fails and fires `error`), and `--ext media` reports an
75
+ actionable error instead of linking. Enabling it needs an MSVC-compatible
76
+ FFmpeg build and is deferred until the Windows GUI path itself is validated.
77
+ `<video controls>` is likewise deferred — the intended pattern is TS + a CSS
78
+ overlay, as in `examples/gui/media.ts`.
78
79
  - **D3D12 shaders:** the SPIR-V (Vulkan) and DXIL (Direct3D 12) blobs are built
79
80
  and embedded, so the non-Metal path no longer skips geometry. DXIL remains
80
81
  unusable on **SDL 3.2.10**: its D3D12 backend rejects any graphics pipeline
package/doc/gui/media.md CHANGED
@@ -65,17 +65,31 @@ clip paths are not rendered (see [Known issues](./known-issues.md)).
65
65
 
66
66
  ## Video & audio (M13)
67
67
 
68
- `runtime/ext_gui/media.{h,cpp}` wraps a statically vendored **FFmpeg** build and
69
- `runtime/ext_gui/gui_media.cpp` connects the DOM tree to it. `<video>` and
70
- `<audio>` really decode and play; there is no WebView and no platform media
71
- framework in the path.
68
+ Media playback is split across two extensions so a GUI program does not pay for
69
+ a decoder it never uses:
70
+
71
+ - **`gui`** (`gui.a`) owns the `<video>`/`<audio>` elements, the DOM API, the
72
+ renderer integration and the SDL audio sink, but contains **no FFmpeg**.
73
+ `runtime/ext_gui/media.{h,cpp}` declares the player interface and dispatches
74
+ through a table the media backend installs at startup
75
+ (`runtime/ext_gui/media_api.h`).
76
+ - **`media`** (`media.a`, `runtime/ext_media/`) is the optional decoder
77
+ backend: `runtime/ext_media/media.cpp` wraps a statically vendored **FFmpeg**
78
+ build, and `runtime/ext_media/media_bridge.c` registers it with the engine.
79
+ Enable it with `--ext media` (or `--ext gui,media`). `media` requires `gui`;
80
+ the registry rejects the combination in the wrong order.
81
+
82
+ Without the `media` extension the element still exists and its DOM API is
83
+ callable, but opening a source fails and fires `error`. That is also how Windows
84
+ behaves today: the archive is POSIX-only. There is no WebView and no platform
85
+ media framework in the path either way.
72
86
 
73
87
  - **Decoding.** `avformat_open_input` opens the `src` (a path — `file://` is
74
88
  accepted, but media paths are *not* percent-decoded, unlike images), the
75
89
  best stream is selected for each kind, and a software decoder yields
76
90
  `yuv/rgb` frames which `sws_scale` converts to RGBA8 at the coded size. The
77
91
  supported containers/codecs are the minimal FFmpeg configuration from
78
- `scripts/build-gui.ts`: MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/ASF, MP3/OGG/WAV/FLAC/
92
+ `scripts/build-media.ts`: MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/ASF, MP3/OGG/WAV/FLAC/
79
93
  AAC; H.264, HEVC, VP8, VP9, AV1, MPEG-4, MJPEG video; AAC, MP3, Opus, Vorbis,
80
94
  FLAC, ALAC, AC3/E-AC3, WMAv2 and the raw PCM formats audio.
81
95
  - **Video.** `<video>` is a replaced element like `<img>` (UA default
@@ -83,13 +97,14 @@ framework in the path.
83
97
  same `PaintImage` as an image; the renderer uploads each new decoded frame to
84
98
  an RGBA texture (`ensureMediaTexture`) and draws a textured quad, so the GPU
85
99
  scales it to the CSS box instead of the CPU resizing. Playback timing is the
86
- wall clock (`SDL_GetTicks`), not the frame delta.
100
+ process monotonic clock, not the frame delta.
87
101
  - **Audio.** There is no audio callback: the decoder is pushed on the main
88
- thread as it is ticked and the PCM is queued with
89
- `SDL_OpenAudioDeviceStream` in push mode (`NULL` callback), buffering about
90
- 0.25 s ahead at 48 kHz stereo `SDL_AUDIO_F32`. `volume` maps to
91
- `SDL_SetAudioStreamGain`; audio is best-effort and is skipped when no device
92
- is available (headless CI).
102
+ thread as it is ticked and writes into an `xtgui::AudioSink`. The GUI
103
+ implements that sink in `gui_media.cpp` with `SDL_OpenAudioDeviceStream` in
104
+ push mode (`NULL` callback), buffering about 0.25 s ahead at 48 kHz stereo
105
+ `SDL_AUDIO_F32`. `volume` maps to `SDL_SetAudioStreamGain`; audio is
106
+ best-effort and is skipped when no device is available (headless CI). Keeping
107
+ the sink in `gui.a` means `media.a` depends only on FFmpeg.
93
108
  - **Threading.** The engine is single-threaded and not thread-aware, so all
94
109
  decoding happens on the main thread inside `xt_gui_media_tick` — no decode
95
110
  thread, no cross-thread GC exposure.
package/doc/gui.md CHANGED
@@ -21,9 +21,11 @@ what `examples/gui/pelican-bike` — a playable 2D game — is built on. **M12**
21
21
  DOM node lifetime explicit (what a mutation releases, what it keeps) and gives the
22
22
  engine bounded memory: see [Node lifetime](./gui/dom.md#node-lifetime) and
23
23
  [Memory & GC](./gui/runtime.md#memory--gc). **M13** plays real
24
- `<video>`/`<audio>` — a vendored minimal FFmpeg decodes them on the main thread,
25
- video frames go through the image texture path and audio to an SDL stream, with
26
- the DOM media API exposing `play`/`pause`/`currentTime`/… and media events.
24
+ `<video>`/`<audio>` — the optional `media` extension decodes them on the main
25
+ thread with a vendored minimal FFmpeg, video frames go through the image texture
26
+ path and audio to an SDL stream, with the DOM media API exposing
27
+ `play`/`pause`/`currentTime`/… and media events. Enabling `gui` alone does not
28
+ link FFmpeg.
27
29
  **M14** adds `<canvas>` with a self-written 2D software rasteriser (anti-aliased
28
30
  fills, paths, text and `drawImage`) uploaded through the image texture path —
29
31
  see [Canvas](./gui/canvas.md). **M15** adds CSS `transform` /
@@ -84,7 +86,7 @@ pages above.
84
86
  | 4 | The engine is **not self-hosted** (it is C/C++, not TS), but must not affect the compiler's platform-agnostic design. Delivered as a per-platform prebuilt archive linked via `nativeObjects`. |
85
87
  | 5 | The **event loop is generic**: the runtime exposes a generic main-loop hook and a poll primitive; nothing GUI-specific enters `runtime/`. |
86
88
  | 6 | **Multiple windows** are supported by the core object model. |
87
- | 7 | Third-party low-level libs are **statically linked into `gui.a`** so releases stay self-contained; only OS frameworks are added at link time. |
89
+ | 7 | Third-party low-level libs are **statically linked into the extension archives** so releases stay self-contained; only OS frameworks are added at link time. `gui.a` carries SDL3/FreeType/HarfBuzz, and the optional `media.a` carries FFmpeg and is linked only with `--ext media`. |
88
90
 
89
91
  ## Architecture
90
92
 
@@ -106,6 +108,7 @@ pages above.
106
108
  windowing + input (SDL3) ── multiple windows
107
109
  text (HarfBuzz + FreeType) ── glyph atlas / SDF
108
110
  images (stb_image; SVG via NanoSVG)
111
+ media DOM + audio sink ── optional media.a (FFmpeg) installs the decoder
109
112
  ```
110
113
 
111
114
  The engine never talks to the compiler. The compiler only sees an `Extension`
@@ -120,18 +123,18 @@ with `nativeObjects()`, `linkerFlags()` and `modules()`.
120
123
  | Text shaping | **HarfBuzz** | correct complex-script shaping |
121
124
  | Glyph raster | **FreeType** | glyph outlines → GPU atlas / SDF |
122
125
  | Images | **stb_image** (vendored header) for bitmaps, **NanoSVG** (vendored headers) for SVG | single headers to start; SVG files rasterise on demand |
123
- | Video + audio | **FFmpeg** (statically vendored, minimal build) | decodes the common container/codec set; no WebView or platform media framework |
126
+ | Video + audio | **FFmpeg** (statically vendored, minimal build, optional `media` extension) | decodes the common container/codec set; no WebView or platform media framework |
124
127
 
125
128
  Pinned versions: SDL3 `release-3.2.10`, FreeType `2.13.3`, HarfBuzz `10.1.0`,
126
- FFmpeg `7.1` (overridable with `SDL3_TAG` / `FREETYPE_VERSION` /
127
- `HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
129
+ FFmpeg `7.1` for the optional `media` archive (overridable with `SDL3_TAG` /
130
+ `FREETYPE_VERSION` / `HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
128
131
 
129
132
  Alternative if SDL3 is rejected: **GLFW + OpenGL 3.3** (simpler, but OpenGL is
130
133
  deprecated on macOS and gives no modern GPU abstraction). **wgpu-native** if the
131
134
  engine were Rust.
132
135
 
133
- > These vendored libraries are linked **statically into `gui.a`** so a released
134
- > xbintsc stays "download and run"; only OS frameworks are added by
136
+ > These vendored libraries are linked **statically into `gui.a` / `media.a`** so
137
+ > a released xbintsc stays "download and run"; only OS frameworks are added by
135
138
  > `linkerFlags()`.
136
139
 
137
140
  ## Milestones
@@ -172,15 +175,16 @@ engine were Rust.
172
175
  animated `background-color`/`color`/`border-color`/`border-radius`, retargeting
173
176
  and `win.advance(ms)`. `@keyframes` remain.
174
177
  7. **M7 — CI & releases** ✅ (Linux/macOS build) / 🚧 (run + Windows)
175
- - `compile-examples` builds `gui.a` on Linux and macOS (required), then runs
176
- the example and `tests/e2e/gui-*.test.ts` under Xvfb + lavapipe on Linux
177
- (required). The macOS run is provisional until a WindowServer is confirmed;
178
- it is guarded so a failure is logged without annotating the run.
179
- - The `package` job builds `gui.a` before assembling the release, so it ships
180
- inside the existing runtime archive (`package-release` copies all of
181
- `runtime/`).
182
- - `vendor/` (SDL3/FreeType/HarfBuzz, the slow part) is cached per OS/arch,
183
- keyed by `scripts/build-gui.ts`.
178
+ - `compile-examples` builds `gui.a` and `media.a` on Linux and macOS
179
+ (required), then runs the examples and `tests/e2e/gui-*.test.ts` under
180
+ Xvfb + lavapipe on Linux (required). The macOS run is provisional until a
181
+ WindowServer is confirmed; it is guarded so a failure is logged without
182
+ annotating the run.
183
+ - The `package` job builds `gui.a` and `media.a` before assembling the
184
+ release, so both ship inside the existing runtime archive
185
+ (`package-release` copies all of `runtime/`).
186
+ - `vendor/` (SDL3/FreeType/HarfBuzz for `gui.a`, FFmpeg for `media.a`) is
187
+ cached per OS/arch, keyed by `scripts/build-gui.ts` / `scripts/build-media.ts`.
184
188
  - Windows (build + run) stays provisional in CI: the MSVC-compatible
185
189
  `gui.lib` builds and runs locally through SDL_GPU's Vulkan backend, but the
186
190
  hosted runners have no GPU/Vulkan driver. A failure is logged without
@@ -234,13 +238,15 @@ engine were Rust.
234
238
  `tests/e2e/gui-retention.test.ts` (see [Memory & GC](./gui/runtime.md#memory--gc)).
235
239
  13. **M13 — video & audio** ✅
236
240
  - `<video>` / `<audio>` decode through a statically vendored minimal FFmpeg
237
- (`media.*`, `gui_media.cpp`), frames upload to GPU textures through the
238
- existing image pipeline, audio is pushed to an SDL stream on the main
239
- thread, and the DOM media API plus `loadedmetadata`/`play`/`pause`/
240
- `ended`/`timeupdate`/`error` events are exposed on element handles
241
+ in the optional `media` extension (`runtime/ext_media/media.cpp`), frames
242
+ upload to GPU textures through the existing image pipeline, audio is
243
+ pushed to an SDL stream on the main thread by the GUI's audio sink, and
244
+ the DOM media API plus `loadedmetadata`/`play`/`pause`/`ended`/
245
+ `timeupdate`/`error` events are exposed on element handles
241
246
  (`dom_api_media.cpp`, `dom_api_proto.cpp`). Controlled from TS + CSS;
242
247
  there are no built-in controls. Example in `examples/gui/media.*`, e2e in
243
- `tests/e2e/gui-media.test.ts`.
248
+ `tests/e2e/gui-media.test.ts` (with the `media` extension) and
249
+ `tests/e2e/gui-media-disabled.test.ts` (without it).
244
250
  14. **M14 — `<canvas>` 2D** ✅
245
251
  - `<canvas>` is a replaced element (`xtgui::Canvas2D`, `layout.*`), the
246
252
  `getContext("2d")` handle and drawing API live in `dom_api_canvas.cpp`,
@@ -300,10 +306,11 @@ engine were Rust.
300
306
  pixel size and cached by that size, so scaling and HiDPI stay crisp.
301
307
  - **M6b** ✅ CSS transitions (parsing in `style.*`, an animation clock and
302
308
  transition state in `document.*`, `advance`/`advance(ms)` hooks, e2e coverage).
303
- - **M7** ✅ builds `gui.a` in CI on Linux and macOS (required) and runs the
304
- example plus the GUI e2e suite under Xvfb on Linux; `package` ships `gui.a`,
305
- `vendor/` is cached, and the `ar -M` merge works on GNU/Linux and macOS.
306
- macOS runs and all of Windows remain provisional in CI.
309
+ - **M7** ✅ builds `gui.a` (and the optional `media.a`) in CI on Linux and macOS
310
+ (required) and runs the example plus the GUI e2e suite under Xvfb on Linux;
311
+ `package` ships both archives, `vendor/` is cached, and the `ar -M` merge
312
+ works on GNU/Linux and macOS. macOS runs and all of Windows remain provisional
313
+ in CI.
307
314
  - **M8** ✅ element/document handles (`dom_api.*`), DOM mutation with lazy
308
315
  restyle/relayout (`document.*`, `xt_gui_flush_dom`), element event dispatch
309
316
  with capture/bubble and `stopPropagation` (`dom_api.*`, `gui.cpp`), and e2e
@@ -338,14 +345,17 @@ engine were Rust.
338
345
  ratcheting upwards. `win.stats()` exposes the engine counters and
339
346
  `tests/e2e/gui-retention.test.ts` pins them; a game loop rewriting its sprite
340
347
  every frame now runs in constant memory (see [Memory & GC](./gui/runtime.md#memory--gc)).
341
- - **M13** ✅ `<video>` / `<audio>` support. A minimal static FFmpeg is vendored
342
- and merged into `gui.a` (`scripts/build-gui.ts`), `media.*` decodes video to
343
- RGBA frames and audio to 48 kHz stereo PCM on the main thread, `gui_media.cpp`
344
- owns the players and pumps the media clock and events each frame,
348
+ - **M13** ✅ `<video>` / `<audio>` support. The optional `media` extension
349
+ vendors and builds a minimal static FFmpeg (`scripts/build-media.ts`) and
350
+ merges it into `media.a`; `runtime/ext_media/media.cpp` decodes video to RGBA
351
+ frames and audio to 48 kHz stereo PCM on the main thread,
352
+ `runtime/ext_gui/gui_media.cpp` owns the players, implements the SDL audio
353
+ sink and pumps the media clock and events each frame,
345
354
  `renderer_image.cpp` uploads each new video frame through the image texture
346
355
  path, and `dom_api_media.cpp` exposes `play`/`pause`/`load` plus the media
347
- properties on element handles. `examples/gui/media.*` demonstrates the
348
- TS-plus-CSS-controls pattern and `tests/e2e/gui-media.test.ts` pins it.
356
+ properties on element handles. `gui.a` stays FFmpeg-free; enabling `media`
357
+ without `gui` is rejected by the registry. `examples/gui/media.*` demonstrates
358
+ the TS-plus-CSS-controls pattern and `tests/e2e/gui-media.test.ts` pins it.
349
359
  - **M14** ✅ `<canvas>` support. A new replaced element (`layout.*`) carries an
350
360
  intrinsic size from its `width`/`height` attributes (default `300x150`),
351
361
  `xtgui::Canvas2D` (`canvas.{h,cpp}`) owns the RGBA8 bitmap and drawing state,
@@ -5,33 +5,39 @@
5
5
  > 属于 [GUI 扩展](../gui.md) 的一部分;引擎总览、锁定决策与路线图在那里。
6
6
 
7
7
  - 源码位于 `runtime/ext_gui/`(C/C++),外加 `vendor/` 下的 vendored 库
8
- (被 gitignore;按需获取)。
8
+ (被 gitignore;按需获取)。媒体解码位于 `runtime/ext_media/`,单独构建。
9
9
  - `npm run gui`(`scripts/build-gui.ts`)获取固定版本的 SDL3(`SDL3_TAG`,
10
10
  默认 `release-3.2.10`),构建静态 SDL3,获取并构建静态 FreeType
11
11
  (`FREETYPE_VERSION`)与 HarfBuzz(`HARFBUZZ_VERSION`),编译引擎,并把
12
12
  所有内容合并到 `runtime/lib/<os>-<arch>/gui.a`(Windows 上 MSVC ABI 则为
13
- `gui.lib`),与 `core.a` 采用相同约定,静态打包三者。它还会获取、配置并构建
14
- 一个最小化的静态 **FFmpeg**(`FFMPEG_VERSION`,默认 `7.1`),只启用媒体元素
15
- 所需的 demuxer/parser/decoder,并把那些归档合并进 `gui.a`,同时为引擎定义
16
- `XT_GUI_WITH_FFMPEG`。合并使用 macOS 上的
13
+ `gui.lib`),与 `core.a` 采用相同约定,静态打包三者。合并使用 macOS 上的
17
14
  `libtool` 与其他平台上的 `ar -M`(GNU/LLVM);CMake 归档会在构建根目录或
18
15
  `Release/` 下按任一生成器风格被发现。
19
- - `src/extensions/gui/index.ts` 通过 `nativeObjects()` 暴露该归档(经由
16
+ - `npm run media`(`scripts/build-media.ts`)构建可选的 **FFmpeg** 后端。它获取
17
+ FFmpeg(`FFMPEG_VERSION`,默认 `7.1`),配置一个只启用媒体元素所需
18
+ demuxer/parser/decoder 的最小静态构建,用它编译
19
+ `runtime/ext_media/media.cpp`,并把所有内容合并到
20
+ `runtime/lib/<os>-<arch>/media.a`。`gui.a` **不会**链接 FFmpeg:程序通过
21
+ `--ext media` 显式启用播放能力。
22
+ - `src/extensions/gui/index.ts` 通过 `nativeObjects()` 暴露引擎归档(经由
20
23
  `findRuntimeLibrary`,因此 `.a`/`.lib` 都可用),并通过 `linkerFlags()`
21
- 暴露操作系统框架。
24
+ 暴露操作系统框架。`src/extensions/media/index.ts` 对 `media.a` 做同样的暴露,
25
+ 并把注册桥(`runtime/ext_media/media_bridge.c`)作为普通 runtime 源编译。
22
26
  - `scripts/build-gui.ts` 使用与驱动**相同**的 clang
23
27
  (`src/driver/toolchain-provider.ts`);Windows 上就是 Visual Studio 自带的、
24
28
  面向 MSVC 的 LLVM,并配套使用其同目录下的 `clang++`/`llvm-ar` 来配置 CMake、
25
29
  编译引擎和合并归档。脚本不提供 `gcc`/`g++` 回退,Windows 上也始终使用
26
30
  `llvm-ar`,因此归档不会混入其他目标文件格式或 C++ 运行时。Visual Studio
27
31
  工具链按绝对路径查找,无需开发者命令提示符。Windows 上 SDL_GPU 会选择 Vulkan
28
- 后端,不需要 DXIL/`dxc` 工具链。
29
- - CI 在组装发布归档之前构建 `gui.a`,因此它会随 `runtime/lib/<slug>/` 一起
30
- 发布(发布 tarball 会复制整个 `runtime/` 树)。归档在 Linux(Xvfb 下,
31
- 使用 lavapipe 软件 Vulkan 驱动)与 macOS 上构建并运行示例;Windows 为
32
- 试验性,因为托管 runner 没有 GPU/Vulkan 驱动。
33
- - 现有的缓存指纹已经会对 `nativeObjects()` 的内容做哈希,因此重建 `gui.a`
34
- 会自动使缓存二进制失效。
32
+ 后端,不需要 DXIL/`dxc` 工具链。`scripts/build-media.ts` 对 FFmpeg 与播放器
33
+ 使用同一套已解析的工具链。
34
+ - CI 在组装发布归档之前构建 `gui.a` 与 `media.a`,因此它们会随
35
+ `runtime/lib/<slug>/` 一起发布(发布 tarball 会复制整个 `runtime/` 树)。
36
+ 归档在 Linux(Xvfb 下,使用 lavapipe 软件 Vulkan 驱动)与 macOS 上构建并运行
37
+ 示例;Windows 为试验性,因为托管 runner 没有 GPU/Vulkan 驱动,且 media 归档
38
+ 目前仅支持 POSIX。
39
+ - 现有的缓存指纹已经会对 `nativeObjects()` 的内容与 `runtimeSources()` 的内容
40
+ 做哈希,因此重建任一归档都会自动使缓存二进制失效。
35
41
  - CMake 构建树位于其源码**旁边**(`vendor/sdl-build`、`vendor/freetype-build`、
36
42
  `vendor/harfbuzz-build`),绝不放在源码内部:`CMakeCache.txt` 记录的是配置它
37
43
  的那台机器,而 SDL 的视频后端是 `option()`,因此先运行的那次配置会一直生效。
@@ -47,6 +53,7 @@
47
53
  ```sh
48
54
  npm run runtime # runtime 改动后重建 core.a
49
55
  npm run gui # 构建 runtime/lib/<os>-<arch>/gui.a(首次会获取 SDL3)
56
+ npm run media # 可选:构建 media.a(首次会获取并构建 FFmpeg)
50
57
  xbintsc run examples/gui/hello.ts --ext gui
51
58
  ```
52
59
 
@@ -59,8 +66,9 @@ xbintsc run examples/gui/hello.ts --ext gui
59
66
 
60
67
  `examples/gui/media.ts`(配合 `media.html`)演示 `<video>`/`<audio>`:它在
61
68
  `<video>` 上叠一个 CSS 进度条,并用 `play()`/`pause()`/`currentTime`/
62
- `volume` 驱动播放。媒体 `src` 相对于进程工作目录解析,因此从仓库根目录运行:
63
- `xbintsc run examples/gui/media.ts --ext gui`。几何报告见该目录的 README。
69
+ `volume` 驱动播放。它需要单独的可选 `media` 扩展,因此要构建并同时启用两者。
70
+ 媒体 `src` 相对于进程工作目录解析,因此从仓库根目录运行:
71
+ `xbintsc run examples/gui/media.ts --ext gui,media`。几何报告见该目录的 README。
64
72
 
65
73
  这些测试套件在 Windows 上默认跳过(归档可以在本机通过 Vulkan 运行,但 CI
66
74
  runner 没有可用的 GPU 驱动)。如果本机有 `gui.lib` 与可用的 GPU,可用
@@ -77,10 +85,14 @@ npx vitest run tests/e2e/gui-layout.test.ts tests/e2e/gui-example.test.ts
77
85
  ```sh
78
86
  sudo sh scripts/install-gui-deps.sh # clang、cmake、X11/Wayland 头文件、Vulkan、Xvfb
79
87
  npm run runtime && npm run gui
88
+ npm run media # 可选:<video>/<audio> 播放
80
89
  xvfb-run -a --server-args="-screen 0 1280x720x24" \
81
90
  npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
82
91
  ```
83
92
 
93
+ 媒体 e2e 套件(`tests/e2e/gui-media.test.ts`)只有在 `gui.a` 与 `media.a` 都
94
+ 存在时才会运行;`tests/e2e/gui-media-disabled.test.ts` 覆盖仅 `gui.a` 的构建。
95
+
84
96
  包清单集中在 `scripts/install-gui-deps.sh`(`apt-get install -y clang cmake
85
97
  libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev libxinerama-dev
86
98
  libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols libdecor-0-dev
@@ -54,11 +54,12 @@
54
54
  SDL_GPU 会选择 **Vulkan** 后端(`win.driver()` 返回 `"vulkan"`),因此归档
55
55
  不需要 DXIL/`dxc` 工具链;托管 Windows runner 没有 GPU/Vulkan 驱动,这才是
56
56
  CI 运行仍为试验性的原因。
57
- - **Windows 上的媒体:** 媒体解码器(FFmpeg)只构建并合并进 POSIX 的
58
- `gui.a`;Windows 的 `gui.lib` 不含它们,因此那里的 `<video>`/`<audio>` 为空
59
- 操作(失败会触发 `error`)。启用它需要 MSVC 兼容的 FFmpeg 构建,推迟到
60
- Windows GUI 路径本身验证之后。`<video controls>` 同样推迟——预期模式是 TS +
61
- CSS 叠层,如 `examples/gui/media.ts`。
57
+ - **Windows 上的媒体:** `media` 扩展目前仅支持 POSIX。GUI 引擎本身可在
58
+ Windows 上构建,但没有该扩展时 `<video>`/`<audio>` 为空操作(打开失败并触发
59
+ `error`),`--ext media` 会给出可操作的错误而不是继续链接。启用它需要
60
+ MSVC 兼容的 FFmpeg 构建,推迟到 Windows GUI 路径本身验证之后。
61
+ `<video controls>` 同样推迟——预期模式是 TS + CSS 叠层,如
62
+ `examples/gui/media.ts`。
62
63
  - **D3D12 着色器:** SPIR-V(Vulkan)与 DXIL(Direct3D 12)二进制块均已构建并
63
64
  内嵌,非 Metal 路径不再跳过几何绘制。但在 **SDL 3.2.10** 上 DXIL 仍不可用:
64
65
  其 D3D12 后端会拒绝任何着色器中声明了 uniform buffer 的图形管线,而引擎正是
@@ -55,25 +55,36 @@ opacity、`display`——与任何其他元素一样。盒子内部的艺术内
55
55
 
56
56
  ## 视频与音频(M13)
57
57
 
58
- `runtime/ext_gui/media.{h,cpp}` 封装一个静态 vendored 的 **FFmpeg** 构建,
59
- `runtime/ext_gui/gui_media.cpp` 把 DOM 树接到它上面。`<video>` 与 `<audio>`
60
- 真正解码并播放;路径中没有 WebView,也没有平台媒体框架。
58
+ 媒体播放横跨两个扩展,因此只做界面的 GUI 程序不必为用不到的解码器付出体积:
59
+
60
+ - **`gui`**(`gui.a`)负责 `<video>`/`<audio>` 元素、DOM API、渲染器集成与 SDL
61
+ 音频 sink,但**不含 FFmpeg**。`runtime/ext_gui/media.{h,cpp}` 声明播放器接口,
62
+ 并通过媒体后端在启动时安装的函数表分发(`runtime/ext_gui/media_api.h`)。
63
+ - **`media`**(`media.a`、`runtime/ext_media/`)是可选的解码后端:
64
+ `runtime/ext_media/media.cpp` 封装静态 vendored 的 **FFmpeg** 构建,
65
+ `runtime/ext_media/media_bridge.c` 把它注册进引擎。用 `--ext media`(或
66
+ `--ext gui,media`)启用。`media` 依赖 `gui`;注册表会拒绝错误的组合。
67
+
68
+ 没有 `media` 扩展时,元素仍然存在、DOM API 仍可调用,但打开源会失败并触发
69
+ `error`。Windows 目前也是如此:该归档仅支持 POSIX。两条路径都没有 WebView,也
70
+ 没有平台媒体框架。
61
71
 
62
72
  - **解码。** `avformat_open_input` 打开 `src`(是路径——接受 `file://`,但与
63
73
  图像不同,媒体路径*不做*百分号解码),为每种媒体选取最佳流,软件解码器
64
74
  产出 `yuv/rgb` 帧,`sws_scale` 按编码尺寸转换为 RGBA8。支持的容器/编码即
65
- `scripts/build-gui.ts` 的最小 FFmpeg 配置:MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/
75
+ `scripts/build-media.ts` 的最小 FFmpeg 配置:MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/
66
76
  ASF,MP3/OGG/WAV/FLAC/AAC;视频 H.264、HEVC、VP8、VP9、AV1、MPEG-4、MJPEG;
67
77
  音频 AAC、MP3、Opus、Vorbis、FLAC、ALAC、AC3/E-AC3、WMAv2 及原始 PCM 格式。
68
78
  - **视频。** `<video>` 与 `<img>` 一样是替换元素(UA 默认 `300x150`,解码
69
79
  尺寸通过 `setMediaSize` 覆盖)。绘制产出与图像相同的 `PaintImage`;渲染器把
70
80
  每个新解码帧上传到 RGBA 纹理(`ensureMediaTexture`)并绘制带纹理四边形,
71
- 因此由 GPU 缩放到 CSS 盒,而非 CPU 缩放。播放计时基于墙钟
72
- (`SDL_GetTicks`),不是帧间隔。
73
- - **音频。** 没有音频回调:解码器随主线程 tick 被推送,PCM 以 push 模式
74
- (`NULL` 回调)通过 `SDL_OpenAudioDeviceStream` 排队,在前方缓冲约 0.25 秒,
75
- 格式为 48 kHz 立体声 `SDL_AUDIO_F32`。`volume` 映射到
76
- `SDL_SetAudioStreamGain`;音频是尽力而为,无设备时(无头 CI)跳过。
81
+ 因此由 GPU 缩放到 CSS 盒,而非 CPU 缩放。播放计时基于进程单调时钟,不是帧间隔。
82
+ - **音频。** 没有音频回调:解码器随主线程 tick 被推送,并写入
83
+ `xtgui::AudioSink`。GUI 在 `gui_media.cpp` 中用 push 模式(`NULL` 回调)的
84
+ `SDL_OpenAudioDeviceStream` 实现该 sink,在前方缓冲约 0.25 秒,格式为
85
+ 48 kHz 立体声 `SDL_AUDIO_F32`。`volume` 映射到 `SDL_SetAudioStreamGain`;
86
+ 音频是尽力而为,无设备时(无头 CI)跳过。把 sink 留在 `gui.a` 中意味着
87
+ `media.a` 只依赖 FFmpeg。
77
88
  - **线程。** 引擎是单线程且非线程感知,因此所有解码都在主线程的
78
89
  `xt_gui_media_tick` 内完成——没有解码线程,也没有跨线程 GC 暴露。
79
90
  - **DOM API。** 元素句柄直接暴露媒体接口(见