xbintsc 0.5.21 → 0.5.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (98) hide show
  1. package/AGENTS.md +1 -1
  2. package/README.md +15 -5
  3. package/README.zh-CN.md +11 -4
  4. package/dist/src/cli/hints.js +11 -0
  5. package/dist/src/cli/hints.js.map +1 -1
  6. package/dist/src/cli/main.js +1 -1
  7. package/dist/src/driver/compiler.js +1 -0
  8. package/dist/src/driver/compiler.js.map +1 -1
  9. package/dist/src/driver/version.d.ts +1 -1
  10. package/dist/src/driver/version.js +1 -1
  11. package/dist/src/extensions/catalog.js +2 -1
  12. package/dist/src/extensions/catalog.js.map +1 -1
  13. package/dist/src/extensions/media/index.d.ts +27 -0
  14. package/dist/src/extensions/media/index.js +60 -0
  15. package/dist/src/extensions/media/index.js.map +1 -0
  16. package/dist/src/extensions/registry.d.ts +12 -0
  17. package/dist/src/extensions/registry.js +16 -0
  18. package/dist/src/extensions/registry.js.map +1 -1
  19. package/dist/src/index.d.ts +1 -0
  20. package/dist/src/index.js +1 -0
  21. package/dist/src/index.js.map +1 -1
  22. package/dist/tests/cli/hints.test.js +15 -0
  23. package/dist/tests/cli/hints.test.js.map +1 -1
  24. package/dist/tests/cli/process.test.js +12 -0
  25. package/dist/tests/cli/process.test.js.map +1 -1
  26. package/dist/tests/e2e/gui-canvas.test.js +159 -0
  27. package/dist/tests/e2e/gui-canvas.test.js.map +1 -1
  28. package/dist/tests/e2e/gui-example-canvas.test.d.ts +11 -0
  29. package/dist/tests/e2e/gui-example-canvas.test.js +72 -0
  30. package/dist/tests/e2e/gui-example-canvas.test.js.map +1 -0
  31. package/dist/tests/e2e/gui-helpers.d.ts +3 -0
  32. package/dist/tests/e2e/gui-helpers.js +2 -0
  33. package/dist/tests/e2e/gui-helpers.js.map +1 -1
  34. package/dist/tests/e2e/gui-media-disabled.test.d.ts +9 -0
  35. package/dist/tests/e2e/gui-media-disabled.test.js +42 -0
  36. package/dist/tests/e2e/gui-media-disabled.test.js.map +1 -0
  37. package/dist/tests/e2e/gui-media.test.d.ts +6 -4
  38. package/dist/tests/e2e/gui-media.test.js +15 -12
  39. package/dist/tests/e2e/gui-media.test.js.map +1 -1
  40. package/dist/tests/e2e/unicode-output.test.d.ts +16 -0
  41. package/dist/tests/e2e/unicode-output.test.js +65 -0
  42. package/dist/tests/e2e/unicode-output.test.js.map +1 -0
  43. package/dist/tests/extensions/media.test.d.ts +1 -0
  44. package/dist/tests/extensions/media.test.js +58 -0
  45. package/dist/tests/extensions/media.test.js.map +1 -0
  46. package/dist/tests/extensions/registry.test.js +8 -0
  47. package/dist/tests/extensions/registry.test.js.map +1 -1
  48. package/dist/tests/lexer/identifiers.test.js +12 -0
  49. package/dist/tests/lexer/identifiers.test.js.map +1 -1
  50. package/dist/tests/lexer/scanner.test.js +8 -0
  51. package/dist/tests/lexer/scanner.test.js.map +1 -1
  52. package/dist/tests/lexer/strings.test.js +30 -0
  53. package/dist/tests/lexer/strings.test.js.map +1 -1
  54. package/doc/ai/cli.md +1 -1
  55. package/doc/ai/extensions.md +35 -22
  56. package/doc/ai/troubleshooting.md +9 -0
  57. package/doc/ai/zh-CN/cli.md +1 -1
  58. package/doc/ai/zh-CN/extensions.md +22 -9
  59. package/doc/ai/zh-CN/troubleshooting.md +8 -0
  60. package/doc/gui/build.md +37 -21
  61. package/doc/gui/canvas.md +47 -21
  62. package/doc/gui/known-issues.md +7 -6
  63. package/doc/gui/media.md +26 -11
  64. package/doc/gui.md +50 -34
  65. package/doc/zh-CN/gui/build.md +28 -16
  66. package/doc/zh-CN/gui/canvas.md +36 -17
  67. package/doc/zh-CN/gui/known-issues.md +6 -5
  68. package/doc/zh-CN/gui/media.md +21 -10
  69. package/doc/zh-CN/gui.md +43 -34
  70. package/package.json +2 -1
  71. package/runtime/ext_gui/canvas.cpp +114 -199
  72. package/runtime/ext_gui/canvas.h +101 -2
  73. package/runtime/ext_gui/canvas_gradient.cpp +112 -0
  74. package/runtime/ext_gui/canvas_internal.h +6 -6
  75. package/runtime/ext_gui/canvas_raster.cpp +27 -5
  76. package/runtime/ext_gui/canvas_stroke.cpp +252 -0
  77. package/runtime/ext_gui/canvas_style.cpp +240 -0
  78. package/runtime/ext_gui/dom_api.cpp +1 -0
  79. package/runtime/ext_gui/dom_api_canvas.cpp +38 -4
  80. package/runtime/ext_gui/dom_api_canvas_extra.cpp +232 -0
  81. package/runtime/ext_gui/dom_api_internal.h +31 -0
  82. package/runtime/ext_gui/dom_api_proto.cpp +24 -0
  83. package/runtime/ext_gui/gui_media.cpp +69 -2
  84. package/runtime/ext_gui/media.cpp +34 -488
  85. package/runtime/ext_gui/media.h +54 -19
  86. package/runtime/ext_gui/media_api.h +65 -0
  87. package/runtime/ext_media/media.cpp +507 -0
  88. package/runtime/ext_media/media_bridge.c +35 -0
  89. package/scripts/build-gui.ts +6 -72
  90. package/scripts/build-media.ts +245 -0
  91. package/src/cli/hints.ts +11 -0
  92. package/src/cli/main.ts +1 -1
  93. package/src/driver/compiler.ts +1 -0
  94. package/src/driver/version.ts +1 -1
  95. package/src/extensions/catalog.ts +2 -1
  96. package/src/extensions/media/index.ts +67 -0
  97. package/src/extensions/registry.ts +24 -0
  98. package/src/index.ts +1 -0
package/doc/gui/canvas.md CHANGED
@@ -13,8 +13,11 @@ video frames (`renderer_image.cpp`), so the GPU scales it into the CSS box.
13
13
 
14
14
  The engine-side object is `xtgui::Canvas2D`
15
15
  (`runtime/ext_gui/canvas.{h,cpp}`) — one RGBA8 bitmap plus the 2D drawing state.
16
- The rasteriser lives in `canvas_raster.cpp`; the DOM bindings live in
17
- `dom_api_canvas.cpp`, and the per-window lifetime in `gui_canvas.cpp`.
16
+ The rasteriser lives in `canvas_raster.cpp`, stroke geometry (caps, joins,
17
+ dashes) in `canvas_stroke.cpp`, gradients in `canvas_gradient.cpp`, and drawing
18
+ state parsing in `canvas_style.cpp`; the DOM bindings live in
19
+ `dom_api_canvas.cpp` / `dom_api_canvas_extra.cpp`, and the per-window lifetime in
20
+ `gui_canvas.cpp`.
18
21
 
19
22
  ## Getting a context
20
23
 
@@ -37,6 +40,14 @@ horizontal coverage** pass, and fills paths with the **non-zero winding rule**
37
40
  (`b == 0 && c == 0` in the transform) takes a direct coverage fast path.
38
41
  Blending is non-premultiplied source-over, scaled by `globalAlpha`.
39
42
 
43
+ A fill or stroke paints with a solid colour **or a `CanvasGradient`**. Gradient
44
+ coordinates are user-space coordinates, so the rasteriser maps each device pixel
45
+ back through the inverse of the current transform before sampling; this keeps a
46
+ gradient attached to the shape when the shape is translated, scaled or rotated.
47
+ Strokes are flattened into segment quads plus the join/cap shapes selected by
48
+ `lineJoin`/`lineCap`; a dash pattern first splits the polyline into its "on"
49
+ pieces, then those pieces are stroked with the same caps.
50
+
40
51
  Canvas pixels are **not** touched by the garbage collector: a canvas is owned by
41
52
  the window, keyed by its DOM node (the same shape as the `<video>`/`<audio>`
42
53
  decode table). It is released when the node is dropped (`loadHTML`, window
@@ -48,24 +59,39 @@ matching GPU texture.
48
59
  State and transforms: `save` / `restore` (which restore the transform, line
49
60
  width, alpha, font/text settings **and** the fill/stroke styles),
50
61
  `translate`, `rotate`, `scale`, `transform(a,b,c,d,e,f)`, `setTransform`,
51
- `resetTransform`.
62
+ `resetTransform`, `getTransform` (returns an object with `a`…`f` and `is2D`).
63
+ Transforms compose in canvas order: `translate(x, y); rotate(a)` rotates around
64
+ the translated origin.
52
65
 
53
66
  Styles and settings: `fillStyle` / `strokeStyle` (CSS colors understood by the
54
- cascade's color parser), `lineWidth`, `globalAlpha`, `font` (a CSS font
55
- shorthand resolving to family, size and bold/italic — default
56
- `10px sans-serif`), `textAlign` (`start`/`left`, `center`, `right`/`end`) and
57
- `textBaseline` (`alphabetic`, `top`, `middle`, `bottom`, `hanging`,
58
- `ideographic`). Reading a property returns the value last assigned; assigning an
59
- unparseable color keeps the previous one.
60
-
61
- Paths: `beginPath`, `closePath`, `moveTo`, `lineTo`, `rect`, `arc`,
62
- `quadraticCurveTo`, `bezierCurveTo`, then `fill` (multi-loop, non-zero winding)
63
- or `stroke`. Bezier and quadratic segments are flattened into line segments;
64
- arcs are flattened too and each subpath is stroked as segment quads plus a disc
65
- at each joint.
67
+ cascade's color parser, or a `CanvasGradient`), `lineWidth`, `globalAlpha`,
68
+ `lineCap` (`butt`/`round`/`square`), `lineJoin` (`miter`/`round`/`bevel`),
69
+ `miterLimit`, `lineDashOffset`, `font` (a CSS font shorthand resolving to
70
+ family, size and bold/italic — default `10px sans-serif`), `textAlign`
71
+ (`start`/`left`, `center`, `right`/`end`) and `textBaseline` (`alphabetic`,
72
+ `top`, `middle`, `bottom`, `hanging`, `ideographic`). Reading a property returns
73
+ the value last assigned; assigning an unparseable color keeps the previous one.
74
+
75
+ Gradients: `createLinearGradient(x0, y0, x1, y1)` and
76
+ `createRadialGradient(x0, y0, r0, x1, y1, r1)` return a `CanvasGradient` with
77
+ `addColorStop(offset, color)`. Stops stay live, so a stop added after
78
+ `fillStyle = gradient` affects later fills, and the `fillStyle`/`strokeStyle`
79
+ getter returns the same gradient handle. Radial gradients interpolate by
80
+ distance from the **outer** circle's centre (`x1, y1`), so `x0`/`y0` are
81
+ currently ignored: use concentric circles (`x0 == x1`, `y0 == y1`).
82
+
83
+ Paths: `beginPath`, `closePath`, `moveTo`, `lineTo`, `rect`, `roundRect`,
84
+ `arc`, `ellipse`, `quadraticCurveTo`, `bezierCurveTo`, then `fill` (multi-loop,
85
+ non-zero winding) or `stroke`. Bezier and quadratic segments are flattened into
86
+ line segments; arcs and ellipses are flattened too. `roundRect` accepts one
87
+ radius number (the array form is not implemented).
66
88
 
67
89
  Rectangles: `clearRect`, `fillRect`, `strokeRect`.
68
90
 
91
+ Dashes: `setLineDash(segments)` / `getLineDash()` / `lineDashOffset` implement
92
+ the standard pattern rules (an odd-length list is duplicated, negative or
93
+ non-finite entries are ignored, an all-zero list means a solid line).
94
+
69
95
  Text: `measureText(text)` (returns `{ width }`), `fillText(text, x, y)` and
70
96
  `strokeText(text, x, y)`. Text is shaped with HarfBuzz and rasterised with
71
97
  FreeType (the same stack as HTML text) at the transform's approximate scale.
@@ -87,12 +113,12 @@ Out-of-bounds pixels are transparent.
87
113
 
88
114
  ## Not implemented
89
115
 
90
- Gradients and patterns, shadows, `clip()`, `globalCompositeOperation`, line
91
- joins/caps/dashes/mitre, `setLineDash`, `createLinearGradient`/`createPattern`,
92
- `putImageData`, `createImageData`, `isPointInPath`, `getTransform`,
93
- `setTransform` with a `DOMMatrix`, `drawImage` with an image smoothing toggle,
94
- `toDataURL`/`toBlob`, `<canvas>` as a CSS `background-image`, and offscreen
95
- canvases (`OffscreenCanvas`, `transferControlToOffscreen`).
116
+ Patterns (`createPattern`), shadows, `clip()`, `globalCompositeOperation`,
117
+ `putImageData`, `createImageData`, `isPointInPath`, `setTransform` with a
118
+ `DOMMatrix`, `drawImage` with an image smoothing toggle, `toDataURL`/`toBlob`,
119
+ `<canvas>` as a CSS `background-image`, and offscreen canvases
120
+ (`OffscreenCanvas`, `transferControlToOffscreen`). `roundRect` takes a single
121
+ radius, and radial gradients ignore the inner circle's offset.
96
122
 
97
123
  See also [Media](./media.md) for the image/video/audio decode stack that
98
124
  `drawImage` and the texture upload share.
@@ -69,12 +69,13 @@
69
69
  (`win.driver()` reports `"vulkan"`), so the archive needs no DXIL/`dxc`
70
70
  toolchain; the hosted Windows runners have no GPU/Vulkan driver, which is why
71
71
  the CI run stays provisional.
72
- - **Media on Windows:** the media decoders (FFmpeg) are only built and merged
73
- into the POSIX `gui.a`; the Windows `gui.lib` is produced without them, so
74
- `<video>`/`<audio>` are inert there (failures fire `error`). Enabling it needs
75
- an MSVC-compatible FFmpeg build and is deferred until the Windows GUI path
76
- itself is validated. `<video controls>` is likewise deferred — the intended
77
- pattern is TS + a CSS overlay, as in `examples/gui/media.ts`.
72
+ - **Media on Windows:** the `media` extension is POSIX-only for now. The GUI
73
+ engine itself builds on Windows, but without the extension `<video>`/`<audio>`
74
+ are inert there (an open fails and fires `error`), and `--ext media` reports an
75
+ actionable error instead of linking. Enabling it needs an MSVC-compatible
76
+ FFmpeg build and is deferred until the Windows GUI path itself is validated.
77
+ `<video controls>` is likewise deferred — the intended pattern is TS + a CSS
78
+ overlay, as in `examples/gui/media.ts`.
78
79
  - **D3D12 shaders:** the SPIR-V (Vulkan) and DXIL (Direct3D 12) blobs are built
79
80
  and embedded, so the non-Metal path no longer skips geometry. DXIL remains
80
81
  unusable on **SDL 3.2.10**: its D3D12 backend rejects any graphics pipeline
package/doc/gui/media.md CHANGED
@@ -65,17 +65,31 @@ clip paths are not rendered (see [Known issues](./known-issues.md)).
65
65
 
66
66
  ## Video & audio (M13)
67
67
 
68
- `runtime/ext_gui/media.{h,cpp}` wraps a statically vendored **FFmpeg** build and
69
- `runtime/ext_gui/gui_media.cpp` connects the DOM tree to it. `<video>` and
70
- `<audio>` really decode and play; there is no WebView and no platform media
71
- framework in the path.
68
+ Media playback is split across two extensions so a GUI program does not pay for
69
+ a decoder it never uses:
70
+
71
+ - **`gui`** (`gui.a`) owns the `<video>`/`<audio>` elements, the DOM API, the
72
+ renderer integration and the SDL audio sink, but contains **no FFmpeg**.
73
+ `runtime/ext_gui/media.{h,cpp}` declares the player interface and dispatches
74
+ through a table the media backend installs at startup
75
+ (`runtime/ext_gui/media_api.h`).
76
+ - **`media`** (`media.a`, `runtime/ext_media/`) is the optional decoder
77
+ backend: `runtime/ext_media/media.cpp` wraps a statically vendored **FFmpeg**
78
+ build, and `runtime/ext_media/media_bridge.c` registers it with the engine.
79
+ Enable it with `--ext media` (or `--ext gui,media`). `media` requires `gui`;
80
+ the registry rejects the combination in the wrong order.
81
+
82
+ Without the `media` extension the element still exists and its DOM API is
83
+ callable, but opening a source fails and fires `error`. That is also how Windows
84
+ behaves today: the archive is POSIX-only. There is no WebView and no platform
85
+ media framework in the path either way.
72
86
 
73
87
  - **Decoding.** `avformat_open_input` opens the `src` (a path — `file://` is
74
88
  accepted, but media paths are *not* percent-decoded, unlike images), the
75
89
  best stream is selected for each kind, and a software decoder yields
76
90
  `yuv/rgb` frames which `sws_scale` converts to RGBA8 at the coded size. The
77
91
  supported containers/codecs are the minimal FFmpeg configuration from
78
- `scripts/build-gui.ts`: MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/ASF, MP3/OGG/WAV/FLAC/
92
+ `scripts/build-media.ts`: MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/ASF, MP3/OGG/WAV/FLAC/
79
93
  AAC; H.264, HEVC, VP8, VP9, AV1, MPEG-4, MJPEG video; AAC, MP3, Opus, Vorbis,
80
94
  FLAC, ALAC, AC3/E-AC3, WMAv2 and the raw PCM formats audio.
81
95
  - **Video.** `<video>` is a replaced element like `<img>` (UA default
@@ -83,13 +97,14 @@ framework in the path.
83
97
  same `PaintImage` as an image; the renderer uploads each new decoded frame to
84
98
  an RGBA texture (`ensureMediaTexture`) and draws a textured quad, so the GPU
85
99
  scales it to the CSS box instead of the CPU resizing. Playback timing is the
86
- wall clock (`SDL_GetTicks`), not the frame delta.
100
+ process monotonic clock, not the frame delta.
87
101
  - **Audio.** There is no audio callback: the decoder is pushed on the main
88
- thread as it is ticked and the PCM is queued with
89
- `SDL_OpenAudioDeviceStream` in push mode (`NULL` callback), buffering about
90
- 0.25 s ahead at 48 kHz stereo `SDL_AUDIO_F32`. `volume` maps to
91
- `SDL_SetAudioStreamGain`; audio is best-effort and is skipped when no device
92
- is available (headless CI).
102
+ thread as it is ticked and writes into an `xtgui::AudioSink`. The GUI
103
+ implements that sink in `gui_media.cpp` with `SDL_OpenAudioDeviceStream` in
104
+ push mode (`NULL` callback), buffering about 0.25 s ahead at 48 kHz stereo
105
+ `SDL_AUDIO_F32`. `volume` maps to `SDL_SetAudioStreamGain`; audio is
106
+ best-effort and is skipped when no device is available (headless CI). Keeping
107
+ the sink in `gui.a` means `media.a` depends only on FFmpeg.
93
108
  - **Threading.** The engine is single-threaded and not thread-aware, so all
94
109
  decoding happens on the main thread inside `xt_gui_media_tick` — no decode
95
110
  thread, no cross-thread GC exposure.
package/doc/gui.md CHANGED
@@ -21,9 +21,11 @@ what `examples/gui/pelican-bike` — a playable 2D game — is built on. **M12**
21
21
  DOM node lifetime explicit (what a mutation releases, what it keeps) and gives the
22
22
  engine bounded memory: see [Node lifetime](./gui/dom.md#node-lifetime) and
23
23
  [Memory & GC](./gui/runtime.md#memory--gc). **M13** plays real
24
- `<video>`/`<audio>` — a vendored minimal FFmpeg decodes them on the main thread,
25
- video frames go through the image texture path and audio to an SDL stream, with
26
- the DOM media API exposing `play`/`pause`/`currentTime`/… and media events.
24
+ `<video>`/`<audio>` — the optional `media` extension decodes them on the main
25
+ thread with a vendored minimal FFmpeg, video frames go through the image texture
26
+ path and audio to an SDL stream, with the DOM media API exposing
27
+ `play`/`pause`/`currentTime`/… and media events. Enabling `gui` alone does not
28
+ link FFmpeg.
27
29
  **M14** adds `<canvas>` with a self-written 2D software rasteriser (anti-aliased
28
30
  fills, paths, text and `drawImage`) uploaded through the image texture path —
29
31
  see [Canvas](./gui/canvas.md). **M15** adds CSS `transform` /
@@ -84,7 +86,7 @@ pages above.
84
86
  | 4 | The engine is **not self-hosted** (it is C/C++, not TS), but must not affect the compiler's platform-agnostic design. Delivered as a per-platform prebuilt archive linked via `nativeObjects`. |
85
87
  | 5 | The **event loop is generic**: the runtime exposes a generic main-loop hook and a poll primitive; nothing GUI-specific enters `runtime/`. |
86
88
  | 6 | **Multiple windows** are supported by the core object model. |
87
- | 7 | Third-party low-level libs are **statically linked into `gui.a`** so releases stay self-contained; only OS frameworks are added at link time. |
89
+ | 7 | Third-party low-level libs are **statically linked into the extension archives** so releases stay self-contained; only OS frameworks are added at link time. `gui.a` carries SDL3/FreeType/HarfBuzz, and the optional `media.a` carries FFmpeg and is linked only with `--ext media`. |
88
90
 
89
91
  ## Architecture
90
92
 
@@ -106,6 +108,7 @@ pages above.
106
108
  windowing + input (SDL3) ── multiple windows
107
109
  text (HarfBuzz + FreeType) ── glyph atlas / SDF
108
110
  images (stb_image; SVG via NanoSVG)
111
+ media DOM + audio sink ── optional media.a (FFmpeg) installs the decoder
109
112
  ```
110
113
 
111
114
  The engine never talks to the compiler. The compiler only sees an `Extension`
@@ -120,18 +123,18 @@ with `nativeObjects()`, `linkerFlags()` and `modules()`.
120
123
  | Text shaping | **HarfBuzz** | correct complex-script shaping |
121
124
  | Glyph raster | **FreeType** | glyph outlines → GPU atlas / SDF |
122
125
  | Images | **stb_image** (vendored header) for bitmaps, **NanoSVG** (vendored headers) for SVG | single headers to start; SVG files rasterise on demand |
123
- | Video + audio | **FFmpeg** (statically vendored, minimal build) | decodes the common container/codec set; no WebView or platform media framework |
126
+ | Video + audio | **FFmpeg** (statically vendored, minimal build, optional `media` extension) | decodes the common container/codec set; no WebView or platform media framework |
124
127
 
125
128
  Pinned versions: SDL3 `release-3.2.10`, FreeType `2.13.3`, HarfBuzz `10.1.0`,
126
- FFmpeg `7.1` (overridable with `SDL3_TAG` / `FREETYPE_VERSION` /
127
- `HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
129
+ FFmpeg `7.1` for the optional `media` archive (overridable with `SDL3_TAG` /
130
+ `FREETYPE_VERSION` / `HARFBUZZ_VERSION` / `FFMPEG_VERSION`).
128
131
 
129
132
  Alternative if SDL3 is rejected: **GLFW + OpenGL 3.3** (simpler, but OpenGL is
130
133
  deprecated on macOS and gives no modern GPU abstraction). **wgpu-native** if the
131
134
  engine were Rust.
132
135
 
133
- > These vendored libraries are linked **statically into `gui.a`** so a released
134
- > xbintsc stays "download and run"; only OS frameworks are added by
136
+ > These vendored libraries are linked **statically into `gui.a` / `media.a`** so
137
+ > a released xbintsc stays "download and run"; only OS frameworks are added by
135
138
  > `linkerFlags()`.
136
139
 
137
140
  ## Milestones
@@ -172,15 +175,16 @@ engine were Rust.
172
175
  animated `background-color`/`color`/`border-color`/`border-radius`, retargeting
173
176
  and `win.advance(ms)`. `@keyframes` remain.
174
177
  7. **M7 — CI & releases** ✅ (Linux/macOS build) / 🚧 (run + Windows)
175
- - `compile-examples` builds `gui.a` on Linux and macOS (required), then runs
176
- the example and `tests/e2e/gui-*.test.ts` under Xvfb + lavapipe on Linux
177
- (required). The macOS run is provisional until a WindowServer is confirmed;
178
- it is guarded so a failure is logged without annotating the run.
179
- - The `package` job builds `gui.a` before assembling the release, so it ships
180
- inside the existing runtime archive (`package-release` copies all of
181
- `runtime/`).
182
- - `vendor/` (SDL3/FreeType/HarfBuzz, the slow part) is cached per OS/arch,
183
- keyed by `scripts/build-gui.ts`.
178
+ - `compile-examples` builds `gui.a` and `media.a` on Linux and macOS
179
+ (required), then runs the examples and `tests/e2e/gui-*.test.ts` under
180
+ Xvfb + lavapipe on Linux (required). The macOS run is provisional until a
181
+ WindowServer is confirmed; it is guarded so a failure is logged without
182
+ annotating the run.
183
+ - The `package` job builds `gui.a` and `media.a` before assembling the
184
+ release, so both ship inside the existing runtime archive
185
+ (`package-release` copies all of `runtime/`).
186
+ - `vendor/` (SDL3/FreeType/HarfBuzz for `gui.a`, FFmpeg for `media.a`) is
187
+ cached per OS/arch, keyed by `scripts/build-gui.ts` / `scripts/build-media.ts`.
184
188
  - Windows (build + run) stays provisional in CI: the MSVC-compatible
185
189
  `gui.lib` builds and runs locally through SDL_GPU's Vulkan backend, but the
186
190
  hosted runners have no GPU/Vulkan driver. A failure is logged without
@@ -234,13 +238,15 @@ engine were Rust.
234
238
  `tests/e2e/gui-retention.test.ts` (see [Memory & GC](./gui/runtime.md#memory--gc)).
235
239
  13. **M13 — video & audio** ✅
236
240
  - `<video>` / `<audio>` decode through a statically vendored minimal FFmpeg
237
- (`media.*`, `gui_media.cpp`), frames upload to GPU textures through the
238
- existing image pipeline, audio is pushed to an SDL stream on the main
239
- thread, and the DOM media API plus `loadedmetadata`/`play`/`pause`/
240
- `ended`/`timeupdate`/`error` events are exposed on element handles
241
+ in the optional `media` extension (`runtime/ext_media/media.cpp`), frames
242
+ upload to GPU textures through the existing image pipeline, audio is
243
+ pushed to an SDL stream on the main thread by the GUI's audio sink, and
244
+ the DOM media API plus `loadedmetadata`/`play`/`pause`/`ended`/
245
+ `timeupdate`/`error` events are exposed on element handles
241
246
  (`dom_api_media.cpp`, `dom_api_proto.cpp`). Controlled from TS + CSS;
242
247
  there are no built-in controls. Example in `examples/gui/media.*`, e2e in
243
- `tests/e2e/gui-media.test.ts`.
248
+ `tests/e2e/gui-media.test.ts` (with the `media` extension) and
249
+ `tests/e2e/gui-media-disabled.test.ts` (without it).
244
250
  14. **M14 — `<canvas>` 2D** ✅
245
251
  - `<canvas>` is a replaced element (`xtgui::Canvas2D`, `layout.*`), the
246
252
  `getContext("2d")` handle and drawing API live in `dom_api_canvas.cpp`,
@@ -300,10 +306,11 @@ engine were Rust.
300
306
  pixel size and cached by that size, so scaling and HiDPI stay crisp.
301
307
  - **M6b** ✅ CSS transitions (parsing in `style.*`, an animation clock and
302
308
  transition state in `document.*`, `advance`/`advance(ms)` hooks, e2e coverage).
303
- - **M7** ✅ builds `gui.a` in CI on Linux and macOS (required) and runs the
304
- example plus the GUI e2e suite under Xvfb on Linux; `package` ships `gui.a`,
305
- `vendor/` is cached, and the `ar -M` merge works on GNU/Linux and macOS.
306
- macOS runs and all of Windows remain provisional in CI.
309
+ - **M7** ✅ builds `gui.a` (and the optional `media.a`) in CI on Linux and macOS
310
+ (required) and runs the example plus the GUI e2e suite under Xvfb on Linux;
311
+ `package` ships both archives, `vendor/` is cached, and the `ar -M` merge
312
+ works on GNU/Linux and macOS. macOS runs and all of Windows remain provisional
313
+ in CI.
307
314
  - **M8** ✅ element/document handles (`dom_api.*`), DOM mutation with lazy
308
315
  restyle/relayout (`document.*`, `xt_gui_flush_dom`), element event dispatch
309
316
  with capture/bubble and `stopPropagation` (`dom_api.*`, `gui.cpp`), and e2e
@@ -338,14 +345,17 @@ engine were Rust.
338
345
  ratcheting upwards. `win.stats()` exposes the engine counters and
339
346
  `tests/e2e/gui-retention.test.ts` pins them; a game loop rewriting its sprite
340
347
  every frame now runs in constant memory (see [Memory & GC](./gui/runtime.md#memory--gc)).
341
- - **M13** ✅ `<video>` / `<audio>` support. A minimal static FFmpeg is vendored
342
- and merged into `gui.a` (`scripts/build-gui.ts`), `media.*` decodes video to
343
- RGBA frames and audio to 48 kHz stereo PCM on the main thread, `gui_media.cpp`
344
- owns the players and pumps the media clock and events each frame,
348
+ - **M13** ✅ `<video>` / `<audio>` support. The optional `media` extension
349
+ vendors and builds a minimal static FFmpeg (`scripts/build-media.ts`) and
350
+ merges it into `media.a`; `runtime/ext_media/media.cpp` decodes video to RGBA
351
+ frames and audio to 48 kHz stereo PCM on the main thread,
352
+ `runtime/ext_gui/gui_media.cpp` owns the players, implements the SDL audio
353
+ sink and pumps the media clock and events each frame,
345
354
  `renderer_image.cpp` uploads each new video frame through the image texture
346
355
  path, and `dom_api_media.cpp` exposes `play`/`pause`/`load` plus the media
347
- properties on element handles. `examples/gui/media.*` demonstrates the
348
- TS-plus-CSS-controls pattern and `tests/e2e/gui-media.test.ts` pins it.
356
+ properties on element handles. `gui.a` stays FFmpeg-free; enabling `media`
357
+ without `gui` is rejected by the registry. `examples/gui/media.*` demonstrates
358
+ the TS-plus-CSS-controls pattern and `tests/e2e/gui-media.test.ts` pins it.
349
359
  - **M14** ✅ `<canvas>` support. A new replaced element (`layout.*`) carries an
350
360
  intrinsic size from its `width`/`height` attributes (default `300x150`),
351
361
  `xtgui::Canvas2D` (`canvas.{h,cpp}`) owns the RGBA8 bitmap and drawing state,
@@ -354,7 +364,13 @@ engine were Rust.
354
364
  `getContext("2d")` handle and its API are bound in `dom_api_canvas.cpp`, and
355
365
  `gui_canvas.cpp` owns the per-window lifetime. The bitmap uploads through the
356
366
  image texture path (`renderer_image.cpp`) and the display list carries a
357
- `canvas_id` (`paint.*`). See [Canvas](./gui/canvas.md); e2e in
367
+ `canvas_id` (`paint.*`). The drawing surface was later widened with linear /
368
+ radial gradients (`canvas_gradient.cpp`), `ellipse`/`roundRect`, line
369
+ caps/joins/miter limits and dash patterns (`canvas_style.cpp`,
370
+ `canvas_stroke.cpp`), and `getTransform`; `transform` now post-multiplies, so
371
+ `translate(x, y); rotate(a)` rotates around the translated origin, and
372
+ `examples/gui/pelican-canvas` draws an animated pelican on a bicycle entirely
373
+ with these APIs. See [Canvas](./gui/canvas.md); e2e in
358
374
  `tests/e2e/gui-canvas.test.ts`.
359
375
  - **M15** ✅ CSS `transform` / `transform-origin`. A new translation unit
360
376
  (`transform.{h,cpp}`) parses `translate`/`translateX`/`translateY`/`scale`/
@@ -5,33 +5,39 @@
5
5
  > 属于 [GUI 扩展](../gui.md) 的一部分;引擎总览、锁定决策与路线图在那里。
6
6
 
7
7
  - 源码位于 `runtime/ext_gui/`(C/C++),外加 `vendor/` 下的 vendored 库
8
- (被 gitignore;按需获取)。
8
+ (被 gitignore;按需获取)。媒体解码位于 `runtime/ext_media/`,单独构建。
9
9
  - `npm run gui`(`scripts/build-gui.ts`)获取固定版本的 SDL3(`SDL3_TAG`,
10
10
  默认 `release-3.2.10`),构建静态 SDL3,获取并构建静态 FreeType
11
11
  (`FREETYPE_VERSION`)与 HarfBuzz(`HARFBUZZ_VERSION`),编译引擎,并把
12
12
  所有内容合并到 `runtime/lib/<os>-<arch>/gui.a`(Windows 上 MSVC ABI 则为
13
- `gui.lib`),与 `core.a` 采用相同约定,静态打包三者。它还会获取、配置并构建
14
- 一个最小化的静态 **FFmpeg**(`FFMPEG_VERSION`,默认 `7.1`),只启用媒体元素
15
- 所需的 demuxer/parser/decoder,并把那些归档合并进 `gui.a`,同时为引擎定义
16
- `XT_GUI_WITH_FFMPEG`。合并使用 macOS 上的
13
+ `gui.lib`),与 `core.a` 采用相同约定,静态打包三者。合并使用 macOS 上的
17
14
  `libtool` 与其他平台上的 `ar -M`(GNU/LLVM);CMake 归档会在构建根目录或
18
15
  `Release/` 下按任一生成器风格被发现。
19
- - `src/extensions/gui/index.ts` 通过 `nativeObjects()` 暴露该归档(经由
16
+ - `npm run media`(`scripts/build-media.ts`)构建可选的 **FFmpeg** 后端。它获取
17
+ FFmpeg(`FFMPEG_VERSION`,默认 `7.1`),配置一个只启用媒体元素所需
18
+ demuxer/parser/decoder 的最小静态构建,用它编译
19
+ `runtime/ext_media/media.cpp`,并把所有内容合并到
20
+ `runtime/lib/<os>-<arch>/media.a`。`gui.a` **不会**链接 FFmpeg:程序通过
21
+ `--ext media` 显式启用播放能力。
22
+ - `src/extensions/gui/index.ts` 通过 `nativeObjects()` 暴露引擎归档(经由
20
23
  `findRuntimeLibrary`,因此 `.a`/`.lib` 都可用),并通过 `linkerFlags()`
21
- 暴露操作系统框架。
24
+ 暴露操作系统框架。`src/extensions/media/index.ts` 对 `media.a` 做同样的暴露,
25
+ 并把注册桥(`runtime/ext_media/media_bridge.c`)作为普通 runtime 源编译。
22
26
  - `scripts/build-gui.ts` 使用与驱动**相同**的 clang
23
27
  (`src/driver/toolchain-provider.ts`);Windows 上就是 Visual Studio 自带的、
24
28
  面向 MSVC 的 LLVM,并配套使用其同目录下的 `clang++`/`llvm-ar` 来配置 CMake、
25
29
  编译引擎和合并归档。脚本不提供 `gcc`/`g++` 回退,Windows 上也始终使用
26
30
  `llvm-ar`,因此归档不会混入其他目标文件格式或 C++ 运行时。Visual Studio
27
31
  工具链按绝对路径查找,无需开发者命令提示符。Windows 上 SDL_GPU 会选择 Vulkan
28
- 后端,不需要 DXIL/`dxc` 工具链。
29
- - CI 在组装发布归档之前构建 `gui.a`,因此它会随 `runtime/lib/<slug>/` 一起
30
- 发布(发布 tarball 会复制整个 `runtime/` 树)。归档在 Linux(Xvfb 下,
31
- 使用 lavapipe 软件 Vulkan 驱动)与 macOS 上构建并运行示例;Windows 为
32
- 试验性,因为托管 runner 没有 GPU/Vulkan 驱动。
33
- - 现有的缓存指纹已经会对 `nativeObjects()` 的内容做哈希,因此重建 `gui.a`
34
- 会自动使缓存二进制失效。
32
+ 后端,不需要 DXIL/`dxc` 工具链。`scripts/build-media.ts` 对 FFmpeg 与播放器
33
+ 使用同一套已解析的工具链。
34
+ - CI 在组装发布归档之前构建 `gui.a` 与 `media.a`,因此它们会随
35
+ `runtime/lib/<slug>/` 一起发布(发布 tarball 会复制整个 `runtime/` 树)。
36
+ 归档在 Linux(Xvfb 下,使用 lavapipe 软件 Vulkan 驱动)与 macOS 上构建并运行
37
+ 示例;Windows 为试验性,因为托管 runner 没有 GPU/Vulkan 驱动,且 media 归档
38
+ 目前仅支持 POSIX。
39
+ - 现有的缓存指纹已经会对 `nativeObjects()` 的内容与 `runtimeSources()` 的内容
40
+ 做哈希,因此重建任一归档都会自动使缓存二进制失效。
35
41
  - CMake 构建树位于其源码**旁边**(`vendor/sdl-build`、`vendor/freetype-build`、
36
42
  `vendor/harfbuzz-build`),绝不放在源码内部:`CMakeCache.txt` 记录的是配置它
37
43
  的那台机器,而 SDL 的视频后端是 `option()`,因此先运行的那次配置会一直生效。
@@ -47,6 +53,7 @@
47
53
  ```sh
48
54
  npm run runtime # runtime 改动后重建 core.a
49
55
  npm run gui # 构建 runtime/lib/<os>-<arch>/gui.a(首次会获取 SDL3)
56
+ npm run media # 可选:构建 media.a(首次会获取并构建 FFmpeg)
50
57
  xbintsc run examples/gui/hello.ts --ext gui
51
58
  ```
52
59
 
@@ -59,8 +66,9 @@ xbintsc run examples/gui/hello.ts --ext gui
59
66
 
60
67
  `examples/gui/media.ts`(配合 `media.html`)演示 `<video>`/`<audio>`:它在
61
68
  `<video>` 上叠一个 CSS 进度条,并用 `play()`/`pause()`/`currentTime`/
62
- `volume` 驱动播放。媒体 `src` 相对于进程工作目录解析,因此从仓库根目录运行:
63
- `xbintsc run examples/gui/media.ts --ext gui`。几何报告见该目录的 README。
69
+ `volume` 驱动播放。它需要单独的可选 `media` 扩展,因此要构建并同时启用两者。
70
+ 媒体 `src` 相对于进程工作目录解析,因此从仓库根目录运行:
71
+ `xbintsc run examples/gui/media.ts --ext gui,media`。几何报告见该目录的 README。
64
72
 
65
73
  这些测试套件在 Windows 上默认跳过(归档可以在本机通过 Vulkan 运行,但 CI
66
74
  runner 没有可用的 GPU 驱动)。如果本机有 `gui.lib` 与可用的 GPU,可用
@@ -77,10 +85,14 @@ npx vitest run tests/e2e/gui-layout.test.ts tests/e2e/gui-example.test.ts
77
85
  ```sh
78
86
  sudo sh scripts/install-gui-deps.sh # clang、cmake、X11/Wayland 头文件、Vulkan、Xvfb
79
87
  npm run runtime && npm run gui
88
+ npm run media # 可选:<video>/<audio> 播放
80
89
  xvfb-run -a --server-args="-screen 0 1280x720x24" \
81
90
  npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
82
91
  ```
83
92
 
93
+ 媒体 e2e 套件(`tests/e2e/gui-media.test.ts`)只有在 `gui.a` 与 `media.a` 都
94
+ 存在时才会运行;`tests/e2e/gui-media-disabled.test.ts` 覆盖仅 `gui.a` 的构建。
95
+
84
96
  包清单集中在 `scripts/install-gui-deps.sh`(`apt-get install -y clang cmake
85
97
  libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev libxinerama-dev
86
98
  libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols libdecor-0-dev
@@ -10,7 +10,9 @@
10
10
  CSS 盒中。
11
11
 
12
12
  引擎侧的对象是 `xtgui::Canvas2D`(`runtime/ext_gui/canvas.{h,cpp}`)——一块 RGBA8 位图
13
- 加上 2D 绘制状态。光栅器位于 `canvas_raster.cpp`;DOM 绑定位于 `dom_api_canvas.cpp`;
13
+ 加上 2D 绘制状态。光栅器位于 `canvas_raster.cpp`,描边几何(端点、连接、虚线)位于
14
+ `canvas_stroke.cpp`,渐变位于 `canvas_gradient.cpp`,绘制状态解析位于
15
+ `canvas_style.cpp`;DOM 绑定位于 `dom_api_canvas.cpp` / `dom_api_canvas_extra.cpp`;
14
16
  按窗口的生命周期位于 `gui_canvas.cpp`。
15
17
 
16
18
  ## 获取上下文
@@ -31,6 +33,11 @@ const ctx = canvas.getContext("2d"); // 其他上下文类型返回 null
31
33
  以相反方向环绕时能形成空洞)。轴对齐矩形(变换中 `b == 0 && c == 0`)走直接的覆盖率
32
34
  快速路径。混合为非预乘 source-over,并按 `globalAlpha` 缩放。
33
35
 
36
+ 填充或描边可以使用纯色,也可以使用 **`CanvasGradient`**。渐变的坐标属于用户空间,
37
+ 因此光栅器会先用当前变换的逆矩阵把每个设备像素映射回用户空间再采样——形状被平移、
38
+ 缩放或旋转时,渐变仍然贴在形状上。描边被展平为线段四边形,再按 `lineJoin`/`lineCap`
39
+ 补上连接/端点形状;虚线模式先把折线切成“实线”片段,然后用同样的端点规则描边。
40
+
34
41
  Canvas 像素**不参与垃圾回收**:canvas 由窗口拥有、以 DOM 节点为键(与
35
42
  `<video>`/`<audio>` 解码表结构相同)。当节点被丢弃(`loadHTML`、窗口关闭/退出,或丢弃
36
43
  子树的变更操作)时随之释放,并释放对应的 GPU 纹理。
@@ -39,21 +46,34 @@ Canvas 像素**不参与垃圾回收**:canvas 由窗口拥有、以 DOM 节点
39
46
 
40
47
  状态与变换:`save` / `restore`(会恢复变换、线宽、透明度、字体/文本设置**以及**
41
48
  填充/描边样式)、`translate`、`rotate`、`scale`、`transform(a,b,c,d,e,f)`、
42
- `setTransform`、`resetTransform`。
43
-
44
- 样式与设置:`fillStyle` / `strokeStyle`(层叠所用的颜色解析器能识别的 CSS 颜色)、
45
- `lineWidth`、`globalAlpha`、`font`(CSS 字体简写,解析出字体族、字号与粗体/斜体——
46
- 默认 `10px sans-serif`)、`textAlign`(`start`/`left`、`center`、`right`/`end`)与
47
- `textBaseline`(`alphabetic`、`top`、`middle`、`bottom`、`hanging`、`ideographic`)。
48
- 读取属性返回上次赋予的值;赋予无法解析的颜色会保留原值。
49
-
50
- 路径:`beginPath`、`closePath`、`moveTo`、`lineTo`、`rect`、`arc`、
51
- `quadraticCurveTo`、`bezierCurveTo`,然后 `fill`(多环、非零环绕)或 `stroke`。贝塞尔
52
- 与二次曲线段会被展平为线段;弧线同样被展平,每条子路径按线段四边形描边,并在每个连接
53
- 处补一个圆盘。
49
+ `setTransform`、`resetTransform`、`getTransform`(返回带 `a`…`f` 与 `is2D` 的对象)。
50
+ 变换按 canvas 的顺序组合:`translate(x, y); rotate(a)` 围绕平移后的原点旋转。
51
+
52
+ 样式与设置:`fillStyle` / `strokeStyle`(层叠所用的颜色解析器能识别的 CSS 颜色,或
53
+ `CanvasGradient`)、`lineWidth`、`globalAlpha`、`lineCap`(`butt`/`round`/`square`)、
54
+ `lineJoin`(`miter`/`round`/`bevel`)、`miterLimit`、`lineDashOffset`、`font`
55
+ (CSS 字体简写,解析出字体族、字号与粗体/斜体——默认 `10px sans-serif`)、
56
+ `textAlign`(`start`/`left`、`center`、`right`/`end`)与 `textBaseline`
57
+ (`alphabetic`、`top`、`middle`、`bottom`、`hanging`、`ideographic`)。读取属性返回
58
+ 上次赋予的值;赋予无法解析的颜色会保留原值。
59
+
60
+ 渐变:`createLinearGradient(x0, y0, x1, y1)` 与
61
+ `createRadialGradient(x0, y0, r0, x1, y1, r1)` 返回带
62
+ `addColorStop(offset, color)` 的 `CanvasGradient`。色标是**活的**:在
63
+ `fillStyle = gradient` 之后追加的色标会影响后续填充,且 `fillStyle`/`strokeStyle`
64
+ 的读取返回同一个渐变句柄。径向渐变按到**外圆**圆心(`x1, y1`)的距离插值,因此目前
65
+ 会忽略 `x0`/`y0`——请使用同心圆(`x0 == x1`、`y0 == y1`)。
66
+
67
+ 路径:`beginPath`、`closePath`、`moveTo`、`lineTo`、`rect`、`roundRect`、`arc`、
68
+ `ellipse`、`quadraticCurveTo`、`bezierCurveTo`,然后 `fill`(多环、非零环绕)或
69
+ `stroke`。贝塞尔与二次曲线段会被展平为线段;弧线与椭圆同样被展平。`roundRect` 只接受
70
+ 单个圆角半径数字(数组形式尚未实现)。
54
71
 
55
72
  矩形:`clearRect`、`fillRect`、`strokeRect`。
56
73
 
74
+ 虚线:`setLineDash(segments)` / `getLineDash()` / `lineDashOffset` 实现标准规则
75
+ (奇数长度列表会复制一遍,负数/非有限值会被忽略,全零列表表示实线)。
76
+
57
77
  文本:`measureText(text)`(返回 `{ width }`)、`fillText(text, x, y)` 与
58
78
  `strokeText(text, x, y)`。文本经 HarfBuzz 排版、FreeType 光栅化(与 HTML 文本相同的
59
79
  技术栈),按变换的近似缩放绘制。**仅支持平移与均匀缩放**——旋转或斜切变换仍会把每个
@@ -70,11 +90,10 @@ Canvas 像素**不参与垃圾回收**:canvas 由窗口拥有、以 DOM 节点
70
90
 
71
91
  ## 尚未实现
72
92
 
73
- 渐变与图案、阴影、`clip()`、`globalCompositeOperation`、线段连接/端点/虚线/斜接、
74
- `setLineDash`、`createLinearGradient`/`createPattern`、`putImageData`、
75
- `createImageData`、`isPointInPath`、`getTransform`、以 `DOMMatrix` 调用的
93
+ 图案(`createPattern`)、阴影、`clip()`、`globalCompositeOperation`、
94
+ `putImageData`、`createImageData`、`isPointInPath`、以 `DOMMatrix` 调用的
76
95
  `setTransform`、可切换图像平滑的 `drawImage`、`toDataURL`/`toBlob`、把 `<canvas>` 用作
77
96
  CSS `background-image`,以及离屏画布(`OffscreenCanvas`、
78
- `transferControlToOffscreen`)。
97
+ `transferControlToOffscreen`)。`roundRect` 只接受单个半径,径向渐变会忽略内圆偏移。
79
98
 
80
99
  另见[媒体](./media.md):`drawImage` 与纹理上传所共享的图像/视频/音频解码栈。
@@ -54,11 +54,12 @@
54
54
  SDL_GPU 会选择 **Vulkan** 后端(`win.driver()` 返回 `"vulkan"`),因此归档
55
55
  不需要 DXIL/`dxc` 工具链;托管 Windows runner 没有 GPU/Vulkan 驱动,这才是
56
56
  CI 运行仍为试验性的原因。
57
- - **Windows 上的媒体:** 媒体解码器(FFmpeg)只构建并合并进 POSIX 的
58
- `gui.a`;Windows 的 `gui.lib` 不含它们,因此那里的 `<video>`/`<audio>` 为空
59
- 操作(失败会触发 `error`)。启用它需要 MSVC 兼容的 FFmpeg 构建,推迟到
60
- Windows GUI 路径本身验证之后。`<video controls>` 同样推迟——预期模式是 TS +
61
- CSS 叠层,如 `examples/gui/media.ts`。
57
+ - **Windows 上的媒体:** `media` 扩展目前仅支持 POSIX。GUI 引擎本身可在
58
+ Windows 上构建,但没有该扩展时 `<video>`/`<audio>` 为空操作(打开失败并触发
59
+ `error`),`--ext media` 会给出可操作的错误而不是继续链接。启用它需要
60
+ MSVC 兼容的 FFmpeg 构建,推迟到 Windows GUI 路径本身验证之后。
61
+ `<video controls>` 同样推迟——预期模式是 TS + CSS 叠层,如
62
+ `examples/gui/media.ts`。
62
63
  - **D3D12 着色器:** SPIR-V(Vulkan)与 DXIL(Direct3D 12)二进制块均已构建并
63
64
  内嵌,非 Metal 路径不再跳过几何绘制。但在 **SDL 3.2.10** 上 DXIL 仍不可用:
64
65
  其 D3D12 后端会拒绝任何着色器中声明了 uniform buffer 的图形管线,而引擎正是
@@ -55,25 +55,36 @@ opacity、`display`——与任何其他元素一样。盒子内部的艺术内
55
55
 
56
56
  ## 视频与音频(M13)
57
57
 
58
- `runtime/ext_gui/media.{h,cpp}` 封装一个静态 vendored 的 **FFmpeg** 构建,
59
- `runtime/ext_gui/gui_media.cpp` 把 DOM 树接到它上面。`<video>` 与 `<audio>`
60
- 真正解码并播放;路径中没有 WebView,也没有平台媒体框架。
58
+ 媒体播放横跨两个扩展,因此只做界面的 GUI 程序不必为用不到的解码器付出体积:
59
+
60
+ - **`gui`**(`gui.a`)负责 `<video>`/`<audio>` 元素、DOM API、渲染器集成与 SDL
61
+ 音频 sink,但**不含 FFmpeg**。`runtime/ext_gui/media.{h,cpp}` 声明播放器接口,
62
+ 并通过媒体后端在启动时安装的函数表分发(`runtime/ext_gui/media_api.h`)。
63
+ - **`media`**(`media.a`、`runtime/ext_media/`)是可选的解码后端:
64
+ `runtime/ext_media/media.cpp` 封装静态 vendored 的 **FFmpeg** 构建,
65
+ `runtime/ext_media/media_bridge.c` 把它注册进引擎。用 `--ext media`(或
66
+ `--ext gui,media`)启用。`media` 依赖 `gui`;注册表会拒绝错误的组合。
67
+
68
+ 没有 `media` 扩展时,元素仍然存在、DOM API 仍可调用,但打开源会失败并触发
69
+ `error`。Windows 目前也是如此:该归档仅支持 POSIX。两条路径都没有 WebView,也
70
+ 没有平台媒体框架。
61
71
 
62
72
  - **解码。** `avformat_open_input` 打开 `src`(是路径——接受 `file://`,但与
63
73
  图像不同,媒体路径*不做*百分号解码),为每种媒体选取最佳流,软件解码器
64
74
  产出 `yuv/rgb` 帧,`sws_scale` 按编码尺寸转换为 RGBA8。支持的容器/编码即
65
- `scripts/build-gui.ts` 的最小 FFmpeg 配置:MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/
75
+ `scripts/build-media.ts` 的最小 FFmpeg 配置:MP4/MOV/MKV/AVI/FLV/MPEG-TS/M4V/
66
76
  ASF,MP3/OGG/WAV/FLAC/AAC;视频 H.264、HEVC、VP8、VP9、AV1、MPEG-4、MJPEG;
67
77
  音频 AAC、MP3、Opus、Vorbis、FLAC、ALAC、AC3/E-AC3、WMAv2 及原始 PCM 格式。
68
78
  - **视频。** `<video>` 与 `<img>` 一样是替换元素(UA 默认 `300x150`,解码
69
79
  尺寸通过 `setMediaSize` 覆盖)。绘制产出与图像相同的 `PaintImage`;渲染器把
70
80
  每个新解码帧上传到 RGBA 纹理(`ensureMediaTexture`)并绘制带纹理四边形,
71
- 因此由 GPU 缩放到 CSS 盒,而非 CPU 缩放。播放计时基于墙钟
72
- (`SDL_GetTicks`),不是帧间隔。
73
- - **音频。** 没有音频回调:解码器随主线程 tick 被推送,PCM 以 push 模式
74
- (`NULL` 回调)通过 `SDL_OpenAudioDeviceStream` 排队,在前方缓冲约 0.25 秒,
75
- 格式为 48 kHz 立体声 `SDL_AUDIO_F32`。`volume` 映射到
76
- `SDL_SetAudioStreamGain`;音频是尽力而为,无设备时(无头 CI)跳过。
81
+ 因此由 GPU 缩放到 CSS 盒,而非 CPU 缩放。播放计时基于进程单调时钟,不是帧间隔。
82
+ - **音频。** 没有音频回调:解码器随主线程 tick 被推送,并写入
83
+ `xtgui::AudioSink`。GUI 在 `gui_media.cpp` 中用 push 模式(`NULL` 回调)的
84
+ `SDL_OpenAudioDeviceStream` 实现该 sink,在前方缓冲约 0.25 秒,格式为
85
+ 48 kHz 立体声 `SDL_AUDIO_F32`。`volume` 映射到 `SDL_SetAudioStreamGain`;
86
+ 音频是尽力而为,无设备时(无头 CI)跳过。把 sink 留在 `gui.a` 中意味着
87
+ `media.a` 只依赖 FFmpeg。
77
88
  - **线程。** 引擎是单线程且非线程感知,因此所有解码都在主线程的
78
89
  `xt_gui_media_tick` 内完成——没有解码线程,也没有跨线程 GC 暴露。
79
90
  - **DOM API。** 元素句柄直接暴露媒体接口(见