xbintsc 0.3.69 → 0.3.81

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (115) hide show
  1. package/README.md +57 -4
  2. package/README.zh-CN.md +51 -2
  3. package/dist/src/driver/compiler.js +6 -1
  4. package/dist/src/driver/compiler.js.map +1 -1
  5. package/dist/src/extensions/gui/index.js +10 -2
  6. package/dist/src/extensions/gui/index.js.map +1 -1
  7. package/dist/tests/e2e/gui-advanced.test.d.ts +6 -0
  8. package/dist/tests/e2e/gui-advanced.test.js +261 -0
  9. package/dist/tests/e2e/gui-advanced.test.js.map +1 -0
  10. package/dist/tests/e2e/gui-canvas.test.d.ts +9 -0
  11. package/dist/tests/e2e/gui-canvas.test.js +233 -0
  12. package/dist/tests/e2e/gui-canvas.test.js.map +1 -0
  13. package/dist/tests/e2e/gui-example-svg.test.d.ts +21 -0
  14. package/dist/tests/e2e/gui-example-svg.test.js +105 -0
  15. package/dist/tests/e2e/gui-example-svg.test.js.map +1 -0
  16. package/dist/tests/e2e/gui-example.test.d.ts +1 -1
  17. package/dist/tests/e2e/gui-example.test.js +2 -2
  18. package/dist/tests/e2e/gui-helpers.d.ts +3 -0
  19. package/dist/tests/e2e/gui-helpers.js +5 -0
  20. package/dist/tests/e2e/gui-helpers.js.map +1 -1
  21. package/dist/tests/e2e/gui-layout.test.js +73 -0
  22. package/dist/tests/e2e/gui-layout.test.js.map +1 -1
  23. package/dist/tests/e2e/gui-media.test.d.ts +8 -0
  24. package/dist/tests/e2e/gui-media.test.js +119 -0
  25. package/dist/tests/e2e/gui-media.test.js.map +1 -0
  26. package/dist/tests/e2e/gui-paint.test.js +40 -0
  27. package/dist/tests/e2e/gui-paint.test.js.map +1 -1
  28. package/dist/tests/e2e/gui-transform.test.d.ts +7 -0
  29. package/dist/tests/e2e/gui-transform.test.js +198 -0
  30. package/dist/tests/e2e/gui-transform.test.js.map +1 -0
  31. package/doc/ai/cli.md +5 -0
  32. package/doc/ai/extensions.md +20 -3
  33. package/doc/ai/zh-CN/extensions.md +14 -3
  34. package/doc/gui/animation.md +97 -0
  35. package/doc/gui/api.md +92 -0
  36. package/doc/gui/build.md +91 -0
  37. package/doc/gui/canvas.md +98 -0
  38. package/doc/gui/dom.md +128 -0
  39. package/doc/gui/input.md +25 -0
  40. package/doc/gui/known-issues.md +78 -0
  41. package/doc/gui/layout.md +108 -0
  42. package/doc/gui/media.md +87 -0
  43. package/doc/gui/rendering.md +113 -0
  44. package/doc/gui/runtime.md +88 -0
  45. package/doc/gui-scripts.md +7 -3
  46. package/doc/gui.md +109 -612
  47. package/doc/implemented.md +1 -0
  48. package/doc/zh-CN/gui/animation.md +93 -0
  49. package/doc/zh-CN/gui/api.md +88 -0
  50. package/doc/zh-CN/gui/build.md +84 -0
  51. package/doc/zh-CN/gui/canvas.md +80 -0
  52. package/doc/zh-CN/gui/dom.md +119 -0
  53. package/doc/zh-CN/gui/input.md +23 -0
  54. package/doc/zh-CN/gui/known-issues.md +60 -0
  55. package/doc/zh-CN/gui/layout.md +91 -0
  56. package/doc/zh-CN/gui/media.md +71 -0
  57. package/doc/zh-CN/gui/rendering.md +92 -0
  58. package/doc/zh-CN/gui/runtime.md +76 -0
  59. package/doc/zh-CN/gui-scripts.md +6 -2
  60. package/doc/zh-CN/gui.md +89 -532
  61. package/doc/zh-CN/implemented.md +1 -0
  62. package/llms.txt +15 -2
  63. package/package.json +1 -1
  64. package/runtime/ext_gui/canvas.cpp +580 -0
  65. package/runtime/ext_gui/canvas.h +170 -0
  66. package/runtime/ext_gui/canvas_internal.h +59 -0
  67. package/runtime/ext_gui/canvas_raster.cpp +254 -0
  68. package/runtime/ext_gui/document.cpp +45 -4
  69. package/runtime/ext_gui/document.h +15 -1
  70. package/runtime/ext_gui/dom_api.cpp +3 -0
  71. package/runtime/ext_gui/dom_api_canvas.cpp +548 -0
  72. package/runtime/ext_gui/dom_api_internal.h +73 -0
  73. package/runtime/ext_gui/dom_api_media.cpp +274 -0
  74. package/runtime/ext_gui/dom_api_node.cpp +6 -4
  75. package/runtime/ext_gui/dom_api_proto.cpp +80 -0
  76. package/runtime/ext_gui/gui.cpp +6 -0
  77. package/runtime/ext_gui/gui_canvas.cpp +95 -0
  78. package/runtime/ext_gui/gui_engine.h +49 -0
  79. package/runtime/ext_gui/gui_events.cpp +21 -5
  80. package/runtime/ext_gui/gui_media.cpp +193 -0
  81. package/runtime/ext_gui/image.cpp +31 -12
  82. package/runtime/ext_gui/image.h +20 -7
  83. package/runtime/ext_gui/layout.cpp +130 -6
  84. package/runtime/ext_gui/layout.h +54 -1
  85. package/runtime/ext_gui/layout_flex.cpp +263 -99
  86. package/runtime/ext_gui/layout_flow.cpp +85 -9
  87. package/runtime/ext_gui/layout_flow_internal.h +84 -5
  88. package/runtime/ext_gui/layout_inline.cpp +54 -37
  89. package/runtime/ext_gui/media.cpp +514 -0
  90. package/runtime/ext_gui/media.h +104 -0
  91. package/runtime/ext_gui/nanosvg.h +3278 -0
  92. package/runtime/ext_gui/nanosvgrast.h +1472 -0
  93. package/runtime/ext_gui/paint.cpp +156 -38
  94. package/runtime/ext_gui/paint.h +42 -6
  95. package/runtime/ext_gui/renderer.cpp +117 -12
  96. package/runtime/ext_gui/renderer.h +49 -9
  97. package/runtime/ext_gui/renderer_image.cpp +205 -78
  98. package/runtime/ext_gui/renderer_internal.h +12 -9
  99. package/runtime/ext_gui/renderer_text.cpp +46 -39
  100. package/runtime/ext_gui/style.h +21 -0
  101. package/runtime/ext_gui/style_print.cpp +17 -0
  102. package/runtime/ext_gui/style_ua.cpp +7 -0
  103. package/runtime/ext_gui/style_values.cpp +83 -1
  104. package/runtime/ext_gui/svg.cpp +182 -0
  105. package/runtime/ext_gui/svg.h +46 -0
  106. package/runtime/ext_gui/text.cpp +156 -25
  107. package/runtime/ext_gui/text.h +11 -2
  108. package/runtime/ext_gui/transform.cpp +412 -0
  109. package/runtime/ext_gui/transform.h +112 -0
  110. package/runtime/ext_gui/window.cpp +10 -5
  111. package/scripts/build-gui.ts +148 -6
  112. package/scripts/check-file-length.ts +5 -1
  113. package/scripts/install-gui-deps.sh +28 -0
  114. package/src/driver/compiler.ts +6 -1
  115. package/src/extensions/gui/index.ts +10 -2
package/doc/gui/api.md ADDED
@@ -0,0 +1,92 @@
1
+ # GUI API
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there.
5
+
6
+ ## TS-facing API
7
+
8
+ ```ts
9
+ import { createWindow, run, quit } from "gui";
10
+
11
+ const win = createWindow({ title: "Demo", width: 900, height: 600 });
12
+ win.setBackground("#14161c");
13
+ win.loadHTML(INDEX_HTML); // parses HTML/CSS and computes styles
14
+ win.on("ready", () => console.log("first frame presented"));
15
+ win.on("close", () => console.log("window closed"));
16
+
17
+ run(); // drives the main loop until all windows close
18
+ ```
19
+
20
+ Methods implemented on a window handle: `setTitle` / `setSize` / `loadHTML` /
21
+ `getHTML` / `setBackground` / `close` / `isOpen` / `on` / `off`, plus `driver()`,
22
+ which reports the SDL_GPU backend in use (`"vulkan"`, `"direct3d12"`, `"metal"`,
23
+ … — SDL picks the best one the device supports). The Web spellings are available
24
+ too: `addEventListener` / `removeEventListener` are aliases of `on` / `off` (their
25
+ `options` argument is accepted and ignored) and the `closed` getter mirrors
26
+ `isOpen()`, so a document written against the browser API runs unchanged.
27
+ Events emitted:
28
+ `ready` (after the first presented frame), `load`, `close`, `resize` and the
29
+ input events `mousemove`, `mousedown`, `mouseup`, `click`, `wheel`, `keydown`,
30
+ `keyup`.
31
+
32
+ On Windows a GUI program is linked as a **Windows-subsystem** executable, so
33
+ launching it does not flash a console window. That also means stdout/stderr go
34
+ nowhere when it is started from Explorer; set `app.console: true` in
35
+ `xbintsc.config.json` to keep the console subsystem (useful while debugging).
36
+
37
+ Input handlers receive a single payload object (lifecycle handlers receive none):
38
+
39
+ ```ts
40
+ win.on("click", (e) => console.log(e.x, e.y, e.button, e.target));
41
+ win.on("wheel", (e) => console.log(e.deltaX, e.deltaY));
42
+ win.on("keydown", (e) => console.log(e.key, e.code, e.ctrl, e.shift, e.alt, e.meta));
43
+ ```
44
+
45
+ | Field | Events | Meaning |
46
+ | --- | --- | --- |
47
+ | `x`, `y` / `clientX`, `clientY` | pointer, wheel | viewport-relative logical pixels |
48
+ | `button` | pointer | `0` left, `1` middle, `2` right (`-1` for move) |
49
+ | `clicks` | pointer | click count reported by the OS |
50
+ | `deltaX`, `deltaY` | wheel | scroll amount (`deltaY` positive = down) |
51
+ | `key`, `code` | keyboard | key name and physical scancode name |
52
+ | `repeat`, `ctrl`, `shift`, `alt`, `meta` | keyboard | modifiers |
53
+ | `target` | pointer, wheel | deepest element under the point as a CSS-like descriptor (`div#main.card`), or absent |
54
+
55
+ ## Diagnostics
56
+
57
+ So the HTML/CSS/paint pipeline is testable without a GPU, a window handle
58
+ exposes read-only hooks:
59
+
60
+ ```ts
61
+ win.computedStyle(selector, property) // e.g. ("#main", "width") -> "60%"
62
+ win.queryCount(selector) // number of matching elements
63
+ win.getBoundingClientRect(selector) // { x, y, width, height } (border box, transformed)
64
+ win.documentTree() // serialized DOM (debugging)
65
+ win.layoutTree() // serialized layout boxes (debugging)
66
+ win.paintCount() // number of shapes in the display list
67
+ win.paintList() // serialized display list (debugging)
68
+ win.measureText(text, fontSize?, family?) // shaped advance width in pixels
69
+ win.fontMetrics(fontSize?, family?) // { ascent, descent, lineHeight, ready }
70
+ win.hitTest(x, y) // deepest element descriptor, or ""
71
+ win.sendEvent(type, options?) // synthesise input (testing)
72
+ win.advance(ms) // step the CSS transition clock (testing)
73
+ win.stats() // engine/runtime counters (memory debugging)
74
+ ```
75
+
76
+ `win.stats()` returns `{ heapBytes, heapAllocations, treeNodes, detachedRoots,`
77
+ `detachedNodes, handles, handleSlots, listeners }`: the live runtime heap, the
78
+ size of the DOM tree, the detached-node pool, and the element-handle tables. In a
79
+ steady state every one of them is constant — a climbing `detachedNodes` or
80
+ `handles` means the program removes nodes faster than it releases them (see
81
+ [Node lifetime](./dom.md#node-lifetime)).
82
+
83
+ They are used by `tests/e2e/gui-*.test.ts` to assert parsing, selector matching,
84
+ specificity, inheritance, `!important` and layout geometry. They will stay useful
85
+ afterwards for debugging.
86
+
87
+ ## Multiple windows
88
+
89
+ Multiple windows fall out of the object model: `createWindow` returns a native
90
+ object handle; each handle owns its own `SDL_Window`/GPU surface and its own DOM
91
+ tree. `run()` drives one shared main loop that ticks every window and exits when
92
+ the last one closes.
@@ -0,0 +1,91 @@
1
+ # Packaging and build
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there.
5
+
6
+ - Sources live in `runtime/ext_gui/` (C/C++) plus vendored libs under `vendor/`
7
+ (gitignored; fetched on demand).
8
+ - `npm run gui` (`scripts/build-gui.ts`) fetches a pinned SDL3 (`SDL3_TAG`,
9
+ default `release-3.2.10`), builds a static SDL3, fetches and builds static
10
+ FreeType (`FREETYPE_VERSION`) and HarfBuzz (`HARFBUZZ_VERSION`), compiles the
11
+ engine and merges everything into `runtime/lib/<os>-<arch>/gui.a` (or
12
+ `gui.lib` with the MSVC ABI on Windows), same convention as `core.a`,
13
+ statically bundling all three. It also fetches, configures and builds a
14
+ minimal static **FFmpeg** (`FFMPEG_VERSION`, default `7.1`) with only the
15
+ demuxers/parsers/decoders the media elements need and merges those archives
16
+ into `gui.a`, defining `XT_GUI_WITH_FFMPEG` for the engine. The merge uses
17
+ `libtool` on macOS and `ar -M` (GNU/LLVM) elsewhere; CMake archives are
18
+ discovered under the build root or `Release/` for either generator style.
19
+ - `src/extensions/gui/index.ts` exposes the archive through `nativeObjects()`
20
+ (via `findRuntimeLibrary`, so `.a`/`.lib` both work) and the OS frameworks
21
+ through `linkerFlags()`.
22
+ - CI builds `gui.a` before assembling the release archive so it ships inside
23
+ `runtime/lib/<slug>/` (the release tarball copies the whole `runtime/` tree).
24
+ The archive is built and the example is run on Linux (under Xvfb, with the
25
+ lavapipe software Vulkan driver) and macOS; Windows is provisional.
26
+ - The existing cache fingerprint already hashes `nativeObjects()` contents, so
27
+ rebuilding `gui.a` invalidates cached binaries automatically.
28
+ - CMake build trees live *next to* their sources (`vendor/sdl-build`,
29
+ `vendor/freetype-build`, `vendor/harfbuzz-build`), never inside them: a
30
+ `CMakeCache.txt` records the machine it was configured on, and SDL's video
31
+ backends are `option()`s, so the first configure to run wins. A cached
32
+ `vendor/SDL` tree that also carried `build/` therefore restored one machine's
33
+ `SDL_X11`/`SDL_WAYLAND` choices into another's configure — SDL then aborts
34
+ with "SDL could not find X11 or Wayland development libraries" even though the
35
+ headers are installed. The CI cache is keyed per OS/arch and omits the build
36
+ trees; `npm run gui` also checks the X11/Wayland headers up front on
37
+ Debian/Ubuntu and reports the missing development packages by name (set
38
+ `xbintsc_SKIP_HEADER_CHECK=1` when they live under a prefix that check does
39
+ not know).
40
+
41
+ ## Running the example
42
+
43
+ ```sh
44
+ npm run runtime # rebuild core.a after a runtime change
45
+ npm run gui # build runtime/lib/<os>-<arch>/gui.a (fetches SDL3 once)
46
+ xbintsc run examples/gui/hello.ts --ext gui
47
+ ```
48
+
49
+ Two examples live here: `hello.ts` (HTML/CSS + a reactive counter) and
50
+ [`pelican-bike/`](../../examples/gui/pelican-bike), a playable 2D game that uses
51
+ positioned layers, CSS sprites, `requestAnimationFrame` and keyboard input — see
52
+ its README for controls and the geometry report it prints.
53
+
54
+ `media.ts` (with `media.html`) demonstrates `<video>`/`<audio>`: it overlays a
55
+ CSS progress bar on the video and drives playback with `play()` / `pause()` /
56
+ `currentTime` / `volume`. Media `src` paths resolve relative to the process
57
+ working directory, so run it from the repository root:
58
+ `xbintsc run examples/gui/media.ts --ext gui`.
59
+
60
+ Set `XT_GUI_AUTOCLOSE_MS=<n>` to close all windows after `n` milliseconds,
61
+ which the e2e test (`tests/e2e/gui-*.test.ts`) uses to run headlessly.
62
+
63
+ Those suites are skipped on Windows by default (the archive is built there but
64
+ the run is not validated in CI). With a `gui.lib` and a working GPU, opt in with
65
+ `xbintsc_GUI_TESTS=1`:
66
+
67
+ ```bat
68
+ set xbintsc_GUI_TESTS=1
69
+ npx vitest run tests/e2e/gui-layout.test.ts tests/e2e/gui-example.test.ts
70
+ ```
71
+
72
+ On a headless Linux box, install the SDL3 build headers and run under Xvfb with a
73
+ software Vulkan driver:
74
+
75
+ ```sh
76
+ sudo sh scripts/install-gui-deps.sh # clang, cmake, X11/Wayland headers, Vulkan, Xvfb
77
+ npm run runtime && npm run gui
78
+ xvfb-run -a --server-args="-screen 0 1280x720x24" \
79
+ npx tsx src/cli/main.ts run examples/gui/hello.ts --ext gui
80
+ ```
81
+
82
+ `scripts/install-gui-deps.sh` holds the package list (`apt-get install -y clang
83
+ cmake libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev
84
+ libxinerama-dev libxfixes-dev libxkbcommon-dev libwayland-dev wayland-protocols
85
+ libdecor-0-dev libasound2-dev libpulse-dev libdbus-1-dev libudev-dev libdrm-dev
86
+ libgbm-dev libgl1-mesa-dev libegl1-mesa-dev libvulkan-dev mesa-vulkan-drivers
87
+ xvfb`) so that every Linux job which builds `gui.a` installs the same set: the
88
+ X11 backend is used by default, Wayland is enabled too but not yet exercised.
89
+ Install it before `npm run gui` anywhere the archive is built — SDL3 cannot
90
+ find a window backend without those headers.
91
+
@@ -0,0 +1,98 @@
1
+ # Canvas: `<canvas>` and the 2D context (M14)
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there.
5
+
6
+ `<canvas>` is a **replaced element** laid out like `<img>`: its intrinsic size
7
+ comes from the `width`/`height` attributes (default `300x150`, the CSS default
8
+ canvas size), an explicit CSS `width`/`height` wins, and the aspect ratio is
9
+ preserved when only one axis is constrained. Paint emits one `PaintImage` per
10
+ canvas, carrying a `canvas_id` instead of a `src`/`media_id`; the renderer
11
+ uploads the bitmap to an RGBA texture through the same image path as bitmaps and
12
+ video frames (`renderer_image.cpp`), so the GPU scales it into the CSS box.
13
+
14
+ The engine-side object is `xtgui::Canvas2D`
15
+ (`runtime/ext_gui/canvas.{h,cpp}`) — one RGBA8 bitmap plus the 2D drawing state.
16
+ The rasteriser lives in `canvas_raster.cpp`; the DOM bindings live in
17
+ `dom_api_canvas.cpp`, and the per-window lifetime in `gui_canvas.cpp`.
18
+
19
+ ## Getting a context
20
+
21
+ ```ts
22
+ const canvas: any = document.getElementById("c");
23
+ const ctx = canvas.getContext("2d"); // or null for any other context kind
24
+ ```
25
+
26
+ `getContext("2d")` returns a context handle whose methods and accessors live on
27
+ a shared prototype. The handle is cached on the element, so every call returns
28
+ the **same** object and the drawing state (transform, styles, path) is per
29
+ canvas, not per call. `ctx.canvas` is the owning element handle.
30
+
31
+ ## Drawing model
32
+
33
+ The rasteriser is deliberately small and self-written (no Skia/Cairo). It
34
+ anti-aliases with **4 vertical sub-scanlines** per pixel row and an **analytic
35
+ horizontal coverage** pass, and fills paths with the **non-zero winding rule**
36
+ (so `arc`-shaped holes work when wound the other way). An axis-aligned rectangle
37
+ (`b == 0 && c == 0` in the transform) takes a direct coverage fast path.
38
+ Blending is non-premultiplied source-over, scaled by `globalAlpha`.
39
+
40
+ Canvas pixels are **not** touched by the garbage collector: a canvas is owned by
41
+ the window, keyed by its DOM node (the same shape as the `<video>`/`<audio>`
42
+ decode table). It is released when the node is dropped (`loadHTML`, window
43
+ close/quit, or a mutation that discards the subtree), which also releases the
44
+ matching GPU texture.
45
+
46
+ ## Supported API
47
+
48
+ State and transforms: `save` / `restore` (which restore the transform, line
49
+ width, alpha, font/text settings **and** the fill/stroke styles),
50
+ `translate`, `rotate`, `scale`, `transform(a,b,c,d,e,f)`, `setTransform`,
51
+ `resetTransform`.
52
+
53
+ Styles and settings: `fillStyle` / `strokeStyle` (CSS colors understood by the
54
+ cascade's color parser), `lineWidth`, `globalAlpha`, `font` (a CSS font
55
+ shorthand resolving to family, size and bold/italic — default
56
+ `10px sans-serif`), `textAlign` (`start`/`left`, `center`, `right`/`end`) and
57
+ `textBaseline` (`alphabetic`, `top`, `middle`, `bottom`, `hanging`,
58
+ `ideographic`). Reading a property returns the value last assigned; assigning an
59
+ unparseable color keeps the previous one.
60
+
61
+ Paths: `beginPath`, `closePath`, `moveTo`, `lineTo`, `rect`, `arc`,
62
+ `quadraticCurveTo`, `bezierCurveTo`, then `fill` (multi-loop, non-zero winding)
63
+ or `stroke`. Bezier and quadratic segments are flattened into line segments;
64
+ arcs are flattened too and each subpath is stroked as segment quads plus a disc
65
+ at each joint.
66
+
67
+ Rectangles: `clearRect`, `fillRect`, `strokeRect`.
68
+
69
+ Text: `measureText(text)` (returns `{ width }`), `fillText(text, x, y)` and
70
+ `strokeText(text, x, y)`. Text is shaped with HarfBuzz and rasterised with
71
+ FreeType (the same stack as HTML text) at the transform's approximate scale.
72
+ **Only translation and uniform scale are honoured** — a rotated or skewed
73
+ transform still places each glyph at its transformed baseline point but does not
74
+ rotate the glyph bitmaps. `strokeText` currently fills with the stroke style
75
+ rather than producing an outline.
76
+
77
+ Images: `drawImage(source, …)` in the 3-argument (`dx, dy`), 5-argument
78
+ (`dx, dy, dw, dh`) and 9-argument (`dx, dy, dw, dh, sx, sy, sw, sh`) forms.
79
+ `source` may be an `<img>` element (decoded through the image module) or another
80
+ `<canvas>` element (its current bitmap). Sampling is nearest-neighbour, so a
81
+ large upscale of a small source is blocky — fine for icons and pixel art, not a
82
+ photo pipeline.
83
+
84
+ Test hook: `getImageData(x, y, w, h)` copies a region to
85
+ `{ width, height, data }` where `data` is a row-major RGBA8 byte array.
86
+ Out-of-bounds pixels are transparent.
87
+
88
+ ## Not implemented
89
+
90
+ Gradients and patterns, shadows, `clip()`, `globalCompositeOperation`, line
91
+ joins/caps/dashes/mitre, `setLineDash`, `createLinearGradient`/`createPattern`,
92
+ `putImageData`, `createImageData`, `isPointInPath`, `getTransform`,
93
+ `setTransform` with a `DOMMatrix`, `drawImage` with an image smoothing toggle,
94
+ `toDataURL`/`toBlob`, `<canvas>` as a CSS `background-image`, and offscreen
95
+ canvases (`OffscreenCanvas`, `transferControlToOffscreen`).
96
+
97
+ See also [Media](./media.md) for the image/video/audio decode stack that
98
+ `drawImage` and the texture upload share.
package/doc/gui/dom.md ADDED
@@ -0,0 +1,128 @@
1
+ # DOM, HTML, CSS and AOT scripts
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there. The scripting surface has its own
5
+ > document, [gui-scripts.md](../gui-scripts.md).
6
+
7
+ ## HTML/CSS subset (M3a)
8
+
9
+ **HTML parser** (`runtime/ext_gui/dom.{h,cpp}`): tags/attributes/text,
10
+ comments and doctype skipped, entity decoding (named + numeric/hex), void
11
+ elements, raw-text elements (`<style>`/`<script>`), and the common implicit
12
+ close rules (`li`, `dt`/`dd`, `option`, `p`, headings, table cells/rows).
13
+
14
+ **CSS parser** (`runtime/ext_gui/css.{h,cpp}`): comments and at-rules skipped
15
+ (for now), rules with multiple selectors, and declarations. Selectors: type,
16
+ `.class`, `#id`, attribute (`=`, `~=`, `|=`, `^=`, `$=`, `*=`), the four
17
+ combinators (descendant, child, adjacent and general sibling) and the
18
+ pseudo-classes `:first-child`, `:last-child`, `:only-child`, `:empty`,
19
+ `:root`, `:not(...)`, `:nth-child(an+b)`, `:disabled`, `:checked`, plus the
20
+ stateful `:hover` and `:focus` (see [Input](./input.md)). Values:
21
+ lengths (`px`, `%`, `em`, `rem`, `vw`, `vh`, `pt`, `pc`, `in`, `cm`, `mm`, `q`),
22
+ colors (hex, `rgb()`/`rgba()`, a named subset), numbers, keywords and shorthands
23
+ (`margin`/`padding`/`border`/`flex`). `transform` (`translate`/`translate3d`/
24
+ `translateX`/`translateY`, `scale`/`scale3d`/`scaleX`/`scaleY`, `rotate`/
25
+ `rotateZ`, `matrix`; the `3d` spellings' `z` argument is dropped) and
26
+ `transform-origin` are parsed into the computed style as a list of operations
27
+ plus two (unresolved) origin lengths; an unsupported function invalidates the
28
+ whole declaration ([Layout](./layout.md#transforms-m15)).
29
+
30
+ **Cascade & computed style** (`runtime/ext_gui/style.{h,cpp}`): a small built-in
31
+ UA stylesheet, author rules sorted by `(!important, specificity, source order)`,
32
+ then inline `style=""` (highest specificity, but non-`!important` inline loses to
33
+ `!important`), plus CSS inheritance of the text properties. Relative lengths are
34
+ kept unresolved until layout, except `font-size` (resolved against the *parent*
35
+ font size) and `line-height`.
36
+
37
+ ## Interactive DOM (M8)
38
+
39
+ `win.document` returns the document handle; element handles have stable identity
40
+ and read/write properties, attributes, traversal and geometry:
41
+
42
+ ```ts
43
+ const doc = win.document;
44
+ const box = doc.querySelector("#box");
45
+ const inner = doc.querySelector("#inner");
46
+ inner.textContent = "hi"; // write
47
+ inner.classList.add("hot");
48
+ inner.style.setProperty("color", "#0f0");
49
+ inner.setAttribute("data-role", "lead");
50
+ console.log(inner.id, inner.tagName, doc.querySelector("#box") === box);
51
+
52
+ const created = doc.createElement("div");
53
+ created.textContent = "added";
54
+ box.appendChild(created);
55
+ box.removeChild(created);
56
+
57
+ inner.addEventListener("click", (e) => console.log(e.target.id, e.currentTarget.id));
58
+ inner.click(); // synthesise a click at the element
59
+ console.log(box.offsetWidth, box.offsetHeight); // rounded border box
60
+ console.log(box.contains(inner)); // descendant test
61
+ ```
62
+
63
+ Mutations mark the document dirty; the engine restyles + relayouts lazily (before
64
+ the next read or frame). `inner = …`/`innerHTML = …` do **not** run scripts.
65
+ Element events support capture and bubble phases, `stopPropagation`, `once`, and
66
+ bubble up to `document`/`window`. The legacy `win.on(type, fn)` payload keeps its
67
+ **string** `e.target` (`div#id.class`); the element `Event.target` is a handle
68
+ whose descriptor matches the same string.
69
+
70
+ ### Node lifetime
71
+
72
+ A mutation decides whether the nodes it removes can still be reached, and the
73
+ engine releases exactly those it cannot:
74
+
75
+ | Mutation | Removed nodes |
76
+ | --- | --- |
77
+ | `innerHTML = …`, `inner = …`, `textContent = …`, `replaceChild` | **released** — the script has no way to refer to them again |
78
+ | `removeChild`, `remove` | **kept** in the document's detached pool — the DOM keeps a removed node alive and usable, so it can be re-attached |
79
+
80
+ Released nodes free their C++ subtree as well as their handle, so the counters
81
+ from `win.stats()` (`detachedNodes`, `handles`, `handleSlots`) return to their
82
+ previous values. That is what makes the usual game pattern — rewriting a sprite
83
+ with `innerHTML` every frame — run in constant memory; before, every replaced
84
+ subtree stayed pooled forever and a loop writing ten nodes per frame leaked
85
+ several megabytes per second.
86
+
87
+ The one case the engine cannot see is a node the script creates and then drops
88
+ without ever attaching it: `document.createElement("div")` parks the node in the
89
+ pool *and* keeps its handle reachable, because the mark-sweep collector has no
90
+ weak references, so the engine cannot tell that the script threw the handle
91
+ away. A loop that creates elements without attaching them therefore grows the
92
+ pool. Reuse the elements (or build the content with `innerHTML`, which releases
93
+ what it replaces) instead of creating fresh ones per frame.
94
+
95
+ ## AOT scripts (M9)
96
+
97
+ Import an `.html` file that contains inline `<script lang="ts">` bodies; the
98
+ loader compiles each body into a native function and `win.loadHTML(page)` runs
99
+ them once the document is parsed (no JavaScript engine, no runtime `eval`):
100
+
101
+ ```ts
102
+ import { createWindow, run } from "gui";
103
+ import page from "./page.html";
104
+
105
+ const win = createWindow({ title: "counter", width: 320, height: 240 });
106
+ win.loadHTML(page);
107
+ run();
108
+ ```
109
+
110
+ ```html
111
+ <button id="b">0</button>
112
+ <script lang="ts">
113
+ const b = document.getElementById("b");
114
+ let n = 0;
115
+ b.addEventListener("click", () => { b.textContent = String(++n); });
116
+ </script>
117
+ ```
118
+
119
+ The two parameters (`window`, `document`) are ordinary function arguments, so
120
+ script locals need no global-object machinery. External scripts work too:
121
+ `<script src="./counter.ts">` is read at compile time, its imports are hoisted
122
+ (rewritten to resolve from the HTML file) and its body is wrapped the same way —
123
+ so a script module can `import` helpers and still see `document`. Scripts are
124
+ **compile-time assets**: HTML created at runtime (`innerHTML`, fetched over the
125
+ network) never executes, and `src` URLs (`https://…`, `data:…`) are ignored.
126
+ Everything runs in document order after parsing (effectively deferred). See
127
+ `doc/gui-scripts.md` for the full design.
128
+
@@ -0,0 +1,25 @@
1
+ # Input and events (M5)
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there. The public event API is in
5
+ > [api.md](./api.md).
6
+
7
+ `LayoutTree::hitTest` returns the deepest box containing a point (probing later
8
+ siblings first so the topmost element wins), and `xt_dom_describe` turns the
9
+ element into the `div#id.class` descriptor carried by event payloads. A
10
+ transformed box maps the point through its inverse matrix first, so a rotated,
11
+ scaled or translated element is hit exactly where it is drawn — and `:hover`
12
+ follows the artwork, not the box flow left behind.
13
+
14
+ - SDL pointer/wheel/key events are routed to the owning window, hit tested, and
15
+ delivered to the handlers registered with `on`. `mousedown` also moves focus.
16
+ - `:hover` matches the hovered element **and its ancestors** (so hovering a child
17
+ lights up its parents); `:focus` matches the focused element. When either
18
+ changes, `XtDocument::setHover`/`setFocus` recompute styles and layout and mark
19
+ the window's geometry dirty, so the change is painted on the next frame.
20
+ - `win.hitTest(x, y)` and `win.sendEvent(type, options)` expose hit testing and
21
+ synthetic input so the whole path is testable headlessly (the e2e suite drives
22
+ clicks, wheels and keys without a real mouse).
23
+
24
+ Still to do for full input: text selection, drag, IME and clipboard.
25
+
@@ -0,0 +1,78 @@
1
+ # Known issues and open questions
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there.
5
+
6
+ ## Known issues
7
+
8
+ - **Intermittent crash at auto-close.** `XT_GUI_AUTOCLOSE_MS` can end a run
9
+ with a native access violation in roughly two runs out of five; a clean exit
10
+ looks identical otherwise. Windows reports it as the NTSTATUS exit code
11
+ `0xC0000005`; on Linux/macOS the process dies with a signal — an access
12
+ violation is `SIGSEGV`, so `spawnSync` reports `status: null` plus a
13
+ `signal`. It only reproduces with a *running animation-frame loop* (a static
14
+ document closes fine) and it is **not** the game's own code: no exception is
15
+ reported, the crash usually lands inside the first second regardless of the
16
+ deadline, and raising `XT_GC_THRESHOLD` hides it, so it looks like a
17
+ GC/teardown race. Interactive runs (no auto-close) are stable for minutes.
18
+ A leading cause was that the engine cached `xt_value`s in C++ containers
19
+ the mark-sweep collector cannot see — the AOT `<script>` registry, element
20
+ handles, element listeners and the queued/currently-running
21
+ `requestAnimationFrame` callbacks. They are now marked by GC root providers
22
+ (`xt_gui_script_gc_scan` and `xt_gui_gc_scan_roots`), which removes that
23
+ class of use-after-free. Since the M12 node-lifetime work it has not
24
+ reproduced at all: 44 auto-close runs of `examples/gui/pelican-bike` (deadlines
25
+ 800 ms–4 s, with the default, a 1 MiB and a 64 MiB collector threshold, where
26
+ the old report said roughly two in five) all exited cleanly, which points at
27
+ the same root cause — a detached-node pool and handle table that grew with the
28
+ frame count while the collector's threshold ratcheted up. It stays listed
29
+ because the archive is built per platform and the failure is timing-sensitive;
30
+ `tests/e2e/gui-example.test.ts` still treats either crash spelling as an
31
+ acceptable outcome, and the behavioural assertions live in
32
+ `tests/e2e/gui-example-browser.test.ts` (no GPU, deterministic).
33
+ - **`display: none -> flex` restyle.** A hidden element being shown by a
34
+ *class change* on a flex container has also been observed to crash the same
35
+ way; `examples/gui/pelican-bike` reveals its game-over overlay with `opacity`
36
+ instead.
37
+
38
+ ## Open questions
39
+
40
+ - Whether Linux ships X11, Wayland, or both in the first cut. (Decision:
41
+ X11 first, Wayland later.)
42
+ - **SVG:** SVG *files* are supported through the image loader
43
+ (`<img src="logo.svg">`): `runtime/ext_gui/svg.cpp` wraps the vendored
44
+ **NanoSVG** parser/rasteriser and `image.{h,cpp}` treats an SVG as a vector
45
+ image, rasterised at the drawn pixel size. Inline `<svg>` **trees** in a
46
+ document are still not supported: the HTML parser knows elements and
47
+ attributes, the layout engine knows boxes, and paint knows
48
+ rectangles/text/images, so `<svg>` subtrees would need a whole second
49
+ geometry+paint path. Vector art therefore has to arrive as an SVG/PNG file or
50
+ be drawn from boxes, as `examples/gui/pelican-bike` does.
51
+ - **Windows: the C++ runtime must not be named on the link line.** `gui.lib` is
52
+ built with the *static* CRT (`CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded`) and
53
+ every engine object records `libcpmt` as a dependent library, which is also
54
+ clang's Windows default and what the xbintsc runtime objects use
55
+ (`scripts/build-gui.ts`). Asking for the DLL runtime as well — the gui
56
+ extension used to pass `-lmsvcprt` — makes the two copies of the same symbols
57
+ collide: `LNK2038: mismatch detected for 'RuntimeLibrary'` plus a wall of
58
+ `LNK2005: already defined`, surfaced as `clang: linker command failed with exit
59
+ code 1169`. `src/extensions/gui/index.ts` now lists only the OS libraries
60
+ (`user32`, `gdi32`, `d3d12`, …). A Windows link is also the only place the
61
+ MSVC environment matters, so build from an x64 Native Tools prompt.
62
+ - Windows: an MSVC-compatible `gui.lib` (COFF objects + `ar -M`/`llvm-ar`) is
63
+ produced by `scripts/build-gui.ts` but is not validated in CI, so the Windows
64
+ step is provisional (the job stays green) and `package` skips it. Building and
65
+ running `examples/gui/pelican-svg` and `examples/gui/pelican-bike` on a
66
+ developer machine works. It also needs an SDL3 build with the D3D12/DXIL
67
+ backend (DXIL requires `dxc`).
68
+ - **Media on Windows:** the media decoders (FFmpeg) are only built and merged
69
+ into the POSIX `gui.a`; the Windows `gui.lib` is produced without them, so
70
+ `<video>`/`<audio>` are inert there (failures fire `error`). Enabling it needs
71
+ an MSVC-compatible FFmpeg build and is deferred until the Windows GUI path
72
+ itself is validated. `<video controls>` is likewise deferred — the intended
73
+ pattern is TS + a CSS overlay, as in `examples/gui/media.ts`.
74
+ - **D3D12 shaders:** the SPIR-V (Vulkan) and DXIL (Direct3D 12) blobs are built
75
+ and embedded, so the non-Metal path no longer skips geometry. DXIL remains
76
+ unusable on **SDL 3.2.10**: its D3D12 backend rejects any graphics pipeline
77
+ whose shaders declare a uniform buffer, and the engine passes the viewport that
78
+ way. Re-check after an SDL upgrade.
@@ -0,0 +1,108 @@
1
+ # Layout
2
+
3
+ > Part of the [GUI extension](../gui.md); the engine overview, locked
4
+ > decisions and roadmap live there.
5
+
6
+ ## Layout engine (M3)
7
+
8
+ `runtime/ext_gui/layout.{h,cpp}` turns the styled DOM into a `LayoutBox` tree
9
+ with absolute (viewport-relative) geometry:
10
+
11
+ - **Block flow** — block-level children stack vertically; `display: none`
12
+ generates no box; `width: auto` fills the containing block, `height: auto`
13
+ wraps the content. The box model (margin/padding/border) is resolved,
14
+ including percentages against the containing block width. Adjoining vertical
15
+ margins between siblings collapse to `max(positive) + min(negative)`; a
16
+ parent's bottom margin does not collapse with its last child's (see the
17
+ limitations below).
18
+ - **Inline flow** — consecutive inline-level children form an anonymous inline
19
+ formatting context with greedy, word-based line breaking, `text-align` and
20
+ `line-height`. Inline elements get the union of their descendants' geometry;
21
+ `display: inline-block` is laid out atomically with a shrink-to-fit width.
22
+ Each line shrinks to the band left free by any floats (`float: left|right`)
23
+ that overlap it, so later lines wrap around them.
24
+ Each text fragment remembers the run it covers, so paint can shape it. Text is
25
+ measured with the HarfBuzz/FreeType stack (see [Text stack](./rendering.md#text-stack-m4b)).
26
+ - **Flexbox** — `row`/`column` (and the `-reverse` variants) with `gap`,
27
+ `flex-basis`/`flex-grow`/`flex-shrink`, `justify-content` and `align-items`
28
+ (including `stretch` when the cross size is definite). `flex-wrap: wrap`
29
+ breaks items into several lines, `align-content`
30
+ (`stretch`/`flex-start`/`flex-end`/`center`/`space-between`/`space-around`)
31
+ distributes the lines on the cross axis and `wrap-reverse` flips that axis.
32
+ Item margins are part of the item's outer size on both axes, so a negative
33
+ `margin-top` pulls an item up instead of being ignored. Whitespace-only text
34
+ between items (source indentation) does not become a flex item.
35
+ - **Positioning** — `position: relative` offsets a box (and its subtree) by
36
+ `top`/`right`/`bottom`/`left` without changing the space it reserved;
37
+ `position: absolute` (and `fixed`, which resolves against the viewport) takes
38
+ the box out of flow and places it against the padding box of the nearest
39
+ positioned ancestor — the viewport when there is none. An `auto` inset keeps
40
+ the static position on that axis, an `auto` width is shrink-to-fit unless both
41
+ `left` and `right` are set (then it fills the space between them), and
42
+ percentages resolve against the containing block. Stacking is approximated by
43
+ sorting each parent's children by `z-index` (negative → `auto` → positive)
44
+ before painting; this is a single level, not a full stacking context, so a
45
+ child cannot escape its parent's level.
46
+ Note the consequence of that shrink-to-fit rule: an absolutely positioned box
47
+ with `width: auto` and no explicit size gets **no** available width for its own
48
+ inline content (its preferred width comes from its text or its non-absolute
49
+ children only), so inline runs inside it are laid out at zero available width
50
+ and each one wraps onto its own line. Give such a box an explicit `width`, or
51
+ place its children with `left`/`width` (as
52
+ `examples/gui/pelican-bike` does for its sprite runs), instead of relying on
53
+ line flow inside it.
54
+ - **Floats and clearance** — `float: left|right` takes a box out of flow,
55
+ shrink-to-fits its width, places it against the current line and lets later
56
+ inline lines wrap around it; `clear: left|right|both` pushes a following block
57
+ below the relevant floats, and a block that contains only floats grows to
58
+ enclose them.
59
+ - **`overflow` clipping** — a box with `overflow: hidden` (or `scroll`/`auto`)
60
+ clips its subtree to its padding box, intersected with any ancestor clip; the
61
+ renderer applies the result with a GPU scissor rectangle. There is no
62
+ scrolling yet, so `scroll`/`auto` clip like `hidden`.
63
+
64
+ Not yet implemented: parent/child and empty-block margin collapsing, anonymous
65
+ flex items from non-whitespace text, full `z-index` stacking contexts, and
66
+ `float` affecting block children (only inline content wraps; a block must
67
+ `clear`).
68
+
69
+ ## Transforms (M15)
70
+
71
+ `transform` and `transform-origin` are a **paint-time** effect: the box model
72
+ above is untouched (nothing reflows, no geometry is recomputed), and a second
73
+ pass after `xt_layout_run` gives every `LayoutBox` a `Matrix2D world` — the
74
+ transform of the whole ancestor chain — plus its `inverse`:
75
+
76
+ - **Only element boxes contribute.** A text box shares its parent element's
77
+ `style` pointer, so letting both contribute would apply the element's list
78
+ twice; the pass composes an element's own list, its ancestors', and nothing
79
+ else.
80
+ - **`transform-origin` is resolved here**, not in the cascade: the computed style
81
+ keeps the two (possibly `%`) `Length`s (default `50% 50%`) and layout resolves
82
+ them against the *border box*, applying `T(origin) · M · T(-origin)`. That is
83
+ why `computedStyle(el, "transform-origin")` still reports `50% 92%`. The
84
+ keywords are positional as in CSS: `top` alone is `center top`, and the
85
+ keyword pair may be given in either order (`bottom right` = `right bottom`).
86
+ - **Lengths resolve against the box**: `%` in `translate()` is a share of the
87
+ border box's width/height, `em`/`rem` against the root font size (16px), and
88
+ `vw`/`vh` against the viewport.
89
+ - **Composition order** follows CSS: the list applies right to left, so
90
+ `rotate(90deg) translate(10px)` moves the box *down* the screen, not right.
91
+
92
+ Everything downstream follows the matrices:
93
+
94
+ | Consumer | Behaviour |
95
+ | --- | --- |
96
+ | Paint | each op carries the matrix (see [Rendering](./rendering.md#paint-and-display-list-m4am4b)); a clipped subtree clips to the bounding box of its padding box through the matrix |
97
+ | Hit testing | `hitTestBox` maps the point through each box's inverse before the containment test, so a rotated or scaled element is clickable where it is *drawn* |
98
+ | `getBoundingClientRect` | the axis-aligned bounding box of the **transformed** border box (`xt_layout_transformed_bounds`) |
99
+ | Layout geometry | unchanged — `layoutTree()` and `offsetWidth`/`offsetHeight` keep reporting the untransformed box |
100
+
101
+ Unsupported transform functions (`skew`, `perspective`, …) invalidate the whole
102
+ declaration, so a box is either correctly transformed or not transformed at all
103
+ rather than partially applied.
104
+
105
+ **Document** (`runtime/ext_gui/document.{h,cpp}`): owns the DOM tree, gathers
106
+ `<style>` text into one stylesheet, computes styles and layout for a viewport and
107
+ offers `querySelector`/`querySelectorAll`/`styleOf`/`boxOf`.
108
+