@datamoc/mw_games 0.40.0 → 0.41.0

package/tools.md

@@ -0,0 +1,216 @@
+# Tools
+
+Developer-side scripts in `tools/`. Nothing here reaches the player: these author
+content, verify it, or package it, and the shipped game is still a folder that opens
+by double-clicking. Run from the repo root.
+
+## Content and authoring
+
+| tool | what it does | run |
+| --- | --- | --- |
+| `i18n-edit` | Split-screen terminal editor for `mwg/i18n` catalogs: reference language left, translation right, either side any language, per-string sound cues, JSON/FTL auto-detected, with a `--check` mode for CI. Panes preview markdown as bold/italic and flag placeholder drift (`{dmg:03d}` dropped or reshaped); `v` toggles raw source, `Ctrl+P` previews the row's cue through the OS player (`MWG_SFX_PLAYER` overrides) | `npx mwg-i18n <base> <target> [--check]` in a game (shipped, item 384), `npm run i18n:edit -- <base> <target>` here |
+| `make-example-assets` | Generates `examples/assets` (tileset, sounds). Borrowed art stays out for licence reasons, so this draws everything; a game can prototype with the same set before its own art exists (item 389) | `npm run assets` here; `npx mwg-placeholder-assets <folder> [--force]` in a game (refuses to overwrite without `--force`), or `writePlaceholderAssets` from `@datamoc/mw_games/tools/make-example-assets` |
+| `soundfonts` | Picks a SoundFont other than the bundled `microgm-opt` (WTFPL, 2.6 MB, the default). `list` prints the catalogue (`data/soundfont-catalog.json`, a licence-classified snapshot of the marmooo free-soundfonts collection: public domain, MIT and CC-BY fonts fetch freely, share-alike, copyleft and unclear ones need `--accept-licence`); `fetch <font>` downloads only the instruments the game's MIDI files use (`--midi`) or a list (`--programs 0,24,128:0`), and merges them into one SoundFont 3 (the compressed samples untouched, no decoding, no `ffmpeg`) beside a `.credits.txt` that keeps the licence with the file; the game decodes it where it runs with `loadSoundFont`. `fetch --sf2` and `convert <in.sf3> <out.sf2>` decode to a plain SoundFont 2 instead, about eight times larger, which needs `ffmpeg` on PATH (developer machines only). The result goes in the asset folder, where the build subsets it to the programs the game plays | `npx mwg-soundfonts list`, `npx mwg-soundfonts fetch GeneralUser_GS_v2.0.3 --midi assets/*.mid --accept-licence`, or `fetchSoundFont` from `@datamoc/mw_games/tools/soundfonts` |
+| `extract-rgssad` | Decrypts an RPG Maker XP/VX archive to plain files (vendored MIT decoder, developer-side only); an entry whose name would land outside `outDir` is refused (item 389) | `npm run extract:rgssad -- <archive.rgssad> <outDir>` here; `npx mwg-extract-rgssad <archive> <outDir>` or `extractRgssad` from `@datamoc/mw_games/tools/extract-rgssad` in a game |
+| `multiplayer-server` | The reference lockstep server for `core.LockstepClient`: rooms, one shared tick, `null` for a missed input, messages capped at `maxMessageBytes` (64 KB) and read through `parseInbound`, room names bounded, `maxClientsPerRoom` (64). Speaks `ws:`, so a deployment puts it behind a TLS proxy (items 129, 385) | `npx mwg-lockstep-server [port] [--host=]` (127.0.0.1 unless `--host`), or `createLockstepServer(options)` from `@datamoc/mw_games/tools/multiplayer-server`; needs the optional peer `ws` |
+| `mwl` | Compiles MWL content (`[game]`/`[scenario]`/`[side]`/`[unit]`/`[event]`/`[campaign]`) into a game-data module, an i18n catalog and an asset manifest | `npm run mwl -- build <contentDir> -o <outDir>`, or `node node_modules/@datamoc/mw_games/tools/mwl.mjs build <contentDir> -o <outDir>` from an install. It is a command line, not an importable module (it dispatches when loaded), so it has no package subpath; script against the typed library instead: `@datamoc/mw_games/mwl` |
+| `extract-html` | Splits the resources embedded in an HTML file (scripts, CSS, images) back out into files | `npm run extract:html -- <page.html> -o <outDir>`, or `extractHtml` from `@datamoc/mw_games/tools/extract-html` |
+| `compile-openings` | Compiles opening tables (`eco`, `name`, `pgn` TSV rows) into a position-keyed book JSON a game checks in beside its own code: SAN resolved against the real move generator, every position keeping its continuations, the longest line ending there naming it. Rows are vendored CC0 data (`data/openings/`), never fetched | `node tools/compile-openings.mjs <tsv folder> <book.json>`, or `npx mwg-openings <tsv folder> <book.json>` in a game |
+| `build-tablebases` | Solves pawnless white-extras-against-lone-king endings by retrograde analysis and ships them compiled as Huffman-coded JSON under `data/tablebases` (3-piece: `KQK KRK KBK KNK`; 4-piece like `KBNK KBBK` generate the same way but stay a tool run away, full 4-piece tables are tens of megabytes each). One `threads.spawn` task per ending over capped lanes (4-piece lanes stay narrow: hundreds of megabytes of arrays each); captures convert into the already-solved smaller tables, so 3-piece endings solve first. Golden counts: KQK 0/200896/99448/223944, KRK 0/201700/98644/223944, KBK/KNK all draws, KBBK 0/5628080/21071792/6854560, KBNK 0/11188168/8816020/13550244 (unknown/loss/draw/win) | `node tools/build-tablebases.mjs <outDir> [ending ...] [--jobs=N]`, or `npx mwg-tablebases <outDir> [ending ...]` in a game |
+| `play-uci` | A UCI adapter over the tournament engine for GUIs and rating runs: `position`/`go` on stdin, `bestmove` on stdout. Move choice runs book, then tablebase, then `searchTourney`; a Node tool, never part of the browser bundle | `node tools/play-uci.mjs [--book <book.json>] [--tables <dir>] [--depth N]`, or `npx mwg-uci` in a game |
+
+## Shipping (`file://`)
+
+| tool | what it does | run |
+| --- | --- | --- |
+| `compile-resources` | Turns an asset folder into `data:` URI scripts under `window.__MWG_ASSETS__`, grouped by top-level folder. `toWebp: true` converts `.png`/`.jpg`/`.jpeg` to WebP first (lossless by default, only kept when actually smaller, key unchanged) via the optional `sharp` devDependency, dynamically imported only when used (item 350) | called by the example builds; `node tools/compile-resources.mjs <asset folder> <output folder> [--to-webp] [--webp-lossy[=quality]]` |
+| `webp-convert` | Converts one raster image buffer (PNG/JPEG) to WebP, lossless by default; the standalone half of `compile-resources`'s `toWebp` option (item 350) | `node tools/webp-convert.mjs <file.png\|file.jpg> [--lossy[=quality]]` |
+| `emit-page` | Finishes a vite build so it opens from `file://` with no server: compiles the assets, rewrites the entry to a classic deferred script with the asset scripts before it, then optionally precompresses and writes a single-file variant (item 381) | `emitPage(options)` from `@datamoc/mw_games/tools/emit-page`, the `mwg-emit <folder> [--dist=] [--assets=]` command, or called by each `example:*:build` script here |
+| `init` | `mwg-init <folder>` writes a starter project (vite config using `mwgPage`, one scene, tsconfig, scripts) that builds to a double-clickable page | `npx mwg-init my-game` |
+| `verify-scenarios` | `mwg-scenarios <built index.html>` runs every game-owned scenario in one Chrome page, using registered scene controls for bot turns, assertions and cleanup | `npx mwg-scenarios dist/index.html --turns 3` |
+
+| `vite` | The Vite plugin a game uses instead of either: `mwgPage(options)` sets the `file://` build configuration (IIFE `game.js`, `base: './'`, `assets/` as the public folder) and runs `emitPage` when `vite build` finishes (item 381) | `plugins: [mwgPage()]` from `@datamoc/mw_games/tools/vite`; `mwgPage({ singleFile: { compress: true } })` also writes one self-contained `standalone.html` (about 250 KB for Pixi + mwg + a small game) |
+| `browser-smoke` | Opens a built page from `file://` in headless Chrome and fails unless it rendered: no page error, an mwg game ready on a WebGL/WebGPU canvas, non-flat pixels, then a screenshot (item 383). `find-chrome` beside it picks the system Chrome, then the newest Chromium Playwright installed, then PATH; `CHROME_PATH` overrides | `vite build && npx mwg-smoke dist` in a game's CI (needs the optional peer `playwright-core`), or `smokePage(options)` from `@datamoc/mw_games/tools/browser-smoke` |
+| `csp` | The Content-Security-Policy `emitPage` writes into every page by default (item 374): scripts only from the page's own folder plus the hashed inline ones, images/fonts/media only `data:`/`blob:`, network (`connect-src`) only to declared origins, `'unsafe-eval'` kept because Pixi 8 needs it | automatic; `csp: { connect: ['https://api.example'] }` or `csp: false` on `emitPage`/`mwgPage`, `--connect=`/`--no-csp` on `mwg-emit`, or `withContentSecurityPolicy(html)` from `@datamoc/mw_games/tools/csp` |
+| `classic-html` | Rewrites a bundler build's `<script type="module">` entry tag to a classic deferred script, the shipped, reusable half of what `emit-page` does for this repo's own examples (item 306) | `import { toClassicScript } from '@datamoc/mw_games/tools/classic-html'` in a game's own build script, or `node tools/classic-html.mjs dist/index.html` here |
+| `compress-dist` | Writes `.gz`/`.br` siblings for large build outputs, into the folder it is handed, plus `.xz` where smaller behind the opt-in `xz` option (needs the system `xz` binary). Files compress in parallel over async `zlib`, `MWG_COMPRESS_JOBS` names the lane count, rows keep walk order | used by the packaging flow |
+| `single-file` | Inlines every `<script src>` an `emit-page` output loads into one additional standalone HTML file, no sibling `.js` at all; `compress: true` compresses each script first (gzip by default, or `algorithm: 'brotli'`), unpacked at load through the browser's own `DecompressionStream` behind a splash screen. Brotli needs `DecompressionStream('br')`, unsupported in a directly-tested current Chrome (153) as of this writing - gzip is the broadly-supported default, brotli an advanced option that degrades to a clear, catchable error rather than hanging (items 349, 350) | `node tools/single-file.mjs <dist folder> [--compress[=level]] [--brotli[=quality]] [--no-splash] [--output name.html]`, `emit-page --single-file [--single-file-compress[=level]] [--single-file-brotli[=quality]]`, or `buildSingleFile` from `@datamoc/mw_games/tools/single-file`. `output` is a file name written inside `dist`, not a path: one with a separator is refused |
+| `asset-report` | Lists every file a built page ships, with raw/gzip/brotli weights and each file's share of the compressed page, plus the embedded source assets with byte-identical duplicates grouped (item 422) | `mwg-assets dist [--assets=assets] [--json=report.json]`, or `assetReport(options)` from `@datamoc/mw_games/tools/asset-report` |
+
+### Scenario verification contract
+
+Expose `window.__MWG_SCENARIOS__` in a verification build with `controls` (a
+`createSceneControls()` registry from `@datamoc/mw_games/testing`) and `scenarios`:
+
+```js
+window.__MWG_SCENARIOS__ = {
+  controls,
+  scenarios: [
+    { id: 'tutorial', enter: 'enterTutorial', turn: ['playHumanBot', 'playEnemyBots', 'advanceDialogue'],
+      verify: 'checkTutorial', leave: 'destroyScenario' },
+  ],
+};
+```
+
+Each name is a registered step. The game implements scenario entry, bots for all sides,
+dialogue advancement and checks for map, units, outcomes or campaign references. A check
+throws on failure; an optional `leave` step handles cleanup even after a failed entry or
+turn. The tool validates the whole selected plan before entering any scenario, runs each
+turn's steps in order, then checks it. Empty plans and filters matching nothing fail.
+Exceptions and console errors fail the report. A timed-out step aborts the run and closes
+the browser because its promise may still be changing scene state.
+
+Run `mwg-scenarios dist/index.html --turns 3 --timeout-ms 120000 --only tutorial`.
+The page opens from `file://` once, and the JSON report names every attempted scenario,
+completed turn count and failure. Install the optional `playwright-core` development peer
+to use this tool. `CHROME_PATH` selects Chrome; `MWG_VISUAL_CHROME_ARGS` supplies runner flags.
+
+## Packaging as a native app
+
+The packaging scaffolding is part of this repository, not of the published package. Its
+`files` list ships `dist/`, the `tools/*.mjs` helpers (see the table above), the licence and the markdown
+documents, and none of its packaging scripts (`desktop:*`, `mobile:*`, `cap:*`) do: they all
+point into this checkout. A game that installs `@datamoc/mw_games` from the registry
+therefore supplies its own native shell; the recipe for each target is below.
+
+Build the game without `compress-dist`'s `.gz`/`.br` siblings for any native target: a
+packaged app has no server to negotiate `Content-Encoding` with, so they are dead weight
+(the opt-in `.xz` siblings are the same), and on Android the asset merge fails outright
+(`Duplicate resources` from AAPT, which reads `game.js` and `game.js.gz` as colliding).
+
+### Desktop (Windows, WebView2)
+
+The reference host is `desktop/MwgDesktopHost` - two files, `MwgDesktopHost.csproj` and
+`Program.cs`. Copy that one folder into your project and point it at the game's built
+`dist/`:
+
+```powershell
+cp -r <a clone of mwg>/desktop/MwgDesktopHost ./desktop/MwgDesktopHost
+dotnet run --project desktop/MwgDesktopHost -- dist/index.html
+```
+
+The argument is resolved against the directory the command runs from, so it can be relative;
+with no argument the host loads this repository's own tower-defense example, which is what
+`npm run desktop:run` does. It needs the .NET 8 SDK (`winget install Microsoft.DotNet.SDK.8`)
+and the WebView2 Runtime, already present on current Windows.
+
+The host serves the page through `CoreWebView2.SetVirtualHostNameToFolderMapping`, mapping
+`https://mwg.local/` to the page's own folder rather than navigating to `file://`. That real
+origin is what lets `assets.fetchWithByteProgress`, a `LockstepClient` WebSocket connection,
+or any other genuine network call work inside the app; `file://` blocks all of them. A game
+that needs none of that can use `desktop/webview2` instead, which opens the same built page
+from `file://` and takes the same page argument.
+
+`dotnet publish -c Release -r win-x64 --self-contained true` inside the host project produces
+a distributable build.
+
+### Mobile (Capacitor, Android)
+
+`android/` and the `cap:*` scripts are this repository's own scaffold, so copy the shape of
+them rather than the files: a game creates its own Capacitor project. Toolchain first, since
+these are the two that stop a clean machine:
+
+- **JDK 21** (`winget install EclipseAdoptium.Temurin.21.JDK`), with `JAVA_HOME` pointing at it.
+  Capacitor's Android Gradle Plugin will not compile under JDK 17: `invalid source release: 21`.
+- **Android SDK** with `platform-tools`, and either `ANDROID_HOME` set or an
+  `android/local.properties` written by hand with `sdk.dir=C:/path/to/Android/Sdk`. Forward
+  slashes work; backslashes have to be doubled, and a properties file with single backslashes
+  silently loses every separator and fails much later as an unreadable path.
+
+Then, from the game's own folder:
+
+```powershell
+npm install --save-dev @capacitor/cli @capacitor/core @capacitor/android
+# capacitor.config.json: { "appId": "com.example.yourgame", "appName": "Your Game", "webDir": "dist" }
+npx cap add android        # once: scaffolds android/
+npx cap sync android       # after every web rebuild: copies dist/ into the project
+cd android && ./gradlew assembleDebug      # gradlew.bat on Windows
+adb install -r app/build/outputs/apk/debug/app-debug.apk
+adb shell am start -n com.example.yourgame/.MainActivity
+```
+
+`assembleDebug` produces an installable APK; a release build needs signing, which is yours to
+arrange. `adb` comes with the SDK's `platform-tools`; an emulator is
+`emulator -avd <name>`, and a cold boot takes a minute before `adb install` will accept it.
+
+Three things about the generated project are worth knowing before the first bad build, all
+three because `cap add android` writes them once and never revisits them:
+
+- **Do not ship `.gz`/`.br` sidecars.** The asset merge fails outright:
+  `mergeDebugAssets FAILED ... [public/game.js.gz]: Duplicate resources`, because AAPT reads
+  `game.js` and `game.js.gz` as colliding. Build the web output without `compress-dist` for a
+  packaged app anyway: there is no server to negotiate `Content-Encoding`.
+- **`ignoreAssetsPattern`,** in `android/app/build.gradle`: the generated line ends `!*~`. Add
+  `:!*.gz:!*.br` to it, which is what this repository's own `android/` carries. That is the
+  second half of the same problem, and it is why MWG tracks `android/` in git at all - a fresh
+  `cap add android` would lose it.
+- **`versionCode`/`versionName`,** in the same file: written as `1` and `"1.0"` and never
+  revised, so an app that ships twice without stamping them cannot be installed over itself
+  (`INSTALL_FAILED_VERSION_DOWNGRADE`; Android refuses an update whose `versionCode` has not
+  increased). Stamp both from the release that produced the build.
+
+On some Windows hosts the Capacitor CLI itself refuses to start: Node 26 can return `ENOMEM`
+from `os.userInfo()`, which Capacitor's terminal helper calls at load. The framework's own
+`cap:cli` script works around it, and a game that hits it can too:
+
+```js
+// tools/capacitor-node-workaround.cjs, then: node --require ./tools/capacitor-node-workaround.cjs
+// node_modules/@capacitor/cli/bin/capacitor add android
+const os = require('node:os');
+try {
+	os.userInfo();
+} catch (error) {
+	if (error?.syscall !== 'uv_os_get_passwd') throw error;
+	os.userInfo = () => ({
+		uid: -1,
+		gid: -1,
+		username: process.env.USERNAME || 'unknown',
+		homedir: process.env.USERPROFILE || process.cwd(),
+		shell: process.env.COMSPEC || null,
+	});
+}
+```
+
+Verified end to end from a folder outside this repository, against the published package: a
+game built with `tools/classic-html`'s `toClassicScript`, `cap add android`, `cap sync`, a real
+`gradlew assembleDebug`, then installed and launched on a `Medium_Phone_API_35` emulator, where
+the game rendered and its pixels were checked rather than assumed. The duplicate-resource
+failure above and the `ignoreAssetsPattern` fix that clears it were both reproduced on that
+same build.
+
+## Verification and benchmarks
+
+| tool | what it does | run |
+| --- | --- | --- |
+| `benchmark-browser` | Measures a built page's frame rate in headless Chrome: mean fps, 95th-percentile frame time, JS heap, a per-page history compared with its best run (item 387) | `npm run benchmark:browser` here; `npx mwg-bench dist [--frames=] [--min-fps=] [--max-p95=] [--history=]` in a game (optional peer `playwright-core`), or `measurePage(options)` from `@datamoc/mw_games/tools/benchmark-browser` |
+| `benchmark-simulation` | Headless throughput for `mwg/simulation` (no frame, canvas, or Pixi involved) | `npm run benchmark:simulation` |
+| `benchmark-animation` | CSS/SVG element animation vs Pixi sprites vs both, at increasing counts, sampling rAF intervals in headless Chrome. A measurement, not a gate | `npm run benchmark:animation` |
+| `graphics-capabilities` | Probes the host's WebGL capabilities headless | `npm run graphics:capabilities` |
+| `visual-smoke` | Opens one built example from `file://` in headless Chrome, fails on a page error or a blank canvas, and writes a screenshot. Runs per pull request in CI | `npm run visual:smoke:ui` |
+| `package-smoke` | `npm pack`, installs the tarball outside the repo, builds the tutorial's tiny game and opens both that and the standalone global from `file://` | `npm run package:smoke` |
+| `motion-smoke` | Opens the interface example three ways (no preference, an emulated `prefers-reduced-motion: reduce`, and a flip while the page is open) and asserts the demo's own state: the preference arrives, stops the motion, and is seen when it changes | `npm run motion:smoke` |
+| `build-all-examples` | Runs every `example:*:build` script, so a vite/emit-page regression in any example is caught, not just the benchmarked ones | `npm run examples:build` |
+| `bundle-size` | The size budget: raw and gzipped bytes against a committed baseline, failing past a tolerance (2 % by default). Here, the global bundle and the published `dist`; in a game, every file a player receives plus their total (item 386) | `npm run size:check`/`size:update` here; `npx mwg-size dist [--baseline=size-budget.json] [--update] [--tolerance=0.02]` in a game |
+| `audit-adoptions` | The consumer adoption auditor: reads the `api-surface.json` manifest plus a consumer-authored reasons file and config (type-only directories, subpath deny-lists, per-pattern import budgets), then reports exports neither imported nor reasoned, shadowing collisions, and config violations, with `file:line` and a non-zero exit | `npx mwg-audit-adoptions --manifest api-surface.json --imports imports.json --locals locals.json --config audit.json`, or `auditAdoptions(manifest, imports, locals, config)` from `@datamoc/mw_games/tools/audit-adoptions` |
+| `browser-driver` | The dependency-free browser harness: a self-served static directory on an OS-chosen free port, with Chrome over CDP and Firefox over WebDriver BiDi behind one surface (`goto`/`waitReady`/`tap`/`evaluate`/`script`/`screenshot`/`consoleErrors`/`close`), reusing the `browser-smoke` readiness contract and no pixel-diff baseline | `driveChrome()`/`driveFirefox()` plus `serveDir(root)` from `@datamoc/mw_games/tools/browser-driver` |
+
+## Project chores
+
+| tool | what it does | run |
+| --- | --- | --- |
+| `roadmap-progress` | Reports CLOSED.md + ROADMAP.md completion: numbered items in batches of 25, plus the 1.0 exit checklist on its own row (checks to run, not capabilities, so they stay out of the numbered totals), optionally in a browser | `npm run roadmap:progress` |
+| `build-webpage-examples` / `build-webpage-docs` | Generate the live examples and Documentation page under `webpage/` (`REFERENCE.md` guide + TypeDoc under `documentation/api/`) | `npm run webpage:examples`, `npm run webpage:docs` |
+| `make-example-diagrams` / `make-architecture-diagrams` | Rebuild the generated diagrams under `webpage/assets` (never hand-drawn) | `npm run webpage:diagrams` |
+| `api-report` | Regenerates `API_REPORT.md` from the built declarations; `--check` compares instead. A CI gate | `npm run api:report`, `npm run api:check` |
+| `project-stats` | Writes `PROJECT_STATS.json`/`PROJECT_STATS.md` and the website statistics page; `--check` compares. A CI gate | `npm run stats:write`, `npm run stats:check` |
+| `ci-status` | Waits for every workflow run of `HEAD` and fails if any of them did | `npm run ci:status` |
+| `npm-audit` | Dependency audit that fails for high or critical advisories | `npm run audit` |
+| `sbom` | Two CycloneDX SBOMs, both reproducible (no timestamp, a content-derived serial). The lockfile SBOM of any npm project (`buildSbom`; `mwg-sbom [folder] [--out=] [--check]`; this repository's own committed `sbom.cdx.json`, gated in CI and now shipped in the package). And the SBOM of what a build ships (`buildArtifactSbom`, written as `dist/sbom.cdx.json` by `mwgPage()`): only the packages whose modules reached the bundle, from vite's module graph, plus each shipped file's SHA-256 (items 315, 380) | `npm run sbom`, `npm run sbom:check` here; `npx mwg-sbom` or `mwgPage()` in a game |
+
+### Selecting embedded assets
+
+The Vite plugin accepts `mwgPage({ assets: 'assets', assetFiles: ['hero.svg', 'levels/one.json'] })`.
+Paths are relative to the asset folder. Omission embeds every supported file as before; an
+empty list embeds none. Unknown files and unsupported formats named explicitly fail the build.
+The same `assetFiles` option is available on `emitPage` and `compileResources`. Dev serving is
+unchanged. Include `credits.json` in an explicit list to generate its `CREDITS.txt` attribution.