@inneranimalmedia/agentsam-sdk 2.6.5 → 2.6.7

package/skills/grok-building-games/references/ai-pathfinding.md

@@ -0,0 +1,122 @@
+# Game AI & Pathfinding (A* on grids, steering, FSM, behavior trees, navmesh)
+
+Consolidated from Red Blob Games (the canonical A*/grid reference), Craig Reynolds' steering work, and Game AI Pro / behavior-tree canon (see Sources). Focus: what an AI builder needs so enemies/agents **move sensibly and don't jitter, get stuck, or tank the frame rate.**
+
+---
+
+## 1. Pathfinding: A* on a grid (the default)
+
+**A\*** finds the shortest path by expanding the node that minimizes `f = g + h`, where `g` = cost so far and `h` = heuristic estimate to goal. It's Dijkstra + a goal-directed heuristic.
+
+Rules:
+- **Heuristic must be admissible** (never overestimate true cost) or A* may return a non-optimal path.
+  - **4-directional grid → Manhattan distance** `|dx|+|dy|`.
+  - **8-directional grid → octile/Chebyshev distance** (diagonal cost ≈ 1.414). Using Manhattan on an 8-dir grid overestimates → wrong paths; using Euclidean on a 4-dir grid underestimates → slow.
+- **Use a binary heap / priority queue** for the open set. A linear scan of the open set is the #1 A* perf bug — turns O(E log V) into O(V²) and stutters on big maps.
+- **Track a closed set / best-g per node** so you don't reprocess nodes; update `g` if a cheaper path to a node is found.
+- **Weighted tiles:** give terrain costs (mud=5, road=1) via `g`; A* naturally routes around expensive terrain.
+- **Diagonal corner-cutting:** forbid diagonal moves that clip through two wall corners, or agents visually cut through walls.
+- **Reconstruct the path** by following `cameFrom` parents from goal back to start, then reverse.
+
+Related tools (Red Blob Games):
+- **Breadth-First Search / Dijkstra maps ("flow fields"):** when *many* agents chase *one* goal (tower defense, RTS), compute one BFS/Dijkstra field from the goal and have every agent follow the gradient — far cheaper than one A* per agent.
+- **Greedy Best-First** is faster but not optimal; A* is the balanced default.
+- **JPS (Jump Point Search)** optimizes A* on uniform-cost grids.
+
+Libraries: **PathFinding.js** (grid A*/JPS/BFS, easy), or roll your own for control.
+
+**Path-following gotchas:** smooth the raw grid path (string-pulling / line-of-sight simplification) so agents don't zig-zag along cell centers; recompute paths sparingly (on target move / periodically, not every frame); handle "no path exists" explicitly.
+
+---
+
+## 2. Steering behaviors (smooth local movement)
+
+A* gives *where* to go; **steering** (Craig Reynolds) makes agents *move there naturally* by accumulating steering forces on velocity:
+- **Seek / Arrive** — move toward a target; Arrive decelerates within a slowing radius (prevents overshoot/orbit).
+- **Flee / Evade** — away from a threat.
+- **Pursuit** — seek the target's *predicted future* position.
+- **Wander** — smooth random roaming (jitter a point on a circle ahead), not teleporting random targets.
+- **Obstacle avoidance** — steer around obstacles detected by a look-ahead feeler.
+- **Separation / Alignment / Cohesion** = **Flocking (boids)** for crowds/schools/swarms.
+- **Path following** — follow a computed A* path smoothly rather than snapping cell-to-cell.
+
+Rules: **combine forces** (weighted sum or priority), **clamp to max force and max speed**, apply with **delta time**. Steering handles the smooth "how"; A*/flow fields handle the strategic "where." Use both together — A* alone looks robotic; steering alone gets stuck on walls.
+
+---
+
+## 3. Finite State Machines (FSM) — decision-making for simple agents
+
+- Agent is in exactly one **state** (Idle, Patrol, Chase, Attack, Flee); **transitions** fire on conditions (saw player → Chase; lost player → Patrol; low HP → Flee).
+- Each state has `enter()`, `update(dt)`, `exit()`. Keep transition logic centralized/table-driven.
+- Great for enemies with a handful of behaviors. **Downside:** transitions explode combinatorially (n states → up to n² transitions) — becomes spaghetti past ~6–8 states.
+- **Hierarchical FSMs** (states containing sub-states) tame some of that.
+
+---
+
+## 4. Behavior Trees (BT) — scalable decision-making
+
+When FSMs get unwieldy, use a **behavior tree**: a tree of nodes evaluated each tick, each returning **Success / Failure / Running**.
+- **Composites:** **Sequence** (run children in order, fail-fast — "AND"), **Selector/Fallback** (try children until one succeeds — "OR"), **Parallel**.
+- **Decorators:** Inverter, Repeat, Cooldown, Succeeder, condition guards.
+- **Leaves:** Actions (MoveTo, Attack, Reload) and Conditions (IsPlayerVisible, HasAmmo).
+- **`Running` is the key concept:** long actions (walking a path) return `Running` and resume next tick — don't block.
+- Far more **modular/reusable/authorable** than FSMs; the industry standard for complex NPCs. Consider a lib (behaviortree.js) or a small hand-rolled implementation.
+- **Utility AI / GOAP** are alternatives for emergent/goal-driven behavior at higher complexity.
+
+**Rule:** simple enemies → FSM; complex/multi-goal NPCs → behavior tree. Don't build a BT framework for a single 3-state slime.
+
+---
+
+## 5. Navmesh basics (3D / open spaces)
+
+- Grids are natural for tile games; for **open 3D/continuous** worlds, a **navigation mesh** (convex polygons covering walkable surfaces) is more efficient and gives smoother paths than a dense grid.
+- Run A* over the polygon graph (portal edges), then **string-pull ("funnel algorithm")** through polygon portals for a smooth, natural path.
+- In JS: **recast-navigation-js** (WASM port of industry-standard Recast/Detour) bakes navmeshes from level geometry and provides crowd/agent avoidance; three-pathfinding for a simpler Three.js navmesh follower.
+- Navmeshes handle multi-level geometry (ramps, bridges) that flat grids can't; support off-mesh links (jumps, ladders).
+
+---
+
+## 6. Performance & correctness rules
+- **Don't A* every agent every frame.** Cache paths; recompute on target movement or a throttled interval; **time-slice** pathfinding across frames or use a **web worker** for big searches so the main thread doesn't hitch.
+- **Flow field / Dijkstra map** when many agents share one goal — one computation for all.
+- **Binary heap** open set, **typed arrays** for large grids, reuse buffers (no per-search allocation churn).
+- **Run AI on a fixed timestep** (decoupled from render) for determinism/netcode; often at a lower rate than render (e.g. 10–20Hz for decisions) with steering interpolating between.
+- **Stagger** AI updates across agents (LOD-AI): distant/off-screen agents think less often.
+- Handle **dynamic obstacles**: mark blocked cells / re-path when the map changes; use local steering avoidance for moving agents.
+
+---
+
+## 7. Bug-prevention checklist
+- **Non-admissible heuristic** (Euclidean on 4-dir, Manhattan on 8-dir) → non-optimal or slow paths; match heuristic to movement.
+- **Linear open-set scan** → severe stutter on big maps; use a binary heap.
+- **A* per agent per frame** → frame drops; cache/throttle/flow-field/worker.
+- **Following raw grid path** → robotic zig-zag; smooth/string-pull and add steering.
+- **No Arrive/slowing radius** → agents overshoot and orbit the target.
+- **Corner-cutting diagonals** → agents clip through wall corners; forbid unsafe diagonals.
+- **FSM sprawl** → unmaintainable transitions; switch to a behavior tree.
+- **Blocking actions in a BT** → tree stalls; return `Running` for long actions.
+- **Ignoring "no path"** → crashes/agents freeze; handle unreachable goals explicitly.
+- **Not re-pathing on map change** → agents walk into new walls; invalidate cached paths.
+- **AI tied to render frame rate** → non-deterministic/uneven; fixed timestep, staggered.
+
+---
+
+## Defaults to apply
+- **Default AI stack:** **A\* on a grid** (binary-heap open set, movement-matched heuristic, weighted tiles) for "where," **steering behaviors** (seek/arrive/avoid/separation) for smooth "how," and an **FSM for simple enemies / behavior tree for complex NPCs** for "what to do."
+- **Use flow fields (BFS/Dijkstra map) when many agents chase one target** (tower defense/RTS) — big perf win over per-agent A*.
+- **For 3D/open worlds, generate navmeshes via recast-navigation-js**; smooth paths with the funnel algorithm.
+- **Bake in the perf guardrails**: cache paths, throttle/time-slice/worker pathfinding, run AI on a fixed lower-rate tick with steering interpolation, and stagger distant agents. Always handle "no path found."
+
+---
+
+## Sources
+- Red Blob Games — Introduction to A* (canonical, interactive): https://www.redblobgames.com/pathfinding/a-star/introduction.html ; A* Implementation Guide: https://www.redblobgames.com/pathfinding/a-star/implementation.html
+- Red Blob Games — Grids/heuristics & "Pathfinding for Tower Defense" (flow fields): https://www.redblobgames.com/pathfinding/tower-defense/ ; grid pathfinding tricks: https://www.redblobgames.com/pathfinding/grids/algorithms.html
+- Craig Reynolds — "Steering Behaviors For Autonomous Characters": https://www.red3d.com/cwr/steer/ ; Boids: https://www.red3d.com/cwr/boids/
+- Amit Patel / Red Blob — heuristics guide: https://theory.stanford.edu/~amitp/GameProgramming/Heuristics.html
+- PathFinding.js (grid A*/JPS/BFS): https://github.com/qiao/PathFinding.js
+- Jump Point Search (Harabor & Grastien): https://harablog.wordpress.com/2011/09/07/jump-point-search/
+- Behavior Trees — Chris Simpson, "Behavior trees for AI: How they work": https://www.gamedeveloper.com/programming/behavior-trees-for-ai-how-they-work ; behaviortree.js: https://github.com/Calamari/BehaviorTree.js
+- Game Programming Patterns — State (FSM): https://gameprogrammingpatterns.com/state.html
+- Recast/Detour navmesh: https://github.com/recastnavigation/recastnavigation ; recast-navigation-js: https://github.com/isaac-mason/recast-navigation-js ; three-pathfinding: https://github.com/donmccurdy/three-pathfinding
+- GOAP (Jeff Orkin, F.E.A.R.): https://alumni.media.mit.edu/~jorkin/goap.html

package/skills/grok-building-games/references/audio.md

@@ -0,0 +1,129 @@
+# Game Audio in the Browser (Web Audio API, mobile unlock, Howler.js, gain buses, spatial, latency)
+
+Consolidated from MDN Web Audio docs and Howler.js docs (see Sources). Focus: what an AI builder needs so audio **actually plays on mobile, stays low-latency, and mixes cleanly** — the #1 audio bug in browser games is "no sound on iOS."
+
+---
+
+## 1. The autoplay unlock — the single most important rule
+
+Browsers block audio until the user interacts with the page. An `AudioContext` created on page load starts in the **`"suspended"`** state and will silently play nothing until resumed.
+
+**Rule: create/resume the AudioContext (or unlock your audio lib) from inside the FIRST real user gesture** (`click`/`touchend`/`keydown`), and call `resume()` **synchronously** in that handler's call stack (iOS Safari is strict — an `await` before `resume()` can break the gesture chain).
+
+```js
+let audioCtx;
+function unlockAudio() {
+  if (!audioCtx) audioCtx = new (window.AudioContext || window.webkitAudioContext)();
+  if (audioCtx.state === 'suspended') audioCtx.resume(); // call synchronously in the gesture
+}
+// Attach once; remove after first success. `pointerdown`/`touchend`/`keydown` all count.
+window.addEventListener('pointerdown', unlockAudio, { once: true });
+window.addEventListener('keydown', unlockAudio, { once: true });
+```
+
+- Show a "tap to start" / "click to play" screen so the first gesture reliably unlocks audio (and any pointer lock / fullscreen you need). Games should not expect audio before the player clicks in.
+- Also **resume on `visibilitychange`/focus** — contexts can get re-suspended when a tab is backgrounded on mobile.
+- Listen to the `statechange` event if you need to reflect mute/paused UI.
+
+---
+
+## 2. Web Audio API vs `<audio>` vs Howler
+
+- **Don't use `<audio>` / `Audio()` elements for game SFX.** They have high latency, limited simultaneous playback, and inconsistent mobile behavior. Fine for background music streaming only.
+- **Web Audio API** is the correct foundation: decode once into an `AudioBuffer`, then fire many low-latency, overlapping `AudioBufferSourceNode`s. Gives you precise scheduling, gain buses, and spatialization.
+- **Howler.js** is the recommended default library — it wraps Web Audio (with `<audio>` fallback), auto-handles the **mobile unlock**, sprite support, fades, spatial audio, and pooling. Use it unless you specifically need raw Web Audio control. (Note: `AudioBufferSourceNode` is one-shot — you create a new source per play; that's normal and cheap. Howler manages this for you.)
+
+**Preload & decode up front:** `fetch → arrayBuffer → audioCtx.decodeAudioData` during a loading screen. Decoding on first play causes an audible hitch.
+
+---
+
+## 3. Gain buses (music / SFX / master) — do this from the start
+
+Route everything through a small mixer graph so you can control category volumes and a master mute:
+
+```
+source → (per-sound gain) → sfxBus ─┐
+music  → musicGain ────────────────┼→ masterGain → audioCtx.destination
+```
+
+```js
+const master = audioCtx.createGain();
+const musicBus = audioCtx.createGain();
+const sfxBus   = audioCtx.createGain();
+musicBus.connect(master); sfxBus.connect(master); master.connect(audioCtx.destination);
+```
+
+- Set volumes with **`gain.setTargetAtTime(value, audioCtx.currentTime, 0.02)`** or `setValueAtTime`, **not** a raw `gain.value =` mid-play — abrupt jumps cause clicks/pops. Ramp fades over ~10–50ms.
+- Perceived loudness is logarithmic: map a 0–1 slider to gain with a curve (e.g. `gain = slider²`) so sliders feel linear.
+- Howler equivalent: per-sound `volume`, category via `Howler` groups / separate `Howl` instances, and `Howler.volume()` for master.
+
+---
+
+## 4. Latency (keep audio tight)
+
+- Prefer **Web Audio buffer sources** over `<audio>` for anything reactive — element playback latency ruins hit feedback.
+- Optionally construct the context with **`{ latencyHint: 'interactive' }`** (default) for lowest latency; use `'playback'` only for pure music.
+- **Reuse decoded buffers**; never `decodeAudioData` in the gameplay hot path.
+- Schedule to `audioCtx.currentTime` for sample-accurate timing (e.g. rhythm games): `source.start(when)`. Don't rely on `setTimeout` for musical timing — use the Web Audio clock ("A Tale of Two Clocks").
+- Avoid creating hundreds of nodes per frame; cap concurrent voices and stop/disconnect finished sources (set `source.onended` to disconnect) to avoid graph buildup.
+
+---
+
+## 5. Variation & layering (feel)
+
+- **Pitch-randomize** repeated SFX so footsteps/gunfire don't sound robotic: set `source.playbackRate.value = 1 + (Math.random()*2-1)*0.1` (Howler: `rate`). Randomize volume slightly too.
+- **Layer** a sound from multiple samples (thump + body + transient) — see `game-feel-juice.md`.
+- **Round-robin / random pick** from a few variants of the same sound.
+- Use **audio sprites** (one file, offset+duration segments) to cut HTTP requests and decode overhead — Howler supports `sprite` natively.
+
+---
+
+## 6. Spatial / positional audio (for 3D or top-down games)
+
+- **`PannerNode`** positions a source in 3D relative to `audioCtx.listener`. Connect `source → panner → bus`. Update `panner.positionX/Y/Z` and `listener.positionX/Y/Z` (+ listener `forward`/`up`) each frame to match camera/player.
+- `panningModel`: `'HRTF'` (realistic, headphone-friendly, costlier) or `'equalpower'` (cheap stereo). `distanceModel`: `'inverse'`/`'linear'`/`'exponential'` with `refDistance`, `maxDistance`, `rolloffFactor` for falloff.
+- For simple **2D stereo pan** (left/right by screen x), a `StereoPannerNode` (pan −1..1) is cheaper and sufficient — don't reach for HRTF in a 2D game.
+- Howler exposes `pos()`, `orientation()`, and `pannerAttr()` for this.
+- Update positions in the render loop; set the position AudioParams via `.value` (or `setValueAtTime`) to avoid zipper noise on fast movement.
+
+---
+
+## 7. Formats & assets
+- Use **compressed formats**: `.webm`/`.ogg` (Opus/Vorbis) with an `.mp3`/`.m4a` fallback for Safari. Howler takes an array of sources and picks a supported one.
+- Keep SFX short and mono (mono halves size and works with panners); keep music stereo but compressed.
+- Watch total download; long music tracks can stream via `html5: true` in Howler (uses `<audio>`, saves memory, slightly higher latency — fine for music).
+
+---
+
+## 8. Bug-prevention checklist
+- **No sound on mobile/iOS** → didn't resume `AudioContext` synchronously inside a user gesture; add a tap-to-start unlock.
+- **Silence after backgrounding tab** → context re-suspended; resume on `visibilitychange`/focus.
+- **Clicks/pops on volume change or start/stop** → setting `gain.value` abruptly; ramp with `setTargetAtTime` and fade in/out a few ms.
+- **Laggy hit sounds** → using `<audio>` elements; use Web Audio buffer sources.
+- **Hitch on first play of a sound** → decoding at play time; preload + `decodeAudioData` during loading.
+- **Robotic repeated sounds** → no pitch/volume variation; randomize `playbackRate`.
+- **Growing memory / stuck voices** → not disconnecting finished sources or capping concurrent voices.
+- **Works in Chrome, silent in Safari** → format not supported (provide ogg/opus **and** mp3/m4a) or async break before `resume()`.
+- **No volume controls** → always expose master/music/sfx sliders + mute (accessibility + player expectation).
+
+---
+
+## Defaults to apply
+- **Default to Howler.js** for generated games (handles unlock, sprites, fades, spatial, pooling) — fall back to raw Web Audio only when we need custom DSP/scheduling.
+- **Always wire the mixer graph**: master + music + sfx gain buses with sliders and a mute, from the first commit. Map sliders through a `x²` curve.
+- **Always add a "tap to start" gate** that unlocks audio (and pointer lock/fullscreen) on the first gesture, and re-resume on visibility change — kills the #1 "no audio on mobile" bug.
+- **Preload + decode on the loading screen**, reuse buffers, pitch-randomize repeated SFX, ramp all gain changes. Provide ogg/opus + mp3 fallbacks.
+
+---
+
+## Sources
+- MDN — Web Audio API Best Practices (autoplay, gesture, gain, controls): https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Best_practices
+- MDN — `AudioContext.resume()`: https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/resume
+- MDN — `BaseAudioContext.decodeAudioData()`: https://developer.mozilla.org/en-US/docs/Web/API/BaseAudioContext/decodeAudioData
+- MDN — `GainNode` / `AudioParam.setTargetAtTime`: https://developer.mozilla.org/en-US/docs/Web/API/GainNode , https://developer.mozilla.org/en-US/docs/Web/API/AudioParam/setTargetAtTime
+- MDN — `PannerNode` + Web audio spatialization basics: https://developer.mozilla.org/en-US/docs/Web/API/PannerNode , https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Web_audio_spatialization_basics
+- MDN — `StereoPannerNode`: https://developer.mozilla.org/en-US/docs/Web/API/StereoPannerNode
+- MDN — Autoplay guide for media & Web Audio: https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide
+- Chris Wilson — "A Tale of Two Clocks" (Web Audio scheduling): https://web.dev/articles/audio-scheduling
+- Howler.js — docs & repo: https://howlerjs.com/ , https://github.com/goldfire/howler.js
+- Chrome autoplay policy for Web Audio: https://developer.chrome.com/blog/autoplay/#webaudio

package/skills/grok-building-games/references/babylon.md

@@ -0,0 +1,202 @@
+# Babylon.js — Deep Engine Guide (Havok physics, cameras, SceneOptimizer, WebGPU, assets, GUI, freezing)
+
+Babylon.js is the "batteries-included" 3D engine: built-in physics, inspector, GUI, node materials, camera behaviors, SceneOptimizer, and a first-class WebGPU backend. Use it when you want an integrated engine rather than assembling Three.js + libraries.
+
+This file assumes the general loop/orientation/perf rules and the short Babylon perf bullets in `threejs-foundational.md` and goes **much deeper** without repeating them.
+
+---
+
+## 1. Engine setup & the render loop
+
+```js
+import { Engine, Scene, Vector3 } from "@babylonjs/core";
+const engine = new Engine(canvas, true, { preserveDrawingBuffer: false, stencil: true, powerPreference: "high-performance" });
+const scene = new Scene(engine);
+engine.runRenderLoop(() => scene.render());
+window.addEventListener("resize", () => engine.resize());
+```
+
+- `engine.runRenderLoop` is Babylon's RAF loop; hook game logic into **`scene.onBeforeRenderObservable`** and read delta with **`engine.getDeltaTime()`** (ms) or `scene.getAnimationRatio()` (frame-rate compensation factor). Scale movement by delta as always.
+- Use **`scene.onBeforeRenderObservable.add(fn)`** for per-frame logic instead of stuffing everything in the render loop closure — it's observable, disposable, and composable.
+- `engine.setHardwareScalingLevel(n)` renders at 1/n resolution (n>1 = lower res, big mobile win; n<1 = supersample).
+
+---
+
+## 2. WebGPU (real, production-ready — prefer it with fallback)
+
+Babylon has a mature **`WebGPUEngine`** that dramatically cuts CPU-side draw-call overhead and enables compute shaders.
+
+```js
+import { WebGPUEngine, Engine } from "@babylonjs/core";
+async function createEngine(canvas) {
+  if (await WebGPUEngine.IsSupportedAsync) {
+    const e = new WebGPUEngine(canvas, { antialias: true });
+    await e.initAsync();            // REQUIRED before creating a Scene
+    return e;
+  }
+  return new Engine(canvas, true, { powerPreference: "high-performance" });   // WebGL2 fallback
+}
+```
+
+- `WebGPUEngine.initAsync()` **must be awaited** before you create the scene — a top setup bug.
+- **Snapshot rendering** (`engine.snapshotRendering = true`, WebGPU only) records the command stream for a static scene and replays it, slashing CPU cost for scenes whose draw list doesn't change. Set `snapshotRenderingMode` and refresh when the scene structure changes.
+- WebGPU supports **compute shaders** (particles, culling, GPU picking) — not available on WebGL.
+
+---
+
+## 3. Cameras & Camera Behaviors (free polish)
+
+- **ArcRotateCamera** — orbit/inspection. Set limits and inertia so it feels good: `lowerRadiusLimit`/`upperRadiusLimit`, `lowerBetaLimit`/`upperBetaLimit`, `wheelPrecision`, `panningSensibility`, `inertia`. `camera.attachControl(canvas, true)`.
+- **UniversalCamera** — FPS/free camera (WASD + mouse; combines FreeCamera + touch/gamepad). `camera.applyGravity`, `camera.checkCollisions`, `camera.ellipsoid` for simple capsule collision against meshes with `mesh.checkCollisions = true` (built-in, no physics engine needed).
+- **FollowCamera** — third-person chase; set `radius`, `heightOffset`, `rotationOffset`, `cameraAcceleration`, `maxCameraSpeed`.
+- **Keep `camera.maxZ` as small as practical** and `minZ` as large as practical — improves depth precision, culling, and overdraw.
+
+**Behaviors** attach polished motion for free (this is a Babylon differentiator):
+- **FramingBehavior** — auto-frames a target mesh nicely (`camera.useFramingBehavior = true`).
+- **BouncingBehavior** — soft bounce at radius limits.
+- **AutoRotationBehavior** — idle turntable spin (great for product/menu scenes).
+Enable via `camera.useAutoRotationBehavior = true` etc., then tune the behavior object.
+
+---
+
+## 4. Physics — Havok (V2 plugin, the current default)
+
+Havok is the recommended physics engine (WASM, fast, deterministic-ish). Use the **Physics V2** API (`PhysicsAggregate` / `PhysicsBody` / `PhysicsShape`).
+
+```js
+import { HavokPlugin, PhysicsAggregate, PhysicsShapeType } from "@babylonjs/core";
+import HavokPhysics from "@babylonjs/havok";
+
+const havok = await HavokPhysics();                 // MUST await — loads the .wasm
+scene.enablePhysics(new Vector3(0, -9.81, 0), new HavokPlugin(true, havok));
+
+// dynamic body:
+new PhysicsAggregate(sphere, PhysicsShapeType.SPHERE, { mass: 1, restitution: 0.6, friction: 0.5 }, scene);
+// static body: mass 0
+new PhysicsAggregate(ground, PhysicsShapeType.BOX, { mass: 0 }, scene);
+```
+
+- **`await HavokPhysics()` is mandatory** and the #1 init error. First `HavokPlugin` arg (`_useDeltaForWorldStep`, usually `true`) makes physics use engine delta for smoother variable framerate.
+- **Bundler gotcha (Vite/webpack):** the `HavokPhysics.wasm` file must be served. Use `await HavokPhysics({ locateFile: f => \`/assets/${f}\` })` or configure the bundler to copy the wasm.
+- **PhysicsAggregate** is the easy path: it builds the matching `PhysicsShape` + `PhysicsBody` automatically. For fine control (compound shapes, sharing shapes across many bodies) build `PhysicsBody` + `PhysicsShape` yourself.
+- Read/write motion: `aggregate.body.setLinearVelocity(v)`, `applyImpulse`, `setMassProperties`, `body.setMotionType(PhysicsMotionType.ANIMATED/DYNAMIC/STATIC)`.
+- Collisions/triggers: `body.setCollisionCallbackEnabled(true)` + `body.getCollisionObservable().add(...)`; use `PhysicsShapeType.CONTAINER`/mesh shapes sparingly (mesh colliders are expensive — prefer primitives or convex hulls).
+- **glTF imports:** apply physics to the actual mesh, not the `__root__` transform node — often re-parent or use the child mesh; otherwise transforms get double-applied.
+- Debug: `new BABYLON.Debug.PhysicsViewer(scene).showBody(aggregate.body)`.
+- Older engines (Cannon, Ammo, Oimo) still exist via the V1 plugin, but **Havok V2 is the recommended choice** for new games. Ammo remains an option if you need its specific features/soft bodies.
+
+---
+
+## 5. Asset loading — containers & glTF (do it right for level switching)
+
+- **glTF/GLB is the format.** `import "@babylonjs/loaders";` then `await ImportMeshAsync("model.glb", scene)` (or `SceneLoader.ImportMeshAsync` / `AppendSceneAsync`). Use Draco + KTX2/Basis compression.
+- **AssetContainer is the key tool for reusable/levels:** `LoadAssetContainerAsync(url, scene)` loads meshes/materials/textures **without adding them to the scene**. Then `container.addAllToScene()` / `container.removeAllFromScene()` and, crucially, **`container.dispose()`** to fully free GPU resources when unloading a level. This gives clean load/unload cycles and lets you **instantiate** copies:
+  - `container.instantiateModelsToScene(name => name, cloneMaterials)` clones a loaded model (with skeletons/animations) cheaply for spawning many enemies from one load.
+- **AssetsManager** gives progress-tracked batch loading with tasks (mesh, texture, binary) and `onProgress`/`onFinish` — good for a loading screen.
+- **Babylon does not GC GPU resources** any more than Three.js does — dispose meshes/materials/textures or (better) dispose the AssetContainer when leaving a level.
+
+---
+
+## 6. Materials & Node Material Editor
+
+- **PBRMaterial** for realistic; **StandardMaterial** for simpler/cheaper. Reuse material instances across meshes (shared material = fewer state changes).
+- **Node Material** — visual shader graph (Node Material Editor at nme.babylonjs.com). Export as JSON and load with `NodeMaterial.ParseFromSnippetAsync("snippetId", scene)` or from file. Great for custom effects (dissolve, water, force-fields) without writing GLSL/WGSL by hand, and it compiles to both WebGL and WebGPU.
+- **Material plugins** and `material.freeze()` (see freezing below) reduce per-frame shader/uniform recompute.
+
+---
+
+## 7. GUI (built-in — no DOM juggling)
+
+`import { AdvancedDynamicTexture, Button, ... } from "@babylonjs/gui";`
+
+- **Fullscreen UI:** `AdvancedDynamicTexture.CreateFullscreenUI("ui")` — a 2D overlay in screen space for HUD/menus (buttons, sliders, text, stack panels, grids). Resolution-independent, handles pointer input, works on mobile.
+- **In-world UI:** apply an ADT to a mesh's material for diegetic screens/labels; use `linkWithMesh` to attach floating labels/health bars that track 3D objects.
+- Prefer Babylon GUI over hand-rolled DOM overlays for game HUDs — it integrates with the scene, scales with the canvas, and doesn't fight CSS.
+
+---
+
+## 8. Performance — FREEZING is the Babylon superpower
+
+For anything static, tell Babylon to stop recomputing it. This is the highest-leverage Babylon optimization and under-used in generated code.
+
+- **`mesh.freezeWorldMatrix()`** — mesh never moves ⇒ skip world-matrix recompute each frame. Call `unfreezeWorldMatrix()` if it needs to move again. Also set `mesh.doNotSyncBoundingInfo = true` for static meshes.
+- **`material.freeze()`** — material params fixed ⇒ skip uniform/shader re-evaluation. `unfreeze()` to change it.
+- **`scene.freezeActiveMeshes()`** — locks the active-mesh (culling) list; use when the set of visible meshes doesn't change (e.g., fixed camera or after setup). `unfreezeActiveMeshes()` when it changes. Huge CPU savings on large static scenes.
+- **`scene.blockMaterialDirtyMechanism = true`** while bulk-creating materials to avoid repeated recompiles.
+
+Other big levers (deeper than the three.js file's bullets):
+- **`scene.performancePriority = ScenePerformancePriority.Intermediate` (or `Aggressive`)** — auto-applies a bundle of optimizations (freezes active meshes, skips some checks). `Aggressive` disables per-mesh picking/some features — verify nothing you need breaks.
+- **Thin instances** (`mesh.thinInstanceAdd(matrix)` / `thinInstanceSetBuffer`) for thousands of identical static objects → one draw call, cheaper than regular instances. Regular **instances** (`mesh.createInstance()`) when you need per-instance picking/parenting.
+- **`scene.autoClear = false`** (and `autoClearDepthAndStencil`) when the scene fully covers the viewport — skips the clear.
+- **`scene.skipPointerMovePicking = true`** if you don't need hover/move picking (picking on every pointer move is a silent CPU cost). Set `mesh.isPickable = false` on non-interactive meshes.
+- **`scene.skipFrustumClipping`**, `mesh.alwaysSelectAsActiveMesh` — micro-tune culling for known-visible meshes.
+- **Merge meshes** (`Mesh.MergeMeshes([...], true, true, undefined, false, true)`) for static geometry sharing a material.
+- **LOD:** `mesh.addLODLevel(distance, lowMesh)` and `addLODLevel(farDist, null)` to cull entirely far away.
+- Keep **`camera.maxZ` low**; limit lights (each real-time light multiplies shader cost); use **shadow generators sparingly** with small map sizes and `RefreshRate` for static lights.
+- Avoid per-frame allocations: use **`TmpVectors.Vector3[0]`** scratch vectors and the `...ToRef()` method variants (`addToRef`, `scaleToRef`) instead of returning new objects.
+
+---
+
+## 9. SceneOptimizer — adaptive quality to hit a target FPS
+
+Instead of guessing settings per device, let Babylon degrade quality until it reaches a target framerate.
+
+```js
+import { SceneOptimizer, SceneOptimizerOptions } from "@babylonjs/core";
+// preset tiers try progressively harder optimizations to reach the target FPS:
+SceneOptimizer.OptimizeAsync(scene,
+  SceneOptimizerOptions.HighDegradationAllowed(60),   // target 60 fps
+  () => console.log("reached target"),
+  () => console.log("could not reach target"));
+```
+
+- Presets: `LowDegradationAllowed`, `ModerateDegradationAllowed`, `HighDegradationAllowed(targetFps)`. They apply, in escalating order, optimizations like: reduce hardware scaling, merge meshes, disable shadows, reduce texture size, disable post-processes, cut particle counts.
+- You can build a custom `SceneOptimizerOptions` with specific `SceneOptimization` steps and priorities. Great for shipping one build that adapts from low-end mobile to desktop automatically.
+
+---
+
+## 10. Inspector & profiling
+
+- **`import "@babylonjs/inspector"; scene.debugLayer.show()`** — the in-browser inspector: scene tree, per-mesh stats, materials, textures, and a **Statistics/Performance** tab (draw calls, active meshes, frame breakdown, GPU/CPU time). This is the fastest way to find what's slow.
+- Watch **draw calls** and **active meshes** counts; use the profiler to see whether you're CPU-bound (draw calls, matrix updates → freeze/instance) or GPU-bound (overdraw, shader cost, texture size → LOD/compression/fewer lights).
+
+---
+
+## 11. Common Babylon pitfalls (checklist)
+
+- Forgetting `await WebGPUEngine.initAsync()` / `await HavokPhysics()` → cryptic init failures.
+- Havok `.wasm` not served by the bundler → physics silently absent; use `locateFile` / copy the wasm.
+- Not disposing meshes/materials/textures or the AssetContainer on level switch → GPU memory leak.
+- Applying physics to a glTF `__root__` node → double transforms / wrong positions.
+- Static scene running full cost because nothing is frozen → freeze world matrices, materials, active meshes.
+- Picking on every pointer move (`skipPointerMovePicking` not set; meshes left `isPickable`) → CPU drain.
+- Too many real-time lights / large shadow maps → shader + fill cost explosion.
+- Per-frame `new Vector3()`/`Matrix` allocations → GC churn; use `TmpVectors` + `...ToRef()`.
+- Huge `camera.maxZ` → z-fighting and wasted culling range.
+- Mesh colliders for everything instead of primitive/convex shapes → physics perf collapse.
+
+---
+
+## Defaults to apply
+
+- **Reach for Babylon when the game wants an integrated engine** (built-in physics, GUI, inspector, camera behaviors) rather than the assemble-it-yourself Three.js route.
+- **Always await `initAsync()` (WebGPU) and `HavokPhysics()`**, and wire the **WebGPU-with-WebGL2-fallback** pattern by default.
+- **Emit freezing by default for static content**: `freezeWorldMatrix()` on non-moving meshes, `material.freeze()` on static materials, and `scene.freezeActiveMeshes()` / `performancePriority` once the scene is set up. This is the single biggest Babylon win most generated code misses.
+- **Ship `SceneOptimizer.OptimizeAsync(scene, HighDegradationAllowed(60))`** so one build adapts from low-end mobile to desktop instead of hardcoding quality.
+- **Use AssetContainers** for level load/unload (`instantiateModelsToScene` to spawn many from one load; `dispose()` to free on unload) to avoid the memory leaks that plague generated 3D games.
+- **Use Babylon GUI** (fullscreen ADT) for HUDs/menus instead of DOM overlays; use **camera Behaviors** (Framing/AutoRotation) for free menu/product polish.
+- Set **`skipPointerMovePicking`, low `maxZ`, `isPickable=false`** on non-interactive meshes as defaults.
+
+---
+
+## Sources
+- Babylon.js docs — Optimizing your scene (freeze APIs, autoClear, performancePriority, active meshes): https://doc.babylonjs.com/features/featuresDeepDive/scene/optimize_your_scene
+- Babylon.js docs — Physics V2 / Havok (`enablePhysics`, `PhysicsAggregate`, `PhysicsBody/Shape`): https://doc.babylonjs.com/features/featuresDeepDive/physics/usingPhysicsEngine and https://github.com/BabylonJS/Documentation/blob/master/content/features/featuresDeepDive/physics/v2/usingPhysicsEngine.md
+- `@babylonjs/havok` npm (init pattern, `await HavokPhysics()`, `locateFile`): https://www.npmjs.com/package/@babylonjs/havok
+- Babylon.js docs — WebGPU (`WebGPUEngine`, `initAsync`, snapshot rendering): https://doc.babylonjs.com/setup/support/webGPU
+- Babylon.js docs — Cameras & Camera Behaviors: https://doc.babylonjs.com/features/featuresDeepDive/cameras/camera_introduction and https://doc.babylonjs.com/features/featuresDeepDive/behaviors/cameraBehaviors
+- Babylon.js docs — Asset Containers & glTF loading: https://doc.babylonjs.com/features/featuresDeepDive/importers/assetContainers and https://doc.babylonjs.com/features/featuresDeepDive/importers/loadingFileTypes
+- Babylon.js docs — SceneOptimizer: https://doc.babylonjs.com/features/featuresDeepDive/scene/sceneOptimizer
+- Babylon.js docs — GUI: https://doc.babylonjs.com/features/featuresDeepDive/gui/gui ; Node Material: https://doc.babylonjs.com/features/featuresDeepDive/materials/node_material/nodeMaterial
+- Babylon.js docs — Inspector: https://doc.babylonjs.com/toolsAndResources/inspector
+- Babylon.js forum — optimization best practices thread: https://forum.babylonjs.com/t/best-practices-for-optimizing-babylon-js-scenes-not-just-on-lower-end-devices/58688

package/skills/grok-building-games/references/collision-physics.md

@@ -0,0 +1,122 @@
+# Collision & Physics for Browser Games (2D AABB/SAT/broadphase, swept collision, 3D Rapier/cannon, fixed timestep)
+
+Consolidated from MDN game collision docs, Gaffer On Games, Rapier/cannon-es docs, and the SAT/swept-AABB canon (see Sources). Focus: what an AI builder needs so collisions are **correct and stable** — the classic bugs are tunneling through walls at speed, jitter, sinking into the ground, and O(n²) slowdowns.
+
+---
+
+## 1. Do you even need a physics engine?
+
+- **Simple arcade games** (platformers, top-down, breakout, shooters) → hand-rolled **AABB** collision + resolution is often better: fully deterministic, no dependency, and you control feel. Most 2D games don't need Box2D/Matter.
+- **Rigid-body simulation** (stacking, ragdolls, joints, realistic bouncing, vehicles) → use an engine: **2D → Matter.js / Planck.js (Box2D) / Rapier2D**; **3D → Rapier (rapier.js, WASM, fast, favored) / cannon-es / Ammo.js**.
+- **Rule:** don't pull in a full rigid-body engine for a game that just needs "does the player box overlap the wall box." Physics engines add non-determinism and tuning overhead. Match the tool to the game.
+
+---
+
+## 2. 2D collision detection primitives
+
+- **AABB (axis-aligned bounding box)** — the workhorse. Two boxes overlap iff they overlap on **both** axes:
+  ```js
+  const hit = a.x < b.x+b.w && a.x+a.w > b.x && a.y < b.y+b.h && a.y+a.h > b.y;
+  ```
+  Cheap, ideal for tile maps, most 2D gameplay. Only valid for **non-rotated** boxes.
+- **Circle/distance** — for round objects: compare squared distance to summed radii (`dx*dx+dy*dy < (r1+r2)²` — avoid `sqrt`).
+- **SAT (Separating Axis Theorem)** — for **rotated boxes / convex polygons**: two convex shapes don't intersect iff there exists an axis (a face normal of either shape) on which their projections don't overlap. Test all face normals; if any gap exists → no collision. The **minimum overlap axis gives the collision normal + penetration depth** (the MTV, minimum translation vector) for resolution.
+  - SAT only works on **convex** shapes — decompose concave shapes into convex pieces.
+  - For circles vs polygon, also test the axis from circle center to nearest vertex.
+
+---
+
+## 3. Broadphase: don't test every pair (O(n²) killer)
+
+Checking all pairs is O(n²) — fine for dozens, catastrophic for hundreds/thousands. Split into **broadphase** (cheap culling of pairs that can't touch) → **narrowphase** (exact AABB/SAT on survivors).
+
+- **Uniform spatial hash grid** — bucket objects into cells by position; only test objects sharing a cell. Best default for many similar-sized, evenly-distributed objects (bullets, particles, .io games). Cell size ≈ average object size (2–3×).
+- **Quadtree** — recursively subdivide space; good for **non-uniform / clustered / varied-size** objects. Rebuild or update each frame.
+- **Sort and sweep (sweep-and-prune)** — sort AABB endpoints on an axis; good when objects move coherently.
+- **Rule:** the moment you have >~100 dynamic colliders, add a broadphase. Physics engines do this internally; hand-rolled collision needs it explicitly.
+
+---
+
+## 4. Tunneling & swept collision (the top correctness bug)
+
+**Tunneling:** a fast object moves so far in one frame that it passes *through* a thin wall without ever overlapping it on any frame. Discrete AABB checks miss it entirely.
+
+Fixes (in order of preference):
+- **Swept collision (continuous / CCD):** instead of testing the box at its new position, test the **motion path**. **Swept AABB** computes the fraction of the frame `t (0..1)` at which the moving box first touches the target, then moves the object to exactly that contact point and resolves — no overlap ever occurs. Essential for bullets, fast platformers, thin walls.
+- **Substepping / smaller fixed timestep:** break a big move into several small steps and test each. Simpler than swept, good enough for moderate speeds.
+- **Raycast** fast/small projectiles (bullets) instead of moving a body — cast a ray along the travel and hit the first collider.
+- **Minimum wall thickness** ≥ max per-step travel is a cheap band-aid but fragile.
+- In engines, **enable CCD** on fast bodies (Rapier `setCcdEnabled(true)`, cannon-es CCD options).
+
+---
+
+## 5. Collision *resolution* (not just detection)
+
+- **AABB resolution:** compute penetration on each axis; push the object out along the **axis of least penetration** (this makes walls/floors behave). Zero the velocity component on the axis you resolved (hitting the floor kills downward velocity).
+- **Resolve axes separately** for platformers ("move X, resolve X, then move Y, resolve Y") — prevents catching on tile seams and gives clean wall-slide behavior.
+- **Ground detection:** you're grounded if resolving downward this frame; set a small `coyoteTime` (see input skill) for feel.
+- **Avoid sinking/jitter:** apply a tiny "skin"/epsilon and don't over-correct; with engines, tune restitution/friction and let the solver settle. Resting jitter usually means variable timestep or fighting corrections.
+- With **SAT**, resolve along the MTV (normal × penetration depth); reflect/scale velocity along the normal for bounces.
+
+---
+
+## 6. Fixed timestep — the stability foundation (tie-in)
+
+**Physics MUST run on a fixed timestep with an accumulator** ("Fix Your Timestep!", Gaffer On Games). Variable `dt` makes collision/physics non-deterministic and unstable (tunneling worsens, springs explode, replays/netcode desync).
+```js
+const STEP = 1/60; let acc = 0;
+function frame(dt){ acc += Math.min(dt, 0.25);        // clamp to avoid spiral-of-death after tab-out
+  while (acc >= STEP){ physicsStep(STEP); acc -= STEP; }
+  const alpha = acc / STEP; render(alpha);            // interpolate between last two states
+}
+```
+- **Clamp accumulated dt** (e.g. ≤0.25s) so a backgrounded tab doesn't trigger a "spiral of death" of catch-up steps.
+- **Interpolate rendering** between the previous and current physics state using `alpha` for smooth visuals at any display rate.
+- All physics engines expect a fixed step — call `world.step()` at a fixed rate, not per rAF with variable dt. Rapier is deterministic given fixed steps + same inputs (good for netcode).
+
+---
+
+## 7. 3D character controllers
+
+Falling capsules from raw rigid-body dynamics feel bad for players — use a **kinematic character controller**:
+- **Rapier `KinematicCharacterController`** (favored): create via `world.createCharacterController(offset)`, use a kinematic body + capsule/collider, then each fixed step call `computeColliderMovement(collider, desiredTranslation)` and read `computedMovement()` to get the collision-corrected move. Built-in **autostep** (stairs), **snap-to-ground** (don't float off slopes/ramps), **max slope climb angle**, and slide-along-walls. Set gravity/jump yourself (kinematic = you control motion).
+- **cannon-es**: `PointerLockControlsCannon` example / a sphere or capsule body; more manual.
+- Rules: use a **capsule** (not a box) so the player slides over small steps and around corners; apply movement each **fixed step**; do ground checks via the controller's grounded flag or a short downward ray; keep the visual mesh synced to the body each frame (with interpolation).
+
+---
+
+## 8. Bug-prevention checklist
+- **Variable timestep for physics** → non-deterministic, jitter, worse tunneling, netcode desync; fixed step + accumulator.
+- **No dt clamp** → "spiral of death" after tab-out; clamp accumulated dt.
+- **Fast objects passing through walls** → tunneling; use swept AABB / CCD / raycast / substeps.
+- **SAT on concave shapes** → wrong results; decompose into convex pieces.
+- **AABB test on rotated boxes** → false/missed hits; use SAT for rotation.
+- **O(n²) pair testing** → frame drops at scale; add spatial hash/quadtree broadphase.
+- **Resolving both axes at once** → catching on tile seams / clipping corners; resolve X then Y.
+- **Not zeroing velocity on the resolved axis** → gravity accumulates, object shudders into ground.
+- **Player floats off ramps / trips on stairs** → missing snap-to-ground / autostep; use a character controller (Rapier) with those enabled.
+- **Box character controller** → snags on edges; use a capsule.
+- **`sqrt` in hot distance checks** → wasted perf; compare squared distances.
+- **Reading physics state without render interpolation** → visible stutter at high refresh rates; interpolate with `alpha`.
+
+---
+
+## Defaults to apply
+- **Right-size the solution:** hand-rolled **AABB + separate-axis resolution + swept collision** for arcade/2D/platformers; **Rapier** (2D or 3D) when we need real rigid bodies, joints, or a character controller. Don't default to a heavy engine.
+- **Always run physics on a fixed timestep with a clamped accumulator + render interpolation** — this one pattern prevents tunneling instability and jitter, and it ties directly into the fixed-timestep game loop (see `threejs-foundational.md`).
+- **Ship anti-tunneling by default** for fast objects (swept AABB or CCD/raycast for projectiles) and a **broadphase (spatial hash or quadtree)** once there are many colliders.
+- **For 3D player movement, generate a Rapier `KinematicCharacterController` with a capsule, autostep, snap-to-ground, and slope limits** — not a raw dynamic body — so movement feels right out of the box.
+
+---
+
+## Sources
+- MDN — 2D collision detection (AABB, circle, SAT): https://developer.mozilla.org/en-US/docs/Games/Techniques/2D_collision_detection
+- Gaffer On Games — "Fix Your Timestep!": https://gafferongames.com/post/fix_your_timestep/ ; "Collision Response and Coulomb Friction": https://gafferongames.com/post/collision_response_and_coulomb_friction/
+- Swept AABB collision — jitter physics / "Swept AABB Collision Detection and Response" (gamedev): https://www.gamedev.net/tutorials/programming/general-and-gameplay-programming/swept-aabb-collision-detection-and-response-r3084/
+- SAT reference — Metanet/N tutorial & dyn4j "SAT (Separating Axis Theorem)": https://dyn4j.org/2010/01/sat/ ; https://www.metanetsoftware.com/technique/tutorialA.html
+- Spatial hashing / broadphase — "Spatial Hashing" (gamedev) & quadtree collision (GameDev Academy): https://gameprogrammingpatterns.com/spatial-partition.html
+- Rapier — docs (rigid bodies, CCD, character controller): https://rapier.rs/docs/ ; character controller: https://rapier.rs/docs/user_guides/javascript/character_controller ; JS bindings: https://github.com/dimforge/rapier.js
+- cannon-es — repo/examples (incl. PointerLockControlsCannon): https://github.com/pmndrs/cannon-es ; https://pmndrs.github.io/cannon-es/
+- Matter.js: https://brm.io/matter-js/ ; Planck.js (Box2D): https://github.com/piqnt/planck.js
+- Box2D docs (continuous collision / solver background): https://box2d.org/documentation/
+- Game Programming Patterns — Spatial Partition & Game Loop: https://gameprogrammingpatterns.com/spatial-partition.html , https://gameprogrammingpatterns.com/game-loop.html