melonjs 20.7.0 → 20.9.0

package/skills/melonjs-3d-assets/SKILL.md

@@ -33,13 +33,34 @@ or it renders flat. See `melonjs-3d` for the camera.
 |---|---|---|
 | `scale` | `1` | pixels per glTF unit, applied to the whole scene. Blender's metre-scale export usually needs 20–100. |
 | `container` | `game.world` | where the nodes are added |
-| `onLoaded` | `app.onLevelLoaded` | called with the **level id**, not the scene |
+| `onLoaded` | `game.onLevelLoaded` | called with the **level id**, not the scene |
 | `async` | `false` | return a promise that settles once the scene is in the world, instead of a boolean |
 | `rightHanded` | `true` | see below |
 | `lights` | `true` | instantiate authored `KHR_lights_punctual` lights as `Light3d` world children |
 | `lightIntensityScale` | — | keep authored intensity ratios instead of normalising every light to 1 |
+
+Those `game.` defaults are the engine's, not a recommendation. `game` is
+whichever `Application` finished `init()` last, so **pass `container`
+explicitly** from a `Stage`, where the right one is in your hand:
+
+```js
+level.load("sceneId", { container: this.app.world, scale: 40 });
+```
 | `castGroundShadow` | inherits the app setting (**on**) | blob shadows for this scene's meshes |
 | `shadowGroundY` | each object's own base | world Y of the floor the blobs land on |
+| `shadowOpacity` | `0.45` | how dark the blobs are |
+| `shadowScale` | `1` | blob footprint multiplier; `0` or less hides them |
+| `shadowOffset` | `0` | slide them along the light, in blob radii; needs `shadowGroundY`, which is the only way a scene load can name its floor |
+| `shadowStretch` | `1` | lengthen them along it, clamped to 3 |
+| `shadowDirectionX` / `shadowDirectionZ` | `0` | the ground direction they are cast along |
+| `shadowGroundNormal` | world up | up normal of the floor, for a scene that is not level |
+| `shadowLight` | — | a `Light3d` to take the direction from, re-read every draw |
+
+`shadowGroundPoint` is **not** on that list, and a scene load drops it without
+a warning. One floor POINT shared by every part of a scene would stack the
+whole thing's blobs on a single spot, which is nonsense where one floor
+HEIGHT is perfectly sensible. Ground that is not flat is a per-object answer:
+set it on the parts you want once the load has finished.
 
 `onLoaded` receives the level id — it is a "done" signal, not a handle on the
 scene. You need it, or `async`: with the game loop running, `level.load` stops
@@ -79,10 +100,11 @@ world.addChild(new Trigger(x, y, {
 }));
 ```
 
-`scale`, `rightHanded`, `lights`, `lightIntensityScale`, `castGroundShadow` and
-`shadowGroundY` all travel, and so do `container`, `onLoaded`, `flatten` and
-`setViewportBounds`. That list is **hardcoded** in `Trigger` — a `level.load`
-option outside it is dropped without a warning.
+Every `level.load` option above travels, including the whole ground-shadow set,
+and so do `container`, `onLoaded`, `flatten` and `setViewportBounds`. The list is
+**derived** from `level.loadOptions` rather than restated, so an option added to
+a level format arrives here for free. It used to be hand-written and had fallen
+three releases behind the glTF path (#1649).
 
 **`async` is deliberately excluded.** A trigger with a `color` + `duration`
 sets it itself, because it needs the promise to sequence its fade and reveal; a
@@ -129,6 +151,74 @@ asset fails to load. Export uncompressed.
 `"auto"`) drives sampling; glTF materials that declare their own sampler carry
 it through. Pixel-art models want `"nearest"`.
 
+### `baseColorFactor` is LINEAR; a `tint` is sRGB
+**Pass the factor as a setting and the encoding is done for you.** `Mesh`
+takes `baseColorFactor` — `[r, g, b]` in 0..1, linear, which is the form the
+loader hands back — so a mesh built from a node passes the node's own field
+straight through:
+
+```js
+const node = loader.getGLTF("palm").nodes[0];
+const mesh = new Mesh(x, y, {
+    vertices: node.vertices, normals: node.normals,
+    uvs: node.uvs, indices: node.indices, normalize: false,
+    baseColorFactor: node.baseColorFactor,   // encoded, not scaled
+});
+```
+
+The setting's type admits `undefined`, because the loader declares the field
+optional and that line has to compile under `exactOptionalPropertyTypes`.
+`Sprite3d` forwards it too, and `mesh.setBaseColorFactor(f)` applies one after
+construction. For an ordinary sRGB colour use `setTint` instead — the rest of
+this section is why the two are different.
+
+
+The loader handles this for you. It only bites when you build meshes yourself
+from `loader.getGLTF(name).nodes` — which is what you do for per-node control,
+since there is no public per-node `Mesh` factory — and reach for the obvious
+`* 255`:
+
+```js
+// WRONG — every untextured material renders far too dark
+mesh.tint.setColor(f[0] * 255, f[1] * 255, f[2] * 255, 1);
+```
+
+glTF defines `pbrMetallicRoughness.baseColorFactor` in **linear** space (spec
+3.9.2), while a melonJS `tint` is 8-bit **sRGB** — the same space as a CSS
+colour or a texel out of a PNG. They are not the same number, and the gap is
+widest exactly where art tends to live:
+
+| linear | `* 255` | correct | 
+|---|---|---|
+| 0.05 | 13 | 63 |
+| 0.20 | 51 | 124 |
+| 0.42 | 107 | 173 |
+| 0.72 | 184 | 221 |
+| 1.00 | 255 | 255 |
+
+`Color.setLinear` does the encoding for you — it is the linear-space sibling of
+`setFloat`, which takes the same `0..1` range but treats it as already-sRGB:
+
+```js
+mesh.tint.setLinear(...node.baseColorFactor);   // encodes; what you want
+mesh.tint.setFloat(...node.baseColorFactor);    // does NOT encode; too dark
+```
+
+It clamps, so the slightly out-of-range factors some exporters emit cannot turn
+into `NaN`, and it keeps the encoded value as a float rather than rounding
+through 8 bits.
+
+Measured on a scene whose every material went through `* 255`: mean frame luma
+85.5 against 147.8 once encoded, with each channel sitting 60 to 73 counts low.
+
+**Fix it early.** It is nearly invisible as a bug, because the scene still
+looks coherent — just moody — so lighting and bloom thresholds get tuned
+against the wrong values. Correcting the encoding afterwards blows the frame
+out and forces a lighting retune, which is a much bigger job than the encode.
+
+This applies only to factors coming out of glTF. Colours **you** author for the
+screen, a HUD tint or a glow, are already sRGB: `* 255` is right for those.
+
 ## Lights
 
 Authored `KHR_lights_punctual` lights — sun, point, spot — become `Light3d`
@@ -223,11 +313,21 @@ Omit `shadowGroundY` and each blob sits at its own object's base at full
 strength, which is right for props already resting on the ground. Set it when
 things jump or fly, so the shadow stays on the floor and shrinks with height.
 
-A blob is centred on its caster's x/z and is never offset by light direction, so
-a wide flat-bottomed prop hides its own shadow under itself. Raising
-`shadowGroundY` to force one into view floats the blob up over the object as a
-dark ring rather than sliding it clear — the setting is for things that leave
-the ground, not a visibility knob.
+A blob sits on its caster's x/z by default, so a wide flat-bottomed prop hides
+its own shadow under itself. The knob for that is `shadowScale`, which widens
+the blob until its edge shows past the prop, or `shadowOffset` plus a direction,
+which slides it clear. **Not** `shadowGroundY`: raising that floats the blob up
+over the object as a dark ring instead of sliding it out, because it is for
+things that leave the ground, not a visibility control.
+
+For a light that is not overhead, name one: `shadowLight` takes the direction
+from a `Light3d` and is re-read every draw, so a day or night cycle carries
+every shadow in the scene with it. Nothing is inferred — a scene with no named
+light and no `shadowDirectionX`/`Z` gets no direction, because a scene with
+several lights has no non-arbitrary answer. `shadowStretch` lengthens the blob
+along that direction and fades it as it pulls; `shadowGroundNormal` lays it on
+a floor that is not level. All of these apply to an `InstancedMesh` too, as one
+value for the whole set.
 
 ## OBJ/MTL
 
@@ -423,6 +523,7 @@ need a prefix.
 | one part of a rig renders rigid while its curves exist | it was re-parented mid-pose, baking the inverse into `matrix_parent_inverse` |
 | a merged mesh lights wrong along one axis | it inherited the active object's non-uniform scale on join — apply transforms |
 | an authored palette comes out uniformly dark, but white is correct | linear values written to a strip that is saved verbatim |
+| every untextured glTF material renders too dark, white unaffected | `baseColorFactor` is linear and a `tint` is sRGB — `tint.setLinear(...)`, not `* 255` or `setFloat` |
 | `getGLTF(name).nodes[0]` geometry lands in the wrong place | `nodes` is per primitive, each with its own `world`; merge to one primitive and export at the origin |
 | a prop casts no visible shadow | wide and flat-bottomed — the blob is under it; `shadowGroundY` haloes it rather than revealing it |
 | shadows only show on casters near the camera | `shadowGroundY` is on the wrong side — Y-down means the floor is a **greater** y, so a `pos.y - lift` puts the blob inside the caster and the depth test leaves only a hairline ring |

package/skills/melonjs-audio/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: melonjs-audio
-description: "Use this skill for sound and music in melonJS — loading audio, playing effects, background tracks, audio sprites, volume and muting, spatial audio, and procedural tone/noise generation. Covers the mandatory audio.init format list, the directory-not-file src convention, playTrack versus play, and browser autoplay unlocking. Triggers on: audio, sound, audio.init, audio.play, playTrack, stopTrack, audio.load, sfx, music, volume, mute, sprite audio, stereo, panner, audio.tone, audio.noise, getAudioContext, autoplay."
+description: "Use this skill for sound and music in melonJS — loading audio, playing effects, background tracks, audio sprites, volume and muting, spatial audio, procedural tone/noise generation, and effects such as reverb or a compressor on the whole mix. Covers the mandatory audio.init format list, the directory-not-file src convention, playTrack versus play, the fixed listener and pixel-scale distances of positional sound, and browser autoplay unlocking. Triggers on: audio, sound, audio.init, audio.play, playTrack, stopTrack, audio.load, sfx, music, volume, mute, fade, rate, pitch, sprite audio, stereo, panner, position, listener, refDistance, reverb, echo, delay, compressor, distortion, filter, audio effect, audio.tone, audio.noise, getAudioContext, getMasterGain, hasAudio, hasFormat, autoplay, no sound, sound stopped working, context suspended, silent after idle, clips silent but tones play."
 license: MIT
 ---
 
@@ -29,6 +29,10 @@ await loader.preload([
    gives you a 404. A `data:audio/...` URL is the one exception: it is used
    verbatim, prefix and extension skipped.
 
+`audio.hasAudio()` says whether the browser can play audio at all, and
+`audio.hasFormat("ogg")` whether it can play one codec — useful for choosing
+the `init` list or hiding a sound toggle on a device with no audio.
+
 Order matters — melonJS tries the listed formats left to right, so put the
 preferred one first. Two formats (`"webm,mp3"` or `"mp3,ogg"`) is the usual
 belt-and-braces for codec coverage. Accepted tokens: `mp3`, `mpeg`, `opus`,
@@ -51,7 +55,33 @@ loops, and it records the name as the current track — which is what
 `app.pauseOnBlur` / `resumeOnFocus` / `stopOnBlur` act on, via
 `state.pause(true)` → `pauseTrack()`. `stopTrack()`, `pauseTrack()`,
 `resumeTrack()` and `getCurrentTrack()` all read that same single slot, so there
-is only ever one current track. Stop it in `onDestroyEvent`.
+is only ever one current *track name*.
+
+**`playTrack` stops the track it replaces** (20.9.0 and later; before that it
+only assigned the slot, and the previous bed kept looping underneath where
+`stopTrack()` could no longer reach it). So a scene just states its own music
+and nothing cancels anything:
+
+```js
+class PlayScreen extends Stage {
+    onResetEvent() { audio.playTrack("battle", 0.6); }
+}
+```
+
+Calling it with the track already playing **restarts** it, the same as `play()`
+on any other clip. A scene that wants the bed to carry on across a screen
+change should not call it again, rather than rely on it doing nothing.
+
+`stopTrack()` is for where nothing follows: a scene whose successor has no
+music of its own, or a game going quiet. `state.change` destroys the outgoing
+stage **before** it resets the incoming one, so a `stopTrack()` in
+`onDestroyEvent` and a `playTrack()` in the next `onResetEvent` are in that
+order and never cancel each other.
+
+To end a track on an event rather than on a screen change, say a run ending
+while the player stays on the same screen, `audio.fade(name, from, 0, ms)`
+takes it down gently. A fade to zero is silence, not a stop, so the voice is
+still live until something stops it or replaces it.
 
 Every per-clip function throws `audio clip <name> does not exist` if the clip was
 never loaded, and `play` additionally throws when a named sprite region is
@@ -72,8 +102,9 @@ audio.play("sfx", { sprite: "jump" });
 
 The optional third element marks a region as looping. `play()`'s second argument
 takes either the original `loop` boolean or a `PlayOptions` object
-(`sprite`, `loop`, `onend`, `volume`) — both forms work. Object fields win over
-the positional `onend` / `volume` arguments when both are supplied.
+(`sprite`, `loop`, `onend`, `volume`, plus the spatial `follow`, `at` and
+`stopWithTarget`) — both forms work. Object fields win over the positional
+`onend` / `volume` arguments when both are supplied.
 
 ## Clip state
 
@@ -88,7 +119,9 @@ audio.state("theme");           // "unloaded" | "loading" | "loaded"
 `stop()`, `pause()` and `resume()` take an optional instance `id` too; omit it
 and the whole group is affected. `audio.stop()` with no arguments at all stops
 every sound. `audio.seek(name)` reads the position, `audio.seek(name, s, id?)`
-writes it; `audio.rate` and `audio.fade(name, from, to, ms, id?)` round it out.
+writes it; `audio.rate(name, r, id?)` changes speed and pitch together
+(`0.5`..`4.0`, `1` is normal) and `audio.fade(name, from, to, ms, id?)` ramps
+the volume (`0`..`1`, over milliseconds).
 
 ## Volume and muting
 
@@ -105,6 +138,14 @@ audio.muted();                  // global mute state
 `audio.unload(name)` / `audio.unloadAll()` free the decoded buffers; `unload`
 also clears the current-track pointer if it named that clip.
 
+### Losing focus mutes everything
+
+When the window loses focus the engine mutes all audio, and unmutes it on
+focus. That covers sound effects, procedural `tone` / `noise` and the mixer
+effects, not just the music track that `state.pause()` pauses. Set
+`pauseOnBlur: false` in the application settings if a clip has to keep playing
+in the background.
+
 ## Spatial audio
 
 ```js
@@ -120,10 +161,113 @@ Called with just the clip name, each of the four returns the current value —
 `stereo` gives `0` and `position` gives `[0, 0, 0]` before they have ever been
 set.
 
+### Let the engine place the sound: `follow` and `at`
+
+The shortest path to positional audio is a play option. Both take **world
+pixels**, the same numbers as `pos.x` / `pos.y`, and the engine converts to
+audio space for you:
+
+```js
+// tracks the renderable every frame, and stops when it is destroyed
+audio.play("ufo", { follow: this, stopWithTarget: true });
+
+// pinned to a fixed world point
+audio.play("portal", { at: { x: 1200, y: 480 } });
+```
+
+`follow` and `at` are mutually exclusive: passing both throws, rather than
+quietly picking one. `follow` re-reads the target's **absolute** position each
+frame, so a renderable nested in a moving container is placed correctly. `stopWithTarget` is what you
+want for a sound that belongs to an entity: without it a looping clip on a
+destroyed renderable keeps playing from wherever it last was. `audio.unfollow(id)`
+detaches a sound and leaves it playing where it is.
+
+### Move the listener, don't do the subtraction yourself
+
+```js
+audio.setListener(player);                  // tracked every frame
+audio.listener(x, y, z);                    // or place it by hand
+audio.listener();                           // → [x, y, z] in world pixels
+audio.listenerOrientation(fx, fy, fz, ux, uy, uz);
+audio.setListener(null);                    // stop tracking
+```
+
+With a listener in the world, source positions are **absolute world
+coordinates**, not offsets. The older idiom — passing `source.pos.x -
+player.pos.x` every frame with the ear stuck at the origin — still works, but
+only if you leave the listener at `(0, 0, 0)`; mixing the two double-counts the
+player's position.
+
+`setListener` places the ear at the target's absolute position after the world
+updates. A plain renderable contributes position only, so a side-scroller keeps
+the default "facing into the screen" and gets left-right panning plus distance
+falloff from the one call.
+
+A `Camera3d` target also contributes its **orientation**, taken from its own
+basis, so turning the camera swings the stereo image with it. That is the whole
+of what a posed 3D scene needs:
+
+```js
+audio.setListener(app.viewport);                 // in a 3D scene this is a Camera3d
+audio.play("drone", { follow: saucer });         // scene coordinates, directly
+```
+
+**Y is down.** Every one of these calls takes and returns melonJS world
+coordinates, where y grows downward; the Y-up flip Web Audio wants happens
+inside the engine. Pass `pos.y` as it is.
+
+### The defaults are tuned for pixels
+
+`audio.setSpatialDefaults()` / `getSpatialDefaults()` set the panner attributes
+a sound placed through `follow` or `at` starts from, and the shipped values
+assume a 2D game measured in pixels:
+
+| attribute | default |
+|---|---|
+| `refDistance` | `240` |
+| `maxDistance` | `10000` |
+| `panningModel` | `"equalpower"` |
+| `distanceModel` | `"inverse"` |
+
+With those, a placed sound is at full volume out to 240 px and then halves
+each time the distance doubles: about half at 480, a quarter at 960, roughly
+an eighth at 2000. `maxDistance` is where it stops getting quieter.
+
+**They apply to placed sounds only.** A plain `audio.play()` followed by a
+bare `audio.position()` keeps WebAudio's own defaults, where `refDistance` is
+`1` METRE and a source 100 px out is already near silent. That path is
+unchanged, which is what keeps old code working.
+
+Widen or tighten the field once, globally:
+
+```js
+audio.setSpatialDefaults({ refDistance: 400, maxDistance: 20000 });
+```
+
+`audio.panner(name, attrs, id?)` still overrides per clip or per voice, and it
+merges over the current values rather than replacing them, so writing one field
+no longer resets the rest. Called without an `id` it writes the **group**
+defaults, which is what a clip-wide tuning call looks like.
+
+For a 3D scene where front-to-back matters, `panningModel: "HRTF"` is worth the
+cost; for a 2D one it mostly muddies the stereo image.
+
+### `stereo` and `position`: last writer wins
+
+Both drive the same per-instance panner node, and the engine swaps the node to
+match whichever you called **last**: `stereo()` after `position()` gives you
+plain left-right panning, and `position()` after `stereo()` gives you 3D
+falloff. Neither silently does nothing any more, and the order you call them in
+no longer matters.
+
+Still pick one per voice and stay with it. A 2D game usually wants `stereo()`
+alone; reach for `position()` (or `follow` / `at`) when distance falloff
+matters.
+
 **Spatial audio is WebAudio-only.** On a clip loaded with `stream: true` (or
 `html5: true`) `stereo` / `position` / `orientation` return early and do
 nothing — which is exactly the case for the long music tracks people reach for
-first.
+first. `follow` and `at` are equally inert on a streamed clip.
 
 ## Procedural sound
 
@@ -167,6 +311,42 @@ custom Web Audio work; both return `null` when audio is unavailable, so guard.
 Connect a custom graph to the master gain rather than `ctx.destination` if you
 want it to respect `setVolume` / `muteAll`.
 
+## Effects on all game audio: reverb, echo, compressor
+
+There is no built-in reverb, delay, distortion or compressor, and no per-clip
+filter on file playback. There does not need to be one for the common case:
+every buffered clip and every `tone` / `noise` passes through the master gain
+on its way to the speakers, so an effect inserted **after** it applies to the
+whole mix.
+
+```js
+const ctx = audio.getAudioContext();   // creates the context if needed
+const master = audio.getMasterGain();
+if (ctx && master) {                   // null when audio is unavailable
+    const comp = ctx.createDynamicsCompressor();
+    master.disconnect();               // master -> speakers becomes
+    master.connect(comp);              // master -> compressor -> speakers
+    comp.connect(ctx.destination);
+}
+```
+
+Any Web Audio node or chain works the same way: a `ConvolverNode` with an
+impulse response for reverb, a `DelayNode` with a feedback gain for echo, a
+`BiquadFilterNode` for an underwater low-pass, a `WaveShaperNode` for
+distortion. Do it once, at startup; the master gain lives as long as the
+context.
+
+- **`setVolume` / `muteAll` still work**, because they act on the master gain,
+  which is now upstream of the effect.
+- **Streamed clips are not affected.** A `stream: true` (`html5: true`) clip
+  plays through an `<audio>` element and never reaches the master gain — which
+  is usually the music. Preload the music buffered if it must go through the
+  effect.
+- **It is all or nothing.** The per-instance nodes are not public, so an effect
+  on one clip only (muffling one sound behind a wall) is not possible through
+  this route; `noise` has its own `filter` for generated sounds.
+- To remove it, reverse the wiring: `master.disconnect(); master.connect(ctx.destination);`.
+
 ## Autoplay: sound is silent until the first gesture
 
 Browsers create the audio context in the `suspended` state and keep it there
@@ -174,8 +354,36 @@ until a user interaction. Nothing errors — audio simply does not play, which
 reads as a bug. melonJS installs one-shot `touchstart` / `touchend` / `click` /
 `keydown` unlock listeners on `document`, so the first real gesture anywhere
 unlocks it and fires the clip's `on.unlock` callback. Design for it: start music
-on the first click or key press, not on load. `tone` / `noise` additionally call
-`ctx.resume()` themselves, best-effort, on every invocation.
+on the first click or key press, not on load. The usual way to guarantee one is
+a gate screen — a splash, a "press any key", or a health/photosensitivity notice
+— whose dismissal *is* the gesture, so the menu behind it can start its music
+unconditionally. Note the unlock listens for `keydown` too, so a keyboard-only
+player is covered.
+
+### The context can stop again, long after it was unlocked
+
+Unlocking is only the first half. A context that has been running can be
+stopped later: melonJS suspends it itself after thirty idle seconds to save
+power, and browsers stop it for their own reasons, a backgrounded tab among
+them. Recovery is automatic and you do not have to write it: a user gesture
+or the tab coming back to the front revives it, and a deliberate idle
+suspension is left alone so the power saving still works.
+
+**The symptom worth memorising**, because it points straight at the cause:
+if `tone` and `noise` are audible while loaded clips are silent, the context
+is healthy and the engine's playback state is wrong. The generators only need
+a live context, where a clip goes through the engine's state machine and can
+be parked waiting for a resume. It is never "the context died".
+
+Both routes ask the same owner to resume, so hearing a tone tells you the
+hardware is back; it tells you nothing about whether the clips waiting on the
+engine's state have been let go.
+
+That asymmetry was a real Safari-only bug: thirty idle seconds before the
+first click left the engine waiting on a suspend the browser never confirmed,
+so every clip queued for the life of the page while generated sound played
+normally. Fixed in 20.9, and the lesson generalises: a clip that will not play
+is a state question, not a hardware one.
 
 Playback also starts **asynchronously**: `play()` returns an id while the
 instance is still starting, so `playing()` can legitimately be `false` on the
@@ -209,6 +417,10 @@ same switch under its backend name.) `on` accepts the full lifecycle set:
 `play`, `pause`, `stop`, `end`, `fade`, `seek`, `rate`, `volume`, `mute`,
 `unlock`.
 
+Outside a preload, `audio.load(asset, onload?, onerror?)` takes the same
+descriptor and loads one clip at runtime — for a level's sounds fetched when
+the level starts. The loader calls the same function, so the same rules apply.
+
 Re-preloading a manifest that already contains a loaded clip is a no-op —
 `audio.unload(name)` first if you genuinely want to reload it.
 
@@ -241,12 +453,21 @@ either way — blank screen, empty console.
 | nothing plays until the user clicks | browser autoplay policy — expected, design around it |
 | `playing()` is false right after `play()` | playback starts asynchronously |
 | music keeps playing after leaving a scene | used `play` instead of `playTrack`, or no `stopTrack` in teardown |
+| two music tracks playing at once, and `stopTrack()` only silences one | melonJS before 20.9.0: `playTrack` did not stop the previous track, so the scene that started it had to `stopTrack()` on the way out |
+| the music restarts when returning to a screen | `playTrack` with the track already playing restarts it; don't call it again if the bed should carry on |
 | `stereo` / `position` do nothing | the clip was loaded with `stream: true` — spatial is WebAudio-only |
 | one format works, another does not | that codec is not in the `audio.init()` list |
 | audio behind a login fails, or the preload hangs | `loader.setOptions({ withCredentials: true })` — the same switch every asset type uses |
 | a streamed clip ignores `withCredentials` | `stream: true` plays through an `<audio>` element; preload it buffered instead |
 | `audio.tone(440, {...})` does nothing / errors | `tone` takes a single options object with a required `duration` |
 | `tone` / `noise` are silent with no error | no WebAudio context — `getAudioContext()` returned `null` |
+| a positioned sound is silent, or only audible right next to the player | placed by hand with `audio.position()`, which keeps WebAudio's metre defaults (`refDistance: 1`) — use `follow` / `at`, or set `refDistance` in pixels with `audio.panner()` |
+| a positioned sound does not follow the player | nothing is placing it — `audio.play(name, { follow: entity })`, and `audio.setListener(player)` once |
+| a followed sound pans to the wrong side as the player moves | the listener was moved **and** positions are still relative to the player — with a listener in the world, pass absolute world coordinates |
+| a looping sound outlives the entity that owns it | `follow` without `stopWithTarget: true` |
+| a voice panned with `stereo()` loses its 3D placement, or the reverse | one panner node per voice and last writer wins — pick one per voice and stay with it |
+| a reverb / compressor on the master gain does not affect the music | the music is `stream: true`, which bypasses the master gain — preload it buffered |
+| sound effects keep playing after the window loses focus | `pauseOnBlur` and `stopOnBlur` are both off, so the engine is deliberately leaving the game running in the background |
 
 ## Related skills
 

package/skills/melonjs-camera-and-drawing/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: melonjs-camera-and-drawing
-description: "Use this skill for camera control and immediate-mode drawing in melonJS — following a target, viewport bounds, shake and fade, secondary cameras, and drawing shapes, lines and gradients inside a custom draw(). Covers world versus screen coordinate conversion, clipping and masking, and baking with CanvasRenderTarget. Triggers on: Camera2d, viewport, follow, setBounds, shake, fadeIn, fadeOut, worldToLocal, localToWorld, colorMatrix, renderer.fill, renderer.stroke, Rect, Ellipse, Polygon, Line, Gradient, clipRect, mask, CanvasRenderTarget, NoiseTexture2d."
+description: "Use this skill for camera control and immediate-mode drawing in melonJS — following a target, viewport bounds, shake and fade, secondary cameras, and drawing shapes, lines and gradients inside a custom draw(). Covers world versus screen coordinate conversion, clipping and masking, baking with CanvasRenderTarget, and reading pixels back. Triggers on: Camera2d, viewport, follow, setBounds, shake, fadeIn, fadeOut, worldToLocal, localToWorld, colorMatrix, renderer.fill, renderer.stroke, Rect, Ellipse, Polygon, Line, Gradient, clipRect, mask, CanvasRenderTarget, NoiseTexture2d, lookAt, setBasis, toImageData, getImageData, toBlob, toDataURL, screenshot, read pixels."
 license: MIT
 ---
 
@@ -24,6 +24,44 @@ frame-rate independent since 19.7. `AXIS.HORIZONTAL` / `AXIS.VERTICAL` /
 
 Call it from the followed entity's `onActivateEvent`, using `this.parentApp`.
 
+**For anything that is NOT the camera, the same smoothing is `math.damp`.**
+`follow()` is frame-rate independent for you; a value you smooth yourself is
+not, and `lerp(current, target, alpha)` once per frame moves further on a slow
+frame than on a fast one, so the feel changes with the frame rate:
+
+```js
+import { math } from "melonjs";
+
+// dt is milliseconds, damp wants seconds
+this.speed = math.damp(this.speed, targetSpeed, 5, dt / 1000);
+this.turret.angle = math.damp(this.turret.angle, wanted, 8, dt / 1000);
+```
+
+`damp(current, target, lambda, dt)`. Higher `lambda` converges faster; it is
+the rate, not a per-frame fraction.
+
+**For a DIRECTION rather than a scalar, the tool is `slerp`**, on `Vector2d`
+and `Vector3d` alike. Two reasons, and the second is the one that bites:
+
+```js
+// a turret, a ship, a camera: a facing that chases another facing
+facing.slerp(toTarget, 1 - Math.exp((-6 * dt) / 1000));
+```
+
+1. Damping a facing component by component drags it through the inside of the
+   circle, so it slows at the ends, rushes the middle and comes out shorter
+   than it went in.
+2. **Damping a scalar heading goes the wrong way round the wrap.** From 170
+   degrees to -170 degrees is 20 degrees apart, and the numbers are 340 apart,
+   so the turret sweeps all the way round the back. `slerp` works on the
+   directions, takes the shorter arc by construction, and has no wrap to cross.
+
+See `melonjs-3d` for the rest, including what happens between two exactly
+opposite directions. Use it for camera pans you drive yourself,
+turret tracking, zoom, input smoothing and any value that should chase another
+one — it is the tool that is easy to miss and easy to hand-write slightly
+wrong.
+
 ## Bounds are the thing people forget
 
 The camera's bounds default to the **renderer size**, so it cannot scroll until
@@ -53,6 +91,26 @@ The fade pair is named from the *colour's* point of view, not the scene's:
 (screen starts at `color` and clears). Both default to a 1000 ms duration and
 take an optional completion callback.
 
+**`fadeIn` does not HOLD the colour.** The effect completes, the camera's own
+update destroys it, and the screen snaps back — which is why the engine's
+idiom does the work in the completion callback and then calls `fadeOut`. To
+keep a filled screen across a state change, own the effect: a completed one
+marked `isPersistent` is left alone, and taking it away again is yours to do.
+
+```js
+const hide = camera.addCameraEffect(
+    new FadeEffect(camera, { color: "#000", duration: 400, direction: "in" }),
+);
+hide.isPersistent = true;          // survives completion AND Camera2d#reset()
+// ... later, once the new stage is up
+camera.removeCameraEffect(hide);   // which destroys it for you
+```
+
+That is exactly what `state.transition()` is built on, so a hand-rolled
+transition behaves the same way. Before 20.9 a persistent effect was destroyed
+on completion like any other, which is why a transition used to flash the
+scene it had just covered.
+
 `colorMatrix` (`brightness` / `contrast` / `saturate` / `hueRotate` / `sepia` /
 `invertColors`, all chainable) is the cheap way to grade a whole camera: the
 camera appends it as a final post-effect pass by itself whenever the matrix is
@@ -102,6 +160,24 @@ coordinate that puts your label on the wrong side of the screen. Skip it.
 Do not hand-roll this with `Matrix3d` — the perspective divide and the behind-
 camera case are exactly what gets it wrong.
 
+## Pointing a 3D camera
+
+A 2D camera's only rotation is `roll`. A 3D one can be pointed at a target, and
+optionally banked at the same time:
+
+```js
+camera.lookAt(target);                  // pitch and yaw; roll untouched
+camera.lookAt(target, up);              // and bank, where `up` is what should
+                                        // appear UP ON SCREEN
+camera.setBasis(right, up, forward);    // or pose it from a basis outright
+```
+
+`up` has no default, so the one-argument form behaves exactly as it always
+did. Because this engine renders Y-down, a level horizon is `(0, -1, 0)`.
+`setBasis` is the writable counterpart of `getBasis` and takes the camera's
+own columns, whose `up` points down the screen: the opposite sense, one
+negation apart. See `melonjs-3d` for the full account.
+
 ## Secondary cameras
 
 Cameras are first-class — a minimap is a second camera with `autoResize = false`
@@ -195,9 +271,57 @@ hand `.canvas` on for exactly that reason.
 
 This is the sanctioned performance idiom before reaching for a custom shader.
 
+**For a solid fill you do not need a canvas at all.** `Renderer.getWhitePixel()`
+is a shared, lazily-created 1x1 white canvas, and stretching it is how you get
+a coloured rectangle, a bar, a flat screen overlay, or a blank carrier for an
+effect that paints from scratch:
+
+```js
+const bar = new Sprite(0, 0, { image: Renderer.getWhitePixel() });
+bar.anchorPoint.set(0, 0);
+bar.setTint("#39d0ff");
+bar.scale(width, height);     // a 1x1 texture, so scale IS the size in pixels
+```
+
+It is also the right texture under a `ShaderEffect` whose body ignores the
+incoming colour, and it keeps the quad batched with everything else using it.
+When you DO need a real canvas, `Renderer.createCanvas(width, height)` is the
+engine's allocator: it hands back an `OffscreenCanvas` where that is safe and
+throws on a zero dimension, instead of `document.createElement("canvas")`.
+
 `renderer.toFrameTexture()` is the related tool for capturing the *current frame*
 into a texture for a shader to sample.
 
+## Reading pixels back
+
+To grab what a render target holds, as pixels or as a file:
+
+```js
+const pixels = await rt.toImageData();      // ImageData, any backend
+const blob   = await rt.toBlob();           // Blob, "image/png" by default
+const url    = await rt.toDataURL();        // data: URL string
+const bitmap = await rt.toImageBitmap();    // ImageBitmap
+```
+
+All four are promises, and that is not decoration. **WebGPU cannot read a
+texture back synchronously** — it has to map a buffer, which only completes on
+a later tick. So `toImageData()` is the portable readback and the other three
+are built on it.
+
+There is also a synchronous `getImageData()` on `CanvasRenderTarget` and
+`WebGLRenderTarget`, because those two genuinely can. It is **not** part of the
+`RenderTarget` contract and it **throws on a WebGPU target**. Reach for it only
+in code that already knows which backend it is on; anything backend-agnostic
+should await `toImageData()`.
+
+```js
+// fine in canvas-only or WebGL-only code
+const data = rt.getImageData(0, 0, w, h);
+
+// the version that works wherever the game runs
+const data = await rt.toImageData(0, 0, w, h);
+```
+
 ## Symptom → cause
 
 | symptom | cause |
@@ -212,7 +336,9 @@ into a texture for a shader to sample.
 | clipping in the wrong place inside a container | container clips in local coordinates, after its own translate |
 | `container.clipping = true` does nothing | container has no explicit size, so `width`/`height` are `Infinity` |
 | baked target draws nothing / throws | passed the `CanvasRenderTarget`, not its `.canvas` |
+| `getImageData()` throws on one machine but not another | that one is on the WebGPU renderer, which cannot read back synchronously — `await toImageData()` instead |
 | frame rate drops with many static draws | bake into a `CanvasRenderTarget` instead |
+| a 3D camera cannot be given an arbitrary up | `Camera3d#lookAt(target, up)` or `setBasis(right, up, forward)`; see `melonjs-3d` |
 
 ## Related skills