@supersoniks/concorde 4.9.96 → 4.9.97-visual-stack.0

package/src/core/components/functional/shader/shader.md

@@ -0,0 +1,207 @@
+# Shader
+
+`sonic-shader` — renderer WebGL2 **compatible ShaderToy** (pass `Image` + buffers `A–D` + `Common`).
+
+For WGSL / WebGPU (1 pass, same DP glue): [`sonic-webgpu`](#core/components/functional/webgpu/webgpu.md/webgpu).
+
+Import opt-in (hors bundle `functional`) :
+
+<sonic-code language="javascript">
+  <template>
+import "@supersoniks/concorde/shader";
+import type {
+  SonicShaderConfig,
+  SonicShaderOutput,
+  ShaderExtractOutput,
+  ShaderRoi,
+  ShaderMouseMode,
+  ShaderHoverParam,
+} from "@supersoniks/concorde/shader";
+  </template>
+</sonic-code>
+
+CDN (after the core): `concorde-shader.bundle.js` — see [CDN addons](#docs/_getting-started/cdn-addons.md/cdn-addons).
+
+Uniforms supportés : `iResolution`, `iTime`, `iTimeDelta`, `iFrame`, `iFrameRate`, `iMouse`, `iDate`, `iChannel0–3`, `iChannelResolution`, `iChannelTime`, `iSampleRate`, `uParam0–3`.
+
+Textures `channel0–3` : images **ou** vidéos (`.webm` / `.mp4` / …).
+
+## Multipass sur une photo
+
+Buffer A simule une height-map d’ondes (ripple) ; Image réfracte la photo via le gradient — **sans teinte**. Glisser la souris pour faire des vagues.
+
+<docs-lit-demo for="docs-shader-multipass-demo"></docs-lit-demo>
+
+Photo + depth parallax (HF): [Visual stack](#docs/_misc/visual-stack.md/visual-stack).
+
+## Chaînage d’effets (officiel)
+
+**Règle :** brancher des passes dans **un** `sonic-shader` (buffers `A–D` → `Image`), pas deux composants via dataProvider.
+
+| Besoin | Mécanisme |
+|--------|-----------|
+| Sortie d’un effet → entrée du suivant | `image-ch0="buffer-a"` (ou `buffer-b`… / `self`) — **même contexte GPU**, FBO ping-pong |
+| Params / sync UI | `dataProvider` → `uParam0…3` **et** canaux (`channel0`, `bufferACh0`…) — string URL ou `SonicMediaRef` |
+| Hors viewport | `release-offscreen` (défaut) : libère textures/FBO, garde le contexte |
+
+Les passes intermédiaires sont **déjà offscreen** (FBO). Pas besoin d’un second `sonic-shader` invisible.
+
+Deux instances séparées ne partagent pas leurs textures WebGL (contextes distincts). Un bus texture via DP n’existe pas (et un `toBlob`/`dataURL` ne serait pas temps réel).
+
+Démo pédagogique : Buffer&nbsp;A = Sobel → Image = overlay + `param0`.
+
+<docs-lit-demo for="docs-shader-chain-demo"></docs-lit-demo>
+
+Exemple de câblage :
+
+<sonic-code language="html">
+  <template>
+&lt;sonic-shader
+  play
+  buffer-a-ch0="/photo.jpg"
+  image-ch0="buffer-a"
+  image-ch1="/photo.jpg"
+&gt;&lt;/sonic-shader&gt;
+&lt;!-- .bufferA = effet A ; .image = effet B qui lit iChannel0 = buffer A --&gt;
+  </template>
+</sonic-code>
+
+## Rollover / roll-out (`hover-param`)
+
+Au survol, le composant anime un `uParamN` de `0` → `1` (et l’inverse au leave), avec easing `hover-speed`. Le shader lit ce param pour mixer l’effet on/off (zoom, aberration, ripple…). `play-on-hover` n’anime que pendant le hover / la transition.
+
+Type : `ShaderHoverParam` = `0` | `1` | `2` | `3` | `"param0"`…`"param3"` | `""`.
+
+<docs-lit-demo for="docs-shader-hover-demo"></docs-lit-demo>
+
+## Sortie / sampling (`sample` + dataProvider)
+
+`dataProvider` est **bidirectionnel** : `param0…` en entrée ; en sortie `mouseX` / `mouseY` / `hovering` / `frame` (+ `param0…` si `out-data-provider` est séparé, ou le slot `hover-param` seul si le DP est partagé avec le formulaire). Avec `sample`, lit la couleur sous le curseur → `sampleR/G/B/A` + `sampleLuma` (`SonicShaderOutput`). Event `out` (payload complet, y compris les params). Optionnel : `out-data-provider` pour séparer l’écriture.
+
+**Publication paresseuse** : rien n’est écrit (ni `readPixels`) tant qu’aucun champ méta/sample n’a de lecteur DP (`@handle(frame)` / `@subscribe` / `sonic-value`…) et qu’aucun listener `out` n’est branché. Un `sonic-input name="param0"` ne compte **pas** comme consommateur out (évite le write-back qui remettait les champs à 0). Dès qu’un vrai consommateur apparaît, les maj partent au rythme de `out-interval`. Avec `sample`, le `readPixels` suit le même gate.
+
+Pattern typique — écoute `frame` + lecture massive :
+
+<sonic-code language="typescript">
+  <template>
+@handle(shaderOutKey.frame)
+onFrame() {
+  const snap = get(shaderOutKey); // mouseX/Y, sampleR/G/B, luma…
+}
+  </template>
+</sonic-code>
+
+<docs-lit-demo for="docs-shader-out-demo"></docs-lit-demo>
+
+## Extraction patches / singularities (`extract`)
+
+A GPU pass writes a **score mask** (R channel) into a buffer; `extract` scans it, crops **patches** around peaks, and publishes scalars + `blobUrl` for downstream use.
+
+```mermaid
+flowchart LR
+  Src[source] --> BufA[BufferA feature map]
+  BufA --> BufB[BufferB blur / refine]
+  BufB --> BufC[BufferC score mask]
+  BufB --> Image[Image overlay]
+  Src --> Image
+  BufC --> Extract[extract findPeaks]
+  Image --> Extract
+  Extract --> Patches[blobUrl patches]
+```
+
+| Input | Role |
+|-------|------|
+| `rois` (DP) | Optional manual ROIs `{ x, y, w?, h?, patch? }` |
+| `extract-mask` | Score buffer (R channel) — e.g. `buffer-c` |
+| `extract-source` | Color pass to crop (`image` by default) |
+
+`detect` scores: mask R channel (prefer a soft map, not hard-clipped), then **renormalized** to the best peak in the batch (`1` = strongest). `findPeaks` sorts and keeps the top ones (NMS ≈ `min(patch/3, 16)` px).
+
+| Output | Role |
+|--------|------|
+| `extracts` / `extractCount` / `extractSeq` | Items + count + sequence |
+| Event `extract` | `ShaderExtractOutput` |
+| `readRegion` / `extractNow()` | On-demand API |
+
+**Lazy**: no `readPixels` without a DP reader / `extract` event listener.
+
+<sonic-code language="typescript">
+  <template>
+@handle(extractKey.extractSeq)
+onExtract() {
+  const extracts = get(extractKey.extracts) ?? [];
+}
+  </template>
+</sonic-code>
+
+Live Harris → patches → WebGPU: [Visual stack](#docs/_misc/visual-stack.md/visual-stack).
+
+## API typée (`SonicShaderConfig`)
+
+| Champ / attribut | Type | Rôle |
+|------------------|------|------|
+| `image` | `string` | GLSL pass Image (**requis**, propriété JS) |
+| `bufferA`…`bufferD` / `buffer-a`… | `string` | Passes multipass |
+| `common` | `string` | GLSL partagé |
+| `channel0`…`channel3` | `string` | URLs textures (toutes passes) |
+| `imageCh0`… / `image-ch0`… | `string` | Override canal : URL, `self`, `buffer-a`…, `none` |
+| `play` | `boolean` | Boucle RAF (défaut `true`) |
+| `playOnHover` / `play-on-hover` | `boolean` | Anime seulement au hover / transition |
+| `mouse` | `boolean` | Pointeur → `iMouse` (défaut `true`) |
+| `mouseMode` / `mouse-mode` | `ShaderMouseMode` | `direct` \| `target` |
+| `mouseSpeed` / `mouse-speed` | `number` | Poursuite target (1/s, défaut `8`) |
+| `hoverParam` / `hover-param` | `ShaderHoverParam` | Slot `uParamN` on/off rollover |
+| `hoverSpeed` / `hover-speed` | `number` | Easing hover (1/s, défaut `10`) |
+| `param0`…`param3` | `number` | → `uParam0`…`uParam3` |
+| `dataProvider` | `string` | Publisher entrée (`param0`…) + sortie par défaut |
+| `outDataProvider` / `out-data-provider` | `string` | Publisher de sortie (défaut = `dataProvider`) |
+| `sample` | `boolean` | Lit la couleur sous le pointeur → `sampleR…` / `sampleLuma` |
+| `outInterval` / `out-interval` | `number` | Intervalle publication + event `out` (ms, défaut `80`) |
+| `extract` | `boolean` | Active extraction patches / singularités |
+| `extractInterval` / `extract-interval` | `number` | Throttle extract ms (défaut = out-interval) |
+| `extractSource` / `extract-source` | `string` | Pass couleur : `image` \| `buffer-a`… |
+| `extractMask` / `extract-mask` | `string` | Pass mask : `none` \| `buffer-a`… |
+| `extractThreshold` / `extract-threshold` | `number` | Seuil peaks 0–1 (défaut `0.5`) |
+| `extractMax` / `extract-max` | `number` | Cap détection (défaut `8`) |
+| `extractPatch` / `extract-patch` | `number` | Patch px par défaut (défaut `16`, max 128) |
+| `extractBitmaps` / `extract-bitmaps` | `boolean` | Génère `blobUrl` par item |
+| `extractDataProvider` / `extract-data-provider` | `string` | Publisher extract (défaut = out) |
+| `aspectRatio` / `aspect-ratio` | `string` | Ratio CSS forcé |
+| `dprMax` / `dpr-max` | `number` | Plafond DPR (défaut `2`) |
+| `releaseOffscreen` / `release-offscreen` | `boolean` | Hors viewport : libère textures/FBO (garde le contexte). `dispose` + loseContext au disconnect (défaut `true`) |
+
+## Markup
+
+<sonic-code language="html">
+  <template>
+&lt;sonic-shader
+  mouse
+  mouse-mode="target"
+  mouse-speed="6"
+  hover-param="0"
+  hover-speed="12"
+  play
+  play-on-hover
+  release-offscreen
+  channel0="/docs/shader-samples/color.jpg"
+&gt;&lt;/sonic-shader&gt;
+  </template>
+</sonic-code>
+
+Assigne le GLSL via `.image` (et `.bufferA`, …) depuis JS / Lit.
+
+## Events
+
+`error` — compile / link / WebGL (`detail.message`, `detail.pass`, `detail.infoLog`).
+
+`out` — snapshot `SonicShaderOutput` (throttlé par `out-interval`, **seulement s’il y a un listener**) : souris, params, samples.
+
+`extract` — snapshot `ShaderExtractOutput` (throttlé, paresseux) : patches manuels + détectés.
+
+## Notes
+
+- WebGL2 requis. Hors viewport (`release-offscreen`) : textures/FBO libérés, contexte conservé (évite loseContext sur le même canvas). Slot navigateur (~8–16) libéré au disconnect.
+- **Chaînage d’effets** : multipass dans un seul composant (`buffer-a`… → `image-chN`). Pas de bus texture entre deux `sonic-shader` via dataProvider.
+- Vidéos : lecture auto muted + loop.
+- Hors scope : Sound, Cubemap, VR ; chaînage multi-instances / contexte GPU partagé.
+- Pas d’import dans le bundle `functional` core.

package/src/core/components/functional/shader/shader.spec.ts

@@ -0,0 +1,288 @@
+import { expect, describe, it, vi, afterEach, beforeAll } from "vitest";
+import TestUtils from "@supersoniks/concorde/test-utils/TestUtils";
+import { parseChannelSource } from "./gl-runtime";
+import { buildFragmentSource } from "./shadertoy-prelude";
+import "./shader";
+import { SonicShader } from "./shader";
+
+describe("parseChannelSource", () => {
+  it("résout self / buffers / url / fallback", () => {
+    expect(parseChannelSource("self", "")).toEqual({ kind: "self" });
+    expect(parseChannelSource("buffer-a", "")).toEqual({
+      kind: "buffer",
+      buffer: "bufferA",
+    });
+    expect(parseChannelSource("B", "")).toEqual({
+      kind: "buffer",
+      buffer: "bufferB",
+    });
+    expect(parseChannelSource("/tex.png", "")).toEqual({
+      kind: "url",
+      url: "/tex.png",
+    });
+    expect(parseChannelSource("", "/fallback.png")).toEqual({
+      kind: "url",
+      url: "/fallback.png",
+    });
+    expect(parseChannelSource("none", "")).toEqual({ kind: "none" });
+  });
+});
+
+describe("buildFragmentSource", () => {
+  it("injecte common + mainImage wrapper", () => {
+    const src = buildFragmentSource(
+      "float k = 1.0;",
+      "void mainImage(out vec4 o, in vec2 p){ o = vec4(k); }",
+    );
+    expect(src).toContain("#version 300 es");
+    expect(src).toContain("float k = 1.0;");
+    expect(src).toContain("mainImage(sonicFragColor");
+    expect(src).toContain("uniform sampler2D iChannel0");
+  });
+});
+
+describe("parseHoverParam", () => {
+  it("accepte 0–3 / paramN / vide", async () => {
+    const { parseHoverParam } = await import("./types");
+    expect(parseHoverParam(0)).toBe(0);
+    expect(parseHoverParam("param2")).toBe(2);
+    expect(parseHoverParam("")).toBeNull();
+    expect(parseHoverParam("nope")).toBeNull();
+  });
+});
+
+describe("parseExtractPass", () => {
+  it("résout image / buffers / none", async () => {
+    const { parseExtractPass } = await import("./types");
+    expect(parseExtractPass("image")).toBe("image");
+    expect(parseExtractPass("buffer-a")).toBe("bufferA");
+    expect(parseExtractPass("B")).toBe("bufferB");
+    expect(parseExtractPass("none")).toBeNull();
+    expect(parseExtractPass("")).toBeNull();
+  });
+});
+
+describe("findPeaks", () => {
+  it("trouve un max local et respecte NMS / maxCount", async () => {
+    const { findPeaks } = await import("./types");
+    const w = 8;
+    const h = 8;
+    const data = new Uint8Array(w * h * 4);
+    // Peak fort au centre
+    const set = (x: number, y: number, v: number) => {
+      const i = (y * w + x) * 4;
+      data[i] = v;
+    };
+    set(3, 3, 255);
+    set(3, 4, 200);
+    set(6, 6, 240);
+    const peaks = findPeaks(data, w, h, {
+      threshold: 0.5,
+      maxCount: 2,
+      minDistance: 3,
+    });
+    expect(peaks.length).toBeGreaterThanOrEqual(1);
+    expect(peaks[0].score).toBeCloseTo(1, 2);
+    expect(peaks.length).toBeLessThanOrEqual(2);
+  });
+
+  it("ignore sous le seuil", async () => {
+    const { findPeaks } = await import("./types");
+    const w = 4;
+    const h = 4;
+    const data = new Uint8Array(w * h * 4);
+    data[ (2 * w + 2) * 4 ] = 40; // ~0.16
+    expect(
+      findPeaks(data, w, h, { threshold: 0.5, maxCount: 4, minDistance: 1 }),
+    ).toEqual([]);
+  });
+});
+
+describe("roiToPixels", () => {
+  it("clamp un point + patch", async () => {
+    const { roiToPixels } = await import("./types");
+    const r = roiToPixels({ x: 0.5, y: 0.5, patch: 16 }, 100, 80, 16, 128);
+    expect(r).not.toBeNull();
+    expect(r!.w).toBe(16);
+    expect(r!.h).toBe(16);
+    expect(r!.x).toBeGreaterThanOrEqual(0);
+    expect(r!.y).toBeGreaterThanOrEqual(0);
+    expect(r!.x + r!.w).toBeLessThanOrEqual(100);
+    expect(r!.y + r!.h).toBeLessThanOrEqual(80);
+  });
+
+  it("clamp un rectangle normalisé hors bornes", async () => {
+    const { roiToPixels } = await import("./types");
+    const r = roiToPixels(
+      { x: -0.1, y: -0.1, w: 0.5, h: 0.5 },
+      64,
+      64,
+      16,
+      128,
+    );
+    expect(r).not.toBeNull();
+    expect(r!.x).toBe(0);
+    expect(r!.y).toBe(0);
+    expect(r!.w).toBeGreaterThan(0);
+    expect(r!.h).toBeGreaterThan(0);
+  });
+
+  it("refuse un ROI vide après clamp", async () => {
+    const { roiToPixels } = await import("./types");
+    expect(
+      roiToPixels({ x: 2, y: 2, w: 0.1, h: 0.1 }, 10, 10, 16, 128),
+    ).toBeNull();
+  });
+});
+
+describe("meanRgba", () => {
+  it("moyenne un buffer RGBA", async () => {
+    const { meanRgba } = await import("./types");
+    const data = new Uint8Array([255, 0, 0, 255, 0, 255, 0, 255]);
+    const m = meanRgba(data);
+    expect(m.r).toBeCloseTo(0.5, 2);
+    expect(m.g).toBeCloseTo(0.5, 2);
+    expect(m.b).toBeCloseTo(0, 2);
+  });
+});
+
+describe("SonicShader", () => {
+  afterEach(() => {
+    document.body.innerHTML = "";
+  });
+
+  beforeAll(() => {
+    if (typeof globalThis.ResizeObserver === "undefined") {
+      globalThis.ResizeObserver = class {
+        observe() {}
+        unobserve() {}
+        disconnect() {}
+      } as typeof ResizeObserver;
+    }
+    if (typeof globalThis.IntersectionObserver === "undefined") {
+      globalThis.IntersectionObserver = class {
+        observe() {}
+        unobserve() {}
+        disconnect() {}
+        takeRecords() {
+          return [];
+        }
+        root = null;
+        rootMargin = "";
+        thresholds = [];
+      } as typeof IntersectionObserver;
+    }
+  });
+
+  it("enregistre le custom element", () => {
+    expect(customElements.get("sonic-shader")).toBe(SonicShader);
+  });
+
+  it("applique param0…3 depuis dataProvider", async () => {
+    const { PublisherManager } = await import(
+      "@supersoniks/concorde/utils"
+    );
+    const id = "shader-form-test";
+    const pub = PublisherManager.get(id);
+    pub.set({
+      param0: 0.11,
+      param1: 0.22,
+      param2: 0.33,
+      param3: 0.44,
+    });
+    const el = TestUtils.bootstrap(
+      `<sonic-shader dataProvider="${id}"></sonic-shader>`,
+      true,
+    )[0] as SonicShader;
+    await el.updateComplete;
+    await new Promise((r) => setTimeout(r, 0));
+    expect(el.param0).toBe(0.11);
+    expect(el.param1).toBe(0.22);
+    expect(el.param2).toBe(0.33);
+    expect(el.param3).toBe(0.44);
+
+    // Nested field update (comme sonic-input) doit remonter via onInternalMutation
+    pub.param0.set(0.99);
+    await new Promise((r) => setTimeout(r, 0));
+    expect(el.param0).toBe(0.99);
+  });
+
+  it("applique les canaux depuis dataProvider (url | MediaRef)", async () => {
+    const { PublisherManager } = await import(
+      "@supersoniks/concorde/utils"
+    );
+    const id = "shader-channel-dp";
+    const pub = PublisherManager.get(id);
+    pub.set({
+      bufferACh0: { url: "https://example.test/depth.png" },
+      imageCh0: "buffer-a",
+      imageCh1: "https://example.test/color.jpg",
+      param0: 0.5,
+    });
+    const el = TestUtils.bootstrap(
+      `<sonic-shader dataProvider="${id}"></sonic-shader>`,
+      true,
+    )[0] as SonicShader;
+    await el.updateComplete;
+    await new Promise((r) => setTimeout(r, 0));
+    expect(el.bufferACh0).toBe("https://example.test/depth.png");
+    expect(el.imageCh0).toBe("buffer-a");
+    expect(el.imageCh1).toBe("https://example.test/color.jpg");
+    expect(el.param0).toBe(0.5);
+  });
+
+  it("ne réécrit pas param0…3 sur un dataProvider partagé form+out", async () => {
+    const { PublisherManager } = await import(
+      "@supersoniks/concorde/utils"
+    );
+    const id = "shader-form-no-writeback";
+    const pub = PublisherManager.get(id);
+    pub.set({ param0: 0.42, param1: 0.7, frame: 0 });
+    const el = TestUtils.bootstrap(
+      `<sonic-shader dataProvider="${id}"></sonic-shader>`,
+      true,
+    )[0] as SonicShader;
+    await el.updateComplete;
+    await new Promise((r) => setTimeout(r, 0));
+
+    // sonic-input écoute param0 ; un lecteur frame active vraiment l’out
+    const onParam = () => {};
+    pub.param0.onAssign(onParam);
+    pub.frame.onAssign(() => {});
+
+    // Props GPU locales à 0 ≠ valeurs form — ne doivent pas écraser le DP
+    el.param0 = 0;
+    el.param1 = 0;
+    const internal = el as unknown as {
+      lastOutAt: number;
+      maybePublishOut: (now: number, time: number) => void;
+    };
+    internal.lastOutAt = 0;
+    internal.maybePublishOut(performance.now(), 1);
+
+    expect(pub.param0.get()).toBe(0.42);
+    expect(pub.param1.get()).toBe(0.7);
+    pub.param0.offAssign(onParam);
+  });
+
+  it("émet error si WebGL2 est indisponible", async () => {
+    const onError = vi.fn();
+    const el = TestUtils.bootstrap(
+      `<sonic-shader style="width:64px;height:64px"></sonic-shader>`,
+      true,
+    )[0] as SonicShader;
+    el.addEventListener("error", onError);
+    el.image =
+      "void mainImage(out vec4 fragColor, in vec2 fragCoord){ fragColor=vec4(1.0); }";
+    await el.updateComplete;
+    await new Promise((r) => setTimeout(r, 0));
+
+    // jsdom / happy-dom: pas de WebGL2 → error au firstUpdated
+    if (onError.mock.calls.length > 0) {
+      expect(onError.mock.calls[0][0].detail.message).toMatch(/WebGL2/i);
+    } else {
+      // Environnement avec WebGL2 : l’élément monte sans error
+      expect(el.shadowRoot?.querySelector("canvas")).toBeTruthy();
+    }
+  });
+});