react-native-webrtc-kaleidoscope 2.0.0 → 2.1.1

package/NOTICE.md

@@ -4,29 +4,29 @@ react-native-webrtc-kaleidoscope ships under the MIT license (see [LICENSE](./LI
 
 ## Runtime peer dependency
 
-- **react-native-webrtc** — MIT license. <https://github.com/react-native-webrtc/react-native-webrtc>. The `track._setVideoEffects(...)` JS surface, the Android `ProcessorProvider` registry, and the iOS `RTCVideoFrameProcessor` protocol all originate upstream. This package is a consumer of, not a fork of, that codebase.
+- **react-native-webrtc**: MIT license. <https://github.com/react-native-webrtc/react-native-webrtc>. The `track._setVideoEffects(...)` JS surface, the Android `ProcessorProvider` registry, and the iOS `RTCVideoFrameProcessor` protocol all originate upstream. This package is a consumer of, not a fork of, that codebase.
 
-## Segmentation backends (used by the `blur` and `background-image` effects)
+## Segmentation backends (used by the segmentation mask that drives the `blur` and image layers)
 
 All three platforms run Google's MediaPipe selfie segmentation. The native targets use the MediaPipe Tasks Image Segmenter with a model bundled in this package; the web target uses the legacy MediaPipe Selfie Segmentation Solution loaded at runtime from a CDN.
 
-- **MediaPipe Tasks Vision** (Android) — Apache License 2.0. The `com.google.mediapipe:tasks-vision` AAR, `ImageSegmenter` API. <https://github.com/google-ai-edge/mediapipe>.
-- **MediaPipe Tasks Vision** (iOS) — Apache License 2.0. The `MediaPipeTasksVision` CocoaPod, `ImageSegmenter` API. <https://github.com/google-ai-edge/mediapipe>.
-- **MediaPipe Selfie Segmentation** (web) — Apache License 2.0. The `@mediapipe/selfie_segmentation` Solution, loaded at runtime from `cdn.jsdelivr.net` (not bundled in this package). <https://github.com/google-ai-edge/mediapipe>.
+- **MediaPipe Tasks Vision** (Android): Apache License 2.0. The `com.google.mediapipe:tasks-vision` AAR, `ImageSegmenter` API. <https://github.com/google-ai-edge/mediapipe>.
+- **MediaPipe Tasks Vision** (iOS): Apache License 2.0. The `MediaPipeTasksVision` CocoaPod, `ImageSegmenter` API. <https://github.com/google-ai-edge/mediapipe>.
+- **MediaPipe Selfie Segmentation** (web): Apache License 2.0. The `@mediapipe/selfie_segmentation` Solution, loaded at runtime from `cdn.jsdelivr.net` (not bundled in this package). <https://github.com/google-ai-edge/mediapipe>.
 
 ### Bundled model
 
 The native targets redistribute a TensorFlow Lite model inside the published package; it ships in `android/src/main/assets/selfie_segmenter.tflite` and in the iOS `Kaleidoscope.bundle`.
 
-- **MediaPipe SelfieSegmenter** (`selfie_segmenter.tflite`) — distributed by Google as part of MediaPipe Solutions.
+- **MediaPipe SelfieSegmenter** (`selfie_segmenter.tflite`): distributed by Google as part of MediaPipe Solutions.
   - Source: <https://storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter/float16/latest/selfie_segmenter.tflite>
   - Model card: <https://storage.googleapis.com/mediapipe-assets/Model%20Card%20MediaPipe%20Selfie%20Segmentation.pdf>
   - The MediaPipe project and SDK are published by Google under the Apache License 2.0. The `selfie_segmenter.tflite` weights are redistributed here as published by Google as part of MediaPipe Solutions; Google does not attach a separate license to the hosted model file, so its use is governed by the model card above. The web target does not bundle this file; it fetches its model from the CDN at runtime.
 
 ## Algorithmic and architectural references
 
-- **mrousavy/FaceBlurApp** — segmentation + GPU composite at 60–120 FPS. Different plumbing (camera-preview pipeline, not a WebRTC track), but the algorithmic shape of "mask + blurred copy + composite" is the same.
-- **Volcomix/virtual-background** — TFLite WASM reference for the web blur composite.
+- **mrousavy/FaceBlurApp**: segmentation + GPU composite at 60–120 FPS. Different plumbing (camera-preview pipeline, not a WebRTC track), but the algorithmic shape of "mask + blurred copy + composite" is the same.
+- **Volcomix/virtual-background**: TFLite WASM reference for the web blur composite.
 
 ## Trademarks
 

package/README.md

@@ -12,29 +12,36 @@
 [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/simiancraft/react-native-webrtc-kaleidoscope/badge)](https://securityscorecards.dev/viewer/?uri=github.com/simiancraft/react-native-webrtc-kaleidoscope)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
 
-> Creative, shader-based video effects for React Native video calls: blur or replace a person's background, with more camera effects to come. Works with `react-native-webrtc` and LiveKit, managed-Expo-friendly.
+<p align="center">
+  <code>kaleidoscope</code> &nbsp;•&nbsp; <code>transform</code> &nbsp;•&nbsp; <code>mask</code>
+</p>
+
+> Creative, shader-based video effects for React Native video calls: blur or replace a person's background in a teleconference. Works with `react-native-webrtc` and LiveKit, managed-Expo-friendly.
+
+**Bind a track once, then drive it with three verbs.** `kaleidoscope` swaps the background effect (blur, a still image, or a procedural shader), `transform` reorients the frame, and `mask` tunes the segmentation edge. The effects you can command are a typed preset book you declare in your own project; at `expo prebuild`, only the assets you actually reference ship in your native bundle.
 
 ## Status
 
-**Active development; not yet production-ready.** Published to npm at `1.0.0` (semantic-release cut the first tag at 1.0.0; the number reflects release automation, not a maturity claim). The npm presentation, marketing, and release-quality polish will come in a later pass; right now the README's job is to tell the truth about what works.
+**Active development; not yet production-ready.** Published to npm (the badge shows the current version, which tracks release automation rather than maturity). Presentation and install-doc polish come later; for now the README documents what works.
 
 ### What works today
 
-- **Transform** (flip X / flip Y / rotate 90° CW / CCW) — orientation utilities, identical across platforms.
-- **Blur** (background blur, person stays sharp).
-- **Background replacement** (composite a still image behind the segmented person; eleven bundled presets, plus arbitrary URLs on web; library-side asset pipeline).
-- **Runtime tuning** of the GLSL effects; see the [Use](#use) section.
+- **`kaleidoscope`**: the art axis. Blur, a bundled background image (ten themed backgrounds plus a calibration grid, or your own assets), or a procedural shader, commanded by preset name. The generative shaders that ship are `plasma`, `clouds`, `godrays`, `fireflies`, `nebula`, `simianlights`, `anamorphic-lensflare`, `light-beams-and-motes`, and `corporate-blobs`.
+- **`transform`**: absolute flips and rotation (snapped to 90°), reapplied as full state each call.
+- **`mask`**: the segmentation edge (hardness, threshold) shared by every art effect.
+- **Tree-shaking**: declare a preset book; only the assets you reference ship in your native bundle (web tree-shakes by import).
+- **Drop-in UI (optional)**: a preset-driven `PresetBookMenu` (the menu) under `react-native-webrtc-kaleidoscope/preset-book-menu`, plus a headless live-editor kit (`PresetControlPanel`, mask and transform controls, a theme provider) under `react-native-webrtc-kaleidoscope/preset-control-panel`. Controlled and NativeWind-ready. See [Drop-in UI](#drop-in-ui-optional).
 
 | Platform | Transform | Blur | Background replacement | Notes |
 |---|---|---|---|---|
 | Web (Chrome / Edge) | ✓ | ✓ | ✓ | MediaStreamTrackProcessor + MediaPipe Selfie Segmentation (WASM, CDN) |
 | Android (API 24+) | ✓ | ✓ | ✓ | OpenGL ES 3.0 + MediaPipe Selfie Segmentation (Tasks) |
-| iOS (≥ 15) | ✓ | ✓ | ✓ | Metal + Vision (person segmentation), verified on device. Older A11 devices (iPhone X) run at a lower frame rate |
-| Safari / Firefox | — | — | — | No Insertable Streams; `applyVideoEffects` throws a clear capability error |
+| iOS (≥ 15) | ✓ | ✓ | ✓ | Metal + MediaPipe Selfie Segmentation (Tasks), verified on device. Older A11 devices (iPhone X) run at a lower frame rate |
+| Safari / Firefox | n/a | n/a | n/a | No Insertable Streams; the effects throw a clear capability error |
 
 ### Coming soon
 
-- **Procedural backgrounds** (animated shaders behind the person, not just still images). Same composite path; the only new piece is each effect's background producer.
+- **Animated image backgrounds**: a bundled still that moves (beyond the procedural shaders, which already render behind the person today). Same composite path; the new piece is a per-effect background producer for animated images.
 - A careful pass over the npm presentation, install docs, and demo polish before any "we recommend you use this" framing.
 
 ## Install
@@ -55,12 +62,14 @@ bun add @livekit/react-native @livekit/react-native-webrtc react-native-webrtc-k
 
 Pick one fork. Installing both upstream `react-native-webrtc` and `@livekit/react-native-webrtc` in the same app will cause native class collisions; that's the consumer's problem to resolve.
 
-**Native wiring.** `@livekit/react-native` hands you a `LocalVideoTrack`; apply effects to its underlying `MediaStreamTrack`:
+**Native wiring.** `@livekit/react-native` hands you a `LocalVideoTrack`; bind effects to its underlying `MediaStreamTrack`:
 
 ```ts
-import { applyVideoEffects } from 'react-native-webrtc-kaleidoscope';
+import { bindKaleidoscope } from 'react-native-webrtc-kaleidoscope';
+import { presets } from './kaleidoscope.preset-book';
 
-applyVideoEffects(localCameraTrack.mediaStreamTrack, ['blur']);
+const { kaleidoscope } = bindKaleidoscope(localCameraTrack.mediaStreamTrack, { presets });
+kaleidoscope('blur-soft');
 ```
 
 **Web wiring.** On web, LiveKit owns the `RTCRtpSender`, so you cannot swap the track yourself; go through LiveKit's processor API instead. The opt-in `/livekit` subpath ships a ready-made processor (it needs `livekit-client`, which a LiveKit app already has):
@@ -71,7 +80,7 @@ import { KaleidoscopeProcessor } from 'react-native-webrtc-kaleidoscope/livekit'
 await localVideoTrack.setProcessor(new KaleidoscopeProcessor(['blur']), true);
 ```
 
-The second argument shows the processed stream in your local preview. The processor tears down its Insertable-Streams pipeline on camera flip (`restart`) and unpublish (`destroy`), so repeated flips do not leak generators.
+The `true` second argument to `setProcessor` shows the processed stream in your local preview. The processor tears down its Insertable-Streams pipeline on camera flip (`restart`) and unpublish (`destroy`), so repeated flips do not leak generators.
 
 ## Configure
 
@@ -95,68 +104,181 @@ bunx expo prebuild
 
 ## Use
 
+First declare a **preset book** in your project: a flat catalog of **composites**, the only things you can command. A composite is `{ name, taxonomy, thumbnail?, layers }`, where `taxonomy` is the picker's grouping path, root first (`[group]` or `[group, category]`, e.g. `['Backgrounds', 'Office']`); everything is a layer stack, painted back to front. A layer is `{ id, shader, target?, blend? }` plus the shader's fields: `image` takes a `source`; `direct` samples the ingest-normalized (upright, non-mirrored) camera frame for its target (`target: 'subject'` is the masked person, `target: 'background'` the full frame); `blur` and the generative shaders (`plasma`, `clouds`, …) take `uniforms`. `target` defaults to `'background'` (fullscreen); `'subject'` stencils to the segmented person. Each layer's `id` is unique within its composite. Declare the book `as const satisfies PresetBook` for per-layer typing.
+
+```ts
+// kaleidoscope.preset-book.ts
+import type { PresetBook } from 'react-native-webrtc-kaleidoscope';
+import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark';
+// Packaged composites ship ready to use; import and spread them in.
+import { wizardTower } from 'react-native-webrtc-kaleidoscope/composites/wizard-tower';
+
+export const presets = {
+  // Replace your background with an image, you composited over it.
+  'office-dark': {
+    name: 'Dark Office',
+    taxonomy: ['Backgrounds', 'Office'],
+    thumbnail: officeDark,
+    layers: [
+      { id: 'office-dark', shader: 'image', source: officeDark },
+      { id: 'you', shader: 'direct', target: 'subject' },
+    ],
+  },
+  // Blur the background, stay sharp.
+  'blur-heavy': {
+    name: 'Heavy',
+    taxonomy: ['Effects', 'Blur'],
+    layers: [
+      { id: 'bg', shader: 'blur', target: 'background', uniforms: { sigma: 7 } },
+      { id: 'you', shader: 'direct', target: 'subject' },
+    ],
+  },
+  // A packaged multi-layer composite (clouds + a cut-out image + you).
+  'wizard-tower': wizardTower,
+} as const satisfies PresetBook;
+```
+
+Then bind a track once and drive it with the three verbs:
+
 ```ts
 import { mediaDevices } from 'react-native-webrtc';
-import {
-  applyVideoEffects,
-  setBlurSigma,
-  setMaskHardness,
-  setMaskThreshold,
-} from 'react-native-webrtc-kaleidoscope';
-import { darkOffice } from 'react-native-webrtc-kaleidoscope/backgrounds/dark-office';
+import { bindKaleidoscope } from 'react-native-webrtc-kaleidoscope';
+import { presets } from './kaleidoscope.preset-book';
 
 const stream = await mediaDevices.getUserMedia({ video: true });
 const [track] = stream.getVideoTracks();
 
-applyVideoEffects(track, ['flip-x']); // also flip-y, rotate-cw, rotate-ccw
-applyVideoEffects(track, ['blur']);
-applyVideoEffects(track, [{ name: 'background-image', source: darkOffice }]);
-applyVideoEffects(track, []); // clear all effects
+const { kaleidoscope, transform, mask } = bindKaleidoscope(track, {
+  presets,
+  // Web rebuilds the pipeline per command and yields a NEW track; read it here
+  // (attach to <video> or replaceTrack). Native mutates the bound track in place.
+  onTrack: (out) => {
+    /* setPreviewTrack(out) */
+  },
+});
+
+// kaleidoscope, the art axis. Pass a preset id (autocompletes from your book):
+kaleidoscope('wizard-tower');
+// Override a layer's uniforms live, addressed by layer id, while the patched
+// preset is active (merged, no pipeline rebuild). `shader` types the uniforms:
+kaleidoscope('blur-heavy', [{ id: 'bg', shader: 'blur', uniforms: { sigma: 5 } }]);
+kaleidoscope(null);                          // clear the art
+
+// transform, absolute geometry. Every call is the full state from identity;
+// re-pass what you want to keep. rotate snaps to the nearest 90°.
+transform({ flip: { x: true }, rotate: 90 });
+transform();                                 // reset to identity
+
+// mask, the segmentation edge shared by every art effect. Both required, 0..1.
+mask({ hardness: 0.5, threshold: 0.5 });
+```
+
+That is the whole runtime surface: `kaleidoscope`, `transform`, `mask`.
+
+Numeric shader uniforms are normalized 0..1 by convention where practical; JSDoc documents each option's expected range as an IntelliSense hint (ranges are not enforced at runtime; validate in your own layer if you forward them to end users).
+
+**Tuning note:** all three platforms run MediaPipe selfie segmentation (Tasks Image Segmenter on native, the Selfie Segmentation Solution on web), so the mask edge that suits one may differ slightly from another. `mask({ hardness, threshold })` defaults to `0.5 / 0.5`; nudge it to match your camera and lighting.
+
+## Drop-in UI (optional)
+
+Build your own controls against the three verbs, or import a ready-made, headless picker from `react-native-webrtc-kaleidoscope/preset-book-menu` that reads your preset book directly:
 
-// Runtime tuning (effects pick up the new values on the next frame):
-setBlurSigma(5);         // Gaussian σ; clamped to [0.5, 7], default 5.
-setMaskHardness(0.5);    // smoothstep transition width; clamped to [0, 1]. 0 = soft halo, 1 = near-step. Default 0.5.
-setMaskThreshold(0.7);   // smoothstep center; clamped to [0.05, 0.95]. Higher rejects low-confidence pixels. Default 0.7.
+```tsx
+import { useEffect, useState } from 'react';
+import { PresetBookMenu } from 'react-native-webrtc-kaleidoscope/preset-book-menu';
+import { presets } from './kaleidoscope.preset-book';
+
+// `kaleidoscope` is the verb returned by bindKaleidoscope(track, { presets }).
+function BackgroundControls({ kaleidoscope }) {
+  const [art, setArt] = useState<keyof typeof presets | null>(null);
+  useEffect(() => {
+    if (art) kaleidoscope(art);
+    else kaleidoscope(null);
+  }, [art, kaleidoscope]);
+
+  return <PresetBookMenu presets={presets} value={art} onSelect={setArt} />;
+}
+```
+
+`PresetBookMenu` is a two-level browser driven by each composite's `taxonomy`: a tab row across the top, one tab per **group** (`taxonomy[0]`, e.g. Effects, Worlds, Backgrounds, Shaders), and a left-hand menu of **categories** (`taxonomy[1]`) under the active group; the tile grid is filtered by both. A flat (depth-1) group shows no category menu. Every preset renders as a uniform tile: a wallpaper when the composite has a `thumbnail`, a recessed button of the same footprint when it does not, so a thumbnail-less preset never breaks the grid. The same pieces are exported as standalone primitives (`PresetGrid`, `PresetTile`, plus the `usePresetBookMenu` hook and `PresetBookMenuLayout`), so you can lay out your own. Selection is controlled (`value` + `onSelect(id)`, narrowed to your book's keys); the components are presentational: they emit the selected id, you apply it via `kaleidoscope`.
+
+**Styling, three tiers.** Sensible defaults out of the box; override with an RN `style` prop, a `className` prop, or a `renderTile` render-prop slot for full control.
+
+**NativeWind-ready.** The components accept `className`. To turn it on, import the opt-in registration once in your NativeWind interop setup (`nativewind` is an optional peer dependency; the core `./preset-book-menu` import never pulls it in):
+
+```ts
+import { registerKaleidoscopeNativeWind } from 'react-native-webrtc-kaleidoscope/nativewind';
+registerKaleidoscopeNativeWind();
 ```
 
-Effects chain in array order.
+### Live controls (the editor)
 
-**Tuning note:** all three platforms run MediaPipe selfie segmentation (Tasks Image Segmenter on native, the Selfie Segmentation Solution on web), but optimal values still differ slightly because the input downscale and the API variant differ per platform, producing different confidence distributions. Working defaults on a typical well-lit scene:
+For a tuning or admin panel, `react-native-webrtc-kaleidoscope/preset-control-panel` ships a headless editor that reads the active preset and renders a control per tunable uniform, plus the mask and transform panels:
 
-| Platform | Blur sigma | Mask hardness | Mask threshold |
-|---|---|---|---|
-| Web (MediaPipe) | 25 | 0.2 | 0.85 |
-| Android (MediaPipe) | 30 | 0.2 | 0.6 |
+```tsx
+import {
+  KaleidoscopeThemeProvider,
+  PresetControlPanel,
+  MaskControlPanel,
+  TransformControlPanel,
+} from 'react-native-webrtc-kaleidoscope/preset-control-panel';
+
+// `controls` is the object from bindKaleidoscope(track, { presets }).
+<KaleidoscopeThemeProvider>
+  <PresetControlPanel presets={presets} value={art} onPatch={(p) => controls.kaleidoscope(art, [p])} />
+  <MaskControlPanel hardness={h} threshold={t} onChange={setMask} />
+  <TransformControlPanel flip={flip} rotate={rotate} onChange={setTransform} />
+</KaleidoscopeThemeProvider>
+```
 
-The library ships defaults (5, 0.5, 0.7) and consumers tune at runtime via the API above; whether to ship the dialed-in per-platform values as defaults is an open question.
+Each preset supplies its editor as a `controls` component on the book entry. The packaged composites export theirs at `react-native-webrtc-kaleidoscope/composites/<name>/controls`; for your own presets, compose `CompositeLayerControlPanel` over a shader's control descriptor (or `makeControls` for a custom widget). See [catalog/shaders/README.md](./catalog/shaders/README.md).
+
+Like the picker, the editor is controlled and presentational: it emits patches and you apply them. `KaleidoscopeThemeProvider` themes every control at once (a slot bank; `style` works everywhere, `className` via the same opt-in NativeWind interop). The sliders need `@react-native-community/slider` (an optional peer; a native module, so installing it needs a dev-client rebuild).
+
+Live per-layer tuning runs on web today; on native the editor renders but the live per-layer uniform channel is in progress. Mask and transform are live on every platform.
+
+## Worlds
+
+Packaged composites: a multi-layer stack (a generative shader or a cut-out image, the masked person on top), imported and spread into your book (e.g. `import { wizardTower } from 'react-native-webrtc-kaleidoscope/composites/wizard-tower'`). They carry their own `taxonomy: ['Worlds', <group>]`, so the menu groups them under the Worlds tab.
+
+| Wizard Tower | Observation Deck | Fairy Cave |
+|---|---|---|
+| <img src="composites/wizard-tower/wizard-tower.thumb.webp" width="220" alt="wizard-tower" /> | <img src="composites/observation-deck/observation-deck.thumb.webp" width="220" alt="observation-deck" /> | <img src="composites/fairy-cave/fairy-cave.thumb.webp" width="220" alt="fairy-cave" /> |
+| Underwater | Nebula | Simianlights |
+| <img src="composites/underwater/underwater.thumb.webp" width="220" alt="underwater" /> | <img src="composites/nebula/nebula.thumb.webp" width="220" alt="nebula" /> | <img src="composites/simianlights/simianlights.thumb.webp" width="220" alt="simianlights" /> |
+| Corporate Blobs | | |
+| <img src="composites/corporate-blobs/corporate-blobs.thumb.webp" width="220" alt="corporate-blobs" /> | | |
+
+The `clouds` (Sky) composite also ships; its preview tile is pending.
 
 ## Background presets
 
-Eleven backgrounds ship for the `background-image` effect, imported per preset (e.g. `import { darkOffice } from 'react-native-webrtc-kaleidoscope/backgrounds/dark-office'`). On web a preset can also be any image URL or data URI; native resolves bundled preset names only.
+The bundled backgrounds ship as `image` layers, filed by category and imported per image (e.g. `import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark'`). On web an image can also be any image URL or data URI; native resolves bundled image ids only.
 
-| Theme | Light | Dark |
+| Category | Light | Dark |
 |---|---|---|
-| Office | <img src="src/backgrounds/light-office.webp" width="220" alt="light-office" /> | <img src="src/backgrounds/dark-office.webp" width="220" alt="dark-office" /> |
-| Home | <img src="src/backgrounds/home-light.webp" width="220" alt="home-light" /> | <img src="src/backgrounds/home-dark.webp" width="220" alt="home-dark" /> |
-| Nature | <img src="src/backgrounds/nature-light.webp" width="220" alt="nature-light" /> | <img src="src/backgrounds/nature-dark.webp" width="220" alt="nature-dark" /> |
-| Stylized | <img src="src/backgrounds/stylized-light.webp" width="220" alt="stylized-light" /> | <img src="src/backgrounds/stylized-dark.webp" width="220" alt="stylized-dark" /> |
-| Simiancraft | <img src="src/backgrounds/simiancraft-light.webp" width="220" alt="simiancraft-light" /> | <img src="src/backgrounds/simiancraft-dark.webp" width="220" alt="simiancraft-dark" /> |
+| Office | <img src="images/office/office-light.thumb.webp" width="220" alt="office-light" /> | <img src="images/office/office-dark.thumb.webp" width="220" alt="office-dark" /> |
+| Home | <img src="images/home/home-light.thumb.webp" width="220" alt="home-light" /> | <img src="images/home/home-dark.thumb.webp" width="220" alt="home-dark" /> |
+| Nature | <img src="images/nature/landscape-light.thumb.webp" width="220" alt="landscape-light" /> | <img src="images/nature/landscape-dark.thumb.webp" width="220" alt="landscape-dark" /> |
+| Sci-Fi | <img src="images/sci-fi/sci-fi-light.thumb.webp" width="220" alt="sci-fi-light" /> | |
+| Underwater | | <img src="images/underwater/oceanscape-dark.thumb.webp" width="220" alt="oceanscape-dark" /> |
+| Simiancraft | <img src="images/simiancraft/simiancraft-light.thumb.webp" width="220" alt="simiancraft-light" /> | <img src="images/simiancraft/simiancraft-dark.thumb.webp" width="220" alt="simiancraft-dark" /> |
 
-Plus **`debug-resolutions`**, a viewport/resolution calibration grid for verifying background cover-fit:
+The `simiancraft` category also ships two transparent brand images, `simiancraft-light-transparency` and `simiancraft-dark-transparency` (alpha preserved). Plus **`debug-resolutions`**, a viewport/resolution calibration grid for verifying background cover-fit:
 
-<img src="src/backgrounds/debug-resolutions.webp" width="220" alt="debug-resolutions" />
+<img src="images/debug/debug-resolutions.thumb.webp" width="220" alt="debug-resolutions" />
 
-See [`src/backgrounds/README.md`](./src/backgrounds/README.md) for sizing and how to add a preset.
+See [`catalog/images/README.md`](./catalog/images/README.md) for the folder layout, the two image formats, and how to add one.
 
 ## Web and native differences
 
 The API surface is the same across platforms, but the runtimes differ in ways worth knowing before you wire effects in:
 
-- **Effect parameters.** Web reads tuning from the global setters (`setBlurSigma`, `setMaskHardness`, `setMaskThreshold`) on the next frame. Native currently ignores per-call `EffectSpec` parameters such as `{ name: 'blur', sigma: 12 }`; tuning is global through the same setters. Per-call uniforms through the native registry are a follow-up.
-- **Background source.** `background-image.source` is a bundled preset name on native (the upstream `_setVideoEffects` registry is keyed by flat strings, not URIs), but on web it accepts either a preset name or an arbitrary image URL or data URI.
-- **Background presets ship as tree-shakeable files.** The bundled backgrounds (see [Background presets](#background-presets)) are importable per preset: `import { darkOffice } from 'react-native-webrtc-kaleidoscope/backgrounds/dark-office'`. Each preset is its own file behind its own subpath export, and the package sets `sideEffects: false`, so an unused preset is dropped by web bundlers — and, since Metro doesn't tree-shake, simply never imported on native. Web resolves the bundled WebP to a URL; native loads its own bundled copy by name. Web also still accepts an arbitrary image URL or data URI. See [`src/backgrounds/README.md`](./src/backgrounds/README.md).
-- **Segmentation model on web.** The web blur and background-image effects load MediaPipe Selfie Segmentation from the jsdelivr CDN (`cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation`) on first use. A strict Content-Security-Policy must allow that origin for `script-src`, `connect-src`, and the WASM fetch, and the effects do not work offline. Mirror needs no model.
-- **Browser support on web.** Effects use Insertable Streams (`MediaStreamTrackProcessor` and `MediaStreamTrackGenerator`), which ship in Chromium-based browsers (Chrome, Edge); Safari and Firefox lack the API, so `applyVideoEffects` throws a clear capability error and the demo falls back to the unprocessed track.
+- **Output track.** On web each `kaleidoscope`/`transform` command rebuilds the Insertable-Streams pipeline and yields a NEW `MediaStreamTrack`, surfaced via `onTrack`; on native the bound track is mutated in place. `mask` updates the segmentation edge the running composite reads each frame, with no rebuild on either platform.
+- **Image source.** An `image` layer's `source` is a bundled preset name on native (the upstream `_setVideoEffects` registry is keyed by flat strings, not URIs), but on web it accepts either a preset name or an arbitrary image URL or data URI.
+- **Background presets ship as tree-shakeable files.** The bundled backgrounds (see [Background presets](#background-presets)) are importable per image: `import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark'`. Each image is its own file behind its own subpath export, and the package sets `sideEffects: false`, so an unused preset is dropped by web bundlers; since Metro doesn't tree-shake, it is simply never imported on native. Web resolves the bundled WebP to a URL; native loads its own bundled copy by name. Web also still accepts an arbitrary image URL or data URI. See [`catalog/images/README.md`](./catalog/images/README.md).
+- **Segmentation model on web.** The web compositor loads MediaPipe Selfie Segmentation from the jsdelivr CDN (`cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation`) on first use. A strict Content-Security-Policy must allow that origin for `script-src`, `connect-src`, and the WASM fetch, and the effects do not work offline. The `transform` ops need no model.
+- **Browser support on web.** Effects use Insertable Streams (`MediaStreamTrackProcessor` and `MediaStreamTrackGenerator`), which ship in Chromium-based browsers (Chrome, Edge); Safari and Firefox lack the API, so the effects throw a clear capability error and the demo falls back to the unprocessed track.
 
 ## What this isn't
 
@@ -167,14 +289,20 @@ The API surface is the same across platforms, but the runtimes differ in ways wo
 
 ## Architecture
 
-The codebase lives across four surfaces:
+The canonical assets live in three root, folder-per-item directories, out of the TypeScript build path; the build and the prebuild copy read from them:
+
+- `catalog/shaders/<name>/`: each shader's `.frag` plus its typed `.ts` (uniforms + control descriptor). All shaders share one vertex stage, `catalog/shaders/_shared/passthrough.vert`; there is no per-shader `.vert`. `catalog/shaders/_shared/` also holds the per-layer composite frags (`composite-camera.frag`, `composite-image.frag`, `composite-subject.frag`, `composite-masked.frag`, `composite-blit.frag`) and `transform.frag`. `bun run build:shaders` codegens the web and Android sources and transpiles the iOS Metal from these. See [catalog/shaders/README.md](./catalog/shaders/README.md) to add or extend a shader.
+- `catalog/images/<category>/`: images filed by category, several per folder. Each image is a quad: `<leaf>.webp`, its `<leaf>.thumb.webp`, and the `<leaf>.ts` / `<leaf>.web.ts` loader pair, behind a `./images/<category>/<leaf>` subpath export.
+- `catalog/composites/<name>/`: each packaged composite (a `KaleidoscopePreset`), behind a `./composites/<name>` subpath export.
+
+The code lives across the platform surfaces:
 
-- `src/` — JS facade and shared types. `applyVideoEffects(track, effects)` plus runtime tuning setters.
-- `src/web/` — WebGL2 pipeline. MediaPipe segmentation + GLSL composite. One shader file per stage in `src/web/shaders.ts`.
-- `android/` — OpenGL ES 3.0 pipeline. MediaPipe Tasks segmentation (async, worker-thread, last-known-mask cache) + GLSL composite. Shaders inline in `gpu/Shaders.kt` as `const val` strings.
-- `ios/` — Metal pipeline (Swift) with MediaPipe Tasks segmentation (`selfie_segmenter.tflite`, the same model the Android target bundles). The canonical GLSL in `shaders/` transpiles to Metal Shading Language via `scripts/build-shaders.ts`. Implemented and verified on device.
+- `src/`: JS facade and shared types. `bindKaleidoscope` returns the `kaleidoscope` / `transform` / `mask` verbs; the preset-book types live in `src/kaleidoscope/`.
+- `web-driver/`: WebGL2 pipeline. MediaPipe segmentation + the layered compositor (`web-driver/effects/composite.ts`).
+- `android/`: OpenGL ES 3.0 pipeline. MediaPipe Tasks segmentation + the layered compositor (`effects/CompositeFactory.kt`); codegen lands in `gpu/ShadersGenerated.kt`, the hand-written layer GLSL in `effects/LayerShaders.kt`.
+- `ios/`: Metal pipeline (Swift) with MediaPipe Tasks segmentation (`selfie_segmenter.tflite`, the same model Android bundles); the canonical GLSL transpiles to Metal via `scripts/build-shaders.ts`.
 
-The composite shader (`shaders/composite.frag`) is the same GLSL source for every effect category (blur, background-image, future procedural backgrounds). Per-effect difference is upstream of the composite: how the `uBackground` texture gets produced.
+Every effect is a LAYER in one compositor: an `image` image, a `direct` passthrough (the masked person, or the raw camera), a camera-sampling `blur`, or a generative shader, composited back to front with per-layer blend. There is one registered native effect, `composite`; its layer stack is delivered out of band and reconciled each command.
 
 See [`PATTERNS.md`](./PATTERNS.md) for the file-layout conventions, texture-orientation contract, and recipe for adding new effects, shaders, presets, or tunable parameters.
 
@@ -183,6 +311,7 @@ See [`PATTERNS.md`](./PATTERNS.md) for the file-layout conventions, texture-orie
 - [CONTRIBUTING.md](./CONTRIBUTING.md): setup, scripts, commit conventions.
 - [AGENTS.md](./AGENTS.md): agent and contributor orientation.
 - [PATTERNS.md](./PATTERNS.md): codebase conventions and how-to-extend.
+- [catalog/shaders/README.md](./catalog/shaders/README.md): adding and extending shaders.
 - [SECURITY.md](./SECURITY.md): security policy and reporting.
 - [NOTICE.md](./NOTICE.md): third-party attributions.
 - Sibling projects: [chromonym](https://github.com/simiancraft/chromonym) and [unitforge](https://github.com/simiancraft/unitforge); same OSS-hygiene template.

package/android/build.gradle

@@ -31,6 +31,14 @@ android {
   kotlinOptions {
     jvmTarget = '17'
   }
+
+  // JVM unit tests (testDebugUnitTest) for pure-logic units like the composite
+  // layer-stack parser. returnDefaultValues makes android.util.Log a no-op stub
+  // instead of throwing; the real org.json on the test classpath (below) shadows
+  // the stubbed android.jar one so JSON parsing actually runs.
+  testOptions {
+    unitTests.returnDefaultValues = true
+  }
 }
 
 dependencies {
@@ -58,6 +66,11 @@ dependencies {
   // Person/background mask: MediaPipe Tasks ImageSegmenter (Mask.kt), the same
   // model family the web and iOS sides run.
   implementation 'com.google.mediapipe:tasks-vision:0.10.14'
+
+  // Unit tests (JVM, testDebugUnitTest). The real org.json shadows the stubbed
+  // android.jar copy so CompositeLayers' JSONArray/JSONObject parsing runs for real.
+  testImplementation 'junit:junit:4.13.2'
+  testImplementation 'org.json:json:20231013'
 }
 
 // Helper: gracefully read root project ext properties with a default.

package/android/src/main/java/com/simiancraft/kaleidoscope/CompositeLayers.kt

@@ -0,0 +1,127 @@
+// Composite-layer spec side-channel for the native composite compositor.
+//
+// The Expo Module's setCompositeLayers(json) JS function writes here; the
+// CompositeFactory reads the current layer stack each frame and composites it.
+// This delivers the spec without re-registering: it carries the whole ordered
+// layer stack (the composite is one registered effect name, "composite", whose
+// contents JS swaps as the active composite changes).
+//
+// The wire shape is a JSON array of layer objects (see parse()). JS sends it as a
+// String across the Expo bridge; we parse it once at set() time into immutable
+// Kotlin models the GL thread can read without locking. @Volatile on the parsed
+// reference is the memory barrier; the JS-thread write publishes, the GL-thread
+// read observes the latest stable snapshot.
+//
+// A malformed layer is skipped with a log rather than crashing the render thread;
+// an unparseable whole payload leaves the previous composite in place.
+
+package com.simiancraft.kaleidoscope
+
+import android.util.Log
+import org.json.JSONArray
+import org.json.JSONObject
+
+/** One parsed composite layer. The `shader` discriminant decides which fields matter. */
+internal data class CompositeLayer(
+    val id: String, // unique within a composite; the live-tuning / patch address
+    val shader: String,
+    val target: String, // "background" | "subject"
+    val blend: String?, // "normal" | "additive" | null (base = opaque)
+    val source: String?, // image id for an `image` layer; null otherwise
+    val uniforms: Map<String, FloatArray>, // generative-layer uniforms, by name
+)
+
+internal object CompositeLayers {
+    private const val TAG = "Kaleidoscope.CompositeLayers"
+
+    @Volatile
+    private var layers: List<CompositeLayer> = emptyList()
+
+    /** The current parsed layer stack; a stable snapshot safe to read on the GL thread. */
+    fun get(): List<CompositeLayer> = layers
+
+    /**
+     * Parse and store the composite layer stack from the JS-supplied JSON string.
+     * Called on the Expo module thread (JS-driven), not the GL thread. Leaves the
+     * previous composite in place on a whole-payload parse failure.
+     */
+    fun set(json: String) {
+        val parsed =
+            try {
+                parse(json)
+            } catch (t: Throwable) {
+                Log.e(TAG, "failed to parse composite layers; keeping previous composite", t)
+                return
+            }
+        layers = parsed
+        Log.i(TAG, "composite layers set: ${parsed.size} layer(s) [${parsed.joinToString(",") { it.shader }}]")
+    }
+
+    /** Clear the active composite (used when a non-composite effect takes over). */
+    fun clear() {
+        layers = emptyList()
+    }
+
+    private fun parse(json: String): List<CompositeLayer> {
+        val arr = JSONArray(json)
+        val out = ArrayList<CompositeLayer>(arr.length())
+        for (i in 0 until arr.length()) {
+            val obj = arr.optJSONObject(i) ?: continue
+            val shader = obj.optString("shader", "")
+            if (shader.isEmpty()) {
+                Log.w(TAG, "layer $i has no shader; skipping")
+                continue
+            }
+            // `id` is always present on the wire now (serializeCompositeLayers emits it);
+            // fall back to the array index so a malformed payload missing it still
+            // yields a stable, unique-per-stack address rather than a collision.
+            val id = if (obj.has("id") && !obj.isNull("id")) obj.getString("id") else i.toString()
+            val target = obj.optString("target", "background")
+            val blend = if (obj.has("blend") && !obj.isNull("blend")) obj.getString("blend") else null
+            val source = if (obj.has("source") && !obj.isNull("source")) obj.getString("source") else null
+            val uniforms = parseUniforms(obj.optJSONObject("uniforms"))
+            out.add(CompositeLayer(id, shader, target, blend, source, uniforms))
+        }
+        return out
+    }
+
+    // name -> FloatArray. A JSON number -> [f]; a JSON array of numbers -> FloatArray.
+    // A non-numeric or malformed value is skipped with a log (the shader keeps its
+    // GLSL default for that name).
+    private fun parseUniforms(obj: JSONObject?): Map<String, FloatArray> {
+        if (obj == null) return emptyMap()
+        val out = HashMap<String, FloatArray>(obj.length())
+        val keys = obj.keys()
+        while (keys.hasNext()) {
+            val key = keys.next()
+            val value = obj.opt(key)
+            val floats = normalize(value)
+            if (floats == null) {
+                Log.w(TAG, "skipping uniform '$key': unsupported type ${value?.javaClass?.simpleName}")
+                continue
+            }
+            out[key] = floats
+        }
+        return out
+    }
+
+    private fun normalize(value: Any?): FloatArray? =
+        when (value) {
+            is Number -> floatArrayOf(value.toFloat())
+            is JSONArray -> {
+                val out = FloatArray(value.length())
+                var ok = true
+                for (i in 0 until value.length()) {
+                    val n = value.opt(i)
+                    if (n is Number) {
+                        out[i] = n.toFloat()
+                    } else {
+                        ok = false
+                        break
+                    }
+                }
+                if (ok) out else null
+            }
+            else -> null
+        }
+}