react-native-webrtc-kaleidoscope 2.7.2 → 2.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +251 -194
- package/package.json +3 -1
package/README.md
CHANGED
|
@@ -1,20 +1,36 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<
|
|
2
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/">
|
|
3
|
+
<img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/evidence/issue-verification/evidence/2026-06-14-readme-demo/showcase.gif" alt="One person held in frame by the live segmentation mask while the background cycles through a parametric blur, painted worlds, and animated generative shaders" width="720" />
|
|
4
|
+
</a>
|
|
3
5
|
</p>
|
|
4
6
|
|
|
5
7
|
<p align="center">
|
|
6
|
-
<a
|
|
7
|
-
|
|
8
|
-
|
|
8
|
+
<sub><b>This is the real camera, not a still pasted over a video.</b> The live segmentation mask holds one person while shipped presets swap in behind. Every icon below is a live preset, in the order the loop plays them; click one to run it on your own camera.</sub>
|
|
9
|
+
</p>
|
|
10
|
+
|
|
11
|
+
<p align="center">
|
|
12
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=blur-medium" title="Blur">🌫️</a>
|
|
13
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=kaleidoscope-mandala" title="Kaleidoscope mandala">🔮</a>
|
|
14
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=wizard-tower" title="Wizard's tower">🧙</a>
|
|
15
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-classic" title="Outrun grid">🌆</a>
|
|
16
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=observation-deck" title="Observation deck">🛸</a>
|
|
17
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-cobalt" title="Cobalt data-mesh">🔷</a>
|
|
18
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=fairy-grotto" title="Fairy grotto">🧚</a>
|
|
19
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simianlights-hearth" title="Simianlights hearth">🪔</a>
|
|
20
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simiancraft-dark" title="Simiancraft">🐒</a>
|
|
21
|
+
<sub><a href="#presets">+ dozens more</a></sub>
|
|
9
22
|
</p>
|
|
10
23
|
|
|
11
24
|
<p align="center">
|
|
12
|
-
<
|
|
25
|
+
<a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/">
|
|
26
|
+
<img src="https://img.shields.io/badge/▶%20Live%20demo-blur%20yourself%2C%20swap%20the%20room-8b5cf6?style=for-the-badge" alt="Live demo" />
|
|
27
|
+
</a>
|
|
13
28
|
</p>
|
|
14
29
|
|
|
15
|
-
|
|
30
|
+
<h1>
|
|
31
|
+
<img src="./docs/kaleidoscope-logo-thumb.webp" alt="kaleidoscope logo" width="36" /> react-native-webrtc-kaleidoscope
|
|
32
|
+
</h1>
|
|
16
33
|
|
|
17
|
-
[](#status)
|
|
18
34
|
[](https://www.npmjs.com/package/react-native-webrtc-kaleidoscope)
|
|
19
35
|
[](https://www.npmjs.com/package/react-native-webrtc-kaleidoscope)
|
|
20
36
|
[](https://github.com/simiancraft/react-native-webrtc-kaleidoscope/actions/workflows/ci.yml)
|
|
@@ -22,140 +38,123 @@
|
|
|
22
38
|
[](https://securityscorecards.dev/viewer/?uri=github.com/simiancraft/react-native-webrtc-kaleidoscope)
|
|
23
39
|
[](./LICENSE)
|
|
24
40
|
|
|
25
|
-
|
|
26
|
-
<code>kaleidoscope</code> • <code>transform</code> • <code>mask</code>
|
|
27
|
-
</p>
|
|
28
|
-
|
|
29
|
-
> Creative, shader-based video effects for React Native video calls: blur or replace a person's background in a teleconference. Works with `react-native-webrtc` and LiveKit, managed-Expo-friendly.
|
|
30
|
-
|
|
31
|
-
**Bind a track once, then drive it with three verbs.** `kaleidoscope` swaps the background effect (blur, a still image, or a procedural shader), `transform` reorients the frame, and `mask` tunes the segmentation edge. The effects you can command are a typed preset book you declare in your own project; at `expo prebuild`, only the assets you actually reference ship in your native bundle.
|
|
41
|
+
> **Blur yourself, swap the room; live, on the device.** Real-time background blur and replacement for React Native and web video calls: bundled images, animated generative shaders, or painted worlds, each stenciled to the person by an on-device segmentation mask. Works with `react-native-webrtc` and LiveKit on Android, iOS, and Chromium browsers; managed-Expo-friendly.
|
|
32
42
|
|
|
33
|
-
|
|
43
|
+
Every other turnkey option we could find is a feature welded to one vendor's calling SDK (Stream, Agora, 100ms, and the rest). This one attaches to `react-native-webrtc` instead, so it rides whatever stack you already run, LiveKit included. And where those vendors ship blur and a static image, this paints animated, generative-shader backgrounds and whole worlds.
|
|
34
44
|
|
|
35
|
-
|
|
45
|
+
What you get:
|
|
36
46
|
|
|
37
|
-
|
|
47
|
+
- **[Bind a track once, then drive it with three verbs.](#the-three-verbs)** `kaleidoscope` swaps the background, `transform` reorients the frame, and `mask` tunes the segmentation edge. That is the entire runtime surface.
|
|
48
|
+
- **[Hand it to an agent and it wires itself.](#with-an-agent)** Point a coding agent at [`llms.txt`](./llms.txt) and it installs the package, writes the config plugin, provisions a preset book, and gets an effect on screen.
|
|
49
|
+
- **[Drop-in components, not a pipeline to assemble.](#quick-start)** A picker that reads your preset book, a headless live-editor, and a persistence provider; one import each, wire a callback, done.
|
|
50
|
+
- **[Dozens of presets out of the box.](#presets)** Blur, painted worlds, bundled rooms, and animated shaders; every one is a starting point you can retune.
|
|
51
|
+
- **[See what every effect costs.](#performance)** Each shader carries a measured GPU-cost annotation and the resolution tier is one knob, so you size the spend before you ship it.
|
|
52
|
+
- **[Bench, meter, and thumbnail tools included.](#authoring-tooling)** A SPIR-V cost bench, a live GPU-time meter, a thumbnail renderer, and this README's preset gallery all regenerate from one command set.
|
|
53
|
+
- **[Ship only the presets you reference.](#only-ship-what-you-use)** Per-asset subpath exports plus `sideEffects: false`; web tree-shakes by import and `expo prebuild` copies only the assets your book names into the native bundle.
|
|
54
|
+
- **[A new shader is one folder.](#make-your-own-presets)** Every effect is a layer in one [compositor](#architecture); drop a `.frag` into `catalog/shaders/` and it codegens to web, Android, and iOS, masked and oriented for free.
|
|
38
55
|
|
|
39
|
-
|
|
40
|
-
- **`transform`**: absolute flips and rotation (snapped to 90°), reapplied as full state each call.
|
|
41
|
-
- **`mask`**: the segmentation edge (hardness, threshold) shared by every art effect.
|
|
42
|
-
- **Tree-shaking**: declare a preset book; only the assets you reference ship in your native bundle (web tree-shakes by import).
|
|
43
|
-
- **Drop-in UI (optional)**: a preset-driven `PresetBookMenu` (the menu) under `react-native-webrtc-kaleidoscope/preset-book-menu`, plus a headless live-editor kit (`PresetControlPanel`, mask and transform controls, a theme provider) under `react-native-webrtc-kaleidoscope/preset-control-panel`. Controlled and NativeWind-ready. See [Drop-in UI](#drop-in-ui-optional).
|
|
56
|
+
## Presets
|
|
44
57
|
|
|
45
|
-
|
|
46
|
-
|---|---|---|---|---|
|
|
47
|
-
| Web (Chrome / Edge) | ✓ | ✓ | ✓ | MediaStreamTrackProcessor + MediaPipe Selfie Segmentation (WASM, CDN) |
|
|
48
|
-
| Android (API 24+) | ✓ | ✓ | ✓ | OpenGL ES 3.0 + MediaPipe Selfie Segmentation (Tasks) |
|
|
49
|
-
| iOS (≥ 15) | ✓ | ✓ | ✓ | Metal + MediaPipe Selfie Segmentation (Tasks), verified on device. Older A11 devices (iPhone X) run at a lower frame rate |
|
|
50
|
-
| Safari / Firefox | n/a | n/a | n/a | No Insertable Streams; the effects throw a clear capability error |
|
|
51
|
-
|
|
52
|
-
### Coming soon
|
|
58
|
+
Like a synthesizer, the fastest way to judge it is to hear the patches. The demo book ships the gallery below; **every tile is a live link**, so click one and it opens on your own camera. Bring your own with a few lines (see [Make your own presets](#make-your-own-presets)).
|
|
53
59
|
|
|
54
|
-
|
|
60
|
+
<!-- PRESET-WAFFLE:START -->
|
|
55
61
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
62
|
+
<table cellspacing="0" cellpadding="2">
|
|
63
|
+
<tr><td align="right" valign="middle"><sub><b>Blur</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=blur-low" title="Low"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/blur-low.thumb.webp" alt="Low" title="Low" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=blur-medium" title="Medium"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/blur-medium.thumb.webp" alt="Medium" title="Medium" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=blur-high" title="High"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/blur-high.thumb.webp" alt="High" title="High" width="58" /></a></td></tr>
|
|
64
|
+
<tr><td align="right" valign="middle"><sub><b>Wizard Tower</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=wizard-tower" title="Wizard Tower"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/wizard-tower/wizard-tower.thumb.webp" alt="Wizard Tower" title="Wizard Tower" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=wizard-tower-night" title="Night"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/wizard-tower-night/wizard-tower-night.thumb.webp" alt="Night" title="Night" width="58" /></a></td></tr>
|
|
65
|
+
<tr><td align="right" valign="middle"><sub><b>Spaceship</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=observation-deck" title="Observation Deck"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/observation-deck/observation-deck.thumb.webp" alt="Observation Deck" title="Observation Deck" width="58" /></a></td></tr>
|
|
66
|
+
<tr><td align="right" valign="middle"><sub><b>Fairy Cave</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=fairy-cave" title="Fairy Cave"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/fairy-cave/fairy-cave.thumb.webp" alt="Fairy Cave" title="Fairy Cave" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=fairy-grotto" title="Grotto"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/fairy-grotto/fairy-grotto.thumb.webp" alt="Grotto" title="Grotto" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=fairy-hollow" title="Hollow"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/fairy-hollow/fairy-hollow.thumb.webp" alt="Hollow" title="Hollow" width="58" /></a></td></tr>
|
|
67
|
+
<tr><td align="right" valign="middle"><sub><b>Ocean</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=underwater" title="Underwater"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/underwater/underwater.thumb.webp" alt="Underwater" title="Underwater" width="58" /></a></td></tr>
|
|
68
|
+
<tr><td align="right" valign="middle"><sub><b>Corporate</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=corporate-blobs" title="Blobs"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/corporate-blobs/corporate-blobs.thumb.webp" alt="Blobs" title="Blobs" width="58" /></a></td></tr>
|
|
69
|
+
<tr><td align="right" valign="middle"><sub><b>Simiancraft</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simiancraft-light" title="Simiancraft Light"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/simiancraft-light.thumb.webp" alt="Simiancraft Light" title="Simiancraft Light" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simiancraft-dark" title="Simiancraft Dark"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/simiancraft-dark.thumb.webp" alt="Simiancraft Dark" title="Simiancraft Dark" width="58" /></a></td></tr>
|
|
70
|
+
<tr><td align="right" valign="middle"><sub><b>Office</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=office-dark" title="Dark Office"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/office-dark.thumb.webp" alt="Dark Office" title="Dark Office" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=office-light" title="Light Office"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/office-light.thumb.webp" alt="Light Office" title="Light Office" width="58" /></a></td></tr>
|
|
71
|
+
<tr><td align="right" valign="middle"><sub><b>Nature</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=landscape-light" title="Nature Light"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/landscape-light.thumb.webp" alt="Nature Light" title="Nature Light" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=landscape-dark" title="Nature Dark"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/landscape-dark.thumb.webp" alt="Nature Dark" title="Nature Dark" width="58" /></a></td></tr>
|
|
72
|
+
<tr><td align="right" valign="middle"><sub><b>Home</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=home-light" title="Home Light"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/home-light.thumb.webp" alt="Home Light" title="Home Light" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=home-dark" title="Home Dark"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/home-dark.thumb.webp" alt="Home Dark" title="Home Dark" width="58" /></a></td></tr>
|
|
73
|
+
<tr><td align="right" valign="middle"><sub><b>Sci-Fi</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=sci-fi-light" title="Landscape"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/sci-fi-light.thumb.webp" alt="Landscape" title="Landscape" width="58" /></a></td></tr>
|
|
74
|
+
<tr><td align="right" valign="middle"><sub><b>Ocean</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=oceanscape-dark" title="Underwater"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/oceanscape-dark.thumb.webp" alt="Underwater" title="Underwater" width="58" /></a></td></tr>
|
|
75
|
+
<tr><td align="right" valign="middle"><sub><b>Debug</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=debug-resolutions" title="Resolutions"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/debug-resolutions.thumb.webp" alt="Resolutions" title="Resolutions" width="58" /></a></td></tr>
|
|
76
|
+
<tr><td align="right" valign="middle"><sub><b>User</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=wolf-cave" title="Wolf Cave"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/wolf-cave.thumb.webp" alt="Wolf Cave" title="Wolf Cave" width="58" /></a></td></tr>
|
|
77
|
+
<tr><td align="right" valign="middle"><sub><b>Sky</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=clouds" title="Daytime"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/clouds/clouds.thumb.webp" alt="Daytime" title="Daytime" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=clouds-dawn" title="Dawn"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/clouds-dawn.thumb.webp" alt="Dawn" title="Dawn" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=clouds-dusk" title="Dusk"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/clouds-dusk.thumb.webp" alt="Dusk" title="Dusk" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=clouds-night" title="Night"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/clouds-night.thumb.webp" alt="Night" title="Night" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=clouds-otherworld" title="Otherworld"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/clouds-otherworld.thumb.webp" alt="Otherworld" title="Otherworld" width="58" /></a></td></tr>
|
|
78
|
+
<tr><td align="right" valign="middle"><sub><b>Plasma</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=plasma-ocean" title="Ocean"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/plasma-ocean.thumb.webp" alt="Ocean" title="Ocean" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=plasma-sunset" title="Sunset"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/plasma-sunset.thumb.webp" alt="Sunset" title="Sunset" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=plasma-mint" title="Mint"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/plasma-mint.thumb.webp" alt="Mint" title="Mint" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=plasma-fast" title="Fast"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/plasma-fast.thumb.webp" alt="Fast" title="Fast" width="58" /></a></td></tr>
|
|
79
|
+
<tr><td align="right" valign="middle"><sub><b>Kaleidoscope</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=kaleidoscope-stained-glass" title="Stained Glass"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/kaleidoscope-stained-glass.thumb.webp" alt="Stained Glass" title="Stained Glass" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=kaleidoscope-mandala" title="Mandala"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/kaleidoscope-mandala.thumb.webp" alt="Mandala" title="Mandala" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=kaleidoscope-prism" title="Prism"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/kaleidoscope-prism.thumb.webp" alt="Prism" title="Prism" width="58" /></a></td></tr>
|
|
80
|
+
<tr><td align="right" valign="middle"><sub><b>Neo-Memphis</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=neo-memphis-jazz-cup" title="Jazz Cup"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/neo-memphis-jazz-cup.thumb.webp" alt="Jazz Cup" title="Jazz Cup" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=neo-memphis-bauhaus" title="Bauhaus"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/neo-memphis-bauhaus.thumb.webp" alt="Bauhaus" title="Bauhaus" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=neo-memphis-confetti" title="Confetti"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/neo-memphis-confetti.thumb.webp" alt="Confetti" title="Confetti" width="58" /></a></td></tr>
|
|
81
|
+
<tr><td align="right" valign="middle"><sub><b>Halftone</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=halftone-boardroom" title="Boardroom"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/halftone-boardroom.thumb.webp" alt="Boardroom" title="Boardroom" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=halftone-press" title="Press"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/halftone-press.thumb.webp" alt="Press" title="Press" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=halftone-ripple" title="Ripple"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/halftone-ripple.thumb.webp" alt="Ripple" title="Ripple" width="58" /></a></td></tr>
|
|
82
|
+
<tr><td align="right" valign="middle"><sub><b>Aurora</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=aurora-corporate-silk" title="Corporate Silk"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/aurora-corporate-silk.thumb.webp" alt="Corporate Silk" title="Corporate Silk" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=aurora-dusk" title="Dusk"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/aurora-dusk.thumb.webp" alt="Dusk" title="Dusk" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=aurora-polar" title="Polar"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/aurora-polar.thumb.webp" alt="Polar" title="Polar" width="58" /></a></td></tr>
|
|
83
|
+
<tr><td align="right" valign="middle"><sub><b>Outrun</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-classic" title="Classic"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/outrun-classic.thumb.webp" alt="Classic" title="Classic" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-miami" title="Miami"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/outrun-miami.thumb.webp" alt="Miami" title="Miami" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-circuit" title="Circuit"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/outrun-circuit.thumb.webp" alt="Circuit" title="Circuit" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-acid" title="Acid"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/outrun-acid.thumb.webp" alt="Acid" title="Acid" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=outrun-vapor" title="Vapor"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/outrun-vapor.thumb.webp" alt="Vapor" title="Vapor" width="58" /></a></td></tr>
|
|
84
|
+
<tr><td align="right" valign="middle"><sub><b>Nebula</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=nebula" title="Nebula"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/nebula/nebula.thumb.webp" alt="Nebula" title="Nebula" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=nebula-ember" title="Ember"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/nebula-ember.thumb.webp" alt="Ember" title="Ember" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=nebula-drift" title="Drift"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/nebula-drift.thumb.webp" alt="Drift" title="Drift" width="58" /></a></td></tr>
|
|
85
|
+
<tr><td align="right" valign="middle"><sub><b>Simianlights</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simianlights" title="Simianlights"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/catalog/composites/simianlights/simianlights.thumb.webp" alt="Simianlights" title="Simianlights" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simianlights-glacier" title="Glacier"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/simianlights-glacier.thumb.webp" alt="Glacier" title="Glacier" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=simianlights-hearth" title="Hearth"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/simianlights-hearth.thumb.webp" alt="Hearth" title="Hearth" width="58" /></a></td></tr>
|
|
86
|
+
<tr><td align="right" valign="middle"><sub><b>Interior</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=interior-home" title="Home"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/interior-home.thumb.webp" alt="Home" title="Home" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=interior-office" title="Office"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/interior-office.thumb.webp" alt="Office" title="Office" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=interior-ab-shaft" title="A/B 1-shaft"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/interior-ab-shaft.thumb.webp" alt="A/B 1-shaft" title="A/B 1-shaft" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=interior-ab-3beam" title="A/B 3-beam"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/interior-ab-3beam.thumb.webp" alt="A/B 3-beam" title="A/B 3-beam" width="58" /></a></td></tr>
|
|
87
|
+
<tr><td align="right" valign="middle"><sub><b>Data-Mesh</b></sub></td><td valign="middle"><a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-datafield" title="Datafield"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/data-mesh-datafield.thumb.webp" alt="Datafield" title="Datafield" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-boardroom" title="Boardroom"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/data-mesh-boardroom.thumb.webp" alt="Boardroom" title="Boardroom" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-acid" title="Acid"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/data-mesh-acid.thumb.webp" alt="Acid" title="Acid" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-cobalt" title="Cobalt"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/data-mesh-cobalt.thumb.webp" alt="Cobalt" title="Cobalt" width="58" /></a> <a href="https://simiancraft.github.io/react-native-webrtc-kaleidoscope/?preset=data-mesh-slate" title="Slate"><img src="https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/demo/assets/thumbnails/data-mesh-slate.thumb.webp" alt="Slate" title="Slate" width="58" /></a></td></tr>
|
|
88
|
+
</table>
|
|
61
89
|
|
|
62
|
-
|
|
90
|
+
<sub>64 presets across 4 families (Effects, Worlds, Backgrounds, Shaders); click any to open it live, or <a href="#make-your-own-presets">make your own</a> in a few lines.</sub>
|
|
63
91
|
|
|
64
|
-
|
|
92
|
+
<!-- PRESET-WAFFLE:END -->
|
|
65
93
|
|
|
66
|
-
|
|
94
|
+
<sub>Runs on Android, iOS, and Chromium browsers (Chrome, Edge), against either `react-native-webrtc` or LiveKit. Platform specifics and the Safari/Firefox fallback are in [Platform support](#platform-support).</sub>
|
|
67
95
|
|
|
68
|
-
|
|
69
|
-
bun add @livekit/react-native @livekit/react-native-webrtc react-native-webrtc-kaleidoscope
|
|
70
|
-
```
|
|
96
|
+
## Quick start
|
|
71
97
|
|
|
72
|
-
|
|
98
|
+
Two paths to a working integration: hand it to a coding agent, or wire it yourself. Either way the shape is the same: install, add the config plugin, declare a preset book, bind a track.
|
|
73
99
|
|
|
74
|
-
|
|
100
|
+
### With an agent
|
|
75
101
|
|
|
76
|
-
|
|
77
|
-
import { bindKaleidoscope } from 'react-native-webrtc-kaleidoscope';
|
|
78
|
-
import { presets } from './kaleidoscope.preset-book';
|
|
102
|
+
Point your coding agent (Claude Code, Cursor, Copilot, …) at [`llms.txt`](./llms.txt). It is written for exactly this: a top-to-bottom integration guide that installs the package, writes the config-plugin entry, provisions a runnable preset book, and gets an effect on screen, with a six-file starting set lifted from the working `demo/`.
|
|
79
103
|
|
|
80
|
-
const { kaleidoscope } = bindKaleidoscope(localCameraTrack.mediaStreamTrack, { presets });
|
|
81
|
-
kaleidoscope('blur-soft');
|
|
82
104
|
```
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
```ts
|
|
87
|
-
import { KaleidoscopeProcessor } from 'react-native-webrtc-kaleidoscope/livekit';
|
|
88
|
-
|
|
89
|
-
await localVideoTrack.setProcessor(new KaleidoscopeProcessor(['blur']), true);
|
|
105
|
+
Read https://raw.githubusercontent.com/simiancraft/react-native-webrtc-kaleidoscope/main/llms.txt
|
|
106
|
+
and integrate react-native-webrtc-kaleidoscope into this Expo app: add the config
|
|
107
|
+
plugin, create a starter preset book, and show the PresetBookMenu over my camera track.
|
|
90
108
|
```
|
|
91
109
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
To tune the segmentation mask edge on the processor path, use `setMaskTuning`; it is the processor-path twin of the binding's `mask` verb. The mask edge is page-shared state every kaleidoscope pipeline reads per frame, so the write reaches every active processor on the next frame with no rebuild:
|
|
110
|
+
### Manually
|
|
95
111
|
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
setMaskTuning({ hardness: 0.2, threshold: 0.85 });
|
|
112
|
+
```sh
|
|
113
|
+
bun add react-native-webrtc react-native-webrtc-kaleidoscope
|
|
100
114
|
```
|
|
101
115
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
Add the config plugin to `app.config.ts`:
|
|
116
|
+
`react-native-webrtc` is a peer dependency; install it explicitly. (Using LiveKit instead? See [Using LiveKit](#using-livekit).) Add the config plugin to `app.config.ts`, then rebuild native code:
|
|
105
117
|
|
|
106
118
|
```ts
|
|
107
|
-
export default {
|
|
108
|
-
expo: {
|
|
109
|
-
plugins: ['react-native-webrtc-kaleidoscope'],
|
|
110
|
-
},
|
|
111
|
-
};
|
|
119
|
+
export default { expo: { plugins: ['react-native-webrtc-kaleidoscope'] } };
|
|
112
120
|
```
|
|
113
121
|
|
|
114
|
-
(`react-native-webrtc` 124.x does not ship a config plugin upstream; do not list it in `plugins`. If you are on a fork that adds one, add it explicitly.)
|
|
115
|
-
|
|
116
|
-
Then rebuild native code:
|
|
117
|
-
|
|
118
122
|
```sh
|
|
119
123
|
bunx expo prebuild
|
|
120
124
|
```
|
|
121
125
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
First declare a **preset book** in your project: a flat catalog of **composites**, the only things you can command. A composite is `{ name, taxonomy, thumbnail?, layers }`, where `taxonomy` is the picker's grouping path, root first (`[group]` or `[group, category]`, e.g. `['Backgrounds', 'Office']`); everything is a layer stack, painted back to front. A layer is `{ id, shader, target?, blend? }` plus the shader's fields: `image` takes a `source`; `direct` samples the ingest-normalized (upright, non-mirrored) camera frame for its target (`target: 'subject'` is the masked person, `target: 'background'` the full frame); `blur` and the generative shaders (`plasma`, `clouds`, …) take `uniforms`. `target` defaults to `'background'` (fullscreen); `'subject'` stencils to the segmented person. Each layer's `id` is unique within its composite. Declare the book `as const satisfies PresetBook` for per-layer typing.
|
|
126
|
+
Declare a **preset book**: a flat catalog of the effects you can command. A rudimentary one is three entries:
|
|
125
127
|
|
|
126
128
|
```ts
|
|
127
129
|
// kaleidoscope.preset-book.ts
|
|
128
|
-
import type {
|
|
130
|
+
import type { KaleidoscopePresetBook } from 'react-native-webrtc-kaleidoscope';
|
|
129
131
|
import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark';
|
|
130
|
-
// Packaged composites ship ready to use; import and spread them in.
|
|
131
132
|
import { wizardTower } from 'react-native-webrtc-kaleidoscope/composites/wizard-tower';
|
|
132
133
|
|
|
133
134
|
export const presets = {
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
taxonomy: ['Backgrounds', 'Office'],
|
|
138
|
-
thumbnail: officeDark,
|
|
135
|
+
'blur-soft': {
|
|
136
|
+
name: 'Soft blur',
|
|
137
|
+
taxonomy: ['Effects', 'Blur'],
|
|
139
138
|
layers: [
|
|
140
|
-
{ id: '
|
|
139
|
+
{ id: 'bg', shader: 'blur', target: 'background', uniforms: { sigma: 5 } },
|
|
141
140
|
{ id: 'you', shader: 'direct', target: 'subject' },
|
|
142
141
|
],
|
|
143
142
|
},
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
143
|
+
'office-dark': {
|
|
144
|
+
name: 'Dark office',
|
|
145
|
+
taxonomy: ['Backgrounds', 'Office'],
|
|
146
|
+
thumbnail: officeDark,
|
|
148
147
|
layers: [
|
|
149
|
-
{ id: '
|
|
148
|
+
{ id: 'office', shader: 'image', source: officeDark },
|
|
150
149
|
{ id: 'you', shader: 'direct', target: 'subject' },
|
|
151
150
|
],
|
|
152
151
|
},
|
|
153
|
-
// A packaged multi-layer
|
|
152
|
+
// A packaged multi-layer world, imported and spread in.
|
|
154
153
|
'wizard-tower': wizardTower,
|
|
155
|
-
} as const satisfies
|
|
154
|
+
} as const satisfies KaleidoscopePresetBook;
|
|
156
155
|
```
|
|
157
156
|
|
|
158
|
-
|
|
157
|
+
Bind a track once and drive it:
|
|
159
158
|
|
|
160
159
|
```ts
|
|
161
160
|
import { mediaDevices } from 'react-native-webrtc';
|
|
@@ -165,63 +164,133 @@ import { presets } from './kaleidoscope.preset-book';
|
|
|
165
164
|
const stream = await mediaDevices.getUserMedia({ video: true });
|
|
166
165
|
const [track] = stream.getVideoTracks();
|
|
167
166
|
|
|
168
|
-
const { kaleidoscope
|
|
167
|
+
const { kaleidoscope } = bindKaleidoscope(track, {
|
|
169
168
|
presets,
|
|
170
|
-
// Web rebuilds the pipeline per command and yields a NEW track; read it here
|
|
171
|
-
//
|
|
172
|
-
onTrack: (out) => {
|
|
173
|
-
/* setPreviewTrack(out) */
|
|
174
|
-
},
|
|
169
|
+
// Web rebuilds the pipeline per command and yields a NEW track; read it here.
|
|
170
|
+
// Native mutates the bound track in place.
|
|
171
|
+
onTrack: (out) => {/* setPreviewTrack(out) */},
|
|
175
172
|
});
|
|
176
173
|
|
|
177
|
-
|
|
174
|
+
kaleidoscope('wizard-tower'); // autocompletes from your book
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
For a ready-made gallery, drop in the picker; it reads your book directly:
|
|
178
|
+
|
|
179
|
+
```tsx
|
|
180
|
+
import { PresetBookMenu } from 'react-native-webrtc-kaleidoscope/preset-book-menu';
|
|
181
|
+
import { presets } from './kaleidoscope.preset-book';
|
|
182
|
+
|
|
183
|
+
<PresetBookMenu presets={presets} value={art} onSelect={setArt} />;
|
|
184
|
+
// route onSelect into kaleidoscope() and the picker is wired.
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Want the selection, live tweaks, and mask edge to survive a reload? Wrap your app in the [persistence provider](#persistence). Want the styled UI and a live tuning panel? See [Drop-in UI](#drop-in-ui).
|
|
188
|
+
|
|
189
|
+
### Using LiveKit
|
|
190
|
+
|
|
191
|
+
If your project uses `@livekit/react-native` it pulls in `@livekit/react-native-webrtc`, a fork that preserves the same `videoEffects` native classes and `_setVideoEffects` JS API. Kaleidoscope works against either fork; the Android Gradle script picks whichever one autolinking surfaced. Pick **one** fork, never both, or the native classes collide.
|
|
192
|
+
|
|
193
|
+
```sh
|
|
194
|
+
bun add @livekit/react-native @livekit/react-native-webrtc react-native-webrtc-kaleidoscope
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
On native, `@livekit/react-native` hands you a `LocalVideoTrack`; bind to its underlying `MediaStreamTrack`:
|
|
198
|
+
|
|
199
|
+
```ts
|
|
200
|
+
const { kaleidoscope } = bindKaleidoscope(localCameraTrack.mediaStreamTrack, { presets });
|
|
201
|
+
kaleidoscope('blur-soft');
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
On web, LiveKit owns the `RTCRtpSender`, so you cannot swap the track yourself; go through LiveKit's processor API. The opt-in `/livekit` subpath ships a ready-made processor (it needs `livekit-client`, which a LiveKit app already has):
|
|
205
|
+
|
|
206
|
+
```ts
|
|
207
|
+
import { KaleidoscopeProcessor, setMaskTuning } from 'react-native-webrtc-kaleidoscope/livekit';
|
|
208
|
+
|
|
209
|
+
await localVideoTrack.setProcessor(new KaleidoscopeProcessor(['blur']), true);
|
|
210
|
+
setMaskTuning({ hardness: 0.2, threshold: 0.85 }); // the processor-path twin of the `mask` verb
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
The processor constructor takes raw effect names (`'blur'`, a shader basename), not preset-book ids. The `true` shows the processed stream in your local preview. The processor tears down its Insertable-Streams pipeline on camera flip (`restart`) and unpublish (`destroy`), so repeated flips do not leak generators.
|
|
214
|
+
|
|
215
|
+
## Concepts
|
|
216
|
+
|
|
217
|
+
Four nouns, learned in the order you meet them.
|
|
218
|
+
|
|
219
|
+
- **Preset book**: the file you author (`kaleidoscope.preset-book.ts`); a flat, typed map of the effects your app can command. This is your point of entry; everything else hangs off it. Declare it `as const satisfies KaleidoscopePresetBook` for per-layer typing and autocompleting ids.
|
|
220
|
+
- **Preset**: one named entry in the book, `{ name, taxonomy, thumbnail?, layers, controls? }`. It is what `kaleidoscope(id)` applies. `taxonomy` is the picker's grouping path, root first (`[group, category]`, e.g. `['Backgrounds', 'Office']`).
|
|
221
|
+
- **Layer**: one entry in a preset's stack, `{ id, shader, target?, blend? }` plus the shader's own fields. Layers paint back to front. `id` is unique within the preset and is how you address it for live uniform patches.
|
|
222
|
+
- **Composite**: what a preset becomes at runtime, the layer stack rendered into the output frame. There is exactly one registered native effect, `composite`; "one effect" is just a composite with a single layer.
|
|
223
|
+
|
|
224
|
+
A layer's `shader` is what it draws: `image` (a bundled WebP, takes a `source`), `direct` (the ingest-normalized camera frame, upright and non-mirrored), `blur`, or a generative shader (`plasma`, `clouds`, …; these take `uniforms`). Its `target` is where it lands: `background` (fullscreen, the default) or `subject` (stenciled to the segmented person), and the two combine: `direct` + `subject` is the masked person, `direct` + `background` is the raw full frame. Its `blend` is how it stacks: opaque base, `normal` (alpha-over), or `additive`.
|
|
225
|
+
|
|
226
|
+
## The three verbs
|
|
227
|
+
|
|
228
|
+
<p align="center">
|
|
229
|
+
<code>kaleidoscope</code> • <code>transform</code> • <code>mask</code>
|
|
230
|
+
</p>
|
|
231
|
+
|
|
232
|
+
`bindKaleidoscope(track, { presets })` returns three functions. That is the whole runtime API.
|
|
233
|
+
|
|
234
|
+
```ts
|
|
235
|
+
const { kaleidoscope, transform, mask } = bindKaleidoscope(track, { presets, onTrack });
|
|
236
|
+
|
|
237
|
+
// kaleidoscope: the art axis. Pass a preset id (autocompletes from your book):
|
|
178
238
|
kaleidoscope('wizard-tower');
|
|
179
|
-
// Override a layer's uniforms live, addressed by
|
|
180
|
-
//
|
|
181
|
-
|
|
239
|
+
// Override a layer's uniforms live, addressed by id, while the preset is active
|
|
240
|
+
// (merged over the baked values, no pipeline rebuild). `shader` is a compile-time
|
|
241
|
+
// discriminant that types `uniforms`; the layer is reached by `id`, not by shader:
|
|
242
|
+
kaleidoscope('blur-soft', [{ id: 'bg', shader: 'blur', uniforms: { sigma: 9 } }]);
|
|
182
243
|
kaleidoscope(null); // clear the art
|
|
183
244
|
|
|
184
|
-
// transform
|
|
245
|
+
// transform: absolute geometry. Every call is the full state from identity;
|
|
185
246
|
// re-pass what you want to keep. rotate snaps to the nearest 90°.
|
|
186
247
|
transform({ flip: { x: true }, rotate: 90 });
|
|
187
248
|
transform(); // reset to identity
|
|
188
249
|
|
|
189
|
-
// mask
|
|
250
|
+
// mask: one segmentation edge for the whole composite (not per layer). Both 0..1.
|
|
190
251
|
mask({ hardness: 0.5, threshold: 0.5 });
|
|
191
252
|
```
|
|
192
253
|
|
|
193
|
-
|
|
254
|
+
Many uniforms are normalized `0..1` by convention; others (`sigma`, scales, counts) carry natural units, as the examples above show. JSDoc documents each option's expected range as an IntelliSense hint (ranges are not enforced at runtime; validate in your own layer if you forward them to end users).
|
|
194
255
|
|
|
195
|
-
|
|
256
|
+
**Tuning note.** All three platforms run MediaPipe selfie segmentation, so the mask edge that suits one may differ slightly from another. `mask` defaults to `0.5 / 0.5`; nudge `hardness` and `threshold` to match your camera and lighting.
|
|
196
257
|
|
|
197
|
-
|
|
258
|
+
## Make your own presets
|
|
198
259
|
|
|
199
|
-
|
|
260
|
+
A preset is a composition: **every preset is a back-to-front stack of N layers**, and the compositor does not care what produces a layer's texture, which is exactly what makes it extensible. To author one, stack layers in the order you want them painted, lowest first, the masked person (`{ shader: 'direct', target: 'subject' }`) usually last so it sits on top.
|
|
200
261
|
|
|
201
|
-
|
|
262
|
+
```ts
|
|
263
|
+
// A generative shader behind the person, with an additive glow layer on top of it.
|
|
264
|
+
'aurora-night': {
|
|
265
|
+
name: 'Aurora night',
|
|
266
|
+
taxonomy: ['Shaders', 'Aurora'],
|
|
267
|
+
layers: [
|
|
268
|
+
{ id: 'sky', shader: 'clouds', target: 'background', uniforms: { coverage: 0.4 } },
|
|
269
|
+
{ id: 'glow', shader: 'godrays', target: 'background', blend: 'additive' },
|
|
270
|
+
{ id: 'you', shader: 'direct', target: 'subject' },
|
|
271
|
+
],
|
|
272
|
+
},
|
|
273
|
+
```
|
|
202
274
|
|
|
203
|
-
|
|
204
|
-
import { useEffect, useState } from 'react';
|
|
205
|
-
import { PresetBookMenu } from 'react-native-webrtc-kaleidoscope/preset-book-menu';
|
|
206
|
-
import { presets } from './kaleidoscope.preset-book';
|
|
275
|
+
The packaged `wolf-cave` preset in the [demo book](./demo/kaleidoscope.preset-book.ts) is a worked example of a multi-layer world (a generative shader, a cut-out image, and the masked person).
|
|
207
276
|
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
useEffect(() => {
|
|
212
|
-
if (art) kaleidoscope(art);
|
|
213
|
-
else kaleidoscope(null);
|
|
214
|
-
}, [art, kaleidoscope]);
|
|
277
|
+
- **Bundled images** ship as tree-shakeable `image` layers, filed by category and imported per image (`import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark'`). On web a `source` can also be any image URL or data URI; native resolves bundled ids only. See [`catalog/images/README.md`](./catalog/images/README.md).
|
|
278
|
+
- **New shaders** drop a single `.frag` + typed `.ts` into `catalog/shaders/<name>/`; `bun run build:shaders` codegens the web and Android sources and transpiles the iOS Metal. The canonical upright frame and the mask stencil come for free; you write zero orientation code. See [`catalog/shaders/README.md`](./catalog/shaders/README.md).
|
|
279
|
+
- **Packaged composites** (the Worlds) live in `catalog/composites/<name>/` behind a `./composites/<name>` subpath export; import and spread one into your book.
|
|
215
280
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
281
|
+
After adding a preset to the demo book, regenerate its thumbnail and this README's gallery: `bun run thumbs && bun run gen:waffle` (see [Authoring tooling](#authoring-tooling)).
|
|
282
|
+
|
|
283
|
+
## Drop-in UI
|
|
284
|
+
|
|
285
|
+
Build your own controls against the three verbs, or import the headless, controlled components. All are presentational: they emit a selection or a patch, you apply it.
|
|
286
|
+
|
|
287
|
+
### The picker
|
|
219
288
|
|
|
220
|
-
`PresetBookMenu` is a two-level browser driven by each
|
|
289
|
+
`PresetBookMenu` (from `react-native-webrtc-kaleidoscope/preset-book-menu`) is a two-level browser driven by each preset's `taxonomy`: a tab row across the top, one tab per **group** (`taxonomy[0]`), and a left-hand menu of **categories** (`taxonomy[1]`) under the active group; the tile grid filters by both. A flat (depth-1) group shows no category menu. Every preset renders as a uniform tile: a wallpaper when it has a `thumbnail`, a recessed button of the same footprint when it does not, so a thumbnail-less preset never breaks the grid. The same pieces ship as standalone primitives (`PresetGrid`, `PresetTile`, the `usePresetBookMenu` hook, `PresetBookMenuLayout`) for custom layouts.
|
|
221
290
|
|
|
222
291
|
**Styling, three tiers.** Sensible defaults out of the box; override with an RN `style` prop, a `className` prop, or a `renderTile` render-prop slot for full control.
|
|
223
292
|
|
|
224
|
-
**NativeWind-ready.** The components accept `className`.
|
|
293
|
+
**NativeWind-ready.** The components accept `className`. Turn it on by importing the opt-in registration once (`nativewind` is an optional peer; the core `./preset-book-menu` import never pulls it in):
|
|
225
294
|
|
|
226
295
|
```ts
|
|
227
296
|
import { registerKaleidoscopeNativeWind } from 'react-native-webrtc-kaleidoscope/nativewind';
|
|
@@ -240,7 +309,6 @@ import {
|
|
|
240
309
|
TransformControlPanel,
|
|
241
310
|
} from 'react-native-webrtc-kaleidoscope/preset-control-panel';
|
|
242
311
|
|
|
243
|
-
// `controls` is the object from bindKaleidoscope(track, { presets }).
|
|
244
312
|
<KaleidoscopeThemeProvider>
|
|
245
313
|
<PresetControlPanel presets={presets} value={art} onPatch={(p) => controls.kaleidoscope(art, [p])} />
|
|
246
314
|
<MaskControlPanel hardness={h} threshold={t} onChange={setMask} />
|
|
@@ -248,28 +316,19 @@ import {
|
|
|
248
316
|
</KaleidoscopeThemeProvider>
|
|
249
317
|
```
|
|
250
318
|
|
|
251
|
-
Each preset supplies its editor as a `controls` component on the book entry
|
|
319
|
+
Each preset supplies its editor as a `controls` component on the book entry; packaged composites export theirs at `react-native-webrtc-kaleidoscope/composites/<name>/controls`. For your own presets, compose `CompositeLayerControlPanel` over a shader's control descriptor (or `makeControls` for a custom widget). `KaleidoscopeThemeProvider` themes every control at once. The sliders need `@react-native-community/slider` (an optional peer; a native module, so it needs a dev-client rebuild). Live per-layer tuning runs on web today; on native the editor renders while the live per-layer uniform channel is in progress. Mask and transform are live on every platform.
|
|
252
320
|
|
|
253
|
-
|
|
321
|
+
### Persistence
|
|
254
322
|
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
### Persistence (the selection that survives a reload)
|
|
258
|
-
|
|
259
|
-
`react-native-webrtc-kaleidoscope/persistence` ships a provider + hook that keep the person's selection across launches: the last applied preset id, the per-layer uniform patches they dialed in through the control panels (kept per preset, so tweaks to several presets all survive), and the mask edge.
|
|
323
|
+
`react-native-webrtc-kaleidoscope/persistence` ships a provider + hook that keep the person's selection across launches: the last applied preset id, the per-layer uniform patches they dialed in (kept per preset), and the mask edge.
|
|
260
324
|
|
|
261
325
|
```tsx
|
|
262
326
|
// App root:
|
|
263
327
|
import { KaleidoscopeStateProvider } from 'react-native-webrtc-kaleidoscope/persistence';
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
<KaleidoscopeStateProvider presets={presets}>
|
|
267
|
-
<App />
|
|
268
|
-
</KaleidoscopeStateProvider>;
|
|
328
|
+
<KaleidoscopeStateProvider presets={presets}><App /></KaleidoscopeStateProvider>;
|
|
269
329
|
|
|
270
330
|
// In the screen that binds the track:
|
|
271
331
|
import { useKaleidoscopeState } from 'react-native-webrtc-kaleidoscope/persistence';
|
|
272
|
-
|
|
273
332
|
const { hydrated, presetId, mask, setPreset, setMask, setPatch, patchesFor, reset } =
|
|
274
333
|
useKaleidoscopeState<typeof presets>();
|
|
275
334
|
|
|
@@ -280,89 +339,87 @@ useEffect(() => {
|
|
|
280
339
|
}, [hydrated, controls, presetId]);
|
|
281
340
|
```
|
|
282
341
|
|
|
283
|
-
Route the picker's `onSelect` into `setPreset`, the editor's `onPatch` into `setPatch(presetId, patch)
|
|
284
|
-
|
|
285
|
-
The default store is [`@react-native-async-storage/async-storage`](https://github.com/react-native-async-storage/async-storage) (an optional peer; install it alongside the library when you use this subpath; on web it is localStorage-backed). To back it with something else (MMKV, a server), pass any `{ load, save }` pair as the `store` prop; the stored shape is versioned and parses tolerantly, so a malformed payload reads as empty rather than throwing.
|
|
342
|
+
Route the picker's `onSelect` into `setPreset`, the editor's `onPatch` into `setPatch(presetId, patch)`, and the mask panel into `setMask`; every write persists. The default store is [`@react-native-async-storage/async-storage`](https://github.com/react-native-async-storage/async-storage) (an optional peer; localStorage-backed on web). Back it with anything else (MMKV, a server) by passing a `{ load, save }` pair as the `store` prop; the stored shape is versioned and parses tolerantly, so a malformed payload reads as empty rather than throwing.
|
|
286
343
|
|
|
287
|
-
##
|
|
344
|
+
## Performance
|
|
288
345
|
|
|
289
|
-
|
|
346
|
+
The product is the masked-background composite, and its cost is something you can see, not guess.
|
|
290
347
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
| Underwater | Nebula | Simianlights |
|
|
295
|
-
| <img src="composites/underwater/underwater.thumb.webp" width="220" alt="underwater" /> | <img src="composites/nebula/nebula.thumb.webp" width="220" alt="nebula" /> | <img src="composites/simianlights/simianlights.thumb.webp" width="220" alt="simianlights" /> |
|
|
296
|
-
| Corporate Blobs | | |
|
|
297
|
-
| <img src="composites/corporate-blobs/corporate-blobs.thumb.webp" width="220" alt="corporate-blobs" /> | | |
|
|
348
|
+
- **Annotated shader cost.** Each generative shader's `.ts` carries a measured GPU cost annotation (relative to `plasma` as the cheap baseline), so you know what a preset spends before you ship it.
|
|
349
|
+
- **One resolution knob.** Raw shader compute scales with output resolution, handled by the resolution tier (`targetShortSide`), not by per-effect orientation tricks. Drop the tier on weak GPUs; the mask and composite logic are unchanged.
|
|
350
|
+
- **Bounded work per frame.** Compositing is per-layer through a single mask stencil; a new shader inherits the pipeline's frame budget rather than adding a pass of its own.
|
|
298
351
|
|
|
299
|
-
|
|
352
|
+
## Authoring tooling
|
|
300
353
|
|
|
301
|
-
|
|
354
|
+
The kit ships the same tools used to build it. All regenerate from the command line.
|
|
302
355
|
|
|
303
|
-
|
|
356
|
+
| Command | What it does |
|
|
357
|
+
|---|---|
|
|
358
|
+
| `bun run bench:shader` | SPIR-V weighted op-cost bench for a shader (good for no-loop shaders; rank by the meter for loop-bound ones). |
|
|
359
|
+
| `bun run shader:view` | WebGL2 A/B viewer with a live GPU-time meter for tuning a shader against the camera. |
|
|
360
|
+
| `bun run thumbs` | Render a `320×180` WebP thumbnail per preset in a book (the gallery tiles and picker wallpapers). |
|
|
361
|
+
| `bun run gen:waffle` | Regenerate this README's [preset gallery](#presets) from the demo book + thumbnails. Run after adding a preset; `--check` gates staleness. |
|
|
304
362
|
|
|
305
|
-
|
|
306
|
-
|---|---|---|
|
|
307
|
-
| Office | <img src="images/office/office-light.thumb.webp" width="220" alt="office-light" /> | <img src="images/office/office-dark.thumb.webp" width="220" alt="office-dark" /> |
|
|
308
|
-
| Home | <img src="images/home/home-light.thumb.webp" width="220" alt="home-light" /> | <img src="images/home/home-dark.thumb.webp" width="220" alt="home-dark" /> |
|
|
309
|
-
| Nature | <img src="images/nature/landscape-light.thumb.webp" width="220" alt="landscape-light" /> | <img src="images/nature/landscape-dark.thumb.webp" width="220" alt="landscape-dark" /> |
|
|
310
|
-
| Sci-Fi | <img src="images/sci-fi/sci-fi-light.thumb.webp" width="220" alt="sci-fi-light" /> | |
|
|
311
|
-
| Underwater | | <img src="images/underwater/oceanscape-dark.thumb.webp" width="220" alt="oceanscape-dark" /> |
|
|
312
|
-
| Simiancraft | <img src="images/simiancraft/simiancraft-light.thumb.webp" width="220" alt="simiancraft-light" /> | <img src="images/simiancraft/simiancraft-dark.thumb.webp" width="220" alt="simiancraft-dark" /> |
|
|
363
|
+
## Only ship what you use
|
|
313
364
|
|
|
314
|
-
|
|
365
|
+
Install size and bundle size are different numbers, and you pay for the second.
|
|
315
366
|
|
|
316
|
-
|
|
367
|
+
- **Per-asset subpath exports.** Each bundled image and packaged composite is its own file behind its own subpath (`./images/<category>/<leaf>`, `./composites/<name>`), and the package sets `sideEffects: false`. A web bundler drops every preset you do not import.
|
|
368
|
+
- **Native ships only what your book references.** Metro does not tree-shake, so an unused preset is simply never imported; and `expo prebuild` copies only the assets your preset book actually names into the native bundle. Declare ten rooms, reference two, ship two.
|
|
369
|
+
- **Assets are WebP.** Backgrounds are 720p WebP; each platform decodes it natively (BitmapFactory on Android, ImageIO / MTKTextureLoader on iOS), so a full image set is a couple of megabytes, not sixteen.
|
|
317
370
|
|
|
318
|
-
|
|
371
|
+
## For LLMs and agents
|
|
319
372
|
|
|
320
|
-
|
|
373
|
+
Feeding this repo into Claude / Cursor / Copilot, or shipping it into an app with an agent? Read [`llms.txt`](./llms.txt) first: the same scope as this README in a denser, parseable shape, with a copy-paste starting fileset that runs on all three platforms. It is the file to hand an agent for a hands-off integration (see [Quick start → With an agent](#with-an-agent)).
|
|
321
374
|
|
|
322
|
-
|
|
375
|
+
## Architecture
|
|
323
376
|
|
|
324
|
-
|
|
325
|
-
- **Image source.** An `image` layer's `source` is a bundled preset name on native (the upstream `_setVideoEffects` registry is keyed by flat strings, not URIs), but on web it accepts either a preset name or an arbitrary image URL or data URI.
|
|
326
|
-
- **Background presets ship as tree-shakeable files.** The bundled backgrounds (see [Background presets](#background-presets)) are importable per image: `import { officeDark } from 'react-native-webrtc-kaleidoscope/images/office/office-dark'`. Each image is its own file behind its own subpath export, and the package sets `sideEffects: false`, so an unused preset is dropped by web bundlers; since Metro doesn't tree-shake, it is simply never imported on native. Web resolves the bundled WebP to a URL; native loads its own bundled copy by name. Web also still accepts an arbitrary image URL or data URI. See [`catalog/images/README.md`](./catalog/images/README.md).
|
|
327
|
-
- **Segmentation model on web.** The web compositor loads MediaPipe Selfie Segmentation from the jsdelivr CDN (`cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation`) on first use. A strict Content-Security-Policy must allow that origin for `script-src`, `connect-src`, and the WASM fetch, and the effects do not work offline. The `transform` ops need no model.
|
|
328
|
-
- **Browser support on web.** Effects use Insertable Streams (`MediaStreamTrackProcessor` and `MediaStreamTrackGenerator`), which ship in Chromium-based browsers (Chrome, Edge); Safari and Firefox lack the API, so the effects throw a clear capability error and the demo falls back to the unprocessed track.
|
|
329
|
-
- **Android revokes the camera ~60 s into the background.** Android 11+ disables camera access for backgrounded apps by device policy (`ERROR_CAMERA_DISABLED`); `react-native-webrtc` logs the event but never restarts capture, so after a long background the preview stays black on resume and no effect or preset change can recover it (the frame source is dead, not the pipeline). Re-acquire `getUserMedia` when the app returns from the background; the demo's [`use-loopback-stream.ts`](./demo/src/use-loopback-stream.ts) shows the `AppState` pattern, and effects re-bind to the new track through the normal verbs.
|
|
377
|
+
Every effect is a **layer in one compositor**: a bundled `image`, a `direct` passthrough (the masked person or the raw camera), a camera-sampling `blur`, or a generative shader, composited back to front with per-layer blend. There is one registered native effect, `composite`; its layer stack is delivered out of band and reconciled each command. Adding a background source is adding a layer kind, not a new effect, which is why [a new shader](#make-your-own-presets) reaches all three platforms from one folder.
|
|
330
378
|
|
|
331
|
-
|
|
379
|
+
Canonical assets live in three root, folder-per-item directories, out of the TypeScript build path:
|
|
332
380
|
|
|
333
|
-
-
|
|
334
|
-
-
|
|
335
|
-
-
|
|
336
|
-
- **Not a streaming protocol replacement.** The transformed track plugs into the consumer's existing `RTCPeerConnection` pipeline.
|
|
381
|
+
- `catalog/shaders/<name>/`: each shader's `.frag` plus its typed `.ts` (uniforms + control descriptor). All share one vertex stage; `bun run build:shaders` codegens the web and Android sources and transpiles the iOS Metal.
|
|
382
|
+
- `catalog/images/<category>/`: images filed by category; each is a `<leaf>.webp`, its `<leaf>.thumb.webp`, and the `<leaf>.ts` / `<leaf>.web.ts` loader pair, behind a subpath export.
|
|
383
|
+
- `catalog/composites/<name>/`: each packaged composite, behind a `./composites/<name>` subpath export.
|
|
337
384
|
|
|
338
|
-
|
|
385
|
+
The code spans the platform surfaces: `src/` (JS facade + shared types), `web-driver/` (WebGL2 pipeline), `android/` (OpenGL ES 3.0), and `ios/` (Metal). Orientation is normalized exactly once at the ingest, so effects do zero orientation work. The full contract, including the texture-orientation convention and the mask buffer-ownership rule, is in [`PATTERNS.md`](./PATTERNS.md).
|
|
339
386
|
|
|
340
|
-
|
|
387
|
+
## Platform support
|
|
341
388
|
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
389
|
+
| Platform | Transform | Blur | Background replacement | Notes |
|
|
390
|
+
|---|---|---|---|---|
|
|
391
|
+
| Web (Chrome / Edge) | ✓ | ✓ | ✓ | MediaStreamTrackProcessor + MediaPipe Selfie Segmentation (WASM, CDN) |
|
|
392
|
+
| Android (API 24+) | ✓ | ✓ | ✓ | OpenGL ES 3.0 + MediaPipe Selfie Segmentation (Tasks) |
|
|
393
|
+
| iOS (≥ 15) | ✓ | ✓ | ✓ | Metal + MediaPipe Selfie Segmentation (Tasks), verified on device. Older A11 devices (iPhone X) run at a lower frame rate |
|
|
394
|
+
| Safari / Firefox | n/a | n/a | n/a | No Insertable Streams; the effects throw a clear capability error and the demo falls back to the unprocessed track |
|
|
345
395
|
|
|
346
|
-
|
|
396
|
+
A few runtime differences worth knowing before you wire effects in:
|
|
347
397
|
|
|
348
|
-
-
|
|
349
|
-
-
|
|
350
|
-
-
|
|
351
|
-
- `ios/`: Metal pipeline (Swift) with MediaPipe Tasks segmentation (`selfie_segmenter.tflite`, the same model Android bundles); the canonical GLSL transpiles to Metal via `scripts/build-shaders.ts`.
|
|
398
|
+
- **Output track.** On web each `kaleidoscope` / `transform` command rebuilds the Insertable-Streams pipeline and yields a NEW `MediaStreamTrack` via `onTrack`; on native the bound track is mutated in place. `mask` updates the running composite with no rebuild on either platform.
|
|
399
|
+
- **Segmentation model on web.** The web compositor loads MediaPipe Selfie Segmentation from the jsDelivr CDN on first use. A strict Content-Security-Policy must allow that origin for `script-src`, `connect-src`, and the WASM fetch, and the effects do not work offline. `transform` needs no model.
|
|
400
|
+
- **Android revokes the camera ~60 s into the background.** Android 11+ disables camera access for backgrounded apps by device policy; `react-native-webrtc` logs it but never restarts capture, so after a long background the preview stays black on resume. Re-acquire `getUserMedia` when the app returns from the background; the demo's [`use-loopback-stream.ts`](./demo/src/use-loopback-stream.ts) shows the `AppState` pattern, and effects re-bind to the new track through the normal verbs.
|
|
352
401
|
|
|
353
|
-
|
|
402
|
+
## What this isn't
|
|
354
403
|
|
|
355
|
-
|
|
404
|
+
- **Not a fork of `react-native-webrtc`.** A thin layer over its undocumented `_setVideoEffects` registry on native, and `MediaStreamTrackProcessor` on web. Install alongside it.
|
|
405
|
+
- **Not a managed cloud SaaS.** Effects run locally on the device; the track stays peer-to-peer. No service, no API key, no per-minute billing.
|
|
406
|
+
- **Not a face-filter SDK.** Effects are background segmentation and frame transforms, not facial AR.
|
|
407
|
+
- **Not a streaming protocol replacement.** The transformed track plugs into your existing `RTCPeerConnection` pipeline.
|
|
356
408
|
|
|
357
409
|
## Reference
|
|
358
410
|
|
|
411
|
+
- [CHANGELOG.md](./CHANGELOG.md): release history (semantic-release, Conventional Commits).
|
|
359
412
|
- [CONTRIBUTING.md](./CONTRIBUTING.md): setup, scripts, commit conventions.
|
|
360
|
-
- [AGENTS.md](./AGENTS.md):
|
|
361
|
-
- [PATTERNS.md](./PATTERNS.md): codebase conventions and how
|
|
413
|
+
- [AGENTS.md](./AGENTS.md): contributor and agent orientation for working on the repo.
|
|
414
|
+
- [PATTERNS.md](./PATTERNS.md): codebase conventions, the orientation contract, and how to extend.
|
|
362
415
|
- [catalog/shaders/README.md](./catalog/shaders/README.md): adding and extending shaders.
|
|
416
|
+
- [catalog/images/README.md](./catalog/images/README.md): the image folder layout and formats.
|
|
417
|
+
- [llms.txt](./llms.txt): dense, agent-oriented integration guide.
|
|
363
418
|
- [SECURITY.md](./SECURITY.md): security policy and reporting.
|
|
364
419
|
- [NOTICE.md](./NOTICE.md): third-party attributions.
|
|
365
420
|
|
|
366
421
|
---
|
|
367
422
|
|
|
368
423
|
MIT licensed. © 2026 Jesse Harlin / [Simiancraft](https://github.com/simiancraft).
|
|
424
|
+
|
|
425
|
+
<p align="center"><sub>Crafted with care by <a href="https://simiancraft.com">Simiancraft</a>.</sub></p>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "react-native-webrtc-kaleidoscope",
|
|
3
|
-
"version": "2.7.
|
|
3
|
+
"version": "2.7.3",
|
|
4
4
|
"description": "Live video effects (blur, background replacement, generative backgrounds, flip/rotate) for react-native-webrtc, packaged as a managed-Expo-friendly Expo Module. Working on web, Android, and iOS. Active development.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"react-native",
|
|
@@ -587,6 +587,8 @@
|
|
|
587
587
|
"bench:shader": "bun run scripts/shader-cost.ts",
|
|
588
588
|
"shader:view": "bun run scripts/shader-view.ts",
|
|
589
589
|
"thumbs": "bun run build && bun tools/thumbnails/make-thumbnails.ts --book demo/kaleidoscope.preset-book.ts --out demo/assets/thumbnails --repo",
|
|
590
|
+
"gen:waffle": "bun run scripts/gen-preset-waffle.ts",
|
|
591
|
+
"check:waffle": "bun run scripts/gen-preset-waffle.ts --check",
|
|
590
592
|
"demo": "bun run build && cd demo && bun run start",
|
|
591
593
|
"demo:wsl": "bun run build && cd demo && bun run start:wsl",
|
|
592
594
|
"demo:ios": "bun run build && cd demo && bun run ios",
|