longpipe 0.0.4 → 0.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -42,7 +42,7 @@ await pipeline.ready // optional — resolves once the effect i
|
|
|
42
42
|
`pipeline.stream` is wired synchronously and emits the unprocessed input until the model is ready (~1–3s on cold start, depending on hardware), so the consumer sees live video the whole time.
|
|
43
43
|
|
|
44
44
|
|
|
45
|
-
##
|
|
45
|
+
## Performance
|
|
46
46
|
|
|
47
47
|
Longpipe uses custom-trained models built with an EfficientNetLite encoder, and a UNet style decoder, breaking it down into 7 different variations/performance presets, which vary number of channels, encoder size as well as input size. Even at 0.0.1, the first version of Longpipe (across all variants) have much higher segmentation accuracy than alternative open source models like Mediapipe and Bodypix, while also having much better real-world performance due to implmenting the model as pure WebGPU/WebGL shaders and using a zero-copy fully GPU pipeline.
|
|
48
48
|
|
|
@@ -51,7 +51,7 @@ Using average alpha pixel error on the P3M-10K valdation dataset (499 landscape
|
|
|
51
51
|
|
|
52
52
|
<img width="640" alt="lp-quality" src="https://github.com/user-attachments/assets/c7044937-95af-496f-893a-a05a50d7c914" />
|
|
53
53
|
|
|
54
|
-
###
|
|
54
|
+
### Speed
|
|
55
55
|
With the pure GPU zero copy pipeline, Longpipe achieves better real world performance than mediapipe using much larger models.
|
|
56
56
|
|
|
57
57
|
<img width="640" alt="lp-speed" src="https://github.com/user-attachments/assets/ce658b8d-dc62-4d6b-90f8-cd62dfef2a59" />
|
|
@@ -157,8 +157,9 @@ Works on Chromium (Chrome, Edge), Firefox, and Safari (desktop and iOS). WebGPU
|
|
|
157
157
|
|
|
158
158
|
Two layers:
|
|
159
159
|
|
|
160
|
-
- **Model** (`src/model/`) — EfficientNet-Lite-0 encoder with a U-Net decoder, written as TypeScript op classes. Each layer is a class; weights load as binary tensors at init; the backend (WebGPU or WebGL2) is injected at construction. BatchNorm is fused into conv weights at export — there is no BN op at inference.
|
|
161
|
-
-
|
|
160
|
+
- **Model** (`src/model/`) — EfficientNet-Lite-0 encoder with a U-Net decoder, written as TypeScript op classes. Each layer is a class; weights load as binary tensors at init; the backend (WebGPU or WebGL2) is injected at construction. BatchNorm is fused into conv weights at export — there is no BN op at inference.
|
|
161
|
+
-
|
|
162
|
+
- **Pipeline** (`src/pipeline/`) — capability detection, per-browser frame transport selection, worker spawn, audio passthrough, autotune, and the adaptive controller. Designed to absorb the browser/codec/canvas plumbing complexity so consumers don't have to.
|
|
162
163
|
|
|
163
164
|
Six trained presets cover the hardware range:
|
|
164
165
|
|
package/package.json
CHANGED