oidn-web 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -4
- package/dist/{oidn.mjs → oidn.js} +3520 -3407
- package/dist/{oidn.umd.js → oidn.umd.cjs} +474 -464
- package/lib/UNet.d.ts +22 -5
- package/lib/UNet.js +101 -74
- package/lib/UNet.js.map +1 -1
- package/lib/WGPUComputePass.d.ts +13 -3
- package/lib/WGPUComputePass.js +85 -29
- package/lib/WGPUComputePass.js.map +1 -1
- package/lib/backend.js +6 -1
- package/lib/backend.js.map +1 -1
- package/lib/helper.d.ts +3 -0
- package/lib/helper.js +6 -0
- package/lib/helper.js.map +1 -1
- package/lib/process.d.ts +14 -5
- package/lib/process.js +207 -86
- package/lib/process.js.map +1 -1
- package/package.json +3 -1
- package/src/UNet.ts +154 -100
- package/src/WGPUComputePass.ts +100 -38
- package/src/backend.ts +8 -2
- package/src/helper.ts +8 -0
- package/src/process.ts +227 -98
package/README.md
CHANGED
|
@@ -1,17 +1,24 @@
|
|
|
1
1
|
# Open Image Denoise on the Web
|
|
2
2
|
|
|
3
3
|
This library brings the state-of-the-art AI-based denoising library [Open Image Denoise](https://github.com/RenderKit/oidn) to the web.
|
|
4
|
+
Currently it's only available on the browsers support WebGPU.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
+
It's used in the [Vector to 3D](https://www.figma.com/community/plugin/1264600219316901594/) Figma plugin for high quality rendering and denoising.
|
|
7
|
+
|
|
8
|
+
| 2000 Samples | 3 Samples | 3 Samples + Denoised |
|
|
9
|
+
| :----------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------: |
|
|
10
|
+
|  |  |  |
|
|
6
11
|
|
|
7
12
|
## How it Works.
|
|
8
13
|
|
|
9
14
|
It uses [tfjs](https://github.com/tensorflow/tfjs) to build the UNet model used by the OIDN. Then use the model to do prediction with a WebGPU backend from the image data.
|
|
10
15
|
|
|
11
|
-
Currently it's only available on the browser with WebGPU enabled.
|
|
12
|
-
|
|
13
16
|
## How to Use
|
|
14
17
|
|
|
18
|
+
[Basic Example](https://oidn-web-example.vercel.app/) ([Code](https://github.com/pissang/oidn-web-example/blob/main/src/main.js))
|
|
19
|
+
|
|
20
|
+
[Use with three-gpu-pathtracer](https://oidn-web-example.vercel.app/three-gpu-pathtracer.html) ([Code](https://github.com/pissang/oidn-web-example/blob/main/src/three-gpu-pathtracer.js))
|
|
21
|
+
|
|
15
22
|
### Install
|
|
16
23
|
|
|
17
24
|
```shell
|
|
@@ -69,7 +76,7 @@ initUNetFromURL('./weights/rt_hdr.tza', undefined, {
|
|
|
69
76
|
});
|
|
70
77
|
```
|
|
71
78
|
|
|
72
|
-
###
|
|
79
|
+
### Use auxiliary images
|
|
73
80
|
|
|
74
81
|
```ts
|
|
75
82
|
import { UNet, initUNetFromURL } from 'oidn-web';
|
|
@@ -98,6 +105,8 @@ initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', undefined, {
|
|
|
98
105
|
|
|
99
106
|
If you already have a WebGPU path tracer. You can integrate the oidn-web into your pipeline. It supports input/output gpu buffers to avoid the cost of syncing between CPU and GPU.
|
|
100
107
|
|
|
108
|
+
`hdr` and `aux` are required in the WebGPU pipeline.
|
|
109
|
+
|
|
101
110
|
```ts
|
|
102
111
|
initUNetFromURL(
|
|
103
112
|
'./weights/rt_hdr_alb_nrm.tza',
|
|
@@ -143,3 +152,7 @@ initUNetFromURL('./weights/rt_hdr_alb_nrm_small.tza', ...);
|
|
|
143
152
|
```
|
|
144
153
|
|
|
145
154
|
Other combinations can be found in the [oidn-weights](https://github.com/RenderKit/oidn-weights)
|
|
155
|
+
|
|
156
|
+
## Credits
|
|
157
|
+
|
|
158
|
+
Huge thanks to Max Liani for his series: https://maxliani.wordpress.com/2023/03/17/dnnd-1-a-deep-neural-network-dive/. My work is mostly inspired by it.
|