oidn-web 0.4.0 → 0.5.0

package/CHANGELOG.md

@@ -2,6 +2,50 @@
 
 All notable changes to this project are documented in this file.
 
+## [0.5.0] - 2026-10-09
+
+oidn-web 0.5.0 simplifies device setup, makes tiled execution always settle, and switches FP16 convolutions to implicit GEMM. It contains breaking changes for TypeScript callers that pass `adapterInfo`, for code that constructs `UNet` directly, and for code that relies on square tiles or per-frame tile pacing.
+
+### Breaking changes
+
+- `initUNetFromURL` and `initUNetFromBuffer` take `{ device }` as the second argument. `adapterInfo` was only needed by the removed TensorFlow.js backend and is no longer accepted by the type. Extra properties are ignored at runtime, but TypeScript rejects an object literal that still contains `adapterInfo`.
+- `new UNet(tensors, device, options)` takes a `GPUDevice` instead of a `{ device, adapterInfo }` object.
+- `tileExecute` now continues on the event loop between tiles by default instead of waiting for `requestAnimationFrame`. Pass `scheduling: 'animation-frame'` to keep pacing tiles to display frames.
+- Tiles are balanced rectangles with overlap only on edges shared with another tile, instead of fixed-size squares. Tile count, position, and size reported to `progress` differ from 0.4.0. The default `dynamicTile.initialTileSize` is now 432.
+- `UNetExecutionStats` no longer has `tileWidth` and `tileHeight`. Use `tileColumns`, `tileRows`, `tileOverlap`, `inputPixelCount`, and `inputShapeCount` instead.
+- FP16 convolutions use implicit GEMM by default. Output differs slightly from 0.4.0 because accumulation order changed. Set `kernel: 'direct'` to reproduce the 0.4.0 FP16 path.
+
+### Added
+
+- `hdrTransfer: 'log'` for RTLightmap HDR models, alongside the default PU transfer.
+- `error` callback on `tileExecute` for asynchronous failures, WebGPU device loss, and exceptions thrown by `progress` or `done`. `progress` and `done` may return promises.
+- `scheduling: 'event-loop' | 'animation-frame'`, `tileOverlap`, and `wholeImage` options on `tileExecute`.
+- `prepareForImage(width, height)` to create per-shape GPU resources before the first denoise.
+- `planTileGrid` and its `TilePlan`, `PlannedTile`, and `TileRect` types.
+- Experimental `gemm` tuning options. These are intended for benchmarking and are not covered by semver.
+- `hdrTransfer` and `activeExecutionCount` in `getRuntimeInfo()`, and the active GEMM configuration under `kernel.gemm`.
+
+### Changed
+
+- Implicit GEMM tiles are selected by output alignment and GPU limits, with optimized addressing, weight layout, register tiles, shared-memory layout, and pooling access.
+- The final RGB convolution uses an adaptive shared-memory cache.
+- Adaptive tile sizing uses the smoothed P75 tile GPU time, excludes the cold first tile, ignores cancelled and single-tile work, and buckets input shapes to at most two sizes.
+- `animation-frame` scheduling falls back to a 100 ms timer so execution still completes in hidden tabs.
+
+### Fixed
+
+- Tiled execution always settles with `done` or `error`, or stops silently after abort, including when `requestAnimationFrame` never fires or the device is lost.
+- Completed executions release their device-loss listeners.
+- Edge tiles whose size is not a multiple of 16 replicate edge pixels into the padded model input.
+- The npm package no longer includes benchmark results, tests, and scripts.
+
+### Migration notes
+
+- Replace `{ device, adapterInfo }` with `{ device }`.
+- Replace `new UNet(tensors, { device, adapterInfo }, options)` with `new UNet(tensors, device, options)`.
+- Interactive renderers that share the GPU with OIDN should pass `scheduling: 'animation-frame'`.
+- Pass an `error` callback to handle failures. Without one, failures are logged to the console.
+
 ## [0.4.0] - 2026-08-20
 
 oidn-web 0.4.0 replaces the TensorFlow.js inference stack with a purpose-built, model-driven WebGPU runtime. Existing `initUNetFromURL` and `initUNetFromBuffer` integrations remain supported while gaining native FP16, adaptive scheduling, runtime diagnostics, and stronger model validation.
@@ -46,4 +90,5 @@ oidn-web 0.4.0 replaces the TensorFlow.js inference stack with a purpose-built,
 - When supplying an existing `GPUDevice`, request `shader-f16` before creating the device if FP16 inference is desired.
 - Set `dynamicTile: false` to restore fixed-size tiling.
 
+[0.5.0]: https://github.com/pissang/oidn-web/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/pissang/oidn-web/compare/v0.3.5...v0.4.0

package/README.md

@@ -15,9 +15,9 @@ The OIDN U-Net runs directly on WebGPU with model-driven WGSL compute
 pipelines. Convolution activations use a blocked
 four-channel layout, decoder `upsample + concat + conv` patterns are fused,
 and all network dispatches for a tile are submitted in one command buffer.
-FP32 convolutions use channel-specialized implicit-GEMM tiles; FP16 uses
-channel-specialized vector FMA and separate max-pool passes, selected from the
-same model descriptor.
+FP16 and FP32 convolutions use channel-specialized implicit-GEMM tiles by
+default, with a direct convolution for the final output layer and separate
+max-pool passes.
 
 TZA half-float weights stay half-float when the device enables `shader-f16`.
 FP16 products are accumulated in short half-precision groups and periodically
@@ -50,8 +50,8 @@ initUNetFromURL('./weights/rt_ldr.tza').then((unet) => {
     .getContext('2d')
     .getImageData(0, 0, width, height);
 
-  // Tile execute the denoising.
-  // If the resolution is high. It will split the input into tiles and execute one tile per frame.
+  // Tile execute the denoising. High resolutions use balanced rectangular
+  // tiles with overlap only at boundaries shared by another tile.
   const abortDenoising = unet.tileExecute({
     // The color input for LDR image is 4 channels.
     // In the format of Uint8ClampedArray or Uint8Array.
@@ -89,6 +89,22 @@ initUNetFromURL('./weights/rt_hdr.tza', undefined, {
 });
 ```
 
+HDR transfer defaults to the PU curve used by the regular RT models. Models
+trained for the RTLightmap filter use the logarithmic curve from upstream OIDN;
+select it explicitly when loading such weights:
+
+```ts
+const lightmap = await initUNetFromURL('./weights/rtlightmap_hdr.tza', undefined, {
+  hdr: true,
+  hdrTransfer: 'log'
+});
+```
+
+The `log` transfer maps `y` to `log(1 + y) / log(65505)` and reverses this
+before writing HDR output. For GPU-buffer inputs, callers may continue to
+pre-scale the complete image once and leave the runtime's `inputScale` at its
+existing default of `1`.
+
 ### Use auxiliary images
 
 ```ts
@@ -124,9 +140,8 @@ If you already have a WebGPU path tracer. You can integrate the oidn-web into yo
 initUNetFromURL(
   './weights/rt_hdr_alb_nrm.tza',
   {
-    // Share GPUDevice and GPUAdapterInfo with the native WGSL runtime.
-    device,
-    adapterInfo
+    // Share the GPUDevice with the native WGSL runtime.
+    device
   },
   {
     aux: true,
@@ -180,7 +195,7 @@ const device = await adapter.requestDevice({ requiredFeatures });
 
 const unet = await initUNetFromURL(
   modelUrl,
-  { device, adapterInfo },
+  { device },
   {
     aux: true,
     hdr: true,
@@ -230,40 +245,75 @@ npm run model:inspect -- weights/rt_hdr_alb_nrm.tza
 ```
 
 ```ts
-const unet = await initUNetFromURL(newModelUrl, backend, {
+const unet = await initUNetFromURL(newModelUrl, undefined, {
   aux: true,
   hdr: true,
   modelSpec: newOidnModelSpec
 });
 ```
 
-### GPU backpressure and dynamic tiles
+### GPU backpressure, scheduling, and dynamic tiles
 
 `tileExecute` waits for the submitted GPU work of a tile before scheduling the
 next tile. This keeps at most one OIDN tile in flight, which makes cancellation
 responsive instead of leaving queued denoising work ahead of interactive
 rendering.
 
-Tile sizing is adaptive by default. `maxTileSize` is a hard upper bound; the
-completed GPU time of a tiled execution adjusts the tile size used by the next
-execution. Single-tile images do not affect the estimate. The default range
-starts at 384 pixels, does not go below 256, and targets about 16 ms of GPU work
-per tile.
+Between tiles, JavaScript yields according to `scheduling`:
+
+- `'event-loop'` (default) continues on the next macrotask. Execution finishes
+  as soon as the GPU allows and also completes in background tabs.
+- `'animation-frame'` waits for the next display frame, so at most one tile
+  runs per frame. Use it when OIDN shares the GPU with an interactive renderer
+  and frame rate matters more than denoise latency. If no frame arrives within
+  100 ms (for example in a hidden tab), the next tile runs anyway.
+
+Asynchronous failures, including WebGPU device loss and exceptions thrown by
+`progress` or `done`, are reported to the optional `error` callback. Unless the
+returned abort function is called first, every execution ends by calling `done`
+or `error`; without an `error` callback, failures are logged to the console.
+
+Tile sizing is adaptive by default. `maxTileSize` is a hard upper bound. Each
+execution partitions the image into balanced rectangular output regions and
+adds model context only on edges shared with another tile. Input shapes are
+bucketed to at most two sizes so the native execution cache remains stable.
+The smoothed P75 GPU time of completed tiled executions adjusts the maximum tile
+size used by the next execution. The potentially cold first tile is excluded,
+cancelled and single-tile work is ignored, and a layout is held for at least two
+complete executions. The default range starts at 432 pixels, does not go below
+256, changes in 16-pixel steps, and targets about 16 ms of GPU work per tile.
 
 ```ts
-initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', backend, {
+const unet = await initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', undefined, {
   aux: true,
   hdr: true,
   maxTileSize: 512,
   dynamicTile: {
     minTileSize: 256,
-    initialTileSize: 384,
+    initialTileSize: 432,
     targetTileTimeMs: 16
   }
 });
 
+// An interactive renderer can pace tiles to display frames. A smaller halo
+// lowers per-tile cost; the default overlap is half of the model receptive
+// field rounded up to 16 pixels.
+const abortDenoising = unet.tileExecute({
+  color,
+  albedo,
+  normal,
+  tileOverlap: 80,
+  scheduling: 'animation-frame',
+  done(denoised) {
+    // ...
+  },
+  error(reason) {
+    console.error(reason);
+  }
+});
+
 // Restore fixed-size behavior when deterministic tiling is preferred.
-initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', backend, {
+initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', undefined, {
   aux: true,
   hdr: true,
   maxTileSize: 512,
@@ -271,6 +321,20 @@ initUNetFromURL('./weights/rt_hdr_alb_nrm.tza', backend, {
 });
 ```
 
+### Warm up and whole-image execution
+
+The native runtime creates per-shape GPU buffers and pipelines the first time
+it sees a tile input shape. Call `prepareForImage` with the expected image size while a loading
+state is still visible, so the first `tileExecute` does not stall:
+
+```ts
+await unet.prepareForImage(width, height);
+```
+
+Pass `wholeImage: true` to `tileExecute` (and to `prepareForImage`) to run the
+complete image as one tile and skip tiling overhead. It ignores `maxTileSize`,
+so the image must fit the device's buffer and dispatch limits.
+
 ### Benchmark the native runtime against TFJS
 
 The browser benchmark automatically finds the nearest ancestor whose package