@mauriciobenjamin700/ort-vision-sdk-web 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +68 -2
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/dist/preprocess/pipeline.d.ts +114 -0
- package/dist/preprocess/pipeline.d.ts.map +1 -1
- package/dist/preprocess/pipeline.js +161 -0
- package/dist/preprocess/pipeline.js.map +1 -1
- package/dist/tasks/base.d.ts.map +1 -1
- package/dist/tasks/base.js +22 -1
- package/dist/tasks/base.js.map +1 -1
- package/dist/tasks/classifier.d.ts +22 -0
- package/dist/tasks/classifier.d.ts.map +1 -1
- package/dist/tasks/classifier.js +39 -5
- package/dist/tasks/classifier.js.map +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,64 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.7.0] - 2026-08-08
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **`Classifier.warmup(runs = 1)`.** The classifier was the one task left
|
|
15
|
+
without it — the documentation already claimed all four had it. The cost it
|
|
16
|
+
moves is the same one the other three move (WebGPU compiling shaders, the WASM
|
|
17
|
+
backend faulting in its arenas), and it lands worse here: an app running a
|
|
18
|
+
detector and then a classifier could warm the first and not the second, so an
|
|
19
|
+
unavoidable first-run cost sat right before the answer appears on screen.
|
|
20
|
+
|
|
21
|
+
- **`ResizePipeline`, the classification counterpart of `LetterboxPipeline`**
|
|
22
|
+
(plus `resizeToTensorData`, its one-shot form, and `writePlanarFloat32`, the
|
|
23
|
+
loop both pipelines share). A classifier stretches to the model's input rather
|
|
24
|
+
than letterboxing into it — no padding, no scale to invert afterwards — so it
|
|
25
|
+
could not reuse the letterbox path as it stands.
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- **`Classifier` preprocessing allocates nothing per frame and walks the image
|
|
30
|
+
once.** It was still on the composable route (`resize` → `normalize` →
|
|
31
|
+
`toCHW`), which allocates an `RGBImage` and two `Float32Array`s and scans each
|
|
32
|
+
end to end: roughly 1.4 MB of fresh garbage per 224×224 `predict()`, produced
|
|
33
|
+
exactly when a phone near its ONNX memory ceiling can least afford it. It now
|
|
34
|
+
goes through `ResizePipeline`, which resizes with one `drawImage` and writes
|
|
35
|
+
normalized planar float32 into a buffer held across calls.
|
|
36
|
+
|
|
37
|
+
**The output is bit-identical** to the composable path — asserted value by
|
|
38
|
+
value against `normalize` → `toCHW`, including the `mean=[0,0,0]`,
|
|
39
|
+
`std=[1,1,1]` configuration an Ultralytics `-cls` export uses, where the old
|
|
40
|
+
route walked the whole buffer and allocated another one to change no value at
|
|
41
|
+
all.
|
|
42
|
+
|
|
43
|
+
### Fixed
|
|
44
|
+
|
|
45
|
+
- **The web reference stopped listing `decodeYoloV8`, `decodeYoloV8Anchors` and
|
|
46
|
+
`decodeYoloV8Seg`.** They were removed in 0.6.0; the page kept advertising
|
|
47
|
+
them as current API. It documents both preprocessing pipelines now.
|
|
48
|
+
|
|
49
|
+
## [0.6.1] - 2026-08-08
|
|
50
|
+
|
|
51
|
+
### Fixed
|
|
52
|
+
|
|
53
|
+
- **The threshold quoted in a `NoDetectionsError` now reads the same in both
|
|
54
|
+
SDKs.** This side rendered a whole threshold as `confThreshold=1` where the
|
|
55
|
+
Python SDK rendered `conf_threshold=1.0`, and wrote `0.0000001` where Python
|
|
56
|
+
switched to `1e-07`. A fused pipeline is built once and runs under both
|
|
57
|
+
runtimes from the same file, so the two SDKs were describing one run with two
|
|
58
|
+
different numbers. Both sides now render six decimals with the trailing zeros
|
|
59
|
+
trimmed, and the same table in both test suites fails if either side drifts.
|
|
60
|
+
|
|
61
|
+
- **The changelog's link table is no longer stuck at 0.2.0.** `[Unreleased]`
|
|
62
|
+
compared against `web-v0.2.0`, so following it showed four releases of shipped
|
|
63
|
+
history as unreleased work, and `[0.3.0]` through `[0.6.0]` had no link
|
|
64
|
+
definition at all — they rendered as literal `[0.6.0]` text. The table is
|
|
65
|
+
rebuilt through 0.6.1. `[0.1.0]` loses its definition on purpose: no
|
|
66
|
+
`web-v0.1.0` tag was ever pushed, so that link was a 404.
|
|
67
|
+
|
|
10
68
|
## [0.6.0] - 2026-08-07
|
|
11
69
|
|
|
12
70
|
### Added
|
|
@@ -444,6 +502,14 @@ console.log(r.boxes.xyxy, r.boxes.cls, r.boxes.conf, r.names);
|
|
|
444
502
|
- Execution-provider resolution defaulting to `["webgpu", "wasm"]`.
|
|
445
503
|
- Public types mirroring the Python SDK: `BoundingBox`, `ClassProbability`, `ClassificationResult`, `DetectionResult`, `RGBImage`.
|
|
446
504
|
|
|
447
|
-
[Unreleased]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.
|
|
505
|
+
[Unreleased]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.7.0...HEAD
|
|
506
|
+
[0.7.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.6.1...web-v0.7.0
|
|
507
|
+
[0.6.1]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.6.0...web-v0.6.1
|
|
508
|
+
[0.6.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.5.1...web-v0.6.0
|
|
509
|
+
[0.5.1]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.5.0...web-v0.5.1
|
|
510
|
+
[0.5.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.4.0...web-v0.5.0
|
|
511
|
+
[0.4.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.3.0...web-v0.4.0
|
|
512
|
+
[0.3.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.2.2...web-v0.3.0
|
|
513
|
+
[0.2.2]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.2.1...web-v0.2.2
|
|
514
|
+
[0.2.1]: https://github.com/mauriciobenjamin700/ort-vision-sdk/compare/web-v0.2.0...web-v0.2.1
|
|
448
515
|
[0.2.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/releases/tag/web-v0.2.0
|
|
449
|
-
[0.1.0]: https://github.com/mauriciobenjamin700/ort-vision-sdk/releases/tag/web-v0.1.0
|
package/dist/index.d.ts
CHANGED
|
@@ -14,7 +14,7 @@ export { type Speed, SpeedTimer } from "./core/timing.js";
|
|
|
14
14
|
export { type ImageInput, loadImage } from "./io/image.js";
|
|
15
15
|
export { FUSION_KIND_DETECT_CLASSIFY, INPUT_IMAGE, INPUT_PAD, INPUT_SCALE, INPUT_SOURCE, METADATA_PREFIX, OUTPUT_BOXES, OUTPUT_CLASSES, OUTPUT_NUM_DETECTIONS, OUTPUT_PROBS, OUTPUT_SCORES, type CropSource, type FusionSpec, readFusionSpec, } from "./fusion.js";
|
|
16
16
|
export { type LetterboxResult, fromCv2, letterbox, normalize, resize, toCHW, toCv2, toFloat32, toFloat32Tensor, toTensor, } from "./preprocess/image.js";
|
|
17
|
-
export { type FusedLetterboxResult, LetterboxPipeline, letterboxToTensorData, zeroTensorData, } from "./preprocess/pipeline.js";
|
|
17
|
+
export { type FusedLetterboxResult, type FusedResizeResult, LetterboxPipeline, ResizePipeline, letterboxToTensorData, resizeToTensorData, writePlanarFloat32, zeroTensorData, } from "./preprocess/pipeline.js";
|
|
18
18
|
export { type TopKResult, softmax, topK, } from "./postprocess/classification.js";
|
|
19
19
|
export { type DecodeYoloAnchorsOptions, type DecodeYoloOptions, type DecodedAnchors, type DecodedDetection, batchedNms, decodeYolo, decodeYoloAnchors, nms, } from "./postprocess/detection.js";
|
|
20
20
|
export { type DecodeYoloSegOptions, type DecodedSegmentation, decodeYoloSeg, } from "./postprocess/segmentation.js";
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EACL,WAAW,EACX,IAAI,EACJ,QAAQ,EACR,KAAK,gBAAgB,EACrB,KAAK,oBAAoB,EACzB,KAAK,eAAe,EACpB,KAAK,kBAAkB,GACxB,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,KAAK,EACL,qBAAqB,EACrB,qBAAqB,EACrB,gBAAgB,EAChB,KAAK,EACL,KAAK,EACL,mBAAmB,GACpB,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,YAAY,EACZ,KAAK,SAAS,EACd,KAAK,oBAAoB,EACzB,aAAa,EACb,aAAa,GACd,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,WAAW,EACX,cAAc,EACd,cAAc,EACd,aAAa,EACb,cAAc,EACd,iBAAiB,EACjB,cAAc,EACd,yBAAyB,GAC1B,MAAM,sBAAsB,CAAC;AAE9B,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,iBAAiB,EACtB,UAAU,GACX,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,aAAa,EAClB,KAAK,uBAAuB,EAC5B,wBAAwB,EACxB,kBAAkB,EAClB,mBAAmB,EACnB,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAC1E,OAAO,EAAE,KAAK,KAAK,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAE1D,OAAO,EAAE,KAAK,UAAU,EAAE,SAAS,EAAE,MAAM,eAAe,CAAC;AAE3D,OAAO,EACL,2BAA2B,EAC3B,WAAW,EACX,SAAS,EACT,WAAW,EACX,YAAY,EACZ,eAAe,EACf,YAAY,EACZ,cAAc,EACd,qBAAqB,EACrB,YAAY,EACZ,aAAa,EACb,KAAK,UAAU,EACf,KAAK,UAAU,EACf,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,KAAK,eAAe,EACpB,OAAO,EACP,SAAS,EACT,SAAS,EACT,MAAM,EACN,KAAK,EACL,KAAK,EACL,SAAS,EACT,eAAe,EACf,QAAQ,GACT,MAAM,uBAAuB,CAAC;AAE/B,OAAO,EACL,KAAK,oBAAoB,EACzB,iBAAiB,EACjB,qBAAqB,EACrB,cAAc,GACf,MAAM,0BAA0B,CAAC;AAElC,OAAO,EACL,KAAK,UAAU,EACf,OAAO,EACP,IAAI,GACL,MAAM,iCAAiC,CAAC;AAEzC,OAAO,EACL,KAAK,wBAAwB,EAC7B,KAAK,iBAAiB,EACtB,KAAK,cAAc,EACnB,KAAK,gBAAgB,EACrB,UAAU,EACV,UAAU,EACV,iBAAiB,EACjB,GAAG,GACJ,MAAM,4BAA4B,CAAC;AAEpC,OAAO,EACL,KAAK,oBAAoB,EACzB,KAAK,mBAAmB,EACxB,aAAa,GACd,MAAM,+BAA+B,CAAC;AAEvC,OAAO,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAChE,OAAO,EACL,KAAK,iBAAiB,EACtB,KAAK,wBAAwB,EAC7B,UAAU,GACX,MAAM,uBAAuB,CAAC;AAC/B,OAAO,EACL,KAAK,qBAAqB,EAC1B,KAAK,4BAA4B,EACjC,cAAc,GACf,MAAM,2BAA2B,CAAC;AACnC,OAAO,EACL,KAAK,YAAY,EACjB,KAAK,eAAe,EACpB,KAAK,sBAAsB,EAC3B,QAAQ,GACT,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EACL,KAAK,aAAa,EAClB,KAAK,gBAAgB,EACrB,KAAK,uBAAuB,EAC5B,SAAS,GACV,MAAM,sBAAsB,CAAC;AAE9B,eAAO,MAAM,OAAO,EAAE,MAAgB,CAAC"}
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EACL,WAAW,EACX,IAAI,EACJ,QAAQ,EACR,KAAK,gBAAgB,EACrB,KAAK,oBAAoB,EACzB,KAAK,eAAe,EACpB,KAAK,kBAAkB,GACxB,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,KAAK,EACL,qBAAqB,EACrB,qBAAqB,EACrB,gBAAgB,EAChB,KAAK,EACL,KAAK,EACL,mBAAmB,GACpB,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,YAAY,EACZ,KAAK,SAAS,EACd,KAAK,oBAAoB,EACzB,aAAa,EACb,aAAa,GACd,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,WAAW,EACX,cAAc,EACd,cAAc,EACd,aAAa,EACb,cAAc,EACd,iBAAiB,EACjB,cAAc,EACd,yBAAyB,GAC1B,MAAM,sBAAsB,CAAC;AAE9B,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,iBAAiB,EACtB,UAAU,GACX,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EACL,KAAK,WAAW,EAChB,KAAK,aAAa,EAClB,KAAK,uBAAuB,EAC5B,wBAAwB,EACxB,kBAAkB,EAClB,mBAAmB,EACnB,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAC1E,OAAO,EAAE,KAAK,KAAK,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAE1D,OAAO,EAAE,KAAK,UAAU,EAAE,SAAS,EAAE,MAAM,eAAe,CAAC;AAE3D,OAAO,EACL,2BAA2B,EAC3B,WAAW,EACX,SAAS,EACT,WAAW,EACX,YAAY,EACZ,eAAe,EACf,YAAY,EACZ,cAAc,EACd,qBAAqB,EACrB,YAAY,EACZ,aAAa,EACb,KAAK,UAAU,EACf,KAAK,UAAU,EACf,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,KAAK,eAAe,EACpB,OAAO,EACP,SAAS,EACT,SAAS,EACT,MAAM,EACN,KAAK,EACL,KAAK,EACL,SAAS,EACT,eAAe,EACf,QAAQ,GACT,MAAM,uBAAuB,CAAC;AAE/B,OAAO,EACL,KAAK,oBAAoB,EACzB,KAAK,iBAAiB,EACtB,iBAAiB,EACjB,cAAc,EACd,qBAAqB,EACrB,kBAAkB,EAClB,kBAAkB,EAClB,cAAc,GACf,MAAM,0BAA0B,CAAC;AAElC,OAAO,EACL,KAAK,UAAU,EACf,OAAO,EACP,IAAI,GACL,MAAM,iCAAiC,CAAC;AAEzC,OAAO,EACL,KAAK,wBAAwB,EAC7B,KAAK,iBAAiB,EACtB,KAAK,cAAc,EACnB,KAAK,gBAAgB,EACrB,UAAU,EACV,UAAU,EACV,iBAAiB,EACjB,GAAG,GACJ,MAAM,4BAA4B,CAAC;AAEpC,OAAO,EACL,KAAK,oBAAoB,EACzB,KAAK,mBAAmB,EACxB,aAAa,GACd,MAAM,+BAA+B,CAAC;AAEvC,OAAO,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAChE,OAAO,EACL,KAAK,iBAAiB,EACtB,KAAK,wBAAwB,EAC7B,UAAU,GACX,MAAM,uBAAuB,CAAC;AAC/B,OAAO,EACL,KAAK,qBAAqB,EAC1B,KAAK,4BAA4B,EACjC,cAAc,GACf,MAAM,2BAA2B,CAAC;AACnC,OAAO,EACL,KAAK,YAAY,EACjB,KAAK,eAAe,EACpB,KAAK,sBAAsB,EAC3B,QAAQ,GACT,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EACL,KAAK,aAAa,EAClB,KAAK,gBAAgB,EACrB,KAAK,uBAAuB,EAC5B,SAAS,GACV,MAAM,sBAAsB,CAAC;AAE9B,eAAO,MAAM,OAAO,EAAE,MAAgB,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -14,7 +14,7 @@ export { SpeedTimer } from "./core/timing.js";
|
|
|
14
14
|
export { loadImage } from "./io/image.js";
|
|
15
15
|
export { FUSION_KIND_DETECT_CLASSIFY, INPUT_IMAGE, INPUT_PAD, INPUT_SCALE, INPUT_SOURCE, METADATA_PREFIX, OUTPUT_BOXES, OUTPUT_CLASSES, OUTPUT_NUM_DETECTIONS, OUTPUT_PROBS, OUTPUT_SCORES, readFusionSpec, } from "./fusion.js";
|
|
16
16
|
export { fromCv2, letterbox, normalize, resize, toCHW, toCv2, toFloat32, toFloat32Tensor, toTensor, } from "./preprocess/image.js";
|
|
17
|
-
export { LetterboxPipeline, letterboxToTensorData, zeroTensorData, } from "./preprocess/pipeline.js";
|
|
17
|
+
export { LetterboxPipeline, ResizePipeline, letterboxToTensorData, resizeToTensorData, writePlanarFloat32, zeroTensorData, } from "./preprocess/pipeline.js";
|
|
18
18
|
export { softmax, topK, } from "./postprocess/classification.js";
|
|
19
19
|
export { batchedNms, decodeYolo, decodeYoloAnchors, nms, } from "./postprocess/detection.js";
|
|
20
20
|
export { decodeYoloSeg, } from "./postprocess/segmentation.js";
|
|
@@ -23,5 +23,5 @@ export { Classifier, } from "./tasks/classifier.js";
|
|
|
23
23
|
export { DetectClassify, } from "./tasks/detectClassify.js";
|
|
24
24
|
export { Detector, } from "./tasks/detector.js";
|
|
25
25
|
export { Segmenter, } from "./tasks/segmenter.js";
|
|
26
|
-
export const VERSION = "0.
|
|
26
|
+
export const VERSION = "0.7.0";
|
|
27
27
|
//# sourceMappingURL=index.js.map
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EACL,WAAW,EACX,IAAI,EACJ,QAAQ,GAKT,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,KAAK,EACL,qBAAqB,EACrB,qBAAqB,EACrB,gBAAgB,EAChB,KAAK,EACL,KAAK,EACL,mBAAmB,GACpB,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,YAAY,EAGZ,aAAa,EACb,aAAa,GACd,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,WAAW,EACX,cAAc,EACd,cAAc,EACd,aAAa,EACb,cAAc,EACd,iBAAiB,EACjB,cAAc,EACd,yBAAyB,GAC1B,MAAM,sBAAsB,CAAC;AAE9B,OAAO,EAGL,UAAU,GACX,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAIL,wBAAwB,EACxB,kBAAkB,EAClB,mBAAmB,EACnB,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAC1E,OAAO,EAAc,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAE1D,OAAO,EAAmB,SAAS,EAAE,MAAM,eAAe,CAAC;AAE3D,OAAO,EACL,2BAA2B,EAC3B,WAAW,EACX,SAAS,EACT,WAAW,EACX,YAAY,EACZ,eAAe,EACf,YAAY,EACZ,cAAc,EACd,qBAAqB,EACrB,YAAY,EACZ,aAAa,EAGb,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,OAAO,EAEL,OAAO,EACP,SAAS,EACT,SAAS,EACT,MAAM,EACN,KAAK,EACL,KAAK,EACL,SAAS,EACT,eAAe,EACf,QAAQ,GACT,MAAM,uBAAuB,CAAC;AAE/B,OAAO,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,EACL,WAAW,EACX,IAAI,EACJ,QAAQ,GAKT,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,KAAK,EACL,qBAAqB,EACrB,qBAAqB,EACrB,gBAAgB,EAChB,KAAK,EACL,KAAK,EACL,mBAAmB,GACpB,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,YAAY,EAGZ,aAAa,EACb,aAAa,GACd,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,WAAW,EACX,cAAc,EACd,cAAc,EACd,aAAa,EACb,cAAc,EACd,iBAAiB,EACjB,cAAc,EACd,yBAAyB,GAC1B,MAAM,sBAAsB,CAAC;AAE9B,OAAO,EAGL,UAAU,GACX,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAIL,wBAAwB,EACxB,kBAAkB,EAClB,mBAAmB,EACnB,gBAAgB,EAChB,gBAAgB,GACjB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAC1E,OAAO,EAAc,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAE1D,OAAO,EAAmB,SAAS,EAAE,MAAM,eAAe,CAAC;AAE3D,OAAO,EACL,2BAA2B,EAC3B,WAAW,EACX,SAAS,EACT,WAAW,EACX,YAAY,EACZ,eAAe,EACf,YAAY,EACZ,cAAc,EACd,qBAAqB,EACrB,YAAY,EACZ,aAAa,EAGb,cAAc,GACf,MAAM,aAAa,CAAC;AAErB,OAAO,EAEL,OAAO,EACP,SAAS,EACT,SAAS,EACT,MAAM,EACN,KAAK,EACL,KAAK,EACL,SAAS,EACT,eAAe,EACf,QAAQ,GACT,MAAM,uBAAuB,CAAC;AAE/B,OAAO,EAGL,iBAAiB,EACjB,cAAc,EACd,qBAAqB,EACrB,kBAAkB,EAClB,kBAAkB,EAClB,cAAc,GACf,MAAM,0BAA0B,CAAC;AAElC,OAAO,EAEL,OAAO,EACP,IAAI,GACL,MAAM,iCAAiC,CAAC;AAEzC,OAAO,EAKL,UAAU,EACV,UAAU,EACV,iBAAiB,EACjB,GAAG,GACJ,MAAM,4BAA4B,CAAC;AAEpC,OAAO,EAGL,aAAa,GACd,MAAM,+BAA+B,CAAC;AAEvC,OAAO,EAAE,UAAU,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAChE,OAAO,EAGL,UAAU,GACX,MAAM,uBAAuB,CAAC;AAC/B,OAAO,EAGL,cAAc,GACf,MAAM,2BAA2B,CAAC;AACnC,OAAO,EAIL,QAAQ,GACT,MAAM,qBAAqB,CAAC;AAC7B,OAAO,EAIL,SAAS,GACV,MAAM,sBAAsB,CAAC;AAE9B,MAAM,CAAC,MAAM,OAAO,GAAW,OAAO,CAAC"}
|
|
@@ -95,6 +95,120 @@ export declare class LetterboxPipeline {
|
|
|
95
95
|
*/
|
|
96
96
|
private _ensureSource;
|
|
97
97
|
}
|
|
98
|
+
/**
|
|
99
|
+
* Write an RGBA buffer as normalized planar float32.
|
|
100
|
+
*
|
|
101
|
+
* Shared by {@link ResizePipeline} and exported for a custom pipeline that
|
|
102
|
+
* already holds pixels and wants the SDK's exact arithmetic.
|
|
103
|
+
*
|
|
104
|
+
* The expression is `(value / 255 - mean) / std`, evaluated in that order on
|
|
105
|
+
* purpose: folding it into a single multiply-add (`value * k + b`) computes the
|
|
106
|
+
* same quantity but rounds differently, and the tasks' output is asserted to be
|
|
107
|
+
* bit-identical to the composable `normalize` → `toCHW` path.
|
|
108
|
+
*
|
|
109
|
+
* @param rgba Source pixels, 4 bytes per pixel, `width * height` long.
|
|
110
|
+
* @param width Image width in pixels.
|
|
111
|
+
* @param height Image height in pixels.
|
|
112
|
+
* @param mean Per-channel RGB mean, already in `[0, 1]`.
|
|
113
|
+
* @param std Per-channel RGB standard deviation.
|
|
114
|
+
* @param out Destination buffer, `3 * width * height` long.
|
|
115
|
+
* @param stride Bytes per source pixel: 4 for canvas RGBA, 3 for packed RGB.
|
|
116
|
+
*/
|
|
117
|
+
export declare function writePlanarFloat32(rgba: Uint8ClampedArray | Uint8Array, width: number, height: number, mean: readonly [number, number, number], std: readonly [number, number, number], out: Float32Array, stride?: number): void;
|
|
118
|
+
/** Planar tensor data produced by {@link ResizePipeline}. */
|
|
119
|
+
export interface FusedResizeResult {
|
|
120
|
+
/** CHW float32, normalized, length `3 * targetHeight * targetWidth`. */
|
|
121
|
+
readonly data: Float32Array;
|
|
122
|
+
/**
|
|
123
|
+
* Whether {@link data} is the pipeline's reusable buffer.
|
|
124
|
+
*
|
|
125
|
+
* `true` means the next {@link ResizePipeline.run} overwrites it, so a caller
|
|
126
|
+
* keeping the values past its own inference has to copy them.
|
|
127
|
+
*/
|
|
128
|
+
readonly reused: boolean;
|
|
129
|
+
}
|
|
130
|
+
/**
|
|
131
|
+
* Reusable stretch-resize → normalized tensor pipeline for one target size.
|
|
132
|
+
*
|
|
133
|
+
* The classification counterpart of {@link LetterboxPipeline}. A classifier
|
|
134
|
+
* stretches to the model's square input instead of letterboxing into it — no
|
|
135
|
+
* padding, no scale to invert later, because nothing is mapped back onto the
|
|
136
|
+
* source image afterwards. That difference is why it cannot simply reuse the
|
|
137
|
+
* letterbox path.
|
|
138
|
+
*
|
|
139
|
+
* What it does share is the technique. The composable route
|
|
140
|
+
* (`resize` → `normalize` → `toCHW`) allocates an `RGBImage` and two
|
|
141
|
+
* `Float32Array`s and walks each end to end on every call: about 1.4 MB of
|
|
142
|
+
* fresh garbage per 224×224 `predict()`, produced at the exact moment a phone
|
|
143
|
+
* near its memory ceiling can least afford it. Here one `drawImage` resizes,
|
|
144
|
+
* and one loop reads the resulting RGBA and writes normalized planar float32
|
|
145
|
+
* into a buffer held across calls.
|
|
146
|
+
*
|
|
147
|
+
* Create one per task, not per frame.
|
|
148
|
+
*/
|
|
149
|
+
export declare class ResizePipeline {
|
|
150
|
+
private readonly _targetWidth;
|
|
151
|
+
private readonly _targetHeight;
|
|
152
|
+
private readonly _mean;
|
|
153
|
+
private readonly _std;
|
|
154
|
+
private readonly _buffer;
|
|
155
|
+
private _target;
|
|
156
|
+
private _targetContext;
|
|
157
|
+
private _source;
|
|
158
|
+
private _sourceContext;
|
|
159
|
+
private _bufferInUse;
|
|
160
|
+
/**
|
|
161
|
+
* @param targetWidth Model input width in pixels.
|
|
162
|
+
* @param targetHeight Model input height in pixels.
|
|
163
|
+
* @param mean Per-channel RGB mean in `[0, 1]`. Defaults to no shift.
|
|
164
|
+
* @param std Per-channel RGB standard deviation. Defaults to no scaling.
|
|
165
|
+
*/
|
|
166
|
+
constructor(targetWidth: number, targetHeight: number, mean?: readonly [number, number, number], std?: readonly [number, number, number]);
|
|
167
|
+
/** The `[width, height]` this pipeline resizes into. */
|
|
168
|
+
get targetSize(): readonly [number, number];
|
|
169
|
+
/**
|
|
170
|
+
* Resize an image to the target size and write it as normalized planar float32.
|
|
171
|
+
*
|
|
172
|
+
* An image that already arrives at the target size skips the canvas entirely
|
|
173
|
+
* and is read straight out of its packed RGB — which is both faster and what
|
|
174
|
+
* keeps the result identical to `resize()`, whose own fast path returns the
|
|
175
|
+
* input untouched.
|
|
176
|
+
*
|
|
177
|
+
* @param image Source image in the SDK's canonical HWC RGB layout.
|
|
178
|
+
*/
|
|
179
|
+
run(image: RGBImage): FusedResizeResult;
|
|
180
|
+
/**
|
|
181
|
+
* Mark the reusable buffer free again.
|
|
182
|
+
*
|
|
183
|
+
* Call it once the tensor built from a {@link run} result has been handed to
|
|
184
|
+
* ONNX Runtime and the run has resolved — after that the values are inside
|
|
185
|
+
* the WASM heap and the buffer can be overwritten.
|
|
186
|
+
*/
|
|
187
|
+
release(): void;
|
|
188
|
+
/**
|
|
189
|
+
* Build the target canvas on first use.
|
|
190
|
+
*
|
|
191
|
+
* Lazily, so a pipeline constructed where no canvas implementation exists
|
|
192
|
+
* (a Node test, a worker without OffscreenCanvas) only fails if it is asked
|
|
193
|
+
* to resize something.
|
|
194
|
+
*/
|
|
195
|
+
private _ensureTarget;
|
|
196
|
+
/** Grow the scratch source canvas to fit an image, reusing it when possible. */
|
|
197
|
+
private _ensureSource;
|
|
198
|
+
}
|
|
199
|
+
/**
|
|
200
|
+
* Resize an image into normalized planar float32 without keeping any state.
|
|
201
|
+
*
|
|
202
|
+
* The allocation-free path is {@link ResizePipeline}; this is the one-shot
|
|
203
|
+
* form, for a caller who wants the fused behaviour without owning a pipeline.
|
|
204
|
+
*
|
|
205
|
+
* @param image Source image in the SDK's canonical HWC RGB layout.
|
|
206
|
+
* @param targetWidth Model input width in pixels.
|
|
207
|
+
* @param targetHeight Model input height in pixels.
|
|
208
|
+
* @param mean Per-channel RGB mean in `[0, 1]`. Defaults to no shift.
|
|
209
|
+
* @param std Per-channel RGB standard deviation. Defaults to no scaling.
|
|
210
|
+
*/
|
|
211
|
+
export declare function resizeToTensorData(image: RGBImage, targetWidth: number, targetHeight: number, mean?: readonly [number, number, number], std?: readonly [number, number, number]): FusedResizeResult;
|
|
98
212
|
/**
|
|
99
213
|
* Build a zero-filled CHW tensor payload for a warm-up run.
|
|
100
214
|
*
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../../src/preprocess/pipeline.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAGH,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAIvC,wEAAwE;AACxE,MAAM,WAAW,oBAAoB;IACnC,wEAAwE;IACxE,QAAQ,CAAC,IAAI,EAAE,YAAY,CAAC;IAC5B,kEAAkE;IAClE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,oCAAoC;IACpC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,kCAAkC;IAClC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;OAKG;IACH,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;CAC1B;AAED;;;;;;GAMG;AACH,qBAAa,iBAAiB;IAC5B,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAoC;IAC1D,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAW;IACnC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAY;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAe;IACvC,OAAO,CAAC,OAAO,CAAyB;IACxC,OAAO,CAAC,cAAc,CAA0B;IAChD,OAAO,CAAC,YAAY,CAAS;IAE7B;;;;OAIG;gBAED,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAmB;IAe3D,4DAA4D;IAC5D,IAAI,UAAU,IAAI,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAE1C;IAED;;;;;;;;;;OAUG;IACH,GAAG,CAAC,KAAK,EAAE,QAAQ,GAAG,oBAAoB;IA4C1C;;;;;;OAMG;IACH,OAAO,IAAI,IAAI;IAIf;;;;;;;;OAQG;IACH,OAAO,CAAC,aAAa;CAOtB;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,YAAY,CAE1E;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,qBAAqB,CACnC,KAAK,EAAE,QAAQ,EACf,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAmB,GACxD,oBAAoB,CAEtB"}
|
|
1
|
+
{"version":3,"file":"pipeline.d.ts","sourceRoot":"","sources":["../../src/preprocess/pipeline.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAGH,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAIvC,wEAAwE;AACxE,MAAM,WAAW,oBAAoB;IACnC,wEAAwE;IACxE,QAAQ,CAAC,IAAI,EAAE,YAAY,CAAC;IAC5B,kEAAkE;IAClE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,oCAAoC;IACpC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,kCAAkC;IAClC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;OAKG;IACH,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;CAC1B;AAED;;;;;;GAMG;AACH,qBAAa,iBAAiB;IAC5B,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAoC;IAC1D,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAW;IACnC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAY;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAe;IACvC,OAAO,CAAC,OAAO,CAAyB;IACxC,OAAO,CAAC,cAAc,CAA0B;IAChD,OAAO,CAAC,YAAY,CAAS;IAE7B;;;;OAIG;gBAED,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAmB;IAe3D,4DAA4D;IAC5D,IAAI,UAAU,IAAI,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAE1C;IAED;;;;;;;;;;OAUG;IACH,GAAG,CAAC,KAAK,EAAE,QAAQ,GAAG,oBAAoB;IA4C1C;;;;;;OAMG;IACH,OAAO,IAAI,IAAI;IAIf;;;;;;;;OAQG;IACH,OAAO,CAAC,aAAa;CAOtB;AAED;;;;;;;;;;;;;;;;;;GAkBG;AACH,wBAAgB,kBAAkB,CAChC,IAAI,EAAE,iBAAiB,GAAG,UAAU,EACpC,KAAK,EAAE,MAAM,EACb,MAAM,EAAE,MAAM,EACd,IAAI,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,EACvC,GAAG,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,EACtC,GAAG,EAAE,YAAY,EACjB,MAAM,GAAE,MAAU,GACjB,IAAI,CAaN;AAED,6DAA6D;AAC7D,MAAM,WAAW,iBAAiB;IAChC,wEAAwE;IACxE,QAAQ,CAAC,IAAI,EAAE,YAAY,CAAC;IAC5B;;;;;OAKG;IACH,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;CAC1B;AAED;;;;;;;;;;;;;;;;;;GAkBG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAoC;IAC1D,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAoC;IACzD,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAe;IACvC,OAAO,CAAC,OAAO,CAAyB;IACxC,OAAO,CAAC,cAAc,CAA0B;IAChD,OAAO,CAAC,OAAO,CAAyB;IACxC,OAAO,CAAC,cAAc,CAA0B;IAChD,OAAO,CAAC,YAAY,CAAS;IAE7B;;;;;OAKG;gBAED,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAa,EACnD,GAAG,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAa;IAYpD,wDAAwD;IACxD,IAAI,UAAU,IAAI,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAE1C;IAED;;;;;;;;;OASG;IACH,GAAG,CAAC,KAAK,EAAE,QAAQ,GAAG,iBAAiB;IAuBvC;;;;;;OAMG;IACH,OAAO,IAAI,IAAI;IAIf;;;;;;OAMG;IACH,OAAO,CAAC,aAAa;IAUrB,gFAAgF;IAChF,OAAO,CAAC,aAAa;CAOtB;AAED;;;;;;;;;;;GAWG;AACH,wBAAgB,kBAAkB,CAChC,KAAK,EAAE,QAAQ,EACf,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAa,EACnD,GAAG,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAa,GACjD,iBAAiB,CAEnB;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,YAAY,CAE1E;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,qBAAqB,CACnC,KAAK,EAAE,QAAQ,EACf,WAAW,EAAE,MAAM,EACnB,YAAY,EAAE,MAAM,EACpB,IAAI,GAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAmB,GACxD,oBAAoB,CAEtB"}
|
|
@@ -127,6 +127,167 @@ export class LetterboxPipeline {
|
|
|
127
127
|
return this._sourceContext;
|
|
128
128
|
}
|
|
129
129
|
}
|
|
130
|
+
/**
|
|
131
|
+
* Write an RGBA buffer as normalized planar float32.
|
|
132
|
+
*
|
|
133
|
+
* Shared by {@link ResizePipeline} and exported for a custom pipeline that
|
|
134
|
+
* already holds pixels and wants the SDK's exact arithmetic.
|
|
135
|
+
*
|
|
136
|
+
* The expression is `(value / 255 - mean) / std`, evaluated in that order on
|
|
137
|
+
* purpose: folding it into a single multiply-add (`value * k + b`) computes the
|
|
138
|
+
* same quantity but rounds differently, and the tasks' output is asserted to be
|
|
139
|
+
* bit-identical to the composable `normalize` → `toCHW` path.
|
|
140
|
+
*
|
|
141
|
+
* @param rgba Source pixels, 4 bytes per pixel, `width * height` long.
|
|
142
|
+
* @param width Image width in pixels.
|
|
143
|
+
* @param height Image height in pixels.
|
|
144
|
+
* @param mean Per-channel RGB mean, already in `[0, 1]`.
|
|
145
|
+
* @param std Per-channel RGB standard deviation.
|
|
146
|
+
* @param out Destination buffer, `3 * width * height` long.
|
|
147
|
+
* @param stride Bytes per source pixel: 4 for canvas RGBA, 3 for packed RGB.
|
|
148
|
+
*/
|
|
149
|
+
export function writePlanarFloat32(rgba, width, height, mean, std, out, stride = 4) {
|
|
150
|
+
const plane = width * height;
|
|
151
|
+
const m0 = mean[0];
|
|
152
|
+
const m1 = mean[1];
|
|
153
|
+
const m2 = mean[2];
|
|
154
|
+
const s0 = std[0];
|
|
155
|
+
const s1 = std[1];
|
|
156
|
+
const s2 = std[2];
|
|
157
|
+
for (let pixel = 0, offset = 0; pixel < plane; pixel++, offset += stride) {
|
|
158
|
+
out[pixel] = (rgba[offset] * INV_255 - m0) / s0;
|
|
159
|
+
out[plane + pixel] = (rgba[offset + 1] * INV_255 - m1) / s1;
|
|
160
|
+
out[2 * plane + pixel] = (rgba[offset + 2] * INV_255 - m2) / s2;
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
/**
|
|
164
|
+
* Reusable stretch-resize → normalized tensor pipeline for one target size.
|
|
165
|
+
*
|
|
166
|
+
* The classification counterpart of {@link LetterboxPipeline}. A classifier
|
|
167
|
+
* stretches to the model's square input instead of letterboxing into it — no
|
|
168
|
+
* padding, no scale to invert later, because nothing is mapped back onto the
|
|
169
|
+
* source image afterwards. That difference is why it cannot simply reuse the
|
|
170
|
+
* letterbox path.
|
|
171
|
+
*
|
|
172
|
+
* What it does share is the technique. The composable route
|
|
173
|
+
* (`resize` → `normalize` → `toCHW`) allocates an `RGBImage` and two
|
|
174
|
+
* `Float32Array`s and walks each end to end on every call: about 1.4 MB of
|
|
175
|
+
* fresh garbage per 224×224 `predict()`, produced at the exact moment a phone
|
|
176
|
+
* near its memory ceiling can least afford it. Here one `drawImage` resizes,
|
|
177
|
+
* and one loop reads the resulting RGBA and writes normalized planar float32
|
|
178
|
+
* into a buffer held across calls.
|
|
179
|
+
*
|
|
180
|
+
* Create one per task, not per frame.
|
|
181
|
+
*/
|
|
182
|
+
export class ResizePipeline {
|
|
183
|
+
_targetWidth;
|
|
184
|
+
_targetHeight;
|
|
185
|
+
_mean;
|
|
186
|
+
_std;
|
|
187
|
+
_buffer;
|
|
188
|
+
_target = null;
|
|
189
|
+
_targetContext = null;
|
|
190
|
+
_source = null;
|
|
191
|
+
_sourceContext = null;
|
|
192
|
+
_bufferInUse = false;
|
|
193
|
+
/**
|
|
194
|
+
* @param targetWidth Model input width in pixels.
|
|
195
|
+
* @param targetHeight Model input height in pixels.
|
|
196
|
+
* @param mean Per-channel RGB mean in `[0, 1]`. Defaults to no shift.
|
|
197
|
+
* @param std Per-channel RGB standard deviation. Defaults to no scaling.
|
|
198
|
+
*/
|
|
199
|
+
constructor(targetWidth, targetHeight, mean = [0, 0, 0], std = [1, 1, 1]) {
|
|
200
|
+
if (targetWidth <= 0 || targetHeight <= 0) {
|
|
201
|
+
throw new Error(`Invalid resize target ${targetWidth}x${targetHeight}.`);
|
|
202
|
+
}
|
|
203
|
+
this._targetWidth = targetWidth;
|
|
204
|
+
this._targetHeight = targetHeight;
|
|
205
|
+
this._mean = mean;
|
|
206
|
+
this._std = std;
|
|
207
|
+
this._buffer = new Float32Array(3 * targetHeight * targetWidth);
|
|
208
|
+
}
|
|
209
|
+
/** The `[width, height]` this pipeline resizes into. */
|
|
210
|
+
get targetSize() {
|
|
211
|
+
return [this._targetWidth, this._targetHeight];
|
|
212
|
+
}
|
|
213
|
+
/**
|
|
214
|
+
* Resize an image to the target size and write it as normalized planar float32.
|
|
215
|
+
*
|
|
216
|
+
* An image that already arrives at the target size skips the canvas entirely
|
|
217
|
+
* and is read straight out of its packed RGB — which is both faster and what
|
|
218
|
+
* keeps the result identical to `resize()`, whose own fast path returns the
|
|
219
|
+
* input untouched.
|
|
220
|
+
*
|
|
221
|
+
* @param image Source image in the SDK's canonical HWC RGB layout.
|
|
222
|
+
*/
|
|
223
|
+
run(image) {
|
|
224
|
+
const targetWidth = this._targetWidth;
|
|
225
|
+
const targetHeight = this._targetHeight;
|
|
226
|
+
const reused = !this._bufferInUse;
|
|
227
|
+
const data = reused ? this._buffer : new Float32Array(3 * targetHeight * targetWidth);
|
|
228
|
+
this._bufferInUse = true;
|
|
229
|
+
if (image.width === targetWidth && image.height === targetHeight) {
|
|
230
|
+
writePlanarFloat32(image.data, targetWidth, targetHeight, this._mean, this._std, data, 3);
|
|
231
|
+
return { data, reused };
|
|
232
|
+
}
|
|
233
|
+
const source = this._ensureSource(image.width, image.height);
|
|
234
|
+
source.putImageData(rgbToImageData(image), 0, 0);
|
|
235
|
+
const context = this._ensureTarget();
|
|
236
|
+
context.drawImage(this._source, 0, 0, targetWidth, targetHeight);
|
|
237
|
+
const rgba = context.getImageData(0, 0, targetWidth, targetHeight).data;
|
|
238
|
+
writePlanarFloat32(rgba, targetWidth, targetHeight, this._mean, this._std, data);
|
|
239
|
+
return { data, reused };
|
|
240
|
+
}
|
|
241
|
+
/**
|
|
242
|
+
* Mark the reusable buffer free again.
|
|
243
|
+
*
|
|
244
|
+
* Call it once the tensor built from a {@link run} result has been handed to
|
|
245
|
+
* ONNX Runtime and the run has resolved — after that the values are inside
|
|
246
|
+
* the WASM heap and the buffer can be overwritten.
|
|
247
|
+
*/
|
|
248
|
+
release() {
|
|
249
|
+
this._bufferInUse = false;
|
|
250
|
+
}
|
|
251
|
+
/**
|
|
252
|
+
* Build the target canvas on first use.
|
|
253
|
+
*
|
|
254
|
+
* Lazily, so a pipeline constructed where no canvas implementation exists
|
|
255
|
+
* (a Node test, a worker without OffscreenCanvas) only fails if it is asked
|
|
256
|
+
* to resize something.
|
|
257
|
+
*/
|
|
258
|
+
_ensureTarget() {
|
|
259
|
+
if (this._target === null) {
|
|
260
|
+
this._target = createCanvas(this._targetWidth, this._targetHeight);
|
|
261
|
+
this._targetContext = get2DContext(this._target, { willReadFrequently: true });
|
|
262
|
+
this._targetContext.imageSmoothingEnabled = true;
|
|
263
|
+
this._targetContext.imageSmoothingQuality = "high";
|
|
264
|
+
}
|
|
265
|
+
return this._targetContext;
|
|
266
|
+
}
|
|
267
|
+
/** Grow the scratch source canvas to fit an image, reusing it when possible. */
|
|
268
|
+
_ensureSource(width, height) {
|
|
269
|
+
if (this._source === null || this._source.width !== width || this._source.height !== height) {
|
|
270
|
+
this._source = createCanvas(width, height);
|
|
271
|
+
this._sourceContext = get2DContext(this._source);
|
|
272
|
+
}
|
|
273
|
+
return this._sourceContext;
|
|
274
|
+
}
|
|
275
|
+
}
|
|
276
|
+
/**
|
|
277
|
+
* Resize an image into normalized planar float32 without keeping any state.
|
|
278
|
+
*
|
|
279
|
+
* The allocation-free path is {@link ResizePipeline}; this is the one-shot
|
|
280
|
+
* form, for a caller who wants the fused behaviour without owning a pipeline.
|
|
281
|
+
*
|
|
282
|
+
* @param image Source image in the SDK's canonical HWC RGB layout.
|
|
283
|
+
* @param targetWidth Model input width in pixels.
|
|
284
|
+
* @param targetHeight Model input height in pixels.
|
|
285
|
+
* @param mean Per-channel RGB mean in `[0, 1]`. Defaults to no shift.
|
|
286
|
+
* @param std Per-channel RGB standard deviation. Defaults to no scaling.
|
|
287
|
+
*/
|
|
288
|
+
export function resizeToTensorData(image, targetWidth, targetHeight, mean = [0, 0, 0], std = [1, 1, 1]) {
|
|
289
|
+
return new ResizePipeline(targetWidth, targetHeight, mean, std).run(image);
|
|
290
|
+
}
|
|
130
291
|
/**
|
|
131
292
|
* Build a zero-filled CHW tensor payload for a warm-up run.
|
|
132
293
|
*
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"pipeline.js","sourceRoot":"","sources":["../../src/preprocess/pipeline.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAEH,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,cAAc,EAAiC,MAAM,mBAAmB,CAAC;AAG9G,MAAM,OAAO,GAAG,CAAC,GAAG,GAAG,CAAC;AAqBxB;;;;;;GAMG;AACH,MAAM,OAAO,iBAAiB;IACX,YAAY,CAAS;IACrB,aAAa,CAAS;IACtB,KAAK,CAAoC;IACzC,OAAO,CAAW;IAClB,cAAc,CAAY;IAC1B,OAAO,CAAe;IAC/B,OAAO,GAAoB,IAAI,CAAC;IAChC,cAAc,GAAqB,IAAI,CAAC;IACxC,YAAY,GAAG,KAAK,CAAC;IAE7B;;;;OAIG;IACH,YACE,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC;QAEzD,IAAI,WAAW,IAAI,CAAC,IAAI,YAAY,IAAI,CAAC,EAAE,CAAC;YAC1C,MAAM,IAAI,KAAK,CAAC,4BAA4B,WAAW,IAAI,YAAY,GAAG,CAAC,CAAC;QAC9E,CAAC;QACD,IAAI,CAAC,YAAY,GAAG,WAAW,CAAC;QAChC,IAAI,CAAC,aAAa,GAAG,YAAY,CAAC;QAClC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,WAAW,EAAE,YAAY,CAAC,CAAC;QACvD,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,kBAAkB,EAAE,IAAI,EAAE,CAAC,CAAC;QAC/E,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,IAAI,CAAC;QACjD,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,MAAM,CAAC;QACnD,IAAI,CAAC,OAAO,GAAG,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;IAClE,CAAC;IAED,4DAA4D;IAC5D,IAAI,UAAU;QACZ,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;IACjD,CAAC;IAED;;;;;;;;;;OAUG;IACH,GAAG,CAAC,KAAe;QACjB,MAAM,WAAW,GAAG,IAAI,CAAC,YAAY,CAAC;QACtC,MAAM,YAAY,GAAG,IAAI,CAAC,aAAa,CAAC;QACxC,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,WAAW,GAAG,KAAK,CAAC,KAAK,EAAE,YAAY,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;QAC/E,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC,CAAC;QACpD,MAAM,YAAY,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,KAAK,CAAC,CAAC;QACtD,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC;QAC5D,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,YAAY,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC;QAE7D,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC;QAC7D,MAAM,CAAC,YAAY,CAAC,cAAc,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QAEjD,MAAM,OAAO,GAAG,IAAI,CAAC,cAAc,CAAC;QACpC,IAAI,OAAO,GAAG,CAAC,IAAI,MAAM,GAAG,CAAC,IAAI,WAAW,KAAK,WAAW,IAAI,YAAY,KAAK,YAAY,EAAE,CAAC;YAC9F,OAAO,CAAC,SAAS,GAAG,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC;YAC9E,OAAO,CAAC,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC;QACpD,CAAC;QACD,OAAO,CAAC,SAAS,CACf,IAAI,CAAC,OAA4B,EACjC,CAAC,EACD,CAAC,EACD,KAAK,CAAC,KAAK,EACX,KAAK,CAAC,MAAM,EACZ,OAAO,EACP,MAAM,EACN,WAAW,EACX,YAAY,CACb,CAAC;QAEF,MAAM,IAAI,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC,IAAI,CAAC;QACxE,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,YAAY,CAAC;QAClC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;QACtF,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC;QAEzB,MAAM,KAAK,GAAG,WAAW,GAAG,YAAY,CAAC;QACzC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,EAAE,KAAK,GAAG,KAAK,EAAE,KAAK,EAAE,EAAE,MAAM,IAAI,CAAC,EAAE,CAAC;YACpE,IAAI,CAAC,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,CAAY,GAAG,OAAO,CAAC;YACjD,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,CAAC;YAC7D,IAAI,CAAC,CAAC,GAAG,KAAK,GAAG,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,CAAC;QACnE,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;IAClD,CAAC;IAED;;;;;;OAMG;IACH,OAAO;QACL,IAAI,CAAC,YAAY,GAAG,KAAK,CAAC;IAC5B,CAAC;IAED;;;;;;;;OAQG;IACK,aAAa,CAAC,KAAa,EAAE,MAAc;QACjD,IAAI,IAAI,CAAC,OAAO,KAAK,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,KAAK,KAAK,KAAK,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;YAC5F,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;YAC3C,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,IAAI,CAAC,cAA2B,CAAC;IAC1C,CAAC;CACF;AAED;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,KAAa,EAAE,MAAc;IAC1D,OAAO,IAAI,YAAY,CAAC,CAAC,GAAG,MAAM,GAAG,KAAK,CAAC,CAAC;AAC9C,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,qBAAqB,CACnC,KAAe,EACf,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC;IAEzD,OAAO,IAAI,iBAAiB,CAAC,WAAW,EAAE,YAAY,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;AAC3E,CAAC"}
|
|
1
|
+
{"version":3,"file":"pipeline.js","sourceRoot":"","sources":["../../src/preprocess/pipeline.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAEH,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,cAAc,EAAiC,MAAM,mBAAmB,CAAC;AAG9G,MAAM,OAAO,GAAG,CAAC,GAAG,GAAG,CAAC;AAqBxB;;;;;;GAMG;AACH,MAAM,OAAO,iBAAiB;IACX,YAAY,CAAS;IACrB,aAAa,CAAS;IACtB,KAAK,CAAoC;IACzC,OAAO,CAAW;IAClB,cAAc,CAAY;IAC1B,OAAO,CAAe;IAC/B,OAAO,GAAoB,IAAI,CAAC;IAChC,cAAc,GAAqB,IAAI,CAAC;IACxC,YAAY,GAAG,KAAK,CAAC;IAE7B;;;;OAIG;IACH,YACE,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC;QAEzD,IAAI,WAAW,IAAI,CAAC,IAAI,YAAY,IAAI,CAAC,EAAE,CAAC;YAC1C,MAAM,IAAI,KAAK,CAAC,4BAA4B,WAAW,IAAI,YAAY,GAAG,CAAC,CAAC;QAC9E,CAAC;QACD,IAAI,CAAC,YAAY,GAAG,WAAW,CAAC;QAChC,IAAI,CAAC,aAAa,GAAG,YAAY,CAAC;QAClC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,WAAW,EAAE,YAAY,CAAC,CAAC;QACvD,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,kBAAkB,EAAE,IAAI,EAAE,CAAC,CAAC;QAC/E,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,IAAI,CAAC;QACjD,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,MAAM,CAAC;QACnD,IAAI,CAAC,OAAO,GAAG,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;IAClE,CAAC;IAED,4DAA4D;IAC5D,IAAI,UAAU;QACZ,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;IACjD,CAAC;IAED;;;;;;;;;;OAUG;IACH,GAAG,CAAC,KAAe;QACjB,MAAM,WAAW,GAAG,IAAI,CAAC,YAAY,CAAC;QACtC,MAAM,YAAY,GAAG,IAAI,CAAC,aAAa,CAAC;QACxC,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,WAAW,GAAG,KAAK,CAAC,KAAK,EAAE,YAAY,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;QAC/E,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC,CAAC;QACpD,MAAM,YAAY,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,KAAK,CAAC,CAAC;QACtD,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC;QAC5D,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,YAAY,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC;QAE7D,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC;QAC7D,MAAM,CAAC,YAAY,CAAC,cAAc,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QAEjD,MAAM,OAAO,GAAG,IAAI,CAAC,cAAc,CAAC;QACpC,IAAI,OAAO,GAAG,CAAC,IAAI,MAAM,GAAG,CAAC,IAAI,WAAW,KAAK,WAAW,IAAI,YAAY,KAAK,YAAY,EAAE,CAAC;YAC9F,OAAO,CAAC,SAAS,GAAG,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC;YAC9E,OAAO,CAAC,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC;QACpD,CAAC;QACD,OAAO,CAAC,SAAS,CACf,IAAI,CAAC,OAA4B,EACjC,CAAC,EACD,CAAC,EACD,KAAK,CAAC,KAAK,EACX,KAAK,CAAC,MAAM,EACZ,OAAO,EACP,MAAM,EACN,WAAW,EACX,YAAY,CACb,CAAC;QAEF,MAAM,IAAI,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC,IAAI,CAAC;QACxE,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,YAAY,CAAC;QAClC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;QACtF,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC;QAEzB,MAAM,KAAK,GAAG,WAAW,GAAG,YAAY,CAAC;QACzC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,EAAE,KAAK,GAAG,KAAK,EAAE,KAAK,EAAE,EAAE,MAAM,IAAI,CAAC,EAAE,CAAC;YACpE,IAAI,CAAC,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,CAAY,GAAG,OAAO,CAAC;YACjD,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,CAAC;YAC7D,IAAI,CAAC,CAAC,GAAG,KAAK,GAAG,KAAK,CAAC,GAAI,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,CAAC;QACnE,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;IAClD,CAAC;IAED;;;;;;OAMG;IACH,OAAO;QACL,IAAI,CAAC,YAAY,GAAG,KAAK,CAAC;IAC5B,CAAC;IAED;;;;;;;;OAQG;IACK,aAAa,CAAC,KAAa,EAAE,MAAc;QACjD,IAAI,IAAI,CAAC,OAAO,KAAK,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,KAAK,KAAK,KAAK,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;YAC5F,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;YAC3C,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,IAAI,CAAC,cAA2B,CAAC;IAC1C,CAAC;CACF;AAED;;;;;;;;;;;;;;;;;;GAkBG;AACH,MAAM,UAAU,kBAAkB,CAChC,IAAoC,EACpC,KAAa,EACb,MAAc,EACd,IAAuC,EACvC,GAAsC,EACtC,GAAiB,EACjB,SAAiB,CAAC;IAElB,MAAM,KAAK,GAAG,KAAK,GAAG,MAAM,CAAC;IAC7B,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACnB,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACnB,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACnB,MAAM,EAAE,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IAClB,MAAM,EAAE,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IAClB,MAAM,EAAE,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IAClB,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,EAAE,KAAK,GAAG,KAAK,EAAE,KAAK,EAAE,EAAE,MAAM,IAAI,MAAM,EAAE,CAAC;QACzE,GAAG,CAAC,KAAK,CAAC,GAAG,CAAE,IAAI,CAAC,MAAM,CAAY,GAAG,OAAO,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC;QAC5D,GAAG,CAAC,KAAK,GAAG,KAAK,CAAC,GAAG,CAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC;QACxE,GAAG,CAAC,CAAC,GAAG,KAAK,GAAG,KAAK,CAAC,GAAG,CAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAY,GAAG,OAAO,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC;IAC9E,CAAC;AACH,CAAC;AAeD;;;;;;;;;;;;;;;;;;GAkBG;AACH,MAAM,OAAO,cAAc;IACR,YAAY,CAAS;IACrB,aAAa,CAAS;IACtB,KAAK,CAAoC;IACzC,IAAI,CAAoC;IACxC,OAAO,CAAe;IAC/B,OAAO,GAAoB,IAAI,CAAC;IAChC,cAAc,GAAqB,IAAI,CAAC;IACxC,OAAO,GAAoB,IAAI,CAAC;IAChC,cAAc,GAAqB,IAAI,CAAC;IACxC,YAAY,GAAG,KAAK,CAAC;IAE7B;;;;;OAKG;IACH,YACE,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,EACnD,MAAyC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;QAElD,IAAI,WAAW,IAAI,CAAC,IAAI,YAAY,IAAI,CAAC,EAAE,CAAC;YAC1C,MAAM,IAAI,KAAK,CAAC,yBAAyB,WAAW,IAAI,YAAY,GAAG,CAAC,CAAC;QAC3E,CAAC;QACD,IAAI,CAAC,YAAY,GAAG,WAAW,CAAC;QAChC,IAAI,CAAC,aAAa,GAAG,YAAY,CAAC;QAClC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC,IAAI,GAAG,GAAG,CAAC;QAChB,IAAI,CAAC,OAAO,GAAG,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;IAClE,CAAC;IAED,wDAAwD;IACxD,IAAI,UAAU;QACZ,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;IACjD,CAAC;IAED;;;;;;;;;OASG;IACH,GAAG,CAAC,KAAe;QACjB,MAAM,WAAW,GAAG,IAAI,CAAC,YAAY,CAAC;QACtC,MAAM,YAAY,GAAG,IAAI,CAAC,aAAa,CAAC;QACxC,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,YAAY,CAAC;QAClC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,YAAY,CAAC,CAAC,GAAG,YAAY,GAAG,WAAW,CAAC,CAAC;QACtF,IAAI,CAAC,YAAY,GAAG,IAAI,CAAC;QAEzB,IAAI,KAAK,CAAC,KAAK,KAAK,WAAW,IAAI,KAAK,CAAC,MAAM,KAAK,YAAY,EAAE,CAAC;YACjE,kBAAkB,CAAC,KAAK,CAAC,IAAI,EAAE,WAAW,EAAE,YAAY,EAAE,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;YAC1F,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;QAC1B,CAAC;QAED,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC;QAC7D,MAAM,CAAC,YAAY,CAAC,cAAc,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QAEjD,MAAM,OAAO,GAAG,IAAI,CAAC,aAAa,EAAE,CAAC;QACrC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,OAA4B,EAAE,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC;QAEtF,MAAM,IAAI,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC,IAAI,CAAC;QACxE,kBAAkB,CAAC,IAAI,EAAE,WAAW,EAAE,YAAY,EAAE,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACjF,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;IAC1B,CAAC;IAED;;;;;;OAMG;IACH,OAAO;QACL,IAAI,CAAC,YAAY,GAAG,KAAK,CAAC;IAC5B,CAAC;IAED;;;;;;OAMG;IACK,aAAa;QACnB,IAAI,IAAI,CAAC,OAAO,KAAK,IAAI,EAAE,CAAC;YAC1B,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,IAAI,CAAC,YAAY,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;YACnE,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,kBAAkB,EAAE,IAAI,EAAE,CAAC,CAAC;YAC/E,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,IAAI,CAAC;YACjD,IAAI,CAAC,cAAc,CAAC,qBAAqB,GAAG,MAAM,CAAC;QACrD,CAAC;QACD,OAAO,IAAI,CAAC,cAA2B,CAAC;IAC1C,CAAC;IAED,gFAAgF;IACxE,aAAa,CAAC,KAAa,EAAE,MAAc;QACjD,IAAI,IAAI,CAAC,OAAO,KAAK,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,KAAK,KAAK,KAAK,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;YAC5F,IAAI,CAAC,OAAO,GAAG,YAAY,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;YAC3C,IAAI,CAAC,cAAc,GAAG,YAAY,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;QACnD,CAAC;QACD,OAAO,IAAI,CAAC,cAA2B,CAAC;IAC1C,CAAC;CACF;AAED;;;;;;;;;;;GAWG;AACH,MAAM,UAAU,kBAAkB,CAChC,KAAe,EACf,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,EACnD,MAAyC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;IAElD,OAAO,IAAI,cAAc,CAAC,WAAW,EAAE,YAAY,EAAE,IAAI,EAAE,GAAG,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;AAC7E,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,KAAa,EAAE,MAAc;IAC1D,OAAO,IAAI,YAAY,CAAC,CAAC,GAAG,MAAM,GAAG,KAAK,CAAC,CAAC;AAC9C,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,qBAAqB,CACnC,KAAe,EACf,WAAmB,EACnB,YAAoB,EACpB,OAA0C,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC;IAEzD,OAAO,IAAI,iBAAiB,CAAC,WAAW,EAAE,YAAY,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;AAC3E,CAAC"}
|
package/dist/tasks/base.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"base.d.ts","sourceRoot":"","sources":["../../src/tasks/base.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAEhD,8BAAsB,UAAU;IACR,SAAS,CAAC,QAAQ,CAAC,QAAQ,EAAE,UAAU;IAA7D,SAAS,aAAgC,QAAQ,EAAE,UAAU;IAE7D,+DAA+D;IAC/D,IAAI,OAAO,IAAI,UAAU,CAExB;CACF;
|
|
1
|
+
{"version":3,"file":"base.d.ts","sourceRoot":"","sources":["../../src/tasks/base.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAEhD,8BAAsB,UAAU;IACR,SAAS,CAAC,QAAQ,CAAC,QAAQ,EAAE,UAAU;IAA7D,SAAS,aAAgC,QAAQ,EAAE,UAAU;IAE7D,+DAA+D;IAC/D,IAAI,OAAO,IAAI,UAAU,CAExB;CACF;AAwBD;;;;;;;;;;;;;;GAcG;AACH,wBAAgB,iBAAiB,CAC/B,KAAK,EAAE,MAAM,EACb,OAAO,EAAE;IACP,QAAQ,CAAC,YAAY,EAAE,OAAO,CAAC;IAC/B,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,GAAG,SAAS,CAAC;IAChD,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;CAC9B,GACA,IAAI,CAUN"}
|
package/dist/tasks/base.js
CHANGED
|
@@ -15,6 +15,27 @@ export class VisionTask {
|
|
|
15
15
|
return this._session;
|
|
16
16
|
}
|
|
17
17
|
}
|
|
18
|
+
/**
|
|
19
|
+
* Render a confidence threshold the way the Python SDK renders it.
|
|
20
|
+
*
|
|
21
|
+
* JavaScript and Python disagree on how a number becomes text. A whole
|
|
22
|
+
* threshold is `1` here and `1.0` there; JavaScript holds off on exponent
|
|
23
|
+
* notation until `1e-7` while Python switches at `1e-5`. A fused pipeline is
|
|
24
|
+
* built once and runs under both runtimes from the same file, so a message that
|
|
25
|
+
* quotes the threshold has to quote it identically — otherwise the two SDKs
|
|
26
|
+
* describe the same run with two different numbers.
|
|
27
|
+
*
|
|
28
|
+
* Six decimals with the trailing zeros trimmed covers every threshold a caller
|
|
29
|
+
* can meaningfully set and agrees byte for byte with `_format_threshold` in the
|
|
30
|
+
* Python SDK's `tasks/base.py`. The pairing is fixed by a shared table in both
|
|
31
|
+
* test suites, so a change on one side that is not mirrored on the other fails.
|
|
32
|
+
*
|
|
33
|
+
* @param value The threshold to render.
|
|
34
|
+
* @returns The threshold as text, without trailing zeros or a trailing dot.
|
|
35
|
+
*/
|
|
36
|
+
function formatThreshold(value) {
|
|
37
|
+
return value.toFixed(6).replace(/0+$/, "").replace(/\.$/, "");
|
|
38
|
+
}
|
|
18
39
|
/**
|
|
19
40
|
* Turn an empty result into an error, when the caller asked for that.
|
|
20
41
|
*
|
|
@@ -37,6 +58,6 @@ export function requireDetections(count, options) {
|
|
|
37
58
|
const narrowed = options.classes === undefined
|
|
38
59
|
? ""
|
|
39
60
|
: ` among classes [${[...options.classes].sort((a, b) => a - b).join(", ")}]`;
|
|
40
|
-
throw new NoDetectionsError(`No detections${where}${narrowed}: nothing cleared confThreshold=${options.confThreshold}.`);
|
|
61
|
+
throw new NoDetectionsError(`No detections${where}${narrowed}: nothing cleared confThreshold=${formatThreshold(options.confThreshold)}.`);
|
|
41
62
|
}
|
|
42
63
|
//# sourceMappingURL=base.js.map
|
package/dist/tasks/base.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/tasks/base.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAC;AAG1D,MAAM,OAAgB,UAAU;IACW;IAAzC,YAAyC,QAAoB;QAApB,aAAQ,GAAR,QAAQ,CAAY;IAAG,CAAC;IAEjE,+DAA+D;IAC/D,IAAI,OAAO;QACT,OAAO,IAAI,CAAC,QAAQ,CAAC;IACvB,CAAC;CACF;AAED;;;;;;;;;;;;;;GAcG;AACH,MAAM,UAAU,iBAAiB,CAC/B,KAAa,EACb,OAKC;IAED,IAAI,CAAC,OAAO,CAAC,YAAY,IAAI,KAAK,GAAG,CAAC;QAAE,OAAO;IAC/C,MAAM,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,OAAO,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACxD,MAAM,QAAQ,GACZ,OAAO,CAAC,OAAO,KAAK,SAAS;QAC3B,CAAC,CAAC,EAAE;QACJ,CAAC,CAAC,mBAAmB,CAAC,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC;IAClF,MAAM,IAAI,iBAAiB,CACzB,gBAAgB,KAAK,GAAG,QAAQ,mCAAmC,OAAO,CAAC,aAAa,GAAG,
|
|
1
|
+
{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/tasks/base.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAC;AAG1D,MAAM,OAAgB,UAAU;IACW;IAAzC,YAAyC,QAAoB;QAApB,aAAQ,GAAR,QAAQ,CAAY;IAAG,CAAC;IAEjE,+DAA+D;IAC/D,IAAI,OAAO;QACT,OAAO,IAAI,CAAC,QAAQ,CAAC;IACvB,CAAC;CACF;AAED;;;;;;;;;;;;;;;;;GAiBG;AACH,SAAS,eAAe,CAAC,KAAa;IACpC,OAAO,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;AAChE,CAAC;AAED;;;;;;;;;;;;;;GAcG;AACH,MAAM,UAAU,iBAAiB,CAC/B,KAAa,EACb,OAKC;IAED,IAAI,CAAC,OAAO,CAAC,YAAY,IAAI,KAAK,GAAG,CAAC;QAAE,OAAO;IAC/C,MAAM,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,OAAO,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACxD,MAAM,QAAQ,GACZ,OAAO,CAAC,OAAO,KAAK,SAAS;QAC3B,CAAC,CAAC,EAAE;QACJ,CAAC,CAAC,mBAAmB,CAAC,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC;IAClF,MAAM,IAAI,iBAAiB,CACzB,gBAAgB,KAAK,GAAG,QAAQ,mCAAmC,eAAe,CAAC,OAAO,CAAC,aAAa,CAAC,GAAG,CAC7G,CAAC;AACJ,CAAC"}
|
|
@@ -79,6 +79,28 @@ export declare class Classifier extends VisionTask {
|
|
|
79
79
|
private readonly _std;
|
|
80
80
|
private readonly _applySoftmax;
|
|
81
81
|
private constructor();
|
|
82
|
+
private _pipelineCache;
|
|
83
|
+
/**
|
|
84
|
+
* Run the model once on a zero tensor, paying one-time costs up front.
|
|
85
|
+
*
|
|
86
|
+
* The first inference of a session is not representative: WebGPU compiles its
|
|
87
|
+
* shaders on it and the WASM backend faults in its arenas. Calling this while
|
|
88
|
+
* a loading spinner is still up moves that cost somewhere the user is already
|
|
89
|
+
* waiting — which matters most for a classifier running as the last step of
|
|
90
|
+
* an on-device analysis, where the delay lands right before the answer shows.
|
|
91
|
+
*
|
|
92
|
+
* @param runs How many warm-up inferences to run. One is enough for WASM;
|
|
93
|
+
* WebGPU sometimes settles on the second.
|
|
94
|
+
*/
|
|
95
|
+
warmup(runs?: number): Promise<void>;
|
|
96
|
+
/**
|
|
97
|
+
* The fused preprocessing pipeline, built on first use.
|
|
98
|
+
*
|
|
99
|
+
* Lazily, because constructing it reserves the output buffer: a task built in
|
|
100
|
+
* an environment without a canvas implementation stays constructible, and only
|
|
101
|
+
* fails if it is actually asked to preprocess something.
|
|
102
|
+
*/
|
|
103
|
+
private get _pipeline();
|
|
82
104
|
/** Load the model and resolve labels. */
|
|
83
105
|
static create(model: ModelSource, options?: ClassifierOptions): Promise<Classifier>;
|
|
84
106
|
/** Class labels indexed by class id. */
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"classifier.d.ts","sourceRoot":"","sources":["../../src/tasks/classifier.ts"],"names":[],"mappings":"AAAA;;GAEG;AAIH,OAAO,EAAE,KAAK,WAAW,EAAE,KAAK,iBAAiB,EAAc,MAAM,oBAAoB,CAAC;AAE1F,OAAO,EAAE,KAAK,UAAU,EAAa,MAAM,gBAAgB,CAAC;AAG5D,OAAO,EAAE,KAAK,SAAS,EAAiB,MAAM,cAAc,CAAC;
|
|
1
|
+
{"version":3,"file":"classifier.d.ts","sourceRoot":"","sources":["../../src/tasks/classifier.ts"],"names":[],"mappings":"AAAA;;GAEG;AAIH,OAAO,EAAE,KAAK,WAAW,EAAE,KAAK,iBAAiB,EAAc,MAAM,oBAAoB,CAAC;AAE1F,OAAO,EAAE,KAAK,UAAU,EAAa,MAAM,gBAAgB,CAAC;AAG5D,OAAO,EAAE,KAAK,SAAS,EAAiB,MAAM,cAAc,CAAC;AAI7D,OAAO,EAAE,qBAAqB,EAAS,MAAM,eAAe,CAAC;AAC7D,OAAO,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAUvC,MAAM,WAAW,iBAAkB,SAAQ,iBAAiB;IAC1D;;;;;;;OAOG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,SAAS,CAAC;IAC5B;;;;;OAKG;IACH,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;IAC7B;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC/C,yEAAyE;IACzE,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;IAClD,gEAAgE;IAChE,QAAQ,CAAC,GAAG,CAAC,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;IACjD;;;;OAIG;IACH,QAAQ,CAAC,YAAY,CAAC,EAAE,OAAO,CAAC;CACjC;AAED,MAAM,WAAW,wBAAwB;IACvC;;;;OAIG;IACH,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;CACxB;AAED;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,qBAAa,UAAW,SAAQ,UAAU;IAGtC,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,MAAM;IACvB,OAAO,CAAC,QAAQ,CAAC,UAAU;IAC3B,OAAO,CAAC,QAAQ,CAAC,KAAK;IACtB,OAAO,CAAC,QAAQ,CAAC,IAAI;IACrB,OAAO,CAAC,QAAQ,CAAC,aAAa;IAPhC,OAAO;IAYP,OAAO,CAAC,cAAc,CAA+B;IAErD;;;;;;;;;;;OAWG;IACG,MAAM,CAAC,IAAI,GAAE,MAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAQ7C;;;;;;OAMG;IACH,OAAO,KAAK,SAAS,GAMpB;IAED,yCAAyC;WAC5B,MAAM,CACjB,KAAK,EAAE,WAAW,EAClB,OAAO,GAAE,iBAAsB,GAC9B,OAAO,CAAC,UAAU,CAAC;IA0BtB,wCAAwC;IACxC,IAAI,MAAM,IAAI,SAAS,MAAM,EAAE,CAE9B;IAED,uEAAuE;IACvE,IAAI,KAAK,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAE5C;IAED;;;;;;OAMG;IACH,IAAI,SAAS,IAAI,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAEzC;IAED,+CAA+C;IAC/C,IAAI,UAAU,IAAI,MAAM,CAEvB;IAED,4EAA4E;IACtE,IAAI,CACR,KAAK,EAAE,UAAU,EACjB,OAAO,GAAE,wBAA6B,GACrC,OAAO,CAAC,qBAAqB,EAAE,CAAC;IAInC,4CAA4C;IACtC,OAAO,CACX,KAAK,EAAE,UAAU,EACjB,OAAO,GAAE,wBAA6B,GACrC,OAAO,CAAC,qBAAqB,EAAE,CAAC;IAkEnC,OAAO,CAAC,WAAW;IAMnB,OAAO,CAAC,YAAY;CAGrB"}
|
package/dist/tasks/classifier.js
CHANGED
|
@@ -8,7 +8,8 @@ import { classificationNumClasses, resolveInputSize } from "../core/graph.js";
|
|
|
8
8
|
import { modelNames } from "../core/metadata.js";
|
|
9
9
|
import { resolveLabels } from "../labels.js";
|
|
10
10
|
import { softmax, topK } from "../postprocess/classification.js";
|
|
11
|
-
import {
|
|
11
|
+
import { toFloat32Tensor } from "../preprocess/image.js";
|
|
12
|
+
import { ResizePipeline, zeroTensorData } from "../preprocess/pipeline.js";
|
|
12
13
|
import { ClassificationResults, Probs } from "../results.js";
|
|
13
14
|
import { VisionTask } from "./base.js";
|
|
14
15
|
const IMAGENET_MEAN = [0.485, 0.456, 0.406];
|
|
@@ -50,6 +51,40 @@ export class Classifier extends VisionTask {
|
|
|
50
51
|
this._std = _std;
|
|
51
52
|
this._applySoftmax = _applySoftmax;
|
|
52
53
|
}
|
|
54
|
+
_pipelineCache = null;
|
|
55
|
+
/**
|
|
56
|
+
* Run the model once on a zero tensor, paying one-time costs up front.
|
|
57
|
+
*
|
|
58
|
+
* The first inference of a session is not representative: WebGPU compiles its
|
|
59
|
+
* shaders on it and the WASM backend faults in its arenas. Calling this while
|
|
60
|
+
* a loading spinner is still up moves that cost somewhere the user is already
|
|
61
|
+
* waiting — which matters most for a classifier running as the last step of
|
|
62
|
+
* an on-device analysis, where the delay lands right before the answer shows.
|
|
63
|
+
*
|
|
64
|
+
* @param runs How many warm-up inferences to run. One is enough for WASM;
|
|
65
|
+
* WebGPU sometimes settles on the second.
|
|
66
|
+
*/
|
|
67
|
+
async warmup(runs = 1) {
|
|
68
|
+
const [tw, th] = this._inputSize;
|
|
69
|
+
for (let i = 0; i < runs; i++) {
|
|
70
|
+
const tensor = toFloat32Tensor(zeroTensorData(tw, th), [1, 3, th, tw]);
|
|
71
|
+
await this._session.run({ [this._session.inputName]: tensor });
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
/**
|
|
75
|
+
* The fused preprocessing pipeline, built on first use.
|
|
76
|
+
*
|
|
77
|
+
* Lazily, because constructing it reserves the output buffer: a task built in
|
|
78
|
+
* an environment without a canvas implementation stays constructible, and only
|
|
79
|
+
* fails if it is actually asked to preprocess something.
|
|
80
|
+
*/
|
|
81
|
+
get _pipeline() {
|
|
82
|
+
if (this._pipelineCache === null) {
|
|
83
|
+
const [tw, th] = this._inputSize;
|
|
84
|
+
this._pipelineCache = new ResizePipeline(tw, th, this._mean, this._std);
|
|
85
|
+
}
|
|
86
|
+
return this._pipelineCache;
|
|
87
|
+
}
|
|
53
88
|
/** Load the model and resolve labels. */
|
|
54
89
|
static async create(model, options = {}) {
|
|
55
90
|
const session = await OrtSession.create(model, options);
|
|
@@ -102,6 +137,7 @@ export class Classifier extends VisionTask {
|
|
|
102
137
|
const tensor = this._preprocess(original);
|
|
103
138
|
timer.stage("preprocess");
|
|
104
139
|
const outputs = await this._session.run({ [this._session.inputName]: tensor });
|
|
140
|
+
this._pipeline.release();
|
|
105
141
|
timer.stage("inference");
|
|
106
142
|
const firstOutputName = this._session.outputNames[0];
|
|
107
143
|
if (firstOutputName === undefined) {
|
|
@@ -149,10 +185,8 @@ export class Classifier extends VisionTask {
|
|
|
149
185
|
}
|
|
150
186
|
_preprocess(image) {
|
|
151
187
|
const [tw, th] = this._inputSize;
|
|
152
|
-
const
|
|
153
|
-
|
|
154
|
-
const chw = toCHW(normalized, resized.width, resized.height, 3);
|
|
155
|
-
return toFloat32Tensor(chw, [1, 3, resized.height, resized.width]);
|
|
188
|
+
const { data } = this._pipeline.run(image);
|
|
189
|
+
return toFloat32Tensor(data, [1, 3, th, tw]);
|
|
156
190
|
}
|
|
157
191
|
_postprocess(raw) {
|
|
158
192
|
return this._applySoftmax ? softmax(raw) : new Float32Array(raw);
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"classifier.js","sourceRoot":"","sources":["../../src/tasks/classifier.ts"],"names":[],"mappings":"AAAA;;GAEG;AAIH,OAAO,EAA4C,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAC1F,OAAO,EAAE,UAAU,EAAE,MAAM,mBAAmB,CAAC;AAC/C,OAAO,EAAmB,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAC5D,OAAO,EAAE,wBAAwB,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAC9E,OAAO,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AACjD,OAAO,EAAkB,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7D,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,kCAAkC,CAAC;AACjE,OAAO,
|
|
1
|
+
{"version":3,"file":"classifier.js","sourceRoot":"","sources":["../../src/tasks/classifier.ts"],"names":[],"mappings":"AAAA;;GAEG;AAIH,OAAO,EAA4C,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAC1F,OAAO,EAAE,UAAU,EAAE,MAAM,mBAAmB,CAAC;AAC/C,OAAO,EAAmB,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAC5D,OAAO,EAAE,wBAAwB,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAC9E,OAAO,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AACjD,OAAO,EAAkB,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7D,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,kCAAkC,CAAC;AACjE,OAAO,EAAE,eAAe,EAAE,MAAM,wBAAwB,CAAC;AACzD,OAAO,EAAE,cAAc,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAC3E,OAAO,EAAE,qBAAqB,EAAE,KAAK,EAAE,MAAM,eAAe,CAAC;AAC7D,OAAO,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAOvC,MAAM,aAAa,GAAsC,CAAC,KAAK,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;AAC/E,MAAM,YAAY,GAAsC,CAAC,KAAK,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;AAgD9E;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,MAAM,OAAO,UAAW,SAAQ,UAAU;IAGrB;IACA;IACA;IACA;IACA;IACA;IAPnB,YACE,OAAmB,EACF,OAA0B,EAC1B,MAAwC,EACxC,UAAqC,EACrC,KAAwC,EACxC,IAAuC,EACvC,aAAsB;QAEvC,KAAK,CAAC,OAAO,CAAC,CAAC;QAPE,YAAO,GAAP,OAAO,CAAmB;QAC1B,WAAM,GAAN,MAAM,CAAkC;QACxC,eAAU,GAAV,UAAU,CAA2B;QACrC,UAAK,GAAL,KAAK,CAAmC;QACxC,SAAI,GAAJ,IAAI,CAAmC;QACvC,kBAAa,GAAb,aAAa,CAAS;IAGzC,CAAC;IAEO,cAAc,GAA0B,IAAI,CAAC;IAErD;;;;;;;;;;;OAWG;IACH,KAAK,CAAC,MAAM,CAAC,OAAe,CAAC;QAC3B,MAAM,CAAC,EAAE,EAAE,EAAE,CAAC,GAAG,IAAI,CAAC,UAAU,CAAC;QACjC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,EAAE,CAAC,EAAE,EAAE,CAAC;YAC9B,MAAM,MAAM,GAAG,eAAe,CAAC,cAAc,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC;YACvE,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;QACjE,CAAC;IACH,CAAC;IAED;;;;;;OAMG;IACH,IAAY,SAAS;QACnB,IAAI,IAAI,CAAC,cAAc,KAAK,IAAI,EAAE,CAAC;YACjC,MAAM,CAAC,EAAE,EAAE,EAAE,CAAC,GAAG,IAAI,CAAC,UAAU,CAAC;YACjC,IAAI,CAAC,cAAc,GAAG,IAAI,cAAc,CAAC,EAAE,EAAE,EAAE,EAAE,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QAC1E,CAAC;QACD,OAAO,IAAI,CAAC,cAAc,CAAC;IAC7B,CAAC;IAED,yCAAyC;IACzC,MAAM,CAAC,KAAK,CAAC,MAAM,CACjB,KAAkB,EAClB,UAA6B,EAAE;QAE/B,MAAM,OAAO,GAAG,MAAM,UAAU,CAAC,MAAM,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;QACxD,MAAM,UAAU,GACd,OAAO,CAAC,UAAU,IAAI,wBAAwB,CAAC,OAAO,CAAC,WAAW,CAAC,IAAI,SAAS,CAAC;QACnF,MAAM,MAAM,GAAG,aAAa,CAAC,OAAO,CAAC,MAAM,IAAI,UAAU,CAAC,OAAO,CAAC,QAAQ,CAAC,EAAE;YAC3E,UAAU;SACX,CAAC,CAAC;QACH,MAAM,KAAK,GAA2B,EAAE,CAAC;QACzC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACvC,KAAK,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,CAAW,CAAC;QACjC,CAAC;QACD,OAAO,IAAI,UAAU,CACnB,OAAO,EACP,MAAM,EACN,KAAK,EACL,gBAAgB,CAAC;YACf,UAAU,EAAE,OAAO,CAAC,UAAU;YAC9B,SAAS,EAAE,OAAO,CAAC,SAAS;YAC5B,QAAQ,EAAE,CAAC,GAAG,EAAE,GAAG,CAAC;SACrB,CAAC,EACF,OAAO,CAAC,IAAI,IAAI,aAAa,EAC7B,OAAO,CAAC,GAAG,IAAI,YAAY,EAC3B,OAAO,CAAC,YAAY,IAAI,IAAI,CAC7B,CAAC;IACJ,CAAC;IAED,wCAAwC;IACxC,IAAI,MAAM;QACR,OAAO,IAAI,CAAC,OAAO,CAAC;IACtB,CAAC;IAED,uEAAuE;IACvE,IAAI,KAAK;QACP,OAAO,IAAI,CAAC,MAAM,CAAC;IACrB,CAAC;IAED;;;;;;OAMG;IACH,IAAI,SAAS;QACX,OAAO,IAAI,CAAC,UAAU,CAAC;IACzB,CAAC;IAED,+CAA+C;IAC/C,IAAI,UAAU;QACZ,OAAO,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC;IAC7B,CAAC;IAED,4EAA4E;IAC5E,KAAK,CAAC,IAAI,CACR,KAAiB,EACjB,UAAoC,EAAE;QAEtC,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;IACtC,CAAC;IAED,4CAA4C;IAC5C,KAAK,CAAC,OAAO,CACX,KAAiB,EACjB,UAAoC,EAAE;QAEtC,MAAM,KAAK,GAAG,IAAI,UAAU,EAAE,CAAC;QAC/B,MAAM,IAAI,GAAG,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC;QACtD,MAAM,QAAQ,GAAG,MAAM,SAAS,CAAC,KAAK,CAAC,CAAC;QACxC,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;QACpB,MAAM,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC;QAC1C,KAAK,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;QAC1B,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;QAC/E,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,CAAC;QACzB,KAAK,CAAC,KAAK,CAAC,WAAW,CAAC,CAAC;QACzB,MAAM,eAAe,GAAG,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC;QACrD,IAAI,eAAe,KAAK,SAAS,EAAE,CAAC;YAClC,MAAM,IAAI,KAAK,CAAC,kCAAkC,CAAC,CAAC;QACtD,CAAC;QACD,MAAM,GAAG,GAAG,OAAO,CAAC,eAAe,CAAC,CAAC;QACrC,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,MAAM,IAAI,KAAK,CAAC,2BAA2B,eAAe,6BAA6B,CAAC,CAAC;QAC3F,CAAC;QACD,MAAM,SAAS,GAAG,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,IAAoB,CAAC,CAAC;QAE9D,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,OAAO,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC;QAClE,MAAM,aAAa,GAAuB,EAAE,CAAC;QAC7C,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACxC,MAAM,EAAE,GAAG,OAAO,CAAC,CAAC,CAAW,CAAC;YAChC,MAAM,SAAS,GAAG,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC,IAAI,SAAS,EAAE,EAAE,CAAC;YACpD,aAAa,CAAC,IAAI,CAAC;gBACjB,OAAO,EAAE,EAAE;gBACX,SAAS;gBACT,WAAW,EAAE,MAAM,CAAC,CAAC,CAAW;gBAChC,GAAG,EAAE,EAAE;gBACP,IAAI,EAAE,SAAS;gBACf,IAAI,EAAE,MAAM,CAAC,CAAC,CAAW;aAC1B,CAAC,CAAC;QACL,CAAC;QACD,IAAI,aAAa,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC/B,MAAM,IAAI,KAAK,CAAC,sDAAsD,CAAC,CAAC;QAC1E,CAAC;QAED,MAAM,GAAG,GAAG,aAAa,CAAC,CAAC,CAAqB,CAAC;QACjD,MAAM,MAAM,GAAyB;YACnC,OAAO,EAAE,GAAG,CAAC,OAAO;YACpB,SAAS,EAAE,GAAG,CAAC,SAAS;YACxB,UAAU,EAAE,GAAG,CAAC,WAAW;YAC3B,GAAG,EAAE,GAAG,CAAC,OAAO;YAChB,IAAI,EAAE,GAAG,CAAC,SAAS;YACnB,IAAI,EAAE,GAAG,CAAC,WAAW;YACrB,KAAK,EAAE,QAAQ;YACf,aAAa;SACd,CAAC;QAEF,MAAM,IAAI,GAA8B,CAAC,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,KAAK,CAAC,CAAC;QAC1E,MAAM,KAAK,GAAG,IAAI,KAAK,CAAC,SAAS,CAAC,CAAC;QACnC,KAAK,CAAC,KAAK,CAAC,aAAa,CAAC,CAAC;QAC3B,OAAO;YACL,IAAI,qBAAqB,CACvB,KAAK,EACL,MAAM,EACN,IAAI,CAAC,MAAM,EACX,QAAQ,EACR,IAAI,EACJ,IAAI,EACJ,KAAK,CAAC,KAAK,EAAE,CACd;SACF,CAAC;IACJ,CAAC;IAEO,WAAW,CAAC,KAAe;QACjC,MAAM,CAAC,EAAE,EAAE,EAAE,CAAC,GAAG,IAAI,CAAC,UAAU,CAAC;QACjC,MAAM,EAAE,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QAC3C,OAAO,eAAe,CAAC,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC;IAC/C,CAAC;IAEO,YAAY,CAAC,GAAiB;QACpC,OAAO,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,CAAC;IACnE,CAAC;CACF"}
|
package/package.json
CHANGED