web-sdk-pp-detection 0.1.0 → 0.1.1

package/README.md

@@ -2,7 +2,7 @@
 
 [English](#english) | [在线 Demo](https://chenmohan123.github.io/web-sdk-PP-Detection/)
 
-基于 ONNX Runtime Web 的浏览器端 PP-Detection 版面分析 SDK，支持 PC、移动端与各类 H5 页面。
+基于 ONNX Runtime Web 的浏览器端 PP-Detection 目标检测 SDK，支持 PC、移动端与各类 H5 页面。
 
 ## 安装
 
@@ -25,16 +25,15 @@ const detector = await createPPDetection({
 const result = await detector.detect(file, {
   threshold: 0.5,
   classThresholds: {
-    formula: 0.4,
-    table: 0.55,
-    text: 0.6
+    person: 0.6,
+    car: 0.5
   }
 });
 console.log(result.detections, result.runtime, result.timings);
 await detector.dispose();
 ```
 
-`classThresholds` 按 manifest 标签名称覆盖置信度过滤阈值，未配置的类别回退到 `threshold`。全局 `threshold` 仍用于 mask 二值化和多边形提取。未知类别名称或超出 `0` 到 `1` 的值会被拒绝。
+`classThresholds` 按 manifest 标签名称覆盖目标检测置信度过滤阈值，未配置的类别回退到全局 `threshold`。未知类别名称或超出 `0` 到 `1` 的值会被拒绝。
 
 当前发布版本内置 PicoDet 1.0.1 FP32 stable manifest；使用默认模型时可以省略 `model`。自定义模型仍应传入经过验证的 runtime manifest 或清单对象。
 
@@ -42,7 +41,7 @@ await detector.dispose();
 
 ## 运行后端与精度
 
-- `backend: "auto"` 优先使用 WebGPU；只有显式设置 `allowFallback: true` 时，WebGPU 会话失败才会尝试 WASM（CPU）。也可手动指定 `"webgpu"` 或 `"wasm"`。
+- `backend: "auto"` 优先使用 WebGPU；设置 `allowFallback: true` 后，WebGPU 会话或推理失败才会尝试 WASM（CPU）。也可手动指定 `"webgpu"` 或 `"wasm"`。
 - `precision: "auto"` 选择清单中可用的默认稳定精度；当前默认 PicoDet 仅验证 FP32，因此不会臆测切换到 FP16。已验证其他精度时可手动指定 `"fp16"`、`"int8"` 等。
 - 默认 PicoDet 1.0.1 FP32 已发布可下载的 stable ONNX 资产，并通过 Linux WASM 与 Windows NVIDIA WebGPU 七张 fixture 验证；FP16、INT8、INT4 和 FP8 仍需独立证据。
 - 使用默认模型时，显式请求清单中未声明的组合会抛出 `CAPABILITY_UNSUPPORTED`，不会改写无效组合；自定义清单可在单独验证后声明其他组合。上游模型是 float32，不支持 FP64；FP32 约为 FP16 两倍大小并可能更慢、更占显存。
@@ -75,7 +74,7 @@ const detector = await createPPDetection({
 
 ## English
 
-A browser-first PP-Detection document layout analysis SDK powered by ONNX Runtime Web for desktop, mobile, and H5 pages.
+A browser-first PP-Detection object detection SDK powered by ONNX Runtime Web for desktop, mobile, and H5 pages.
 
 ### Installation
 
@@ -98,16 +97,15 @@ const detector = await createPPDetection({
 const result = await detector.detect(file, {
   threshold: 0.5,
   classThresholds: {
-    formula: 0.4,
-    table: 0.55,
-    text: 0.6
+    person: 0.6,
+    car: 0.5
   }
 });
 console.log(result.detections, result.runtime, result.timings);
 await detector.dispose();
 ```
 
-`classThresholds` overrides confidence filtering for matching manifest label names and falls back to `threshold` for unspecified classes. The global `threshold` still controls mask binarization and polygon extraction. Unknown class names and values outside `0` through `1` are rejected.
+`classThresholds` overrides object-detection confidence filtering for matching manifest label names and falls back to the global `threshold` for unspecified classes. Unknown class names and values outside `0` through `1` are rejected.
 
 The release includes a built-in PicoDet 1.0.1 FP32 stable manifest, so `model` may be omitted for the default model. Pass a verified runtime or custom manifest when using another model.
 
@@ -115,7 +113,7 @@ Detailed initialization timings are available through `detector.loadTimings`. `t
 
 ### Backend and precision
 
-- `backend: "auto"` prefers WebGPU; only `allowFallback: true` permits a failed WebGPU session to try WASM (CPU). Use `"webgpu"` or `"wasm"` for an explicit choice.
+- `backend: "auto"` prefers WebGPU; with `allowFallback: true`, a failed WebGPU session or inference attempts WASM (CPU). Use `"webgpu"` or `"wasm"` for an explicit choice.
 - `precision: "auto"` selects the available default stable precision from the manifest. The default PicoDet 1.0.1 validates FP32 only, so the SDK does not guess an FP16 switch. When another precision has been validated, select it explicitly with `"fp16"`, `"int8"`, and so on.
 - The default PicoDet 1.0.1 FP32 variant is a downloadable stable ONNX asset and has passed seven-fixture validation on Linux WASM and Windows NVIDIA WebGPU. FP16, INT8, INT4, and FP8 still require independent release-source and backend evidence.
 - `detect` accepts image Blob/File, Canvas/ImageData, `HTMLVideoElement`, or a single `VideoFrame`. Hosts own camera/video permissions and frame pacing; await each frame Promise and call `dispose()` when media stops.

package/dist/browser-global.js

@@ -15323,14 +15323,15 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
   function mapError(error, phase) {
     if (error instanceof PPDetectionError) return error;
     const message = error instanceof Error ? error.message : String(error);
+    const details = { phase, causeMessage: message };
     if (/abort|cancel/i.test(message))
-      return new PPDetectionError("ABORTED", "\u63A8\u7406\u5DF2\u53D6\u6D88", { phase }, { cause: error });
+      return new PPDetectionError("ABORTED", "\u63A8\u7406\u5DF2\u53D6\u6D88", details, { cause: error });
     if (/memory|out.of.memory|allocation/i.test(message))
-      return new PPDetectionError("OUT_OF_MEMORY", "\u8FD0\u884C\u65F6\u5185\u5B58\u4E0D\u8DB3", { phase }, { cause: error });
+      return new PPDetectionError("OUT_OF_MEMORY", "\u8FD0\u884C\u65F6\u5185\u5B58\u4E0D\u8DB3", details, { cause: error });
     return new PPDetectionError(
       phase === "create" ? "SESSION_CREATE_FAILED" : "INFERENCE_FAILED",
       phase === "create" ? "\u521B\u5EFA ONNX Runtime \u4F1A\u8BDD\u5931\u8D25" : "ONNX Runtime \u63A8\u7406\u5931\u8D25",
-      { phase },
+      details,
       { cause: error }
     );
   }
@@ -15488,14 +15489,13 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
   function fail(code, message, details) {
     throw new PPDetectionError(code, message, details);
   }
-  function candidatesForBackend(requested, capabilities, allowFallback) {
+  function candidatesForBackend(requested, capabilities) {
     if (requested === "wasm") return ["wasm"];
     if (requested === "webgpu")
       return capabilities.webgpu ? ["webgpu"] : fail("CAPABILITY_UNSUPPORTED", "\u8BF7\u6C42\u7684 webgpu \u4E0D\u53EF\u7528", { requestedBackend: requested });
     const available = [];
     if (capabilities.webgpu) available.push("webgpu");
     available.push("wasm");
-    if (!allowFallback) return available.slice(0, 1);
     return available;
   }
   function selectExecutionPlan(options, capabilities, manifest) {
@@ -15513,19 +15513,18 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
         requestedPrecision
       });
     }
-    const candidates = candidatesForBackend(
-      requestedBackend,
-      capabilities,
-      options.allowFallback === true
-    ).filter((backend) => variant3.backends.includes(backend));
-    if (candidates.length === 0) {
+    const candidates = candidatesForBackend(requestedBackend, capabilities).filter(
+      (backend) => variant3.backends.includes(backend)
+    );
+    const selectedCandidates = options.allowFallback === true ? candidates : candidates.slice(0, 1);
+    if (selectedCandidates.length === 0) {
       fail("CAPABILITY_UNSUPPORTED", "\u6CA1\u6709\u4E0E\u6A21\u578B\u53D8\u4F53\u5339\u914D\u7684\u53EF\u7528\u540E\u7AEF", {
         requestedBackend,
         requestedPrecision,
         availableBackends: variant3.backends
       });
     }
-    const actualBackend = candidates[0];
+    const actualBackend = selectedCandidates[0];
     return {
       variantId: variant3.id,
       requestedBackend,
@@ -15533,7 +15532,7 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
       requestedPrecision,
       actualPrecision: variant3.precision,
       executionMode,
-      candidates: candidates.map((backend) => ({
+      candidates: selectedCandidates.map((backend) => ({
         variantId: variant3.id,
         backend,
         precision: variant3.precision,
@@ -15688,7 +15687,7 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
   };
 
   // src/index.ts
-  var CURRENT_SDK_VERSION = "0.1.0";
+  var CURRENT_SDK_VERSION = "0.1.1";
   function probePPDetectionCapabilities(options = {}) {
     return probeCapabilities(options);
   }
@@ -15814,7 +15813,6 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
       cache: options.cache === false ? false : options.cache === "memory" ? "memory" : void 0
     });
     let executor;
-    let activeBridge;
     try {
       const loadStartedAt = now4();
       options.onProgress?.({ phase: "model", status: "start" });
@@ -15847,29 +15845,16 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
       }
       options.onProgress?.({ phase: "model", status: "complete" });
       const fallbacks = [];
-      let selectedPlan = plan;
-      let sessionMs = 0;
-      for (const candidate of plan.candidates) {
-        const candidatePlan = {
-          ...plan,
-          variantId: candidate.variantId,
-          actualBackend: candidate.backend,
-          actualPrecision: candidate.precision,
-          executionMode: candidate.executionMode,
-          candidates: [candidate]
-        };
+      const createExecutorForPlan = async (candidatePlan) => {
         options.onProgress?.({ phase: "session", status: "start" });
+        let bridge;
         try {
-          if (candidate.executionMode === "worker") {
+          if (candidatePlan.executionMode === "worker") {
             if (typeof Worker !== "function")
               throw new PPDetectionError("CAPABILITY_UNSUPPORTED", "\u5F53\u524D\u73AF\u5883\u4E0D\u652F\u6301 Worker");
-            const worker = new Worker(workerUrl(), {
-              type: "module"
-            });
-            const bridge = new WorkerBridge(worker);
-            activeBridge = bridge;
-            const workerModelBytes = modelBytes.slice(0);
-            await bridge.load(workerModelBytes, candidatePlan, {
+            const worker = new Worker(workerUrl(), { type: "module" });
+            bridge = new WorkerBridge(worker);
+            await bridge.load(modelBytes.slice(0), candidatePlan, {
               onProgress: (event) => options.onProgress?.({
                 phase: "session",
                 status: event.status
@@ -15879,35 +15864,65 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
                 numThreads: options.ort?.wasm?.numThreads
               }
             });
-            executor = {
-              run(input, signal) {
-                return bridge.run(
-                  { [input.inputName]: { data: input.data, dims: input.dims } },
-                  { signal }
-                );
-              },
-              dispose: () => bridge.dispose()
-            };
-            activeBridge = void 0;
-          } else {
-            const session = await createOrtSession(modelBytes, candidatePlan, {
-              ort: options.ort?.module,
-              wasmPaths: options.ort?.wasm?.paths,
-              numThreads: options.ort?.wasm?.numThreads
-            });
-            sessionMs = session.sessionMs;
-            executor = {
+            const activeBridge = bridge;
+            options.onProgress?.({ phase: "session", status: "complete" });
+            return {
               run(input, signal) {
-                return session.run(
+                return activeBridge.run(
                   { [input.inputName]: { data: input.data, dims: input.dims } },
                   { signal }
                 );
               },
-              dispose: () => session.dispose()
+              dispose: () => activeBridge.dispose()
             };
           }
-          selectedPlan = candidatePlan;
+          const session = await createOrtSession(modelBytes, candidatePlan, {
+            ort: options.ort?.module,
+            wasmPaths: options.ort?.wasm?.paths,
+            numThreads: options.ort?.wasm?.numThreads
+          });
+          sessionMs = session.sessionMs;
           options.onProgress?.({ phase: "session", status: "complete" });
+          return {
+            run(input, signal) {
+              return session.run(
+                { [input.inputName]: { data: input.data, dims: input.dims } },
+                { signal }
+              );
+            },
+            dispose: () => session.dispose()
+          };
+        } catch (error) {
+          try {
+            await bridge?.dispose();
+          } catch {
+          }
+          if (error instanceof PPDetectionError) throw error;
+          const message = error instanceof Error ? error.message : String(error);
+          throw new PPDetectionError(
+            "SESSION_CREATE_FAILED",
+            "\u521B\u5EFA ONNX Runtime \u4F1A\u8BDD\u5931\u8D25",
+            { phase: "create", causeMessage: message },
+            { cause: error }
+          );
+        }
+      };
+      let selectedPlan = plan;
+      let sessionMs = 0;
+      let selectedCandidateIndex = -1;
+      for (const [candidateIndex, candidate] of plan.candidates.entries()) {
+        const candidatePlan = {
+          ...plan,
+          variantId: candidate.variantId,
+          actualBackend: candidate.backend,
+          actualPrecision: candidate.precision,
+          executionMode: candidate.executionMode,
+          candidates: [candidate]
+        };
+        try {
+          executor = await createExecutorForPlan(candidatePlan);
+          selectedPlan = candidatePlan;
+          selectedCandidateIndex = candidateIndex;
           break;
         } catch (error) {
           const mapped = error instanceof PPDetectionError ? error : new PPDetectionError("SESSION_CREATE_FAILED", String(error));
@@ -15917,11 +15932,6 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
           } catch {
           }
           executor = void 0;
-          try {
-            await activeBridge?.dispose();
-          } catch {
-          }
-          activeBridge = void 0;
           if (!hasNext) {
             throw mapped;
           }
@@ -15939,22 +15949,67 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
         }
       }
       if (!executor) throw new PPDetectionError("SESSION_CREATE_FAILED", "\u65E0\u6CD5\u521B\u5EFA\u68C0\u6D4B Session");
-      const loadedExecutor = executor;
+      const runtime = {
+        requestedBackend: options.backend ?? "auto",
+        backend: selectedPlan.actualBackend,
+        precision: selectedPlan.actualPrecision,
+        mode: selectedPlan.executionMode,
+        fallbacks,
+        capabilities
+      };
+      let activeExecutor = executor;
+      const fallbackExecutor = {
+        async run(input, signal) {
+          try {
+            return await activeExecutor.run(input, signal);
+          } catch (error) {
+            if (options.allowFallback !== true || selectedCandidateIndex < 0 || selectedCandidateIndex >= plan.candidates.length - 1) {
+              throw error;
+            }
+            const failedCandidate = plan.candidates[selectedCandidateIndex];
+            const mapped = error instanceof PPDetectionError ? error : new PPDetectionError("INFERENCE_FAILED", String(error), {}, { cause: error });
+            if (mapped.code === "ABORTED") throw mapped;
+            const nextCandidate = plan.candidates[selectedCandidateIndex + 1];
+            const nextPlan = {
+              ...plan,
+              variantId: nextCandidate.variantId,
+              actualBackend: nextCandidate.backend,
+              actualPrecision: nextCandidate.precision,
+              executionMode: nextCandidate.executionMode,
+              candidates: [nextCandidate]
+            };
+            await activeExecutor.dispose();
+            const nextExecutor = await createExecutorForPlan(nextPlan);
+            activeExecutor = nextExecutor;
+            selectedCandidateIndex += 1;
+            selectedPlan = nextPlan;
+            runtime.backend = nextPlan.actualBackend;
+            runtime.precision = nextPlan.actualPrecision;
+            runtime.mode = nextPlan.executionMode;
+            const fallback = {
+              cause: mapped.cause ?? mapped,
+              code: mapped.code,
+              message: mapped.message,
+              precision: failedCandidate.precision,
+              provider: failedCandidate.backend,
+              stage: "inference",
+              variantId: failedCandidate.variantId
+            };
+            fallbacks.push(fallback);
+            options.onProgress?.({ phase: "fallback", status: "complete", fallback });
+            return activeExecutor.run(input, signal);
+          }
+        },
+        dispose: () => activeExecutor.dispose()
+      };
       loadTimings = { ...loadTimings, sessionMs, totalMs: now4() - loadStartedAt };
       const detector = new PPDetectionDetectorImplementation({
         capabilities,
         manifest: runtimeManifest,
         model: modelInfo(runtimeManifest, variant3.id, actualSource),
-        runtime: {
-          requestedBackend: options.backend ?? "auto",
-          backend: selectedPlan.actualBackend,
-          precision: selectedPlan.actualPrecision,
-          mode: selectedPlan.executionMode,
-          fallbacks,
-          capabilities
-        },
+        runtime,
         loadTimings,
-        loadExecutor: () => Promise.resolve(loadedExecutor),
+        loadExecutor: () => Promise.resolve(fallbackExecutor),
         onProgress: options.onProgress,
         clearCurrentModelCache: () => modelManager.clearCurrentModelCache(),
         clearAllCache: () => modelManager.clearAllCache(),
@@ -15969,10 +16024,6 @@ ${u}`, c = n.createShaderModule({ code: d, label: e.name });
         await executor?.dispose();
       } catch {
       }
-      try {
-        await activeBridge?.dispose();
-      } catch {
-      }
       try {
         await modelManager.dispose();
       } catch {