@kubohiroya/turbowarp-camera-source 0.11.0 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.ja.md CHANGED
@@ -26,13 +26,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
26
26
  次のURLをunsandboxed custom extensionとして読み込みます。
27
27
 
28
28
  ```text
29
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.11.0/dist/camera-source.js
29
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.13.0/dist/camera-source.js
30
30
  ```
31
31
 
32
32
  npm hostでは次を使います。
33
33
 
34
34
  ```bash
35
- pnpm add @kubohiroya/turbowarp-camera-source@0.11.0
35
+ pnpm add @kubohiroya/turbowarp-camera-source@0.13.0
36
36
  ```
37
37
 
38
38
  ## Quick start
@@ -55,6 +55,7 @@ stop shared camera [pose]
55
55
  <!-- BEGIN GENERATED BLOCKS -->
56
56
 
57
57
  - `start shared camera [CAMERA_ID] with device ID [DEVICE_ID]`: 名前付き共有カメラを開始します。
58
+ - `start shared camera [CAMERA_ID] requested by this page`: ページのクエリパラメータが指定するカメラを開始します。`cameraDeviceId`は厳密に、`cameraWidth`・`cameraHeight`・`cameraFrameRate`は理想値として要求します。指定がなければ、デバイスを指定しない`start shared camera`と同じです。複数のUSBカメラを使うアプリが、同じoriginの別のアプリ(レンズ校正アプリなど)をそのうち1台のために開くときの指定方法です。
58
59
  - `stop shared camera [CAMERA_ID]`: 名前付き共有カメラを停止し、MediaStreamTrackを解放します。
59
60
  - `shared camera [CAMERA_ID] is running?`: 指定した共有カメラが起動中かを返します。
60
61
  - `shared camera [CAMERA_ID] error code`: 直近の開始失敗コードを返します。開始成功後は空文字列です。
@@ -85,6 +86,9 @@ stop shared camera [pose]
85
86
  - `camera conditions generation for [CAMERA_ID]`: 幾何に影響する条件が変わるたびに増える整数を返します。frame rateでは動きません。
86
87
  - `save camera profile for [CAMERA_ID] to browser storage`: そのカメラに登録済みのプロファイルを、このoriginのブラウザストレージ(IndexedDB)へ保存し、同じoriginの他ウィンドウへ通知します。書き込み完了まで待ちます。結果は`saved`/`no-profile`/`unavailable`です。
87
88
  - `restore stored camera profile for [CAMERA_ID]`: 保存済みプロファイルのうち、現在のカメラ構成に`compatible`な最新のものをこのcamera IDで登録します。fail closedで、`undetermined`や`incompatible`なものは登録せず、既存の登録にも触れません。結果は`restored`/`none`/`incompatible`/`undetermined`/`unavailable`です。
89
+ - `restore stored camera profile calibrated on the device of [CAMERA_ID]`: `restore stored camera profile`と同じですが、このカメラが今動いているデバイスで校正したプロファイルだけを候補にします。同じ型番のカメラどうしで校正を取り違えません。device IDはブラウザストレージと同じくoriginとブラウザのプロファイルごとです。カメラが動いていなければ`undetermined`、`none`のときは他のデバイスのプロファイルが何件あったかを説明に含めます。
90
+ - `bind camera profile for [CAMERA_ID] to its current device`: 登録済みプロファイルに、このカメラが今動いているデバイスを記録します。ファイルから読んだものなど、操作者がこのカメラのものと指定したプロファイルのためのブロックです。現在のカメラに`compatible`なときだけ記録し、そうでなければ何も変えません。記録を残すには続けて保存します。
91
+ - `camera profile for [CAMERA_ID] belongs to its current device?`: 登録済みプロファイルが、このカメラが今動いているデバイスで校正された、または結び付けられたものかを返します。
88
92
  - `stored camera profile result for [CAMERA_ID]`: そのcamera IDで直近に行った保存または復元の結果を返します。未実行なら空文字列です。
89
93
  - `stored camera profile detail for [CAMERA_ID]`: 直近の保存・復元の説明を返します。保存・復元したプロファイル、またはどの保存済みプロファイルもカメラに適合しない理由です。
90
94
  - `stored camera profiles generation`: このウィンドウまたは同じoriginの他ウィンドウでプロファイルが保存されるたびに増えるカウンタです。プロジェクトを読み込んでも戻りません。
@@ -155,6 +159,8 @@ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いま
155
159
 
156
160
  **プロファイルを自分でスケールせず、内部行列を要求すること。** 校正時と違う解像度で届く場合、必要な計算は何が起きたかで変わる。単純な縮小なら`fx`・`fy`・`cx`・`cy`を比率倍するが、cropなら焦点距離はそのままで主点が平行移動する。trackの`resizeMode`を見られるのは本拡張だけなので、両者を区別できるのも本拡張だけ。**利用側がそれぞれ推測すれば、それぞれ違う推測をし、同じカメラがどの拡張から尋ねたかで違う幾何を返すことになる。** `camera intrinsics JSON`は現在のフレームに適合済みの数値を返し、cropやアスペクト変更で主点を置けない場合は空文字を返す。
157
161
 
162
+ **フレームの撮影時刻。** カメラがフレームを1枚表示すると、`getFrameSource()`は`frameTime`も返す。`timestampUs`はページの単調時計でのUnix epochからのマイクロ秒、`source`は`requestVideoFrameCallback`の`captureTime`なら`capture`、ブラウザが`presentationTime`しか報告しなければ`presentation`、`presentedFrames`は使用済みのフレームと新しいフレームを見分けるための番号である。同じページのカメラは同じ時計を共有するので、フレームをそのまま並べて比べられる。ほかのコンピュータとは同期していない。時刻はframe sourceを取得した時点で表示されていたフレームのものなので、画素を読む直前に取得すること。無いときは不明であって、現在時刻ではない。
163
+
158
164
  **previewを反転してもフレームは反転しない。** `getFrameSource()`が返す`previewFlip`は、stageがどう描いているかだけを述べる。その裏にあるフレームは常にカメラが撮ったままである。**反転表示したpreview上で拾った座標は、これらのフレームと使う前に戻さなければならない** — previewの座標をそのままsolveへ渡すと、左右反転した姿勢に収束し、しかも再投影誤差は小さいまま出る。`horizontal`は左右反転で、`cv::flip(…, 1)`・ffmpegの`hflip`・CSSの`scaleX(-1)`と同じ。回転は別の関心事なので、同じenumには混ぜていない。
159
165
 
160
166
  **契約は書き写さずimportすること。** 型は専用のentry pointとして公開してある。
@@ -243,8 +249,14 @@ if <(stored camera profile result for [pose]) = [restored]> then
243
249
 
244
250
  `register camera profile [PROFILE_JSON] as [CAMERA_ID]`はファイルに対して同じ付け替えを行う。校正アプリで`default`として解いたプロファイルを、利用側の作品で`pose`として登録できる。
245
251
 
252
+ **同じ型番のカメラが2台あるとき。** 2台は同じラベルと同じ撮影条件を報告するので、互いの校正が`compatible`になる。`restore stored camera profile calibrated on the device of [CAMERA_ID]`は、記録されたdevice IDがそのカメラの今のデバイスと一致するプロファイルだけを候補にする。device IDはoriginとブラウザのプロファイルごとで、ストレージの範囲と同じである。ファイルから持ち込んだプロファイルには別のブラウザのdevice IDが入っているので、カメラに登録し、`bind camera profile for [CAMERA_ID] to its current device`(`compatible`なものだけを結び付ける)を実行してから保存する。1台のカメラのために校正アプリを開くアプリは、クエリパラメータ`cameraDeviceId`・`cameraWidth`・`cameraHeight`・`cameraFrameRate`でカメラと使うサイズを指定し、`start shared camera [CAMERA_ID] requested by this page`がそのとおりに開始する。
253
+
246
254
  ## 互換性
247
255
 
256
+ 0.13.0ではframe sourceに`frameTime`を追加した。既存のブロックとcapabilityのメンバーは変えていない。
257
+
258
+ 0.12.0ではデバイスで絞る復元、`bind camera profile ... to its current device`、`camera profile ... belongs to its current device?`、`start shared camera [CAMERA_ID] requested by this page`を追加した。既存のブロックとcapabilityのメンバーは変えていない。
259
+
248
260
  0.11.0では校正ファイルをROSの`camera_info` YAMLで読み書きするようにした。`camera profile YAML`を追加し、`register camera profile`はYAMLも受け取り、処理だけを持つ関数を`./profile`で公開した。既存のブロックとcapabilityのメンバーは変えていない。
249
261
 
250
262
  0.10.0では校正プロファイルのブラウザストレージと`register camera profile as`を追加した。既存のブロックとcapabilityのメンバーは変えていない。
package/README.md CHANGED
@@ -30,13 +30,13 @@ or select separate cameras for separate roles.
30
30
  Load this URL as an unsandboxed custom extension:
31
31
 
32
32
  ```text
33
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.11.0/dist/camera-source.js
33
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.13.0/dist/camera-source.js
34
34
  ```
35
35
 
36
36
  For npm hosts:
37
37
 
38
38
  ```bash
39
- pnpm add @kubohiroya/turbowarp-camera-source@0.11.0
39
+ pnpm add @kubohiroya/turbowarp-camera-source@0.13.0
40
40
  ```
41
41
 
42
42
  ## Quick Start
@@ -69,6 +69,16 @@ Starts or keeps a named shared MediaDevices camera stream.
69
69
  | `CAMERA_ID` | String, default: `default` |
70
70
  | `DEVICE_ID` | String, default: `` |
71
71
 
72
+ ### `start shared camera [CAMERA_ID] requested by this page`
73
+
74
+ Starts the camera the page's query parameters name: `cameraDeviceId` exactly, and `cameraWidth`, `cameraHeight` and `cameraFrameRate` as ideal values. Without them it behaves like `start shared camera` with no device. An app with several USB cameras opens another app on the same origin, such as the lens calibration app, for one of them this way.
75
+
76
+ | Property | Value |
77
+ |---|---|
78
+ | Type | Command |
79
+ | Opcode | `startRequestedSharedCamera` |
80
+ | `CAMERA_ID` | String, default: `default` |
81
+
72
82
  ### `stop shared camera [CAMERA_ID]`
73
83
 
74
84
  Stops a named shared camera stream and releases its tracks.
@@ -367,6 +377,36 @@ Registers the newest saved profile that is `compatible` with the camera as confi
367
377
  | Opcode | `restoreStoredCameraProfile` |
368
378
  | `CAMERA_ID` | String, default: `default` |
369
379
 
380
+ ### `restore stored camera profile calibrated on the device of [CAMERA_ID]`
381
+
382
+ Like `restore stored camera profile`, but considers only profiles calibrated on the device this camera is running on now, so two cameras of the same model never receive each other's calibration. Device IDs are scoped to the origin and browser profile, as browser storage is. A camera that is not running gives `undetermined`; `none` says how many saved profiles belong to other devices.
383
+
384
+ | Property | Value |
385
+ |---|---|
386
+ | Type | Command |
387
+ | Opcode | `restoreStoredCameraProfileForDevice` |
388
+ | `CAMERA_ID` | String, default: `default` |
389
+
390
+ ### `bind camera profile for [CAMERA_ID] to its current device`
391
+
392
+ Records the device this camera is running on in its registered profile, for a profile the operator has assigned to this camera, such as one loaded from a file. Only a profile `compatible` with the camera as it is now is bound; otherwise nothing changes. Save the profile afterwards to keep the binding.
393
+
394
+ | Property | Value |
395
+ |---|---|
396
+ | Type | Command |
397
+ | Opcode | `bindCameraProfileToDevice` |
398
+ | `CAMERA_ID` | String, default: `default` |
399
+
400
+ ### `camera profile for [CAMERA_ID] belongs to its current device?`
401
+
402
+ Whether the registered profile was calibrated on, or bound to, the device this camera is running on now.
403
+
404
+ | Property | Value |
405
+ |---|---|
406
+ | Type | Boolean |
407
+ | Opcode | `cameraProfileOnDevice` |
408
+ | `CAMERA_ID` | String, default: `default` |
409
+
370
410
  ### `stored camera profile result for [CAMERA_ID]`
371
411
 
372
412
  Returns the outcome of the last save or restore for the camera ID, or an empty string before either has run.
@@ -475,6 +515,8 @@ const assessment = calibration.assessProfile("pose");
475
515
 
476
516
  **Ask for intrinsics rather than scaling a profile yourself.** When the camera delivers a size the calibration was not solved at, the arithmetic depends on what happened: a pure downscale multiplies `fx`, `fy`, `cx` and `cy`, while a crop leaves the focal lengths alone and shifts the principal point instead. Only this extension sees the track's `resizeMode`, so only it can tell the two apart — and if every consumer guesses, they guess differently and the same camera yields different geometry depending on which extension asked. `camera intrinsics JSON` returns numbers already adapted to the current frame, or an empty string when the difference is a crop or an aspect change and the principal point cannot be placed.
477
517
 
518
+ **Frame capture time.** `getFrameSource()` also carries `frameTime` once the camera has presented a frame: `timestampUs` in microseconds since the Unix epoch on the page's monotonic clock, `source` (`capture` for `captureTime` from `requestVideoFrameCallback`, `presentation` where the browser reports only `presentationTime`), and `presentedFrames`, which tells a frame already used from a new one. Cameras on one page share that clock, so their frames can be lined up directly; it is not synchronized with any other computer. Take the frame source immediately before reading pixels: the time belongs to the frame presented when the source was taken. Absent means unknown, never now.
519
+
478
520
  **Turning the preview over does not turn the frames over.** `getFrameSource()` reports `previewFlip`, which describes how the stage is drawing the image and nothing else; the frames behind it are always the ones the camera captured. Coordinates picked off a flipped preview must be turned back before they are used with these frames — a solve fed the preview's coordinates converges on a left-right reflected pose and reports a small reprojection error while doing it. `horizontal` is the left-right mirror, matching `cv::flip(…, 1)`, ffmpeg's `hflip` and CSS `scaleX(-1)`; rotation is a separate concern and is deliberately not folded into the same enum.
479
521
 
480
522
  **Import these declarations rather than re-writing them.** The contract is published as its own
@@ -597,8 +639,24 @@ camera is left as it was. The result is `restored`, `none`, `incompatible`, `und
597
639
  `register camera profile [PROFILE_JSON] as [CAMERA_ID]` does the same rebinding for a file: a profile
598
640
  solved under `default` in a calibration app can be registered as `pose` in the app that uses it.
599
641
 
642
+ **Two cameras of the same model.** They report the same label and the same capture conditions, so
643
+ each one's calibration is `compatible` with the other. `restore stored camera profile calibrated on
644
+ the device of [CAMERA_ID]` only considers profiles whose recorded device ID is the one the camera is
645
+ running on. Device IDs are scoped to the origin and browser profile, which is also where the storage
646
+ lives. A profile brought in from a file carries another browser's device ID: register it for the
647
+ camera, `bind camera profile for [CAMERA_ID] to its current device` (which binds only a `compatible`
648
+ profile), and save it. An app that opens the calibration app for one camera names that camera and the size it uses in the
649
+ `cameraDeviceId`, `cameraWidth`, `cameraHeight` and `cameraFrameRate` query parameters, and
650
+ `start shared camera [CAMERA_ID] requested by this page` starts it that way.
651
+
600
652
  ## Compatibility
601
653
 
654
+ Version 0.13.0 adds `frameTime` to the frame source. No existing block or capability member changes.
655
+
656
+ Version 0.12.0 adds device-scoped restore, `bind camera profile ... to its current device`,
657
+ `camera profile ... belongs to its current device?` and `start shared camera [CAMERA_ID] requested by this page`.
658
+ No existing block or capability member changes.
659
+
602
660
  Version 0.11.0 writes and reads calibration files as ROS `camera_info` YAML: `camera profile YAML`
603
661
  is new, `register camera profile` also accepts YAML, and the pure functions are published under
604
662
  `./profile`. No existing block or capability member changes.
@@ -35,6 +35,15 @@
35
35
  }
36
36
  }
37
37
  },
38
+ {
39
+ "opcode": "startRequestedSharedCamera",
40
+ "blockType": "COMMAND",
41
+ "text": "start shared camera [CAMERA_ID] requested by this page",
42
+ "arguments": { "CAMERA_ID": {
43
+ "type": "STRING",
44
+ "defaultValue": "default"
45
+ } }
46
+ },
38
47
  {
39
48
  "opcode": "stopSharedCamera",
40
49
  "blockType": "COMMAND",
@@ -306,6 +315,33 @@
306
315
  "defaultValue": "default"
307
316
  } }
308
317
  },
318
+ {
319
+ "opcode": "restoreStoredCameraProfileForDevice",
320
+ "blockType": "COMMAND",
321
+ "text": "restore stored camera profile calibrated on the device of [CAMERA_ID]",
322
+ "arguments": { "CAMERA_ID": {
323
+ "type": "STRING",
324
+ "defaultValue": "default"
325
+ } }
326
+ },
327
+ {
328
+ "opcode": "bindCameraProfileToDevice",
329
+ "blockType": "COMMAND",
330
+ "text": "bind camera profile for [CAMERA_ID] to its current device",
331
+ "arguments": { "CAMERA_ID": {
332
+ "type": "STRING",
333
+ "defaultValue": "default"
334
+ } }
335
+ },
336
+ {
337
+ "opcode": "cameraProfileOnDevice",
338
+ "blockType": "BOOLEAN",
339
+ "text": "camera profile for [CAMERA_ID] belongs to its current device?",
340
+ "arguments": { "CAMERA_ID": {
341
+ "type": "STRING",
342
+ "defaultValue": "default"
343
+ } }
344
+ },
309
345
  {
310
346
  "opcode": "storedCameraProfileResult",
311
347
  "blockType": "REPORTER",
@@ -2200,6 +2236,55 @@
2200
2236
  });
2201
2237
  }
2202
2238
  //#endregion
2239
+ //#region src/frame-time.ts
2240
+ function finite(value) {
2241
+ return typeof value === "number" && Number.isFinite(value) ? value : void 0;
2242
+ }
2243
+ /** The time of one presented frame, or the previous one when the metadata carries no time. */
2244
+ function frameTimeFrom(metadata, timeOriginMs, previous) {
2245
+ const capture = finite(metadata.captureTime);
2246
+ const presentation = finite(metadata.presentationTime);
2247
+ const milliseconds = capture ?? presentation;
2248
+ if (milliseconds === void 0) return previous;
2249
+ return Object.freeze({
2250
+ timestampUs: Math.round((timeOriginMs + milliseconds) * 1e3),
2251
+ source: capture === void 0 ? "presentation" : "capture",
2252
+ presentedFrames: finite(metadata.presentedFrames) ?? (previous?.presentedFrames ?? 0) + 1
2253
+ });
2254
+ }
2255
+ /**
2256
+ * Follows one video element for as long as its camera runs.
2257
+ *
2258
+ * Nothing is recorded where the browser has no `requestVideoFrameCallback`; frame sources then carry
2259
+ * no time, which a consumer has to treat as unknown rather than as now.
2260
+ */
2261
+ var FrameTimeWatcher = class {
2262
+ constructor(element, timeOriginMs) {
2263
+ this.disposed = false;
2264
+ this.element = element;
2265
+ this.timeOriginMs = timeOriginMs;
2266
+ this.schedule();
2267
+ }
2268
+ latest() {
2269
+ return this.frame;
2270
+ }
2271
+ dispose() {
2272
+ this.disposed = true;
2273
+ if (this.handle !== void 0) {
2274
+ this.element.cancelVideoFrameCallback(this.handle);
2275
+ this.handle = void 0;
2276
+ }
2277
+ }
2278
+ schedule() {
2279
+ if (this.disposed || typeof this.element.requestVideoFrameCallback !== "function") return;
2280
+ this.handle = this.element.requestVideoFrameCallback((_now, metadata) => {
2281
+ this.handle = void 0;
2282
+ this.frame = frameTimeFrom(metadata, this.timeOriginMs, this.frame);
2283
+ this.schedule();
2284
+ });
2285
+ }
2286
+ };
2287
+ //#endregion
2203
2288
  //#region src/extension.ts
2204
2289
  var blockDefinitions = block_definitions_default.blocks;
2205
2290
  var menuDefinitions = block_definitions_default.menus;
@@ -2212,6 +2297,38 @@
2212
2297
  function normalizeId(value, fallback = defaultCameraId) {
2213
2298
  return String(value ?? "").trim() || fallback;
2214
2299
  }
2300
+ /**
2301
+ * The camera a page was opened for, from its query parameters.
2302
+ *
2303
+ * Returns only what was named, so a page opened without parameters starts a camera exactly as the
2304
+ * plain block does. A size or rate that is not a positive number is ignored rather than guessed.
2305
+ */
2306
+ function requestedCamera(search) {
2307
+ let parameters;
2308
+ try {
2309
+ parameters = new URLSearchParams(search);
2310
+ } catch {
2311
+ return {};
2312
+ }
2313
+ const deviceId = optionalText(parameters.get("cameraDeviceId"));
2314
+ const video = {};
2315
+ if (deviceId) video.deviceId = { exact: deviceId };
2316
+ const ideal = (name) => {
2317
+ const value = Number(parameters.get(name) ?? "");
2318
+ return Number.isFinite(value) && value > 0 ? value : void 0;
2319
+ };
2320
+ const width = ideal("cameraWidth");
2321
+ const height = ideal("cameraHeight");
2322
+ const frameRate = ideal("cameraFrameRate");
2323
+ if (width !== void 0) video.width = { ideal: width };
2324
+ if (height !== void 0) video.height = { ideal: height };
2325
+ if (frameRate !== void 0) video.frameRate = { ideal: frameRate };
2326
+ if (Object.keys(video).length === 0) return {};
2327
+ return {
2328
+ ...deviceId ? { deviceId } : {},
2329
+ video
2330
+ };
2331
+ }
2215
2332
  function optionalText(value) {
2216
2333
  return String(value ?? "").trim();
2217
2334
  }
@@ -2556,7 +2673,77 @@
2556
2673
  * so trying a candidate that loses never displaces the registered one even for a moment.
2557
2674
  */
2558
2675
  async restoreStoredCameraProfile(args = {}) {
2676
+ await this.restoreStoredProfile(normalizeId(args.CAMERA_ID), false);
2677
+ }
2678
+ /**
2679
+ * Like `restore stored camera profile`, but only from profiles calibrated on the device the camera
2680
+ * is running on now.
2681
+ *
2682
+ * Compatibility alone cannot tell two cameras of the same model apart: they report the same label
2683
+ * and the same capture conditions, so either one's calibration reads as `compatible` for the other.
2684
+ * What differs is the device id, which is scoped to this origin and browser profile. That scope is
2685
+ * exactly where browser storage lives, so a profile saved here and a camera opened here agree on it
2686
+ * for as long as the browser keeps both.
2687
+ *
2688
+ * A camera that is not running has no device id, and the answer is then `undetermined`. A profile
2689
+ * brought in from another browser carries another id and is never a candidate; register it with
2690
+ * the file and `bind camera profile ... to its current device` first.
2691
+ */
2692
+ async restoreStoredCameraProfileForDevice(args = {}) {
2693
+ await this.restoreStoredProfile(normalizeId(args.CAMERA_ID), true);
2694
+ }
2695
+ /**
2696
+ * Records the device the camera is running on in its registered profile.
2697
+ *
2698
+ * For the case where the operator has said which camera a profile belongs to, typically by loading
2699
+ * a file for it. Only a profile `compatible` with the camera as it is now is bound: a profile that
2700
+ * does not fit is not made more trustworthy by naming a device in it. A label the camera reports is
2701
+ * written too when the profile has none; one it already has has been compared and matched.
2702
+ */
2703
+ bindCameraProfileToDevice(args = {}) {
2704
+ const cameraId = normalizeId(args.CAMERA_ID);
2705
+ const profile = this.profiles.get(cameraId);
2706
+ if (!profile) return;
2707
+ const conditions = this.conditionsOf(cameraId);
2708
+ if (conditions.deviceId.length === 0) return;
2709
+ if (evaluateProfileCompatibility(profile, conditions).state !== "compatible") return;
2710
+ const document = JSON.parse(serializeCameraIntrinsicProfile(profile));
2711
+ const label = profile.device?.label ?? conditions.label;
2712
+ document["device"] = {
2713
+ ...label === void 0 ? {} : { label },
2714
+ deviceId: conditions.deviceId
2715
+ };
2716
+ this.profiles.register(document);
2717
+ }
2718
+ /** Whether the registered profile was calibrated on, or bound to, the device the camera runs on. */
2719
+ cameraProfileOnDevice(args = {}) {
2559
2720
  const cameraId = normalizeId(args.CAMERA_ID);
2721
+ const deviceId = this.profiles.get(cameraId)?.device?.deviceId;
2722
+ return deviceId !== void 0 && deviceId.length > 0 && deviceId === this.conditionsOf(cameraId).deviceId;
2723
+ }
2724
+ /**
2725
+ * Starts a camera the way the page's query parameters ask, or as `start shared camera` does when
2726
+ * they ask nothing.
2727
+ *
2728
+ * How one app hands a camera to another it opens on the same origin. An app running several USB
2729
+ * cameras opens the lens calibration app for one of them, and the calibration has to be solved on
2730
+ * that device at the size the app will use: a camera the browser picks may be the other camera of
2731
+ * the same model, and a profile solved at another size is `incompatible`.
2732
+ *
2733
+ * `cameraDeviceId` is required exactly. `cameraWidth`, `cameraHeight` and `cameraFrameRate` are
2734
+ * asked for as `ideal`, the way the opening app asked, so the camera settles on the same mode.
2735
+ */
2736
+ async startRequestedSharedCamera(args = {}) {
2737
+ const cameraId = normalizeId(args.CAMERA_ID);
2738
+ if (this.blockLeases.has(cameraId)) return;
2739
+ const lease = await this.acquireCamera({
2740
+ owner: "camera-source-block",
2741
+ cameraId,
2742
+ ...requestedCamera(globalThis.location?.search ?? "")
2743
+ });
2744
+ this.blockLeases.set(cameraId, lease);
2745
+ }
2746
+ async restoreStoredProfile(cameraId, sameDevice) {
2560
2747
  let records;
2561
2748
  try {
2562
2749
  records = await this.profileStore.list();
@@ -2568,14 +2755,26 @@
2568
2755
  return;
2569
2756
  }
2570
2757
  const conditions = this.conditionsOf(cameraId);
2758
+ if (sameDevice && conditions.deviceId.length === 0) {
2759
+ this.storedProfileOutcomes.set(cameraId, {
2760
+ state: "undetermined",
2761
+ detail: "The camera is not running, so the device a saved profile was calibrated on cannot be compared."
2762
+ });
2763
+ return;
2764
+ }
2571
2765
  let incompatible;
2572
2766
  let undetermined;
2767
+ let otherDevices = 0;
2573
2768
  for (const record of [...records].sort(newestFirst)) {
2574
2769
  const parsed = parseJson(record.document);
2575
2770
  if (!parsed.ok) continue;
2576
2771
  const document = rebindCameraId(parsed.value, cameraId);
2577
2772
  const result = readCameraProfileDocument(document);
2578
2773
  if (!result.ok) continue;
2774
+ if (sameDevice && result.profile.device?.deviceId !== conditions.deviceId) {
2775
+ otherDevices += 1;
2776
+ continue;
2777
+ }
2579
2778
  const report = evaluateProfileCompatibility(result.profile, conditions);
2580
2779
  if (report.state === "compatible") {
2581
2780
  this.profiles.register(document);
@@ -2591,7 +2790,7 @@
2591
2790
  const decisive = incompatible ?? undetermined;
2592
2791
  this.storedProfileOutcomes.set(cameraId, decisive === void 0 ? {
2593
2792
  state: "none",
2594
- detail: ""
2793
+ detail: otherDevices === 0 ? "" : `${otherDevices} saved profile(s) were calibrated on other devices and were not considered.`
2595
2794
  } : {
2596
2795
  state: decisive.state,
2597
2796
  detail: compatibilityDetail(decisive)
@@ -2809,7 +3008,8 @@
2809
3008
  video: null,
2810
3009
  preview: null,
2811
3010
  startPromise: null,
2812
- activeDeviceId: ""
3011
+ activeDeviceId: "",
3012
+ frameTimes: null
2813
3013
  };
2814
3014
  this.sessions.set(cameraId, session);
2815
3015
  return session;
@@ -2828,6 +3028,7 @@
2828
3028
  if (!session.active) throw new Error("Camera acquisition was cancelled.");
2829
3029
  session.stream = stream;
2830
3030
  session.video = video;
3031
+ session.frameTimes = new FrameTimeWatcher(video, globalThis.performance?.timeOrigin ?? 0);
2831
3032
  this.watchStreamEnd(session, stream);
2832
3033
  this.updateActiveDevice(session);
2833
3034
  this.cameraFailures.delete(session.cameraId);
@@ -2885,13 +3086,15 @@
2885
3086
  }
2886
3087
  getFrameSource(session) {
2887
3088
  if (!session.video || !session.stream) throw new Error("Shared camera is not running.");
3089
+ const frameTime = session.frameTimes?.latest();
2888
3090
  return Object.freeze({
2889
3091
  kind: "video",
2890
3092
  element: session.video,
2891
3093
  width: session.video.videoWidth,
2892
3094
  height: session.video.videoHeight,
2893
3095
  previewFlip: this.previewFlip(session),
2894
- deviceId: session.activeDeviceId
3096
+ deviceId: session.activeDeviceId,
3097
+ ...frameTime === void 0 ? {} : { frameTime }
2895
3098
  });
2896
3099
  }
2897
3100
  stopWhenUnused(session) {
@@ -2904,6 +3107,8 @@
2904
3107
  session.active = false;
2905
3108
  this.nextPreviewBlockRevision(cameraId);
2906
3109
  session.preview?.dispose();
3110
+ session.frameTimes?.dispose();
3111
+ session.frameTimes = null;
2907
3112
  session.stream?.getTracks().forEach((track) => track.stop());
2908
3113
  if (session.video) session.video.srcObject = null;
2909
3114
  session.stream = null;
@@ -2,6 +2,16 @@
2
2
  "formatVersion": 1,
3
3
  "id": "kubohiroyacamerasource",
4
4
  "blocks": [
5
+ {
6
+ "opcode": "bindCameraProfileToDevice",
7
+ "blockType": "COMMAND",
8
+ "arguments": [
9
+ {
10
+ "id": "CAMERA_ID",
11
+ "type": "STRING"
12
+ }
13
+ ]
14
+ },
5
15
  {
6
16
  "opcode": "cameraConditionsGeneration",
7
17
  "blockType": "REPORTER",
@@ -167,6 +177,16 @@
167
177
  }
168
178
  ]
169
179
  },
180
+ {
181
+ "opcode": "cameraProfileOnDevice",
182
+ "blockType": "BOOLEAN",
183
+ "arguments": [
184
+ {
185
+ "id": "CAMERA_ID",
186
+ "type": "STRING"
187
+ }
188
+ ]
189
+ },
170
190
  {
171
191
  "opcode": "cameraProfileRegistered",
172
192
  "blockType": "BOOLEAN",
@@ -256,6 +276,16 @@
256
276
  }
257
277
  ]
258
278
  },
279
+ {
280
+ "opcode": "restoreStoredCameraProfileForDevice",
281
+ "blockType": "COMMAND",
282
+ "arguments": [
283
+ {
284
+ "id": "CAMERA_ID",
285
+ "type": "STRING"
286
+ }
287
+ ]
288
+ },
259
289
  {
260
290
  "opcode": "saveCameraProfile",
261
291
  "blockType": "COMMAND",
@@ -281,6 +311,16 @@
281
311
  }
282
312
  ]
283
313
  },
314
+ {
315
+ "opcode": "startRequestedSharedCamera",
316
+ "blockType": "COMMAND",
317
+ "arguments": [
318
+ {
319
+ "id": "CAMERA_ID",
320
+ "type": "STRING"
321
+ }
322
+ ]
323
+ },
284
324
  {
285
325
  "opcode": "startSharedCamera",
286
326
  "blockType": "COMMAND",
package/dist/runtime.d.ts CHANGED
@@ -66,6 +66,33 @@ export interface CameraFrameSource {
66
66
  */
67
67
  readonly previewFlip: Flip;
68
68
  readonly deviceId: string;
69
+ /**
70
+ * When the frame the element is showing was captured, as of the moment this source was taken.
71
+ *
72
+ * Absent until the camera has presented a frame, and where the browser has no
73
+ * `requestVideoFrameCallback`; absent means unknown, never now. The element can present a newer
74
+ * frame between taking the source and reading pixels from it, so the time belongs to the frame
75
+ * presented when the source was taken — take the source immediately before reading pixels.
76
+ */
77
+ readonly frameTime?: CameraFrameTime;
78
+ }
79
+ /**
80
+ * The capture time of a presented frame.
81
+ *
82
+ * `timestampUs` is microseconds since the Unix epoch on the page's monotonic clock
83
+ * (`performance.timeOrigin + time`). Cameras on one page share that clock, so their frames can be
84
+ * compared directly; it is not synchronized with any other computer.
85
+ */
86
+ export interface CameraFrameTime {
87
+ readonly timestampUs: number;
88
+ /**
89
+ * `capture` is when the frame left the device (`captureTime`). `presentation` is when the browser
90
+ * presented it (`presentationTime`), later by the capture and decode pipeline, and used only where
91
+ * the browser reports no capture time.
92
+ */
93
+ readonly source: 'capture' | 'presentation';
94
+ /** Increases with every presented frame, so a frame already used can be recognized. */
95
+ readonly presentedFrames: number;
69
96
  }
70
97
  export interface CameraLease {
71
98
  getFrameSource(): CameraFrameSource;
@@ -1 +1 @@
1
- {"version":3,"file":"runtime.d.ts","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,eAAO,MAAM,uBAAuB,2BAA2B,CAAC;AAEhE;;;;;GAKG;AACH,eAAO,MAAM,sBAAsB,+BAA+B,CAAC;AAEnE;;;;;;;GAOG;AACH,eAAO,MAAM,yBAAyB,qCAAqC,CAAC;AAE5E,wFAAwF;AACxF,eAAO,MAAM,6BAA6B,EAAG,CAAU,CAAC;AAExD;;;;;;GAMG;AACH,MAAM,MAAM,IAAI,GAAG,MAAM,GAAG,YAAY,GAAG,UAAU,GAAG,MAAM,CAAC;AAE/D,MAAM,WAAW,oBAAoB;IACnC,0FAA0F;IAC1F,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,iGAAiG;IACjG,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,qBAAqB,GAAG,OAAO,CAAC;IACxC,gGAAgG;IAChG,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,gBAAgB,CAAC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,WAAW;IAC1B,cAAc,IAAI,iBAAiB,CAAC;IACpC,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAED;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,aAAa,CAAC,OAAO,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;CACrE;AAED;;;;;;;GAOG;AACH,YAAY,EACV,iBAAiB,EACjB,wBAAwB,EACzB,MAAM,yBAAyB,CAAC;AAEjC,YAAY,EACV,kBAAkB,EAClB,qBAAqB,EACrB,gBAAgB,EAChB,kBAAkB,EAClB,gBAAgB,EAChB,wBAAwB,EACxB,gBAAgB,EAChB,WAAW,EACX,eAAe,EACf,YAAY,EACZ,gBAAgB,EAChB,aAAa,EACd,MAAM,wBAAwB,CAAC;AAEhC,YAAY,EACV,cAAc,EACd,eAAe,EACf,iBAAiB,EACjB,gBAAgB,EACjB,MAAM,6BAA6B,CAAC;AAErC,YAAY,EACV,iBAAiB,EACjB,oBAAoB,EACpB,mBAAmB,EACnB,kBAAkB,EAClB,YAAY,EACb,MAAM,gCAAgC,CAAC;AAExC,YAAY,EAAC,gBAAgB,EAAC,MAAM,6BAA6B,CAAC;AAElE,OAAO,KAAK,EAAC,wBAAwB,EAAC,MAAM,yBAAyB,CAAC;AAEtE,oGAAoG;AACpG,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,OAAO,GAAG,mBAAmB,GAAG,SAAS,CAMzF;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,0BAA0B,CACxC,OAAO,EAAE,OAAO,GACf,wBAAwB,GAAG,SAAS,CAMtC"}
1
+ {"version":3,"file":"runtime.d.ts","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,eAAO,MAAM,uBAAuB,2BAA2B,CAAC;AAEhE;;;;;GAKG;AACH,eAAO,MAAM,sBAAsB,+BAA+B,CAAC;AAEnE;;;;;;;GAOG;AACH,eAAO,MAAM,yBAAyB,qCAAqC,CAAC;AAE5E,wFAAwF;AACxF,eAAO,MAAM,6BAA6B,EAAG,CAAU,CAAC;AAExD;;;;;;GAMG;AACH,MAAM,MAAM,IAAI,GAAG,MAAM,GAAG,YAAY,GAAG,UAAU,GAAG,MAAM,CAAC;AAE/D,MAAM,WAAW,oBAAoB;IACnC,0FAA0F;IAC1F,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,iGAAiG;IACjG,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,qBAAqB,GAAG,OAAO,CAAC;IACxC,gGAAgG;IAChG,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,gBAAgB,CAAC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B;;;;;;;OAOG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,eAAe,CAAC;CACtC;AAED;;;;;;GAMG;AACH,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B;;;;OAIG;IACH,QAAQ,CAAC,MAAM,EAAE,SAAS,GAAG,cAAc,CAAC;IAC5C,uFAAuF;IACvF,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;CAClC;AAED,MAAM,WAAW,WAAW;IAC1B,cAAc,IAAI,iBAAiB,CAAC;IACpC,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAED;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,aAAa,CAAC,OAAO,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;CACrE;AAED;;;;;;;GAOG;AACH,YAAY,EACV,iBAAiB,EACjB,wBAAwB,EACzB,MAAM,yBAAyB,CAAC;AAEjC,YAAY,EACV,kBAAkB,EAClB,qBAAqB,EACrB,gBAAgB,EAChB,kBAAkB,EAClB,gBAAgB,EAChB,wBAAwB,EACxB,gBAAgB,EAChB,WAAW,EACX,eAAe,EACf,YAAY,EACZ,gBAAgB,EAChB,aAAa,EACd,MAAM,wBAAwB,CAAC;AAEhC,YAAY,EACV,cAAc,EACd,eAAe,EACf,iBAAiB,EACjB,gBAAgB,EACjB,MAAM,6BAA6B,CAAC;AAErC,YAAY,EACV,iBAAiB,EACjB,oBAAoB,EACpB,mBAAmB,EACnB,kBAAkB,EAClB,YAAY,EACb,MAAM,gCAAgC,CAAC;AAExC,YAAY,EAAC,gBAAgB,EAAC,MAAM,6BAA6B,CAAC;AAElE,OAAO,KAAK,EAAC,wBAAwB,EAAC,MAAM,yBAAyB,CAAC;AAEtE,oGAAoG;AACpG,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,OAAO,GAAG,mBAAmB,GAAG,SAAS,CAMzF;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,0BAA0B,CACxC,OAAO,EAAE,OAAO,GACf,wBAAwB,GAAG,SAAS,CAMtC"}
@@ -1 +1 @@
1
- {"version":3,"file":"runtime.js","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,MAAM,CAAC,MAAM,uBAAuB,GAAG,wBAAwB,CAAC;AAEhE;;;;;GAKG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG,4BAA4B,CAAC;AAEnE;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,yBAAyB,GAAG,kCAAkC,CAAC;AAE5E,wFAAwF;AACxF,MAAM,CAAC,MAAM,6BAA6B,GAAG,CAAU,CAAC;AAwGxD,oGAAoG;AACpG,MAAM,UAAU,uBAAuB,CAAC,OAAgB;IACtD,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,sBAAsB,CAAC,CAAC;IAC/E,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,aAAa,EAAC,GAAG,SAAsC,CAAC;IAC/D,OAAO,OAAO,aAAa,KAAK,UAAU,CAAC,CAAC,CAAE,SAAiC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC9F,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,0BAA0B,CACxC,OAAgB;IAEhB,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,yBAAyB,CAAC,CAAC;IAClF,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,cAAc,EAAC,GAAG,SAAuC,CAAC;IACjE,OAAO,OAAO,cAAc,KAAK,UAAU,CAAC,CAAC,CAAE,SAAsC,CAAC,CAAC,CAAC,SAAS,CAAC;AACpG,CAAC"}
1
+ {"version":3,"file":"runtime.js","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,MAAM,CAAC,MAAM,uBAAuB,GAAG,wBAAwB,CAAC;AAEhE;;;;;GAKG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG,4BAA4B,CAAC;AAEnE;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,yBAAyB,GAAG,kCAAkC,CAAC;AAE5E,wFAAwF;AACxF,MAAM,CAAC,MAAM,6BAA6B,GAAG,CAAU,CAAC;AAoIxD,oGAAoG;AACpG,MAAM,UAAU,uBAAuB,CAAC,OAAgB;IACtD,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,sBAAsB,CAAC,CAAC;IAC/E,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,aAAa,EAAC,GAAG,SAAsC,CAAC;IAC/D,OAAO,OAAO,aAAa,KAAK,UAAU,CAAC,CAAC,CAAE,SAAiC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC9F,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,0BAA0B,CACxC,OAAgB;IAEhB,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,yBAAyB,CAAC,CAAC;IAClF,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,cAAc,EAAC,GAAG,SAAuC,CAAC;IACjE,OAAO,OAAO,cAAc,KAAK,UAAU,CAAC,CAAC,CAAE,SAAsC,CAAC,CAAC,CAAC,SAAS,CAAC;AACpG,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kubohiroya/turbowarp-camera-source",
3
- "version": "0.11.0",
3
+ "version": "0.13.0",
4
4
  "author": "Hiroya Kubo <hiroya@cuc.ac.jp>",
5
5
  "description": "A TurboWarp capability extension for sharing one MediaDevices camera stream across extensions.",
6
6
  "type": "module",