@kubohiroya/turbowarp-camera-source 0.12.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.ja.md CHANGED
@@ -26,13 +26,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
26
26
  次のURLをunsandboxed custom extensionとして読み込みます。
27
27
 
28
28
  ```text
29
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.12.0/dist/camera-source.js
29
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.14.0/dist/camera-source.js
30
30
  ```
31
31
 
32
32
  npm hostでは次を使います。
33
33
 
34
34
  ```bash
35
- pnpm add @kubohiroya/turbowarp-camera-source@0.12.0
35
+ pnpm add @kubohiroya/turbowarp-camera-source@0.14.0
36
36
  ```
37
37
 
38
38
  ## Quick start
@@ -122,6 +122,32 @@ await qrLease.release();
122
122
  await poseLease.release();
123
123
  ```
124
124
 
125
+ consumerが`MediaStream`を既に持っている場合は`adoptCamera()`を使います。`<usermedia>` capability
126
+ elementで取得したもの、別フレームから渡されたもの、テスト用の作りものでも同じです。Camera Sourceは
127
+ それを`cameraId`のもとで共有し、どのconsumerも同じ`getFrameSource()`から読みます。
128
+
129
+ **採用したstreamのtrackをCamera Sourceは止めません。** 最後のleaseを解放するとセッションは終わり、
130
+ streamはdetachされます。カメラを使い終えたときに止めるのは呼び出し側です。trackの停止は取り消せず、
131
+ 同じstreamを見ているほかのconsumer全員からフレームを奪うので、それが正しいときを知りうるのは
132
+ streamを開いた側だけです。自分で取得したstreamは従来どおりCamera Sourceが止めます。
133
+
134
+ ```js
135
+ if (typeof cameraSource.adoptCamera === "function") {
136
+ const hostLease = await cameraSource.adoptCamera({
137
+ owner: "app-shell",
138
+ cameraId: "pose",
139
+ stream: streamTheHostAlreadyHolds,
140
+ });
141
+ const frame = hostLease.getFrameSource();
142
+ await hostLease.release();
143
+ }
144
+ ```
145
+
146
+ このメソッドはfeature detectしてください。このsurfaceはversionではなく形で絞られるため、現行の宣言に
147
+ 対してコンパイルしたconsumerが、それより前のビルドに出会うことがあります。既に別のstreamが動いている
148
+ `cameraId`へ2つめを採用しようとすると拒否します。呼び出し側に他人のフレームを返すと、自分のstreamが
149
+ 使われていると誤解させるためです。同じstreamをもう一度採用した場合はそのセッションに参加します。
150
+
125
151
  `preview: true`を指定したleaseは、GPU-backed stage previewを利用します。専用video skinは共有
126
152
  `HTMLVideoElement`を`texImage2D(video)`でアップロードし、preview drawableのX scaleで左右反転します。
127
153
  `drawImage()`、`getImageData()`、CPUでのフレームピクセル走査は行いません。既定値は`false`で、同じ
@@ -159,6 +185,8 @@ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いま
159
185
 
160
186
  **プロファイルを自分でスケールせず、内部行列を要求すること。** 校正時と違う解像度で届く場合、必要な計算は何が起きたかで変わる。単純な縮小なら`fx`・`fy`・`cx`・`cy`を比率倍するが、cropなら焦点距離はそのままで主点が平行移動する。trackの`resizeMode`を見られるのは本拡張だけなので、両者を区別できるのも本拡張だけ。**利用側がそれぞれ推測すれば、それぞれ違う推測をし、同じカメラがどの拡張から尋ねたかで違う幾何を返すことになる。** `camera intrinsics JSON`は現在のフレームに適合済みの数値を返し、cropやアスペクト変更で主点を置けない場合は空文字を返す。
161
187
 
188
+ **フレームの撮影時刻。** カメラがフレームを1枚表示すると、`getFrameSource()`は`frameTime`も返す。`timestampUs`はページの単調時計でのUnix epochからのマイクロ秒、`source`は`requestVideoFrameCallback`の`captureTime`なら`capture`、ブラウザが`presentationTime`しか報告しなければ`presentation`、`presentedFrames`は使用済みのフレームと新しいフレームを見分けるための番号である。同じページのカメラは同じ時計を共有するので、フレームをそのまま並べて比べられる。ほかのコンピュータとは同期していない。時刻はframe sourceを取得した時点で表示されていたフレームのものなので、画素を読む直前に取得すること。無いときは不明であって、現在時刻ではない。
189
+
162
190
  **previewを反転してもフレームは反転しない。** `getFrameSource()`が返す`previewFlip`は、stageがどう描いているかだけを述べる。その裏にあるフレームは常にカメラが撮ったままである。**反転表示したpreview上で拾った座標は、これらのフレームと使う前に戻さなければならない** — previewの座標をそのままsolveへ渡すと、左右反転した姿勢に収束し、しかも再投影誤差は小さいまま出る。`horizontal`は左右反転で、`cv::flip(…, 1)`・ffmpegの`hflip`・CSSの`scaleX(-1)`と同じ。回転は別の関心事なので、同じenumには混ぜていない。
163
191
 
164
192
  **契約は書き写さずimportすること。** 型は専用のentry pointとして公開してある。
@@ -251,6 +279,12 @@ if <(stored camera profile result for [pose]) = [restored]> then
251
279
 
252
280
  ## 互換性
253
281
 
282
+ 0.14.0では呼び出し側が持っているstreamを共有する`adoptCamera`と、`./runtime`に`CameraAdoptOptions`・
283
+ `CameraLeaseOptions`の宣言を追加した。`CameraAcquireOptions`のメンバーは変えていない。既存のブロックと
284
+ capabilityのメンバーは変えていない。
285
+
286
+ 0.13.0ではframe sourceに`frameTime`を追加した。既存のブロックとcapabilityのメンバーは変えていない。
287
+
254
288
  0.12.0ではデバイスで絞る復元、`bind camera profile ... to its current device`、`camera profile ... belongs to its current device?`、`start shared camera [CAMERA_ID] requested by this page`を追加した。既存のブロックとcapabilityのメンバーは変えていない。
255
289
 
256
290
  0.11.0では校正ファイルをROSの`camera_info` YAMLで読み書きするようにした。`camera profile YAML`を追加し、`register camera profile`はYAMLも受け取り、処理だけを持つ関数を`./profile`で公開した。既存のブロックとcapabilityのメンバーは変えていない。
package/README.md CHANGED
@@ -30,13 +30,13 @@ or select separate cameras for separate roles.
30
30
  Load this URL as an unsandboxed custom extension:
31
31
 
32
32
  ```text
33
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.12.0/dist/camera-source.js
33
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.14.0/dist/camera-source.js
34
34
  ```
35
35
 
36
36
  For npm hosts:
37
37
 
38
38
  ```bash
39
- pnpm add @kubohiroya/turbowarp-camera-source@0.12.0
39
+ pnpm add @kubohiroya/turbowarp-camera-source@0.14.0
40
40
  ```
41
41
 
42
42
  ## Quick Start
@@ -469,6 +469,35 @@ await qrLease.release();
469
469
  await poseLease.release();
470
470
  ```
471
471
 
472
+ Use `adoptCamera()` instead when the consumer already holds a `MediaStream` — one obtained
473
+ through the `<usermedia>` capability element, from another frame, or from a test fixture.
474
+ Camera Source shares it under a `cameraId` exactly as an acquired stream, and every consumer
475
+ reads it through the same `getFrameSource()`.
476
+
477
+ **Camera Source never stops the tracks of an adopted stream.** Releasing the last lease ends
478
+ the session and detaches the stream, leaving the caller to stop it when it is finished with the
479
+ camera. Stopping a track cannot be undone and takes the frames away from every other consumer
480
+ of the same stream, so the side that opened it is the only side that can know when that is
481
+ right. An acquired stream is still stopped by Camera Source as before.
482
+
483
+ ```js
484
+ if (typeof cameraSource.adoptCamera === "function") {
485
+ const hostLease = await cameraSource.adoptCamera({
486
+ owner: "app-shell",
487
+ cameraId: "pose",
488
+ stream: streamTheHostAlreadyHolds,
489
+ });
490
+ const frame = hostLease.getFrameSource();
491
+ await hostLease.release();
492
+ }
493
+ ```
494
+
495
+ Feature detect the method. This surface is narrowed by shape rather than by a version, so a
496
+ consumer compiled against the current declarations can still meet a build that predates it.
497
+ Adopting a second, different stream under a `cameraId` that already runs one is refused:
498
+ answering the caller with somebody else's frames would leave it believing its own stream is in
499
+ use. Adopting the same stream again joins the session.
500
+
472
501
  `preview: true` opts that lease into the GPU-backed stage preview. The dedicated video skin uploads
473
502
  the shared `HTMLVideoElement` with `texImage2D(video)` and mirrors by changing the preview drawable's
474
503
  X scale. It does not call `drawImage()`, `getImageData()`, or scan frame pixels on the CPU. The option
@@ -515,6 +544,8 @@ const assessment = calibration.assessProfile("pose");
515
544
 
516
545
  **Ask for intrinsics rather than scaling a profile yourself.** When the camera delivers a size the calibration was not solved at, the arithmetic depends on what happened: a pure downscale multiplies `fx`, `fy`, `cx` and `cy`, while a crop leaves the focal lengths alone and shifts the principal point instead. Only this extension sees the track's `resizeMode`, so only it can tell the two apart — and if every consumer guesses, they guess differently and the same camera yields different geometry depending on which extension asked. `camera intrinsics JSON` returns numbers already adapted to the current frame, or an empty string when the difference is a crop or an aspect change and the principal point cannot be placed.
517
546
 
547
+ **Frame capture time.** `getFrameSource()` also carries `frameTime` once the camera has presented a frame: `timestampUs` in microseconds since the Unix epoch on the page's monotonic clock, `source` (`capture` for `captureTime` from `requestVideoFrameCallback`, `presentation` where the browser reports only `presentationTime`), and `presentedFrames`, which tells a frame already used from a new one. Cameras on one page share that clock, so their frames can be lined up directly; it is not synchronized with any other computer. Take the frame source immediately before reading pixels: the time belongs to the frame presented when the source was taken. Absent means unknown, never now.
548
+
518
549
  **Turning the preview over does not turn the frames over.** `getFrameSource()` reports `previewFlip`, which describes how the stage is drawing the image and nothing else; the frames behind it are always the ones the camera captured. Coordinates picked off a flipped preview must be turned back before they are used with these frames — a solve fed the preview's coordinates converges on a left-right reflected pose and reports a small reprojection error while doing it. `horizontal` is the left-right mirror, matching `cv::flip(…, 1)`, ffmpeg's `hflip` and CSS `scaleX(-1)`; rotation is a separate concern and is deliberately not folded into the same enum.
519
550
 
520
551
  **Import these declarations rather than re-writing them.** The contract is published as its own
@@ -649,6 +680,12 @@ profile), and save it. An app that opens the calibration app for one camera name
649
680
 
650
681
  ## Compatibility
651
682
 
683
+ Version 0.14.0 adds `adoptCamera` for sharing a stream the caller already holds, and the
684
+ `CameraAdoptOptions` and `CameraLeaseOptions` declarations to `./runtime`. `CameraAcquireOptions`
685
+ keeps every member it had. No existing block or capability member changes.
686
+
687
+ Version 0.13.0 adds `frameTime` to the frame source. No existing block or capability member changes.
688
+
652
689
  Version 0.12.0 adds device-scoped restore, `bind camera profile ... to its current device`,
653
690
  `camera profile ... belongs to its current device?` and `start shared camera [CAMERA_ID] requested by this page`.
654
691
  No existing block or capability member changes.
@@ -712,10 +749,9 @@ The generated JavaScript is a single, non-minified TurboWarp extension file with
712
749
  - `src/block-definitions.json`: canonical block metadata used by both the extension and README generator
713
750
  - `src/extension.ts`: extension implementation
714
751
  - `src/video-preview.ts`: GPU-backed `HTMLVideoElement` preview skin and drawable lifecycle
715
- - `src/extension-manifest.ts`: canonical manifest generator and Vite output plugin
716
752
  - `src/index.ts`: extension registration entry point
717
753
  - `src/globals.d.ts`: Scratch API declarations used by the project
718
- - `schemas/extension-manifest.schema.json`: JSON Schema for the generated API contract
754
+ - the manifest generator and its JSON Schema come from [`@kubohiroya/turbowarp-extension-manifest`](https://github.com/kubohiroya/turbowarp-extension-manifest)
719
755
  - `scripts/generate-readme.ts`: updates the generated README block section
720
756
  - `tests/`: unit tests
721
757
  - `vite.config.ts`: TurboWarp-compatible Vite build configuration
@@ -727,7 +763,7 @@ Each build emits `dist/extension-manifest.json` with `formatVersion: 1`. It reco
727
763
  block opcodes and types, argument IDs and types, and menu references in a deterministic order. Tools
728
764
  such as `sb3-toolchain` can compare this contract before updating an embedded extension or migrating
729
765
  its ID. See [the architecture document](docs/architecture.md) and the
730
- [JSON Schema](schemas/extension-manifest.schema.json) for the v1 contract.
766
+ [JSON Schema](https://github.com/kubohiroya/turbowarp-extension-manifest/blob/main/schemas/extension-manifest.schema.json) for the v1 contract.
731
767
 
732
768
  After changing runtime or block metadata, regenerate and verify the tracked release artifacts:
733
769
 
@@ -2236,6 +2236,55 @@
2236
2236
  });
2237
2237
  }
2238
2238
  //#endregion
2239
+ //#region src/frame-time.ts
2240
+ function finite(value) {
2241
+ return typeof value === "number" && Number.isFinite(value) ? value : void 0;
2242
+ }
2243
+ /** The time of one presented frame, or the previous one when the metadata carries no time. */
2244
+ function frameTimeFrom(metadata, timeOriginMs, previous) {
2245
+ const capture = finite(metadata.captureTime);
2246
+ const presentation = finite(metadata.presentationTime);
2247
+ const milliseconds = capture ?? presentation;
2248
+ if (milliseconds === void 0) return previous;
2249
+ return Object.freeze({
2250
+ timestampUs: Math.round((timeOriginMs + milliseconds) * 1e3),
2251
+ source: capture === void 0 ? "presentation" : "capture",
2252
+ presentedFrames: finite(metadata.presentedFrames) ?? (previous?.presentedFrames ?? 0) + 1
2253
+ });
2254
+ }
2255
+ /**
2256
+ * Follows one video element for as long as its camera runs.
2257
+ *
2258
+ * Nothing is recorded where the browser has no `requestVideoFrameCallback`; frame sources then carry
2259
+ * no time, which a consumer has to treat as unknown rather than as now.
2260
+ */
2261
+ var FrameTimeWatcher = class {
2262
+ constructor(element, timeOriginMs) {
2263
+ this.disposed = false;
2264
+ this.element = element;
2265
+ this.timeOriginMs = timeOriginMs;
2266
+ this.schedule();
2267
+ }
2268
+ latest() {
2269
+ return this.frame;
2270
+ }
2271
+ dispose() {
2272
+ this.disposed = true;
2273
+ if (this.handle !== void 0) {
2274
+ this.element.cancelVideoFrameCallback(this.handle);
2275
+ this.handle = void 0;
2276
+ }
2277
+ }
2278
+ schedule() {
2279
+ if (this.disposed || typeof this.element.requestVideoFrameCallback !== "function") return;
2280
+ this.handle = this.element.requestVideoFrameCallback((_now, metadata) => {
2281
+ this.handle = void 0;
2282
+ this.frame = frameTimeFrom(metadata, this.timeOriginMs, this.frame);
2283
+ this.schedule();
2284
+ });
2285
+ }
2286
+ };
2287
+ //#endregion
2239
2288
  //#region src/extension.ts
2240
2289
  var blockDefinitions = block_definitions_default.blocks;
2241
2290
  var menuDefinitions = block_definitions_default.menus;
@@ -2469,9 +2518,31 @@
2469
2518
  async acquireCamera(options = {}) {
2470
2519
  const cameraId = normalizeId(options.cameraId);
2471
2520
  const session = this.session(cameraId);
2472
- const token = Symbol(String(options.owner ?? "camera-lease"));
2473
2521
  if (session.startPromise) await session.startPromise;
2474
2522
  else if (!session.stream) await this.start(session, options);
2523
+ return this.lease(session, options);
2524
+ }
2525
+ /**
2526
+ * Shares a stream the caller already holds, under a camera id, without opening a camera.
2527
+ *
2528
+ * Refuses when the id already runs a different stream rather than quietly joining it: the caller
2529
+ * asked for its own stream to be shared, and answering with somebody else's would leave it
2530
+ * believing a stream is in use that nobody reads. Adopting the same stream twice is the one case
2531
+ * that joins, because that is what the caller asked for and already got.
2532
+ */
2533
+ async adoptCamera(options) {
2534
+ const stream = options?.stream;
2535
+ if (typeof stream !== "object" || stream === null || typeof stream.getTracks !== "function") throw new TypeError("Camera Source adoptCamera requires a MediaStream.");
2536
+ const cameraId = normalizeId(options.cameraId);
2537
+ const session = this.session(cameraId);
2538
+ if (session.startPromise) await session.startPromise;
2539
+ if (session.stream && session.stream !== stream) throw new Error(`Camera id ${cameraId} already runs another stream. Release its leases before adopting.`);
2540
+ if (!session.stream) await this.adopt(session, stream);
2541
+ return this.lease(session, options);
2542
+ }
2543
+ lease(session, options) {
2544
+ const cameraId = session.cameraId;
2545
+ const token = Symbol(String(options.owner ?? "camera-lease"));
2475
2546
  session.leases.add(token);
2476
2547
  try {
2477
2548
  if (options.preview === true) {
@@ -2959,11 +3030,49 @@
2959
3030
  video: null,
2960
3031
  preview: null,
2961
3032
  startPromise: null,
2962
- activeDeviceId: ""
3033
+ activeDeviceId: "",
3034
+ frameTimes: null,
3035
+ adopted: false
2963
3036
  };
2964
3037
  this.sessions.set(cameraId, session);
2965
3038
  return session;
2966
3039
  }
3040
+ /**
3041
+ * Brings a caller's stream into a session, taking the same path `start` takes after acquisition.
3042
+ *
3043
+ * `adopted` is set before anything can fail, so no failure path can reach the tracks.
3044
+ */
3045
+ async adopt(session, stream) {
3046
+ session.adopted = true;
3047
+ session.startPromise = (async () => {
3048
+ let video = null;
3049
+ try {
3050
+ if (!session.active) throw new Error("Camera adoption was cancelled.");
3051
+ video = document.createElement("video");
3052
+ video.muted = true;
3053
+ video.playsInline = true;
3054
+ video.srcObject = stream;
3055
+ await video.play();
3056
+ if (!session.active) throw new Error("Camera adoption was cancelled.");
3057
+ session.stream = stream;
3058
+ session.video = video;
3059
+ session.frameTimes = new FrameTimeWatcher(video, globalThis.performance?.timeOrigin ?? 0);
3060
+ this.watchStreamEnd(session, stream);
3061
+ this.updateActiveDevice(session);
3062
+ this.cameraFailures.delete(session.cameraId);
3063
+ } catch (error) {
3064
+ if (video) video.srcObject = null;
3065
+ throw error;
3066
+ }
3067
+ })();
3068
+ try {
3069
+ await session.startPromise;
3070
+ } catch (error) {
3071
+ if (this.sessions.get(session.cameraId) === session) this.stopCameraSession(session.cameraId);
3072
+ this.cameraFailures.set(session.cameraId, cameraFailure(error));
3073
+ throw error;
3074
+ }
3075
+ }
2967
3076
  async start(session, options) {
2968
3077
  session.startPromise = (async () => {
2969
3078
  const stream = await mediaDevices().getUserMedia(videoConstraints(options));
@@ -2978,6 +3087,7 @@
2978
3087
  if (!session.active) throw new Error("Camera acquisition was cancelled.");
2979
3088
  session.stream = stream;
2980
3089
  session.video = video;
3090
+ session.frameTimes = new FrameTimeWatcher(video, globalThis.performance?.timeOrigin ?? 0);
2981
3091
  this.watchStreamEnd(session, stream);
2982
3092
  this.updateActiveDevice(session);
2983
3093
  this.cameraFailures.delete(session.cameraId);
@@ -3035,13 +3145,15 @@
3035
3145
  }
3036
3146
  getFrameSource(session) {
3037
3147
  if (!session.video || !session.stream) throw new Error("Shared camera is not running.");
3148
+ const frameTime = session.frameTimes?.latest();
3038
3149
  return Object.freeze({
3039
3150
  kind: "video",
3040
3151
  element: session.video,
3041
3152
  width: session.video.videoWidth,
3042
3153
  height: session.video.videoHeight,
3043
3154
  previewFlip: this.previewFlip(session),
3044
- deviceId: session.activeDeviceId
3155
+ deviceId: session.activeDeviceId,
3156
+ ...frameTime === void 0 ? {} : { frameTime }
3045
3157
  });
3046
3158
  }
3047
3159
  stopWhenUnused(session) {
@@ -3054,7 +3166,9 @@
3054
3166
  session.active = false;
3055
3167
  this.nextPreviewBlockRevision(cameraId);
3056
3168
  session.preview?.dispose();
3057
- session.stream?.getTracks().forEach((track) => track.stop());
3169
+ session.frameTimes?.dispose();
3170
+ session.frameTimes = null;
3171
+ if (!session.adopted) session.stream?.getTracks().forEach((track) => track.stop());
3058
3172
  if (session.video) session.video.srcObject = null;
3059
3173
  session.stream = null;
3060
3174
  session.video = null;
package/dist/runtime.d.ts CHANGED
@@ -38,17 +38,37 @@ export declare const cameraSourceCapabilityVersion: 1;
38
38
  * rotation and not a flip, and once both exist their order matters.
39
39
  */
40
40
  export type Flip = 'none' | 'horizontal' | 'vertical' | 'both';
41
- export interface CameraAcquireOptions {
41
+ /** What taking a lease needs, however the stream behind it was obtained. */
42
+ export interface CameraLeaseOptions {
42
43
  /** Free text naming the caller. Appears in diagnostics; never used to decide anything. */
43
44
  owner?: string;
44
45
  /** The role name of the stream, such as `pose` or `qr`. Callers naming the same one share it. */
45
46
  cameraId?: string;
46
- deviceId?: string;
47
- video?: MediaTrackConstraints | boolean;
48
47
  /** How this consumer wants the stage preview drawn. Only meaningful together with `preview`. */
49
48
  previewFlip?: Flip;
50
49
  preview?: boolean;
51
50
  }
51
+ export interface CameraAcquireOptions extends CameraLeaseOptions {
52
+ deviceId?: string;
53
+ video?: MediaTrackConstraints | boolean;
54
+ }
55
+ /**
56
+ * What adopting a stream the caller already holds needs.
57
+ *
58
+ * No `deviceId` or `video`: the stream exists, so there is nothing left to constrain. Which device
59
+ * it came from is readable from the track, and `CameraFrameSource.deviceId` reports it.
60
+ */
61
+ export interface CameraAdoptOptions extends CameraLeaseOptions {
62
+ /**
63
+ * The stream to share. Camera Source never stops its tracks.
64
+ *
65
+ * Releasing the last lease ends the session and detaches the stream, leaving the caller to stop
66
+ * it when it is finished with the camera. Stopping a track cannot be undone and takes the frames
67
+ * away from every other consumer of the same stream, so the side that opened it is the only side
68
+ * that can know when that is right.
69
+ */
70
+ stream: MediaStream;
71
+ }
52
72
  export interface CameraFrameSource {
53
73
  readonly kind: 'video';
54
74
  readonly element: HTMLVideoElement;
@@ -66,6 +86,33 @@ export interface CameraFrameSource {
66
86
  */
67
87
  readonly previewFlip: Flip;
68
88
  readonly deviceId: string;
89
+ /**
90
+ * When the frame the element is showing was captured, as of the moment this source was taken.
91
+ *
92
+ * Absent until the camera has presented a frame, and where the browser has no
93
+ * `requestVideoFrameCallback`; absent means unknown, never now. The element can present a newer
94
+ * frame between taking the source and reading pixels from it, so the time belongs to the frame
95
+ * presented when the source was taken — take the source immediately before reading pixels.
96
+ */
97
+ readonly frameTime?: CameraFrameTime;
98
+ }
99
+ /**
100
+ * The capture time of a presented frame.
101
+ *
102
+ * `timestampUs` is microseconds since the Unix epoch on the page's monotonic clock
103
+ * (`performance.timeOrigin + time`). Cameras on one page share that clock, so their frames can be
104
+ * compared directly; it is not synchronized with any other computer.
105
+ */
106
+ export interface CameraFrameTime {
107
+ readonly timestampUs: number;
108
+ /**
109
+ * `capture` is when the frame left the device (`captureTime`). `presentation` is when the browser
110
+ * presented it (`presentationTime`), later by the capture and decode pipeline, and used only where
111
+ * the browser reports no capture time.
112
+ */
113
+ readonly source: 'capture' | 'presentation';
114
+ /** Increases with every presented frame, so a frame already used can be recognized. */
115
+ readonly presentedFrames: number;
69
116
  }
70
117
  export interface CameraLease {
71
118
  getFrameSource(): CameraFrameSource;
@@ -79,6 +126,15 @@ export interface CameraLease {
79
126
  */
80
127
  export interface CameraSourceRuntime {
81
128
  acquireCamera(options?: CameraAcquireOptions): Promise<CameraLease>;
129
+ /**
130
+ * Shares a stream the caller already obtained, instead of opening a camera.
131
+ *
132
+ * Optional because this surface is narrowed by shape rather than by a version, so a consumer
133
+ * compiled against these declarations can still meet a build that predates the method. Feature
134
+ * detect it: `typeof runtime.adoptCamera === 'function'`. A caller reaching for it is already
135
+ * feature detecting whatever produced the stream.
136
+ */
137
+ adoptCamera?(options: CameraAdoptOptions): Promise<CameraLease>;
82
138
  }
83
139
  /**
84
140
  * The profile contract, re-exported so a consumer states it rather than restating it.
@@ -1 +1 @@
1
- {"version":3,"file":"runtime.d.ts","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,eAAO,MAAM,uBAAuB,2BAA2B,CAAC;AAEhE;;;;;GAKG;AACH,eAAO,MAAM,sBAAsB,+BAA+B,CAAC;AAEnE;;;;;;;GAOG;AACH,eAAO,MAAM,yBAAyB,qCAAqC,CAAC;AAE5E,wFAAwF;AACxF,eAAO,MAAM,6BAA6B,EAAG,CAAU,CAAC;AAExD;;;;;;GAMG;AACH,MAAM,MAAM,IAAI,GAAG,MAAM,GAAG,YAAY,GAAG,UAAU,GAAG,MAAM,CAAC;AAE/D,MAAM,WAAW,oBAAoB;IACnC,0FAA0F;IAC1F,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,iGAAiG;IACjG,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,qBAAqB,GAAG,OAAO,CAAC;IACxC,gGAAgG;IAChG,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,gBAAgB,CAAC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,WAAW;IAC1B,cAAc,IAAI,iBAAiB,CAAC;IACpC,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAED;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,aAAa,CAAC,OAAO,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;CACrE;AAED;;;;;;;GAOG;AACH,YAAY,EACV,iBAAiB,EACjB,wBAAwB,EACzB,MAAM,yBAAyB,CAAC;AAEjC,YAAY,EACV,kBAAkB,EAClB,qBAAqB,EACrB,gBAAgB,EAChB,kBAAkB,EAClB,gBAAgB,EAChB,wBAAwB,EACxB,gBAAgB,EAChB,WAAW,EACX,eAAe,EACf,YAAY,EACZ,gBAAgB,EAChB,aAAa,EACd,MAAM,wBAAwB,CAAC;AAEhC,YAAY,EACV,cAAc,EACd,eAAe,EACf,iBAAiB,EACjB,gBAAgB,EACjB,MAAM,6BAA6B,CAAC;AAErC,YAAY,EACV,iBAAiB,EACjB,oBAAoB,EACpB,mBAAmB,EACnB,kBAAkB,EAClB,YAAY,EACb,MAAM,gCAAgC,CAAC;AAExC,YAAY,EAAC,gBAAgB,EAAC,MAAM,6BAA6B,CAAC;AAElE,OAAO,KAAK,EAAC,wBAAwB,EAAC,MAAM,yBAAyB,CAAC;AAEtE,oGAAoG;AACpG,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,OAAO,GAAG,mBAAmB,GAAG,SAAS,CAMzF;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,0BAA0B,CACxC,OAAO,EAAE,OAAO,GACf,wBAAwB,GAAG,SAAS,CAMtC"}
1
+ {"version":3,"file":"runtime.d.ts","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,eAAO,MAAM,uBAAuB,2BAA2B,CAAC;AAEhE;;;;;GAKG;AACH,eAAO,MAAM,sBAAsB,+BAA+B,CAAC;AAEnE;;;;;;;GAOG;AACH,eAAO,MAAM,yBAAyB,qCAAqC,CAAC;AAE5E,wFAAwF;AACxF,eAAO,MAAM,6BAA6B,EAAG,CAAU,CAAC;AAExD;;;;;;GAMG;AACH,MAAM,MAAM,IAAI,GAAG,MAAM,GAAG,YAAY,GAAG,UAAU,GAAG,MAAM,CAAC;AAE/D,4EAA4E;AAC5E,MAAM,WAAW,kBAAkB;IACjC,0FAA0F;IAC1F,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,iGAAiG;IACjG,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,gGAAgG;IAChG,WAAW,CAAC,EAAE,IAAI,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,MAAM,WAAW,oBAAqB,SAAQ,kBAAkB;IAC9D,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,qBAAqB,GAAG,OAAO,CAAC;CACzC;AAED;;;;;GAKG;AACH,MAAM,WAAW,kBAAmB,SAAQ,kBAAkB;IAC5D;;;;;;;OAOG;IACH,MAAM,EAAE,WAAW,CAAC;CACrB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,gBAAgB,CAAC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B;;;;;;;OAOG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,eAAe,CAAC;CACtC;AAED;;;;;;GAMG;AACH,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B;;;;OAIG;IACH,QAAQ,CAAC,MAAM,EAAE,SAAS,GAAG,cAAc,CAAC;IAC5C,uFAAuF;IACvF,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;CAClC;AAED,MAAM,WAAW,WAAW;IAC1B,cAAc,IAAI,iBAAiB,CAAC;IACpC,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CAC1B;AAED;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,aAAa,CAAC,OAAO,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;IACpE;;;;;;;OAOG;IACH,WAAW,CAAC,CAAC,OAAO,EAAE,kBAAkB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;CACjE;AAED;;;;;;;GAOG;AACH,YAAY,EACV,iBAAiB,EACjB,wBAAwB,EACzB,MAAM,yBAAyB,CAAC;AAEjC,YAAY,EACV,kBAAkB,EAClB,qBAAqB,EACrB,gBAAgB,EAChB,kBAAkB,EAClB,gBAAgB,EAChB,wBAAwB,EACxB,gBAAgB,EAChB,WAAW,EACX,eAAe,EACf,YAAY,EACZ,gBAAgB,EAChB,aAAa,EACd,MAAM,wBAAwB,CAAC;AAEhC,YAAY,EACV,cAAc,EACd,eAAe,EACf,iBAAiB,EACjB,gBAAgB,EACjB,MAAM,6BAA6B,CAAC;AAErC,YAAY,EACV,iBAAiB,EACjB,oBAAoB,EACpB,mBAAmB,EACnB,kBAAkB,EAClB,YAAY,EACb,MAAM,gCAAgC,CAAC;AAExC,YAAY,EAAC,gBAAgB,EAAC,MAAM,6BAA6B,CAAC;AAElE,OAAO,KAAK,EAAC,wBAAwB,EAAC,MAAM,yBAAyB,CAAC;AAEtE,oGAAoG;AACpG,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,OAAO,GAAG,mBAAmB,GAAG,SAAS,CAMzF;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,0BAA0B,CACxC,OAAO,EAAE,OAAO,GACf,wBAAwB,GAAG,SAAS,CAMtC"}
@@ -1 +1 @@
1
- {"version":3,"file":"runtime.js","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,MAAM,CAAC,MAAM,uBAAuB,GAAG,wBAAwB,CAAC;AAEhE;;;;;GAKG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG,4BAA4B,CAAC;AAEnE;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,yBAAyB,GAAG,kCAAkC,CAAC;AAE5E,wFAAwF;AACxF,MAAM,CAAC,MAAM,6BAA6B,GAAG,CAAU,CAAC;AAwGxD,oGAAoG;AACpG,MAAM,UAAU,uBAAuB,CAAC,OAAgB;IACtD,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,sBAAsB,CAAC,CAAC;IAC/E,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,aAAa,EAAC,GAAG,SAAsC,CAAC;IAC/D,OAAO,OAAO,aAAa,KAAK,UAAU,CAAC,CAAC,CAAE,SAAiC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC9F,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,0BAA0B,CACxC,OAAgB;IAEhB,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,yBAAyB,CAAC,CAAC;IAClF,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,cAAc,EAAC,GAAG,SAAuC,CAAC;IACjE,OAAO,OAAO,cAAc,KAAK,UAAU,CAAC,CAAC,CAAE,SAAsC,CAAC,CAAC,CAAC,SAAS,CAAC;AACpG,CAAC"}
1
+ {"version":3,"file":"runtime.js","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,0FAA0F;AAC1F,MAAM,CAAC,MAAM,uBAAuB,GAAG,wBAAwB,CAAC;AAEhE;;;;;GAKG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG,4BAA4B,CAAC;AAEnE;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,yBAAyB,GAAG,kCAAkC,CAAC;AAE5E,wFAAwF;AACxF,MAAM,CAAC,MAAM,6BAA6B,GAAG,CAAU,CAAC;AAmKxD,oGAAoG;AACpG,MAAM,UAAU,uBAAuB,CAAC,OAAgB;IACtD,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,sBAAsB,CAAC,CAAC;IAC/E,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,aAAa,EAAC,GAAG,SAAsC,CAAC;IAC/D,OAAO,OAAO,aAAa,KAAK,UAAU,CAAC,CAAC,CAAE,SAAiC,CAAC,CAAC,CAAC,SAAS,CAAC;AAC9F,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,0BAA0B,CACxC,OAAgB;IAEhB,IAAI,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACtE,MAAM,SAAS,GAAI,OAAmC,CAAC,yBAAyB,CAAC,CAAC;IAClF,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,EAAC,cAAc,EAAC,GAAG,SAAuC,CAAC;IACjE,OAAO,OAAO,cAAc,KAAK,UAAU,CAAC,CAAC,CAAE,SAAsC,CAAC,CAAC,CAAC,SAAS,CAAC;AACpG,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kubohiroya/turbowarp-camera-source",
3
- "version": "0.12.0",
3
+ "version": "0.14.0",
4
4
  "author": "Hiroya Kubo <hiroya@cuc.ac.jp>",
5
5
  "description": "A TurboWarp capability extension for sharing one MediaDevices camera stream across extensions.",
6
6
  "type": "module",
@@ -61,6 +61,7 @@
61
61
  },
62
62
  "devDependencies": {
63
63
  "@eslint/js": "^9.39.5",
64
+ "@kubohiroya/turbowarp-extension-manifest": "0.2.0",
64
65
  "@kubohiroya/vite-plugin-turbowarp-extension": "0.4.0",
65
66
  "@types/node": "^24.13.3",
66
67
  "ajv": "^8.20.0",
@@ -1,49 +0,0 @@
1
- {
2
- "$schema": "https://json-schema.org/draft/2020-12/schema",
3
- "$id": "https://raw.githubusercontent.com/kubohiroya/turbowarp-extension-template/main/schemas/extension-manifest.schema.json",
4
- "title": "TurboWarp Extension API Manifest",
5
- "type": "object",
6
- "additionalProperties": false,
7
- "required": ["formatVersion", "id", "blocks", "menus"],
8
- "properties": {
9
- "formatVersion": {"const": 1},
10
- "id": {"type": "string", "pattern": "^[a-z0-9]+$"},
11
- "blocks": {
12
- "type": "array",
13
- "items": {
14
- "type": "object",
15
- "additionalProperties": false,
16
- "required": ["opcode", "blockType", "arguments"],
17
- "properties": {
18
- "opcode": {"type": "string", "minLength": 1},
19
- "blockType": {"type": "string", "minLength": 1},
20
- "arguments": {
21
- "type": "array",
22
- "items": {
23
- "type": "object",
24
- "additionalProperties": false,
25
- "required": ["id", "type"],
26
- "properties": {
27
- "id": {"type": "string", "minLength": 1},
28
- "type": {"type": "string", "minLength": 1},
29
- "menu": {"type": "string", "minLength": 1}
30
- }
31
- }
32
- }
33
- }
34
- }
35
- },
36
- "menus": {
37
- "type": "array",
38
- "items": {
39
- "type": "object",
40
- "additionalProperties": false,
41
- "required": ["id", "acceptReporters"],
42
- "properties": {
43
- "id": {"type": "string", "minLength": 1},
44
- "acceptReporters": {"type": "boolean"}
45
- }
46
- }
47
- }
48
- }
49
- }