@kubohiroya/turbowarp-camera-source 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.ja.md CHANGED
@@ -26,13 +26,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
26
26
  次のURLをunsandboxed custom extensionとして読み込みます。
27
27
 
28
28
  ```text
29
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.3.0/dist/camera-source.js
29
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.5.0/dist/camera-source.js
30
30
  ```
31
31
 
32
32
  npm hostでは次を使います。
33
33
 
34
34
  ```bash
35
- pnpm add @kubohiroya/turbowarp-camera-source@0.3.0
35
+ pnpm add @kubohiroya/turbowarp-camera-source@0.5.0
36
36
  ```
37
37
 
38
38
  ## Quick start
@@ -41,6 +41,10 @@ pnpm add @kubohiroya/turbowarp-camera-source@0.3.0
41
41
 
42
42
  ```text
43
43
  start shared camera [pose] with device ID []
44
+ show shared camera [pose] preview mirrored [true]
45
+ shared camera [pose] frame width
46
+ shared camera [pose] frame rate
47
+ hide shared camera [pose] preview
44
48
  start shared camera [qr] with device ID []
45
49
  shared camera [pose] is running?
46
50
  stop shared camera [pose]
@@ -52,6 +56,11 @@ stop shared camera [pose]
52
56
  - `stop shared camera [CAMERA_ID]`: 名前付き共有カメラを停止し、MediaStreamTrackを解放します。
53
57
  - `shared camera [CAMERA_ID] is running?`: 指定した共有カメラが起動中かを返します。
54
58
  - `shared camera [CAMERA_ID] device ID`: 指定した共有カメラのdevice IDを返します。
59
+ - `show shared camera [CAMERA_ID] preview mirrored [MIRRORED]`: GPU-backed stage previewを表示します。
60
+ - `hide shared camera [CAMERA_ID] preview`: ブロックが所有するpreview leaseだけを解放して非表示にします。
61
+ - `shared camera [CAMERA_ID] frame width`: 実際のframe幅をpixel単位で返します。
62
+ - `shared camera [CAMERA_ID] frame height`: 実際のframe高さをpixel単位で返します。
63
+ - `shared camera [CAMERA_ID] frame rate`: 有効なvideo trackのframe rateを返します。
55
64
  - `refresh camera devices`: カメラデバイス一覧を更新します。
56
65
  - `camera device count`: 更新済みカメラデバイス数を返します。
57
66
  - `camera device ID at [INDEX]`: 1始まりの位置でカメラdevice IDを返します。
@@ -72,16 +81,55 @@ const qrLease = await cameraSource.acquireCamera({
72
81
  cameraId: "qr",
73
82
  deviceId: qrDeviceId,
74
83
  });
84
+ const previewLease = await cameraSource.acquireCamera({
85
+ owner: "camera-preview",
86
+ cameraId: "pose",
87
+ preview: true,
88
+ mirrored: true,
89
+ });
75
90
  const frame = qrLease.getFrameSource();
91
+ await previewLease.release();
76
92
  await qrLease.release();
77
93
  await poseLease.release();
78
94
  ```
79
95
 
80
- `cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。`frame.element`は`HTMLVideoElement`です。各カメラは最後のleaseが解放されるまで維持されます。
96
+ `preview: true`を指定したleaseは、GPU-backed stage previewを利用します。専用video skinは共有
97
+ `HTMLVideoElement`を`texImage2D(video)`でアップロードし、preview drawableのX scaleで左右反転します。
98
+ `drawImage()`、`getImageData()`、CPUでのフレームピクセル走査は行いません。既定値は`false`で、同じ
99
+ カメラの最後のpreview leaseが解放されるまで表示を維持します。
100
+ 複数のpreview leaseが同じカメラを共有する場合、1つ以上の有効なleaseが`mirrored: true`を要求している間は
101
+ previewを左右反転します。
102
+
103
+ TurboWarp作品からは`show shared camera ... preview`と`hide shared camera ... preview`ブロックで同じ
104
+ 表示経路を操作できます。showブロックが所有するpreview leaseはcamera IDごとに1つだけです。同じ設定で
105
+ 繰り返し実行してもleaseやdrawableは増えず、`MIRRORED`を変更した場合は既存camera sessionを維持したまま
106
+ leaseを置換します。`MIRRORED`には`true`または`false`など、TurboWarpがbooleanとして解釈できる値を指定します。
107
+ hideはブロック所有の表示leaseだけを解放するため、ほかのブロックや拡張が所有する処理用leaseは継続します。
108
+ 名前付きcameraの停止、作品の停止・再読込、runtime disposeではブロック所有resourceをすべて解放します。
109
+
110
+ frame widthとheightは、video elementが現在受信している寸法を優先し、最初のframeより前はvideo track設定を
111
+ fallbackとして使います。frame rateは有効なvideo track設定を返します。cameraが未起動の場合、またはブラウザが
112
+ 値を提供しない場合、これら3つのreporterは`0`を返します。
113
+
114
+ `frame.element`はpreviewを左右反転した場合も未加工の`HTMLVideoElement`です。そのためWebGPUや
115
+ WebCodecsのconsumerは、CPU canvasやpreviewを経由せず、同じsourceを
116
+ `GPUQueue.copyExternalImageToTexture()`や短時間だけ保持する`VideoFrame`へ渡せます。consumerが生成した
117
+ `VideoFrame`はconsumer自身が`close()`する必要があります。
118
+
119
+ この表示経路は明示的なCPU readbackを避けますが、end-to-endの完全なzero-copyは保証しません。ブラウザ内部の
120
+ 色変換やGPU転送が発生する可能性があります。またpreviewはprivateかつnoninteractiveで、表示専用です。
121
+ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いません。
122
+
123
+ `cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。各カメラは最後のleaseが解放されるまで維持されます。
81
124
 
82
125
  ## 互換性
83
126
 
84
- Extension IDは`kubohiroyacamerasource`のままです。ブロックopcode、camera lease ownership、device selection、release semanticsは0.2.0でも変更しません。
127
+ 0.5.0ではGPU-backed previewの表示・非表示・mirror設定と、実際のframe width/height/rateを
128
+ TurboWarpブロックとして公開します。既定動作は変更しません。
129
+
130
+ 0.4.0ではopt-inのGPU-backed video previewを追加します。Extension ID、ブロックopcode、camera lease
131
+ ownership、device selection、release semanticsは変更しません。`preview: true`を指定しないconsumerは、
132
+ 従来のcamera-source動作を維持します。
85
133
 
86
134
  ## 開発
87
135
 
package/README.md CHANGED
@@ -30,13 +30,13 @@ or select separate cameras for separate roles.
30
30
  Load this URL as an unsandboxed custom extension:
31
31
 
32
32
  ```text
33
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.3.0/dist/camera-source.js
33
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.5.0/dist/camera-source.js
34
34
  ```
35
35
 
36
36
  For npm hosts:
37
37
 
38
38
  ```bash
39
- pnpm add @kubohiroya/turbowarp-camera-source@0.3.0
39
+ pnpm add @kubohiroya/turbowarp-camera-source@0.5.0
40
40
  ```
41
41
 
42
42
  ## Quick Start
@@ -45,6 +45,10 @@ Use role names for camera streams. Consumers that use the same `cameraId` share
45
45
 
46
46
  ```text
47
47
  start shared camera [pose] with device ID []
48
+ show shared camera [pose] preview mirrored [true]
49
+ shared camera [pose] frame width
50
+ shared camera [pose] frame rate
51
+ hide shared camera [pose] preview
48
52
  start shared camera [qr] with device ID []
49
53
  shared camera [pose] is running?
50
54
  stop shared camera [pose]
@@ -95,6 +99,57 @@ Returns the active device ID for a named shared camera when available.
95
99
  | Opcode | `cameraDeviceIdReporter` |
96
100
  | `CAMERA_ID` | String, default: `default` |
97
101
 
102
+ ### `show shared camera [CAMERA_ID] preview mirrored [MIRRORED]`
103
+
104
+ Shows the named shared camera with the GPU-backed stage preview.
105
+
106
+ | Property | Value |
107
+ |---|---|
108
+ | Type | Command |
109
+ | Opcode | `showCameraPreview` |
110
+ | `CAMERA_ID` | String, default: `default` |
111
+ | `MIRRORED` | String, default: `true` |
112
+
113
+ ### `hide shared camera [CAMERA_ID] preview`
114
+
115
+ Hides the block-owned preview without stopping leases owned by other consumers.
116
+
117
+ | Property | Value |
118
+ |---|---|
119
+ | Type | Command |
120
+ | Opcode | `hideCameraPreview` |
121
+ | `CAMERA_ID` | String, default: `default` |
122
+
123
+ ### `shared camera [CAMERA_ID] frame width`
124
+
125
+ Returns the active video frame width in pixels, or zero while the camera is not running.
126
+
127
+ | Property | Value |
128
+ |---|---|
129
+ | Type | Reporter |
130
+ | Opcode | `cameraFrameWidth` |
131
+ | `CAMERA_ID` | String, default: `default` |
132
+
133
+ ### `shared camera [CAMERA_ID] frame height`
134
+
135
+ Returns the active video frame height in pixels, or zero while the camera is not running.
136
+
137
+ | Property | Value |
138
+ |---|---|
139
+ | Type | Reporter |
140
+ | Opcode | `cameraFrameHeight` |
141
+ | `CAMERA_ID` | String, default: `default` |
142
+
143
+ ### `shared camera [CAMERA_ID] frame rate`
144
+
145
+ Returns the active video track frame rate, or zero when it is unavailable.
146
+
147
+ | Property | Value |
148
+ |---|---|
149
+ | Type | Reporter |
150
+ | Opcode | `cameraFrameRate` |
151
+ | `CAMERA_ID` | String, default: `default` |
152
+
98
153
  ### `refresh camera devices`
99
154
 
100
155
  Refreshes the browser camera device list.
@@ -154,11 +209,56 @@ const qrLease = await cameraSource.acquireCamera({
154
209
  cameraId: "qr",
155
210
  deviceId: qrDeviceId,
156
211
  });
212
+ const previewLease = await cameraSource.acquireCamera({
213
+ owner: "camera-preview",
214
+ cameraId: "pose",
215
+ preview: true,
216
+ mirrored: true,
217
+ });
218
+ const frame = qrLease.getFrameSource();
219
+ await previewLease.release();
220
+ await qrLease.release();
221
+ await poseLease.release();
157
222
  ```
158
223
 
224
+ `preview: true` opts that lease into the GPU-backed stage preview. The dedicated video skin uploads
225
+ the shared `HTMLVideoElement` with `texImage2D(video)` and mirrors by changing the preview drawable's
226
+ X scale. It does not call `drawImage()`, `getImageData()`, or scan frame pixels on the CPU. The option
227
+ defaults to `false`; the preview remains visible until the final preview lease for that camera is
228
+ released.
229
+ If multiple preview leases share a camera, the preview is mirrored while any active preview lease
230
+ requests `mirrored: true`.
231
+
232
+ TurboWarp projects can manage the same display path with the `show shared camera ... preview`
233
+ and `hide shared camera ... preview` blocks. The show block owns exactly one preview lease per
234
+ camera ID. Calling it repeatedly with the same setting is idempotent; changing `MIRRORED` replaces
235
+ that lease without creating another drawable. `MIRRORED` accepts TurboWarp boolean text such as
236
+ `true` and `false`. Hiding the preview releases only the block-owned display lease, so processing
237
+ leases held by other blocks or extensions continue running. Stopping the named camera, stopping or
238
+ loading a project, and disposing the runtime release the block-owned camera and preview resources.
239
+
240
+ The frame width and height reporters prefer the dimensions currently delivered by the video
241
+ element and fall back to the active track settings before its first frame. The frame-rate reporter
242
+ uses the active track setting. All three reporters return `0` when the camera is not running or the
243
+ browser does not provide the value.
244
+
245
+ `frame.element` is the original, unmodified `HTMLVideoElement`, even when the preview is mirrored.
246
+ WebGPU and WebCodecs consumers can therefore use that same source without going through the preview
247
+ or a CPU canvas, for example with `GPUQueue.copyExternalImageToTexture()` or a short-lived
248
+ `VideoFrame`. Consumers own and must close any `VideoFrame` they create.
249
+
250
+ The display path avoids explicit CPU readback, but it does not promise end-to-end zero-copy:
251
+ browser-internal color conversion and GPU transfer may still occur. The preview is private and
252
+ noninteractive; it is intended for display, not Scratch touching or color-sensing queries.
253
+
159
254
  ## Compatibility
160
255
 
161
- The extension ID remains `kubohiroyacamerasource`, and the block opcodes are unchanged. Camera lease ownership, device selection, and release semantics are unchanged in 0.2.0.
256
+ Version 0.5.0 exposes GPU-backed preview visibility, mirroring, and actual frame width, height,
257
+ and rate as TurboWarp blocks. The default behavior remains unchanged.
258
+
259
+ Version 0.4.0 adds the opt-in GPU-backed video preview without changing the extension ID,
260
+ block opcodes, camera lease ownership, device selection, or release semantics. Consumers that do
261
+ not request `preview: true` retain the existing camera-source behavior.
162
262
 
163
263
  ## Development
164
264
 
@@ -194,6 +294,7 @@ The generated JavaScript is a single, non-minified TurboWarp extension file with
194
294
  - `src/config.ts`: extension metadata
195
295
  - `src/block-definitions.json`: canonical block metadata used by both the extension and README generator
196
296
  - `src/extension.ts`: extension implementation
297
+ - `src/video-preview.ts`: GPU-backed `HTMLVideoElement` preview skin and drawable lifecycle
197
298
  - `src/extension-manifest.ts`: canonical manifest generator and Vite output plugin
198
299
  - `src/index.ts`: extension registration entry point
199
300
  - `src/globals.d.ts`: Scratch API declarations used by the project
@@ -66,6 +66,62 @@
66
66
  "defaultValue": "default"
67
67
  } }
68
68
  },
69
+ {
70
+ "opcode": "showCameraPreview",
71
+ "blockType": "COMMAND",
72
+ "text": "show shared camera [CAMERA_ID] preview mirrored [MIRRORED]",
73
+ "description": "Shows the named shared camera with the GPU-backed stage preview.",
74
+ "arguments": {
75
+ "CAMERA_ID": {
76
+ "type": "STRING",
77
+ "defaultValue": "default"
78
+ },
79
+ "MIRRORED": {
80
+ "type": "STRING",
81
+ "defaultValue": "true"
82
+ }
83
+ }
84
+ },
85
+ {
86
+ "opcode": "hideCameraPreview",
87
+ "blockType": "COMMAND",
88
+ "text": "hide shared camera [CAMERA_ID] preview",
89
+ "description": "Hides the block-owned preview without stopping leases owned by other consumers.",
90
+ "arguments": { "CAMERA_ID": {
91
+ "type": "STRING",
92
+ "defaultValue": "default"
93
+ } }
94
+ },
95
+ {
96
+ "opcode": "cameraFrameWidth",
97
+ "blockType": "REPORTER",
98
+ "text": "shared camera [CAMERA_ID] frame width",
99
+ "description": "Returns the active video frame width in pixels, or zero while the camera is not running.",
100
+ "arguments": { "CAMERA_ID": {
101
+ "type": "STRING",
102
+ "defaultValue": "default"
103
+ } }
104
+ },
105
+ {
106
+ "opcode": "cameraFrameHeight",
107
+ "blockType": "REPORTER",
108
+ "text": "shared camera [CAMERA_ID] frame height",
109
+ "description": "Returns the active video frame height in pixels, or zero while the camera is not running.",
110
+ "arguments": { "CAMERA_ID": {
111
+ "type": "STRING",
112
+ "defaultValue": "default"
113
+ } }
114
+ },
115
+ {
116
+ "opcode": "cameraFrameRate",
117
+ "blockType": "REPORTER",
118
+ "text": "shared camera [CAMERA_ID] frame rate",
119
+ "description": "Returns the active video track frame rate, or zero when it is unavailable.",
120
+ "arguments": { "CAMERA_ID": {
121
+ "type": "STRING",
122
+ "defaultValue": "default"
123
+ } }
124
+ },
69
125
  {
70
126
  "opcode": "refreshCameraDevices",
71
127
  "blockType": "COMMAND",
@@ -103,6 +159,169 @@
103
159
  ]
104
160
  };
105
161
  //#endregion
162
+ //#region src/video-preview.ts
163
+ var videoLayer = "video";
164
+ var haveCurrentData = 2;
165
+ function videoSkinClass(renderer) {
166
+ const BaseSkin = renderer.exports?.Skin;
167
+ if (typeof BaseSkin !== "function") throw new Error("Camera preview requires renderer.exports.Skin.");
168
+ return class VideoSkin extends BaseSkin {
169
+ constructor(id, skinRenderer, video, onFrame) {
170
+ super(id, skinRenderer);
171
+ this.texture = null;
172
+ this.textureSize = [0, 0];
173
+ this.dirty = true;
174
+ this.disposed = false;
175
+ this.videoFrameCallbackId = null;
176
+ this.animationFrameId = null;
177
+ this.lastCurrentTime = NaN;
178
+ this.handleVideoFrame = () => {
179
+ if (this.disposed) return;
180
+ this.dirty = true;
181
+ this.onFrame(this.syncMetrics());
182
+ this.scheduleFrame();
183
+ };
184
+ this.handleAnimationFrame = () => {
185
+ if (this.disposed) return;
186
+ if (this.video.currentTime !== this.lastCurrentTime) {
187
+ this.dirty = true;
188
+ this.onFrame(this.syncMetrics());
189
+ }
190
+ this.scheduleFrame();
191
+ };
192
+ this.video = video;
193
+ this.onFrame = onFrame;
194
+ this.private = true;
195
+ this.syncMetrics();
196
+ this.scheduleFrame();
197
+ }
198
+ get size() {
199
+ return this.textureSize;
200
+ }
201
+ getTexture() {
202
+ if (this.disposed || this.video.readyState < haveCurrentData || this.video.videoWidth === 0 || this.video.videoHeight === 0) return null;
203
+ if (this.syncMetrics()) this.onFrame(true);
204
+ if (!this.dirty && this.video.currentTime === this.lastCurrentTime) return this.texture;
205
+ const gl = renderer.gl;
206
+ if (!this.texture) {
207
+ this.texture = gl.createTexture();
208
+ if (!this.texture) throw new Error("Camera preview could not create a WebGL texture.");
209
+ gl.bindTexture(gl.TEXTURE_2D, this.texture);
210
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
211
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
212
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
213
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
214
+ } else gl.bindTexture(gl.TEXTURE_2D, this.texture);
215
+ gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true);
216
+ try {
217
+ gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, this.video);
218
+ } finally {
219
+ gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
220
+ }
221
+ this.dirty = false;
222
+ this.lastCurrentTime = this.video.currentTime;
223
+ return this.texture;
224
+ }
225
+ dispose() {
226
+ this.disposed = true;
227
+ if (this.videoFrameCallbackId !== null && this.video.cancelVideoFrameCallback) this.video.cancelVideoFrameCallback(this.videoFrameCallbackId);
228
+ if (this.animationFrameId !== null && typeof globalThis.cancelAnimationFrame === "function") globalThis.cancelAnimationFrame(this.animationFrameId);
229
+ if (this.texture) renderer.gl.deleteTexture(this.texture);
230
+ this.texture = null;
231
+ super.dispose();
232
+ }
233
+ useNearest() {
234
+ return false;
235
+ }
236
+ updateSilhouette() {}
237
+ syncMetrics() {
238
+ const width = this.video.videoWidth;
239
+ const height = this.video.videoHeight;
240
+ if (width === this.textureSize[0] && height === this.textureSize[1]) return false;
241
+ this.textureSize = [width, height];
242
+ this.rotationCenter[0] = width / 2;
243
+ this.rotationCenter[1] = height / 2;
244
+ return true;
245
+ }
246
+ scheduleFrame() {
247
+ if (typeof this.video.requestVideoFrameCallback === "function") this.videoFrameCallbackId = this.video.requestVideoFrameCallback(this.handleVideoFrame);
248
+ else if (typeof globalThis.requestAnimationFrame === "function") this.animationFrameId = globalThis.requestAnimationFrame(this.handleAnimationFrame);
249
+ }
250
+ };
251
+ }
252
+ function assertRenderer(renderer) {
253
+ if (!renderer || !Array.isArray(renderer._allSkins) || !Number.isInteger(renderer._nextSkinId) || typeof renderer.createDrawable !== "function" || typeof renderer.destroyDrawable !== "function" || typeof renderer.destroySkin !== "function" || typeof renderer.getNativeSize !== "function") throw new Error("Camera preview requires a compatible TurboWarp renderer.");
254
+ }
255
+ function createVideoPreview(renderer, video, mirrored, requestRedraw) {
256
+ assertRenderer(renderer);
257
+ const VideoSkin = videoSkinClass(renderer);
258
+ const skinId = renderer._nextSkinId++;
259
+ let drawableId;
260
+ let disposed = false;
261
+ let previewMirrored = mirrored;
262
+ let lastLayout = "";
263
+ const updateLayout = () => {
264
+ if (drawableId === void 0 || video.videoWidth === 0 || video.videoHeight === 0) return;
265
+ const [stageWidth, stageHeight] = renderer.getNativeSize();
266
+ const layout = [
267
+ video.videoWidth,
268
+ video.videoHeight,
269
+ stageWidth,
270
+ stageHeight,
271
+ previewMirrored
272
+ ].join(":");
273
+ if (layout === lastLayout) return;
274
+ const scale = Math.max(stageWidth / video.videoWidth, stageHeight / video.videoHeight) * 100;
275
+ renderer.updateDrawableScale(drawableId, [previewMirrored ? -scale : scale, scale]);
276
+ lastLayout = layout;
277
+ };
278
+ const skin = new VideoSkin(skinId, renderer, video, (metricsChanged) => {
279
+ if (metricsChanged) skin.emitWasAltered();
280
+ updateLayout();
281
+ requestRedraw();
282
+ });
283
+ renderer._allSkins[skinId] = skin;
284
+ try {
285
+ drawableId = renderer.createDrawable(videoLayer);
286
+ if (drawableId === void 0) throw new Error("Camera preview could not create a video drawable.");
287
+ renderer.updateDrawableSkinId(drawableId, skinId);
288
+ renderer.updateDrawablePosition(drawableId, [0, 0]);
289
+ updateLayout();
290
+ renderer.updateDrawableVisible(drawableId, true);
291
+ renderer.markSkinAsPrivate?.(skinId);
292
+ renderer.markDrawableAsNoninteractive?.(drawableId);
293
+ requestRedraw();
294
+ } catch (error) {
295
+ try {
296
+ if (drawableId !== void 0) renderer.destroyDrawable(drawableId, videoLayer);
297
+ } finally {
298
+ renderer.destroySkin(skinId);
299
+ }
300
+ throw error;
301
+ }
302
+ return Object.freeze({
303
+ setMirrored: (nextMirrored) => {
304
+ if (disposed || previewMirrored === nextMirrored) return;
305
+ previewMirrored = nextMirrored;
306
+ updateLayout();
307
+ requestRedraw();
308
+ },
309
+ dispose: () => {
310
+ if (disposed) return;
311
+ disposed = true;
312
+ try {
313
+ if (drawableId !== void 0) renderer.destroyDrawable(drawableId, videoLayer);
314
+ } finally {
315
+ try {
316
+ renderer.destroySkin(skinId);
317
+ } finally {
318
+ requestRedraw();
319
+ }
320
+ }
321
+ }
322
+ });
323
+ }
324
+ //#endregion
106
325
  //#region src/extension.ts
107
326
  var blockDefinitions = block_definitions_default.blocks;
108
327
  var defaultCameraId = "default";
@@ -139,8 +358,22 @@
139
358
  constructor() {
140
359
  this.sessions = /* @__PURE__ */ new Map();
141
360
  this.blockLeases = /* @__PURE__ */ new Map();
361
+ this.blockPreviewLeases = /* @__PURE__ */ new Map();
362
+ this.blockPreviewRevisions = /* @__PURE__ */ new Map();
142
363
  this.devices = [];
364
+ this.dispose = () => {
365
+ this.stopAllCameras();
366
+ Scratch.vm.runtime.off?.("PROJECT_STOP_ALL", this.handleProjectBoundary);
367
+ Scratch.vm.runtime.off?.("PROJECT_LOADED", this.handleProjectBoundary);
368
+ Scratch.vm.runtime.off?.("RUNTIME_DISPOSED", this.dispose);
369
+ };
370
+ this.handleProjectBoundary = () => {
371
+ this.stopAllCameras();
372
+ };
143
373
  Scratch.vm.runtime.ext_kubohiroyacamerasource = this;
374
+ Scratch.vm.runtime.on?.("PROJECT_STOP_ALL", this.handleProjectBoundary);
375
+ Scratch.vm.runtime.on?.("PROJECT_LOADED", this.handleProjectBoundary);
376
+ Scratch.vm.runtime.on?.("RUNTIME_DISPOSED", this.dispose);
144
377
  }
145
378
  getInfo() {
146
379
  return {
@@ -156,6 +389,20 @@
156
389
  cameraDeviceIdReporter(args = {}) {
157
390
  return this.sessions.get(normalizeId(args.CAMERA_ID))?.activeDeviceId ?? "";
158
391
  }
392
+ cameraFrameWidth(args = {}) {
393
+ const session = this.sessions.get(normalizeId(args.CAMERA_ID));
394
+ if (!session?.stream) return 0;
395
+ return session.video?.videoWidth || this.trackSetting(session, "width");
396
+ }
397
+ cameraFrameHeight(args = {}) {
398
+ const session = this.sessions.get(normalizeId(args.CAMERA_ID));
399
+ if (!session?.stream) return 0;
400
+ return session.video?.videoHeight || this.trackSetting(session, "height");
401
+ }
402
+ cameraFrameRate(args = {}) {
403
+ const session = this.sessions.get(normalizeId(args.CAMERA_ID));
404
+ return session?.stream ? this.trackSetting(session, "frameRate") : 0;
405
+ }
159
406
  async startSharedCamera(args = {}) {
160
407
  const cameraId = normalizeId(args.CAMERA_ID);
161
408
  if (this.blockLeases.has(cameraId)) return;
@@ -175,6 +422,18 @@
175
422
  if (session.startPromise) await session.startPromise;
176
423
  else if (!session.stream) await this.start(session, options);
177
424
  session.leases.add(token);
425
+ try {
426
+ if (options.preview === true) {
427
+ session.previewLeases.set(token, options.mirrored === true);
428
+ this.ensurePreview(session);
429
+ session.preview?.setMirrored(this.previewMirrored(session));
430
+ }
431
+ } catch (error) {
432
+ session.leases.delete(token);
433
+ session.previewLeases.delete(token);
434
+ if (session.leases.size === 0) this.stopCameraSession(session.cameraId);
435
+ throw error;
436
+ }
178
437
  let released = false;
179
438
  return Object.freeze({
180
439
  getFrameSource: () => this.getFrameSource(session),
@@ -182,10 +441,45 @@
182
441
  if (released) return;
183
442
  released = true;
184
443
  session.leases.delete(token);
444
+ session.previewLeases.delete(token);
445
+ if (session.previewLeases.size === 0) {
446
+ session.preview?.dispose();
447
+ session.preview = null;
448
+ } else session.preview?.setMirrored(this.previewMirrored(session));
185
449
  if (session.leases.size === 0) this.stopCameraSession(session.cameraId);
186
450
  }
187
451
  });
188
452
  }
453
+ async showCameraPreview(args = {}) {
454
+ const cameraId = normalizeId(args.CAMERA_ID);
455
+ const mirrored = Scratch.Cast.toBoolean(args.MIRRORED ?? true);
456
+ if (this.blockPreviewLeases.get(cameraId)?.mirrored === mirrored) return;
457
+ const revision = this.nextPreviewBlockRevision(cameraId);
458
+ const lease = await this.acquireCamera({
459
+ owner: "camera-source-preview-block",
460
+ cameraId,
461
+ preview: true,
462
+ mirrored
463
+ });
464
+ if (this.blockPreviewRevisions.get(cameraId) !== revision) {
465
+ await lease.release();
466
+ return;
467
+ }
468
+ const current = this.blockPreviewLeases.get(cameraId);
469
+ this.blockPreviewLeases.set(cameraId, {
470
+ lease,
471
+ mirrored
472
+ });
473
+ await current?.lease.release();
474
+ }
475
+ async hideCameraPreview(args = {}) {
476
+ const cameraId = normalizeId(args.CAMERA_ID);
477
+ this.nextPreviewBlockRevision(cameraId);
478
+ const existing = this.blockPreviewLeases.get(cameraId);
479
+ if (!existing) return;
480
+ this.blockPreviewLeases.delete(cameraId);
481
+ await existing.lease.release();
482
+ }
189
483
  stopSharedCamera(args = {}) {
190
484
  this.stopCameraSession(normalizeId(args.CAMERA_ID));
191
485
  }
@@ -204,6 +498,7 @@
204
498
  stopAllCameras() {
205
499
  for (const cameraId of [...this.sessions.keys()]) this.stopCameraSession(cameraId);
206
500
  this.blockLeases.clear();
501
+ this.blockPreviewLeases.clear();
207
502
  }
208
503
  session(cameraId) {
209
504
  const existing = this.sessions.get(cameraId);
@@ -211,8 +506,11 @@
211
506
  const session = {
212
507
  cameraId,
213
508
  leases: /* @__PURE__ */ new Set(),
509
+ previewLeases: /* @__PURE__ */ new Map(),
510
+ active: true,
214
511
  stream: null,
215
512
  video: null,
513
+ preview: null,
216
514
  startPromise: null,
217
515
  mirrored: false,
218
516
  activeDeviceId: ""
@@ -224,26 +522,52 @@
224
522
  session.mirrored = options.mirrored === true;
225
523
  session.startPromise = (async () => {
226
524
  const stream = await mediaDevices().getUserMedia(videoConstraints(options));
227
- const video = document.createElement("video");
228
- video.muted = true;
229
- video.playsInline = true;
230
- video.srcObject = stream;
231
- await video.play();
232
- session.stream = stream;
233
- session.video = video;
234
- this.updateActiveDevice(session);
525
+ let video = null;
526
+ try {
527
+ if (!session.active) throw new Error("Camera acquisition was cancelled.");
528
+ video = document.createElement("video");
529
+ video.muted = true;
530
+ video.playsInline = true;
531
+ video.srcObject = stream;
532
+ await video.play();
533
+ if (!session.active) throw new Error("Camera acquisition was cancelled.");
534
+ session.stream = stream;
535
+ session.video = video;
536
+ this.updateActiveDevice(session);
537
+ } catch (error) {
538
+ stream.getTracks().forEach((track) => track.stop());
539
+ if (video) video.srcObject = null;
540
+ throw error;
541
+ }
235
542
  })();
236
543
  try {
237
544
  await session.startPromise;
238
545
  } catch (error) {
239
- this.stopCameraSession(session.cameraId);
546
+ if (this.sessions.get(session.cameraId) === session) this.stopCameraSession(session.cameraId);
240
547
  throw error;
241
548
  }
242
549
  }
550
+ nextPreviewBlockRevision(cameraId) {
551
+ const revision = (this.blockPreviewRevisions.get(cameraId) ?? 0) + 1;
552
+ this.blockPreviewRevisions.set(cameraId, revision);
553
+ return revision;
554
+ }
243
555
  updateActiveDevice(session) {
244
556
  const settings = (session.stream?.getVideoTracks()[0] ?? null)?.getSettings();
245
557
  session.activeDeviceId = typeof settings?.deviceId === "string" ? settings.deviceId : "";
246
558
  }
559
+ trackSetting(session, name) {
560
+ const value = session.stream?.getVideoTracks()[0]?.getSettings()[name];
561
+ return typeof value === "number" && Number.isFinite(value) ? value : 0;
562
+ }
563
+ ensurePreview(session) {
564
+ if (session.preview || !session.video) return;
565
+ const runtime = Scratch.vm.runtime;
566
+ session.preview = createVideoPreview(runtime.renderer, session.video, this.previewMirrored(session), () => runtime.requestRedraw?.());
567
+ }
568
+ previewMirrored(session) {
569
+ return [...session.previewLeases.values()].some(Boolean);
570
+ }
247
571
  getFrameSource(session) {
248
572
  if (!session.video || !session.stream) throw new Error("Shared camera is not running.");
249
573
  return Object.freeze({
@@ -258,15 +582,21 @@
258
582
  stopCameraSession(cameraId) {
259
583
  const session = this.sessions.get(cameraId);
260
584
  if (!session) return;
585
+ session.active = false;
586
+ this.nextPreviewBlockRevision(cameraId);
587
+ session.preview?.dispose();
261
588
  session.stream?.getTracks().forEach((track) => track.stop());
262
589
  if (session.video) session.video.srcObject = null;
263
590
  session.stream = null;
264
591
  session.video = null;
265
592
  session.startPromise = null;
593
+ session.preview = null;
266
594
  session.activeDeviceId = "";
267
595
  session.leases.clear();
596
+ session.previewLeases.clear();
268
597
  this.sessions.delete(cameraId);
269
598
  this.blockLeases.delete(cameraId);
599
+ this.blockPreviewLeases.delete(cameraId);
270
600
  }
271
601
  toScratchBlock(block) {
272
602
  return {
@@ -37,6 +37,46 @@
37
37
  }
38
38
  ]
39
39
  },
40
+ {
41
+ "opcode": "cameraFrameHeight",
42
+ "blockType": "REPORTER",
43
+ "arguments": [
44
+ {
45
+ "id": "CAMERA_ID",
46
+ "type": "STRING"
47
+ }
48
+ ]
49
+ },
50
+ {
51
+ "opcode": "cameraFrameRate",
52
+ "blockType": "REPORTER",
53
+ "arguments": [
54
+ {
55
+ "id": "CAMERA_ID",
56
+ "type": "STRING"
57
+ }
58
+ ]
59
+ },
60
+ {
61
+ "opcode": "cameraFrameWidth",
62
+ "blockType": "REPORTER",
63
+ "arguments": [
64
+ {
65
+ "id": "CAMERA_ID",
66
+ "type": "STRING"
67
+ }
68
+ ]
69
+ },
70
+ {
71
+ "opcode": "hideCameraPreview",
72
+ "blockType": "COMMAND",
73
+ "arguments": [
74
+ {
75
+ "id": "CAMERA_ID",
76
+ "type": "STRING"
77
+ }
78
+ ]
79
+ },
40
80
  {
41
81
  "opcode": "isCameraRunning",
42
82
  "blockType": "BOOLEAN",
@@ -52,6 +92,20 @@
52
92
  "blockType": "COMMAND",
53
93
  "arguments": []
54
94
  },
95
+ {
96
+ "opcode": "showCameraPreview",
97
+ "blockType": "COMMAND",
98
+ "arguments": [
99
+ {
100
+ "id": "CAMERA_ID",
101
+ "type": "STRING"
102
+ },
103
+ {
104
+ "id": "MIRRORED",
105
+ "type": "STRING"
106
+ }
107
+ ]
108
+ },
55
109
  {
56
110
  "opcode": "startSharedCamera",
57
111
  "blockType": "COMMAND",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kubohiroya/turbowarp-camera-source",
3
- "version": "0.3.0",
3
+ "version": "0.5.0",
4
4
  "author": "Hiroya Kubo <hiroya@cuc.ac.jp>",
5
5
  "description": "A TurboWarp capability extension for sharing one MediaDevices camera stream across extensions.",
6
6
  "type": "module",
@@ -29,9 +29,27 @@
29
29
  "access": "public",
30
30
  "registry": "https://registry.npmjs.org/"
31
31
  },
32
+ "packageManager": "pnpm@11.18.0",
32
33
  "engines": {
33
34
  "node": ">=22.18.0"
34
35
  },
36
+ "scripts": {
37
+ "dev": "vite build --watch",
38
+ "typecheck": "tsc --noEmit",
39
+ "build": "vite build",
40
+ "lint": "eslint src tests vite.config.ts",
41
+ "format": "eslint src tests vite.config.ts --fix",
42
+ "test": "vitest run",
43
+ "docs": "node scripts/generate-readme.ts",
44
+ "docs:check": "pnpm run docs && git diff --exit-code -- README.md",
45
+ "docs:pages:check": "node scripts/check-pages.ts",
46
+ "repo:check": "node scripts/check-repo.ts",
47
+ "pack:check": "npm pack --dry-run --ignore-scripts",
48
+ "release:check": "pnpm publish --dry-run --access public --no-git-checks",
49
+ "check": "pnpm run typecheck && pnpm run lint && pnpm run test && pnpm run docs:check && pnpm run docs:pages:check && pnpm run check:dist && pnpm run repo:check && pnpm run pack:check",
50
+ "check:dist": "pnpm run build && node scripts/check-dist.ts",
51
+ "prepack": "pnpm run check"
52
+ },
35
53
  "devDependencies": {
36
54
  "@eslint/js": "^9.39.5",
37
55
  "@kubohiroya/vite-plugin-turbowarp-extension": "0.4.0",
@@ -49,20 +67,5 @@
49
67
  "camera",
50
68
  "media-devices",
51
69
  "getusermedia"
52
- ],
53
- "scripts": {
54
- "dev": "vite build --watch",
55
- "typecheck": "tsc --noEmit",
56
- "build": "vite build",
57
- "lint": "eslint src tests vite.config.ts",
58
- "test": "vitest run",
59
- "docs": "node scripts/generate-readme.ts",
60
- "docs:check": "pnpm run docs && git diff --exit-code -- README.md",
61
- "docs:pages:check": "node scripts/check-pages.ts",
62
- "repo:check": "node scripts/check-repo.ts",
63
- "pack:check": "npm pack --dry-run --ignore-scripts",
64
- "release:check": "pnpm publish --dry-run --access public --no-git-checks",
65
- "check": "pnpm run typecheck && pnpm run lint && pnpm run test && pnpm run docs:check && pnpm run docs:pages:check && pnpm run check:dist && pnpm run repo:check && pnpm run pack:check",
66
- "check:dist": "pnpm run build && node scripts/check-dist.ts"
67
- }
68
- }
70
+ ]
71
+ }