@kubohiroya/turbowarp-camera-source 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.ja.md CHANGED
@@ -26,13 +26,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
26
26
  次のURLをunsandboxed custom extensionとして読み込みます。
27
27
 
28
28
  ```text
29
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.2.0/dist/camera-source.js
29
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.4.0/dist/camera-source.js
30
30
  ```
31
31
 
32
32
  npm hostでは次を使います。
33
33
 
34
34
  ```bash
35
- pnpm add @kubohiroya/turbowarp-camera-source@0.2.0
35
+ pnpm add @kubohiroya/turbowarp-camera-source@0.4.0
36
36
  ```
37
37
 
38
38
  ## Quick start
@@ -72,16 +72,41 @@ const qrLease = await cameraSource.acquireCamera({
72
72
  cameraId: "qr",
73
73
  deviceId: qrDeviceId,
74
74
  });
75
+ const previewLease = await cameraSource.acquireCamera({
76
+ owner: "camera-preview",
77
+ cameraId: "pose",
78
+ preview: true,
79
+ mirrored: true,
80
+ });
75
81
  const frame = qrLease.getFrameSource();
82
+ await previewLease.release();
76
83
  await qrLease.release();
77
84
  await poseLease.release();
78
85
  ```
79
86
 
80
- `cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。`frame.element`は`HTMLVideoElement`です。各カメラは最後のleaseが解放されるまで維持されます。
87
+ `preview: true`を指定したleaseは、GPU-backed stage previewを利用します。専用video skinは共有
88
+ `HTMLVideoElement`を`texImage2D(video)`でアップロードし、preview drawableのX scaleで左右反転します。
89
+ `drawImage()`、`getImageData()`、CPUでのフレームピクセル走査は行いません。既定値は`false`で、同じ
90
+ カメラの最後のpreview leaseが解放されるまで表示を維持します。
91
+ 複数のpreview leaseが同じカメラを共有する場合、1つ以上の有効なleaseが`mirrored: true`を要求している間は
92
+ previewを左右反転します。
93
+
94
+ `frame.element`はpreviewを左右反転した場合も未加工の`HTMLVideoElement`です。そのためWebGPUや
95
+ WebCodecsのconsumerは、CPU canvasやpreviewを経由せず、同じsourceを
96
+ `GPUQueue.copyExternalImageToTexture()`や短時間だけ保持する`VideoFrame`へ渡せます。consumerが生成した
97
+ `VideoFrame`はconsumer自身が`close()`する必要があります。
98
+
99
+ この表示経路は明示的なCPU readbackを避けますが、end-to-endの完全なzero-copyは保証しません。ブラウザ内部の
100
+ 色変換やGPU転送が発生する可能性があります。またpreviewはprivateかつnoninteractiveで、表示専用です。
101
+ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いません。
102
+
103
+ `cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。各カメラは最後のleaseが解放されるまで維持されます。
81
104
 
82
105
  ## 互換性
83
106
 
84
- Extension IDは`kubohiroyacamerasource`のままです。ブロックopcode、camera lease ownership、device selection、release semanticsは0.2.0でも変更しません。
107
+ 0.4.0ではopt-inのGPU-backed video previewを追加します。Extension ID、ブロックopcode、camera lease
108
+ ownership、device selection、release semanticsは変更しません。`preview: true`を指定しないconsumerは、
109
+ 従来のcamera-source動作を維持します。
85
110
 
86
111
  ## 開発
87
112
 
package/README.md CHANGED
@@ -30,13 +30,13 @@ or select separate cameras for separate roles.
30
30
  Load this URL as an unsandboxed custom extension:
31
31
 
32
32
  ```text
33
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.2.0/dist/camera-source.js
33
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.4.0/dist/camera-source.js
34
34
  ```
35
35
 
36
36
  For npm hosts:
37
37
 
38
38
  ```bash
39
- pnpm add @kubohiroya/turbowarp-camera-source@0.2.0
39
+ pnpm add @kubohiroya/turbowarp-camera-source@0.4.0
40
40
  ```
41
41
 
42
42
  ## Quick Start
@@ -154,11 +154,40 @@ const qrLease = await cameraSource.acquireCamera({
154
154
  cameraId: "qr",
155
155
  deviceId: qrDeviceId,
156
156
  });
157
+ const previewLease = await cameraSource.acquireCamera({
158
+ owner: "camera-preview",
159
+ cameraId: "pose",
160
+ preview: true,
161
+ mirrored: true,
162
+ });
163
+ const frame = qrLease.getFrameSource();
164
+ await previewLease.release();
165
+ await qrLease.release();
166
+ await poseLease.release();
157
167
  ```
158
168
 
169
+ `preview: true` opts that lease into the GPU-backed stage preview. The dedicated video skin uploads
170
+ the shared `HTMLVideoElement` with `texImage2D(video)` and mirrors by changing the preview drawable's
171
+ X scale. It does not call `drawImage()`, `getImageData()`, or scan frame pixels on the CPU. The option
172
+ defaults to `false`; the preview remains visible until the final preview lease for that camera is
173
+ released.
174
+ If multiple preview leases share a camera, the preview is mirrored while any active preview lease
175
+ requests `mirrored: true`.
176
+
177
+ `frame.element` is the original, unmodified `HTMLVideoElement`, even when the preview is mirrored.
178
+ WebGPU and WebCodecs consumers can therefore use that same source without going through the preview
179
+ or a CPU canvas, for example with `GPUQueue.copyExternalImageToTexture()` or a short-lived
180
+ `VideoFrame`. Consumers own and must close any `VideoFrame` they create.
181
+
182
+ The display path avoids explicit CPU readback, but it does not promise end-to-end zero-copy:
183
+ browser-internal color conversion and GPU transfer may still occur. The preview is private and
184
+ noninteractive; it is intended for display, not Scratch touching or color-sensing queries.
185
+
159
186
  ## Compatibility
160
187
 
161
- The extension ID remains `kubohiroyacamerasource`, and the block opcodes are unchanged. Camera lease ownership, device selection, and release semantics are unchanged in 0.2.0.
188
+ Version 0.4.0 adds the opt-in GPU-backed video preview without changing the extension ID,
189
+ block opcodes, camera lease ownership, device selection, or release semantics. Consumers that do
190
+ not request `preview: true` retain the existing camera-source behavior.
162
191
 
163
192
  ## Development
164
193
 
@@ -194,11 +223,12 @@ The generated JavaScript is a single, non-minified TurboWarp extension file with
194
223
  - `src/config.ts`: extension metadata
195
224
  - `src/block-definitions.json`: canonical block metadata used by both the extension and README generator
196
225
  - `src/extension.ts`: extension implementation
226
+ - `src/video-preview.ts`: GPU-backed `HTMLVideoElement` preview skin and drawable lifecycle
197
227
  - `src/extension-manifest.ts`: canonical manifest generator and Vite output plugin
198
228
  - `src/index.ts`: extension registration entry point
199
229
  - `src/globals.d.ts`: Scratch API declarations used by the project
200
230
  - `schemas/extension-manifest.schema.json`: JSON Schema for the generated API contract
201
- - `scripts/generate-readme.mjs`: updates the generated README block section
231
+ - `scripts/generate-readme.ts`: updates the generated README block section
202
232
  - `tests/`: unit tests
203
233
  - `vite.config.ts`: TurboWarp-compatible Vite build configuration
204
234
  - `dist/`: tracked TurboWarp JavaScript and extension API manifest
@@ -7,205 +7,476 @@
7
7
  (function (Scratch) {
8
8
  'use strict';
9
9
 
10
- const extensionConfig = {
11
- id: "kubohiroyacamerasource",
12
- name: "Camera Source"
10
+ //#region src/config.ts
11
+ var extensionConfig = {
12
+ id: "kubohiroyacamerasource",
13
+ slug: "camera-source",
14
+ name: "Camera Source",
15
+ description: "Share one MediaDevices camera stream between TurboWarp extensions.",
16
+ author: "Hiroya Kubo",
17
+ license: "MPL-2.0",
18
+ unsandboxed: true
13
19
  };
14
- const extensionName = "Camera Source";
15
- const blocks = [{ "opcode": "startSharedCamera", "blockType": "COMMAND", "text": "start shared camera [CAMERA_ID] with device ID [DEVICE_ID]", "description": "Starts or keeps a named shared MediaDevices camera stream.", "arguments": { "CAMERA_ID": { "type": "STRING", "defaultValue": "default" }, "DEVICE_ID": { "type": "STRING", "defaultValue": "" } } }, { "opcode": "stopSharedCamera", "blockType": "COMMAND", "text": "stop shared camera [CAMERA_ID]", "description": "Stops a named shared camera stream and releases its tracks.", "arguments": { "CAMERA_ID": { "type": "STRING", "defaultValue": "default" } } }, { "opcode": "isCameraRunning", "blockType": "BOOLEAN", "text": "shared camera [CAMERA_ID] is running?", "description": "Reports whether a named shared camera stream is active.", "arguments": { "CAMERA_ID": { "type": "STRING", "defaultValue": "default" } } }, { "opcode": "cameraDeviceIdReporter", "blockType": "REPORTER", "text": "shared camera [CAMERA_ID] device ID", "description": "Returns the active device ID for a named shared camera when available.", "arguments": { "CAMERA_ID": { "type": "STRING", "defaultValue": "default" } } }, { "opcode": "refreshCameraDevices", "blockType": "COMMAND", "text": "refresh camera devices", "description": "Refreshes the browser camera device list.", "arguments": {} }, { "opcode": "cameraDeviceCount", "blockType": "REPORTER", "text": "camera device count", "description": "Returns the number of known camera devices after refresh.", "arguments": {} }, { "opcode": "cameraDeviceIdAt", "blockType": "REPORTER", "text": "camera device ID at [INDEX]", "description": "Returns the one-based camera device ID at the requested index.", "arguments": { "INDEX": { "type": "STRING", "defaultValue": "1" } } }, { "opcode": "cameraDeviceLabelAt", "blockType": "REPORTER", "text": "camera device label at [INDEX]", "description": "Returns the one-based camera device label at the requested index when the browser exposes it.", "arguments": { "INDEX": { "type": "STRING", "defaultValue": "1" } } }];
16
- const definitions = {
17
- extensionName,
18
- blocks
20
+ var block_definitions_default = {
21
+ extensionName: "Camera Source",
22
+ blocks: [
23
+ {
24
+ "opcode": "startSharedCamera",
25
+ "blockType": "COMMAND",
26
+ "text": "start shared camera [CAMERA_ID] with device ID [DEVICE_ID]",
27
+ "description": "Starts or keeps a named shared MediaDevices camera stream.",
28
+ "arguments": {
29
+ "CAMERA_ID": {
30
+ "type": "STRING",
31
+ "defaultValue": "default"
32
+ },
33
+ "DEVICE_ID": {
34
+ "type": "STRING",
35
+ "defaultValue": ""
36
+ }
37
+ }
38
+ },
39
+ {
40
+ "opcode": "stopSharedCamera",
41
+ "blockType": "COMMAND",
42
+ "text": "stop shared camera [CAMERA_ID]",
43
+ "description": "Stops a named shared camera stream and releases its tracks.",
44
+ "arguments": { "CAMERA_ID": {
45
+ "type": "STRING",
46
+ "defaultValue": "default"
47
+ } }
48
+ },
49
+ {
50
+ "opcode": "isCameraRunning",
51
+ "blockType": "BOOLEAN",
52
+ "text": "shared camera [CAMERA_ID] is running?",
53
+ "description": "Reports whether a named shared camera stream is active.",
54
+ "arguments": { "CAMERA_ID": {
55
+ "type": "STRING",
56
+ "defaultValue": "default"
57
+ } }
58
+ },
59
+ {
60
+ "opcode": "cameraDeviceIdReporter",
61
+ "blockType": "REPORTER",
62
+ "text": "shared camera [CAMERA_ID] device ID",
63
+ "description": "Returns the active device ID for a named shared camera when available.",
64
+ "arguments": { "CAMERA_ID": {
65
+ "type": "STRING",
66
+ "defaultValue": "default"
67
+ } }
68
+ },
69
+ {
70
+ "opcode": "refreshCameraDevices",
71
+ "blockType": "COMMAND",
72
+ "text": "refresh camera devices",
73
+ "description": "Refreshes the browser camera device list.",
74
+ "arguments": {}
75
+ },
76
+ {
77
+ "opcode": "cameraDeviceCount",
78
+ "blockType": "REPORTER",
79
+ "text": "camera device count",
80
+ "description": "Returns the number of known camera devices after refresh.",
81
+ "arguments": {}
82
+ },
83
+ {
84
+ "opcode": "cameraDeviceIdAt",
85
+ "blockType": "REPORTER",
86
+ "text": "camera device ID at [INDEX]",
87
+ "description": "Returns the one-based camera device ID at the requested index.",
88
+ "arguments": { "INDEX": {
89
+ "type": "STRING",
90
+ "defaultValue": "1"
91
+ } }
92
+ },
93
+ {
94
+ "opcode": "cameraDeviceLabelAt",
95
+ "blockType": "REPORTER",
96
+ "text": "camera device label at [INDEX]",
97
+ "description": "Returns the one-based camera device label at the requested index when the browser exposes it.",
98
+ "arguments": { "INDEX": {
99
+ "type": "STRING",
100
+ "defaultValue": "1"
101
+ } }
102
+ }
103
+ ]
19
104
  };
20
- const blockDefinitions = definitions.blocks;
21
- const defaultCameraId = "default";
105
+ //#endregion
106
+ //#region src/video-preview.ts
107
+ var videoLayer = "video";
108
+ var haveCurrentData = 2;
109
+ function videoSkinClass(renderer) {
110
+ const BaseSkin = renderer.exports?.Skin;
111
+ if (typeof BaseSkin !== "function") throw new Error("Camera preview requires renderer.exports.Skin.");
112
+ return class VideoSkin extends BaseSkin {
113
+ constructor(id, skinRenderer, video, onFrame) {
114
+ super(id, skinRenderer);
115
+ this.texture = null;
116
+ this.textureSize = [0, 0];
117
+ this.dirty = true;
118
+ this.disposed = false;
119
+ this.videoFrameCallbackId = null;
120
+ this.animationFrameId = null;
121
+ this.lastCurrentTime = NaN;
122
+ this.handleVideoFrame = () => {
123
+ if (this.disposed) return;
124
+ this.dirty = true;
125
+ this.onFrame(this.syncMetrics());
126
+ this.scheduleFrame();
127
+ };
128
+ this.handleAnimationFrame = () => {
129
+ if (this.disposed) return;
130
+ if (this.video.currentTime !== this.lastCurrentTime) {
131
+ this.dirty = true;
132
+ this.onFrame(this.syncMetrics());
133
+ }
134
+ this.scheduleFrame();
135
+ };
136
+ this.video = video;
137
+ this.onFrame = onFrame;
138
+ this.private = true;
139
+ this.syncMetrics();
140
+ this.scheduleFrame();
141
+ }
142
+ get size() {
143
+ return this.textureSize;
144
+ }
145
+ getTexture() {
146
+ if (this.disposed || this.video.readyState < haveCurrentData || this.video.videoWidth === 0 || this.video.videoHeight === 0) return null;
147
+ if (this.syncMetrics()) this.onFrame(true);
148
+ if (!this.dirty && this.video.currentTime === this.lastCurrentTime) return this.texture;
149
+ const gl = renderer.gl;
150
+ if (!this.texture) {
151
+ this.texture = gl.createTexture();
152
+ if (!this.texture) throw new Error("Camera preview could not create a WebGL texture.");
153
+ gl.bindTexture(gl.TEXTURE_2D, this.texture);
154
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
155
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
156
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
157
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
158
+ } else gl.bindTexture(gl.TEXTURE_2D, this.texture);
159
+ gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true);
160
+ try {
161
+ gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, this.video);
162
+ } finally {
163
+ gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
164
+ }
165
+ this.dirty = false;
166
+ this.lastCurrentTime = this.video.currentTime;
167
+ return this.texture;
168
+ }
169
+ dispose() {
170
+ this.disposed = true;
171
+ if (this.videoFrameCallbackId !== null && this.video.cancelVideoFrameCallback) this.video.cancelVideoFrameCallback(this.videoFrameCallbackId);
172
+ if (this.animationFrameId !== null && typeof globalThis.cancelAnimationFrame === "function") globalThis.cancelAnimationFrame(this.animationFrameId);
173
+ if (this.texture) renderer.gl.deleteTexture(this.texture);
174
+ this.texture = null;
175
+ super.dispose();
176
+ }
177
+ useNearest() {
178
+ return false;
179
+ }
180
+ updateSilhouette() {}
181
+ syncMetrics() {
182
+ const width = this.video.videoWidth;
183
+ const height = this.video.videoHeight;
184
+ if (width === this.textureSize[0] && height === this.textureSize[1]) return false;
185
+ this.textureSize = [width, height];
186
+ this.rotationCenter[0] = width / 2;
187
+ this.rotationCenter[1] = height / 2;
188
+ return true;
189
+ }
190
+ scheduleFrame() {
191
+ if (typeof this.video.requestVideoFrameCallback === "function") this.videoFrameCallbackId = this.video.requestVideoFrameCallback(this.handleVideoFrame);
192
+ else if (typeof globalThis.requestAnimationFrame === "function") this.animationFrameId = globalThis.requestAnimationFrame(this.handleAnimationFrame);
193
+ }
194
+ };
195
+ }
196
+ function assertRenderer(renderer) {
197
+ if (!renderer || !Array.isArray(renderer._allSkins) || !Number.isInteger(renderer._nextSkinId) || typeof renderer.createDrawable !== "function" || typeof renderer.destroyDrawable !== "function" || typeof renderer.destroySkin !== "function" || typeof renderer.getNativeSize !== "function") throw new Error("Camera preview requires a compatible TurboWarp renderer.");
198
+ }
199
+ function createVideoPreview(renderer, video, mirrored, requestRedraw) {
200
+ assertRenderer(renderer);
201
+ const VideoSkin = videoSkinClass(renderer);
202
+ const skinId = renderer._nextSkinId++;
203
+ let drawableId;
204
+ let disposed = false;
205
+ let previewMirrored = mirrored;
206
+ let lastLayout = "";
207
+ const updateLayout = () => {
208
+ if (drawableId === void 0 || video.videoWidth === 0 || video.videoHeight === 0) return;
209
+ const [stageWidth, stageHeight] = renderer.getNativeSize();
210
+ const layout = [
211
+ video.videoWidth,
212
+ video.videoHeight,
213
+ stageWidth,
214
+ stageHeight,
215
+ previewMirrored
216
+ ].join(":");
217
+ if (layout === lastLayout) return;
218
+ const scale = Math.max(stageWidth / video.videoWidth, stageHeight / video.videoHeight) * 100;
219
+ renderer.updateDrawableScale(drawableId, [previewMirrored ? -scale : scale, scale]);
220
+ lastLayout = layout;
221
+ };
222
+ const skin = new VideoSkin(skinId, renderer, video, (metricsChanged) => {
223
+ if (metricsChanged) skin.emitWasAltered();
224
+ updateLayout();
225
+ requestRedraw();
226
+ });
227
+ renderer._allSkins[skinId] = skin;
228
+ try {
229
+ drawableId = renderer.createDrawable(videoLayer);
230
+ if (drawableId === void 0) throw new Error("Camera preview could not create a video drawable.");
231
+ renderer.updateDrawableSkinId(drawableId, skinId);
232
+ renderer.updateDrawablePosition(drawableId, [0, 0]);
233
+ updateLayout();
234
+ renderer.updateDrawableVisible(drawableId, true);
235
+ renderer.markSkinAsPrivate?.(skinId);
236
+ renderer.markDrawableAsNoninteractive?.(drawableId);
237
+ requestRedraw();
238
+ } catch (error) {
239
+ try {
240
+ if (drawableId !== void 0) renderer.destroyDrawable(drawableId, videoLayer);
241
+ } finally {
242
+ renderer.destroySkin(skinId);
243
+ }
244
+ throw error;
245
+ }
246
+ return Object.freeze({
247
+ setMirrored: (nextMirrored) => {
248
+ if (disposed || previewMirrored === nextMirrored) return;
249
+ previewMirrored = nextMirrored;
250
+ updateLayout();
251
+ requestRedraw();
252
+ },
253
+ dispose: () => {
254
+ if (disposed) return;
255
+ disposed = true;
256
+ try {
257
+ if (drawableId !== void 0) renderer.destroyDrawable(drawableId, videoLayer);
258
+ } finally {
259
+ try {
260
+ renderer.destroySkin(skinId);
261
+ } finally {
262
+ requestRedraw();
263
+ }
264
+ }
265
+ }
266
+ });
267
+ }
268
+ //#endregion
269
+ //#region src/extension.ts
270
+ var blockDefinitions = block_definitions_default.blocks;
271
+ var defaultCameraId = "default";
22
272
  function mediaDevices() {
23
- const devices = globalThis.navigator?.mediaDevices;
24
- if (!devices || typeof devices.getUserMedia !== "function") {
25
- throw new Error("Camera Source requires navigator.mediaDevices.getUserMedia.");
26
- }
27
- return devices;
273
+ const devices = globalThis.navigator?.mediaDevices;
274
+ if (!devices || typeof devices.getUserMedia !== "function") throw new Error("Camera Source requires navigator.mediaDevices.getUserMedia.");
275
+ return devices;
28
276
  }
29
277
  function normalizeId(value, fallback = defaultCameraId) {
30
- const text = String(value ?? "").trim();
31
- return text || fallback;
278
+ return String(value ?? "").trim() || fallback;
32
279
  }
33
280
  function optionalText(value) {
34
- return String(value ?? "").trim();
281
+ return String(value ?? "").trim();
35
282
  }
36
283
  function indexFrom(value) {
37
- const parsed = Number.parseInt(String(value ?? ""), 10);
38
- return Number.isFinite(parsed) ? parsed - 1 : -1;
284
+ const parsed = Number.parseInt(String(value ?? ""), 10);
285
+ return Number.isFinite(parsed) ? parsed - 1 : -1;
39
286
  }
40
287
  function videoConstraints(options) {
41
- if (typeof options.video === "object" && options.video !== null) {
42
- return { audio: false, video: options.video };
43
- }
44
- if (options.deviceId) {
45
- return { audio: false, video: { deviceId: { exact: options.deviceId } } };
46
- }
47
- return { audio: false, video: options.video ?? true };
48
- }
49
- class CameraSourceExtension {
50
- constructor() {
51
- this.sessions = /* @__PURE__ */ new Map();
52
- this.blockLeases = /* @__PURE__ */ new Map();
53
- this.devices = [];
54
- Scratch.vm.runtime.ext_kubohiroyacamerasource = this;
55
- }
56
- getInfo() {
57
- return {
58
- id: extensionConfig.id,
59
- name: Scratch.translate(definitions.extensionName),
60
- blocks: blockDefinitions.map((block) => this.toScratchBlock(block))
61
- };
62
- }
63
- isCameraRunning(args = {}) {
64
- const session = this.sessions.get(normalizeId(args.CAMERA_ID));
65
- return Boolean(session?.stream);
66
- }
67
- cameraDeviceIdReporter(args = {}) {
68
- return this.sessions.get(normalizeId(args.CAMERA_ID))?.activeDeviceId ?? "";
69
- }
70
- async startSharedCamera(args = {}) {
71
- const cameraId = normalizeId(args.CAMERA_ID);
72
- if (this.blockLeases.has(cameraId)) return;
73
- const deviceId = optionalText(args.DEVICE_ID);
74
- const options = { owner: "camera-source-block", cameraId };
75
- if (deviceId) options.deviceId = deviceId;
76
- const lease = await this.acquireCamera(options);
77
- this.blockLeases.set(cameraId, lease);
78
- }
79
- async acquireCamera(options = {}) {
80
- const cameraId = normalizeId(options.cameraId);
81
- const session = this.session(cameraId);
82
- const token = Symbol(String(options.owner ?? "camera-lease"));
83
- if (session.startPromise) {
84
- await session.startPromise;
85
- } else if (!session.stream) {
86
- await this.start(session, options);
87
- }
88
- session.leases.add(token);
89
- let released = false;
90
- return Object.freeze({
91
- getFrameSource: () => this.getFrameSource(session),
92
- release: async () => {
93
- if (released) return;
94
- released = true;
95
- session.leases.delete(token);
96
- if (session.leases.size === 0) this.stopCameraSession(session.cameraId);
97
- }
98
- });
99
- }
100
- stopSharedCamera(args = {}) {
101
- this.stopCameraSession(normalizeId(args.CAMERA_ID));
102
- }
103
- async refreshCameraDevices() {
104
- this.devices = (await mediaDevices().enumerateDevices()).filter(
105
- (device) => device.kind === "videoinput"
106
- );
107
- }
108
- cameraDeviceCount() {
109
- return this.devices.length;
110
- }
111
- cameraDeviceIdAt(args) {
112
- return this.devices[indexFrom(args.INDEX)]?.deviceId ?? "";
113
- }
114
- cameraDeviceLabelAt(args) {
115
- return this.devices[indexFrom(args.INDEX)]?.label ?? "";
116
- }
117
- stopAllCameras() {
118
- for (const cameraId of [...this.sessions.keys()]) {
119
- this.stopCameraSession(cameraId);
120
- }
121
- this.blockLeases.clear();
122
- }
123
- session(cameraId) {
124
- const existing = this.sessions.get(cameraId);
125
- if (existing) return existing;
126
- const session = {
127
- cameraId,
128
- leases: /* @__PURE__ */ new Set(),
129
- stream: null,
130
- video: null,
131
- startPromise: null,
132
- mirrored: false,
133
- activeDeviceId: ""
134
- };
135
- this.sessions.set(cameraId, session);
136
- return session;
137
- }
138
- async start(session, options) {
139
- session.mirrored = options.mirrored === true;
140
- session.startPromise = (async () => {
141
- const stream = await mediaDevices().getUserMedia(videoConstraints(options));
142
- const video = document.createElement("video");
143
- video.muted = true;
144
- video.playsInline = true;
145
- video.srcObject = stream;
146
- await video.play();
147
- session.stream = stream;
148
- session.video = video;
149
- this.updateActiveDevice(session);
150
- })();
151
- try {
152
- await session.startPromise;
153
- } catch (error) {
154
- this.stopCameraSession(session.cameraId);
155
- throw error;
156
- }
157
- }
158
- updateActiveDevice(session) {
159
- const track = session.stream?.getVideoTracks()[0] ?? null;
160
- const settings = track?.getSettings();
161
- session.activeDeviceId = typeof settings?.deviceId === "string" ? settings.deviceId : "";
162
- }
163
- getFrameSource(session) {
164
- if (!session.video || !session.stream) {
165
- throw new Error("Shared camera is not running.");
166
- }
167
- return Object.freeze({
168
- kind: "video",
169
- element: session.video,
170
- width: session.video.videoWidth,
171
- height: session.video.videoHeight,
172
- mirrored: session.mirrored,
173
- deviceId: session.activeDeviceId
174
- });
175
- }
176
- stopCameraSession(cameraId) {
177
- const session = this.sessions.get(cameraId);
178
- if (!session) return;
179
- session.stream?.getTracks().forEach((track) => track.stop());
180
- if (session.video) session.video.srcObject = null;
181
- session.stream = null;
182
- session.video = null;
183
- session.startPromise = null;
184
- session.activeDeviceId = "";
185
- session.leases.clear();
186
- this.sessions.delete(cameraId);
187
- this.blockLeases.delete(cameraId);
188
- }
189
- toScratchBlock(block) {
190
- return {
191
- opcode: block.opcode,
192
- blockType: Scratch.BlockType[block.blockType],
193
- text: Scratch.translate(block.text),
194
- arguments: Object.fromEntries(
195
- Object.entries(block.arguments).map(([name, argument]) => [
196
- name,
197
- {
198
- type: Scratch.ArgumentType[argument.type],
199
- defaultValue: argument.defaultValue
200
- }
201
- ])
202
- )
203
- };
204
- }
205
- }
206
- if (!Scratch.extensions.unsandboxed) {
207
- throw new Error(`${extensionConfig.name} must run unsandboxed.`);
288
+ if (typeof options.video === "object" && options.video !== null) return {
289
+ audio: false,
290
+ video: options.video
291
+ };
292
+ if (options.deviceId) return {
293
+ audio: false,
294
+ video: { deviceId: { exact: options.deviceId } }
295
+ };
296
+ return {
297
+ audio: false,
298
+ video: options.video ?? true
299
+ };
208
300
  }
301
+ var CameraSourceExtension = class {
302
+ constructor() {
303
+ this.sessions = /* @__PURE__ */ new Map();
304
+ this.blockLeases = /* @__PURE__ */ new Map();
305
+ this.devices = [];
306
+ Scratch.vm.runtime.ext_kubohiroyacamerasource = this;
307
+ }
308
+ getInfo() {
309
+ return {
310
+ id: extensionConfig.id,
311
+ name: Scratch.translate(block_definitions_default.extensionName),
312
+ blocks: blockDefinitions.map((block) => this.toScratchBlock(block))
313
+ };
314
+ }
315
+ isCameraRunning(args = {}) {
316
+ const session = this.sessions.get(normalizeId(args.CAMERA_ID));
317
+ return Boolean(session?.stream);
318
+ }
319
+ cameraDeviceIdReporter(args = {}) {
320
+ return this.sessions.get(normalizeId(args.CAMERA_ID))?.activeDeviceId ?? "";
321
+ }
322
+ async startSharedCamera(args = {}) {
323
+ const cameraId = normalizeId(args.CAMERA_ID);
324
+ if (this.blockLeases.has(cameraId)) return;
325
+ const deviceId = optionalText(args.DEVICE_ID);
326
+ const options = {
327
+ owner: "camera-source-block",
328
+ cameraId
329
+ };
330
+ if (deviceId) options.deviceId = deviceId;
331
+ const lease = await this.acquireCamera(options);
332
+ this.blockLeases.set(cameraId, lease);
333
+ }
334
+ async acquireCamera(options = {}) {
335
+ const cameraId = normalizeId(options.cameraId);
336
+ const session = this.session(cameraId);
337
+ const token = Symbol(String(options.owner ?? "camera-lease"));
338
+ if (session.startPromise) await session.startPromise;
339
+ else if (!session.stream) await this.start(session, options);
340
+ session.leases.add(token);
341
+ try {
342
+ if (options.preview === true) {
343
+ session.previewLeases.set(token, options.mirrored === true);
344
+ this.ensurePreview(session);
345
+ session.preview?.setMirrored(this.previewMirrored(session));
346
+ }
347
+ } catch (error) {
348
+ session.leases.delete(token);
349
+ session.previewLeases.delete(token);
350
+ if (session.leases.size === 0) this.stopCameraSession(session.cameraId);
351
+ throw error;
352
+ }
353
+ let released = false;
354
+ return Object.freeze({
355
+ getFrameSource: () => this.getFrameSource(session),
356
+ release: async () => {
357
+ if (released) return;
358
+ released = true;
359
+ session.leases.delete(token);
360
+ session.previewLeases.delete(token);
361
+ if (session.previewLeases.size === 0) {
362
+ session.preview?.dispose();
363
+ session.preview = null;
364
+ } else session.preview?.setMirrored(this.previewMirrored(session));
365
+ if (session.leases.size === 0) this.stopCameraSession(session.cameraId);
366
+ }
367
+ });
368
+ }
369
+ stopSharedCamera(args = {}) {
370
+ this.stopCameraSession(normalizeId(args.CAMERA_ID));
371
+ }
372
+ async refreshCameraDevices() {
373
+ this.devices = (await mediaDevices().enumerateDevices()).filter((device) => device.kind === "videoinput");
374
+ }
375
+ cameraDeviceCount() {
376
+ return this.devices.length;
377
+ }
378
+ cameraDeviceIdAt(args) {
379
+ return this.devices[indexFrom(args.INDEX)]?.deviceId ?? "";
380
+ }
381
+ cameraDeviceLabelAt(args) {
382
+ return this.devices[indexFrom(args.INDEX)]?.label ?? "";
383
+ }
384
+ stopAllCameras() {
385
+ for (const cameraId of [...this.sessions.keys()]) this.stopCameraSession(cameraId);
386
+ this.blockLeases.clear();
387
+ }
388
+ session(cameraId) {
389
+ const existing = this.sessions.get(cameraId);
390
+ if (existing) return existing;
391
+ const session = {
392
+ cameraId,
393
+ leases: /* @__PURE__ */ new Set(),
394
+ previewLeases: /* @__PURE__ */ new Map(),
395
+ stream: null,
396
+ video: null,
397
+ preview: null,
398
+ startPromise: null,
399
+ mirrored: false,
400
+ activeDeviceId: ""
401
+ };
402
+ this.sessions.set(cameraId, session);
403
+ return session;
404
+ }
405
+ async start(session, options) {
406
+ session.mirrored = options.mirrored === true;
407
+ session.startPromise = (async () => {
408
+ const stream = await mediaDevices().getUserMedia(videoConstraints(options));
409
+ const video = document.createElement("video");
410
+ video.muted = true;
411
+ video.playsInline = true;
412
+ video.srcObject = stream;
413
+ await video.play();
414
+ session.stream = stream;
415
+ session.video = video;
416
+ this.updateActiveDevice(session);
417
+ })();
418
+ try {
419
+ await session.startPromise;
420
+ } catch (error) {
421
+ this.stopCameraSession(session.cameraId);
422
+ throw error;
423
+ }
424
+ }
425
+ updateActiveDevice(session) {
426
+ const settings = (session.stream?.getVideoTracks()[0] ?? null)?.getSettings();
427
+ session.activeDeviceId = typeof settings?.deviceId === "string" ? settings.deviceId : "";
428
+ }
429
+ ensurePreview(session) {
430
+ if (session.preview || !session.video) return;
431
+ const runtime = Scratch.vm.runtime;
432
+ session.preview = createVideoPreview(runtime.renderer, session.video, this.previewMirrored(session), () => runtime.requestRedraw?.());
433
+ }
434
+ previewMirrored(session) {
435
+ return [...session.previewLeases.values()].some(Boolean);
436
+ }
437
+ getFrameSource(session) {
438
+ if (!session.video || !session.stream) throw new Error("Shared camera is not running.");
439
+ return Object.freeze({
440
+ kind: "video",
441
+ element: session.video,
442
+ width: session.video.videoWidth,
443
+ height: session.video.videoHeight,
444
+ mirrored: session.mirrored,
445
+ deviceId: session.activeDeviceId
446
+ });
447
+ }
448
+ stopCameraSession(cameraId) {
449
+ const session = this.sessions.get(cameraId);
450
+ if (!session) return;
451
+ session.preview?.dispose();
452
+ session.stream?.getTracks().forEach((track) => track.stop());
453
+ if (session.video) session.video.srcObject = null;
454
+ session.stream = null;
455
+ session.video = null;
456
+ session.startPromise = null;
457
+ session.preview = null;
458
+ session.activeDeviceId = "";
459
+ session.leases.clear();
460
+ session.previewLeases.clear();
461
+ this.sessions.delete(cameraId);
462
+ this.blockLeases.delete(cameraId);
463
+ }
464
+ toScratchBlock(block) {
465
+ return {
466
+ opcode: block.opcode,
467
+ blockType: Scratch.BlockType[block.blockType],
468
+ text: Scratch.translate(block.text),
469
+ arguments: Object.fromEntries(Object.entries(block.arguments).map(([name, argument]) => [name, {
470
+ type: Scratch.ArgumentType[argument.type],
471
+ defaultValue: argument.defaultValue
472
+ }]))
473
+ };
474
+ }
475
+ };
476
+ //#endregion
477
+ //#region src/index.ts
478
+ if (extensionConfig.unsandboxed && !Scratch.extensions.unsandboxed) throw new Error(`${extensionConfig.name} must run unsandboxed.`);
209
479
  Scratch.extensions.register(new CameraSourceExtension());
480
+ //#endregion
210
481
 
211
482
  })(Scratch);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kubohiroya/turbowarp-camera-source",
3
- "version": "0.2.0",
3
+ "version": "0.4.0",
4
4
  "author": "Hiroya Kubo <hiroya@cuc.ac.jp>",
5
5
  "description": "A TurboWarp capability extension for sharing one MediaDevices camera stream across extensions.",
6
6
  "type": "module",
@@ -29,34 +29,17 @@
29
29
  "access": "public",
30
30
  "registry": "https://registry.npmjs.org/"
31
31
  },
32
- "packageManager": "pnpm@11.18.0",
33
32
  "engines": {
34
- "node": ">=22"
35
- },
36
- "scripts": {
37
- "dev": "vite build --watch",
38
- "typecheck": "tsc --noEmit",
39
- "build": "vite build",
40
- "lint": "eslint src tests vite.config.ts",
41
- "test": "vitest run",
42
- "docs": "node scripts/generate-readme.mjs",
43
- "docs:check": "pnpm run docs && git diff --exit-code -- README.md",
44
- "docs:pages:check": "node scripts/check-pages.mjs",
45
- "repo:check": "node scripts/check-repo.mjs",
46
- "pack:check": "npm pack --dry-run --ignore-scripts",
47
- "release:check": "pnpm publish --dry-run --access public --no-git-checks",
48
- "check": "pnpm run typecheck && pnpm run lint && pnpm run test && pnpm run docs:check && pnpm run docs:pages:check && pnpm run check:dist && pnpm run repo:check && pnpm run pack:check",
49
- "check:dist": "pnpm run build && node scripts/check-dist.mjs",
50
- "prepack": "pnpm run check"
33
+ "node": ">=22.18.0"
51
34
  },
52
35
  "devDependencies": {
53
36
  "@eslint/js": "^9.39.5",
54
- "@kubohiroya/vite-plugin-turbowarp-extension": "0.1.1",
37
+ "@kubohiroya/vite-plugin-turbowarp-extension": "0.4.0",
55
38
  "@types/node": "^24.13.3",
56
39
  "eslint": "^9.39.5",
57
40
  "typescript": "^5.9.3",
58
41
  "typescript-eslint": "^8.66.0",
59
- "vite": "^7.3.6",
42
+ "vite": "^8.2.2",
60
43
  "vitest": "^4.1.10"
61
44
  },
62
45
  "keywords": [
@@ -66,5 +49,20 @@
66
49
  "camera",
67
50
  "media-devices",
68
51
  "getusermedia"
69
- ]
70
- }
52
+ ],
53
+ "scripts": {
54
+ "dev": "vite build --watch",
55
+ "typecheck": "tsc --noEmit",
56
+ "build": "vite build",
57
+ "lint": "eslint src tests vite.config.ts",
58
+ "test": "vitest run",
59
+ "docs": "node scripts/generate-readme.ts",
60
+ "docs:check": "pnpm run docs && git diff --exit-code -- README.md",
61
+ "docs:pages:check": "node scripts/check-pages.ts",
62
+ "repo:check": "node scripts/check-repo.ts",
63
+ "pack:check": "npm pack --dry-run --ignore-scripts",
64
+ "release:check": "pnpm publish --dry-run --access public --no-git-checks",
65
+ "check": "pnpm run typecheck && pnpm run lint && pnpm run test && pnpm run docs:check && pnpm run docs:pages:check && pnpm run check:dist && pnpm run repo:check && pnpm run pack:check",
66
+ "check:dist": "pnpm run build && node scripts/check-dist.ts"
67
+ }
68
+ }