@kubohiroya/turbowarp-camera-source 0.19.4 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja.md +50 -6
- package/README.md +60 -10
- package/assets/block-icon.svg +1 -0
- package/dist/camera-source.js +5230 -5081
- package/dist/runtime.d.ts +18 -0
- package/dist/runtime.d.ts.map +1 -1
- package/dist/runtime.js.map +1 -1
- package/dist/snapshot.d.ts +59 -0
- package/dist/snapshot.d.ts.map +1 -0
- package/dist/snapshot.js +137 -0
- package/dist/snapshot.js.map +1 -0
- package/lib-editor/src/config.d.ts +1 -0
- package/lib-editor/src/config.js +5 -1
- package/package.json +8 -7
package/README.ja.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
|
-
# TurboWarp
|
|
1
|
+
# TurboWarp Camera Source
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
<img src="assets/block-icon.svg" alt="" width="48" height="48">
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[English](README.md) | 日本語
|
|
6
|
+
|
|
7
|
+
TurboWarp Camera Sourceは、MediaDevicesのカメラストリームを名前付きで複数のTurboWarp拡張へ共有するcapability拡張です。leaseで起動と停止を管理し、TM、QR読み取り、将来の画像入力が同じ物理カメラを共有する場合も、姿勢認識用とQR/画像認識用で別カメラを使う場合も、`getUserMedia()`の競合を避けます。
|
|
6
8
|
|
|
7
9
|
**[English guide](https://kubohiroya.github.io/turbowarp-camera-source/)** ·
|
|
8
10
|
**[日本語ガイド](https://kubohiroya.github.io/turbowarp-camera-source/ja/)**
|
|
@@ -32,13 +34,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
|
|
|
32
34
|
次のURLをunsandboxed custom extensionとして読み込みます。
|
|
33
35
|
|
|
34
36
|
```text
|
|
35
|
-
https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.
|
|
37
|
+
https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.21.0/dist/camera-source.js
|
|
36
38
|
```
|
|
37
39
|
|
|
38
40
|
npm hostでは次を使います。
|
|
39
41
|
|
|
40
42
|
```bash
|
|
41
|
-
pnpm add @kubohiroya/turbowarp-camera-source@0.
|
|
43
|
+
pnpm add @kubohiroya/turbowarp-camera-source@0.21.0
|
|
42
44
|
```
|
|
43
45
|
|
|
44
46
|
## 最初の使い方
|
|
@@ -62,7 +64,7 @@ stop shared camera [pose]
|
|
|
62
64
|
|
|
63
65
|
- `start shared camera [CAMERA_ID] with device ID [DEVICE_ID]`: 名前付き共有カメラを開始します。
|
|
64
66
|
- `start shared camera [CAMERA_ID] requested by this page`: ページのクエリパラメータが指定するカメラを開始します。`cameraDeviceId`は厳密に、`cameraWidth`・`cameraHeight`・`cameraFrameRate`は理想値として要求します。指定がなければ、デバイスを指定しない`start shared camera`と同じです。複数のUSBカメラを使うアプリが、同じoriginの別のアプリ(レンズ校正アプリなど)をそのうち1台のために開くときの指定方法です。
|
|
65
|
-
- `start shared camera [CAMERA_ID] from the browser's camera button`: ブラウザ自身のカメラボタンをステージに置き、操作者が答えるまで待って、返ってきたstreamをこのカメラIDで共有します(`getUserMedia`は呼びません)。エディタで開いた`.sb3`のように、作品の上に要素を置くhostが居ない場合のためのブロックです。そのカメラIDがすでにstreamを持つとき——hostが採用したpackaged作品——は、訊き直さずそのstreamに合流します。拒否されても作品は動き続けます。カメラは起動せず、`shared camera error code`は`usermedia-cancelled`になり、`start shared camera`も使えます。要素を持たないブラウザでは`usermedia-unsupported
|
|
67
|
+
- `start shared camera [CAMERA_ID] from the browser's camera button`: ブラウザ自身のカメラボタンをステージに置き、操作者が答えるまで待って、返ってきたstreamをこのカメラIDで共有します(`getUserMedia`は呼びません)。エディタで開いた`.sb3`のように、作品の上に要素を置くhostが居ない場合のためのブロックです。そのカメラIDがすでにstreamを持つとき——hostが採用したpackaged作品——は、訊き直さずそのstreamに合流します。拒否されても作品は動き続けます。カメラは起動せず、`shared camera error code`は`usermedia-cancelled`になり、`start shared camera`も使えます。要素を持たないブラウザでは`usermedia-unsupported`を報告します。要素はカメラだけでなくマイクも返しますが、そのtrackは共有の前に止めて外すので、誰も聞き取りません。ブラウザ自身の許可画面にマイクが出ることは、将来の`<camera>`要素でなければ変えられません。
|
|
66
68
|
- `hide the camera button for shared camera [CAMERA_ID]`: まだ答えを待っているカメラボタンを取り除きます。待っていたスクリプトは続行し、カメラのエラーコードは`usermedia-cancelled`になります。停止ボタンや別の作品の読み込みでも取り除かれます。
|
|
67
69
|
- `browser's camera button available?`: このブラウザがカメラボタンの要素(`<usermedia>`、Chrome 151以降)を持つかどうかです。ボタンを出す前に確認し、偽のときは`start shared camera`に落としてください。
|
|
68
70
|
- `stop shared camera [CAMERA_ID]`: 名前付き共有カメラを停止し、MediaStreamTrackを解放します。
|
|
@@ -186,6 +188,47 @@ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いま
|
|
|
186
188
|
|
|
187
189
|
`cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。各カメラは最後のleaseが解放されるまで維持されます。
|
|
188
190
|
|
|
191
|
+
### 静止画の取得
|
|
192
|
+
|
|
193
|
+
画素ではなくファイルが欲しいconsumer——アセット管理、ダウンロード、最新フレームを配るサーバ——は、leaseに
|
|
194
|
+
1枚求められます。画像を自分で処理するconsumerは、encodeを挟まず`frame.element`を読んでください。
|
|
195
|
+
|
|
196
|
+
```js
|
|
197
|
+
if (typeof lease.captureFrame === "function") {
|
|
198
|
+
const shot = await lease.captureFrame({type: "image/jpeg", quality: 0.8, maxWidth: 1280});
|
|
199
|
+
// shot.bytesはUint8Array。shot.type・shot.width・shot.height・shot.capturedAtがそれを説明します。
|
|
200
|
+
}
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
`adoptCamera()`と同じ理由でfeature detectしてください。この面はversionではなく形で狭めているので、現在の宣言
|
|
204
|
+
でcompileしたconsumerが、このメソッドより古いbuildに出会うことがあります。
|
|
205
|
+
|
|
206
|
+
leaseが既に持っているstreamから描くので、2つ目のカメラを開くことはありません。sessionについても何も変えず、
|
|
207
|
+
1枚も取らなかった場合と同じようにleaseを解放できます。`type`の既定は`image/jpeg`。`quality`は0〜1へ丸めますが、
|
|
208
|
+
そもそもqualityでなかった値は、ブラウザの既定でencodeせずに拒否します。`maxWidth`・`maxHeight`は上限で、
|
|
209
|
+
アスペクト比は保たれ、厳しい側が決め、**フレームを引き伸ばすことはありません**——640 pxのフレームに4000 pxを
|
|
210
|
+
求めるのは、存在しない細部を求めることです。要求したtypeをencodeできないブラウザは失敗ではなくPNGを返すので、
|
|
211
|
+
typeを照合し、違っていれば誤った名前で渡す代わりに報告します。
|
|
212
|
+
|
|
213
|
+
**取得される静止画はカメラが撮ったそのものです。** `previewFlip`は描画上の選択であり、適用しません。
|
|
214
|
+
プロファイルのintrinsicsは撮影された画像を記述するので、黙って左右反転した静止画は、同じカメラで解いた全ての
|
|
215
|
+
校正と食い違い、しかもその食い違いが見えません——左右反転した部屋の写真は、部屋の写真に見えます。操作者が見て
|
|
216
|
+
いたものが欲しい場合は`flip`で求め、それが何だったかは`frame.previewFlip`で分かります。
|
|
217
|
+
|
|
218
|
+
ここにHTTP・アセット・保存の概念は入りません。10秒ごとのライブカメラは外側で組み立てます。
|
|
219
|
+
|
|
220
|
+
```js
|
|
221
|
+
const lease = await cameraSource.acquireCamera({owner: "live-camera", cameraId: "live"});
|
|
222
|
+
const timer = setInterval(async () => {
|
|
223
|
+
const shot = await lease.captureFrame({type: "image/jpeg", quality: 0.7, maxWidth: 1280});
|
|
224
|
+
await assets.replace("live-camera", shot.bytes, shot.type); // 最新フレームを保持する何か
|
|
225
|
+
}, 10_000);
|
|
226
|
+
// clearInterval(timer); await lease.release();
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
あとは保持側が配ります——`turbowarp-http-server`が`GET /camera/image.jpg`に答える、ダウンロードさせる、
|
|
230
|
+
アップロードする——が、そのいずれもCamera Sourceは知りません。
|
|
231
|
+
|
|
189
232
|
### 校正プロファイル
|
|
190
233
|
|
|
191
234
|
内部校正プロファイルの契約は本拡張が持つが、**プロファイルを作ることはしない**。チェスボード校正も、操作者が貼り付けたJSONも、将来の別方式も、検証を通る文書の生産者というだけで、ここからは区別できない。
|
|
@@ -328,6 +371,7 @@ ownership、device selection、release semanticsは変更しません。`preview
|
|
|
328
371
|
## 開発
|
|
329
372
|
|
|
330
373
|
```bash
|
|
374
|
+
npm install --global corepack@latest
|
|
331
375
|
corepack enable
|
|
332
376
|
pnpm install --frozen-lockfile
|
|
333
377
|
pnpm run check
|
package/README.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
|
-
# TurboWarp
|
|
1
|
+
# TurboWarp Camera Source
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
<img src="assets/block-icon.svg" alt="" width="48" height="48">
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
English | [日本語](README.ja.md)
|
|
6
|
+
|
|
7
|
+
TurboWarp Camera Source is a TurboWarp extension capability for sharing named
|
|
6
8
|
`MediaDevices` camera streams across consumers such as pose recognition, image
|
|
7
9
|
classification, and QR scanning. It owns camera startup and shutdown through
|
|
8
10
|
leases so multiple extensions can intentionally share the same physical camera
|
|
@@ -18,7 +20,7 @@ or select separate cameras for separate roles.
|
|
|
18
20
|
- Keeps each stream alive until the final lease is released.
|
|
19
21
|
- Lists browser camera devices after permission is granted.
|
|
20
22
|
|
|
21
|
-
## Requirements and
|
|
23
|
+
## Requirements and safety
|
|
22
24
|
|
|
23
25
|
- TurboWarp custom extensions loaded with **Run extension without sandbox**.
|
|
24
26
|
- A secure browser context such as HTTPS or localhost.
|
|
@@ -36,16 +38,16 @@ the original description when no translation is available. Translation does not
|
|
|
36
38
|
Load this URL as an unsandboxed custom extension:
|
|
37
39
|
|
|
38
40
|
```text
|
|
39
|
-
https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.
|
|
41
|
+
https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.21.0/dist/camera-source.js
|
|
40
42
|
```
|
|
41
43
|
|
|
42
44
|
For npm hosts:
|
|
43
45
|
|
|
44
46
|
```bash
|
|
45
|
-
pnpm add @kubohiroya/turbowarp-camera-source@0.
|
|
47
|
+
pnpm add @kubohiroya/turbowarp-camera-source@0.21.0
|
|
46
48
|
```
|
|
47
49
|
|
|
48
|
-
## Quick
|
|
50
|
+
## Quick start
|
|
49
51
|
|
|
50
52
|
Use role names for camera streams. Consumers that use the same `cameraId` share one camera; different names can bind to different devices.
|
|
51
53
|
|
|
@@ -60,7 +62,7 @@ shared camera [pose] is running?
|
|
|
60
62
|
stop shared camera [pose]
|
|
61
63
|
```
|
|
62
64
|
|
|
63
|
-
## Block
|
|
65
|
+
## Block reference
|
|
64
66
|
|
|
65
67
|
<!-- BEGIN GENERATED BLOCKS -->
|
|
66
68
|
|
|
@@ -87,7 +89,7 @@ Starts the camera the page's query parameters name: `cameraDeviceId` exactly, an
|
|
|
87
89
|
|
|
88
90
|
### `start shared camera [CAMERA_ID] from the browser's camera button`
|
|
89
91
|
|
|
90
|
-
Places the browser's own camera button on the stage and waits for the operator to answer it, then shares the stream it returns under this camera id without calling getUserMedia. For a .sb3 opened in the editor, where no host above the project can place the element. When the camera id already runs a stream -- a packaged work whose host adopted one -- this joins that stream instead of asking again. A refusal leaves the work running: the camera does not start, `shared camera error code` reads `usermedia-cancelled`, and `start shared camera` is still available. Browsers without the element report `usermedia-unsupported`.
|
|
92
|
+
Places the browser's own camera button on the stage and waits for the operator to answer it, then shares the stream it returns under this camera id without calling getUserMedia. For a .sb3 opened in the editor, where no host above the project can place the element. When the camera id already runs a stream -- a packaged work whose host adopted one -- this joins that stream instead of asking again. A refusal leaves the work running: the camera does not start, `shared camera error code` reads `usermedia-cancelled`, and `start shared camera` is still available. Browsers without the element report `usermedia-unsupported`. The element answers with a microphone as well as a camera; its track is stopped and taken out of the stream before the camera is shared, so nothing listens on it. The browser's own prompt still names the microphone, which only a future `<camera>` element could change.
|
|
91
93
|
|
|
92
94
|
| Property | Value |
|
|
93
95
|
|---|---|
|
|
@@ -563,6 +565,53 @@ The display path avoids explicit CPU readback, but it does not promise end-to-en
|
|
|
563
565
|
browser-internal color conversion and GPU transfer may still occur. The preview is private and
|
|
564
566
|
noninteractive; it is intended for display, not Scratch touching or color-sensing queries.
|
|
565
567
|
|
|
568
|
+
### Still images
|
|
569
|
+
|
|
570
|
+
A consumer that wants a file rather than pixels — an asset store, a download, a server publishing
|
|
571
|
+
the latest frame — can ask the lease for one. A consumer that is going to process the image itself
|
|
572
|
+
should read `frame.element` instead and skip the encode.
|
|
573
|
+
|
|
574
|
+
```js
|
|
575
|
+
if (typeof lease.captureFrame === "function") {
|
|
576
|
+
const shot = await lease.captureFrame({type: "image/jpeg", quality: 0.8, maxWidth: 1280});
|
|
577
|
+
// shot.bytes is a Uint8Array; shot.type, shot.width, shot.height and shot.capturedAt describe it.
|
|
578
|
+
}
|
|
579
|
+
```
|
|
580
|
+
|
|
581
|
+
Feature detect the method, for the same reason as `adoptCamera()`: this surface is narrowed by
|
|
582
|
+
shape rather than by a version, so a consumer compiled against the current declarations can still
|
|
583
|
+
meet a build that predates it.
|
|
584
|
+
|
|
585
|
+
It draws from the stream the lease already holds, so it never opens a second camera, and it changes
|
|
586
|
+
nothing about the session — the lease is released the same way whether or not anything was captured.
|
|
587
|
+
`type` defaults to `image/jpeg`; `quality` is clamped into 0–1 and a value that was never a quality
|
|
588
|
+
is refused rather than encoded at the browser's default. `maxWidth` and `maxHeight` are upper
|
|
589
|
+
bounds: the aspect ratio is kept, the tighter bound decides, and **the frame is never enlarged** — a
|
|
590
|
+
caller asking for 4000 px of a 640 px frame is asking for detail that does not exist. A browser that
|
|
591
|
+
cannot encode the type asked for returns PNG rather than failing, so the type is checked and a
|
|
592
|
+
mismatch is reported instead of handed over under the wrong name.
|
|
593
|
+
|
|
594
|
+
**The snapshot is the camera's own capture.** `previewFlip` is a drawing choice and is not applied:
|
|
595
|
+
a profile's intrinsics describe the captured image, so a snapshot that silently mirrored itself
|
|
596
|
+
would disagree with every calibration solved against the same camera, and disagree invisibly — a
|
|
597
|
+
mirrored photograph of a room looks like a photograph of a room. A caller that wants what the
|
|
598
|
+
operator was looking at asks for it with `flip`, reading `frame.previewFlip` to find out what that
|
|
599
|
+
was.
|
|
600
|
+
|
|
601
|
+
Nothing here knows about HTTP, assets or storage. A ten-second live camera is composed outside:
|
|
602
|
+
|
|
603
|
+
```js
|
|
604
|
+
const lease = await cameraSource.acquireCamera({owner: "live-camera", cameraId: "live"});
|
|
605
|
+
const timer = setInterval(async () => {
|
|
606
|
+
const shot = await lease.captureFrame({type: "image/jpeg", quality: 0.7, maxWidth: 1280});
|
|
607
|
+
await assets.replace("live-camera", shot.bytes, shot.type); // whatever holds the latest frame
|
|
608
|
+
}, 10_000);
|
|
609
|
+
// clearInterval(timer); await lease.release();
|
|
610
|
+
```
|
|
611
|
+
|
|
612
|
+
The store then serves it — `turbowarp-http-server` answering `GET /camera/image.jpg`, a download, an
|
|
613
|
+
upload — without Camera Source knowing any of that happened.
|
|
614
|
+
|
|
566
615
|
### Calibration profiles
|
|
567
616
|
|
|
568
617
|
This extension owns the contract for intrinsic calibration profiles but never produces one. Chessboard calibration, an operator pasting JSON, and any future calibrator are all just producers of a document that passes validation, and nothing here can tell them apart.
|
|
@@ -765,6 +814,7 @@ not request `preview: true` retain the existing camera-source behavior.
|
|
|
765
814
|
## Development
|
|
766
815
|
|
|
767
816
|
```bash
|
|
817
|
+
npm install --global corepack@latest
|
|
768
818
|
corepack enable
|
|
769
819
|
pnpm install --frozen-lockfile
|
|
770
820
|
pnpm run check
|
|
@@ -776,7 +826,7 @@ For continuous rebuilding during development:
|
|
|
776
826
|
pnpm run dev
|
|
777
827
|
```
|
|
778
828
|
|
|
779
|
-
## Build
|
|
829
|
+
## Build workflow
|
|
780
830
|
|
|
781
831
|
```text
|
|
782
832
|
TypeScript source
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48"><circle cx="17" cy="24" r="9" fill="none" stroke="#4C97FF" stroke-width="3"/><circle cx="17" cy="24" r="3.4" fill="#4C97FF"/><path d="M30 24h10M29 17l9-5M29 31l9 5" fill="none" stroke="#59C059" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/></svg>
|