@kubohiroya/turbowarp-camera-source 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.ja.md CHANGED
@@ -26,13 +26,13 @@ TurboWarp-Camera-Sourceは、MediaDevicesのカメラストリームを名前付
26
26
  次のURLをunsandboxed custom extensionとして読み込みます。
27
27
 
28
28
  ```text
29
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.6.0/dist/camera-source.js
29
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.7.0/dist/camera-source.js
30
30
  ```
31
31
 
32
32
  npm hostでは次を使います。
33
33
 
34
34
  ```bash
35
- pnpm add @kubohiroya/turbowarp-camera-source@0.6.0
35
+ pnpm add @kubohiroya/turbowarp-camera-source@0.7.0
36
36
  ```
37
37
 
38
38
  ## Quick start
@@ -52,13 +52,15 @@ stop shared camera [pose]
52
52
 
53
53
  ## ブロック一覧
54
54
 
55
+ <!-- BEGIN GENERATED BLOCKS -->
56
+
55
57
  - `start shared camera [CAMERA_ID] with device ID [DEVICE_ID]`: 名前付き共有カメラを開始します。
56
58
  - `stop shared camera [CAMERA_ID]`: 名前付き共有カメラを停止し、MediaStreamTrackを解放します。
57
59
  - `shared camera [CAMERA_ID] is running?`: 指定した共有カメラが起動中かを返します。
58
60
  - `shared camera [CAMERA_ID] error code`: 直近の開始失敗コードを返します。開始成功後は空文字列です。
59
61
  - `shared camera [CAMERA_ID] error`: 直近の開始失敗メッセージを返します。開始成功後は空文字列です。
60
62
  - `shared camera [CAMERA_ID] device ID`: 指定した共有カメラのdevice IDを返します。
61
- - `show shared camera [CAMERA_ID] preview mirrored [MIRRORED]`: GPU-backed stage previewを表示します。
63
+ - `show shared camera [CAMERA_ID] preview flipped [PREVIEW_FLIP]`: GPU-backed stage previewを表示します。`none`/`horizontal`/`vertical`/`both`から選び、reporterも渡せます。
62
64
  - `hide shared camera [CAMERA_ID] preview`: ブロックが所有するpreview leaseだけを解放して非表示にします。
63
65
  - `shared camera [CAMERA_ID] frame width`: 実際のframe幅をpixel単位で返します。
64
66
  - `shared camera [CAMERA_ID] frame height`: 実際のframe高さをpixel単位で返します。
@@ -68,6 +70,23 @@ stop shared camera [pose]
68
70
  - `camera device ID at [INDEX]`: 1始まりの位置でカメラdevice IDを返します。
69
71
  - `camera device label at [INDEX]`: 1始まりの位置でカメララベルを返します。
70
72
 
73
+ 次の12ブロックは`calibrationProfilesV1`フラグがONのときだけパレットに出ます(既定OFF)。
74
+
75
+ - `register camera profile [PROFILE_JSON]`: 内部校正プロファイルを検証して登録します。現行の`twcs/camera-intrinsics`と、旧`twrmc/camera-calibration`のどちらの文書も受け取ります。
76
+ - `forget camera profile for [CAMERA_ID]`: 登録済みプロファイルを破棄します。
77
+ - `camera [CAMERA_ID] is calibrated?`: プロファイルが登録されているかを返します。未登録は正常な状態です。
78
+ - `camera profile JSON for [CAMERA_ID]`: 保存された文書をそのまま返します。表示や書き出し用であって、投影に使う数値ではありません。
79
+ - `camera profile error`: 直近の登録失敗のコードを返します。成功後は空文字列です。
80
+ - `camera profile error detail`: 直近の登録失敗の位置と理由を返します。
81
+ - `camera profile compatibility for [CAMERA_ID]`: `compatible`/`incompatible`/`undetermined`を返します。
82
+ - `camera profile compatibility detail for [CAMERA_ID]`: 判定を決めた項目の理由を返します。
83
+ - `camera profile adaptation for [CAMERA_ID]`: 現在のフレームサイズへ読み替えられたか(`exact`/`scaled`/`unavailable`)を返します。
84
+ - `camera intrinsics JSON for [CAMERA_ID]`: 現在のフレームに適合済みの内部行列を返します。適合していなければ空文字列です。
85
+ - `camera conditions JSON for [CAMERA_ID]`: カメラが今報告している撮影条件を返します。
86
+ - `camera conditions generation for [CAMERA_ID]`: 幾何に影響する条件が変わるたびに増える整数を返します。frame rateでは動きません。
87
+
88
+ <!-- END GENERATED BLOCKS -->
89
+
71
90
  ## Runtime API
72
91
 
73
92
  他のunsandboxed拡張は`Scratch.vm.runtime.ext_kubohiroyacamerasource`を参照できます。
@@ -124,8 +143,37 @@ Scratchのtouchingまたはcolor sensing用の画像sourceとしては扱いま
124
143
 
125
144
  `cameraId`は`pose`や`qr`のような作品内の役割名です。`deviceId`を指定すると、その役割をブラウザが公開する特定のカメラデバイスへ割り当てられます。各カメラは最後のleaseが解放されるまで維持されます。
126
145
 
146
+ ### 校正プロファイル
147
+
148
+ 内部校正プロファイルの契約は本拡張が持つが、**プロファイルを作ることはしない**。チェスボード校正も、操作者が貼り付けたJSONも、将来の別方式も、検証を通る文書の生産者というだけで、ここからは区別できない。
149
+
150
+ この面は`config/feature-flags.ts`の起動時固定フラグで囲ってあり、**既定はOFF**。カメラ取得とpreviewはどちらでも変わらない。
151
+
152
+ ```js
153
+ globalThis.__TWCS_FEATURE_FLAGS__ = {calibrationProfilesV1: true};
154
+ ```
155
+
156
+ **プロファイルはカメラの投影の仕方を述べるもので、どこに立っているかは述べない。** 共通world座標での姿勢は配置を解く拡張の責務で、両者を1つの文書に混ぜたことが、従来の`twrmc/camera-calibration`形式を出自のアプリ以外で使えなくしていた。旧文書は読めるが、その姿勢は配置として再公開せずに捨てる。
157
+
158
+ **プロファイルを自分でスケールせず、内部行列を要求すること。** 校正時と違う解像度で届く場合、必要な計算は何が起きたかで変わる。単純な縮小なら`fx`・`fy`・`cx`・`cy`を比率倍するが、cropなら焦点距離はそのままで主点が平行移動する。trackの`resizeMode`を見られるのは本拡張だけなので、両者を区別できるのも本拡張だけ。**利用側がそれぞれ推測すれば、それぞれ違う推測をし、同じカメラがどの拡張から尋ねたかで違う幾何を返すことになる。** `camera intrinsics JSON`は現在のフレームに適合済みの数値を返し、cropやアスペクト変更で主点を置けない場合は空文字を返す。
159
+
160
+ **previewを反転してもフレームは反転しない。** `getFrameSource()`が返す`previewFlip`は、stageがどう描いているかだけを述べる。その裏にあるフレームは常にカメラが撮ったままである。**反転表示したpreview上で拾った座標は、これらのフレームと使う前に戻さなければならない** — previewの座標をそのままsolveへ渡すと、左右反転した姿勢に収束し、しかも再投影誤差は小さいまま出る。`horizontal`は左右反転で、`cv::flip(…, 1)`・ffmpegの`hflip`・CSSの`scaleX(-1)`と同じ。回転は別の関心事なので、同じenumには混ぜていない。
161
+
162
+ **契約は書き写さずimportすること。** 型は専用のentry pointとして公開してある。
163
+
164
+ ```ts
165
+ import {readCameraSourceRuntime} from "@kubohiroya/turbowarp-camera-source/runtime";
166
+ import type {CameraFrameSource, CameraLease} from "@kubohiroya/turbowarp-camera-source/runtime";
167
+ ```
168
+
169
+ このmoduleはlogicを持たず拡張本体を引き込まない。手書きの写しは何とも照合されないので、こちらの形が変わっても利用側のリポジトリでは型検査が通り続け、ブラウザで実行したときに壊れる。
170
+
171
+ **「判定できない」は「適合する」とは別の答え。** 適合性は`compatible`/`incompatible`/`undetermined`の3値で、不明が`compatible`へ格上げされることはない。プロファイルが現在の構成に適合していない限り内部行列は渡さない。渡してしまえば、利用側は別の構成の数値で投影し、**もっともらしく間違った幾何**を得ることになる。
172
+
127
173
  ## 互換性
128
174
 
175
+ 未リリースの変更は破壊的である。`CameraFrameSource.mirrored`は`previewFlip`へ置き換えた。軸を名指しでき、かつ「描画の話であって画素の話ではない」と言えるためである。previewブロックの`MIRRORED`引数は`PREVIEW_FLIP`へ置き換え、booleanはもう読まない。`acquireCamera`は`previewFlip`を受け取り、`mirrored`は受け付けない。frame sourceのinterfaceを手書きで持っている利用側は、公開した`./runtime`のimportへ移ること。そうしていればこの変更はビルド時に分かった。
176
+
129
177
  0.6.0ではcamera IDごとの開始失敗情報を公開し、inactive streamまたは終了したvideo trackを停止状態として
130
178
  扱います。既存のcamera ID、lease ownership、preview動作は変更しません。
131
179
 
package/README.md CHANGED
@@ -30,13 +30,13 @@ or select separate cameras for separate roles.
30
30
  Load this URL as an unsandboxed custom extension:
31
31
 
32
32
  ```text
33
- https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.6.0/dist/camera-source.js
33
+ https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-camera-source@0.7.0/dist/camera-source.js
34
34
  ```
35
35
 
36
36
  For npm hosts:
37
37
 
38
38
  ```bash
39
- pnpm add @kubohiroya/turbowarp-camera-source@0.6.0
39
+ pnpm add @kubohiroya/turbowarp-camera-source@0.7.0
40
40
  ```
41
41
 
42
42
  ## Quick Start
@@ -119,16 +119,16 @@ Returns the active device ID for a named shared camera when available.
119
119
  | Opcode | `cameraDeviceIdReporter` |
120
120
  | `CAMERA_ID` | String, default: `default` |
121
121
 
122
- ### `show shared camera [CAMERA_ID] preview mirrored [MIRRORED]`
122
+ ### `show shared camera [CAMERA_ID] preview flipped [PREVIEW_FLIP]`
123
123
 
124
- Shows the named shared camera with the GPU-backed stage preview.
124
+ Draws the named camera on the stage, turned over as asked. The flip is how the preview is drawn and never changes the frames a consumer is handed.
125
125
 
126
126
  | Property | Value |
127
127
  |---|---|
128
128
  | Type | Command |
129
129
  | Opcode | `showCameraPreview` |
130
130
  | `CAMERA_ID` | String, default: `default` |
131
- | `MIRRORED` | String, default: `true` |
131
+ | `PREVIEW_FLIP` | String, default: `horizontal` |
132
132
 
133
133
  ### `hide shared camera [CAMERA_ID] preview`
134
134
 
@@ -208,6 +208,124 @@ Returns the one-based camera device label at the requested index when the browse
208
208
  | Opcode | `cameraDeviceLabelAt` |
209
209
  | `INDEX` | String, default: `1` |
210
210
 
211
+ ### `register camera profile [PROFILE_JSON]`
212
+
213
+ Validates a twcs/camera-intrinsics version 1 document and stores it against the camera it names. Nothing is stored unless the whole document passes, and the profile in force for that camera is replaced.
214
+
215
+ | Property | Value |
216
+ |---|---|
217
+ | Type | Command |
218
+ | Opcode | `registerCameraProfile` |
219
+ | `PROFILE_JSON` | String, default: `{}` |
220
+
221
+ ### `forget camera profile for [CAMERA_ID]`
222
+
223
+ Removes the stored calibration profile for one camera.
224
+
225
+ | Property | Value |
226
+ |---|---|
227
+ | Type | Command |
228
+ | Opcode | `forgetCameraProfile` |
229
+ | `CAMERA_ID` | String, default: `default` |
230
+
231
+ ### `camera [CAMERA_ID] is calibrated?`
232
+
233
+ Reports whether a calibration profile is stored for the camera. An uncalibrated camera is an ordinary state and not an error.
234
+
235
+ | Property | Value |
236
+ |---|---|
237
+ | Type | Boolean |
238
+ | Opcode | `cameraProfileRegistered` |
239
+ | `CAMERA_ID` | String, default: `default` |
240
+
241
+ ### `camera profile JSON for [CAMERA_ID]`
242
+
243
+ Returns the stored profile as JSON, or an empty string when the camera has none.
244
+
245
+ | Property | Value |
246
+ |---|---|
247
+ | Type | Reporter |
248
+ | Opcode | `cameraProfileJson` |
249
+ | `CAMERA_ID` | String, default: `default` |
250
+
251
+ ### `camera profile error`
252
+
253
+ Returns the code of the last rejected profile document, or an empty string when the last one was accepted.
254
+
255
+ | Property | Value |
256
+ |---|---|
257
+ | Type | Reporter |
258
+ | Opcode | `cameraProfileError` |
259
+
260
+ ### `camera profile error detail`
261
+
262
+ Returns why the last profile document was rejected, naming the member at fault.
263
+
264
+ | Property | Value |
265
+ |---|---|
266
+ | Type | Reporter |
267
+ | Opcode | `cameraProfileErrorDetail` |
268
+
269
+ ### `camera profile compatibility for [CAMERA_ID]`
270
+
271
+ Returns compatible, incompatible, undetermined, or an empty string when the camera has no profile. Undetermined is a distinct answer from compatible and never resolves upward into it.
272
+
273
+ | Property | Value |
274
+ |---|---|
275
+ | Type | Reporter |
276
+ | Opcode | `cameraProfileCompatibility` |
277
+ | `CAMERA_ID` | String, default: `default` |
278
+
279
+ ### `camera profile compatibility detail for [CAMERA_ID]`
280
+
281
+ Returns the findings that decided the verdict, so an operator sees the one thing that has to change.
282
+
283
+ | Property | Value |
284
+ |---|---|
285
+ | Type | Reporter |
286
+ | Opcode | `cameraProfileCompatibilityDetail` |
287
+ | `CAMERA_ID` | String, default: `default` |
288
+
289
+ ### `camera profile adaptation for [CAMERA_ID]`
290
+
291
+ Returns exact, scaled, or unavailable. A frame size that differs by a pure scale can be projected with scaled intrinsics; a crop or an aspect change cannot, because the principal point cannot be placed.
292
+
293
+ | Property | Value |
294
+ |---|---|
295
+ | Type | Reporter |
296
+ | Opcode | `cameraProfileAdaptation` |
297
+ | `CAMERA_ID` | String, default: `default` |
298
+
299
+ ### `camera intrinsics JSON for [CAMERA_ID]`
300
+
301
+ Returns the intrinsics to project the current frames with, already adapted to the frame size, or an empty string when they cannot be established. Consumers use this rather than scaling a profile themselves, because only this extension can tell a scale from a crop.
302
+
303
+ | Property | Value |
304
+ |---|---|
305
+ | Type | Reporter |
306
+ | Opcode | `cameraProfileIntrinsicsJson` |
307
+ | `CAMERA_ID` | String, default: `default` |
308
+
309
+ ### `camera conditions JSON for [CAMERA_ID]`
310
+
311
+ Returns what the track reports about itself: frame size, resize mode, zoom, focus and frame rate. Read only; this extension never changes a shared camera configuration.
312
+
313
+ | Property | Value |
314
+ |---|---|
315
+ | Type | Reporter |
316
+ | Opcode | `cameraConditionsJson` |
317
+ | `CAMERA_ID` | String, default: `default` |
318
+
319
+ ### `camera conditions generation for [CAMERA_ID]`
320
+
321
+ Returns a number that increases whenever something affecting the geometry changes. A consumer holding a derived result compares this instead of re-checking every condition.
322
+
323
+ | Property | Value |
324
+ |---|---|
325
+ | Type | Reporter |
326
+ | Opcode | `cameraConditionsGeneration` |
327
+ | `CAMERA_ID` | String, default: `default` |
328
+
211
329
  <!-- END GENERATED BLOCKS -->
212
330
 
213
331
  ## Runtime API
@@ -233,7 +351,7 @@ const previewLease = await cameraSource.acquireCamera({
233
351
  owner: "camera-preview",
234
352
  cameraId: "pose",
235
353
  preview: true,
236
- mirrored: true,
354
+ previewFlip: "horizontal",
237
355
  });
238
356
  const frame = qrLease.getFrameSource();
239
357
  await previewLease.release();
@@ -246,14 +364,14 @@ the shared `HTMLVideoElement` with `texImage2D(video)` and mirrors by changing t
246
364
  X scale. It does not call `drawImage()`, `getImageData()`, or scan frame pixels on the CPU. The option
247
365
  defaults to `false`; the preview remains visible until the final preview lease for that camera is
248
366
  released.
249
- If multiple preview leases share a camera, the preview is mirrored while any active preview lease
250
- requests `mirrored: true`.
367
+ If multiple preview leases share a camera, the preview is turned over while any active preview
368
+ lease asks for a flip.
251
369
 
252
370
  TurboWarp projects can manage the same display path with the `show shared camera ... preview`
253
371
  and `hide shared camera ... preview` blocks. The show block owns exactly one preview lease per
254
- camera ID. Calling it repeatedly with the same setting is idempotent; changing `MIRRORED` replaces
255
- that lease without creating another drawable. `MIRRORED` accepts TurboWarp boolean text such as
256
- `true` and `false`. Hiding the preview releases only the block-owned display lease, so processing
372
+ camera ID. Calling it repeatedly with the same setting is idempotent; changing `PREVIEW_FLIP`
373
+ replaces that lease without creating another drawable. `PREVIEW_FLIP` takes `none`, `horizontal`,
374
+ `vertical` or `both`, and accepts a reporter so a project can hold the choice in a variable. Hiding the preview releases only the block-owned display lease, so processing
257
375
  leases held by other blocks or extensions continue running. Stopping the named camera, stopping or
258
376
  loading a project, and disposing the runtime release the block-owned camera and preview resources.
259
377
 
@@ -271,8 +389,53 @@ The display path avoids explicit CPU readback, but it does not promise end-to-en
271
389
  browser-internal color conversion and GPU transfer may still occur. The preview is private and
272
390
  noninteractive; it is intended for display, not Scratch touching or color-sensing queries.
273
391
 
392
+ ### Calibration profiles
393
+
394
+ This extension owns the contract for intrinsic calibration profiles but never produces one. Chessboard calibration, an operator pasting JSON, and any future calibrator are all just producers of a document that passes validation, and nothing here can tell them apart.
395
+
396
+ The surface is behind a startup-fixed flag in `config/feature-flags.ts`, **off by default**. Camera acquisition and preview are untouched by it either way:
397
+
398
+ ```js
399
+ globalThis.__TWCS_FEATURE_FLAGS__ = {calibrationProfilesV1: true};
400
+ ```
401
+
402
+ A versioned capability is published for other extensions:
403
+
404
+ ```js
405
+ const calibration = Scratch.vm.runtime.kubohiroyaCameraSourceCapability.requireVersion(1);
406
+ calibration.registerProfile(document);
407
+ const assessment = calibration.assessProfile("pose");
408
+ ```
409
+
410
+ **A profile says how a camera projects, never where it stands.** Camera pose in a shared world frame belongs to whichever extension solves placement; mixing the two into one document is what made the earlier `twrmc/camera-calibration` format unusable outside the application it came from. Legacy documents can be read, but their pose is dropped rather than republished as a placement.
411
+
412
+ **Ask for intrinsics rather than scaling a profile yourself.** When the camera delivers a size the calibration was not solved at, the arithmetic depends on what happened: a pure downscale multiplies `fx`, `fy`, `cx` and `cy`, while a crop leaves the focal lengths alone and shifts the principal point instead. Only this extension sees the track's `resizeMode`, so only it can tell the two apart — and if every consumer guesses, they guess differently and the same camera yields different geometry depending on which extension asked. `camera intrinsics JSON` returns numbers already adapted to the current frame, or an empty string when the difference is a crop or an aspect change and the principal point cannot be placed.
413
+
414
+ **Turning the preview over does not turn the frames over.** `getFrameSource()` reports `previewFlip`, which describes how the stage is drawing the image and nothing else; the frames behind it are always the ones the camera captured. Coordinates picked off a flipped preview must be turned back before they are used with these frames — a solve fed the preview's coordinates converges on a left-right reflected pose and reports a small reprojection error while doing it. `horizontal` is the left-right mirror, matching `cv::flip(…, 1)`, ffmpeg's `hflip` and CSS `scaleX(-1)`; rotation is a separate concern and is deliberately not folded into the same enum.
415
+
416
+ **Import these declarations rather than re-writing them.** The contract is published as its own
417
+ entry point, so a consumer states what it expects and finds out at build time when that changes.
418
+
419
+ ```ts
420
+ import {readCameraSourceRuntime} from "@kubohiroya/turbowarp-camera-source/runtime";
421
+ import type {CameraFrameSource, CameraLease} from "@kubohiroya/turbowarp-camera-source/runtime";
422
+ ```
423
+
424
+ The module holds no logic and pulls in none of the extension. A hand-written copy of an interface
425
+ compiles perfectly against nothing: when the shape here changes, the copy keeps type-checking in its
426
+ own repository and fails in a browser instead.
427
+
428
+ **"Cannot tell" is a distinct answer from "fits".** Compatibility is `compatible`, `incompatible` or `undetermined`, and unknown never resolves upward. Intrinsics are withheld unless the profile actually fits the camera as configured now: handing them over regardless would let a consumer project with numbers from a different configuration and get plausible, wrong geometry back.
429
+
274
430
  ## Compatibility
275
431
 
432
+ Unreleased changes are breaking. `CameraFrameSource.mirrored` is replaced by `previewFlip`, which
433
+ names the axis and says that it describes the drawing rather than the pixels. The preview block's
434
+ `MIRRORED` argument is replaced by `PREVIEW_FLIP` and the boolean is no longer read. `acquireCamera`
435
+ takes `previewFlip` and no longer accepts `mirrored`. Consumers holding a hand-written copy of the
436
+ frame source interface should move to the published `./runtime` entry point, which would have caught
437
+ this at build time.
438
+
276
439
  Version 0.6.0 exposes per-camera failure details and treats inactive or ended video tracks as
277
440
  stopped. Existing camera IDs, lease ownership, and preview behavior remain unchanged.
278
441