@fabi-play/protocol 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -74,3 +74,57 @@ from this package.
74
74
  ## License
75
75
 
76
76
  Fabi Play Kit License: free to use, for making games and tools for Fabi Play only. See [LICENSE](./LICENSE).
77
+
78
+ ## The TV remote
79
+
80
+ A manifest's optional `inputs` (`GameInput[]`, from `GAME_INPUTS`: `body`, `remote`; absent `["body"]`, see
81
+ `gameInputs`) says whether the game can be played with the TV remote when there is no phone; `parseManifest`
82
+ requires `body` in it, no repeats and no unknown values. The screen tells the game how a session is played in
83
+ `init.input` (absent `'body'`) and, with `'remote'`, whether the remote points (`init.pointer`, LG's Magic Remote:
84
+ plain pointer and mouse events in the game's page). The screen turns keys into gestures (↑ or OK `jump`, ↓ held
85
+ `squat`/`stand`, ← → lanes, OK twice `clap`, Back the platform's pause) and plays one child at a time: init's
86
+ `players` is the child playing now (one id). Several children take turns only with games whose manifest says `"turns": true`
87
+ (valid only with `remote` in `inputs`; it means the game follows the turn contract; without it, one child only): `init.turns`
88
+ lists every child in turn order (`parseTurnOrder`; absent: `players`). The game plays a round per child; when a
89
+ child's round ends (a crash too) and turns remain, it asks `break('turn')`, and the screen always answers
90
+ `break-start` (its turn screen), `{ type: 'turn', players: [2] }` (exactly one id, the next in the order;
91
+ `parseTurnPlayers` checks its shape) and `break-done`, the next child's puppet recentred in the middle lane. After
92
+ the last round the game ends with every child's scores. `level` and `gameover` breaks never swap players; hosts
93
+ before turns answer `turn` as an unknown reason, at once. Games keep scores by player id. See "Playing with the TV remote" in the
94
+ [`@fabi-play/sdk` README](https://www.npmjs.com/package/@fabi-play/sdk).
95
+
96
+ ### The arrows as a cursor
97
+
98
+ A manifest's optional `"remoteCursor": true` (valid only with `remote` in `inputs`) asks the screen, in remote mode,
99
+ to turn the arrows into a free cursor instead of gestures. The screen keeps which arrows are held and runs
100
+ `KeyCursor` every animation frame (`update(held, dtMs)` → `{ x, y }`, 0–1 of the game's frame: slow at first, faster
101
+ the longer an arrow is held, opposite arrows cancel, diagonals no faster, clamped to the edges, a long frame counted
102
+ as 100 ms; `reset()` at a turn), draws the cursor above the game, and sends `{ type: 'cursor', x, y, down }`
103
+ (`parseCursor`) when it moves and when OK goes down or up. init says `pointer: true`, as for a Magic Remote. The
104
+ SDK turns the messages into pointer and mouse events in the game's page, so the game sees a mouse either way. Back
105
+ stays the platform's pause; OK is the cursor's button, not a jump.
106
+
107
+ ## Pictures to the phone
108
+
109
+ A game can save a picture on the family's phone. The path, and the messages:
110
+
111
+ 1. The screen tells the game whether it can: `init.pictures` and, when it changes, `{ type: 'capabilities',
112
+ pictures }`. It is true while a phone whose `hello` said `pictures: true` is linked (the app; the web phone
113
+ doesn't say it).
114
+ 2. The game sends `{ type: 'picture', data, title?, id? }` (GameMessage): `data` a base64 data URL,
115
+ `data:image/jpeg` or `data:image/png`, at most `MAX_PICTURE_BYTES` (3 MiB of image), whose bytes really start as
116
+ that type (`checkPicture`; `parseGameMessage` drops anything else); `title` up to `MAX_PICTURE_TITLE` (60)
117
+ characters, cleaned by `parsePictureTitle`.
118
+ 3. The screen sends it to the phone over their reliable, ordered data channel, in pieces, because data channels
119
+ don't carry multi-megabyte messages (16 KiB is what every browser takes): `chunkPicture(dataUrl, id, title)`
120
+ gives `{ type: 'picture-start', id, mime, size, title? }` (`size` in bytes of the image), then
121
+ `{ type: 'picture-chunk', id, seq, data }` (`PICTURE_CHUNK_CHARS`, 16 000 base64 characters, each but the last,
122
+ `seq` from 0; each JSON message stays under 16 KiB), then `{ type: 'picture-end', id }` (ScreenMessage).
123
+ 4. The phone feeds them to a `PictureAssembler` (chunks in any order; a missing chunk, a picture over the limit, a
124
+ chunk past its size, bytes that are not the type the start said, all fail with a reason) and answers
125
+ `{ type: 'picture-saved', id }` or `{ type: 'picture-failed', id }` (ControllerMessage).
126
+ 5. The screen answers the game `{ type: 'picture-result', id, result }` (`parsePictureResult`): `'sent'`,
127
+ `'unavailable'`, `'too-big'` or `'error'`.
128
+
129
+ Privacy: the picture travels only TV → phone over the direct link (WebRTC, encrypted), never through or to a server,
130
+ and no copy is kept anywhere else. Games can send only a JPEG or PNG image this way.
package/dist/index.d.ts CHANGED
@@ -2,13 +2,195 @@
2
2
  // Generated by scripts/build-lib.mjs (fabiplay-kit) from the sources: do not edit.
3
3
 
4
4
  /**
5
- * The messages of Fabi Play, between three parties:
5
+ * A picture from the game to the family's phone (a drawing, the photo of a wall), saved in the parent's gallery.
6
6
  *
7
- * phone (controller) ──link──▶ screen (host) ──postMessage──▶ game (sandboxed frame)
7
+ * game ──postMessage { type: 'picture', data }──▶ screen ──data channel, in chunks──▶ phone (gallery)
8
8
  *
9
- * The camera stays on the phone: it sends only body landmarks, never images. The game gets the landmarks and the
10
- * movements recognised by the host, and can only answer with what is listed in GameMessage.
9
+ * Privacy: the picture goes only from the TV to the phone, over the direct link they already share (WebRTC, encrypted
10
+ * end to end), never through or to a server, and no copy is kept anywhere else. A game can send only a JPEG or PNG
11
+ * image this way (a base64 data URL whose bytes really start as one), at most MAX_PICTURE_BYTES: nothing else, no
12
+ * text, no other file. The phone side decides where it goes (the gallery, with the parent's permission).
13
+ *
14
+ * WebRTC data channels do not carry multi-megabyte messages reliably (16 KiB is the size every browser takes), so
15
+ * the screen sends a picture as picture-start, picture-chunk × n (PICTURE_CHUNK_CHARS of base64 each, the last
16
+ * shorter), picture-end; chunkPicture makes them, PictureAssembler puts them back together on the phone, which
17
+ * answers picture-saved or picture-failed.
18
+ */
19
+ /** The image types a picture may be. No WebP, GIF or SVG: JPEG for paintings and photos, PNG for flat art. */
20
+ export type PictureMime = "image/jpeg" | "image/png";
21
+ export declare const PICTURE_MIMES: readonly PictureMime[];
22
+ /** The largest picture, in bytes of the image (3 MiB; its base64 is a third bigger). */
23
+ export declare const MAX_PICTURE_BYTES: number;
24
+ /** The SDK scales a canvas down to this many pixels on its longer side before encoding it. */
25
+ export declare const MAX_PICTURE_SIDE = 1920;
26
+ /** The JPEG quality the SDK encodes a canvas with first (lower ones only if the picture would be too big). */
27
+ export declare const PICTURE_JPEG_QUALITY = 0.85;
28
+ /** The longest title of a picture (its file name on the phone), in characters. */
29
+ export declare const MAX_PICTURE_TITLE = 60;
30
+ /**
31
+ * Base64 characters per picture-chunk: a multiple of 4 (each chunk decodes on its own), small enough that the whole
32
+ * JSON message stays under 16 KiB.
33
+ */
34
+ export declare const PICTURE_CHUNK_CHARS = 16000;
35
+ /** What became of a picture the game asked to save (HostMessage picture-result, Play.savePicture). */
36
+ export type PictureResult = "sent" | "unavailable" | "too-big" | "error";
37
+ export declare const PICTURE_RESULTS: readonly PictureResult[];
38
+ /** A picture, checked: its type, its base64 (no data: prefix) and its size in bytes. */
39
+ export interface PictureData {
40
+ mime: PictureMime;
41
+ base64: string;
42
+ size: number;
43
+ }
44
+ /**
45
+ * A picture's data URL checked: 'ok' with its parts, 'too-big' over MAX_PICTURE_BYTES, 'invalid' for anything that
46
+ * is not a base64 JPEG or PNG data URL whose bytes start as that type.
47
+ */
48
+ export declare function checkPicture(v: unknown): {
49
+ ok: PictureData;
50
+ } | {
51
+ error: "too-big" | "invalid";
52
+ };
53
+ /** A picture's title, or undefined: a string, trimmed, without control characters, cut to MAX_PICTURE_TITLE. */
54
+ export declare function parsePictureTitle(v: unknown): string | undefined;
55
+ /** The three messages of a picture from the screen to the phone (members of ScreenMessage). */
56
+ export type PictureStart = {
57
+ type: "picture-start";
58
+ id: number;
59
+ mime: PictureMime;
60
+ size: number;
61
+ title?: string;
62
+ };
63
+ export type PictureChunk = {
64
+ type: "picture-chunk";
65
+ id: number;
66
+ seq: number;
67
+ data: string;
68
+ };
69
+ export type PictureEnd = {
70
+ type: "picture-end";
71
+ id: number;
72
+ };
73
+ export type PictureMessage = PictureStart | PictureChunk | PictureEnd;
74
+ /** A picture's id on the link: an integer from 0 to 2³¹ − 1, chosen by the screen. */
75
+ export declare const isPictureId: (v: unknown) => v is number;
76
+ /** How many chunks a picture of this many bytes travels in. */
77
+ export declare const pictureChunks: (size: number) => number;
78
+ /**
79
+ * A picture as the messages the screen sends the phone, in order: picture-start, its chunks (seq from 0), picture-end.
80
+ * Throws RangeError when dataUrl is not a picture checkPicture takes, or id not a picture id. The screen sends them on
81
+ * the reliable, ordered channel, waiting while its bufferedAmount is high.
82
+ */
83
+ export declare function chunkPicture(dataUrl: string, id: number, title?: string): PictureMessage[];
84
+ /** The picture-* screen messages checked (see parseScreenMessage), or null. */
85
+ export declare function parsePictureMessage(data: Record<string, unknown>): PictureMessage | null;
86
+ /** A picture put back together on the phone: dataUrl for an <Image>, base64 to write the file. */
87
+ export interface AssembledPicture {
88
+ id: number;
89
+ mime: PictureMime;
90
+ size: number;
91
+ title?: string;
92
+ base64: string;
93
+ dataUrl: string;
94
+ }
95
+ /**
96
+ * Why a picture could not be put back together: 'too-big' (its start says over MAX_PICTURE_BYTES, or chunks run
97
+ * past its size), 'missing' (a chunk never came, or the picture ended short), 'invalid' (a chunk out of place, a
98
+ * repeated chunk that differs, bytes that are not the type its start said), 'unknown' (a chunk or end with no start),
99
+ * 'replaced' (dropped for newer pictures: at most PICTURE_ASSEMBLER_MAX at once).
100
+ */
101
+ export type PictureFailure = "too-big" | "missing" | "invalid" | "unknown" | "replaced";
102
+ /** What PictureAssembler.push says: nothing yet (null), a picture, or a failed one (answer picture-failed with its id). */
103
+ export type AssemblerResult = null | {
104
+ ok: AssembledPicture;
105
+ } | {
106
+ failed: PictureFailure;
107
+ id: number;
108
+ };
109
+ /** Pictures put together at once; a start beyond this drops the oldest ('replaced'). */
110
+ export declare const PICTURE_ASSEMBLER_MAX = 2;
111
+ /**
112
+ * The phone's side: feed it every picture-* message (parsed with parseScreenMessage), in any order; it gives back the
113
+ * picture at its picture-end, or why it failed. Chunks may come out of order or twice (the same data); a picture
114
+ * whose end comes before all its chunks fails 'missing'. Pure, no DOM: decoding and saving are the app's.
115
+ *
116
+ * @example
117
+ * const pictures = new PictureAssembler()
118
+ * channel.onmessage = (e) => {
119
+ * const m = parseScreenMessage(JSON.parse(e.data))
120
+ * if (!m || !m.type.startsWith('picture-')) return
121
+ * const r = pictures.push(m as PictureMessage)
122
+ * if (r && 'ok' in r) saveToGallery(r.ok).then(() => send({ type: 'picture-saved', id: r.ok.id }), () => send({ type: 'picture-failed', id: r.ok.id }))
123
+ * else if (r) send({ type: 'picture-failed', id: r.id })
124
+ * }
11
125
  */
126
+ export declare class PictureAssembler {
127
+ private readonly pending;
128
+ /** Pictures being put together now. */
129
+ get size(): number;
130
+ push(m: PictureMessage): AssemblerResult;
131
+ /** Forgets every picture being put together (the link closed). */
132
+ clear(): void;
133
+ private fail;
134
+ }
135
+ /**
136
+ * The TV remote's arrows as a free cursor, for games whose manifest says "remoteCursor": true (see
137
+ * GameManifest.remoteCursor). The screen keeps which arrows are held and calls update every animation frame; the
138
+ * cursor glides, slowly at first for precision, then faster the longer an arrow is held, and stops at the edges.
139
+ * Pure: no DOM, no clock of its own (the screen passes the time between frames), so it is tested on its own.
140
+ *
141
+ * const cursor = new KeyCursor({ aspect: innerWidth / innerHeight })
142
+ * // every frame, with the arrows held now (the screen's KeyGate knows them):
143
+ * const { x, y } = cursor.update(held, dt)
144
+ * host.cursor(x, y, okHeld)
145
+ */
146
+ /** The arrows, as the screen names its keys. */
147
+ export type CursorKey = "UP" | "DOWN" | "LEFT" | "RIGHT";
148
+ /** The arrows held now: the key names (other names are ignored), or one flag per arrow. */
149
+ export type CursorKeys = Iterable<string> | Partial<Record<CursorKey, boolean>>;
150
+ /** Options of KeyCursor; the defaults are what the TV uses. */
151
+ export interface KeyCursorOptions {
152
+ /** The screen's width / height, so the cursor moves as fast across as down (default 16 / 9). */
153
+ aspect?: number;
154
+ /** Speed when an arrow is first held, in screen heights per second (default 0.12: a tap moves a few pixels). */
155
+ slow?: number;
156
+ /** Top speed, in screen heights per second (default 1.1: a 1080p screen's width in about 1.5 s). */
157
+ fast?: number;
158
+ /** How long an arrow stays at the slow speed before it speeds up, ms (default 180). */
159
+ holdMs?: number;
160
+ /** How long the speed then takes to reach fast, ms (default 1000; it eases in). */
161
+ rampMs?: number;
162
+ /** Where it starts, 0–1 (default the centre). */
163
+ x?: number;
164
+ y?: number;
165
+ }
166
+ export declare class KeyCursor {
167
+ /** 0 left, 1 right: fractions of the game's frame. */
168
+ x: number;
169
+ /** 0 top, 1 bottom. */
170
+ y: number;
171
+ private readonly aspect;
172
+ private readonly slow;
173
+ private readonly fast;
174
+ private readonly holdMs;
175
+ private readonly rampMs;
176
+ /** How long the current direction has been held, ms; the direction it is. */
177
+ private held;
178
+ private dir;
179
+ constructor(o?: KeyCursorOptions);
180
+ /** The speed after holding a direction this long, in screen heights per second. */
181
+ speed(heldMs: number): number;
182
+ /**
183
+ * One frame: keys, the arrows held now; dt, ms since the last update (over 100 ms counts as 100). Opposite arrows
184
+ * cancel; a diagonal is as fast as a straight move. The speed starts slow again whenever the direction changes or
185
+ * no arrow is held. Returns where the cursor is now, 0–1, clamped to the edges.
186
+ */
187
+ update(keys: CursorKeys, dt: number): {
188
+ x: number;
189
+ y: number;
190
+ };
191
+ /** Puts the cursor at x, y (clamped), slow again: a new turn, a game that starts. Default the centre. */
192
+ reset(x?: number, y?: number): void;
193
+ }
12
194
  export declare const PROTOCOL_VERSION = 1;
13
195
  /** MediaPipe Pose landmarks: 33 points, the ones the games use by name. */
14
196
  export declare const LANDMARKS = 33;
@@ -96,6 +278,8 @@ export type ControllerMessage =
96
278
  * aspect: width / height of the camera image, so the screen draws the bodies undistorted. fov: the camera's field
97
279
  * of view for that image (degrees, h across its width, v down its height), when the phone knows it (the app reads
98
280
  * it from the camera); the screen estimates each child's distance with it, or with DEFAULT_FOV when absent.
281
+ * pictures: true when this phone can receive pictures from games and save them (the app, with its gallery); absent
282
+ * on phones that cannot (the web phone, apps before pictures): the screen then tells games it can't save.
99
283
  */
100
284
  {
101
285
  type: "hello";
@@ -104,6 +288,7 @@ export type ControllerMessage =
104
288
  device: string;
105
289
  aspect: number;
106
290
  fov?: Fov;
291
+ pictures?: boolean;
107
292
  }
108
293
  /** A camera frame read on the phone: t in ms on the phone's clock, one body per player seen. */
109
294
  | {
@@ -156,6 +341,15 @@ export type ControllerMessage =
156
341
  type: "video";
157
342
  state: VideoState;
158
343
  reason?: VideoBlock;
344
+ }
345
+ /**
346
+ * The answer to a picture the screen sent (picture-start … picture-end, same id): saved (in the gallery, or kept to
347
+ * save once the parent allows it), or failed (it could not be put together, or saving failed for good). When saving
348
+ * fails the phone tells the parent itself; the children are never told.
349
+ */
350
+ | {
351
+ type: "picture-saved" | "picture-failed";
352
+ id: number;
159
353
  };
160
354
  export type ScreenMessage =
161
355
  /**
@@ -211,7 +405,14 @@ export type ScreenMessage =
211
405
  * confirmed band (absent: not known, e.g. a phone answered elsewhere and the TV forgot), zone the distances (metres)
212
406
  * the children should play at, for the parents' area.
213
407
  */
214
- | ScreenSizeNews;
408
+ | ScreenSizeNews
409
+ /**
410
+ * A picture from the game for the phone to save (see chunkPicture, PictureAssembler): picture-start (its type, its
411
+ * size in bytes, a title for the file), then picture-chunk × n (base64, PICTURE_CHUNK_CHARS each but the last, seq
412
+ * from 0), then picture-end; the phone answers picture-saved or picture-failed. Only over the direct link to the
413
+ * phone, never through a server; only to a phone whose hello said pictures.
414
+ */
415
+ | PictureMessage;
215
416
  /**
216
417
  * Mirror mode on or off, for every app at build time: the one switch for the phone's camera video on the TV. On:
217
418
  * the parent's switch on the phone decides, within the rules of docs/test.md section 13. Off: the phones (web page and app) never send video and never open a camera for it, their video switch and
@@ -307,10 +508,15 @@ export declare const wire: <T extends object>(m: T) => Wire<T>;
307
508
  /**
308
509
  * Why the game stops, when it asks for a break:
309
510
  * - level: a level is over (the best moment);
310
- * - gameover: the game is over and a new one may start.
311
- * Never at the game's own pause menu: a child who stops for a moment must not get an ad for it.
511
+ * - gameover: the game is over and a new one may start;
512
+ * - turn: with input 'remote' and turns remaining, the child playing now finished their round (whatever happened in
513
+ * it: a crash or a fall ends a round too). The screen always answers with break-start (its turn screen is the
514
+ * break, and may host an ad), then turn (the next child), then break-done. Hosts before turns answer it as an
515
+ * unknown reason (break-done at once), but they never send turns either.
516
+ * Only 'turn' ever swaps players; 'level' and 'gameover' never do. Never at the game's own pause menu: a child who
517
+ * stops for a moment must not get an ad for it.
312
518
  */
313
- export type BreakReason = "level" | "gameover";
519
+ export type BreakReason = "level" | "gameover" | "turn";
314
520
  export declare const BREAK_REASONS: BreakReason[];
315
521
  /**
316
522
  * The movements the host recognises, per player:
@@ -366,7 +572,9 @@ export type Backdrop = "video" | "none";
366
572
  * image is cropped to fill the screen), with any backdrop, so avatars and hands are where the children stand (a
367
573
  * "virtual mirror"). The screen decides it: with 'video', where the video is drawn (cover or contain); with 'none',
368
574
  * the whole image contained at 92% of the viewport's height, bottom-aligned. Games don't compute it: they just use
369
- * it. Body points are in player space (0–1 of the camera image, already mirrored like the video), so a point p is on
575
+ * it. It is not fixed: with automatic framing (the default, see GameManifest.framing) the screen zooms in on children
576
+ * standing far away, up to 1.8×, so the image may reach past every edge; it changes only at calibration and in breaks
577
+ * (pause, level, turn), never during play, and arrives in a backdrop message. Body points are in player space (0–1 of the camera image, already mirrored like the video), so a point p is on
370
578
  * screen at (x + p.x·w, y + p.y·h): see toScreen.
371
579
  */
372
580
  export interface BackdropView {
@@ -418,6 +626,28 @@ export type HostMessage =
418
626
  /** view: where the camera image is, with any backdrop (see BackdropView); absent if the screen doesn't know. */
419
627
  /** stats: a debug host (the simulator) asks the game to say how often it draws (stats messages); absent on the TV. */
420
628
  /** device: what the screen knows about the device (a starting tier, the display's rate); absent if it knows nothing. */
629
+ /**
630
+ * input: how this session is played (see GameInput); absent = 'body'. Only games whose manifest lists 'remote' ever
631
+ * get 'remote'. With 'remote', players is exactly one id: the child playing now (see turn).
632
+ */
633
+ /**
634
+ * turns: only with input 'remote', and only to games whose manifest says "turns": true: every child who
635
+ * will play this game, one id each, in turn order, players[0] first (see parseTurnOrder); absent = players (one
636
+ * child, no turns). A round per child: at the end of each child's round, while turns remain, the game calls
637
+ * break('turn'); the screen answers break-start, turn (the next child), break-done. After the last child's round
638
+ * the game calls end() with every child's scores. An order that is not valid, or doesn't start with players[0],
639
+ * makes the session one child only.
640
+ */
641
+ /**
642
+ * pointer: only with input 'remote', true when the TV's remote also points (LG's Magic Remote): it reaches the game
643
+ * as ordinary pointer and mouse events (hover, click with OK); or, for a game whose manifest says "remoteCursor":
644
+ * true, when the screen moves a cursor with the arrows (cursor messages, which the SDK turns into the same pointer
645
+ * and mouse events). Absent = false.
646
+ */
647
+ /**
648
+ * pictures: a phone that can save pictures is linked now (its hello said pictures), so the game may send picture
649
+ * messages; it changes with a capabilities message. Absent = false.
650
+ */
421
651
  {
422
652
  type: "init";
423
653
  v: number;
@@ -430,8 +660,12 @@ export type HostMessage =
430
660
  view?: BackdropView;
431
661
  stats?: boolean;
432
662
  device?: DeviceHint;
663
+ input?: GameInput;
664
+ turns?: number[];
665
+ pointer?: boolean;
666
+ pictures?: boolean;
433
667
  }
434
- /** The backdrop or the view changed mid-game (video switched on or off, the camera image's fit changed). */
668
+ /** The backdrop or the view changed mid-game (video switched on or off, the camera image's fit or framing changed). */
435
669
  | {
436
670
  type: "backdrop";
437
671
  backdrop: Backdrop;
@@ -470,7 +704,62 @@ export type HostMessage =
470
704
  | {
471
705
  type: "break-done";
472
706
  shown: boolean;
707
+ }
708
+ /**
709
+ * Turns (input 'remote' only): from now on this child plays, in place of the one before. One remote is one child,
710
+ * so with several children the screen plays one at a time: the same game goes on with the next child in init's
711
+ * turns (players [2] after [1], …), exactly one id. Sent only inside the break of a break('turn') the game asked
712
+ * for (after break-start, before break-done); never at other breaks, so a round never changes hands midway. The
713
+ * new child's puppet starts recentred, in the middle lane (lane 0), with no lane gesture. Ids keep their meaning
714
+ * (slots, avatars): keep each player's score by id across turns, and end with the scores of every child who
715
+ * played. Never sent with input 'body', where init's players stay for the whole game.
716
+ */
717
+ | {
718
+ type: "turn";
719
+ players: number[];
720
+ }
721
+ /**
722
+ * What the screen can do for the game changed mid-game: pictures, whether a phone that can save pictures is linked
723
+ * now (one joined, or the last one left). init carries the first value.
724
+ */
725
+ | {
726
+ type: "capabilities";
727
+ pictures: boolean;
728
+ }
729
+ /**
730
+ * The answer to a picture message, with its id (absent if the picture had none): 'sent', the phone has it (saved,
731
+ * or saved once the parent allows it); 'unavailable', no phone that saves pictures is linked; 'too-big', over
732
+ * MAX_PICTURE_BYTES; 'error', anything else (the link dropped, the phone failed, no answer in time).
733
+ */
734
+ | {
735
+ type: "picture-result";
736
+ id?: number;
737
+ result: PictureResult;
738
+ }
739
+ /**
740
+ * Remote mode with "remoteCursor": true: the cursor the arrows move (see KeyCursor), x and y in fractions of the
741
+ * game's frame (0–1), down while OK is held. Sent when it moves and when OK goes down or up; the SDK turns it into
742
+ * pointer and mouse events in the game's page (pointermove, pointerdown, pointerup, click, …), as a Magic Remote's.
743
+ * The screen draws the cursor itself, above the game.
744
+ */
745
+ | {
746
+ type: "cursor";
747
+ x: number;
748
+ y: number;
749
+ down: boolean;
473
750
  };
751
+ /** A cursor message's position and button, or undefined: x, y finite (clamped to 0–1), down a boolean. */
752
+ export declare function parseCursor(v: unknown): Extract<HostMessage, {
753
+ type: "cursor";
754
+ }> | undefined;
755
+ /** A picture-result message, or undefined: result one of PICTURE_RESULTS, id (if any) an integer. */
756
+ export declare function parsePictureResult(v: unknown): Extract<HostMessage, {
757
+ type: "picture-result";
758
+ }> | undefined;
759
+ /** The players of a turn message, or undefined if they are not valid: exactly one integer id from 1 to MAX_PLAYERS. A copy. */
760
+ export declare function parseTurnPlayers(v: unknown): number[] | undefined;
761
+ /** init's turns, or undefined if they are not valid: 1 to MAX_PLAYERS distinct integer ids from 1 to MAX_PLAYERS, in order. A copy. */
762
+ export declare function parseTurnOrder(v: unknown): number[] | undefined;
474
763
  /** Why a game is paused (see HostMessage pause). */
475
764
  export type PauseReason = "user" | "distance";
476
765
  export declare const PAUSE_REASONS: PauseReason[];
@@ -512,7 +801,20 @@ export type GameMessage =
512
801
  | ({
513
802
  type: "stats";
514
803
  renderFps: number;
515
- } & GameStats);
804
+ } & GameStats)
805
+ /**
806
+ * Save a picture on the family's phone (a drawing, the photo of a wall). data: a base64 data URL, data:image/jpeg
807
+ * or data:image/png, at most MAX_PICTURE_BYTES, whose bytes really are that type (checkPicture); title: a name for
808
+ * it (MAX_PICTURE_TITLE characters); id: the game's own, echoed in picture-result. Anything else is dropped: a game
809
+ * can send only an image this way. The screen sends it to the phone over their direct link only, never to a
810
+ * server, and answers picture-result.
811
+ */
812
+ | {
813
+ type: "picture";
814
+ data: string;
815
+ title?: string;
816
+ id?: number;
817
+ };
516
818
  /** The movements a game is played with, as its manifest lists them. */
517
819
  export type Movement = "hands" | "jump" | "squat" | "clap" | "freeze" | "lanes" | "shapes";
518
820
  /**
@@ -566,7 +868,53 @@ export interface GameManifest {
566
868
  backdrop?: GameBackdrop[];
567
869
  /** Optional: Fabi explains the game on the TV before calibration (see GameIntro). Absent: straight to calibration. */
568
870
  intro?: GameIntro;
871
+ /** Optional: how it can be played; absent = ['body']. Must include 'body' (Fabi Play is a movement platform); add 'remote' to be offered without a phone. */
872
+ inputs?: GameInput[];
873
+ /**
874
+ * Optional, only with 'remote' in inputs: true when the game follows the turns contract, so the screen may let
875
+ * several children play with the remote, a round each (init.turns): play a round with the child in players; when
876
+ * it ends (a crash too) and turns remain, ask break('turn') (the screen answers break-start, turn, break-done);
877
+ * after the last round, end with every child's scores. Absent or false: remote mode is one child only.
878
+ */
879
+ turns?: boolean;
880
+ /**
881
+ * Optional: 'auto' (the default) lets the screen frame the children: when they stand far from the camera it zooms
882
+ * in on them (up to 1.8×), so every child's avatar is about the same size and reaches the same things, near or
883
+ * far; the view changes only at calibration and in breaks. 'fixed' keeps the whole camera image, as before (for
884
+ * a game drawn on exact spots of the live video).
885
+ */
886
+ framing?: GameFraming;
887
+ /**
888
+ * Optional, only with 'remote' in inputs: true and, in remote mode, the arrows move a free cursor instead of making
889
+ * gestures (they speed up when held, see KeyCursor) and OK presses and releases it: the game gets it as pointer and
890
+ * mouse events, the same as a Magic Remote's (init.pointer true), so a game written for the pointer works with
891
+ * both. Back is still the platform's pause. For games picked or drawn with a pointer (a painting game). Absent or
892
+ * false: arrows and OK make gestures.
893
+ */
894
+ remoteCursor?: boolean;
569
895
  }
896
+ /** How the screen frames the camera image for a game: see GameManifest.framing. */
897
+ export type GameFraming = "auto" | "fixed";
898
+ export declare const GAME_FRAMINGS: GameFraming[];
899
+ /** The framing of a manifest, with the default. */
900
+ export declare function gameFraming(m: Pick<GameManifest, "framing">): GameFraming;
901
+ /**
902
+ * How a game can be played: with the body (camera on the phone), or with the TV remote (keys become gestures).
903
+ *
904
+ * 'remote' is for TV apps without a phone (Samsung, LG, Android TV, Fire TV): one remote is one child, and with
905
+ * several children the screen runs turns, only for games whose manifest says "turns": true (see GameManifest.turns,
906
+ * BreakReason turn, HostMessage turn). The screen turns keys into the gestures the body makes (↑ or OK jump;
907
+ * ↓ held squat, stand on release; ← → lanes; OK twice quickly clap; Back is the platform's pause, never the game's)
908
+ * and animates a puppet body in the frames, so with one child most games need no code. On LG the Magic
909
+ * Remote also points: it reaches the game as ordinary pointer and mouse events (init.pointer says when). A game that
910
+ * lists 'remote' must play with those keys alone and, where it has things to pick (bubbles, answers, buttons), let a
911
+ * pointer click them too (pointer events, large hit areas, a hover highlight). Games aimed with the hands should not
912
+ * list it unless they offer a selection the keys can drive.
913
+ */
914
+ export type GameInput = "body" | "remote";
915
+ export declare const GAME_INPUTS: GameInput[];
916
+ /** The inputs of a manifest, with the default. */
917
+ export declare function gameInputs(m: Pick<GameManifest, "inputs">): GameInput[];
570
918
  /** The backdrops a game may declare in its manifest (its own background is always supported). */
571
919
  export type GameBackdrop = Exclude<Backdrop, "none">;
572
920
  export declare const GAME_BACKDROPS: GameBackdrop[];
package/dist/index.js CHANGED
@@ -1,3 +1,128 @@
1
+ var __defProp = Object.defineProperty;
2
+ var __defNormalProp = (obj, key, value) => key in obj ? __defProp(obj, key, { enumerable: true, configurable: true, writable: true, value }) : obj[key] = value;
3
+ var __publicField = (obj, key, value) => __defNormalProp(obj, typeof key !== "symbol" ? key + "" : key, value);
4
+
5
+ // src/picture.ts
6
+ var PICTURE_MIMES = ["image/jpeg", "image/png"];
7
+ var MAX_PICTURE_BYTES = 3 * 1024 * 1024;
8
+ var MAX_PICTURE_SIDE = 1920;
9
+ var PICTURE_JPEG_QUALITY = 0.85;
10
+ var MAX_PICTURE_TITLE = 60;
11
+ var PICTURE_CHUNK_CHARS = 16e3;
12
+ var PICTURE_RESULTS = ["sent", "unavailable", "too-big", "error"];
13
+ var BASE64 = /^[A-Za-z0-9+/]*={0,2}$/;
14
+ var MAGIC = { "image/jpeg": "/9j/", "image/png": "iVBORw0KGg" };
15
+ var PREFIX = /^data:(image\/(?:jpeg|png));base64,/;
16
+ var decodedSize = (b64) => b64.length / 4 * 3 - (b64.endsWith("==") ? 2 : b64.endsWith("=") ? 1 : 0);
17
+ var MAX_BASE64 = Math.ceil(MAX_PICTURE_BYTES / 3) * 4;
18
+ var isImage = (mime, b64) => b64.length % 4 === 0 && b64.length > 0 && b64.startsWith(MAGIC[mime]) && BASE64.test(b64);
19
+ function checkPicture(v) {
20
+ if (typeof v !== "string") return { error: "invalid" };
21
+ const m = PREFIX.exec(v);
22
+ if (!m) return { error: "invalid" };
23
+ const mime = m[1];
24
+ const base64 = v.slice(m[0].length);
25
+ if (base64.length > MAX_BASE64) return { error: "too-big" };
26
+ if (!isImage(mime, base64)) return { error: "invalid" };
27
+ const size = decodedSize(base64);
28
+ return size > MAX_PICTURE_BYTES ? { error: "too-big" } : { ok: { mime, base64, size } };
29
+ }
30
+ function parsePictureTitle(v) {
31
+ if (typeof v !== "string") return void 0;
32
+ const t = Array.from(v.replace(/[\p{Cc}\p{Cf}]/gu, "").trim()).slice(0, MAX_PICTURE_TITLE).join("").trim();
33
+ return t || void 0;
34
+ }
35
+ var isPictureId = (v) => Number.isInteger(v) && v >= 0 && v <= 2147483647;
36
+ var pictureChunks = (size) => Math.ceil(Math.ceil(size / 3) * 4 / PICTURE_CHUNK_CHARS);
37
+ function chunkPicture(dataUrl, id, title) {
38
+ if (!isPictureId(id)) throw new RangeError("picture: id is not an integer from 0 to 2^31 - 1");
39
+ const checked = checkPicture(dataUrl);
40
+ if ("error" in checked) throw new RangeError(`picture: ${checked.error === "too-big" ? `over ${MAX_PICTURE_BYTES} bytes` : "not a base64 JPEG or PNG data URL"}`);
41
+ const { mime, base64, size } = checked.ok;
42
+ const name = parsePictureTitle(title);
43
+ const out = [{ type: "picture-start", id, mime, size, ...name ? { title: name } : {} }];
44
+ for (let at = 0, seq = 0; at < base64.length; at += PICTURE_CHUNK_CHARS, seq++) out.push({ type: "picture-chunk", id, seq, data: base64.slice(at, at + PICTURE_CHUNK_CHARS) });
45
+ out.push({ type: "picture-end", id });
46
+ return out;
47
+ }
48
+ function parsePictureMessage(data) {
49
+ if (!isPictureId(data.id)) return null;
50
+ const id = data.id;
51
+ switch (data.type) {
52
+ case "picture-start": {
53
+ const { mime, size } = data;
54
+ if (!PICTURE_MIMES.includes(mime) || !Number.isInteger(size) || size < 1) return null;
55
+ const title = parsePictureTitle(data.title);
56
+ return { type: "picture-start", id, mime, size, ...title ? { title } : {} };
57
+ }
58
+ case "picture-chunk": {
59
+ const { seq, data: d } = data;
60
+ return Number.isInteger(seq) && seq >= 0 && typeof d === "string" && d.length > 0 && d.length <= PICTURE_CHUNK_CHARS && BASE64.test(d) ? { type: "picture-chunk", id, seq, data: d } : null;
61
+ }
62
+ case "picture-end":
63
+ return { type: "picture-end", id };
64
+ default:
65
+ return null;
66
+ }
67
+ }
68
+ var PICTURE_ASSEMBLER_MAX = 2;
69
+ var PictureAssembler = class {
70
+ constructor() {
71
+ __publicField(this, "pending", /* @__PURE__ */ new Map());
72
+ }
73
+ /** Pictures being put together now. */
74
+ get size() {
75
+ return this.pending.size;
76
+ }
77
+ push(m) {
78
+ switch (m.type) {
79
+ case "picture-start": {
80
+ this.pending.delete(m.id);
81
+ if (m.size > MAX_PICTURE_BYTES) return { failed: "too-big", id: m.id };
82
+ let dropped = null;
83
+ if (this.pending.size >= PICTURE_ASSEMBLER_MAX) {
84
+ const oldest = this.pending.keys().next().value;
85
+ this.pending.delete(oldest);
86
+ dropped = { failed: "replaced", id: oldest };
87
+ }
88
+ this.pending.set(m.id, { start: m, chunks: new Array(pictureChunks(m.size)), received: 0 });
89
+ return dropped;
90
+ }
91
+ case "picture-chunk": {
92
+ const p = this.pending.get(m.id);
93
+ if (!p) return { failed: "unknown", id: m.id };
94
+ const last = p.chunks.length - 1;
95
+ const expected = m.seq < last ? PICTURE_CHUNK_CHARS : Math.ceil(p.start.size / 3) * 4 - last * PICTURE_CHUNK_CHARS;
96
+ if (m.seq > last) return this.fail(m.id, "too-big");
97
+ if (m.data.length !== expected) return this.fail(m.id, m.data.length > expected ? "too-big" : "invalid");
98
+ const had = p.chunks[m.seq];
99
+ if (had !== void 0) return had === m.data ? null : this.fail(m.id, "invalid");
100
+ p.chunks[m.seq] = m.data;
101
+ p.received++;
102
+ return null;
103
+ }
104
+ case "picture-end": {
105
+ const p = this.pending.get(m.id);
106
+ if (!p) return { failed: "unknown", id: m.id };
107
+ this.pending.delete(m.id);
108
+ if (p.received < p.chunks.length) return { failed: "missing", id: m.id };
109
+ const base64 = p.chunks.join("");
110
+ const { id, mime, size, title } = p.start;
111
+ if (!isImage(mime, base64) || decodedSize(base64) !== size) return { failed: "invalid", id };
112
+ return { ok: { id, mime, size, ...title ? { title } : {}, base64, dataUrl: `data:${mime};base64,${base64}` } };
113
+ }
114
+ }
115
+ }
116
+ /** Forgets every picture being put together (the link closed). */
117
+ clear() {
118
+ this.pending.clear();
119
+ }
120
+ fail(id, why) {
121
+ this.pending.delete(id);
122
+ return { failed: why, id };
123
+ }
124
+ };
125
+
1
126
  // src/sha256.ts
2
127
  function utf8(s) {
3
128
  const out = [];
@@ -133,6 +258,76 @@ function sha256(bytes) {
133
258
  return out;
134
259
  }
135
260
 
261
+ // src/key-cursor.ts
262
+ var MAX_DT_MS = 100;
263
+ var clamp01 = (v) => v <= 0 ? 0 : v >= 1 ? 1 : v;
264
+ var finite = (v, d) => typeof v === "number" && Number.isFinite(v) && v > 0 ? v : d;
265
+ var KeyCursor = class {
266
+ constructor(o = {}) {
267
+ /** 0 left, 1 right: fractions of the game's frame. */
268
+ __publicField(this, "x");
269
+ /** 0 top, 1 bottom. */
270
+ __publicField(this, "y");
271
+ __publicField(this, "aspect");
272
+ __publicField(this, "slow");
273
+ __publicField(this, "fast");
274
+ __publicField(this, "holdMs");
275
+ __publicField(this, "rampMs");
276
+ /** How long the current direction has been held, ms; the direction it is. */
277
+ __publicField(this, "held", 0);
278
+ __publicField(this, "dir", "");
279
+ this.aspect = finite(o.aspect, 16 / 9);
280
+ this.slow = finite(o.slow, 0.12);
281
+ this.fast = Math.max(finite(o.fast, 1.1), this.slow);
282
+ this.holdMs = typeof o.holdMs === "number" && o.holdMs >= 0 ? o.holdMs : 180;
283
+ this.rampMs = finite(o.rampMs, 1e3);
284
+ this.x = clamp01(o.x ?? 0.5);
285
+ this.y = clamp01(o.y ?? 0.5);
286
+ }
287
+ /** The speed after holding a direction this long, in screen heights per second. */
288
+ speed(heldMs) {
289
+ const t = clamp01((heldMs - this.holdMs) / this.rampMs);
290
+ return this.slow + (this.fast - this.slow) * t * t;
291
+ }
292
+ /**
293
+ * One frame: keys, the arrows held now; dt, ms since the last update (over 100 ms counts as 100). Opposite arrows
294
+ * cancel; a diagonal is as fast as a straight move. The speed starts slow again whenever the direction changes or
295
+ * no arrow is held. Returns where the cursor is now, 0–1, clamped to the edges.
296
+ */
297
+ update(keys, dt) {
298
+ const k = held(keys);
299
+ const dx = (k.RIGHT ? 1 : 0) - (k.LEFT ? 1 : 0);
300
+ const dy = (k.DOWN ? 1 : 0) - (k.UP ? 1 : 0);
301
+ const dir = `${dx},${dy}`;
302
+ if (dir !== this.dir) {
303
+ this.dir = dir;
304
+ this.held = 0;
305
+ }
306
+ const ms = Number.isFinite(dt) && dt > 0 ? Math.min(dt, MAX_DT_MS) : 0;
307
+ if ((dx || dy) && ms) {
308
+ const v = (this.speed(this.held) + this.speed(this.held + ms)) / 2;
309
+ this.held += ms;
310
+ const step = v * ms / 1e3 / (dx && dy ? Math.SQRT2 : 1);
311
+ this.x = clamp01(this.x + dx * step / this.aspect);
312
+ this.y = clamp01(this.y + dy * step);
313
+ }
314
+ return { x: this.x, y: this.y };
315
+ }
316
+ /** Puts the cursor at x, y (clamped), slow again: a new turn, a game that starts. Default the centre. */
317
+ reset(x = 0.5, y = 0.5) {
318
+ this.x = clamp01(Number.isFinite(x) ? x : 0.5);
319
+ this.y = clamp01(Number.isFinite(y) ? y : 0.5);
320
+ this.held = 0;
321
+ this.dir = "";
322
+ }
323
+ };
324
+ function held(keys) {
325
+ if (typeof keys[Symbol.iterator] !== "function") return keys;
326
+ const out = {};
327
+ for (const name of keys) if (name === "UP" || name === "DOWN" || name === "LEFT" || name === "RIGHT") out[name] = true;
328
+ return out;
329
+ }
330
+
136
331
  // src/index.ts
137
332
  var PROTOCOL_VERSION = 1;
138
333
  var LANDMARKS = 33;
@@ -204,7 +399,7 @@ var TOO_CLOSE_KEY = "fabi-play.too-close-pause";
204
399
  var PHASES = ["setup", "intro", "calibrate", "play", "pause", "break", "end"];
205
400
  var NS = "fabi-play";
206
401
  var wire = (m) => ({ ns: NS, ...m });
207
- var BREAK_REASONS = ["level", "gameover"];
402
+ var BREAK_REASONS = ["level", "gameover", "turn"];
208
403
  function parseView(v) {
209
404
  if (!isObject(v)) return void 0;
210
405
  const { x, y, w, h } = v;
@@ -227,7 +422,32 @@ function parseGameStats(v) {
227
422
  if (DEVICE_TIERS.includes(v.tier)) out.tier = v.tier;
228
423
  return out;
229
424
  }
425
+ function parseCursor(v) {
426
+ if (!isObject(v) || !isNumber(v.x) || !isNumber(v.y) || typeof v.down !== "boolean") return void 0;
427
+ const c = (n) => Math.min(Math.max(n, 0), 1);
428
+ return { type: "cursor", x: c(v.x), y: c(v.y), down: v.down };
429
+ }
430
+ function parsePictureResult(v) {
431
+ if (!isObject(v) || !PICTURE_RESULTS.includes(v.result)) return void 0;
432
+ return { type: "picture-result", ...Number.isInteger(v.id) ? { id: v.id } : {}, result: v.result };
433
+ }
434
+ var isPlayerId = (id) => Number.isInteger(id) && id >= 1 && id <= MAX_PLAYERS;
435
+ function parseTurnPlayers(v) {
436
+ return Array.isArray(v) && v.length === 1 && isPlayerId(v[0]) ? [v[0]] : void 0;
437
+ }
438
+ function parseTurnOrder(v) {
439
+ if (!Array.isArray(v) || v.length < 1 || v.length > MAX_PLAYERS || !v.every(isPlayerId) || new Set(v).size !== v.length) return void 0;
440
+ return [...v];
441
+ }
230
442
  var PAUSE_REASONS = ["user", "distance"];
443
+ var GAME_FRAMINGS = ["auto", "fixed"];
444
+ function gameFraming(m) {
445
+ return m.framing ?? "auto";
446
+ }
447
+ var GAME_INPUTS = ["body", "remote"];
448
+ function gameInputs(m) {
449
+ return m.inputs?.length ? m.inputs : ["body"];
450
+ }
231
451
  var GAME_BACKDROPS = ["video"];
232
452
  var MOVEMENTS = ["hands", "jump", "squat", "clap", "freeze", "lanes", "shapes"];
233
453
  function parseManifest(data) {
@@ -248,6 +468,13 @@ function parseManifest(data) {
248
468
  if (m.sdk !== PROTOCOL_VERSION) errors.push(`sdk: this version of Fabi Play uses protocol ${PROTOCOL_VERSION}`);
249
469
  if (m.backdrop !== void 0 && !(Array.isArray(m.backdrop) && m.backdrop.every((b) => GAME_BACKDROPS.includes(b)) && new Set(m.backdrop).size === m.backdrop.length))
250
470
  errors.push(`backdrop: optional, from ${GAME_BACKDROPS.join(", ")}, without repeats`);
471
+ if (m.inputs !== void 0 && !(Array.isArray(m.inputs) && m.inputs.includes("body") && m.inputs.every((i) => GAME_INPUTS.includes(i)) && new Set(m.inputs).size === m.inputs.length))
472
+ errors.push(`inputs: optional, from ${GAME_INPUTS.join(", ")}, without repeats, always with body`);
473
+ if (m.turns !== void 0 && !(typeof m.turns === "boolean" && (!m.turns || Array.isArray(m.inputs) && m.inputs.includes("remote"))))
474
+ errors.push('turns: optional, true or false; true only with "remote" in inputs');
475
+ if (m.remoteCursor !== void 0 && !(typeof m.remoteCursor === "boolean" && (!m.remoteCursor || Array.isArray(m.inputs) && m.inputs.includes("remote"))))
476
+ errors.push('remoteCursor: optional, true or false; true only with "remote" in inputs');
477
+ if (m.framing !== void 0 && !GAME_FRAMINGS.includes(m.framing)) errors.push(`framing: optional, ${GAME_FRAMINGS.join(" or ")}`);
251
478
  if (m.intro !== void 0) errors.push(...introErrors(m.intro));
252
479
  return errors.length ? { errors } : { manifest: m };
253
480
  }
@@ -342,6 +569,11 @@ function parseGameMessage(data) {
342
569
  }
343
570
  case "stats":
344
571
  return isNumber(data.renderFps) && data.renderFps >= 0 && data.renderFps < 1e3 ? { type: "stats", renderFps: data.renderFps, ...parseGameStats(data) } : null;
572
+ case "picture": {
573
+ if (!("ok" in checkPicture(data.data))) return null;
574
+ const title = parsePictureTitle(data.title);
575
+ return { type: "picture", data: data.data, ...title ? { title } : {}, ...Number.isInteger(data.id) ? { id: data.id } : {} };
576
+ }
345
577
  default:
346
578
  return null;
347
579
  }
@@ -361,7 +593,8 @@ function parseControllerMessage(data) {
361
593
  camera: data.camera,
362
594
  device: data.device.slice(0, 80),
363
595
  aspect: isNumber(data.aspect) ? Math.min(Math.max(data.aspect, 0.3), 3) : 0.75,
364
- ...fov ? { fov } : {}
596
+ ...fov ? { fov } : {},
597
+ ...data.pictures === true ? { pictures: true } : {}
365
598
  };
366
599
  }
367
600
  case "pose":
@@ -385,6 +618,9 @@ function parseControllerMessage(data) {
385
618
  return INTRO_ACTIONS.includes(data.action) ? { type: "intro", action: data.action } : null;
386
619
  case "video":
387
620
  return VIDEO_STATES.includes(data.state) ? { type: "video", state: data.state, ...VIDEO_BLOCKS.includes(data.reason) ? { reason: data.reason } : {} } : null;
621
+ case "picture-saved":
622
+ case "picture-failed":
623
+ return Number.isInteger(data.id) && data.id >= 0 ? { type: data.type, id: data.id } : null;
388
624
  default:
389
625
  return null;
390
626
  }
@@ -434,6 +670,10 @@ function parseScreenMessage(data) {
434
670
  ...zone ? { zone } : {}
435
671
  };
436
672
  }
673
+ case "picture-start":
674
+ case "picture-chunk":
675
+ case "picture-end":
676
+ return parsePictureMessage(data);
437
677
  default:
438
678
  return null;
439
679
  }
@@ -545,11 +785,17 @@ export {
545
785
  DEVICE_TIERS,
546
786
  FABI_VOICE,
547
787
  GAME_BACKDROPS,
788
+ GAME_FRAMINGS,
789
+ GAME_INPUTS,
548
790
  INTRO_MAX_CHARS,
549
791
  INTRO_MAX_STEPS,
550
792
  INTRO_MOVES,
793
+ KeyCursor,
551
794
  LANDMARKS,
552
795
  MAX_AVATAR_LENGTH,
796
+ MAX_PICTURE_BYTES,
797
+ MAX_PICTURE_SIDE,
798
+ MAX_PICTURE_TITLE,
553
799
  MAX_PLAYERS,
554
800
  MIRROR_ENABLED,
555
801
  MOVEMENTS,
@@ -557,40 +803,58 @@ export {
557
803
  P,
558
804
  PAUSE_REASONS,
559
805
  PHASES,
806
+ PICTURE_ASSEMBLER_MAX,
807
+ PICTURE_CHUNK_CHARS,
808
+ PICTURE_JPEG_QUALITY,
809
+ PICTURE_MIMES,
810
+ PICTURE_RESULTS,
560
811
  PLAYER_COLORS,
561
812
  POSE_CODEC_VERSION,
562
813
  POSE_CODEC_VERSIONS,
563
814
  POSE_MAX_BYTES,
564
815
  PROTOCOL_VERSION,
816
+ PictureAssembler,
565
817
  SCREEN_SIZES,
566
818
  SCREEN_SIZE_KEY,
567
819
  TOO_CLOSE_KEY,
568
820
  VIDEO_BLOCKS,
569
821
  VIDEO_STATES,
570
822
  acceptsBinaryPose,
823
+ checkPicture,
824
+ chunkPicture,
571
825
  decodePose,
572
826
  defaultAvatarFor,
573
827
  encodePose,
828
+ gameFraming,
829
+ gameInputs,
574
830
  gamesFor,
575
831
  introText,
576
832
  isAgeRange,
577
833
  isAvatarName,
578
834
  isBody,
579
835
  isGameId,
836
+ isPictureId,
580
837
  isScreenSizeId,
581
838
  legacyStorageKey,
582
839
  migrateStorageKey,
583
840
  migrateStorageKeyAsync,
584
841
  parseControllerMessage,
842
+ parseCursor,
585
843
  parseDeviceHint,
586
844
  parseFov,
587
845
  parseGameMessage,
588
846
  parseGameStats,
589
847
  parseIntro,
590
848
  parseManifest,
849
+ parsePictureMessage,
850
+ parsePictureResult,
851
+ parsePictureTitle,
591
852
  parseScreenMessage,
853
+ parseTurnOrder,
854
+ parseTurnPlayers,
592
855
  parseView,
593
856
  parseZone,
857
+ pictureChunks,
594
858
  poseBytes,
595
859
  screenSize,
596
860
  screenSizesOf,
@@ -5,6 +5,20 @@
5
5
  "type": "object",
6
6
  "required": ["id", "version", "title", "description", "ageRanges", "movements", "players", "minutes", "entry", "sdk"],
7
7
  "additionalProperties": false,
8
+ "allOf": [
9
+ {
10
+ "anyOf": [
11
+ { "not": { "required": ["turns"], "properties": { "turns": { "const": true } } } },
12
+ { "required": ["inputs"], "properties": { "inputs": { "contains": { "const": "remote" } } } }
13
+ ]
14
+ },
15
+ {
16
+ "anyOf": [
17
+ { "not": { "required": ["remoteCursor"], "properties": { "remoteCursor": { "const": true } } } },
18
+ { "required": ["inputs"], "properties": { "inputs": { "contains": { "const": "remote" } } } }
19
+ ]
20
+ }
21
+ ],
8
22
  "properties": {
9
23
  "$schema": {
10
24
  "type": "string",
@@ -78,6 +92,27 @@
78
92
  "uniqueItems": true,
79
93
  "items": { "enum": ["video"] }
80
94
  },
95
+ "inputs": {
96
+ "type": "array",
97
+ "description": "Optional: how the game can be played; absent = [\"body\"]. Must include \"body\" (Fabi Play is a movement platform). Add \"remote\" to be offered on TVs without a phone: the TV remote's keys become gestures (up or OK jump, down held squat, left and right lanes, OK twice clap), one child at a time; on LG the Magic Remote also points, so let what the game has to pick be clicked too.",
98
+ "minItems": 1,
99
+ "uniqueItems": true,
100
+ "contains": { "const": "body" },
101
+ "items": { "enum": ["body", "remote"] },
102
+ "examples": [["body", "remote"]]
103
+ },
104
+ "turns": {
105
+ "type": "boolean",
106
+ "description": "Optional, true only with \"remote\" in inputs: the game follows the turns contract, so several children may play with the remote, a round each. When a child's round ends (a crash too) and turns remain, the game asks break('turn') (the screen answers break-start, turn, break-done); after the last round it ends with every child's scores. Absent or false: remote mode is one child only."
107
+ },
108
+ "remoteCursor": {
109
+ "type": "boolean",
110
+ "description": "Optional, true only with \"remote\" in inputs: in remote mode the arrows move a free cursor (it speeds up while held) and OK presses it, instead of making gestures. The game gets it as pointer and mouse events, the same as a Magic Remote's (init.pointer true), so a game written for the pointer works with both. Back is still Fabi Play's pause. Absent or false: arrows and OK make gestures."
111
+ },
112
+ "framing": {
113
+ "enum": ["auto", "fixed"],
114
+ "description": "Optional, \"auto\" by default: the screen zooms in on children standing far away (up to 1.8×), so every child's avatar is about the same size near or far; the view changes only at calibration and in breaks, never during play. \"fixed\": always the whole camera image (for a game drawn on exact spots of the live video)."
115
+ },
81
116
  "intro": {
82
117
  "type": "object",
83
118
  "description": "Optional: Fabi explains the game on the TV before calibration, in 1 to 4 steps. Each step: one sentence (Italian and English, up to 90 characters; or one per age range) over an image of the game or a move shown with the children's avatars. The voice is generated by Fabi Play at publication from the text. Without intro the game goes straight to calibration; an invalid one is skipped by the TV.",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@fabi-play/protocol",
3
- "version": "0.2.0",
3
+ "version": "0.4.0",
4
4
  "description": "Messages between the phone, the screen and the games, with their checks.",
5
5
  "keywords": [
6
6
  "fabi-play",