@voqalize/avatar 0.0.1 → 0.2.0

package/docs/contract-protocol.md

@@ -10,7 +10,7 @@ saying and looking at, and tells the widget; the widget's only job is to look
 right while rendering that. Nothing in this contract lets the client decide
 call content, and nothing in it requires the server to know what a face looks
 like. The whole protocol is: **a state enum, an emotion enum, a gaze enum, an
-interjection id, and a stream of timed viseme letters.**
+interjection id, a hand-gesture id, and a stream of timed viseme letters.**
 
 Everything below is reachable from one import:
 
@@ -19,7 +19,8 @@ import { createAvatar } from './src/avatar.js';
 const avatar = createAvatar({ mount: '#avatar' });   // also: avatar, face, theme, mouthGain, gestureGain
 ```
 
-All setters are chainable. Unknown state and interjection ids **throw**;
+All setters are chainable. Unknown state, interjection and gesture ids
+**throw**;
 unknown emotion falls back to `neutral` silently; unknown gaze falls back to
 `USER` silently.
 
@@ -154,6 +155,44 @@ until `attachAudio` gives them a voice.
 A repeated `interject(id)` while that clip is already playing is collapsed to a
 no-op; a *different* id replaces the running clip immediately.
 
+## Hand gestures — `gesture(id)`
+
+A hand rising into the bottom of the frame, plus the face half that makes it
+belong to somebody. **This is a separate verb from `interject` on purpose.**
+The four ids below are disjoint from `INTERJECTION_IDS`, and `interject('WAVE')`
+still means exactly what it always meant — the face alone. A server that
+upgrades the widget gets no hand until it asks for one.
+
+| id | dur | what it does |
+|---|---|---|
+| `HI` | 1250 | open palm rises and waves — greeting. Face half: `WAVE` |
+| `BYE` | 1550 | the same wave, one swing longer and a touch slower — parting. Face half: `WAVE` |
+| `THUMBS_UP` | 1300 | fist, back of hand to camera, thumb up — approval. Face half: `THUMBS_UP` |
+| `ONE_MOMENT` | 1700 | a single raised index finger, held — "one moment". Face half: `ONE_MOMENT`, which speaks |
+
+Calling `gesture(id)` fires the face half as an `interject()` on the caller's
+behalf — do **not** send both; the second one replaces the first mid-clip. It
+also suppresses autonomous backchannels for the gesture's duration plus 500 ms,
+because a nod landing on top of a deliberate hand is the listening engine
+talking over the server.
+
+What the widget guarantees, and why it is stated here rather than left to the
+drawing: **nothing but a single digit ever passes the mouth.** Mouth sync is
+the headline feature, so a gesture is free to fire mid-speech. The hand also
+never leaves the frame sideways and never shows a wrist — see
+`docs/contract-avatar.md` § The hand for the rules and the per-avatar check.
+
+Degradation is total and silent. An avatar mounted with `hand: false` — a face
+drawn in some other idiom, or a tile too small to spend the pixels — plays the
+face half and nothing else, which is the same fallback every id already had
+before the hand existed. `api.gesturing` is the id in flight, or `null` — which
+is what it always reads under `hand: false`, and `gestureEnd` correspondingly
+never fires there: both describe the *hand*, and there is no hand. An unknown
+id throws, as `interject` does.
+
+`setHandSide(+1 | -1)` picks which side of the frame the hand enters from;
+`+1` (the viewer's right) is the default.
+
 ## Speech — `speak({ cues, audio?, clock? })`, `pushCues(cues)`, `stopSpeaking()`
 
 The headline feature. A **cue** is:
@@ -195,19 +234,18 @@ TTS produces them; the merged track is re-normalized each push.
 `speak()` auto-enters `SPEAKING` (keeping the current gaze) and kills any
 spoken interjection in flight. `speakEnd` fires when the track completes.
 
-## The user's voice — `setUserAudio(source?)`, `setUserSpeaking(bool)`
+## The user's voice — `setUserSpeaking(bool)`
 
 Backchannels only create rapport when they are *contingent* on the speaker —
 identical nods on a timer measurably read as distracting
-(docs/research-biomechanics.md §3.5). Give the widget the user's voice and the
-listening engine does the rest:
+(docs/research-biomechanics.md §3.5). Tell the widget when the user holds the
+floor and the listening engine does the rest.
 
-- `setUserAudio(streamOrElement)` — the widget runs its own coarse VAD on the
-  signal (RMS hysteresis: 80 ms on, 250 ms off). This audio never drives the
-  mouth; it is the *input* side.
-- `setUserSpeaking(bool)` — the host's (or server endpointer's) own turn
-  signal. Wins over the level VAD when both are driven; pass `null` to hand
-  back.
+`setUserSpeaking(bool)` is the whole input side: the turn signal from the
+server's endpointer, which is the one that already decides where a turn ends.
+Pass `null` to hand back. The widget used to be able to derive this itself from
+the user's `MediaStream` (`setUserAudio`), a second VAD racing the server's —
+`docs/removed.md` § Client-side VAD.
 
 While `LISTENING`/`WAITING_FOR_USER` with a signal supplied: an
 acknowledgement fires 250–600 ms after a user pause onset, on about half of
@@ -224,12 +262,12 @@ up.
 
 ### The mouth priority rule (invariant)
 
-**Server viseme track > clip mouth track > amplitude fallback.** While a
-server track plays, it owns the mouth outright: an interjection fired
-mid-utterance contributes head and brows only, and its mouth track is dropped.
-The amplitude fallback (`setAudioFallback(mediaElementOrStream)`) is the
-zero-server-work tier: an energy/spectral guesser that runs only when no cue
-track is live. Anything that degrades this ordering is a regression.
+**Server viseme track > clip mouth track.** While a server track plays, it
+owns the mouth outright: an interjection fired mid-utterance contributes head
+and brows only, and its mouth track is dropped. There used to be a third tier —
+an amplitude/spectral guesser off the bot's own audio element, for a server
+that sent no cues at all (`docs/removed.md` § Amplitude lipsync). Anything that
+degrades this ordering is a regression.
 
 ## Events, gains, introspection
 
@@ -240,6 +278,8 @@ track is live. Anything that degrades this ordering is a regression.
   its own (fires with the interjection id)
 - `on('performEnd', fn)` — a performance's last action has fired (see
   *Composing behavior*)
+- `on('gestureEnd', fn)` — the hand has left the frame (fires with the gesture
+  id). Only where a hand is mounted — see *Hand gestures*
 - `setMouthGain(g)` — scales viseme excursion away from rest (1 = as authored;
   useful when the avatar renders small). Never drags a closed mouth open.
 - `setGestureGain(g)` — scales clip deltas; small gestures under-render
@@ -252,7 +292,7 @@ track is live. Anything that degrades this ordering is a regression.
   "fidgety" starts moves with tile size and with the audience, so this is
   deliberately a host decision rather than a constant.
 - Getters: `state`, `emotion`, `gaze`, `speaking`, `performing`, `clip`,
-  `params` (the live smoothed vector), `audioLevel`, `svg`.
+  `gesturing`, `params` (the live smoothed vector), `svg`.
 - `setOverrides({channel: value})` — direct parameter injection, post-clamp.
   For tuning UIs and tests, not production.
 - `blink(double?)`, `destroy()`.
@@ -298,6 +338,7 @@ and tuned on the rig. A backend wanting a new move asks for a new enum entry
 | `emotion` | `name`, `i?` 0..1 (default 1) | `setEmotion(name, i)` |
 | `gaze` | `name` | `setGaze(name)` |
 | `interject` | `id` | `interject(id)` |
+| `gesture` | `id` | `gesture(id)` — the hand *and* its face half |
 
 The natural unit a server assembles is audio + cue track + action track on
 **one clock** (`demo/perf-clips.json` scripts every demo turn this way, and
@@ -345,22 +386,33 @@ package (`pip install voqalize-avatar`), a pipecat `FrameProcessor` that infers
 the base states from stock frames and emits the envelope below as RTVI
 server-messages. Design and rationale:
 [design-library-split.md](design-library-split.md). A host driving the widget
-through that stack never calls the API above directly; it mounts
-`@voqalize/avatar/pipecat`, whose `AvatarClient` dispatches those messages:
+through that stack never calls the API above directly; it renders
+`<Avatar client={pipecatClient} />` from `@voqalize/avatar`, and the dispatcher
+inside it turns these messages into the calls above:
 
 ```json
-{ "type": "avatar", "v": 1, "cmd": "state", "name": "THINKING" }
+{ "type": "avatar", "cmd": "state", "name": "THINKING" }
 ```
 
+`{"type": "avatar"}` is the whole membership test — an RTVI server-message in
+that envelope is the avatar's, and one outside it is not, whoever sent it.
+There is no protocol version field; forward compatibility is the ignore-unknown
+rule below, which a version number would not have improved
+(`docs/removed.md` § The `v` field).
+
 | `cmd` | payload → widget call |
 |---|---|
 | `state` | `name`, `emotion?`, `gaze?` → `setState` |
 | `interject` | `id` → `interject` |
-| `perform` | `actions`, `ctx` → `perform` |
+| `gesture` | `id` → `gesture` (a hand gesture id, not an interjection id) |
 | `cues` | `ctx`, `from_ms`, `cues`, `final?` → splice, then `speak`/`pushCues` |
 | `speech` | `event: start\|stop`, `ctx` → anchor / release the turn clock |
 | `user` | `speaking` → `setUserSpeaking` |
-| `hint` | `kind: eager_eot` → app hook (listening engine may ack now) |
+
+That is the whole wire vocabulary. `perform` (a timeline as one message) and
+`hint` (an advisory with no rendering) were both on it and are not any more —
+`docs/removed.md` § The `perform` command and § The `hint` command. `perform()`
+itself is untouched; what went away is a *server* being able to send one.
 
 Semantics the envelope adds on top of this contract:
 
@@ -376,9 +428,9 @@ Semantics the envelope adds on top of this contract:
   residual error lands video-first — the +125 ms side of the asymmetric
   tolerance window, not the −45 ms one.
 - **Explicit instructions override heuristics.** An application that knows
-  something the pipeline cannot infer pushes the same `{cmd, ...}` payload from
-  its own code; it dispatches into the same handler. The backend's state
-  heuristics are the default layer underneath, not a competing one.
+  something the pipeline cannot infer pushes the same envelope from its own
+  code; it dispatches into the same handler. The backend's state heuristics are
+  the default layer underneath, not a competing one.
 - Unknown `cmd`s are ignored — the server may grow vocabulary ahead of
   deployed clients.
 
@@ -390,12 +442,20 @@ Agreed direction, not yet landed; backend work can anticipate it:
   applications need them — same `setState` surface, one STATES entry each.
 - Backend heuristics for the states still unmapped server-side: `CANT_HEAR`
   from the STT's own confidence signal plus user volume; `DISTRACTED`,
-  `SEARCHING_SCREEN` and `TYPING` from tool-call names, which today only the
-  opt-in `tool_states` map reaches.
+  `SEARCHING_SCREEN` and `TYPING` from tool-call names. A tool call shows
+  `THINKING` today and only that — an application that wants to distinguish its
+  own tools subclasses `AvatarStateMachine` and says so.
 
 For servers written against a pre-2026-08 version of this contract, the
 renames were: `createKiran`→`createAvatar`, gaze `CANDIDATE`→`USER` and
 `CODE_AREA`→`SCREEN_WORK`, state `WAITING_FOR_ANSWER`→`WAITING_FOR_USER`.
 Everything else in this document — the descriptor (`api.meta`), `perform()`,
-the listening engine, the compound states and the disagree family — landed
-2026-08 and is current.
+the listening engine, the compound states, the disagree family and the hand
+gestures — landed 2026-08 and is current. `gesture` is the newest verb
+(2026-08-07); a widget older than it ignores the `cmd` and drops the
+`perform()` verb with a warning, which is the forward-compat rule working as
+intended, so a backend may send it unconditionally.
+
+The 0.2 release cut the wire down to the six commands in the table and the
+package down to one React component. Everything it removed, and how to get any
+of it back, is [removed.md](removed.md).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@voqalize/avatar",
-  "version": "0.0.1",
+  "version": "0.2.0",
   "description": "A JavaScript-programmable 2-D talking head for AI voice calls: server-synced lipsync, state through motion, zero dependencies.",
   "license": "AGPL-3.0-only",
   "author": "Voqalize",
@@ -33,16 +33,8 @@
   "sideEffects": false,
   "exports": {
     ".": {
-      "types": "./src/avatar.d.ts",
-      "default": "./src/avatar.js"
-    },
-    "./pipecat": {
-      "types": "./client/dist/pipecat.d.ts",
-      "default": "./client/dist/pipecat.js"
-    },
-    "./react": {
-      "types": "./client/dist/react.d.ts",
-      "default": "./client/dist/react.js"
+      "types": "./client/dist/index.d.ts",
+      "default": "./client/dist/index.js"
     },
     "./package.json": "./package.json"
   },

package/src/avatar.d.ts

@@ -89,6 +89,12 @@ export type AvatarInterjectionId =
   | "GOT_IT"
   | "TAKE_YOUR_TIME";
 
+/** `HAND_GESTURE_IDS` — see docs/contract-protocol.md § Hand gestures. A hand at
+ * the frame edge plus the face half that makes it belong to somebody. Disjoint
+ * from the interjection ids on purpose: `interject("WAVE")` is still the face
+ * alone. */
+export type AvatarHandGestureId = "HI" | "BYE" | "THUMBS_UP" | "ONE_MOMENT";
+
 /** Rhubarb Lip Sync letter — see docs/contract-protocol.md § Speech. */
 export type VisemeLetter = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "X";
 
@@ -108,7 +114,7 @@ export interface GazeCustom {
 /** A `perform()` timeline action — see docs/contract-protocol.md § Composing behavior. */
 export interface AvatarAction {
   t: number;
-  do: "state" | "emotion" | "gaze" | "interject";
+  do: "state" | "emotion" | "gaze" | "interject" | "gesture";
   name?: string;
   id?: string;
   i?: number;
@@ -139,7 +145,8 @@ export interface PerformHandle {
   stop: () => void;
 }
 
-export type AvatarEventName = "state" | "speakEnd" | "clipEnd" | "backchannel" | "performEnd";
+export type AvatarEventName =
+  | "state" | "speakEnd" | "clipEnd" | "backchannel" | "performEnd" | "gestureEnd";
 
 /** What a host needs to frame an avatar it has never seen: the drawing's own
  * window, and where the mouth is inside it. See CLAUDE.md § The two
@@ -158,19 +165,24 @@ export interface AvatarApi {
   pushCues(cues: Cue[]): AvatarApi;
   stopSpeaking(): AvatarApi;
   interject(id: AvatarInterjectionId | (string & {})): AvatarApi;
+  gesture(id: AvatarHandGestureId | (string & {})): AvatarApi;
+  /** +1 the viewer's right (the character's own left hand), -1 the other. */
+  setHandSide(dir: number): AvatarApi;
   perform(actions: AvatarAction[], o?: PerformOptions): PerformHandle;
-  setAudioFallback(source?: HTMLMediaElement | MediaStream | null): AvatarApi;
-  setUserAudio(source?: HTMLMediaElement | MediaStream | null): AvatarApi;
   setUserSpeaking(speaking: boolean | null): AvatarApi;
   setMouthGain(g: number): AvatarApi;
   readonly mouthGain: number;
   setGestureGain(g: number): AvatarApi;
   readonly gestureGain: number;
+  setMotionGain(g: number): AvatarApi;
+  readonly motionGain: number;
   blink(double?: boolean): AvatarApi;
+  /** Advance one frame by hand — only meaningful under `{ manual: true }`. */
+  step(dt: number): AvatarApi;
   setOverrides(o: Record<string, number> | null): AvatarApi;
   on(event: "state", fn: (name: AvatarStateName) => void): AvatarApi;
   on(event: "speakEnd" | "performEnd", fn: () => void): AvatarApi;
-  on(event: "clipEnd" | "backchannel", fn: (id: string) => void): AvatarApi;
+  on(event: "clipEnd" | "backchannel" | "gestureEnd", fn: (id: string) => void): AvatarApi;
   on(event: AvatarEventName, fn: (...args: unknown[]) => void): AvatarApi;
   readonly state: AvatarStateName;
   readonly emotion: AvatarEmotionName;
@@ -178,8 +190,9 @@ export interface AvatarApi {
   readonly speaking: boolean;
   readonly performing: boolean;
   readonly clip: string | null;
+  /** The hand gesture in flight, or null — always null under `hand: false`. */
+  readonly gesturing: AvatarHandGestureId | null;
   readonly params: Record<string, number>;
-  readonly audioLevel: number;
   readonly userSpeaking: boolean;
   readonly svg: SVGSVGElement;
   readonly meta: AvatarMeta;
@@ -209,6 +222,15 @@ export interface CreateAvatarOptions {
   theme?: unknown;
   mouthGain?: number;
   gestureGain?: number;
+  motionGain?: number;
+  /** Withhold the frame-edge hand entirely — for a face drawn in some other
+   * idiom, or a tile too small to spend the pixels. `gesture()` then degrades
+   * to the gesture's face half. Default true. */
+  hand?: boolean;
+  /** Which hand the character gestures with: +1 the viewer's right. */
+  handSide?: number;
+  /** Withhold the rAF loop so a tool can drive frames itself via `step(dt)`. */
+  manual?: boolean;
 }
 
 export function createAvatar(opts: CreateAvatarOptions): AvatarApi;
@@ -226,6 +248,16 @@ export const GAZE_TARGETS: Record<string, { x: number; y: number }>;
 export const EMOTION_NAMES: AvatarEmotionName[];
 export const INTERJECTIONS: Record<string, unknown>;
 export const INTERJECTION_IDS: AvatarInterjectionId[];
+export const HAND_GESTURES: Record<string, Record<string, unknown>>;
+export const HAND_GESTURE_IDS: AvatarHandGestureId[];
+/** Asserts the two framing rules against a face's own window. Throws on
+ * violation — `tools/sweep.mjs` runs it for every registered avatar. */
+export function checkHandFraming(meta: AvatarMeta): {
+  ok: true;
+  wristDrop: number;
+  outboardLimit: number;
+  worst: Record<string, number>;
+};
 /** The subset that has (or expects) audio — the rest are silent gestures. */
 export const SPOKEN_IDS: AvatarInterjectionId[];
 export const VISEME_LETTERS: VisemeLetter[];

package/src/avatar.js

@@ -8,10 +8,10 @@
  *   avatar.speak({ audio, cues })        // cues are {t, v, i?} in ms
  *   avatar.pushCues(moreCues)            // streaming top-up
  *   avatar.interject('OKAY')
+ *   avatar.gesture('HI')                 // a hand at the frame edge + its face
  *   avatar.perform(beats, { audio })     // timed {t, do, ...} verbs, same clock
- *   avatar.setUserAudio(micStream)       // or setUserSpeaking(bool) — the
- *                                        // user's voice, so listening is
- *                                        // contingent instead of timed
+ *   avatar.setUserSpeaking(bool)         // the user has the floor, so listening
+ *                                        // is contingent instead of timed
  *
  * Per frame the mixer runs a fixed layer order. Earlier layers are overwritten
  * by later ones on the channels they touch; the gesture and idle layers are
@@ -36,7 +36,7 @@ import { ClipPlayer } from './clips.js';
 import { INTERJECTIONS } from './interjections.js';
 import { VisemeTrack, shapeFor, SILENT } from './visemes.js';
 import { PerformTrack } from './perform.js';
-import { AudioFallback } from './audio-fallback.js';
+import { createHand, HAND_GESTURES } from './hand.js';
 
 // Each state's `idle` is a profile for the liveness layer (see DEFAULT_PROFILE
 // in idle.js). Blink gaps come from docs/research-biomechanics.md §5: the rate
@@ -282,11 +282,6 @@ export function createAvatar(opts = {}) {
   const gaze = new GazeLayer();
   const idle = new IdleLayer();
   const speech = new VisemeTrack();
-  const fallback = new AudioFallback();
-  // A second analyser for the USER's voice — same machinery, opposite
-  // direction: this one never touches the mouth, it feeds the listening engine.
-  const userAudio = new AudioFallback();
-  let userAudioOn = false;
 
   let gazeOverrideByClip = null;
   const clip = new ClipPlayer({
@@ -295,10 +290,18 @@ export function createAvatar(opts = {}) {
   });
   const backchannel = new ListeningEngine((id) => { interject(id); emit('backchannel', id); });
   const performTrack = new PerformTrack();
+  // The hand is a sibling of the mixer, not a layer inside it: it writes SVG
+  // directly rather than parameter channels, because a hand at the frame edge is
+  // not part of the rig's body (see hand.js, and CLAUDE.md constraint 9 for the
+  // arm chain this replaces). `hand: false` opts out — for a face drawn in some
+  // other idiom, or a tile too small to spend the pixels.
+  const hand = opts.hand === false ? null : createHand(face.svg, face.theme, meta, { dir: opts.handSide });
 
   gaze.onLargeShift = () => idle.blink();
 
-  const listeners = { state: [], speakEnd: [], clipEnd: [], backchannel: [], performEnd: [] };
+  const listeners = {
+    state: [], speakEnd: [], clipEnd: [], backchannel: [], performEnd: [], gestureEnd: [],
+  };
   const emit = (ev, ...a) => listeners[ev] && listeners[ev].forEach((f) => f(...a));
   clip.onEnd = (c) => { if (c) emit('clipEnd', c.id); };
   speech.onEnd = () => { emit('speakEnd'); };
@@ -339,7 +342,6 @@ export function createAvatar(opts = {}) {
   let glanceUntil = 0;
   let speakClock = null;
   let speakStart = 0;
-  let useFallback = false;
 
   const cur = Object.assign({}, REST);
   const target = Object.assign({}, REST);
@@ -426,7 +428,6 @@ export function createAvatar(opts = {}) {
       }
     }
     backchannel.enabled = !!st.backchannel && !clip.playing;
-    if (userAudioOn) { userAudio.sample(dt); backchannel.observeLevel(userAudio.level); }
     backchannel.update(dt);
     // Engagement posture: forward lean while the user holds the floor, spent
     // only in the states that are *about* the user holding the floor. The
@@ -439,12 +440,14 @@ export function createAvatar(opts = {}) {
     // costs nothing; with no user signal the static pose carries the state.
     else if (stateName === 'CANT_HEAR') target.torsoLean += 0.10 * backchannel.engage;
 
-    // 4. mouth. Server track wins; then clip track; then a fallback analyser.
+    // 4. mouth. The server's viseme track wins; a clip's mouth track fills the
+    // gaps. There is deliberately no third leg: with no cues the mouth stays
+    // shut, and a still mouth under speech is the *visible* symptom of a
+    // backend that could not align — see docs/removed.md § Amplitude lipsync.
     const clipOut = clip.update(dtMs);
     let mouth = speech.sample();
     let mouthOwner = mouth ? 'speech' : null;
     if (!mouth && clipOut.ownsMouth && clipOut.mouth) { mouth = clipOut.mouth; mouthOwner = 'clip'; }
-    if (!mouth && useFallback) { mouth = fallback.sample(dt); mouthOwner = 'fallback'; }
     if (mouth) {
       const shape = mouth.letter !== SILENT
         ? shapeFor(mouth.letter, mouth.intensity)
@@ -504,6 +507,16 @@ export function createAvatar(opts = {}) {
     for (const c of CHANNELS) cur[c] = approach(cur[c], target[c], TAU[c], dt);
 
     face.apply(cur);
+
+    // 9. the hand, outside all of the above. It writes a transform on its own
+    //    group rather than parameter channels, so it neither smooths nor
+    //    composes — its timelines are authored as delivered motion, not as
+    //    targets to chase. `elapsed` is the mixer's own clock, so a manual
+    //    stepper gets a reproducible gesture for free.
+    if (hand) {
+      const done = hand.update(elapsed * 1000);
+      if (done) emit('gestureEnd', done.id);
+    }
   }
 
   const REST_SHAPE = shapeFor(SILENT, 1);
@@ -591,28 +604,39 @@ export function createAvatar(opts = {}) {
     return api;
   }
 
-  function setAudioFallback(source) {
-    if (!source) { fallback.detach(); useFallback = false; return api; }
-    fallback.attach(source);
-    useFallback = true;
-    return api;
-  }
-
   /**
-   * Give the listening engine the USER's voice, so backchannels become
-   * contingent on their pauses instead of running on a timer. Either works,
-   * and the flag wins when both are driven:
-   *   setUserAudio(streamOrElement)  — the widget runs its own coarse VAD
-   *   setUserSpeaking(bool)          — the host (or server endpointer) decides;
-   *                                    pass null to hand back to the level VAD
+   * A hand gesture: the hand at the frame edge, plus the face half that makes it
+   * belong to somebody.
+   *
+   * The face half is not a convenience — a hand rising to the jaw over a head
+   * and shoulders sitting perfectly still is a cut-out, not a gesture. Each
+   * entry in HAND_GESTURES names an interjection that already exists and was
+   * already tuned (`WAVE`, `THUMBS_UP`, `ONE_MOMENT`); firing it here is the
+   * library composing two authored things, not the client inventing motion.
+   *
+   * The two halves stay separable in both directions. `interject('WAVE')` is
+   * still the face alone and is unchanged by this — a server that upgrades gets
+   * no new behaviour until it asks for one — and on an avatar mounted with
+   * `hand: false` this call degrades to exactly that interjection, which is the
+   * same graceful failure the arm removal already forced every id through.
    */
-  function setUserAudio(source) {
-    if (!source) { userAudio.detach(); userAudioOn = false; return api; }
-    userAudio.attach(source);
-    userAudioOn = true;
+  function gesture(id) {
+    const def = HAND_GESTURES[id];
+    if (!def) throw new Error(`unknown hand gesture: ${id}`);
+    if (hand) hand.play(id, elapsed * 1000);
+    if (def.face) interject(def.face);
+    // A hand in frame is a deliberate move; a backchannel landing on top of it
+    // is the listening engine talking over the server.
+    backchannel.reset(def.dur / 1000 + 0.5);
     return api;
   }
 
+  /**
+   * Tell the listening engine whether the USER holds the floor, so backchannels
+   * become contingent on their pauses instead of running on a timer. The server
+   * owns this — it has the endpointer — and sends it as the `user` command;
+   * `null` hands back to the no-signal timer fallback.
+   */
   function setUserSpeaking(b) { backchannel.setUserSpeaking(b); return api; }
 
   // What one action does when its moment comes. Enum validity is checked here,
@@ -627,6 +651,7 @@ export function createAvatar(opts = {}) {
       else if (a.do === 'emotion') setEmotion(a.name, a.i ?? 1);
       else if (a.do === 'gaze') setGaze(a.name);
       else if (a.do === 'interject') interject(a.id);
+      else if (a.do === 'gesture') gesture(a.id);
     } catch (e) {
       console.warn(`perform: ${a.do} at ${a.t}ms skipped — ${e.message}`);
     }
@@ -636,8 +661,8 @@ export function createAvatar(opts = {}) {
 
   /**
    * Play a timed action track — the composition surface a server assembles
-   * turns from. Verbs: state / emotion / gaze / interject (see perform.js for
-   * hygiene, docs/contract-protocol.md for the schema).
+   * turns from. Verbs: state / emotion / gaze / interject / gesture (see
+   * perform.js for hygiene, docs/contract-protocol.md for the schema).
    *
    * Clock resolution mirrors speak(): explicit `clock` fn, else the audio
    * element's own time, else ms elapsed since this call. perform() never
@@ -667,8 +692,12 @@ export function createAvatar(opts = {}) {
 
   const api = {
     setState, setEmotion, setGaze, speak, pushCues, stopSpeaking, interject,
-    perform,
-    setAudioFallback, setUserAudio, setUserSpeaking,
+    gesture, perform,
+    /** Which hand the character gestures with: +1 the viewer's right (its own
+     *  left), -1 the other. Both are anatomically real — the thumb splays away
+     *  from the body either way — so this is a character choice, not a fix. */
+    setHandSide: (d) => { if (hand) hand.setDir(d); return api; },
+    setUserSpeaking,
     /** Articulation gain: 1 is the VISEME_SHAPES table as authored. */
     setMouthGain: (g) => { mouthGain = g; return api; },
     get mouthGain() { return mouthGain; },
@@ -691,8 +720,9 @@ export function createAvatar(opts = {}) {
     get speaking() { return speech.playing; },
     get performing() { return performTrack.playing; },
     get clip() { return clip.id; },
+    /** The hand gesture in flight, or null. `null` forever if `hand: false`. */
+    get gesturing() { return hand ? hand.id : null; },
     get params() { return cur; },
-    get audioLevel() { return fallback.level; },
     get userSpeaking() { return backchannel.speaking; },
     svg: face.svg,
     meta,
@@ -701,7 +731,11 @@ export function createAvatar(opts = {}) {
      *  page behind a transparent mount — needs the same colours the drawing
      *  used, and guessing them per avatar is how the two drift apart. */
     theme: face.theme,
-    destroy() { cancelAnimationFrame(raf); fallback.detach(); userAudio.detach(); face.destroy(); },
+    destroy() {
+      cancelAnimationFrame(raf);
+      if (hand) hand.destroy();
+      face.destroy();
+    },
   };
 
   setState('IDLE');
@@ -712,6 +746,7 @@ export function createAvatar(opts = {}) {
 export { INTERJECTIONS, INTERJECTION_IDS, SPOKEN_IDS, attachAudio } from './interjections.js';
 export { GAZE_NAMES, GAZE_TARGETS } from './gaze.js';
 export { normalizeActions } from './perform.js';
+export { HAND_GESTURES, HAND_GESTURE_IDS, checkHandFraming } from './hand.js';
 export { EMOTION_NAMES } from './emotions.js';
 export {
   VISEME_LETTERS, VISEME_SHAPES, normalizeCues, textToCues,