summer-engine 1.5.0 → 2.2.0

package/skills/animation/animation-tree/SKILL.md

@@ -0,0 +1,260 @@
+---
+name: animation-tree
+description: Use when the user has clips on a character and needs them to play in response to gameplay — idle/walk/run blend, attacks that interrupt locomotion, hit reactions that override everything. Designs and wires AnimationTree state machines and blend trees in Godot 4.5. Trigger on "AnimationTree", "state machine", "blend tree", "transition", "play animation when", "character keeps T-posing", "wire animations".
+license: MIT
+compatibility: [Cursor, Claude Code, Windsurf, Codex]
+category: animation
+user-invocable: false
+allowed-tools: Read Grep Edit Write summer_get_scene_tree summer_inspect_node summer_inspect_resource summer_add_node summer_set_prop summer_set_resource_property summer_save_scene summer_get_script_errors
+paths: ["**/*.gd", "**/*.tscn", "**/*.tres"]
+---
+
+# AnimationTree — Wire Clips Into Behavior
+
+Generation gives you clips. The AnimationTree gives the character life. This skill is **not generative** — no FAL, no MCP AI calls. It composes existing clips on an `AnimationPlayer` into an `AnimationTree` with a `StateMachine` for distinct states (idle / attack / hit / dead) and `BlendSpace` nodes for continuous parameters (walk-run blend by speed). Done right, the character moves like a shipped game; done wrong, you get T-poses, snapping, or animations that "eat" each other.
+
+The canonical Godot 4.5 stack:
+
+```
+Character (CharacterBody3D)
+├── Skeleton3D (from the Meshy rig)
+├── AnimationPlayer            # holds the raw clips (idle, walk, run, attack_sword, ...)
+└── AnimationTree              # references AnimationPlayer, drives behavior
+```
+
+## When to use this skill
+
+- Character has 3+ clips and switching between them via `AnimationPlayer.play("walk")` is causing pops, T-poses, or "starts halfway through".
+- Need a walk → run blend that's continuous, not boolean.
+- Need attack to interrupt locomotion but only on the upper body, while feet keep walking.
+- Need hit-react one-shot that returns to whatever state was playing before.
+- Building any non-trivial NPC or player character past the prototype phase.
+
+## When NOT to use this skill
+
+- Single-clip looping background prop (banner waving, water surface). Just `AnimationPlayer.play("loop")` is fine.
+- Cinematic with hand-authored timing. Use `AnimationPlayer` directly with `play_with_capture`; AnimationTree is overkill.
+- Character has zero clips — generate first via `summer:animation/generate-motion`.
+- Procedural-only animation (no clips, all IK). That's `summer:animation/procedural-animation`.
+
+## Canonical state machine
+
+This is the production-default for a humanoid character that can move, attack, take a hit, and die. Start here, prune what you don't need, extend with named states for specials.
+
+```
+                     ┌─────────┐
+              ┌─────►│  Dead   │  (terminal — no exit)
+              │      └─────────┘
+              │           ▲
+              │           │ on_death
+   ┌──────────┴───────────┴──────────┐
+   │              Hit                │  (one-shot, returns to prev)
+   └──────────▲──────────────────────┘
+              │ on_damaged                  ┌──────────┐
+   ┌──────────┴──────────┐                  │  Attack  │  (one-shot)
+   │     Locomotion      │ on_attack ──────►└──────────┘
+   │   (BlendSpace1D)    │ ◄────── on_attack_done
+   │   idle ↔ walk ↔ run │
+   └─────────────────────┘
+```
+
+- **Locomotion** is a single AnimationNodeBlendSpace1D parameterized by `move_speed` (0..max). Three points on the line: idle (0), walk (~3), run (~6). Crossfades for free.
+- **Attack** is an AnimationNodeOneShot wrapping `attack_sword` (or whichever clip). Fires from the script via `parameters/attack/request = ONE_SHOT_REQUEST_FIRE`. Auto-returns to Locomotion when the clip ends.
+- **Hit** same pattern, higher priority — interrupts attack. Fires on `damaged` signal.
+- **Dead** is a transition-target with no exit. Trigger via `state_machine.travel("dead")`.
+
+## Steps via MCP
+
+### 1. Inspect the current character
+
+```
+summer_get_scene_tree
+summer_inspect_node "./World/Goblin"
+```
+
+Confirm there's a `Skeleton3D` and an `AnimationPlayer` with at least `idle` + `walk` + `run` populated. If clips are missing, route to `summer:animation/generate-motion`.
+
+### 2. Add the AnimationTree
+
+```
+summer_add_node(parent: "./World/Goblin", type: "AnimationTree", name: "AnimationTree")
+summer_set_prop(path: "./World/Goblin/AnimationTree", key: "anim_player", value: "../AnimationPlayer")
+summer_set_prop(path: "./World/Goblin/AnimationTree", key: "active", value: "true")
+```
+
+`anim_player` must be a relative NodePath; `../AnimationPlayer` from a sibling. Don't use absolute paths — breaks scene reuse.
+
+### 3. Build the tree resource
+
+The `tree_root` is a single `AnimationNodeStateMachine`. Inside it, a `Locomotion` node which is itself an `AnimationNodeBlendSpace1D`, plus `Attack`, `Hit`, `Dead` nodes.
+
+This is more readable to edit by hand or in the Godot editor, then commit the `.tres`. The MCP tool `summer_set_resource_property` can build it node-by-node, but the tree-resource model is verbose enough that **writing the `.tres` and importing it is faster** for the canonical structure. Skeleton:
+
+```
+[gd_resource type="AnimationNodeStateMachine" load_steps=8 format=3]
+
+[sub_resource type="AnimationNodeBlendSpace1D" id="locomotion"]
+blend_point_0/node = SubResource("idle_clip")
+blend_point_0/pos = 0.0
+blend_point_1/node = SubResource("walk_clip")
+blend_point_1/pos = 3.0
+blend_point_2/node = SubResource("run_clip")
+blend_point_2/pos = 6.0
+min_space = 0.0
+max_space = 6.0
+
+[sub_resource type="AnimationNodeOneShot" id="attack_oneshot"]
+fadein_time = 0.05
+fadeout_time = 0.10
+mix_mode = 0   # 0=BLEND, 1=ADD
+
+[sub_resource type="AnimationNodeOneShot" id="hit_oneshot"]
+fadein_time = 0.0    # snap in — hit reactions are immediate
+fadeout_time = 0.15
+
+# ... AnimationNodeAnimation sub_resources point at clip names ...
+
+[resource]
+states/Locomotion/node = SubResource("locomotion")
+states/Attack/node = SubResource("attack_oneshot")
+states/Hit/node = SubResource("hit_oneshot")
+states/Dead/node = SubResource("dead_clip")
+transitions = [
+  "Locomotion", "Attack",   SubResource("trans_to_attack"),
+  "Attack",     "Locomotion", SubResource("trans_back"),
+  "Locomotion", "Hit",      SubResource("trans_to_hit"),
+  "Attack",     "Hit",      SubResource("trans_to_hit"),
+  "Hit",        "Locomotion", SubResource("trans_back"),
+  "Locomotion", "Dead",     SubResource("trans_to_dead"),
+  "Attack",     "Dead",     SubResource("trans_to_dead"),
+  "Hit",        "Dead",     SubResource("trans_to_dead")
+]
+graph_offset = Vector2(0, 0)
+```
+
+Each `AnimationNodeStateMachineTransition` carries `xfade_time` (0.15 default), `switch_mode` (Immediate / Sync / At End), and `advance_mode` (Auto / Enabled / Disabled).
+
+Ask before writing the `.tres`: "May I write `res://characters/goblin/animation_tree.tres` with the canonical 4-state machine + Locomotion blendspace?"
+
+### 4. Drive it from the controller script
+
+```gdscript
+@onready var tree: AnimationTree = $AnimationTree
+@onready var sm: AnimationNodeStateMachinePlayback = tree["parameters/playback"]
+
+func _physics_process(delta: float) -> void:
+    var horiz_speed: float = Vector2(velocity.x, velocity.z).length()
+    tree.set("parameters/Locomotion/blend_position", horiz_speed)
+
+func attack() -> void:
+    tree.set("parameters/Attack/request", AnimationNodeOneShot.ONE_SHOT_REQUEST_FIRE)
+
+func take_hit() -> void:
+    tree.set("parameters/Hit/request", AnimationNodeOneShot.ONE_SHOT_REQUEST_FIRE)
+
+func die() -> void:
+    sm.travel("Dead")
+```
+
+`tree.set(...)` writes a parameter path. `sm.travel(...)` switches state.
+
+### 5. Connect signals
+
+```
+summer_connect_signal(
+  source: "./World/Goblin",
+  signal: "damaged",
+  target: "./World/Goblin",
+  method: "take_hit"
+)
+```
+
+If the goblin has an HP component emitting `died`, connect that to `die`.
+
+### 6. Save & verify
+
+```
+summer_save_scene
+summer_get_script_errors
+```
+
+Then `summer_play` and watch — the goblin should idle in place, blend up to walk/run as it moves, fire attack on input/AI, snap to hit on damage, end in Dead.
+
+## Confirmation gates
+
+- **Before writing the `.tres`:** show the state list and transition table.
+- **Before connecting signals:** show source → target → method.
+- **Before pressing play:** state which behaviors should be visible (idle plays, walk crossfades, attack interrupts).
+
+## Reference card
+
+### Parameter paths cheat-sheet (Godot 4.5)
+
+| Goal | Path |
+|---|---|
+| Set blendspace1D position | `parameters/Locomotion/blend_position` |
+| Fire a OneShot | `parameters/<NodeName>/request` (set to `ONE_SHOT_REQUEST_FIRE`) |
+| Abort a OneShot | `parameters/<NodeName>/request = ONE_SHOT_REQUEST_ABORT` |
+| Read OneShot active state | `parameters/<NodeName>/active` |
+| Get StateMachine playback | `parameters/playback` (returns `AnimationNodeStateMachinePlayback`) |
+| Switch state | `state_machine.travel("StateName")` |
+| Read current state | `state_machine.get_current_node()` |
+| Force state without transition | `state_machine.start("StateName")` |
+
+### Transition `switch_mode` decision
+
+- **Immediate (0):** snap to new state at xfade. Use for hit reactions, deaths.
+- **Sync (1):** match current play position. Use for walk ↔ run (avoids foot-pop mid-stride).
+- **At End (2):** finish current clip first. Use for attack-string combos (`attack_1` → `attack_2` only at end of `attack_1`).
+
+### Pitfalls
+
+- **Character T-poses on play.** `AnimationTree.active = true` was missed. Set it in `_ready()` or via inspector.
+- **Locomotion blendspace stuck on idle even when moving.** The `blend_position` parameter isn't being written — confirm `tree.set("parameters/Locomotion/blend_position", speed)` runs every frame in `_physics_process`. Also confirm the path matches your state name exactly (case-sensitive).
+- **Walk-to-run pops.** Transition `switch_mode` is `Immediate`. Set to `Sync` so the cycle phase matches across the crossfade.
+- **Attack one-shot fires every frame as long as input is held.** Use `Input.is_action_just_pressed`, not `is_action_pressed`. Or store a `_can_attack` flag with cooldown.
+- **Hit reaction interrupts itself when hit twice in 100ms.** Set `fadein_time = 0.0` and gate from script: `if sm.get_current_node() == "Hit": return`.
+- **Attack plays but locomotion is invisible (lower body T-pose).** OneShot's `mix_mode` is `BLEND` (full body). For upper-body-only attacks, set `mix_mode = ADD` and use an additive clip (recorded as the offset from idle), or split the rig with a SkeletonModification3D mask.
+- **Dead state replays from start every time HP < 0.** Add a guard in the controller: `if is_dead: return`.
+
+### One-shot vs StateMachine — which to use
+
+- **OneShot:** the action is a *temporary override* of an underlying state (attack while running, hit-react while attacking). The base keeps playing.
+- **StateMachine state:** the action is a *replacement* (dead, stunned, ragdoll). Nothing else plays.
+- Combat in 2026-feel games is almost always OneShot for attacks + StateMachine state for "dead". Don't model attacks as StateMachine states unless you need precise transition control between attack-string variants.
+
+### BlendSpace1D vs BlendSpace2D
+
+- 1D for speed-only locomotion (idle / walk / run).
+- 2D for omnidirectional movement (idle / forward / strafe-left / strafe-right / back). Eight or nine clips arranged on a unit circle. Necessary for top-down or strafing third-person; overkill for FPS hands.
+
+## Anti-patterns
+
+- Skipping AnimationTree and calling `AnimationPlayer.play("walk")` from script. Works for one clip; breaks the moment you need a crossfade.
+- Modeling every attack as a StateMachine state. The transition matrix explodes (N states × N transitions). Use one OneShot per attack and switch its inner clip via `parameters/Attack/animation`.
+- Writing the `.tres` by hand without using sub_resource IDs — Godot loses references and the inspector is unhappy. Always use `[sub_resource id="..."]` blocks.
+- Driving `blend_position` from raw input (`Input.get_axis(...)`). Drive from actual velocity — that way the blend matches the body's real motion, including knockback and slope-slide.
+
+## Edge cases
+
+- **Player rotates 180° instantly while sprinting.** The blendspace doesn't know about turning — locomotion looks like the character moonwalks for one frame. Add a small additive turn-lean clip layered on top, or add a brief "turn" state with switch_mode=AtEnd.
+- **Animation library uses root motion.** Set `root_motion_track` on the AnimationTree (path to the root bone), then in `_physics_process` use `tree.get_root_motion_position()` and apply to the CharacterBody3D. Don't combine with input-driven velocity — pick one.
+- **Character has no AnimationPlayer.** Add one first. AnimationTree without anim_player target is a no-op.
+
+## Fallback (no MCP)
+
+Open the scene in Godot 4.5 editor, add `AnimationTree` node, set `Anim Player` to the AnimationPlayer, set `Active = on`, double-click `Tree Root` and edit visually. Drag `AnimationNodeStateMachine` as the root, drop nodes inside, draw transitions by Shift-dragging. Save the scene. The visual editor produces the same `.tres` — slower but discoverable.
+
+## Handoff
+
+- For NPC behavior firing these states (decide-when-to-attack), hand off to `summer:ai-and-npcs/design-npc`.
+- For first-person hand animations on the player rig, the same pattern applies — hand off to `summer:character-controllers/fps-controller` for the state inputs.
+- For procedural overlays on top of the tree (look-at the player while idling), hand off to `summer:animation/procedural-animation`.
+- For lipsync layered on the face during dialogue states, hand off to `summer:animation/facial-and-lipsync`.
+
+## See also
+
+- `summer:animation/generate-motion` — produce the clips this tree references.
+- `summer:animation/retarget` — share one tree across many characters.
+- `_shared/gd-style.md` — typed GDScript conventions in the controller snippets.
+- `_shared/mcp-tools-reference.md` — `summer_connect_signal` and `summer_set_resource_property` rules.

package/skills/animation/facial-and-lipsync/SKILL.md

@@ -0,0 +1,282 @@
+---
+name: facial-and-lipsync
+description: Use when the user has a voice-over (or wants one) and needs the character's mouth to actually move with the words — phoneme extraction from audio, mapping to viseme blendshapes, plus emotional facial expressions (smile, frown, surprise). Trigger on "lipsync", "lip sync", "talking head", "phonemes", "viseme", "facial animation", "blendshapes", "make him talk", "dialogue animation".
+license: MIT
+compatibility: [Cursor, Claude Code, Windsurf, Codex]
+category: animation
+user-invocable: false
+allowed-tools: Read Grep Edit Write summer_search_assets summer_inspect_resource summer_inspect_node summer_generate_audio summer_generate_motion summer_add_node summer_set_prop summer_set_resource_property summer_save_scene summer_get_script_errors
+paths: ["**/*.gd", "**/*.tscn", "**/*.tres"]
+---
+
+# Facial Animation & Lipsync
+
+Half generative, half authored. The *generative* half: an audio file (TTS via `summer_generate_audio` with `capability: "text_to_speech"`) goes through a phoneme-extraction tool (Summer does NOT wrap one — you run it externally), out comes a viseme timeline — a list of `{ phoneme, start_time, duration }` triples. The *authored* half: the character's face mesh must have BlendShapes named for the standard viseme set, or the timeline has nothing to drive. Without both halves, you get a talking robot.
+
+The 2026 production stack:
+
+1. **Audio in** — `.wav`/`.mp3` from `summer_generate_audio({capability: "text_to_speech", ...})` or imported VO.
+2. **Phoneme extraction** — done OUTSIDE Summer. Recommended: **Rhubarb Lip Sync** (open source, the industry standard for game-dev lipsync; outputs JSON with mouth-shape cues over time). Alternatives: gentle, allosaurus, or a Whisper-phoneme model on Replicate / Hugging Face. There is no `summer_*` MCP for this in the current engine.
+3. **Viseme mapping** — phonemes / Rhubarb cues → Oculus / Apple ARKit viseme set (15 standard shapes covers English).
+4. **BlendShape driver** — at runtime, lerp the mesh's BlendShape weights along the timeline, synced to audio playback.
+5. **Optional emotional layer** — separate BlendShape track for `smile`, `brow_raise`, `eye_squint` etc., authored in code or at clip-edit time.
+
+## When to use this skill
+
+- NPC has voice lines and the mouth doesn't move.
+- Cinematic with VO that needs lipsync.
+- "Make him sing the lyrics."
+- Animating expressions on a face that has BlendShapes (Meshy character heads ship with the ARKit-52 set by default).
+- Reactive face: smile when player gives gold, scowl when attacked.
+
+## When NOT to use this skill
+
+- Character has no face / no BlendShapes (helmeted soldier, robot, mascot). Hard-skip to body language via `summer:animation/generate-motion`.
+- The dialogue is written but not yet voiced — generate audio first via `summer_generate_audio({capability: "text_to_speech", ...})`. Lipsync without audio has nothing to sync to.
+- Pre-rendered cinematic from external DCC. Lipsync is in the rendered video, not the engine.
+
+## Steps
+
+### 1. Confirm the head has BlendShapes
+
+```
+summer_inspect_node "./World/NPC/Head"     # or wherever the head MeshInstance3D lives
+```
+
+Look for `mesh.blend_shape_count` > 0 and a list of names. The Meshy ARKit-52 set (default for Meshy character heads) has names like `jawOpen`, `mouthClose`, `mouthFunnel`, `mouthPucker`, `mouthLeft`, `mouthRight`, `mouthSmile_L`, `mouthSmile_R`, `mouthFrown_L`, `mouthFrown_R`, `browInnerUp`, `browOuterUp_L`, `browDown_L`, `eyeBlink_L`, `eyeWide_L`, `eyeSquint_L` (mirrored on R).
+
+If the head has zero BlendShapes, stop. Tell the user: "This head mesh has no BlendShapes — facial animation is impossible without re-meshing. Options: regenerate the character with the `face_blendshapes: arkit` option (`summer_generate_3d({ kind: \"image-to-3d\", options: { rig: true, face_blendshapes: \"arkit\" } })`), use an emotional body-language overlay instead, or hand off the character to a 3D artist for shape-key authoring." Don't proceed.
+
+Note: the `face_blendshapes` option is a Meshy passthrough; if your target backend doesn't support it, the rig will use the default skeleton — adjust manually in the Meshy dashboard.
+
+### 2. Get the audio
+
+If the user has VO already, use it. If not, generate via TTS. The `voiceId` is an ElevenLabs voice ID — the user has to pick one from the ElevenLabs voice library (https://elevenlabs.io/app/voice-library) or upload/clone their own. There is no "voice name" string; it's always an ID like `21m00Tcm4TlvDq8ikWAM` (Rachel) or a custom-cloned ID.
+
+```
+summer_generate_audio({
+  capability: "text_to_speech",
+  text: "Welcome to the village, traveler.",
+  voiceId: "<elevenlabs_voice_id>"
+})
+// returns { jobId, ... } — poll with summer_check_job for the audio asset
+```
+
+If the user hasn't picked a voice yet, stop and ask: "Which ElevenLabs voice ID should I use? You can browse the library at elevenlabs.io/app/voice-library and copy the ID, or paste a custom-clone ID from your account."
+
+### 3. Extract phonemes (the generative step — runs OUTSIDE Summer)
+
+Summer does not wrap a phoneme-extraction MCP tool. Run it externally, then import the result. **Recommended: Rhubarb Lip Sync** — an open-source CLI built for game-dev lipsync. Mouth shapes A–H map cleanly to viseme shapes.
+
+Install once:
+
+```
+# macOS
+brew install rhubarb-lip-sync
+# Windows / Linux: download release from https://github.com/DanielSWolf/rhubarb-lip-sync
+```
+
+Run per VO line:
+
+```
+rhubarb -f json -o welcome_traveler.json welcome_traveler.wav
+```
+
+Output JSON shape:
+
+```json
+{
+  "metadata": { "duration": 2.34 },
+  "mouthCues": [
+    { "start": 0.00, "end": 0.08, "value": "B" },
+    { "start": 0.08, "end": 0.18, "value": "C" },
+    { "start": 0.18, "end": 0.25, "value": "D" },
+    { "start": 0.25, "end": 0.30, "value": "B" },
+    { "start": 0.30, "end": 0.42, "value": "A" },
+    { "start": 0.42, "end": 0.50, "value": "B" },
+    { "start": 2.30, "end": 2.34, "value": "X" }
+  ]
+}
+```
+
+Rhubarb mouth-shape → ARKit viseme cheat sheet (apply during the bake step):
+
+| Rhubarb | ARKit viseme | Notes |
+|---|---|---|
+| A | `viseme_PP` | closed (P, B, M) |
+| B | `viseme_kk` | slightly open, neutral |
+| C | `viseme_E` | open, lips spread (EH, IH) |
+| D | `viseme_aa` | wide open (AA, AH) |
+| E | `viseme_O` | rounded (OW, ER) |
+| F | `viseme_U` | small rounded (UW, OO) |
+| G | `viseme_FF` | F, V (lower lip + teeth) |
+| H | `viseme_RR` | L, R (tongue raised) |
+| X | `viseme_sil` | silence / closed neutral |
+
+**Manual fallback (no Rhubarb available):** for short lines you can hand-type a `mouthCues` array by listening to the clip and tagging vowel/consonant boundaries. Painful past ~3s of audio; use Rhubarb for anything longer.
+
+**Cloud fallback:** a Whisper-phoneme model on Replicate or a Hugging Face inference endpoint will emit ARPAbet phonemes with timestamps. Then map ARPAbet → ARKit visemes via the table in the Reference card section. More accurate than Rhubarb on noisy audio; slower and costs cents per minute.
+
+Cost: Rhubarb is free + ~5s CPU per 30s clip locally. Cloud fallback ~$0.02 / minute, ~5s wall-clock for a 30s clip.
+
+### 4. Persist the viseme track as an AnimationLibrary entry
+
+Convert Rhubarb's `mouthCues` into a Godot Animation resource — one track per BlendShape, keyframes at each viseme transition. This makes lipsync replayable via the same AnimationPlayer/AnimationTree as body motion.
+
+```gdscript
+# scripts/lipsync_baker.gd — run once per VO line at edit time
+const RHUBARB_TO_VISEME := {
+    "A": "viseme_PP",  "B": "viseme_kk", "C": "viseme_E",
+    "D": "viseme_aa",  "E": "viseme_O",  "F": "viseme_U",
+    "G": "viseme_FF",  "H": "viseme_RR", "X": "viseme_sil",
+}
+
+static func bake_from_rhubarb(rhubarb_json_path: String, head_path: NodePath) -> Animation:
+    var f := FileAccess.open(rhubarb_json_path, FileAccess.READ)
+    var data: Dictionary = JSON.parse_string(f.get_as_text())
+    var cues: Array = data["mouthCues"]
+    var anim := Animation.new()
+    anim.length = float(data["metadata"]["duration"])
+    var visemes := ["viseme_aa", "viseme_E", "viseme_I", "viseme_O", "viseme_U",
+                    "viseme_PP", "viseme_FF", "viseme_TH", "viseme_DD", "viseme_kk",
+                    "viseme_CH", "viseme_SS", "viseme_nn", "viseme_RR", "viseme_sil"]
+    var tracks := {}
+    for v in visemes:
+        var idx := anim.add_track(Animation.TYPE_BLEND_SHAPE)
+        anim.track_set_path(idx, NodePath(str(head_path) + ":" + v))
+        tracks[v] = idx
+    # For each cue, set the active viseme to 1.0 and others to 0.0 at cue.start
+    for cue in cues:
+        var active_viseme: String = RHUBARB_TO_VISEME.get(cue["value"], "viseme_sil")
+        for v in visemes:
+            var weight: float = 1.0 if v == active_viseme else 0.0
+            anim.track_insert_key(tracks[v], float(cue["start"]), weight)
+    return anim
+```
+
+Bake once, save into the character's AnimationLibrary as `dialogue_<line_id>`, and play via the AnimationTree.
+
+If you used the Whisper/ARPAbet cloud fallback instead of Rhubarb, swap `bake_from_rhubarb` for a variant that consumes `{ phoneme, start, duration }` triples and applies the ARPAbet → viseme table from the Reference card.
+
+### 5. Wire into the AnimationTree
+
+Add a OneShot node `Lipsync` that overlays the viseme animation as an *additive* layer over the base face. Fire from the dialogue system:
+
+```gdscript
+@onready var tree: AnimationTree = $AnimationTree
+@onready var audio: AudioStreamPlayer3D = $VoicePlayer
+
+func say(line_id: String) -> void:
+    var clip_id := "dialogue_" + line_id
+    tree.set("parameters/Lipsync/animation", clip_id)
+    audio.stream = load("res://audio/" + line_id + ".ogg")
+    audio.play()
+    tree.set("parameters/Lipsync/request", AnimationNodeOneShot.ONE_SHOT_REQUEST_FIRE)
+```
+
+Sync is preserved as long as both fire on the same frame. ~16ms drift is the threshold of perception; AnimationTree + AudioStreamPlayer3D are both sample-accurate, so drift only happens if the engine hitches mid-line.
+
+### 6. Add the emotional layer (authored, not generative)
+
+A second OneShot or persistent additive track for expressions. Key the relevant BlendShapes (`mouthSmile_L`, `mouthSmile_R`, `browInnerUp`, etc.) at design time:
+
+```gdscript
+func smile(intensity: float) -> void:
+    var head: MeshInstance3D = $Head
+    head.set_blend_shape_value(head.find_blend_shape_by_name("mouthSmile_L"), intensity)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("mouthSmile_R"), intensity)
+
+func surprise(intensity: float) -> void:
+    var head: MeshInstance3D = $Head
+    head.set_blend_shape_value(head.find_blend_shape_by_name("browInnerUp"), intensity)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("browOuterUp_L"), intensity * 0.7)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("browOuterUp_R"), intensity * 0.7)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("eyeWide_L"), intensity)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("eyeWide_R"), intensity)
+    head.set_blend_shape_value(head.find_blend_shape_by_name("jawOpen"), intensity * 0.3)
+```
+
+Driven from gameplay events; orthogonal to the lipsync layer.
+
+## Confirmation gates
+
+- **Before extracting:** show audio length, est. cost, est. wait. Wait for OK.
+- **Before baking the Animation resource:** confirm the head's BlendShape names match the standard set. If they're custom-named, ask for the mapping.
+- **Before saving the scene:** confirm the AnimationTree changes (added Lipsync OneShot, library entry).
+
+## Reference card
+
+### ARKit-52 viseme subset (the 15 that matter for English lipsync)
+
+| Viseme | Triggered by phonemes (ARPAbet) | Mouth shape |
+|---|---|---|
+| `viseme_sil` | silence | closed neutral |
+| `viseme_PP` | P, B, M | lips pressed |
+| `viseme_FF` | F, V | lower lip + upper teeth |
+| `viseme_TH` | TH, DH | tongue tip + teeth |
+| `viseme_DD` | T, D, N, S, Z | tongue + alveolar |
+| `viseme_kk` | K, G, NG | back-tongue, mouth slightly open |
+| `viseme_CH` | CH, JH, SH, ZH | lips rounded forward |
+| `viseme_SS` | S, Z (sometimes split from DD) | teeth nearly closed |
+| `viseme_nn` | N, L | tongue-tip + open mouth |
+| `viseme_RR` | R, ER | mouth slightly rounded |
+| `viseme_aa` | AA, AH, AE | wide open |
+| `viseme_E` | EH, EY, IH | mid open + lips spread |
+| `viseme_I` | IY, IH | narrow + spread |
+| `viseme_O` | OW, AO | rounded |
+| `viseme_U` | UW, UH | rounded + small |
+
+### Phoneme → viseme mapping (for hand-rolled extraction)
+
+If the user has a CMUDict-style phoneme list and wants to map manually, this is the table to apply.
+
+### Pitfalls
+
+- **Mouth never closes between words.** No `viseme_sil` keyframes at silence intervals. The bake step must scan the audio for silence (RMS below threshold for 100ms+) and insert sil keys, OR the phoneme extractor must emit silence markers. Rhubarb emits `X` cues for silence (mapped above to `viseme_sil`); cloud Whisper-phoneme models often don't — add a silence-detection pass if you go that route.
+- **Lipsync drifts behind audio.** Audio playback latency on some platforms is 30–60ms. Either delay the audio start by 1 frame, or pre-shift the animation by the platform's known latency. On desktop Linux audio output can be 60ms behind; on Steam Deck ~20ms.
+- **Visemes pop on/off.** Crossfade between viseme keyframes — set `Animation.TRACK_INTERPOLATION_LINEAR` (default) and ensure each viseme's weight ramps from previous to current. Default bake does this; if you wrote custom keyframes with NEAREST interp, switch.
+- **Smile fights lipsync.** Both write to mouth BlendShapes. Solve by additive layering: lipsync layer outputs deltas from neutral, smile layer outputs deltas from neutral, sum them, clamp 0..1. ARKit shapes are sum-safe up to ~1.5; clamp prevents over-rotation.
+- **Eyes look dead.** Lipsync is mouth-only; without blinks and saccades the face is uncanny. Add an idle blink track (every 4–8s, jittered) and a small saccade track (random eye movement up to 5°). Both can be one-shots fired by a `Timer`.
+- **Phoneme extraction returns gibberish.** Audio is too noisy, mismatched language code, or compressed too aggressively. Re-export the source as 22kHz mono WAV before sending. Don't lipsync from a 64kbps MP3.
+- **Custom rig has different BlendShape names.** The bake assumes ARKit-52. If the rig uses Preston Blair or a custom set, write a name-mapping `Dictionary` and apply it during bake. Only do this once per character.
+
+### Quality bar
+
+- ~12 phonemes/sec is normal English speech; the bake produces ~25 keyframes/sec across all tracks. Below 60fps playback on weak hardware: enable `BlendShape track interpolation = NEAREST` (loses smoothness, gains 30% perf). Reserve for mobile / very crowded crowd scenes.
+- Lipsync alone is ~70% of "alive". Add idle blink + idle micro-head-bob (`summer:animation/procedural-animation`) and you're at ~95%. The remaining 5% is brow articulation tied to dialogue sentiment, which is bespoke per scene.
+
+## Anti-patterns
+
+- Driving BlendShape weights from `_process` with hand-coded curves. The Animation track is sample-accurate and re-uses the AnimationTree's interpolation; manual code drifts and stutters.
+- Using a single "talking" loop instead of phoneme-driven lipsync. The 1990s look. Always extract phonemes; the cost is trivial.
+- Forgetting to gate audio + animation on the same frame. If you `play()` audio in `_ready` and fire the OneShot in `_process`, you'll see ~16ms drift at line start. Both calls in the same function, same frame.
+- Using lipsync for non-talking sounds (groans, screams). Lipsync needs phonemes; non-verbal vocals confuse the extractor. For grunts, drive `jawOpen` from the audio's RMS envelope instead — much simpler.
+
+## Edge cases
+
+- **Multilingual VO.** Rhubarb supports English best; for other languages, pass `--recognizer phonetic` (language-agnostic, less accurate) or use a multilingual Whisper-phoneme model on Replicate / HF. Cross-language lipsync (extract as English on Spanish audio) gives ~70% accuracy — bad enough that subtitles are needed regardless.
+- **Singing.** Phoneme extractor handles sustained vowels well, but consonant timing is loose. For sung dialogue, manually keyframe consonants and let extracted vowels fill in.
+- **Aside / muttering at low volume.** Phoneme extractor needs > -40 dB. Boost the source clip before extracting if the line is intentionally quiet, then play it at the original volume in-engine.
+- **Stylized character with no jaw bone (e.g., a cartoon ball).** No bone, but BlendShapes can still drive a "morph open" shape. Same pipeline; skip the jaw-bone-track and only animate BlendShapes.
+
+## Fallback (no Rhubarb)
+
+If Rhubarb won't install on the user's platform, run a Whisper-phoneme model on Replicate (e.g., `cjwbw/whisper-phoneme`) or `aeneas-align` for forced alignment. Download the JSON, swap the bake function for one that consumes ARPAbet phonemes (table in Reference card). Same end result — replayable BlendShape Animation resource.
+
+For projects that can't use any cloud or third-party tool, Godot 4.5's `AudioStreamGenerator` with hand-rolled vowel/consonant detection from RMS + zero-crossings gives ~50% accuracy — enough for a stylized character but not photoreal.
+
+## Handoff
+
+- For voice generation upstream, `summer:audio/generate-voice` (which wraps `summer_generate_audio({capability: "text_to_speech", ...})`).
+- For dialogue scripts and conversation flow, `summer:ai-and-npcs/design-npc`.
+- For the AnimationTree this layer composes into, `summer:animation/animation-tree`.
+- For idle blinks, saccades, and head-tracking that complement lipsync, `summer:animation/procedural-animation`.
+- For full performance capture (face + body together), out of scope — see Meshy's mocap docs or external pipelines.
+
+## See also
+
+- `summer:audio/generate-voice` — TTS upstream of this skill.
+- `summer:animation/animation-tree` — wire the lipsync OneShot into the character's tree.
+- `summer:animation/procedural-animation` — eye blinks, saccades, head idle.
+- `_shared/mcp-tools-reference.md` — `summer_generate_audio` schema (TTS).
+- Rhubarb Lip Sync — https://github.com/DanielSWolf/rhubarb-lip-sync (external tool; phoneme extraction).