kinescript 1.2.0__tar.gz → 1.3.1__tar.gz

{kinescript-1.2.0 → kinescript-1.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kinescript
-Version: 1.2.0
+Version: 1.3.1
 Summary: Render motion graphics, video edits and voice-over from JSON scenes on the GPU, with checks an agent can read.
 Author: Dr. Know How
 License-Expression: MIT
@@ -133,9 +133,17 @@ Sounds are a top-level `audio` list, timed like layers and mixed by ffmpeg while
   word: `"at": "vo1:render"` starts a layer as the voice says "render", and `{"type": "captions", "voice": "vo1"}`
   captions it, lighting each word as it is said.
 - **Voice-over from text.** Write what a voice says in `text` and `kine voice` speaks it through a plugin, writing
-  the file and its word times. [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
-  is the first. `[[go]]` in the text names the next word, so a cue to `vo1:go` survives a rewrite of the line.
-  Only a line whose text, speaker or rate changed is spoken again, and rendering never calls a speech engine.
+  the file and its word times: [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
+  (local, fast),
+  [Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox) (local,
+  cloned voices) or [ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs)
+  (cloud, needs an API key), picked per sound with `backend`. `[[go]]` in the text names the next
+  word, so a cue to `vo1:go` survives a rewrite of the line. Only a line whose text, speaker, rate, backend or
+  options changed is spoken again, and rendering never calls a speech engine.
+- **Delivery tags.** `[whispers]`, `[laughs]` or `[sighs]` in the text tell ElevenLabs how to say what follows.
+  Chatterbox takes them too but acts them only some of the time: in test takes it made a sound where `[laughs]`
+  goes in six of nine and whispered in none of seven. Tags are not words, so cues, marks and captions skip them,
+  and Kokoro says the line without them.
 - **A mix without listening.** Give a sound a `role` and it is set to that role's level: voice -18 LUFS, music
   -16, ambience -30, effects peaking at -6 dBFS. Music and ambience dip 14 dB while a voice speaks, an effect's
   leading silence is skipped so it lands on its cue, and the whole mix ends at -16 LUFS with peaks under
@@ -161,14 +169,20 @@ A narrated beat with a click on a marked word, over music that ducks under the v
 
 `kine voice scene.json` speaks it once. The
 [voice demo](https://drknowhow.github.io/kinescript/#sound) on the project site plays the result with sound, next
-to its scene file. The Kokoro plugin needs Python 3.10 to 3.12:
+to its scene file. Install the plugin for the voice you want next to Kinescript:
 
 ```bash
-pip install "kinescript-kokoro @ git+https://github.com/drknowhow/kinescript#subdirectory=plugins/kinescript-kokoro"
+pip install kinescript-kokoro       # local and fast; Python 3.10 to 3.12
+pip install kinescript-chatterbox   # local, copies a voice from a clip, tags hit or miss; installs PyTorch
+pip install kinescript-elevenlabs   # ElevenLabs' API, acts on tags; needs ELEVENLABS_API_KEY
 ```
 
-On a newer Python, install it into another one and point `KINE_VOICE_CMD` at it
-([details](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)).
+A plugin can also live in another Python, for Kokoro on Python 3.13 or newer or to keep PyTorch out of
+Kinescript's environment: point `KINE_VOICE_CMD`, or `KINE_VOICE_CMD_<NAME>` for one backend, at it. Each plugin's
+README has the details:
+[Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro),
+[Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox),
+[ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs).
 
 ## Built for agents
 
@@ -300,7 +314,8 @@ Within one process, such as the MCP server, the workers stay warm for two minute
 - Audio is mixing and levelling (trim, gain over time, fades, speed, loop, pan, role levels, ducking, a loudness
   target), with no effects such as EQ, compression of single sounds or reverb.
 - Kinescript does not speak by itself: `kine voice` needs a plugin or a command. The Kokoro plugin times words
-  for English voices only and needs Python 3.10 to 3.12.
+  for English voices only and needs Python 3.10 to 3.12; the Chatterbox plugin speaks and times English only;
+  the ElevenLabs plugin needs an API key and a network connection.
 - Video is decoded and resampled by ffmpeg to the scene frame rate. Jumping backwards or far ahead reopens the
   decoder, which is slower than sequential playback.
 - Text layout is single-script left-to-right with greedy wrapping. No shaping for complex scripts or bidi.
@@ -313,7 +328,8 @@ kinescript/   engine: timing, props, scene, components, render (Skia), space, re
               mesh and models (3D), output (encoding and the audio mix),
               verify (check, look, probe), schema, docs, mcp, cli
 examples/     example scenes and the generator for sample media
-plugins/      kinescript-kokoro, the Kokoro voice for kine voice (installed on its own)
+plugins/      voices for kine voice, each installed on its own: kinescript-kokoro, kinescript-chatterbox,
+              kinescript-elevenlabs
 marketing/    the launch video as a scene and its builder; build_site.py renders the site
 site/         the project site: demo scenes next to the videos rendered from them
 brand/        logo files, generated art and the brand guide

{kinescript-1.2.0 → kinescript-1.3.1}/README.md

@@ -86,9 +86,17 @@ Sounds are a top-level `audio` list, timed like layers and mixed by ffmpeg while
   word: `"at": "vo1:render"` starts a layer as the voice says "render", and `{"type": "captions", "voice": "vo1"}`
   captions it, lighting each word as it is said.
 - **Voice-over from text.** Write what a voice says in `text` and `kine voice` speaks it through a plugin, writing
-  the file and its word times. [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
-  is the first. `[[go]]` in the text names the next word, so a cue to `vo1:go` survives a rewrite of the line.
-  Only a line whose text, speaker or rate changed is spoken again, and rendering never calls a speech engine.
+  the file and its word times: [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
+  (local, fast),
+  [Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox) (local,
+  cloned voices) or [ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs)
+  (cloud, needs an API key), picked per sound with `backend`. `[[go]]` in the text names the next
+  word, so a cue to `vo1:go` survives a rewrite of the line. Only a line whose text, speaker, rate, backend or
+  options changed is spoken again, and rendering never calls a speech engine.
+- **Delivery tags.** `[whispers]`, `[laughs]` or `[sighs]` in the text tell ElevenLabs how to say what follows.
+  Chatterbox takes them too but acts them only some of the time: in test takes it made a sound where `[laughs]`
+  goes in six of nine and whispered in none of seven. Tags are not words, so cues, marks and captions skip them,
+  and Kokoro says the line without them.
 - **A mix without listening.** Give a sound a `role` and it is set to that role's level: voice -18 LUFS, music
   -16, ambience -30, effects peaking at -6 dBFS. Music and ambience dip 14 dB while a voice speaks, an effect's
   leading silence is skipped so it lands on its cue, and the whole mix ends at -16 LUFS with peaks under
@@ -114,14 +122,20 @@ A narrated beat with a click on a marked word, over music that ducks under the v
 
 `kine voice scene.json` speaks it once. The
 [voice demo](https://drknowhow.github.io/kinescript/#sound) on the project site plays the result with sound, next
-to its scene file. The Kokoro plugin needs Python 3.10 to 3.12:
+to its scene file. Install the plugin for the voice you want next to Kinescript:
 
 ```bash
-pip install "kinescript-kokoro @ git+https://github.com/drknowhow/kinescript#subdirectory=plugins/kinescript-kokoro"
+pip install kinescript-kokoro       # local and fast; Python 3.10 to 3.12
+pip install kinescript-chatterbox   # local, copies a voice from a clip, tags hit or miss; installs PyTorch
+pip install kinescript-elevenlabs   # ElevenLabs' API, acts on tags; needs ELEVENLABS_API_KEY
 ```
 
-On a newer Python, install it into another one and point `KINE_VOICE_CMD` at it
-([details](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)).
+A plugin can also live in another Python, for Kokoro on Python 3.13 or newer or to keep PyTorch out of
+Kinescript's environment: point `KINE_VOICE_CMD`, or `KINE_VOICE_CMD_<NAME>` for one backend, at it. Each plugin's
+README has the details:
+[Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro),
+[Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox),
+[ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs).
 
 ## Built for agents
 
@@ -253,7 +267,8 @@ Within one process, such as the MCP server, the workers stay warm for two minute
 - Audio is mixing and levelling (trim, gain over time, fades, speed, loop, pan, role levels, ducking, a loudness
   target), with no effects such as EQ, compression of single sounds or reverb.
 - Kinescript does not speak by itself: `kine voice` needs a plugin or a command. The Kokoro plugin times words
-  for English voices only and needs Python 3.10 to 3.12.
+  for English voices only and needs Python 3.10 to 3.12; the Chatterbox plugin speaks and times English only;
+  the ElevenLabs plugin needs an API key and a network connection.
 - Video is decoded and resampled by ffmpeg to the scene frame rate. Jumping backwards or far ahead reopens the
   decoder, which is slower than sequential playback.
 - Text layout is single-script left-to-right with greedy wrapping. No shaping for complex scripts or bidi.
@@ -266,7 +281,8 @@ kinescript/   engine: timing, props, scene, components, render (Skia), space, re
               mesh and models (3D), output (encoding and the audio mix),
               verify (check, look, probe), schema, docs, mcp, cli
 examples/     example scenes and the generator for sample media
-plugins/      kinescript-kokoro, the Kokoro voice for kine voice (installed on its own)
+plugins/      voices for kine voice, each installed on its own: kinescript-kokoro, kinescript-chatterbox,
+              kinescript-elevenlabs
 marketing/    the launch video as a scene and its builder; build_site.py renders the site
 site/         the project site: demo scenes next to the videos rendered from them
 brand/        logo files, generated art and the brand guide

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/__init__.py

@@ -9,5 +9,5 @@ except ImportError as e:  # on Linux the wheel loads the system's EGL and GL, ev
 from .output import render, still
 from .scene import Scene, SceneError, load
 
-__version__ = "1.2.0"
+__version__ = "1.3.1"
 __all__ = ["Scene", "SceneError", "load", "render", "still", "__version__"]

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/components.py

@@ -182,6 +182,17 @@ def _fill_template(v, params):
 _counter = [0]
 
 
+def _own(layers, pid: str) -> None:
+    """Put the instance id in front of the ids of a scene component's layers, nested ones too, so it can be used
+    more than once."""
+    for k in layers:
+        if not isinstance(k, dict):
+            continue
+        if "id" in k and not str(k["id"]).startswith(f"{pid}-"):
+            k["id"] = f"{pid}-{k['id']}"
+        _own(k.get("children") or [], pid)
+
+
 def expand(raw, user: dict, theme: dict, scene=None):
     if not isinstance(raw, dict):
         return raw
@@ -201,7 +212,7 @@ def expand(raw, user: dict, theme: dict, scene=None):
         _unused(name, raw, names, scene)
         for i, k in enumerate(body):
             k.setdefault("id", f"{params['id']}-{i}")
-            k["id"] = f"{params['id']}-{k['id']}" if not str(k["id"]).startswith(params["id"]) else k["id"]
+        _own(body, params["id"])
         out = {"type": "group", "x": raw.get("x", 0), "y": raw.get("y", 0), "children": body}
     elif name in BUILTIN:
         out = BUILTIN[name](params, theme)

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/docs.py

@@ -76,7 +76,8 @@ KEYS EVERY LAYER HAS
   x, y          position of the anchor. A number (px), "50%" (of the canvas, or of the group's
                 w and h), "50%-120", "10vw", "5vh", or an expression.
   anchor        center (default) | topleft | top | topright | left | right | bottomleft | bottom |
-                bottomright. Video, shader, and image without x default to topleft.
+                bottomright. Video and shader default to topleft, even with x and y set; an image
+                defaults to topleft without x and to center with it.
   opacity       0..1 (default 1)
   scale, scaleX, scaleY   1 = normal size
   rotate        degrees, clockwise
@@ -92,7 +93,8 @@ KEYS EVERY LAYER HAS
   enter, exit   a motion (see ENTER AND EXIT)
   hidden        true skips the layer (and its sound)
   sound         a file played when the layer starts, timed by the layer (see SOUND)
-  allowOffscreen  true stops check warning when the layer leaves the canvas
+  allowOffscreen  true stops check warning when the layer leaves the canvas; on a group or
+                  sequence it covers the layers inside
   note          free text, ignored
 
 LAYER TYPES (defaults in brackets)
@@ -280,7 +282,8 @@ COMPONENTS  {"use": "<name>", "id": "...", "at": ..., "dur": ..., ...parameters}
   scale, rotate, hidden, sound.
   Your own: "components": {"badge": {"params": {"label": "NEW"}, "layers": [
               {"type": "text", "text": "{{label}}", "size": 40}]}}
-  then a layer {"use": "badge", "label": "BETA", "x": 200, "y": 100}.
+  then a layer {"use": "badge", "label": "BETA", "x": 200, "y": 100}. Its layers' ids, nested ones too,
+  get the instance id in front ("b1-title" for "title" in {"use": "badge", "id": "b1"}).
 
 SOUND
   Top-level "audio" lists the sounds. Each one:
@@ -327,8 +330,25 @@ SOUND
   It writes voice/vo1.wav (or the sound's src) and its words file, so cues and captions work as above.
   [[name]] gives the next word that id: "at": "vo1:go" starts as it says Render, whatever the line
   says later. speaker is the backend's voice (Kokoro: af_heart, af_bella, am_adam, bf_emma, bm_george),
-  rate (0.5 to 2, default 1) how fast it speaks. kine check reports a sound with text that is not spoken
-  yet (voice-missing) and one whose text, speaker or rate changed since it was spoken (voice-stale).
+  rate (0.5 to 2, default 1) how fast it speaks. With Kokoro, a speaker of "af_heart,af_bella" blends
+  the two voices, [word](/phonemes/) in text says that word with the IPA phonemes you give (for a name it
+  says wrong), and [word](-1) or [word](+1) lowers or raises its stress; captions and word ids keep the
+  word. kine check reports a sound with text that is not spoken yet (voice-missing) and one whose text,
+  speaker, rate, backend or options changed since it was spoken (voice-stale).
+  A tag in the text says how to speak what follows, for a backend that acts on tags: [whispers], [laughs],
+  [sighs]. Tags are not words: marks and word ids skip them, a backend without tags says the line without
+  them, and kine voice lists what a backend could not do under "ignored". A sound picks its backend with
+  backend (kokoro, chatterbox, elevenlabs, each a plugin) and hands it settings with options:
+    {"id": "vo2", "role": "voice", "text": "[whispers] Nobody knows. [laughs] Yet.", "backend": "elevenlabs",
+     "speaker": "George", "options": {"stability": 0.3}}
+  chatterbox takes [laugh], [chuckle], [sigh], [gasp], [cough], [clear throat], [whispering], [angry],
+  [happy], [sarcastic], [surprised], [crying] and a few more ([laughs] reads as [laugh]), but acts them
+  only some of the time ([laugh] made a sound in 6 of 9 test takes, [whispering] whispered in 0 of 7),
+  so use elevenlabs when a line needs its tags heard; its speaker
+  is a clip of 5 s or more whose voice it copies, and "options": {"model": "standard", "exaggeration": 0.8}
+  trades the tags for stronger emotion. elevenlabs needs an ELEVENLABS_API_KEY and uses the account's
+  quota; its speaker is a voice name or id, its options model (eleven_v4), stability, style and
+  similarity_boost. kokoro has no tags.
   You cannot hear the mix, so read it: kine check measures it and names each sound problem with its
   time range, and kine look --audio draws it under the frames, with each sound's loudness in the JSON.
   Sound is mixed into .mp4, .mov and .webm, and into --from/--to renders. -o mix.wav (or .m4a,
@@ -372,8 +392,10 @@ COMMANDS
   kine new scene.json             write a starter scene
   kine check scene.json           JSON list of problems. error: must fix. warning: should fix:
                                   text off the canvas or inside the outer 5% (safe-area),
-                                  overlapping text, contrast under 3:1, or text shown for less
-                                  than about 0.4 s + 0.3 s per word (read-time). With sound (it is
+                                  overlapping text, contrast under 3:1, text shown for less
+                                  than about 0.4 s + 0.3 s per word (read-time), or a video,
+                                  image or shader more than half off the canvas for 1 s or more
+                                  (media-offscreen). With sound (it is
                                   mixed and measured, see SOUND): a voice cut off (audio-cut), two
                                   voices at once (voice-overlap), music under 10 LU below a voice
                                   (voice-masked), an sfx whose first sound is late (sfx-late), a
@@ -396,11 +418,13 @@ COMMANDS
                                   .wav, .m4a, .mp3 or .flac: the sound alone.
                                   --scale 0.5 halves the size; --from 2 --to 4 renders a part.
   kine voice scene.json           speaks each sound's text into its file and times its words; a sound
-                                  is spoken again only when its text, speaker or rate changed
-                                  (--force: all of them, --only vo1: one). Needs a voice backend,
-                                  the Kokoro plugin (with none installed, kine voice prints the pip
-                                  command), or KINE_VOICE_CMD set to a command that reads JSON
-                                  requests on stdin. Rendering never speaks.
+                                  is spoken again only when its text, speaker, rate, backend or
+                                  options changed (--force: all of them, --only vo1: one). Needs a
+                                  voice backend plugin (with none installed, kine voice prints the
+                                  pip command); --backend picks one for sounds without their own.
+                                  KINE_VOICE_CMD (or KINE_VOICE_CMD_CHATTERBOX for one backend) runs
+                                  a backend in another Python, as JSON requests on stdin.
+                                  Rendering never speaks.
   kine watch scene.json           re-makes the contact sheet on every save
   kine doctor                     checks ffmpeg, the GPU and fonts
   kine mcp                        the same tools for MCP clients

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/mcp.py

@@ -43,7 +43,9 @@ TOOLS = [
                      "required": ["scene"]}},
     {"name": "kine_voice", "description": "Speak each sound's `text` into its file (voice/<id>.wav unless it has a "
      "src) and write its word times, so layers can cue from words (\"vo1:render\") and captions can follow it. Only "
-     "sounds whose text, speaker or rate changed are spoken again, unless force. Run it before check and render.",
+     "sounds whose text, speaker, rate, backend or options changed are spoken again, unless force. A sound's own "
+     "\"backend\" picks its voice plugin; backend here is the one for sounds without. Tags such as [laughs] are "
+     "acted by backends that can and listed under \"ignored\" by the rest. Run it before check and render.",
      "inputSchema": {"type": "object", "properties": {"scene": _SCENE, "base": {"type": "string"},
                                                       "only": {"type": "array", "items": {"type": "string"}},
                                                       "force": {"type": "boolean", "default": False},

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/scene.py

@@ -232,8 +232,9 @@ class Scene:
                 continue
             rec = V.made(near) if a.get("text") else None
             if rec and rec.get("hash") != V.fingerprint(a):
-                self.note("warning", "voice-stale", sid, f"the text, speaker or rate of '{sid}' changed after {src} "
-                          "was spoken, so it says the old text", "run kine voice on this scene to speak it again")
+                self.note("warning", "voice-stale", sid, f"the text, speaker, rate, backend or options of '{sid}' "
+                          f"changed after {src} was spoken, so it says the old text",
+                          "run kine voice on this scene to speak it again")
             given = a.get("words")
             if given is None and not near.exists():
                 continue

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/schema.py

@@ -48,13 +48,14 @@ TWEEN = {"from", "to", "dur", "delay", "ease", "keys"}
 TWEEN_ALIAS = {"duration": "dur", "easing": "ease", "start": "from", "end": "to", "begin": "from"}
 GRADIENT = {"linear", "radial", "stops"}
 AUDIO = {"src", "at", "trim", "gain", "volume", "fadeIn", "fadeOut", "id", "role", "dur", "until", "loop", "pan",
-         "words", "text", "speaker", "rate"}
+         "words", "text", "speaker", "rate", "backend", "options"}
 AUDIO_ALIAS = {"source": "src", "file": "src", "path": "src", "start": "at", "fade_in": "fadeIn",
                "fade_out": "fadeOut", "fadein": "fadeIn", "fadeout": "fadeOut", "duration": "dur", "length": "dur",
                "end": "until", "type": "role", "kind": "role", "balance": "pan", "repeat": "loop",
-               "script": "text", "say": "text", "tts": "text", "narrator": "speaker", "pace": "rate"}
+               "script": "text", "say": "text", "tts": "text", "narrator": "speaker", "pace": "rate",
+               "engine": "backend", "settings": "options", "voiceSettings": "options"}
 # A layer's "sound": cued from the layer, so it has no id, until, loop, words or text of its own.
-SOUND = AUDIO - {"id", "until", "loop", "words", "text", "speaker", "rate"}
+SOUND = AUDIO - {"id", "until", "loop", "words", "text", "speaker", "rate", "backend", "options"}
 MIX = {"loudness", "peak", "levels", "duck"}
 MIX_ALIAS = {"lufs": "loudness", "target": "loudness", "ceiling": "peak", "truePeak": "peak", "ducking": "duck",
              "level": "levels"}
@@ -598,6 +599,12 @@ def _audio_values(a: dict, rep: Report, lid, where: str):
         rep.add("error", "bad-value", lid, f"{where}text is {t!r}; it is the words to speak, as a string")
     if r is not None and (isinstance(r, bool) or not isinstance(r, (int, float)) or not 0.5 <= r <= 2):
         rep.add("error", "bad-value", lid, f"{where}rate is {r!r}; it is how fast the text is spoken, from 0.5 to 2")
+    b, o = a.get("backend"), a.get("options")
+    if b is not None and not (isinstance(b, str) and b.strip()):
+        rep.add("error", "bad-value", lid, f"{where}backend is {b!r}; it names the voice backend, like \"kokoro\"")
+    if o is not None and not isinstance(o, dict):
+        rep.add("error", "bad-value", lid, f"{where}options is {o!r}; it is an object the backend reads, like "
+                "{\"exaggeration\": 0.7}")
     if not a.get("src") and not a.get("text"):
         rep.add("error", "missing-key", lid, f"{where}an audio entry needs 'src' (or 'text' to speak)",
                 "\"src\": \"music.mp3\"")

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/verify.py

@@ -19,6 +19,10 @@ from .scene import Scene, SceneError, load
 # Model triangles one frame may draw before check warns. 100,000 took 71 ms a frame at 1080p on
 # the GPU in the 3D spike (2026-10-29), against about 15 ms at 20,000.
 TRIANGLE_BUDGET = 60_000
+# Layers check warns about when more than half of one stays off the canvas for MEDIA_OFF_FOR seconds; a slide or
+# a transition passes through faster.
+MEDIA = ("video", "image", "shader")
+MEDIA_OFF_FOR = 1.0
 
 
 def _times(scene: Scene, count: int, times=None) -> list[float]:
@@ -149,6 +153,7 @@ def check(src, step: float = 0.25, safe: float = 0.05) -> dict:
     pool.shutdown(wait=False)
     r = Renderer(scene, track=True, scale=1.0)
     found: dict[tuple, dict] = {}
+    off: dict[str, list] = {}  # media layer (or its sequence) id -> (time, box, share off, layer) when mostly off
 
     def add(level, code, lid, T, msg, fix="", until=None):
         key = (code, lid)
@@ -183,8 +188,10 @@ def check(src, step: float = 0.25, safe: float = 0.05) -> dict:
             if b["alpha"] < 0.3:
                 continue
             l, t, rr, bb = b["rect"]
-            layer = scene.by_id.get(lid)
-            allow = layer is not None and layer.raw.get("allowOffscreen")
+            layer = up = scene.by_id.get(lid)
+            allow = False
+            while up is not None and not allow:  # set on a group or sequence, it covers what is inside
+                allow, up = bool(up.raw.get("allowOffscreen")), up.parent
             if b["type"] in ("text", "counter", "captions"):
                 texts.append((lid, b))
                 if b.get("mirrored"):
@@ -200,7 +207,13 @@ def check(src, step: float = 0.25, safe: float = 0.05) -> dict:
                 if px < H / 45 and b["alpha"] > 0.9:
                     add("warning", "tiny-text", lid, T, f"text renders at ~{px:.0f}px, small for {H}p video",
                         f"use at least {math.ceil(H / 45)}px")
-            elif b["type"] not in ("video", "group") and not allow:
+            elif b["type"] in MEDIA:
+                area = (rr - l) * (bb - t)
+                seen = max(0.0, min(rr, W) - max(l, 0.0)) * max(0.0, min(bb, H) - max(t, 0.0))
+                if area > 0 and seen < area / 2 and not allow and layer is not None:
+                    seq = layer.parent if layer.parent is not None and layer.parent.raw.get("span") else None
+                    off.setdefault(seq.id if seq else lid, []).append((T, b["rect"], 1 - seen / area, layer))
+            elif b["type"] != "group" and not allow:
                 if rr < 0 or bb < 0 or l > W or t > H:
                     add("info", "offscreen", lid, T, "layer is visible but entirely off canvas", "check x/y")
         for i in range(len(texts)):
@@ -213,6 +226,17 @@ def check(src, step: float = 0.25, safe: float = 0.05) -> dict:
                     if ix * iy / area > 0.15:
                         add("warning", "text-overlap", f"{texts[i][0]}+{texts[j][0]}", T,
                             "two text layers overlap", "offset one of them or separate them in time")
+    for lid, seen in off.items():
+        if len({s[0] for s in seen}) * step < MEDIA_OFF_FOR:  # frames, once each where a sequence's clips overlap
+            continue
+        T, rect, share, layer = seen[0]
+        ty = layer.type
+        fix = ("a video or shader is placed by its top-left corner unless anchor says otherwise: set \"anchor\": "
+               "\"center\" to centre it on x and y, or move it inward"
+               if ty in ("video", "shader") and "anchor" not in layer.raw else
+               "move it inward, or set allowOffscreen if it is meant to be cut off")
+        add("warning", "media-offscreen", lid, T, f"{ty} box {rect} is {share:.0%} off the {W}x{H} canvas", fix,
+            until=seen[-1][0])
     # Contrast: render the frame without the text layer and its successors, compare luminance under its box.
     for layer in scene.all_layers():
         if layer.type not in ("text", "counter"):

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript/voice.py

@@ -5,30 +5,41 @@ A sound with text is spoken by a backend:
     {"id": "vo1", "role": "voice", "text": "Write a scene. [[check]]Check it.", "speaker": "af_heart"}
 
 writes voice/vo1.wav (or its src) and voice/vo1.words.json. [[name]] gives the next word that id, so a cue can
-say "vo1:check" whatever the word is. A sound is spoken again only when its text, speaker or rate changed since
-its file was made (or with force); `kine check` warns when they changed and the file was not remade.
+say "vo1:check" whatever the word is. A tag such as [laughs] or [whispers] tells a backend that acts on tags how
+to say what follows; it is not a word, and other backends never see it. A sound is spoken again only when its text,
+speaker, rate, backend or options changed since its file was made (or with force); `kine check` warns when they
+changed and the file was not remade.
 
 A backend is a function registered under the entry-point group "kinescript.voice" (kinescript-kokoro is one) that
 takes {"text", "speaker", "rate", "out"}, writes a WAV to out, and returns {"words": [{"word", "start", "end"}],
-"speaker", "version", "seconds"} (seconds, the file's length, is optional). KINE_VOICE_CMD names a command instead, for a backend in another Python: it reads a JSON
-list of those requests on stdin and prints the list of results. Scene text never reaches a shell.
+"speaker", "version", "seconds"} (seconds, the file's length, is optional). The request also carries "tagged" (the
+text with its tags), "options" (the sound's options, as given) and "base" (the scene's folder, for files the options
+or speaker name). A backend that acts on tags returns "tags": true, anything it could not do in "ignored", and
+words it could not time in "note".
+A sound picks its backend with "backend"; KINE_VOICE_CMD_<NAME> runs that backend in another Python, and
+KINE_VOICE_CMD runs the default one (or the backend whose module it runs, python -m kinescript_<name>). A command
+reads a JSON list of those requests on stdin and prints the list of results. Scene text never reaches a shell.
 """
 from __future__ import annotations
 
+import contextlib
 import hashlib
 import json
 import os
 import re
 import shlex
 import subprocess
+import sys
 import time
 from pathlib import Path
 
 from . import words as W
 
 GROUP = "kinescript.voice"
-KOKORO = "kinescript-kokoro @ git+https://github.com/drknowhow/kinescript#subdirectory=plugins/kinescript-kokoro"
+KOKORO = "kinescript-kokoro"
 MARK = re.compile(r"\[\[\s*(\w+)\s*\]\]")
+# [laughs], [clears throat]: in single brackets, and neither a mark ([[name]]) nor a pronunciation ([word](/ph/)).
+TAG = re.compile(r"(?<!\[)\[(?!\[)[^\[\]()]+\](?![\](])")
 
 
 def default_src(sid: str) -> str:
@@ -36,17 +47,29 @@ def default_src(sid: str) -> str:
 
 
 def fingerprint(a: dict) -> str:
-    """What the file was made from: text, speaker and rate. A change in any means it must be spoken again."""
+    """What the file was made from: text, speaker, rate, and the backend and options when the sound names them.
+    A change in any means it must be spoken again."""
     spec = {"text": a.get("text"), "speaker": a.get("speaker"), "rate": a.get("rate", 1)}
+    spec.update({k: a[k] for k in ("backend", "options") if a.get(k)})  # absent: the same hash as before they existed
     return hashlib.sha256(json.dumps(spec, sort_keys=True).encode()).hexdigest()[:16]
 
 
+def tags(text: str) -> list[str]:
+    return TAG.findall(text)
+
+
+def untag(text: str) -> str:
+    """The text without its tags: what a backend that does not act on tags says, and the words marks count."""
+    return re.sub(r"\s+", " ", TAG.sub("", text)).strip()
+
+
 def split_marks(text: str) -> tuple[str, dict[int, str]]:
-    """'Go [[x]]now' -> ('Go now', {1: 'x'}): the text to speak, and the word index each mark names."""
+    """'Go [[x]]now' -> ('Go now', {1: 'x'}): the text to speak, and the word index each mark names. Tags stay in
+    the text and are not counted as words."""
     out, marks = [], {}
     for k, part in enumerate(MARK.split(text)):
         if k % 2:
-            n = len(" ".join(out).split())
+            n = len(untag(" ".join(out)).split())
             if n in marks:
                 raise ValueError(f"two marks, [[{marks[n]}]] and [[{part}]], name the same word")
             marks[n] = part
@@ -124,39 +147,61 @@ def backends() -> dict:
     return {e.name: e for e in entry_points(group=GROUP)}
 
 
-def _command() -> list[str] | None:
-    cmd = os.environ.get("KINE_VOICE_CMD", "").strip()
-    if not cmd:
-        return None
+def _argv(cmd: str) -> list[str]:
     if cmd.startswith("["):
         return [str(x) for x in json.loads(cmd)]
     return [t.strip('"') for t in shlex.split(cmd, posix=os.name != "nt")]
 
 
+def _env_name(name: str) -> str:
+    return "KINE_VOICE_CMD_" + re.sub(r"\W", "_", name).upper()
+
+
+def _command(name: str | None = None) -> tuple[list[str], str] | None:
+    """The command that runs backend `name` in another Python, and the variable it came from: KINE_VOICE_CMD_<NAME>,
+    else KINE_VOICE_CMD when no backend is named or it runs that backend's module (-m kinescript_<name>)."""
+    own = os.environ.get(_env_name(name), "").strip() if name else ""
+    if own:
+        return _argv(own), _env_name(name)
+    cmd = os.environ.get("KINE_VOICE_CMD", "").strip()
+    if cmd and (not name or f"kinescript_{name}" in _argv(cmd)):
+        return _argv(cmd), "KINE_VOICE_CMD"
+    return None
+
+
 def _speaker(name: str | None):
     """The function that speaks a list of requests, and its name."""
-    cmd = _command()
-    if cmd and not name:
+    found = backends()
+    cmd = _command(name)
+    if cmd and (not name or name not in found or cmd[1] != "KINE_VOICE_CMD"):
+        argv, var = cmd
+
         def run(reqs):
-            res = subprocess.run(cmd, input=json.dumps(reqs), capture_output=True, text=True, encoding="utf-8")
+            res = subprocess.run(argv, input=json.dumps(reqs), capture_output=True, text=True, encoding="utf-8")
             if res.returncode:
-                raise RuntimeError(f"KINE_VOICE_CMD failed ({res.returncode}): {res.stderr.strip()[-600:]}")
+                raise RuntimeError(f"{var} failed ({res.returncode}): {res.stderr.strip()[-600:]}")
             return json.loads(res.stdout)
-        return run, Path(cmd[-1]).name
-    found = backends()
+        return run, name or Path(argv[-1]).name
     if name:
         if name not in found:
-            raise ValueError(f"no voice backend '{name}'; installed: {', '.join(found) or 'none'}")
+            raise ValueError(f"no voice backend '{name}': pip install kinescript-{name}, or set {_env_name(name)} to a "
+                             f"command that runs it in another Python; installed: {', '.join(found) or 'none'}")
         key = name
     elif len(found) == 1:
         key = next(iter(found))
     elif found:
         raise ValueError(f"several voice backends are installed ({', '.join(found)}); pick one with --backend")
     else:
-        raise ValueError(f"no voice backend is installed: pip install \"{KOKORO}\" (Python 3.10 to 3.12), "
+        raise ValueError(f"no voice backend is installed: pip install {KOKORO} (Python 3.10 to 3.12), "
                          "or set KINE_VOICE_CMD to a command that speaks")
-    fn = found[key].load()
-    return (lambda reqs: [fn(r) for r in reqs]), key
+    # What a backend prints goes to stderr: kine voice answers in JSON on stdout, and kine mcp speaks JSON-RPC there.
+    with contextlib.redirect_stdout(sys.stderr):
+        fn = found[key].load()
+
+    def run(reqs):
+        with contextlib.redirect_stdout(sys.stderr):
+            return [fn(r) for r in reqs]
+    return run, key
 
 
 def generate(scene, only=None, force: bool = False, backend: str | None = None, base=None) -> dict:
@@ -181,24 +226,36 @@ def generate(scene, only=None, force: bool = False, backend: str | None = None,
             continue
         src.parent.mkdir(parents=True, exist_ok=True)
         todo.append((sid, a, src, words_file, text, marks))
-    if todo:
-        speak, name = _speaker(backend)
+    groups: dict = {}  # a sound's own backend, else the one asked for, else the default
+    for job in todo:
+        groups.setdefault(job[1].get("backend") or backend, []).append(job)
+    for want, jobs in groups.items():
+        speak, name = _speaker(want)
         t0 = time.perf_counter()
-        results = speak([{"text": text, "speaker": a.get("speaker"), "rate": a.get("rate", 1), "out": str(src)}
-                         for _, a, src, _, text, _ in todo])
-        took = (time.perf_counter() - t0) / len(todo)
-        for (sid, a, src, words_file, text, marks), res in zip(todo, results):
+        results = speak([{"text": untag(text), "tagged": text, "speaker": a.get("speaker"), "rate": a.get("rate", 1),
+                          "options": a.get("options") or {}, "out": str(src), "base": str(Path(base).resolve())}
+                         for _, a, src, _, text, _ in jobs])
+        took = (time.perf_counter() - t0) / len(jobs)
+        for (sid, a, src, words_file, text, marks), res in zip(jobs, results):
             if not src.exists():
                 raise RuntimeError(f"the voice backend {name} did not write {src} for '{sid}'")
             timed = [{"word": w["word"], "start": w["start"], "end": w["end"]} for w in W.read(res.get("words") or [])]
-            said = place_marks(text, marks, timed)
+            said = place_marks(untag(text), marks, timed)
             W.read(said)  # the ids are valid and unique before the file is written
             rec = {"hash": fingerprint(a), "backend": name, "version": res.get("version"),
                    "speaker": res.get("speaker") or a.get("speaker")}
             words_file.write_text(json.dumps({"words": said, "made": rec}, indent=1, ensure_ascii=False),
                                   encoding="utf-8")
             secs = res.get("seconds") or (said[-1]["end"] if said else 0.0)
-            out.append({"id": sid, "src": str(a.get("src") or default_src(sid)), "status": "made", "words": len(said),
-                        "seconds": round(float(secs), 2), "took": round(took, 2)})
+            item = {"id": sid, "src": str(a.get("src") or default_src(sid)), "status": "made", "words": len(said),
+                    "seconds": round(float(secs), 2), "took": round(took, 2)}
+            ignored = list(res.get("ignored") or []) + ([] if res.get("tags") else tags(text))
+            if ignored:
+                item["ignored"] = ignored  # said without these: the backend cannot do them
+            if res.get("note"):
+                item["note"] = str(res["note"])
+            elif not said and untag(text).strip():
+                item["note"] = f"{name} gave no word times, so word cues and voice captions of '{sid}' do not work"
+            out.append(item)
     order = {sid: k for k, (sid, _) in enumerate(entries)}
     return {"ok": True, "voices": sorted(out, key=lambda v: order[v["id"]])}

{kinescript-1.2.0 → kinescript-1.3.1}/kinescript.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kinescript
-Version: 1.2.0
+Version: 1.3.1
 Summary: Render motion graphics, video edits and voice-over from JSON scenes on the GPU, with checks an agent can read.
 Author: Dr. Know How
 License-Expression: MIT
@@ -133,9 +133,17 @@ Sounds are a top-level `audio` list, timed like layers and mixed by ffmpeg while
   word: `"at": "vo1:render"` starts a layer as the voice says "render", and `{"type": "captions", "voice": "vo1"}`
   captions it, lighting each word as it is said.
 - **Voice-over from text.** Write what a voice says in `text` and `kine voice` speaks it through a plugin, writing
-  the file and its word times. [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
-  is the first. `[[go]]` in the text names the next word, so a cue to `vo1:go` survives a rewrite of the line.
-  Only a line whose text, speaker or rate changed is spoken again, and rendering never calls a speech engine.
+  the file and its word times: [Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)
+  (local, fast),
+  [Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox) (local,
+  cloned voices) or [ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs)
+  (cloud, needs an API key), picked per sound with `backend`. `[[go]]` in the text names the next
+  word, so a cue to `vo1:go` survives a rewrite of the line. Only a line whose text, speaker, rate, backend or
+  options changed is spoken again, and rendering never calls a speech engine.
+- **Delivery tags.** `[whispers]`, `[laughs]` or `[sighs]` in the text tell ElevenLabs how to say what follows.
+  Chatterbox takes them too but acts them only some of the time: in test takes it made a sound where `[laughs]`
+  goes in six of nine and whispered in none of seven. Tags are not words, so cues, marks and captions skip them,
+  and Kokoro says the line without them.
 - **A mix without listening.** Give a sound a `role` and it is set to that role's level: voice -18 LUFS, music
   -16, ambience -30, effects peaking at -6 dBFS. Music and ambience dip 14 dB while a voice speaks, an effect's
   leading silence is skipped so it lands on its cue, and the whole mix ends at -16 LUFS with peaks under
@@ -161,14 +169,20 @@ A narrated beat with a click on a marked word, over music that ducks under the v
 
 `kine voice scene.json` speaks it once. The
 [voice demo](https://drknowhow.github.io/kinescript/#sound) on the project site plays the result with sound, next
-to its scene file. The Kokoro plugin needs Python 3.10 to 3.12:
+to its scene file. Install the plugin for the voice you want next to Kinescript:
 
 ```bash
-pip install "kinescript-kokoro @ git+https://github.com/drknowhow/kinescript#subdirectory=plugins/kinescript-kokoro"
+pip install kinescript-kokoro       # local and fast; Python 3.10 to 3.12
+pip install kinescript-chatterbox   # local, copies a voice from a clip, tags hit or miss; installs PyTorch
+pip install kinescript-elevenlabs   # ElevenLabs' API, acts on tags; needs ELEVENLABS_API_KEY
 ```
 
-On a newer Python, install it into another one and point `KINE_VOICE_CMD` at it
-([details](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro)).
+A plugin can also live in another Python, for Kokoro on Python 3.13 or newer or to keep PyTorch out of
+Kinescript's environment: point `KINE_VOICE_CMD`, or `KINE_VOICE_CMD_<NAME>` for one backend, at it. Each plugin's
+README has the details:
+[Kokoro](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-kokoro),
+[Chatterbox](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-chatterbox),
+[ElevenLabs](https://github.com/drknowhow/kinescript/tree/main/plugins/kinescript-elevenlabs).
 
 ## Built for agents
 
@@ -300,7 +314,8 @@ Within one process, such as the MCP server, the workers stay warm for two minute
 - Audio is mixing and levelling (trim, gain over time, fades, speed, loop, pan, role levels, ducking, a loudness
   target), with no effects such as EQ, compression of single sounds or reverb.
 - Kinescript does not speak by itself: `kine voice` needs a plugin or a command. The Kokoro plugin times words
-  for English voices only and needs Python 3.10 to 3.12.
+  for English voices only and needs Python 3.10 to 3.12; the Chatterbox plugin speaks and times English only;
+  the ElevenLabs plugin needs an API key and a network connection.
 - Video is decoded and resampled by ffmpeg to the scene frame rate. Jumping backwards or far ahead reopens the
   decoder, which is slower than sequential playback.
 - Text layout is single-script left-to-right with greedy wrapping. No shaping for complex scripts or bidi.
@@ -313,7 +328,8 @@ kinescript/   engine: timing, props, scene, components, render (Skia), space, re
               mesh and models (3D), output (encoding and the audio mix),
               verify (check, look, probe), schema, docs, mcp, cli
 examples/     example scenes and the generator for sample media
-plugins/      kinescript-kokoro, the Kokoro voice for kine voice (installed on its own)
+plugins/      voices for kine voice, each installed on its own: kinescript-kokoro, kinescript-chatterbox,
+              kinescript-elevenlabs
 marketing/    the launch video as a scene and its builder; build_site.py renders the site
 site/         the project site: demo scenes next to the videos rendered from them
 brand/        logo files, generated art and the brand guide

{kinescript-1.2.0 → kinescript-1.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "kinescript"
-version = "1.2.0"
+version = "1.3.1"
 description = "Render motion graphics, video edits and voice-over from JSON scenes on the GPU, with checks an agent can read."
 readme = "README.md"
 license = "MIT"

{kinescript-1.2.0 → kinescript-1.3.1}/tests/test_core.py

@@ -326,6 +326,36 @@ def test_check_flags_offscreen_text_and_overlap():
     assert "text-offscreen" in codes and "text-overlap" in codes and not res["ok"]
 
 
+def test_check_flags_media_mostly_off_canvas(clip):
+    # A video sits by its top-left corner, so x and y at the middle without an anchor push it off to the bottom right.
+    placed = {"id": "v", "type": "video", "src": clip, "x": "50%", "y": "50%", "w": 320, "h": 180, "dur": 2}
+    res = check(scene([placed], duration=2))
+    hit = [i for i in res["issues"] if i["code"] == "media-offscreen"]
+    assert len(hit) == 1 and hit[0]["layer"] == "v" and hit[0]["level"] == "warning" and "anchor" in hit[0]["fix"]
+    assert "75% off" in hit[0]["message"] and hit[0]["to"] - hit[0]["from"] >= 1.5
+    for fixed in ({"anchor": "center"}, {"allowOffscreen": True}):
+        res = check(scene([dict(placed, **fixed)], duration=2))
+        assert not [i for i in res["issues"] if i["code"] == "media-offscreen"], fixed
+    # A sequence's clips get its x, y and anchor, so they are reported once, as the sequence; its allowOffscreen
+    # covers them.
+    seq = {"id": "cut", "type": "sequence", "x": "50%", "y": "50%", "w": 320, "h": 180,
+           "clips": [{"src": clip, "trim": [0, 1.5]}, {"src": clip, "trim": [1.5, 3]}]}
+    res = check(scene([seq], duration=3))
+    hit = [i for i in res["issues"] if i["code"] == "media-offscreen"]
+    assert [i["layer"] for i in hit] == ["cut"] and hit[0]["to"] > 2.5
+    res = check(scene([dict(seq, allowOffscreen=True)], duration=3))
+    assert not [i for i in res["issues"] if i["code"] == "media-offscreen"]
+
+
+def test_check_lets_media_pass_through_the_edge():
+    # Sliding in from off the canvas is mostly off for under a second, which is not a misplaced layer.
+    shader = {"id": "s", "type": "shader", "preset": "score", "w": 320, "h": 180, "dur": 2}
+    res = check(scene([dict(shader, x=[[0, -320], [0.5, 0, "outCubic"]])], duration=2))
+    assert not [i for i in res["issues"] if i["code"] == "media-offscreen"]
+    res = check(scene([dict(shader, x=-240)], duration=2))  # left there, the same layer is flagged
+    assert [i["layer"] for i in res["issues"] if i["code"] == "media-offscreen"] == ["s"]
+
+
 def test_check_flags_low_contrast():
     res = check(scene([{"id": "a", "type": "text", "text": "Grey", "x": 160, "y": 90, "size": 30, "fill": "#20232a"}],
                       bg="#1b1d22", duration=2))
@@ -359,6 +389,16 @@ def test_theme_names_work_inside_a_scenes_own_components():
     assert [(s.by_id[i].raw["fill"], s.by_id[i].raw["stroke"]) for i in ("t1-w", "t2-w")] == [("#9299a8", "#123456")] * 2
 
 
+def test_a_component_holding_a_group_can_be_used_twice():
+    spec = scene([{"use": "card", "id": "a", "label": "A"}, {"use": "card", "id": "ab", "label": "B"}])
+    spec["components"] = {"card": {"params": {"label": ""}, "layers": [
+        {"id": "box", "type": "group", "children": [{"id": "abc", "type": "text", "text": "{{label}}"},
+                                                    {"type": "group", "children": [{"id": "dot", "type": "circle", "r": 4}]}]}]}}
+    s = k.Scene(spec)
+    assert {"a-box", "a-abc", "a-dot", "ab-box", "ab-abc", "ab-dot"} <= set(s.by_id)
+    assert s.by_id["ab-abc"].raw["text"] == "B"
+
+
 def test_sequence_crossfades_and_video_renders(clip):
     spec = scene([{"type": "sequence", "id": "cut", "clips": [{"src": clip, "trim": [0, 1.5]}, {"src": clip, "trim": [1, 3]}],
                    "transition": {"type": "fade", "dur": 0.5}}])
@@ -886,6 +926,196 @@ def test_marks_find_their_word_when_the_voice_reads_it_differently():
         split_marks("[[a]]x [[a]]y")
 
 
+def test_tags_are_not_words_and_reach_only_backends_that_act_on_them(monkeypatch, tmp_path):
+    from kinescript import voice
+    text, marks = voice.split_marks("Well [sighs] okay, [[go]]go [laughs]now.")
+    assert (text, marks) == ("Well [sighs] okay, go [laughs]now.", {2: "go"})
+    assert voice.untag(text) == "Well okay, go now."
+    # marks and pronunciations are not tags
+    assert voice.tags("Meet [Kinescript](/kˈɪn/) [[a]]now [clears throat] ok") == ["[clears throat]"]
+    got = []
+
+    def entry(acts):
+        def speak(req):
+            got.append((req["text"], req["tagged"]))
+            return {**_fake_speak(req), "tags": acts}
+        return type("E", (), {"load": lambda self: speak})()
+    monkeypatch.delenv("KINE_VOICE_CMD", raising=False)
+    monkeypatch.setattr(voice, "backends", lambda: {"plain": entry(False), "acts": entry(True)})
+    spec = scene([{"type": "rect"}], duration="auto")
+    spec["audio"] = [{"id": "a", "text": "Well [sighs] okay. [[go]]Go.", "backend": "plain"},
+                     {"id": "b", "text": "Well [sighs] okay. [[go]]Go.", "backend": "acts"}]
+    a, b = voice.generate(spec, base=tmp_path)["voices"]
+    assert got == [("Well okay. Go.", "Well [sighs] okay. Go.")] * 2
+    assert a["ignored"] == ["[sighs]"] and "ignored" not in b  # said without it, and said so
+    words = json.loads((tmp_path / "voice" / "b.words.json").read_text())["words"]
+    assert [w["word"] for w in words] == ["Well", "okay.", "Go."] and words[2]["id"] == "go"
+
+
+def test_what_a_backend_prints_stays_off_stdout(monkeypatch, tmp_path, capsys):
+    # kine voice answers in JSON on stdout and kine mcp speaks JSON-RPC there; Chatterbox prints as it loads
+    from kinescript import voice
+
+    def speak(req):
+        print("loaded PerthNet")
+        return _fake_speak(req)
+    monkeypatch.delenv("KINE_VOICE_CMD", raising=False)
+    monkeypatch.setattr(voice, "backends", lambda: {"loud": type("E", (), {"load": lambda self: speak})()})
+    spec = scene([{"type": "rect"}], duration="auto")
+    spec["audio"] = [{"id": "a", "text": "hello"}]
+    assert voice.generate(spec, base=tmp_path)["voices"][0]["status"] == "made"
+    out, err = capsys.readouterr()
+    assert out == "" and "loaded PerthNet" in err
+
+
+def test_each_sound_picks_its_backend_and_options(monkeypatch, tmp_path):
+    import hashlib
+    from kinescript import voice
+    seen = []
+
+    def entry(name):
+        def speak(req):
+            seen.append((name, req["options"], req["base"]))
+            return {**_fake_speak(req), **({"note": "half untimed"} if name == "two" else {})}
+        return type("E", (), {"load": lambda self: speak})()
+    monkeypatch.delenv("KINE_VOICE_CMD", raising=False)
+    monkeypatch.setattr(voice, "backends", lambda: {"one": entry("one"), "two": entry("two")})
+    spec = scene([{"type": "rect"}], duration="auto")
+    spec["audio"] = [{"id": "a", "text": "first"},
+                     {"id": "b", "text": "second", "engine": "two", "settings": {"seed": 3}}]  # aliases read
+    with pytest.raises(ValueError, match="several voice backends are installed"):
+        voice.generate(spec, base=tmp_path)
+    got = voice.generate(spec, base=tmp_path, backend="one")
+    assert [v.get("note") for v in got["voices"]] == [None, "half untimed"]  # the backend's own note
+    assert seen == [("one", {}, str(tmp_path.resolve())), ("two", {"seed": 3}, str(tmp_path.resolve()))]
+    assert json.loads((tmp_path / "voice" / "b.words.json").read_text())["made"]["backend"] == "two"
+    # options are part of what the file was made from; a sound without backend or options keeps its old hash
+    f = tmp_path / "s.json"
+    spec["audio"] = [{"id": "b", "text": "second", "backend": "two", "options": {"seed": 4}}]
+    f.write_text(json.dumps(spec))
+    assert [i["code"] for i in k.load(str(f)).issues] == ["voice-stale"]
+    old = {"text": "x", "speaker": None, "rate": 1}
+    assert voice.fingerprint({"text": "x"}) == hashlib.sha256(json.dumps(old, sort_keys=True).encode()).hexdigest()[:16]
+    with pytest.raises(ValueError, match="no voice backend 'three': pip install kinescript-three, or set "
+                                         "KINE_VOICE_CMD_THREE"):
+        voice.generate({**spec, "audio": [{"id": "c", "text": "x", "backend": "three"}]}, base=tmp_path)
+    # a backend in another Python, by name: its own variable, or KINE_VOICE_CMD when it runs that module
+    tool = tmp_path / "speak.py"
+    tool.write_text("import json, sys\nsys.path.insert(0, %r)\nfrom test_core import _fake_speak\n"
+                    "print(json.dumps([_fake_speak(r) for r in json.load(sys.stdin)]))\n" % str(Path(__file__).parent))
+    monkeypatch.setenv("KINE_VOICE_CMD_THREE", json.dumps([sys.executable, str(tool)]))
+    got = voice.generate({**spec, "audio": [{"id": "c", "text": "x y", "backend": "three"}]}, base=tmp_path)
+    assert got["voices"][0]["words"] == 2
+    assert voice._command("kokoro") is None
+    monkeypatch.setenv("KINE_VOICE_CMD", "py -m kinescript_kokoro")
+    assert voice._command("kokoro") == (["py", "-m", "kinescript_kokoro"], "KINE_VOICE_CMD")
+    assert voice._command("other") is None and voice._command(None)[1] == "KINE_VOICE_CMD"
+
+
+def _plugin(monkeypatch, name):
+    monkeypatch.syspath_prepend(str(Path(__file__).parents[1] / "plugins" / f"kinescript-{name}"))
+
+
+def test_forced_alignment_places_words_around_a_laugh_and_a_number(monkeypatch):
+    _plugin(monkeypatch, "chatterbox")
+    from kinescript_chatterbox.align import ctc_path, times, units
+    vocab = {"<pad>": 0, "|": 1, "A": 2, "B": 3, "C": 4, "D": 5}
+    words, seq = units("AB [laugh] C 12.", vocab)
+    assert words == ["AB", "C", "12."] and [s[0] for s in seq] == [2, 3, -2, 4, 1, -1]
+
+    def frames(said):  # what each 20 ms frame sounds like most: a vocabulary id, 0 for nothing
+        lp = np.full((len(said), len(vocab)), np.log(0.02))
+        lp[np.arange(len(said)), said] = np.log(0.9)
+        return lp
+    # silence, A A B, a laugh (heard as D D D), silence, C C, a word gap, "twelve" (D D D), silence
+    said = [0, 0, 2, 2, 3, 5, 5, 5, 0, 4, 4, 1, 5, 5, 5, 0]
+    got = times(ctc_path(frames(said), seq, 0), seq, words, offset=1)
+    assert got == [{"word": "AB", "start": 1.04, "end": 1.1}, {"word": "C", "start": 1.18, "end": 1.22},
+                   {"word": "12.", "start": 1.24, "end": 1.3}]
+    # no laugh at all: the tag takes no time
+    said = [0, 2, 3, 0, 4, 1, 5, 0]
+    assert [w["start"] for w in times(ctc_path(frames(said), seq, 0), seq, words)] == [0.02, 0.08, 0.12]
+    with pytest.raises(ValueError, match="too short"):
+        ctc_path(frames([2, 3]), seq, 0)
+    # a line of more than 127 states (63 letters and gaps) backtracks without overflowing
+    words, seq = units(" ".join(["AB"] * 30), vocab)
+    said = [0] + [2, 3, 1] * 30
+    assert [w["start"] for w in times(ctc_path(frames(said), seq, 0), seq, words)][-1] == 1.76
+
+
+def test_forced_alignment_reads_numbers_as_they_are_said(monkeypatch):
+    _plugin(monkeypatch, "chatterbox")
+    from kinescript_chatterbox.align import spoken, units
+    said = {w: " ".join(spoken(w)) for w in ["250", "2026", "2005", "1,000", "3.5", "v1.2.0", "$5", "40%", "21st",
+                                             "100th", "Render.", "—"]}
+    assert said == {"250": "TWO HUNDRED FIFTY", "2026": "TWENTY TWENTY SIX", "2005": "TWO THOUSAND FIVE",
+                    "1,000": "ONE THOUSAND", "3.5": "THREE POINT FIVE", "v1.2.0": "V ONE POINT TWO POINT ZERO",
+                    "$5": "FIVE DOLLARS", "40%": "FORTY PERCENT", "21st": "TWENTY FIRST", "100th": "ONE HUNDREDTH",
+                    "Render.": "RENDER", "—": ""}
+    vocab = {"<pad>": 0, "|": 1, **{c: 2 + i for i, c in enumerate("ABCDEFGHIJKLMNOPQRSTUVWXYZ'")}}
+    words, seq = units("took 12 — now", vocab)
+    # each word of a number is its own run of letters, a lone dash says nothing
+    assert words == ["took", "12", "now"]
+    assert "".join("|" if s[0] == 1 else "?" if s[0] < 0 else chr(63 + s[0]) for s in seq) == "TOOK|TWELVE|NOW"
+
+
+def test_chatterbox_reads_tags_written_for_other_backends(monkeypatch):
+    _plugin(monkeypatch, "chatterbox")
+    import kinescript_chatterbox as cb
+    monkeypatch.setattr(cb, "TURBO_TAGS", {"[laugh]", "[sigh]", "[chuckle]", "[clear throat]", "[whispering]"})
+    written = ["[laugh]", "[Laughs]", "[laughing]", "[sighs]", "[chuckles]", "[whispers]", "[clears throat]", "[sings]"]
+    assert [cb.turbo_tag(t) for t in written] == ["[laugh]", "[laugh]", "[laugh]", "[sigh]", "[chuckle]",
+                                                  "[whispering]", "[clear throat]", None]
+    long = "One two three. " * 30
+    assert all(len(c) <= 300 for c in cb.chunks(long)) and " ".join(cb.chunks(long)) == long.strip()
+
+
+def test_elevenlabs_times_words_from_its_character_times(monkeypatch, tmp_path):
+    import base64
+    import wave
+    _plugin(monkeypatch, "elevenlabs")
+    import kinescript_elevenlabs as el
+    text = "Hi [laughs] there, all."
+    chars = list(text)
+    starts = [0.1 * k for k in range(len(chars))]
+    words = el.words_from_chars(chars, starts, [s + 0.08 for s in starts])
+    assert [w["word"] for w in words] == ["Hi", "there,", "all."]
+    assert words[1] == {"word": "there,", "start": 1.2, "end": 1.68}  # first to last letter, not the comma
+    calls = []
+
+    def call(path, key, body=None, ctype="application/json"):
+        calls.append((path, json.loads(body) if ctype == "application/json" and body else body))
+        if path == "/v1/voices":
+            return {"voices": [{"voice_id": "a" * 20, "name": "Aria - calm"}]}
+        if path.startswith("/v1/forced-alignment"):
+            return {"words": [{"text": "Hi", "start": 0.0, "end": 0.2}, {"text": " ", "start": 0.2, "end": 0.3},
+                              {"text": "there,", "start": 0.3, "end": 0.6}]}
+        timing = None if "multilingual" in json.loads(body)["model_id"] else {
+            "characters": chars, "character_start_times_seconds": starts,
+            "character_end_times_seconds": [s + 0.08 for s in starts]}
+        return {"audio_base64": base64.b64encode(b"\0\0" * 24000).decode(), "alignment": timing}
+    monkeypatch.setattr(el, "_call", call)
+    monkeypatch.setenv("ELEVENLABS_API_KEY", "k")
+    out = tmp_path / "v.wav"
+    req = {"text": "Hi there, all.", "tagged": text, "speaker": "Aria", "rate": 1.1, "out": str(out),
+           "options": {"stability": 0.3, "model": "eleven_v4", "loud": 1}}
+    res = el.synth(req)
+    assert calls[1] == (f"/v1/text-to-speech/{'a' * 20}/with-timestamps?output_format=pcm_24000",
+                        {"text": text, "model_id": "eleven_v4", "voice_settings": {"stability": 0.3, "speed": 1.1}})
+    assert res["tags"] and res["ignored"] == ["options.loud"] and res["seconds"] == 1.0 and len(res["words"]) == 3
+    with wave.open(str(out)) as f:
+        assert (f.getframerate(), f.getnframes()) == (24000, 24000)
+    # a model without tags gets the text without them; with no character times, forced alignment times the words
+    calls.clear()
+    res = el.synth({**req, "rate": 2, "speaker": "b" * 20, "options": {"model": "eleven_multilingual_v2"}})
+    assert calls[0][1]["text"] == "Hi there, all." and not res["tags"]
+    assert res["ignored"] == ["rate 2: ElevenLabs speaks at 0.7 to 1.2"] and calls[1][0] == "/v1/forced-alignment"
+    assert [w["word"] for w in res["words"]] == ["Hi", "there,"]
+    monkeypatch.delenv("ELEVENLABS_API_KEY")
+    with pytest.raises(RuntimeError, match="set ELEVENLABS_API_KEY"):
+        el.synth(req)
+
+
 def test_kokoro_tokens_become_the_words_of_the_text(monkeypatch):
     from types import SimpleNamespace as T
     monkeypatch.syspath_prepend(str(Path(__file__).parents[1] / "plugins" / "kinescript-kokoro"))

{kinescript-1.2.0 → kinescript-1.3.1}/tests/test_packaging.py

@@ -13,7 +13,10 @@ from urllib.parse import unquote, urlparse
 import pytest
 
 ROOT = Path(__file__).resolve().parents[1]
-README = ROOT / "README.md"
+# Every package this repository publishes to PyPI: the folder with its pyproject.toml and README.md.
+PACKAGES = {"kinescript": ROOT, **{f"kinescript-{p}": ROOT / "plugins" / f"kinescript-{p}"
+                                   for p in ("kokoro", "chatterbox", "elevenlabs")}}
+each_package = pytest.mark.parametrize("pkg", list(PACKAGES))
 
 # Link targets that work on PyPI. Everything else (docs/hero.gif, ../x, /x, //host/x, a bare filename) is
 # resolved against pypi.org and breaks.
@@ -49,14 +52,15 @@ def relative_targets(text):
     return [t for t in link_targets(text) if not ABSOLUTE.match(t)]
 
 
-def _readme():
-    return README.read_text(encoding="utf-8")
+def _readme(pkg="kinescript"):
+    return (PACKAGES[pkg] / "README.md").read_text(encoding="utf-8")
 
 
 # --- README.md ---------------------------------------------------------------------------------------------
 
-def test_readme_has_no_relative_links_or_images():
-    bad = relative_targets(_readme())
+@each_package
+def test_readme_has_no_relative_links_or_images(pkg):
+    bad = relative_targets(_readme(pkg))
     assert not bad, (
         "README.md is the PyPI long description, so a relative target resolves against pypi.org and breaks, "
         "and PyPI drops any other scheme (data:, javascript:). "
@@ -72,10 +76,11 @@ REPO_FILE = re.compile(
 )
 
 
-def test_readme_points_at_files_that_exist_in_the_repo():
+@each_package
+def test_readme_points_at_files_that_exist_in_the_repo(pkg):
     # The README's images are served from the main branch. PyPI keeps the README of every release forever, so a
     # file that is renamed or deleted breaks the page of every release that links to it.
-    hits = [m for m in map(REPO_FILE.match, link_targets(_readme())) if m]
+    hits = [m for m in map(REPO_FILE.match, link_targets(_readme(pkg))) if m]
     missing = [m.group(0) for m in hits if not (ROOT / unquote(m.group(1))).exists()]
     assert not missing, f"README.md links to repo files that do not exist: {missing}"
 
@@ -102,8 +107,9 @@ def _render(text):
     return rm.render(text)
 
 
-def test_readme_renders_the_way_pypi_renders_it():
-    text = _readme()
+@each_package
+def test_readme_renders_the_way_pypi_renders_it(pkg):
+    text = _readme(pkg)
     out = _render(text)
     assert out, "readme_renderer returned nothing: PyPI would show the plain README text instead of HTML"
     out = unquote(html.unescape(out))  # comrak percent-encodes some characters in URLs
@@ -115,24 +121,26 @@ def test_readme_renders_the_way_pypi_renders_it():
 
 # --- pyproject.toml ----------------------------------------------------------------------------------------
 
-def _project():
+def _project(pkg="kinescript"):
     try:
         import tomllib
     except ModuleNotFoundError:  # Python 3.10
         tomllib = pytest.importorskip("tomli")
-    return tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))["project"]
+    return tomllib.loads((PACKAGES[pkg] / "pyproject.toml").read_text(encoding="utf-8"))["project"]
 
 
-def test_summary_is_one_short_plain_line():
-    summary = _project()["description"]
+@each_package
+def test_summary_is_one_short_plain_line(pkg):
+    summary = _project(pkg)["description"]
     assert 20 <= len(summary) <= 120, f"{len(summary)} characters: {summary!r}"
     assert "\n" not in summary and not re.search(r"[`*\[\]<>]", summary), "the summary is plain text, not Markdown"
     assert not summary.endswith(" "), summary
 
 
-def test_classifiers_are_all_valid():
+@each_package
+def test_classifiers_are_all_valid(pkg):
     trove = pytest.importorskip("trove_classifiers")
-    listed = _project()["classifiers"]
+    listed = _project(pkg)["classifiers"]
     unknown = [c for c in listed if c not in trove.classifiers]
     deprecated = [c for c in listed if c in trove.deprecated_classifiers]
     assert not unknown, f"PyPI rejects the upload for classifiers it does not know: {unknown}"
@@ -142,13 +150,15 @@ def test_classifiers_are_all_valid():
     assert not [c for c in listed if c.startswith("License ::")]
 
 
-def test_keywords_are_unique():
-    words = [k.lower() for k in _project()["keywords"]]
+@each_package
+def test_keywords_are_unique(pkg):
+    words = [k.lower() for k in _project(pkg)["keywords"]]
     assert words and len(set(words)) == len(words)
 
 
-def test_project_links_are_complete_and_verifiable():
-    urls = _project()["urls"]
+@each_package
+def test_project_links_are_complete_and_verifiable(pkg):
+    urls = _project(pkg)["urls"]
     # These labels get an icon on PyPI (see docs.pypi.org/project_metadata, "Link icons").
     for label in ("Homepage", "Documentation", "Source", "Issues", "Changelog"):
         assert label in urls, f"missing project link {label!r}"