@slatesvideo/shared 0.6.4 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/api-url.d.ts +5 -3
- package/dist/api-url.js +5 -3
- package/dist/index.d.ts +1 -0
- package/dist/index.js +1 -0
- package/dist/manual/content.d.ts +2 -0
- package/dist/manual/content.js +3 -0
- package/dist/manual/index.d.ts +5 -0
- package/dist/manual/index.js +20 -0
- package/dist/operations/index.d.ts +21 -10
- package/dist/operations/index.js +145 -41
- package/dist/prompts/agent-doctrine.js +1 -0
- package/dist/prompts/shot-spec.d.ts +21 -8
- package/dist/prompts/shot-spec.js +24 -8
- package/dist/skills/content.js +5 -5
- package/package.json +1 -1
- package/skills/slates-model-selection.md +3 -2
- package/skills/slates-prompting-elevenlabs.md +1 -1
- package/skills/slates-prompting-inworld-tts.md +174 -166
- package/skills/slates-prompting-lip-sync.md +1 -1
- package/skills/slates-prompting-seed-audio.md +1 -1
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@slatesvideo/shared",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.5",
|
|
4
4
|
"description": "Shared operations layer for the Slates MCP server and CLI: auth, cloud/desktop clients, and the single tool surface both consume. Most users want @slatesvideo/mcp-server or @slatesvideo/cli instead.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -105,10 +105,11 @@ Both tools are **Kling-only**. Every entry in them is a real Kling endpoint that
|
|
|
105
105
|
|
|
106
106
|
| Job | Model | Why |
|
|
107
107
|
|---|---|---|
|
|
108
|
-
| **Default — a whole audio scene in one pass**: room tone, ambience beds, crowds, nature, layered dialogue + effects, spoken lines | **Seed Audio 1.0** (`seed-audio`) | One plain sentence in, a complete scene out. The continuity-bed workhorse
|
|
108
|
+
| **Default — a whole audio scene in one pass**: room tone, ambience beds, crowds, nature, layered dialogue + effects, spoken lines inside a scene | **Seed Audio 1.0** (`seed-audio`) | One plain sentence in, a complete scene out. The continuity-bed workhorse; dialogue is performed inside the room, not cast. |
|
|
109
|
+
| **One named voice saying one line** — a character's own voice, a narrator, a clean VO to lip-sync against | **Inworld TTS-2** (`inworld-tts-2`) | The prompt IS the words, spoken verbatim and billed per character. Voice = the character's clip (cloned for the take), a description, or a preset. No room tone — mix it on the timeline. |
|
|
109
110
|
| **One effect that lands on a known frame**, or a seamless loop | **Sound Effects v2** (`eleven-sfx`) | The only surface with an exact duration control and a real loop mode. |
|
|
110
111
|
|
|
111
|
-
**There is no music model
|
|
112
|
+
**There is no music model.** A song is imported (Slates reads audio files and puts them on the timeline), not generated. A line that has to be spoken in a SPECIFIC voice is generated on Inworld TTS-2 and lip-synced against; a line that belongs to a scene is performed by Seed Audio inside it.
|
|
112
113
|
|
|
113
114
|
### Named audio escalation triggers
|
|
114
115
|
|
|
@@ -28,7 +28,7 @@ description: How to prompt ElevenLabs Sound Effects v2 in Slates. Read before ca
|
|
|
28
28
|
- `A heavy oak door slams shut in a stone hallway, brief reverberant tail.` (1.5s)
|
|
29
29
|
- `Steady rain on a tin awning, no thunder, no wind gusts, seamless loop.` (18s)
|
|
30
30
|
|
|
31
|
-
**Hard constraint:** it is billed per second and the duration is never left for the model to pick — that would make the charge non-deterministic. It is NOT a speech surface:
|
|
31
|
+
**Hard constraint:** it is billed per second and the duration is never left for the model to pick — that would make the charge non-deterministic. It is NOT a speech surface: a line in a specific voice is `inworld-tts-2`, and dialogue inside a scene is Seed Audio, which casts and performs the line in the room.
|
|
32
32
|
<!-- @card:end -->
|
|
33
33
|
|
|
34
34
|
<!-- @banned:start -->
|
|
@@ -1,166 +1,174 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: slates-prompting-inworld-tts
|
|
3
|
-
description: How to use Inworld Realtime TTS-2, the VOICE seat. Read before calling slates_generate_audio with model inworld-tts-2. Speech in a SPECIFIC voice, billed per character - the prompt is the words spoken, verbatim
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Inworld Realtime TTS-2 — the voice seat
|
|
7
|
-
|
|
8
|
-
<!-- @card:start -->
|
|
9
|
-
<!-- slates-only -->
|
|
10
|
-
<!-- MACHINE-READ. Everything between the @card markers is extracted by
|
|
11
|
-
src/prompts/craft-cards.ts and returned on every cost estimate for this
|
|
12
|
-
model, so it is the ONE piece of positive craft guidance the agent cannot
|
|
13
|
-
skip. Keep it under 2,400 characters (the build fails above that) and keep
|
|
14
|
-
the rationale and the worked examples in the body below. -->
|
|
15
|
-
<!-- /slates-only -->
|
|
16
|
-
**Card — Inworld TTS-2.** Speech in a SPECIFIC voice. The prompt is the words spoken, verbatim — not a description of them.
|
|
17
|
-
|
|
18
|
-
**IDENTITY, NOT ACOUSTICS — the rule that decides whether cloning works**
|
|
19
|
-
A reference carries WHO is speaking: timbre, pitch, accent, age, vowel shape. It does NOT carry WHERE they are — room tone, distance, phone EQ, reverb and mic character are *acoustics*, and this model reproduces the identity while discarding the room. So:
|
|
20
|
-
1. **A noisy reference does not give a noisy read — it gives a WORSE identity.** Music, a second speaker or heavy reverb corrupt what is being extracted.
|
|
21
|
-
2. **You cannot get "on a payphone" by cloning a payphone recording.** Acoustics come from the MIX, or from `seed-audio` which renders a room.
|
|
22
|
-
|
|
23
|
-
**DIRECTION GOES IN SQUARE BRACKETS. PARENTHESES ARE SPOKEN ALOUD.** `[whispering] I hope nobody notices` is whispered; `(quietly) I hope nobody notices` says the word "quietly" out loud. Verified by ear — the easiest way to ruin a take.
|
|
24
|
-
|
|
25
|
-
- **Plain English works inside them** — it is natural-language steering, not a fixed vocabulary: `[very quiet]`, `[whisper in a hushed style]`, `[very slow]`, `[say excitedly]`. Non-verbals are their own tags: `[laugh]`, `[sigh]`, `[breathe]`, `[clear throat]`.
|
|
26
|
-
- **A tag it does not recognise is still consumed, and still changes the read.** Never spoken, never an error — so a mistyped tag fails SILENTLY and only listening catches it.
|
|
27
|
-
- **Tags persist across sentences** until changed; `[reset]` returns to normal.
|
|
28
|
-
- **Punctuation is the timing.** `Wait. Stop.` differs from `Wait, stop.`
|
|
29
|
-
- **One line, one take.** Split a paragraph so a bad clause costs one re-roll.
|
|
30
|
-
- **Spell numbers and titles aloud:** `twenty twenty-six`, `Doctor Reyes`.
|
|
31
|
-
|
|
32
|
-
**Route elsewhere when:** the scene needs dialogue mixed with effects and room tone in one pass (`seed-audio`), or it is a single non-speech sound (`eleven-sfx`). This surface makes ONE voice saying ONE thing, cleanly.
|
|
33
|
-
|
|
34
|
-
**Hard constraints:** no duration parameter — length falls out of the text.
|
|
35
|
-
<!-- @card:end -->
|
|
36
|
-
|
|
37
|
-
<!-- @banned:start -->
|
|
38
|
-
<!-- slates-only -->
|
|
39
|
-
<!-- MACHINE-READ. Every `backticked` token between the @banned markers is
|
|
40
|
-
extracted by src/prompts/banned-tokens.ts and returned on this model's cost
|
|
41
|
-
estimate, and every submitted prompt is matched against it. Keep entries
|
|
42
|
-
backticked and prose outside the backticks. -->
|
|
43
|
-
<!-- /slates-only -->
|
|
44
|
-
**Never use** — the prompt on this surface is SPOKEN ALOUD, so anything that describes the audio instead of being the audio gets read out as words:
|
|
45
|
-
|
|
46
|
-
- `SFX`, `Ambient noise`, `Background music` as labels — this model speaks; it does not render a scene. Use `seed-audio` for those.
|
|
47
|
-
- shot language: `wide shot`, `slow push in`, `warm tungsten` — video-prompt words, and here they would literally be said aloud
|
|
48
|
-
- `voiceover`, `narrator says`, `he says` as stage directions wrapping the line — write only the words that should come out of the speaker
|
|
49
|
-
<!-- @banned:end -->
|
|
50
|
-
|
|
51
|
-
## Why the prompt is not a prompt
|
|
52
|
-
|
|
53
|
-
On every other surface in Slates the prompt DESCRIBES what you want and the model interprets it. Here the prompt IS the deliverable: each character is spoken aloud and each character is billed. `a gravelly man says he is tired` produces a voice saying the words "a gravelly man says he is tired".
|
|
54
|
-
|
|
55
|
-
That also means the two numbers a user cares about are the same number. The text length sets the price (in 250-character buckets) and sets the length of the audio. There is nothing to choose and nothing to reconcile.
|
|
56
|
-
|
|
57
|
-
## Steering the delivery
|
|
58
|
-
|
|
59
|
-
`VERIFIED BY EAR, 2026-09-05.` Every claim in this section was listened to, not
|
|
60
|
-
inferred — an earlier draft of this skill documented tag forms that had only been
|
|
61
|
-
probed for an HTTP 200, which proves the request was accepted and nothing about
|
|
62
|
-
whether it was obeyed.
|
|
63
|
-
|
|
64
|
-
**Square brackets are consumed. Parentheses are read aloud.** That is the whole
|
|
65
|
-
rule, and getting it wrong is not a subtle degradation — the audience hears a
|
|
66
|
-
narrator say the word "quietly" in the middle of your line.
|
|
67
|
-
|
|
68
|
-
| Written | What comes out |
|
|
69
|
-
|---|---|
|
|
70
|
-
| `[whispering] I really hope nobody notices that.` | whispered, tag not spoken ✅ |
|
|
71
|
-
| `[very quiet] I really hope nobody notices that.` | very quiet, tag not spoken ✅ |
|
|
72
|
-
| `[whisper in a hushed style] …` | hushed, tag not spoken ✅ |
|
|
73
|
-
| `[very slow] …` | slowed right down, tag not spoken ✅ |
|
|
74
|
-
| `[laugh] …` | an actual laugh, then the line ✅ |
|
|
75
|
-
| `(quietly, under his breath) …` | 🚨 **the words "quietly, under his breath" are SPOKEN** |
|
|
76
|
-
|
|
77
|
-
**Plain English works — it is natural-language steering, not a fixed vocabulary.**
|
|
78
|
-
Both the documented phrasings (`[whisper in a hushed style]`) and ordinary adverbs
|
|
79
|
-
(`[whispering]`) were obeyed. Write the direction the way you would say it to an
|
|
80
|
-
actor.
|
|
81
|
-
|
|
82
|
-
The eight dimensions the model steers on, with a working example of each:
|
|
83
|
-
|
|
84
|
-
| Dimension | Example |
|
|
85
|
-
|---|---|
|
|
86
|
-
| Emotion | `[say excitedly]`, `[sound sad]`, `[sound terrified]` |
|
|
87
|
-
| Articulation | `[say with force]`, `[articulate clearly]` |
|
|
88
|
-
| Intonation | `[say with a rising pitch]` |
|
|
89
|
-
| Volume | `[very quiet]`, `[very loud]` |
|
|
90
|
-
| Pitch | `[say in a low tone]` |
|
|
91
|
-
| Range | `[say playfully]`, `[say with no pitch variation]` |
|
|
92
|
-
| Speed | `[very fast]`, `[very slow]` |
|
|
93
|
-
| Vocal style | `[whisper in a hushed style]`, `[give a nasal quality]` |
|
|
94
|
-
|
|
95
|
-
Non-verbals sit inline where they happen: `[laugh]`, `[sigh]`, `[cough]`,
|
|
96
|
-
`[breathe]`, `[yawn]`, `[clear throat]`.
|
|
97
|
-
|
|
98
|
-
###
|
|
99
|
-
|
|
100
|
-
1. 🚨 **A tag it does not recognise is still consumed, and still changes the read.**
|
|
101
|
-
`[zzzqqq]` is not spoken and does not error — it produces a different, arbitrary
|
|
102
|
-
delivery. So a typo in a tag is SILENT: there is no rejection, no warning, and no
|
|
103
|
-
way to catch it except listening to the take. Treat an unexpected performance as
|
|
104
|
-
a possible misspelled tag before you blame the voice.
|
|
105
|
-
2. **Tags persist across sentences.** A `[very slow]` at the top governs everything
|
|
106
|
-
after it until something changes it. Use `[reset]` to go back to normal rather
|
|
107
|
-
than assuming the next sentence starts clean.
|
|
108
|
-
3. **Do not stack opposing directions.** `[whisper in a hushed style]` together with
|
|
109
|
-
`[very loud]` produces unpredictable results — the model is resolving a
|
|
110
|
-
contradiction, and which side wins is not something you can rely on.
|
|
111
|
-
4. **Tags COUNT toward the billed characters**, even though they are never spoken.
|
|
112
|
-
They are part of the text sent to the vendor, so the vendor charges for them and
|
|
113
|
-
so do we — billing what was actually sent is the only honest basis. It rarely
|
|
114
|
-
matters (a 13-character tag inside a 250-character bucket), but a line sitting
|
|
115
|
-
just under a bucket boundary can be pushed into the next one by a long
|
|
116
|
-
direction. Prefer `[very slow]` over `[say this one very slowly please]`.
|
|
117
|
-
|
|
118
|
-
## Identity versus acoustics, at length
|
|
119
|
-
|
|
120
|
-
This is the distinction that decides whether the feature feels good, and it is worth being precise about because the failure is quiet — you get a usable clip that is subtly not the person.
|
|
121
|
-
|
|
122
|
-
**What a reference clip transfers:** vocal timbre, pitch range, accent and regional vowels, apparent age, speech rate tendencies, and the particular rasp or breathiness of the source speaker.
|
|
123
|
-
|
|
124
|
-
**What it does not transfer:** the room, the microphone, the codec, the distance from the mic, any processing on the source, and any other sound present in it.
|
|
125
|
-
|
|
126
|
-
So the ideal reference is boring: one person, close to a microphone, no music, no second speaker, no heavy reverb, five to fifteen seconds, speaking normally rather than performing. A phone voice memo in a quiet room beats a beautifully produced clip with a music bed underneath it.
|
|
127
|
-
|
|
128
|
-
**Two failure modes, both common:**
|
|
129
|
-
|
|
130
|
-
- *"I cloned my podcast intro and it doesn't sound like me."* The intro had music under it. The model averaged the music into the identity. Re-clone from a clean stretch.
|
|
131
|
-
- *"I want the line to sound like it's coming through a car radio."* Clone the clean voice, then EQ and process the returned clip on the timeline. A radio-sounding reference makes a worse voice, not a radio effect.
|
|
132
|
-
|
|
133
|
-
##
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
**A
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
```
|
|
158
|
-
|
|
159
|
-
**
|
|
160
|
-
|
|
161
|
-
```
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
1
|
+
---
|
|
2
|
+
name: slates-prompting-inworld-tts
|
|
3
|
+
description: How to use Inworld Realtime TTS-2, the VOICE seat. Read before calling slates_generate_audio with model inworld-tts-2. Speech in a SPECIFIC voice, billed per character - the prompt is the words spoken, verbatim. Covers the identity-versus-acoustics rule (what a reference clip does and does not carry), how to write a line so it is performed rather than read, when to reach for seed-audio instead, and the voice-consent rule.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Inworld Realtime TTS-2 — the voice seat
|
|
7
|
+
|
|
8
|
+
<!-- @card:start -->
|
|
9
|
+
<!-- slates-only -->
|
|
10
|
+
<!-- MACHINE-READ. Everything between the @card markers is extracted by
|
|
11
|
+
src/prompts/craft-cards.ts and returned on every cost estimate for this
|
|
12
|
+
model, so it is the ONE piece of positive craft guidance the agent cannot
|
|
13
|
+
skip. Keep it under 2,400 characters (the build fails above that) and keep
|
|
14
|
+
the rationale and the worked examples in the body below. -->
|
|
15
|
+
<!-- /slates-only -->
|
|
16
|
+
**Card — Inworld TTS-2.** Speech in a SPECIFIC voice. The prompt is the words spoken, verbatim — not a description of them. Text length determines the bill.
|
|
17
|
+
|
|
18
|
+
**IDENTITY, NOT ACOUSTICS — the rule that decides whether cloning works**
|
|
19
|
+
A reference carries WHO is speaking: timbre, pitch, accent, age, vowel shape. It does NOT carry WHERE they are — room tone, distance, phone EQ, reverb and mic character are *acoustics*, and this model reproduces the identity while discarding the room. So:
|
|
20
|
+
1. **A noisy reference does not give a noisy read — it gives a WORSE identity.** Music, a second speaker or heavy reverb corrupt what is being extracted. Use a clean single-speaker recording.
|
|
21
|
+
2. **You cannot get "on a payphone" by cloning a payphone recording.** Acoustics come from the MIX, or from `seed-audio` which renders a room.
|
|
22
|
+
|
|
23
|
+
**DIRECTION GOES IN SQUARE BRACKETS. PARENTHESES ARE SPOKEN ALOUD.** `[whispering] I hope nobody notices` is whispered; `(quietly) I hope nobody notices` says the word "quietly" out loud. Verified by ear — the easiest way to ruin a take.
|
|
24
|
+
|
|
25
|
+
- **Plain English works inside them** — it is natural-language steering, not a fixed vocabulary: `[very quiet]`, `[whisper in a hushed style]`, `[very slow]`, `[say excitedly]`. Non-verbals are their own tags: `[laugh]`, `[sigh]`, `[breathe]`, `[clear throat]`.
|
|
26
|
+
- **A tag it does not recognise is still consumed, and still changes the read.** Never spoken, never an error — so a mistyped tag fails SILENTLY and only listening catches it.
|
|
27
|
+
- **Tags persist across sentences** until changed; `[reset]` returns to normal.
|
|
28
|
+
- **Punctuation is the timing.** `Wait. Stop.` differs from `Wait, stop.`
|
|
29
|
+
- **One line, one take.** Split a paragraph so a bad clause costs one re-roll.
|
|
30
|
+
- **Spell numbers and titles aloud:** `twenty twenty-six`, `Doctor Reyes`.
|
|
31
|
+
|
|
32
|
+
**Route elsewhere when:** the scene needs dialogue mixed with effects and room tone in one pass (`seed-audio`), or it is a single non-speech sound (`eleven-sfx`). This surface makes ONE voice saying ONE thing, cleanly.
|
|
33
|
+
|
|
34
|
+
**Hard constraints:** no duration parameter — length falls out of the text. Exactly one voice source: a preset `voiceId` from `slates_list_voices`, a clip as `voiceReferenceAssetId` (a character's voice clip to speak AS the character, or any clean clip of one speaker), or `voiceDescription`.
|
|
35
|
+
<!-- @card:end -->
|
|
36
|
+
|
|
37
|
+
<!-- @banned:start -->
|
|
38
|
+
<!-- slates-only -->
|
|
39
|
+
<!-- MACHINE-READ. Every `backticked` token between the @banned markers is
|
|
40
|
+
extracted by src/prompts/banned-tokens.ts and returned on this model's cost
|
|
41
|
+
estimate, and every submitted prompt is matched against it. Keep entries
|
|
42
|
+
backticked and prose outside the backticks. -->
|
|
43
|
+
<!-- /slates-only -->
|
|
44
|
+
**Never use** — the prompt on this surface is SPOKEN ALOUD, so anything that describes the audio instead of being the audio gets read out as words:
|
|
45
|
+
|
|
46
|
+
- `SFX`, `Ambient noise`, `Background music` as labels — this model speaks; it does not render a scene. Use `seed-audio` for those.
|
|
47
|
+
- shot language: `wide shot`, `slow push in`, `warm tungsten` — video-prompt words, and here they would literally be said aloud
|
|
48
|
+
- `voiceover`, `narrator says`, `he says` as stage directions wrapping the line — write only the words that should come out of the speaker
|
|
49
|
+
<!-- @banned:end -->
|
|
50
|
+
|
|
51
|
+
## Why the prompt is not a prompt
|
|
52
|
+
|
|
53
|
+
On every other surface in Slates the prompt DESCRIBES what you want and the model interprets it. Here the prompt IS the deliverable: each character is spoken aloud and each character is billed. `a gravelly man says he is tired` produces a voice saying the words "a gravelly man says he is tired".
|
|
54
|
+
|
|
55
|
+
That also means the two numbers a user cares about are the same number. The text length sets the price (in 250-character buckets) and sets the length of the audio. There is nothing to choose and nothing to reconcile.
|
|
56
|
+
|
|
57
|
+
## Steering the delivery
|
|
58
|
+
|
|
59
|
+
`VERIFIED BY EAR, 2026-09-05.` Every claim in this section was listened to, not
|
|
60
|
+
inferred — an earlier draft of this skill documented tag forms that had only been
|
|
61
|
+
probed for an HTTP 200, which proves the request was accepted and nothing about
|
|
62
|
+
whether it was obeyed.
|
|
63
|
+
|
|
64
|
+
**Square brackets are consumed. Parentheses are read aloud.** That is the whole
|
|
65
|
+
rule, and getting it wrong is not a subtle degradation — the audience hears a
|
|
66
|
+
narrator say the word "quietly" in the middle of your line.
|
|
67
|
+
|
|
68
|
+
| Written | What comes out |
|
|
69
|
+
|---|---|
|
|
70
|
+
| `[whispering] I really hope nobody notices that.` | whispered, tag not spoken ✅ |
|
|
71
|
+
| `[very quiet] I really hope nobody notices that.` | very quiet, tag not spoken ✅ |
|
|
72
|
+
| `[whisper in a hushed style] …` | hushed, tag not spoken ✅ |
|
|
73
|
+
| `[very slow] …` | slowed right down, tag not spoken ✅ |
|
|
74
|
+
| `[laugh] …` | an actual laugh, then the line ✅ |
|
|
75
|
+
| `(quietly, under his breath) …` | 🚨 **the words "quietly, under his breath" are SPOKEN** |
|
|
76
|
+
|
|
77
|
+
**Plain English works — it is natural-language steering, not a fixed vocabulary.**
|
|
78
|
+
Both the documented phrasings (`[whisper in a hushed style]`) and ordinary adverbs
|
|
79
|
+
(`[whispering]`) were obeyed. Write the direction the way you would say it to an
|
|
80
|
+
actor.
|
|
81
|
+
|
|
82
|
+
The eight dimensions the model steers on, with a working example of each:
|
|
83
|
+
|
|
84
|
+
| Dimension | Example |
|
|
85
|
+
|---|---|
|
|
86
|
+
| Emotion | `[say excitedly]`, `[sound sad]`, `[sound terrified]` |
|
|
87
|
+
| Articulation | `[say with force]`, `[articulate clearly]` |
|
|
88
|
+
| Intonation | `[say with a rising pitch]` |
|
|
89
|
+
| Volume | `[very quiet]`, `[very loud]` |
|
|
90
|
+
| Pitch | `[say in a low tone]` |
|
|
91
|
+
| Range | `[say playfully]`, `[say with no pitch variation]` |
|
|
92
|
+
| Speed | `[very fast]`, `[very slow]` |
|
|
93
|
+
| Vocal style | `[whisper in a hushed style]`, `[give a nasal quality]` |
|
|
94
|
+
|
|
95
|
+
Non-verbals sit inline where they happen: `[laugh]`, `[sigh]`, `[cough]`,
|
|
96
|
+
`[breathe]`, `[yawn]`, `[clear throat]`.
|
|
97
|
+
|
|
98
|
+
### Four rules that are not obvious
|
|
99
|
+
|
|
100
|
+
1. 🚨 **A tag it does not recognise is still consumed, and still changes the read.**
|
|
101
|
+
`[zzzqqq]` is not spoken and does not error — it produces a different, arbitrary
|
|
102
|
+
delivery. So a typo in a tag is SILENT: there is no rejection, no warning, and no
|
|
103
|
+
way to catch it except listening to the take. Treat an unexpected performance as
|
|
104
|
+
a possible misspelled tag before you blame the voice.
|
|
105
|
+
2. **Tags persist across sentences.** A `[very slow]` at the top governs everything
|
|
106
|
+
after it until something changes it. Use `[reset]` to go back to normal rather
|
|
107
|
+
than assuming the next sentence starts clean.
|
|
108
|
+
3. **Do not stack opposing directions.** `[whisper in a hushed style]` together with
|
|
109
|
+
`[very loud]` produces unpredictable results — the model is resolving a
|
|
110
|
+
contradiction, and which side wins is not something you can rely on.
|
|
111
|
+
4. **Tags COUNT toward the billed characters**, even though they are never spoken.
|
|
112
|
+
They are part of the text sent to the vendor, so the vendor charges for them and
|
|
113
|
+
so do we — billing what was actually sent is the only honest basis. It rarely
|
|
114
|
+
matters (a 13-character tag inside a 250-character bucket), but a line sitting
|
|
115
|
+
just under a bucket boundary can be pushed into the next one by a long
|
|
116
|
+
direction. Prefer `[very slow]` over `[say this one very slowly please]`.
|
|
117
|
+
|
|
118
|
+
## Identity versus acoustics, at length
|
|
119
|
+
|
|
120
|
+
This is the distinction that decides whether the feature feels good, and it is worth being precise about because the failure is quiet — you get a usable clip that is subtly not the person.
|
|
121
|
+
|
|
122
|
+
**What a reference clip transfers:** vocal timbre, pitch range, accent and regional vowels, apparent age, speech rate tendencies, and the particular rasp or breathiness of the source speaker.
|
|
123
|
+
|
|
124
|
+
**What it does not transfer:** the room, the microphone, the codec, the distance from the mic, any processing on the source, and any other sound present in it.
|
|
125
|
+
|
|
126
|
+
So the ideal reference is boring: one person, close to a microphone, no music, no second speaker, no heavy reverb, five to fifteen seconds, speaking normally rather than performing. A phone voice memo in a quiet room beats a beautifully produced clip with a music bed underneath it.
|
|
127
|
+
|
|
128
|
+
**Two failure modes, both common:**
|
|
129
|
+
|
|
130
|
+
- *"I cloned my podcast intro and it doesn't sound like me."* The intro had music under it. The model averaged the music into the identity. Re-clone from a clean stretch.
|
|
131
|
+
- *"I want the line to sound like it's coming through a car radio."* Clone the clean voice, then EQ and process the returned clip on the timeline. A radio-sounding reference makes a worse voice, not a radio effect.
|
|
132
|
+
|
|
133
|
+
## Getting the voice onto the call
|
|
134
|
+
|
|
135
|
+
Exactly one source per call, and none of them requires a character to exist first:
|
|
136
|
+
|
|
137
|
+
- **A preset:** `slates_list_voices` lists stock voices with gender, age, accent and tags — filter by any of them, or search the descriptions ("gravelly", "narration"). Pass the chosen `voiceId`. Presets clone nothing, so they are the fastest path and avoid the clone-creation rate ceiling.
|
|
138
|
+
- **Speak AS a character:** `voiceReferenceAssetId: <its voiceAssetId>` (the clip on the row `slates_list_characters` returns). The seat clones the clip for that take and discards the vendor voice afterwards, so there is nothing to reconcile — but cloning shares a ceiling of two new voices a minute across every Slates user, so a run of lines in one cloned voice pauses between takes rather than failing. Send each line once; do not re-send one that already came back. Any other clean clip of one speaker works the same way.
|
|
139
|
+
- **A voice with no recording:** `voiceDescription` (7–1000 characters of words). If it will be used again, keep the returned clip on a character with `slates_update_character` (`voiceAssetId`) so later lines clone the same clip instead of designing a new voice each time — a convenience, never a requirement.
|
|
140
|
+
|
|
141
|
+
## Consent
|
|
142
|
+
|
|
143
|
+
Cloning a real person's voice needs that person's explicit, documented permission, scoped to what you are making. Clone from original human recordings only — never from another model's output. This is the same gate the real-face route applies to likeness, and it applies here for the same reason.
|
|
144
|
+
|
|
145
|
+
## Worked examples
|
|
146
|
+
|
|
147
|
+
**A line with a direction**
|
|
148
|
+
|
|
149
|
+
```
|
|
150
|
+
[very quiet] I heard what you said in there. I'm not going to pretend I didn't.
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
**A line that needs its numbers spoken**
|
|
154
|
+
|
|
155
|
+
```
|
|
156
|
+
The vote was three hundred and twelve to eighty-nine. It carried at four minutes past midnight.
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**A paragraph, split into three takes** — so one bad clause costs one re-roll:
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
1. You keep asking me why I stayed.
|
|
163
|
+
2. It wasn't loyalty. It wasn't even fear, not by the end.
|
|
164
|
+
3. [very slow] It was that I couldn't picture the version of me that left.
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**What NOT to send**
|
|
168
|
+
|
|
169
|
+
```
|
|
170
|
+
(gravelly, tired) a tired old man narrates the opening of the film, wide shot, warm tungsten
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Every word of that is spoken aloud — **including the parenthetical**, which is the
|
|
174
|
+
trap: it looks like a stage direction and is treated as dialogue. Describe the voice when you are CHOOSING one (`voiceDescription`, or the desktop's voice picker); the prompt is only ever the words.
|
|
@@ -28,7 +28,7 @@ description: How to set up lip-sync — Kling-only (dedicated lip-sync and avata
|
|
|
28
28
|
- `Soft rim light, warm office, gentle confident smile between sentences.`
|
|
29
29
|
- `Cool blue evening light through a window, focused intent expression.` (Or `.` — an empty prompt is fine when you have nothing to add.)
|
|
30
30
|
|
|
31
|
-
**Hard constraint:** it is Kling-only and always 5 seconds. For a generated PERFORMANCE instead — head movement, gesture, delivery energy, with the dialogue as a native conditioning signal — that is a normal Seedance video generation with the clip attached as a video reference, not a mode of this tool.
|
|
31
|
+
**Hard constraint:** it is Kling-only and always 5 seconds. For a generated PERFORMANCE instead — head movement, gesture, delivery energy, with the dialogue as a native conditioning signal — that is a normal Seedance video generation with the clip attached as a video reference, not a mode of this tool. A real recording, or a cloned/cast voice rendered on `inworld-tts-2`, for production; this tool's built-in TTS is for scratch.
|
|
32
32
|
<!-- @card:end -->
|
|
33
33
|
|
|
34
34
|
<!-- @banned:start -->
|
|
@@ -48,7 +48,7 @@ ByteDance's one-pass audio scene model, carried on fal (`bytedance/seed-audio-1.
|
|
|
48
48
|
## Where it routes
|
|
49
49
|
|
|
50
50
|
- **Scene audio, room tone, ambience beds, crowd/nature soundscapes** — anything where several sounds share a space. One generation, not three layered ones.
|
|
51
|
-
- **Dialogue and scratch VO.**
|
|
51
|
+
- **Dialogue and scratch VO inside a scene.** The line is performed in the room, by a voice the scene casts. When WHO is speaking matters — a character's own voice, a clean narrator — that is `inworld-tts-2` instead. Lock the read by re-rolling until a take is right, then lip-sync against it with `slates_generate_lip_sync`.
|
|
52
52
|
- **NOT** a single effect that must land on a known frame — that is `eleven-sfx`, which takes an exact duration.
|
|
53
53
|
- **NOT** music. Slates has no music model; import a track and drop it on an audio track.
|
|
54
54
|
- **AUDIO-ONLY.** It cannot produce images or video.
|