@gentbajko/slopify 3.19.0 → 3.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/extension/slopify-studio-chrome.zip +0 -0
- package/dist/extension/slopify-studio-firefox.zip +0 -0
- package/dist/patch-notes/3.20.0.md +10 -0
- package/dist/patch-notes/index.json +6 -0
- package/dist/slices/admission/schema.js +2 -1
- package/dist/slices/narration/breaths-detect.js +51 -21
- package/dist/slices/narration/breaths-model.js +21 -8
- package/dist/slices/narration/breaths.js +15 -13
- package/dist/slices/narration/clean-pieces.js +69 -0
- package/dist/slices/play-drafts/convert.js +3 -2
- package/dist/slices/play-drafts/schema.js +3 -2
- package/dist/slices/project-templates/from-project.js +1 -0
- package/dist/slices/rebuild/preview-details.js +2 -1
- package/dist/slices/rebuild/recipe-breaths.js +21 -11
- package/dist/slices/rebuild/runtime-local.js +13 -12
- package/dist/tutorials/Editing-a-Project.md +3 -2
- package/dist/web/assets/index-DNPct8pl.js +326 -0
- package/dist/web/assets/{pdf-Dxrlh3NL.js → pdf-tyJWWiKA.js} +1 -1
- package/dist/web/index.html +1 -1
- package/package.json +1 -1
- package/dist/web/assets/index-OxnO0Bfh.js +0 -326
|
Binary file
|
|
Binary file
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Slopify 3.20.0
|
|
2
|
+
|
|
3
|
+
Released 11 October 2026.
|
|
4
|
+
|
|
5
|
+
- **Remove breaths is now Soften breaths, and it no longer clips words.** 3.19.0 muted every breath it found to silence. On a full-length narration some of those stretches held the edge of a word, which came out clipped ("Highs" as "Hi…s"), and the drops to silence were abrupt. Breaths are now turned down 12 dB with gentle fades, and any stretch with a word sound in it is left alone. See `docs/breaths.md`.
|
|
6
|
+
- **Soften S sounds.** A new switch under Soften breaths (Play → Video and style, Edit project → Pauses and volume) takes the sharp edge off s and sh sounds with a very light de-esser. It is off by default and works with or without Soften breaths.
|
|
7
|
+
|
|
8
|
+
## Upgrading from 3.19.0
|
|
9
|
+
|
|
10
|
+
- A project that had Remove breaths on joins its narration again once, from the pieces it already has. No speech is made again. If you used Remove breaths, remake those videos.
|
|
@@ -185,8 +185,9 @@ export const runDraftSchema = z.object({
|
|
|
185
185
|
// Pauses between sentences and paragraphs (`narration/pauses-model.ts`); checked by admission.
|
|
186
186
|
sentencePauseSeconds: z.number().optional(),
|
|
187
187
|
paragraphPauseSeconds: z.number().optional(),
|
|
188
|
-
//
|
|
188
|
+
// Soften breaths and Soften S sounds (`narration/breaths-model.ts`); absent is off.
|
|
189
189
|
removeBreaths: z.boolean().optional(),
|
|
190
|
+
deEss: z.boolean().optional(),
|
|
190
191
|
// Absent is English (`kernel/ports/languages.ts`).
|
|
191
192
|
language: languageSchema.optional(),
|
|
192
193
|
});
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { setImmediate as nextTurn } from "node:timers/promises";
|
|
2
|
-
// The breath detector behind
|
|
2
|
+
// The breath detector behind Soften breaths (`breaths-model.ts` says why). It reads 16-bit mono
|
|
3
3
|
// PCM at 48 kHz in 20 ms frames, 10 ms apart, and gives each frame three numbers: its level
|
|
4
4
|
// (RMS in dB), how often it crosses zero (noise crosses often, a voiced vowel seldom) and its
|
|
5
5
|
// spectral centroid (where its sound sits: an in-breath low, an "s" or "f" high).
|
|
@@ -8,15 +8,22 @@ import { setImmediate as nextTurn } from "node:timers/promises";
|
|
|
8
8
|
// 2. A run of breath-like frames 140 ms or longer is a breath. It grows back over the quiet
|
|
9
9
|
// frames before it until silence, a voiced word's tail or a fricative, and forward over the
|
|
10
10
|
// quiet after it until silence or the next word's voice.
|
|
11
|
-
// 3.
|
|
12
|
-
//
|
|
11
|
+
// 3. A stretch whose loudest 10 ms (RMS over its non-overlapping 10 ms blocks) is above -26 dB
|
|
12
|
+
// at the calibration (8.74 dB under the voice's level) holds a word sound, and is left alone. 3.19.0 muted those too
|
|
13
|
+
// and clipped word edges ("Highs" came out "Hi…s").
|
|
14
|
+
// 4. Each other breath is turned down 12 dB, with 45 ms linear fades inside its own stretch (a
|
|
15
|
+
// fade outside it once landed on the next word's first sound). Silencing it, as 3.19.0 did,
|
|
16
|
+
// dropped to nothing abruptly; a stretch of 90 ms or less is left as it is.
|
|
13
17
|
// Levels are relative to the voice's own: the median level of its voiced frames. A quieter or
|
|
14
|
-
// louder voice finds the same breaths. The thresholds are the
|
|
15
|
-
// approved by ear
|
|
18
|
+
// louder voice finds the same breaths. The thresholds are the prototype's absolute ones, which
|
|
19
|
+
// Gent approved by ear on the finished video's audio: its median voiced frame (read as
|
|
20
|
+
// `speechLevel` reads it, over the whole 2.5 h video) is -17.26 dB. The levelled pieces the
|
|
21
|
+
// join works on sit about 4 dB lower, and every threshold moves down with them.
|
|
16
22
|
export const breathRate = 48_000;
|
|
17
23
|
const frameLength = 960; // 20 ms
|
|
18
24
|
const hop = 480; // 10 ms
|
|
19
|
-
|
|
25
|
+
// Calibrated on the finished video's audio, which is what was approved by ear.
|
|
26
|
+
const referenceSpeech = -17.26;
|
|
20
27
|
const breathCeiling = -22;
|
|
21
28
|
const breathFloor = -55;
|
|
22
29
|
const voicedFloor = -42;
|
|
@@ -27,23 +34,31 @@ const breathFrames = 14; // 140 ms
|
|
|
27
34
|
// A voiced frame for the reference: under the noisy crossing rate and within 30 dB of the
|
|
28
35
|
// voice's loud frames (the 95th percentile).
|
|
29
36
|
const voicedSpan = 30;
|
|
30
|
-
|
|
31
|
-
const
|
|
37
|
+
// A stretch with a 10 ms block louder than this (at the calibration) is left alone.
|
|
38
|
+
const wordCeiling = -26;
|
|
39
|
+
const softGain = 10 ** (-12 / 20);
|
|
40
|
+
const fadeSamples = 2160; // 45 ms
|
|
32
41
|
// ceiling: 200 frames (2 s of audio, about 15 ms of work) between yields, so a long piece never
|
|
33
42
|
// holds the server's event loop long enough for the page to notice.
|
|
34
43
|
const framesPerTurn = 200;
|
|
35
|
-
// Where the breaths are in a piece, read once from its PCM in order.
|
|
36
|
-
export async function
|
|
44
|
+
// Where the breaths are in a piece, and which hold a word sound, read once from its PCM in order.
|
|
45
|
+
export async function scanBreaths(chunks) {
|
|
37
46
|
return breathsIn(await featuresOf(chunks));
|
|
38
47
|
}
|
|
39
|
-
// The
|
|
40
|
-
export function
|
|
48
|
+
// The breaths to soften in a piece.
|
|
49
|
+
export async function findBreaths(chunks) {
|
|
50
|
+
return (await scanBreaths(chunks)).stretches;
|
|
51
|
+
}
|
|
52
|
+
// The chunk starting `offset` samples into the piece with the stretches turned down; a new array.
|
|
53
|
+
export function softenChunk(chunk, offset, stretches) {
|
|
41
54
|
const out = Int16Array.from(chunk);
|
|
42
55
|
const end = offset + chunk.length;
|
|
43
56
|
for (const stretch of stretches) {
|
|
44
57
|
if (stretch.end <= offset || stretch.start >= end)
|
|
45
58
|
continue;
|
|
46
|
-
|
|
59
|
+
if (stretch.end - stretch.start <= 2 * fadeSamples)
|
|
60
|
+
continue;
|
|
61
|
+
const fade = fadeSamples;
|
|
47
62
|
const from = Math.max(stretch.start, offset);
|
|
48
63
|
const to = Math.min(stretch.end, end);
|
|
49
64
|
for (let at = from; at < to; at += 1) {
|
|
@@ -56,17 +71,18 @@ export function muteChunk(chunk, offset, stretches) {
|
|
|
56
71
|
function gainAt(stretch, fade, at) {
|
|
57
72
|
const into = at - stretch.start;
|
|
58
73
|
if (into < fade)
|
|
59
|
-
return
|
|
74
|
+
return 1 + ((softGain - 1) * into) / (fade - 1);
|
|
60
75
|
const out = at - (stretch.end - fade);
|
|
61
76
|
if (out >= 0)
|
|
62
|
-
return
|
|
63
|
-
return
|
|
77
|
+
return softGain + ((1 - softGain) * out) / (fade - 1);
|
|
78
|
+
return softGain;
|
|
64
79
|
}
|
|
65
80
|
async function featuresOf(chunks) {
|
|
66
81
|
let capacity = 4096;
|
|
67
82
|
let db = new Float32Array(capacity);
|
|
68
83
|
let crossings = new Float32Array(capacity);
|
|
69
84
|
let centroid = new Float32Array(capacity);
|
|
85
|
+
let blockDb = new Float32Array(capacity + 1);
|
|
70
86
|
let count = 0;
|
|
71
87
|
const work = fftWorkspace();
|
|
72
88
|
const frame = new Float64Array(frameLength);
|
|
@@ -84,6 +100,7 @@ async function featuresOf(chunks) {
|
|
|
84
100
|
db = grown(db, capacity);
|
|
85
101
|
crossings = grown(crossings, capacity);
|
|
86
102
|
centroid = grown(centroid, capacity);
|
|
103
|
+
blockDb = grown(blockDb, capacity + 1);
|
|
87
104
|
}
|
|
88
105
|
const start = count * hop - base;
|
|
89
106
|
for (let at = 0; at < frameLength; at += 1)
|
|
@@ -91,6 +108,9 @@ async function featuresOf(chunks) {
|
|
|
91
108
|
db[count] = levelOf(frame);
|
|
92
109
|
crossings[count] = crossingsOf(frame);
|
|
93
110
|
centroid[count] = powerSpectrumCentroid(frame, work);
|
|
111
|
+
// A frame's halves are blocks count and count + 1; the next frame reads the second again.
|
|
112
|
+
blockDb[count] = levelOf(frame.subarray(0, hop));
|
|
113
|
+
blockDb[count + 1] = levelOf(frame.subarray(hop));
|
|
94
114
|
count += 1;
|
|
95
115
|
if (count % framesPerTurn === 0)
|
|
96
116
|
await nextTurn();
|
|
@@ -99,7 +119,7 @@ async function featuresOf(chunks) {
|
|
|
99
119
|
carry = pending.slice(keep);
|
|
100
120
|
base += keep;
|
|
101
121
|
}
|
|
102
|
-
return { count, db, crossings, centroid };
|
|
122
|
+
return { count, db, crossings, centroid, blockDb };
|
|
103
123
|
}
|
|
104
124
|
function grown(values, capacity) {
|
|
105
125
|
const next = new Float32Array(capacity);
|
|
@@ -109,13 +129,14 @@ function grown(values, capacity) {
|
|
|
109
129
|
function breathsIn(features) {
|
|
110
130
|
const speech = speechLevel(features);
|
|
111
131
|
if (speech === undefined)
|
|
112
|
-
return [];
|
|
132
|
+
return { stretches: [], skipped: [] };
|
|
113
133
|
const level = (prototype) => prototype - referenceSpeech + speech;
|
|
114
134
|
const ceiling = level(breathCeiling);
|
|
115
135
|
const floor = level(breathFloor);
|
|
116
136
|
const voicedAbove = level(voicedFloor);
|
|
117
137
|
const silentBelow = level(silenceFloor);
|
|
118
|
-
const
|
|
138
|
+
const wordAbove = level(wordCeiling);
|
|
139
|
+
const { count, db, crossings, centroid, blockDb } = features;
|
|
119
140
|
const at = (values, index) => values[index] ?? 0;
|
|
120
141
|
const voiced = (index) => at(db, index) > voicedAbove && at(crossings, index) < noisyCrossings;
|
|
121
142
|
const hit = (index) => at(crossings, index) >= noisyCrossings &&
|
|
@@ -123,6 +144,14 @@ function breathsIn(features) {
|
|
|
123
144
|
at(db, index) < ceiling &&
|
|
124
145
|
at(centroid, index) < breathCentroid;
|
|
125
146
|
const stretches = [];
|
|
147
|
+
const skipped = [];
|
|
148
|
+
// Blocks a to b: the stretch from a * hop to (b - 1) * hop + frameLength = (b + 1) * hop.
|
|
149
|
+
const holdsWord = (a, b) => {
|
|
150
|
+
for (let block = a; block <= b; block += 1)
|
|
151
|
+
if (at(blockDb, block) > wordAbove)
|
|
152
|
+
return true;
|
|
153
|
+
return false;
|
|
154
|
+
};
|
|
126
155
|
let i = 0;
|
|
127
156
|
while (i < count) {
|
|
128
157
|
if (!hit(i)) {
|
|
@@ -142,11 +171,12 @@ function breathsIn(features) {
|
|
|
142
171
|
let b = j;
|
|
143
172
|
while (b < count && at(db, b) > silentBelow && !voiced(b))
|
|
144
173
|
b += 1;
|
|
145
|
-
|
|
174
|
+
const stretch = { start: a * hop, end: (b - 1) * hop + frameLength };
|
|
175
|
+
(holdsWord(a, b) ? skipped : stretches).push(stretch);
|
|
146
176
|
}
|
|
147
177
|
i = j;
|
|
148
178
|
}
|
|
149
|
-
return stretches;
|
|
179
|
+
return { stretches, skipped };
|
|
150
180
|
}
|
|
151
181
|
// The voice's own level: the median of its voiced frames, those under the noisy crossing rate
|
|
152
182
|
// within 30 dB of its loud frames. Undefined for a piece without a voiced frame.
|
|
@@ -1,13 +1,17 @@
|
|
|
1
|
-
//
|
|
2
|
-
//
|
|
3
|
-
// words. Switched on, the join finds those
|
|
4
|
-
// (`breaths.ts`, `breaths-detect.ts`)
|
|
5
|
-
//
|
|
6
|
-
//
|
|
7
|
-
//
|
|
1
|
+
// Soften breaths (the config keeps its first name, `removeBreaths`). Some voices (Inworld's
|
|
2
|
+
// TTS-2 among them) take a loud, audible in-breath after almost every comma and full stop:
|
|
3
|
+
// about a third of a second, only 12 to 15 dB under the words. Switched on, the join finds those
|
|
4
|
+
// breaths in every narration piece and turns them down 12 dB (`breaths.ts`, `breaths-detect.ts`),
|
|
5
|
+
// leaving alone any stretch with a word sound in it: nothing is cut out, so every word and its
|
|
6
|
+
// timing stay exactly where they were, and the pauses between sentences (`pauses-model.ts`) are
|
|
7
|
+
// planned from the pieces as spoken just the same. It happens in the join, so the plain narration
|
|
8
|
+
// the word timing reads and the levelled one the video plays lose the same breaths.
|
|
9
|
+
//
|
|
10
|
+
// Soften S sounds (`deEss`): each narration piece in the join goes through ffmpeg's de-esser,
|
|
11
|
+
// set very light (Gent approved these values by ear), after the breaths and before the pauses.
|
|
8
12
|
//
|
|
9
13
|
// Browser-safe: Play and Edit project read it. Off is absent from the config, which is what
|
|
10
|
-
// every project saved before
|
|
14
|
+
// every project saved before them was, and the joins keep their fingerprints then.
|
|
11
15
|
export function breathsOn(config) {
|
|
12
16
|
return config.removeBreaths === true;
|
|
13
17
|
}
|
|
@@ -15,3 +19,12 @@ export function breathsOn(config) {
|
|
|
15
19
|
export function breathsOfForm(form) {
|
|
16
20
|
return form.removeBreaths === true ? { removeBreaths: true } : {};
|
|
17
21
|
}
|
|
22
|
+
// ffmpeg's `deesser` options: intensity, most de-essing, frequency (each 0 to 1).
|
|
23
|
+
export const deEsser = { intensity: 0.35, max: 0.5, frequency: 0.5 };
|
|
24
|
+
export function deEssOn(config) {
|
|
25
|
+
return config.deEss === true;
|
|
26
|
+
}
|
|
27
|
+
// Play's switch as a run's config holds it: only on is written.
|
|
28
|
+
export function deEssOfForm(form) {
|
|
29
|
+
return form.deEss === true ? { deEss: true } : {};
|
|
30
|
+
}
|
|
@@ -2,19 +2,20 @@ import { mkdirSync, rmSync } from "node:fs";
|
|
|
2
2
|
import { open, stat } from "node:fs/promises";
|
|
3
3
|
import { join } from "node:path";
|
|
4
4
|
import { runFfmpeg } from "../video/ffmpeg.js";
|
|
5
|
-
import { breathRate,
|
|
6
|
-
//
|
|
5
|
+
import { breathRate, scanBreaths, softenChunk } from "./breaths-detect.js";
|
|
6
|
+
// Soften breaths, made in the join (`breaths-model.ts` says why there). Each narration piece is
|
|
7
7
|
// decoded by ffmpeg to 16-bit mono PCM at 48 kHz in a file beside it, read once to find its
|
|
8
|
-
// breaths (`breaths-detect.ts`) and once more to write it again as a WAV with them
|
|
8
|
+
// breaths (`breaths-detect.ts`) and once more to write it again as a WAV with them turned down. Both
|
|
9
9
|
// reads go in chunks, so a long piece never sits in memory whole, and the detector yields to
|
|
10
10
|
// the event loop as it goes, so the server stays responsive while it works. A piece without a
|
|
11
11
|
// breath is written again all the same: the join takes every piece in the one format.
|
|
12
12
|
// ceiling: one second of audio per read.
|
|
13
13
|
const chunkSamples = breathRate;
|
|
14
|
-
export async function
|
|
14
|
+
export async function softenBreaths(run, files, directory) {
|
|
15
15
|
mkdirSync(directory, { recursive: true, mode: 0o700 });
|
|
16
16
|
const written = [];
|
|
17
|
-
let
|
|
17
|
+
let softened = 0;
|
|
18
|
+
let skipped = 0;
|
|
18
19
|
for (const [index, file] of files.entries()) {
|
|
19
20
|
run.signal.throwIfAborted();
|
|
20
21
|
const name = `breaths-${String(index + 1).padStart(4, "0")}`;
|
|
@@ -28,22 +29,23 @@ export async function removeBreaths(run, files, directory) {
|
|
|
28
29
|
log: run.log,
|
|
29
30
|
onProgress: () => { },
|
|
30
31
|
});
|
|
31
|
-
const
|
|
32
|
+
const scan = await scanBreaths(pcmChunks(raw));
|
|
32
33
|
run.signal.throwIfAborted();
|
|
33
|
-
await
|
|
34
|
-
|
|
34
|
+
await writeSoftened(raw, output, scan.stretches);
|
|
35
|
+
softened += scan.stretches.length;
|
|
36
|
+
skipped += scan.skipped.length;
|
|
35
37
|
}
|
|
36
38
|
catch (error) {
|
|
37
39
|
if (run.signal.aborted)
|
|
38
40
|
throw error;
|
|
39
|
-
throw new Error(`
|
|
41
|
+
throw new Error(`Soften breaths could not process narration piece ${String(index + 1)} (${error instanceof Error ? error.message : String(error)}), so the narration can't be joined together. Use Try again; if it fails again, turn off Soften breaths in Edit project → Pauses and volume and use Save changes, then use Download diagnostics in Settings and report it.`, { cause: error });
|
|
40
42
|
}
|
|
41
43
|
finally {
|
|
42
44
|
rmSync(raw, { force: true });
|
|
43
45
|
}
|
|
44
46
|
written.push(output);
|
|
45
47
|
}
|
|
46
|
-
return { files: written,
|
|
48
|
+
return { files: written, softened, skipped };
|
|
47
49
|
}
|
|
48
50
|
export function decodeArgs(input, output) {
|
|
49
51
|
return [
|
|
@@ -94,7 +96,7 @@ async function readChunk(handle) {
|
|
|
94
96
|
out[at] = buffer.readInt16LE(at * 2);
|
|
95
97
|
return out;
|
|
96
98
|
}
|
|
97
|
-
async function
|
|
99
|
+
async function writeSoftened(raw, output, stretches) {
|
|
98
100
|
const size = (await stat(raw)).size;
|
|
99
101
|
const bytes = size - (size % 2);
|
|
100
102
|
const target = await open(output, "w", 0o600);
|
|
@@ -102,9 +104,9 @@ async function writeMuted(raw, output, stretches) {
|
|
|
102
104
|
await target.write(wavHeader(bytes));
|
|
103
105
|
let offset = 0;
|
|
104
106
|
for await (const chunk of pcmChunks(raw)) {
|
|
105
|
-
const
|
|
107
|
+
const soft = softenChunk(chunk, offset, stretches);
|
|
106
108
|
// Typed arrays are little-endian on every platform Slopify ships for, as WAV is.
|
|
107
|
-
await target.write(Buffer.from(
|
|
109
|
+
await target.write(Buffer.from(soft.buffer, soft.byteOffset, soft.byteLength));
|
|
108
110
|
offset += chunk.length;
|
|
109
111
|
}
|
|
110
112
|
}
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
import { mkdirSync } from "node:fs";
|
|
2
|
+
import { join } from "node:path";
|
|
3
|
+
import { runFfmpeg } from "../video/ffmpeg.js";
|
|
4
|
+
import { softenBreaths } from "./breaths.js";
|
|
5
|
+
import { deEsser } from "./breaths-model.js";
|
|
6
|
+
export async function cleanPieces(run, files, directory, cleaning) {
|
|
7
|
+
if (!cleaning.breaths && !cleaning.deEss)
|
|
8
|
+
return undefined;
|
|
9
|
+
const breaths = cleaning.breaths
|
|
10
|
+
? await softenBreaths(run, files, join(directory, "breaths"))
|
|
11
|
+
: undefined;
|
|
12
|
+
const soft = breaths?.files ?? files;
|
|
13
|
+
const cleaned = cleaning.deEss ? await deEssPieces(run, soft, join(directory, "deess")) : soft;
|
|
14
|
+
return {
|
|
15
|
+
files: cleaned,
|
|
16
|
+
...(breaths === undefined
|
|
17
|
+
? {}
|
|
18
|
+
: { breaths: { softened: breaths.softened, skipped: breaths.skipped } }),
|
|
19
|
+
};
|
|
20
|
+
}
|
|
21
|
+
export async function deEssPieces(run, files, directory) {
|
|
22
|
+
mkdirSync(directory, { recursive: true, mode: 0o700 });
|
|
23
|
+
const written = [];
|
|
24
|
+
for (const [index, file] of files.entries()) {
|
|
25
|
+
run.signal.throwIfAborted();
|
|
26
|
+
const output = join(directory, `deess-${String(index + 1).padStart(4, "0")}.wav`);
|
|
27
|
+
try {
|
|
28
|
+
await runFfmpeg({
|
|
29
|
+
bin: run.bin,
|
|
30
|
+
args: deEssArgs(file, output),
|
|
31
|
+
signal: run.signal,
|
|
32
|
+
log: run.log,
|
|
33
|
+
onProgress: () => { },
|
|
34
|
+
});
|
|
35
|
+
}
|
|
36
|
+
catch (error) {
|
|
37
|
+
if (run.signal.aborted)
|
|
38
|
+
throw error;
|
|
39
|
+
throw new Error(`Soften S sounds could not process narration piece ${String(index + 1)} (${error instanceof Error ? error.message : String(error)}), so the narration can't be joined together. Use Try again; if it fails again, turn off Soften S sounds in Edit project → Pauses and volume and use Save changes, then use Download diagnostics in Settings and report it.`, { cause: error });
|
|
40
|
+
}
|
|
41
|
+
written.push(output);
|
|
42
|
+
}
|
|
43
|
+
return written;
|
|
44
|
+
}
|
|
45
|
+
export function deEssFilter() {
|
|
46
|
+
return `deesser=i=${String(deEsser.intensity)}:m=${String(deEsser.max)}:f=${String(deEsser.frequency)}`;
|
|
47
|
+
}
|
|
48
|
+
// The piece's own rate and channels, written as 16-bit WAV like the other cleaned pieces.
|
|
49
|
+
export function deEssArgs(input, output) {
|
|
50
|
+
return [
|
|
51
|
+
"-hide_banner",
|
|
52
|
+
"-nostdin",
|
|
53
|
+
"-loglevel",
|
|
54
|
+
"error",
|
|
55
|
+
"-nostats",
|
|
56
|
+
"-y",
|
|
57
|
+
"-i",
|
|
58
|
+
input,
|
|
59
|
+
"-map",
|
|
60
|
+
"0:a:0",
|
|
61
|
+
"-af",
|
|
62
|
+
deEssFilter(),
|
|
63
|
+
"-c:a",
|
|
64
|
+
"pcm_s16le",
|
|
65
|
+
"-f",
|
|
66
|
+
"wav",
|
|
67
|
+
output,
|
|
68
|
+
];
|
|
69
|
+
}
|
|
@@ -4,7 +4,7 @@ import { runDraftSchema } from "../admission/schema.js";
|
|
|
4
4
|
import { draftDocumentThemeOf } from "../document/model.js";
|
|
5
5
|
import { defaultExpectedWords, everyMinutesMax, everyMinutesMin, expectedWordsMax, imagesPerHourMax, imagesPerHourMin, perHourFromMinutes, wordCount, } from "../images/scale.js";
|
|
6
6
|
import { defaultLoudness, loudnessOfForm, } from "../loudness/model.js";
|
|
7
|
-
import { breathsOfForm } from "../narration/breaths-model.js";
|
|
7
|
+
import { breathsOfForm, deEssOfForm } from "../narration/breaths-model.js";
|
|
8
8
|
import { pausesOfForm } from "../narration/pauses-model.js";
|
|
9
9
|
import { reviewSettingsFromForm } from "../reviews/model.js";
|
|
10
10
|
import { stageMakesItems } from "../reviews/rules.js";
|
|
@@ -293,8 +293,9 @@ export function toAdmissionDraft(input) {
|
|
|
293
293
|
...pickLoudness(sources.audio, loudnessOfForm(form.loudness, input.loudness ?? defaultLoudness)),
|
|
294
294
|
// Pauses between sentences: the default for a new run, with a narration to pace.
|
|
295
295
|
...(sources.audio === "generate" ? pausesOfForm(form) : {}),
|
|
296
|
-
//
|
|
296
|
+
// Soften breaths and Soften S sounds: only while on, with a narration made here to clean.
|
|
297
297
|
...(sources.audio === "generate" ? breathsOfForm(form) : {}),
|
|
298
|
+
...(sources.audio === "generate" ? deEssOfForm(form) : {}),
|
|
298
299
|
edgeSilenceSeconds: measure("edgeSilenceSeconds", sources.audio !== "off", defaultEdgeSilenceSeconds, edgeSilenceSecondsProblem),
|
|
299
300
|
};
|
|
300
301
|
const parsed = runDraftSchema.safeParse(draft);
|
|
@@ -86,9 +86,10 @@ export const playDraftFormSchema = z
|
|
|
86
86
|
// paragraph minimum (`narration/pauses-model.ts`).
|
|
87
87
|
sentencePause: text.optional(),
|
|
88
88
|
paragraphPause: text.optional(),
|
|
89
|
-
//
|
|
90
|
-
// before
|
|
89
|
+
// Soften breaths and Soften S sounds (`narration/breaths-model.ts`). Absent on drafts and
|
|
90
|
+
// templates saved before them: off.
|
|
91
91
|
removeBreaths: z.boolean().optional(),
|
|
92
|
+
deEss: z.boolean().optional(),
|
|
92
93
|
// Absent on drafts and templates saved before the YouTube description: off, built-in prompt.
|
|
93
94
|
youtubeDescription: z.boolean().optional(),
|
|
94
95
|
showFigures: z.boolean().optional(),
|
|
@@ -237,6 +237,7 @@ export function documentFromProject(deps, revision) {
|
|
|
237
237
|
? {}
|
|
238
238
|
: { paragraphPause: String(config.paragraphPauseSeconds) }),
|
|
239
239
|
...(config.removeBreaths === true ? { removeBreaths: true } : {}),
|
|
240
|
+
...(config.deEss === true ? { deEss: true } : {}),
|
|
240
241
|
imagePrompts,
|
|
241
242
|
...(config.imageScale === undefined ? {} : { imageScale: imageScaleForm(config.imageScale) }),
|
|
242
243
|
...(config.imageScenes === true ? { imageScenes: true } : {}),
|
|
@@ -159,7 +159,8 @@ function inputChanges(parent, view) {
|
|
|
159
159
|
add("Show tables and figures on screen", before.showFigures === true ? "On" : "Off", after.showFigures === true ? "On" : "Off");
|
|
160
160
|
add("Pause between sentences (seconds)", before.sentencePauseSeconds ?? 0, after.sentencePauseSeconds ?? 0);
|
|
161
161
|
add("Pause between paragraphs (seconds)", before.paragraphPauseSeconds ?? 0, after.paragraphPauseSeconds ?? 0);
|
|
162
|
-
add("
|
|
162
|
+
add("Soften breaths", before.removeBreaths === true ? "On" : "Off", after.removeBreaths === true ? "On" : "Off");
|
|
163
|
+
add("Soften S sounds", before.deEss === true ? "On" : "Off", after.deEss === true ? "On" : "Off");
|
|
163
164
|
add("Level the volume", before.loudness === undefined ? "Off" : loudnessSummary(before.loudness), after.loudness === undefined ? "Off" : loudnessSummary(after.loudness));
|
|
164
165
|
add("YouTube description", before.youtubeDescription === true ? "On" : "Off", after.youtubeDescription === true ? "On" : "Off");
|
|
165
166
|
add("Description prompt", before.descriptionPrompt, after.descriptionPrompt);
|
|
@@ -1,16 +1,26 @@
|
|
|
1
|
-
import { breathsOn } from "../narration/breaths-model.js";
|
|
2
|
-
//
|
|
3
|
-
//
|
|
4
|
-
// every join made before
|
|
5
|
-
//
|
|
6
|
-
|
|
1
|
+
import { breathsOn, deEsser, deEssOn } from "../narration/breaths-model.js";
|
|
2
|
+
// Soften breaths and Soften S sounds (`narration/breaths-model.ts`) are made in the narration
|
|
3
|
+
// joins, so a join carries them in its values, after the pauses when they are set. Nothing at
|
|
4
|
+
// all while they are off, so every join made before them keeps its fingerprint. The levelled
|
|
5
|
+
// join copies these values, so it softens the same.
|
|
6
|
+
// v2: 3.19.0's v1 muted breaths to silence and could clip word edges; its joins are remade.
|
|
7
|
+
export const breathsValue = "breaths-v2";
|
|
8
|
+
export const deEssValue = "deess-v1";
|
|
7
9
|
export function breathValues(config) {
|
|
8
|
-
return
|
|
10
|
+
return [
|
|
11
|
+
...(breathsOn(config) ? [[breathsValue]] : []),
|
|
12
|
+
...(deEssOn(config) ? [[deEssValue, deEsser.intensity, deEsser.max, deEsser.frequency]] : []),
|
|
13
|
+
];
|
|
9
14
|
}
|
|
10
15
|
// Whether a join's values ask for it: any value after the join's own two.
|
|
11
16
|
export function breathsAsked(values) {
|
|
12
|
-
return (
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
17
|
+
return laterValues(values).some((value) => value.length === 1 && value[0] === breathsValue);
|
|
18
|
+
}
|
|
19
|
+
export function deEssAsked(values) {
|
|
20
|
+
return laterValues(values).some((value) => value[0] === deEssValue);
|
|
21
|
+
}
|
|
22
|
+
function laterValues(values) {
|
|
23
|
+
return Array.isArray(values)
|
|
24
|
+
? values.slice(2).filter((value) => Array.isArray(value))
|
|
25
|
+
: [];
|
|
16
26
|
}
|
|
@@ -5,14 +5,14 @@ import { referenceKey, thumbnailVariant } from "../admission/model.js";
|
|
|
5
5
|
import { plainText } from "../article/plain.js";
|
|
6
6
|
import { splitEndMatter } from "../article/split.js";
|
|
7
7
|
import { levelPieces } from "../loudness/level-pieces.js";
|
|
8
|
-
import {
|
|
8
|
+
import { cleanPieces } from "../narration/clean-pieces.js";
|
|
9
9
|
import { joinNarration } from "../narration/concat.js";
|
|
10
10
|
import { applyPauses, planPieces } from "../narration/pauses.js";
|
|
11
11
|
import { allocateAsset, discardPreparedAssets, sealAsset } from "../storage/assets.js";
|
|
12
12
|
import { outputPath } from "../storage/layout.js";
|
|
13
13
|
import { probeDurationMs, runFfmpeg } from "../video/ffmpeg.js";
|
|
14
14
|
import { turnJoinArgs } from "../voices/join.js";
|
|
15
|
-
import { breathsAsked } from "./recipe-breaths.js";
|
|
15
|
+
import { breathsAsked, deEssAsked } from "./recipe-breaths.js";
|
|
16
16
|
import { levelOperation } from "./recipe-loudness.js";
|
|
17
17
|
import { executeFigureCard } from "./runtime-figure-card.js";
|
|
18
18
|
import { publishNarrationText } from "./runtime-narration-text.js";
|
|
@@ -136,27 +136,28 @@ async function concatenate(deps, context, piece) {
|
|
|
136
136
|
try {
|
|
137
137
|
const run = { bin: deps.ffmpeg, log: deps.log, signal: context.signal };
|
|
138
138
|
const levelled = level ? await levelPieces(run, files, piecesDirectory) : undefined;
|
|
139
|
-
//
|
|
140
|
-
// are still planned from the pieces as spoken.
|
|
141
|
-
const
|
|
142
|
-
|
|
143
|
-
:
|
|
144
|
-
|
|
139
|
+
// Soften breaths, then Soften S sounds: made in what is joined, before the pauses; nothing
|
|
140
|
+
// moves, so the pauses are still planned from the pieces as spoken.
|
|
141
|
+
const cleaned = await cleanPieces(run, levelled?.files ?? files, piecesDirectory, {
|
|
142
|
+
breaths: breathsAsked(values),
|
|
143
|
+
deEss: deEssAsked(values),
|
|
144
|
+
});
|
|
145
|
+
if (cleaned?.breaths !== undefined)
|
|
145
146
|
deps.log.write("info", "narration.breaths", {
|
|
146
147
|
projectId: context.work.projectId,
|
|
147
148
|
stage: "audio",
|
|
148
|
-
detail: `${String(
|
|
149
|
+
detail: `${String(cleaned.breaths.softened)} breaths softened, ${String(cleaned.breaths.skipped)} left alone (a word sound in them) in ${String(files.length)} pieces`,
|
|
149
150
|
});
|
|
150
151
|
// Pauses between sentences: planned from the pieces as spoken, made in what is joined.
|
|
151
152
|
const pauses = pausesIn(values);
|
|
152
153
|
const paced = pauses === undefined
|
|
153
154
|
? undefined
|
|
154
|
-
: await applyPauses(run,
|
|
155
|
+
: await applyPauses(run, cleaned?.files ?? levelled?.files ?? files, paceScaled(await planPieces(run, files.map((file, at) => ({
|
|
155
156
|
file,
|
|
156
157
|
text: pieceText(plan.recipes.find((one) => one.key === recipe.dependsOn[at])),
|
|
157
158
|
next: pauses.next[at] ?? "end",
|
|
158
159
|
})), pauses.settings), operation === "concat-turns-v1" ? values : undefined), join(piecesDirectory, "paced"));
|
|
159
|
-
const joined = paced?.files ??
|
|
160
|
+
const joined = paced?.files ?? cleaned?.files ?? levelled?.files ?? files;
|
|
160
161
|
const durationMs = operation === "concat-turns-v1"
|
|
161
162
|
? await joinTurns(run, joined, values, pending.absolutePath)
|
|
162
163
|
: await joinNarration({ bin: deps.ffmpeg, log: deps.log }, {
|
|
@@ -164,7 +165,7 @@ async function concatenate(deps, context, piece) {
|
|
|
164
165
|
output: pending.absolutePath,
|
|
165
166
|
listPath: join(dirname(pending.absolutePath), "parts.txt"),
|
|
166
167
|
signal: context.signal,
|
|
167
|
-
reencode: level || paced !== undefined ||
|
|
168
|
+
reencode: level || paced !== undefined || cleaned !== undefined,
|
|
168
169
|
});
|
|
169
170
|
rmSync(piecesDirectory, { recursive: true, force: true });
|
|
170
171
|
context.signal.throwIfAborted();
|
|
@@ -134,7 +134,8 @@ See [Narration Aliases and Glossary](Narration-Aliases-and-Glossary).
|
|
|
134
134
|
| --- | --- | --- |
|
|
135
135
|
| **Pause between sentences (seconds)** | The least quiet after each sentence. Adds silence where the pause is shorter; never shortens one or cuts a word. About 0.25 s is brisk, 0.35 to 0.45 natural, 0.5 to 0.8 an audiobook's pace, 0.8 and up sleep content. 0 keeps the voice's own. | 0.4; 0 to 2 in steps of 0.05 |
|
|
136
136
|
| **Pause between paragraphs (seconds)** | The least quiet at the end of a paragraph, when you want more than between sentences. At 0 paragraphs get the sentence pause. Try 0.8 to 1.2 for a slower read. A speaker's turn gap is set under Speakers instead. | 0; 0 to 2 |
|
|
137
|
-
| **
|
|
137
|
+
| **Soften breaths** | Turns the audible in-breaths a voice takes between sentences and after commas down by 12 dB, with soft fades (Inworld's TTS-2 takes a loud one almost every time). A breath with any word sound in it is left alone; words, pauses and captions stay where they were. | Off |
|
|
138
|
+
| **Soften S sounds** | Takes the sharp edge off s and sh sounds with a very light de-esser; the voice keeps its brightness. Works with or without Soften breaths. | Off |
|
|
138
139
|
| **Level the volume** | Brings every narration piece to one loudness before they are joined, then sets the video, shorts and audio files to a finished loudness. | On for new runs (Settings → General → **Level the volume for new runs**) |
|
|
139
140
|
| **Video volume** | How loud the video and shorts are, in dB from the recommended level (0 dB is -14 LUFS, what YouTube and Spotify play at). -6 dB is about 50%, +4 dB about 158%. Peaks stay under -1.5 dBTP. | 0 dB; -10 to +4 dB in 0.5 dB steps |
|
|
140
141
|
| **Audio files volume** | How loud the WAV, MP3 and M4B are, in dB from -18 LUFS (what audiobook shops such as ACX ask for). Peaks stay under -3 dBTP. | 0 dB; -10 to +4 dB |
|
|
@@ -143,7 +144,7 @@ You can type a volume in dB or as a percentage of the recommended level.
|
|
|
143
144
|
|
|
144
145
|
What changing them remakes:
|
|
145
146
|
|
|
146
|
-
- **Pauses or
|
|
147
|
+
- **Pauses, Soften breaths or Soften S sounds**: the narration is joined again from the pieces it already has and timed again, which remakes what is timed from it: captions, the video and, when on, the YouTube description and the shorts' pick (those two are text calls). No speech is made again.
|
|
147
148
|
- **Level the volume on or off**: the levelled narration is joined again and the exports rendered again. No speech is made again, and word timing, description, shorts' pick and reviews are not redone.
|
|
148
149
|
- **A volume only**: the exports are mastered again, nothing else.
|
|
149
150
|
|