studio-engine 0.119.1 → 0.120.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 327664450f0c0b1de0a01b75231b2f2fa470ab66d865f3b585eb55281e3318d3
4
- data.tar.gz: 0634a4d5e519e0a808558cd3341d192ab7c0162373ad46d376e6cd6753a1f432
3
+ metadata.gz: 602092c1d8ddd3ce61cb6f192162ea1cae13af1313769742fa88d5f954327232
4
+ data.tar.gz: 6a2ca70d13097ebfe23e52c7b14d551b430f92ef07da51bed9c1b80f1baa522c
5
5
  SHA512:
6
- metadata.gz: 75317c813644a660eb548ddd2ee7727371b7df3520867ef0037449a7d68bb57a72c5e0c87eb767b56426459f627809f17dd2dacd8d56baf9ffd41226337c7368
7
- data.tar.gz: ba9ee2c20c1edf6659186b89a30a44a80c5cc3e4c4afd3b44910bdc934922c82d0250995996746b8ecc12976690d6ca5f8880dcae6f1caab7a149471a8808307
6
+ metadata.gz: 5e8a55b6e21888c67df80712062478ae4b635a4a95d2e062771ab00570f5cd9ba417601e05ab39f14791be16619c3780799d965d2c02491952386001dd6b6f18
7
+ data.tar.gz: 7abef1323bfc65d590af8c5cef094d7136471e6f0356703b6a1fd73c1bb3137d76b70051b500f6ddab24c4d092d3812f3ae82e766123dbf728c15b28dbd410a4
data/CHANGELOG.md CHANGED
@@ -4,6 +4,26 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
4
4
 
5
5
  ## Unreleased
6
6
 
7
+ ## 0.120.0 — 2026-10-30
8
+
9
+ ### Added
10
+
11
+ - **Voice profile on `/profile`, opt-in.** `config.voice_profile = true` adds a
12
+ "Your voice" row: the signed-in user reads three phrases while a live level
13
+ meter shows the microphone being heard, silent takes are refused, and each
14
+ take becomes a speaker embedding on the device (WeSpeaker ResNet34 int8 from
15
+ Hugging Face at a pinned revision, through transformers.js 3.7.6 from
16
+ jsDelivr). Save posts only the three vectors (`POST /profile/voice`); the
17
+ server averages them onto the user's own record. Re-record and Delete
18
+ (`DELETE /profile/voice`) are on the row, with a consent line saying what is
19
+ stored. A new host-table migration, `add_voice_profile_to_users`, adds
20
+ `voice_embedding`, `voice_embedding_model` and `voice_embedding_updated_at`.
21
+ `Studio::VoiceMatch.identify(embedding, among:, model:)` returns the closest
22
+ user at or above 0.6 cosine, else nil, comparing only voices from the same
23
+ model. `config.voice_embedder = "fake"` is the test embedder
24
+ (`window.studioFakeVoice`). docs/VOICE_PROFILE.md lists the CSP and privacy
25
+ changes a host makes first.
26
+
7
27
  ## 0.119.1 — 2026-10-29
8
28
 
9
29
  ### Fixed
data/README.md CHANGED
@@ -45,6 +45,7 @@ resolved.
45
45
  - **Transactional emails**: `Studio::EmailCatalog` — every email an app sends, its type, a live preview, and its banner — plus the shared `/admin/emails` page. Every app inherits the standard emails and their artwork on day one, and can register its own workflows and upload its own banners. See [Transactional emails](#transactional-emails).
46
46
  - **Site footer**: `<%= studio_site_footer %>` in the layout renders the public site's footer. It is on by default: with nothing configured it prints the app's name, logo and the legal links the app has routes for, and no address. `config.site_footer` declares the app's own facts (brand, social profiles, link columns, an address with a live map, legal links), every part optional; `config.site_footer_address` adds a location in one setting; `config.site_footer = false` turns the footer off. Leaflet is vendored and served by the engine. See [`docs/SITE_FOOTER.md`](docs/SITE_FOOTER.md).
47
47
  - **Booking**: a simple scheduler on Google Calendar appointment schedules, with or without the footer. With `config.booking_url`, `studio_booking_frame` embeds Google's booking page, `studio_booking_link` opens it in a popup, and `config.draw_booking_routes = true` draws `/schedule` (or name the app's own page with `config.booking_path`). `config.booking_crop` crops the frame to the slot picker at rest, from numbers `bin/booking-crop-measure` reads off the app's own schedule. See [`docs/BOOKING.md`](docs/BOOKING.md).
48
+ - **Voice profile**: an opt-in row on `/profile` ("Set up your voice"): the person reads three phrases with a live level meter, their device turns each take into a speaker embedding (WeSpeaker via transformers.js), and only the averaged vector is saved on their user, never audio. `Studio::VoiceMatch.identify(embedding, among: User.all, model:)` answers who is speaking. Turn it on with `config.voice_profile = true` after installing the migration. See [`docs/VOICE_PROFILE.md`](docs/VOICE_PROFILE.md).
48
49
  - **Surveys**: define a survey in app code (`config/surveys/*.rb`, `Studio.define_survey`) with six question types — emoji scale, 1–5 rating, single and multiple choice, short and long text — and the engine serves it at `/surveys/:slug`: one question per screen, autosave, resume, keyboard shortcuts, and a no-JavaScript fallback. `config.on_survey_completed` and `config.survey_ref_resolver` let the app fire its own goals and attribute email arrivals. An admin panel at `/admin/surveys` breaks results down per question and exports CSV. Opt in with `config.draw_survey_routes = true` and `config.draw_admin_survey_routes = true`. See [`docs/SURVEYS.md`](docs/SURVEYS.md).
49
50
 
50
51
  ## Tailwind
@@ -202,8 +202,64 @@ module Studio
202
202
  end
203
203
  end
204
204
 
205
+ # POST /profile/voice — save (or replace) the signed-in user's voice.
206
+ #
207
+ # THE BODY IS VECTORS, NEVER AUDIO. The page records three phrases, turns
208
+ # each into a speaker embedding on the device (studio/voice_profile) and
209
+ # drops the takes; this receives `voice[model]` and `voice[embeddings]` (a
210
+ # JSON array of Studio::VoiceMatch::TAKES vectors) and averages them here.
211
+ #
212
+ # The model must be the one this app runs (Studio.voice_embedder), so a page
213
+ # cannot save vectors from an embedder the matcher would never compare, and
214
+ # the test embedder can never be saved where the real one runs. A take that
215
+ # embedded to all zeros (silence the browser let through) is refused by
216
+ # VoiceMatch.enrol.
217
+ #
218
+ # Answers JSON for the page's fetch and redirects for anything else.
219
+ def save_voice
220
+ return voice_refused("This app has no voice profile to set up.", :not_found) unless row_rendered?(:voice)
221
+
222
+ embedder = Studio.voice_embedder.to_s
223
+ model = params.dig(:voice, :model).to_s
224
+ unless model == Studio::VoiceMatch.model_id(embedder)
225
+ return voice_refused("That voice came from a different model. Reload the page and record again.")
226
+ end
227
+
228
+ vector = Studio::VoiceMatch.enrol(params.dig(:voice, :embeddings), size: Studio::VoiceMatch.size(embedder))
229
+ if vector.nil?
230
+ return voice_refused("We couldn't use those recordings. Read all three phrases again.")
231
+ end
232
+
233
+ rescue_and_log(target: current_user) do
234
+ current_user.update!(voice_embedding: vector, voice_embedding_model: model,
235
+ voice_embedding_updated_at: Time.current)
236
+ respond_to do |format|
237
+ format.json { render json: { url: profile_path }, status: :ok }
238
+ format.any { redirect_to profile_path, notice: "Your voice is set up.", status: :see_other }
239
+ end
240
+ end
241
+ end
242
+
243
+ # DELETE /profile/voice — remove the stored voice entirely: the vector, its
244
+ # model and its date. Nothing of it is kept.
245
+ def delete_voice
246
+ return unsupported("voice profile") unless row_rendered?(:voice)
247
+
248
+ rescue_and_log(target: current_user) do
249
+ current_user.update!(voice_embedding: nil, voice_embedding_model: nil, voice_embedding_updated_at: nil)
250
+ redirect_to profile_path, notice: "Your voice was deleted.", status: :see_other
251
+ end
252
+ end
253
+
205
254
  private
206
255
 
256
+ def voice_refused(message, status = :unprocessable_entity)
257
+ respond_to do |format|
258
+ format.json { render json: { error: message }, status: status }
259
+ format.any { redirect_to profile_path, alert: message, status: :see_other }
260
+ end
261
+ end
262
+
207
263
  # Tell the host, and NEVER let it break the subscription.
208
264
  #
209
265
  # The subscription is the durable fact; the reaction is not. turf-monster's
@@ -0,0 +1,20 @@
1
+ // voice-profile: the voice row on /profile (studio/profiles/_voice_section), on
2
+ // the engine's Stimulus application (studio/stimulus registers it lazily, the
3
+ // first time a page renders the row). It sits on the row's wrapper:
4
+ //
5
+ // <div data-studio-controller="voice-profile" data-voice-save-url="/profile/voice" ...>
6
+ //
7
+ // The recording, the meter, the embedding and the save are studio/voice_profile.
8
+ import { Controller } from "@hotwired/stimulus"
9
+ import { mountVoiceProfile } from "studio/voice_profile"
10
+
11
+ export default class extends Controller {
12
+ connect() {
13
+ this.voice = mountVoiceProfile(this.element)
14
+ }
15
+
16
+ disconnect() {
17
+ if (this.voice) this.voice.destroy()
18
+ this.voice = null
19
+ }
20
+ }
@@ -57,7 +57,9 @@ export const schema = {
57
57
  // knowledge transcript: a recording whose controller failed still plays from
58
58
  // its own controls, and its page says that click-to-play did not load. So is the
59
59
  // agent chat: a chat whose controller failed still renders, and its section
60
- // carries data-studio-controller-failed="agent-chat".
60
+ // carries data-studio-controller-failed="agent-chat". So is the voice row: a
61
+ // row whose controller failed keeps its saved state and its Delete, and says
62
+ // that recording did not load (data-studio-controller-failed="voice-profile").
61
63
  export const LAZY = {
62
64
  "board": () => import("studio/controllers/board_controller"),
63
65
  "geo-settings": () => import("studio/controllers/geo_settings_controller"),
@@ -67,7 +69,8 @@ export const LAZY = {
67
69
  "footer-map": () => import("studio/controllers/footer_map_controller"),
68
70
  "survey": () => import("studio/controllers/survey_controller"),
69
71
  "knowledge-transcript": () => import("studio/controllers/knowledge_transcript_controller"),
70
- "agent-chat": () => import("studio/controllers/agent_chat_controller")
72
+ "agent-chat": () => import("studio/controllers/agent_chat_controller"),
73
+ "voice-profile": () => import("studio/controllers/voice_profile_controller")
71
74
  }
72
75
 
73
76
  export const application = Application.start(document.documentElement, schema)
@@ -0,0 +1,529 @@
1
+ // studio/voice_profile: the voice row on /profile (studio/profiles/_voice_section).
2
+ // The signed-in person reads three short phrases; each take is recorded, turned
3
+ // into a speaker embedding ON THIS DEVICE, and thrown away. Save posts only the
4
+ // vectors; the server averages them into one voice on the person's own record
5
+ // (Studio::VoiceMatch). No audio leaves the device for this.
6
+ //
7
+ // <section data-studio-controller="voice-profile" data-voice-state="idle"
8
+ // data-voice-save-url="/profile/voice" data-voice-embedder='{"name":"wespeaker",...}'>
9
+ //
10
+ // The controller (studio/controllers/voice_profile_controller) calls
11
+ // mountVoiceProfile(element) and destroys it on disconnect. Ported from
12
+ // Voicebox's voice setup (McRitchie-Studio/voicebox#10), where the embedder, the
13
+ // level meter and the silence threshold were measured.
14
+ //
15
+ // STATES, on data-voice-state: idle (the summary) -> ready (setup open, phrase
16
+ // n waiting) -> recording -> thinking -> ready ... -> complete -> saving. Each
17
+ // phrase carries data-phrase-state: waiting, current or done. A take the meter
18
+ // heard nothing in is refused (data-voice-last-take="silent") and the phrase
19
+ // stays current. While recording, --studio-voice-level (0..1) on the root drives
20
+ // the ring around the record button.
21
+ //
22
+ // EMBEDDERS, named by the page (Studio::VoiceMatch.browser_config):
23
+ // wespeaker WeSpeaker ResNet34 (VoxCeleb), int8, through transformers.js on
24
+ // onnxruntime-web, from jsDelivr and Hugging Face at a pinned
25
+ // revision. Loaded when the setup opens; the browser caches it.
26
+ // fake Tests only: a fixed vector per voice name (window.studioFakeVoice),
27
+ // because Chrome's fake microphone cannot play chosen audio.
28
+ //
29
+ // It imports nothing statically; the model runtime is a dynamic import of the
30
+ // transformers.js URL the page names (Studio::VoiceMatch::TRANSFORMERS_URL), made
31
+ // only once someone opens the setup. test/javascript/voice_profile.test.mjs
32
+ // loads it as a data: module.
33
+
34
+ // The model hears 16 kHz mono; a take is cut to its first MAX_SECONDS.
35
+ export const SAMPLE_RATE = 16000
36
+ export const MAX_SECONDS = 10
37
+
38
+ // The loudest frame (RMS, 0..1) a take must reach to count as heard: 0.01 is
39
+ // -40 dBFS. A muted microphone sits at 0, a quiet room near 0.002-0.006, and
40
+ // speech at arm's length peaks well above 0.05.
41
+ export const SILENCE_RMS = 0.01
42
+
43
+ // A phrase is about four seconds; the take stops itself after TAKE_MS.
44
+ export const TAKE_MS = 6000
45
+
46
+ const MIME_CANDIDATES = [
47
+ "audio/webm;codecs=opus",
48
+ "audio/webm",
49
+ "audio/mp4;codecs=mp4a.40.2",
50
+ "audio/mp4",
51
+ "audio/ogg;codecs=opus"
52
+ ]
53
+
54
+ export const COPY = {
55
+ idle: ["", ""],
56
+ ready: ["Record phrase {n}", "Tap Record, then read phrase {n} out loud."],
57
+ recording: ["Stop", "Listening... read phrase {n}, then tap Stop."],
58
+ thinking: ["One moment...", "Learning your voice..."],
59
+ complete: ["All three done", "All three done. Tap Save."],
60
+ saving: ["Saving...", "Saving..."]
61
+ }
62
+
63
+ export const MESSAGES = {
64
+ silent: "I couldn't hear that one. Check your microphone and read it again.",
65
+ failed: "That one didn't work. Tap Record and read it again.",
66
+ denied: "Allow the microphone to set up your voice.",
67
+ unsupported: "This browser can't record here. Try Safari or Chrome.",
68
+ network: "We couldn't reach the server. Tap Save again.",
69
+ notSaved: "That didn't save. Tap Save again."
70
+ }
71
+
72
+ export class SilentTakeError extends Error {}
73
+
74
+ export function rootMeanSquare(samples) {
75
+ if (!samples || samples.length === 0) return 0
76
+ let sum = 0
77
+ for (let i = 0; i < samples.length; i++) sum += samples[i] * samples[i]
78
+ return Math.sqrt(sum / samples.length)
79
+ }
80
+
81
+ // The loudest 50 ms window's RMS, 0..1, at 16 kHz.
82
+ export function peakWindowRms(samples, windowSize = 800) {
83
+ let peak = 0
84
+ if (!samples) return peak
85
+ for (let start = 0; start < samples.length; start += windowSize) {
86
+ const end = Math.min(samples.length, start + windowSize)
87
+ let sum = 0
88
+ for (let i = start; i < end; i++) sum += samples[i] * samples[i]
89
+ const rms = Math.sqrt(sum / Math.max(1, end - start))
90
+ if (rms > peak) peak = rms
91
+ }
92
+ return peak
93
+ }
94
+
95
+ // The meter's reading, 0..1: logarithmic from -40 dBFS to -10 dBFS, where
96
+ // speech lives.
97
+ export function meterLevel(rms) {
98
+ if (!(rms > 0)) return 0
99
+ const db = 20 * Math.log10(rms)
100
+ return Math.min(1, Math.max(0, (db + 40) / 30))
101
+ }
102
+
103
+ // Heard, unless the meter measured the take and it stayed under SILENCE_RMS. A
104
+ // take the meter never measured (no Web Audio, a context that never ran) is
105
+ // let through, and the embedder's own check on the decoded samples decides.
106
+ export function takeWasHeard(peakRms, framesMeasured) {
107
+ if (!framesMeasured) return true
108
+ return peakRms >= SILENCE_RMS
109
+ }
110
+
111
+ export function pickMimeType(isTypeSupported) {
112
+ if (typeof isTypeSupported !== "function") return ""
113
+ return MIME_CANDIDATES.find((type) => {
114
+ try { return isTypeSupported(type) } catch (_) { return false }
115
+ }) || ""
116
+ }
117
+
118
+ // The words for a state, with {n} the phrase being read (1-based).
119
+ export function copyFor(state, step) {
120
+ const [button, status] = COPY[state] || ["Try again", ""]
121
+ const n = String(step + 1)
122
+ return { button: button.replace("{n}", n), status: status.replace("{n}", n) }
123
+ }
124
+
125
+ // Each phrase's state for a step (how many takes are kept).
126
+ export function phraseStates(total, step) {
127
+ return Array.from({ length: total }, (_, i) => (i < step ? "done" : (i === step ? "current" : "waiting")))
128
+ }
129
+
130
+ // mulberry32: a small seeded generator, so the fake's vectors are the same in
131
+ // every run and every browser.
132
+ function seededRandom(seed) {
133
+ let a = seed >>> 0
134
+ return () => {
135
+ a = (a + 0x6D2B79F5) >>> 0
136
+ let t = a
137
+ t = Math.imul(t ^ (t >>> 15), t | 1)
138
+ t ^= t + Math.imul(t ^ (t >>> 7), t | 61)
139
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296
140
+ }
141
+ }
142
+
143
+ function hashString(text) {
144
+ let h = 2166136261
145
+ for (let i = 0; i < text.length; i++) {
146
+ h ^= text.charCodeAt(i)
147
+ h = Math.imul(h, 16777619)
148
+ }
149
+ return h >>> 0
150
+ }
151
+
152
+ // The fake embedder's vector for a voice name: the same name always gives the
153
+ // same vector, and two names give nearly unrelated ones (cosine near 0).
154
+ export function fakeVector(voice, size = 256) {
155
+ const random = seededRandom(hashString(String(voice)))
156
+ return Array.from({ length: size }, () => random() * 2 - 1)
157
+ }
158
+
159
+ // A recorded Blob as 16 kHz mono floats: the browser decodes the container
160
+ // (webm/opus or mp4/aac) and resamples it while rendering offline.
161
+ export async function decodeToMono16k(blob, win = globalThis) {
162
+ const bytes = await blob.arrayBuffer()
163
+ const Offline = win.OfflineAudioContext || win.webkitOfflineAudioContext
164
+ const probe = new Offline(1, 1, SAMPLE_RATE)
165
+ const decoded = await probe.decodeAudioData(bytes)
166
+ const seconds = Math.min(decoded.duration, MAX_SECONDS)
167
+ const length = Math.max(1, Math.ceil(seconds * SAMPLE_RATE))
168
+ const offline = new Offline(1, length, SAMPLE_RATE)
169
+ const source = offline.createBufferSource()
170
+ source.buffer = decoded
171
+ source.connect(offline.destination) // a mono destination downmixes
172
+ source.start()
173
+ const rendered = await offline.startRendering()
174
+ return rendered.getChannelData(0)
175
+ }
176
+
177
+ class WespeakerEmbedder {
178
+ constructor(config) { this.config = config }
179
+
180
+ get name() { return this.config.name }
181
+ get id() { return this.config.id }
182
+
183
+ // Starts the download (runtime ~4.3 MB, model ~6.7 MB, cached after the first
184
+ // time) without waiting for it. Safe to call more than once; a failed load
185
+ // can be retried.
186
+ warm() {
187
+ if (!this.loading) {
188
+ this.loading = (async () => {
189
+ // By full URL (the page's config.runtime), never an importmap pin.
190
+ const transformers = await import(this.config.runtime)
191
+ // The model comes from the Hugging Face hub at a pinned revision, never
192
+ // from this app's own /models path.
193
+ transformers.env.allowLocalModels = false
194
+ const options = { revision: this.config.revision }
195
+ const [processor, model] = await Promise.all([
196
+ transformers.AutoProcessor.from_pretrained(this.config.model, options),
197
+ transformers.AutoModel.from_pretrained(this.config.model, { ...options, dtype: this.config.dtype })
198
+ ])
199
+ return { processor, model }
200
+ })()
201
+ this.loading.catch(() => { this.loading = null })
202
+ }
203
+ return this.loading
204
+ }
205
+
206
+ async embed(blob) {
207
+ const samples = await decodeToMono16k(blob)
208
+ if (peakWindowRms(samples) < SILENCE_RMS) throw new SilentTakeError("silent take")
209
+ const { processor, model } = await this.warm()
210
+ const inputs = await processor(samples)
211
+ const output = await model(inputs)
212
+ const tensor = output.embeddings || output.last_hidden_state || Object.values(output)[0]
213
+ return Array.from(tensor.data)
214
+ }
215
+ }
216
+
217
+ class FakeEmbedder {
218
+ constructor(config, win) {
219
+ this.config = config
220
+ this.win = win
221
+ }
222
+
223
+ get name() { return this.config.name }
224
+ get id() { return this.config.id }
225
+
226
+ warm() { return Promise.resolve() }
227
+
228
+ async embed(blob) {
229
+ if (!blob || blob.size === 0) throw new SilentTakeError("empty take")
230
+ return fakeVector((this.win && this.win.studioFakeVoice) || "default", this.config.size)
231
+ }
232
+ }
233
+
234
+ const EMBEDDERS = { wespeaker: WespeakerEmbedder, fake: FakeEmbedder }
235
+
236
+ // The embedder the page names, or null when it names none this script knows.
237
+ export function embedderFor(config, win = globalThis) {
238
+ if (!config || !config.name) return null
239
+ const Embedder = EMBEDDERS[config.name]
240
+ return Embedder ? new Embedder(config, win) : null
241
+ }
242
+
243
+ function parseJson(text) {
244
+ try { return JSON.parse(text || "null") } catch (_) { return null }
245
+ }
246
+
247
+ // Binds the voice row. Returns { destroy }.
248
+ export function mountVoiceProfile(root, win = window) {
249
+ const doc = root.ownerDocument
250
+ const phrases = () => Array.from(root.querySelectorAll("[data-voice-phrase]"))
251
+ const one = (selector) => root.querySelector(selector)
252
+ const embedder = embedderFor(parseJson(root.getAttribute("data-voice-embedder")), win)
253
+ const saveUrl = root.getAttribute("data-voice-save-url")
254
+
255
+ let vectors = []
256
+ let state = "idle"
257
+ let stream = null
258
+ let recorder = null
259
+ let chunks = []
260
+ let timer = null
261
+ let audioContext = null
262
+ let analyser = null
263
+ let levelBuffer = null
264
+ let meterFrame = null
265
+ let meterTimer = null
266
+ let peakRms = 0
267
+ let frames = 0
268
+ let starting = false
269
+ let destroyed = false
270
+
271
+ function setState(next, message = null) {
272
+ state = next
273
+ root.dataset.voiceState = next
274
+ const states = phraseStates(phrases().length, vectors.length)
275
+ phrases().forEach((phrase, i) => { phrase.dataset.phraseState = states[i] })
276
+ const words = copyFor(next, vectors.length)
277
+ const label = one("[data-voice-record-label]")
278
+ if (label && words.button) label.textContent = words.button
279
+ const status = one("[data-voice-status]")
280
+ if (status) status.textContent = message || words.status
281
+ const button = one("[data-voice-record]")
282
+ if (button) {
283
+ button.disabled = !["ready", "recording"].includes(next)
284
+ button.setAttribute("aria-pressed", next === "recording" ? "true" : "false")
285
+ }
286
+ const save = one("[data-voice-save]")
287
+ if (save) save.disabled = next !== "complete"
288
+ }
289
+
290
+ function settle(message = null) {
291
+ setState(vectors.length >= phrases().length ? "complete" : "ready", message)
292
+ }
293
+
294
+ function supported() {
295
+ return !!(win.MediaRecorder && win.navigator && win.navigator.mediaDevices && win.navigator.mediaDevices.getUserMedia)
296
+ }
297
+
298
+ function open() {
299
+ if (state !== "idle") return
300
+ vectors = []
301
+ const setup = one("[data-voice-setup]")
302
+ if (setup) setup.hidden = false
303
+ const summary = one("[data-voice-summary]")
304
+ if (summary) summary.hidden = true
305
+ if (!embedder || !supported()) {
306
+ setState("ready", MESSAGES.unsupported)
307
+ const button = one("[data-voice-record]")
308
+ if (button) button.disabled = true
309
+ return
310
+ }
311
+ embedder.warm().catch(() => {})
312
+ setState("ready")
313
+ }
314
+
315
+ function cancel() {
316
+ stopTake(false)
317
+ vectors = []
318
+ const setup = one("[data-voice-setup]")
319
+ if (setup) setup.hidden = true
320
+ const summary = one("[data-voice-summary]")
321
+ if (summary) summary.hidden = false
322
+ setState("idle")
323
+ }
324
+
325
+ function paintLevel(level) {
326
+ root.style.setProperty("--studio-voice-level", level.toFixed(3))
327
+ root.dataset.voiceLevel = level.toFixed(2)
328
+ }
329
+
330
+ function sampleLevel() {
331
+ if (!analyser || !audioContext || audioContext.state !== "running") return
332
+ analyser.getFloatTimeDomainData(levelBuffer)
333
+ const rms = rootMeanSquare(levelBuffer)
334
+ frames += 1
335
+ if (rms > peakRms) {
336
+ peakRms = rms
337
+ root.dataset.voicePeak = rms.toFixed(4)
338
+ }
339
+ if (rms >= SILENCE_RMS) root.dataset.voiceHeard = "true"
340
+ paintLevel(meterLevel(rms))
341
+ }
342
+
343
+ function startMeter(source) {
344
+ peakRms = 0
345
+ frames = 0
346
+ root.dataset.voiceHeard = "false"
347
+ root.dataset.voicePeak = "0"
348
+ paintLevel(0)
349
+ const AudioContextClass = win.AudioContext || win.webkitAudioContext
350
+ if (!AudioContextClass) return
351
+ try {
352
+ audioContext = new AudioContextClass()
353
+ const input = audioContext.createMediaStreamSource(source)
354
+ analyser = audioContext.createAnalyser()
355
+ analyser.fftSize = 1024
356
+ levelBuffer = new Float32Array(analyser.fftSize)
357
+ input.connect(analyser)
358
+ if (audioContext.state === "suspended" && audioContext.resume) audioContext.resume().catch(() => {})
359
+ } catch (_) {
360
+ closeMeter()
361
+ return
362
+ }
363
+ const frame = () => {
364
+ sampleLevel()
365
+ meterFrame = win.requestAnimationFrame(frame)
366
+ }
367
+ meterFrame = win.requestAnimationFrame(frame)
368
+ // A background tab stops animation frames; the timer keeps measuring.
369
+ meterTimer = win.setInterval(sampleLevel, 100)
370
+ }
371
+
372
+ function closeMeter() {
373
+ if (meterFrame) win.cancelAnimationFrame(meterFrame)
374
+ if (meterTimer) win.clearInterval(meterTimer)
375
+ meterFrame = null
376
+ meterTimer = null
377
+ analyser = null
378
+ if (audioContext && audioContext.state !== "closed") audioContext.close().catch(() => {})
379
+ audioContext = null
380
+ paintLevel(0)
381
+ }
382
+
383
+ function release() {
384
+ if (stream) stream.getTracks().forEach((track) => track.stop())
385
+ stream = null
386
+ }
387
+
388
+ async function startTake() {
389
+ try {
390
+ stream = await win.navigator.mediaDevices.getUserMedia({ audio: true })
391
+ } catch (error) {
392
+ setState("ready", error && error.name === "NotAllowedError" ? MESSAGES.denied : MESSAGES.failed)
393
+ return
394
+ }
395
+ if (destroyed || state !== "ready") { release(); return }
396
+ chunks = []
397
+ try {
398
+ startMeter(stream)
399
+ const MediaRecorderClass = win.MediaRecorder
400
+ const mimeType = pickMimeType(MediaRecorderClass.isTypeSupported && MediaRecorderClass.isTypeSupported.bind(MediaRecorderClass))
401
+ try {
402
+ recorder = mimeType ? new MediaRecorderClass(stream, { mimeType }) : new MediaRecorderClass(stream)
403
+ } catch (_) {
404
+ recorder = new MediaRecorderClass(stream)
405
+ }
406
+ recorder.addEventListener("dataavailable", (event) => {
407
+ if (event.data && event.data.size > 0) chunks.push(event.data)
408
+ })
409
+ recorder.addEventListener("stop", () => finishTake())
410
+ recorder.start(250)
411
+ } catch (_) {
412
+ closeMeter()
413
+ release()
414
+ setState("ready", MESSAGES.failed)
415
+ return
416
+ }
417
+ setState("recording")
418
+ timer = win.setTimeout(() => stopTake(true), TAKE_MS)
419
+ }
420
+
421
+ function stopTake(keep = true) {
422
+ if (timer) win.clearTimeout(timer)
423
+ timer = null
424
+ if (!recorder || recorder.state === "inactive") return
425
+ if (!keep) recorder.discard = true
426
+ recorder.stop()
427
+ }
428
+
429
+ async function finishTake() {
430
+ const discarded = recorder && recorder.discard
431
+ sampleLevel()
432
+ const heard = takeWasHeard(peakRms, frames)
433
+ closeMeter()
434
+ release()
435
+ const type = (recorder && recorder.mimeType) || "audio/webm"
436
+ const blob = new Blob(chunks, { type })
437
+ chunks = []
438
+ recorder = null
439
+ if (discarded || destroyed) return
440
+ if (!heard || blob.size === 0) {
441
+ root.dataset.voiceLastTake = "silent"
442
+ settle(MESSAGES.silent)
443
+ return
444
+ }
445
+ setState("thinking")
446
+ try {
447
+ const vector = await embedder.embed(blob)
448
+ if (destroyed || state !== "thinking") return
449
+ vectors.push(vector)
450
+ root.dataset.voiceLastTake = "kept"
451
+ settle()
452
+ } catch (error) {
453
+ if (destroyed || state !== "thinking") return
454
+ const silent = error instanceof SilentTakeError
455
+ root.dataset.voiceLastTake = silent ? "silent" : "failed"
456
+ settle(silent ? MESSAGES.silent : MESSAGES.failed)
457
+ }
458
+ }
459
+
460
+ function toggle() {
461
+ if (state === "recording") stopTake(true)
462
+ else if (state === "ready" && !starting) {
463
+ starting = true
464
+ startTake().finally(() => { starting = false })
465
+ }
466
+ }
467
+
468
+ function csrfToken() {
469
+ const meta = doc.querySelector("meta[name='csrf-token']")
470
+ return meta ? meta.content : ""
471
+ }
472
+
473
+ async function save() {
474
+ if (state !== "complete") return
475
+ setState("saving")
476
+ let response
477
+ let body
478
+ try {
479
+ response = await win.fetch(saveUrl, {
480
+ method: "POST",
481
+ credentials: "same-origin",
482
+ headers: { "Accept": "application/json", "Content-Type": "application/json", "X-CSRF-Token": csrfToken() },
483
+ body: JSON.stringify({ voice: { model: embedder.id, embeddings: vectors } })
484
+ })
485
+ body = await response.json().catch(() => null)
486
+ } catch (_) {
487
+ setState("complete", MESSAGES.network)
488
+ return
489
+ }
490
+ // fetch resolves on a 4xx or 5xx: only an ok answer with a url is saved.
491
+ if (!response.ok || !body || !body.url) {
492
+ setState("complete", (body && body.error) || MESSAGES.notSaved)
493
+ return
494
+ }
495
+ vectors = []
496
+ root.dataset.voiceSaved = "true"
497
+ win.location.assign(body.url)
498
+ }
499
+
500
+ const handlers = {
501
+ open: () => open(),
502
+ cancel: () => cancel(),
503
+ record: () => toggle(),
504
+ save: () => save()
505
+ }
506
+
507
+ function onClick(event) {
508
+ const target = event.target.closest && event.target.closest("[data-voice-action]")
509
+ if (!target || !root.contains(target)) return
510
+ const handler = handlers[target.getAttribute("data-voice-action")]
511
+ if (!handler) return
512
+ event.preventDefault()
513
+ handler()
514
+ }
515
+
516
+ root.addEventListener("click", onClick)
517
+ root.dataset.voiceReady = "true"
518
+ setState("idle")
519
+
520
+ return {
521
+ destroy() {
522
+ destroyed = true
523
+ root.removeEventListener("click", onClick)
524
+ stopTake(false)
525
+ closeMeter()
526
+ release()
527
+ }
528
+ }
529
+ }
@@ -0,0 +1,108 @@
1
+ <%# The voice row: "Set up your voice". Rendered only when Studio.voice_profile
2
+ is on and the user has the voice columns (lib/studio/profile_sections.rb).
3
+
4
+ Locals: user (required); embedder (optional, defaults to
5
+ Studio.voice_embedder: the browser e2e lab passes "fake" rather than write
6
+ the process-wide setting).
7
+
8
+ THE DEVICE DOES THE LISTENING. Three phrases are recorded one at a time;
9
+ studio/voice_profile turns each take into a speaker embedding in the browser,
10
+ drops the audio, and Save posts only the vectors to /profile/voice, where
11
+ Studio::VoiceMatch averages them into one voice on this user. Nothing here
12
+ uploads a recording.
13
+
14
+ WITHOUT JAVASCRIPT the row still says whether a voice is saved and still
15
+ deletes it: the summary and the Delete form are plain HTML. Only recording
16
+ needs the controller (lazy, studio/stimulus). If it fails to load, the row
17
+ carries data-studio-controller-failed="voice-profile" and says so.
18
+
19
+ SCOPED CSS rather than utilities, as in _identity_styles: the meter ring and
20
+ the phrase marks must render in an app whose Tailwind build never saw this
21
+ file. The ring reads --studio-voice-level (0..1), which the module writes
22
+ while a take records. %>
23
+ <% embedder = local_assigns.fetch(:embedder, Studio.voice_embedder).to_s %>
24
+ <% config = Studio::VoiceMatch.browser_config(embedder) %>
25
+ <% saved = user.voice_embedding.present? %>
26
+ <% current = saved && user.voice_embedding_model.to_s == config["id"] %>
27
+
28
+ <style>
29
+ .studio-voice-consent { font-size: .8125rem; color: var(--color-text-secondary, inherit); margin-bottom: 1rem; }
30
+ .studio-voice-lead { margin-bottom: .5rem; }
31
+ .studio-voice-row { display: flex; align-items: center; justify-content: space-between; gap: .75rem; flex-wrap: wrap; }
32
+ .studio-voice-actions { display: flex; align-items: center; gap: .5rem; flex-wrap: wrap; }
33
+ .studio-voice-phrases { list-style: none; padding: 0; margin: 0 0 1rem; display: grid; gap: .5rem; }
34
+ .studio-voice-phrase { display: flex; align-items: center; gap: .75rem; padding: .75rem; border-radius: .75rem;
35
+ border: 1px solid var(--color-border, rgb(127 127 127 / .25)); }
36
+ .studio-voice-phrase[data-phrase-state="current"] { outline: 2px solid rgb(var(--color-primary-500-rgb, 99 102 241)); outline-offset: 1px; }
37
+ .studio-voice-mark { flex-shrink: 0; width: 1.75rem; height: 1.75rem; border-radius: 9999px; display: flex;
38
+ align-items: center; justify-content: center; font-size: .8125rem; font-weight: 700;
39
+ border: 1px solid var(--color-border, rgb(127 127 127 / .25)); color: var(--color-text-secondary, inherit); }
40
+ .studio-voice-phrase[data-phrase-state="done"] .studio-voice-mark { background: #10b981; border-color: #10b981; color: #fff; }
41
+ .studio-voice-controls { display: flex; flex-direction: column; align-items: center; gap: .75rem; text-align: center; }
42
+ .studio-voice-record { min-width: 12rem; border-radius: 9999px; padding: .875rem 1.75rem; font-weight: 700;
43
+ color: #fff; background: rgb(var(--color-primary-500-rgb, 99 102 241));
44
+ box-shadow: 0 0 0 calc(var(--studio-voice-level, 0) * 14px) rgb(var(--color-primary-500-rgb, 99 102 241) / .3);
45
+ transition: box-shadow 80ms linear; }
46
+ .studio-voice-record:disabled { opacity: .5; cursor: not-allowed; }
47
+ [data-voice-state="recording"] .studio-voice-record { background: #dc2626; }
48
+ @media (prefers-reduced-motion: reduce) { .studio-voice-record { transition: none; } }
49
+ .studio-voice-status { min-height: 2.5rem; max-width: 24rem; color: var(--color-text-body, inherit); }
50
+ .studio-voice-failed { display: none; font-size: .8125rem; color: var(--color-text-secondary, inherit); }
51
+ [data-studio-controller-failed~="voice-profile"] .studio-voice-failed { display: block; }
52
+ </style>
53
+
54
+ <div data-studio-controller="voice-profile"
55
+ data-voice-state="idle"
56
+ data-voice-save-url="<%= profile_voice_path %>"
57
+ data-voice-embedder="<%= config.to_json %>">
58
+ <p class="studio-voice-consent" data-voice-consent>
59
+ This saves a voice pattern, not a recording. Your device listens to three short phrases and keeps
60
+ only numbers that describe how you sound, so your agent can tell it's you. Delete it any time.
61
+ </p>
62
+
63
+ <div class="studio-voice-row" data-voice-summary>
64
+ <p class="text-body min-w-0" data-voice-saved="<%= current ? "current" : (saved ? "stale" : "none") %>">
65
+ <% if current %>
66
+ Your voice is set up<% if user.voice_embedding_updated_at %> (<%= user.voice_embedding_updated_at.to_date.to_fs(:long) %>)<% end %>.
67
+ <% elsif saved %>
68
+ Your saved voice came from an older model. Record it again to keep using it.
69
+ <% else %>
70
+ Read three short phrases so your agent knows your voice.
71
+ <% end %>
72
+ </p>
73
+ <div class="studio-voice-actions">
74
+ <button type="button" class="btn btn-primary btn-sm" data-voice-action="open">
75
+ <%= saved ? "Re-record" : "Set up your voice" %>
76
+ </button>
77
+ <% if saved %>
78
+ <%= button_to "Delete", profile_voice_path, method: :delete,
79
+ class: "btn btn-sm", form: { class: "inline", data: { voice_delete: true } } %>
80
+ <% end %>
81
+ </div>
82
+ </div>
83
+ <p class="studio-voice-failed">Recording didn't load on this page. Reload to try again.</p>
84
+
85
+ <div data-voice-setup hidden>
86
+ <p class="studio-voice-lead text-sm font-semibold text-heading">Read each one out loud</p>
87
+ <ol class="studio-voice-phrases">
88
+ <% Studio::VoiceMatch::PHRASES.each_with_index do |phrase, i| %>
89
+ <li class="studio-voice-phrase" data-voice-phrase data-phrase-state="waiting">
90
+ <span class="studio-voice-mark" aria-hidden="true"><%= i + 1 %></span>
91
+ <span class="text-body">&ldquo;<%= phrase %>&rdquo;</span>
92
+ </li>
93
+ <% end %>
94
+ </ol>
95
+
96
+ <div class="studio-voice-controls">
97
+ <button type="button" class="studio-voice-record" data-voice-action="record" data-voice-record
98
+ aria-pressed="false" disabled>
99
+ <span data-voice-record-label>Record phrase 1</span>
100
+ </button>
101
+ <p class="studio-voice-status" data-voice-status role="status" aria-live="polite"></p>
102
+ <div class="studio-voice-actions">
103
+ <button type="button" class="btn btn-primary btn-sm" data-voice-action="save" data-voice-save disabled>Save</button>
104
+ <button type="button" class="btn btn-sm" data-voice-action="cancel">Cancel</button>
105
+ </div>
106
+ </div>
107
+ </div>
108
+ </div>
@@ -0,0 +1,44 @@
1
+ # The voice row on /profile (Studio.voice_profile): one speaker embedding on the
2
+ # user's own record, the model it came from, and when it was saved.
3
+ #
4
+ # A HOST-TABLE MIGRATION, the engine's second after
5
+ # add_standard_user_profile_columns, and built the same way for the same reason:
6
+ # one definition the apps install (`bin/rails studio_engine:install:migrations`)
7
+ # instead of a hand-written copy per app.
8
+ #
9
+ # voice_embedding jsonb the averaged unit vector (256 floats for
10
+ # the wespeaker embedder). A pattern of the
11
+ # voice, never audio: the takes stay on the
12
+ # device.
13
+ # voice_embedding_model string which embedder and revision made it
14
+ # (Studio::VoiceMatch::EMBEDDERS ids). Vectors
15
+ # from different models are not comparable,
16
+ # so the matcher filters on this.
17
+ # voice_embedding_updated_at datetime
18
+ #
19
+ # Every add is `if_not_exists`, and an app with no users table is skipped, as in
20
+ # the standard-columns migration.
21
+ #
22
+ # THE DOWN DROPS THEM, unlike the standard-columns migration, which refuses. That
23
+ # one could not tell its columns from columns the host already owned
24
+ # (first_name, birth_year). These three names are the engine's own coinage, and
25
+ # no consumer had any of them when this shipped (mcritchie-studio,
26
+ # turf-monster, mcritchie-industries, cyvasse and moms-app checked 2026-10-29),
27
+ # so a drop removes only what this migration added.
28
+ class AddVoiceProfileToUsers < ActiveRecord::Migration[7.2]
29
+ def up
30
+ return unless table_exists?(:users)
31
+
32
+ add_column :users, :voice_embedding, :jsonb, if_not_exists: true
33
+ add_column :users, :voice_embedding_model, :string, if_not_exists: true
34
+ add_column :users, :voice_embedding_updated_at, :datetime, if_not_exists: true
35
+ end
36
+
37
+ def down
38
+ return unless table_exists?(:users)
39
+
40
+ remove_column :users, :voice_embedding_updated_at, if_exists: true
41
+ remove_column :users, :voice_embedding_model, if_exists: true
42
+ remove_column :users, :voice_embedding, if_exists: true
43
+ end
44
+ end
data/lib/studio/engine.rb CHANGED
@@ -299,6 +299,14 @@ module Studio
299
299
  app.config.middleware.use Studio::GoogleSignIn::Middleware
300
300
  end
301
301
 
302
+ # The voice row's vectors stay out of every consumer's request log and
303
+ # User#inspect: POST /profile/voice carries voice[embeddings], and the user
304
+ # holds voice_embedding (Voicebox logged vectors until it filtered them).
305
+ # `(\A|\.)` also masks the JSON param wrapper's copy (profile.voice.embeddings).
306
+ initializer "studio.voice_profile.filter_parameters" do |app|
307
+ app.config.filter_parameters += [/(\A|\.)voice\.embeddings\z/, /\Avoice_embedding\z/]
308
+ end
309
+
302
310
  initializer "studio.footer_only_eager_load" do
303
311
  next if Studio.active_record?
304
312
 
@@ -6,6 +6,7 @@
6
6
  # and the engine fails to boot. Declaring the dependency where it is used makes
7
7
  # the pair immune to a future reorder of that require list.
8
8
  require_relative "newsletter"
9
+ require_relative "voice_match"
9
10
 
10
11
  # Resolves Studio.profile_sections — the rows that make up the shared /profile
11
12
  # page — for a given view context. Pure Ruby (no Rails dependency) so the unit
@@ -100,6 +101,16 @@ module Studio
100
101
  requires: Studio::Newsletter::COLUMNS,
101
102
  modals: "studio/profiles/newsletter_modals" },
102
103
 
104
+ # The voice: three phrases read once, so an agent can tell who is
105
+ # talking. OPT-IN (Studio.voice_profile), because it fetches a speaker
106
+ # model from Hugging Face and stores a voice pattern, and a host's CSP and
107
+ # privacy page must say so first. Gated on the columns too: the engine
108
+ # migration add_voice_profile_to_users installs them.
109
+ { key: :voice, title: "Your voice", page: :show,
110
+ partial: "studio/profiles/voice_section",
111
+ requires: Studio::VoiceMatch::USER_COLUMNS,
112
+ if: -> { Studio.voice_profile } },
113
+
103
114
  # --- the edit page ------------------------------------------------------
104
115
  # These are FIELDS in one form with one Save, so they carry no buttons of
105
116
  # their own. Email included: it needed a separate action only while it was
@@ -1,3 +1,3 @@
1
1
  module Studio
2
- VERSION = "0.119.1"
2
+ VERSION = "0.120.0"
3
3
  end
@@ -0,0 +1,238 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+
5
+ # Who is speaking: compares one take's speaker embedding with the voice
6
+ # embeddings saved on users' own records (the voice row on /profile, opt-in with
7
+ # Studio.voice_profile). Pure arithmetic on arrays of floats, with no Rails
8
+ # dependency, so the threshold logic is unit-tested without a browser, a model or
9
+ # a database.
10
+ #
11
+ # THE EMBEDDINGS COME FROM THE BROWSER. The profile page records three short
12
+ # phrases, turns each take into a vector on the person's own device
13
+ # (app/javascript/studio/voice_profile.js), drops the audio, and posts only the
14
+ # vectors. The server averages them here and stores one vector on the user. No
15
+ # audio is ever uploaded for this.
16
+ #
17
+ # Ported from Voicebox's SpeakerMatch (McRitchie-Studio/voicebox#10), where the
18
+ # embedder, the threshold and the averaging were measured.
19
+ module Studio
20
+ module VoiceMatch
21
+ # The speaker embedders the browser may run, by name. `id` is what is stored
22
+ # on the user beside the vector (users.voice_embedding_model): vectors from
23
+ # two models, or from two revisions of one model, are not comparable, so the
24
+ # matcher only ever compares a take with users whose id is the same.
25
+ #
26
+ # "wespeaker" is WeSpeaker's ResNet34 speaker model (trained on VoxCeleb),
27
+ # int8-quantized, run by transformers.js on onnxruntime-web. The model is
28
+ # about 6.7 MB and the runtime about 4.3 MB; the browser caches both after the
29
+ # first load. Its revision is pinned so THRESHOLD always describes the vectors
30
+ # it was tuned on. Changing the revision is changing the id, and every saved
31
+ # voice stops matching until its owner re-records.
32
+ #
33
+ # "fake" is for tests only: a deterministic vector seeded by a voice name the
34
+ # test sets on the page (window.studioFakeVoice), because Chrome's fake
35
+ # microphone cannot play chosen audio.
36
+ TRANSFORMERS_VERSION = "3.7.6"
37
+
38
+ # Where the browser imports transformers.js from, by full URL. NOT an
39
+ # importmap pin: every pin a host draws must be a fingerprinted local asset
40
+ # (the hub's AssetUrlsTest), and a ~1 MB runtime that fetches its own
41
+ # onnxruntime WebAssembly from the same jsDelivr path anyway gains nothing
42
+ # from being vendored. A host's CSP names this package path.
43
+ TRANSFORMERS_URL =
44
+ "https://cdn.jsdelivr.net/npm/@huggingface/transformers@#{TRANSFORMERS_VERSION}/dist/transformers.min.js"
45
+
46
+ EMBEDDERS = {
47
+ "wespeaker" => {
48
+ "id" => "wespeaker-voxceleb-resnet34-LM@6a61a18",
49
+ "size" => 256,
50
+ "model" => "onnx-community/wespeaker-voxceleb-resnet34-LM",
51
+ "revision" => "6a61a1833ff2583aabeba044f5c8221f00b67ceb",
52
+ "dtype" => "q8",
53
+ "runtime" => TRANSFORMERS_URL
54
+ }.freeze,
55
+ "fake" => {
56
+ "id" => "fake-256",
57
+ "size" => 256
58
+ }.freeze
59
+ }.freeze
60
+
61
+ # The columns the voice lives in on the host's users table, installed by the
62
+ # engine migration add_voice_profile_to_users.
63
+ USER_COLUMNS = %i[voice_embedding voice_embedding_model voice_embedding_updated_at].freeze
64
+
65
+ # What the setup asks the person to read: three short, phonetically varied
66
+ # sentences of about four seconds each. TAKES is their count, and the server
67
+ # refuses a save with any other number of vectors.
68
+ PHRASES = [
69
+ "The quick brown fox jumps over the lazy dog.",
70
+ "I'd like a cup of coffee and a slice of toast, please.",
71
+ "My voice is my own, and today is a good day to talk."
72
+ ].freeze
73
+
74
+ TAKES = PHRASES.size
75
+
76
+ # The cosine similarity a take must reach to be named. Measured in Voicebox
77
+ # (2026-10-29) on the wespeaker embedder, enrolling five synthetic voices from
78
+ # three phrases each and probing with a fourth: same voice 0.85 - 0.91,
79
+ # different voice 0.04 - 0.53 (the closest pair two adult women). 0.6 sits
80
+ # above every different-voice score and well under every same-voice one.
81
+ THRESHOLD = 0.6
82
+
83
+ # user: the closest user at or above the threshold, else nil. score: that
84
+ # closest similarity, or nil when there was nothing to compare.
85
+ Result = Struct.new(:user, :score, keyword_init: true) do
86
+ def unknown? = user.nil?
87
+ end
88
+
89
+ NONE = Result.new(user: nil, score: nil).freeze
90
+
91
+ module_function
92
+
93
+ # The settings the page hands the browser for one embedder.
94
+ def browser_config(name)
95
+ config = EMBEDDERS.fetch(name.to_s)
96
+ { "name" => name.to_s }.merge(config)
97
+ end
98
+
99
+ def embedder!(name)
100
+ EMBEDDERS.fetch(name.to_s) do
101
+ raise ArgumentError, "Studio.voice_embedder must be one of #{EMBEDDERS.keys.join(', ')} (got #{name.inspect})"
102
+ end
103
+ end
104
+
105
+ def model_id(name) = embedder!(name).fetch("id")
106
+
107
+ def size(name) = embedder!(name).fetch("size")
108
+
109
+ # The closest user to `embedding` among `among`, or nil when nobody reaches
110
+ # the threshold. Only users whose saved vector came from `model` are
111
+ # compared. `among` is anything enumerable of users; an ActiveRecord scope is
112
+ # narrowed to that model in SQL first.
113
+ def identify(embedding, among:, model:, threshold: THRESHOLD)
114
+ match(embedding, among: among, model: model, threshold: threshold).user
115
+ end
116
+
117
+ # identify, with the score: a caller that wants to say "someone new" needs to
118
+ # tell "nobody close" (a score below the threshold) from "nothing to compare".
119
+ def match(embedding, among:, model:, threshold: THRESHOLD)
120
+ probe = vector(embedding)
121
+ return NONE if probe.nil? || model.to_s.empty?
122
+
123
+ best = nil
124
+ best_score = nil
125
+ candidates(among, model).each do |user|
126
+ next unless user.respond_to?(:voice_embedding_model) && user.voice_embedding_model.to_s == model.to_s
127
+
128
+ saved = vector(user.voice_embedding)
129
+ next if saved.nil? || saved.size != probe.size
130
+
131
+ score = cosine(probe, saved)
132
+ next if score.nil?
133
+
134
+ if best_score.nil? || score > best_score
135
+ best = user
136
+ best_score = score
137
+ end
138
+ end
139
+ return NONE if best_score.nil?
140
+
141
+ Result.new(user: best_score >= threshold ? best : nil, score: best_score.round(4))
142
+ end
143
+
144
+ # Cosine similarity, -1..1; nil when the sizes differ or either vector has
145
+ # no length.
146
+ def cosine(a, b)
147
+ return nil unless a.size == b.size
148
+
149
+ dot = 0.0
150
+ norm_a = 0.0
151
+ norm_b = 0.0
152
+ a.each_with_index do |x, i|
153
+ y = b[i]
154
+ dot += x * y
155
+ norm_a += x * x
156
+ norm_b += y * y
157
+ end
158
+ return nil if norm_a.zero? || norm_b.zero?
159
+
160
+ dot / Math.sqrt(norm_a * norm_b)
161
+ end
162
+
163
+ # One profile vector from the setup takes: each take scaled to unit length,
164
+ # so a loud phrase does not outweigh a quiet one, then averaged and scaled to
165
+ # unit length again. nil when no take has any length (silence embeds to
166
+ # zeros), or the takes disagree on size.
167
+ def average(vectors)
168
+ units = Array(vectors).map { |v| normalize(v) }.compact
169
+ return nil if units.empty?
170
+ return nil unless units.all? { |u| u.size == units.first.size }
171
+
172
+ sum = Array.new(units.first.size, 0.0)
173
+ units.each { |u| u.each_with_index { |x, i| sum[i] += x } }
174
+ normalize(sum)
175
+ end
176
+
177
+ def normalize(vector)
178
+ return nil if vector.nil? || vector.empty?
179
+
180
+ norm = Math.sqrt(vector.sum { |x| x * x })
181
+ return nil if norm.zero? || !norm.finite?
182
+
183
+ vector.map { |x| x / norm }
184
+ end
185
+
186
+ # A vector the browser sent, as floats, or nil when it is not `size` finite
187
+ # numbers. Accepts a JSON string or an already-parsed array.
188
+ def parse(raw, size:)
189
+ values = raw.is_a?(String) ? JSON.parse(raw) : raw
190
+ return nil unless values.is_a?(Array) && values.size == size
191
+ return nil unless values.all? { |x| x.is_a?(Numeric) && x.to_f.finite? }
192
+
193
+ values.map(&:to_f)
194
+ rescue JSON::ParserError
195
+ nil
196
+ end
197
+
198
+ # The setup's takes, validated: exactly TAKES vectors of `size` finite
199
+ # numbers, each with some length. Returns the averaged unit vector, or nil.
200
+ def enrol(raw_takes, size:)
201
+ takes = raw_takes.is_a?(String) ? JSON.parse(raw_takes) : raw_takes
202
+ return nil unless takes.is_a?(Array) && takes.size == TAKES
203
+
204
+ vectors = takes.map { |take| parse(take, size: size) }
205
+ return nil if vectors.any?(&:nil?)
206
+ return nil if vectors.any? { |v| normalize(v).nil? }
207
+
208
+ average(vectors)
209
+ rescue JSON::ParserError
210
+ nil
211
+ end
212
+
213
+ # A stored embedding (jsonb array, or a JSON string on an adapter without
214
+ # json columns) as floats, or nil.
215
+ def vector(raw)
216
+ values = raw.is_a?(String) ? JSON.parse(raw) : raw
217
+ return nil unless values.is_a?(Array) && !values.empty?
218
+ return nil unless values.all? { |x| x.is_a?(Numeric) && x.to_f.finite? }
219
+
220
+ values.map(&:to_f)
221
+ rescue JSON::ParserError
222
+ nil
223
+ end
224
+
225
+ def candidates(among, model)
226
+ return Array(among) unless among.respond_to?(:where)
227
+
228
+ # A scope or the model class itself. Narrowed in SQL only when the table
229
+ # has the column; the per-user check in #match holds either way.
230
+ klass = among.respond_to?(:klass) ? among.klass : among
231
+ if klass.respond_to?(:column_names) && klass.column_names.include?("voice_embedding_model")
232
+ among.where(voice_embedding_model: model.to_s)
233
+ else
234
+ among
235
+ end
236
+ end
237
+ end
238
+ end
data/lib/studio.rb CHANGED
@@ -46,6 +46,7 @@ require "studio/loader"
46
46
  require "studio/auth_labels"
47
47
  require "studio/profile_sections"
48
48
  require "studio/profile_image"
49
+ require "studio/voice_match"
49
50
  require "studio/oauth_identity"
50
51
  require "studio/google_sign_in"
51
52
  require "studio/newsletter"
@@ -556,6 +557,25 @@ module Studio
556
557
  # wants the page gone still sets this false.
557
558
  mattr_accessor :draw_profile_routes, default: true
558
559
 
560
+ # The voice row on /profile: "Set up your voice". The signed-in user reads
561
+ # three short phrases; each take becomes a speaker embedding ON THEIR DEVICE
562
+ # (studio/voice_profile, a WeSpeaker model run by transformers.js), the audio
563
+ # is dropped, and only the vectors are posted. The server averages them into
564
+ # one vector on the user's own record (users.voice_embedding, installed by the
565
+ # engine migration add_voice_profile_to_users). Studio::VoiceMatch.identify
566
+ # answers "who is speaking" from an embedding.
567
+ #
568
+ # OFF BY DEFAULT. Turning it on is a privacy decision (a voice pattern is
569
+ # biometric data in several jurisdictions) and it makes the page fetch a model
570
+ # from Hugging Face and its runtime from jsDelivr, which a host's CSP and
571
+ # privacy page must both name. docs/VOICE_PROFILE.md lists what to change.
572
+ mattr_accessor :voice_profile, default: false
573
+
574
+ # Which speaker embedder the browser runs: "wespeaker" (the real one) or
575
+ # "fake" (tests only; a fixed vector per window.studioFakeVoice). The server
576
+ # refuses a voice saved by any other embedder, so a page cannot pick its own.
577
+ mattr_accessor :voice_embedder, default: "wespeaker"
578
+
559
579
  # How long a freshly minted magic link stays live.
560
580
  mattr_accessor :magic_link_ttl, default: 15.minutes
561
581
 
@@ -1926,6 +1946,13 @@ module Studio
1926
1946
  post "profile/newsletter", to: "studio/profiles#subscribe_newsletter",
1927
1947
  as: :profile_newsletter
1928
1948
  delete "profile/newsletter", to: "studio/profiles#unsubscribe_newsletter"
1949
+
1950
+ # The voice row. POST saves (or replaces) the voice from the setup's
1951
+ # vectors; DELETE removes it. Drawn with the page, like the newsletter
1952
+ # pair; the actions refuse unless the row renders (Studio.voice_profile
1953
+ # on, and the user has the columns).
1954
+ post "profile/voice", to: "studio/profiles#save_voice", as: :profile_voice
1955
+ delete "profile/voice", to: "studio/profiles#delete_voice"
1929
1956
  end
1930
1957
 
1931
1958
  resources :error_logs, only: [:index, :show]
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: studio-engine
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.119.1
4
+ version: 0.120.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Alex McRitchie
@@ -512,6 +512,7 @@ files:
512
512
  - app/javascript/studio/controllers/style_modals_controller.js
513
513
  - app/javascript/studio/controllers/survey_controller.js
514
514
  - app/javascript/studio/controllers/toast_controller.js
515
+ - app/javascript/studio/controllers/voice_profile_controller.js
515
516
  - app/javascript/studio/cropper.js
516
517
  - app/javascript/studio/email_banner.js
517
518
  - app/javascript/studio/email_banner_scale.js
@@ -545,6 +546,7 @@ files:
545
546
  - app/javascript/studio/theme_editor.js
546
547
  - app/javascript/studio/toast.js
547
548
  - app/javascript/studio/vendor/stimulus.js
549
+ - app/javascript/studio/voice_profile.js
548
550
  - app/jobs/error_log_cleanup_job.rb
549
551
  - app/jobs/studio/email_delivery_job.rb
550
552
  - app/jobs/studio/health_report_job.rb
@@ -738,6 +740,7 @@ files:
738
740
  - app/views/studio/profiles/_pencil_icon.html.erb
739
741
  - app/views/studio/profiles/_plain_save.html.erb
740
742
  - app/views/studio/profiles/_save_controls.html.erb
743
+ - app/views/studio/profiles/_voice_section.html.erb
741
744
  - app/views/studio/profiles/edit.html.erb
742
745
  - app/views/studio/profiles/show.html.erb
743
746
  - app/views/studio/public_users/not_found.html.erb
@@ -806,6 +809,7 @@ files:
806
809
  - db/migrate/20261006120003_ensure_image_caches_table.rb
807
810
  - db/migrate/20261008120000_add_recording_to_studio_knowledge_docs.rb
808
811
  - db/migrate/20261010120000_add_stored_in_to_image_caches.rb
812
+ - db/migrate/20261029120000_add_voice_profile_to_users.rb
809
813
  - docs/BOOKING.md
810
814
  - docs/SITE_FOOTER.md
811
815
  - lib/active_storage/service/studio_trash_s3_service.rb
@@ -889,6 +893,7 @@ files:
889
893
  - lib/studio/ui_primitives.rb
890
894
  - lib/studio/username_generator.rb
891
895
  - lib/studio/version.rb
896
+ - lib/studio/voice_match.rb
892
897
  - lib/tasks/studio_email.rake
893
898
  - lib/tasks/studio_knowledge.rake
894
899
  - lib/tasks/studio_node.rake