studio-engine 0.121.0 → 0.122.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +43 -0
- data/app/controllers/studio/voice_takes_controller.rb +55 -2
- data/app/javascript/studio/microphone_chip.js +7 -3
- data/app/javascript/studio/voice_profile.js +11 -1
- data/app/javascript/studio/voice_recorder.js +13 -2
- data/app/views/studio/voice/_microphone_help.html.erb +3 -1
- data/app/views/studio/voice/_styles.html.erb +4 -0
- data/lib/studio/version.rb +1 -1
- data/lib/studio/voice_transcription.rb +100 -34
- data/lib/studio.rb +17 -2
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: cd0f77b33c7833a6cb5061bc17b11baad3db5ad3b2e6ad232b31cc363d124e49
|
|
4
|
+
data.tar.gz: 942b24a25b2dbdb8d7e1af5a918c323718fcb2438eced8b46dd1593f1563d9a0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2d81f612a81ddb2d5e9ccdf6c85d4e4cea1bd47043ba5e0c9732202f55e39489b44059646e8f42848530525c2c4c8f23ecd299afeb322b43adb3dd3bb513ff1a
|
|
7
|
+
data.tar.gz: 3f6a3214f4a5a80cfc2c73453dd0429161f28879ba3bd41857160ab02c2f70bb633213d317c19d9026e786afb7d99c37ddc5e6e1da2793d8edf495fa976f216d
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,49 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.122.0 — 2026-11-01
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`POST /voice/takes` is rate-limited per user.** `Studio.voice_takes_rate_limit`
|
|
12
|
+
(default `{ to: 20, within: 600 }`, a fixed window per user id; `store:`
|
|
13
|
+
optional; `false` turns it off). Over it: a JSON `429`
|
|
14
|
+
`{error, reason: "rate_limited", retry_after}` with `Retry-After`, before the
|
|
15
|
+
take is read or transcribed. On a `NullStore` cache it counts in a per-process
|
|
16
|
+
memory store rather than going inert.
|
|
17
|
+
- **One deadline per take at the vendor.** `Studio.voice_transcribe_deadline`
|
|
18
|
+
(25 s) covers both models together; every socket timeout is what is left of
|
|
19
|
+
it, and a hard timeout backs it (the old 10 s + 120 s per attempt could hold a
|
|
20
|
+
Puma thread about 260 s).
|
|
21
|
+
- `Studio::VoiceTranscription::HttpError#reason` (`unauthorized`,
|
|
22
|
+
`rate_limited`, `server_error`, `rejected`, `timeout`, `unreachable`).
|
|
23
|
+
- The recorder's `failed.busy` line: a microphone another app holds
|
|
24
|
+
(`NotReadableError`) reads "Your microphone is busy in another app. Close it
|
|
25
|
+
there, then try again.", not "I couldn't find a microphone".
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- **Voice reads its own key.** `Studio.voice_openai_api_key`, else
|
|
30
|
+
`STUDIO_VOICE_OPENAI_API_KEY`; the app's shared `OPENAI_API_KEY` is no longer
|
|
31
|
+
read. An app that turned voice on with `OPENAI_API_KEY` alone sets
|
|
32
|
+
`STUDIO_VOICE_OPENAI_API_KEY` (until then a take answers the 503
|
|
33
|
+
`not_configured`).
|
|
34
|
+
- The `whisper-1` fallback no longer runs after a timeout, a vendor 429, or no
|
|
35
|
+
answer at all, nor with under 2 s of the deadline left.
|
|
36
|
+
- `HttpError` messages, and so `ErrorLog` rows, carry the model, the status and
|
|
37
|
+
a reason code, never the vendor's error text. The row's `target` is the user
|
|
38
|
+
who sent the take.
|
|
39
|
+
- Microphone picker: Esc in the open list returns focus to the chip; options
|
|
40
|
+
drop `aria-pressed` (the one in use keeps `aria-current`); options and the
|
|
41
|
+
help link wear the chip's 3px focus ring; the help's address-bar drawing shows
|
|
42
|
+
a drawn bar instead of printing `request.host`.
|
|
43
|
+
|
|
44
|
+
### Fixed
|
|
45
|
+
|
|
46
|
+
- A `voice` param that is a string or a list answers `422`, not `500`.
|
|
47
|
+
- The no-live-calls trap reads `URI#hostname`, so an IPv6 loopback endpoint
|
|
48
|
+
(`http://[::1]:…`) counts as this machine.
|
|
49
|
+
|
|
7
50
|
## 0.121.0 — 2026-11-01
|
|
8
51
|
|
|
9
52
|
### Added
|
|
@@ -19,6 +19,10 @@ module Studio
|
|
|
19
19
|
# CSRF-protected (send the page's token as X-CSRF-Token), JSON, no-store.
|
|
20
20
|
# The speaker is matched only against the signed-in user's own saved voice.
|
|
21
21
|
#
|
|
22
|
+
# RATE-LIMITED per user (Studio.voice_takes_rate_limit, 20 takes in 10
|
|
23
|
+
# minutes): over it, a JSON 429 {error, reason: "rate_limited", retry_after}
|
|
24
|
+
# and a Retry-After header, before anything is read or transcribed.
|
|
25
|
+
#
|
|
22
26
|
# NOTHING IS KEPT: the upload is read once, handed to the transcriber, and its
|
|
23
27
|
# tempfile is deleted before the answer; no audio or vector is stored or
|
|
24
28
|
# logged (voice[audio] and voice[embedding] are in the engine's parameter
|
|
@@ -27,13 +31,22 @@ module Studio
|
|
|
27
31
|
before_action :require_authentication
|
|
28
32
|
protect_from_forgery with: :exception
|
|
29
33
|
before_action :no_store
|
|
34
|
+
before_action :limit_takes
|
|
30
35
|
|
|
31
36
|
CANT_HEAR = "I couldn't hear any words in that. Check your microphone and try again."
|
|
32
37
|
UNAVAILABLE = "I couldn't transcribe that just now. Try again in a moment."
|
|
38
|
+
TOO_MANY = "That's a lot of recordings in a short time. Wait a moment, then try again."
|
|
39
|
+
|
|
40
|
+
# The counter's fallback when the app's Rails.cache counts nothing (a
|
|
41
|
+
# NullStore, as many apps run in development): per process, so a limit
|
|
42
|
+
# still holds.
|
|
43
|
+
FALLBACK_STORE = ActiveSupport::Cache::MemoryStore.new(size: 4.megabytes)
|
|
33
44
|
|
|
34
45
|
def create
|
|
35
46
|
voice = params[:voice]
|
|
36
|
-
|
|
47
|
+
# A string (voice=abc) or an array is no take: refuse it, never index it.
|
|
48
|
+
voice = nil unless voice.is_a?(ActionController::Parameters)
|
|
49
|
+
audio = voice ? voice[:audio] : nil
|
|
37
50
|
return refuse("Record something first.") unless audio.respond_to?(:read) && audio.respond_to?(:size)
|
|
38
51
|
|
|
39
52
|
begin
|
|
@@ -68,6 +81,41 @@ module Studio
|
|
|
68
81
|
|
|
69
82
|
private
|
|
70
83
|
|
|
84
|
+
# The per-user limit, a fixed window: the window's number is in the key, so
|
|
85
|
+
# its end (retry_after) is known exactly. Rails' own rate_limit macro fixes
|
|
86
|
+
# to:/within: when the class loads and counts nothing on a NullStore; this
|
|
87
|
+
# reads the config per request and never goes inert. It instruments the same
|
|
88
|
+
# rate_limit.action_controller event.
|
|
89
|
+
def limit_takes
|
|
90
|
+
limit = Studio.voice_takes_rate_limit
|
|
91
|
+
return unless limit && current_user
|
|
92
|
+
|
|
93
|
+
limit = limit.to_h.transform_keys(&:to_sym)
|
|
94
|
+
|
|
95
|
+
to = limit[:to].to_i
|
|
96
|
+
within = limit[:within].to_i
|
|
97
|
+
return unless to.positive? && within.positive?
|
|
98
|
+
|
|
99
|
+
epoch = Time.now.to_i
|
|
100
|
+
window = epoch / within
|
|
101
|
+
key = "rate-limit:#{controller_path}:#{current_user.id}:#{window}"
|
|
102
|
+
count = rate_limit_store(limit).increment(key, 1, expires_in: within)
|
|
103
|
+
return if count.nil? || count <= to
|
|
104
|
+
|
|
105
|
+
retry_after = within - (epoch % within)
|
|
106
|
+
ActiveSupport::Notifications.instrument("rate_limit.action_controller",
|
|
107
|
+
request: request, count: count, to: to, within: within,
|
|
108
|
+
by: current_user.id, name: nil, scope: controller_path, cache_key: key) do
|
|
109
|
+
response.headers["Retry-After"] = retry_after.to_s
|
|
110
|
+
refuse(TOO_MANY, :too_many_requests, reason: "rate_limited", retry_after: retry_after)
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def rate_limit_store(limit)
|
|
115
|
+
store = limit[:store] || (Rails.cache if defined?(Rails.cache))
|
|
116
|
+
store.nil? || store.is_a?(ActiveSupport::Cache::NullStore) ? FALLBACK_STORE : store
|
|
117
|
+
end
|
|
118
|
+
|
|
71
119
|
def refuse(message, status = :unprocessable_entity, **extra)
|
|
72
120
|
render json: { error: message, **extra }, status: status
|
|
73
121
|
end
|
|
@@ -84,8 +132,13 @@ module Studio
|
|
|
84
132
|
nil
|
|
85
133
|
end
|
|
86
134
|
|
|
135
|
+
# An ErrorLog row on the user who sent the take. The error's message is the
|
|
136
|
+
# engine's own words (model, status, reason code), never the vendor's.
|
|
87
137
|
def capture(error)
|
|
88
|
-
|
|
138
|
+
return unless defined?(::ErrorLog) && ::ErrorLog.respond_to?(:capture!)
|
|
139
|
+
|
|
140
|
+
log = ::ErrorLog.capture!(error)
|
|
141
|
+
log.update(target: current_user) if log.respond_to?(:target=) && current_user
|
|
89
142
|
rescue StandardError
|
|
90
143
|
nil
|
|
91
144
|
end
|
|
@@ -193,7 +193,8 @@ export function mountMicrophoneChip(root, { win = window } = {}) {
|
|
|
193
193
|
button.type = "button"
|
|
194
194
|
button.className = "studio-mic-option"
|
|
195
195
|
button.dataset.microphoneOption = choice.deviceId
|
|
196
|
-
|
|
196
|
+
// aria-current marks the one in use; aria-pressed would make each option
|
|
197
|
+
// a toggle, which picking is not.
|
|
197
198
|
if (inUse) button.setAttribute("aria-current", "true")
|
|
198
199
|
const mark = doc.createElement("span")
|
|
199
200
|
mark.className = "studio-mic-check"
|
|
@@ -333,12 +334,15 @@ export function mountMicrophoneChip(root, { win = window } = {}) {
|
|
|
333
334
|
if (chip && target.closest("[data-microphone-chip]") === chip) toggle()
|
|
334
335
|
}
|
|
335
336
|
|
|
336
|
-
// Esc closes the list
|
|
337
|
-
//
|
|
337
|
+
// Esc closes the list and hands focus back to the chip, so a keyboard user
|
|
338
|
+
// is not left on an option that just vanished. An Esc inside the help dialog
|
|
339
|
+
// is the dialog's: it shuts the dialog and leaves the list where it was.
|
|
338
340
|
function onKey(event) {
|
|
339
341
|
if (event.key !== "Escape") return
|
|
340
342
|
if (dialog && dialog.contains(event.target)) return
|
|
343
|
+
if (!panel || panel.hidden) return
|
|
341
344
|
closePanel()
|
|
345
|
+
if (chip) chip.focus()
|
|
342
346
|
}
|
|
343
347
|
|
|
344
348
|
const onDeviceChange = () => refresh()
|
|
@@ -58,6 +58,7 @@ export const MESSAGES = {
|
|
|
58
58
|
silent: "I couldn't hear that one. Check your microphone and read it again.",
|
|
59
59
|
failed: "That one didn't work. Tap Record and read it again.",
|
|
60
60
|
denied: "Allow the microphone to set up your voice.",
|
|
61
|
+
busy: "Your microphone is busy in another app. Close it there, then try again.",
|
|
61
62
|
unsupported: "This browser can't record here. Try Safari or Chrome.",
|
|
62
63
|
network: "We couldn't reach the server. Tap Save again.",
|
|
63
64
|
notSaved: "That didn't save. Tap Save again."
|
|
@@ -65,6 +66,15 @@ export const MESSAGES = {
|
|
|
65
66
|
|
|
66
67
|
export class SilentTakeError extends Error {}
|
|
67
68
|
|
|
69
|
+
// Why the microphone would not open: refused, held by another app
|
|
70
|
+
// (NotReadableError; Chrome's older TrackStartError), or anything else.
|
|
71
|
+
export function openFailure(error) {
|
|
72
|
+
const name = error && error.name
|
|
73
|
+
if (name === "NotAllowedError") return MESSAGES.denied
|
|
74
|
+
if (name === "NotReadableError" || name === "TrackStartError") return MESSAGES.busy
|
|
75
|
+
return MESSAGES.failed
|
|
76
|
+
}
|
|
77
|
+
|
|
68
78
|
// The words for a state, with {n} the phrase being read (1-based).
|
|
69
79
|
export function copyFor(state, step) {
|
|
70
80
|
const [button, status] = COPY[state] || ["Try again", ""]
|
|
@@ -304,7 +314,7 @@ export function mountVoiceProfile(root, win = window) {
|
|
|
304
314
|
try {
|
|
305
315
|
stream = await win.navigator.mediaDevices.getUserMedia({ audio: true })
|
|
306
316
|
} catch (error) {
|
|
307
|
-
setState("ready", error
|
|
317
|
+
setState("ready", openFailure(error))
|
|
308
318
|
return
|
|
309
319
|
}
|
|
310
320
|
if (destroyed || state !== "ready") { release(); return }
|
|
@@ -61,7 +61,8 @@ export const COPY = {
|
|
|
61
61
|
silent: { button: "Record", status: "I couldn't hear anything. Check your microphone and try again.",
|
|
62
62
|
named: "I couldn't hear anything from %{microphone}. Pick another microphone and try again." },
|
|
63
63
|
failed: { button: "Try again", status: "That didn't record. Try again.", start: "I couldn't start recording. Try again.",
|
|
64
|
-
missing: "I couldn't find a microphone. Plug one in and try again."
|
|
64
|
+
missing: "I couldn't find a microphone. Plug one in and try again.",
|
|
65
|
+
busy: "Your microphone is busy in another app. Close it there, then try again." },
|
|
65
66
|
denied: { button: "Record", status: "Allow the microphone to record." },
|
|
66
67
|
unsupported: { button: "Record", status: "This browser can't record here. Try Safari or Chrome." }
|
|
67
68
|
}
|
|
@@ -78,6 +79,16 @@ export function formatElapsed(ms) {
|
|
|
78
79
|
return `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, "0")}`
|
|
79
80
|
}
|
|
80
81
|
|
|
82
|
+
// The line for a microphone that would not open (NotAllowedError is "denied",
|
|
83
|
+
// not this). NotReadableError is a microphone another app holds (Chrome's
|
|
84
|
+
// older name, TrackStartError); everything else reads as no microphone found.
|
|
85
|
+
export const BUSY_ERRORS = ["NotReadableError", "TrackStartError"]
|
|
86
|
+
|
|
87
|
+
export function failedLine(error, copy = COPY) {
|
|
88
|
+
const busy = !!error && BUSY_ERRORS.includes(error.name)
|
|
89
|
+
return busy ? (copy.failed.busy || COPY.failed.busy) : copy.failed.missing
|
|
90
|
+
}
|
|
91
|
+
|
|
81
92
|
// The page's words over the defaults, one state at a time.
|
|
82
93
|
export function mergeCopy(overrides) {
|
|
83
94
|
const merged = {}
|
|
@@ -193,7 +204,7 @@ export function mountVoiceRecorder(root, { win = window, open = openMicrophone,
|
|
|
193
204
|
if (destroyed) return
|
|
194
205
|
const denied = error && error.name === "NotAllowedError"
|
|
195
206
|
if (denied) announce(MICROPHONE_EVENTS.denied, {}, win)
|
|
196
|
-
setState(denied ? "denied" : "failed", denied ? null : copy
|
|
207
|
+
setState(denied ? "denied" : "failed", denied ? null : failedLine(error, copy))
|
|
197
208
|
return
|
|
198
209
|
}
|
|
199
210
|
stream = opened.stream
|
|
@@ -27,7 +27,9 @@
|
|
|
27
27
|
<line x1="19" y1="20" x2="33" y2="20"/><circle cx="23" cy="20" r="2.5" class="mock-knob"/>
|
|
28
28
|
<line x1="19" y1="28" x2="33" y2="28"/><circle cx="29" cy="28" r="2.5" class="mock-knob"/>
|
|
29
29
|
</g>
|
|
30
|
-
|
|
30
|
+
<%# The address is a drawn bar, not this app's host: the drawing is the
|
|
31
|
+
same on every app, and renders without a request. %>
|
|
32
|
+
<rect x="48" y="19" width="132" height="10" rx="5" class="mock-address"/>
|
|
31
33
|
<rect x="12" y="54" width="232" height="106" rx="10" class="mock-panel"/>
|
|
32
34
|
<g class="mock-ink" stroke-width="2" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
|
33
35
|
<rect x="26" y="68" width="8" height="13" rx="4"/><path d="M23 77a7 7 0 0 0 14 0M30 84v4"/>
|
|
@@ -32,6 +32,9 @@
|
|
|
32
32
|
border: 1px solid transparent; background: transparent; padding: .5rem .75rem; text-align: left; font-size: .875rem;
|
|
33
33
|
color: var(--color-text, inherit); cursor: pointer; }
|
|
34
34
|
.studio-mic-option:hover { background-color: rgb(var(--color-primary-500-rgb, 99 102 241) / .08); }
|
|
35
|
+
/* The same 3px ring as the chip on every control in the picker. */
|
|
36
|
+
.studio-mic-option:focus-visible, .studio-mic-help-link:focus-visible {
|
|
37
|
+
outline: 3px solid rgb(var(--color-primary-500-rgb, 99 102 241) / .5); outline-offset: 2px; }
|
|
35
38
|
.studio-mic-option[aria-current="true"] { border-color: rgb(var(--color-primary-500-rgb, 99 102 241));
|
|
36
39
|
background-color: rgb(var(--color-primary-500-rgb, 99 102 241) / .12); font-weight: 600; }
|
|
37
40
|
.studio-mic-check { width: 1rem; flex-shrink: 0; font-weight: 700; color: rgb(var(--color-primary-500-rgb, 99 102 241)); }
|
|
@@ -64,6 +67,7 @@
|
|
|
64
67
|
.studio-mic-mock .mock-ink { stroke: currentColor; }
|
|
65
68
|
.studio-mic-mock .mock-knob { fill: currentColor; stroke: none; }
|
|
66
69
|
.studio-mic-mock .mock-text { fill: currentColor; font-family: inherit; }
|
|
70
|
+
.studio-mic-mock .mock-address { fill: currentColor; fill-opacity: .3; }
|
|
67
71
|
.studio-mic-mock .mock-on, .studio-mic-mock .mock-badge { fill: rgb(var(--color-primary-500-rgb, 99 102 241)); }
|
|
68
72
|
.studio-mic-mock .mock-knob-on, .studio-mic-mock .mock-badge-text { fill: var(--color-surface, Canvas); }
|
|
69
73
|
|
data/lib/studio/version.rb
CHANGED
|
@@ -5,6 +5,7 @@ require "net/http"
|
|
|
5
5
|
require "openssl"
|
|
6
6
|
require "securerandom"
|
|
7
7
|
require "stringio"
|
|
8
|
+
require "timeout"
|
|
8
9
|
require "uri"
|
|
9
10
|
|
|
10
11
|
# Understand: one finished take (the recorder's blob and its mime type, plus the
|
|
@@ -38,17 +39,36 @@ require "uri"
|
|
|
38
39
|
# logger and environment, so the unit suite runs it without an app.
|
|
39
40
|
module Studio
|
|
40
41
|
module VoiceTranscription
|
|
41
|
-
# A transcriber call that failed
|
|
42
|
+
# A transcriber call that failed.
|
|
42
43
|
class Error < StandardError; end
|
|
43
44
|
|
|
44
|
-
# The vendor
|
|
45
|
-
# an auth failure from a model that fell
|
|
45
|
+
# The vendor failed. The status rides along (0 when nothing answered) with a
|
|
46
|
+
# reason code, so a caller can tell an auth failure from a model that fell
|
|
47
|
+
# over. The message names only the model, the status and the reason, never
|
|
48
|
+
# the vendor's own words: it reaches ErrorLog, and a vendor's error text can
|
|
49
|
+
# echo what it was sent.
|
|
46
50
|
class HttpError < Error
|
|
47
|
-
|
|
51
|
+
# Why, as a code: unauthorized (401/403), rate_limited (429),
|
|
52
|
+
# server_error (5xx), rejected (another non-2xx), timeout (the deadline
|
|
53
|
+
# ran out), unreachable (no answer: refused, reset, DNS, TLS).
|
|
54
|
+
REASONS = %w[unauthorized rate_limited server_error rejected timeout unreachable].freeze
|
|
48
55
|
|
|
49
|
-
|
|
56
|
+
attr_reader :status, :reason
|
|
57
|
+
|
|
58
|
+
def initialize(message, status:, reason: nil)
|
|
50
59
|
super(message)
|
|
51
60
|
@status = status
|
|
61
|
+
@reason = reason || self.class.reason_for(status)
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def self.reason_for(status)
|
|
65
|
+
case status.to_i
|
|
66
|
+
when 401, 403 then "unauthorized"
|
|
67
|
+
when 429 then "rate_limited"
|
|
68
|
+
when 500..599 then "server_error"
|
|
69
|
+
when 0 then "unreachable"
|
|
70
|
+
else "rejected"
|
|
71
|
+
end
|
|
52
72
|
end
|
|
53
73
|
end
|
|
54
74
|
|
|
@@ -64,7 +84,8 @@ module Studio
|
|
|
64
84
|
|
|
65
85
|
NO_LIVE_CALLS_ENV = "VOICE_NO_LIVE_CALLS"
|
|
66
86
|
TRANSCRIBER_ENV = "STUDIO_VOICE_TRANSCRIBER"
|
|
67
|
-
|
|
87
|
+
# Voice's own key. Deliberately NOT the app's shared OPENAI_API_KEY.
|
|
88
|
+
API_KEY_ENV = "STUDIO_VOICE_OPENAI_API_KEY"
|
|
68
89
|
|
|
69
90
|
# What a page shows when the app has no transcriber key. Says what is wrong
|
|
70
91
|
# without naming the variable to a visitor.
|
|
@@ -123,24 +144,38 @@ module Studio
|
|
|
123
144
|
# The real speech-to-text: OpenAI's POST /v1/audio/transcriptions, one
|
|
124
145
|
# multipart file. Ported from Voicebox's Transcription::OpenAI: the model is
|
|
125
146
|
# told the language (left to guess, a model handed near-silence answers a
|
|
126
|
-
# stock word in any language), and a
|
|
127
|
-
# on the fallback
|
|
147
|
+
# stock word in any language), and a model that fell over (a 5xx, or a 4xx
|
|
148
|
+
# other than auth and 429) is retried once on the fallback from the start of
|
|
149
|
+
# the audio.
|
|
150
|
+
#
|
|
151
|
+
# ONE DEADLINE for the whole take (Studio.voice_transcribe_deadline, 25 s):
|
|
152
|
+
# both attempts share it, every socket timeout is what is left of it, and a
|
|
153
|
+
# hard Timeout backs it, so a vendor that trickles bytes cannot hold a web
|
|
154
|
+
# thread past it. Nothing is retried after a timeout (the deadline is spent),
|
|
155
|
+
# a 429 (the account is throttled; a second model only spends more), an auth
|
|
156
|
+
# failure, or no answer at all; nor with under MIN_RETRY_SECONDS left.
|
|
128
157
|
class OpenAI
|
|
129
158
|
ENDPOINT = "https://api.openai.com/v1/audio/transcriptions"
|
|
159
|
+
# The cap on connecting, inside the deadline.
|
|
130
160
|
OPEN_TIMEOUT = 10
|
|
131
|
-
|
|
132
|
-
|
|
161
|
+
NO_RETRY = %w[unauthorized rate_limited timeout unreachable].freeze
|
|
162
|
+
MIN_RETRY_SECONDS = 2
|
|
133
163
|
|
|
134
|
-
|
|
164
|
+
# The hard backstop's exception; turned into HttpError(reason: "timeout").
|
|
165
|
+
class DeadlineExceeded < Timeout::Error; end
|
|
166
|
+
|
|
167
|
+
attr_reader :model, :fallback_model, :language, :endpoint, :deadline
|
|
135
168
|
|
|
136
169
|
def initialize(api_key: VoiceTranscription.api_key, model: Studio.voice_transcribe_model,
|
|
137
170
|
fallback_model: Studio.voice_transcribe_fallback_model,
|
|
138
|
-
language: Studio.voice_transcribe_language, endpoint: ENDPOINT
|
|
171
|
+
language: Studio.voice_transcribe_language, endpoint: ENDPOINT,
|
|
172
|
+
deadline: Studio.voice_transcribe_deadline)
|
|
139
173
|
@api_key = api_key
|
|
140
174
|
@model = model.to_s
|
|
141
175
|
@fallback_model = fallback_model.to_s.strip.empty? ? nil : fallback_model.to_s
|
|
142
176
|
@language = language.to_s.strip.empty? ? nil : language.to_s
|
|
143
177
|
@endpoint = endpoint
|
|
178
|
+
@deadline = deadline.to_f.positive? ? deadline.to_f : 25.0
|
|
144
179
|
end
|
|
145
180
|
|
|
146
181
|
def configured? = !@api_key.to_s.strip.empty?
|
|
@@ -149,13 +184,24 @@ module Studio
|
|
|
149
184
|
def transcribe(io:, filename:, content_type:)
|
|
150
185
|
raise NotConfigured, "#{API_KEY_ENV} is not set, so this app cannot transcribe voice" unless configured?
|
|
151
186
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
187
|
+
ends_at = now + deadline
|
|
188
|
+
begin
|
|
189
|
+
request_text(model, io, filename, content_type, ends_at)
|
|
190
|
+
rescue HttpError => e
|
|
191
|
+
raise unless retry?(e, ends_at)
|
|
155
192
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
193
|
+
VoiceTranscription.log_warn("#{model} transcription failed (HTTP #{e.status}, #{e.reason}); " \
|
|
194
|
+
"retrying on #{fallback_model}")
|
|
195
|
+
io.rewind
|
|
196
|
+
request_text(fallback_model, io, filename, content_type, ends_at)
|
|
197
|
+
end
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def retry?(error, ends_at)
|
|
201
|
+
return false if fallback_model.nil? || fallback_model == model
|
|
202
|
+
return false if NO_RETRY.include?(error.reason)
|
|
203
|
+
|
|
204
|
+
ends_at - now >= MIN_RETRY_SECONDS
|
|
159
205
|
end
|
|
160
206
|
|
|
161
207
|
# The multipart fields, apart from the socket so a test reads them.
|
|
@@ -184,36 +230,52 @@ module Studio
|
|
|
184
230
|
|
|
185
231
|
private
|
|
186
232
|
|
|
187
|
-
def
|
|
188
|
-
|
|
233
|
+
def now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
234
|
+
|
|
235
|
+
def request_text(model_name, io, filename, content_type, ends_at)
|
|
236
|
+
perform(model_name, io, filename, content_type, ends_at).fetch("text", "").to_s.strip
|
|
189
237
|
end
|
|
190
238
|
|
|
191
239
|
# The only place a socket opens, so the only place the trap sits.
|
|
192
|
-
|
|
240
|
+
# uri.hostname, not uri.host: an IPv6 literal's host keeps its brackets
|
|
241
|
+
# ("[::1]"), which neither the loopback check nor the connect reads.
|
|
242
|
+
def perform(model_name, io, filename, content_type, ends_at)
|
|
193
243
|
uri = URI.parse(endpoint)
|
|
194
|
-
VoiceTranscription.guard_live_call!("OpenAI", host: uri.
|
|
244
|
+
VoiceTranscription.guard_live_call!("OpenAI", host: uri.hostname)
|
|
245
|
+
|
|
246
|
+
left = ends_at - now
|
|
247
|
+
raise HttpError.new("OpenAI #{model_name} was not tried: the take's deadline ran out", status: 0, reason: "timeout") if left <= 0
|
|
195
248
|
|
|
196
249
|
request = build_request(model_name, io, filename, content_type)
|
|
197
|
-
response =
|
|
198
|
-
|
|
199
|
-
|
|
250
|
+
response = Timeout.timeout(left, DeadlineExceeded) do
|
|
251
|
+
Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https", open_timeout: [OPEN_TIMEOUT, left].min,
|
|
252
|
+
read_timeout: left, write_timeout: left) do |http|
|
|
253
|
+
http.request(request)
|
|
254
|
+
end
|
|
200
255
|
end
|
|
201
256
|
interpret(response, model_name)
|
|
202
|
-
rescue Timeout::Error
|
|
203
|
-
raise HttpError.new("OpenAI #{model_name}
|
|
257
|
+
rescue Timeout::Error => e
|
|
258
|
+
raise HttpError.new("OpenAI #{model_name} did not answer within the take's #{deadline.round(1)} s deadline (#{e.class})",
|
|
259
|
+
status: 0, reason: "timeout")
|
|
260
|
+
rescue SystemCallError, IOError, OpenSSL::SSL::SSLError, SocketError => e
|
|
261
|
+
raise HttpError.new("OpenAI #{model_name} could not be reached (#{e.class})", status: 0, reason: "unreachable")
|
|
204
262
|
end
|
|
205
263
|
|
|
264
|
+
# A failure carries the status and its reason code, never the vendor's
|
|
265
|
+
# words (HttpError).
|
|
206
266
|
def interpret(response, model_name)
|
|
267
|
+
status = response.code.to_i
|
|
268
|
+
unless response.is_a?(Net::HTTPSuccess)
|
|
269
|
+
reason = HttpError.reason_for(status)
|
|
270
|
+
raise HttpError.new("OpenAI #{model_name} answered HTTP #{status} (#{reason})", status: status, reason: reason)
|
|
271
|
+
end
|
|
272
|
+
|
|
207
273
|
parsed = begin
|
|
208
274
|
JSON.parse(response.body.to_s)
|
|
209
275
|
rescue JSON::ParserError
|
|
210
276
|
{}
|
|
211
277
|
end
|
|
212
|
-
parsed
|
|
213
|
-
return parsed if response.is_a?(Net::HTTPSuccess)
|
|
214
|
-
|
|
215
|
-
detail = parsed.dig("error", "message") || response.body.to_s[0, 200]
|
|
216
|
-
raise HttpError.new("OpenAI #{model_name} answered HTTP #{response.code}: #{detail}", status: response.code.to_i)
|
|
278
|
+
parsed.is_a?(Hash) ? parsed : {}
|
|
217
279
|
end
|
|
218
280
|
end
|
|
219
281
|
|
|
@@ -269,7 +331,8 @@ module Studio
|
|
|
269
331
|
klass.new
|
|
270
332
|
end
|
|
271
333
|
|
|
272
|
-
# Studio.voice_openai_api_key, else
|
|
334
|
+
# Studio.voice_openai_api_key, else STUDIO_VOICE_OPENAI_API_KEY. Never the
|
|
335
|
+
# app's shared OPENAI_API_KEY: voice spends on its own key and budget.
|
|
273
336
|
def api_key
|
|
274
337
|
key = Studio.voice_openai_api_key.to_s.strip
|
|
275
338
|
key.empty? ? ENV[API_KEY_ENV].to_s.strip : key
|
|
@@ -285,9 +348,12 @@ module Studio
|
|
|
285
348
|
rails_env == "test"
|
|
286
349
|
end
|
|
287
350
|
|
|
351
|
+
# This machine. host is URI#hostname, so IPv6 arrives unbracketed.
|
|
352
|
+
LOOPBACK_HOSTS = %w[127.0.0.1 localhost ::1].freeze
|
|
353
|
+
|
|
288
354
|
def guard_live_call!(vendor, host:)
|
|
289
355
|
return unless live_calls_blocked?
|
|
290
|
-
return if
|
|
356
|
+
return if LOOPBACK_HOSTS.include?(host.to_s)
|
|
291
357
|
|
|
292
358
|
raise LiveCallAttempted, "A live #{vendor} call was attempted with #{NO_LIVE_CALLS_ENV} armed. " \
|
|
293
359
|
"Tests drive the mock transcriber (Studio.voice_transcriber = \"mock\")."
|
data/lib/studio.rb
CHANGED
|
@@ -589,10 +589,25 @@ module Studio
|
|
|
589
589
|
# VOICE_NO_LIVE_CALLS is armed in every test environment.
|
|
590
590
|
mattr_accessor :voice_transcriber, default: nil
|
|
591
591
|
|
|
592
|
-
# The OpenAI key for the real transcriber. nil reads
|
|
593
|
-
#
|
|
592
|
+
# The OpenAI key for the real transcriber, voice's OWN key. nil reads
|
|
593
|
+
# STUDIO_VOICE_OPENAI_API_KEY, and never the app's shared OPENAI_API_KEY (on
|
|
594
|
+
# the hub that is the image key, with its own budget). Missing, the endpoint
|
|
595
|
+
# answers 503 with a line the page can show; nothing crashes.
|
|
594
596
|
mattr_accessor :voice_openai_api_key, default: nil
|
|
595
597
|
|
|
598
|
+
# One take's whole budget at the vendor, in seconds: the first model AND the
|
|
599
|
+
# fallback share it, so a take never holds a web thread longer than this
|
|
600
|
+
# (Studio::VoiceTranscription::OpenAI). A timeout or a vendor 429 is not
|
|
601
|
+
# retried.
|
|
602
|
+
mattr_accessor :voice_transcribe_deadline, default: 25
|
|
603
|
+
|
|
604
|
+
# POST /voice/takes per signed-in user: at most `to` takes in any `within`
|
|
605
|
+
# seconds (a fixed window), else a JSON 429 with retry_after. Every take is a
|
|
606
|
+
# paid vendor call. `store:` (optional) is the counter's cache; nil uses
|
|
607
|
+
# Rails.cache, or a per-process memory store when Rails.cache is a NullStore.
|
|
608
|
+
# false turns the limit off.
|
|
609
|
+
mattr_accessor :voice_takes_rate_limit, default: { to: 20, within: 600 }
|
|
610
|
+
|
|
596
611
|
# The speech-to-text models (Voicebox's measured pair) and the language both
|
|
597
612
|
# are told to hear. A blank fallback turns the one retry off.
|
|
598
613
|
mattr_accessor :voice_transcribe_model, default: "gpt-4o-transcribe"
|