cohere-transcribe 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +5 -5
- data/ext/cohere_transcribe_native/CMakeLists.txt +1 -1
- data/ext/cohere_transcribe_native/README.md +1 -1
- data/ext/cohere_transcribe_native/extconf.rb +12 -2
- data/lib/cohere/transcribe/asr/native.rb +72 -8
- data/lib/cohere/transcribe/audio/decoder.rb +130 -82
- data/lib/cohere/transcribe/audio/ffmpeg_native.rb +68 -39
- data/lib/cohere/transcribe/cli.rb +1 -0
- data/lib/cohere/transcribe/configuration.rb +2 -0
- data/lib/cohere/transcribe/dense_converter.rb +29 -4
- data/lib/cohere/transcribe/doctor.rb +2 -2
- data/lib/cohere/transcribe/hub.rb +26 -5
- data/lib/cohere/transcribe/input.rb +4 -4
- data/lib/cohere/transcribe/python_text.rb +21 -3
- data/lib/cohere/transcribe/pytorch_checkpoint.rb +68 -11
- data/lib/cohere/transcribe/runtime/engine.rb +11 -1
- data/lib/cohere/transcribe/runtime/preparation.rb +81 -13
- data/lib/cohere/transcribe/runtime/resources.rb +105 -17
- data/lib/cohere/transcribe/state/io.rb +12 -2
- data/lib/cohere/transcribe/state/manifest.rb +30 -16
- data/lib/cohere/transcribe/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f494c8ad68293a60a51ea1510bd3ce55b81867f9014d4a5dcbd5c8499e970bde
|
|
4
|
+
data.tar.gz: efe16c2e6c4b0c2e4924716fc4023dffe0480375a075b8138c974b41dddb9115
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d43d30ab5c72a800e94148d368475aaf97a8f3a8ca97bd8ea24804783dcfea3450cef86accfdacd487064f6c7636c3811ca054d6f51c75a040dcc050a6634027
|
|
7
|
+
data.tar.gz: 7e506a692ae9025e4d825f5330a94685a2500f296745dbdf671bdf632799873278d88f21f93e1d36b9a382b8c8e1eae834a82267f7851da2e41b2371b6312f25
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.1.2] - 2026-07-16
|
|
4
|
+
|
|
5
|
+
- Correct macOS creation-mode delivery for descriptor-relative staged files and turn concurrent output replacement during preflight into a normal reprocessing decision.
|
|
6
|
+
- Correct ZIP and ZIP64 boundary handling, verify streamed PyTorch tensor storage checksums before conversion, and make vocabulary validation bounded for malformed declared shapes.
|
|
7
|
+
- Close abandoned and partially constructed native sessions exactly once, release native decoder metadata allocations deterministically, and guarantee decoded FFmpeg buffers are released when interruption arrives at the native return boundary.
|
|
8
|
+
- Make preparation groups size-aware so long files retain the full single-file memory ceiling without overlapping another group, unify stereo downmix energy across audio backends, and return typed public errors for binary-encoded Unix paths.
|
|
9
|
+
- Keep diagnostics and early command-line interruption contained, revalidate symbolic Hub revisions while online with an explicit offline cache mode, require CMake 3.15+, and install and exercise the built source gem on both CI platforms.
|
|
10
|
+
|
|
3
11
|
## [0.1.1] - 2026-07-15
|
|
4
12
|
|
|
5
13
|
- Fix CUDA source-gem installation across Unix Makefiles and Ninja by making NVCC's generated response file available from each compiler working directory, including when the gem is installed beneath a path containing spaces.
|
data/README.md
CHANGED
|
@@ -8,7 +8,7 @@ The core Dense path supports the pinned Arabic/English model, compatible Hub fin
|
|
|
8
8
|
|
|
9
9
|
- 64-bit Ruby `>= 4.0, < 5.0` (the Dense model exceeds a 32-bit address space)
|
|
10
10
|
- 64-bit Linux or macOS; the packaged native source build does not currently support Windows
|
|
11
|
-
- CMake 3.
|
|
11
|
+
- CMake 3.15+ and a C++20 compiler when installing from source
|
|
12
12
|
- FFmpeg 4–8 shared libraries for the full accepted container/codec set
|
|
13
13
|
- `libsndfile` and `libsamplerate` shared libraries for the fallback used when the native FFmpeg adapter is unavailable
|
|
14
14
|
- A Hugging Face token when the selected model requires authentication
|
|
@@ -187,7 +187,7 @@ The segment runtime uses CrispASR's padded Cohere encoder and ragged greedy deco
|
|
|
187
187
|
|
|
188
188
|
Native inference failures carry a thread-local error kind and diagnostic message across the C ABI. Ruby maps invalid arguments and invariant violations to fatal failures, allocator failures to OOM, and ordinary runtime failures to isolatable errors. Only typed OOM failures teach a smaller adaptive batch cap; fatal failures open the retained session's circuit breaker.
|
|
189
189
|
|
|
190
|
-
For multi-file runs, `preprocess_workers` concurrently decodes and segments one ordered preparation group while `pipeline_preparation` permits exactly one next group to overlap current ASR. Each pipelined group is capped at the smaller of half `audio_memory_gb` and 512 MiB of retained mono float PCM,
|
|
190
|
+
For multi-file runs, `preprocess_workers` concurrently decodes and segments one ordered preparation group while `pipeline_preparation` permits exactly one next group to overlap current ASR. Each ordinary pipelined group is capped at the smaller of half `audio_memory_gb` and 512 MiB of retained mono float PCM, with metadata-based size estimates assigning per-file ceilings instead of dividing the cap equally. A file whose estimate exceeds the group cap, or whose size cannot be estimated, is prepared alone with the full `audio_memory_gb` ceiling and no adjacent-group overlap. Native codec transients can add short-lived overhead. Disable `pipeline_preparation` for a fully sequential path. Automatic worker selection uses one worker for one file and at most two otherwise; explicit counts are capped by available processors and the group size. Decode/VAD failures remain isolated per file, and results, progress events, and publication always follow input order. Word alignment does not retain those prepared waveforms across its phase barrier. It re-decodes one file at a time through the concrete backend recorded during ASR (with one look-ahead only when the adjacent PCM pair fits `audio_memory_gb`) and rejects backend or sample-count drift. Resumable ASR checkpoints enter the alignment phase directly without reopening Dense or repeating decode/VAD preparation.
|
|
191
191
|
|
|
192
192
|
`pin_memory` remains accepted for Python API/CLI compatibility, but resolves to `false` in this ggml runtime. Ruby passes float PCM directly through the native session ABI, so there is no PyTorch host tensor to pin and no nonblocking tensor transfer for the option to accelerate.
|
|
193
193
|
|
|
@@ -215,13 +215,13 @@ CohereLabs/cohere-transcribe-arabic-07-2026
|
|
|
215
215
|
0a8193caa4f3f92131471ab08824e488141cb392
|
|
216
216
|
```
|
|
217
217
|
|
|
218
|
-
Hub artifacts reuse the standard Hugging Face cache. Dense Safetensors and `pytorch_model.bin` weights are streamed into a GGUF conversion once and reused from `~/.cache/cohere-transcribe`. PyTorch metadata is decoded by a restricted, allowlist-only Ruby reader; it never imports Python or executes pickle globals. Current `torch.save` ZIP files and the preceding raw-storage stream format are supported, including sharded indexes and strided tensors. Ancient tar-format checkpoints are rejected because they cannot be interpreted with the same restricted weights-only contract; re-save those as Safetensors or a current state dict. The retained-session identity includes the resolved device and dtype as well as the model identity. Each converted artifact has an independent cache key for its output dtype and a SHA-256 source fingerprint over the model ID/revision, relative source paths, sizes, mtimes, and ctimes, so even a same-size, mtime-preserving rewrite invalidates a local fine-tune through its changed ctime. A sidecar completion marker binds that source fingerprint, output dtype, and cache layout to the converted GGUF's device, inode, size, mtime, and ctime. This marker is stored beside the model as `*.complete.json`. The cache accepts only regular, non-symlink GGUF/marker files; conversion locks use no-follow opens where available and verify that the opened descriptor still matches the path's device and inode.
|
|
218
|
+
Hub artifacts reuse the standard Hugging Face cache. Symbolic branches and tags are revalidated while online; set `HF_HUB_OFFLINE=1` to resolve them only from an already complete cached snapshot. Dense Safetensors and `pytorch_model.bin` weights are streamed into a GGUF conversion once and reused from `~/.cache/cohere-transcribe`. PyTorch metadata is decoded by a restricted, allowlist-only Ruby reader; it never imports Python or executes pickle globals. Current `torch.save` ZIP files and the preceding raw-storage stream format are supported, including sharded indexes and strided tensors. Ancient tar-format checkpoints are rejected because they cannot be interpreted with the same restricted weights-only contract; re-save those as Safetensors or a current state dict. The retained-session identity includes the resolved device and dtype as well as the model identity. Each converted artifact has an independent cache key for its output dtype and a SHA-256 source fingerprint over the model ID/revision, relative source paths, sizes, mtimes, and ctimes, so even a same-size, mtime-preserving rewrite invalidates a local fine-tune through its changed ctime. A sidecar completion marker binds that source fingerprint, output dtype, and cache layout to the converted GGUF's device, inode, size, mtime, and ctime. This marker is stored beside the model as `*.complete.json`. The cache accepts only regular, non-symlink GGUF/marker files; conversion locks use no-follow opens where available and verify that the opened descriptor still matches the path's device and inode.
|
|
219
219
|
|
|
220
220
|
Word mode uses `onnx-community/mms-300m-1130-forced-aligner-ONNX@2100fb247d8e43962eef24491597fbeb8b469531`, an ONNX export of `MahmoudAshraf/mms-300m-1130-forced-aligner@49402e9577b1158620820667c218cd494cc44486`. The default `align_dtype: "fp32"` works with the CPU provider even when Dense ASR runs on CUDA. `align_dtype: "fp16"` requires a CUDA-enabled ONNX Runtime; point `COHERE_TRANSCRIBE_ONNXRUNTIME_LIBRARY` at that runtime when it is not the one supplied by the installed `onnxruntime` gem. The runtime verifies the complete model file against a pinned byte size and SHA-256 before loading it. These downloaded model weights are CC-BY-NC-4.0 and are not distributed inside the gem; see `lib/cohere/transcribe/alignment/ATTRIBUTION.md` for provenance and notices.
|
|
221
221
|
|
|
222
222
|
Useful environment variables:
|
|
223
223
|
|
|
224
|
-
- `HF_TOKEN`, `HF_HOME`, `HF_HUB_CACHE`, `HF_ENDPOINT`
|
|
224
|
+
- `HF_TOKEN`, `HF_HOME`, `HF_HUB_CACHE`, `HF_ENDPOINT`, `HF_HUB_OFFLINE`
|
|
225
225
|
- `COHERE_TRANSCRIBE_CACHE`
|
|
226
226
|
- `COHERE_TRANSCRIBE_NATIVE_LIBRARY`
|
|
227
227
|
- `COHERE_TRANSCRIBE_AUDIO_LIBRARY`
|
|
@@ -254,7 +254,7 @@ Installed-gem WER measurements use the standalone runner documented in [`benchma
|
|
|
254
254
|
|
|
255
255
|
### Releasing
|
|
256
256
|
|
|
257
|
-
Add a repository Actions secret named `RUBYGEMS_AUTH_TOKEN` containing a RubyGems API key with permission to push `cohere-transcribe`. Publishing a GitHub Release tagged `v0.1.
|
|
257
|
+
Add a repository Actions secret named `RUBYGEMS_AUTH_TOKEN` containing a RubyGems API key with permission to push `cohere-transcribe`. Publishing a GitHub Release tagged `v0.1.2` runs the release workflow, verifies that the tag matches `Cohere::Transcribe::VERSION`, builds the exact `cohere-transcribe-0.1.2.gem` artifact, and pushes it to RubyGems.org.
|
|
258
258
|
|
|
259
259
|
Normal CI runs the Ruby suite, style checks, signature validation, native CPU build and ABI smoke checks, and source-gem build on Linux and macOS. It does not run the installed-gem WER or performance benchmarks.
|
|
260
260
|
|
|
@@ -12,7 +12,7 @@ The gemspec should register `ext/cohere_transcribe_native/extconf.rb` in `spec.e
|
|
|
12
12
|
|
|
13
13
|
Packaged builds define `GGML_BACKTRACE_NO_DEBUGGER`: fatal ggml diagnostics use the platform's in-process backtrace implementation and never fork or execute `gdb`/`lldb`. The upstream debugger-attaching behavior remains unchanged when the vendored ggml snapshot is built outside this gem's CMake project.
|
|
14
14
|
|
|
15
|
-
CPU is the default. Optional build flags are:
|
|
15
|
+
CPU is the default. Source builds require CMake 3.15 or newer and a C++20 compiler. CUDA builds require CMake 3.18 or newer. Optional build flags are:
|
|
16
16
|
|
|
17
17
|
- `COHERE_TRANSCRIBE_NATIVE=1`: tune CPU code for the build machine.
|
|
18
18
|
- `COHERE_TRANSCRIBE_OPENMP=1`: enable OpenMP.
|
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
require "fileutils"
|
|
4
4
|
require "rbconfig"
|
|
5
|
+
require "rubygems/version"
|
|
5
6
|
require "shellwords"
|
|
7
|
+
require "open3"
|
|
6
8
|
|
|
7
9
|
EXTENSION_DIR = File.expand_path(__dir__)
|
|
8
10
|
GEM_ROOT = File.expand_path("../..", EXTENSION_DIR)
|
|
@@ -22,8 +24,16 @@ abort "COHERE_TRANSCRIBE_CUDA=1 is only supported on Linux" if cuda && !host_os.
|
|
|
22
24
|
abort "enable either CUDA or Metal, not both" if cuda && metal
|
|
23
25
|
|
|
24
26
|
cmake = ENV.fetch("CMAKE", "cmake")
|
|
25
|
-
|
|
26
|
-
|
|
27
|
+
begin
|
|
28
|
+
cmake_output, cmake_status = Open3.capture2e(cmake, "--version")
|
|
29
|
+
rescue SystemCallError
|
|
30
|
+
cmake_output = nil
|
|
31
|
+
cmake_status = nil
|
|
32
|
+
end
|
|
33
|
+
cmake_version_match = cmake_output&.match(/cmake version (\d+(?:\.\d+)+)/i)
|
|
34
|
+
cmake_version = cmake_version_match && cmake_version_match[1]
|
|
35
|
+
unless cmake_status&.success? && cmake_version && Gem::Version.new(cmake_version) >= Gem::Version.new("3.15")
|
|
36
|
+
abort "cmake 3.15 or newer is required to build cohere-transcribe's native runtime"
|
|
27
37
|
end
|
|
28
38
|
|
|
29
39
|
build_dir = File.expand_path(
|
|
@@ -297,6 +297,50 @@ module Cohere
|
|
|
297
297
|
4 => :error
|
|
298
298
|
}.freeze
|
|
299
299
|
|
|
300
|
+
# Keeps the native pointer independently of the Ruby session wrapper so
|
|
301
|
+
# ObjectSpace cleanup never retains the object it is meant to collect.
|
|
302
|
+
# Taking the pointer before calling the foreign close function makes
|
|
303
|
+
# explicit close, constructor rollback, and finalization idempotent.
|
|
304
|
+
class SessionOwnership
|
|
305
|
+
def initialize(library)
|
|
306
|
+
@library = library
|
|
307
|
+
@mutex = Mutex.new
|
|
308
|
+
@session = nil
|
|
309
|
+
end
|
|
310
|
+
|
|
311
|
+
def install(session)
|
|
312
|
+
@mutex.synchronize do
|
|
313
|
+
raise TranscriptionRuntimeError, "Native session ownership is already installed" if @session
|
|
314
|
+
|
|
315
|
+
@session = session
|
|
316
|
+
end
|
|
317
|
+
end
|
|
318
|
+
|
|
319
|
+
def close
|
|
320
|
+
session = @mutex.synchronize do
|
|
321
|
+
current = @session
|
|
322
|
+
@session = nil
|
|
323
|
+
current
|
|
324
|
+
end
|
|
325
|
+
return unless session
|
|
326
|
+
|
|
327
|
+
@library.call(:session_close, session)
|
|
328
|
+
nil
|
|
329
|
+
end
|
|
330
|
+
|
|
331
|
+
def finalize
|
|
332
|
+
close
|
|
333
|
+
rescue Exception # rubocop:disable Lint/RescueException -- finalizers must not escape during GC or shutdown
|
|
334
|
+
nil
|
|
335
|
+
end
|
|
336
|
+
end
|
|
337
|
+
private_constant :SessionOwnership
|
|
338
|
+
|
|
339
|
+
def self.finalizer_for(ownership)
|
|
340
|
+
proc { |_object_id| ownership.finalize }
|
|
341
|
+
end
|
|
342
|
+
private_class_method :finalizer_for
|
|
343
|
+
|
|
300
344
|
attr_reader :backend, :batch_capacity, :compute_backend, :device, :last_batch_metrics, :model_path
|
|
301
345
|
|
|
302
346
|
def initialize(model_path, options, threads: nil, library: NativeLibrary.load)
|
|
@@ -306,12 +350,24 @@ module Cohere
|
|
|
306
350
|
|
|
307
351
|
@mutex = Mutex.new
|
|
308
352
|
@closed = false
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
threads: thread_count
|
|
353
|
+
@session_ownership = SessionOwnership.new(@library)
|
|
354
|
+
ObjectSpace.define_finalizer(
|
|
355
|
+
self,
|
|
356
|
+
self.class.send(:finalizer_for, @session_ownership)
|
|
314
357
|
)
|
|
358
|
+
thread_count = normalize_threads(threads)
|
|
359
|
+
# An asynchronous exception may become deliverable immediately after
|
|
360
|
+
# the foreign open returns. Defer it until the pointer belongs to the
|
|
361
|
+
# independent cleanup state, then let the constructor rescue close it.
|
|
362
|
+
Thread.handle_interrupt(Exception => :never) do
|
|
363
|
+
session = @library.open_session(
|
|
364
|
+
model_path: @model_path,
|
|
365
|
+
device: options.device.to_s,
|
|
366
|
+
threads: thread_count
|
|
367
|
+
)
|
|
368
|
+
@session_ownership.install(session)
|
|
369
|
+
@session = session
|
|
370
|
+
end
|
|
315
371
|
@default_max_new_tokens = Integer(options.max_new_tokens)
|
|
316
372
|
@library.call(:session_set_max_new_tokens, @session, @default_max_new_tokens)
|
|
317
373
|
@library.call(:session_set_beam_size, @session, 1)
|
|
@@ -341,8 +397,12 @@ module Cohere
|
|
|
341
397
|
close
|
|
342
398
|
raise TranscriptionRuntimeError, "Native runtime reported an invalid Cohere batch capacity"
|
|
343
399
|
end
|
|
344
|
-
rescue
|
|
345
|
-
|
|
400
|
+
rescue Exception # rubocop:disable Lint/RescueException -- native ownership must roll back on Interrupt too
|
|
401
|
+
begin
|
|
402
|
+
close if defined?(@session_ownership)
|
|
403
|
+
rescue Exception # rubocop:disable Lint/RescueException -- preserve the constructor failure
|
|
404
|
+
nil
|
|
405
|
+
end
|
|
346
406
|
raise
|
|
347
407
|
end
|
|
348
408
|
|
|
@@ -449,9 +509,13 @@ module Cohere
|
|
|
449
509
|
@mutex.synchronize do
|
|
450
510
|
return if @closed
|
|
451
511
|
|
|
452
|
-
@library.call(:session_close, @session) if @library && @session
|
|
453
512
|
@session = nil
|
|
454
513
|
@closed = true
|
|
514
|
+
begin
|
|
515
|
+
@session_ownership&.close
|
|
516
|
+
ensure
|
|
517
|
+
ObjectSpace.undefine_finalizer(self)
|
|
518
|
+
end
|
|
455
519
|
end
|
|
456
520
|
nil
|
|
457
521
|
end
|
|
@@ -131,16 +131,60 @@ module Cohere
|
|
|
131
131
|
end
|
|
132
132
|
return nil unless SoundFileABI::AVAILABLE
|
|
133
133
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
134
|
+
SoundFileABI::SFInfo.malloc(Fiddle::RUBY_FREE) do |info|
|
|
135
|
+
handle = SoundFileABI.sf_open(source.to_s, SFM_READ, info.to_ptr)
|
|
136
|
+
return nil if handle.null?
|
|
137
137
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
138
|
+
frames = Integer(info.frames)
|
|
139
|
+
sample_rate = Integer(info.samplerate)
|
|
140
|
+
return nil unless frames >= 0 && sample_rate.positive?
|
|
141
|
+
|
|
142
|
+
seconds = frames.fdiv(sample_rate)
|
|
143
|
+
return seconds if seconds.finite? && seconds >= 0.0
|
|
144
|
+
|
|
145
|
+
nil
|
|
146
|
+
end
|
|
147
|
+
rescue Fiddle::DLError, SystemCallError, TranscriptionRuntimeError
|
|
148
|
+
nil
|
|
149
|
+
ensure
|
|
150
|
+
SoundFileABI.sf_close(handle) if defined?(handle) && handle && !handle.null?
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
# Best-effort upper bound for the buffers governed by max_decoded_bytes.
|
|
154
|
+
# The preparation scheduler uses it only for grouping; decode performs
|
|
155
|
+
# the authoritative check again against the per-file ceiling.
|
|
156
|
+
def estimate_decoded_bytes(path, backend: "auto", sample_rate: SAMPLE_RATE)
|
|
157
|
+
requested = backend
|
|
158
|
+
return unless requested.is_a?(String) && BACKENDS.include?(requested)
|
|
159
|
+
return unless sample_rate.is_a?(Integer) && sample_rate.positive?
|
|
160
|
+
|
|
161
|
+
source = Pathname(path).expand_path
|
|
162
|
+
return unless source.file?
|
|
163
|
+
|
|
164
|
+
native_ffmpeg_available = %w[auto librosa].include?(requested) && FFmpegNative.available?
|
|
165
|
+
use_ffmpeg = requested == "ffmpeg" || requested == "torchcodec" ||
|
|
166
|
+
(%w[auto librosa].include?(requested) && native_ffmpeg_available)
|
|
167
|
+
if use_ffmpeg
|
|
168
|
+
duration = FFmpegNative.duration(source)
|
|
169
|
+
return unless duration&.finite? && duration >= 0.0
|
|
170
|
+
|
|
171
|
+
return ((duration * sample_rate).ceil + 1) * Fiddle::SIZEOF_FLOAT
|
|
172
|
+
end
|
|
173
|
+
return unless SoundFileABI::AVAILABLE
|
|
174
|
+
|
|
175
|
+
SoundFileABI::SFInfo.malloc(Fiddle::RUBY_FREE) do |info|
|
|
176
|
+
handle = SoundFileABI.sf_open(source.to_s, SFM_READ, info.to_ptr)
|
|
177
|
+
next if handle.null?
|
|
178
|
+
|
|
179
|
+
frames = Integer(info.frames)
|
|
180
|
+
channels = Integer(info.channels)
|
|
181
|
+
source_rate = Integer(info.samplerate)
|
|
182
|
+
next unless frames >= 0 && channels.positive? && source_rate.positive?
|
|
141
183
|
|
|
142
|
-
|
|
143
|
-
|
|
184
|
+
input_bytes = frames * channels * Fiddle::SIZEOF_FLOAT
|
|
185
|
+
output_frames = (frames * sample_rate.fdiv(source_rate)).ceil + 64
|
|
186
|
+
[input_bytes, output_frames * Fiddle::SIZEOF_FLOAT].max
|
|
187
|
+
end
|
|
144
188
|
rescue Fiddle::DLError, SystemCallError, TranscriptionRuntimeError
|
|
145
189
|
nil
|
|
146
190
|
ensure
|
|
@@ -195,64 +239,67 @@ module Cohere
|
|
|
195
239
|
raise TranscriptionRuntimeError, "libsndfile is required for native audio decoding: #{sound_file_error}"
|
|
196
240
|
end
|
|
197
241
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
begin
|
|
203
|
-
frames = Integer(info.frames)
|
|
204
|
-
channels = Integer(info.channels)
|
|
205
|
-
source_rate = Integer(info.samplerate)
|
|
206
|
-
unless frames >= 0 && channels.positive? && source_rate.positive?
|
|
207
|
-
raise TranscriptionRuntimeError, "Decoder returned invalid audio metadata for #{source}"
|
|
208
|
-
end
|
|
209
|
-
|
|
210
|
-
input_bytes = frames * channels * Fiddle::SIZEOF_FLOAT
|
|
211
|
-
output_frames = (frames * sample_rate.fdiv(source_rate)).ceil + 64
|
|
212
|
-
projected_bytes = [input_bytes, output_frames * Fiddle::SIZEOF_FLOAT].max
|
|
213
|
-
if max_decoded_bytes && projected_bytes > max_decoded_bytes
|
|
214
|
-
raise TranscriptionRuntimeError,
|
|
215
|
-
"Decoded audio exceeds the configured memory limit for #{source} " \
|
|
216
|
-
"(#{projected_bytes} > #{max_decoded_bytes} bytes)"
|
|
217
|
-
end
|
|
218
|
-
|
|
219
|
-
raw = Fiddle::Pointer.malloc([input_bytes, 1].max, Fiddle::RUBY_FREE)
|
|
220
|
-
read_frames = SoundFileABI.sf_readf_float(handle, raw, frames)
|
|
221
|
-
raise TranscriptionRuntimeError, "Cannot decode #{source}: #{SoundFileABI.sf_strerror(handle)}" if read_frames.negative?
|
|
222
|
-
raise TranscriptionRuntimeError, "Decoder returned more frames than allocated for #{source}" if read_frames > frames
|
|
242
|
+
SoundFileABI::SFInfo.malloc(Fiddle::RUBY_FREE) do |info|
|
|
243
|
+
handle = SoundFileABI.sf_open(source.to_s, SFM_READ, info.to_ptr)
|
|
244
|
+
raise TranscriptionRuntimeError, "Cannot decode #{source}: #{SoundFileABI.sf_strerror(handle)}" if handle.null?
|
|
223
245
|
|
|
224
|
-
frames = read_frames
|
|
225
246
|
begin
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
247
|
+
frames = Integer(info.frames)
|
|
248
|
+
channels = Integer(info.channels)
|
|
249
|
+
source_rate = Integer(info.samplerate)
|
|
250
|
+
unless frames >= 0 && channels.positive? && source_rate.positive?
|
|
251
|
+
raise TranscriptionRuntimeError, "Decoder returned invalid audio metadata for #{source}"
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
input_bytes = frames * channels * Fiddle::SIZEOF_FLOAT
|
|
255
|
+
output_frames = (frames * sample_rate.fdiv(source_rate)).ceil + 64
|
|
256
|
+
projected_bytes = [input_bytes, output_frames * Fiddle::SIZEOF_FLOAT].max
|
|
257
|
+
if max_decoded_bytes && projected_bytes > max_decoded_bytes
|
|
258
|
+
raise TranscriptionRuntimeError,
|
|
259
|
+
"Decoded audio exceeds the configured memory limit for #{source} " \
|
|
260
|
+
"(#{projected_bytes} > #{max_decoded_bytes} bytes)"
|
|
261
|
+
end
|
|
262
|
+
|
|
263
|
+
raw = Fiddle::Pointer.malloc([input_bytes, 1].max, Fiddle::RUBY_FREE)
|
|
264
|
+
read_frames = SoundFileABI.sf_readf_float(handle, raw, frames)
|
|
265
|
+
raise TranscriptionRuntimeError, "Cannot decode #{source}: #{SoundFileABI.sf_strerror(handle)}" if read_frames.negative?
|
|
266
|
+
raise TranscriptionRuntimeError, "Decoder returned more frames than allocated for #{source}" if read_frames > frames
|
|
267
|
+
|
|
268
|
+
frames = read_frames
|
|
269
|
+
begin
|
|
270
|
+
require "numo/narray"
|
|
271
|
+
rescue LoadError => e
|
|
272
|
+
raise TranscriptionRuntimeError, "numo-narray is required for decoded audio: #{e.message}"
|
|
273
|
+
end
|
|
274
|
+
interleaved = if frames.zero?
|
|
275
|
+
nil
|
|
276
|
+
else
|
|
277
|
+
Numo::SFloat.from_binary(raw[0, frames * channels * Fiddle::SIZEOF_FLOAT])
|
|
278
|
+
end
|
|
279
|
+
mono = if frames.zero?
|
|
280
|
+
Numo::SFloat.zeros(0)
|
|
281
|
+
elsif channels == 1
|
|
282
|
+
interleaved
|
|
283
|
+
elsif channels == 2
|
|
284
|
+
(interleaved.reshape(frames, channels).sum(1) * Math.sqrt(0.5)).cast_to(Numo::SFloat)
|
|
285
|
+
else
|
|
286
|
+
interleaved.reshape(frames, channels).mean(1).cast_to(Numo::SFloat)
|
|
287
|
+
end
|
|
288
|
+
samples = source_rate == sample_rate ? mono : resample(mono, source_rate, sample_rate, max_decoded_bytes)
|
|
289
|
+
validate_finite!(samples)
|
|
290
|
+
Decoded.new(
|
|
291
|
+
samples: samples.freeze,
|
|
292
|
+
sample_rate: sample_rate,
|
|
293
|
+
backend: "libsndfile",
|
|
294
|
+
fallback_reason: if %w[auto libsndfile].include?(requested)
|
|
295
|
+
nil
|
|
296
|
+
else
|
|
297
|
+
"Ruby #{requested} compatibility mode uses the native libsndfile ABI"
|
|
298
|
+
end
|
|
299
|
+
)
|
|
300
|
+
ensure
|
|
301
|
+
SoundFileABI.sf_close(handle)
|
|
229
302
|
end
|
|
230
|
-
interleaved = if frames.zero?
|
|
231
|
-
nil
|
|
232
|
-
else
|
|
233
|
-
Numo::SFloat.from_binary(raw[0, frames * channels * Fiddle::SIZEOF_FLOAT])
|
|
234
|
-
end
|
|
235
|
-
mono = if frames.zero?
|
|
236
|
-
Numo::SFloat.zeros(0)
|
|
237
|
-
elsif channels == 1
|
|
238
|
-
interleaved
|
|
239
|
-
else
|
|
240
|
-
interleaved.reshape(frames, channels).mean(1).cast_to(Numo::SFloat)
|
|
241
|
-
end
|
|
242
|
-
samples = source_rate == sample_rate ? mono : resample(mono, source_rate, sample_rate, max_decoded_bytes)
|
|
243
|
-
validate_finite!(samples)
|
|
244
|
-
Decoded.new(
|
|
245
|
-
samples: samples.freeze,
|
|
246
|
-
sample_rate: sample_rate,
|
|
247
|
-
backend: "libsndfile",
|
|
248
|
-
fallback_reason: if %w[auto libsndfile].include?(requested)
|
|
249
|
-
nil
|
|
250
|
-
else
|
|
251
|
-
"Ruby #{requested} compatibility mode uses the native libsndfile ABI"
|
|
252
|
-
end
|
|
253
|
-
)
|
|
254
|
-
ensure
|
|
255
|
-
SoundFileABI.sf_close(handle)
|
|
256
303
|
end
|
|
257
304
|
end
|
|
258
305
|
|
|
@@ -277,26 +324,27 @@ module Cohere
|
|
|
277
324
|
input_string = samples.to_binary
|
|
278
325
|
input_pointer = Fiddle::Pointer[input_string]
|
|
279
326
|
output_pointer = Fiddle::Pointer.malloc([bytes, 1].max, Fiddle::RUBY_FREE)
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
327
|
+
SampleRateABI::SRCData.malloc(Fiddle::RUBY_FREE) do |data|
|
|
328
|
+
data.data_in = input_pointer
|
|
329
|
+
data.data_out = output_pointer
|
|
330
|
+
data.input_frames = samples.length
|
|
331
|
+
data.output_frames = output_capacity
|
|
332
|
+
data.input_frames_used = 0
|
|
333
|
+
data.output_frames_gen = 0
|
|
334
|
+
data.end_of_input = 1
|
|
335
|
+
data.src_ratio = ratio
|
|
336
|
+
error_code = SampleRateABI.src_simple(data.to_ptr, SRC_SINC_FASTEST, 1)
|
|
337
|
+
raise TranscriptionRuntimeError, "Audio resampling failed: #{SampleRateABI.src_strerror(error_code)}" unless error_code.zero?
|
|
338
|
+
|
|
339
|
+
generated = Integer(data.output_frames_gen)
|
|
340
|
+
consumed = Integer(data.input_frames_used)
|
|
341
|
+
unless consumed == samples.length && generated.between?(0, output_capacity)
|
|
342
|
+
raise TranscriptionRuntimeError,
|
|
343
|
+
"Audio resampling returned invalid frame counts " \
|
|
344
|
+
"(consumed #{consumed}/#{samples.length}, generated #{generated}/#{output_capacity})"
|
|
345
|
+
end
|
|
346
|
+
return Numo::SFloat.from_binary(output_pointer[0, generated * Fiddle::SIZEOF_FLOAT])
|
|
298
347
|
end
|
|
299
|
-
Numo::SFloat.from_binary(output_pointer[0, generated * Fiddle::SIZEOF_FLOAT])
|
|
300
348
|
end
|
|
301
349
|
|
|
302
350
|
def validate_finite!(samples)
|
|
@@ -111,6 +111,28 @@ module Cohere
|
|
|
111
111
|
end
|
|
112
112
|
|
|
113
113
|
def decode(path, sample_rate:, max_decoded_bytes:)
|
|
114
|
+
worker = Thread.new do
|
|
115
|
+
decode_owned(path, sample_rate: sample_rate, max_decoded_bytes: max_decoded_bytes)
|
|
116
|
+
end
|
|
117
|
+
worker.name = "cohere-ffmpeg-decode" if worker.respond_to?(:name=)
|
|
118
|
+
worker.report_on_exception = false
|
|
119
|
+
completed = false
|
|
120
|
+
result = worker.value
|
|
121
|
+
completed = true
|
|
122
|
+
result
|
|
123
|
+
ensure
|
|
124
|
+
if defined?(worker) && worker && !completed
|
|
125
|
+
Thread.handle_interrupt(Exception => :never) do
|
|
126
|
+
@functions.fetch(:cancel).call if worker.alive?
|
|
127
|
+
rescue StandardError
|
|
128
|
+
nil
|
|
129
|
+
ensure
|
|
130
|
+
worker.join
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
def decode_owned(path, sample_rate:, max_decoded_bytes:)
|
|
114
136
|
output_slot = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP, Fiddle::RUBY_FREE)
|
|
115
137
|
output_slot[0, Fiddle::SIZEOF_VOIDP] = [0].pack("J")
|
|
116
138
|
count_slot = Fiddle::Pointer.malloc(Fiddle::SIZEOF_INT64_T, Fiddle::RUBY_FREE)
|
|
@@ -119,46 +141,51 @@ module Cohere
|
|
|
119
141
|
message[0, ERROR_CAPACITY] = "\0" * ERROR_CAPACITY
|
|
120
142
|
maximum = max_decoded_bytes || 0
|
|
121
143
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
144
|
+
address = 0
|
|
145
|
+
Thread.handle_interrupt(Exception => :never) do
|
|
146
|
+
status = @functions.fetch(:decode).call(
|
|
147
|
+
c_string(path.to_s),
|
|
148
|
+
sample_rate,
|
|
149
|
+
maximum,
|
|
150
|
+
output_slot,
|
|
151
|
+
count_slot,
|
|
152
|
+
message,
|
|
153
|
+
ERROR_CAPACITY
|
|
154
|
+
)
|
|
155
|
+
address = output_slot[0, Fiddle::SIZEOF_VOIDP].unpack1("J")
|
|
156
|
+
begin
|
|
157
|
+
Thread.handle_interrupt(Exception => :immediate) do
|
|
158
|
+
count = count_slot[0, Fiddle::SIZEOF_INT64_T].unpack1("q")
|
|
159
|
+
detail = message.to_s.force_encoding(Encoding::UTF_8).scrub
|
|
160
|
+
unless status.zero?
|
|
161
|
+
raise Interrupt, "Native FFmpeg decode was cancelled for #{path}: #{detail}" if status == CANCELLED_STATUS
|
|
162
|
+
|
|
163
|
+
raise TranscriptionRuntimeError, "Cannot decode #{path} through native FFmpeg: #{detail}"
|
|
164
|
+
end
|
|
165
|
+
if count.negative? || (count.positive? && address.zero?)
|
|
166
|
+
raise TranscriptionRuntimeError, "Native FFmpeg returned invalid output metadata for #{path}"
|
|
167
|
+
end
|
|
168
|
+
|
|
169
|
+
bytes = count * Fiddle::SIZEOF_FLOAT
|
|
170
|
+
if max_decoded_bytes && bytes > max_decoded_bytes
|
|
171
|
+
raise TranscriptionRuntimeError,
|
|
172
|
+
"Native FFmpeg exceeded the configured decoded-audio memory limit for #{path}"
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
begin
|
|
176
|
+
require "numo/narray"
|
|
177
|
+
if count.zero?
|
|
178
|
+
Numo::SFloat.zeros(0)
|
|
179
|
+
else
|
|
180
|
+
Numo::SFloat.from_binary(Fiddle::Pointer.new(address)[0, bytes])
|
|
181
|
+
end
|
|
182
|
+
rescue LoadError => e
|
|
183
|
+
raise TranscriptionRuntimeError, "numo-narray is required for decoded audio: #{e.message}"
|
|
184
|
+
end
|
|
185
|
+
end
|
|
186
|
+
ensure
|
|
187
|
+
@functions.fetch(:free).call(address) unless address.zero?
|
|
157
188
|
end
|
|
158
|
-
rescue LoadError => e
|
|
159
|
-
raise TranscriptionRuntimeError, "numo-narray is required for decoded audio: #{e.message}"
|
|
160
|
-
ensure
|
|
161
|
-
@functions.fetch(:free).call(address) unless address.zero?
|
|
162
189
|
end
|
|
163
190
|
end
|
|
164
191
|
|
|
@@ -194,6 +221,8 @@ module Cohere
|
|
|
194
221
|
|
|
195
222
|
private
|
|
196
223
|
|
|
224
|
+
private :decode_owned
|
|
225
|
+
|
|
197
226
|
def c_string(value)
|
|
198
227
|
Fiddle::Pointer["#{value}\0"]
|
|
199
228
|
end
|
|
@@ -207,6 +207,8 @@ module Cohere
|
|
|
207
207
|
|
|
208
208
|
valid = valid_repository_id?(text)
|
|
209
209
|
invalid!("#{option} must be a valid Hugging Face repository ID or local directory") unless valid
|
|
210
|
+
rescue EncodingError
|
|
211
|
+
invalid!("#{option} must be a valid Hugging Face repository ID or local directory")
|
|
210
212
|
rescue ArgumentError => e
|
|
211
213
|
invalid!(e.message)
|
|
212
214
|
end
|