cohere-transcribe 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: a989c36d7ff926d3e9a0e45bbe66a0ece7c6760deaadefa415bf4c975ae64d93
+  data.tar.gz: 413ce9a40e4915f6167f8058d5548f95d87dd65b2d1177233d906524fa0cd2e2
+SHA512:
+  metadata.gz: 9adf6b3e0aab57f248556a856793d582e8a1fcb64de892229ba40757ee499fe679f15c4c167727cadef4fa9d3dd28d162866c997cf72334a3b8f85c2943dde60
+  data.tar.gz: 2a1f3fd82aec431e244e8a2cc4315c1abf008d332b16402b29297397816f81fb8328f9236722d66a648287119832d0c50f4b0428152d6385bd6b19e7ff21fa7e

data/CHANGELOG.md

@@ -0,0 +1,21 @@
+# Changelog
+
+## [0.1.0] - 2026-07-15
+
+- Release the independent Ruby gem with output schema 8 and profile schema 9.
+- License original gem code and documentation under Apache License 2.0 and package Ruby-specific notices for every retained third-party component.
+- Add Ruby 4 Linux/macOS CI with native ABI checks, monthly dependency updates, and GitHub Release publishing to RubyGems.org; WER and performance benchmarks remain outside CI.
+- Unify directory-sync, staged-output cleanup, rollback reporting, backup preservation, target-mode lookup, and finite-JSON behavior across transcript, profile, checkpoint, and manifest publication.
+- Add bounded parallel feature extraction, fused Conformer SiGLU execution, flattened decoder projections, device-side token selection, and the fast padded-encoder projection path. Across three fresh-process runs of the 69.34-minute audio benchmark, the final CUDA path has a median throughput of 118.82x real time; the matching Python package has a median throughput of 119.64x.
+- Add a standalone installed-gem WER benchmark that remains outside the test suite and CI. On the balanced-500 corpus the final public API scores 22.7534% lexical WER and the native B24 lane scores 23.1724%, compared with 22.9551% for the retained Python BF16 reference.
+- Refresh the reproducible Ruby 4 source gem, isolated install checks, native ABI checks, and real CPU/CUDA Dense-model coverage.
+- Port the complete public Ruby API and 54-option CLI from the Python package.
+- Add native Dense Safetensors/PyTorch-state-dict-to-GGUF conversion for F16, BF16, and F32 weights, including compatible Hub and local Dense fine-tunes. Cache keys fingerprint dtype and source metadata, while completion markers bind the source fingerprint to the converted file's filesystem identity.
+- Add 24-row logical native ASR batches: padded encoder work is microbatched at eight rows and gathered into one ragged decoder call. Include bounded adaptive growth, typed native failure diagnostics, OOM-only cap learning and recursive failure splitting, exact EOS/token-limit provenance, targeted token retries, and repetition stopping.
+- Add ordered, memory-bounded decode/VAD worker groups with one-group ASR look-ahead, worker-confined Silero sessions, and deterministic error isolation.
+- Make packed-compatible Silero tuning effective through the packaged ONNX substitute: `vad_threads` configures CPU intra-op execution, `vad_block_frames` bounds temporal calls exactly, and `vad_batch_size` caps independent-file session concurrency. Report exact single-stream call/frame telemetry and ONNX provider introspection in JSON and profiles.
+- Add two-phase multi-file word execution with one Dense session, a hard Dense/MMS eviction barrier, bounded concrete-backend PCM reload, and direct checkpoint-to-alignment resume.
+- Add source-bound ASR checkpoints and preflight verification. Verified skips avoid PCM decode, while render-only segment/none resumes avoid decode, VAD, and Dense model loading. Profiles receive private batch/checkpoint telemetry and raw-versus-selected speech-span measurements without changing the public statistics type.
+- Add exact Silero v6 ONNX, Auditok-compatible energy, and fixed-window VAD.
+- Add subprocess-free FFmpeg 4–8 C-ABI audio decoding for all 15 accepted media extensions, cooperative cancellation and decode/probe deadlines, metadata duration probing, plus libsndfile/libsamplerate fallback, word/segment timing, subtitle rendering, transactional publication, profiling, diagnostics, and immutable result types.
+- Add source-buildable native runtime packaging for Ruby 4 on Linux and macOS.

data/LICENSE.txt

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

data/NOTICE

@@ -0,0 +1,5 @@
+cohere-transcribe Ruby gem
+Copyright 2026 Ali Hamdi Ali Fadel
+
+This product includes software developed for the cohere-transcribe Ruby gem:
+https://github.com/AliOsm/cohere-transcribe-ruby

data/README.md

@@ -0,0 +1,265 @@
+# cohere-transcribe
+
+Ruby 4 transcription for native Transformers Cohere ASR checkpoints. The gem mirrors the Python package's public API and command-line interface while keeping inference entirely in Ruby and native C/C++ ABIs—there is no Python runtime, worker, subprocess, or model server.
+
+The core Dense path supports the pinned Arabic/English model, compatible Hub fine-tunes, and local Dense checkpoints. It includes 16 kHz audio decoding/resampling, Silero v6 ONNX VAD, Auditok-compatible energy VAD, fixed-window segmentation, native batched inference, exact decoder stop/retry metadata, pinned MMS CTC word alignment, subtitle cues, transactional publication, progress callbacks, and reusable model sessions.
+
+## Requirements
+
+- 64-bit Ruby `>= 4.0, < 5.0` (the Dense model exceeds a 32-bit address space)
+- 64-bit Linux or macOS; the packaged native source build does not currently support Windows
+- CMake 3.14+ and a C++20 compiler when installing from source
+- FFmpeg 4–8 shared libraries for the full accepted container/codec set
+- `libsndfile` and `libsamplerate` shared libraries for the fallback used when the native FFmpeg adapter is unavailable
+- A Hugging Face token when the selected model requires authentication
+- Enough storage for the source checkpoint and its cached F16, BF16, or F32 GGUF conversion; word alignment additionally downloads a pinned 1.18 GiB FP32 MMS ONNX model (or 603 MiB FP16 model on CUDA)
+
+The native extension builds a portable CPU backend by default. Accelerator availability depends on the build flags and platform; set `COHERE_TRANSCRIBE_NATIVE_LIBRARY` to use a separately built CrispASR library when needed.
+
+Dense conversion and inference support F16, BF16, and F32 GGUF weights for the pinned checkpoint and compatible Dense fine-tunes. CPU execution resolves to FP32. CUDA `auto` selects BF16 when the loaded runtime reports hardware support and otherwise selects FP16; MPS `auto` selects FP16. An explicit unsupported BF16 accelerator request fails instead of silently changing precision.
+
+Audio decoding never launches the `ffmpeg` executable. The packaged `libcohere_audio` adapter dynamically binds a compatible libavformat/libavcodec/libavutil/libswresample tuple and decodes the first audio stream directly to mono 16 kHz float PCM. This path covers AAC, AIFF, ALAC, FLAC, M4A, MP3, MP4, Ogg/Vorbis, Opus, WAV, WebM, and WMA. Explicit `torchcodec` and `librosa` compatibility modes use the same FFmpeg codec runtime through that C ABI, preserving the complete accepted-format set while reporting the concrete decoder in result provenance. If the adapter is absent, `auto` and `librosa` can fall back to libsndfile/libsamplerate for the formats those libraries support.
+
+The FFmpeg adapter gives duration probes and full decodes monotonic deadlines (30 seconds and one hour, respectively) and shares a generation-based cooperative cancellation hook. It supplies FFmpeg's public `AVIOInterruptCB` to input I/O and also checks cancellation between codec and resampler calls. Cancellation therefore takes effect when FFmpeg invokes that callback or returns control; it does not forcibly unwind an uninterruptible codec or system call.
+
+## Installation
+
+The release artifact is a source gem. RubyGems installs the Ruby dependencies automatically and then builds the native runtime locally. Before continuing, confirm that `ruby --version` reports a 64-bit Ruby 4.x installation.
+
+### Ubuntu or Debian: CPU
+
+```bash
+sudo apt update
+sudo apt install build-essential cmake ffmpeg libsndfile1 libsamplerate0
+gem install cohere-transcribe
+```
+
+This creates the portable CPU build. The `ffmpeg` package supplies the shared codec libraries used by the native audio adapter; the gem does not invoke the `ffmpeg` executable or require FFmpeg development headers.
+
+### Ubuntu or Debian: CUDA
+
+Install the NVIDIA driver and CUDA toolkit by following NVIDIA's [CUDA installation guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/), and confirm that `nvcc` is available. CUDA builds require CMake 3.18 or newer.
+
+```bash
+sudo apt update
+sudo apt install build-essential cmake ffmpeg libsndfile1 libsamplerate0
+COHERE_TRANSCRIBE_CUDA=1 gem install cohere-transcribe
+```
+
+If the same gem version is already installed as a CPU build, add `--force` to the CUDA installation command so RubyGems rebuilds the native extension.
+
+### macOS: CPU or Metal
+
+Install Apple's command-line build tools and the runtime libraries from Homebrew:
+
+```bash
+xcode-select --install
+brew install cmake ffmpeg libsndfile libsamplerate
+```
+
+For CPU:
+
+```bash
+gem install cohere-transcribe
+```
+
+For Metal:
+
+```bash
+COHERE_TRANSCRIBE_METAL=1 gem install cohere-transcribe
+```
+
+The Metal build additionally requires Xcode's `metal` and `metallib` tools; Apple documents the separately installable [Metal toolchain](https://developer.apple.com/documentation/Xcode/downloading-and-installing-additional-xcode-components). Add `--force` when replacing an existing CPU installation of the same version.
+
+### Bundler
+
+Inside an application, `bundle add` can replace `gem install`; the CUDA and Metal environment flags work the same way:
+
+```bash
+bundle add cohere-transcribe
+```
+
+### Verify the installation
+
+```bash
+cohere-transcribe-doctor
+cohere-transcribe-doctor --model-access
+```
+
+The first command checks the local runtime without loading ASR weights. The second also resolves the selected model metadata and may require `HF_TOKEN`. Model weights are downloaded on first use rather than during `gem install`.
+
+See [the native runtime build notes](ext/cohere_transcribe_native/README.md) for host CPU tuning, OpenMP, parallel build jobs, custom CMake arguments, and external native-library overrides.
+
+## Ruby API
+
+One-shot transcription creates and closes a session automatically:
+
+```ruby
+require "cohere/transcribe"
+
+options = Cohere::Transcribe::TranscriptionOptions.new(
+  language: "ar",
+  device: "auto",
+  vad: "silero",
+  vad_engine: "auto",
+  alignment: "word"
+)
+
+run = Cohere::Transcribe.transcribe("speech.wav", options: options)
+result = run.single
+
+puts result.text
+result.words.each do |word|
+  puts "%.2f..%.2f %s" % [word.start, word.end, word.text]
+end
+```
+
+Retain the model for repeated calls with `Transcriber`:
+
+```ruby
+transcriber = Cohere::Transcribe::Transcriber.new(options)
+begin
+  first = transcriber.transcribe("first.wav")
+  second = transcriber.transcribe(["second.wav", "recordings/"])
+ensure
+  transcriber.close
+end
+```
+
+Inputs may be a string, a path-like object, or an ordered array of files and directories. Directory expansion is deterministic, recursive by default, and deduplicates canonical paths. A run preserves that expanded order and provides `successful`, `failed`, `skipped`, `single`, and `ok?` helpers.
+
+Per-file media failures are returned as failed results so the rest of a batch can finish. Pass `raise_on_error: true` to raise `BatchTranscriptionError`; its `run` still contains every completed result.
+
+Progress callbacks receive immutable `ProgressEvent` values:
+
+```ruby
+callback = ->(event) do
+  if event.message
+    warn event.message
+  elsif event.total
+    warn "#{event.stage}: #{event.current}/#{event.total}"
+  end
+end
+
+run = Cohere::Transcribe.transcribe("speech.wav", progress: callback)
+```
+
+## Durable outputs
+
+The in-memory API writes nothing by default. Add `PublicationOptions` to create TXT, SRT, VTT, and JSON files:
+
+```ruby
+publication = Cohere::Transcribe::PublicationOptions.new(
+  formats: %w[txt srt vtt json],
+  output_dir: "transcripts",
+  existing: "error", # error, overwrite, or skip
+  profile_json: "transcripts/profile.json"
+)
+
+options = Cohere::Transcribe::TranscriptionOptions.new(publication: publication)
+run = Cohere::Transcribe.transcribe("recordings", options: options)
+```
+
+All formats for one source are staged before commit. Existing outputs are preserved if staging fails, and directory-relative structure is retained beneath `output_dir`.
+
+Durable publication binds the planned output root and parent directory inodes, then performs staging, backup, commit, rollback, and cleanup relative to a retained directory descriptor. This prevents a concurrent directory rename or symlink replacement from redirecting transcript, checkpoint, manifest, or profile bytes. Publication fails closed if the planned path identity changes or if the platform lacks `O_NOFOLLOW` and the POSIX `openat`, `renameat`, and `unlinkat` primitives; the supported Linux and macOS targets provide them.
+
+Publication state records the source's canonical path, device, inode, size, nanosecond mtime, and nanosecond ctime, plus checksums for the state payload and published artifacts. With `existing: "skip"`, a verified manifest is resolved before PCM decode, VAD, or a Dense model session is opened; a best-effort metadata duration probe supplies the skipped result's duration without materializing PCM.
+
+When publication is enabled, completed Dense ASR is checkpointed before timing and rendering. A later run validates the source snapshot, ASR contract, and checkpoint contents during the same preflight. Render-only changes can then rebuild segment/none-aligned outputs without decoding audio, running VAD, or opening Dense. Word-aligned resumes likewise skip Dense and VAD preparation, but re-decode through the recorded concrete audio backend because MMS needs the full waveform.
+
+`profile_json` writes the Python-compatible profile schema while keeping controller details private to the profile rather than expanding the stable `TranscriptionStatistics` value. Its ASR section includes effective batch minimum/maximum, final size/cap, batch history, and checkpoint written/resumed counts. Per-file rows retain raw VAD span counts/durations and selected-audio duration even when `vad_merge` combines those spans; JSON transcript output similarly keeps the original spans in `segmentation_details.speech_spans`.
+
+## CLI
+
+```bash
+cohere-transcribe interview.wav
+cohere-transcribe recordings/ --language ar --formats txt srt vtt json
+cohere-transcribe speech.wav --vad auditok --alignment segment
+cohere-transcribe speech.wav --vad none --max-dur 30 --text-only
+```
+
+Run `cohere-transcribe --help` for the complete Python-compatible 54-option interface. Exit status is `0` on success, `1` when files fail, `2` for command-line errors, `130` for interruption, and `143` for termination.
+
+## Native batching and generation controls
+
+The segment runtime uses CrispASR's padded Cohere encoder and ragged greedy decoder for true native batches. `batch_size`, `batch_max_size`, `batch_audio_seconds`, and `adaptive_batch` control row count, padded-audio budget, and bounded growth. One native session call accepts up to 24 logical rows. Internally it runs consecutive padded-encoder microbatches of at most eight rows, gathers their valid encoder states, and feeds all logical rows to one ragged decoder call. The Ruby controller caps itself to the capacity reported by the loaded session, and the session ABI rejects calls above it. Allocation failures are split recursively without discarding successful rows. Word-aligned requests retain the same native ASR batching. They run as two explicit phases: every file completes ASR with one retained Dense session, then Dense is evicted once before one retained MMS session aligns and publishes the completed files. The phases therefore never co-reside the 2B ASR checkpoint and 300M aligner.
+
+Native inference failures carry a thread-local error kind and diagnostic message across the C ABI. Ruby maps invalid arguments and invariant violations to fatal failures, allocator failures to OOM, and ordinary runtime failures to isolatable errors. Only typed OOM failures teach a smaller adaptive batch cap; fatal failures open the retained session's circuit breaker.
+
+For multi-file runs, `preprocess_workers` concurrently decodes and segments one ordered preparation group while `pipeline_preparation` permits exactly one next group to overlap current ASR. Each pipelined group is capped at the smaller of half `audio_memory_gb` and 512 MiB of retained mono float PCM, so the current and next groups remain within the configured PCM budget; native codec transients can add short-lived overhead. Disable `pipeline_preparation` to use the full per-file budget on a sequential path, which is the escape hatch for a single file larger than the half-budget pipeline slot. Automatic worker selection uses one worker for one file and at most two otherwise; explicit counts are capped by available processors and the group size. Decode/VAD failures remain isolated per file, and results, progress events, and publication always follow input order. Word alignment does not retain those prepared waveforms across its phase barrier. It re-decodes one file at a time through the concrete backend recorded during ASR (with one look-ahead only when the adjacent PCM pair fits `audio_memory_gb`) and rejects backend or sample-count drift. Resumable ASR checkpoints enter the alignment phase directly without reopening Dense or repeating decode/VAD preparation.
+
+`pin_memory` remains accepted for Python API/CLI compatibility, but resolves to `false` in this ggml runtime. Ruby passes float PCM directly through the native session ABI, so there is no PyTorch host tensor to pin and no nonblocking tensor transfer for the option to accelerate.
+
+Generated-token counts come directly from decoder IDs rather than rendered word estimates. Rows that reach `max_new_tokens` without EOS follow `truncation_policy` and retry only the affected rows up to `max_retry_tokens`. `stop_repetition_loops` applies the same conservative 96-token, four-repeat, 8–32-token-period guard as the Python runtime, and all stop/retry decisions are recorded per segment in provenance.
+
+## VAD and timing modes
+
+- `vad: "silero"` runs the packaged Silero v6 ONNX graph with recurrent state and the same sample-domain timestamp state machine as Silero 6.2.1. `auto` and `onnx` select it directly; `torch` and `jit` requests use the equivalent ONNX graph and record that executor substitution in provenance.
+
+The packed-Torch tuning options remain effective without Python. For requested `auto` or `torch`, `vad_block_frames` is the exact maximum number of temporal frames sent to one sequence-ONNX call, and `vad_threads` sets ONNX Runtime's CPU intra-op SessionOption (one thread when omitted). The reference profile schema has no ONNX-session thread field, so the effective value is disclosed in its legacy `vad.torch_intraop_threads` slot. Requested `onnx` or `jit` retains the reference sequence runner's 256-frame block and ignores these packed-only knobs; `vad_threads` remains rejected for those engines as in Python.
+
+The packaged graph has temporal input `[seq_len, 576]` but recurrent h/c inputs fixed at `[1, 1, 128]`; it has no file-batch axis, lengths, or mask. Concatenating files would therefore leak recurrent state between recordings. Ruby instead uses one thread-confined session per active file and makes `vad_batch_size` an upper bound on that independent-file concurrency for requested `auto`/`torch`. The effective count is also bounded by `preprocess_workers`, CPU availability, group size, and the sequential preparation mode. Profiles consequently report `max_files_per_call: 1`, exact temporal model-call/frame counts, and provider options matching ONNX Runtime introspection (`CPUExecutionProvider: {}`), while the configured batch/block values and effective temporal block remain visible.
+- `vad: "auditok"` uses a native Ruby implementation of Auditok's 50 ms PCM16 log-RMS tokenizer.
+- `vad: "none"` creates bounded fixed windows using `max_dur`.
+- `alignment: "word"` computes full-file emissions with the exact pinned MMS-300M forced-aligner ONNX export, then runs a pure-Ruby float32 CTC Viterbi kernel. An unalignable segment alone falls back to bounded uniform timing without dropping transcript words.
+- `alignment: "segment"` distributes words uniformly over the segment's speech spans.
+- `alignment: "none"` returns plain text without words or cues.
+
+## Models and cache
+
+The default model and revision are pinned:
+
+```text
+CohereLabs/cohere-transcribe-arabic-07-2026
+0a8193caa4f3f92131471ab08824e488141cb392
+```
+
+Hub artifacts reuse the standard Hugging Face cache. Dense Safetensors and `pytorch_model.bin` weights are streamed into a GGUF conversion once and reused from `~/.cache/cohere-transcribe`. PyTorch metadata is decoded by a restricted, allowlist-only Ruby reader; it never imports Python or executes pickle globals. Current `torch.save` ZIP files and the preceding raw-storage stream format are supported, including sharded indexes and strided tensors. Ancient tar-format checkpoints are rejected because they cannot be interpreted with the same restricted weights-only contract; re-save those as Safetensors or a current state dict. The retained-session identity includes the resolved device and dtype as well as the model identity. Each converted artifact has an independent cache key for its output dtype and a SHA-256 source fingerprint over the model ID/revision, relative source paths, sizes, mtimes, and ctimes, so even a same-size, mtime-preserving rewrite invalidates a local fine-tune through its changed ctime. A sidecar completion marker binds that source fingerprint, output dtype, and cache layout to the converted GGUF's device, inode, size, mtime, and ctime. This marker is stored beside the model as `*.complete.json`. The cache accepts only regular, non-symlink GGUF/marker files; conversion locks use no-follow opens where available and verify that the opened descriptor still matches the path's device and inode.
+
+Word mode uses `onnx-community/mms-300m-1130-forced-aligner-ONNX@2100fb247d8e43962eef24491597fbeb8b469531`, an ONNX export of `MahmoudAshraf/mms-300m-1130-forced-aligner@49402e9577b1158620820667c218cd494cc44486`. The default `align_dtype: "fp32"` works with the CPU provider even when Dense ASR runs on CUDA. `align_dtype: "fp16"` requires a CUDA-enabled ONNX Runtime; point `COHERE_TRANSCRIBE_ONNXRUNTIME_LIBRARY` at that runtime when it is not the one supplied by the installed `onnxruntime` gem. The runtime verifies the complete model file against a pinned byte size and SHA-256 before loading it. These downloaded model weights are CC-BY-NC-4.0 and are not distributed inside the gem; see `lib/cohere/transcribe/alignment/ATTRIBUTION.md` for provenance and notices.
+
+Useful environment variables:
+
+- `HF_TOKEN`, `HF_HOME`, `HF_HUB_CACHE`, `HF_ENDPOINT`
+- `COHERE_TRANSCRIBE_CACHE`
+- `COHERE_TRANSCRIBE_NATIVE_LIBRARY`
+- `COHERE_TRANSCRIBE_AUDIO_LIBRARY`
+- `COHERE_TRANSCRIBE_GGML_LIBRARY`
+- `COHERE_TRANSCRIBE_SNDFILE_LIBRARY`
+- `COHERE_TRANSCRIBE_SAMPLERATE_LIBRARY`
+- `COHERE_TRANSCRIBE_ONNXRUNTIME_LIBRARY`
+- `COHERE_TRANSCRIBE_THREADS`
+
+Advanced deployments may pin the four dynamically loaded codec libraries with `COHERE_TRANSCRIBE_AVFORMAT_LIBRARY`, `COHERE_TRANSCRIBE_AVCODEC_LIBRARY`, `COHERE_TRANSCRIBE_AVUTIL_LIBRARY`, and `COHERE_TRANSCRIBE_SWRESAMPLE_LIBRARY`; set all four together.
+
+### Saved quantization and adapters
+
+Saved bitsandbytes checkpoints and PEFT/LoRA adapters are detected and rejected explicitly; they are not silently treated as Dense weights.
+
+The official [Transformers bitsandbytes contract](https://huggingface.co/docs/transformers/quantization/bitsandbytes) defines INT8 execution with an FP16 outlier path and INT4 NF4/FP4 storage with optional nested quantization. Those layouts and execution rules are not GGML Q8/Q4 tensor formats. A bounded-memory converter could dequantize either format to this gem's Dense GGUF contract, but the result would be a roughly Dense-sized model with Dense inference behavior, not native bitsandbytes support or equivalent memory/performance characteristics.
+
+Likewise, the official [PEFT LoRA contract](https://huggingface.co/docs/peft/main/package_reference/lora) is broader than a pair of low-rank matrices on `Linear` weights: supported layouts can include embedding and convolution adapters, DoRA magnitude vectors, saved module replacements, bias updates, and sparse trainable-token state. A future adapter importer must define and validate a narrower mergeable profile, merge it before GGUF conversion, and bind the adapter configuration and weights into the conversion fingerprint and completion marker. Until that contract exists, explicit rejection avoids a plausible-looking but incomplete merge.
+
+## Development
+
+```bash
+bundle install
+bundle exec rake test
+```
+
+The test suite contains pure-Ruby unit coverage, ABI-boundary tests, real audio decode/resample tests, exact Silero differential tests against Python, converter fixtures, and optional live Dense-model smoke tests.
+
+Installed-gem WER measurements use the standalone runner documented in [`benchmark/README.md`](benchmark/README.md). The inference benchmark is intentionally separate from `bundle exec rake` and CI.
+
+### Releasing
+
+Add a repository Actions secret named `RUBYGEMS_AUTH_TOKEN` containing a RubyGems API key with permission to push `cohere-transcribe`. Publishing a GitHub Release tagged `v0.1.0` runs the release workflow, verifies that the tag matches `Cohere::Transcribe::VERSION`, builds the exact `cohere-transcribe-0.1.0.gem` artifact, and pushes it to RubyGems.org.
+
+Normal CI runs the Ruby suite, style checks, signature validation, native CPU build and ABI smoke checks, and source-gem build on Linux and macOS. It does not run the installed-gem WER or performance benchmarks.
+
+## License
+
+The gem's original Ruby code and documentation are licensed under Apache License 2.0. Packaged third-party components retain their own terms: CC-BY-NC-4.0 for the retained Fairseq MMS and ctc-forced-aligner normalization/span behavior, BSD-2-Clause for the TorchAudio forced-alignment port, Uroman's permissive license for generated romanization data, and MIT for Auditok-derived segmentation, CrispASR, GGML, Silero VAD, and faster-whisper assets. See [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and the license files beside those components.
+
+MMS model weights downloaded when word alignment is used are not distributed in the gem and remain separately licensed CC-BY-NC-4.0. Other runtime-downloaded models remain subject to the terms published by their owners.

data/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,67 @@
+# Third-party notices
+
+This source gem contains components under several licenses. The Apache License 2.0 in [`LICENSE.txt`](LICENSE.txt) applies to original cohere-transcribe Ruby code, native adapter code, and documentation. It does not replace the licenses identified below.
+
+## Fairseq MMS and ctc-forced-aligner
+
+Normalization, punctuation, repeated-token merging, and span behavior in `lib/cohere/transcribe/alignment/text.rb` and `lib/cohere/transcribe/alignment/ctc.rb` retain modified behavior from Fairseq MMS and ctc-forced-aligner.
+
+- Fairseq MMS revision: `728b947019fd186753197add48c39cbb24ea43e2`
+- Fairseq MMS source: <https://github.com/facebookresearch/fairseq/tree/728b947019fd186753197add48c39cbb24ea43e2/examples/mms>
+- ctc-forced-aligner revision: `11855d1de76af2b490dd2e8e2db2661805ae90a0`
+- ctc-forced-aligner source: <https://github.com/MahmoudAshraf97/ctc-forced-aligner/tree/11855d1de76af2b490dd2e8e2db2661805ae90a0>
+- License: Creative Commons Attribution-NonCommercial 4.0 International; see [`lib/cohere/transcribe/alignment/LICENSE.ctc-forced-aligner`](lib/cohere/transcribe/alignment/LICENSE.ctc-forced-aligner)
+
+The Fairseq source identifies Facebook, Inc. and its affiliates as the copyright holder. The ctc-forced-aligner source identifies Mahmoud Ashraf as its author. The Ruby implementation removes unsupported language and split modes, unused metadata, confidence aggregation, model-loading helpers, and native-extension integration while retaining the Arabic/English behavior used by this gem.
+
+## TorchAudio
+
+The float32 CTC dynamic program in `lib/cohere/transcribe/alignment/ctc.rb` is a Ruby port of the TorchAudio 2.11.0 `forced_align` CPU kernel's state transitions and tie-breaking behavior.
+
+- Source: <https://github.com/pytorch/audio/blob/v2.11.0/src/libtorchaudio/forced_align/cpu/compute.cpp>
+- Copyright: 2017 Facebook Inc. (Soumith Chintala)
+- License: BSD-2-Clause; see [`lib/cohere/transcribe/alignment/LICENSE.torchaudio`](lib/cohere/transcribe/alignment/LICENSE.torchaudio)
+
+## Uroman
+
+`lib/cohere/transcribe/alignment/uroman_data.rb` and the Uroman rules in `lib/cohere/transcribe/alignment/text.rb` are generated or ported from Uroman 1.3.1.1.
+
+- Source: <https://github.com/isi-nlp/uroman>
+- Copyright: 2015-2020 Ulf Hermjakob, USC Information Sciences Institute
+- License: the Uroman permissive license; see [`lib/cohere/transcribe/alignment/LICENSE.uroman`](lib/cohere/transcribe/alignment/LICENSE.uroman)
+
+This project uses the universal romanizer software "uroman" written by Ulf Hermjakob, USC Information Sciences Institute (2015-2020). See the packaged license for the requested publication acknowledgement and bibliography.
+
+## Auditok
+
+The energy tokenizer in `lib/cohere/transcribe/audio/segmentation.rb` retains the 50 ms PCM16 energy and state-transition behavior of Auditok 0.4.2 while implementing it directly in Ruby.
+
+- Source: <https://github.com/amsehili/auditok>
+- Copyright: 2015-2026 Mohamed El Amine SEHILI
+- License: MIT; see [`lib/cohere/transcribe/audio/LICENSE.auditok`](lib/cohere/transcribe/audio/LICENSE.auditok)
+
+## Silero VAD and faster-whisper
+
+`lib/cohere/transcribe/vad/silero_vad_v6.onnx`, the Ruby timestamp state machine, and the sequence runner use Silero VAD 6.2.1 behavior and the exact ONNX export distributed by faster-whisper.
+
+- Silero VAD revision: `7e30209a3e901f9842f81b225f3e93d8199902b1`
+- Silero VAD source: <https://github.com/snakers4/silero-vad/tree/v6.2.1>
+- faster-whisper revision: `ed9a06cd89a93e47838f564998a6c09b655d7f43`
+- faster-whisper source: <https://github.com/SYSTRAN/faster-whisper/tree/ed9a06cd89a93e47838f564998a6c09b655d7f43>
+- License: MIT; see [`lib/cohere/transcribe/vad/LICENSE.silero-vad`](lib/cohere/transcribe/vad/LICENSE.silero-vad), [`lib/cohere/transcribe/vad/LICENSE.faster-whisper`](lib/cohere/transcribe/vad/LICENSE.faster-whisper), and [`lib/cohere/transcribe/vad/ATTRIBUTION.md`](lib/cohere/transcribe/vad/ATTRIBUTION.md)
+
+## CrispASR and GGML
+
+The native runtime vendors a deliberately reduced CrispASR 0.8.9 snapshot and its nested GGML snapshot. The gem adds a Ruby-facing session adapter, audio adapter, bounded batching, cancellation, and packaging changes while retaining the upstream source licenses and author records.
+
+- CrispASR revision: `a68dd64092c4c44b41799a15976f5c6d27af13d7`
+- CrispASR source: <https://github.com/CrispStrobe/CrispASR/tree/a68dd64092c4c44b41799a15976f5c6d27af13d7>
+- GGML revision: `0714117daca2471b00e09554c7eaa74a06b0b2c5`
+- GGML source: <https://github.com/CrispStrobe/ggml/tree/0714117daca2471b00e09554c7eaa74a06b0b2c5>
+- License: MIT; see [`vendor/crispasr/LICENSE`](vendor/crispasr/LICENSE), [`vendor/crispasr/ggml/LICENSE`](vendor/crispasr/ggml/LICENSE), and [`vendor/crispasr/UPSTREAM.md`](vendor/crispasr/UPSTREAM.md)
+
+## Runtime downloads and separately installed software
+
+Model weights downloaded at runtime are not included in the gem and retain their own terms. The default Cohere ASR model is licensed under Apache License 2.0. The default MMS alignment model is licensed under Creative Commons Attribution-NonCommercial 4.0 International. Custom models and adapters remain subject to the terms published by their owners.
+
+Ruby gems installed as dependencies and system libraries loaded at runtime are not copied into this source gem and retain their respective licenses.

data/exe/cohere-transcribe

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+
+require "cohere/transcribe/cli"
+
+exit Cohere::Transcribe::CLI.cli(ARGV)

data/exe/cohere-transcribe-doctor

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+
+require "cohere/transcribe/doctor"
+
+exit Cohere::Transcribe::Doctor.main(ARGV)