stenograf 0.2.2__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {stenograf-0.2.2 → stenograf-0.3.0}/PKG-INFO +31 -6
- {stenograf-0.2.2 → stenograf-0.3.0}/PLAN.md +148 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/README.md +29 -5
- stenograf-0.3.0/eval/README.md +306 -0
- stenograf-0.3.0/eval/bias.py +858 -0
- stenograf-0.3.0/eval/bias_confidence.py +190 -0
- stenograf-0.3.0/eval/bias_data.py +230 -0
- stenograf-0.3.0/eval/bias_fluid.py +346 -0
- stenograf-0.3.0/eval/bias_score.py +452 -0
- stenograf-0.3.0/eval/bias_tts.py +314 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/pyproject.toml +21 -1
- stenograf-0.3.0/src/stenograf/asr/biasing.py +368 -0
- stenograf-0.3.0/src/stenograf/asr/parakeet.py +255 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/asr/parakeet_onnx.py +100 -2
- stenograf-0.3.0/src/stenograf/asr/tokens.py +146 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/linux.py +25 -5
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/macos.py +41 -8
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/streaming.py +38 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/windows.py +11 -7
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/run.py +4 -3
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/start.py +42 -6
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/transcribe.py +8 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/config.py +3 -3
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/glossary.py +76 -11
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/loaders.py +81 -9
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/settings.py +38 -4
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/flow.py +10 -1
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/setup.py +6 -6
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/fake_parec.py +6 -1
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/fake_stenocap.py +21 -3
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_asr_tokens.py +15 -0
- stenograf-0.3.0/tests/test_biasing.py +157 -0
- stenograf-0.3.0/tests/test_biasing_loops.py +207 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_capture_linux.py +25 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_capture_macos.py +32 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_capture_windows.py +14 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_cli.py +106 -17
- stenograf-0.3.0/tests/test_eval_bias.py +312 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_glossary.py +60 -5
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_settings.py +2 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_ui.py +6 -1
- {stenograf-0.2.2 → stenograf-0.3.0}/uv.lock +87 -33
- stenograf-0.2.2/eval/README.md +0 -142
- stenograf-0.2.2/src/stenograf/asr/parakeet.py +0 -128
- stenograf-0.2.2/src/stenograf/asr/tokens.py +0 -47
- {stenograf-0.2.2 → stenograf-0.3.0}/.claude/skills/verify/SKILL.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/.github/workflows/ci.yml +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/.github/workflows/release.yml +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/.gitignore +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/CLAUDE.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/LICENSE +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/PLAN-AEC.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/PLAN-CLEANUP.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/adjudicate.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/aec_rig.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/aec_score.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/backends.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/common.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/der.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/diarize.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/extract.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/live.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/manifest.example.json +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/parity.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/rttm.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/scan_languages.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/score.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/eval/transcribe.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/hatch_build.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/install.ps1 +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/install.sh +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/README.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/helper/Info.plist +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/helper/build.sh +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/helper/main.swift +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/spike/Info.plist +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/spike/build.sh +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/spike/main.swift +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/.gitignore +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/Cargo.lock +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/Cargo.toml +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/build.ps1 +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/build.sh +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/src/main.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/VENDOR.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/Cargo.toml +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/LICENSE +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/README.md +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/binarize.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/clustering/ahc.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/clustering/plda.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/clustering/vbx.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/clustering.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/coreml/array.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/coreml/path.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/coreml/runtime.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/coreml.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/batch.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/chunk.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/fbank.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/load/sessions.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/load.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/native/loaders.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/native.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/paths.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/run.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/session.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/tail.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding/tensor.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/embedding.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/native.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/parallel/batch.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/parallel/single.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/parallel.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/run.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation/tensor.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference/segmentation.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/inference.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/lib.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/linalg.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/metrics.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/models.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/builder.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding/collect.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding/error.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding/gpu.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding/orchestrate.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding/prep.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/chunk_embedding.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/clustering.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/concurrent.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/config.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/post_inference.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/queued.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/tests.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/types/data.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/types/error.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/types/extract.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/types/layout.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline/types.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/pipeline.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/powerset.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/reconstruct.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/segment.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/native/stenodiar/vendor/speakrs/src/utils.rs +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/__main__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/aec.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/asr/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/asr/base.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/asr/providers.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/asr/registry.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/audio.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/base.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/capture/file.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/doctor_cmd.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/format.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/notes.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/profiles.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/cli/settings_cmd.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/diarization/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/diarization/base.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/diarization/sherpa.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/diarization/speakrs.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/doctor.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/lid.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/live.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/models.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/backend.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/command.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/export.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/generate.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/mlx.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/model.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/ollama.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/notes/prompt.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/output.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/pipeline.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/profiles.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/recording.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/session.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/shortcut.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/transcript.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/__init__.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/_fps.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/app.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/doctor.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/home.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/meeting.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/notes.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/settings.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/transcribe.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/ui/widgets.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/vad.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/src/stenograf/view.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/conftest.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_aec.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_asr_parakeet.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_asr_parakeet_onnx.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_asr_providers.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_asr_registry.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_audio.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_build_hook.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_capture_file.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_capture_streaming.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_cli_notes.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_config.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_diarization_sherpa.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_diarization_sherpa_unit.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_diarization_speakrs.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_doctor.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_echo_dedup.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_eval_der.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_lid.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_live.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_live_orchestration.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_meeting.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_models.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_notes_backends.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_notes_export.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_notes_generate.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_notes_model.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_output.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_pipeline.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_profiles.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_recording.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_session.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_shortcut.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_transcript.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_vad.py +0 -0
- {stenograf-0.2.2 → stenograf-0.3.0}/tests/test_view.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: stenograf
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: Accuracy-first local meeting transcription for German and English. Audio stays in RAM; only text is written.
|
|
5
5
|
Project-URL: Repository, https://github.com/daniel-om-weber/stenograf
|
|
6
6
|
Project-URL: Issues, https://github.com/daniel-om-weber/stenograf/issues
|
|
@@ -26,6 +26,7 @@ Requires-Dist: onnx-asr[hub]>=0.11
|
|
|
26
26
|
Requires-Dist: onnxruntime-directml>=1.24; sys_platform == 'win32'
|
|
27
27
|
Requires-Dist: onnxruntime>=1.18.1; sys_platform != 'win32'
|
|
28
28
|
Requires-Dist: parakeet-mlx>=0.5.2; sys_platform == 'darwin' and platform_machine == 'arm64'
|
|
29
|
+
Requires-Dist: sentencepiece>=0.2
|
|
29
30
|
Requires-Dist: sherpa-onnx-core<1.13,>=1.12.40
|
|
30
31
|
Requires-Dist: sherpa-onnx<1.13,>=1.12.40
|
|
31
32
|
Requires-Dist: soundcard>=0.4.4; sys_platform == 'win32'
|
|
@@ -287,16 +288,40 @@ dir (not the model cache) and are never uploaded.
|
|
|
287
288
|
|
|
288
289
|
## Vocabulary
|
|
289
290
|
|
|
290
|
-
Domain terms and attendee names
|
|
291
|
-
|
|
291
|
+
Domain terms and attendee names steer the transcript twice: they boost the
|
|
292
|
+
decoder toward those spellings *while* it transcribes, and then correct the
|
|
293
|
+
near-misses it still got wrong in the finalized text.
|
|
292
294
|
|
|
293
295
|
```sh
|
|
294
296
|
steno transcribe rec.mov --attendee "Anja Müller" --glossary Kubernetes,gRPC
|
|
295
297
|
steno transcribe rec.mov --glossary-file terms.txt
|
|
296
298
|
```
|
|
297
299
|
|
|
298
|
-
|
|
299
|
-
|
|
300
|
+
Write a term the way it appears in a sentence. A capitalized term (`Kubernetes`,
|
|
301
|
+
`iOS`) is imposed verbatim; an all-lowercase one asserts the spelling but leaves
|
|
302
|
+
the model's capitalization alone. In the correction pass a term and its
|
|
303
|
+
transcription must share a word count — `gRPC` can fix `G R P C` spoken as one
|
|
304
|
+
word, but not a term split across word boundaries.
|
|
305
|
+
|
|
306
|
+
**An acronym you pronounce as a word needs both spellings.** Biasing rewards the
|
|
307
|
+
exact token path you write, so `NIRS` only ever rewards `N-I-R-S` — a spelling
|
|
308
|
+
the acoustics never take when you say "nirs". List `NIRS` *and* `Nirs`: the
|
|
309
|
+
decoder can reach the second, and the correction pass snaps it back to the first,
|
|
310
|
+
since the two normalize to the same word. Only case variants of the same letters
|
|
311
|
+
work — a respelling (`Ekmo` for `ECMO`) is reachable but never snaps back.
|
|
312
|
+
|
|
313
|
+
Both layers are on whenever there are terms. `[asr] boost` scales the decoder
|
|
314
|
+
biasing (default 1.0; `0` turns it off, and much above ~3 it starts rewriting
|
|
315
|
+
words that were never in your list), and `[vocab] glossary_threshold` is the
|
|
316
|
+
similarity a word must reach before the correction pass touches it (default
|
|
317
|
+
0.95 — deliberately strict, because a loose threshold corrupts words the model
|
|
318
|
+
already had right).
|
|
319
|
+
|
|
320
|
+
Reach for the spellings before the knobs. On an 85-minute German meeting, adding
|
|
321
|
+
the pronounceable twins fixed three more terms for four extra changed words in
|
|
322
|
+
the rest of the transcript, where raising `boost` to 1.5 bought two and disturbed
|
|
323
|
+
forty-six — at 2.0 the correct compound "Medizintechnikgruppe" decayed into
|
|
324
|
+
"Medizinischechnikgruppe", pulled apart by a listed term it half-matched.
|
|
300
325
|
|
|
301
326
|
## Settings
|
|
302
327
|
|
|
@@ -321,7 +346,7 @@ formats = ["md", "json", "txt"] # default --format list (srt/vtt for subtitles
|
|
|
321
346
|
[vocab] # standing vocabulary — MERGED with per-run
|
|
322
347
|
glossary_file = "~/steno/glossary.txt" # --glossary/--attendee flags
|
|
323
348
|
attendees = ["Anja Müller"]
|
|
324
|
-
glossary_threshold = 0.
|
|
349
|
+
glossary_threshold = 0.95
|
|
325
350
|
|
|
326
351
|
[output]
|
|
327
352
|
dir = "~/Documents/Meetings" # where meeting folders are created
|
|
@@ -1284,6 +1284,154 @@ rest. Two naming landmines for future screens: never define `_running` or
|
|
|
1284
1284
|
`_render` on a Screen — both shadow Textual internals (`MessagePump._running`,
|
|
1285
1285
|
`Widget._render`) and break the screen silently.
|
|
1286
1286
|
|
|
1287
|
+
### Contextual-biasing evaluation harness (built 2026-07-13)
|
|
1288
|
+
|
|
1289
|
+
Decode-time biasing (`stenograf/asr/biasing.py`, a from-scratch port of NVIDIA's
|
|
1290
|
+
TurboBias/GPU-PB) shipped with a tree verified against NeMo's golden vectors but
|
|
1291
|
+
an *effect* resting on one TTS clip and three meeting WAVs. That is enough to
|
|
1292
|
+
prove the mechanism fires and not enough to set `[asr] boost` or to defend the
|
|
1293
|
+
two places we deliberately diverge from NeMo. `eval/bias*.py` replaces the
|
|
1294
|
+
anecdotes with numbers, with **zero hand labeling** — every reference and word
|
|
1295
|
+
list comes from corpora that already ship them. Usage in `eval/README.md`.
|
|
1296
|
+
|
|
1297
|
+
**The harness.** `bias_score.py` is a pure scorer (B-WER / U-WER / entity
|
|
1298
|
+
recall+precision / false insertions / surface damage), a faithful port of is21's
|
|
1299
|
+
own alignment — which turns the **44 hypothesis+result file pairs they publish
|
|
1300
|
+
into a free correctness oracle**: `tests/test_eval_bias.py` reproduces every one
|
|
1301
|
+
of them to the digit (skips until `bias_data.py --fetch is21` has run).
|
|
1302
|
+
`bias_data.py` downloads the English lists and **derives the German benchmark**
|
|
1303
|
+
(none exists) from MLS's free train transcripts by the identical recipe — 3,394
|
|
1304
|
+
utts, a 299,699-word distractor pool, 16.4 % rare-token density (vs 11 % in
|
|
1305
|
+
English: German compounding fattens the tail, which makes it a *better* biasing
|
|
1306
|
+
benchmark). `bias.py` drives the real shipped backend through `create_backend`.
|
|
1307
|
+
|
|
1308
|
+
**What it found** (500-utt pinned subsample, `parakeet` MLX, N=100 unless noted):
|
|
1309
|
+
|
|
1310
|
+
- **English correctness gate — PASS.** Baseline B-WER 12.73 % vs U-WER 1.03 %
|
|
1311
|
+
(rare words 12× harder); biasing → **B-WER −34.9 % relative, U-WER exactly
|
|
1312
|
+
flat**. The is21 authors report ~30 % relative with U-WER flat, so the port
|
|
1313
|
+
reproduces the literature on a far stronger model. B-WER also degrades
|
|
1314
|
+
monotonically with list size (12.22 → 12.84 % for N=100 → 2000), which is the
|
|
1315
|
+
bug detector: it proves the distractors are really in the tree and competing.
|
|
1316
|
+
- **`boost = 1.0` CONFIRMED as the default.** German B-WER by boost: 0.5 →
|
|
1317
|
+
−16.5 %, **1.0 → −27.0 % (U-WER −3.2 %, i.e. it *improves*)**, 2.0 → −39.4 %
|
|
1318
|
+
but U-WER **+6.7 %** and false insertions 3 → 27, 3.0 → no further B-WER gain
|
|
1319
|
+
(−39.2 %) with U-WER **+38.4 %** and 105 false insertions. The knee is exactly
|
|
1320
|
+
between 1 and 2; 3 is the catastrophe NVIDIA warns about, measured.
|
|
1321
|
+
- **`unk_score = 1.0` (our divergence from NeMo's code) CONFIRMED.** NeMo's
|
|
1322
|
+
`unk_score=0` wins on B-WER (−33.7 % vs −27.0 %) but rewrites **33 words'
|
|
1323
|
+
surface forms vs our 6** — and the rewrites are exactly the predicted failure:
|
|
1324
|
+
spurious capitalization (`ihrer`→`Ihrer`, `war`→`War`, `fertig`→`Fertig`),
|
|
1325
|
+
because with `unk_score < context_score` the root's arcs stop being uniform and
|
|
1326
|
+
the tree yanks the decoder toward glossary-ish spellings at *every word start*.
|
|
1327
|
+
The `DEFAULT_UNK_SCORE` docstring predicted this mechanism; the sweep measured it.
|
|
1328
|
+
- **Compound-tail tokenization — NULL on this benchmark, and that is a fact about
|
|
1329
|
+
the corpus, not the feature.** Identical B-WER and recall with it on and off
|
|
1330
|
+
(12.22 %, 87.8 %). MLS German is 19th-century audiobook prose, where a glossary
|
|
1331
|
+
term almost never appears *inside* a compound — the case the feature exists for.
|
|
1332
|
+
It is confirmed elsewhere (the real-model measurement that motivated it, and the
|
|
1333
|
+
TTS reachability tier) and shown here to do **no harm**. Tier 2 is simply the
|
|
1334
|
+
wrong corpus to price it; a German *technical-speech* set would be the right one,
|
|
1335
|
+
and none exists.
|
|
1336
|
+
- **The post-correction layer was over-correcting, and `glossary_threshold` moved
|
|
1337
|
+
0.82 → 0.95** (`--post`, added 2026-07-13; costs no decoding — post-correction is
|
|
1338
|
+
a pure text transform over the other arms' cached hypotheses). We ship *two*
|
|
1339
|
+
glossary layers and had measured only one. At 0.82, post-correction posts a B-WER
|
|
1340
|
+
drop twice biasing's (−52.8 % German, −55.5 % English) **and damages the
|
|
1341
|
+
transcript**: U-WER +9.9 % German / **+86.3 % English**, 85 false insertions vs
|
|
1342
|
+
biasing's 3. The shipped stack (both layers) was worse than the `boost = 2.0`
|
|
1343
|
+
config we had already rejected for exactly this failure (+6.5 % vs +6.7 % U-WER;
|
|
1344
|
+
84 vs 27 insertions). Fuzzy matching answers to no acoustics, so it cannot
|
|
1345
|
+
distinguish a misrecognition from a correct word that merely looks like a term —
|
|
1346
|
+
and **B-WER alone cannot see the difference**. At 0.95 the layer is a free win:
|
|
1347
|
+
B-WER −30.3 % German / −36.3 % English (*better* than biasing alone: −27.0 % /
|
|
1348
|
+
−34.9 %), U-WER flat or improving, false insertions at biasing's own level (4/3),
|
|
1349
|
+
and zero added insertions on real meeting audio at a realistic 30-term glossary.
|
|
1350
|
+
The surface-damage column then caught a **second** bug, and was the only metric
|
|
1351
|
+
that could: it sat at ~1190 for the post arms at *every* threshold, and a quantity
|
|
1352
|
+
that ignores the knob gating it is not being produced by that knob. The word lists
|
|
1353
|
+
are 100 % lowercase and `glossary.py` imposed a term's spelling verbatim, so it
|
|
1354
|
+
de-capitalized German nouns wholesale. It now applies `biasing.surface_forms`' rule
|
|
1355
|
+
— capitals in a term are a deliberate spelling and win; an all-lowercase term
|
|
1356
|
+
asserts nothing about case and leaves the model's alone (`_recase`). The column
|
|
1357
|
+
fell **1190 → 2 with B-WER, U-WER and false insertions bit-identical**: every
|
|
1358
|
+
WER-shaped metric normalizes case away and would have shipped it.
|
|
1359
|
+
- **The fuzzy layer is not redundant with a better-tuned decoder — it is the
|
|
1360
|
+
*cheaper* knob** (`--boosts 0.5 1.0 1.25 1.5 1.75 2.0 --post 0.95`, the question
|
|
1361
|
+
"can we tune `boost` well enough to delete `glossary.py`?"). Answer: no. Matched on
|
|
1362
|
+
B-WER, lower-boost+post dominates higher-boost-alone on **both** damage metrics —
|
|
1363
|
+
at −37.6 % B-WER, `boost=1.5 + post` costs 8 false insertions and U-WER −2.3 %
|
|
1364
|
+
where `boost=1.75` alone costs **16** and −0.6 %; at −40 %, `boost=1.75 + post`
|
|
1365
|
+
(−40.7 %, U-WER −1.0 %, 16 FI) beats `boost=2.0` alone, which *fails* the U-WER
|
|
1366
|
+
gate (−39.4 %, **+6.7 %**, 27 FI) to get less. The layers fail differently, which
|
|
1367
|
+
is the whole point: boost makes the decoder credulous *everywhere*, so its damage
|
|
1368
|
+
is diffuse (U-WER, across the transcript); post@0.95 only fires on near-exact
|
|
1369
|
+
strings, so its damage is narrow. Buying a targeted fix with a diffuse knob is a
|
|
1370
|
+
bad trade. On **real meeting audio** the asymmetry is starker still: post@0.95 adds
|
|
1371
|
+
**0** false insertions at every boost level, while boost 1.0 → 1.75 takes them from
|
|
1372
|
+
2 → 11. The fuzzy layer is free; boost is not. Only at ≈ −30 % B-WER is it a tie
|
|
1373
|
+
(`boost=1.25` alone ≈ `boost=1.0 + post`), so the layer is discardable only if we
|
|
1374
|
+
are content to stop there. **Shipped defaults stay at `boost = 1.0` + threshold
|
|
1375
|
+
0.95** — the conservative end of the frontier, consistent with over-correction
|
|
1376
|
+
being worse than a missed correction. The rest of the frontier, if we ever want
|
|
1377
|
+
recall: `boost=1.5 + post` = B-WER −37.6 % for 7 false insertions on real audio.
|
|
1378
|
+
- **Head-to-head vs TypeWhisper's engine (`eval/bias_fluid.py`, English only).** Its
|
|
1379
|
+
Parakeet plugin calls FluidAudio, which runs *the same TDT 0.6b v3 we do*, so model,
|
|
1380
|
+
audio, lists and scorer are all held constant and the biasing **mechanism** is the
|
|
1381
|
+
only variable: our tree over token logits *inside* the greedy loop vs their second
|
|
1382
|
+
CTC model rescoring the *finished* transcript. **German is not runnable at all** —
|
|
1383
|
+
their spotter (`parakeet-ctc-110m`) has 1024 tokens and *zero* non-ASCII ones, so
|
|
1384
|
+
German terms tokenize to `<unk>` and are silently kept; TypeWhisper pairs it with the
|
|
1385
|
+
multilingual transcriber with no language check. **As TypeWhisper ships it, the
|
|
1386
|
+
vocabulary destroys the transcript**: B-WER −75.3 % bought with U-WER **+305.6 %** and
|
|
1387
|
+
**375 false insertions** (287/500 utterances altered; `glowing`→`unloving`,
|
|
1388
|
+
`pound`→`compound` — correct words snapped onto *distractors*). Not tuning: their own
|
|
1389
|
+
defaults give 374, their documented `cbw 3.0` gives 362. **But the mechanism is fine —
|
|
1390
|
+
the shipped config is the bug.** At `minSimilarity 0.85` the same engine reaches B-WER
|
|
1391
|
+
**−32.2 %, U-WER +2.4 %, 8 false insertions**, i.e. *parity* with our −34.9 % / +0.0 %
|
|
1392
|
+
/ 2. On accuracy this benchmark does not separate the two approaches; what separates
|
|
1393
|
+
them is one pass vs two models, streaming vs batch-only, and German. Two defaults do
|
|
1394
|
+
the damage and both are worst where real users live: `rescorerConfig(forVocabSize:)`
|
|
1395
|
+
gives **≤10 terms the *loosest* threshold (0.50)** — a real glossary is 10–30 terms —
|
|
1396
|
+
and the **spotter rescue** (on by default) wrecks small lists outright (762 false
|
|
1397
|
+
insertions on an oracle list of only-spoken words; `--vocab-disable-spotter-rescue`
|
|
1398
|
+
drops it to 104, and minSimilarity does not gate it). Caveat, stated in the README:
|
|
1399
|
+
is21's "rare words" are ordinary English words, not the entities FluidAudio is built
|
|
1400
|
+
for, so this benchmark is harsher on their design than their target domain is.
|
|
1401
|
+
- **Glossary size is a risk knob in its own right** (`--tier distractor`, biasing
|
|
1402
|
+
with words known to be absent, so any change at all is a false insertion): 10
|
|
1403
|
+
bogus terms → **0 edits**, 50 → 2, 100 → 5, 500 → 17. Damage scales with list
|
|
1404
|
+
length because more phrases mean more prefixes for the decoder to fall into
|
|
1405
|
+
mid-word. Real meeting glossaries are 10–30 terms, where it is clean — but this
|
|
1406
|
+
is the argument for keeping them to the terms that actually come up.
|
|
1407
|
+
- **TTS (Tier 5) is a reachability diagnostic and cannot be more.** All four error
|
|
1408
|
+
classes are reachable (casing, compound, glued, multi-token name) and the
|
|
1409
|
+
absent-term probes stay clean. But the tier had to be *redesigned mid-flight*:
|
|
1410
|
+
it originally demanded the term be recovered under both German and English
|
|
1411
|
+
pronunciation, and measurement killed that. A German acoustic model fed English
|
|
1412
|
+
phonemes can lose the word entirely ("Kafka-Consumer" → heard as "Kafketon
|
|
1413
|
+
Zoomer") or produce a *different real word* — the English rendering of "Ada" is
|
|
1414
|
+
heard as "Ede", so the audio genuinely says "Ede", and demanding recovery there
|
|
1415
|
+
would be demanding the decoder overwrite what was said. **Requiring the fix would
|
|
1416
|
+
be requiring a false insertion.** Two voices of the *same* engine also landed on
|
|
1417
|
+
opposite sides of that line for one sentence. Verdict is therefore per error
|
|
1418
|
+
*class*, and every non-recovery prints what the engine actually said.
|
|
1419
|
+
|
|
1420
|
+
**Landmines paid for here.** `load_dataset(repo, config, split="test")` downloads
|
|
1421
|
+
the whole config — for LibriSpeech "clean" that is train-clean-360 too, 30+ GB to
|
|
1422
|
+
read a 350 MB test split; name the parquet instead. `datasets` ≥ 4 decodes audio
|
|
1423
|
+
through **torchcodec**, i.e. it would drag torch into a torch-free project — so the
|
|
1424
|
+
harness reads parquet with pyarrow + soundfile and decodes only the sampled rows.
|
|
1425
|
+
Piper's VITS decoder is **stochastic**: regenerating fixtures produced different
|
|
1426
|
+
audio and flipped a verdict, so synthesis noise is zeroed and the fixture set is
|
|
1427
|
+
pinned. `de_DE-kerstin-low` ships a 130-phoneme map with no combining cedilla — it
|
|
1428
|
+
cannot say the German ich-laut and Piper drops it with a log line, silently
|
|
1429
|
+
mangling "Ich"/"Küche"; the generator now refuses any voice that cannot render a
|
|
1430
|
+
fixture.
|
|
1431
|
+
|
|
1432
|
+
**Open.** The sweep did not re-run the winning config over the *full* test set (the
|
|
1433
|
+
tables say so); `[asr] boost` keeps its 1.0 default, now with evidence behind it.
|
|
1434
|
+
|
|
1287
1435
|
---
|
|
1288
1436
|
|
|
1289
1437
|
## 6. Key sources
|
|
@@ -251,16 +251,40 @@ dir (not the model cache) and are never uploaded.
|
|
|
251
251
|
|
|
252
252
|
## Vocabulary
|
|
253
253
|
|
|
254
|
-
Domain terms and attendee names
|
|
255
|
-
|
|
254
|
+
Domain terms and attendee names steer the transcript twice: they boost the
|
|
255
|
+
decoder toward those spellings *while* it transcribes, and then correct the
|
|
256
|
+
near-misses it still got wrong in the finalized text.
|
|
256
257
|
|
|
257
258
|
```sh
|
|
258
259
|
steno transcribe rec.mov --attendee "Anja Müller" --glossary Kubernetes,gRPC
|
|
259
260
|
steno transcribe rec.mov --glossary-file terms.txt
|
|
260
261
|
```
|
|
261
262
|
|
|
262
|
-
|
|
263
|
-
|
|
263
|
+
Write a term the way it appears in a sentence. A capitalized term (`Kubernetes`,
|
|
264
|
+
`iOS`) is imposed verbatim; an all-lowercase one asserts the spelling but leaves
|
|
265
|
+
the model's capitalization alone. In the correction pass a term and its
|
|
266
|
+
transcription must share a word count — `gRPC` can fix `G R P C` spoken as one
|
|
267
|
+
word, but not a term split across word boundaries.
|
|
268
|
+
|
|
269
|
+
**An acronym you pronounce as a word needs both spellings.** Biasing rewards the
|
|
270
|
+
exact token path you write, so `NIRS` only ever rewards `N-I-R-S` — a spelling
|
|
271
|
+
the acoustics never take when you say "nirs". List `NIRS` *and* `Nirs`: the
|
|
272
|
+
decoder can reach the second, and the correction pass snaps it back to the first,
|
|
273
|
+
since the two normalize to the same word. Only case variants of the same letters
|
|
274
|
+
work — a respelling (`Ekmo` for `ECMO`) is reachable but never snaps back.
|
|
275
|
+
|
|
276
|
+
Both layers are on whenever there are terms. `[asr] boost` scales the decoder
|
|
277
|
+
biasing (default 1.0; `0` turns it off, and much above ~3 it starts rewriting
|
|
278
|
+
words that were never in your list), and `[vocab] glossary_threshold` is the
|
|
279
|
+
similarity a word must reach before the correction pass touches it (default
|
|
280
|
+
0.95 — deliberately strict, because a loose threshold corrupts words the model
|
|
281
|
+
already had right).
|
|
282
|
+
|
|
283
|
+
Reach for the spellings before the knobs. On an 85-minute German meeting, adding
|
|
284
|
+
the pronounceable twins fixed three more terms for four extra changed words in
|
|
285
|
+
the rest of the transcript, where raising `boost` to 1.5 bought two and disturbed
|
|
286
|
+
forty-six — at 2.0 the correct compound "Medizintechnikgruppe" decayed into
|
|
287
|
+
"Medizinischechnikgruppe", pulled apart by a listed term it half-matched.
|
|
264
288
|
|
|
265
289
|
## Settings
|
|
266
290
|
|
|
@@ -285,7 +309,7 @@ formats = ["md", "json", "txt"] # default --format list (srt/vtt for subtitles
|
|
|
285
309
|
[vocab] # standing vocabulary — MERGED with per-run
|
|
286
310
|
glossary_file = "~/steno/glossary.txt" # --glossary/--attendee flags
|
|
287
311
|
attendees = ["Anja Müller"]
|
|
288
|
-
glossary_threshold = 0.
|
|
312
|
+
glossary_threshold = 0.95
|
|
289
313
|
|
|
290
314
|
[output]
|
|
291
315
|
dir = "~/Documents/Meetings" # where meeting folders are created
|
|
@@ -0,0 +1,306 @@
|
|
|
1
|
+
# Phase 0 — model evaluation harness
|
|
2
|
+
|
|
3
|
+
**Result (2026-07-06): Parakeet-TDT-0.6B-v3 is the default for finalize + live.**
|
|
4
|
+
Blind adjudication of 161 disagreement sites: Parakeet tied Whisper 42:42,
|
|
5
|
+
Voxtral lost to both, Canary lost every pairing ~1:2. Details in PLAN.md §2;
|
|
6
|
+
raw judgments in out/adjudication-results-2026-07-06.json (gitignored).
|
|
7
|
+
|
|
8
|
+
Goal: decide the default finalize-pass ASR model on *real* meeting audio, not
|
|
9
|
+
read-speech leaderboards. Candidates:
|
|
10
|
+
|
|
11
|
+
| Model | Runtime | Role | Status |
|
|
12
|
+
|---|---|---|---|
|
|
13
|
+
| Parakeet-TDT-0.6B-v3 | parakeet-mlx | **default candidate** (finalize *and* live) | wired up |
|
|
14
|
+
| Voxtral Small 24B (4-bit) | mlx-voxtral | max-accuracy challenger | wired up (text only — no timestamps), ~14 GB download |
|
|
15
|
+
| Whisper large-v3 | mlx-whisper | mature fallback | wired up |
|
|
16
|
+
| Canary-1B-v2 | NeMo on MPS | accuracy-ceiling reference only | wired up, needs `uv sync --group eval-canary` (slow; never shippable) |
|
|
17
|
+
|
|
18
|
+
Canary was demoted from expected default (July 2026 research): no MLX/CoreML
|
|
19
|
+
runtime emits its word timestamps — PyPI `canary-mlx` is an abandoned template,
|
|
20
|
+
mlx-audio's Canary returns hardcoded 0.0 timestamps, and onnx-asr's timestamp
|
|
21
|
+
support covers TDT/CTC/RNNT only. NeMo-on-MPS is the sole real path and is too
|
|
22
|
+
slow/heavy to ship, so it serves purely as the accuracy ceiling in this eval.
|
|
23
|
+
|
|
24
|
+
## Workflow
|
|
25
|
+
|
|
26
|
+
Everything under `eval/audio/`, `eval/out/`, and `eval/refs/` contains private
|
|
27
|
+
meeting content and is gitignored.
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
# 0. See what language/content each recording holds (Whisper-tiny probes)
|
|
31
|
+
uv run --group eval eval/scan_languages.py
|
|
32
|
+
|
|
33
|
+
# 1. Define eval segments in manifest.json (id, source file, start/end seconds,
|
|
34
|
+
# language), then cut them to mono 16 kHz WAV:
|
|
35
|
+
uv run --group eval eval/extract.py
|
|
36
|
+
# To pick bounds by listening, first extract a full recording:
|
|
37
|
+
uv run --group eval eval/extract.py --full "my-meeting-recording.mov"
|
|
38
|
+
|
|
39
|
+
# 2. Judge the models. Primary path — blind disagreement adjudication:
|
|
40
|
+
# align all hypotheses, review only the spots where models disagree
|
|
41
|
+
# (audio snippet + shuffled unlabeled variants, ~4s each):
|
|
42
|
+
uv run --group eval eval/adjudicate.py # → eval/out/adjudication.html
|
|
43
|
+
# Open in a browser, judge (keys 1–9, 0 = unsure), download results, then:
|
|
44
|
+
uv run --group eval eval/adjudicate.py --score ~/Downloads/adjudication-results.json
|
|
45
|
+
#
|
|
46
|
+
# Secondary path — full hand-corrected references (slow, and beware anchor
|
|
47
|
+
# bias: a reference corrected from model X's draft flatters model X; the
|
|
48
|
+
# de-1 attempt measured Whisper at 1.3% WER for exactly that reason).
|
|
49
|
+
# Fix eval/refs/<id>.draft.txt while listening, rename to <id>.txt;
|
|
50
|
+
# only *.txt files are scored by score.py.
|
|
51
|
+
|
|
52
|
+
# 3. Run every candidate (one process per backend, for clean peak-memory numbers):
|
|
53
|
+
uv run --group eval eval/transcribe.py --backend parakeet
|
|
54
|
+
uv run --group eval eval/transcribe.py --backend whisper
|
|
55
|
+
uv run --group eval eval/transcribe.py --backend voxtral
|
|
56
|
+
|
|
57
|
+
# 4. Score:
|
|
58
|
+
uv run --group eval eval/score.py # → eval/out/report.md
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Target coverage: ~10 min hand-corrected reference per language (German +
|
|
62
|
+
English), including one in-room far-field sample, per PLAN.md Phase 0.
|
|
63
|
+
|
|
64
|
+
## Diarization scoring (Phase 3, Task 0d)
|
|
65
|
+
|
|
66
|
+
Measures the *diarizer*, not the ASR: **DER** (Diarization Error Rate) and
|
|
67
|
+
**word attribution** (of the finalized words, the fraction placed on the right
|
|
68
|
+
speaker). Nothing speaker-centric — re-ID threshold tuning, clustering/embedding
|
|
69
|
+
upgrades — is measurable without this, so it is the gating prerequisite.
|
|
70
|
+
|
|
71
|
+
References are **hand-labelled** speaker turns in NIST RTTM. The audio is a mono
|
|
72
|
+
downmix of every speaker (`extract.py`'s `-ac 1`), so one RTTM per segment labels
|
|
73
|
+
all distinct voices in it.
|
|
74
|
+
|
|
75
|
+
```sh
|
|
76
|
+
# 1. Seed a draft from the current diarizer to correct while listening (much
|
|
77
|
+
# faster than labelling boundaries from scratch). Pass the real speaker count
|
|
78
|
+
# you remember — unconstrained estimation over-clusters badly (de-1 → 13
|
|
79
|
+
# speakers), which is exactly the problem this measures.
|
|
80
|
+
uv run eval/diarize.py --bootstrap --segments de-1,de-2,en-1 --num-speakers 3
|
|
81
|
+
# → eval/refs/<id>.draft.rttm (each line: SPEAKER <id> 1 <onset> <dur> <NA> <NA> <spk> <NA> <NA>)
|
|
82
|
+
|
|
83
|
+
# 2. Fix boundaries + merge/rename speakers against the audio, then rename to
|
|
84
|
+
# the scored name (only <id>.rttm is scored; never score a .draft — it
|
|
85
|
+
# flatters the model that produced it):
|
|
86
|
+
mv eval/refs/de-1.draft.rttm eval/refs/de-1.rttm # after correcting
|
|
87
|
+
|
|
88
|
+
# 3. Produce hypotheses (raw diarizer turns + finalized word labels):
|
|
89
|
+
uv run eval/diarize.py --segments de-1,de-2,en-1 # → eval/out/diar/<id>.{rttm,words.json}
|
|
90
|
+
|
|
91
|
+
# 4. Score (DER + word attribution, optimal speaker mapping, 0.25 s collar):
|
|
92
|
+
uv run eval/der.py # → eval/out/diar-report.md
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
`der.py`/`rttm.py` are pure (numpy + scipy) and unit-tested in
|
|
96
|
+
`tests/test_eval_der.py` against hand-computed cases; `diarize.py` drives the
|
|
97
|
+
real stenograf backends. Everything under `eval/refs/` and `eval/out/` stays
|
|
98
|
+
gitignored (private content).
|
|
99
|
+
|
|
100
|
+
## Echo cancellation (PLAN-AEC.md)
|
|
101
|
+
|
|
102
|
+
Layer-0 signal scoring of the AEC path. A meeting run with `--aec-dump DIR`
|
|
103
|
+
writes the clock-aligned `mic.wav`/`lpb.wav`/`enh.wav` triple (near end as
|
|
104
|
+
captured, far-end reference, near end as the ASR receives it); score it with:
|
|
105
|
+
|
|
106
|
+
```sh
|
|
107
|
+
uv run --group eval eval/aec_score.py DIR --scenario st # far-end single-talk
|
|
108
|
+
uv run --group eval eval/aec_score.py DIR --scenario dt # double-talk
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Reports ERLE + residual level (energy over 10 ms frames during far-end
|
|
112
|
+
activity) and AECMOS (`speechmos`, the AEC-Challenge metric) — `echo_mos` for
|
|
113
|
+
"is the echo gone", `deg_mos` for "did we damage the local speaker". The
|
|
114
|
+
`--no-aec --aec-dump DIR` combination records the uncancelled baseline.
|
|
115
|
+
|
|
116
|
+
`aec_rig.py` runs a whole scenario on real hardware — plays a speech WAV out
|
|
117
|
+
the speakers while the real pipeline captures — and scores both layers (signal
|
|
118
|
+
metrics + leaked `Local-N` lines in the transcript):
|
|
119
|
+
|
|
120
|
+
```sh
|
|
121
|
+
uv run --group eval eval/aec_rig.py far-only --seconds 60 # pass = 0 leaked lines
|
|
122
|
+
uv run --group eval eval/aec_rig.py far-only --no-aec # uncancelled baseline
|
|
123
|
+
uv run --group eval eval/aec_rig.py double-talk # talk over it yourself
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Runs land in `eval/out/aec/<scenario>-<stamp>/` with the meeting output, the
|
|
127
|
+
dump triple, and `rig.json`. Keep volume, lid angle, and source clip fixed
|
|
128
|
+
across runs you compare. Measured 2026-07-10 (MacBook speakers, volume 63):
|
|
129
|
+
AEC on → 37.6 dB ERLE, −65 dBFS residual, AECMOS echo 4.73, **0 leaked lines
|
|
130
|
+
before any text backstop**; AEC off → −27 dBFS raw echo, AECMOS echo 1.49.
|
|
131
|
+
|
|
132
|
+
## Contextual-biasing evaluation (Phase 5)
|
|
133
|
+
|
|
134
|
+
Decode-time biasing (`stenograf.asr.biasing`) ships with a tree verified against
|
|
135
|
+
NeMo's golden vectors — but its *effect* rested on one TTS clip and three meeting
|
|
136
|
+
WAVs, which is enough to prove the mechanism fires and not enough to set `[asr]
|
|
137
|
+
boost` or to defend our two deliberate divergences from NeMo (`unk_score=1.0`, and
|
|
138
|
+
the German compound-tail tokenization). This harness replaces the anecdotes with
|
|
139
|
+
numbers, and needs **zero hand labeling**: every reference and every word list is
|
|
140
|
+
derived from corpora that already ship them.
|
|
141
|
+
|
|
142
|
+
```sh
|
|
143
|
+
# 0. Fetch/derive the benchmarks (is21 English lists; German built from MLS)
|
|
144
|
+
uv run --group eval eval/bias_data.py --fetch all --sizes 100 500 1000 2000
|
|
145
|
+
|
|
146
|
+
# 1. Correctness gate — the only language with published numbers to check against
|
|
147
|
+
uv run --group eval eval/bias.py --tier english --n 100
|
|
148
|
+
|
|
149
|
+
# 2. The benchmark that sets the shipped defaults (ablates boost/unk/compound-tail)
|
|
150
|
+
uv run --group eval eval/bias.py --tier german --sweep
|
|
151
|
+
|
|
152
|
+
# 3. False insertions with ground truth: bias with words known to be ABSENT,
|
|
153
|
+
# so any change at all is a false insertion. Runs on real meeting audio.
|
|
154
|
+
uv run --group eval eval/bias.py --tier distractor --wav eval/audio/*.wav
|
|
155
|
+
|
|
156
|
+
# 3b. BOTH glossary layers — decode-time biasing vs post-hoc fuzzy correction vs
|
|
157
|
+
# the stack. Works on any tier and re-decodes nothing (post-correction is a
|
|
158
|
+
# pure text transform over the arms' cached hypotheses).
|
|
159
|
+
uv run --group eval eval/bias.py --tier german --post 0.88 0.92 0.95
|
|
160
|
+
|
|
161
|
+
# 4. Reachability probes (synthetic; a diagnostic, never a quality metric)
|
|
162
|
+
uv run --group eval eval/bias_tts.py && uv run --group eval eval/bias.py --tier tts
|
|
163
|
+
|
|
164
|
+
# 5. Head-to-head vs TypeWhisper's engine (FluidAudio), English only — see below
|
|
165
|
+
uv run --group eval eval/bias_fluid.py --cli /path/to/.build/release/fluidaudiocli
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## The head-to-head: FluidAudio (TypeWhisper's Parakeet engine)
|
|
169
|
+
|
|
170
|
+
`bias_fluid.py` runs the *same acoustic model we do* (Parakeet TDT 0.6b v3) through
|
|
171
|
+
FluidAudio's context-biasing path — the engine behind TypeWhisper's Parakeet plugin —
|
|
172
|
+
on the same pinned 500 utterances, the same per-utterance 100-term lists, scored by
|
|
173
|
+
the same `bias_score`. Held constant: model, audio, lists, scorer. The only variable is
|
|
174
|
+
the **mechanism**: our boosting tree over token logits *inside* the greedy TDT loop, vs
|
|
175
|
+
their second CTC model rescoring the *finished* transcript. Each system is reported
|
|
176
|
+
against **its own** unbiased baseline (their encoder is CoreML int8, ours MLX fp32 — the
|
|
177
|
+
absolute WERs are not comparable; the deltas are).
|
|
178
|
+
|
|
179
|
+
**English only, and that is their limit, not our choice.** Their spotter
|
|
180
|
+
(`parakeet-ctc-110m`) has a 1024-token vocabulary with **zero** non-ASCII tokens, so
|
|
181
|
+
German terms tokenize into `<unk>` holes and are silently kept. TypeWhisper pairs it
|
|
182
|
+
with the multilingual transcriber with no language check. Our German tier has no
|
|
183
|
+
opponent — that is a capability they lack, not a number they lose.
|
|
184
|
+
|
|
185
|
+
**Result (2026-07-13).** As TypeWhisper ships it, the vocabulary posts a spectacular
|
|
186
|
+
B-WER −75.3 % and **destroys the transcript doing it**: U-WER **+305.6 %**, **375 false
|
|
187
|
+
insertions**, 287 of 500 utterances altered, with rewrites like `glowing`→`unloving`
|
|
188
|
+
and `pound`→`compound` — correct common words snapped onto *distractors*, terms the
|
|
189
|
+
benchmark included precisely because they are absent. Not a tuning accident: their own
|
|
190
|
+
defaults give 374, their documented `cbw 3.0` gives 362.
|
|
191
|
+
|
|
192
|
+
**But the mechanism is sound and the configuration is the bug.** Forced to
|
|
193
|
+
`minSimilarity 0.85`, the same engine reaches **B-WER −32.2 %, U-WER +2.4 %, 8 false
|
|
194
|
+
insertions** — parity with our −34.9 % / +0.0 % / 2. On accuracy alone this benchmark
|
|
195
|
+
does not separate in-loop boosting from post-decode rescoring. What separates them is
|
|
196
|
+
the rest of the bill: one pass vs two models, streaming vs batch-only, and German.
|
|
197
|
+
|
|
198
|
+
Two defaults do the damage, both worst where real users live:
|
|
199
|
+
|
|
200
|
+
| | |
|
|
201
|
+
|---|---|
|
|
202
|
+
| `rescorerConfig(forVocabSize:)` | minSimilarity **0.60** above 100 terms, **0.55** at 11–100, **0.50** at ≤10 — *smaller glossary, looser matching*. A real meeting glossary is 10–30 terms. |
|
|
203
|
+
| **spotter rescue** (on by default) | wrecks small lists: an oracle list of only genuinely-spoken words yields **762** false insertions and U-WER +1407 %; `--vocab-disable-spotter-rescue` alone drops it to 104. minSimilarity does not gate it — 0.85 leaves the collapse intact. |
|
|
204
|
+
|
|
205
|
+
**Read it fairly.** is21's "rare words" are ordinary English words (`frail`, `idly`,
|
|
206
|
+
`holiness`), not the entities and jargon FluidAudio is designed for; their published
|
|
207
|
+
99.3 % precision on earnings calls is plausibly true in that domain. The bounded claim:
|
|
208
|
+
*on the standard biasing benchmark, at its standard list size, the configuration
|
|
209
|
+
TypeWhisper ships is destructive — and the mechanism underneath it is not.*
|
|
210
|
+
|
|
211
|
+
## The confidence gate we did NOT build (`bias_confidence.py`)
|
|
212
|
+
|
|
213
|
+
The idea worth stealing from FluidAudio is the principle, not the code: **a post-hoc
|
|
214
|
+
correction must answer to the audio**, which is exactly what our fuzzy layer cannot do
|
|
215
|
+
and why over-correction forced the threshold to 0.95. The cheap version of that idea:
|
|
216
|
+
our TDT already emits per-token confidence from the *unbiased* distribution, so gate
|
|
217
|
+
corrections on it — refuse to rewrite a word the model was sure about. No second model,
|
|
218
|
+
no extra compute, works in every language.
|
|
219
|
+
|
|
220
|
+
`bias_confidence.py` tested the claim before anyone implemented it, and **killed it**:
|
|
221
|
+
|
|
222
|
+
| | n | median conf | p10 |
|
|
223
|
+
|---|---|---|---|
|
|
224
|
+
| false insertions | 53 | **0.999** | 0.938 |
|
|
225
|
+
| true fixes | 165 | **0.951** | 0.858 |
|
|
226
|
+
|
|
227
|
+
Directionally right, practically useless. The best gate (c = 0.95) blocks 45 of 53 false
|
|
228
|
+
insertions and destroys **84 of 165 true fixes**; looser points are worse. `der` → `deri`
|
|
229
|
+
(false insertion) scored 1.000 and `finde` → `find` (real fix) scored 0.997 — a greedy
|
|
230
|
+
RNN-T's entropy-normalized confidence saturates at ~1.0 for nearly everything, so it
|
|
231
|
+
cannot carry a gate. One decode pass, no refactor, question closed.
|
|
232
|
+
|
|
233
|
+
What survives is the *other* question. The gate asked "how sure was the model about the
|
|
234
|
+
word it wrote?" (always: certain). The discriminative one is "how much does the model
|
|
235
|
+
dislike the word we want to put there?" — scoring the **candidate** over the span,
|
|
236
|
+
against the tokens actually emitted. That is FluidAudio's idea done with the right model
|
|
237
|
+
(our 600M TDT, not a weak English-only 110M CTC), and it needs forced alignment through
|
|
238
|
+
the joint. Unproven, unbuilt; `bias_confidence.py` is what would price it.
|
|
239
|
+
|
|
240
|
+
**Metrics** (`bias_score.py`, pure — pinned by `tests/test_eval_bias.py`): B-WER
|
|
241
|
+
(WER over reference words in the biasing list — must fall), U-WER (every other word
|
|
242
|
+
— must **not** rise; over-boosting is visible here and nowhere else), entity
|
|
243
|
+
recall/precision/F, false insertions, and surface damage (`Ada` → `ADA`, which every
|
|
244
|
+
WER-shaped metric normalizes away before it can see it). Entity numbers are reported
|
|
245
|
+
strict *and* prefix-tolerant, because in German a term survives inside an inflected
|
|
246
|
+
or compounded word (`Europa` in `Europas`).
|
|
247
|
+
|
|
248
|
+
The scorer is a faithful port of is21's own alignment, so their **44 published
|
|
249
|
+
hypothesis/result file pairs are a free correctness oracle** — `tests/test_eval_bias.py`
|
|
250
|
+
reproduces every one of them to the digit (it skips until `--fetch is21` has run).
|
|
251
|
+
|
|
252
|
+
**What `--post` found (2026-07-13), and why the threshold moved to 0.95.** We ship
|
|
253
|
+
two glossary layers and had only ever measured one. Scored on the same terms, the
|
|
254
|
+
post-correction layer at its old 0.82 default looked spectacular on B-WER (−52.8 %
|
|
255
|
+
German, −55.5 % English — *twice* what decode-time biasing gets) and was in fact
|
|
256
|
+
damaging the transcript: U-WER **+9.9 % German / +86.3 % English**, 85–86 false
|
|
257
|
+
insertions against biasing's 3. The shipped stack (both layers, 0.82) came in at
|
|
258
|
+
U-WER +6.5 % and 84 false insertions — **worse than the `boost = 2.0` config we had
|
|
259
|
+
already rejected** (+6.7 %, 27 insertions). We had a bar; we had just never pointed
|
|
260
|
+
it at the second layer.
|
|
261
|
+
|
|
262
|
+
The cause is structural and worth remembering when adding any correction layer:
|
|
263
|
+
fuzzy text matching answers to no acoustics, so nothing prevents it from snapping a
|
|
264
|
+
correct common word onto a glossary term that merely looks like it, and **B-WER
|
|
265
|
+
alone cannot see the difference** — only U-WER and false insertions can. At 0.95 the
|
|
266
|
+
layer becomes a free win (B-WER −30.3 % German / −36.3 % English, i.e. *better* than
|
|
267
|
+
biasing alone, with U-WER flat and false insertions at biasing's own level), and on
|
|
268
|
+
real meeting audio with a realistic 30-term glossary it inserts nothing that biasing
|
|
269
|
+
did not already insert. One caveat on the tables: damage scales with list length, so
|
|
270
|
+
the N=100 lists overstate the risk for the 10–30-term glossaries real meetings use.
|
|
271
|
+
|
|
272
|
+
**The surface-damage column paid for itself here.** It was the only metric that could
|
|
273
|
+
see a second bug, and it sat at ~1190 for the post arms *at every threshold* — a
|
|
274
|
+
quantity that ignores the knob gating it is not being produced by that knob. Cause:
|
|
275
|
+
these word lists are 100 % lowercase, and `glossary.py` imposed a term's spelling
|
|
276
|
+
verbatim, so it dutifully de-capitalized German nouns. Fixed by teaching it what
|
|
277
|
+
`asr.biasing.surface_forms` already knew — capitals in a term are a deliberate
|
|
278
|
+
spelling and win; an all-lowercase term asserts nothing about case and must not
|
|
279
|
+
overwrite the model's. Re-running the tier drove the column **1190 → 2 with B-WER,
|
|
280
|
+
U-WER and false insertions bit-identical** (they normalize case, so they were blind
|
|
281
|
+
to the whole thing). Every WER-shaped metric would have shipped this.
|
|
282
|
+
|
|
283
|
+
Landmines, each verified and each worth a day: Parakeet emits punctuation and case
|
|
284
|
+
while LibriSpeech/MLS references do not (normalize both sides, and case-match the
|
|
285
|
+
boost phrases); every German noun is capitalized, so rare-by-frequency is the only
|
|
286
|
+
usable definition of a rare word; Common Voice is a dead stub on HuggingFace since
|
|
287
|
+
Mozilla moved it behind their Data Collective; AMI is uppercase, unpunctuated and
|
|
288
|
+
proper-noun-*sparse*, i.e. near-worthless for biasing despite being the closest
|
|
289
|
+
thing to meeting audio.
|
|
290
|
+
|
|
291
|
+
Sweeps run on a **pinned 500-utterance subsample** (the full grid is 5–10 h of
|
|
292
|
+
decoding); only the winning config is re-run over the full test set, and every table
|
|
293
|
+
states which it was. Hypotheses are cached per config, so an interrupted sweep only
|
|
294
|
+
costs the configs it had not reached.
|
|
295
|
+
|
|
296
|
+
## Side quests
|
|
297
|
+
|
|
298
|
+
- ~~Canary-1B-v2 runtime~~ — resolved, see above: no accelerated runtime with
|
|
299
|
+
word timestamps exists; NeMo-on-MPS reference backend wired up instead.
|
|
300
|
+
- **speakrs diarization sanity check** on the same audio (not wired up yet).
|
|
301
|
+
|
|
302
|
+
## Metrics
|
|
303
|
+
|
|
304
|
+
`transcribe.py` records per segment: wall time, speed (×RT), model load time,
|
|
305
|
+
peak RSS, MLX peak memory, detected language. `score.py` adds WER/CER (jiwer)
|
|
306
|
+
after normalization (lowercase, punctuation stripped, umlauts kept).
|