localcaption 0.3.0__tar.gz → 0.4.0__tar.gz

{localcaption-0.3.0 → localcaption-0.4.0}/CHANGELOG.md

@@ -7,6 +7,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-09-05
+
+### Added
+- Pluggable transcription backends. `whisper.cpp` stays the default;
+  `faster-whisper` is available via `pip install localcaption[faster]`.
+  Select with `--backend {whisper-cpp,faster-whisper}` or
+  `$LOCALCAPTION_BACKEND`. `whisper_dir` is only required for whisper.cpp.
+- **Batch mode.** `localcaption --batch urls.txt` transcribes each non-empty,
+  non-`#` line sequentially (URLs or local files). Output goes to
+  `<out>/<videoId>/` so items cannot clobber each other; if
+  `<videoId>/<videoId>.txt` already exists the item is skipped. A summary
+  table is printed at the end. Exit 0 if every item succeeded or was
+  skipped, 1 if any failed. Python helper: `localcaption.batch.transcribe_urls`.
+- **Local auto-summary via Ollama.** `localcaption <url-or-file> --summary`
+  POSTs the `.txt` transcript to a local Ollama instance
+  (`http://localhost:11434/api/generate`) and writes `<id>.summary.md`
+  next to it. `--summary-model` (default `llama3.1:8b`) and
+  `--summary-prompt PATH` override the model and the built-in prompt
+  (TL;DR, key points, notable quotes, action items). If Ollama is down,
+  times out, or returns a bad response, the pipeline still succeeds and
+  prints a warning. `HTTP_PROXY` is ignored for this call.
+- **YouTube chapters.** When yt-dlp reports chapter markers, write
+  `<id>.chapters.json` and a `<id>.chaptered.md` with headings like
+  `## 00:00 Intro`. The raw whisper `<id>.txt` is unchanged.
+- **`localcaption search <term>`.** Grep previously transcribed videos via a
+  JSONL index at `~/.local/share/localcaption/index.jsonl` (override with
+  `LOCALCAPTION_INDEX_PATH`). Matches are ranked by hit count and printed
+  with timestamps when whisper JSON/SRT is available.
+
+### Changed
+- Default whisper model is now `small.en` instead of `base.en`. Better
+  accuracy on accents and proper nouns; `tiny.en` remains the fast
+  fallback via `--model tiny.en`. Installer, `setup.sh`, and
+  `doctor --fix` follow the same default.
+
 ## [0.3.0] - 2026-08-16
 
 ### Added

{localcaption-0.3.0 → localcaption-0.4.0}/CONTRIBUTING.md

@@ -24,11 +24,14 @@ src/localcaption/
 ├── __main__.py        # python -m localcaption
 ├── _logging.py        # tiny stdout logger (no logging-module config)
 ├── audio.py           # ffmpeg → 16 kHz mono WAV (stage 2)
+├── batch.py           # public Python API: transcribe_urls(...)
 ├── cli.py             # argparse entry point (the `localcaption` script)
 ├── download.py        # yt-dlp Python API wrapper (stage 1)
 ├── errors.py          # exception hierarchy
 ├── pipeline.py        # public Python API: transcribe_url(...)
-└── whisper.py         # whisper.cpp invocation (stage 3)
+├── summary.py         # optional local Ollama summary
+├── whisper.py         # Backend protocol + transcribe() dispatcher
+└── backends/          # whisper.cpp (default) and faster-whisper
 scripts/
 └── setup.sh           # bootstraps whisper.cpp + venv + model
 tests/                 # pytest suite

{localcaption-0.3.0 → localcaption-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: localcaption
-Version: 0.3.0
+Version: 0.4.0
 Summary: Fully-local video → transcript pipeline using yt-dlp, ffmpeg, and whisper.cpp. Supports YouTube, Vimeo, Twitch, and 1000+ sites. No API keys.
 Project-URL: Homepage, https://github.com/jatinkrmalik/localcaption
 Project-URL: Repository, https://github.com/jatinkrmalik/localcaption
@@ -51,6 +51,8 @@ Provides-Extra: dev
 Requires-Dist: pytest-cov>=4; extra == 'dev'
 Requires-Dist: pytest>=7; extra == 'dev'
 Requires-Dist: ruff>=0.5; extra == 'dev'
+Provides-Extra: faster
+Requires-Dist: faster-whisper>=1.0; extra == 'faster'
 Description-Content-Type: text/markdown
 
 # localcaption
@@ -81,7 +83,7 @@ Description-Content-Type: text/markdown
 |---|---|
 | Download best audio | [`yt-dlp`](https://github.com/yt-dlp/yt-dlp) (YouTube, Vimeo, Twitch, 1000+ sites) |
 | Re-encode to 16 kHz mono WAV | [`ffmpeg`](https://ffmpeg.org/) |
-| Transcribe locally | [`whisper.cpp`](https://github.com/ggerganov/whisper.cpp) |
+| Transcribe locally | [`whisper.cpp`](https://github.com/ggerganov/whisper.cpp) (default) or [`faster-whisper`](https://github.com/SYSTRAN/faster-whisper) |
 
 Nothing is uploaded to a third-party service. No OpenAI / Google / DeepL keys
 required. Runs happily on a laptop.
@@ -118,8 +120,8 @@ localcaption doctor --fix          # ~2 min on an M-series Mac
 `doctor --fix` is idempotent and end-to-end: it installs missing system
 tools (`ffmpeg`/`cmake` via `brew`/`apt`), clones + builds whisper.cpp at
 the canonical XDG location, downloads the default model, and re-runs the
-diagnostics to confirm everything works. Pick a different model with
-`--model small.en`.
+diagnostics to confirm everything works. Pick a faster model with
+`--model tiny.en`.
 
 Prefer to do it yourself? Two equivalent options:
 
@@ -130,13 +132,13 @@ curl -fsSL https://raw.githubusercontent.com/jatinkrmalik/localcaption/main/scri
 # Option B: DIY, anywhere you like:
 git clone https://github.com/ggerganov/whisper.cpp /path/to/whisper.cpp
 cd /path/to/whisper.cpp && cmake -B build && cmake --build build -j --config Release
-bash models/download-ggml-model.sh base.en
+bash models/download-ggml-model.sh small.en
 export LOCALCAPTION_WHISPER_DIR=/path/to/whisper.cpp   # add to your shell rc
 ```
 
 > 💡 The `install.sh` bootstrap is just `pipx install localcaption` followed
 > by `localcaption doctor --fix`, same logic, single source of truth.
-> Override the default model with `WHISPER_MODEL=small.en bash install.sh`.
+> Override the default model with `WHISPER_MODEL=tiny.en bash install.sh`.
 
 After install, verify everything is wired up:
 
@@ -148,7 +150,7 @@ localcaption doctor --fix          # diagnostic + auto-repair anything missing
 ### Uninstall
 
 To completely remove `localcaption` and everything it installed (the
-binary, whisper.cpp build, and ggml models (about 200 MB total):
+binary, whisper.cpp build, and ggml models (about 500 MB total):
 
 ```bash
 # pipx + whisper.cpp + models, with confirmation prompts:
@@ -159,7 +161,7 @@ bash scripts/uninstall.sh
 ```
 
 Useful flags: `--dry-run` (preview), `--yes` (skip prompts),
-`--keep-models` (uninstall the binary but keep the 200 MB whisper.cpp +
+`--keep-models` (uninstall the binary but keep the ~500 MB whisper.cpp +
 models cache for next time).
 
 Sample output:
@@ -180,7 +182,7 @@ whisper.cpp:
   searching: /Users/you/.local/share/localcaption/whisper.cpp
   ✅ directory exists
   ✅ binary built  (.../build/bin/whisper-cli)
-  ✅ models present  (ggml-base.en.bin)
+  ✅ models present  (ggml-small.en.bin)
 
 All checks passed. You're good to go: localcaption <url>
 ```
@@ -191,7 +193,7 @@ download the default model, then re-verify:
 
 ```bash
 localcaption doctor --fix                      # repair everything
-localcaption doctor --fix --model small.en     # …with a specific model
+localcaption doctor --fix --model tiny.en      # …with a faster/smaller model
 ```
 
 ### Dev install (contributors)
@@ -223,16 +225,28 @@ localcaption "https://vimeo.com/148751763"
 # Local video/audio files
 localcaption /path/to/video.mp4
 localcaption ./recording.wav
+
+# Batch: one URL or local path per line (# comments and blank lines ignored)
+localcaption --batch urls.txt -o transcripts/ -m small.en
+
+# Transcript + local summary (requires a running Ollama)
+localcaption "https://www.youtube.com/watch?v=dQw4w9WgXcQ" --summary
+localcaption ./talk.mp4 --summary --summary-model llama3.1:8b
 ```
 
 | flag | default | what it does |
 |---|---|---|
-| `-m`, `--model` | `base.en` | whisper model name (`tiny.en`, `base.en`, `small.en`, `medium.en`, `large-v3`, …) |
+| `-m`, `--model` | `small.en` | whisper model name (`tiny.en`, `base.en`, `small.en`, `medium.en`, `large-v3`, …) |
 | `-o`, `--out` | `./transcripts` | output directory |
 | `-l`, `--language` | `auto` | ISO language code, or `auto` to let whisper detect it |
-| `--whisper-dir` | auto-detect¹ | path to a built whisper.cpp checkout |
+| `--backend` | `whisper-cpp` | transcription backend: `whisper-cpp` or `faster-whisper`. `$LOCALCAPTION_BACKEND` if the flag is omitted |
+| `--whisper-dir` | auto-detect¹ | path to a built whisper.cpp checkout (whisper-cpp backend) |
 | `--keep-audio` | off | keep the downloaded audio + intermediate WAV in `<out>/.work/` |
 | `--no-print` | off | don't echo the transcript to stdout |
+| `--batch FILE` | off | transcribe each non-empty, non-`#` line in FILE sequentially |
+| `--summary` | off | after transcription, write `<id>.summary.md` via local Ollama |
+| `--summary-model` | `llama3.1:8b` | Ollama model used by `--summary` |
+| `--summary-prompt` | built-in | path to a prompt template (`{transcript}` is replaced if present) |
 
 ¹ `--whisper-dir` resolution order:
    1. The explicit flag value, if given.
@@ -240,10 +254,52 @@ localcaption ./recording.wav
    3. `./whisper.cpp` (dev checkout).
    4. `~/.local/share/localcaption/whisper.cpp` (where `install.sh` puts it).
 
-Outputs `<videoId>.txt`, `.srt`, `.vtt`, and `.json` in the chosen directory. For local files, the output filename is derived from the input file's name.
+Outputs `<videoId>.txt`, `.srt`, `.vtt`, and `.json` in the chosen directory. For local files, the output filename is derived from the input file's name. With `--summary`, also writes `<videoId>.summary.md`. When the source has chapter markers (typical on YouTube), also writes `<videoId>.chapters.json` and `<videoId>.chaptered.md`. The raw whisper `.txt` is left unchanged.
+
+`--batch FILE` writes each item into `<out>/<videoId>/` and skips any video
+whose `.txt` is already there, so you can re-run a list after a failure.
+Local paths are relative to the list file (and `~` is expanded). The process
+is sequential (whisper.cpp already saturates the machine). Exit 0 if
+everything succeeded or was skipped, 1 otherwise.
 
 You can also invoke it as a module: `python -m localcaption <url-or-file>`.
 
+### faster-whisper (optional)
+
+`whisper.cpp` is the default backend and needs no extra Python packages.
+To use [faster-whisper](https://github.com/SYSTRAN/faster-whisper) instead
+(CTranslate2, typically faster on CPU/CUDA, including Windows):
+
+```bash
+pip install 'localcaption[faster]'
+# pipx:
+pipx inject localcaption faster-whisper
+
+localcaption --backend faster-whisper "https://www.youtube.com/watch?v=..."
+# or:
+export LOCALCAPTION_BACKEND=faster-whisper
+```
+
+faster-whisper downloads its own CTranslate2 weights on first use; it does
+not read ggml files from `--whisper-dir`. `--model` names (`base.en`,
+`small.en`, `large-v3`, ...) match the usual Whisper sizes.
+
+### Summaries (optional)
+
+If [Ollama](https://ollama.com) is running locally, `--summary` sends the
+`.txt` transcript to `http://localhost:11434/api/generate` and writes
+`<id>.summary.md` next to it. The built-in prompt asks for a TL;DR, key
+points, notable quotes, and action items.
+
+```bash
+localcaption <url-or-file> --summary
+localcaption <url-or-file> --summary --summary-model mistral
+localcaption <url-or-file> --summary --summary-prompt ./my_prompt.txt
+```
+
+If Ollama isn't reachable, localcaption logs a warning and still exits 0.
+The transcript files are unchanged.
+
 ### Subcommands
 
 | Subcommand | What it does |
@@ -255,12 +311,31 @@ You can also invoke it as a module: `python -m localcaption <url-or-file>`.
 | `localcaption model info <name>` | Show metadata about a single model. |
 | `localcaption model download <name>` | Download a model with progress bar + atomic writes. |
 | `localcaption model rm <name>` | Remove an installed model to free disk space. |
+| `localcaption search <term>` | Grep previously transcribed videos. Ranked matches with timestamps. |
+
+### Search past transcripts
+
+Each successful transcription upserts one JSON line in
+`~/.local/share/localcaption/index.jsonl` (`id`, `url`, `title`, `duration`,
+`chapters`, `transcript`). Re-running the same id replaces that row. Override
+the path with `LOCALCAPTION_INDEX_PATH`.
+
+```bash
+localcaption search "install"
+# vid123  02:30  First, let's install pip
+#         Lecture on Python tooling
+```
+
+Search is a case-insensitive substring. Hits are ranked by how often the term
+appears (title matches get a small boost). Timestamps come from the sibling
+whisper `.json` or `.srt` when those files are still next to the `.txt`.
 
 ### Managing models
 
-`localcaption` ships with a default `base.en` model (~142 MB). For better
-quality or non-English audio, switch models with `--model <name>`. If the
-model isn't already installed, you'll be prompted to download it:
+`localcaption` defaults to `small.en` (~466 MB), downloaded by
+`doctor --fix` or on first use. For a faster run use `--model tiny.en`;
+for non-English audio, pick a multilingual model. If the model isn't
+already installed, you'll be prompted to download it:
 
 ```bash
 $ localcaption --model small.en "https://www.youtube.com/watch?v=..."
@@ -289,9 +364,9 @@ localcaption --model small.en --auto-download "https://www.youtube.com/..."
 
 | Model | Size | Best for |
 |---|---|---|
-| `tiny.en` | 75 MB | Quick drafts, English only, low-resource environments |
-| `base.en` | 142 MB | Current install default, fast & decent |
-| `small.en` | 466 MB | **Recommended for English**, great accuracy/speed balance |
+| `tiny.en` | 75 MB | Fast fallback, English only, low-resource environments |
+| `base.en` | 142 MB | Faster than `small.en`, lower accuracy |
+| `small.en` | 466 MB | **Install default**, English, accuracy/speed balance |
 | `medium.en` | 1.5 GB | High accuracy English, ~3× slower than `small.en` |
 | `large-v3` | 3.0 GB | Best accuracy, multilingual, slow |
 | `large-v3-turbo` | 1.6 GB | Near-large quality at ~half the size, great compromise |
@@ -308,17 +383,37 @@ result = transcribe_url(
     "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
     out_dir=Path("transcripts"),
     whisper_dir=Path("whisper.cpp"),
-    model="base.en",
+    model="small.en",
+    summary=True,  # optional; writes .summary.md via local Ollama
 )
 print(result.transcripts.txt.read_text())
+
+# faster-whisper (pip install 'localcaption[faster]') does not need whisper_dir:
+# transcribe_url(url, out_dir=Path("transcripts"), backend="faster-whisper")
+```
+
+Batch from Python:
+
+```python
+from pathlib import Path
+from localcaption.batch import read_url_list, transcribe_urls
+
+result = transcribe_urls(
+    read_url_list(Path("urls.txt")),
+    out_dir=Path("transcripts"),
+    whisper_dir=Path("whisper.cpp"),
+    model="small.en",
+)
+print(result.summary())
 ```
 
 ## Architecture
 
 `localcaption` is intentionally tiny: an orchestrator (`pipeline.py`) drives
 three single-responsibility stages, each wrapping one external tool. The
-modules are split this way so that a contributor can swap, say, `whisper.cpp`
-for `faster-whisper` without touching `download.py` or `audio.py`.
+transcribe stage is a small `Backend` protocol; `whisper.cpp` is the default
+implementation and `faster-whisper` is an optional extra. Swapping backends
+does not touch `download.py` or `audio.py`.
 
 ### Module map
 
@@ -327,8 +422,9 @@ for `faster-whisper` without touching `download.py` or `audio.py`.
 | Layer | Files | Responsibility |
 |---|---|---|
 | Entry points | `cli.py`, `__main__.py` | argparse, exit codes, stdout formatting |
-| Orchestration | `pipeline.py` | public Python API: `transcribe_url(...)` |
-| Pipeline stages | `download.py`, `audio.py`, `whisper.py` | one external tool each |
+| Orchestration | `pipeline.py`, `batch.py` | public Python API: `transcribe_url(...)`, `transcribe_urls(...)` |
+| Pipeline stages | `download.py`, `audio.py`, `whisper.py`, `backends/`, `summary.py` | download, re-encode, transcribe (pluggable), optional Ollama summary |
+| Chapters & search | `chapters.py`, `index.py` | YouTube chapter sidecars + JSONL search index |
 | Support | `errors.py`, `_logging.py` | exception hierarchy, tiny logger |
 
 ### Runtime sequence
@@ -349,9 +445,11 @@ the subprocess hops to yt-dlp, ffmpeg, and whisper.cpp. The intermediate
 ## Benchmarks
 
 Wall-clock times for the **complete** pipeline (yt-dlp download → ffmpeg
-re-encode → whisper.cpp transcription), measured with the default `base.en`
-model. Numbers will vary with your network speed and CPU/GPU; treat them as
-order-of-magnitude reference, not a competitive benchmark.
+re-encode → whisper.cpp transcription), measured with `base.en` (the
+previous default). These have **not** been re-run on `small.en`; expect
+transcription to take longer. Numbers will vary with your network speed
+and CPU/GPU; treat them as order-of-magnitude reference, not a
+competitive benchmark.
 
 | Video | Length | Wall-clock | Speed vs. realtime | Hardware |
 |---|---|---|---|---|
@@ -364,15 +462,16 @@ order-of-magnitude reference, not a competitive benchmark.
 
 ```bash
 # Apple Silicon, macOS, whisper.cpp built with Metal,
-# model: ggml-base.en, language: auto, no other heavy processes.
+# model: ggml-base.en (matches the table above; not the current default),
+# language: auto, no other heavy processes.
 
-time localcaption --no-print -o /tmp/lc-bench-1 \
+time localcaption --model base.en --no-print -o /tmp/lc-bench-1 \
   "https://www.youtube.com/watch?v=PSRJfaAYkW4"
 
-time localcaption --no-print -o /tmp/lc-bench-2 \
+time localcaption --model base.en --no-print -o /tmp/lc-bench-2 \
   "https://www.youtube.com/watch?v=aircAruvnKk"
 
-time localcaption --no-print -o /tmp/lc-bench-3 \
+time localcaption --model base.en --no-print -o /tmp/lc-bench-3 \
   "https://www.youtube.com/watch?v=BYizgB2FcAQ"
 ```
 
@@ -383,8 +482,8 @@ hardware in the **Hardware** column.
 
 ## Notes
 
-- Bigger models = better quality but slower. `base.en` is a good default;
-  try `small.en` if you have the patience and `tiny.en` for instant results.
+- Bigger models = better quality but slower. `small.en` is the default;
+  use `--model tiny.en` when you want speed over accuracy.
 - Apple Silicon: whisper.cpp's CMake build uses Metal automatically, you'll
   see `ggml_metal_init` in the logs.
 - The pipeline accepts any URL `yt-dlp` supports (Vimeo, Twitch VODs, Twitter/X,
@@ -411,7 +510,7 @@ criteria, and discussion):
 | [#4](https://github.com/jatinkrmalik/localcaption/issues/4) | Speaker diarization with pyannote.audio (`--diarize`) | `stretch`, `help wanted` |
 | [#5](https://github.com/jatinkrmalik/localcaption/issues/5) | YouTube chapters & grep-able search index | `enhancement` |
 | [#6](https://github.com/jatinkrmalik/localcaption/issues/6) | Pluggable transcription backends (faster-whisper / MLX) | `help wanted` |
-| ~~[#1](https://github.com/jatinkrmalik/localcaption/issues/1)~~ | ~~Switch default model from `base.en` to `small.en`~~ | _superseded by #7_ |
+| [#1](https://github.com/jatinkrmalik/localcaption/issues/1) | Switch default model from `base.en` to `small.en` | _unreleased_ ✅ |
 
 **Have an idea?** Open a
 [feature request](https://github.com/jatinkrmalik/localcaption/issues/new/choose),