@steipete/summarize 0.8.2 → 0.10.0

package/docs/language.md

@@ -6,7 +6,7 @@ read_when:
 
 # Output language
 
-By default, `summarize` writes the summary in the **same language as the source content** (`--language auto`).
+By default, `summarize` writes the summary in the **same language as the source content** (`--language auto`). If language detection is uncertain, it falls back to English.
 
 This affects the language of the generated summary text (not extraction/transcription).
 

package/docs/llm.md

@@ -57,7 +57,7 @@ installed, auto mode can use local CLI models when `cli.enabled` is set (see `do
   - Prompts are wrapped in `<instructions>`, `<context>`, `<content>` tags.
   - When `--length` is numeric, we add `Output is X characters.` When `--language` is explicitly set, we add `Output should be <language>.`
 - `--no-cache`
-  - Bypass cache reads and writes (extracted content, transcripts, summaries).
+  - Bypass summary cache reads and writes only (LLM output). Extract/transcript caches still apply.
 - `--cache-stats`
   - Print cache stats and exit.
 - `--clear-cache`
@@ -69,6 +69,8 @@ installed, auto mode can use local CLI models when `cli.enabled` is set (see `do
   - Minimum numeric value: 50 chars.
   - Default: `long`.
   - Output format is Markdown; use short paragraphs and only add bullets when they improve scanability.
+- `--force-summary`
+  - Always run the LLM even when extracted content is shorter than the requested length.
 - `--max-output-tokens <count>`
   - Hard cap for output tokens (optional).
   - If omitted, no max token parameter is sent (provider default).
@@ -78,6 +80,14 @@ installed, auto mode can use local CLI models when `cli.enabled` is set (see `do
   - LLM retry attempts on timeout (default: 1).
 - `--json` (includes prompt + summary in one JSON object)
 
+## Prompt rules
+
+- Video and podcast summaries omit sponsor/ads/promotional segments; do not include them in the summary.
+- Do not mention or acknowledge sponsors/ads, and do not say you skipped or ignored anything.
+- If a standout line is present, include 1-2 short exact excerpts formatted as Markdown italics with single asterisks. Do not use quotation marks of any kind (straight or curly). If a title or excerpt would normally use quotes, remove them and optionally italicize the text instead. Apostrophes in contractions are OK. Never include ad/sponsor/boilerplate excerpts and do not mention them. Avoid sponsor/ad/promo language, brand names like Squarespace, or CTA phrases like discount code.
+- Final check: remove sponsor/ad references or mentions of skipping/ignoring content. Remove any quotation marks. Ensure standout excerpts are italicized; otherwise omit them.
+- Hard rules: never mention sponsor/ads; never output quotation marks of any kind (straight or curly), even for titles.
+
 ## Z.AI
 
 Use `--model zai/<model>` (e.g. `zai/glm-4.7`). Defaults to Z.AI’s base URL and uses chat completions.
@@ -86,3 +96,9 @@ Use `--model zai/<model>` (e.g. `zai/glm-4.7`). Defaults to Z.AI’s base URL an
 
 - Text prompts are checked against the model’s max input tokens (LiteLLM catalog) using a GPT tokenizer.
 - Text files over 10 MB are rejected before tokenization.
+
+## PDF attachments
+
+- For PDF inputs, `--preprocess auto` will send the PDF directly to Anthropic/OpenAI/Gemini when a fixed model supports documents; otherwise we fall back to markitdown.
+- `--preprocess always` forces markitdown (no direct attachments).
+- Streaming is disabled for document attachments.

package/docs/manual-tests.md

@@ -45,6 +45,8 @@ Tip: use `--verbose` to see model attempts + the chosen model.
 
 - YouTube:
   - `summarize https://www.youtube.com/watch?v=dQw4w9WgXcQ --max-output-tokens 200`
+- YouTube summary w/ timestamps (expect `[mm:ss]` in output):
+  - `summarize --timestamps --youtube web --length short https://www.youtube.com/watch?v=I845O57ZSy4`
 - Local video understanding (requires Gemini video-capable model; otherwise expect an error or transcript-only behavior depending on input):
   - `summarize ./path/to/video.mp4 --max-output-tokens 200`
 

package/docs/media.md

@@ -0,0 +1,37 @@
+---
+summary: "Embedded media detection + transcript-first pipeline."
+read_when:
+  - "When changing media detection, embedded captions, or video-mode behavior."
+---
+
+# Media detection + transcript-first
+
+## Detection (HTML)
+
+- Embedded video/audio: `<video>` / `<audio>` tags, `og:video` / `og:audio`, iframe embeds (YouTube/Vimeo/Twitch/Wistia, Spotify/SoundCloud/Podcasts).
+- Captions: `<track kind="captions|subtitles" src=...>`.
+
+## Transcript resolution order
+
+1) Embedded captions (VTT/JSON) when available.
+2) yt-dlp download + Whisper transcription (prefers local whisper.cpp; OpenAI/FAL fallback).
+
+## CLI behavior
+
+- `--video-mode transcript` prefers transcript-first media handling even when a page has text.
+- Direct media URLs (mp4/webm/m4a/etc) skip HTML and transcribe.
+- Local audio/video files are routed through the same transcript-first pipeline.
+- YouTube still uses the YouTube transcript pipeline (captions → yt-dlp fallback).
+ - X/Twitter status URLs with detected video auto-switch to transcript-first (yt-dlp), even in auto mode.
+ - Local media files are capped at 2 GB; remote media URLs are best-effort via yt-dlp (no explicit size limit).
+
+## Chrome extension behavior
+
+- When media is detected on a page, the Summarize button gains a dropdown caret (Page/Video or Page/Audio).
+- Selecting Video/Audio forces URL mode + transcript-first extraction for that run only.
+- Selection is not stored.
+
+## Known limits
+
+- No auth/cookie handling for embedded media; login-gated assets will fail.
+- Captions are best-effort; if captions are missing or unreadable, we fall back to transcription.

package/docs/model-auto.md

@@ -35,7 +35,8 @@ Behavior:
 
 - Uses the order you provide in `model.rules[].candidates[]` (or `bands[].candidates[]`).
 - Filters out candidates that can’t fit the prompt (max input tokens, LiteLLM catalog).
-- For a native candidate, auto mode may add an OpenRouter fallback attempt right after it (when `OPENROUTER_API_KEY` is set and video understanding isn’t required).
+- For a native candidate, auto mode may add an OpenRouter fallback attempt right after it (when `OPENROUTER_API_KEY` is set, video understanding isn’t required, and a matching OpenRouter model id is found). If no unique OpenRouter id matches, the fallback is skipped.
+- To force native-only attempts, unset `OPENROUTER_API_KEY` (or pass an explicit native model id like `xai/...` and keep the key unset).
 
 Notes:
 

package/docs/nvidia-onnx-transcription.md

@@ -0,0 +1,55 @@
+---
+summary: "Local Parakeet/Canary ONNX transcription via external CLI."
+read_when:
+  - "When configuring or changing local ONNX transcription (parakeet/canary)."
+---
+
+# NVIDIA Parakeet/Canary ONNX transcription
+
+Summarize can now run local transcription through NVIDIA's Parakeet-TDT 0.6B-v3 or Canary 1B-v2 ONNX exports by shelling out to a user-provided CLI. Auto selection prefers ONNX when configured; you can still force Whisper or a specific ONNX model.
+
+## How to enable
+
+1) Install a CLI capable of running the ONNX models (e.g. `sherpa-onnx` or a custom wrapper). Homebrew may not have a formula; use upstream binaries or build from source if needed. The CLI must emit the transcribed text on stdout and accept a single WAV input path. **Summarize now downloads the Hugging Face model files automatically on first use** into the cache (see below), so your command template can reference the provided paths.
+2) Set one (or both) command templates:
+
+- Recommended (no shell): provide a JSON array (command + args):
+  - `SUMMARIZE_ONNX_PARAKEET_CMD='["sherpa-onnx", "...", "--tokens", "{vocab}", "--offline-ctc-model", "{model}", "--input-wav", "{input}"]'`
+  - `SUMMARIZE_ONNX_CANARY_CMD='["my-canary-wrapper", "{model_dir}", "{input}"]'`
+- Shell string (advanced): `SUMMARIZE_ONNX_PARAKEET_CMD="sherpa-onnx ... --tokens {vocab} --offline-ctc-model {model} --input-wav {input}"`
+
+Notes:
+
+- If you use the shell string form, **do not quote placeholders** (Summarize shell-escapes substituted paths so spaces work and injection risk is reduced).
+
+Placeholders:
+
+- `{input}` — audio path (added to the end if not present)
+- `{model}` — downloaded `model.onnx` path
+- `{vocab}` — downloaded `vocab.txt` path
+- `{model_dir}` — parent directory containing the downloaded files
+
+3) Pick the ONNX model via CLI or env:
+
+- Auto (default): leave `SUMMARIZE_TRANSCRIBER` unset or set `SUMMARIZE_TRANSCRIBER=auto`
+- CLI: `--transcriber parakeet` or `--transcriber canary`
+- Env: `SUMMARIZE_TRANSCRIBER=parakeet` (or `canary`)
+
+For the Chrome extension, you can pick a permanent default under **Settings → Model → Advanced Overrides → Transcriber**. The selection is sent with every request. Make sure the daemon environment still has your ONNX CLI commands configured (env vars above) so the override can take effect. Alternatively, export the env vars before running `summarize daemon install --token <TOKEN>` so the daemon inherits your ONNX command templates and default transcriber.
+
+### Cache + download details
+
+- Artifacts are stored under `${SUMMARIZE_ONNX_CACHE_DIR || $XDG_CACHE_HOME || ~/.cache}/summarize/onnx/<model>/`.
+- Set `SUMMARIZE_ONNX_MODEL_BASE_URL` to point at a mirror (defaults to the Hugging Face repo for the chosen model).
+- The first run downloads `model.onnx` and `vocab.txt`; subsequent runs reuse cached files.
+
+## Behavior
+
+- Input audio is transcoded to 16kHz mono WAV via `ffmpeg` when available; otherwise the original file is passed to the CLI.
+- Onnx errors (missing command, non-zero exit, empty output) fall back to the existing Whisper flow with a note recorded in the transcript metadata.
+- Progress UI shows "ONNX (Parakeet/Canary)" while the external transcriber runs.
+
+## Notes
+
+- The ONNX inference binary itself is **not** bundled; users must install or provide it separately.
+- This flow remains CPU-only and compatible with existing transcript providers.

package/docs/openai.md

@@ -23,3 +23,8 @@ For the full model/provider matrix, see `docs/llm.md`.
 - `--max-output-tokens <count>`
   - Hard cap for output tokens (optional).
 - `--json` (includes prompt + summary in one JSON object)
+
+## PDF inputs
+
+- When a PDF is provided and `--preprocess auto` is used, summarize sends the PDF as a file input via the OpenAI Responses API.
+- Document streaming is disabled for file inputs; non-streaming calls are used instead.

package/docs/releasing.md

@@ -0,0 +1,26 @@
+---
+summary: "Release checklist + Homebrew tap update."
+---
+
+# Releasing
+
+## Goals
+- Ship npm packages (core first, then CLI).
+- Tag + GitHub release.
+- Update Homebrew tap so `brew install steipete/tap/summarize` matches latest tag.
+
+## Checklist
+1. `scripts/release.sh all` (gates → build → verify → publish → smoke → tag → tap).
+2. Create GitHub release for the new tag (match version, attach notes/assets as needed).
+3. If you didn’t run `tap` in the script, update the Homebrew tap formula for `summarize`:
+   - Bump version to the new tag.
+   - Update tarball URL + SHA256 for the new release.
+4. Verify Homebrew install reflects the new version:
+   - `brew install steipete/tap/summarize`
+   - `summarize --version` matches tag.
+   - Run a feature added in the release (e.g. `summarize daemon install` for v0.8.2).
+5. If anything fails, fix and re-cut the release (no partials).
+
+## Common failure
+- NPM/GitHub release updated, tap not updated → users stuck on old version.
+  Fix: always do step 3–4 before announcing.