@steipete/summarize 0.4.0 → 0.5.0

package/CHANGELOG.md

@@ -1,11 +1,80 @@
 # Changelog
 
+## 0.5.0 - 2025-12-24
+
+### Features
+
+- **Model selection & presets**
+  - Automatic model selection (`--model auto`, now the default):
+    - Chooses models based on input kind (website/YouTube/file/image/video/text) and prompt size.
+    - Skips candidates without API keys; retries next model on request errors.
+    - Adds OpenRouter fallback attempts when `OPENROUTER_API_KEY` is present.
+    - Shows the chosen model in the progress UI.
+  - Named model presets via config (`~/.summarize/config.json` → `models`), selectable as `--model <preset>`.
+  - Built-in preset: `--model free` (OpenRouter `:free` candidates; override via `models.free`).
+- **OpenRouter free preset maintenance**
+  - `summarize refresh-free` regenerates `models.free` by scanning OpenRouter `:free` models and testing availability + latency.
+  - `summarize refresh-free --set-default` also sets `"model": "free"` in `~/.summarize/config.json` (so free becomes your default).
+- **CLI models**
+  - Add `--cli <provider>` flag (equivalent to `--model cli/<provider>`).
+  - `--cli` accepts case-insensitive providers and can be used without a provider to enable CLI auto selection.
+- **Content extraction**
+  - Website extraction detects video-only pages:
+    - YouTube embeds switch to transcript extraction automatically.
+    - Direct video URLs can be downloaded + summarized when `--video-mode auto|understand` and a Gemini key is available.
+- **Env**
+  - `.env` in the current directory is loaded automatically (so API keys work without exporting env vars).
+
+### Changes
+
+- **CLI config**
+  - Auto mode uses CLI models only when `cli.enabled` is set; order follows the list.
+  - `cli.enabled` is an allowlist for CLI usage.
+- **OpenRouter**
+  - Stop sending extra routing headers.
+  - `--model free`: when OpenRouter rejects routing with “No allowed providers”, print the exact provider names to allow and suggest running `summarize refresh-free`.
+  - `--max-output-tokens`: when explicitly set, it is also forwarded to OpenRouter calls.
+- **Refresh Free**
+  - Default extra runs reduced to 2 (total runs = 1 + runs) to reduce rate-limit pressure.
+  - Filter `:free` candidates by recency (default: last 180 days; configurable via `--max-age-days`).
+  - Print `ctx`/`out` in `k` units for readability.
+- **Defaults**
+  - Default summary length is now `xl`.
+
+### Fixes
+
+- **LLM / OpenRouter**
+  - LLM request retries (`--retries`) and clearer timeout errors.
+  - `summarize refresh-free`: detect OpenRouter free-model rate limits and back off + retry.
+- **Streaming**
+  - Normalize + de-dupe overlapping chunks to prevent repeated sections in live Markdown output.
+- **YouTube**
+  - Prefer manual captions over auto-generated when both exist. Thanks @dougvk.
+  - Always summarize YouTube transcripts in auto mode (instead of printing the transcript).
+- **Prompting & metrics**
+  - Don’t “pad” beyond input length when asking for longer summaries.
+  - `--metrics detailed`: fold metrics into finish line and make labels less cryptic.
+
+### Docs
+
+- Add documentation for presets and Refresh Free.
+- Add a “make free the default” quick start for `summarize refresh-free --set-default`.
+- Add a manual end-to-end checklist (`docs/manual-tests.md`).
+- Add a quick CLI smoke checklist (`docs/smoketest.md`).
+- Document CLI ordering and model selection behavior.
+
+### Tests
+
+- Add coverage for presets and Refresh Free regeneration.
+- Add live coverage for the `free` preset.
+- Add regression coverage for YouTube transcript handling and metrics formatting.
+
 ## 0.4.0 - 2025-12-21
 
 ### Changes
 
-- Add URL extraction mode via `--extract` (deprecated alias: `--extract-only`) with `--format md|text`.
-- Rename HTML→Markdown conversion flag to `--markdown-mode` (deprecated alias: `--markdown`).
+- Add URL extraction mode via `--extract` with `--format md|text`.
+- Rename HTML→Markdown conversion flag to `--markdown-mode`.
 - Add `--preprocess off|auto|always` and a `uvx markitdown` fallback for Markdown extraction and unsupported file attachments (when `--format md` is used).
 
 ## 0.3.0 - 2025-12-20
@@ -16,7 +85,6 @@
 - Run yt-dlp after web + Apify in `--youtube auto`, and error early for missing keys in `--youtube yt-dlp`.
 - Require Node 22+.
 - Respect `OPENAI_BASE_URL` when set, even with OpenRouter keys.
-- Apply OpenRouter provider ordering headers to HTML→Markdown conversion.
 - Add OpenRouter configuration tests. Thanks @dougvk for the initial OpenRouter support.
 - Build and ship a Bun bytecode arm64 binary for Homebrew.
 
@@ -33,7 +101,7 @@
 
 ### Changes
 
-- Add native OpenRouter support via `OPENROUTER_API_KEY` with optional provider ordering (`OPENROUTER_PROVIDERS`).
+- Add native OpenRouter support via `OPENROUTER_API_KEY`.
 - Remove map-reduce summarization; reject inputs that exceed the model's context window.
 - Preflight text prompts with the GPT tokenizer and the model’s max input tokens.
 - Reject text files over 10 MB before tokenization.
@@ -101,7 +169,7 @@ First public release.
   - `--max-output-tokens <count>` (optional hard cap)
   - `--timeout <duration>` (default `2m`)
   - `--stream auto|on|off`, `--render auto|md-live|md|plain`
-  - `--extract` (URLs only; no summary; deprecated alias: `--extract-only`)
+  - `--extract` (URLs only; no summary)
   - `--json` (structured output incl. input config, prompt, extracted content, LLM metadata, and metrics)
   - `--metrics off|on|detailed` (default `on`)
   - `--verbose`

package/README.md

@@ -16,7 +16,13 @@ Requires Node 22+.
 - npx (no install):
 
 ```bash
-npx -y @steipete/summarize "https://example.com" --model google/gemini-3-flash-preview
+npx -y @steipete/summarize "https://example.com"
+```
+
+- npm (global install):
+
+```bash
+npm i -g @steipete/summarize
 ```
 
 - Homebrew (custom tap):
@@ -30,7 +36,7 @@ Apple Silicon only (arm64).
 ## Quickstart
 
 ```bash
-summarize "https://example.com" --model google/gemini-3-flash-preview
+summarize "https://example.com"
 ```
 
 Input can be a URL or a local file path:
@@ -73,10 +79,11 @@ Use “gateway-style” ids: `<provider>/<model>`.
 
 Examples:
 
-- `openai/gpt-5.2`
-- `anthropic/claude-opus-4-5`
+- `openai/gpt-5-mini`
+- `anthropic/claude-sonnet-4-5`
 - `xai/grok-4-fast-non-reasoning`
 - `google/gemini-3-flash-preview`
+- `openrouter/openai/gpt-5-mini` (force OpenRouter)
 
 Note: some models/providers don’t support streaming or certain file media types. When that happens, the CLI prints a friendly error (or auto-disables streaming for that model when supported by the provider).
 
@@ -93,6 +100,8 @@ npx -y @steipete/summarize "https://example.com" --length 20k
 - Character targets: `1500`, `20k`, `20000`
 - Optional hard cap: `--max-output-tokens <count>` (e.g. `2000`, `2k`)
   - Provider/model APIs still enforce their own maximum output limits.
+  - If omitted, no max token parameter is sent (provider default).
+  - Prefer `--length` unless you need a hard cap (some providers count “reasoning” into the cap).
 - Minimums: `--length` numeric values must be ≥ 50 chars; `--max-output-tokens` must be ≥ 16.
 
 ## Limits
@@ -106,19 +115,53 @@ npx -y @steipete/summarize "https://example.com" --length 20k
 npx -y @steipete/summarize <input> [flags]
 ```
 
-- `--model <provider/model>`: which model to use (defaults to `google/gemini-3-flash-preview`)
+- `--model <provider/model>`: which model to use (defaults to `auto`)
+- `--model auto`: automatic model selection + fallback (default)
+- `--model <name>`: use a config-defined model (see “Configuration”)
 - `--timeout <duration>`: `30s`, `2m`, `5000ms` (default `2m`)
+- `--retries <count>`: LLM retry attempts on timeout (default `1`)
 - `--length short|medium|long|xl|xxl|<chars>`
-- `--max-output-tokens <count>`: hard cap for LLM output tokens (optional)
+- `--max-output-tokens <count>`: hard cap for LLM output tokens (optional; only sent when set)
+- `--cli [provider]`: use a CLI provider (case-insensitive; equivalent to `--model cli/<provider>`). If omitted, uses auto selection with CLI enabled.
 - `--stream auto|on|off`: stream LLM output (`auto` = TTY only; disabled in `--json` mode)
 - `--render auto|md-live|md|plain`: Markdown rendering (`auto` = best default for TTY)
 - `--format md|text`: website/file content format (default `text`)
-- `--preprocess off|auto|always`: preprocess files (only with `--format md`) for model compatibility (default `auto`)
+- `--preprocess off|auto|always`: controls `uvx markitdown` usage (default `auto`; `always` forces file preprocessing)
+  - Install `uvx`: `brew install uv` (or https://astral.sh/uv/)
 - `--extract`: print extracted content and exit (no summary) — only for URLs
   - Deprecated alias: `--extract-only`
 - `--json`: machine-readable output with diagnostics, prompt, `metrics`, and optional summary
 - `--verbose`: debug/diagnostics on stderr
-- `--metrics off|on|detailed`: metrics output (default `on`; `detailed` prints a breakdown to stderr)
+- `--metrics off|on|detailed`: metrics output (default `on`; `detailed` adds a compact 2nd-line breakdown on stderr)
+
+## Auto model ordering
+
+`--model auto` builds candidate attempts from built-in rules (or your `model.rules` overrides).
+CLI tools are **not** used in auto mode unless you explicitly enable them via `cli.enabled` in config.
+Why: CLI adds ~4s latency per attempt and higher variance.
+Shortcut: `--cli` (with no provider) uses auto selection with CLI enabled.
+
+When enabled, auto prepends CLI attempts in the order listed in `cli.enabled`
+(recommended: `["gemini"]`), then tries the native provider candidates
+(with OpenRouter fallbacks when configured).
+
+Enable CLI attempts:
+
+```json
+{
+  "cli": { "enabled": ["gemini"] }
+}
+```
+
+Disable CLI attempts:
+
+```json
+{
+  "cli": { "enabled": [] }
+}
+```
+
+Note: when `cli.enabled` is set, it’s also an allowlist for explicit `--cli` / `--model cli/...`.
 
 ## Website extraction (Firecrawl + Markdown)
 
@@ -127,6 +170,9 @@ Non-YouTube URLs go through a “fetch → extract” pipeline. When the direct
 - `--firecrawl off|auto|always` (default `auto`)
 - `--extract --format md|text` (default `text`)
 - `--markdown-mode off|auto|llm` (default `auto`; only affects `--format md` for non-YouTube URLs)
+  - `auto`: use an LLM converter when configured; may fall back to `uvx markitdown`
+  - `llm`: force LLM conversion (requires a configured model key)
+  - `off`: disable LLM conversion (still may return Firecrawl Markdown when configured)
 - Plain-text mode: use `--format text`.
 
 ## YouTube transcripts
@@ -153,16 +199,34 @@ Supported keys today:
 
 ```json
 {
-  "model": "openai/gpt-5.2"
+  "model": { "id": "openai/gpt-5-mini" }
 }
 ```
 
+Shorthand (equivalent):
+
+```json
+{
+  "model": "openai/gpt-5-mini"
+}
+```
+
+Also supported:
+
+- `model: { "mode": "auto" }` (automatic model selection + fallback; see `docs/model-auto.md`)
+- `model.rules` (customize candidates / ordering)
+- `models` (define presets selectable via `--model <preset>`)
+- `media.videoMode: "auto"|"transcript"|"understand"`
+
+Note: the config is parsed leniently (JSON5), but **comments are not allowed**.
+Unknown keys are ignored.
+
 Precedence:
 
 1) `--model`
 2) `SUMMARIZE_MODEL`
 3) `~/.summarize/config.json`
-4) default
+4) default (`auto`)
 
 ## Environment variables
 
@@ -176,23 +240,57 @@ Set the key matching your chosen `--model`:
 
 OpenRouter (OpenAI-compatible):
 
-- Set `OPENROUTER_API_KEY=...` to route `openai/...` models through OpenRouter
-- Use OpenRouter models via the `openai/...` prefix, e.g. `--model openai/openai/gpt-oss-20b`
-- Optional: `OPENROUTER_PROVIDERS=...` to specify provider fallback order (e.g. `groq,google-vertex`)
+- Set `OPENROUTER_API_KEY=...`
+- Prefer forcing OpenRouter per model id: `--model openrouter/<author>/<slug>` (e.g. `openrouter/meta-llama/llama-3.1-8b-instruct:free`)
+- Built-in preset: `--model free` (uses a default set of OpenRouter `:free` models).
 
-Example:
+### `summarize refresh-free`
+
+Quick start: make free the default (keep `auto` available)
 
 ```bash
-OPENROUTER_API_KEY=sk-or-... summarize "https://example.com" --model openai/openai/gpt-oss-20b
+# writes ~/.summarize/config.json (models.free) and sets model="free"
+summarize refresh-free --set-default
+
+# now this defaults to free models
+summarize "https://example.com"
+
+# whenever you want best quality instead
+summarize "https://example.com" --model auto
 ```
 
-With provider ordering (falls back through providers in order):
+Regenerates the `free` preset (writes `models.free` into `~/.summarize/config.json`) by:
+
+- Fetching OpenRouter `/models`, filtering `:free`
+- Skipping models that look very small (<27B by default) based on the model id/name (best-effort heuristic)
+- Testing which ones return non-empty text (concurrency 4, timeout 10s)
+- Picking a mix of “smart-ish” (bigger `context_length` / output cap) and fast models
+- Refining timings for the final selection and writing the sorted list back
+
+If `--model free` stops working (rate limits, allowed-provider restrictions, models removed), run:
+
+```bash
+summarize refresh-free
+```
+
+Flags:
+
+- `--runs 2` (default): extra timing runs per selected model (total runs = 1 + runs)
+- `--smart 3` (default): how many “smart-first” picks (rest filled by fastest)
+- `--min-params 27b` (default): ignore models with inferred size smaller than N billion parameters
+- `--max-age-days 180` (default): ignore models older than N days (set 0 to disable)
+- `--set-default`: also sets `"model": "free"` in `~/.summarize/config.json`
+
+Example:
 
 ```bash
-OPENROUTER_API_KEY=sk-or-... OPENROUTER_PROVIDERS="groq,google-vertex" summarize "https://example.com"
+OPENROUTER_API_KEY=sk-or-... summarize "https://example.com" --model openrouter/meta-llama/llama-3.1-8b-instruct:free
 ```
 
-Legacy: `OPENAI_BASE_URL=https://openrouter.ai/api/v1` with `OPENAI_API_KEY` also works.
+If your OpenRouter account enforces an allowed-provider list, make sure at least one provider
+is allowed for the selected model. (When routing fails, `summarize` prints the exact providers to allow.)
+
+Legacy: `OPENAI_BASE_URL=https://openrouter.ai/api/v1` (and either `OPENAI_API_KEY` or `OPENROUTER_API_KEY`) also works.
 
 Optional services: