@steipete/summarize 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/CHANGELOG.md +24 -1
  2. package/README.md +34 -6
  3. package/dist/cli.cjs +4941 -790
  4. package/dist/cli.cjs.map +4 -4
  5. package/dist/esm/content/link-preview/client.js +6 -0
  6. package/dist/esm/content/link-preview/client.js.map +1 -1
  7. package/dist/esm/content/link-preview/transcript/index.js +6 -0
  8. package/dist/esm/content/link-preview/transcript/index.js.map +1 -1
  9. package/dist/esm/content/link-preview/transcript/providers/youtube/yt-dlp.js +213 -0
  10. package/dist/esm/content/link-preview/transcript/providers/youtube/yt-dlp.js.map +1 -0
  11. package/dist/esm/content/link-preview/transcript/providers/youtube.js +40 -2
  12. package/dist/esm/content/link-preview/transcript/providers/youtube.js.map +1 -1
  13. package/dist/esm/flags.js +2 -0
  14. package/dist/esm/flags.js.map +1 -1
  15. package/dist/esm/llm/generate-text.js +51 -14
  16. package/dist/esm/llm/generate-text.js.map +1 -1
  17. package/dist/esm/llm/html-to-markdown.js +3 -2
  18. package/dist/esm/llm/html-to-markdown.js.map +1 -1
  19. package/dist/esm/run.js +40 -9
  20. package/dist/esm/run.js.map +1 -1
  21. package/dist/esm/version.js +1 -1
  22. package/dist/types/content/link-preview/client.d.ts +3 -0
  23. package/dist/types/content/link-preview/content/types.d.ts +1 -1
  24. package/dist/types/content/link-preview/deps.d.ts +3 -0
  25. package/dist/types/content/link-preview/transcript/providers/youtube/yt-dlp.d.ts +15 -0
  26. package/dist/types/content/link-preview/transcript/types.d.ts +4 -0
  27. package/dist/types/flags.d.ts +1 -1
  28. package/dist/types/llm/generate-text.d.ts +8 -2
  29. package/dist/types/llm/html-to-markdown.d.ts +4 -1
  30. package/dist/types/version.d.ts +1 -1
  31. package/docs/youtube.md +5 -2
  32. package/package.json +5 -1
package/CHANGELOG.md CHANGED
@@ -1,10 +1,33 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.3.0 - 2025-12-20
4
+
5
+ ### Changes
6
+
7
+ - Add yt-dlp audio transcription fallback for YouTube; prefer OpenAI Whisper with FAL fallback. Thanks @dougvk.
8
+ - Add `--no-playlist` to yt-dlp downloads to avoid transcript mismatches.
9
+ - Run yt-dlp after web + Apify in `--youtube auto`, and error early for missing keys in `--youtube yt-dlp`.
10
+ - Require Node 22+.
11
+ - Respect `OPENAI_BASE_URL` when set, even with OpenRouter keys.
12
+ - Apply OpenRouter provider ordering headers to HTML→Markdown conversion.
13
+ - Add OpenRouter configuration tests. Thanks @dougvk for the initial OpenRouter support.
14
+ - Build and ship a Bun bytecode arm64 binary for Homebrew.
15
+
16
+ ### Tests
17
+
18
+ - Add coverage for yt-dlp ordering, missing-key errors, and helper paths.
19
+ - Add live coverage for yt-dlp transcript mode and missing-caption YouTube pages.
20
+
21
+ ### Dev
22
+
23
+ - Add `Dockerfile.test` for containerized yt-dlp testing.
24
+
3
25
  ## 0.2.0 - 2025-12-20
4
26
 
5
27
  ### Changes
6
28
 
7
- - Remove map-reduce summarization; reject inputs that exceed the model’s context window.
29
+ - Add native OpenRouter support via `OPENROUTER_API_KEY` with optional provider ordering (`OPENROUTER_PROVIDERS`).
30
+ - Remove map-reduce summarization; reject inputs that exceed the model's context window.
8
31
  - Preflight text prompts with the GPT tokenizer and the model’s max input tokens.
9
32
  - Reject text files over 10 MB before tokenization.
10
33
  - Reject too-small numeric `--length` and `--max-output-tokens` values.
package/README.md CHANGED
@@ -11,6 +11,8 @@ It streams output by default on TTY and renders Markdown to ANSI (via `markdansi
11
11
 
12
12
  ## Install
13
13
 
14
+ Requires Node 22+.
15
+
14
16
  - npx (no install):
15
17
 
16
18
  ```bash
@@ -23,6 +25,8 @@ npx -y @steipete/summarize "https://example.com" --model google/gemini-3-flash-p
23
25
  brew install steipete/tap/summarize
24
26
  ```
25
27
 
28
+ Apple Silicon only (arm64).
29
+
26
30
  ## Quickstart
27
31
 
28
32
  ```bash
@@ -121,11 +125,19 @@ Non-YouTube URLs go through a “fetch → extract” pipeline. When the direct
121
125
  - `--markdown off|auto|llm` (default `auto`; only affects `--extract-only` for non-YouTube URLs)
122
126
  - Plain-text mode: use `--firecrawl off --markdown off`.
123
127
 
124
- ## YouTube transcripts (Apify fallback)
128
+ ## YouTube transcripts
129
+
130
+ `--youtube auto` tries best-effort web transcript endpoints first. When captions aren't available, it falls back to:
131
+
132
+ 1. **Apify** (if `APIFY_API_TOKEN` is set): Uses a scraping actor (`faVsWy9VTSNVIhWpR`)
133
+ 2. **yt-dlp + Whisper** (if `YT_DLP_PATH` is set): Downloads audio via yt-dlp, transcribes with OpenAI Whisper if `OPENAI_API_KEY` is set, otherwise falls back to FAL (`FAL_KEY`)
125
134
 
126
- `--youtube auto` tries best-effort web transcript endpoints first, then falls back to Apify *only if* `APIFY_API_TOKEN` is set.
135
+ Environment variables for yt-dlp mode:
136
+ - `YT_DLP_PATH` - path to yt-dlp binary
137
+ - `OPENAI_API_KEY` - OpenAI Whisper transcription (preferred)
138
+ - `FAL_KEY` - FAL AI Whisper fallback
127
139
 
128
- Apify uses a single actor (`faVsWy9VTSNVIhWpR`). It costs money but tends to be more reliable.
140
+ Apify costs money but tends to be more reliable when captions exist.
129
141
 
130
142
  ## Configuration
131
143
 
@@ -160,13 +172,29 @@ Set the key matching your chosen `--model`:
160
172
 
161
173
  OpenRouter (OpenAI-compatible):
162
174
 
163
- - Set `OPENAI_BASE_URL=https://openrouter.ai/api/v1`
164
- - Prefer `OPENROUTER_API_KEY=...` (instead of reusing `OPENAI_API_KEY`)
165
- - Use OpenRouter models via the `openai/...` prefix, e.g. `--model openai/xiaomi/mimo-v2-flash:free`
175
+ - Set `OPENROUTER_API_KEY=...` to route `openai/...` models through OpenRouter
176
+ - Use OpenRouter models via the `openai/...` prefix, e.g. `--model openai/openai/gpt-oss-20b`
177
+ - Optional: `OPENROUTER_PROVIDERS=...` to specify provider fallback order (e.g. `groq,google-vertex`)
178
+
179
+ Example:
180
+
181
+ ```bash
182
+ OPENROUTER_API_KEY=sk-or-... summarize "https://example.com" --model openai/openai/gpt-oss-20b
183
+ ```
184
+
185
+ With provider ordering (falls back through providers in order):
186
+
187
+ ```bash
188
+ OPENROUTER_API_KEY=sk-or-... OPENROUTER_PROVIDERS="groq,google-vertex" summarize "https://example.com"
189
+ ```
190
+
191
+ Legacy: `OPENAI_BASE_URL=https://openrouter.ai/api/v1` with `OPENAI_API_KEY` also works.
166
192
 
167
193
  Optional services:
168
194
 
169
195
  - `FIRECRAWL_API_KEY` (website extraction fallback)
196
+ - `YT_DLP_PATH` (path to yt-dlp binary for audio extraction)
197
+ - `FAL_KEY` (FAL AI API key for audio transcription via Whisper)
170
198
  - `APIFY_API_TOKEN` (YouTube transcript fallback)
171
199
 
172
200
  ## Model limits