@steipete/summarize 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -1
- package/README.md +34 -6
- package/dist/cli.cjs +4941 -790
- package/dist/cli.cjs.map +4 -4
- package/dist/esm/content/link-preview/client.js +6 -0
- package/dist/esm/content/link-preview/client.js.map +1 -1
- package/dist/esm/content/link-preview/transcript/index.js +6 -0
- package/dist/esm/content/link-preview/transcript/index.js.map +1 -1
- package/dist/esm/content/link-preview/transcript/providers/youtube/yt-dlp.js +213 -0
- package/dist/esm/content/link-preview/transcript/providers/youtube/yt-dlp.js.map +1 -0
- package/dist/esm/content/link-preview/transcript/providers/youtube.js +40 -2
- package/dist/esm/content/link-preview/transcript/providers/youtube.js.map +1 -1
- package/dist/esm/flags.js +2 -0
- package/dist/esm/flags.js.map +1 -1
- package/dist/esm/llm/generate-text.js +51 -14
- package/dist/esm/llm/generate-text.js.map +1 -1
- package/dist/esm/llm/html-to-markdown.js +3 -2
- package/dist/esm/llm/html-to-markdown.js.map +1 -1
- package/dist/esm/run.js +40 -9
- package/dist/esm/run.js.map +1 -1
- package/dist/esm/version.js +1 -1
- package/dist/types/content/link-preview/client.d.ts +3 -0
- package/dist/types/content/link-preview/content/types.d.ts +1 -1
- package/dist/types/content/link-preview/deps.d.ts +3 -0
- package/dist/types/content/link-preview/transcript/providers/youtube/yt-dlp.d.ts +15 -0
- package/dist/types/content/link-preview/transcript/types.d.ts +4 -0
- package/dist/types/flags.d.ts +1 -1
- package/dist/types/llm/generate-text.d.ts +8 -2
- package/dist/types/llm/html-to-markdown.d.ts +4 -1
- package/dist/types/version.d.ts +1 -1
- package/docs/youtube.md +5 -2
- package/package.json +5 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,33 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.0 - 2025-12-20
|
|
4
|
+
|
|
5
|
+
### Changes
|
|
6
|
+
|
|
7
|
+
- Add yt-dlp audio transcription fallback for YouTube; prefer OpenAI Whisper with FAL fallback. Thanks @dougvk.
|
|
8
|
+
- Add `--no-playlist` to yt-dlp downloads to avoid transcript mismatches.
|
|
9
|
+
- Run yt-dlp after web + Apify in `--youtube auto`, and error early for missing keys in `--youtube yt-dlp`.
|
|
10
|
+
- Require Node 22+.
|
|
11
|
+
- Respect `OPENAI_BASE_URL` when set, even with OpenRouter keys.
|
|
12
|
+
- Apply OpenRouter provider ordering headers to HTML→Markdown conversion.
|
|
13
|
+
- Add OpenRouter configuration tests. Thanks @dougvk for the initial OpenRouter support.
|
|
14
|
+
- Build and ship a Bun bytecode arm64 binary for Homebrew.
|
|
15
|
+
|
|
16
|
+
### Tests
|
|
17
|
+
|
|
18
|
+
- Add coverage for yt-dlp ordering, missing-key errors, and helper paths.
|
|
19
|
+
- Add live coverage for yt-dlp transcript mode and missing-caption YouTube pages.
|
|
20
|
+
|
|
21
|
+
### Dev
|
|
22
|
+
|
|
23
|
+
- Add `Dockerfile.test` for containerized yt-dlp testing.
|
|
24
|
+
|
|
3
25
|
## 0.2.0 - 2025-12-20
|
|
4
26
|
|
|
5
27
|
### Changes
|
|
6
28
|
|
|
7
|
-
-
|
|
29
|
+
- Add native OpenRouter support via `OPENROUTER_API_KEY` with optional provider ordering (`OPENROUTER_PROVIDERS`).
|
|
30
|
+
- Remove map-reduce summarization; reject inputs that exceed the model's context window.
|
|
8
31
|
- Preflight text prompts with the GPT tokenizer and the model’s max input tokens.
|
|
9
32
|
- Reject text files over 10 MB before tokenization.
|
|
10
33
|
- Reject too-small numeric `--length` and `--max-output-tokens` values.
|
package/README.md
CHANGED
|
@@ -11,6 +11,8 @@ It streams output by default on TTY and renders Markdown to ANSI (via `markdansi
|
|
|
11
11
|
|
|
12
12
|
## Install
|
|
13
13
|
|
|
14
|
+
Requires Node 22+.
|
|
15
|
+
|
|
14
16
|
- npx (no install):
|
|
15
17
|
|
|
16
18
|
```bash
|
|
@@ -23,6 +25,8 @@ npx -y @steipete/summarize "https://example.com" --model google/gemini-3-flash-p
|
|
|
23
25
|
brew install steipete/tap/summarize
|
|
24
26
|
```
|
|
25
27
|
|
|
28
|
+
Apple Silicon only (arm64).
|
|
29
|
+
|
|
26
30
|
## Quickstart
|
|
27
31
|
|
|
28
32
|
```bash
|
|
@@ -121,11 +125,19 @@ Non-YouTube URLs go through a “fetch → extract” pipeline. When the direct
|
|
|
121
125
|
- `--markdown off|auto|llm` (default `auto`; only affects `--extract-only` for non-YouTube URLs)
|
|
122
126
|
- Plain-text mode: use `--firecrawl off --markdown off`.
|
|
123
127
|
|
|
124
|
-
## YouTube transcripts
|
|
128
|
+
## YouTube transcripts
|
|
129
|
+
|
|
130
|
+
`--youtube auto` tries best-effort web transcript endpoints first. When captions aren't available, it falls back to:
|
|
131
|
+
|
|
132
|
+
1. **Apify** (if `APIFY_API_TOKEN` is set): Uses a scraping actor (`faVsWy9VTSNVIhWpR`)
|
|
133
|
+
2. **yt-dlp + Whisper** (if `YT_DLP_PATH` is set): Downloads audio via yt-dlp, transcribes with OpenAI Whisper if `OPENAI_API_KEY` is set, otherwise falls back to FAL (`FAL_KEY`)
|
|
125
134
|
|
|
126
|
-
|
|
135
|
+
Environment variables for yt-dlp mode:
|
|
136
|
+
- `YT_DLP_PATH` - path to yt-dlp binary
|
|
137
|
+
- `OPENAI_API_KEY` - OpenAI Whisper transcription (preferred)
|
|
138
|
+
- `FAL_KEY` - FAL AI Whisper fallback
|
|
127
139
|
|
|
128
|
-
Apify
|
|
140
|
+
Apify costs money but tends to be more reliable when captions exist.
|
|
129
141
|
|
|
130
142
|
## Configuration
|
|
131
143
|
|
|
@@ -160,13 +172,29 @@ Set the key matching your chosen `--model`:
|
|
|
160
172
|
|
|
161
173
|
OpenRouter (OpenAI-compatible):
|
|
162
174
|
|
|
163
|
-
- Set `
|
|
164
|
-
-
|
|
165
|
-
-
|
|
175
|
+
- Set `OPENROUTER_API_KEY=...` to route `openai/...` models through OpenRouter
|
|
176
|
+
- Use OpenRouter models via the `openai/...` prefix, e.g. `--model openai/openai/gpt-oss-20b`
|
|
177
|
+
- Optional: `OPENROUTER_PROVIDERS=...` to specify provider fallback order (e.g. `groq,google-vertex`)
|
|
178
|
+
|
|
179
|
+
Example:
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
OPENROUTER_API_KEY=sk-or-... summarize "https://example.com" --model openai/openai/gpt-oss-20b
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
With provider ordering (falls back through providers in order):
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
OPENROUTER_API_KEY=sk-or-... OPENROUTER_PROVIDERS="groq,google-vertex" summarize "https://example.com"
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Legacy: `OPENAI_BASE_URL=https://openrouter.ai/api/v1` with `OPENAI_API_KEY` also works.
|
|
166
192
|
|
|
167
193
|
Optional services:
|
|
168
194
|
|
|
169
195
|
- `FIRECRAWL_API_KEY` (website extraction fallback)
|
|
196
|
+
- `YT_DLP_PATH` (path to yt-dlp binary for audio extraction)
|
|
197
|
+
- `FAL_KEY` (FAL AI API key for audio transcription via Whisper)
|
|
170
198
|
- `APIFY_API_TOKEN` (YouTube transcript fallback)
|
|
171
199
|
|
|
172
200
|
## Model limits
|