@steipete/summarize 0.1.2 → 0.2.0

package/CHANGELOG.md

@@ -1,12 +1,52 @@
 # Changelog
 
-All notable changes to this project are documented here.
-
-## 0.1.2 - 2025-12-20
+## 0.2.0 - 2025-12-20
+
+### Changes
+
+- Remove map-reduce summarization; reject inputs that exceed the model’s context window.
+- Preflight text prompts with the GPT tokenizer and the model’s max input tokens.
+- Reject text files over 10 MB before tokenization.
+- Reject too-small numeric `--length` and `--max-output-tokens` values.
+- Cap summaries to the extracted content length when a requested size is larger.
+- Skip summarization for tweets when extracted content is already below the requested length.
+- Use bird CLI for tweet extraction when available and surface it in the status line.
+- Fall back to Nitter for tweet extraction when bird fails; report a clear error when tweet data is unavailable.
+- Compute cost totals via tokentally’s tally helpers.
+- Improve fetch spinner with elapsed time and throughput updates.
+- Show Firecrawl fallback status and reason when scraping kicks in.
+- Enforce a hard deadline for stalled streaming LLM responses.
+- Merge cumulative streaming chunks correctly and keep stream-merge for streaming output.
+- Fall back to non-streaming when streaming requests time out.
+- Preserve parentheses in URL paths when resolving inputs.
+- Stop forcing Firecrawl for --extract-only; only use it as a fallback.
+- Avoid Firecrawl fallback when block keywords only appear in scripts/styles.
+
+### Tests
+
+- Add CLI + live coverage for prompt length capping.
+- Add coverage for cumulative stream merge handling.
+- Add coverage for streaming timeout fallback.
+- Add live coverage for Wikipedia URLs with parentheses.
+- Add coverage for tweet summaries that bypass the LLM when short.
+- Add coverage for content budget paths and TOKENTALLY cache dir overrides.
+
+### Docs
+
+- Update release checklist to all-in-one flow.
+- Fix release script quoting.
+- Document input limits and minimum length/token values.
+
+### Dev
+
+- Add a tokenization benchmark script.
 
 ### Fixes
 
-- Avoid duplicate streamed output when providers emit cumulative chunks instead of deltas.
+- Preserve balanced parentheses/brackets in URL paths (e.g. Wikipedia titles).
+- Avoid Firecrawl fallback when block keywords only appear in scripts/styles.
+- Add a Bird install tip when Twitter/X fetch fails without bird installed.
+- Graceful error when tweet extraction fails after bird + Nitter fallback.
 
 ## 0.1.1 - 2025-12-19
 

package/README.md

@@ -89,6 +89,12 @@ npx -y @steipete/summarize "https://example.com" --length 20k
 - Character targets: `1500`, `20k`, `20000`
 - Optional hard cap: `--max-output-tokens <count>` (e.g. `2000`, `2k`)
   - Provider/model APIs still enforce their own maximum output limits.
+- Minimums: `--length` numeric values must be ≥ 50 chars; `--max-output-tokens` must be ≥ 16.
+
+## Limits
+
+- Text inputs over 10 MB are rejected before tokenization.
+- Text prompts are preflighted against the model’s input limit (LiteLLM catalog), using a GPT tokenizer.
 
 ## Common flags