@steipete/summarize 0.9.0 → 0.11.0

package/docs/timestamps.md

@@ -0,0 +1,124 @@
+---
+summary: "Transcript timestamps plan + clickable chat jumps."
+read_when:
+  - "When planning transcript timestamps or click-to-seek UX."
+---
+
+# Transcript Timestamps Plan
+
+Short scope
+
+- Add `--timestamps` flag to request timed transcripts.
+- Preserve existing plain transcript text; add structured segments + timed text.
+- Chat mode: include timed transcript + prompt for `[mm:ss]` references.
+- Sidepanel: click timestamp → seek media (video or audio), keep play state.
+- Coverage: YouTube, podcasts, embedded captions, generic media; whisper.cpp = no segments unless we add verbose output later.
+
+## 1) API / data model
+
+- New option: `FetchLinkContentOptions.transcriptTimestamps?: boolean`.
+- Thread through provider options (`ProviderFetchOptions`).
+- New types:
+  - `TranscriptSegment`: `{ startMs: number; endMs?: number | null; text: string }`.
+  - `TranscriptResolution.segments?: TranscriptSegment[] | null`.
+  - `ExtractedLinkContent.transcriptSegments?: TranscriptSegment[] | null`.
+  - `ExtractedLinkContent.transcriptTimedText?: string | null` (helper).
+- Keep `TranscriptResolution.text` unchanged (plain transcript).
+
+Notes
+
+- `--timestamps` should only alter output when requested; default output remains stable.
+- For JSON output, include both `transcriptSegments` and `transcriptTimedText` when requested.
+
+## 2) Provider updates
+
+YouTube (youtubei)
+
+- Parse `startMs` (and duration if present) from `transcriptSegmentRenderer`.
+- Build segments array; `text` still plain (join of text).
+
+YouTube (captionTracks json3 / xml)
+
+- json3 provides `events[].tStartMs` and `dDurationMs`; parse segments from `events.segs[].utf8`.
+- XML captions include `start` + `dur`; parse segments when present.
+
+Podcast RSS transcripts
+
+- VTT parser should output segments (start/end + cue text).
+- JSON transcript: support `segments` with `start`/`startMs` + `end`/`endMs` + `text`.
+- Plain text transcripts: `segments = null`.
+
+Generic embedded captions
+
+- When track is VTT/JSON, parse into segments; otherwise `null`.
+
+yt-dlp / whisper / whisper.cpp
+
+- Keep `segments = null` (plain text only).
+- Optional future: request verbose or SRT output from OpenAI/FAL when supported.
+
+## 3) Cache behavior
+
+- Store `segments` in transcript metadata (or dedicated cache field).
+- If `--timestamps` and cached transcript lacks segments, treat as miss and refetch.
+- Keep cache keys stable; only bypass when timestamps requested.
+
+## 4) CLI / daemon
+
+- Add `--timestamps` to CLI help + config.
+- Map to `FetchLinkContentOptions.transcriptTimestamps`.
+- `--extract --json`: include `transcriptSegments` + `transcriptTimedText`.
+- Non-JSON extract: keep plain transcript unless `--timestamps`, then output timed text block.
+
+## 5) Chat prompt + content
+
+- `buildChatPageContent`: when timestamps requested, include `Timed transcript:` block using `[mm:ss]`.
+- `buildChatSystemPrompt`: add instruction:
+  - “When referencing moments, include `[mm:ss]` timestamps from the transcript.”
+
+## 6) Chrome extension UI
+
+Render
+
+- Linkify `[mm:ss]` and `[hh:mm:ss]` in assistant messages.
+- Convert to `timestamp:<seconds>` hrefs (or data attribute).
+
+Seek handler
+
+- On click: prevent default, parse seconds, send `panel:seek` → background → content script.
+- Content script:
+  - Find `<video>` or `<audio>`.
+  - Record `wasPaused = media.paused`.
+  - `media.currentTime = seconds`.
+  - If `!wasPaused`, call `media.play()`; else do nothing.
+- YouTube fallback when no media element:
+  - If `window.ytplayer` / YT IFrame API available, `player.seekTo(seconds, true)`.
+
+## 7) Tests
+
+Core
+
+- youtubei transcript parsing yields segments + plain text.
+- captionTracks json3 + xml yield segments.
+- VTT parser yields segments.
+- Cache: timestamps requested + cached without segments → refetch.
+
+Daemon / CLI
+
+- `--timestamps` propagates into fetch options.
+- JSON extract includes `transcriptSegments` + `transcriptTimedText`.
+
+Chrome extension
+
+- Chat content includes timed transcript when requested.
+- Sidepanel: timestamp link emits `panel:seek`.
+- Content script seek: playing stays playing, paused stays paused; audio + video.
+
+## 8) Changelog
+
+- Entry: `--timestamps` flag, timed transcripts in chat, clickable timestamps in extension, podcast support.
+
+## 9) Notes / open
+
+- “VisPoR” = whisper.cpp: no timestamps unless we add verbose output path.
+- Decide exact format of `transcriptTimedText` (recommend `[mm:ss] text` per line).

package/docs/website.md

@@ -21,6 +21,19 @@ Use this for non-YouTube URLs.
 - With `--format md`, `--markdown-mode auto` may fall back to `uvx markitdown` when available (disable with `--preprocess off`).
 - For podcast URLs (Apple Podcasts, RSS, Spotify episodes), it downloads the episode audio and transcribes via Whisper (prefers local `whisper.cpp` when installed + model available); progress shows “Downloading audio …” then “Transcribing …” (duration uses RSS hints or `ffprobe` when available).
 
+Daemon note:
+
+- `/v1/summarize` accepts `format: "markdown"` plus `markdownMode`/`preprocess` to return extracted Markdown (especially when `extractOnly: true`).
+
+## Short content
+
+- If extracted content is shorter than the requested summary length, summarize returns the content as-is.
+- Use `--force-summary` to override and always run the LLM.
+
+## Twitter/X
+
+- Tweet audio transcription runs only with `--video-mode transcript` (auto mode skips yt-dlp for tweets).
+
 ## Flags
 
 - `--firecrawl off|auto|always`

package/docs/youtube.md

@@ -40,3 +40,23 @@ YouTube URLs use transcript-first extraction.
 ```bash
 pnpm summarize -- --extract "https://www.youtube.com/watch?v=I845O57ZSy4&t=11s"
 ```
+
+## Slides
+
+Use `--slides` to extract slide screenshots for YouTube videos (requires `ffmpeg` and `yt-dlp`).
+Scene detection auto-tunes the threshold using sampled frame hashes:
+
+```bash
+summarize "https://www.youtube.com/watch?v=..." --slides
+summarize "https://www.youtube.com/watch?v=..." --slides --slides-ocr
+```
+
+Slides are written to `./slides/<videoId>/` by default (override with `--slides-dir`). OCR results
+are stored in `slides.json` and included in JSON output (`--json`).
+
+If yt-dlp gets a 403 from YouTube, set `SUMMARIZE_YT_DLP_COOKIES_FROM_BROWSER=chrome` (or
+`chrome:Profile 1`) to pass cookies through to yt-dlp.
+
+Relevant flags:
+
+- `--slides-scene-threshold <value>`: starting threshold for scene detection (auto-tuned as needed)

package/package.json

@@ -1,12 +1,19 @@
 {
   "name": "@steipete/summarize",
-  "version": "0.9.0",
+  "version": "0.11.0",
   "description": "Link → clean text → summary.",
-  "type": "module",
   "bin": {
     "summarize": "./dist/cli.js",
     "summarizer": "./dist/cli.js"
   },
+  "files": [
+    "dist",
+    "docs",
+    "CHANGELOG.md",
+    "README.md",
+    "LICENSE"
+  ],
+  "type": "module",
   "main": "./dist/esm/index.js",
   "module": "./dist/esm/index.js",
   "types": "./dist/types/index.d.ts",
@@ -24,63 +31,65 @@
       "import": "./dist/esm/prompts/index.js"
     }
   },
-  "files": [
-    "dist",
-    "docs",
-    "CHANGELOG.md",
-    "README.md",
-    "LICENSE"
-  ],
-  "engines": {
-    "node": ">=22"
+  "scripts": {
+    "build": "pnpm clean && pnpm -C packages/core build && pnpm build:lib && pnpm build:cli",
+    "build:bun": "bun scripts/build-bun.js",
+    "build:bun:test": "bun scripts/build-bun.js --test",
+    "build:cli": "node scripts/build-cli.mjs",
+    "build:lib": "tsc -p tsconfig.build.json",
+    "check": "pnpm format:check && pnpm lint && pnpm test:coverage",
+    "clean": "rimraf dist packages/core/dist",
+    "docs:list": "tsx scripts/docs-list.ts",
+    "format": "oxfmt --write",
+    "format:check": "oxfmt --check",
+    "lint": "oxlint --type-aware --tsconfig tsconfig.build.json --config .oxlintrc.json .",
+    "lint:fix": "oxlint --type-aware --tsconfig tsconfig.build.json --config .oxlintrc.json --fix . && pnpm format",
+    "prepare": "pnpm build",
+    "release": "bash scripts/release.sh",
+    "s": "tsx src/cli.ts",
+    "summarize": "tsx src/cli.ts",
+    "test": "vitest run",
+    "test:coverage": "vitest run --coverage",
+    "test:coverage:build": "pnpm build && pnpm test:coverage",
+    "test:extension-e2e": "pnpm -C apps/chrome-extension test:e2e",
+    "typecheck": "tsc -p tsconfig.build.json --noEmit"
   },
   "dependencies": {
-    "@mariozechner/pi-ai": "^0.30.2",
-    "commander": "^14.0.2",
-    "file-type": "^21.2.0",
+    "@mariozechner/pi-ai": "^0.52.12",
+    "@steipete/summarize-core": "workspace:*",
+    "commander": "^14.0.3",
+    "file-type": "^21.3.0",
     "gpt-tokenizer": "^3.4.0",
     "json5": "^2.2.3",
-    "markdansi": "^0.2.0",
+    "markdansi": "^0.2.1",
     "mime": "^4.1.0",
-    "ora": "^9.0.0",
-    "osc-progress": "^0.2.0",
+    "ora": "^9.3.0",
+    "osc-progress": "^0.3.0",
     "tokentally": "^0.1.1",
-    "tslog": "^4.10.2",
-    "@steipete/summarize-core": "0.9.0"
+    "tslog": "^4.10.2"
   },
   "devDependencies": {
-    "@biomejs/biome": "^2.3.10",
-    "@fal-ai/client": "^1.8.1",
+    "@fal-ai/client": "^1.9.1",
     "@types/jsdom": "^27.0.0",
-    "@types/node": "^25.0.3",
+    "@types/node": "^25.2.3",
     "@types/sanitize-html": "^2.16.0",
-    "@vitest/coverage-v8": "^4.0.16",
-    "esbuild": "^0.27.2",
-    "oxlint": "^1.35.0",
-    "oxlint-tsgolint": "^0.10.0",
+    "@vitest/coverage-v8": "^4.0.18",
+    "esbuild": "^0.27.3",
+    "oxfmt": "0.32.0",
+    "oxlint": "^1.47.0",
+    "oxlint-tsgolint": "^0.12.2",
+    "rimraf": "^6.1.2",
     "tsx": "^4.21.0",
     "typescript": "^5.9.3",
-    "vitest": "^4.0.16"
+    "vitest": "^4.0.18"
   },
-  "scripts": {
-    "clean": "rm -rf dist packages/core/dist",
-    "build": "pnpm clean && pnpm -C packages/core build && pnpm build:lib && pnpm build:cli",
-    "build:lib": "tsc -p tsconfig.build.json",
-    "build:cli": "node scripts/build-cli.mjs",
-    "build:bun": "bun scripts/build-bun.js",
-    "build:bun:test": "bun scripts/build-bun.js --test",
-    "docs:list": "tsx scripts/docs-list.ts",
-    "typecheck": "tsc -p tsconfig.build.json --noEmit",
-    "summarize": "tsx src/cli.ts",
-    "s": "tsx src/cli.ts",
-    "format": "biome format --write .",
-    "lint": "biome check . && oxlint --type-aware --tsconfig tsconfig.build.json --config .oxlintrc.json .",
-    "lint:fix": "biome check --write . && oxlint --type-aware --tsconfig tsconfig.build.json --config .oxlintrc.json --fix .",
-    "check": "pnpm lint && pnpm test:coverage",
-    "test:extension-e2e": "pnpm -C apps/chrome-extension test:e2e",
-    "test": "vitest run",
-    "test:coverage": "vitest run --coverage",
-    "test:coverage:build": "pnpm build && pnpm test:coverage",
-    "release": "bash scripts/release.sh"
+  "engines": {
+    "node": ">=22"
+  },
+  "packageManager": "pnpm@10.25.0+sha512.5e82639027af37cf832061bcc6d639c219634488e0f2baebe785028a793de7b525ffcd3f7ff574f5e9860654e098fe852ba8ac5dd5cefe1767d23a020a92f501",
+  "pnpm": {
+    "overrides": {
+      "vite": "7.3.0"
+    }
   }
-}
+}