youtube-knowledge-mcp 2.0.2 → 2.2.0

package/README.md

@@ -18,6 +18,7 @@ Supports both **local** (stdio) and **remote** (Streamable HTTP) transports.
 - **Search** videos and channels by keyword
 - **Fetch** videos from a playlist or channel
 - **Video, channel and playlist metadata**, chapters, and top comments
+- **Thumbnails**: look at any video's thumbnail or a channel's avatar or banner as an image, or save every thumbnail of a channel to disk
 - **Transcripts with timestamps**, sliced by time range or chapter, and capped so a
   three-hour video cannot flood your context
 - **Search inside a transcript** and get back `?t=` links that open the video at
@@ -39,6 +40,21 @@ Supports both **local** (stdio) and **remote** (Streamable HTTP) transports.
 - **Read them back**, and **search across all of them** with full-text ranking
 - **Tag, retag and delete**
 
+**Build a brain for a channel** (local mode)
+
+- **Read a whole channel** into a searchable corpus of timestamped passages, in
+  any caption language, resumable and safe to interrupt — a second run continues
+  where it stopped and picks up new uploads
+- **Ask what a creator has said** about anything, across every video, and get
+  back the moments themselves with links that open the video there
+- **Measure the channel**: how much was readable, its upload rhythm, its
+  speaking rate, and the phrases it repeats across videos
+- **Keep a written profile** beside the corpus, grounded in passages you can cite
+
+**Study a channel's thumbnails** (local mode)
+
+- **Save every thumbnail** of a channel, plus its avatar and banner, resumably, at the largest size YouTube serves, with each image's real pixel size recorded
+
 **Built to stay working**
 
 - WebVTT parsed by the W3C reference implementation, not a hand-written matcher
@@ -46,7 +62,9 @@ Supports both **local** (stdio) and **remote** (Streamable HTTP) transports.
   wall of yt-dlp stderr
 - Timeouts, retry with backoff, and a concurrency limit on every yt-dlp call
 - `check_health` diagnoses missing or outdated yt-dlp and ffmpeg
+- Bearer auth, a loopback default bind, an allowlist on every URL and image host, and typed error codes on every failure
 - Structured output on every tool, plus MCP resources, prompts and completions
+- An end-to-end suite runs the built server against real yt-dlp over both transports every week
 
 ## Prerequisites
 
@@ -143,7 +161,11 @@ npm run build
 npm run start:http
 ```
 
-The server listens on `PORT` (default 3000). Set `PORT` environment variable to change.
+The server listens on `127.0.0.1:PORT` (default 3000). To reach it from another
+machine set `MCP_BIND_HOST=0.0.0.0` together with `MCP_AUTH_TOKEN`; the server
+refuses to start exposed and unauthenticated unless you also set
+`MCP_ALLOW_UNAUTHENTICATED=true`. Behind a reverse proxy, set `MCP_TRUST_PROXY`
+to the number of hops and `MCP_PUBLIC_URL` to the address clients use.
 
 #### Docker
 
@@ -199,7 +221,7 @@ a cold start. Any paid plan removes that.
 
 ## MCP Tools
 
-27 tools. The 14 read-only ones work over both transports; the 13 that touch
+37 tools. The 15 read-only ones work over both transports; the 22 that touch
 your filesystem are registered only in local (stdio) mode, so a remote
 deployment cannot reach the host's disk.
 
@@ -209,18 +231,19 @@ available for this video. Call get_transcript again with one of: fr, es, de.`
 
 ### Discovery — remote + local
 
-| Tool                | Key parameters   | Returns                                                          |
-| ------------------- | ---------------- | ---------------------------------------------------------------- |
-| `search_videos`     | `query`, `limit` | Matching videos with durations, channels and view counts         |
-| `search_channels`   | `query`, `limit` | Matching channels with subscriber counts                         |
-| `fetch_videos`      | `url`, `limit`   | Videos in a playlist or channel                                  |
-| `get_video_info`    | `video`          | Title, channel, duration, views, likes, description, tags        |
-| `get_channel_info`  | `channel`        | Name, handle, subscriber count, description                      |
-| `get_playlist_info` | `url`            | Title, channel, video count, last updated                        |
-| `get_chapters`      | `video`          | Chapter titles with start/end times and deep links               |
-| `get_comments`      | `video`, `limit` | Top-level comments by popularity                                 |
-| `list_formats`      | `video`          | Available formats grouped by video+audio, video-only, audio-only |
-| `check_health`      | —                | yt-dlp and ffmpeg status, versions, and staleness warnings       |
+| Tool                | Key parameters                         | Returns                                                                                   |
+| ------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------- |
+| `search_videos`     | `query`, `limit`                       | Matching videos with durations, channels and view counts                                  |
+| `search_channels`   | `query`, `limit`                       | Matching channels with subscriber counts                                                  |
+| `fetch_videos`      | `url`, `limit`                         | Videos in a playlist or channel, with thumbnail URLs                                      |
+| `get_video_info`    | `video`                                | Title, channel, duration, views, likes, description, tags                                 |
+| `get_channel_info`  | `channel`                              | Name, handle, subscriber count, description, avatar and banner                            |
+| `get_playlist_info` | `url`                                  | Title, channel, video count, last updated                                                 |
+| `get_chapters`      | `video`                                | Chapter titles with start/end times and deep links                                        |
+| `get_comments`      | `video`, `limit`                       | Top-level comments by popularity                                                          |
+| `list_formats`      | `video`                                | Available formats grouped by video+audio, video-only, audio-only                          |
+| `get_thumbnail`     | `video`, `channel`, `image`, `quality` | A video thumbnail, channel avatar or banner as an image, with its real size               |
+| `check_health`      | —                                      | yt-dlp and ffmpeg status, versions, staleness warnings, and which session options are set |
 
 ### Transcripts — remote + local
 
@@ -258,22 +281,76 @@ keyframe-aligned cut. All of these require ffmpeg.
 | `save_to_library`       | `videoId`, `title`, `content`, `contentType`, `channel`, `tags` | Path to the saved note              |
 | `list_library`          | `tag`                                                           | Saved items, newest first           |
 | `get_library_item`      | `videoId`, `contentType`                                        | The saved markdown and its metadata |
-| `search_library`        | `query`, `limit`                                                | Ranked matches with excerpts        |
+| `search_library`        | `query`, `limit`, `offset`                                      | Ranked matches with excerpts        |
 | `update_library_tags`   | `videoId`, `add`, `remove`, `replace`                           | The updated tags                    |
 | `delete_library_item`   | `videoId`, `contentType`                                        | What was deleted                    |
 | `rebuild_library_index` | —                                                               | Number of notes reindexed           |
 
+### Channel brains — local only
+
+| Tool                 | Key parameters                                                    | Returns                                          |
+| -------------------- | ----------------------------------------------------------------- | ------------------------------------------------ |
+| `build_brain`        | `channel`, `maxVideos`, `language`, `since`, `minDurationSeconds` | What was read, what was ruled out, and the stats |
+| `ask_brain`          | `channel`, `query`, `limit`, `offset`                             | Passages with timestamps and `?t=` links         |
+| `list_brains`        | —                                                                 | Every brain built locally                        |
+| `get_brain_info`     | `channel`, `includeVideos`                                        | Coverage, statistics and repeated phrases        |
+| `save_brain_profile` | `channel`, `content`                                              | Path to the saved profile                        |
+| `delete_brain`       | `channel`                                                         | What was removed                                 |
+
+`build_brain` is the only one that touches the network. The rest resolve a
+channel from what is already on disk, so they work offline and cost nothing to
+call.
+
+### Channel thumbnails — local only
+
+| Tool                        | Key parameters                               | Returns                                                   |
+| --------------------------- | -------------------------------------------- | --------------------------------------------------------- |
+| `fetch_channel_thumbnails`  | `channel`, `maxVideos`, `tabs`, `quality`    | Every thumbnail saved to disk, plus the avatar and banner |
+| `list_channel_thumbnails`   | `channel`, `tab`, `state`, `limit`, `offset` | Saved entries with paths and decoded sizes                |
+| `delete_channel_thumbnails` | `channel`                                    | What was removed                                          |
+
+One yt-dlp listing per tab, then the images from YouTube's image hosts
+directly, so a five-hundred-video channel costs a handful of listings and not
+five hundred extractions. For each video the fetch tries the largest image
+YouTube serves (`maxresdefault`, 1280 wide where it exists) and keeps it only if
+it decodes wider than the image the listing offered; otherwise it keeps the
+listed image, and failing that the small `hqdefault`. Every width and height in
+the manifest was decoded from the saved bytes, never read off a URL. Shorts are
+listed with portrait thumbnails and saved under `shorts/`. The run is resumable:
+interrupt it and call again, and only what is missing or truncated is fetched.
+
+A brain holds one caption language; pass `language` to read another, and build a
+separate brain per language.
+
+`since` and `minDurationSeconds` describe the brain, not just the call that
+passed them. They are re-applied every time, so narrowing one drops the passages
+of the videos it excludes and widening it reads them back — which is why
+`build_brain` is annotated as destructive. Whether a video qualifies is decided
+from the date and length already recorded, so changing your mind costs no
+requests until there is something new to fetch. Those values come from each
+video's own metadata, never from a guess: a flat channel listing does not carry
+a publication date at all.
+
+`build_brain` also repairs. If the passage file is lost or truncated, the videos
+it can no longer account for are read again on the next call rather than being
+skipped forever as already done.
+
 ## Prompts
 
 Reusable workflows your client can invoke directly: `summarize_video`,
 `extract_skill`, `compare_videos`, `research_topic`, `channel_deep_dive`,
-`clip_from_quote` (find a phrase, then cut the clip around it), and
-`review_library` (local only).
+`clip_from_quote` (find a phrase, then cut the clip around it),
+`study_thumbnails` (save a channel's thumbnails, look at a sample, describe
+what recurs), and — local only — `review_library`, `create_brain` (build a
+channel's corpus, then write its profile from it) and `ask_creator` (answer a
+question strictly from a brain, with citations).
 
 ## Resources
 
 - `youtube://transcript/{videoId}` — a timestamped transcript, fetched and cached on first read
 - `youtube://library/{videoId}/{summary|skill}` — a saved note (local only, and enumerable)
+- `youtube://brain/{channelId}/{manifest|profile}` — what a channel brain covers, or the profile written from it (local only, and enumerable)
+- `youtube://thumbnails/{channelId}/manifest` — what `fetch_channel_thumbnails` saved for a channel (local only, and enumerable)
 
 ## Error codes
 
@@ -289,25 +366,50 @@ them, each prefixed with a code and followed by a next step.
 | `RATE_LIMITED`, `TIMEOUT`                                           | Transient; retried automatically with backoff before surfacing               |
 | `YTDLP_MISSING`, `FFMPEG_MISSING`, `YTDLP_FAILED`                   | A tooling problem; the message says how to fix it                            |
 | `INVALID_INPUT`                                                     | A bad argument, caught before any network call                               |
+| `BOT_CHECK`                                                         | YouTube asked this address to sign in; set the cookie settings below         |
+| `FETCH_FAILED`                                                      | An image could not be downloaded from YouTube's image hosts                  |
 | `CANCELLED`                                                         | The client cancelled the request                                             |
 
 ## Environment variables
 
 All optional.
 
-| Variable                        | Default   | Purpose                                                                                                  |
-| ------------------------------- | --------- | -------------------------------------------------------------------------------------------------------- |
-| `MCP_AUTH_TOKEN`                | unset     | Require this bearer token on the HTTP transport. **Set this if you expose the server beyond localhost.** |
-| `MCP_ALLOWED_HOSTS`             | unset     | Comma-separated Host allowlist; enables DNS-rebinding protection                                         |
-| `MCP_ALLOWED_ORIGINS`           | unset     | Comma-separated Origin allowlist                                                                         |
-| `MCP_BIND_HOST`                 | `0.0.0.0` | Interface to bind                                                                                        |
-| `PORT`                          | `3000`    | HTTP port. The Docker image sets `10000`; Render and similar platforms inject their own                  |
-| `MCP_RATE_LIMIT`                | `60`      | Requests per window, per client                                                                          |
-| `MCP_RATE_WINDOW_MS`            | `60000`   | Rate-limit window                                                                                        |
-| `MCP_SESSION_IDLE_MS`           | `1800000` | Close HTTP sessions idle this long                                                                       |
-| `MCP_MAX_SESSIONS`              | `1000`    | Reject new sessions past this many                                                                       |
-| `YOUTUBE_MCP_MAX_CONCURRENCY`   | `3`       | Concurrent yt-dlp processes                                                                              |
-| `YOUTUBE_MCP_TRANSCRIPT_TTL_MS` | 30 days   | Transcript cache lifetime                                                                                |
+| Variable                           | Default     | Purpose                                                                                                                                                      |
+| ---------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `MCP_AUTH_TOKEN`                   | unset       | Require this bearer token on the HTTP transport. **Set this if you expose the server beyond localhost.**                                                     |
+| `MCP_ALLOWED_HOSTS`                | unset       | Comma-separated Host allowlist; enables DNS-rebinding protection                                                                                             |
+| `MCP_ALLOWED_ORIGINS`              | unset       | Comma-separated Origin allowlist                                                                                                                             |
+| `MCP_BIND_HOST`                    | `127.0.0.1` | Interface to bind. The Docker image sets `0.0.0.0`. A non-loopback bind refuses to start without `MCP_AUTH_TOKEN` unless `MCP_ALLOW_UNAUTHENTICATED=true`    |
+| `MCP_ALLOW_UNAUTHENTICATED`        | `false`     | Explicit consent to listen on a network interface with no token                                                                                              |
+| `MCP_TRUST_PROXY`                  | `false`     | `true`, `false`, or the number of reverse-proxy hops whose `X-Forwarded-*` headers to believe. Off, forwarded headers are ignored for rate limiting and URLs |
+| `MCP_PUBLIC_URL`                   | unset       | The URL clients reach the server at; used in the OAuth metadata it publishes instead of the request's Host                                                   |
+| `MCP_MAX_BODY_BYTES`               | `1048576`   | Largest request body accepted                                                                                                                                |
+| `MCP_MODE`                         | unset       | `http` selects the HTTP transport when no `--http` flag is given                                                                                             |
+| `PORT`                             | `3000`      | HTTP port. The Docker image sets `10000`; Render and similar platforms inject their own                                                                      |
+| `MCP_RATE_LIMIT`                   | `60`        | Requests per window, per client                                                                                                                              |
+| `MCP_RATE_WINDOW_MS`               | `60000`     | Rate-limit window                                                                                                                                            |
+| `MCP_SESSION_IDLE_MS`              | `1800000`   | Close HTTP sessions idle this long                                                                                                                           |
+| `MCP_MAX_SESSIONS`                 | `1000`      | Reject new sessions past this many                                                                                                                           |
+| `YOUTUBE_MCP_MAX_CONCURRENCY`      | `3`         | Concurrent yt-dlp processes                                                                                                                                  |
+| `YOUTUBE_MCP_TRANSCRIPT_TTL_MS`    | 30 days     | Transcript cache lifetime                                                                                                                                    |
+| `YOUTUBE_MCP_COOKIES_FROM_BROWSER` | unset       | Browser whose cookies yt-dlp should read: `brave`, `chrome`, `chromium`, `edge`, `firefox`, `opera`, `safari`, `vivaldi` or `whale`. See Signed-in content   |
+| `YOUTUBE_MCP_COOKIES_FILE`         | unset       | A Netscape-format cookies file inside your home directory, as an alternative to a browser                                                                    |
+| `YOUTUBE_MCP_PROXY`                | unset       | An `http`, `https`, `socks4` or `socks5` proxy URL for yt-dlp                                                                                                |
+| `YOUTUBE_MCP_SLEEP_REQUESTS_S`     | unset       | Seconds yt-dlp sleeps between its own requests, to stay under YouTube's limits                                                                               |
+
+### Signed-in content
+
+Some videos need a signed-in session — age-restricted, members-only, private
+ones you have access to — and YouTube sometimes asks an address to prove it is
+not a bot before serving any video at all. Those failures arrive as
+`LOGIN_REQUIRED`, `AGE_GATED` or `BOT_CHECK`, and each names the fix: set
+`YOUTUBE_MCP_COOKIES_FROM_BROWSER` to a browser you are signed in with, or
+`YOUTUBE_MCP_COOKIES_FILE` to a cookies file, and restart the server.
+
+Cookies are your account. Use them in local (stdio) mode, keep a cookies file
+readable by you alone, and know that content read this way may be personal.
+The server validates both settings at boot, never logs the file's path or
+contents, and never surfaces yt-dlp's output.
 
 ## Library Storage
 
@@ -322,6 +424,18 @@ Content is stored in `~/.youtube-knowledge/`:
 │       ├── metadata.json
 │       ├── summary.md
 │       └── skill.md
+├── brains/               # Channel brains
+│   └── {channel_id}/
+│       ├── manifest.json # What the brain covers, and where a build stopped
+│       ├── chunks.json   # The timestamped passages
+│       └── profile.md    # The written account, if one was saved
+├── thumbnails/           # Saved channel thumbnails
+│   └── {channel_id}/
+│       ├── manifest.json # Every image, its decoded size, where it is
+│       ├── channel/      # avatar.* and banner.*
+│       ├── videos/       # {video_id}.jpg (or .png / .webp)
+│       ├── shorts/
+│       └── streams/
 ├── downloads/            # Full downloads
 ├── clips/                # Extracted clips
 ├── frames/               # Captured stills
@@ -394,6 +508,18 @@ editor can import.
 `save_to_library` stores it; `search_library` searches across everything saved
 with full-text ranking.
 
+### Build a brain for a creator
+
+```
+"Build a brain for @Fireship, then tell me everything they've said about Rust"
+```
+
+`build_brain` reads the channel into timestamped passages — interrupt it and
+call it again to continue. `ask_brain` then answers from what was actually
+said, returning the moments themselves so every claim can be checked against
+the video. Run `build_brain` again a month later and it reads only the new
+uploads.
+
 ## Testing
 
 ```bash
@@ -407,6 +533,11 @@ client over an in-memory transport, exercises the library against a real
 temporary filesystem, and snapshots the tool manifest so any change to the
 public surface shows up as a reviewable diff.
 
+An end-to-end lane (`E2E=1 npm run test:e2e`) drives the built server through
+real MCP clients over both transports against real yt-dlp and the real network,
+in an isolated temporary home. It runs weekly in CI and on every release before
+anything is published; see CONTRIBUTING.md.
+
 ## Development
 
 ```bash
@@ -428,9 +559,11 @@ project layout, coding standards, and how to add a tool.
 
 ## Security
 
-The HTTP transport is unauthenticated unless you set `MCP_AUTH_TOKEN`. See
-[SECURITY.md](SECURITY.md) before exposing it beyond localhost, and to report a
-vulnerability.
+The HTTP transport binds loopback and refuses to listen on a network interface
+without `MCP_AUTH_TOKEN`. Caller-supplied URLs must be on a YouTube host before
+anything is handed to yt-dlp, every yt-dlp target follows a `--` terminator, and
+ids are validated before they become paths. See [SECURITY.md](SECURITY.md) for
+the deployment checklist and how to report a vulnerability.
 
 ## License