yt-briefing 0.9.2 → 0.11.0

package/.claude/skills/yt-search/SKILL.md

@@ -1,12 +1,12 @@
 ---
 name: yt-search
 description: Search WITHIN one YouTube channel by intent — name a channel and what you're after; the engine lists that channel's uploads, ranks them against your intent (metadata only, no transcript yet), then lazily yields ONE matching video at a time with a rich summary. You keep or skip each; at the end it synthesizes a comparison from everything you kept. Channel-scoped, not whole-YouTube. Same transcript engine + proxy as /yt; lazy on purpose (no transcript bursts → no IP block). Summaries and prompts use the language chosen at onboarding.
-argument-hint: A channel (@handle or URL) and a descriptive intent, e.g. "@t3dotgg which terminal for AI coding". Optional --top N (default 10), --scan N, --since YYYY-MM-DD.
+argument-hint: A channel (@handle or URL) and a descriptive intent, e.g. "@t3dotgg which terminal for AI coding". Searches the channel's whole upload history. Optional --top N (default 10).
 ---
 
 ## How it works
 
-`src/yt-search.ts` is the whole engine: list one channel's uploads (cheap — `playlistItems`, ~1 quota unit/page; NOT `search.list`) → re-rank them against your intent on metadata only (title/description, no transcript) → **lazy** one-candidate-at-a-time yield with a rich summary → record keep/skip → on demand synthesize a comparison from everything kept. **Channel-scoped on purpose** — you choose where to look; it does NOT search all of YouTube. Matching is descriptive: the LLM filters the channel's videos by intent. This skill is a thin loop — paste the summary, collect keep/skip, show the final comparison.
+`src/yt-search.ts` is the whole engine: list the channel's **full upload history** (cheap — `playlistItems`, ~1 quota unit/page; NOT `search.list`) → re-rank it against your intent on metadata only (title/description, no transcript; ranked in chunks and merged so a 1000+ video channel never overflows the LLM) → **lazy** one-candidate-at-a-time yield with a rich summary → record keep/skip → on demand synthesize a comparison from everything kept. **Channel-scoped on purpose** — you choose where to look; it does NOT search all of YouTube. Matching is descriptive: the LLM filters the channel's videos by intent. This skill is a thin loop — paste the summary, collect keep/skip, show the final comparison.
 
 **Lazy on purpose:** one transcript per step, never a burst — a burst looks like scraping and gets the IP blocked (same reason `/yt` is lazy). Run the engine bare — stdout is a single JSON line, stderr empty; never redirect.
 
@@ -53,5 +53,5 @@ while true:
 
 - **Verbatim:** paste `summary` and `comparison` exactly as returned; never paste a raw transcript.
 - **Language:** question text + option descriptions follow `output_lang`; button labels stay `Keep` / `Skip`.
-- **Scope:** one channel per search. Listing is cheap; the cost is the lazy transcript fetches, so let the user keep/skip rather than pulling everything. A bare resume (no `--reset`) continues the same ranked queue. `--scan N` (default 50) caps how many recent uploads are considered; `--since` widens by date.
+- **Scope:** one channel per search, ranked across its **whole upload history**. Listing is cheap; the cost is the lazy transcript fetches, so let the user keep/skip rather than pulling everything. A bare resume (no `--reset`) continues the same ranked queue. `--top N` (default 10) caps how many of the top re-ranked matches get triaged — it is the only flag.
 - **Stateless triage:** independent of `/yt` (no channel profiles, no ratings written). For the recurring multi-channel briefing use `/yt`; for one known video use `/yt-transcribe`.

package/README.md

@@ -1,5 +1,12 @@
 # yt-briefing
 
+[![npm](https://img.shields.io/npm/v/yt-briefing)](https://www.npmjs.com/package/yt-briefing)
+[![license](https://img.shields.io/npm/l/yt-briefing)](./LICENSE)
+[![node](https://img.shields.io/node/v/yt-briefing)](https://www.npmjs.com/package/yt-briefing)
+[![mac](https://github.com/michal90r/yt-briefing/actions/workflows/ci-mac.yml/badge.svg)](https://github.com/michal90r/yt-briefing/actions/workflows/ci-mac.yml)
+[![ubuntu](https://github.com/michal90r/yt-briefing/actions/workflows/ci-ubuntu.yml/badge.svg)](https://github.com/michal90r/yt-briefing/actions/workflows/ci-ubuntu.yml)
+[![windows](https://github.com/michal90r/yt-briefing/actions/workflows/ci-windows.yml/badge.svg)](https://github.com/michal90r/yt-briefing/actions/workflows/ci-windows.yml)
+
 Save hours on YouTube. yt-briefing watches the channels you follow so you don't have to. For
 each new video it gives you a short briefing in your own language — every point that matters, with
 only the filler cut, so nothing important is lost. Reading it takes a fraction of the time the

package/dist/lib/skill-install.js

@@ -13,18 +13,31 @@
  * agent's cwd IS the package folder AND the runtime is Bun (which runs TypeScript directly).
  * That's true for the publisher's own day-to-day use, so it stays the default.
  *
- * For everyone else — a Node user, or any install whose cwd won't be the package — we bake an
- * absolute, runtime-correct command instead: `"<this runtime>" "<abs>/dist/X.js"`. The runtime
- * is `process.execPath` of whoever runs the installer (so `init` under Node bakes node, under
- * Bun bakes bun), and it points at the compiled build, so it runs from any cwd. The engine
- * resolves data/.env from its own location regardless. (Requires `dist/` — build once with
- * `bun run build` / `npm run build`; that's exactly how a Node user already got here.)
+ * For everyone else — a Node user, or any install whose cwd won't be the package — we rewrite to
+ * a PORTABLE command: `<runtime> "<project-relative>/dist/X.js"`. The runtime is the bare name
+ * (`node`/`bun`) resolved from PATH, never an absolute binary; the script and `data/` paths are
+ * relative to the PROJECT ROOT, never machine-absolute. The invariant this rests on is the same
+ * one the whole package already relies on (paths.ts derives BASE_DIR/DATA_DIR from
+ * `process.cwd()` when consumed): the agent runs from the project root. So the rewritten skill is
+ * machine-independent — it survives being committed to git and shared across machines (e.g. a
+ * Mac dev box and a Linux VPS), which an absolute `process.execPath`/`<abs>/dist` baking did not.
+ * (Requires `dist/` — build once with `bun run build` / `npm run build`.)
  */
 import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
-import { join, resolve } from 'node:path';
-import { PKG_ROOT, DATA_DIR } from "./paths.js";
-/** Compiled output dir — what a baked (dist) skill command points the runtime at. */
+import { join, resolve, relative, dirname, sep } from 'node:path';
+import { PKG_ROOT, BASE_DIR, DATA_DIR } from "./paths.js";
+/** Compiled output dir — what a rewritten (dist) skill command points the runtime at. */
 const DIST_DIR = join(PKG_ROOT, 'dist');
+/** Consumed as a dependency? Then PKG_ROOT lives under node_modules (mirrors paths.ts). */
+const CONSUMED = PKG_ROOT.split(sep).includes('node_modules');
+/**
+ * The project root the agent runs from — the cwd against which the rewritten skill's relative
+ * paths resolve. When consumed, that's the user's project (parent of `<project>/.yt-briefing`);
+ * in a clone it's the package itself. Matches how paths.ts picks BASE_DIR.
+ */
+const PROJECT_ROOT = CONSUMED ? dirname(BASE_DIR) : PKG_ROOT;
+/** An absolute path expressed relative to PROJECT_ROOT, with POSIX `/` (portable on Windows too). */
+const toProjectRel = (abs) => relative(PROJECT_ROOT, abs).split(sep).join('/');
 /** The skills this package ships — each lives at `.claude/skills/<name>/SKILL.md`. */
 export const SKILLS = ['yt', 'yt-transcribe', 'yt-search'];
 /** True when the installer itself is running under Bun (vs plain Node). */
@@ -56,25 +69,26 @@ export const AGENTS = {
 /**
  * One shipped skill's SKILL.md. `dist=false` (default) returns it verbatim — the
  * `bun run src/X.ts` dev form, correct only when cwd is the package AND the runtime is Bun.
- * `dist=true` rewrites for the consumed case: engine commands become
- * `"<process.execPath>" "<abs>/dist/X.js"` (this machine's runtime, Node or Bun, against the
- * compiled build, so they run from any cwd), and the bare `data/…` paths the agent reads (e.g.
- * `data/config.json`) become the absolute `DATA_DIR`. In dev the agent's cwd IS the package so
- * `data/` resolves; when consumed, DATA_DIR moves to `<project>/.yt-briefing/data`, so the
- * dev-relative paths would miss — hence the rewrite.
+ * `dist=true` rewrites for the consumed case into PORTABLE, project-relative form: engine
+ * commands become `<node|bun> "<rel>/dist/X.js"` (bare runtime from PATH + a path relative to the
+ * project root, so they run on any machine from the project cwd), and the bare `data/…` paths the
+ * agent reads (e.g. `data/config.json`) become the project-relative DATA_DIR (`.yt-briefing/data/`
+ * when consumed). Nothing machine-absolute is baked, so the rewritten skill can be committed and
+ * shared across machines. The runtime name follows whoever runs the installer (Node→`node`,
+ * Bun→`bun`); the compiled `dist/` build runs under either.
  */
 export function skillBody(name, dist = false) {
     const raw = readFileSync(skillSource(name), 'utf8');
     if (!dist)
         return raw;
-    const exe = process.execPath;
-    const cmd = (base) => `"${exe}" "${join(DIST_DIR, base + '.js')}"`;
+    const runtime = isBun ? 'bun' : 'node';
+    const cmd = (base) => `${runtime} "${toProjectRel(join(DIST_DIR, base + '.js'))}"`;
     return raw
         .replace(/bun run src\/yt-sweep\.ts/g, cmd('yt-sweep'))
         .replace(/bun run src\/yt-rating\.ts/g, cmd('yt-rating'))
         .replace(/bun run src\/yt-transcript\.ts/g, cmd('yt-transcript'))
         .replace(/bun run src\/yt-search\.ts/g, cmd('yt-search'))
-        .replace(/data\//g, DATA_DIR + '/');
+        .replace(/data\//g, toProjectRel(DATA_DIR) + '/');
 }
 /**
  * Write every shipped skill into `root`, each under its own `<name>/SKILL.md` subdir

package/dist/yt-search.js

@@ -5,8 +5,9 @@
  *
  * You point it at a channel and describe what you're after ("which terminal does he recommend
  * for AI coding"); the engine:
- *   1. lists that channel's uploads (cheap — playlistItems, 1 quota unit/page; NOT search.list),
- *   2. re-ranks them against your intent on metadata only — title/description, NO transcript yet,
+ *   1. lists that channel's FULL upload history (cheap — playlistItems, 1 quota unit/page; NOT search.list),
+ *   2. re-ranks it against your intent on metadata only — title/description, NO transcript yet
+ *      (the history is ranked in chunks and merged, so a 1000+ video channel never overflows the LLM),
  *   3. yields ONE matching video at a time with a rich summary, lazily — never a burst of
  *      transcript fetches (a burst looks like scraping and gets the IP blocked),
  *   4. records your keep/skip decision; kept summaries accumulate in a cache,
@@ -16,7 +17,7 @@
  * the channel's videos by intent (no exact-keyword needed).
  *
  * Usage (the skill / CLI drives these; one JSON line per call):
- *   yt-search "<intent>" --channel <@handle|url> [--reset] [--top N] [--scan N] [--since DATE] [--lang auto]
+ *   yt-search "<intent>" --channel <@handle|url> [--reset] [--top N] [--lang auto]
  *   yt-search --keep        record the pending candidate, advance, yield next
  *   yt-search --skip        drop the pending candidate, advance, yield next
  *   yt-search --compare     synthesize a comparison from everything kept
@@ -45,7 +46,7 @@ mkdirSync(CACHE_DIR, { recursive: true });
 const RUNTIME = process.execPath;
 const LANG = outputLang();
 const argv = process.argv.slice(2);
-const VALUE_FLAGS = new Set(['--channel', '--top', '--scan', '--since', '--lang']);
+const VALUE_FLAGS = new Set(['--channel', '--top', '--lang']);
 const has = (f) => argv.includes(f);
 const flagVal = (f) => {
     const i = argv.indexOf(f);
@@ -73,8 +74,6 @@ const SKIP = has('--skip');
 const COMPARE = has('--compare');
 const CHANNEL = flagVal('--channel');
 const TOP = Math.max(1, parseInt(flagVal('--top') || '10', 10));
-const SCAN = Math.max(1, parseInt(flagVal('--scan') || '50', 10)); // recent uploads to consider when no --since
-const SINCE = flagVal('--since');
 const LANGTRACK = flagVal('--lang') || 'auto';
 function emit(obj) {
     process.stdout.write(JSON.stringify(obj));
@@ -116,8 +115,12 @@ function parseJsonArray(out) {
         return null;
     }
 }
-/** Re-rank a channel's videos against the intent (metadata only — no transcript). */
-async function rerank(intent, items) {
+// The whole channel history is ranked, so the pool can be large (a years-old channel is 1000+
+// uploads). One LLM prompt would overflow context, so the pool is ranked in chunks and merged.
+const RERANK_CHUNK = 200;
+/** Re-rank ONE chunk against the intent (metadata only). Returns [] on a parse/LLM failure so a
+ *  bad chunk is dropped rather than flooding the queue with unranked videos. */
+async function rerankBatch(intent, items) {
     if (items.length === 0)
         return [];
     const compact = items.map(h => ({ id: h.videoId, title: h.title, published: h.publishedAt, desc: (h.description || '').slice(0, 280) }));
@@ -135,7 +138,7 @@ Set keep=false for anything not relevant to the intent.`;
         const out = await chat(prompt, { system: 'You output ONLY a raw JSON array as instructed.', temperature: 0 });
         const arr = parseJsonArray(out);
         if (!arr)
-            return items; // fall back: keep all, original (newest-first) order
+            return [];
         const byId = new Map(items.map(h => [h.videoId, h]));
         const ranked = [];
         for (const r of arr) {
@@ -145,11 +148,26 @@ Set keep=false for anything not relevant to the intent.`;
             if (h)
                 ranked.push({ ...h, score: typeof r.score === 'number' ? r.score : undefined, reason: r.reason });
         }
-        return ranked.length ? ranked : items;
+        return ranked;
     }
     catch {
-        return items;
+        return [];
+    }
+}
+/** Re-rank the channel's full history against the intent: chunk the pool, rank each chunk, merge
+ *  the keepers globally by score (highest first), dedup by id. */
+async function rerank(intent, items) {
+    if (items.length === 0)
+        return [];
+    const merged = [];
+    for (let i = 0; i < items.length; i += RERANK_CHUNK) {
+        merged.push(...await rerankBatch(intent, items.slice(i, i + RERANK_CHUNK)));
     }
+    const byId = new Map();
+    for (const c of merged)
+        if (!byId.has(c.videoId))
+            byId.set(c.videoId, c);
+    return [...byId.values()].sort((a, b) => (b.score ?? -1) - (a.score ?? -1));
 }
 /** Rich, standalone summary for one candidate — the triage artifact + compare input. */
 async function megaSummary(c, transcript, intent) {
@@ -278,7 +296,7 @@ async function main() {
         emit({ status: 'error', error: `Could not read a channel handle from "${CHANNEL}" — use @name or the channel URL.` });
     let videos;
     try {
-        videos = await fetchChannelVideos(handle, { since: SINCE, limit: SINCE ? null : SCAN, enrich: false });
+        videos = await fetchChannelVideos(handle, { limit: null, enrich: false }); // full channel history
     }
     catch (e) {
         emit({ status: 'error', error: e.message });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "yt-briefing",
-  "version": "0.9.2",
+  "version": "0.11.0",
   "description": "A self-learning YouTube briefing engine: it sweeps the channels you follow, filters noise in two stages (title, then transcript), summarizes the rest in your language, and adapts to your ratings — one video at a time.",
   "type": "module",
   "bin": {
@@ -22,7 +22,8 @@
     "sweep": "bun run src/yt-sweep.ts",
     "rate": "bun run src/yt-rating.ts",
     "transcribe": "bun run src/yt-transcript.ts",
-    "typecheck": "tsc --noEmit"
+    "typecheck": "tsc --noEmit",
+    "test": "vitest run"
   },
   "engines": {
     "node": ">=18",
@@ -51,6 +52,7 @@
   "devDependencies": {
     "@types/node": "^22",
     "typescript": "^5.7",
-    "@types/bun": "latest"
+    "@types/bun": "latest",
+    "vitest": "^3"
   }
 }