yt-briefing 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/yt/SKILL.md +1 -1
- package/README.md +22 -4
- package/dist/bootstrap.js +25 -92
- package/dist/cli.js +13 -3
- package/dist/lib/channels.js +82 -0
- package/dist/lib/llm.js +8 -8
- package/dist/lib/paths.js +8 -8
- package/dist/lib/yt-api.js +3 -3
- package/dist/yt-channel-videos.js +1 -1
- package/dist/yt-channels.js +74 -0
- package/dist/yt-sweep.js +4 -4
- package/dist/yt-transcript.js +8 -8
- package/docs/sync-across-machines.md +5 -5
- package/docs/warp-proxy.md +2 -2
- package/package.json +1 -1
|
@@ -9,7 +9,7 @@ description: Briefing from the YouTube channels you follow — the engine sweeps
|
|
|
9
9
|
|
|
10
10
|
`data/state.md` is the only persistent cursor. Session state — queue, pending, prefetch, background fill — lives in `data/.cache/` (rebuilt each run, never important to keep). Internals — lazy queue build, filters, summary format, LLM model, transcript fetching, prefetch, proxy — are in `README.md`, not here. No manual pre-flight: the engine self-invalidates a stale queue (new day or `--reset`).
|
|
11
11
|
|
|
12
|
-
**Run the engine bare — no redirects.** Its stdout is a pure JSON line and stderr is empty, so `JSON.parse` of the raw output just works; **never** redirect stderr to `/tmp` or any OS temp dir. For timing diagnostics ("why is the sweep slow") run it once with `
|
|
12
|
+
**Run the engine bare — no redirects.** Its stdout is a pure JSON line and stderr is empty, so `JSON.parse` of the raw output just works; **never** redirect stderr to `/tmp` or any OS temp dir. For timing diagnostics ("why is the sweep slow") run it once with `YT_BRIEFING_DEBUG=1` — it appends per-stage timings to the gitignored `data/.cache/sweep.log`.
|
|
13
13
|
|
|
14
14
|
For fast first paint, the engine expands channels in parallel waves until it has the first ratable video, while a detached background process expands the rest in parallel. And while the user rates a video, it warms the **next** video's summary in another background process, so the following step usually emits instantly. All fully internal — the loop below is unchanged.
|
|
15
15
|
|
package/README.md
CHANGED
|
@@ -9,6 +9,16 @@ It also gets better the more you use it. You give each summary a quick rating, w
|
|
|
9
9
|
or not, and from that it learns what to keep showing you and what to drop. Over time the queue
|
|
10
10
|
becomes yours: less noise, more of what you care about.
|
|
11
11
|
|
|
12
|
+
## First run vs later
|
|
13
|
+
|
|
14
|
+
On a channel's first sweep there is no history, so yt-briefing takes the latest video of each
|
|
15
|
+
kind: the newest long-form, the newest short, and the newest live. That gives you a baseline
|
|
16
|
+
without pulling the whole back catalog.
|
|
17
|
+
|
|
18
|
+
After that it works from history. Each rating moves a per-type cursor forward, so later runs
|
|
19
|
+
only surface videos newer than the ones you already handled, and a session just continues where
|
|
20
|
+
the last one left off.
|
|
21
|
+
|
|
12
22
|
## Setup
|
|
13
23
|
|
|
14
24
|
You'll need Node 18+ or Bun, a YouTube Data API v3 key, an LLM key (a
|
|
@@ -43,6 +53,14 @@ npx yt-briefing init # or: bunx yt-briefing init
|
|
|
43
53
|
|
|
44
54
|
`init` asks for your language, your keys, the channels to follow, and which tool runs `/yt`.
|
|
45
55
|
|
|
56
|
+
Add or remove channels anytime:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
npx yt-briefing add @handle https://youtube.com/@another # one or more, handle or URL
|
|
60
|
+
npx yt-briefing remove @handle # also deletes its learned profile
|
|
61
|
+
npx yt-briefing list # show the current list
|
|
62
|
+
```
|
|
63
|
+
|
|
46
64
|
## Run it
|
|
47
65
|
|
|
48
66
|
Open your project in Claude Code or Cursor and run `/yt`. If it's not listed, start a fresh
|
|
@@ -55,9 +73,9 @@ Any OpenAI-compatible endpoint works. Gemini 2.5 Flash is the easy default. It's
|
|
|
55
73
|
and free to start at [Google AI Studio](https://aistudio.google.com/apikey):
|
|
56
74
|
|
|
57
75
|
```ini
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
76
|
+
YT_BRIEFING_LLM_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
|
|
77
|
+
YT_BRIEFING_LLM_API_KEY=<gemini-key>
|
|
78
|
+
YT_BRIEFING_LLM_MODEL=gemini-2.5-flash
|
|
61
79
|
```
|
|
62
80
|
|
|
63
81
|
> On the free tier Gemini sometimes returns a "model is overloaded / high demand" error. Retry,
|
|
@@ -92,7 +110,7 @@ flowing.
|
|
|
92
110
|
|
|
93
111
|
## Sync across machines
|
|
94
112
|
|
|
95
|
-
Your state is plain files in `.yt-briefing/data/`. Version that folder (or point `
|
|
113
|
+
Your state is plain files in `.yt-briefing/data/`. Version that folder (or point `YT_BRIEFING_DATA_DIR`
|
|
96
114
|
at a separate private repo) and commit after each rating. Recipe:
|
|
97
115
|
[docs/sync-across-machines.md](./docs/sync-across-machines.md).
|
|
98
116
|
|
package/dist/bootstrap.js
CHANGED
|
@@ -14,9 +14,11 @@
|
|
|
14
14
|
* Everything it writes is plain Markdown / JSON you can also edit by hand afterwards.
|
|
15
15
|
*/
|
|
16
16
|
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
|
|
17
|
-
import {
|
|
17
|
+
import { join } from 'node:path';
|
|
18
|
+
import { DATA_DIR, BASE_DIR, PKG_ROOT, CHANNELS_DIR, CHANNELS_MD, STATE_MD, CONFIG_JSON, ENV_PATH, profilePath, } from "./lib/paths.js";
|
|
18
19
|
import { AGENTS, installSkill, projectSkillDir, customSkillDirDefault, isPackageDevCwd } from "./lib/skill-install.js";
|
|
19
20
|
import { question } from "./lib/prompt.js";
|
|
21
|
+
import { normalizeHandle, slugify, serializeChannels, serializeState, profileBody, baselineStateRow } from "./lib/channels.js";
|
|
20
22
|
const ask = (q, def = '') => {
|
|
21
23
|
const a = question(def ? `${q} [${def}]:` : `${q}:`).trim();
|
|
22
24
|
return a || def;
|
|
@@ -27,35 +29,6 @@ const askYN = (q, def = true) => {
|
|
|
27
29
|
return def;
|
|
28
30
|
return a.startsWith('y');
|
|
29
31
|
};
|
|
30
|
-
/**
|
|
31
|
-
* Accept any of: `@betterstack`, `betterstack`, `https://www.youtube.com/@betterstack`
|
|
32
|
-
* (with or without a trailing `/videos` etc.) → canonical `@betterstack`. Returns null if
|
|
33
|
-
* no handle can be read (e.g. a bare `/channel/UC…` URL — ask the user for the @handle).
|
|
34
|
-
*/
|
|
35
|
-
function normalizeHandle(input) {
|
|
36
|
-
let s = input.trim();
|
|
37
|
-
if (/youtube\.com/i.test(s) || /^https?:\/\//i.test(s)) {
|
|
38
|
-
const m = s.match(/@[A-Za-z0-9._-]+/); // pull the @handle out of a URL
|
|
39
|
-
s = m ? m[0] : '';
|
|
40
|
-
}
|
|
41
|
-
s = s.replace(/^@+/, '').replace(/[^A-Za-z0-9._-].*$/, ''); // bare token
|
|
42
|
-
return s ? `@${s}` : null;
|
|
43
|
-
}
|
|
44
|
-
/** kebab-case slug that also breaks CamelCase: "BetterStack" → "better-stack". */
|
|
45
|
-
function slugify(s) {
|
|
46
|
-
return s
|
|
47
|
-
.replace(/@/g, '')
|
|
48
|
-
.replace(/([a-z0-9])([A-Z])/g, '$1-$2')
|
|
49
|
-
.replace(/[^A-Za-z0-9]+/g, '-')
|
|
50
|
-
.replace(/-+/g, '-')
|
|
51
|
-
.replace(/^-|-$/g, '')
|
|
52
|
-
.toLowerCase();
|
|
53
|
-
}
|
|
54
|
-
const today = new Date().toISOString().slice(0, 10);
|
|
55
|
-
// Fresh channels start "baseline": the first sweep surfaces the latest upload per type.
|
|
56
|
-
// updated is set this far back so that window actually contains a recent video.
|
|
57
|
-
const BASELINE_LOOKBACK_DAYS = 60;
|
|
58
|
-
const lookback = new Date(Date.now() - BASELINE_LOOKBACK_DAYS * 864e5).toISOString().slice(0, 10);
|
|
59
32
|
function main() {
|
|
60
33
|
console.log('\n yt-briefing — onboarding\n ' + '─'.repeat(40) + '\n');
|
|
61
34
|
if (existsSync(CHANNELS_MD)) {
|
|
@@ -94,15 +67,15 @@ function main() {
|
|
|
94
67
|
}
|
|
95
68
|
else {
|
|
96
69
|
console.log('\n → Custom / local endpoint (e.g. Ollama at http://localhost:11434/v1)');
|
|
97
|
-
llmBaseUrl = ask('
|
|
98
|
-
llmModel = ask('
|
|
99
|
-
llmKey = ask('
|
|
70
|
+
llmBaseUrl = ask(' YT_BRIEFING_LLM_BASE_URL', 'http://localhost:11434/v1');
|
|
71
|
+
llmModel = ask(' YT_BRIEFING_LLM_MODEL', 'llama3.1');
|
|
72
|
+
llmKey = ask(' YT_BRIEFING_LLM_API_KEY (blank for local)', '');
|
|
100
73
|
}
|
|
101
74
|
console.log('\n 3) YouTube Data API (needed to list channel uploads)');
|
|
102
75
|
console.log(' Get a key: https://console.cloud.google.com → YouTube Data API v3');
|
|
103
|
-
const ytKey = ask('
|
|
76
|
+
const ytKey = ask(' YT_BRIEFING_YOUTUBE_API_KEY');
|
|
104
77
|
console.log('\n 4) Proxy (optional — only needed on datacenter/VPS IPs; see docs/warp-proxy.md)');
|
|
105
|
-
const ytProxy = ask('
|
|
78
|
+
const ytProxy = ask(' YT_BRIEFING_PROXY (blank = direct)', '');
|
|
106
79
|
// 5. Channels ----------------------------------------------------------------
|
|
107
80
|
// Just collect a flat list. No categories, no per-channel rules to define up front —
|
|
108
81
|
// each channel's profile LEARNS what to skip as you rate it (## Skip titles / ## Notes).
|
|
@@ -146,68 +119,28 @@ function main() {
|
|
|
146
119
|
mkdirSync(CHANNELS_DIR, { recursive: true });
|
|
147
120
|
// .env
|
|
148
121
|
const envBody = [
|
|
149
|
-
`
|
|
150
|
-
`
|
|
151
|
-
`
|
|
152
|
-
`
|
|
153
|
-
`
|
|
122
|
+
`YT_BRIEFING_LLM_BASE_URL=${llmBaseUrl}`,
|
|
123
|
+
`YT_BRIEFING_LLM_API_KEY=${llmKey}`,
|
|
124
|
+
`YT_BRIEFING_LLM_MODEL=${llmModel}`,
|
|
125
|
+
`YT_BRIEFING_YOUTUBE_API_KEY=${ytKey}`,
|
|
126
|
+
`YT_BRIEFING_PROXY=${ytProxy}`,
|
|
154
127
|
'',
|
|
155
128
|
].join('\n');
|
|
156
129
|
writeFileSync(ENV_PATH, envBody, 'utf8');
|
|
130
|
+
// Secret-safety for the consume layout: drop a .gitignore inside .yt-briefing/ so .env never
|
|
131
|
+
// gets committed regardless of the host project's own ignore rules. data/ stays versionable
|
|
132
|
+
// (for sync). In a dev clone (BASE_DIR === PKG_ROOT) the repo's own .gitignore already covers it.
|
|
133
|
+
if (BASE_DIR !== PKG_ROOT) {
|
|
134
|
+
writeFileSync(join(BASE_DIR, '.gitignore'), '.env\ndata/.cache/\n', 'utf8');
|
|
135
|
+
}
|
|
157
136
|
// config.json
|
|
158
137
|
writeFileSync(CONFIG_JSON, JSON.stringify({ output_lang: outputLang }, null, 2) + '\n', 'utf8');
|
|
159
|
-
// channels.md
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
`updated: ${today}`,
|
|
166
|
-
'---',
|
|
167
|
-
'',
|
|
168
|
-
'# Channels',
|
|
169
|
-
'',
|
|
170
|
-
];
|
|
171
|
-
for (const c of channels) {
|
|
172
|
-
chParts.push(`- [${c.handle}](https://www.youtube.com/${c.handle}) → [[channels/${c.slug}]]`);
|
|
173
|
-
}
|
|
174
|
-
writeFileSync(CHANNELS_MD, chParts.join('\n') + '\n', 'utf8');
|
|
175
|
-
// state.md — one flat table (baseline rows: pointers "—", updated = lookback so the first
|
|
176
|
-
// sweep finds the latest upload per type).
|
|
177
|
-
const stParts = [
|
|
178
|
-
'---',
|
|
179
|
-
'type: yt-state',
|
|
180
|
-
'name: yt-state',
|
|
181
|
-
'description: Per-channel per-type cursor. The sweep reads and updates it.',
|
|
182
|
-
`updated: ${today}`,
|
|
183
|
-
'---',
|
|
184
|
-
'',
|
|
185
|
-
'# State',
|
|
186
|
-
'',
|
|
187
|
-
'| Channel | last_longform_id | last_short_id | last_live_id | updated | session |',
|
|
188
|
-
'|---|---|---|---|---|---|',
|
|
189
|
-
];
|
|
190
|
-
for (const c of channels) {
|
|
191
|
-
stParts.push(`| ${c.handle} | — | — | — | ${lookback} | 0 |`);
|
|
192
|
-
}
|
|
193
|
-
writeFileSync(STATE_MD, stParts.join('\n') + '\n', 'utf8');
|
|
194
|
-
// per-channel profiles
|
|
195
|
-
for (const c of channels) {
|
|
196
|
-
const pParts = [
|
|
197
|
-
'---',
|
|
198
|
-
'type: yt-channel-profile',
|
|
199
|
-
`name: ${c.slug}-profile`,
|
|
200
|
-
`channel: ${c.handle}`,
|
|
201
|
-
`channel_url: https://www.youtube.com/${c.handle}`,
|
|
202
|
-
`updated: ${today}`,
|
|
203
|
-
'sessions_observed: 0',
|
|
204
|
-
'---',
|
|
205
|
-
'',
|
|
206
|
-
`# ${c.handle} — Profile`,
|
|
207
|
-
'',
|
|
208
|
-
];
|
|
209
|
-
writeFileSync(profilePath(c.slug), pParts.join('\n') + '\n', 'utf8');
|
|
210
|
-
}
|
|
138
|
+
// channels.md / state.md / per-channel profiles — via the shared serializers, so the
|
|
139
|
+
// on-disk format is identical to what the add/remove command writes (single source).
|
|
140
|
+
writeFileSync(CHANNELS_MD, serializeChannels(channels), 'utf8');
|
|
141
|
+
writeFileSync(STATE_MD, serializeState(channels.map(c => baselineStateRow(c.handle))), 'utf8');
|
|
142
|
+
for (const c of channels)
|
|
143
|
+
writeFileSync(profilePath(c.slug), profileBody(c.handle, c.slug), 'utf8');
|
|
211
144
|
console.log(' ' + '─'.repeat(40));
|
|
212
145
|
console.log(` Done. Wrote:`);
|
|
213
146
|
console.log(` ${ENV_PATH}`);
|
package/dist/cli.js
CHANGED
|
@@ -7,6 +7,8 @@
|
|
|
7
7
|
*
|
|
8
8
|
* yt-briefing init interactive onboarding wizard
|
|
9
9
|
* yt-briefing install-skill install the /yt skill into a coding agent
|
|
10
|
+
* yt-briefing add|remove <@handle|url> add or remove channels (also list)
|
|
11
|
+
* yt-briefing list list the channels you follow
|
|
10
12
|
* yt-briefing sweep [--reset] advance one step; prints a JSON status line
|
|
11
13
|
* yt-briefing rate --rating 0|1 [...] record a rating for the pending video
|
|
12
14
|
* yt-briefing transcribe <url|id> print a single video's transcript
|
|
@@ -14,6 +16,8 @@
|
|
|
14
16
|
import { spawnSync } from 'node:child_process';
|
|
15
17
|
import { script } from "./lib/paths.js";
|
|
16
18
|
const [cmd, ...rest] = process.argv.slice(2);
|
|
19
|
+
// Subcommand → engine script. Channel actions all route to yt-channels with the action
|
|
20
|
+
// passed through as its first arg.
|
|
17
21
|
const TARGETS = {
|
|
18
22
|
init: 'bootstrap',
|
|
19
23
|
'install-skill': 'install-skill',
|
|
@@ -21,9 +25,15 @@ const TARGETS = {
|
|
|
21
25
|
rate: 'yt-rating',
|
|
22
26
|
transcribe: 'yt-transcript',
|
|
23
27
|
};
|
|
24
|
-
|
|
25
|
-
|
|
28
|
+
const CHANNEL_ACTIONS = new Set(['add', 'remove', 'list']);
|
|
29
|
+
let argv = null;
|
|
30
|
+
if (cmd && CHANNEL_ACTIONS.has(cmd))
|
|
31
|
+
argv = [script('yt-channels'), cmd, ...rest];
|
|
32
|
+
else if (cmd && TARGETS[cmd])
|
|
33
|
+
argv = [script(TARGETS[cmd]), ...rest];
|
|
34
|
+
if (!argv) {
|
|
35
|
+
console.error('Usage: yt-briefing <init|install-skill|add|remove|list|sweep|rate|transcribe> [args...]');
|
|
26
36
|
process.exit(cmd ? 1 : 0);
|
|
27
37
|
}
|
|
28
|
-
const res = spawnSync(process.execPath,
|
|
38
|
+
const res = spawnSync(process.execPath, argv, { stdio: 'inherit' });
|
|
29
39
|
process.exit(res.status ?? 1);
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
export const BASELINE_LOOKBACK_DAYS = 60;
|
|
2
|
+
export const today = () => new Date().toISOString().slice(0, 10);
|
|
3
|
+
// Fresh channels start "baseline": updated is set this far back so the first sweep's window
|
|
4
|
+
// still contains a recent upload per type (which becomes the starting point).
|
|
5
|
+
export const lookbackDate = () => new Date(Date.now() - BASELINE_LOOKBACK_DAYS * 864e5).toISOString().slice(0, 10);
|
|
6
|
+
export const channelUrl = (handle) => `https://www.youtube.com/${handle}`;
|
|
7
|
+
/**
|
|
8
|
+
* Accept any of: `@betterstack`, `betterstack`, `https://www.youtube.com/@betterstack`
|
|
9
|
+
* (with or without a trailing `/videos` etc.) → canonical `@betterstack`. Returns null if no
|
|
10
|
+
* handle can be read (e.g. a bare `/channel/UC…` URL — ask the user for the @handle).
|
|
11
|
+
*/
|
|
12
|
+
export function normalizeHandle(input) {
|
|
13
|
+
let s = input.trim();
|
|
14
|
+
if (/youtube\.com/i.test(s) || /^https?:\/\//i.test(s)) {
|
|
15
|
+
const m = s.match(/@[\p{L}\p{N}._-]+/u); // pull the @handle out of a URL (Unicode-aware)
|
|
16
|
+
s = m ? m[0] : '';
|
|
17
|
+
}
|
|
18
|
+
s = s.replace(/^@+/, '');
|
|
19
|
+
// Keep the leading run of handle chars — letters (any script, e.g. ł), digits, . _ - —
|
|
20
|
+
// so non-ASCII handles like @DziałZagraniczny survive instead of truncating at the first ł.
|
|
21
|
+
const m = s.match(/^[\p{L}\p{N}._-]+/u);
|
|
22
|
+
s = m ? m[0] : '';
|
|
23
|
+
return s ? `@${s}` : null;
|
|
24
|
+
}
|
|
25
|
+
/**
|
|
26
|
+
* kebab-case slug that also breaks CamelCase: "BetterStack" → "better-stack". Diacritics are
|
|
27
|
+
* folded to ASCII so non-Latin handles get readable slugs: "DziałZagraniczny" → "dzial-zagraniczny"
|
|
28
|
+
* (ł has no NFD decomposition, so it's mapped explicitly; the rest go through NFD).
|
|
29
|
+
*/
|
|
30
|
+
export function slugify(s) {
|
|
31
|
+
return s
|
|
32
|
+
.replace(/@/g, '')
|
|
33
|
+
.replace(/ł/g, 'l').replace(/Ł/g, 'L')
|
|
34
|
+
.normalize('NFD').replace(/[̀-ͯ]/g, '') // strip combining diacritics (ą→a, é→e, …)
|
|
35
|
+
.replace(/([a-z0-9])([A-Z])/g, '$1-$2')
|
|
36
|
+
.replace(/[^A-Za-z0-9]+/g, '-')
|
|
37
|
+
.replace(/-+/g, '-')
|
|
38
|
+
.replace(/^-|-$/g, '')
|
|
39
|
+
.toLowerCase();
|
|
40
|
+
}
|
|
41
|
+
/** A fresh state row for a just-added channel: no cursors yet, dated back to the lookback. */
|
|
42
|
+
export const baselineStateRow = (handle) => ({
|
|
43
|
+
handle,
|
|
44
|
+
last_longform_id: null,
|
|
45
|
+
last_short_id: null,
|
|
46
|
+
last_live_id: null,
|
|
47
|
+
updated: lookbackDate(),
|
|
48
|
+
session: 0,
|
|
49
|
+
});
|
|
50
|
+
/** channels.md — a flat list of the channels you follow. */
|
|
51
|
+
export function serializeChannels(entries, date = today()) {
|
|
52
|
+
const parts = [
|
|
53
|
+
'---', 'type: yt-config', 'name: yt-channels',
|
|
54
|
+
'description: The channels you follow. Per-channel learned signal lives in channels/<slug>.md.',
|
|
55
|
+
`updated: ${date}`, '---', '', '# Channels', '',
|
|
56
|
+
];
|
|
57
|
+
for (const c of entries)
|
|
58
|
+
parts.push(`- [${c.handle}](${channelUrl(c.handle)}) → [[channels/${c.slug}]]`);
|
|
59
|
+
return parts.join('\n') + '\n';
|
|
60
|
+
}
|
|
61
|
+
/** state.md — one flat table; baseline rows use "—" pointers and the lookback date. */
|
|
62
|
+
export function serializeState(rows, date = today()) {
|
|
63
|
+
const parts = [
|
|
64
|
+
'---', 'type: yt-state', 'name: yt-state',
|
|
65
|
+
'description: Per-channel per-type cursor. The sweep reads and updates it.',
|
|
66
|
+
`updated: ${date}`, '---', '', '# State', '',
|
|
67
|
+
'| Channel | last_longform_id | last_short_id | last_live_id | updated | session |',
|
|
68
|
+
'|---|---|---|---|---|---|',
|
|
69
|
+
];
|
|
70
|
+
for (const r of rows) {
|
|
71
|
+
parts.push(`| ${r.handle} | ${r.last_longform_id ?? '—'} | ${r.last_short_id ?? '—'} | ${r.last_live_id ?? '—'} | ${r.updated ?? lookbackDate()} | ${r.session} |`);
|
|
72
|
+
}
|
|
73
|
+
return parts.join('\n') + '\n';
|
|
74
|
+
}
|
|
75
|
+
/** A fresh, empty channel profile (learned signal accumulates here as you rate). */
|
|
76
|
+
export function profileBody(handle, slug, date = today()) {
|
|
77
|
+
return [
|
|
78
|
+
'---', 'type: yt-channel-profile', `name: ${slug}-profile`,
|
|
79
|
+
`channel: ${handle}`, `channel_url: ${channelUrl(handle)}`, `updated: ${date}`,
|
|
80
|
+
'sessions_observed: 0', '---', '', `# ${handle} — Profile`, '',
|
|
81
|
+
].join('\n') + '\n';
|
|
82
|
+
}
|
package/dist/lib/llm.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Minimal OpenAI-compatible chat client — the only LLM dependency in yt-briefing.
|
|
3
3
|
*
|
|
4
|
-
* Provider-agnostic: point
|
|
4
|
+
* Provider-agnostic: point YT_BRIEFING_LLM_BASE_URL at any OpenAI-compatible endpoint —
|
|
5
5
|
* OpenRouter (default, "any model, one key"), Google Gemini's OpenAI-compat
|
|
6
6
|
* endpoint, OpenAI itself, a local Ollama, etc. The tool depends only on an API key
|
|
7
7
|
* here — not on any specific vendor and not on a coding agent being installed.
|
|
@@ -11,20 +11,20 @@
|
|
|
11
11
|
* the summaries.
|
|
12
12
|
*
|
|
13
13
|
* Env (see .env.example):
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
14
|
+
* YT_BRIEFING_LLM_BASE_URL default https://openrouter.ai/api/v1
|
|
15
|
+
* YT_BRIEFING_LLM_API_KEY required
|
|
16
|
+
* YT_BRIEFING_LLM_MODEL default google/gemini-2.5-flash
|
|
17
17
|
*/
|
|
18
18
|
const DEFAULT_BASE_URL = "https://openrouter.ai/api/v1";
|
|
19
19
|
const DEFAULT_MODEL = "google/gemini-2.5-flash";
|
|
20
20
|
export function getModel() {
|
|
21
|
-
return process.env.
|
|
21
|
+
return process.env.YT_BRIEFING_LLM_MODEL || DEFAULT_MODEL;
|
|
22
22
|
}
|
|
23
23
|
export async function chat(prompt, opts = {}) {
|
|
24
|
-
const baseUrl = (process.env.
|
|
25
|
-
const apiKey = process.env.
|
|
24
|
+
const baseUrl = (process.env.YT_BRIEFING_LLM_BASE_URL || DEFAULT_BASE_URL).replace(/\/+$/, "");
|
|
25
|
+
const apiKey = process.env.YT_BRIEFING_LLM_API_KEY;
|
|
26
26
|
if (!apiKey)
|
|
27
|
-
throw new Error("
|
|
27
|
+
throw new Error("YT_BRIEFING_LLM_API_KEY not set (see .env.example)");
|
|
28
28
|
const model = opts.model || getModel();
|
|
29
29
|
const messages = [];
|
|
30
30
|
if (opts.system)
|
package/dist/lib/paths.js
CHANGED
|
@@ -7,9 +7,9 @@
|
|
|
7
7
|
* itself) this IS the package root, so the repo layout (`.env`, `data/`) is
|
|
8
8
|
* unchanged. When the package is *consumed as a dependency* — PKG_ROOT sits
|
|
9
9
|
* inside node_modules — that would be wiped on reinstall, so BASE_DIR moves to
|
|
10
|
-
* `<your project>/.yt-briefing/` instead. Override explicitly with
|
|
10
|
+
* `<your project>/.yt-briefing/` instead. Override explicitly with YT_BRIEFING_BASE_DIR.
|
|
11
11
|
* DATA_DIR the mutable state (subscriptions, profiles, cursor, throwaway cache).
|
|
12
|
-
* Defaults to <BASE_DIR>/data; override with
|
|
12
|
+
* Defaults to <BASE_DIR>/data; override with YT_BRIEFING_DATA_DIR to keep it anywhere
|
|
13
13
|
* (e.g. a synced git folder, separate from secrets).
|
|
14
14
|
*
|
|
15
15
|
* BASE_DIR and DATA_DIR are pinned back into the environment so detached child processes —
|
|
@@ -28,15 +28,15 @@ export const PKG_ROOT = resolve(HERE, '../..'); // package root
|
|
|
28
28
|
const SCRIPT_EXT = extname(SELF) || '.ts';
|
|
29
29
|
// Consumed as a dependency? Then PKG_ROOT lives under node_modules and must not hold user state.
|
|
30
30
|
const CONSUMED = PKG_ROOT.split(sep).includes('node_modules');
|
|
31
|
-
export const BASE_DIR = process.env.
|
|
32
|
-
? resolve(process.env.
|
|
31
|
+
export const BASE_DIR = process.env.YT_BRIEFING_BASE_DIR
|
|
32
|
+
? resolve(process.env.YT_BRIEFING_BASE_DIR)
|
|
33
33
|
: CONSUMED ? join(process.cwd(), '.yt-briefing') : PKG_ROOT;
|
|
34
|
-
process.env.
|
|
34
|
+
process.env.YT_BRIEFING_BASE_DIR = BASE_DIR; // pin for children (their cwd differs)
|
|
35
35
|
export const ENV_PATH = join(BASE_DIR, '.env');
|
|
36
|
-
export const DATA_DIR = process.env.
|
|
37
|
-
? resolve(process.env.
|
|
36
|
+
export const DATA_DIR = process.env.YT_BRIEFING_DATA_DIR
|
|
37
|
+
? resolve(process.env.YT_BRIEFING_DATA_DIR)
|
|
38
38
|
: join(BASE_DIR, 'data');
|
|
39
|
-
process.env.
|
|
39
|
+
process.env.YT_BRIEFING_DATA_DIR = DATA_DIR; // pin for children (their cwd differs)
|
|
40
40
|
export const CHANNELS_MD = join(DATA_DIR, 'channels.md');
|
|
41
41
|
export const STATE_MD = join(DATA_DIR, 'state.md');
|
|
42
42
|
export const CONFIG_JSON = join(DATA_DIR, 'config.json');
|
package/dist/lib/yt-api.js
CHANGED
|
@@ -5,14 +5,14 @@
|
|
|
5
5
|
* cost ~7s to load from a cold FS cache on every fresh process. The Data API is a
|
|
6
6
|
* trivial REST surface, so direct fetch keeps cold-start near the runtime's own startup.
|
|
7
7
|
*
|
|
8
|
-
* Auth:
|
|
8
|
+
* Auth: YT_BRIEFING_YOUTUBE_API_KEY — the entrypoint loads it (dotenv.config from ENV_PATH) before
|
|
9
9
|
* calling; this module only reads process.env at call time.
|
|
10
10
|
*/
|
|
11
11
|
const API = 'https://www.googleapis.com/youtube/v3';
|
|
12
12
|
function apiKey() {
|
|
13
|
-
const k = process.env.
|
|
13
|
+
const k = process.env.YT_BRIEFING_YOUTUBE_API_KEY;
|
|
14
14
|
if (!k)
|
|
15
|
-
throw new Error('
|
|
15
|
+
throw new Error('YT_BRIEFING_YOUTUBE_API_KEY env var not set (see .env.example)');
|
|
16
16
|
return k;
|
|
17
17
|
}
|
|
18
18
|
async function get(path, params) {
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
* Filtered OUT: current and upcoming live broadcasts (no transcript yet, never consumable).
|
|
14
14
|
* --no-enrich → skip the videos.list pass; type/durationSeconds omitted; no filtering.
|
|
15
15
|
*
|
|
16
|
-
* Requires:
|
|
16
|
+
* Requires: YT_BRIEFING_YOUTUBE_API_KEY (see .env.example). Thin CLI wrapper over lib/yt-api.ts.
|
|
17
17
|
*
|
|
18
18
|
* Quota: ~1 unit per page (playlistItems.list) + 1 unit per 50 videos (videos.list).
|
|
19
19
|
*/
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Add, remove, or list the channels you follow — keeping channels.md, state.md, and the
|
|
4
|
+
* per-channel profile in sync. Driven by the CLI: `yt-briefing add|remove|list ...`.
|
|
5
|
+
*
|
|
6
|
+
* yt-briefing add @foo https://youtube.com/@bar # add one or more (handle or URL)
|
|
7
|
+
* yt-briefing remove @foo # remove (also deletes its profile)
|
|
8
|
+
* yt-briefing list # show the current list
|
|
9
|
+
*
|
|
10
|
+
* Adding a channel starts it at "baseline" (next sweep surfaces the latest upload per type);
|
|
11
|
+
* removing one drops its row and learned profile. Existing channels' cursors are untouched.
|
|
12
|
+
*/
|
|
13
|
+
import { existsSync, readFileSync, writeFileSync, rmSync } from 'node:fs';
|
|
14
|
+
import { CHANNELS_MD, STATE_MD, profilePath } from "./lib/paths.js";
|
|
15
|
+
import { parseChannels, parseState } from "./lib/yt-lib.js";
|
|
16
|
+
import { normalizeHandle, slugify, serializeChannels, serializeState, profileBody, baselineStateRow, } from "./lib/channels.js";
|
|
17
|
+
const [action, ...args] = process.argv.slice(2);
|
|
18
|
+
if (!existsSync(CHANNELS_MD) || !existsSync(STATE_MD)) {
|
|
19
|
+
console.error('No channel list yet — run onboarding first: yt-briefing init');
|
|
20
|
+
process.exit(1);
|
|
21
|
+
}
|
|
22
|
+
let entries = parseChannels(readFileSync(CHANNELS_MD, 'utf8'));
|
|
23
|
+
if (action === 'list') {
|
|
24
|
+
for (const e of entries)
|
|
25
|
+
console.log(` ${e.handle}`);
|
|
26
|
+
console.log(`\n ${entries.length} channel(s).`);
|
|
27
|
+
process.exit(0);
|
|
28
|
+
}
|
|
29
|
+
if ((action !== 'add' && action !== 'remove') || args.length === 0) {
|
|
30
|
+
console.error('Usage: yt-briefing add|remove <@handle | url> [...] · yt-briefing list');
|
|
31
|
+
process.exit(1);
|
|
32
|
+
}
|
|
33
|
+
let rows = parseState(readFileSync(STATE_MD, 'utf8'));
|
|
34
|
+
let changed = 0;
|
|
35
|
+
for (const raw of args) {
|
|
36
|
+
const handle = normalizeHandle(raw);
|
|
37
|
+
if (!handle) {
|
|
38
|
+
console.log(` ! couldn't read a handle from "${raw}" — use @name or the channel URL`);
|
|
39
|
+
continue;
|
|
40
|
+
}
|
|
41
|
+
const slug = slugify(handle);
|
|
42
|
+
const exists = entries.some(e => e.slug === slug);
|
|
43
|
+
if (action === 'add') {
|
|
44
|
+
if (exists) {
|
|
45
|
+
console.log(` • ${handle} already on the list — skipping`);
|
|
46
|
+
continue;
|
|
47
|
+
}
|
|
48
|
+
entries.push({ handle, slug });
|
|
49
|
+
rows.push(baselineStateRow(handle));
|
|
50
|
+
writeFileSync(profilePath(slug), profileBody(handle, slug), 'utf8');
|
|
51
|
+
console.log(` + ${handle}`);
|
|
52
|
+
changed++;
|
|
53
|
+
}
|
|
54
|
+
else {
|
|
55
|
+
if (!exists) {
|
|
56
|
+
console.log(` • ${handle} not on the list — skipping`);
|
|
57
|
+
continue;
|
|
58
|
+
}
|
|
59
|
+
const entry = entries.find(e => e.slug === slug);
|
|
60
|
+
entries = entries.filter(e => e.slug !== slug);
|
|
61
|
+
rows = rows.filter(r => r.handle !== entry.handle);
|
|
62
|
+
try {
|
|
63
|
+
rmSync(profilePath(slug));
|
|
64
|
+
}
|
|
65
|
+
catch { /* profile already gone */ }
|
|
66
|
+
console.log(` − ${handle} (profile deleted)`);
|
|
67
|
+
changed++;
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
if (changed) {
|
|
71
|
+
writeFileSync(CHANNELS_MD, serializeChannels(entries), 'utf8');
|
|
72
|
+
writeFileSync(STATE_MD, serializeState(rows), 'utf8');
|
|
73
|
+
}
|
|
74
|
+
console.log(`\n ${entries.length} channel(s) now. The next sweep picks up the change.`);
|
package/dist/yt-sweep.js
CHANGED
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
* {"status":"rate_limited"}
|
|
18
18
|
*
|
|
19
19
|
* The engine ONLY writes files under DATA_DIR — it never runs git or any VCS. If you
|
|
20
|
-
* want your briefing state versioned, commit DATA_DIR yourself (or point
|
|
20
|
+
* want your briefing state versioned, commit DATA_DIR yourself (or point YT_BRIEFING_DATA_DIR at
|
|
21
21
|
* a synced folder). Keeping persistence out of the engine is deliberate: it stays a pure
|
|
22
22
|
* data tool with zero host coupling.
|
|
23
23
|
*
|
|
@@ -71,9 +71,9 @@ const prefetchTarget = pfIdx !== -1 ? argv[pfIdx + 1] : null;
|
|
|
71
71
|
const today = new Date().toISOString().slice(0, 10);
|
|
72
72
|
const LANG = outputLang();
|
|
73
73
|
// Diagnostics sink. Default: silent (stdout stays a pure JSON line — the caller never
|
|
74
|
-
// has to redirect anything, so no /tmp). With
|
|
74
|
+
// has to redirect anything, so no /tmp). With YT_BRIEFING_DEBUG set, timing + child stderr append
|
|
75
75
|
// to <DATA_DIR>/.cache/sweep.log (gitignored) — never an OS temp dir.
|
|
76
|
-
const DEBUG = !!process.env.
|
|
76
|
+
const DEBUG = !!process.env.YT_BRIEFING_DEBUG;
|
|
77
77
|
const T0 = Date.now();
|
|
78
78
|
const log = (msg) => { if (DEBUG)
|
|
79
79
|
appendFileSync(LOG_FILE, `⏱ ${msg} (+${Date.now() - T0}ms)\n`); };
|
|
@@ -85,7 +85,7 @@ function run(cmd) {
|
|
|
85
85
|
let stdout = '';
|
|
86
86
|
p.stdout.on('data', d => { stdout += d.toString(); });
|
|
87
87
|
// Child stderr → the gitignored debug log only (never parent stderr / stdout), so a
|
|
88
|
-
// bare invocation emits nothing but the JSON line. Silent unless
|
|
88
|
+
// bare invocation emits nothing but the JSON line. Silent unless YT_BRIEFING_DEBUG.
|
|
89
89
|
if (DEBUG)
|
|
90
90
|
p.stderr.on('data', d => appendFileSync(LOG_FILE, d.toString()));
|
|
91
91
|
else
|
package/dist/yt-transcript.js
CHANGED
|
@@ -12,13 +12,13 @@
|
|
|
12
12
|
* Uses yt-dlp for subtitle extraction (handles all YouTube caption formats).
|
|
13
13
|
*
|
|
14
14
|
* yt-dlp lookup (so a project-local install needs no global PATH pollution):
|
|
15
|
-
* 1. $
|
|
15
|
+
* 1. $YT_BRIEFING_DLP_PATH if set
|
|
16
16
|
* 2. <package>/bin/yt-dlp (yt-dlp.exe on Windows) if present
|
|
17
17
|
* 3. `yt-dlp` on PATH (yt-dlp.exe on Windows)
|
|
18
18
|
* See README.md → Requirements for per-OS install methods.
|
|
19
19
|
*
|
|
20
|
-
*
|
|
21
|
-
* YouTube blocks. Example:
|
|
20
|
+
* YT_BRIEFING_PROXY env (optional): HTTP proxy URL — required on datacenter/VPS IPs which
|
|
21
|
+
* YouTube blocks. Example: YT_BRIEFING_PROXY=http://127.0.0.1:1080 (Cloudflare WARP via
|
|
22
22
|
* Docker). See docs/warp-proxy.md.
|
|
23
23
|
*/
|
|
24
24
|
import { spawnSync } from 'node:child_process';
|
|
@@ -27,13 +27,13 @@ import { join, dirname } from 'node:path';
|
|
|
27
27
|
import { fileURLToPath } from 'node:url';
|
|
28
28
|
import dotenv from 'dotenv';
|
|
29
29
|
import { CACHE_DIR, ENV_PATH } from "./lib/paths.js";
|
|
30
|
-
// Load .env so
|
|
30
|
+
// Load .env so YT_BRIEFING_PROXY is set when run standalone under Node (Bun auto-loads it; Node doesn't).
|
|
31
31
|
// When spawned by yt-sweep, the parent already loaded it and the child inherits the env.
|
|
32
32
|
dotenv.config({ path: ENV_PATH });
|
|
33
33
|
/** Resolve the yt-dlp binary: explicit env → project-local ./bin → PATH (Windows-aware). */
|
|
34
34
|
function resolveYtDlp() {
|
|
35
|
-
if (process.env.
|
|
36
|
-
return process.env.
|
|
35
|
+
if (process.env.YT_BRIEFING_DLP_PATH)
|
|
36
|
+
return process.env.YT_BRIEFING_DLP_PATH;
|
|
37
37
|
const exe = process.platform === 'win32' ? 'yt-dlp.exe' : 'yt-dlp';
|
|
38
38
|
const local = join(dirname(fileURLToPath(import.meta.url)), '..', 'bin', exe);
|
|
39
39
|
return existsSync(local) ? local : exe; // bare name → looked up on PATH
|
|
@@ -61,7 +61,7 @@ function extractVideoId(input) {
|
|
|
61
61
|
process.exit(3);
|
|
62
62
|
}
|
|
63
63
|
const videoId = extractVideoId(rawInput);
|
|
64
|
-
const proxyUrl = process.env.
|
|
64
|
+
const proxyUrl = process.env.YT_BRIEFING_PROXY;
|
|
65
65
|
function vttToText(vtt) {
|
|
66
66
|
let prev = '';
|
|
67
67
|
const parts = [];
|
|
@@ -106,7 +106,7 @@ catch { } };
|
|
|
106
106
|
if (result.error) {
|
|
107
107
|
cleanup();
|
|
108
108
|
const msg = result.error.code === 'ENOENT'
|
|
109
|
-
? `yt-dlp not found (looked for "${YT_DLP}") — install it or set
|
|
109
|
+
? `yt-dlp not found (looked for "${YT_DLP}") — install it or set YT_BRIEFING_DLP_PATH (see README.md → Requirements)`
|
|
110
110
|
: `yt-dlp spawn error: ${result.error.message}`;
|
|
111
111
|
console.error(msg);
|
|
112
112
|
process.exit(3);
|
|
@@ -23,11 +23,11 @@ your project's own git: push from machine A, pull on machine B. Keep `.yt-briefi
|
|
|
23
23
|
git-ignored — secrets stay per machine.
|
|
24
24
|
|
|
25
25
|
Want briefing state in **its own** repo instead (e.g. a laptop and a headless VPS that share
|
|
26
|
-
nothing else)? Point `
|
|
26
|
+
nothing else)? Point `YT_BRIEFING_DATA_DIR` at a folder you control and version that:
|
|
27
27
|
|
|
28
28
|
```bash
|
|
29
29
|
# .env (per machine — secrets never sync)
|
|
30
|
-
|
|
30
|
+
YT_BRIEFING_DATA_DIR=/home/you/yt-briefing-data
|
|
31
31
|
```
|
|
32
32
|
|
|
33
33
|
```bash
|
|
@@ -38,7 +38,7 @@ npx yt-briefing init # onboard into this folder (or move exi
|
|
|
38
38
|
git add -A && git commit -m "initial" && git push -u origin main
|
|
39
39
|
```
|
|
40
40
|
|
|
41
|
-
On the second machine: clone that repo, set the same `
|
|
41
|
+
On the second machine: clone that repo, set the same `YT_BRIEFING_DATA_DIR`, drop in your `.env`.
|
|
42
42
|
|
|
43
43
|
---
|
|
44
44
|
|
|
@@ -71,7 +71,7 @@ Save as `yt-sync.sh` (anywhere), `chmod +x`:
|
|
|
71
71
|
#!/usr/bin/env bash
|
|
72
72
|
# Persist yt-briefing state to git, sync-safe across machines. Best-effort, never blocks.
|
|
73
73
|
set -uo pipefail
|
|
74
|
-
DATA="${
|
|
74
|
+
DATA="${YT_BRIEFING_DATA_DIR:-$PWD/.yt-briefing/data}" # the folder you version (default: in-project)
|
|
75
75
|
cd "$DATA" || exit 0
|
|
76
76
|
|
|
77
77
|
git add channels.md state.md channels/ config.json 2>/dev/null || exit 0
|
|
@@ -113,7 +113,7 @@ yt-briefing rate --rating 0 && /path/to/yt-sync.sh
|
|
|
113
113
|
|
|
114
114
|
> Optional but recommended: also `git pull --rebase` **before** the first sweep of a session
|
|
115
115
|
> (so a machine starts on the latest cursor), e.g. a `PreToolUse` hook matching
|
|
116
|
-
> `*yt-sweep*--reset*`, or just `cd "$
|
|
116
|
+
> `*yt-sweep*--reset*`, or just `cd "$YT_BRIEFING_DATA_DIR" && git pull --rebase` before you start.
|
|
117
117
|
|
|
118
118
|
---
|
|
119
119
|
|
package/docs/warp-proxy.md
CHANGED
|
@@ -38,10 +38,10 @@ either; we use HTTP.
|
|
|
38
38
|
|
|
39
39
|
```bash
|
|
40
40
|
# .env
|
|
41
|
-
|
|
41
|
+
YT_BRIEFING_PROXY=http://127.0.0.1:1080
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
yt-briefing reads `
|
|
44
|
+
yt-briefing reads `YT_BRIEFING_PROXY` and routes all yt-dlp traffic through it. No env var → direct
|
|
45
45
|
fetch (the residential default).
|
|
46
46
|
|
|
47
47
|
### Health check
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "yt-briefing",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "A self-learning YouTube briefing engine: it sweeps the channels you follow, filters noise in two stages (title, then transcript), summarizes the rest in your language, and adapts to your ratings — one video at a time.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|