dsh-baleen 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +50 -0
- package/CONTRIBUTING.md +77 -0
- package/LICENSE +201 -0
- package/README.md +95 -3
- package/SECURITY.md +42 -0
- package/client.js +2191 -0
- package/cordis.patch.yml +9 -0
- package/index.js +685 -0
- package/lib/adapters/bili-wbi.js +100 -0
- package/lib/adapters/bili.js +389 -0
- package/lib/adapters/index.js +15 -0
- package/lib/adapters/rss.js +58 -0
- package/lib/adapters/yt-captions.js +53 -0
- package/lib/adapters/yt.js +460 -0
- package/lib/dedup.js +63 -0
- package/lib/diag.js +49 -0
- package/lib/feed.js +198 -0
- package/lib/jobs.js +64 -0
- package/lib/normalize.js +130 -0
- package/lib/paths.js +29 -0
- package/lib/pool.js +88 -0
- package/lib/preference.js +286 -0
- package/lib/prefilter.js +70 -0
- package/lib/runner.js +144 -0
- package/lib/scheduler.js +133 -0
- package/lib/sql/items.sql +94 -0
- package/lib/store.js +514 -0
- package/lib/tools.js +408 -0
- package/package.json +61 -4
- package/src/client/_footer.txt +490 -0
- package/src/client/_header.txt +30 -0
- package/src/client/css.js +156 -0
- package/src/client/format.js +38 -0
- package/src/client/match.js +20 -0
- package/src/client/view-api.js +136 -0
- package/src/client/view-state.js +81 -0
- package/src/client/views/article.js +84 -0
- package/src/client/views/collections.js +128 -0
- package/src/client/views/digest.js +42 -0
- package/src/client/views/drawer.js +216 -0
- package/src/client/views/library.js +73 -0
- package/src/client/views/settings.js +459 -0
- package/src/client/views/shell.js +40 -0
- package/src/client/views/today.js +105 -0
- package/src/client/views/video.js +75 -0
- package/src/client/vnode.js +8 -0
- package/tools/build-client.mjs +63 -0
- package/tools/client-loader.mjs +157 -0
- package/tools/live-ingest.mjs +68 -0
- package/tools/preview.mjs +195 -0
package/AGENTS.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Baleen — 给协作 agent 的项目说明
|
|
2
|
+
|
|
3
|
+
Baleen 是一个 DSH(DeepSeek Harness 桌面版)插件:定时把信息源灌进本地库,用**读正文/字幕**而不是读标题的方式筛掉劣质内容,每轮自动出一份增量汇报。名字来自鲸须(滤食结构)。
|
|
4
|
+
|
|
5
|
+
设计文档:`docs/superpowers/specs/2026-10-23-baleen-design.md`
|
|
6
|
+
当前实现计划:`docs/superpowers/plans/2026-10-24-baleen-m2-panel.md`(M1 已完成,存档在 `2026-10-23-baleen-m0m1-store-and-rss.md`)
|
|
7
|
+
实测记录:`docs/verification.md`
|
|
8
|
+
|
|
9
|
+
## 硬约束(每个改动都要守)
|
|
10
|
+
|
|
11
|
+
1. **零新增 npm 依赖**为首选。目前只允许 `yaml`(纯 JS、无 install 脚本)。要加别的先停下来问。
|
|
12
|
+
2. **绝不注册平台写操作**。没有评论、私信、点赞、关注、发布。判断那一轮只给只读工具。这是产品红线,不是"以后再说"。
|
|
13
|
+
3. **失败不推进游标**。抓取失败的源必须保留旧游标,让下一轮能补缺口。`test/runner.test.js` 里有一条测试直接盯这件事,别让它变成摆设。
|
|
14
|
+
4. **重试上限 1 次**。超限就记 `failed` 继续下一个源。禁止为了让某个源成功而写临时调试脚本——那是烧钱的思路。
|
|
15
|
+
5. **`body_state` 六值固定**:`full` `summary_only` `subtitle` `transcribed` `missing` `burned_in`。`missing`(没有语音)与 `burned_in`(字在画面里读不到)是两种记录,合并它们等于对用户谎报覆盖面。
|
|
16
|
+
6. **三档词固定**:`值得看` `可以看` `不值当`。不许自创档位,不许写分数。
|
|
17
|
+
7. **日志与诊断脱敏**:不含用户名、不含家目录绝对路径、不含 token/cookie。测试里也不要出现真名,用 `os.homedir()` 做比较。
|
|
18
|
+
8. **两个粗调按钮只写 `tuning`**,永远不改 `themes` / `follow` / `sources`。
|
|
19
|
+
9. 测试只用 `node --test`。不引入 jest / vitest。**注意 `node --test test/` 在 Windows 上会把目录当模块解析而报错**,一律写 `node --test "test/*.test.js"`(`npm test` 已经是这个形式)。
|
|
20
|
+
10. 提交信息用中文;一个任务一次提交。
|
|
21
|
+
11. **收藏只写本机 sqlite**:`collect` / `folder` 三条路由只走面板 → 宿主,**绝不注册成 agent 工具**,也绝不碰平台的收藏/点赞接口(B 站恰好有个同名真动作)。默认夹可改名不可删(路由与 store 两层都拒);夹名超长是**拒绝**而不是截断;已收藏的条目豁免 `runner.js` 的过期再归档,但移出后下一轮照旧清理。
|
|
22
|
+
12. **工具面恰好六个、save_verdicts 是唯一写工具**:`baleen_query / read_item / candidates / run_status / subtitle / save_verdicts`,名字与 spec §10.1 逐字对应(`subtitle` 是 M4a 的视频字幕现取,只读,B 站通道);`save_verdicts` 只写本插件的 items/digests 两表,tier 参数是三档枚举锁死;**平台写动词(评论/私信/点赞/关注/发布/收藏到平台)永远不出现在工具面**——`test/tools.test.js` 的红线测试是事故报警器,红了先停下来想。
|
|
23
|
+
|
|
24
|
+
## 本机的坑(已实证,别再撞)
|
|
25
|
+
|
|
26
|
+
- 工作副本是 **CRLF**。多行锚点的脚本要匹配 `\r?\n`。
|
|
27
|
+
- 编辑工具可能**静默删掉文件内容里的 Windows 绝对路径**。写含路径的文件后要么用脚本拼接,要么读回验证。
|
|
28
|
+
- DSH 客户端 `ctx` 是 **Proxy**:服务名没写进 `package.json` 的 `dsh.client.inject`,`ctx.get()` 和属性访问都会抛。`try { ctx.get(x) } catch {}` 之后再去裸读 `ctx.x` 是炸启动的写法,不是安全降级。
|
|
29
|
+
- 侧栏入口 `sidebar.panellist` 注册的 `id` 必须与 `main` 面板的 key **完全同名**,否则宿主 `layout.selectPanel` 抛。
|
|
30
|
+
- 主题只消费宿主 token(`--dsw-alias-*`),**明暗双主题都是验收项**,不写死颜色。本机实测存在的名字(清单见 `test/css.test.js` 的 `HOST_TOKENS`,来源是已跑通的 dsh-study-desk):`bg-base`、`bg-layer-1/2/3`、`border-l1..l4`、`label-primary/secondary/tertiary/dimmed`、`brand-primary`、`interactive-bg-hover/active`、`state-error-primary`、`state-success-primary`。**`fill-tertiary`、`border-primary`、`link` 这些不存在**——写错名字不是风格问题,是宿主取不到值。也不给 `var()` 加 `light-dark()` 兜底:那等于把硬编码颜色请回来,脱离宿主的样子由 `tools/preview.mjs` 的替身 token 表负责。
|
|
31
|
+
- 面板布局别留两侧大空白;网格用 `auto-fill minmax(260px, 1fr)`。
|
|
32
|
+
- **link 挂载不等于热加载,宿主会重写 cordis.patch.yml**:改完插件代码必须重启 DSH 才装配;宿主运行中重写 patch 会把 link 装配条目丢掉(package.json 依赖还在,patch 条目没了,重启后插件整个不加载)——重启前先 `grep baleen cordis.patch.yml` 确认,没了就重新 `dsh plugin --profile desktop add link:D:/dsh-baleen`。
|
|
33
|
+
- **当前宿主的工具/注入形态是 `ctx.inject`**(2026-10 更新后):工具 `ctx.inject(['tools'], (toolsCtx) => toolsCtx.tools.register(t))`、注入 `ctx.inject(['systemPrompt'], (promptCtx) => promptCtx.systemPrompt.context({ name, order, text: 每轮渲染回调 }))`——study-desk 的 `ctx.on('system-prompt/assemble')` 是旧版事件,新宿主没有;`ctx.get('tools')` 在 Proxy 上抛错会让 apply 整体静默中断(面板正常、无日志、无工具),工具注册段要包 try 并写 diagnostics 日志(tool_registered 探针)。
|
|
34
|
+
- 打包 node 的入口是 `resources/runtime/bin/node.cmd`,需要 `DSH_DESKTOP_NODE_EXECUTABLE` 指向 `DeepSeek Harness.exe`(细节见 `docs/verification.md`)。
|
|
35
|
+
- **宿主 API 实测形态**(抄自本机 dsh-study-desk,别自己发明):取服务用 `ctx.get('webServer') ?? ctx.webServer`;注册路由用 `webServer.register({ kind:'prefix', path, handler })` 且必须包在 `ctx.effect(fn, label)` 里;注册服务用 `ctx.provide`(`ctx.set` 对未 provide 的服务会抛)。
|
|
36
|
+
- **桌面端 UI 跑在自定义协议上,只有 `/api/*` 会被转发到宿主**——路由前缀固定 `/api/dsh-baleen`,改成别的名字面板就读不到数据。
|
|
37
|
+
- 定时器一律走 `ctx.effect(() => { const h = start(); return () => stop(h) })`;测试里通过 `apply(ctx, { setInterval, clearInterval })` 注入替身,否则真 interval 会把 `node --test` 吊住不退出。
|
|
38
|
+
- `node:sqlite` 在本机可用(独立 node 24.15 与 DSH 打包 24.21 都实测通过),不要引第三方 DB。
|
|
39
|
+
- 动 DSH profile 之前:**备份 `cordis.patch.yml` 和 profile `package.json`**,一次只切换一个插件,**永远不要点启动失败弹窗里"禁用第三方插件、备份 profile patch 并重启"那个按钮**(它会重新序列化并清掉文件)。
|
|
40
|
+
- 装完插件先用 `dsh --profile desktop --dump-config` 检查装配,别用重启试错。
|
|
41
|
+
- **`client.js` 是生成产物,别手改**:源在 `src/client/`,改完跑 `node tools/build-client.mjs`(`npm run build:check` 只验证新旧)。`test/build.test.js` 有两条守卫:产物逐字节比对、以及"`src/client` 下每个 `.js` 都在 `PARTS` 里"——新加视图忘了登记 `PARTS`,面板不会有任何变化,也不会报错。
|
|
42
|
+
- 生成方式是把多个源文件**内联进同一个 factory 作用域**(只剥 `import`/`export` 行),所以**两个源文件里的同名 `const` 会直接 `SyntaxError`**。节点工厂统一用 `src/client/vnode.js` 的 `vnode()`,别在视图里再造同名局部函数。
|
|
43
|
+
- 视图返回的是纯对象 `{type, props, children}`,不是 React 元素;装配层用 `toElement()` 桥给 `createElement`。测试桩的 `flattenText` 会**调用函数组件**(桩不 reconcile),所以面板断言看得见内容——但也因此看不见真 React 的键/形状问题,那一层只能在预览门和真机上看。
|
|
44
|
+
|
|
45
|
+
## 界面协作约定
|
|
46
|
+
|
|
47
|
+
- 新建 UI 元素前先看有没有现成参照(`D:/dsh-study-desk` 是本机已验证的写法)。
|
|
48
|
+
- 面板属于 v1,不许在规划时把前端收窄掉。
|
|
49
|
+
- 定稿前给假数据 HTML 预览,明暗两版都能看。
|
|
50
|
+
- 拍板类问题给封闭选项,别让人打字。
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# 参与开发
|
|
2
|
+
|
|
3
|
+
欢迎提 issue 和 PR。这个仓库有几条硬约束,不是风格偏好——它们对应真实踩过的坑,破了会静默坏掉。
|
|
4
|
+
|
|
5
|
+
## 先跑起来
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install
|
|
9
|
+
npm test # node --test,578 条,不需要网络
|
|
10
|
+
npm run build:check # 验证 client.js 与 src/client/ 同步
|
|
11
|
+
node tools/preview.mjs # 出 out/preview-*.html,明暗两版,浏览器直接看
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
不需要装 DSH 就能改前端——预览调的就是 `src/client/views/` 里的真渲染函数。
|
|
15
|
+
|
|
16
|
+
## 硬约束
|
|
17
|
+
|
|
18
|
+
1. **零新增 npm 依赖**是首选。目前只允许 `yaml`(纯 JS、无 install 脚本)。要加别的,先在 issue 里说清楚为什么标准库做不了。
|
|
19
|
+
2. **绝不注册平台写操作**。没有评论、私信、点赞、关注、发布、收藏到平台。判断轮只给只读工具。这是产品红线,PR 碰到这条直接关。
|
|
20
|
+
3. **失败不推进游标**。抓取失败的源必须保留旧游标,下一轮补缺口。`test/runner.test.js` 有测试盯这件事,别把它改成摆设。
|
|
21
|
+
4. **重试上限 1 次**。超限记 `failed` 继续下一个源。
|
|
22
|
+
5. **`body_state` 六值固定**:`full` `summary_only` `subtitle` `transcribed` `missing` `burned_in`。`missing`(没有语音)与 `burned_in`(字烧在画面里)是两种记录,合并等于对用户谎报覆盖面。
|
|
23
|
+
6. **三档词固定**:`值得看` `可以看` `不值当`。不许自创档位,不许写分数。
|
|
24
|
+
7. **日志与诊断脱敏**:不含用户名、不含家目录绝对路径、不含 token/cookie。测试里也别写真名。
|
|
25
|
+
8. **工具面恰好六个**,`save_verdicts` 是唯一写工具。名字与设计文档 §10.1 逐字对应。
|
|
26
|
+
|
|
27
|
+
## 本机实测过的坑(别再撞)
|
|
28
|
+
|
|
29
|
+
### 文件与编码
|
|
30
|
+
|
|
31
|
+
- 工作副本是 **CRLF**。多行锚点的脚本要匹配 `\r?\n`。
|
|
32
|
+
- 编辑工具可能**静默删掉文件内容里的 Windows 绝对路径**。写含路径的文件后要读回验证。
|
|
33
|
+
- `client.js` 是**生成产物,别手改**。源在 `src/client/`,改完跑 `node tools/build-client.mjs`;`build:check` 只做新旧比对。
|
|
34
|
+
- 生成方式是把多个源文件**内联进同一个 factory 作用域**(只剥 `import`/`export` 行),所以**两个源文件里的同名 `const` 会直接 SyntaxError**。节点工厂统一用 `src/client/vnode.js` 的 `vnode()`。
|
|
35
|
+
- 新加视图忘了登记 `tools/build-client.mjs` 的 `PARTS`,面板不会有任何变化,也不会报错——`test/build.test.js` 有条守卫盯这个。
|
|
36
|
+
|
|
37
|
+
### 宿主装配
|
|
38
|
+
|
|
39
|
+
- 宿主 API 实测形态:取服务用 `ctx.get('webServer') ?? ctx.webServer`;注册路由用 `webServer.register({ kind:'prefix', path, handler })` 且必须包在 `ctx.effect(fn, label)` 里;注册服务用 `ctx.provide`(`ctx.set` 对未 provide 的服务会抛)。
|
|
40
|
+
- DSH 客户端 `ctx` 是 **Proxy**:服务名没写进 `package.json` 的 `dsh.client.inject`,`ctx.get()` 和属性访问都会抛。`try { ctx.get(x) } catch {}` 之后再去裸读 `ctx.x` 是炸启动的写法。
|
|
41
|
+
- 工具/注入走 `ctx.inject`:工具 `ctx.inject(['tools'], (c) => c.tools.register(t))`、注入 `ctx.inject(['systemPrompt'], (c) => c.systemPrompt.context({ name, order, text }))`。旧的 `ctx.on('system-prompt/assemble')` 在新宿主上不存在。
|
|
42
|
+
- 定时器一律走 `ctx.effect(() => { const h = start(); return () => stop(h) })`;测试里通过 `apply(ctx, { setInterval, clearInterval })` 注入替身,否则真 interval 会把 `node --test` 吊住不退出。
|
|
43
|
+
- 改完插件代码**必须重启 DSH** 才装配。宿主运行中重写 `cordis.patch.yml` 会把 link 装配条目丢掉,重启前先 `grep baleen cordis.patch.yml` 确认。
|
|
44
|
+
- 动 DSH profile 之前备份 `cordis.patch.yml` 和 profile `package.json`。**永远不要点启动失败弹窗里「禁用第三方插件、备份 profile patch 并重启」那个按钮**(它会重新序列化并清掉文件)。
|
|
45
|
+
|
|
46
|
+
### 样式
|
|
47
|
+
|
|
48
|
+
- 主题只消费宿主 token(`--dsw-alias-*`),**明暗双主题都是验收项**,不写死颜色。本机实测存在的名字见 `test/css.test.js` 的 `HOST_TOKENS`。`fill-tertiary`、`border-primary`、`link` **不存在**——写错不是风格问题,是宿主取不到值。也不给 `var()` 加 `light-dark()` 兜底。
|
|
49
|
+
|
|
50
|
+
### 数据库
|
|
51
|
+
|
|
52
|
+
- `node:sqlite` 在本机可用(node 24.x),不要引第三方 DB。
|
|
53
|
+
- `digests.run_id` 是 `NOT NULL`:测试里手工插 digest 或调 `saveVerdicts` 前必须先 `createRun(db, 'schedule', () => N)`。
|
|
54
|
+
- 加列走 `lib/store.js` 的 `migrate(db)`:先 `PRAGMA table_info` 判断,再 `ALTER TABLE ... ADD COLUMN`,让老库能原地升级。
|
|
55
|
+
|
|
56
|
+
## 测试
|
|
57
|
+
|
|
58
|
+
只用 `node --test`,不引入 jest / vitest。
|
|
59
|
+
|
|
60
|
+
**注意 `node --test test/` 在 Windows 上会把目录当模块解析而报错**,一律写 `node --test "test/*.test.js"`(`npm test` 已经是这个形式)。
|
|
61
|
+
|
|
62
|
+
改完代码至少跑 `npm test` + `npm run build:check`,两个都绿再提 PR。
|
|
63
|
+
|
|
64
|
+
## 提交规范
|
|
65
|
+
|
|
66
|
+
- **提交信息用中文**,一个任务一次提交。
|
|
67
|
+
- **不留 AI 署名**:不写 `Co-authored-by: <AI>`、不写 "Generated with ..." 之类脚注。
|
|
68
|
+
- PR 标题写清「改了什么 + 为什么」,正文贴测试输出。
|
|
69
|
+
- 一次 PR 只做一件事。顺手重构和功能混在一起会被要求拆开。
|
|
70
|
+
|
|
71
|
+
## 分支
|
|
72
|
+
|
|
73
|
+
`main` 是受保护分支:直推被拒,改动走 PR,且必须等 CI 绿。
|
|
74
|
+
|
|
75
|
+
## 许可
|
|
76
|
+
|
|
77
|
+
提交即表示你同意以 [Apache License 2.0](LICENSE) 授权你的贡献。
|
package/LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright [yyyy] [name of copyright owner]
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
package/README.md
CHANGED
|
@@ -1,3 +1,95 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
|
|
1
|
+
# Baleen(`dsh-baleen`)
|
|
2
|
+
|
|
3
|
+
> 每天替你过一遍信息源:灌进一整轮内容,**读正文和字幕**(不是读标题)留下值得看的,把水和标题党排掉,然后把这一轮的判断写成图文汇报放在你面前。
|
|
4
|
+
>
|
|
5
|
+
> 名字来自鲸须——须鲸的滤食结构。这个产品做的同一件事。
|
|
6
|
+
|
|
7
|
+
**当前状态:M0–M5 全部完成并已装载进 DSH 真机验收。** 摄取(RSS / B 站 / YouTube 三通道,读正文与字幕)→ 本地预筛 → 面板六格导航与收藏页 → 判断轮(五个只读工具 + 唯一写工具 save_verdicts,三档枚举锁死)→ 每轮增量汇报 → 设置页四页签,全链路真机跑过。小红书通道未做(需先做代码审查)。
|
|
8
|
+
|
|
9
|
+
## 看面板长什么样(不装 DSH、不联网)
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npm install
|
|
13
|
+
node tools/preview.mjs # 出 out/preview-*.html,每页明暗两块
|
|
14
|
+
open out/preview-today.html # 浏览器直接打开即可
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
预览调的就是 `src/client/views/` 里那批真渲染函数,所以它不是另画一套。两处对不上真机:真 React 的节点形状、以及宿主真实调色板(预览里的 `--dsw-alias-*` 是按观感近似的**替身值**,页面顶部有标注)。
|
|
18
|
+
|
|
19
|
+
## 现在就能试
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npm install # 唯一依赖:yaml(纯 JS,无 install 脚本)
|
|
23
|
+
npm test # 578 个测试,node --test,不需要网络
|
|
24
|
+
cp preference.example.yaml ~/baleen-trial/preference.yaml # 先自己建目录
|
|
25
|
+
BALEEN_HOME=~/baleen-trial node tools/live-ingest.mjs --reset # 真抓一轮公开 feed
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
`tools/live-ingest.mjs` 不经 DSH、用假宿主上下文直接跑一轮,`BALEEN_HOME` 指到临时目录就不会碰你真实的 `~/.dsh/baleen`。
|
|
29
|
+
|
|
30
|
+
## 设计原则(改代码前先读这几条)
|
|
31
|
+
|
|
32
|
+
- **只读世界**:不注册任何平台写操作。没有评论、私信、点赞、关注、发布——不是"暂时没做",是产品红线。
|
|
33
|
+
- **理由必须可信**:每条候选都带"为什么给你"。关键词匹配因此对英文走词边界、对中文走子串——子串匹配曾经把 "building" 判成命中 `UI`,那种假理由比没有理由更糟。**你写关键词时大小写是语义**:`React` / `CSS` / `MCP` 这类含大写的按大小写敏感匹配(散文里的 "how we react to" 不算命中框架),全小写的 `agent` 则不敏感。
|
|
34
|
+
- **读不到要可见**:`body_state` 分 `full / summary_only / subtitle / transcribed / missing / burned_in` 六档;`missing`(没有语音)和 `burned_in`(字烧在画面上)是两回事,汇报里"因无字幕跳过 N 条"必须写出来。
|
|
35
|
+
- **失败不推进游标**:抓取失败的源保留旧游标,下一轮自动补缺口。内容只会推迟,不会永久丢。
|
|
36
|
+
- **首轮收存货**:某个源还没有游标时不看发布日期——低频作者的 feed 里全是几个月的旧文,而刚订阅的你没读过,那正是首轮最值钱的部分。之后才用 `max_age_days`,且**可以按源覆盖**(慢更源 365 天,高频源 14 天)。
|
|
37
|
+
- **成本三道闸**:本地预筛在前 → `max_per_run` → `max_per_day`;工具失败两次就记跳过,不许反复重跑。
|
|
38
|
+
- **默认按偏好过滤**:`require_theme_hit: true` 时,没命中任何主题且不是关注 feed 的内容不进候选。关掉它就退回"启用一个源=它全部内容都进候选"。
|
|
39
|
+
|
|
40
|
+
完整约束见 [AGENTS.md](AGENTS.md),设计见 [docs/superpowers/specs/2026-10-23-baleen-design.md](docs/superpowers/specs/2026-10-23-baleen-design.md),实测记录见 [docs/verification.md](docs/verification.md)。
|
|
41
|
+
|
|
42
|
+
## 数据放在哪
|
|
43
|
+
|
|
44
|
+
插件运行后全部落在自己的家目录(默认 `~/.dsh/baleen/`,可用 `DSH_HOME` 或 `BALEEN_HOME` 覆盖):
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
preference.yaml 你编辑的偏好真源(themes / follow / sources / tuning)
|
|
48
|
+
baleen.sqlite 条目库(items / sources / runs / digests / meta)
|
|
49
|
+
out/ 每轮汇报的 Markdown 副本
|
|
50
|
+
logs/diagnostics.jsonl 脱敏后的运行日志
|
|
51
|
+
run.lock 单实例锁(带 pid,进程被强杀后陈旧锁会被自动接管)
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
不上传、不遥测。凭据只落这个目录(`credentials/`,B 站 cookie 用 `chmod 600` 保护,从不进仓库、不进日志),偏好里的 `themes` / `follow` 只在本机。
|
|
55
|
+
|
|
56
|
+
## 代码地图
|
|
57
|
+
|
|
58
|
+
| 文件 | 责任 |
|
|
59
|
+
|---|---|
|
|
60
|
+
| `index.js` | 宿主半入口:装配、`/api/dsh-baleen` 只读路由、服务注册、调度启动 |
|
|
61
|
+
| `lib/adapters/rss.js` | RSS/Atom 适配器;失败时抛 `status` / `empty` 供上层分类 |
|
|
62
|
+
| `lib/adapters/bili.js` `lib/adapters/bili-wbi.js` | B 站:纯 fetch 直连 + 自研 wbi 签名 + 风控指纹;字幕走 player/wbi/v2 |
|
|
63
|
+
| `lib/adapters/yt.js` `lib/adapters/yt-captions.js` | YouTube:yt-dlp 直连取列表与字幕轨(轨间回退),已入库的 external_id 跳过重抓 |
|
|
64
|
+
| `lib/feed.js` | RSS2 + Atom 容错解析(CDATA、坏日期、双重转义、时长三种写法) |
|
|
65
|
+
| `lib/pool.js` | 并发 4、单 job 超时 20s、**重试固定 1 次**、错误分类 |
|
|
66
|
+
| `lib/normalize.js` | RawItem → items 行:`kind` / `body_state` / 命中理由与得分 |
|
|
67
|
+
| `lib/dedup.js` | 入库唯一入口 `putDraft`:insert / update / skip_seen / duplicate |
|
|
68
|
+
| `lib/prefilter.js` | 本地闸门与 `summarizeGaps`(丢弃统计) |
|
|
69
|
+
| `lib/jobs.js` `lib/runner.js` | 计划与编排,游标只成功才写回 |
|
|
70
|
+
| `lib/scheduler.js` | 时段窗(含跨零点)、文件锁单实例、`triggerNow` |
|
|
71
|
+
| `lib/preference.js` | 偏好读取、逐字段校验、`bumpTuning` 只写 tuning |
|
|
72
|
+
| `lib/diag.js` | 脱敏(家目录、盘符路径、凭据值)与 jsonl 日志 |
|
|
73
|
+
|
|
74
|
+
## 还没做
|
|
75
|
+
|
|
76
|
+
- 小红书通道(扫码 MCP 方案,需先做代码审查)
|
|
77
|
+
- 汇报的图文排版(当前是 Markdown,图片位留白)
|
|
78
|
+
|
|
79
|
+
## 名字提示
|
|
80
|
+
|
|
81
|
+
"Baleen" 在 NLP 圈已有同名 RSS 摄取项目(DistrictDataLabs/baleen 等),带 `dsh-` 前缀才是这个插件。
|
|
82
|
+
|
|
83
|
+
## 安全
|
|
84
|
+
|
|
85
|
+
本插件读本地库、只出只读路由,但它会替你持有 B 站 cookie。凭据怎么存、发现漏洞往哪报,见 [SECURITY.md](SECURITY.md)。
|
|
86
|
+
|
|
87
|
+
## 参与
|
|
88
|
+
|
|
89
|
+
改了代码想提上来,先看 [CONTRIBUTING.md](CONTRIBUTING.md)——里面有本机实测过的坑(CRLF、生成产物、宿主装配)和提交规范。
|
|
90
|
+
|
|
91
|
+
## 许可
|
|
92
|
+
|
|
93
|
+
Apache License 2.0,全文见 [LICENSE](LICENSE)。
|
|
94
|
+
|
|
95
|
+
第三方代码:无。B 站通道的 vendor 方案(BiliStalkerMCP)已在 M4a 落地前废弃,源码未进本仓库——现在的 B 站适配器是自研的纯 fetch 直连。设计参考(DSH 插件形态、面板写法)来自本机已验证的 dsh-study-desk,属于思路借鉴而非代码复制。
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# 安全策略
|
|
2
|
+
|
|
3
|
+
## 报漏洞
|
|
4
|
+
|
|
5
|
+
**不要开公开 issue。** 用 GitHub 的私密报告通道:仓库页 → Security → [Report a vulnerability](https://github.com/slow-stack/baleen/security/advisories/new)。
|
|
6
|
+
|
|
7
|
+
请带上:受影响的版本(`package.json` 的 version 或提交哈希)、能复现的最小步骤、你判断的影响面。三个工作日内给第一次回复;修好之前不在公开渠道描述细节。
|
|
8
|
+
|
|
9
|
+
## 这个插件会碰到的敏感面
|
|
10
|
+
|
|
11
|
+
它不是纯读的工具——它替你持有凭据、跑网络请求、开 HTTP 路由。三块边界值得单独说清。
|
|
12
|
+
|
|
13
|
+
### 1. B 站 cookie
|
|
14
|
+
|
|
15
|
+
字幕增强通道需要登录态 cookie。设计上:
|
|
16
|
+
|
|
17
|
+
- 只存在插件家目录的 `credentials/bili-cookie.txt`(`chmod 600`),**不进仓库**(`.gitignore` 已挡 `credentials/`)、**不进日志**、**不进任何汇报**。
|
|
18
|
+
- 只用于读字幕接口,不注册任何写操作(评论 / 私信 / 点赞 / 关注 / 发布 / 收藏到平台,一个都没有)。
|
|
19
|
+
- 没有这个文件时通道整体降级为「只取简介」,不报错、不阻断其它源。
|
|
20
|
+
|
|
21
|
+
如果你怀疑 cookie 泄漏:B 站网页端退出登录即令其失效,然后重新导出一份覆盖旧文件。**不要**把 cookie 贴进 issue、贴进聊天、贴进提交信息。
|
|
22
|
+
|
|
23
|
+
### 2. 网络与外部命令
|
|
24
|
+
|
|
25
|
+
- 摄取只发 GET(RSS/Atom、B 站只读接口)。YouTube 通道走本机 `yt-dlp` 子进程,参数由适配器拼装,不接受偏好文件里的任意命令。
|
|
26
|
+
- 偏好文件(`preference.yaml`)是**纯数据**,不是脚本。它决定抓哪些源、什么关键词、什么时候抓——但不会被当作 shell 或 eval 的输入。
|
|
27
|
+
- 判断轮(LLM 那一步)只拿到**只读**工具面:`baleen_query / read_item / candidates / run_status / subtitle`。唯一的写工具 `save_verdicts` 只写本插件自己的两张表(`items` / `digests`),tier 参数是三档枚举锁死,不接受任意 SQL、不接受任意路径。
|
|
28
|
+
|
|
29
|
+
### 3. 本地 HTTP 路由
|
|
30
|
+
|
|
31
|
+
- 面板数据走宿主的 `/api/dsh-baleen` 前缀,**只读**(GET)。收藏、归档、清库这些改动全部走宿主侧的面板动作,不是 agent 工具,也没有平台接口。
|
|
32
|
+
- 路由不读请求体里的任意路径:清库/归档清理这类动作只作用于插件自己的 sqlite 文件。
|
|
33
|
+
|
|
34
|
+
## 不在威胁模型内的
|
|
35
|
+
|
|
36
|
+
- **本机已被攻破**:能读你用户目录的攻击者可以直接读 `baleen.sqlite` 和 cookie 文件,插件无法阻止。
|
|
37
|
+
- **上游平台风控**:B 站接口带风控指纹,被判定异常可能导致账号受限。插件做了最小化请求(限速、按 `interval_min` 节流、不并发轰炸),但这不构成对平台条款的豁免——用不用、用哪个账号,你自己判断。
|
|
38
|
+
- **第三方内容本身**:抓回来的正文是**不可信数据**。插件在把它交给判断轮时按数据处理,但内容里的指令性文字永远不该被当成命令执行。
|
|
39
|
+
|
|
40
|
+
## 支持范围
|
|
41
|
+
|
|
42
|
+
只对 `main` 上的最新提交提供修复。发布到 npm 的版本按 [SemVer](https://semver.org/lang/zh-CN/),0.x 阶段不保证向后兼容——升级前看 `CHANGELOG`(如有)或提交记录。
|