@khorsheed/dsh-reader 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/README.en.md +26 -121
- package/README.i18n.yaml +3 -3
- package/README.md +26 -121
- package/cordis.patch.yml +7 -0
- package/lib/client.js +29105 -6692
- package/lib/index.js +3500 -321
- package/lib/typert.host.js +2145 -259
- package/lib/typert.remote-client.d.ts +97 -3
- package/lib/typert.remote-client.js +1590 -250
- package/lib/types/captions.d.ts +99 -0
- package/lib/types/captions.js +236 -0
- package/lib/types/client/MediaPlayer.d.ts +55 -0
- package/lib/types/client/MediaPlayer.js +106 -0
- package/lib/types/client/NotesView.d.ts +57 -0
- package/lib/types/client/NotesView.js +124 -0
- package/lib/types/client/ReaderPane.d.ts +20 -3
- package/lib/types/client/ReaderPane.js +860 -164
- package/lib/types/client/SpeakerLegend.d.ts +18 -0
- package/lib/types/client/SpeakerLegend.js +26 -0
- package/lib/types/client/canvas-sync.d.ts +84 -0
- package/lib/types/client/canvas-sync.js +59 -0
- package/lib/types/client/contract.d.ts +130 -3
- package/lib/types/client/extract-article.d.ts +38 -1
- package/lib/types/client/extract-article.js +561 -33
- package/lib/types/client/icons.d.ts +1 -1
- package/lib/types/client/icons.js +1 -1
- package/lib/types/client/index.js +81 -2
- package/lib/types/client/locales.d.ts +279 -11
- package/lib/types/client/locales.js +278 -10
- package/lib/types/client/notes.d.ts +142 -0
- package/lib/types/client/notes.js +334 -0
- package/lib/types/client/parse-rss.js +40 -0
- package/lib/types/client/preset-visibility.d.ts +93 -0
- package/lib/types/client/preset-visibility.js +115 -0
- package/lib/types/client/selectors.d.ts +5 -0
- package/lib/types/client/selectors.js +5 -0
- package/lib/types/client/session.d.ts +21 -1
- package/lib/types/client/session.js +12 -0
- package/lib/types/client/store.js +4 -1
- package/lib/types/client/translate.d.ts +26 -1
- package/lib/types/client/translate.js +29 -12
- package/lib/types/client/useLocalTranscript.d.ts +43 -0
- package/lib/types/client/useLocalTranscript.js +129 -0
- package/lib/types/client/useMedia.d.ts +68 -0
- package/lib/types/client/useMedia.js +299 -0
- package/lib/types/client/useMediaStates.d.ts +9 -0
- package/lib/types/client/useMediaStates.js +43 -0
- package/lib/types/client/useNotes.d.ts +110 -0
- package/lib/types/client/useNotes.js +423 -0
- package/lib/types/client/usePolish.d.ts +47 -0
- package/lib/types/client/usePolish.js +162 -0
- package/lib/types/document-fetch.d.ts +106 -0
- package/lib/types/document-fetch.js +100 -0
- package/lib/types/index.d.ts +15 -0
- package/lib/types/index.js +3 -0
- package/lib/types/local-transcribe.d.ts +171 -0
- package/lib/types/local-transcribe.js +473 -0
- package/lib/types/media-host.d.ts +91 -0
- package/lib/types/media-host.js +230 -0
- package/lib/types/media.d.ts +100 -0
- package/lib/types/media.js +207 -0
- package/lib/types/notes-types.d.ts +158 -0
- package/lib/types/notes-types.js +8 -0
- package/lib/types/notes.d.ts +213 -0
- package/lib/types/notes.js +718 -0
- package/lib/types/polish.d.ts +41 -0
- package/lib/types/polish.js +136 -0
- package/lib/types/remote.d.ts +118 -0
- package/lib/types/remote.js +151 -1
- package/lib/types/service.d.ts +81 -6
- package/lib/types/service.js +286 -65
- package/lib/types/skill.d.ts +20 -0
- package/lib/types/skill.js +78 -0
- package/lib/types/store.js +30 -8
- package/lib/types/tool.d.ts +23 -0
- package/lib/types/tool.js +40 -0
- package/lib/types/tools.d.ts +69 -0
- package/lib/types/tools.js +400 -0
- package/lib/types/types.d.ts +90 -2
- package/lib/types/types.js +66 -0
- package/lib/types/xiaoyuzhou.d.ts +43 -0
- package/lib/types/xiaoyuzhou.js +134 -0
- package/lib/types/ytdlp.d.ts +111 -0
- package/lib/types/ytdlp.js +269 -0
- package/package.json +33 -20
- package/lib/invariant.js +0 -38
- package/lib/types/invariant.d.ts +0 -29
- package/lib/types/invariant.js +0 -38
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,34 @@
|
|
|
1
1
|
# 变更记录
|
|
2
2
|
|
|
3
|
+
## 0.3.2(2026-10-27)
|
|
4
|
+
|
|
5
|
+
- **微信公众号文章抽取更干净**:以正文容器为准,去掉文末的赞赏卡片。
|
|
6
|
+
- 正文里的图片保持原网页上的显示宽度。
|
|
7
|
+
- **整页抓取**:阅读器自己的抓取正文上限默认改为 200 万字(原来沿用官方 `http` 行给模型用的 10 万字),claude.com 这类正文从约 9.7 万字处才开始的长页不再被截断。见 Agent Note `2026-10-27-reader-fetch-body-cap`。
|
|
8
|
+
- **订阅源和文章不再被新版搜索 API 开关弄坏**:宿主 0.2.1-rc.2 起,网页搜索设置页的「启用新版 Web 搜索 API」会把抓取一起切到 DeepSeek Fetch API(返回 Markdown),订阅源会被当成网页、文章抽不出正文。现在阅读器在官方 web 服务上用自己持有的官方 HTTP 抓取提供方实例抓取,拿到的始终是原始文档;没装提供方包时照旧走 `ctx.web.fetch`。新增配置 `fetchLimits`(`maxResponseBytes` / `maxBodyChars` / `timeoutMs` / `maxRedirects`),抬过 `web-fetch-http` 行上限的部署改配这里。见 Agent Note `2026-10-24-reader-own-http-fetch-provider`。
|
|
9
|
+
|
|
10
|
+
## 0.3.1(2026-10-26)
|
|
11
|
+
|
|
12
|
+
- 修复:重新转写一篇之前翻译过的稿子时,转写进行中每次正文刷新都会重新铺一遍翻译(提示条反复出现);现在转写中不恢复翻译,转完后恢复一次。
|
|
13
|
+
- **让 Agent 润色译文**:地球菜单(没有本机翻译时在标题栏)一键把润色请求填进输入框;模型用 `reader_read` 分批读原文、从原文整段翻译,用新工具 `reader_translate`(每行「¶12:译文」)分批写回,直到全文完成。面板每 5 秒刷新,润色版逐段替换机翻显示(未开机翻时与原文对照),段尾「✦ 润色」点了这一段改回机翻,顶部有进度与「看机翻」(不记住);中途停了给「继续润色剩下的 N 段」。润色存在宿主(`polish/`),跟着原文:说话人改名保留,重新转写作废。
|
|
14
|
+
- **转写在重启后续上**:任务记录存在它的临时目录里(`job.json` + 已转段落 `segments.json`),实例重启后自动续跑——音频不重下,转写从断点接着来,已存稿的直接重跑分说话人。
|
|
15
|
+
- **转写稿分说话人**:本机转写出稿后在后台接着区分说话人——先抽 24 段音色比对(约 1 秒),只有一个人就到此为止;多人才跑完整分人,按时间把每句话归给说话人、换人处分段,段落带说话人颜色,换人的那段开头标名字;页签上方的图例点名字可改名(整期替换,存在宿主上);讨论、考考我读到的原文写成「[mm:ss] 嘉宾:…」。分人中来源行显示「正在区分说话人…」。
|
|
16
|
+
- **详情页减负**:删掉底部「当前会话草稿」预览和「引用 / 记笔记 / 发到侧边对话」一行(划词菜单里有引用与记笔记),文末只留标签;「让Agent考考我」从文末整行大按钮改为页签行右侧的「考考我」小按钮,文末留一行文字链接。添加对话框的「抓取」在输入框为空(含保存成功后清空)时置灰,抓取中显示「抓取中…」。
|
|
17
|
+
- **切回灵感空间不再重载**:右栏切到别的页签再切回,宿主会卸载面板;现在页面内存里留着解析好的卡片墙,切回来立即显示,只重新读取抓取时间变了的源(刷新、添加源照旧全量重读);卡片墙翻译跳过已有译文的卡片,不再重新翻译。
|
|
18
|
+
- **转写稿断句**:引擎按真实停顿切块后,一句话不再被拆进两段;没有句号的长句超过 20 秒遇逗号、30 秒遇停顿即断,最长 45 秒;小于 20 MB 的模型补下载不再询问。已有转写稿点「重新转写」生效。
|
|
19
|
+
- **视频与音频打磨**:卡片与文章卡等高,右侧封面 + 转写状态药丸(新 Remote `mediaStates`,有卡在转写时每 3 秒刷新);详情的控制收成一行(B 站说明收进 ⓘ),页签改叫「转写稿」,时间排进左侧一栏并带 ▶ 与一次性提示;视频往下读时收成迷你条(同一个画面缩小,YouTube 可暂停、显示时间与进度);跟随在手动滚动后停止,「回到正在播放」接上(去掉「跟随」开关);转写中地球置灰,「转完自动翻译」全局勾选(默认不勾),外语转写稿以对照视图自动翻译。见 Agent Note `2026-10-20-reader-media-polish`。
|
|
20
|
+
- **本机转写**:没有字幕的视频和音频(B 站多数视频、播客、小宇宙)打开详情页即在本机转写,经可选的「本机转写」插件(`transcribe` 服务,探测;没装时如实提示);音轨临时下载(yt-dlp 取最小音频格式 / curl 取音频直链)转完即删;转完的部分边转边显示、可读可记,完成后存为正文并标「来源:本机转写」;排队中离开撤出队列、在转的转完,「字幕」关掉或「取消」即停止;首次下载模型确认一次;订阅刷新不转写。见 Agent Note `2026-10-20-reader-local-transcription`。
|
|
21
|
+
- **视频与音频**:YouTube / B 站视频、播客 RSS、小宇宙单集与节目进入灵感空间,类型按每条内容判断;视频以平台字幕为正文(合并成带时间的段落、时间标记可点、当前段高亮跟随、「字幕」开关、播放位置记忆),笔记带时间、`reader_read` 带 `[mm:ss]`;卡片标类型与时长,筛选加视频 / 音频。宿主经自管的独立版 `yt-dlp` 取字幕与频道列表(YouTube 自家频道 RSS 已 404),小宇宙节目页转成 RSS。新增配置 `ytdlpPath`、Remote `mediaTranscript`、状态文件 `media.json` 与 `bin/`。
|
|
22
|
+
- **笔记**:划词「记笔记」(挂进 quote 的选区菜单,另有详情页兜底按钮),摘抄 + 评论存进独立的 `notes.json`;在译文里划的存原文并附译文;正文以 CSS 自定义高亮标出记过的句子;详情页「正文 / 笔记」切换,笔记可编辑、删除、跳回原文。
|
|
23
|
+
- **带着笔记讨论**:选中的笔记合入会话草稿(不覆盖、不发送),文章记下讨论过它的会话,可回到会话。
|
|
24
|
+
- **`reader_read` 工具**(新组合入口 `./tool`,patch 新增 `reader-tool` 行,全会话可见):读最近打开或指定的文章,正文按 ¶ 编号分页、附读者笔记;正文来自浏览器打开文章时交给宿主的原文快照(`snapshots/`,无笔记的文章只留最近 30 篇)。
|
|
25
|
+
- **结论写回与讨论笔记**:`reader_note` 工具(`op=conclusion` 挂在某条笔记下,`op=discussion` 记一条讨论笔记,`kind=check` 为考考我;支持只靠 text 的纯文本写法);读者可追加、改写、删除结论,删除讨论笔记;笔记页显示时每 5 秒重读一次,Agent 写回的内容自动出现。
|
|
26
|
+
- **让Agent考考我**:文末与笔记页的按钮把考考我请求填进输入框(附约定);`reader` 技能写明一次问一组、答完只讲没到位的、记成考考我笔记、不打分。
|
|
27
|
+
- **同步到画布**:装了灵感画布时,笔记 / 讨论笔记可落成正式灵感卡(经画布的 `ctx.canvasInbox`,探测,缺席即隐藏),记住上次的画布,可带讨论结论,`markSynced` 记下落到哪张卡。画布这一波不发布,没装画布时这个按钮不出现。
|
|
28
|
+
- **按模式显隐**:右栏页签按 `./tool` 行双查自隐(画布同款);灵感空间移出基础体验合集,改随「学习套件」安装(写作套件这一波不发布),工具与页签只在授予它的模式的会话里出现。
|
|
29
|
+
- 新增可选 peer:`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-agent`。
|
|
30
|
+
- **README 重写**:改为大白话说明,开头不再以「同步到画布」打头,安装只写学习套件。
|
|
31
|
+
|
|
3
32
|
## 0.3.0(2026-09-30)
|
|
4
33
|
|
|
5
34
|
- **CORP 拦截图救回**:以 `cross-origin-resource-policy: same-origin` 供图的站点(如 claude.dev)会让浏览器跨域 `<img>` 必挂,只有宿主进程抓取能绕过。新增 `fetchImage` Remote——只重抓已经加载失败的图片(公网地址校验钉在 DNS 答案上、逐跳重定向再校验、限 `image/*` + 8MiB + 15s、32MiB LRU),阅读面板把这些图改指 data: URI。健康图片从不经过代理;旧版宿主下图片维持 broken
|
package/README.en.md
CHANGED
|
@@ -2,143 +2,48 @@
|
|
|
2
2
|
|
|
3
3
|
English | [中文](README.md)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Keep the articles, videos and podcasts you mean to get to on one wall, read them right in DSH, take notes, and work through them with the agent.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Papers, articles and videos you come across end up scattered across browser tabs; to talk about one with the agent you copy and paste, and once the chat is over nothing sticks. With the Reader, paste a link or subscribe to an RSS feed and a card appears under "Inspiration" in the right sidebar; open it to read, and translate a foreign-language piece in one click. Select a line that matters to make a note, discuss your notes with the agent, and finally let it quiz you on whether you can explain it — the conclusions go back into your notes.
|
|
8
8
|
|
|
9
|
-
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-1.png" width="
|
|
9
|
+
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-1.png" width="560" alt="the Inspiration wall: a card stream of subscriptions and saved links; the header row holds filter, sort, translate-the-wall, refresh, settings and add, with the search box on its own row">
|
|
10
10
|
|
|
11
|
-
##
|
|
11
|
+
## What it does
|
|
12
12
|
|
|
13
|
-
- **
|
|
14
|
-
- **
|
|
15
|
-
- **
|
|
16
|
-
- **
|
|
17
|
-
- **
|
|
18
|
-
- **
|
|
19
|
-
- **Quote into the
|
|
20
|
-
- **Subscription page and recently read** — one row per source with in-place name and address editing, per-source refresh, pause/resume and remove, plus the cache policy (body retention, translation budget, clear translation cache) and storage readouts on the same page; **recently read** persists 100 rows, survives a restart, and can reopen an entry that has rolled out of its feed's window.
|
|
21
|
-
- **Leave and carry on** — the open article, the reading position anchored by content (the Nth block, this character of its text), the wall's narrowing and the translation are remembered by the whole page across dsh sessions; the place is also written to `sessionStorage`, so a reloaded page finds the reader where they were.
|
|
13
|
+
- **Paste a link and it's saved**: click "Add inspiration" and paste — a feed becomes a subscription, an article is saved on its own; subscriptions refresh daily, and a story reposted in several places keeps just one card. Links it can't fetch are saved too, with the reason.
|
|
14
|
+
- **Videos and podcasts read like articles**: YouTube, Bilibili, podcasts and Xiaoyuzhou all fit; subtitles become the text when there are some, otherwise it transcribes on your computer into a "Transcript", and in a conversation it tells the speakers apart. Click a timestamp to play from there.
|
|
15
|
+
- **Translate in one click**: sentence by sentence, click a sentence to see the original; if the machine translation isn't good enough, "✦ Let the agent polish the translation" has the agent retranslate the whole piece.
|
|
16
|
+
- **Select a line, take a note**: select a sentence and click "Take a note" to keep the original and your thought; noted lines are highlighted in the text, and a note jumps back to its spot.
|
|
17
|
+
- **Discuss with your notes**: "Discuss with these notes" puts them in the composer — send it and talk it over with the agent; ask it to record the conclusion and it goes under that note.
|
|
18
|
+
- **Let the agent quiz you**: after reading, "Let the agent quiz me" has it play a student who hasn't read the piece and ask you to explain it, then point out, against the original, only what you missed. No scores — click again to practice again.
|
|
19
|
+
- **Quote into the chat**: select text to quote it straight into the composer; with the canvas installed, a note can "Sync to canvas" as an inspiration card.
|
|
22
20
|
|
|
23
|
-
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-2.png" width="
|
|
21
|
+
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-2.png" width="560" alt="the detail view: the article rendered as text; with the globe toggle on, sentences are translated side by side and clicking one reveals its original">
|
|
24
22
|
|
|
25
23
|
## Install
|
|
26
24
|
|
|
27
|
-
|
|
28
|
-
dsh plugin --profile web add @khorsheed/dsh-reader
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
Restart the web instance to activate. Uninstall:
|
|
25
|
+
In DSH, open Plugins → Add plugin, enter this package name and click Install:
|
|
32
26
|
|
|
33
|
-
```sh
|
|
34
|
-
dsh plugin --profile web remove @khorsheed/dsh-reader
|
|
35
27
|
```
|
|
36
|
-
|
|
37
|
-
Exactly one directory is written: `$DSH_HOME/state/dsh-reader/` — `state.json` (the subscription list, the latest payloads, the reader's own tags and the recently-read records) and `bodies/` (one file per body over 256 KiB; the global translation memory is its `translation-memory.json`, and a large per-entry segment map gets its own file). Uninstalling leaves the directory behind — delete the whole `dsh-reader/` for a clean removal.
|
|
38
|
-
|
|
39
|
-
## Compatibility
|
|
40
|
-
|
|
41
|
-
- npm release line (`@deepseek-ai/dsh@0.1.5-rc.1`): ✅ works, one designed degrade — the `sidebar.right.pane.tab` seat and the `ctx.web.fetch` egress seam are complete on 0.1.5-rc.1; the in-app Sidebar Browser exits (the script-figure notice, a bot-wall card's action) need the `browser` right-Sidebar tab kind from 0.1.6-alpha.2 and fall back to the ordinary external link on a 0.1.5 host. minHost `0.1.5-rc.1`; this package does no preset self-hide (no `pluginInventory` needed).
|
|
42
|
-
- source line (deepseek-harness master): ✅ (verifiedHost: 0.1.5-rc.1)
|
|
43
|
-
|
|
44
|
-
Note: the host fetch seam truncates a decoded body at 100,000 characters and does so **silently** — that is a deployment-configurable cap (see the patch overlay under 「How it works · the egress seam and storage」), not this package's body budget.
|
|
45
|
-
|
|
46
|
-
## Known Limitations
|
|
47
|
-
|
|
48
|
-
- **Subscribe, do not crawl** — adding a subscription fetches the feed itself and never pre-fetches every entry's body (that would blow up storage and amount to running a crawler); full text arrives a slice per refresh via the backfill, or on the spot when you open an entry.
|
|
49
|
-
- **A PDF (or any non-web file) has no body** — the egress seam accepts only `text/html`, `text/*`, `application/json` and `application/xml`, so `application/pdf` is refused before extraction even starts — and extraction is DOM-based, which a PDF does not have. Such a link is a 「仅链接」 card plus 「阅读原文」; this plugin is not a PDF reader.
|
|
50
|
-
- **A cross-site hop is not followed** — the seam reports the target origin only, never the path, so shorteners and institutional proxies are reported as "this address moves to another site" rather than guessed at.
|
|
51
|
-
- **The unread dot is not persisted** — read state is session state; a reload or restart returns every entry to unread.
|
|
52
|
-
- **Translation needs Chrome 138+ / Edge 148+ on desktop** — Safari, Firefox and mobile have no Translator API, so the globe stays hidden there while everything else keeps working. The trade is deliberate: one 50,000-character article through a host LLM costs tens of thousands of tokens, tens of seconds, and a second copy of the article to the model provider; on-device translation is free, seconds, and stays on the machine (page-translation-grade NMT, not LLM-grade).
|
|
53
|
-
- **Oversized articles cannot be fully extracted** — the seam's default 100,000-character cap (measured: an Anthropic research page ~266k, theverge.com ~899k, newyorker.com ~1.97M characters) leaves the detail view showing the truncation notice plus 「阅读原文」; raising the cap is a deployment decision, and the plugin never works around the host's egress policy itself.
|
|
54
|
-
|
|
55
|
-
## How it works
|
|
56
|
-
|
|
57
|
-
<details>
|
|
58
|
-
<summary>The surfaces (wall / detail / translation / add / subscriptions / recently read — click to expand)</summary>
|
|
59
|
-
|
|
60
|
-
> The package name, tab kind and Remote namespace stay `reader` — they are stable identifiers (renaming them would change a deployed loader row and the on-disk state root). "Inspiration" is the name the reader SEES.
|
|
61
|
-
|
|
62
|
-
| Surface | Contents |
|
|
63
|
-
| --- | --- |
|
|
64
|
-
| The wall | **Content first**: the entry cards own the first screen. **The search field owns its own full-width row**, and the filter, sort and wall-translation controls moved up into the pane's header (with refresh, settings and add) — a field wedged between three buttons is a field the reader cannot see what they typed in on a narrow sidebar; every popover hangs under **the button that opened it** and cannot cover the other one. The filter is a **two-page popover** narrowing the wall four ways — read state (unread), by source, by KIND (a feed or a saved link) and by tag: the root page holds the two read-state rows (unread, and **Hide duplicates**), a by-source drill row that carries the **current value** (so no narrowing is ever hidden) and the three kind rows. **Dedupe is on by default**: an article republished by several feeds folds into one card (three grouping tiers, any hit folds: same entry id — a shared guid shares fetch/read/translation state by design; normalized link — scheme/www/utm-style tracking parameters stripped; folded title plus the same published day; never fuzzy), the surviving card carries a 「N 个重复来源」 badge (hover names the other sources), reading any copy reads the group, the footer notes how many folded, and the backfill never spends a fetch on a hidden copy, and drilling in opens the source page, which owns the panel's height, its own scroll and — past six sources — its own search box, so **however many subscriptions accumulate, the panel's first page stays a fixed handful of rows**. It is the same local predicate as the search box (D14), so filtering costs no round trip, and its value is visible in the search box (`#sourceId` / `#rss` / `#link` / `@tagId`), which is also how it is cleared — and **a field with text in it grows an × at its right edge**, one click away from clearing both the typed term and the filter value. A **text** search is ranked by relevance — exact title, then title contains, then the other fields, with the chosen sort deciding inside each tier (`#sourceId` / `@tagId` selectors are not ranked) — and the footer says how many entries matched (`N matching “X”`), so a narrowed wall can never be mistaken for the whole wall. **The wall translates too**: the globe at the right of the toolbar (its own switch, separate from the detail view's, sharing one translation memory). It decides **per FIELD**: a Chinese title or summary is left exactly as it is, a mixed card has only its English half translated, and Chinese is never translated into Chinese. Only cards that have **entered the viewport** are queued (IntersectionObserver feeds a queue, six cards per pass, grouped by detected language, through the article's own sentinel-batch contract), so scrolling back costs nothing and opening an entry does not re-translate its title. Two views: translation only (**hovering the translated TEXT** swaps just that FIELD back to its original — the title for the title, the summary for the summary; the card's blank space, tile and chevron change nothing) and side by side (each field's original stays under its translation and keeps that FIELD's own typography — a title's original is title-sized, told apart by ink and weight rather than shrunk to caption size). **The text boxes are fixed** — title and summary each reserve two clamped lines — so swapping languages on hover cannot resize a card or shift the grid; only the side-by-side view makes the grid taller, and that is a mode the reader picks. Entry cards are **cards** — a fill, a hairline border, a 12px radius — where hover only lifts the tone (never the geometry) and a keyboard focus draws a ring; unread entries ink their title and keep the **dot** (7px) on the tile's corner — session-only, never persisted — while read entries stay put and go quieter; the feed's own category and the reader's own tag share one pill metric (the reader's tag is outlined, and the row wraps), and tagging opens from the card's `+` into a panel where you tick the vocabulary or **type a name and press Enter to create it** (the native `datalist` is gone, replaced by stylable rows). **Tags are deletable from the filter popover's `By tag` section** (a quiet × at the end of each row removes it from the vocabulary and from every entry, and clears the narrowing when it was the active one) — a vocabulary you can only add to is a one-way door. A saved link whose body could not be read carries a quiet dashed 「仅链接」 badge, and hover and detail both say why. **A saved link's card upgrades to the article's own title once the body parses** (the body's first h1, else the document title, plus the first substantial paragraph as the excerpt) and keeps it in the host annotation — the card still reads that memory after the body's budget eviction, so a wall of papers is scannable without opening each one. A saved link without `publishedAt` sorts by **its source's added time**, so a freshly added link sits at the top instead of sinking. A freshness line sits at the bottom — "Refreshed 3 h ago · daily 10:00" — so a reader can tell whether the snapshot is worth re-fetching |
|
|
65
|
-
| Detail | Where a card leads. The body renders as DOM text — **that is exactly what makes passage quoting work**, and why this surface never hosts an iframe. The body has **no reading measure**: it fills the width the sidebar is given, and paragraphs and lists share one right edge. Toolbar: back, **translate (globe + view menu)**, copy link, open the original in the browser, and — for a single saved link — **a delete button** (a feed entry gets none: the next refresh would bring it back, and a delete that undoes itself is a lie); the **tag row under the title also carries 「重新抓取」** (the same host action as the card's pill, and asking for a fresh copy drops that entry's translation record — the body is about to be a different DOM). Below the body, the other entries "also from this source". **A figure the page draws with its own scripts cannot be fetched**: such a `<figure>` carries nothing but an empty div and a caption in the served HTML (measured: 84 of them on the transformer-circuits.pub paper), and a fetch runs no scripts — so **the caption stays in the body** (it is text the page published, and it says what the picture shows) while one line above the body says 「这一页有 N 张插图由页面自己的脚本绘制,抓取时拿不到画面(图注保留在正文里)」 with the original-page link (opened in the in-app Sidebar Browser when the host has one; and when a future capture package is mounted, a 「渲染抓取」 action appears here — unprobed, it does not render). **And a capture-rendered page keeps its figures**: script-drawn figures arrive as inline SVG with inlined styles and survive through a safe subset (shapes/text/defs/gradients/clips/symbol reuse; `script`/`foreignObject`/SMIL/events/external references never do, and `url(` may only point at a same-document fragment) — instead of leaving only a caption. **Composite figures keep their structure too**: a capture-rendered composite (a diagram plus property cards built from styled boxes) keeps its div/span containers inside the figure with an allowlisted style set (layout/paint properties; `position:fixed`, over-bound `z-index`, external `url()`s and script tricks all stripped), and every unwrapped block container leaves a word boundary — sibling texts can never concatenate. **A picture whose URL IS in the static markup is recovered, though**: lazy-loading sites keep the real URL in a `data-src`-style attribute, a `srcset`, a `<picture>`'s `<source>`, or the `<noscript>` no-JS fallback (the `src` holds a 1px placeholder), and the extractor resolves the candidates in that order, taking the largest srcset one; **the converse shape is kept too — pages that inline their real figures as base64 data URIs (transformer-circuits.pub style)**: placeholder and picture are split by payload size (a 512-char threshold; a 1px gif is ~70, a real chart runs to the thousands), the small one still counts as absent and yields to the lazy attributes, the large one is kept as-is; every emitted `<img>` carries `referrerpolicy="no-referrer"`, because anti-hotlink CDNs answer 403 otherwise. **There is a second refusal a referrer policy cannot fix**: an origin answering `Cross-Origin-Resource-Policy: same-origin` (measured: every `/media` figure on claude.dev) withholds the pixels from a cross-origin page even though a direct fetch gets a 200 — the pane showed the broken-image icon over the full alt text. Those images are fetched host-side instead (public-address validation with the connection pinned to the validated answers, per-hop re-validation, an 8 MiB cap, and an `image/*` content-type gate — the host's own `ctx.web` seam decodes text only and cannot carry bytes): an `<img>` that already failed is re-pointed at the fetched bytes (a `data:` URI), a healthy image never pays the proxy, and one the host cannot have either stays broken (「阅读原文」 remains the way out). **Formulas render as native MathML**: an arXiv HTML paper's `<math alttext="…">` passes through a safe presentation subset (`annotation-xml` and any script never do — the former is an HTML-injection vector), and display math scrolls sideways instead of breaking the layout. **LaTeXML's vector figures are kept too**: arXiv HTML embeds them as `<object type="image/svg+xml" data="…">`, which the normalizer rewrites as an `<img>` carrying the original width/height; a non-image object (a PDF, a movie) is still dropped whole |
|
|
66
|
-
| Translation (on-device) | The toolbar's **globe** is a switch: lit means this body is showing a translation. Its caret holds three views — **translation only** (the default; a one-time line above the body teaches "click any sentence to see its original, click again to hide it"), **side by side** (every paragraph carries its original underneath, foldable per paragraph), and **original only**. The unit is a SENTENCE: clicking one sentence reveals that ONE sentence's original under its paragraph (hairline plus a highlight), and several can be open at once; when the sentence contains an inline link and was cut into several text runs, the reveal still shows the COMPLETE sentence rather than the fragments, the original line itself is NOT a control (it is text meant to be selected for quoting — clicking it does nothing, and a click that ends a selection is ignored, so quoting never folds the sentence away). The English original is set at BODY size (13.5px, the secondary ink, 1.62 leading, 3px between lines) and is spaced by WHERE its container sits: beside a paragraph it hugs that paragraph (4px above, 14px before the next block), while inside a list item, quote or table cell it sits 5px under the translation instead of being pulled up into it by a negative margin. An EXPANDED sentence keeps one quiet marker — a hairline underline, with no standing fill or bar (that is the "this sentence is bound to the line below" cue). The pairing highlight itself is HOVER-ONLY: pointing at a Chinese sentence lights that sentence (including the fragments a link split it into) together with its one original line, and pointing at the original line does the reverse. Nothing carries a standing colour or bar by default — the earlier always-on marks turned a skimmed article into confetti. All of it runs **in the browser** (the `Translator` API in Chrome 138+ / Edge 148+, desktop): the model never leaves the device and spends no tokens, but **the derived sentence pairs do persist** (below), while the globe record and translator sessions never do. Without that API, with an unavailable pair, or on a body that is already Chinese, the globe is **not rendered at all**. **The translations persist in two tiers** (under `$DSH_HOME/state/dsh-reader`, the same directory, sensitivity class and uninstall story as the cached originals): a global **sentence memory** (`<src>→<tgt>:<sentenceHash>` → source + translation, 50k entries) and a **per-entry segment map** (hash → translation, keyed to the body's hash, so a minor refetch still reuses every unchanged sentence). One shared budget — 64 MB by default, tunable through the same cache-policy verb as the body retention — evicts least-recently-used across both tiers, with **no TTL** (a translation costs a gesture plus per-sentence model work to rebuild, so it must not expire by the clock). The net effect: after a reload the globe still waits for its gesture, and after it the whole article re-paints from disk with the model untouched; both tiers carry a schema version that reads as a miss and evicts lazily, so a browser-model upgrade never passes old translations off as the new model's |
|
|
67
|
-
| Add (dialog) | The header's plus, the empty-state card and the dashed card at the end of the wall all open the same **dialog** (the members tab's invite-dialog idiom: transparent overlay plus centered card, Esc and overlay-click close), because adding is a momentary act and the wall should not be replaced by it. **What comes back decides how it is stored** (D15): a feed becomes a subscription, a web page is saved as a single item. **An arXiv abs/pdf link is upgraded to the paper's official HTML version** (`arxiv.org/html/<id>` — LaTeXML-built semantic markup: real tables, real formulas); when the HTML version does not exist (404) the pasted address falls back to the ordinary path. **A DOI link is resolved to its arXiv version first** (Crossref for the title and authors → an arXiv title search → a two-factor gate of title similarity plus author family-name overlap; no match means the ordinary path, never a guess), and a landed card marks the DOI it came from; **an OpenReview link is saved link-only without a fetch** (measured unreadable server-side: the API sits behind a Cloudflare challenge, the forum is a JS SPA), with the reason and the suggestion on the card (paste the arXiv version if one exists). **A URL that cannot be read is still saved**: the verdict changed from "succeeded / failed" to "saved, plus whether it can be previewed", the reason is classified (bot wall / login wall / not a web page / cross-site hop / unreachable / HTTP error) and the card is marked 「仅链接」 — dropping the URL the reader pasted is worse than a card that can only be opened in a browser |
|
|
68
|
-
| Subscriptions (page) | The header's settings button opens the **subscription page**: one **compressed settings row** at the top (daily refresh time and article retention side by side, their long explanations moved into hover titles), a **storage row** (the article cache's readout — "N cached · size", the translation cache's readout — "M maps + S sentences · size" — the **translation budget** input in MB, committed on blur/Enter, and a two-step **clear translation cache** button; the readouts are aggregated on the host, the tables never cross the wire) plus two labelled **chip groups** — `By type` (all / feeds / saved links, each with its count) and `Sort` (added / name / last fetch, added-time descending by default) — then one row per source where **both the name and the feed address are editable in place** (a changed address is a different document, so it refetches on save; clearing the name falls back to the feed's own title), plus its kind, item count, last fetch, failure reason (classified, not the seam's raw sentence), per-source refresh, pause/resume, remove, and the **daily refresh time setting**. Those questions are about SOURCES rather than about content, so they do not live on the wall |
|
|
69
|
-
| Recently read (page) | The header's **clock** opens the **recently read page**: the entries you opened, **newest first** (title, source, "how long ago"), where clicking one **reads it again**; a bin in the page bar **clears the whole list**. It records "what was I just reading", so it is **persisted** (`state.json`'s `recent`, **at most 100 rows**; reopening an entry moves it back to the top instead of duplicating it) and survives a reload and a restart — unlike the pane's own memory (the open article, the reading position, the translation), which is shared **across dsh sessions but not across a reload** (it is per page — see above); `recent` is deployment-level and outlives a restart. **A row stores only the entry id, source id, title, URL and time** — never the body — so the page can still reopen an entry that has **rolled out of its feed's window** (as long as the source is still there). An entry whose source was deleted is still listed but cannot be opened, and says so |
|
|
70
|
-
|
|
71
|
-
**The 「抓取」 pill on every card**: **grey = not fetched (click to fetch) / blue = fetching, or fetched and waiting to be extracted / green = the full text is cached (open it and read) / red = failed**. The failed state **acts per cause**: clicking the red pill opens a small popover — the reason, plus the action that cause answers (a transport failure gets 「重新抓取」; a bot wall, a login wall, a non-web file and a known-unreadable page get **open in a real browser** — the in-app Sidebar Browser when the host has one, else the external browser; an HTTP error, an empty page and a cross-site hop are final, so the popover holds the reason and no action) — while the card itself stays one line tall and never breaks the grid. Once you press it you can **leave the page immediately** — switch panels, close the tab, even reload: the fetch runs on the HOST, the raw page is written to `bodies/` first, and the next time the inspiration space opens, the browser half extracts it and caches the body. So you can queue what you plan to read from the wall instead of waiting in the detail view. **Opening an entry whose body is still missing fetches it automatically** — the same action the pill performs; **an entry that has already been fetched is not fetched again** (the text on screen IS the cached body, and re-fetching it would only replace the DOM — taking the reader's translation with it). **An entry whose feed publishes full text is kept on its first open** — the feed's own text becomes the cached body (no request at all; the text is already on screen), the pill turns green on the same gesture, and the body survives the entry rolling out of the feed's window. To ask for a fresh copy, use **「重新抓取」 in the detail view's tag row** (the same host path; it also drops that entry's translation record — see below).
|
|
72
|
-
|
|
73
|
-
**Leave and come back — and carry on in another conversation**: when another main panel takes over, the host **unmounts** the whole right sidebar (`RightbarRoot` renders only while no other panel is active — it is not hidden), and this pane's store is created per mount. Hopping to the side chat and back used to lose the open article, the reading position, the wall's search/sort/filter and **the translation the reader had already paid for**. No flash of the wall while that happens: the pane knows from its first frame which entry it should be showing, and shows the detail surface with a loading line instead. All of that is now remembered by the **whole page** and restored: **across dsh sessions** — opening the reader in another conversation finds the same article, the same reading position, the same narrowing and the same translation — including **the wall's own translation switch and the card texts it already produced** (the pane is mounted per session; this memory is per page) — and the same-session round trip through the side chat works as it did. The article is reopened (its body answered from the host's cache, so no request), the reading position returns (both scrollers — the wall and the article — and the article's place is remembered by CONTENT, "the Nth block, this far into it, at **this character of its text**", not as a pixel count, because an article's own images carry no dimensions and grow the document as they load; a body replaced or re-fetched under the reader puts them back again, **and an original↔translation switch — which rewrites every block's height — re-anchors to the same sentence through the character offset**) the wall's narrowing stands, and **the translation turns the globe back on by itself whenever this page already built a translator session** (`Translator.create()` needs user activation, so only a session that already exists can restore — with none, nothing is automatic, the record stays, and the next click on the globe still costs nothing). **A translation follows its body**: when the body is replaced under the reader (a re-fetch landing, an expired cache re-read, the element rebuilt), the translation is re-applied to the NEW body as long as this entry's globe is on — from the page's sentence memory, so usually without a request. **The PLACE survives a reload**: the open entry, the article anchors and the wall's offset are written to `sessionStorage`, so a page that restarts still finds the reader where they were. **The globe record and the translator sessions are never written anywhere**, so the globe comes back off after a reload (there is no session to restore it with, and building one needs a gesture) — but **the translated sentence pairs are persisted**, so one click re-paints the whole article from the host's two tiers without a single model request. Closing the tab clears the sessionStorage part (`sessionStorage` is per tab); the translations on the host age out under the budget, or go with the directory on uninstall.
|
|
74
|
-
|
|
75
|
-
**Click semantics (settled as option B)**: a card click opens the detail view. Most reading happens in dsh; when the body is too long or the reader wants the original, the detail view's own button goes to the browser. The trailing chevron is a cue that the card leads somewhere, not a control.
|
|
76
|
-
|
|
77
|
-
**Quoting**: the body is an app-level selection, so `@khorsheed/dsh-quote`'s overlay menu **already covers it** — this package builds no selection menu of its own. The detail view also carries:
|
|
78
|
-
|
|
79
|
-
- **Quote**: the selection (the whole entry when there is none) plus a source label, merged into the current conversation draft through the official input machine (the live draft is read and merged, never overwritten, never sent);
|
|
80
|
-
- **Sent to side chat**: shown only when the composition really has a `sideChat` service (the capability handshake decides), hidden otherwise.
|
|
81
|
-
|
|
82
|
-
</details>
|
|
83
|
-
|
|
84
|
-
<details>
|
|
85
|
-
<summary>Internals (architecture, the egress seam, translation, storage — click to expand)</summary>
|
|
86
|
-
|
|
87
|
-
**Architecture.** Fetching happens on the **host half** through the sanctioned `ctx.web` egress seam; the browser half only parses and renders. The host half provides the `reader` service core (`ctx.provide('reader')`) and mounts its thin Typert Remote face (namespace `reader`); the subscription list and the most recent raw payloads are persisted to `$DSH_HOME/state/dsh-reader/state.json` with atomic `node:fs` writes (not through `ctx.fs` — see below), and the daily refresh is armed by a plugin-owned `setTimeout`, since no host scheduler service exists to ride. The browser half registers the right sidebar in two stages: first its own minted tab type `reader` (implementation id `@khorsheed/dsh-reader`, with a guide card) into the `sidebarRightTabs` registry, then the pane body into the keyed `sidebar.right.pane.tab` seat via `slots.inject`; the zh/en dictionaries register with the plugin, and the Remote is mounted by the client itself via `ctx.remote.$mount`. Feed parsing has to live in the browser — the host has no XML/DOM parser (`globalThis.DOMParser === false`) — so `getBodies` only hands the raw payload across. There is no `static inject`: every optional capability (`fs`, `web`, `sideChat`, the `browser` tab kind, the capture Remote) is probed at the point of use and degrades, never throws.
|
|
88
|
-
|
|
89
|
-
**Why state does not go through `ctx.fs`**: `ctx.fs` is the **sandboxed** filesystem — every mutation is fenced by the FILE POLICY OF THE CALLING SESSION, so a workspace-write session is refused a write under `$DSH_HOME/state` (`FS_SANDBOX_DENIED`, observed on the acceptance instance). The fence is right and stays; the mistake was aiming a session-scoped capability at deployment-owned state. This package therefore writes with `node:fs` (atomic temp-file plus rename), the way this repo's other host-state packages do (`packages/lab/src/state.ts`), and degrades to memory-only when the root is unwritable.
|
|
90
|
-
|
|
91
|
-
**The Remote face.** Namespace `reader`, verbs by group: sources — `capabilities` / `listSources` / `addSource` / `updateSource` / `removeSource` / `refresh`; bodies — `getBodies` / `getEntryBody` / `fetchEntryBody` / `storeEntryBody` / `getRawBody` / `entryFetchStates` / `listBackfillCandidates`; translations — `getEntryTranslation` / `getSentenceTranslations` / `rememberSentences` / `clearTranslations`; tags — `entryTags` / `listTags` / `createTag` / `tagEntry` / `renameTag` / `deleteTag` / `pruneTags`; reading records — `recordRead` / `listRecent` / `clearRecent`; cache policy and readouts — `getCachePolicy` / `setCachePolicy` / `getStorageStats`; plus `quoteToSideChat`. The host half **never parses** a feed — parsing lives in the browser half.
|
|
92
|
-
|
|
93
|
-
**Sources are named by the feed's own title**: the host half cannot parse a feed (it has no XML parser), so a new source starts with a URL-derived label — the `raw.githubusercontent.com/...` you see in the strip. Once the browser half has parsed `<channel><title>`, that real title replaces the display label, so cards read "Anthropic Research"; it falls back to the URL when parsing fails. The `{count} d ago` placeholder the list showed for an errored source was the other face of the same class of bug (an uninterpolated parameter) and is fixed.
|
|
94
|
-
|
|
95
|
-
**Translation uses the browser's own on-device model, not the host's**: `self.Translator` (Chrome 138+ / Edge 148+, desktop only) translates inside the page's own process, so **the model never leaves the device, spends no tokens and needs no network** (the first use downloads a small language pack per pair, and `Translator.create()` requires user activation — the click on the globe is exactly that). **The derived sentence pairs are written to the host side** (the global sentence memory and the per-entry maps under `$DSH_HOME/state/dsh-reader`, a shared 64 MB budget, LRU, no TTL; derived content of pages the reader read, the same sensitivity class as the cached originals beside them — the privacy invariant was always about the network, not the disk).
|
|
96
|
-
|
|
97
|
-
**The translation unit is a sentence — not a paragraph, not a text node**: sentences are cut BEFORE the request, so the pairing is true by construction rather than guessed afterwards — a paragraph-level translation comes back as one block, the model merges and splits sentences, and re-aligning the two sides afterwards can show the wrong original. Batches separate sentences with a sentinel and, when the returned part count does not match, that batch is re-sent one unit per request: **granularity is what the fallback gives up, never correctness**. A sentence that crosses an inline element (a link inside it) becomes two units and the link itself is never touched; nothing inside `<pre>`/`<code>` is translated.
|
|
98
|
-
|
|
99
|
-
**A translator that cannot be built is reported, never faked**: `Translator.availability()` is a HINT, not a promise — on Chrome a pair has been observed answering "available" and then rejecting `create()` with `NotSupportedError` ("Unable to create translator for the given source and target language"; Chromium has the same failure recorded for macOS builds). So the plugin (1) asks the browser's `LanguageDetector` for the source language first — a wrong source is one way a pair cannot be built, and that error names NEITHER language; (2) actually tries the target in both accepted spellings, `zh` then `zh-Hans`; and (3) when every attempt is rejected as unsupported, HIDES the globe and says which pairs it tried while pointing at `chrome://on-device-internals` ("Broker State"). Non-permanent failures (download, quota) keep the globe so the reader can retry, and report the attempted pairs plus the reason.
|
|
100
|
-
|
|
101
|
-
**Restoring the body is byte-exact**: translation only rewrites text nodes (the originals stay in memory), so "original only" and cancel put back exactly the HTML the host sent. That is also why a translation cannot lose links, images or tables — they are never touched.
|
|
102
|
-
|
|
103
|
-
**Some sites refuse automatic fetches, and the button says so honestly**: measured, `openai.com` answers **403 to every non-browser request** (with a proxy, without one, and with a browser User-Agent — it returns a bot-challenge page). That is not an extraction failure and no retry fixes it, so for a 401/403 the detail view says "this publisher refuses automatic fetches" instead of "could not extract". **The status code is not the only signal**: an `openreview.net` PDF link was measured answering `302 → /challenge?redirect=…` and then a **200** challenge page — trusting the status code stores that page as the article, which is exactly how "I added it and there is no content" happened. So the decision is a combination: whether the final URL moved, whether the visible text is too short to be a page, and the words a bot or login page leaves behind; any hit means the payload is NOT stored as a body and the link is saved as 「仅链接」. Newsletter platforms and paywalls behave the same way.
|
|
104
|
-
|
|
105
|
-
**Some feeds publish full text for some posts and only a summary for others**: measured on the OpenAI alignment feed, whose 27 entries run from **155 characters** (a one-line summary) to **62,044** (full text). Nothing is truncated; that is how it is published. **A summary is no longer mistaken for the article**: the feed's `<description>` used to be promoted into `contentHtml`, so the entry looked complete — the backfill skipped it, opening it fetched nothing, and the detail view rendered those 167 characters as the whole paper. Such entries are now flagged `summaryOnly`: opening one fetches the real article on the spot (the detail view says 「这条订阅源只发布了摘要 —— 全文在原文页面上」 while it does, with the original-page link, and the line disappears when the text arrives) and the automatic backfill counts them as text it still owes. **That fetch now happens only while the entry has no cached body**: `summaryOnly` is a property of the FEED entry and is true forever, so this used to re-fetch an already-fetched article on EVERY open — replacing its DOM, and its translation, with the same bytes. The fetch is now gated on the text on screen being the feed's own (`fromFeed`).
|
|
106
|
-
|
|
107
|
-
**The host hands over at most 100,000 characters per payload, so an oversized feed can only be read in part**: this is the egress seam's default (`web-fetch-http`'s `maxBodyChars`), NOT this package's body budget (2 MiB per source, 12 MiB total — see `MAX_BODY_CHARS_PER_SOURCE`). The acceptance instance's feed is 646,905 characters; the host supplies the first 100,000, so only the two entries falling inside that prefix are readable. The plugin **cannot** work around it: `ctx.web.fetch` accepts `{ url }` only — no Range, no custom headers, no pagination — and vendoring `fetch` to bypass it would bypass the host's egress policy, which is not an option. Reading such a feed whole needs **no host code change**: the official patch overlay adds config to that row (`--patch <file>`, or the machine-level `$DSH_HOME/cordis.patch.yml` which applies to every profile):
|
|
108
|
-
|
|
109
|
-
```yaml
|
|
110
|
-
- id: web-fetch-http
|
|
111
|
-
config:
|
|
112
|
-
maxBodyChars: 2000000 # default 100000
|
|
28
|
+
@khorsheed/dsh-learning-suite
|
|
113
29
|
```
|
|
114
30
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
**Full text is filled in automatically — there is no button to press**: after each load the host reports which entries still lack text (at most 8 per run; a failed entry is left alone for 6 hours), and the browser half fetches + extracts them into a 24-hour cache (configurable on the subscription page), **two requests at a time**. An entry being filled keeps showing the feed's own text, the footer reports "Filling in 3/8…", and a completed card carries a "Full text" badge (meaning: the card still shows the feed's own summary, and the full text is cached — open it to read). **Do not want to wait for the slice? Open that entry**: if no body is held locally, opening it fetches once right there and says 「正在抓取正文…」 instead of waiting for the backfill's turn.
|
|
118
|
-
|
|
119
|
-
- Why not fetch them all at once: that would crawl every article of every subscription. This takes a slice per refresh and moves forward.
|
|
120
|
-
- **What arrives is what is shown**: a publisher refusing us (401/403), an unreachable host, and an unreadable page are three different sentences, and the body area always shows the feed's own content plus "read the original" — never a blank page.
|
|
121
|
-
|
|
122
|
-
**A feed the host truncates is salvaged, not discarded**: the egress seam caps a payload by CHARACTER COUNT, and a cap that lands inside a tag makes the whole document malformed — which used to mean **no entries at all**. The parser now keeps every complete `<item>`/`<entry>` (re-wrapping each fragment with the document root's namespace declarations, without which a fragment carrying `content:encoded` is itself malformed), repairs the cut-off block best-effort and flags it **partial** so its arrived text still renders, and the wall shows an "incomplete" line with the original-page link. The shape is pinned by a committed fixture (`tests/fixtures/feed-truncated.xml`). Measured: the acceptance instance's 646 KB feed (of which the host hands over 100,000 characters) now yields **2 cards** — one complete 42 KB body and one partial — where before this fix it yielded **0**.
|
|
31
|
+
The Reader comes with the Learning Suite, along with local transcription and Capture. Then start a new session, pick "学习模式" (learning mode), open "Inspiration" in the right sidebar and paste your first link.
|
|
123
32
|
|
|
124
|
-
|
|
33
|
+
The Reader only shows up in learning-mode sessions; not seeing it in other modes is expected.
|
|
125
34
|
|
|
126
|
-
|
|
35
|
+
## Known limitations
|
|
127
36
|
|
|
128
|
-
|
|
37
|
+
- One-click translation uses the browser's built-in translator, so only desktop Chrome 138+ / Edge 148+ can translate new pieces; phones and other browsers can only view translations already made.
|
|
38
|
+
- PDFs have no readable text and are saved as links only; pages that need a login can't be fetched either.
|
|
39
|
+
- Very long articles may come back only partly; it tells you so and offers "Read the original".
|
|
129
40
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
**A body over 256 KiB is stored as its own file**: `state.json` keeps only its metadata (file name and character count) and the markup itself lives in `$DSH_HOME/state/dsh-reader/bodies/<hash>.html`. The article-retention setting (12/24/168 hours or forever) governs both kinds identically, and eviction deletes the file with the metadata. The reason is that `state.json` is read and rewritten as a whole: a paper whose figures are inlined as base64 extracts to tens of megabytes, and inlining those would make every reader operation pay a multi-megabyte round trip.
|
|
133
|
-
|
|
134
|
-
**A repeated title at the top of the body is dropped**: the detail view already renders the entry title, and the article's own HTML often carries it again (measured: TWO `<h1>`s, preceded by the site name and a logo link). Extraction removes top-level headings that repeat the document title, plus the leading masthead and blank lines, so the body starts at real content ("Authors").
|
|
135
|
-
|
|
136
|
-
**The default filter is "all", not "today"**: a feed's newest entry is often not from today (the acceptance instance's feed was 8 days old), and a default `today` filter made a freshly subscribed wall look empty — which reads as "the subscribe did not work". There is **no today filter at all** now: the four ways to narrow the wall (unread / by source / by kind / by tag) live in the filter popover, and the button's own pressed state plus the `#sourceId` / `#rss` / `#link` / `@tagId` query it writes into the search box keep "what is hidden" visible.
|
|
137
|
-
|
|
138
|
-
**Degrade matrix**: no `fs` → state is memory-only and lost on restart (the handshake reports `hasFs: false` rather than pretending persistence exists); no `web` → a pasted link is still saved, marked 「仅链接 (unreachable)」 with 「阅读原文」, and no exception is thrown and no URL is lost; no `sideChat` → the side-chat action is hidden; no `quote` → the selection overlay is simply absent, while the entry-level Quote button keeps working; no capture package → the 「渲染抓取」 action on the script-figure notice does not render; a host below 0.1.6-alpha.2 (no `browser` right-Sidebar tab kind) → the spots that want a real browser (the script-figure notice, the bot-wall card action) offer the ordinary external link instead, without an error.
|
|
41
|
+
## Compatibility
|
|
139
42
|
|
|
140
|
-
|
|
43
|
+
Requires DSH 0.1.5-rc.1 or later; verified on 0.2.1-rc.3.
|
|
141
44
|
|
|
142
|
-
##
|
|
45
|
+
## For developers
|
|
143
46
|
|
|
144
|
-
|
|
47
|
+
- Every feature in detail, the surfaces, the internals of fetching / translation / storage, degradation and where data is stored: see the [full reference](https://github.com/Khorsheed/dsh-plugins/blob/main/packages/reader/docs/reference.en.md).
|
|
48
|
+
- Entry points: `src/index.ts` (service and sidebar), `src/tool.ts` (model tools such as `reader_read`). You can also install it alone (`@khorsheed/dsh-reader`, or `dsh plugin --profile web add @khorsheed/dsh-reader`), in which case it shows in every session; local transcription and rendered fetch then need `@khorsheed/dsh-transcribe` and `@khorsheed/dsh-capture` installed separately.
|
|
49
|
+
- Part of [dsh-plugins](https://github.com/Khorsheed/dsh-plugins) — file issues and contributions there; changes are in [CHANGELOG.md](CHANGELOG.md).
|
package/README.i18n.yaml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Bilingual-pair consistency record: the git blob hash of each side as of the
|
|
2
2
|
# last confirmed-consistent state. Both languages carry equal authority; after
|
|
3
3
|
# editing either side, bring the other along and re-record with:
|
|
4
|
-
# pnpm run verify-translation-pairing --write packages/
|
|
5
|
-
packages/
|
|
6
|
-
packages/
|
|
4
|
+
# pnpm run verify-translation-pairing --write packages/reader/README.en.md
|
|
5
|
+
packages/reader/README.en.md: 9ca4776636fe7a02a0b61a297f8fc9b3f3cbf391
|
|
6
|
+
packages/reader/README.md: 035bb00f372759cb5292bc4cb537c86cb23f1246
|
package/README.md
CHANGED
|
@@ -2,143 +2,48 @@
|
|
|
2
2
|
|
|
3
3
|
[English](README.en.md) | 中文
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
把想读的文章、视频、播客收在一面墙上,就在 DSH 里读、记笔记,再和 agent 一起把它读透。
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
刷到的论文、文章、视频散落在浏览器标签页里,想跟 agent 聊还得复制粘贴,聊完也没留下什么。装上灵感空间,把链接粘进来、或订阅一个 RSS,右侧栏「灵感空间」里就多一张卡片;点开就能读,外文一键翻成中文。读到要紧的句子划一下记成笔记,带着笔记跟 agent 讨论,最后让它考考你讲不讲得清楚,结论都记回笔记里。
|
|
8
8
|
|
|
9
|
-
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-1.png" width="
|
|
9
|
+
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-1.png" width="560" alt="灵感空间的墙:订阅与保存链接的卡片流,表头一行是筛选、排序、整墙翻译、刷新、设置与新增,搜索框独占一行">
|
|
10
10
|
|
|
11
|
-
##
|
|
11
|
+
## 能做什么
|
|
12
12
|
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
17
|
-
-
|
|
18
|
-
-
|
|
19
|
-
-
|
|
20
|
-
- **订阅管理与最近阅读**——管理页里每个源一行:名称与订阅地址就地编辑、单独刷新、暂停/恢复、删除,缓存策略(正文保留时长、译文预算、清空译文缓存)与存储读数同页;「最近阅读」落盘 100 条、重启还在,滚出 feed 窗口的条目也能接着读。
|
|
21
|
-
- **离开再回来接着读**——打开的文章、按内容锚定的阅读位置(第几个块、块内第几个字符)、墙上的筛选与译文,跨 dsh 会话由整个页面记住;位置另写 `sessionStorage`,页面重启后仍在原处。
|
|
13
|
+
- **粘个链接就收下**:点「新增灵感」粘进来,是订阅源就订阅、是文章就存一篇;订阅每天自动刷新,转载的同一篇只留一张卡。抓不到的也会存下,并写明原因。
|
|
14
|
+
- **视频和播客也能读**:YouTube、B 站、播客和小宇宙都能放进来;有字幕就用字幕,没字幕就在本机转写成「转写稿」,多人对谈还会分出谁在说话。点一段时间就从那里播放。
|
|
15
|
+
- **外文一键翻成中文**:按句翻,点一句看原文;不满意机翻,点「让 Agent 润色译文」,它整篇重译一遍。
|
|
16
|
+
- **划一句就记笔记**:选中句子点「记笔记」,存下原文和你的想法;记过的句子在正文里有底色,点笔记能跳回原处。
|
|
17
|
+
- **带着笔记讨论**:点「带着笔记讨论」,笔记进了输入框,你发出去就能和 agent 聊;聊出的结论让它记回那条笔记下。
|
|
18
|
+
- **让 agent 考考我**:读完点「让Agent考考我」,它扮成没读过的学生,请你讲给它听,再对照原文只指出你没讲到位的地方。不打分,想再练就再点一次。
|
|
19
|
+
- **引用进对话**:选中正文能直接引用进输入框;装了灵感画布,笔记还能「同步到画布」变成一张灵感卡。
|
|
22
20
|
|
|
23
|
-
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-2.png" width="
|
|
21
|
+
<img src="https://raw.githubusercontent.com/Khorsheed/dsh-basic/main/docs/screenshots/reader-2.png" width="560" alt="详情页:正文以 DOM 文本渲染,地球开关点亮后按句对照翻译,点一句展开该句原文">
|
|
24
22
|
|
|
25
23
|
## 安装
|
|
26
24
|
|
|
27
|
-
|
|
28
|
-
dsh plugin --profile web add @khorsheed/dsh-reader
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
装完重启 web 实例后生效。卸载:
|
|
25
|
+
在 DSH 的「插件 → 添加插件」里输入下面的包名,点安装:
|
|
32
26
|
|
|
33
|
-
```sh
|
|
34
|
-
dsh plugin --profile web remove @khorsheed/dsh-reader
|
|
35
27
|
```
|
|
36
|
-
|
|
37
|
-
落盘只有一处目录:`$DSH_HOME/state/dsh-reader/`——`state.json`(订阅清单 + 最近 payload + 读者自己的标签与「最近阅读」记录)与 `bodies/`(超过 256 KiB 的正文各占一个文件;全局翻译记忆是其中的 `translation-memory.json`,超过阈值的条目译文映射也各占一个文件)。卸载后这个目录会留下——想彻底清干净就删掉整个 `dsh-reader/`。
|
|
38
|
-
|
|
39
|
-
## Compatibility
|
|
40
|
-
|
|
41
|
-
- npm 发布线(`@deepseek-ai/dsh@0.1.5-rc.1`):✅ 可用,一处设计内降级——`sidebar.right.pane.tab` 座位、`ctx.web.fetch` 出网缝在 0.1.5-rc.1 上完整;应用内 Sidebar Browser 出口(脚本插图提示、反爬卡片的动作)需要 0.1.6-alpha.2+ 的 `browser` 右栏 tab 类型,0.1.5 宿主上退化为普通外部链接。minHost `0.1.5-rc.1`;本插件不做 preset 自隐(不需要 `pluginInventory`)。
|
|
42
|
-
- 源码线(deepseek-harness master):✅(verifiedHost: 0.1.5-rc.1)。
|
|
43
|
-
|
|
44
|
-
注意:宿主抓取缝默认把解码后的正文截到 100,000 字符且**静默截断**——这是部署配置可以抬升的上限(见「实现原理 · 抓取缝与存储」里的 patch 叠加层写法),不是插件的正文预算。
|
|
45
|
-
|
|
46
|
-
## 已知限制
|
|
47
|
-
|
|
48
|
-
- **只订阅、不爬全文**——新增订阅只抓 feed 本身,不会为每条目预抓正文(存储会爆,而且那是在当爬虫);全文由自动补齐每次刷新补一片,或点开某条时就地抓一次。
|
|
49
|
-
- **PDF 与其它非网页文件没有正文**——宿主的出网缝只认 `text/html`、`text/*`、`application/json`、`application/xml`,`application/pdf` 会被直接拒绝;而正文提取本来就是 DOM 的,PDF 没有 DOM。所以 PDF 链接只能存成「仅链接」的卡片 + 「阅读原文」,本插件不是 PDF 阅读器。
|
|
50
|
-
- **跨站跳转不跟随**——宿主缝只报目标 origin、不报路径,所以换了主机的跳转(短链、ezproxy 之类的代理)一律如实报「跳到另一个站点」,不会跟过去猜。
|
|
51
|
-
- **未读圆点不落盘**——未读标记是会话级状态,刷新或重启后全部回到未读。
|
|
52
|
-
- **翻译只在 Chrome 138+ / Edge 148+ 桌面版可用**——Safari、Firefox、移动端没有 Translator API,这些浏览器里地球按钮不出现,其余功能不受影响(这是刻意的选择:一篇 5 万字符的长文走宿主 LLM 是几万 token + 十几秒 + 把文章再发给模型服务一次;端上翻译零成本、秒级、不出设备,质量是网页翻译级的 NMT)。
|
|
53
|
-
- **超大文章抽不全**——宿主缝默认 100,000 字符上限(实测:Anthropic 研究页 ~266k、theverge.com ~899k、newyorker.com ~1.97M 字符),超限文章显示「受限篇幅」提示 +「阅读原文」;需要全文时由部署方抬升宿主缝上限,插件自己不会绕开宿主的出网策略。
|
|
54
|
-
|
|
55
|
-
## 实现原理
|
|
56
|
-
|
|
57
|
-
<details>
|
|
58
|
-
<summary>各个面(灵感墙 / 详情 / 翻译 / 新增 / 订阅管理 / 最近阅读,点击展开)</summary>
|
|
59
|
-
|
|
60
|
-
> 包名、tab kind、Remote namespace 都还是 `reader` —— 它们是稳定标识(改了等于改部署的 loader 行和落盘目录),「灵感空间」是**用户看到的名字**。
|
|
61
|
-
|
|
62
|
-
| 面 | 内容 |
|
|
63
|
-
| --- | --- |
|
|
64
|
-
| 灵感墙 | **内容优先**:第一屏就是条目卡片流。**搜索框独占一行、铺满整行宽度**,筛选 / 排序 / 整墙翻译三个控件都移到**表头那一行**(和刷新、设置、加号在一起)——搜索框夹在三个按钮之间时,窄侧栏里连自己打了什么都看不全;每个弹层仍然挂在**打开它的那个按钮**下面,不会盖住另一个按钮。**文字搜索按相关性排**:标题完全命中 → 标题包含 → 其它字段命中,同一档里才用你选的排序(`#源id` / `@标签id` 这类选择器不参与排序),所以「搜一个标题却先出来别的」不再发生;搜索生效时底部还会写明**匹配了多少条**(`匹配「X」N 条`),列表有没有被筛过一眼可见。筛选是一个**两级弹层**,按「阅读状态」(未读)、「按来源」、「**按类型**」(订阅源 / 保存的链接)、「按标签」四种方式收窄墙:根页是阅读状态两行(只看未读、**隐藏重复**)、`按来源` 钻取行(那一行带着**当前值**,所以不存在「被藏起来的状态」)和类型三行。**隐藏重复默认开**:同一篇文章被多个源转载时折成一张卡(三层归并,任一命中即同组:同条目 id——共享 guid 按设计共享抓取/已读/译文;链接归一化——剥协议/www/utm 类追踪参数;折叠标题 + 同一发布日;绝不模糊匹配),幸存卡带「N 个重复来源」徽章(悬停列出其它来源),读过一个副本整组即已读,底部注明隐藏了几条,自动补抓也不为被隐藏的副本花请求,点进去是来源页——独占弹层高度、自己滚动、来源超过 6 个时出现自己的搜索框;**所以订阅再多,弹层第一页也恒定是几行**。筛选用的是和搜索同一套本地谓词(D14),所以不花请求,值就写在搜索框里(`#源id` / `#rss` / `#link` / `@标签id`)、看得见也能清掉——**搜索框里有内容时框内右侧出现一个 ×**,点它一次清空(筛选值和手输的关键词一起清)。**墙上也能整墙翻译**:工具条最右的地球(和详情页那个各自独立,共享同一份翻译记忆)。开启后**逐字段判语言**——中文标题/摘要原样保留、混合卡片只翻英文那一半,绝不中译中;只翻**进入过视口**的卡片(IntersectionObserver 喂队列、每批 6 张、按语言分组、沿用正文那套哨兵批量+校验+逐条兜底),滚回去不重复请求,进详情也不再翻一次标题。下拉两档:只看译文(**鼠标悬停在被翻译的文字上**时只把**那个字段**换回原文——标题换标题、摘要换摘要,卡片空白处/瓷贴/箭头一律不动,便于核对又不神经质)/ 双语对照(每个字段的原文常驻在译文下方,且**沿用该字段自身的排版**——标题原文就是标题字号与行高,靠墨色和字重与译文区分,不缩成小注)。**卡片文字盒是固定的**:标题与摘要各按两行预留高度并双向 clamp,所以译文/原文互换(hover 或对照)都不会改变卡片高度、网格不跳动;只有「双语对照」是显式切换、会让网格整体变高一次。条目卡片是**有底、有描边、12px 圆角的卡**:悬停只提亮不位移,键盘聚焦有环;未读条目标题上墨、瓷砖角上一个**圆点**(7px,会话级、不落盘),已读条目留在原位、只是安静一档;feed 自己的分类与读者自建标签共用同一套药丸尺寸(自建标签带描边、可折行),打标签按卡片上的「+」打开面板——里面勾选已存在的标签,或**输入名字回车新建**(原生 datalist 已换成可样式化的行);**标签在筛选弹层的「按标签」区可以删除**(每行右侧一个安静的 ×,删掉即从词表和所有条目上移除,正在用它筛选时收窄也一并清掉)。**保存的链接如果读不到正文,卡片上有一个「仅链接」小徽章**(虚线、安静),悬停与详情会说明原因(见下方「只能存链接的链接」)。**链接卡片的标题在正文解析出来后升级成文章自己的**(正文第一个 h1,否则文档标题;另带第一段实质文字当摘要),随正文写进宿主标注——正文被预算淘汰后卡片仍读这份记忆,一墙论文不用一篇篇点开。没有 `publishedAt` 的保存链接按**它的源加入时间**参与时间排序,所以刚加的那条在最上面,不会沉底。底部一行**新鲜度**:「刷出于 3 小时前 · 每日 10:00」,让你决定要不要按刷新 |
|
|
65
|
-
| 详情 | 点卡片进来。正文以 DOM 文本渲染——**这正是划选引用能工作的原因**,也是这个面不放 iframe 的原因。正文**没有阅读量度**:侧栏给多宽就铺多宽,段落和列表共用同一条右缘。工具条:返回、**翻译(地球 + 视图下拉)**、复制链接、在浏览器打开原文,**单条保存的链接另有一个删除按钮**(订阅源的条目没有:下次刷新它还会回来,给「删除」是假承诺);标题下的**标签行里还有「重新抓取」**(和卡片药丸同一个宿主动作,用它换一份新副本时会清掉这篇的译文记录——正文要换成新的 DOM)。正文下面列"同一来源"的其他条目。**页面用脚本画的插图抓不到**:这类 `<figure>` 在服务器 HTML 里只有一个空 div 和一句图注(实测 transformer-circuits.pub 这篇有 84 张),抓取不执行脚本,所以**图注保留在正文里**(它是页面发布的文字,写明了画的是什么),正文上方另给一行「这一页有 N 张插图由页面自己的脚本绘制,抓取时拿不到画面(图注保留在正文里)—— 阅读原文」(宿主有 Sidebar Browser 时这个出口开在应用内;未来 capture 包在场时这里还会出现「渲染抓取」——探测不到就不渲染)。**capture 渲染回来的图能进正文**:页面脚本绘制的图以「内联 SVG + 内联样式」到达,归一化按安全子集保留(形状/文字/defs/渐变/裁剪/symbol 复用;`script`/`foreignObject`/SMIL/事件/外部引用永不放行,`url(` 只许本文档片段)——而不是只剩图注。**组合图的结构同样保**:capture 渲染的组合图(图 + 样式盒搭的属性卡)在 figure 里保留 div/span 容器与白名单样式(布局/绘制属性;`position:fixed`、`z-index` 超限、外部 `url()`、脚本把戏全剥),而且任何被解包的块级容器都留下词边界——相邻文字绝不粘连。**但静态标记里就有 URL 的图会回收**:懒加载站点把真 URL 放在 `data-src` 等属性、`srcset`、`<picture>` 的 `<source>` 或 `<noscript>` 的 no-JS 兜底里(`src` 里只是 1px 占位图),抽取按这个顺序解析候选、取 srcset 里最大的一张;**反过来,把真图直接内联成 base64 data URI 的页面(transformer-circuits 一类)也保**——占位图与真图按载荷大小分道(阈值 512 字符,1px gif 约 70、真图表数千起),小的照旧当占位、穿透到懒加载属性,大的原样保留;正文输出的每个 `<img>` 都带 `referrerpolicy="no-referrer"`(防盗链的 CDN 否则直接 403)。**还有 403 之外的一种拦法**:源站回 `Cross-Origin-Resource-Policy: same-origin`(实测 claude.dev 的全部 `/media` 插图)时,浏览器跨源拿不到像素——直接抓明明 200,页面上却只有裂图加整句 alt。这种图改由宿主侧代抓(公共地址校验、连接钉死在已校验地址、每跳重验、8 MiB 上限、`image/*` 类型闸——宿主自己的 `ctx.web` 缝只给文本体,给不了二进制):正文里加载失败的 `<img>` 会被改指到宿主取回的字节(data: URI),加载正常的图永不走这条代理,宿主也取不回来的保持裂图原样(「阅读原文」仍是出口)。**LaTeXML 的矢量图也保**:arxiv HTML 把图嵌成 `<object type="image/svg+xml" data="…">`,归一化把它改写成带原始宽高的 `<img>`;非图片的 object(PDF、视频)照旧整棵丢弃。**公式是原生渲染的 MathML**:arxiv HTML 版的 `<math alttext="…">` 按安全子集放行(`annotation-xml` 与任何脚本永不放行——前者是 HTML 注射通道),块级公式横向滚动而不是顶破排版 |
|
|
66
|
-
| 翻译(浏览器本地) | 工具条上的**地球**是开关:点亮 = 正文显示译文。它右边的下拉三档:**只看译文**(默认,正文上方一条一次性提示说明「点任意一句可看原文,再点收起」)、**双语对照**(每段下方常驻原文小字,可整段折起)、**只看原文**。翻译单元是**句子**:点一句只展开那**一句**的原文(段落下、左侧细线 + 高亮),再点收起,可同时展开多句;**即使这句话中间夹着链接、被切成几段文本,展开的仍是完整的那一句**(不会显示成碎片),**原文那一行不是控件**(它是给选中引用用的文本,点它不收起;点句子才收起,而且**正在选区里的点击会被忽略**,所以拖选原文或译文都不会把展开的原文收掉)。英文原文按**正文级**排版(13.5px、secondary 墨色、行距 1.62、行间 3px);它**按容器位置分别排边距**(段落旁:贴住所属段落,上 4px / 下 14px;列表项、引用、表格单元格里则在译文下方 5px,**不会被负边距拽上去贴住译文**),**已展开的那一句保留一条很细的下划线**(只有下划线,没有常驻底色/竖条——那是「这句绑定了下面那行原文」的安静标记);配对高亮**只在 hover 时**出现:鼠标移到某句中文上,这一句(含被链接切开的各段)和它对应那一行原文一起高亮;移到原文行上同理反向。**默认不画任何底色或竖条** —— 试过的常驻标记会把整篇点成花的。**全程在浏览器里跑**(Chrome 138+/Edge 148+ 桌面版的 `Translator` API),模型不出设备、不花 token;**译文的句子对会落盘**(见下),地球开关与翻译会话不落——没有这个 API、语言对不可用、或正文本来就是中文时,地球**不出现**。**译文两层落盘**(`$DSH_HOME/state/dsh-reader`,与缓存正文同一目录、同一敏感级、同一个卸载故事):全局**句子记忆**(按 `<源>→<目标>:<句子哈希>` 存原文+译文,5 万句上限)+ **按条目的段落映射**(哈希→译文,键在正文哈希上,正文小幅重抓后逐句复用仍在);合计预算默认 64 MB(订阅管理页的缓存策略同一个动词可调),按「最近使用」跨两层淘汰,**没有 TTL**(重建一份译文要一次手势加逐句模型,不该按小时过期)。效果是:刷新后地球照样等那一下手势,手势之后整篇从磁盘重绘、模型不沾;句子在两份 schema 版本失配时读作 miss、惰性淘汰(浏览器模型升级后不会把旧译文当成新模型的产出) |
|
|
67
|
-
| 新增(弹窗) | 表头加号、空态卡片、墙上尾部虚线卡片三处都能打开同一个**弹窗**(成员 tab 的邀请弹窗同款做法:透明遮罩 + 居中卡片,Esc / 点遮罩关闭),因为「新增」是个瞬时动作,墙没必要被它替换掉。粘进去的地址**存成什么由抓回来的内容决定**(D15):是 feed 就建订阅,是网页就只存这一篇。**arxiv 的 abs/pdf 链接自动升级到官方 HTML 版**(`arxiv.org/html/<id>`,LaTeXML 生成的语义标记:真表格、真公式),HTML 版不存在(404)时回退到粘进来的地址照常走。**DOI 链接先解析成 arXiv 版再落地**(Crossref 取标题与作者 → arXiv 标题搜索 → 标题相似度 + 作者姓重合的双因子闸;配不上就回普通路径,绝不猜),命中时卡片标注来源 DOI;**OpenReview 链接不抓、直接存仅链接**(实测服务器端读不到:API 在 Cloudflare 挑战后、论坛是 JS SPA),原因和建议写在卡片上(有 arxiv 版就粘 arxiv 链接)。**抓不到也一定存下来**:判定从「成功 / 失败」改成「已保存 + 能不能预览」,原因分档写在弹窗里(反爬 / 需要登录 / 不是网页 / 跨站跳转 / 连不上 / HTTP 错误),对应的卡片标成「仅链接」——把读者粘的 URL 丢掉,比留一张只能打开原文的卡更坏 |
|
|
68
|
-
| 订阅管理(页面) | 表头齿轮进入**订阅管理页**:顶部是一行**压缩设置**(每日刷新时间 + 正文保留并排,长说明收进悬停提示)与一行**存储区**(原文缓存读数「N 篇 · 体量」、译文缓存读数「M 篇映射 + S 句 · 体量」、**译文预算**输入(MB,blur/回车提交)、两步的**清空译文缓存**——读数是宿主聚合的,表本身不过线)与两排带小标题的**筛选片**(`按类型`:全部 / 订阅源 / 保存的链接,各带条数;`排序`:按添加时间 / 按名称 / 按抓取时间,默认添加时间倒序),下面每个源一行,**名称与订阅地址都可以就地编辑**(改地址 = 换一份文档,保存后立刻重抓,名称留空则回退成 feed 自己的标题)、类型、条目数、上次抓取时间、失败原因(按分类说人话,不是宿主缝原话)、单独刷新、暂停/恢复、删除,以及**每日刷新时间的配置**。这些问题是关于「源」而不是关于「内容」的,所以它们不在墙上 |
|
|
69
|
-
| 最近阅读(页面) | 表头**时钟**进入**最近阅读页**:**按打开时间倒序**列出读过的条目(标题 + 来源 + 「多久以前」),点一条**接着读**;右上角一个垃圾桶**清空这份列表**。它记的是「我刚才在读什么」,所以**落盘**(`state.json` 的 `recent`,**最多 100 条**,重开同一条只会把它挪到最上面,不会重复),刷新/重启都还在——这与面板内那份记忆(打开的文章、阅读位置、译文)是两件事:那份**跨 dsh 会话共享、但不跨刷新**(按页面共享,见上),`recent` 则是部署级、重启也在。**一条记录只存条目 id、来源 id、标题、URL、时间**,不存正文;因此即使那条已经**滚出 feed 的窗口**(源还在),这一页依然能把它打开。源被删掉的条目仍然列出,但不可点,并说明只能去原文地址开 |
|
|
70
|
-
|
|
71
|
-
**卡片上的「抓取」**:每张卡在标签行里有一个状态药丸——**灰=还没抓(点一下就开始)/ 蓝=抓取中或已抓待解析 / 绿=已有全文(点开秒读)/ 红=失败**。失败态**按原因给动作**:点红色药丸开一个浮层(原因 + 该原因对应的动作——连不上(网络类)给「重新抓取」;反爬 / 登录墙 / 不是网页 / 服务器端读不到给**在真浏览器里打开**(宿主有 Sidebar Browser 就开在应用内,否则外部浏览器);HTTP 错误 / 空页 / 跨站跳转是终局,只有原因、没有动作),卡片本身保持一行高,不把墙撑破。点「抓取」之后**可以立刻离开这个页面**(切到别的面板、关掉标签、刷新都行):抓取跑在**宿主**上,原始网页先落到 `bodies/`,下次打开灵感空间时由浏览器半抽出正文再缓存。所以你能先在墙上把想看的几篇点掉,再慢慢读,不必在详情页等。**进入详情时如果这篇还没抓,仍然会自动抓一次**(和点卡片上的「抓取」是同一个动作);**已经抓过的就不再重抓**——屏上那份就是缓存里的正文,重抓只会白白把 DOM 换掉、连读者已经翻好的译文一起冲走。**feed 自己发布全文的条目,第一次打开就把这份全文存成正文**(零请求——文字已经在屏上了),药丸在同一次手势里转绿,条目滚出 feed 窗口后正文还在。想换一份新副本,用**详情页标签行里的「重新抓取」**(同一条宿主路径,并清掉这篇的译文记录,见下)。
|
|
72
|
-
|
|
73
|
-
**离开再回来、换个会话接着读**:宿主在另一个主面板接管时会**卸载整个右侧栏**(`RightbarRoot` 只在没有活动主面板时渲染,不是藏起来),而这个面板的 store 每次挂载新建——所以「切到侧边对话再切回来」以前会把打开的正文、阅读位置、墙上的搜索/排序/筛选,以及**已经翻好的译文**全部丢掉。现在这些由**整个页面**记住并恢复:**跨 dsh 会话**——在另一个对话里打开灵感空间,仍然是同一篇文章、同一个阅读位置、同样的收窄和同样的译文——包括**墙上整墙翻译的开关与它翻好的卡片**(面板按会话挂载,这份记忆按页面共享);**同一会话内**从侧边对话回来也一样。恢复时那篇文章被重新打开(正文由宿主缓存作答,不花请求),阅读位置回到原处(**墙和正文各自的位置都算**;正文里的位置按**内容**记——「第几个块、块内多少像素、**块内第几个字符**」而不是一个像素数——因为文章自带的图片没有尺寸、会边加载边把文档撑高;正文被替换或重抓落地后会重新落回原处,**原文↔译文切换改写块的高度时,也会按字符偏移重新落回同一句**),**译文在本页已经建过翻译会话时自动重新点亮地球**(`Translator.create()` 需要用户手势,所以只有本页已有的会话才可能自动恢复;没有会话就不自动恢复,但记录还在,点一下地球依旧零成本)。**译文跟着正文走**:正文在读者脚下被替换(重抓落地、缓存过期后重取、元素被重建)时,只要这一条的地球是亮的,译文会自动重新贴回新正文(靠页面内的句子记忆,通常不花请求)。**刷新(或页面重新初始化)之后,位置还在**:「读到哪」这份**位置**(打开的条目、正文里的锚点、墙的偏移)写进 `sessionStorage`,所以页面重启后文章和阅读位置都会回来;**地球记录与翻译会话不落盘**(没有可用于恢复的会话,而新建一个需要用户手势),刷新后地球是灭的——但**译文的句子对落盘了**:点一下地球,整篇从宿主的两层存储重绘,模型一个请求都不发。**关掉标签页即清空**(`sessionStorage` 的作用域就是这一个标签页;宿主目录里的译文随预算淘汰,或随卸载整个目录一起删)。
|
|
74
|
-
|
|
75
|
-
**点击语义(定案方案 B)**:点卡片 = 进详情页。大多数阅读在 dsh 里完成;正文太长或想直接看原文时,详情页自己的按钮会跳浏览器。卡片尾部的箭头是"这里能进"的提示,不是控件。
|
|
76
|
-
|
|
77
|
-
**引用**:正文是应用级选区,所以 `@khorsheed/dsh-quote` 的浮层菜单**本来就覆盖它**——本插件不自己造选区菜单。详情页底部另有:
|
|
78
|
-
|
|
79
|
-
- **引用**:选区(没选区就是整条)+ 来源标签,经官方输入机合成进当前会话草稿(读现有草稿合并,绝不覆盖、绝不发送);
|
|
80
|
-
- **引用到侧边对话**:只有组合里真有 `sideChat` 服务时才显示(能力握手说了算),否则这一项隐藏。
|
|
81
|
-
|
|
82
|
-
</details>
|
|
83
|
-
|
|
84
|
-
<details>
|
|
85
|
-
<summary>内部机制(架构、抓取缝、翻译、存储,点击展开)</summary>
|
|
86
|
-
|
|
87
|
-
**架构。** 抓取在**宿主半**完成(走 `ctx.web` 这条官方出网缝),浏览器半只做解析和渲染:宿主半提供 `reader` 服务核心(`ctx.provide('reader')`)并挂出薄 Typert Remote 面(namespace `reader`),订阅清单与最近一次抓回的原始 payload 落在 `$DSH_HOME/state/dsh-reader/state.json`(用 `node:fs` 原子写,不走 `ctx.fs`,见下),每日按点自动刷新由插件自带的 `setTimeout` 驱动(宿主没有调度服务可骑)。浏览器半两段式注册右侧栏:先把自己铸造的 tab 类型 `reader`(实现 id `@khorsheed/dsh-reader`,带 guide 卡片)注册进 `sidebarRightTabs` 注册表,再经 `slots.inject('sidebar.right.pane.tab', …)` 把面板本体挂到 keyed 座位;中英文字典随插件注册;Remote 经 `ctx.remote.$mount` 自行挂载。feed 解析必然在浏览器半——宿主侧没有 XML/DOM 解析器(`globalThis.DOMParser === false`),`getBodies` 只把原始 payload 交过去。没有 `static inject`:每个可选能力(`fs`、`web`、`sideChat`、`browser` tab 类型、capture Remote)都在使用时探测,缺席即降级,绝不抛错。
|
|
88
|
-
|
|
89
|
-
**状态为什么不走 `ctx.fs`**:`ctx.fs` 是**沙箱**文件系统,每一次写都由**调用会话**的文件策略把关;workspace-write 会话写 `$DSH_HOME/state` 会被拒(`FS_SANDBOX_DENIED`,验收实例上真实发生过)。这道栅栏本身是对的、要留着;错的是把「属于部署」的状态交给一个「属于会话」的能力。所以本包按仓库里其它宿主态包的写法(`packages/lab/src/state.ts`)用 `node:fs` 原子写(临时文件 + rename),目录不可写时降级为仅内存。
|
|
90
|
-
|
|
91
|
-
**Remote 面。** namespace `reader`,verb 分组:源管理 `capabilities` / `listSources` / `addSource` / `updateSource` / `removeSource` / `refresh`;正文 `getBodies` / `getEntryBody` / `fetchEntryBody` / `storeEntryBody` / `getRawBody` / `entryFetchStates` / `listBackfillCandidates`;译文 `getEntryTranslation` / `getSentenceTranslations` / `rememberSentences` / `clearTranslations`;标签 `entryTags` / `listTags` / `createTag` / `tagEntry` / `renameTag` / `deleteTag` / `pruneTags`;阅读记录 `recordRead` / `listRecent` / `clearRecent`;缓存策略与读数 `getCachePolicy` / `setCachePolicy` / `getStorageStats`;以及 `quoteToSideChat`。宿主半**不解析** feed——解析在浏览器半。
|
|
92
|
-
|
|
93
|
-
**来源名用 feed 自己声明的标题**:宿主半不解析 feed(它没有 XML 解析器),所以新建时只能给一个从 URL 推出来的名字(就是你看到的那串 `raw.githubusercontent.com/...`)。浏览器半解析出 `<channel><title>` 后会用真实标题替换显示名,所以卡片上会出现「Anthropic Research」这样的名字,失败时回退成 URL。列表里报错时显示「{count} 天前」这种占位符也是同一类问题的另一面(参数没被插值),已修。
|
|
94
|
-
|
|
95
|
-
**翻译用的是浏览器自己的端上模型,不是宿主的模型**:`self.Translator`(Chrome 138+/Edge 148+,仅桌面版)在本机进程内翻译,所以**模型不出设备、不花任何 token、不联网**(首次按语言对下一次小语言包,`Translator.create()` 需要用户手势——点地球那一下正好是)。**译文的派生句子对会写到宿主侧**(`$DSH_HOME/state/dsh-reader` 下的全局句子记忆 + 按条目映射,默认 64 MB 预算按最近使用淘汰,无 TTL;敏感级与旁边缓存的正文相同,隐私论据从来是关于网络而不是磁盘)。
|
|
96
|
-
|
|
97
|
-
**翻译单元是句子,不是段落,也不是文本节点**:先按句切好再发出去,配对是**构造出来的真**,不是事后猜的——段落级翻译拿回来是一整段,模型会合句拆句,再对齐就可能给你看错原文。批量发送时句子之间插哨兵,返回的段数对不上就把那一批退回「一句一请求」,**代价是粒度,永远不是正确性**。跨行内元素的句子(例如句中有链接)会变成两个单元,链接本身不动;`<pre>`/`<code>` 里的内容从不翻译。
|
|
98
|
-
|
|
99
|
-
**翻译器建不起来时不会假装**:`Translator.availability()` 只是**提示**,不是承诺 —— Chrome 上确实出现过「availability 说能用、`create()` 却抛 `NotSupportedError`("Unable to create translator for the given source and target language")」的情况(macOS 构建有同类记录)。所以插件会:① 源语言先问浏览器的 `LanguageDetector`(猜错源语言正会让某些语言对建不起来,而那条报错**不说是哪个语言对**);② 目标语言按 `zh` → `zh-Hans` 两种写法依次真建一次;③ 全都以 `NotSupportedError` 拒绝时,**地球自行消失**并明说「这个浏览器无法翻译 en → zh / en → zh-Hans」,同时指向 `chrome://on-device-internals` 的 Broker State 自查;网络/配额这类**非永久**失败则保留地球以便重试,并报出尝试过的语言对与原因(浏览器原文里两个语言都不出现)。
|
|
100
|
-
|
|
101
|
-
**正文还原是逐字节的**:翻译只改文本节点(原文在内存里留着),「只看原文」和取消都能把正文还原成宿主发来的那份 HTML 原样。这也是为什么译文不会丢掉链接、图片、表格——它们根本没被碰过。
|
|
102
|
-
|
|
103
|
-
**有些站点拒绝自动抓取,界面会诚实地告诉你**:实测 `openai.com` 对**任何非浏览器请求都返回 403**(带代理、不带代理、换浏览器 UA 都一样,返回的是一个反机器人挑战页)。这不是抽取失败,也不是能靠重试解决的问题,所以详情页对 401/403 显示「这个站点拒绝自动抓取(原文地址对人以外的请求返回验证页)」而不是「无法提取正文」。**状态码不是唯一线索**:实测 `openreview.net` 的 PDF 链接是 `302 → /challenge?redirect=…` 然后 **200** 的挑战页——只看状态码会把挑战页当成文章存下来(这正是「加进去了但点开没内容」的根因)。所以判定是组合的:最终 URL 与请求 URL 是否变了、可见文本是否少到不像一页、以及反爬/登录页的痕迹词;命中任一条就不再存正文,改存「仅链接」。同类站点(newsletter 平台、付费墙)会有同样表现。
|
|
104
|
-
|
|
105
|
-
**有的 feed 只对部分文章给全文,其余只给摘要**:实测 OpenAI alignment feed 的 27 条正文从 **155 字符**(一句摘要)到 **62,044 字符**(全文)不等——这不是截断,是发布方就这么发的。**摘要不再被当成全文**:以前 feed 的 `<description>` 会被塞进 `contentHtml`,于是条目看起来「已经有正文」——自动补齐跳过它、点开也不再抓,详情页就把那 167 字的摘要当成整篇文章显示。现在解析器给这种条目打上 `summaryOnly`,**打开时照样就地抓一次真文章**(抓取期间正文上方写明「这条订阅源只发布了摘要 —— 全文在原文页面上」,抓到之后这句自动消失),自动补齐也把它列进待补清单。**重抓只在这篇还没有缓存正文时发生**:`summaryOnly` 是 feed 条目的属性、一旦为真就永远为真,所以以前这里会让已经抓过的文章**每次打开都重抓一次**(用同一份正文把 DOM 换掉,译文也就跟着没了);现在只有屏上那段文字是 feed 自己给的(`fromFeed`)才触发抓取。
|
|
106
|
-
|
|
107
|
-
**宿主一次只交接 100,000 字符,所以超大 feed 只能读到一部分**:这是宿主出网缝的默认上限(`web-fetch-http` 的 `maxBodyChars`),**不是插件的正文预算**(插件自己的预算是单源 2 MiB / 总量 12 MiB,见 `MAX_BODY_CHARS_PER_SOURCE`)。你那条 feed 有 646,905 字符,宿主只给前 100,000,所以 21 条里只有落在前面那部分的两条能读到。插件**无法**绕过它:`ctx.web.fetch` 只接受 `{ url }`,没有 Range、没有自定义请求头、没有分页,插件也不会为了绕过它去 vendor `fetch`(那等于绕开宿主的出网策略)。要让 646 KB 的 feed 完整可读,**不需要改宿主代码**:用官方的 patch 叠加层给那一行加 config 即可(`--patch <file>` 或机器级的 `$DSH_HOME/cordis.patch.yml`,后者对所有 profile 生效):
|
|
108
|
-
|
|
109
|
-
```yaml
|
|
110
|
-
- id: web-fetch-http
|
|
111
|
-
config:
|
|
112
|
-
maxBodyChars: 2000000 # 默认 100000
|
|
28
|
+
@khorsheed/dsh-learning-suite
|
|
113
29
|
```
|
|
114
30
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
**全文是自动补齐的,不需要你点**:每次加载后,宿主给出"还缺全文"的条目清单(一次最多 8 条,失败的条目 6 小时内不再重试),浏览器半一条一条抓取+抽取并写入缓存(24 小时,管理页可改),**同时最多 2 个请求**。补齐中的条目仍在墙上显示 feed 自己给的内容,底部一行进度「正在补齐全文 3/8…」;抓到的卡片标一个「已抓全文」(徽章的含义是:卡片上仍是 feed 自己的摘要,全文已经缓存好了,点开就能读)。**不确定要不要等?直接点开那一条**:打开时若本地还没有正文,它会就地抓一次并显示「正在抓取正文…」,不必等补齐轮到它。
|
|
118
|
-
|
|
119
|
-
- 为什么不是"全部一次抓完":那等于把每个订阅源的所有文章都爬一遍。这里是**每次刷新补一片**,滚动前进。
|
|
120
|
-
- **抓不到就展示已有的**:站点拒绝(401/403)、连不上、抽取失败,三种情况分开说明,正文位置永远显示 feed 自己发布的内容 + 「阅读原文」,绝不空白。
|
|
121
|
-
|
|
122
|
-
**被宿主截断的 feed 会抢救,而不是整份丢掉**:宿主缝按字符数截断,截在标签中间会让整份 XML 不合法——而"整份不合法"曾经等于**一条都没有**。现在解析器会:完整闭合的 `<item>`/`<entry>` 全部保留(用文档根上的命名空间声明重新包装,否则带 `content:encoded` 前缀的片段自己也是坏 XML);被截断的那条尽最大努力修复并标成**部分**,正文照常显示已经拿到的部分;墙上给一行「未完整」说明 + 阅读原文。配一个提交在仓库里的 fixture(`tests/fixtures/feed-truncated.xml`)锁住这个形状。实测:验收实例那条 646 KB 的 feed(宿主只给 100,000 字符)现在能出 **2 张卡**——一张完整的 42 KB 正文,一张部分正文——而在这次修复前是 **0 张**。
|
|
123
|
-
|
|
124
|
-
**站点拒绝 / 登录墙 / PDF 都只会变成「仅链接」**:抓回来的页面如果**几乎没有可读文本**(`< 200` 字符可见文本,去掉 script/style 后计)并且带着反爬或登录页的痕迹,就不当作文章存下来——**挑战页本身绝不落成卡片正文**。`application/pdf` 之类更是宿主的出网缝直接拒绝的类型(`WEB_UNSUPPORTED_CONTENT_TYPE`,只认 html/text/json/xml)。这些卡片照常出现在墙上、能打标签、能复制链接、能删除,点开只有原因 + 「阅读原文」。
|
|
31
|
+
灵感空间会跟着「学习套件」一起装好,本机转写和渲染抓取也一并带上。装完开一个新会话,选「学习模式」,在右侧栏打开「灵感空间」,粘进第一个链接。
|
|
125
32
|
|
|
126
|
-
|
|
33
|
+
灵感空间只在学习模式的会话里出现,在其它模式里看不到它是正常的。
|
|
127
34
|
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
**不做条件请求**:宿主缝没有自定义请求头入口,所以没有 ETag/If-None-Match,每次刷新都是完整抓取。**订阅登录墙后面的内容:不做**——没有凭证注入面。
|
|
131
|
-
|
|
132
|
-
**正文超过 256 KiB 会存成单独文件**:`state.json` 只留元数据(文件名 + 字符数),正文本体落在 `$DSH_HOME/state/dsh-reader/bodies/<hash>.html`。【正文保留】设置的 12/24/168 小时 / 永久对**两种正文一视同仁**,淘汰(超出「保留条数」上限或 TTL 到期后重抓)时会连文件一起删掉。这样做是因为 `state.json` 是一份整体读写的 JSON:一篇插图内联成 base64 的论文抽出来几十 MB,直接塞进去会让之后每一次读者操作都付出几 MB 的往返。
|
|
133
|
-
|
|
134
|
-
**正文顶部的重复标题会被去掉**:详情页已经渲染了条目标题,而文章自己的 HTML 里常常再来一两遍(实测那篇有**两个** `<h1>`,前面还跟着站名和 logo 链接)。抽取时会去掉与文档标题重复的一二级标题、开头的站头 chaff 与空行,正文从「Authors」这类真实内容开始。
|
|
35
|
+
## 已知限制
|
|
135
36
|
|
|
136
|
-
|
|
37
|
+
- 一键翻译用的是浏览器自带的翻译,只有电脑上的 Chrome 138+ / Edge 148+ 能新翻;手机和其它浏览器只能看已经翻好的。
|
|
38
|
+
- PDF 读不了正文,只能存成链接;要登录才能看的页面也抓不到。
|
|
39
|
+
- 很长的文章可能只抓到一部分,这时会提示,并给「阅读原文」。
|
|
137
40
|
|
|
138
|
-
|
|
41
|
+
## Compatibility
|
|
139
42
|
|
|
140
|
-
|
|
43
|
+
需要 DSH 0.1.5-rc.1 或更新版本,已在 0.2.1-rc.3 上验证。
|
|
141
44
|
|
|
142
|
-
##
|
|
45
|
+
## 给开发者
|
|
143
46
|
|
|
144
|
-
|
|
47
|
+
- 全部功能细节、各个面、抓取 / 翻译 / 存储的内部机制、降级和存储位置,见[完整参考](https://github.com/Khorsheed/dsh-plugins/blob/main/packages/reader/docs/reference.md)。
|
|
48
|
+
- 入口:`src/index.ts`(服务与侧栏)、`src/tool.ts`(`reader_read` 等模型工具)。也可以单独安装(`@khorsheed/dsh-reader`,或 `dsh plugin --profile web add @khorsheed/dsh-reader`),这时它在所有会话里都可见;本机转写和渲染抓取要另外装 `@khorsheed/dsh-transcribe`、`@khorsheed/dsh-capture`。
|
|
49
|
+
- 隶属 [dsh-plugins](https://github.com/Khorsheed/dsh-plugins),问题与贡献请到该仓库;变更见 [CHANGELOG.md](CHANGELOG.md)。
|