lifeos 1.0.2 → 1.0.3

package/assets/skills/digest/references/run-pipeline.en.md

@@ -0,0 +1,236 @@
+# Run Mode Pipeline
+
+When the user runs `/digest` or `/digest <topic>`, follow this pipeline to fetch updates and generate the digest.
+
+## Preflight
+
+1. Check that Python 3 is available: `python3 --version`
+   - unavailable → ask the user to install Python 3 and stop
+2. Scan `.md` files under `{system directory}/{digest subdirectory}/`
+   - no config files → automatically enter Setup mode (see `setup-guide.md`)
+   - topic specified → load only the matching file
+   - no topic specified → load every config file and run them one by one
+
+## Execution Pipeline (run per topic)
+
+### Phase 1: Parse the Config
+
+Parse the config note according to `config-parser.md` and produce structured data:
+
+```text
+config = {
+  topic: "LLM Agent",
+  period_days: 7,
+  language: "English",
+  modules: {
+    rss: { enabled, feeds },
+    papers: { enabled, sources },
+    web: { enabled, queries, sites },
+    huggingface: { enabled, keywords },
+    github: { enabled, keywords }
+  },
+  categories: [{ name, scope }]
+}
+```
+
+Legacy `arxiv` blocks are still accepted during the transition and are normalized into
+`papers.sources` before execution.
+
+Compute the date range:
+
+- `end_date` = today
+- `start_date` = today - `period_days`
+- `date_range_str` = `MMDD-MMDD` for the filename
+- `date_range_display` = `YYYY-MM-DD ~ YYYY-MM-DD` for the title
+
+### Phase 2: Fetch in Parallel
+
+Run enabled modules in parallel. RSS + arXiv use the Python helper; the rest use agent tools.
+
+#### Task A: RSS + paper sources (Python helper)
+
+For paper sources, the helper should use this runtime contract:
+
+1. normalize `Paper Sources` rows into source adapter inputs
+2. run the source adapters for `arXiv`, `bioRxiv`, `medRxiv`, `ChemRxiv`, `SocArXiv`, and `SSRN`
+3. return normalized papers plus structured per-source errors
+4. if one source fails, keep the successful sources and report the failure in `errors`
+5. legacy `arxiv` config blocks are converted into `arXiv` adapter inputs before execution
+6. the `arXiv` adapter retains the existing OpenAlex fallback behavior when arXiv lookups fail
+7. `SocArXiv` may normalize to `OSF` landing pages; `SSRN` prefers source-hosted SSRN links
+8. keep the transport low-budget: one primary request per source and no pagination
+
+Build the JSON input and send it through stdin:
+
+```bash
+echo '<json_config>' | python3 .agents/skills/digest/references/rss-arxiv-script.py
+```
+
+The payload should include at least:
+
+```json
+{
+  "language": "en",
+  "rss": {...},
+  "papers": {
+    "enabled": true,
+    "sources": [
+      {
+        "source_type": "arXiv",
+        "query": "\"llm agent\"",
+        "scope": "cs.AI",
+        "notes": "Core technical papers"
+      },
+      {
+        "source_type": "bioRxiv",
+        "query": "single-cell",
+        "scope": "Neuroscience",
+        "notes": "Biomedical preprints"
+      },
+      {
+        "source_type": "SocArXiv",
+        "query": "social identity",
+        "scope": "Sociology",
+        "notes": "Social science preprints"
+      }
+    ]
+  },
+  "days": 7
+}
+```
+
+The script returns JSON:
+
+```json
+{
+  "rss_articles": [...],
+  "papers": [...],
+  "stats": { "rss_count": 12, "paper_count": 45 },
+  "errors": [...]
+}
+```
+
+#### Task B: Web Search (WebSearch)
+
+For each query template:
+
+1. replace `{date range}` with the actual date span
+2. run WebSearch
+3. collect the results
+
+For each supplemental site:
+
+1. build a `site:{url} {topic}` query
+2. run WebSearch
+
+Use `defuddle` on high-value results when the article body matters.
+
+#### Task C: HuggingFace Papers (WebFetch)
+
+1. open `https://huggingface.co/papers`
+2. filter results with the configured keywords
+3. collect title, link, and short description
+4. deduplicate against arXiv results with fuzzy title matching
+
+#### Task D: GitHub Trending (WebFetch, optional)
+
+1. open `https://github.com/trending`
+2. filter repositories with the configured keywords
+3. collect repository name, description, stars, and link
+
+### Phase 3: Merge and Deduplicate
+
+1. **Deduplication**
+   - when two items refer to the same paper (title similarity > 80%), keep the richest source
+   - priority: arXiv original > HuggingFace > RSS summary > Web search
+
+2. **Categorization**
+  - match titles and summaries against the configured category system
+  - the "Key Papers / Key Articles" section should contain the 3-5 most important items overall
+  - uncategorized items fall back to the last category, usually "Industry Updates"
+
+3. **Summary writing**
+   - write a 1-2 sentence English summary for each item
+   - include the source link
+
+### Phase 4: Write the Digest
+
+Write `{drafts directory}/{topic_name}-{date_range_str}.md`.
+
+**Frontmatter:**
+
+```yaml
+---
+title: "{topic_display} Weekly Digest · {date_range_display}"
+type: draft
+created: "{YYYY-MM-DD}"
+status: pending
+tags: [digest, {topic_tag}, weekly-digest]
+aliases: []
+---
+```
+
+**Body structure:**
+
+```markdown
+# {topic_display} Weekly Digest · {date_range_display}
+
+> Auto-compiled · RSS {N} items · arXiv {M} items · Web {K} additions · Generated at {HH:MM}
+
+## {category_1}
+
+- **[Title](link)** — one-sentence English summary
+
+## {category_2}
+
+...
+
+---
+
+## Sources
+
+**RSS feeds:** {rss_names_list}
+**Paper sources:** {paper_source_summaries}
+**Legacy arXiv search:** {legacy_arxiv_summary if present}
+**Web search:** {web_sites_list}
+**HuggingFace:** huggingface.co/papers
+**GitHub:** github.com/trending
+```
+
+Only list enabled modules. Omit empty categories.
+
+### Phase 5: Wrap-up
+
+1. call `memory_notify(file_path="{digest file path}")`
+2. print the completion message:
+
+```text
+Weekly digest written to {drafts directory}/{topic_name}-{date_range_str}.md
+RSS {N} items + arXiv {M} items + Web {K} items
+```
+
+## Multi-topic Runs
+
+When `/digest` is called without arguments and multiple config files exist:
+
+- run them in filename order
+- produce one digest per topic
+- print a summary after all topics finish
+
+```text
+All weekly digests generated:
+- LLM-Agent: 00_Drafts/LLM-Agent-0324-0330.md (RSS 12 + arXiv 45 + Web 8)
+- SpatialAI: 00_Drafts/SpatialAI-0324-0330.md (RSS 8 + arXiv 67 + Web 5)
+```
+
+## Error Handling
+
+| Error | Handling |
+|-------|----------|
+| Python unavailable | tell the user to install Python and stop |
+| RSS feed timeout | mark that source as failed and continue |
+| paper source adapter failure | record a structured source error and continue with the other sources |
+| arXiv API unavailable | record a structured arXiv error and try OpenAlex fallback |
+| WebSearch returns nothing | skip that query and continue |
+| config parsing fails | raise an error with the concrete problem |
+| every source fails | do not generate a digest; report the failure reasons |

package/assets/skills/digest/references/run-pipeline.zh.md

@@ -0,0 +1,235 @@
+# Run 模式执行管线
+
+当用户执行 `/digest` 或 `/digest <主题名>` 时，按此管线执行信息抓取和周报生成。
+
+## 前置检查
+
+1. 检查 Python 3 是否可用：`python3 --version`
+   - 不可用 → 提示用户安装 Python 3 并重试
+2. 扫描 `{系统目录}/{信息子目录}/` 下的 `.md` 文件
+   - 无配置文件 → 自动进入 Setup 模式（见 `setup-guide.md`）
+   - 指定了主题名 → 只加载匹配文件
+   - 未指定主题名 → 加载全部配置并逐个执行
+
+## 执行管线（每个主题独立执行）
+
+### Phase 1: 解析配置
+
+按 `config-parser.md` 规范解析配置笔记，产出结构化数据：
+
+```text
+config = {
+  topic: "LLM Agent",
+  period_days: 7,
+  language: "中文",
+  modules: {
+    rss: { enabled, feeds },
+    papers: { enabled, sources },
+    web: { enabled, queries, sites },
+    huggingface: { enabled, keywords },
+    github: { enabled, keywords }
+  },
+  categories: [{ name, scope }]
+}
+```
+
+旧版 `arxiv` 配置块在过渡期内仍然兼容，会在执行前归一化为 `papers.sources`。
+
+计算日期范围：
+
+- `end_date` = 今天
+- `start_date` = 今天 - `period_days`
+- `date_range_str` = `MMDD-MMDD`（用于文件名）
+- `date_range_display` = `YYYY-MM-DD ~ YYYY-MM-DD`（用于标题）
+
+### Phase 2: 并行抓取
+
+按启用的模块执行抓取。RSS + arXiv 通过 Python 脚本批量处理，其余通过 Agent 工具处理。
+
+#### Task A: RSS + paper sources（Python 脚本）
+
+对论文来源，脚本应遵循以下运行契约：
+
+1. 将 `Paper Sources` 行归一化为来源 adapter 输入
+2. 依次运行 `arXiv`、`bioRxiv`、`medRxiv`、`ChemRxiv`、`SocArXiv`、`SSRN` 的来源 adapters
+3. 返回归一化后的论文结果以及结构化的逐来源错误
+4. 某个来源失败时，保留成功来源并把失败写入 `errors`
+5. 旧版 `arxiv` 配置块会先转换成 `arXiv` adapter 输入，再继续执行
+6. `arXiv` adapter 在 arXiv 检索失败时会保留现有的 OpenAlex fallback 行为
+7. `SocArXiv` 可以归一化到 `OSF` 落地页；`SSRN` 优先保留源站 SSRN 链接
+8. 传输层保持低请求预算：每个来源一次主请求，不做分页
+
+构造 JSON 输入并通过 stdin 传给脚本：
+
+```bash
+echo '<json_config>' | python3 .agents/skills/digest/references/rss-arxiv-script.py
+```
+
+JSON 输入从 Phase 1 的配置构造，至少包括：
+
+```json
+{
+  "language": "zh",
+  "rss": {...},
+  "papers": {
+    "enabled": true,
+    "sources": [
+      {
+        "source_type": "arXiv",
+        "query": "\"llm agent\"",
+        "scope": "cs.AI",
+        "notes": "核心技术论文"
+      },
+      {
+        "source_type": "bioRxiv",
+        "query": "single-cell",
+        "scope": "Neuroscience",
+        "notes": "生物医学预印本"
+      },
+      {
+        "source_type": "SocArXiv",
+        "query": "social identity",
+        "scope": "Sociology",
+        "notes": "社会科学预印本"
+      }
+    ]
+  },
+  "days": 7
+}
+```
+
+脚本输出 JSON：
+
+```json
+{
+  "rss_articles": [...],
+  "papers": [...],
+  "stats": { "rss_count": 12, "paper_count": 45 },
+  "errors": [...]
+}
+```
+
+#### Task B: Web 搜索（WebSearch）
+
+对每条搜索查询模板：
+
+1. 将 `{日期范围}` 替换为实际日期
+2. 执行 WebSearch
+3. 收集结果
+
+对补充站点：
+
+1. 生成 `site:{url} {topic}` 查询
+2. 执行 WebSearch
+
+对高价值结果，用 `defuddle` 提取正文摘要。
+
+#### Task C: HuggingFace 热门论文（WebFetch）
+
+1. 用 WebFetch 打开 `https://huggingface.co/papers`
+2. 用配置关键词过滤结果
+3. 记录标题、链接和简要描述
+4. 与 arXiv 结果去重（按标题模糊匹配）
+
+#### Task D: GitHub Trending（WebFetch，仅启用时）
+
+1. 用 WebFetch 打开 `https://github.com/trending`
+2. 用配置关键词过滤结果
+3. 记录仓库名、描述、星标数和链接
+
+### Phase 3: 合并去重
+
+1. **去重规则**
+   - 同一论文（标题相似度 > 80%）只保留最详细来源
+   - 优先级：arXiv 原文 > HuggingFace > RSS 摘要 > Web 搜索
+
+2. **分类归类**
+   - 根据标题和摘要匹配配置中的分类体系
+   - 「重要论文/重要文章」分类从全量结果里选出影响最大的 3-5 条
+   - 无法归类的信息放入最后一个分类（通常是「行业动态」）
+
+3. **摘要生成**
+   - 每条信息用 1-2 句中文提炼核心内容
+   - 附原文链接
+
+### Phase 4: 写入周报
+
+写入 `{草稿目录}/{topic_name}-{date_range_str}.md`。
+
+**Frontmatter：**
+
+```yaml
+---
+title: "{topic_display} 周报 · {date_range_display}"
+type: draft
+created: "{YYYY-MM-DD}"
+status: pending
+tags: [digest, {topic_tag}, weekly-digest]
+aliases: []
+---
+```
+
+**正文结构：**
+
+```markdown
+# {topic_display} 周报 · {date_range_display}
+
+> 自动汇总 · RSS {N} 篇 · arXiv {M} 篇 · Web 补充 {K} 条 · 生成时间 {HH:MM}
+
+## {category_1}
+
+- **[标题](链接)** — 一句话中文摘要
+
+## {category_2}
+
+...
+
+---
+
+## 信息来源
+
+**RSS 订阅：** {rss_names_list}
+**论文来源：** {paper_source_summaries}
+**旧版 arXiv 搜索：** {legacy_arxiv_summary if present}
+**Web 搜索：** {web_sites_list}
+**HuggingFace：** huggingface.co/papers
+**GitHub：** github.com/trending
+```
+
+只列出已启用模块的来源。空分类不输出。
+
+### Phase 5: 收尾
+
+1. 调用 `memory_notify(file_path="{周报文件路径}")`
+2. 输出完成提示：
+
+```text
+✅ {topic_display} 周报已写入：{草稿目录}/{topic_name}-{date_range_str}.md
+   RSS {N} 篇 + arXiv {M} 篇 + Web {K} 条
+```
+
+## 多主题执行
+
+当无参数调用 `/digest` 且存在多个配置文件时：
+
+- 按文件名字母序逐个执行
+- 每个主题独立产出一份周报
+- 全部主题完成后统一输出汇总
+
+```text
+✅ 全部周报已生成：
+- LLM-Agent: 00_草稿/LLM-Agent-0324-0330.md（RSS 12 + arXiv 45 + Web 8）
+- SpatialAI: 00_草稿/SpatialAI-0324-0330.md（RSS 8 + arXiv 67 + Web 5）
+```
+
+## 错误处理
+
+| 错误 | 处理 |
+|------|------|
+| Python 不可用 | 报错提示安装并中止执行 |
+| RSS feed 超时 | 标记失败，继续其他来源 |
+| 论文来源 adapter 失败 | 记录结构化来源错误，继续执行其他来源 |
+| arXiv API 无响应 | 记录结构化 arXiv 错误，并尝试 OpenAlex fallback |
+| WebSearch 无结果 | 跳过该查询，继续 |
+| 配置解析失败 | 报错并提示具体问题 |
+| 所有来源均失败 | 不生成周报，报告失败原因 |

package/assets/skills/digest/references/setup-guide.en.md

@@ -0,0 +1,192 @@
+# Setup Mode Guide
+
+Use this flow when the user first runs `/digest` or explicitly asks for `/digest setup`.
+
+## Trigger Conditions
+
+- no `.md` config files exist under `{system directory}/{digest subdirectory}/`
+- the user explicitly runs `/digest setup` or `/digest setup <topic>`
+
+## Conversation Flow
+
+### Step 1: Define the Topic
+
+If the user did not provide a topic name:
+
+```text
+What area do you want to track with a weekly digest?
+
+Give me a topic name (for example "LLM Agent", "Spatial AI", "Rust ecosystem", or "quant investing"),
+plus the 2-3 subareas you care about most.
+```
+
+If the user already supplied a topic name (for example `/digest setup LLM-Agent`), skip this step and go straight to Step 2.
+
+**Output:** decide `topic_name` (English filename) and `topic_display` (user-facing display name).
+
+### Step 2: Understand Preferences
+
+```text
+For "{topic_display}", help me understand a few preferences:
+
+1. Content type: more academic papers, more industry updates, or both?
+2. Must-read sources: are there blogs, newsletters, or accounts you already follow closely?
+3. Focus areas: which subtopics matter most? These will shape the digest categories.
+```
+
+### Step 3: Generate the Config
+
+Based on the topic and preferences, use agent capabilities to recommend sources and produce a full config note.
+
+**Generation strategy:**
+
+1. **RSS / Newsletter**
+   - verify URLs with WebSearch
+   - recommend 5-15 high-quality sources
+   - prefer sources that expose RSS feeds
+
+2. **Paper Sources**
+   - recommend explicit source rows instead of assuming arXiv-only defaults
+   - use `bioRxiv` / `medRxiv` for biomedical topics
+   - use `ChemRxiv` for chemistry topics
+   - use `arXiv` for technical / AI topics
+   - generate a small number of concrete `Source Type | Query | Scope | Notes` rows
+   - keep `Query` values searchable and source-specific, and write a short note when a source needs
+     special handling
+   - disable paper sources by default for non-academic topics unless the user explicitly wants
+     preprints
+
+3. **Web search**
+   - design 3-5 query templates for important sources without RSS
+   - add 5-10 supplemental sites
+
+4. **HuggingFace**
+   - generate filtering keywords for AI / ML topics
+   - disable by default for non-AI / ML topics
+
+5. **GitHub Trending**
+   - generate filtering keywords for technical topics
+   - disable by default for non-technical topics
+
+6. **Category system**
+   - generate 5-8 categories from the chosen subareas
+   - fix the first category as "Key Papers / Key Articles"
+   - fix the last category as "Industry Updates"
+
+**Config note template:**
+
+```markdown
+---
+title: "{topic_display} Digest"
+type: system
+created: "{YYYY-MM-DD}"
+tags: [digest, subscription]
+aliases: []
+---
+
+# {topic_display} Digest
+
+## Basic Info
+
+| Field | Value |
+|-------|-------|
+| Topic | {topic_display} |
+| Cadence | Weekly |
+| Language | English |
+
+## Sources
+
+### RSS Feeds
+
+- [x] Enabled
+
+| Name | URL | Focus |
+|------|-----|-------|
+| {name} | {url} | {description} |
+...
+
+### Paper Sources
+
+- [x] Enabled
+
+| Source Type | Query | Scope | Notes |
+|-------------|-------|-------|-------|
+| arXiv | {query} | {scope} | {notes} |
+| bioRxiv | {query} | {scope} | {notes} |
+| SocArXiv | {query} | {scope} | {notes} |
+| SSRN | {query} | {scope} | {notes} |
+...
+
+> Legacy compatibility: old notes may still use `### arXiv Search`. New setups should prefer
+> `Paper Sources`, but existing arXiv-only configs remain valid.
+
+### Web Search
+
+- [x] Enabled
+
+| Query Template | Coverage |
+|----------------|----------|
+| {query_template} | {target} |
+...
+
+**Supplemental Sites (covered via Web search):**
+
+| Name | URL | Focus |
+|------|-----|-------|
+| {name} | {url} | {description} |
+...
+
+### HuggingFace Papers
+
+- [{x_or_space}] Enabled
+
+**Filter keywords:** {keyword1}, {keyword2}, ...
+
+### GitHub Trending
+
+- [{x_or_space}] Enabled
+
+**Filter keywords:** {keyword1}, {keyword2}, ...
+
+## Categories
+
+The digest is organized by the following categories. Omit empty categories:
+
+| Category | Coverage |
+|----------|----------|
+| {category} | {scope} |
+...
+
+## Source List
+
+Automatically appended at the end of each digest.
+```
+
+### Step 4: User Review
+
+Write the config note to `{system directory}/{digest subdirectory}/{topic_name}.md`.
+
+```text
+Config note created: {system directory}/{digest subdirectory}/{topic_name}.md
+
+Ask the user to review it in Obsidian:
+- disable modules they do not want with checkboxes
+- keep paper-source queries searchable, then add or remove RSS feeds, Paper Sources rows, and Web
+  search targets
+- adjust the category system
+
+After review, they can run `/digest {topic_name}` to generate the first digest.
+```
+
+## Notes
+
+- keep the setup conversation within 3 rounds whenever possible
+- source recommendations should include concrete URLs, not only names
+- any must-read source mentioned by the user must appear in the config
+- non-technical topics such as finance or history should disable arXiv, HuggingFace, and GitHub by default
+- biomedical topics should usually prioritize `bioRxiv` / `medRxiv`, chemistry should prioritize
+  `ChemRxiv`, social-science topics should usually prioritize `SocArXiv` / `SSRN`, and
+  technical / AI topics should prioritize `arXiv`
+- `SocArXiv` papers may resolve to `OSF` landing pages because that is the source-hosted record
+- paper-source fetching stays intentionally low-budget: one primary request per source and no
+  pagination