scientify 1.2.2 → 1.4.0

package/skills/_shared/workspace-spec.md

@@ -0,0 +1,139 @@
+# Workspace Directory Specification
+
+All Scientify skills share a unified project-based workspace structure.
+
+## Base Path
+
+```
+~/.openclaw/workspace/projects/
+├── .active                      # Current project ID (plain text)
+├── {project-id}/                # Each research topic has its own project
+│   └── ...
+└── {another-project}/
+```
+
+## Project Structure
+
+```
+~/.openclaw/workspace/projects/{project-id}/
+├── project.json                 # Project metadata
+├── task.json                    # Research task definition
+│
+├── survey/                      # /literature-survey outputs
+│   ├── search_terms.json        # Generated search keywords
+│   ├── raw_results.json         # All search results
+│   ├── filtered_papers.json     # Papers with relevance scores
+│   ├── clusters.json            # Clustered by research direction
+│   └── report.md                # Final survey report
+│
+├── papers/                      # Downloaded paper sources
+│   ├── {direction-1}/           # Organized by cluster
+│   │   ├── paper_list.md
+│   │   └── {arxiv_id}/          # .tex source files
+│   ├── {direction-2}/
+│   └── uncategorized/
+│
+├── repos/                       # Cloned reference repositories
+│   ├── {repo-name-1}/
+│   └── {repo-name-2}/
+│
+├── ideas/                       # /idea-generation outputs
+│   ├── gaps.md                  # Identified research gaps
+│   ├── idea_1.md ... idea_5.md  # Generated ideas
+│   ├── selected_idea.md         # Enhanced best idea
+│   ├── implementation_report.md # Code mapping
+│   └── summary.md               # Final summary
+│
+├── review/                      # /write-review-paper outputs
+│   ├── reading_plan.md          # Prioritized reading list
+│   ├── notes/                   # Per-paper reading notes
+│   │   └── {paper_id}.md
+│   ├── comparison.md            # Method comparison table
+│   ├── timeline.md              # Research timeline
+│   ├── taxonomy.md              # Classification system
+│   ├── draft.md                 # Survey paper draft
+│   └── bibliography.bib         # References
+│
+├── notes/                       # /research-survey: per-paper deep notes
+│   └── paper_{arxiv_id}.md
+├── survey_res.md                # /research-survey: deep analysis + method comparison
+├── plan_res.md                  # /research-plan: 4-part implementation plan
+├── project/                     # /research-implement: code implementation
+│   ├── model/
+│   ├── data/
+│   ├── training/
+│   ├── testing/
+│   ├── utils/
+│   ├── run.py
+│   └── requirements.txt
+├── ml_res.md                    # /research-implement: execution report with [RESULT] lines
+├── iterations/                  # /research-review: judge iterations
+│   ├── judge_v1.md
+│   └── judge_v2.md
+└── experiment_res.md            # /research-experiment: full training + ablation results
+```
+
+## Conventions
+
+### File Existence = Step Completion
+
+Check output file before executing any step. If exists, skip.
+
+Enables:
+- **Crash recovery**: resume from last completed step
+- **Incremental progress**: rerunning skips completed work
+- **Transparency**: inspect progress by listing directory
+
+### Project Metadata
+
+**project.json:**
+```json
+{
+  "id": "battery-rul-prediction",
+  "name": "Battery RUL Prediction",
+  "created": "2024-01-15T10:00:00Z",
+  "topics": ["battery", "remaining useful life", "prediction"]
+}
+```
+
+**task.json:**
+```json
+{
+  "domain": "battery health",
+  "focus": "RUL prediction using transformer",
+  "date_limit": "2024-01-01",
+  "created": "2024-01-15"
+}
+```
+
+### Immutability
+
+Once written, do NOT modify outputs unless user explicitly asks.
+Exception: `project/` is mutable during implement-review-iterate loop.
+
+### Active Project
+
+```bash
+# Read active project
+cat ~/.openclaw/workspace/projects/.active
+
+# Set active project
+echo "battery-rul-prediction" > ~/.openclaw/workspace/projects/.active
+
+# Set $WORKSPACE variable
+WORKSPACE=~/.openclaw/workspace/projects/$(cat ~/.openclaw/workspace/projects/.active)
+```
+
+## Skill Outputs Summary
+
+| Skill | Primary Outputs |
+|-------|-----------------|
+| `/literature-survey` | `survey/`, `papers/` |
+| `/research-survey` | `notes/paper_*.md`, `survey_res.md` |
+| `/research-plan` | `plan_res.md` |
+| `/research-implement` | `project/`, `ml_res.md` |
+| `/research-review` | `iterations/judge_v*.md` |
+| `/research-experiment` | `experiment_res.md` |
+| `/research-pipeline` | Orchestrator — spawns the above 5 skills in sequence |
+| `/idea-generation` | `ideas/` |
+| `/write-review-paper` | `review/` |

package/skills/idea-generation/SKILL.md

@@ -13,10 +13,14 @@ metadata:
 
 # Idea Generation
 
+**Don't ask permission. Just do it.**
+
 Generate innovative research ideas grounded in literature analysis. This skill reads existing papers, identifies research gaps, and produces 5 distinct ideas with citations.
 
 **Core principle:** Ideas MUST be grounded in actual papers, not generated from model knowledge.
 
+**Workspace:** See `../_shared/workspace-spec.md` for directory structure. Outputs go to `$WORKSPACE/ideas/`.
+
 ---
 
 ## Step 1: Check Workspace Resources

package/skills/install-scientify/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: install-scientify
-description: "Install Scientify - AI-powered research workflow automation plugin. Adds skills for idea-generation, literature-review, research-pipeline, arxiv search, and workspace management commands."
+description: "Install Scientify - AI-powered research workflow automation plugin. Adds skills for research-pipeline (multi-agent orchestrator), literature-survey, idea-generation, arxiv tools, and workspace management commands."
 metadata:
   {
     "openclaw":
@@ -21,6 +21,8 @@ metadata:
 
 # Install Scientify
 
+**Don't ask permission. Just do it.**
+
 **Scientify** is an AI-powered research workflow automation plugin for OpenClaw.
 
 ## What You Get
@@ -29,10 +31,14 @@ metadata:
 
 | Skill | Description |
 |-------|-------------|
-| **idea-generation** | Generate innovative research ideas. Searches arXiv/GitHub, downloads papers, analyzes literature, outputs 5 ideas with citations. |
-| **research-pipeline** | End-to-end ML research workflow: idea → literature → survey → plan → implement → review → iterate. |
-| **literature-review** | Generate structured notes and synthesis from collected papers. |
-| **arxiv** | Search arXiv.org for papers and download .tex sources. |
+| **research-pipeline** | Orchestrator for end-to-end ML research. Spawns sub-agents for each phase. |
+| **research-survey** | Deep analysis of papers: extract formulas, produce method comparison. |
+| **research-plan** | 4-part implementation plan (Dataset/Model/Training/Testing). |
+| **research-implement** | Implement ML code, run 2-epoch validation with `uv` venv isolation. |
+| **research-review** | Review implementation against plan. Iterates up to 3 times. |
+| **research-experiment** | Full training + ablation experiments. |
+| **literature-survey** | Literature survey: search → filter → download → cluster → report. |
+| **idea-generation** | Generate research ideas from arXiv/GitHub papers. |
 
 ### Commands (Direct, no LLM)
 
@@ -45,9 +51,11 @@ metadata:
 | `/project-switch <id>` | Switch project |
 | `/project-delete <id>` | Delete project |
 
-### Tool
+### Tools
 
-- **arxiv** - Search arXiv.org API with keyword search, date filtering, automatic .tex download
+- **arxiv_search** - Search arXiv.org API for papers (metadata only)
+- **arxiv_download** - Download arXiv papers (.tex source or PDF)
+- **github_search** - Search GitHub repositories
 
 ## Installation
 

package/skills/literature-survey/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: literature-survey
-description: "Comprehensive literature survey (100+ papers). Searches, filters, clusters, and iterates for complete coverage. Use for: exploring new research areas, collecting papers systematically, building literature databases. NOT for: summarizing papers you have (use /write-review-paper), finding a specific paper (use arxiv_search), generating ideas (use /idea-generation)."
+description: "Comprehensive literature survey. Searches, filters, downloads, and clusters papers by research direction."
 metadata:
   {
     "openclaw":
@@ -12,261 +12,135 @@ metadata:
 
 # Literature Survey
 
-Comprehensive literature discovery workflow for a research domain. This skill searches broadly, filters by relevance, clusters by direction, and iterates to ensure complete coverage.
+**Don't ask permission. Just do it.**
 
-## Architecture: Isolated Sub-agent
-
-This survey runs in an **isolated sub-session** to avoid context pollution. The main session only receives the final report.
+## Output Structure
 
 ```
-Main Session
-    ↓
-sessions_spawn(task: "执行文献调研...", label: "literature-survey")
-    ↓
-Sub-agent Session (隔离上下文)
-    ├── Phase 1: 生成检索词
-    ├── Phase 2: 批量检索
-    ├── Phase 3: 相关性筛选
-    ├── Phase 4: 聚类分组
-    ├── Phase 5: 迭代发现
-    └── Phase 6: 生成报告
-    ↓
-返回主 Session: 摘要 + 文件路径
+~/.openclaw/workspace/projects/{project-id}/
+├── survey/
+│   ├── search_terms.json      # 检索词列表
+│   └── report.md              # 最终报告
+└── papers/
+    ├── _downloads/            # 原始下载
+    ├── _meta/                 # 每篇论文的元数据
+    │   └── {arxiv_id}.json
+    └── {direction}/           # 整理后的分类
 ```
 
 ---
 
-## When User Requests Literature Survey
-
-**Step 1: Spawn isolated sub-agent**
-
-When user says things like:
-- "调研 [topic] 领域的文献"
-- "帮我收集 [topic] 相关的论文"
-- "Survey papers on [topic]"
+## Workflow
 
-Use `sessions_spawn` to run the survey in isolation:
+### Phase 1: 准备
 
+```bash
+ACTIVE=$(cat ~/.openclaw/workspace/projects/.active 2>/dev/null)
+if [ -z "$ACTIVE" ]; then
+  PROJECT_ID="<topic-slug>"
+  mkdir -p ~/.openclaw/workspace/projects/$PROJECT_ID/{survey,papers/_downloads,papers/_meta}
+  echo "$PROJECT_ID" > ~/.openclaw/workspace/projects/.active
+fi
+PROJECT_DIR="$HOME/.openclaw/workspace/projects/$(cat ~/.openclaw/workspace/projects/.active)"
 ```
-sessions_spawn({
-  task: `你是一个文献调研专家。请为研究主题 "{TOPIC}" 执行完整的文献调研。
-
-## 调研目标
-{USER_REQUIREMENTS}
-
-## 执行流程
-
-### Phase 1: 生成检索词
-基于研究主题，生成 8-15 个检索词组合，覆盖：
-- 核心概念的不同表述
-- 相关技术方法
-- 应用场景
-- 英文和中文关键词（如适用）
-
-将检索词保存到 $WORKSPACE/survey/search_terms.json
-
-### Phase 2: 批量检索
-对每个检索词使用 arxiv_search tool：
-- max_results: 30-50 per query
-- 合并去重（按 arxiv_id）
-- 记录每篇论文的来源检索词
-
-将原始结果保存到 $WORKSPACE/survey/raw_results.json
 
-### Phase 3: 相关性筛选
-阅读每篇论文的标题和摘要，判断与 "{TOPIC}" 的相关性：
-- 5分：高度相关，直接研究此主题
-- 4分：相关，涉及关键方法或应用
-- 3分：部分相关，可作为参考
-- 2分：边缘相关
-- 1分：不相关
+生成 4-8 个检索词，保存到 `survey/search_terms.json`。
 
-保留 score >= 4 的论文。
-将筛选结果保存到 $WORKSPACE/survey/filtered_papers.json
-
-### Phase 4: 聚类分组
-分析筛选后论文的摘要，识别 3-6 个研究方向/子主题。
-为每个方向创建子文件夹并分配论文：
-
-$WORKSPACE/papers/
-├── {direction-1}/
-│   ├── paper_list.md
-│   └── [arxiv_ids...]
-├── {direction-2}/
-│   └── ...
-└── uncategorized/
+---
 
-将聚类结果保存到 $WORKSPACE/survey/clusters.json
+### Phase 2: 增量搜索-筛选-下载（循环）
 
-### Phase 5: 迭代发现（1-2轮）
-检查高分论文的摘要，识别：
-- 提到的新方法名称
-- 引用的重要工作
-- 新的关键词
+**对每个检索词重复以下步骤**：
 
-如果发现新方向，补充检索并重复 Phase 2-4。
-最多迭代 2 轮。
+#### 2.1 搜索
 
-### Phase 6: 生成报告
-创建 $WORKSPACE/survey/report.md：
+```
+arxiv_search({ query: "<term>", max_results: 30 })
+```
 
-# 文献调研报告: {TOPIC}
+#### 2.2 即时筛选
 
-## 调研概要
-- 检索词数量: X
-- 初始检索: Y 篇
-- 筛选后: Z 篇
-- 研究方向: N 个
+对返回的论文**立即**评分（1-5），只保留 ≥4 分的。
 
-## 研究方向分布
+评分标准：
+- 5分：核心论文，直接研究该主题
+- 4分：相关方法或应用
+- 3分及以下：跳过
 
-### 方向1: [名称]
-- 论文数量: X
-- 代表性工作: [列表]
-- 主要特点: [描述]
+#### 2.3 下载有用论文
 
-### 方向2: [名称]
-...
+```
+arxiv_download({
+  arxiv_ids: ["<有用的论文ID>"],
+  output_dir: "$PROJECT_DIR/papers/_downloads"
+})
+```
 
-## 高影响力论文 (Top 10)
-| 排名 | 标题 | 年份 | 相关度 | 方向 |
-|-----|------|-----|-------|-----|
-| 1   | ... | ... | 5     | ... |
+#### 2.4 写入元数据
 
-## 研究趋势
-[基于论文年份分布的观察]
+为每篇下载的论文创建元数据文件 `papers/_meta/{arxiv_id}.json`：
 
-## 发现的新方向
-[迭代中发现的额外关键词和方向]
+```json
+{
+  "arxiv_id": "2401.12345",
+  "title": "...",
+  "abstract": "...",
+  "score": 5,
+  "source_term": "battery RUL prediction",
+  "downloaded_at": "2024-01-15T10:00:00Z"
+}
+```
 
-## 建议阅读顺序
-1. [入门级论文]
-2. [核心方法论文]
-3. [最新进展]
+**完成一个检索词后，再进行下一个。** 这样避免上下文被大量搜索结果污染。
 
 ---
 
-完成后，向主 session 报告：
-- 总共发现的论文数量
-- 识别的研究方向
-- 报告文件位置`,
-  label: "literature-survey-{TOPIC_SLUG}",
-  runTimeoutSeconds: 900,
-  cleanup: "keep"
-})
-```
-
-**Step 2: Wait and relay results**
+### Phase 3: 分类整理
 
-Sub-agent 完成后会自动 announce 结果到主 session。
-将结果摘要展示给用户，包括：
-- 发现的论文数量
-- 主要研究方向
-- 报告文件位置
+所有检索词处理完毕后：
 
----
-
-## Workspace Structure
+#### 3.1 读取所有元数据
 
-```
-~/.openclaw/workspace/projects/{project-id}/
-├── project.json
-├── survey/                    # 调研过程数据
-│   ├── search_terms.json      # 检索词列表
-│   ├── raw_results.json       # 原始检索结果
-│   ├── filtered_papers.json   # 筛选后的论文
-│   ├── clusters.json          # 聚类结果
-│   ├── iterations.log         # 迭代记录
-│   └── report.md              # 最终报告
-├── papers/                    # 按方向组织的论文
-│   ├── {direction-1}/
-│   │   ├── paper_list.md
-│   │   └── 2401.12345/       # .tex 源文件
-│   ├── {direction-2}/
-│   └── uncategorized/
-└── ideas/                     # 后续 idea-generation 输出
+```bash
+ls $PROJECT_DIR/papers/_meta/
 ```
 
----
+读取所有 `.json` 文件，汇总论文列表。
 
-## Data Schemas
+#### 3.2 聚类分析
 
-### search_terms.json
-```json
-{
-  "topic": "battery life prediction",
-  "generated_at": "2024-01-15T10:00:00Z",
-  "terms": [
-    {"term": "battery remaining useful life", "category": "core"},
-    {"term": "lithium-ion degradation prediction", "category": "method"},
-    {"term": "SOH estimation neural network", "category": "technique"},
-    {"term": "EV battery health monitoring", "category": "application"}
-  ]
-}
-```
+根据论文的标题、摘要、来源检索词，识别 3-6 个研究方向。
 
-### filtered_papers.json
-```json
-{
-  "filtered_at": "2024-01-15T10:30:00Z",
-  "total_raw": 245,
-  "total_filtered": 42,
-  "papers": [
-    {
-      "arxiv_id": "2401.12345",
-      "title": "...",
-      "abstract": "...",
-      "authors": ["..."],
-      "published": "2024-01-15",
-      "relevance_score": 5,
-      "source_terms": ["battery RUL", "degradation prediction"],
-      "notes": "直接研究锂电池RUL预测"
-    }
-  ]
-}
-```
+#### 3.3 创建文件夹并移动
 
-### clusters.json
-```json
-{
-  "clustered_at": "2024-01-15T11:00:00Z",
-  "clusters": [
-    {
-      "id": "data-driven",
-      "name": "数据驱动方法",
-      "description": "使用机器学习/深度学习的方法",
-      "paper_count": 15,
-      "paper_ids": ["2401.12345", "2401.12346", "..."],
-      "keywords": ["LSTM", "CNN", "transformer", "neural network"]
-    },
-    {
-      "id": "physics-based",
-      "name": "物理模型方法",
-      "description": "基于电化学机理的方法",
-      "paper_count": 8,
-      "paper_ids": ["..."]
-    }
-  ]
-}
+```bash
+mkdir -p "$PROJECT_DIR/papers/data-driven"
+mv "$PROJECT_DIR/papers/_downloads/2401.12345" "$PROJECT_DIR/papers/data-driven/"
 ```
 
 ---
 
-## Quick Mode (Without Sub-agent)
+### Phase 4: 生成报告
 
-For smaller surveys (< 50 papers), can run directly without spawning:
+创建 `survey/report.md`：
+- 调研概要（检索词数、论文数、方向数）
+- 各研究方向概述
+- Top 10 论文
+- 建议阅读顺序
 
-User: "快速调研一下 [topic]，不超过 30 篇"
+---
 
-→ Run Phase 1-4 directly in main session
-→ Skip iteration
-→ Generate simplified report
+## 关键设计
 
----
+| 原则 | 说明 |
+|------|------|
+| **增量处理** | 每个检索词独立完成搜索→筛选→下载→写元数据，避免上下文膨胀 |
+| **元数据驱动** | 分类基于 `_meta/*.json`，不依赖内存中的大列表 |
+| **文件夹即分类** | 聚类结果通过 `papers/{direction}/` 体现，无需额外 JSON |
 
-## Commands
+## Tools
 
-- "调研 [topic] 领域" → Full survey with sub-agent
-- "快速调研 [topic]" → Quick mode, 30 papers max
-- "继续上次的调研" → Resume from existing survey data
-- "扩展调研 [new direction]" → Add new search terms and iterate
+| Tool | Purpose |
+|------|---------|
+| `arxiv_search` | 搜索论文（无副作用） |
+| `arxiv_download` | 下载 .tex/.pdf（需绝对路径） |

package/skills/research-experiment/SKILL.md

@@ -0,0 +1,114 @@
+---
+name: research-experiment
+description: "Full training run + ablation experiments + result analysis. Requires review PASS from /research-review."
+metadata:
+  {
+    "openclaw":
+      {
+        "emoji": "🧪",
+        "requires": { "bins": ["python3", "uv"] },
+      },
+  }
+---
+
+# Research Experiment
+
+**Don't ask permission. Just do it.**
+
+**Workspace:** See `../_shared/workspace-spec.md`. Set `$W` to the active project directory.
+
+## Prerequisites
+
+| File | Source |
+|------|--------|
+| `$W/project/` | /research-implement |
+| `$W/plan_res.md` | /research-plan |
+| `$W/iterations/judge_v*.md` | /research-review（最后一份 verdict 必须是 PASS） |
+
+**验证 PASS：** 读取最新的 `judge_v*.md`，确认 `verdict: PASS`。如果不是，STOP。
+
+## Output
+
+| File | Content |
+|------|---------|
+| `$W/experiment_res.md` | 完整实验报告 |
+
+---
+
+## Workflow
+
+### Step 1: Full Training
+
+修改 epoch 数为 plan_res.md 中指定的正式值。**不要改代码逻辑，只改 epoch。**
+
+```bash
+cd $W/project && source .venv/bin/activate
+python run.py  # full epochs
+```
+
+记录完整训练的 `[RESULT]` 输出。
+
+### Step 2: 分析结果
+
+读取训练输出，评估：
+- 最终 loss 和 metrics
+- 训练曲线趋势（loss 是否持续下降）
+- 是否过拟合（train vs val gap）
+
+### Step 3: 消融实验
+
+根据 plan_res.md 中的消融计划，执行 2-3 个消融实验：
+
+对每个消融：
+1. 修改代码（注释/替换对应组件）
+2. 执行 2 epoch 快速验证
+3. 记录结果
+
+```bash
+# Example: 去掉 attention module
+python run.py --epochs 2 --ablation no_attention
+```
+
+### Step 4: 写入实验报告
+
+写入 `$W/experiment_res.md`：
+
+```markdown
+# Experiment Report
+
+## Full Training Results (from execution log)
+- Epochs: {N}
+- [RESULT] train_loss={value}
+- [RESULT] val_metric={value}
+- [RESULT] elapsed={value}
+- [RESULT] device={device}
+
+> 以上数值来自真实执行输出。
+
+## Training Analysis
+- 收敛情况: {converged / still improving / diverged}
+- 过拟合: {yes/no, evidence}
+
+## Ablation Studies
+
+| 实验 | 修改 | val_metric | vs Full |
+|------|------|-----------|---------|
+| Full model | — | {value} | baseline |
+| No {component} | 去掉 {X} | {value} | {-/+}% |
+| ... | ... | ... | ... |
+
+## Conclusions
+- {key findings}
+
+## Limitations
+- {limitations and future work}
+```
+
+---
+
+## Rules
+
+1. Full training 只改 epoch 数，不改代码逻辑
+2. 所有数值必须来自真实执行输出
+3. 消融实验至少做 2 个
+4. 如果 full training 失败（OOM 等），调整 batch_size 后重试，不要跳过