scientify 1.2.2 → 1.4.0

package/skills/research-pipeline/references/prompts/plan.md

@@ -1,142 +0,0 @@
-# Implementation Planning Guide
-
-You are creating a detailed, actionable implementation plan. This plan must be specific enough that the implementation step can follow it without ambiguity.
-
-## Prerequisites
-
-Before planning, you must have:
-
-- `workspace/task.json` — the research idea
-- `workspace/survey_res.md` — the literature survey with theory-to-code mappings
-- `workspace/prepare_res.md` — selected reference repositories
-
-Read ALL of these files thoroughly before writing the plan. Also browse the reference codebases in `workspace/repos/` to understand their structure and reusable components.
-
-## Plan Structure
-
-The plan has four mandatory sections. Write all four to `workspace/plan_res.md`.
-
-### 1. Dataset Plan
-
-```markdown
-## Dataset Plan
-
-### Data Source
-- Dataset name and where to obtain it
-- Size and format
-- Any preprocessing requirements
-
-### Data Loading Pipeline
-1. **Read**: How to load raw data (file format, library to use)
-2. **Preprocess**: Transformations, tokenization, normalization, feature extraction
-3. **DataLoader**: Batch construction, sampling strategy, collate function
-
-### Data Splits
-- Train/validation/test split ratios
-- Any special handling (e.g., cold-start users, temporal splits)
-```
-
-Refer to the reference codebases for data loading patterns. Cite specific files: "See `repos/xyz/data/loader.py` for the graph construction approach."
-
-### 2. Model Plan
-
-```markdown
-## Model Plan
-
-### Architecture Overview
-[High-level description of the model architecture]
-
-### Components (one per atomic definition)
-
-#### [Atomic Definition 1]
-- **Math**: $formula$
-- **Implementation**: Class name, input/output shapes, key methods
-- **Reference**: repos/xyz/model/attention.py, class MultiHeadAttention
-- **Adaptation notes**: [What to change from the reference]
-
-#### [Atomic Definition 2]
-...
-
-### Forward Pass
-[Step-by-step description of the forward pass, connecting all components]
-
-### Parameter Count Estimate
-[Rough estimate to sanity-check the architecture]
-```
-
-Every atomic definition from `survey_res.md` must appear as a component. If a definition doesn't map to a model component, explain why.
-
-### 3. Training Plan
-
-```markdown
-## Training Plan
-
-### Loss Function
-- Formula: $L = ...$
-- Components: [list each loss term and its purpose]
-- Reference: repos/xyz/training/loss.py
-
-### Optimizer
-- Algorithm: [Adam, AdamW, SGD, etc.]
-- Learning rate: [value] with [schedule: cosine, step, warmup, etc.]
-- Weight decay: [value]
-
-### Hyperparameters
-| Parameter | Value | Rationale |
-|-----------|-------|-----------|
-| Batch size | ... | ... |
-| Hidden dim | ... | ... |
-| Num layers | ... | ... |
-| Dropout | ... | ... |
-
-### Training Loop
-1. Forward pass
-2. Compute loss
-3. Backward pass
-4. Gradient clipping (if applicable)
-5. Optimizer step
-6. Logging (every N steps)
-7. Validation (every M epochs)
-
-### Quick Validation
-- Epochs: 2 (for initial validation)
-- Expected behavior: loss should decrease, no NaN/Inf
-
-### Full Training
-- Epochs: [value from reference papers]
-- Early stopping: [criteria]
-- Checkpoint: save best model by validation metric
-```
-
-### 4. Testing Plan
-
-```markdown
-## Testing Plan
-
-### Metrics
-- Primary: [e.g., NDCG@10, BLEU, F1]
-- Secondary: [e.g., Recall@20, Hit Rate]
-- Reference: repos/xyz/evaluation/metrics.py
-
-### Evaluation Protocol
-1. Load best checkpoint
-2. Run inference on test set
-3. Compute metrics
-4. Compare against baselines (from papers)
-
-### Baselines
-| Method | Metric | Value | Source |
-|--------|--------|-------|--------|
-| ... | ... | ... | [paper] |
-
-### Expected Results
-[Reasonable range for the proposed method based on paper claims]
-```
-
-## Quality Criteria
-
-- Every section must reference specific files from `workspace/repos/` where applicable.
-- Hyperparameter values should come from reference papers or standard practice, not guesses.
-- The plan must be implementable end-to-end without additional research.
-- If any information is missing (e.g., dataset not publicly available), flag it explicitly.
-- Do not over-engineer: plan what's needed for a solid implementation, not a production system.

package/skills/research-pipeline/references/prompts/review.md

@@ -1,118 +0,0 @@
-# Code Review Guide
-
-You are reviewing the implementation in `workspace/project/` to verify it correctly implements the research idea. This is a quality gate before full training.
-
-## Review Process
-
-### Phase 1: Verify Against Survey
-
-Read `workspace/survey_res.md` and extract the list of atomic definitions. For each atomic definition:
-
-1. Find the corresponding code in `workspace/project/`.
-2. Compare the code implementation against the mathematical formula.
-3. Check: does the code faithfully implement the math? Watch for:
-   - Missing terms in equations.
-   - Incorrect tensor operations (e.g., sum vs mean, wrong axis).
-   - Hardcoded values where parameters should be used.
-   - Simplifications that change the method's behavior.
-
-### Phase 2: Verify Against Plan
-
-Read `workspace/plan_res.md`. Check each section:
-
-**Dataset Plan:**
-- Is the correct dataset used (not a substitute)?
-- Does the preprocessing match the plan?
-- Is the DataLoader configured correctly (batch size, sampling)?
-
-**Model Plan:**
-- Are all components present?
-- Does the forward pass match the described architecture?
-- Are parameter counts reasonable?
-
-**Training Plan:**
-- Is the loss function correct (all terms present, correct weighting)?
-- Is the optimizer configured as planned?
-- Are hyperparameters matching?
-
-**Testing Plan:**
-- Are the correct metrics implemented?
-- Is the evaluation protocol correct?
-
-### Phase 3: Code Quality
-
-Check for implementation quality issues:
-
-- **Not a toy**: The implementation should be substantive, not a simplified stub.
-- **Correctness**: No obvious bugs (wrong indices, missing gradients, data leakage).
-- **Completeness**: All imports resolved, all functions implemented (no `pass` or `TODO`).
-- **Runnability**: The code should run end-to-end without errors.
-
-### Phase 4: Cross-Reference with Codebases
-
-If needed, compare against reference codebases in `workspace/repos/`:
-
-- Are key algorithmic patterns correctly adapted?
-- Were critical implementation details preserved during adaptation?
-
-## Review Output
-
-Write the review to `workspace/iterations/judge_vN.md` (increment N for each review iteration):
-
-```markdown
-# Review vN
-
-## Verdict: PASS / NEEDS_REVISION
-
-## Atomic Definition Checklist
-
-| Definition | Implemented | Correct | Notes |
-|-----------|-------------|---------|-------|
-| [def 1] | Yes/No | Yes/No | [details] |
-| [def 2] | Yes/No | Yes/No | [details] |
-| ... | ... | ... | ... |
-
-## Plan Compliance
-
-| Section | Status | Notes |
-|---------|--------|-------|
-| Dataset | OK / Issue | ... |
-| Model | OK / Issue | ... |
-| Training | OK / Issue | ... |
-| Testing | OK / Issue | ... |
-
-## Issues (if NEEDS_REVISION)
-
-### Issue 1: [Title]
-- **Location**: `project/model/attention.py`, line ~42
-- **Problem**: [Description of what's wrong]
-- **Expected**: [What the correct implementation should do]
-- **Suggestion**: [Specific fix]
-
-### Issue 2: [Title]
-...
-
-## Summary
-[Brief overall assessment: what's good, what needs work]
-```
-
-## Verdict Criteria
-
-**PASS** if:
-- All atomic definitions are implemented and correct.
-- All plan sections are satisfied.
-- Code runs end-to-end with decreasing loss.
-- No critical bugs.
-
-**NEEDS_REVISION** if:
-- Any atomic definition is missing or incorrectly implemented.
-- Any plan section has significant gaps.
-- Code has bugs that prevent correct execution.
-- Implementation is a toy/stub rather than a genuine attempt.
-
-## Iteration Rules
-
-- Each review is independent: re-evaluate everything, not just previously flagged issues.
-- Be specific in suggestions: cite file names, line numbers, and concrete fixes.
-- After 3 iterations of NEEDS_REVISION, escalate to the user with a summary of remaining issues.
-- Never approve code that doesn't run or produces NaN/Inf.

package/skills/research-pipeline/references/prompts/survey.md

@@ -1,105 +0,0 @@
-# Literature Survey Guide
-
-You are performing a literature survey to bridge theory and implementation. Your goal is to extract actionable knowledge from papers and codebases that will directly inform the implementation.
-
-## Process
-
-### Phase 1: Decompose the Idea
-
-Before reading any papers, break the research idea (from `task.json`) into **atomic academic definitions**. Each atomic definition must be:
-
-- A single, self-contained concept (e.g., "multi-head attention", "contrastive loss", "graph convolution").
-- Have clear mathematical foundations.
-- Be implementable as a code module.
-- Be traceable to specific papers.
-
-Write down your list of atomic definitions before proceeding. This ensures systematic coverage.
-
-### Phase 2: Paper Reading (per paper)
-
-For each paper in `workspace/papers/`:
-
-1. **Skim first**: Read title, abstract, introduction, and conclusion to understand the paper's scope.
-2. **Targeted reading**: For each atomic definition relevant to this paper, find:
-   - The formal definition (usually in a "Method" or "Approach" section).
-   - Mathematical formulas (equations, loss functions, update rules).
-   - Key theoretical claims or properties.
-3. **Search strategically**: Use keyword search within the .tex file. Look for `\begin{equation}`, `\mathcal`, `\text{loss}`, etc.
-
-### Phase 3: Code Reading (per repo)
-
-For each reference codebase in `workspace/repos/`:
-
-1. **Understand structure**: List the directory tree first.
-2. **Find implementations**: Map each mathematical formula to its code implementation:
-   - Model architecture classes → model definition formulas
-   - Loss function implementations → loss formulas
-   - Data processing pipelines → input/output specifications
-3. **Document the mapping**: For each formula, note the exact file, class, and function that implements it.
-
-### Phase 4: Write Notes
-
-For each paper, create `workspace/notes/paper_NNN.md` with this structure:
-
-```markdown
-# [Paper Title]
-ArXiv: [id] | Authors: [first author et al.]
-
-## Core Method
-[1-2 paragraph summary of the paper's main contribution]
-
-## Atomic Definitions Covered
-[List which atomic definitions from Phase 1 this paper addresses]
-
-## Math Formulas
-
-### [Definition Name 1]
-$$formula$$
-- Variables: [explain each variable]
-- Context: [when/where this is applied]
-
-### [Definition Name 2]
-...
-
-## Code Implementation
-
-### [Definition Name 1]
-- **Repo**: repos/[name]
-- **File**: path/to/file.py, class ClassName, method method_name
-- **Key logic**:
-```python
-# Excerpt of the most relevant 10-30 lines
-```
-- **Notes**: [any adaptations, simplifications, or deviations from the paper]
-
-## Key Insights
-- [Insight 1: anything surprising or important for implementation]
-- [Insight 2: ...]
-```
-
-### Phase 5: Synthesize
-
-After all papers are surveyed, write `workspace/survey_res.md`:
-
-```markdown
-# Literature Survey: [Research Idea]
-
-## Atomic Definitions
-[Complete list with brief descriptions]
-
-## Theory-to-Code Mapping
-[For each atomic definition: the formula, which papers define it, which repos implement it, and the recommended implementation approach]
-
-## Implementation Recommendations
-[Which reference implementations to adapt, potential pitfalls, suggested architecture]
-
-## Open Questions
-[Anything unclear that may need user input]
-```
-
-## Quality Criteria
-
-- Every atomic definition must appear in at least one paper note.
-- Every formula must have a corresponding code reference (or be flagged as "no reference found").
-- Do not skip papers. If a paper is not relevant, note why and move on.
-- Err on the side of extracting more detail rather than less. The implementation step depends on this survey.

package/skills/research-pipeline/references/workspace-spec.md

@@ -1,81 +0,0 @@
-# Workspace Directory Specification
-
-All research pipeline artifacts live in a `workspace/` directory. The location is either specified by the user or defaults to the current working directory plus `workspace/`.
-
-## Directory Layout
-
-```
-workspace/
-  task.json                 # Input: research task definition
-  search_results.md         # Step 2: arxiv + github search results
-  prepare_res.md            # Step 3: selected repos and rationale
-  survey_res.md             # Step 5: synthesized literature survey
-  plan_res.md               # Step 6: four-part implementation plan
-  ml_res.md                 # Step 7: implementation report
-  experiment_res.md         # Step 10: full training results
-
-  repos/                    # Step 3: cloned reference repositories
-    repo-name-1/
-    repo-name-2/
-
-  papers/                   # Step 4: downloaded paper sources
-    2401.12345.tex
-    2401.67890.tex
-
-  notes/                    # Step 5: per-paper survey notes
-    paper_001.md
-    paper_002.md
-
-  iterations/               # Steps 8-9: review history
-    judge_v1.md
-    judge_v2.md
-
-  project/                  # Step 7: implementation code
-    model/
-    data/
-    training/
-    testing/
-    utils/
-    run.py
-    requirements.txt
-```
-
-## Conventions
-
-### File Existence = Step Completion
-
-The research pipeline uses file existence as the checkpoint mechanism. Before executing any step, check whether its output file already exists. If it does, skip the step.
-
-This enables:
-- **Crash recovery**: resume from the last completed step.
-- **Incremental progress**: re-running the pipeline skips completed work.
-- **Transparency**: a human can inspect progress by listing the directory.
-
-### Naming Rules
-
-- Markdown files (`.md`) for human-readable outputs.
-- JSON files (`.json`) for structured data (task definition).
-- Paper notes use sequential numbering: `paper_001.md`, `paper_002.md`.
-- Review iterations use version numbering: `judge_v1.md`, `judge_v2.md`.
-
-### Immutability
-
-Once a step's output is written, do NOT modify it unless the user explicitly asks. If a step needs to be re-done, delete the output file first, then re-execute.
-
-Exception: `workspace/project/` is mutable during the implement-review-iterate loop (Steps 7-9).
-
-### task.json Schema
-
-```json
-{
-  "idea": "A 1-3 sentence description of the research idea",
-  "references": ["2401.12345", "paper title string"],
-  "domain": "recommendation systems",
-  "date_limit": "2024-01-01"
-}
-```
-
-- `idea` (required): The core research idea to implement.
-- `references` (optional): ArXiv IDs or paper titles as starting points.
-- `domain` (optional): Research domain for focused searching.
-- `date_limit` (optional): Only consider papers published after this date.