maestro-flow 0.5.31 → 0.5.33

package/.agy/skills/scholar-ideation/phases/02-gap-analysis.md

@@ -0,0 +1,249 @@
+# Phase 2: Gap Analysis
+
+Systematically identify and evaluate research gaps from the literature review to find valuable research opportunities.
+
+## Objective
+
+- Analyze literature across 5 gap types (literature, methodological, application, interdisciplinary, temporal)
+- Evaluate gaps along 5 analysis dimensions
+- Prioritize gaps by importance, novelty, and feasibility
+- Produce a structured gap analysis with actionable research opportunities
+
+## Input
+
+- **literatureResults**: Papers, trends, and key findings from Phase 1
+- **workflowPreferences.topic**: Research topic
+- **workflowPreferences.scope**: Research scope
+
+## Execution
+
+### Step 2.1: Systematic Literature Analysis
+
+Build a comparison matrix from the papers discovered in Phase 1.
+
+**Comparison matrix**:
+
+| Paper | Method | Dataset | Performance | Key Innovation | Limitations |
+|-------|--------|---------|-------------|----------------|-------------|
+| Paper A | Method X | Dataset 1 | Metric: value | Innovation A | Limitation A |
+| Paper B | Method Y | Dataset 2 | Metric: value | Innovation B | Limitation B |
+| ... | ... | ... | ... | ... | ... |
+
+Focus on:
+- What methods are used and how they compare
+- What datasets and benchmarks are standard
+- What performance levels are achieved
+- What each paper identifies as limitations or future work
+
+### Step 2.2: Five-Type Gap Identification
+
+Analyze the literature systematically across five gap types.
+
+**Type 1: Literature Gaps**
+
+Topics or questions not yet sufficiently studied.
+
+Identification methods:
+- Find sub-areas with few papers (< 20 relevant publications)
+- Analyze "Future Work" sections of survey/review papers
+- Identify low-citation but potentially important research directions
+- Discover emerging topics with limited coverage
+
+Questions to answer:
+- Which sub-topics within the field have received minimal attention?
+- Are there important questions that no paper directly addresses?
+- What have review papers flagged as under-explored?
+
+**Type 2: Methodological Gaps**
+
+Limitations and improvement opportunities in existing methods.
+
+Identification methods:
+- Analyze common weaknesses across papers (from comparison matrix Limitations column)
+- Identify scenarios where methods fail or degrade
+- Find computational efficiency or scalability problems
+- Spot gaps between theoretical guarantees and practical performance
+
+Questions to answer:
+- What limitations do ALL current methods share?
+- In what specific scenarios do existing approaches fail?
+- Are there well-known theoretical results with no practical implementation?
+
+**Type 3: Application Gaps**
+
+Theory-to-practice transfer opportunities and new application domains.
+
+Identification methods:
+- Identify theoretical work lacking real-world validation
+- Find successful methods that haven't been applied to relevant new domains
+- Spot disconnects between industry needs and academic research
+- Discover technology transfer possibilities
+
+Questions to answer:
+- Which promising methods have not been tested in practical settings?
+- What industry problems could benefit from existing academic methods?
+- Are there successful cross-domain transfers that haven't been attempted?
+
+**Type 4: Interdisciplinary Gaps**
+
+Research opportunities at the intersection of different fields.
+
+Identification methods:
+- Identify similar problems solved differently across fields
+- Find methods that could transfer across disciplines
+- Discover complex problems requiring multi-disciplinary collaboration
+- Spot emerging cross-disciplinary areas
+
+Questions to answer:
+- Are there related fields with transferable methods?
+- Do similar problems appear in different domains with different solutions?
+- What new fields are emerging at discipline boundaries?
+
+**Type 5: Temporal Gaps**
+
+New research needs arising from changes over time.
+
+Identification methods:
+- Identify new problems created by recent technology advances
+- Find data distribution shifts that affect existing methods
+- Discover new challenges from changing social/regulatory requirements
+- Spot opportunities from recent technological breakthroughs
+
+Questions to answer:
+- What new technologies have created previously impossible research directions?
+- Have recent developments invalidated assumptions in older work?
+- What regulatory or societal changes demand new research?
+
+### Step 2.3: Five-Dimension Analysis
+
+For each identified gap, evaluate along five dimensions.
+
+**Dimension 1: Research Topic Coverage**
+- How many high-quality papers exist on this topic?
+- Thresholds: Sufficient (>100), Moderate (20-100), Under-researched (<20), Unexplored (~0)
+- How many active research groups are working on it?
+
+**Dimension 2: Method Strengths/Weaknesses Comparison**
+- What are the theoretical foundations of existing methods?
+- How do they perform experimentally?
+- What is the computational complexity?
+- How well do they generalize and scale?
+
+**Dimension 3: Experimental Setup Completeness**
+- Are experimental scenarios diverse enough?
+- Do benchmarks cover the full problem space?
+- Are evaluation metrics comprehensive?
+- Are ablation studies thorough?
+
+**Dimension 4: Dataset and Benchmark Availability**
+- Are there public datasets of sufficient quality?
+- Are there standardized benchmarks?
+- Are datasets diverse and representative?
+- Is data annotation quality adequate?
+
+**Dimension 5: Theory-Practice Gap**
+- Do theoretical assumptions match real conditions?
+- Are methods practically deployable?
+- Do theoretical guarantees hold in experiments?
+- Is there industry adoption?
+
+### Step 2.4: Gap Prioritization
+
+Score each identified gap on three criteria (1-5 scale):
+
+**Importance**: Academic and practical value of filling this gap
+- 5: Breakthrough potential, affects the entire field
+- 4: Important, multiple groups would benefit
+- 3: Valuable, relevant to a significant subset
+- 2: Marginal, limited audience
+- 1: Trivial, minimal impact
+
+**Novelty**: Whether others are already addressing this gap
+- 5: Completely unexplored
+- 4: Very few attempts, mostly unsuccessful
+- 3: Some work exists but significant room remains
+- 2: Active area, incremental contribution possible
+- 1: Well-covered, minimal novelty
+
+**Feasibility**: Whether the gap can be addressed with available resources
+- 5: Fully feasible with current resources
+- 4: Feasible with some effort
+- 3: Challenging but possible
+- 2: Difficult, requires significant resources
+- 1: Nearly infeasible with available resources
+
+**Priority decision matrix**:
+
+| Importance | Novelty | Feasibility | Recommendation |
+|-----------|---------|-------------|----------------|
+| High | High | High | Top priority — pursue |
+| High | High | Medium | Worth attempting |
+| High | Medium | High | Safe choice |
+| Medium | High | High | Consider |
+| Low | * | * | Reconsider |
+
+### Step 2.5: Gap Validation
+
+Before finalizing, validate top gaps:
+- Search for very recent work (last 3 months) that might address the gap
+- Check arXiv preprints for ongoing related work
+- Verify no technical or data blockers exist
+- Consider whether the gap is a genuine research opportunity or a dead end
+
+```
+WebSearch: "{gap description keywords} {current year} arxiv"
+```
+
+If validation reveals the gap is being actively closed, downgrade priority or refine the gap definition.
+
+### Step 2.6: Gap Analysis Document
+
+Compile results into a structured analysis:
+
+```markdown
+## Gap Analysis: {Topic}
+
+### Identified Gaps
+
+#### Gap 1: {Title}
+- **Type**: Literature / Methodological / Application / Interdisciplinary / Temporal
+- **Description**: {Detailed description}
+- **Evidence**: {Papers/trends that reveal this gap}
+- **Importance**: {score}/5 — {justification}
+- **Novelty**: {score}/5 — {justification}
+- **Feasibility**: {score}/5 — {justification}
+- **Priority**: {Top / High / Medium / Low}
+
+#### Gap 2: {Title}
+...
+
+### Prioritized Opportunities
+1. {Top priority gap} — {one-line rationale}
+2. {Second priority} — {one-line rationale}
+3. {Third priority} — {one-line rationale}
+
+### Research Direction Recommendations
+- **Recommended focus**: {primary gap to pursue}
+- **Alternative directions**: {backup options}
+- **Combination opportunities**: {gaps that could be addressed together}
+```
+
+Present gap analysis to user for feedback (unless autoYes):
+```
+ask_question:
+  question: "Here is the gap analysis. Which gaps interest you most? Would you like to adjust priorities?"
+  → Incorporate user feedback into final priorities
+```
+
+## Output
+
+- **Variable**: `gapAnalysis` containing:
+  - `gaps[]` — List of identified gaps with type, description, evidence, and scores
+  - `opportunities[]` — Ranked research opportunities
+  - `prioritizedGaps[]` — Top 3-5 gaps sorted by priority
+  - `recommendedFocus` — Primary recommended research direction
+- **TodoWrite**: Mark Phase 2 completed, Phase 3 in_progress
+
+## Next Phase
+
+Return to orchestrator, then proceed to [Phase 3: Research Question Formulation](03-research-question.md).

package/.agy/skills/scholar-ideation/phases/03-research-question.md

@@ -0,0 +1,200 @@
+# Phase 3: Research Question Formulation
+
+Formulate specific, actionable research questions based on the identified gaps and literature analysis.
+
+## Objective
+
+- Transform prioritized gaps into concrete research questions using the SMART framework
+- Generate testable hypotheses for each research question
+- Define research objectives and expected contributions
+- Evaluate questions on importance, novelty, and feasibility
+
+## Input
+
+- **gapAnalysis**: Gaps, opportunities, and priorities from Phase 2
+- **literatureResults**: Papers, trends, and findings from Phase 1
+- **workflowPreferences.scope**: Research scope
+- **workflowPreferences.timeline**: Target timeline
+
+## Execution
+
+### Step 3.1: 5W1H Brainstorming
+
+For the top prioritized gap(s), apply the 5W1H framework to brainstorm research directions.
+
+**What** — What problem or phenomenon to study?
+- Identify the specific research object
+- Define the core problem to solve
+- Set the research scope and boundaries
+- Clarify expected outcomes
+
+**Why** — Why is this problem important?
+- Academic value: Does it advance the field?
+- Practical value: Does it solve real problems?
+- Timeliness: Why research this now?
+- Current limitations: What is missing?
+
+**Who** — Target audience and stakeholders?
+- Academic community: Which researchers care?
+- Industry: Which teams/companies benefit?
+- Users: Who uses the results?
+- Collaborators: Who should be involved?
+
+**When** — Time scope and context?
+- Research duration aligned with `workflowPreferences.timeline`
+- Is the timing right given current technology maturity?
+- When should results be available for maximum impact?
+
+**Where** — Application scenarios and domains?
+- What specific application contexts?
+- Which domains or industries?
+- What experimental settings?
+
+**How** — Preliminary methodology ideas?
+- What research methods fit?
+- What data and resources are needed?
+- How to validate hypotheses?
+- What are the technical challenges?
+
+Document the 5W1H analysis for the top 1-3 gaps.
+
+### Step 3.2: SMART Research Question Formulation
+
+Transform 5W1H insights into formal research questions using SMART criteria.
+
+**Specific**: The question must clearly define:
+- Research object (what exactly is being studied)
+- Improvement direction (what aspect is being improved)
+- Task/scenario (where this applies)
+- Goal (what outcome is expected)
+
+Bad: "How to improve model performance?"
+Good: "How can attention mechanism modifications improve Transformer performance on long-document understanding tasks?"
+
+**Measurable**: Define concrete evaluation criteria:
+- Quantitative metrics: accuracy, F1, BLEU, perplexity, latency
+- Qualitative metrics: human evaluation, case studies
+- Efficiency metrics: training time, inference speed, memory usage
+
+**Achievable**: Verify against available resources:
+- Computational resources (GPU count and type)
+- Data availability (public datasets)
+- Time constraints (aligned with timeline preference)
+- Team capabilities and expertise
+- Is there foundational work to build upon?
+
+**Relevant**: Confirm value alignment:
+- Addresses an identified gap (link back to gap analysis)
+- Has academic impact (advances methodology or understanding)
+- Has practical value (solves real problems)
+- Aligns with research interests and capabilities
+
+**Time-bound**: Set realistic timeline:
+- Short-term (3-6 months): Literature review, problem definition, proof of concept
+- Medium-term (6-12 months): Full experiments, analysis, paper writing
+- Long-term (1-2 years): Systematic research, multiple publications
+
+### Step 3.3: Research Question Types
+
+Classify each formulated question:
+
+**Exploratory questions**: Investigate unknown phenomena
+- Pattern: "What patterns/behaviors does X exhibit in context Y?"
+- Best for: New research areas, complex systems requiring understanding
+- Example: "What attention patterns does a Transformer exhibit when processing documents longer than 10K tokens?"
+
+**Confirmatory questions**: Test hypotheses or theories
+- Pattern: "Does X lead to Y in context Z?"
+- Best for: Verifying assumptions, challenging existing claims
+- Example: "Does increasing model depth improve long-document comprehension performance?"
+
+**Applied questions**: Solve practical problems
+- Pattern: "How can X be used/modified to achieve Y while satisfying constraint Z?"
+- Best for: Practical applications, optimization under constraints
+- Example: "How can sparse attention reduce Transformer memory usage by 50% while maintaining performance on NarrativeQA?"
+
+### Step 3.4: Hypothesis Generation
+
+For each research question, formulate testable hypotheses.
+
+**Hypothesis structure**:
+```
+IF [proposed intervention/method]
+THEN [expected outcome]
+BECAUSE [theoretical reasoning/evidence from literature]
+```
+
+**Example**:
+- **RQ**: How can adaptive sparse attention improve long-text Transformer efficiency?
+- **H1**: IF we use learned sparsity patterns based on content relevance, THEN computational cost decreases by >50% with <2% performance loss, BECAUSE most attention weights are near-zero in practice (evidence: [Paper X, Paper Y])
+- **H2**: IF we combine local and global attention with adaptive selection, THEN the model captures both local syntax and global semantics, BECAUSE existing fixed-pattern approaches (Longformer, BigBird) show this decomposition is effective
+
+### Step 3.5: Research Objectives and Contributions
+
+Define clear objectives and expected contributions for the top research question.
+
+**Primary objective**: One sentence describing the main goal
+**Sub-objectives**: 2-4 specific, measurable sub-goals
+
+**Expected contributions**:
+- **Academic**: New method, theoretical insight, comprehensive analysis
+- **Practical**: Improved efficiency, new tool, better performance
+- **Community**: Open-source code, new benchmark, reproducible experiments
+
+### Step 3.6: Research Question Evaluation
+
+Score each candidate question using the evaluation matrix.
+
+**Importance** (1-5):
+- 5: Breakthrough, affects the entire field
+- 4: Important, multiple groups interested
+- 3: Valuable, significant subset cares
+- 2: Marginal, limited audience
+- 1: Trivial, minimal impact
+
+**Novelty** (1-5):
+- 5: Entirely new question or breakthrough method
+- 4: New question or significantly improved method
+- 3: New perspective or method combination
+- 2: Incremental improvement
+- 1: Duplicates existing work
+
+**Feasibility** (1-5):
+- 5: Fully feasible, resources abundant
+- 4: Mostly feasible, resources adequate
+- 3: Challenging but achievable
+- 2: Difficult, requires breakthroughs
+- 1: Nearly infeasible
+
+**Decision matrix**:
+
+| Importance | Novelty | Feasibility | Recommendation |
+|-----------|---------|-------------|----------------|
+| High | High | High | Prioritize |
+| High | High | Medium | Worth pursuing |
+| High | Medium | High | Safe choice |
+| Medium | High | High | Consider |
+| Low | * | * | Reconsider |
+
+Present candidate research questions to user for selection (unless autoYes):
+```
+ask_question:
+  question: "Here are the candidate research questions with evaluations. Which would you like to pursue? You may also refine them."
+  → Store user selection/refinement
+```
+
+## Output
+
+- **Variable**: `researchQuestions` containing:
+  - `primaryQuestion` — The selected main research question
+  - `subQuestions[]` — Supporting sub-questions
+  - `hypotheses[]` — Testable hypotheses for each question
+  - `objectives` — Primary and sub-objectives
+  - `contributions` — Expected academic, practical, community contributions
+  - `questionType` — Exploratory / Confirmatory / Applied
+  - `evaluation` — Importance, novelty, feasibility scores
+- **TodoWrite**: Mark Phase 3 completed, Phase 4 in_progress
+
+## Next Phase
+
+Return to orchestrator, then proceed to [Phase 4: Method Selection](04-method-selection.md).

package/.agy/skills/scholar-ideation/phases/04-method-selection.md

@@ -0,0 +1,185 @@
+# Phase 4: Method Selection
+
+> **COMPACT SENTINEL [Phase 4: Method Selection]**
+> This phase contains 5 execution steps (Step 4.1 -- 4.5).
+> If you can read this sentinel but cannot find the full Step protocol below, context has been compressed.
+> Recovery: `view_file("phases/04-method-selection.md")`
+
+Select and justify appropriate research methods based on the formulated research questions and identified gaps.
+
+## Objective
+
+- Analyze existing methods and their applicability to the research questions
+- Select the most suitable research approach (theoretical, empirical, system-building, data analysis)
+- Justify method choice with evidence from literature
+- Assess required resources and technical feasibility
+- Identify potential technical risks and mitigation strategies
+
+## Input
+
+- **researchQuestions**: Primary question, sub-questions, hypotheses, objectives from Phase 3
+- **gapAnalysis**: Gaps, opportunities from Phase 2
+- **literatureResults**: Papers and methods from Phase 1
+- **workflowPreferences.timeline**: Target timeline
+
+## Execution
+
+### Step 4.1: Existing Method Analysis
+
+Review methods used in the literature (from Phase 1 comparison matrix) and classify them.
+
+**Method classification**:
+
+| Category | Description | When to Use |
+|----------|-------------|-------------|
+| Theoretical analysis | Mathematical proofs, complexity analysis, convergence guarantees | Questions requiring formal guarantees |
+| Empirical research | Experiment design, benchmarking, ablation studies | Performance evaluation, hypothesis testing |
+| System building | End-to-end systems, tool development, integration | Practical applications, deployment |
+| Data analysis | Visualization, pattern discovery, error analysis | Exploratory studies, model understanding |
+
+**For each relevant method from literature**:
+
+| Method | Used By | Strengths | Weaknesses | Applicable to Our RQ? |
+|--------|---------|-----------|------------|----------------------|
+| Method A | Paper X, Y | Strength 1, 2 | Weakness 1, 2 | Yes/No — reason |
+| Method B | Paper Z | Strength 1 | Weakness 1, 2 | Yes/No — reason |
+
+### Step 4.2: Method-Question Alignment
+
+> **CHECKPOINT**: Before proceeding, verify:
+> 1. This phase is TodoWrite `in_progress` (active phase protection)
+> 2. Full protocol (Step 4.1 -- 4.5) is in active memory, not just sentinel
+> If only sentinel remains -> `view_file("phases/04-method-selection.md")` now.
+
+Map each research question/hypothesis to suitable methods.
+
+**Question type to method mapping**:
+
+| Question Type | Primary Method | Supporting Methods |
+|--------------|----------------|-------------------|
+| Exploratory | Data analysis | Empirical research |
+| Confirmatory | Empirical research | Theoretical analysis |
+| Applied | System building | Empirical research |
+
+**For the primary research question**:
+- Which methods from the literature can be adapted?
+- What modifications are needed?
+- What new methods must be developed?
+- What combination of approaches works best?
+
+### Step 4.3: Resource and Feasibility Assessment
+
+For each candidate method, assess required resources.
+
+**Computational resources**:
+
+| Method | GPU Requirement | Training Time | Storage | Memory |
+|--------|----------------|---------------|---------|--------|
+| Method A | X GPUs | Y hours/days | Z GB | W GB |
+| Method B | ... | ... | ... | ... |
+
+**Data resources**:
+- Required datasets and their availability (public vs. private)
+- Data preprocessing requirements
+- Annotation needs
+- Data volume requirements
+
+**Human resources**:
+- Required expertise and skills
+- Team size considerations
+- Single-person vs. team project suitability
+
+**Time assessment** (aligned with workflowPreferences.timeline):
+
+| Timeline | Suitable Methods | Scope |
+|----------|-----------------|-------|
+| Short (3-6 months) | Small-scale experiments, data analysis, simple theoretical proofs | Proof of concept |
+| Medium (6-12 months) | Full experiments, system prototypes, moderate theory | Complete study |
+| Long (1-2 years) | Large-scale experiments, full systems, deep theory | Comprehensive research |
+
+### Step 4.4: Method Selection and Justification
+
+Based on the analysis, select the primary research method and supporting methods.
+
+**Selection criteria**:
+1. **Alignment**: Does the method directly address the research question?
+2. **Feasibility**: Can it be executed with available resources?
+3. **Innovation potential**: Does it enable novel contributions?
+4. **Risk level**: What is the probability of failure?
+5. **Literature precedent**: Have similar approaches succeeded in related work?
+
+**Method justification template**:
+
+```markdown
+### Selected Method: {Method Name}
+
+**Type**: Theoretical / Empirical / System-building / Data Analysis
+
+**Description**: {What the method involves}
+
+**Justification**:
+- Addresses RQ because: {reason linked to research question}
+- Builds on prior work: {specific papers and their methods}
+- Feasible because: {resource and capability assessment}
+- Novel because: {what is different from existing approaches}
+
+**Technical approach**:
+1. {Step 1 of the method}
+2. {Step 2 of the method}
+3. {Step 3 of the method}
+
+**Datasets**:
+| Dataset | Purpose | Size | Availability |
+|---------|---------|------|--------------|
+| Dataset A | Training | X samples | Public |
+| Dataset B | Evaluation | Y samples | Public |
+
+**Baselines for comparison**:
+- Baseline 1: {method} from {paper} — represents {approach}
+- Baseline 2: {method} from {paper} — represents {approach}
+
+**Evaluation metrics**:
+- Primary: {metric 1, metric 2}
+- Secondary: {metric 3, metric 4}
+- Efficiency: {training time, inference speed, memory}
+
+**Risk assessment**:
+| Risk | Probability | Impact | Mitigation |
+|------|------------|--------|------------|
+| Method doesn't improve over baselines | Medium | High | Prepare alternative approach B |
+| Insufficient data | Low | High | Data augmentation, synthetic data |
+| Computational budget exceeded | Medium | Medium | Start with smaller experiments |
+```
+
+### Step 4.5: Alternative Methods (Backup Plan)
+
+Identify 1-2 backup methods in case the primary approach fails.
+
+**Backup method criteria**:
+- Addresses the same or similar research question
+- Uses different technical approach (de-risks single point of failure)
+- Feasible within remaining timeline if pivot is needed
+- Simpler or more conservative than primary method
+
+Present method selection to user for confirmation (unless autoYes):
+```
+ask_question:
+  question: "Here is the proposed method selection with justification. Do you agree with this approach, or would you like to adjust?"
+  → Incorporate user feedback
+```
+
+## Output
+
+- **Variable**: `selectedMethods` containing:
+  - `primaryMethod` — Selected method with full justification
+  - `backupMethods[]` — Alternative approaches
+  - `datasets[]` — Required datasets with availability
+  - `baselines[]` — Comparison baselines
+  - `evaluationMetrics` — Primary and secondary metrics
+  - `resourceRequirements` — Compute, data, time, personnel needs
+  - `risks[]` — Identified risks with mitigation strategies
+- **TodoWrite**: Mark Phase 4 completed, Phase 5 in_progress
+
+## Next Phase
+
+Return to orchestrator, then proceed to [Phase 5: Research Planning](05-research-planning.md).