@saibolla/ada 0.1.2

package/skills/hypothesis-generation/references/experimental_design_patterns.md

@@ -0,0 +1,329 @@
+# Experimental Design Patterns
+
+## Common Approaches to Testing Scientific Hypotheses
+
+This reference provides patterns and frameworks for designing experiments across scientific domains. Use these patterns to develop rigorous tests for generated hypotheses.
+
+**Note on Report Structure:** When generating hypothesis reports, mention only the key experimental approach (e.g., "in vivo knockout study" or "prospective cohort design") in the main text hypothesis boxes. Include comprehensive experimental protocols with full methods, controls, sample sizes, statistical approaches, feasibility assessments, and resource requirements in **Appendix B: Detailed Experimental Designs**.
+
+## Design Selection Framework
+
+Choose experimental approaches based on:
+- **Nature of hypothesis:** Mechanistic, causal, correlational, descriptive
+- **System studied:** In vitro, in vivo, computational, observational
+- **Feasibility:** Time, cost, ethics, technical capabilities
+- **Evidence needed:** Proof-of-concept, causal demonstration, quantitative relationship
+
+## Laboratory Experimental Designs
+
+### In Vitro Experiments
+
+**When to use:** Testing molecular, cellular, or biochemical mechanisms in controlled systems.
+
+**Common patterns:**
+
+#### 1. Dose-Response Studies
+- **Purpose:** Establish quantitative relationship between input and effect
+- **Design:** Test multiple concentrations/doses of intervention
+- **Key elements:**
+  - Negative control (no treatment)
+  - Positive control (known effective treatment)
+  - Multiple dose levels (typically 5-8 points)
+  - Technical replicates (≥3 per condition)
+  - Appropriate statistical analysis (curve fitting, IC50/EC50 determination)
+
+**Example application:**
+"To test if compound X inhibits enzyme Y, measure enzyme activity at 0, 1, 10, 100, 1000 nM compound X concentrations with n=3 replicates per dose."
+
+#### 2. Gain/Loss of Function Studies
+- **Purpose:** Establish causal role of specific component
+- **Design:** Add (overexpression) or remove (knockout/knockdown) component
+- **Key elements:**
+  - Wild-type control
+  - Gain-of-function condition (overexpression, constitutive activation)
+  - Loss-of-function condition (knockout, knockdown, inhibition)
+  - Rescue experiment (restore function to loss-of-function)
+  - Measure downstream effects
+
+**Example application:**
+"Test if protein X causes phenotype Y by: (1) knocking out X and observing phenotype loss, (2) overexpressing X and observing phenotype enhancement, (3) rescuing knockout with X re-expression."
+
+#### 3. Time-Course Studies
+- **Purpose:** Understand temporal dynamics and sequence of events
+- **Design:** Measure outcomes at multiple time points
+- **Key elements:**
+  - Time 0 baseline
+  - Early time points (capture rapid changes)
+  - Intermediate time points
+  - Late time points (steady state)
+  - Sufficient replication at each time point
+
+**Example application:**
+"Measure protein phosphorylation at 0, 5, 15, 30, 60, 120 minutes after stimulus to determine peak activation timing."
+
+### In Vivo Experiments
+
+**When to use:** Testing hypotheses in whole organisms to assess systemic, physiological, or behavioral effects.
+
+**Common patterns:**
+
+#### 4. Between-Subjects Designs
+- **Purpose:** Compare different groups receiving different treatments
+- **Design:** Randomly assign subjects to treatment groups
+- **Key elements:**
+  - Random assignment to groups
+  - Appropriate sample size (power analysis)
+  - Control group (vehicle, sham, or standard treatment)
+  - Blinding (single or double-blind)
+  - Standardized conditions across groups
+
+**Example application:**
+"Randomly assign 20 mice each to vehicle control or drug treatment groups, measure tumor size weekly for 8 weeks, with experimenters blinded to group assignment."
+
+#### 5. Within-Subjects (Repeated Measures) Designs
+- **Purpose:** Each subject serves as own control, reducing inter-subject variability
+- **Design:** Same subjects measured across multiple conditions/time points
+- **Key elements:**
+  - Baseline measurements
+  - Counterbalancing (if order effects possible)
+  - Washout periods (for sequential treatments)
+  - Appropriate repeated-measures statistics
+
+**Example application:**
+"Measure cognitive performance in same participants at baseline, after training intervention, and at 3-month follow-up."
+
+#### 6. Factorial Designs
+- **Purpose:** Test multiple factors and their interactions simultaneously
+- **Design:** Cross all levels of multiple independent variables
+- **Key elements:**
+  - Clear main effects and interactions
+  - Sufficient power for interaction tests
+  - Full factorial or fractional factorial as appropriate
+
+**Example application:**
+"2×2 design crossing genotype (WT vs. mutant) × treatment (vehicle vs. drug) to test whether drug effect depends on genotype."
+
+### Computational/Modeling Experiments
+
+**When to use:** Testing hypotheses about complex systems, making predictions, or when physical experiments are infeasible.
+
+#### 7. In Silico Simulations
+- **Purpose:** Model complex systems, test theoretical predictions
+- **Design:** Implement computational model and vary parameters
+- **Key elements:**
+  - Well-defined model with explicit assumptions
+  - Parameter sensitivity analysis
+  - Validation against known data
+  - Prediction generation for experimental testing
+
+**Example application:**
+"Build agent-based model of disease spread, vary transmission rate and intervention timing, compare predictions to empirical epidemic data."
+
+#### 8. Bioinformatics/Meta-Analysis
+- **Purpose:** Test hypotheses using existing datasets
+- **Design:** Analyze large-scale data or aggregate multiple studies
+- **Key elements:**
+  - Appropriate statistical corrections (multiple testing)
+  - Validation in independent datasets
+  - Control for confounds and batch effects
+  - Clear inclusion/exclusion criteria
+
+**Example application:**
+"Test if gene X expression correlates with survival across 15 cancer datasets (n>5000 patients total), using Cox regression with clinical covariates."
+
+## Observational Study Designs
+
+### When Physical Manipulation is Impossible or Unethical
+
+#### 9. Cross-Sectional Studies
+- **Purpose:** Examine associations at a single time point
+- **Design:** Measure variables of interest in population at one time
+- **Strengths:** Fast, inexpensive, can establish prevalence
+- **Limitations:** Cannot establish temporality or causation
+- **Key elements:**
+  - Representative sampling
+  - Standardized measurements
+  - Control for confounding variables
+  - Appropriate statistical analysis
+
+**Example application:**
+"Survey 1000 adults to test association between diet pattern and biomarker X, controlling for age, sex, BMI, and physical activity."
+
+#### 10. Cohort Studies (Prospective/Longitudinal)
+- **Purpose:** Establish temporal relationships and potentially causal associations
+- **Design:** Follow group over time, measuring exposures and outcomes
+- **Strengths:** Can establish temporality, calculate incidence
+- **Limitations:** Time-consuming, expensive, subject attrition
+- **Key elements:**
+  - Baseline exposure assessment
+  - Follow-up at defined intervals
+  - Minimize loss to follow-up
+  - Account for time-varying confounders
+
+**Example application:**
+"Follow 5000 initially healthy individuals for 10 years, testing if baseline vitamin D levels predict cardiovascular disease incidence."
+
+#### 11. Case-Control Studies
+- **Purpose:** Efficiently study rare outcomes by comparing cases to controls
+- **Design:** Identify cases with outcome, select matched controls, compare exposures
+- **Strengths:** Efficient for rare diseases, relatively quick
+- **Limitations:** Recall bias, selection bias, cannot calculate incidence
+- **Key elements:**
+  - Clear case definition
+  - Appropriate control selection (matching or statistical adjustment)
+  - Retrospective exposure assessment
+  - Control for confounding
+
+**Example application:**
+"Compare 200 patients with rare disease X to 400 matched controls without X, testing if early-life exposure Y differs between groups."
+
+## Clinical Trial Designs
+
+#### 12. Randomized Controlled Trials (RCTs)
+- **Purpose:** Gold standard for testing interventions in humans
+- **Design:** Randomly assign participants to treatment or control
+- **Key elements:**
+  - Randomization (simple, block, or stratified)
+  - Concealment of allocation
+  - Blinding (participants, providers, assessors)
+  - Intention-to-treat analysis
+  - Pre-registered protocol and analysis plan
+
+**Example application:**
+"Double-blind RCT: randomly assign 300 patients to receive drug X or placebo for 12 weeks, measure primary outcome of symptom improvement."
+
+#### 13. Crossover Trials
+- **Purpose:** Each participant receives all treatments in sequence
+- **Design:** Participants crossed over between treatments with washout
+- **Strengths:** Reduces inter-subject variability, requires fewer participants
+- **Limitations:** Order effects, requires reversible conditions, longer duration
+- **Key elements:**
+  - Adequate washout period
+  - Randomized treatment order
+  - Carryover effect assessment
+
+**Example application:**
+"Crossover trial: participants receive treatment A for 4 weeks, 2-week washout, then treatment B for 4 weeks (randomized order)."
+
+## Advanced Design Considerations
+
+### Sample Size and Statistical Power
+
+**Key questions:**
+- What effect size is meaningful to detect?
+- What statistical test will be used?
+- What alpha (significance level) and beta (power) are appropriate?
+- What is expected variability in the measurement?
+
+**General guidelines:**
+- Conduct formal power analysis before experiment
+- For pilot studies, n≥10 per group minimum
+- For definitive studies, aim for ≥80% power
+- Account for potential attrition in longitudinal studies
+
+### Controls
+
+**Types of controls:**
+- **Negative control:** No intervention (baseline)
+- **Positive control:** Known effective intervention (validates system)
+- **Vehicle control:** Delivery method without active ingredient
+- **Sham control:** Mimics intervention without active component (surgery, etc.)
+- **Historical control:** Prior data (weakest, avoid if possible)
+
+### Blinding
+
+**Levels:**
+- **Open-label:** No blinding (acceptable for objective measures)
+- **Single-blind:** Participants blinded (reduces placebo effects)
+- **Double-blind:** Participants and experimenters blinded (reduces bias in assessment)
+- **Triple-blind:** Participants, experimenters, and analysts blinded (strongest)
+
+### Replication
+
+**Technical replicates:** Repeated measurements on same sample
+- Reduce measurement error
+- Typically 2-3 replicates sufficient
+
+**Biological replicates:** Independent samples/subjects
+- Address biological variability
+- Critical for generalization
+- Minimum: n≥3, preferably n≥5-10 per group
+
+**Experimental replicates:** Repeat entire experiment
+- Validate findings across time, equipment, operators
+- Gold standard for confirming results
+
+### Confound Control
+
+**Strategies:**
+- **Randomization:** Distribute confounds evenly across groups
+- **Matching:** Pair similar subjects across conditions
+- **Blocking:** Group by confound, then randomize within blocks
+- **Statistical adjustment:** Measure confounds and adjust in analysis
+- **Standardization:** Keep conditions constant across groups
+
+## Selecting Appropriate Design
+
+**Decision tree:**
+
+1. **Can variables be manipulated?**
+   - Yes → Experimental design (RCT, lab experiment)
+   - No → Observational design (cohort, case-control, cross-sectional)
+
+2. **What is the system?**
+   - Cells/molecules → In vitro experiments
+   - Whole organisms → In vivo experiments
+   - Humans → Clinical trials or observational studies
+   - Complex systems → Computational modeling
+
+3. **What is the primary goal?**
+   - Mechanism → Gain/loss of function, dose-response
+   - Causation → RCT, cohort study with good controls
+   - Association → Cross-sectional, case-control
+   - Prediction → Modeling, machine learning
+   - Temporal dynamics → Time-course, longitudinal
+
+4. **What are the constraints?**
+   - Time limited → Cross-sectional, in vitro
+   - Budget limited → Computational, observational
+   - Ethical concerns → Observational, in vitro
+   - Rare outcome → Case-control, meta-analysis
+
+## Integrating Multiple Approaches
+
+Strong hypothesis testing often combines multiple designs:
+
+**Example: Testing if microbiome affects cognitive function**
+1. **Observational:** Cohort study showing association between microbiome composition and cognition
+2. **Animal model:** Germ-free mice receiving microbiome transplants show cognitive changes
+3. **Mechanism:** In vitro studies showing microbial metabolites affect neuronal function
+4. **Clinical trial:** RCT of probiotic intervention improving cognitive scores
+5. **Computational:** Model predicting which microbiome profiles should affect cognition
+
+**Triangulation approach:**
+- Each design addresses different aspects/limitations
+- Convergent evidence from multiple approaches strengthens causal claims
+- Start with observational/in vitro, then move to definitive causal tests
+
+## Common Pitfalls
+
+- Insufficient sample size (underpowered)
+- Lack of appropriate controls
+- Confounding variables not accounted for
+- Inappropriate statistical tests
+- P-hacking or multiple testing without correction
+- Lack of blinding when subjective assessments involved
+- Failure to replicate findings
+- Not pre-registering analysis plans (clinical trials)
+
+## Practical Application for Hypothesis Testing
+
+When designing experiments to test hypotheses:
+
+1. **Match design to hypothesis specifics:** Causal claims require experimental manipulation; associations can use observational designs
+2. **Start simple, then elaborate:** Pilot with simple design, then add complexity
+3. **Plan controls carefully:** Controls validate the system and isolate the specific effect
+4. **Consider feasibility:** Balance ideal design with practical constraints
+5. **Plan for multiple experiments:** Rarely does one experiment definitively test a hypothesis
+6. **Pre-specify analysis:** Decide statistical tests before data collection
+7. **Build in validation:** Independent replication, orthogonal methods, convergent evidence

package/skills/hypothesis-generation/references/hypothesis_quality_criteria.md

@@ -0,0 +1,198 @@
+# Hypothesis Quality Criteria
+
+## Framework for Evaluating Scientific Hypotheses
+
+Use these criteria to assess the quality and rigor of generated hypotheses. A robust hypothesis should score well across multiple dimensions.
+
+**Note on Report Structure:** When generating hypothesis reports, provide a brief quality assessment summary in the main text (comparative table with ratings), and include detailed evaluation with strengths, weaknesses, and comprehensive analysis in **Appendix C: Quality Assessment**.
+
+## Core Criteria
+
+### 1. Testability
+
+**Definition:** The hypothesis can be empirically tested through observation or experimentation.
+
+**Evaluation questions:**
+- Can specific experiments or observations test this hypothesis?
+- Are the predicted outcomes measurable?
+- Can the hypothesis be tested with current or near-future methods?
+- Are there multiple independent ways to test it?
+
+**Strong testability examples:**
+- "Increased expression of protein X will reduce cell proliferation rate by >30%"
+- "Patients receiving treatment Y will show 50% reduction in symptom Z within 4 weeks"
+
+**Weak testability examples:**
+- "This process is influenced by complex interactions" (vague, no specific prediction)
+- "The mechanism involves quantum effects" (if no method to test quantum effects exists)
+
+### 2. Falsifiability
+
+**Definition:** Clear conditions or observations would disprove the hypothesis (Popperian criterion).
+
+**Evaluation questions:**
+- What specific observations would prove this hypothesis wrong?
+- Are the falsifying conditions realistic to observe?
+- Is the hypothesis stated clearly enough to be disproven?
+- Can null results meaningfully falsify the hypothesis?
+
+**Strong falsifiability examples:**
+- "If we knock out gene X, phenotype Y will disappear" (can be falsified if phenotype persists)
+- "Drug A will outperform placebo in 80% of patients" (clear falsification threshold)
+
+**Weak falsifiability examples:**
+- "Multiple factors contribute to the outcome" (too vague to falsify)
+- "The effect may vary depending on context" (built-in escape clauses)
+
+### 3. Parsimony (Occam's Razor)
+
+**Definition:** Among competing hypotheses with equal explanatory power, prefer the simpler explanation.
+
+**Evaluation questions:**
+- Does the hypothesis invoke the minimum number of entities/mechanisms needed?
+- Are all proposed elements necessary to explain the phenomenon?
+- Could a simpler mechanism account for the observations?
+- Does it avoid unnecessary assumptions?
+
+**Parsimony considerations:**
+- Simple ≠ simplistic; complexity is justified when evidence demands it
+- Established mechanisms are "simpler" than novel, unproven ones
+- Direct mechanisms are simpler than elaborate multi-step pathways
+- One well-supported mechanism beats multiple speculative ones
+
+### 4. Explanatory Power
+
+**Definition:** The hypothesis accounts for a substantial portion of the observed phenomenon.
+
+**Evaluation questions:**
+- How much of the observed data does this hypothesis explain?
+- Does it account for both typical and atypical observations?
+- Can it explain related phenomena beyond the immediate observation?
+- Does it resolve apparent contradictions in existing data?
+
+**Strong explanatory power indicators:**
+- Explains multiple independent observations
+- Accounts for quantitative relationships, not just qualitative patterns
+- Resolves previously puzzling findings
+- Makes sense of seemingly contradictory results
+
+**Limited explanatory power indicators:**
+- Only explains part of the phenomenon
+- Requires additional hypotheses for complete explanation
+- Leaves major observations unexplained
+
+### 5. Scope
+
+**Definition:** The range of phenomena and contexts the hypothesis can address.
+
+**Evaluation questions:**
+- Does it apply only to the specific case or to broader situations?
+- Can it generalize across conditions, species, or systems?
+- Does it connect to larger theoretical frameworks?
+- What are its boundaries and limitations?
+
+**Broader scope (generally preferable):**
+- Applies across multiple experimental conditions
+- Generalizes to related systems or species
+- Connects phenomenon to established principles
+
+**Narrower scope (acceptable if explicitly defined):**
+- Limited to specific conditions or contexts
+- Requires different mechanisms in different settings
+- Context-dependent with clear boundaries
+
+### 6. Consistency with Established Knowledge
+
+**Definition:** Alignment with well-supported theories, principles, and empirical findings.
+
+**Evaluation questions:**
+- Is it consistent with established physical, chemical, or biological principles?
+- Does it align with or reasonably extend current theories?
+- If contradicting established knowledge, is there strong justification?
+- Does it require violating well-supported laws or findings?
+
+**Levels of consistency:**
+- **Fully consistent:** Applies established mechanisms in new context
+- **Mostly consistent:** Extends current understanding in plausible ways
+- **Partially inconsistent:** Contradicts some findings but has explanatory value
+- **Highly inconsistent:** Requires rejecting well-established principles (requires exceptional evidence)
+
+### 7. Novelty and Insight
+
+**Definition:** The hypothesis offers new understanding beyond merely restating known facts.
+
+**Evaluation questions:**
+- Does it provide new mechanistic insight?
+- Does it challenge assumptions or conventional wisdom?
+- Does it suggest unexpected connections or relationships?
+- Does it open new research directions?
+
+**Novel contributions:**
+- Proposes previously unconsidered mechanisms
+- Reframes the problem in a productive way
+- Connects disparate observations
+- Suggests non-obvious testable predictions
+
+**Note:** Novelty alone doesn't make a hypothesis valuable; it must also be testable, parsimonious, and explanatory.
+
+## Comparative Evaluation
+
+When evaluating multiple competing hypotheses:
+
+### Trade-offs and Balancing
+
+Hypotheses often involve trade-offs:
+- More parsimonious but less explanatory power
+- Broader scope but less testable with current methods
+- Novel insights but less consistent with current knowledge
+
+**Evaluation approach:**
+- No hypothesis needs to be perfect on all dimensions
+- Identify each hypothesis's strengths and weaknesses
+- Consider which criteria are most important for the specific phenomenon
+- Note which hypotheses are most immediately testable
+- Identify which would be most informative if supported
+
+### Distinguishability
+
+**Key question:** Can experiments distinguish between competing hypotheses?
+
+- Identify predictions that differ between hypotheses
+- Prioritize hypotheses that make distinct predictions
+- Note which experiments would most efficiently narrow the field
+- Consider whether hypotheses could all be partially correct
+
+## Common Pitfalls
+
+### Untestable Hypotheses
+- Too vague to generate specific predictions
+- Invoke unobservable or unmeasurable entities
+- Require technology that doesn't exist
+
+### Unfalsifiable Hypotheses
+- Built-in escape clauses ("may or may not occur")
+- Post-hoc explanations that fit any outcome
+- No specification of what would disprove them
+
+### Overly Complex Hypotheses
+- Invoke multiple unproven mechanisms
+- Add unnecessary steps or entities
+- Complexity not justified by explanatory gains
+
+### Just-So Stories
+- Plausible narratives without testable predictions
+- Explain observations but don't predict new ones
+- Impossible to distinguish from alternative stories
+
+## Practical Application
+
+When generating hypotheses:
+
+1. **Draft initial hypotheses** focusing on mechanistic explanations
+2. **Apply quality criteria** to identify weaknesses
+3. **Refine hypotheses** to improve testability and clarity
+4. **Develop specific predictions** to enhance testability and falsifiability
+5. **Compare systematically** across all criteria
+6. **Prioritize for testing** based on distinguishability and feasibility
+
+Remember: The goal is not a perfect hypothesis, but a set of testable, falsifiable, informative hypotheses that advance understanding of the phenomenon.