@synsci/cli-darwin-x64 1.1.97 → 1.1.99

package/bin/skills/scientific-critical-thinking/references/statistical_pitfalls.md

@@ -1,506 +0,0 @@
-# Common Statistical Pitfalls
-
-## P-Value Misinterpretations
-
-### Pitfall 1: P-Value = Probability Hypothesis is True
-**Misconception:** p = .05 means 5% chance the null hypothesis is true.
-
-**Reality:** P-value is the probability of observing data this extreme (or more) *if* the null hypothesis is true. It says nothing about the probability the hypothesis is true.
-
-**Correct interpretation:** "If there were truly no effect, we would observe data this extreme only 5% of the time."
-
-### Pitfall 2: Non-Significant = No Effect
-**Misconception:** p > .05 proves there's no effect.
-
-**Reality:** Absence of evidence ≠ evidence of absence. Non-significant results may indicate:
-- Insufficient statistical power
-- True effect too small to detect
-- High variability
-- Small sample size
-
-**Better approach:**
-- Report confidence intervals
-- Conduct power analysis
-- Consider equivalence testing
-
-### Pitfall 3: Significant = Important
-**Misconception:** Statistical significance means practical importance.
-
-**Reality:** With large samples, trivial effects become "significant." A statistically significant 0.1 IQ point difference is meaningless in practice.
-
-**Better approach:**
-- Report effect sizes
-- Consider practical significance
-- Use confidence intervals
-
-### Pitfall 4: P = .049 vs. P = .051
-**Misconception:** These are meaningfully different because one crosses the .05 threshold.
-
-**Reality:** These represent nearly identical evidence. The .05 threshold is arbitrary.
-
-**Better approach:**
-- Treat p-values as continuous measures of evidence
-- Report exact p-values
-- Consider context and prior evidence
-
-### Pitfall 5: One-Tailed Tests Without Justification
-**Misconception:** One-tailed tests are free extra power.
-
-**Reality:** One-tailed tests assume effects can only go one direction, which is rarely true. They're often used to artificially boost significance.
-
-**When appropriate:** Only when effects in one direction are theoretically impossible or equivalent to null.
-
-## Multiple Comparisons Problems
-
-### Pitfall 6: Multiple Testing Without Correction
-**Problem:** Testing 20 hypotheses at p < .05 gives ~65% chance of at least one false positive.
-
-**Examples:**
-- Testing many outcomes
-- Testing many subgroups
-- Conducting multiple interim analyses
-- Testing at multiple time points
-
-**Solutions:**
-- Bonferroni correction (divide α by number of tests)
-- False Discovery Rate (FDR) control
-- Prespecify primary outcome
-- Treat exploratory analyses as hypothesis-generating
-
-### Pitfall 7: Subgroup Analysis Fishing
-**Problem:** Testing many subgroups until finding significance.
-
-**Why problematic:**
-- Inflates false positive rate
-- Often reported without disclosure
-- "Interaction was significant in women" may be random
-
-**Solutions:**
-- Prespecify subgroups
-- Use interaction tests, not separate tests
-- Require replication
-- Correct for multiple comparisons
-
-### Pitfall 8: Outcome Switching
-**Problem:** Analyzing many outcomes, reporting only significant ones.
-
-**Detection signs:**
-- Secondary outcomes emphasized
-- Incomplete outcome reporting
-- Discrepancy between registration and publication
-
-**Solutions:**
-- Preregister all outcomes
-- Report all planned outcomes
-- Distinguish primary from secondary
-
-## Sample Size and Power Issues
-
-### Pitfall 9: Underpowered Studies
-**Problem:** Small samples have low probability of detecting true effects.
-
-**Consequences:**
-- High false negative rate
-- Significant results more likely to be false positives
-- Overestimated effect sizes (when significant)
-
-**Solutions:**
-- Conduct a priori power analysis
-- Aim for 80-90% power
-- Consider effect size from prior research
-
-### Pitfall 10: Post-Hoc Power Analysis
-**Problem:** Calculating power after seeing results is circular and uninformative.
-
-**Why useless:**
-- Non-significant results always have low "post-hoc power"
-- It recapitulates the p-value without new information
-
-**Better approach:**
-- Calculate confidence intervals
-- Plan replication with adequate sample
-- Conduct prospective power analysis for future studies
-
-### Pitfall 11: Small Sample Fallacy
-**Problem:** Trusting results from very small samples.
-
-**Issues:**
-- High sampling variability
-- Outliers have large influence
-- Assumptions of tests violated
-- Confidence intervals very wide
-
-**Guidelines:**
-- Be skeptical of n < 30
-- Check assumptions carefully
-- Consider non-parametric tests
-- Replicate findings
-
-## Effect Size Misunderstandings
-
-### Pitfall 12: Ignoring Effect Size
-**Problem:** Focusing only on significance, not magnitude.
-
-**Why problematic:**
-- Significance ≠ importance
-- Can't compare across studies
-- Doesn't inform practical decisions
-
-**Solutions:**
-- Always report effect sizes
-- Use standardized measures (Cohen's d, r, η²)
-- Interpret using field conventions
-- Consider minimum clinically important difference
-
-### Pitfall 13: Misinterpreting Standardized Effect Sizes
-**Problem:** Treating Cohen's d = 0.5 as "medium" without context.
-
-**Reality:**
-- Field-specific norms vary
-- Some fields have larger typical effects
-- Real-world importance depends on context
-
-**Better approach:**
-- Compare to effects in same domain
-- Consider practical implications
-- Look at raw effect sizes too
-
-### Pitfall 14: Confusing Explained Variance with Importance
-**Problem:** "Only explains 5% of variance" = unimportant.
-
-**Reality:**
-- Height explains ~5% of variation in NBA player salary but is crucial
-- Complex phenomena have many small contributors
-- Predictive accuracy ≠ causal importance
-
-**Consideration:** Context matters more than percentage alone.
-
-## Correlation and Causation
-
-### Pitfall 15: Correlation Implies Causation
-**Problem:** Inferring causation from correlation.
-
-**Alternative explanations:**
-- Reverse causation (B causes A, not A causes B)
-- Confounding (C causes both A and B)
-- Coincidence
-- Selection bias
-
-**Criteria for causation:**
-- Temporal precedence
-- Covariation
-- No plausible alternatives
-- Ideally: experimental manipulation
-
-### Pitfall 16: Ecological Fallacy
-**Problem:** Inferring individual-level relationships from group-level data.
-
-**Example:** Countries with more chocolate consumption have more Nobel laureates doesn't mean eating chocolate makes you win Nobels.
-
-**Why problematic:** Group-level correlations may not hold at individual level.
-
-### Pitfall 17: Simpson's Paradox
-**Problem:** Trend appears in groups but reverses when combined (or vice versa).
-
-**Example:** Treatment appears worse overall but better in every subgroup.
-
-**Cause:** Confounding variable distributed differently across groups.
-
-**Solution:** Consider confounders and look at appropriate level of analysis.
-
-## Regression and Modeling Pitfalls
-
-### Pitfall 18: Overfitting
-**Problem:** Model fits sample data well but doesn't generalize.
-
-**Causes:**
-- Too many predictors relative to sample size
-- Fitting noise rather than signal
-- No cross-validation
-
-**Solutions:**
-- Use cross-validation
-- Penalized regression (LASSO, ridge)
-- Independent test set
-- Simpler models
-
-### Pitfall 19: Extrapolation Beyond Data Range
-**Problem:** Predicting outside the range of observed data.
-
-**Why dangerous:**
-- Relationships may not hold outside observed range
-- Increased uncertainty not reflected in predictions
-
-**Solution:** Only interpolate; avoid extrapolation.
-
-### Pitfall 20: Ignoring Model Assumptions
-**Problem:** Using statistical tests without checking assumptions.
-
-**Common violations:**
-- Non-normality (for parametric tests)
-- Heteroscedasticity (unequal variances)
-- Non-independence
-- Linearity
-- No multicollinearity
-
-**Solutions:**
-- Check assumptions with diagnostics
-- Use robust methods
-- Transform data
-- Use appropriate non-parametric alternatives
-
-### Pitfall 21: Treating Non-Significant Covariates as Eliminating Confounding
-**Problem:** "We controlled for X and it wasn't significant, so it's not a confounder."
-
-**Reality:** Non-significant covariates can still be important confounders. Significance ≠ confounding.
-
-**Solution:** Include theoretically important covariates regardless of significance.
-
-### Pitfall 22: Collinearity Masking Effects
-**Problem:** When predictors are highly correlated, true effects may appear non-significant.
-
-**Manifestations:**
-- Large standard errors
-- Unstable coefficients
-- Sign changes when adding/removing variables
-
-**Detection:**
-- Variance Inflation Factors (VIF)
-- Correlation matrices
-
-**Solutions:**
-- Remove redundant predictors
-- Combine correlated variables
-- Use regularization methods
-
-## Specific Test Misuses
-
-### Pitfall 23: T-Test for Multiple Groups
-**Problem:** Conducting multiple t-tests instead of ANOVA.
-
-**Why wrong:** Inflates Type I error rate dramatically.
-
-**Correct approach:**
-- Use ANOVA first
-- Follow with planned comparisons or post-hoc tests with correction
-
-### Pitfall 24: Pearson Correlation for Non-Linear Relationships
-**Problem:** Using Pearson's r for curved relationships.
-
-**Why misleading:** r measures linear relationships only.
-
-**Solutions:**
-- Check scatterplots first
-- Use Spearman's ρ for monotonic relationships
-- Consider polynomial or non-linear models
-
-### Pitfall 25: Chi-Square with Small Expected Frequencies
-**Problem:** Chi-square test with expected cell counts < 5.
-
-**Why wrong:** Violates test assumptions, p-values inaccurate.
-
-**Solutions:**
-- Fisher's exact test
-- Combine categories
-- Increase sample size
-
-### Pitfall 26: Paired vs. Independent Tests
-**Problem:** Using independent samples test for paired data (or vice versa).
-
-**Why wrong:**
-- Wastes power (paired data analyzed as independent)
-- Violates independence assumption (independent data analyzed as paired)
-
-**Solution:** Match test to design.
-
-## Confidence Interval Misinterpretations
-
-### Pitfall 27: 95% CI = 95% Probability True Value Inside
-**Misconception:** "95% chance the true value is in this interval."
-
-**Reality:** The true value either is or isn't in this specific interval. If we repeated the study many times, 95% of resulting intervals would contain the true value.
-
-**Better interpretation:** "We're 95% confident this interval contains the true value."
-
-### Pitfall 28: Overlapping CIs = No Difference
-**Problem:** Assuming overlapping confidence intervals mean no significant difference.
-
-**Reality:** Overlapping CIs are less stringent than difference tests. Two CIs can overlap while the difference between groups is significant.
-
-**Guideline:** Overlap of point estimate with other CI is more relevant than overlap of intervals.
-
-### Pitfall 29: Ignoring CI Width
-**Problem:** Focusing only on whether CI includes zero, not precision.
-
-**Why important:** Wide CIs indicate high uncertainty. "Significant" effects with huge CIs are less convincing.
-
-**Consider:** Both significance and precision.
-
-## Bayesian vs. Frequentist Confusions
-
-### Pitfall 30: Mixing Bayesian and Frequentist Interpretations
-**Problem:** Making Bayesian statements from frequentist analyses.
-
-**Examples:**
-- "Probability hypothesis is true" (Bayesian) from p-value (frequentist)
-- "Evidence for null" from non-significant result (frequentist can't support null)
-
-**Solution:**
-- Be clear about framework
-- Use Bayesian methods for Bayesian questions
-- Use Bayes factors to compare hypotheses
-
-### Pitfall 31: Ignoring Prior Probability
-**Problem:** Treating all hypotheses as equally likely initially.
-
-**Reality:** Extraordinary claims need extraordinary evidence. Prior plausibility matters.
-
-**Consider:**
-- Plausibility given existing knowledge
-- Mechanism plausibility
-- Base rates
-
-## Data Transformation Issues
-
-### Pitfall 32: Dichotomizing Continuous Variables
-**Problem:** Splitting continuous variables at arbitrary cutoffs.
-
-**Consequences:**
-- Loss of information and power
-- Arbitrary distinctions
-- Discarding individual differences
-
-**Exceptions:** Clinically meaningful cutoffs with strong justification.
-
-**Better:** Keep continuous or use multiple categories.
-
-### Pitfall 33: Trying Multiple Transformations
-**Problem:** Testing many transformations until finding significance.
-
-**Why problematic:** Inflates Type I error, is a form of p-hacking.
-
-**Better approach:**
-- Prespecify transformations
-- Use theory-driven transformations
-- Correct for multiple testing if exploring
-
-## Missing Data Problems
-
-### Pitfall 34: Listwise Deletion by Default
-**Problem:** Automatically deleting all cases with any missing data.
-
-**Consequences:**
-- Reduced power
-- Potential bias if data not missing completely at random (MCAR)
-
-**Better approaches:**
-- Multiple imputation
-- Maximum likelihood methods
-- Analyze missingness patterns
-
-### Pitfall 35: Ignoring Missing Data Mechanisms
-**Problem:** Not considering why data are missing.
-
-**Types:**
-- MCAR (Missing Completely at Random): Safe to delete
-- MAR (Missing at Random): Can impute
-- MNAR (Missing Not at Random): May bias results
-
-**Solution:** Analyze patterns, use appropriate methods, consider sensitivity analyses.
-
-## Publication and Reporting Issues
-
-### Pitfall 36: Selective Reporting
-**Problem:** Only reporting significant results or favorable analyses.
-
-**Consequences:**
-- Literature appears more consistent than reality
-- Meta-analyses biased
-- Wasted research effort
-
-**Solutions:**
-- Preregistration
-- Report all analyses
-- Use reporting guidelines (CONSORT, PRISMA, etc.)
-
-### Pitfall 37: Rounding to p < .05
-**Problem:** Reporting exact p-values selectively (e.g., p = .049 but p < .05 for .051).
-
-**Why problematic:** Obscures values near threshold, enables p-hacking detection evasion.
-
-**Better:** Always report exact p-values.
-
-### Pitfall 38: No Data Sharing
-**Problem:** Not making data available for verification or reanalysis.
-
-**Consequences:**
-- Can't verify results
-- Can't include in meta-analyses
-- Hinders scientific progress
-
-**Best practice:** Share data unless privacy concerns prohibit.
-
-## Cross-Validation and Generalization
-
-### Pitfall 39: No Cross-Validation
-**Problem:** Testing model on same data used to build it.
-
-**Consequence:** Overly optimistic performance estimates.
-
-**Solutions:**
-- Split data (train/test)
-- K-fold cross-validation
-- Independent validation sample
-
-### Pitfall 40: Data Leakage
-**Problem:** Information from test set leaking into training.
-
-**Examples:**
-- Normalizing before splitting
-- Feature selection on full dataset
-- Including temporal information
-
-**Consequence:** Inflated performance metrics.
-
-**Prevention:** All preprocessing decisions made using only training data.
-
-## Meta-Analysis Pitfalls
-
-### Pitfall 41: Apples and Oranges
-**Problem:** Combining studies with different designs, populations, or measures.
-
-**Balance:** Need homogeneity but also comprehensiveness.
-
-**Solutions:**
-- Clear inclusion criteria
-- Subgroup analyses
-- Meta-regression for moderators
-
-### Pitfall 42: Ignoring Publication Bias
-**Problem:** Published studies overrepresent significant results.
-
-**Consequences:** Overestimated effects in meta-analyses.
-
-**Detection:**
-- Funnel plots
-- Trim-and-fill
-- PET-PEESE
-- P-curve analysis
-
-**Solutions:**
-- Include unpublished studies
-- Register reviews
-- Use bias-correction methods
-
-## General Best Practices
-
-1. **Preregister studies** - Distinguish confirmatory from exploratory
-2. **Report transparently** - All analyses, not just significant ones
-3. **Check assumptions** - Don't blindly apply tests
-4. **Use appropriate tests** - Match test to data and design
-5. **Report effect sizes** - Not just p-values
-6. **Consider practical significance** - Not just statistical
-7. **Replicate findings** - One study is rarely definitive
-8. **Share data and code** - Enable verification
-9. **Use confidence intervals** - Show uncertainty
-10. **Think causally carefully** - Most research is correlational

package/bin/skills/scientific-schematics/QUICK_REFERENCE.md

@@ -1,207 +0,0 @@
-# Scientific Schematics - Quick Reference
-
-**How it works:** Describe your diagram → Nano Banana Pro generates it automatically
-
-## Setup (One-Time)
-
-```bash
-# Get API key from https://openrouter.ai/keys
-export OPENROUTER_API_KEY='sk-or-v1-your_key_here'
-
-# Add to shell profile for persistence
-echo 'export OPENROUTER_API_KEY="sk-or-v1-your_key"' >> ~/.bashrc  # or ~/.zshrc
-```
-
-## Basic Usage
-
-```bash
-# Describe your diagram, Nano Banana Pro creates it
-python scripts/generate_schematic.py "your diagram description" -o output.png
-
-# That's it! Automatic:
-# - Iterative refinement (3 rounds)
-# - Quality review and improvement
-# - Publication-ready output
-```
-
-## Common Examples
-
-### CONSORT Flowchart
-```bash
-python scripts/generate_schematic.py \
-  "CONSORT flow: screened n=500, excluded n=150, randomized n=350" \
-  -o consort.png
-```
-
-### Neural Network
-```bash
-python scripts/generate_schematic.py \
-  "Transformer architecture with encoder and decoder stacks" \
-  -o transformer.png
-```
-
-### Biological Pathway
-```bash
-python scripts/generate_schematic.py \
-  "MAPK pathway: EGFR → RAS → RAF → MEK → ERK" \
-  -o mapk.png
-```
-
-### Circuit Diagram
-```bash
-python scripts/generate_schematic.py \
-  "Op-amp circuit with 1kΩ resistor and 10µF capacitor" \
-  -o circuit.png
-```
-
-## Command Options
-
-| Option | Description | Example |
-|--------|-------------|---------|
-| `-o, --output` | Output file path | `-o figures/diagram.png` |
-| `--iterations N` | Number of refinements (1-2) | `--iterations 2` |
-| `-v, --verbose` | Show detailed output | `-v` |
-| `--api-key KEY` | Provide API key | `--api-key sk-or-v1-...` |
-
-## Prompt Tips
-
-### ✓ Good Prompts (Specific)
-- "CONSORT flowchart with screening (n=500), exclusion (n=150), randomization (n=350)"
-- "Transformer architecture: encoder on left with 6 layers, decoder on right, cross-attention connections"
-- "MAPK signaling: receptor → RAS → RAF → MEK → ERK → nucleus, label each phosphorylation"
-
-### ✗ Avoid (Too Vague)
-- "Make a flowchart"
-- "Neural network"
-- "Pathway diagram"
-
-## Output Files
-
-For input `diagram.png`, you get:
-- `diagram_v1.png` - First iteration
-- `diagram_v2.png` - Second iteration  
-- `diagram_v3.png` - Final iteration
-- `diagram.png` - Copy of final
-- `diagram_review_log.json` - Quality scores and critiques
-
-## Review Log
-
-```json
-{
-  "iterations": [
-    {
-      "iteration": 1,
-      "score": 7.0,
-      "critique": "Good start. Font too small..."
-    },
-    {
-      "iteration": 2,
-      "score": 8.5,
-      "critique": "Much improved. Minor spacing issues..."
-    },
-    {
-      "iteration": 3,
-      "score": 9.5,
-      "critique": "Excellent. Publication ready."
-    }
-  ],
-  "final_score": 9.5
-}
-```
-
-## Python API
-
-```python
-from scripts.generate_schematic_ai import ScientificSchematicGenerator
-
-# Initialize
-gen = ScientificSchematicGenerator(api_key="your_key")
-
-# Generate
-results = gen.generate_iterative(
-    user_prompt="diagram description",
-    output_path="output.png",
-    iterations=2
-)
-
-# Check quality
-print(f"Score: {results['final_score']}/10")
-```
-
-## Troubleshooting
-
-### API Key Not Found
-```bash
-# Check if set
-echo $OPENROUTER_API_KEY
-
-# Set it
-export OPENROUTER_API_KEY='your_key'
-```
-
-### Import Error
-```bash
-# Install requests
-pip install requests
-```
-
-### Low Quality Score
-- Make prompt more specific
-- Include layout details (left-to-right, top-to-bottom)
-- Specify label requirements
-- Increase iterations: `--iterations 2`
-
-## Testing
-
-```bash
-# Verify installation
-python test_ai_generation.py
-
-# Should show: "6/6 tests passed"
-```
-
-## Cost
-
-Typical cost per diagram (max 2 iterations):
-- Simple (1 iteration): $0.05-0.15
-- Complex (2 iterations): $0.10-0.30
-
-## How Nano Banana Pro Works
-
-**Simply describe your diagram in natural language:**
-- ✓ No coding required
-- ✓ No templates needed
-- ✓ No manual drawing
-- ✓ Automatic quality review
-- ✓ Publication-ready output
-- ✓ Works for any diagram type
-
-**Just describe what you want, and it's generated automatically.**
-
-## Getting Help
-
-```bash
-# Show help
-python scripts/generate_schematic.py --help
-
-# Verbose mode for debugging
-python scripts/generate_schematic.py "diagram" -o out.png -v
-```
-
-## Quick Start Checklist
-
-- [ ] Set `OPENROUTER_API_KEY` environment variable
-- [ ] Run `python test_ai_generation.py` (should pass 6/6)
-- [ ] Try: `python scripts/generate_schematic.py "test diagram" -o test.png`
-- [ ] Review output files (test_v1.png, v2, v3, review_log.json)
-- [ ] Read SKILL.md for detailed documentation
-- [ ] Check README.md for examples
-
-## Resources
-
-- Full documentation: `SKILL.md`
-- Detailed guide: `README.md`
-- Implementation details: `IMPLEMENTATION_SUMMARY.md`
-- Example script: `example_usage.sh`
-- Get API key: https://openrouter.ai/keys
-