@brainpilot/skills 0.0.6

package/skills/08_Computational_Neuroscience/neural-decoding-analysis/SKILL.md

@@ -0,0 +1,273 @@
+---
+name: "neural-decoding-analysis"
+description: "Domain-validated methods and decision logic for neural decoding, RSA, temporal generalization, and encoding models in systems neuroscience"
+domain: "systems-neuroscience"
+version: "1.0.0"
+authors:
+ - "Claude Code Skill Generator"
+papers:
+ - "Kriegeskorte et al., 2008"
+ - "Haynes, 2015"
+ - "King & Dehaene, 2014"
+ - "Grootswagers et al., 2017"
+ - "Varoquaux et al., 2017"
+dependencies:
+ required:
+ - research-literacy
+ recommended:
+ - fmri-preprocessing-pipeline-guide
+review_status: "ai-generated"
+---
+
+# Neural Population Decoding Analysis
+
+## Purpose
+
+This skill encodes expert methodological knowledge for multivariate neural decoding analyses in systems neuroscience. It covers cross-validated classification (MVPA), representational similarity analysis (RSA), temporal generalization, and encoding models. The skill provides domain-specific decision logic, parameter recommendations, and pitfall warnings that a machine-learning engineer without neuroscience training would not know.
+
+## When to Use This Skill
+
+- Determining whether stimulus or task information is represented in neural population activity
+- Comparing the representational geometry of brain regions to computational models
+- Characterizing the temporal dynamics of neural representations from EEG/MEG
+- Building encoding models to predict neural responses from stimulus features
+- Designing a decoding analysis pipeline and choosing appropriate methods and parameters
+
+## Research Planning Protocol
+
+Before executing the domain-specific steps below, you MUST:
+
+1. **State the research question** — What specific representational or informational question is this decoding analysis addressing?
+2. **Justify the method choice** — Why decoding/RSA (not univariate analysis, connectivity, etc.)? What alternatives were considered?
+3. **Declare expected outcomes** — What decoding accuracy or representational structure would support vs. refute the hypothesis?
+4. **Note assumptions and limitations** — What does this method assume? Where could it mislead (e.g., confounds, leakage)?
+5. **Present the plan to the user and WAIT for confirmation** before proceeding.
+
+For detailed methodology guidance, see the `research-literacy` skill.
+
+
+## ⚠️ Verification Notice
+
+This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please [open an issue](https://github.com/HaoxuanLiTHUAI/awesome_cognitive_and_neuroscience_skills/issues).
+
+## When to Use Decoding vs. Univariate Analysis
+
+Univariate analysis tests whether the *mean activity level* differs across conditions in a region. Decoding tests whether *spatial patterns of activity* carry information, even when mean activity is identical across conditions (Haynes, 2015). Use decoding when:
+
+- You expect information to be encoded in distributed patterns, not mean amplitude
+- The signal-to-noise ratio per voxel/channel is low but the population carries information
+- You want to compare neural representations to computational model predictions (use RSA)
+- You want to track when information emerges and transforms over time (use temporal generalization)
+
+> **Domain judgment**: High decoding accuracy does NOT mean the decoded region is the *source* of the representation. It means the information is *accessible* from that region's patterns. A downstream region receiving a copy of the signal will also decode well (Haynes, 2015).
+
+## Method Selection Decision Tree
+
+```
+What is your research question?
+|
++-- "Is stimulus/task information present in this brain region's patterns?"
+| --> Cross-validated classification (MVPA)
+| Output: classification accuracy or d-prime
+|
++-- "How are representations organized? Does the geometry match a model?"
+| --> Representational Similarity Analysis (RSA)
+| Output: model-RDM correlation, noise ceiling
+|
++-- "When does information emerge and how does it transform over time?"
+| --> Temporal Generalization (time x time decoding)
+| Output: temporal generalization matrix
+| Best for: EEG, MEG, intracranial recordings
+|
++-- "What stimulus features drive neural responses across the feature space?"
+ --> Encoding Models (voxelwise/channel-wise prediction)
+ Output: prediction accuracy (R^2), feature tuning maps
+```
+
+## Cross-Validated Classification (MVPA)
+
+### Classifier Selection
+
+| Classifier | When to Use | When to Avoid | Source |
+|---|---|---|---|
+| **Linear SVM** | Default choice; robust to high dimensionality; works well with small samples | When you need probabilistic outputs (use logistic regression) | Misaki et al., 2010; Varoquaux et al., 2017 |
+| **LDA** | Fast; good when n_features << n_samples after reduction | Raw high-dimensional data (covariance estimate unstable) | Misaki et al., 2010 |
+| **Logistic Regression** | When you need class probabilities; with L1 for sparse solutions | Rarely a bad choice; comparable to linear SVM | Varoquaux et al., 2017 |
+| **Linear kernel (general)** | Almost always for fMRI/EEG | Nonlinear kernels rarely improve and risk overfitting | Misaki et al., 2010 |
+
+> **Domain judgment**: Linear classifiers are strongly preferred in neuroimaging because (1) fMRI/EEG patterns are high-dimensional relative to sample size, making nonlinear methods prone to overfitting, and (2) linear weights are more interpretable neurally, though see Haufe et al. (2014) on the distinction between classifier weights and activation patterns.
+
+### Cross-Validation Strategy
+
+| Strategy | When to Use | Rationale |
+|---|---|---|
+| **Leave-one-run-out** | fMRI (standard) | Respects temporal autocorrelation within runs; prevents leakage from slow hemodynamic signals (Varoquaux et al., 2017) |
+| **Stratified k-fold** (k=5-10) | EEG/MEG with many trials | Balances class proportions in each fold; k=5 recommended for bias-variance tradeoff (Varoquaux, 2018) |
+| **Leave-one-trial-out** | When few trials available | Maximum training data but high variance; avoid for fMRI due to temporal autocorrelation (Varoquaux et al., 2017) |
+| **Leave-one-subject-out** | Between-subject generalization | Tests whether patterns generalize across individuals |
+
+> **CRITICAL -- Information leakage**: Feature selection, normalization, and dimensionality reduction MUST be performed WITHIN each cross-validation fold, using ONLY training data. Fitting a PCA or z-scoring across all data before splitting inflates accuracy by leaking test-set statistics into training (Kriegeskorte et al., 2009; Varoquaux et al., 2017).
+
+### Chance Level and Statistical Testing
+
+- **Theoretical chance**: 1/n_classes for balanced designs (e.g., 50% for 2-class)
+- **CRITICAL**: Theoretical chance is only valid with infinite samples. With small samples, empirical accuracy on random data can substantially exceed 1/n_classes (Combrisson & Jerbi, 2015)
+- **Use permutation testing**: Shuffle labels 1000+ times, decode each permutation, compute p-value as the proportion of permuted accuracies >= observed accuracy (Combrisson & Jerbi, 2015)
+- **Binomial test**: Acceptable quick alternative for large trial counts, but permutation testing is preferred (Combrisson & Jerbi, 2015)
+- For group-level inference: test accuracy against chance across subjects using a one-sample t-test or Wilcoxon signed-rank test on subject-level accuracies
+
+### Minimum Data Requirements
+
+- **Trials per class**: Minimum **20-30 trials per class** for reliable within-subject decoding (Varoquaux, 2018; Grootswagers et al., 2017)
+- **Cross-validation error bars**: With ~100 samples, expect confidence intervals of approximately +/-10% on accuracy estimates (Varoquaux, 2018)
+- **Trial averaging**: Averaging 5-10 trials before classification improves SNR but reduces effective sample size; balance based on total trial count (Grootswagers et al., 2017)
+
+## Representational Similarity Analysis (RSA)
+
+RSA abstracts from activity patterns to a condition-by-condition dissimilarity matrix (RDM), enabling comparison across brain regions, species, and computational models (Kriegeskorte et al., 2008).
+
+### RDM Construction
+
+| Distance Metric | Properties | When to Use | Source |
+|---|---|---|---|
+| **Correlation distance** (1 - Pearson r) | Invariant to mean and scale | Default for comparing pattern shape; standard in early RSA | Kriegeskorte et al., 2008 |
+| **Euclidean distance** | Sensitive to amplitude | When amplitude differences are meaningful | Kriegeskorte et al., 2008 |
+| **Crossnobis distance** | Cross-validated Mahalanobis; unbiased estimator with interpretable zero | **Preferred for inferential statistics**; requires multi-run data | Walther et al., 2016; Kriegeskorte & Diedrichsen, 2019 |
+
+> **Domain judgment**: The crossnobis estimator is unbiased -- its expected value is zero when two conditions have identical representations, unlike correlation distance or Euclidean distance which are positively biased by noise. This means crossnobis values can be negative (not a true distance), but this property makes it valid for statistical inference without bias correction (Walther et al., 2016).
+
+### Model Comparison
+
+- **Pearson/Spearman correlation**: Correlate model RDM with brain RDM (use Spearman for robustness to outliers; Nili et al., 2014)
+- **Partial correlation**: Control for one model while testing another (essential when models are correlated)
+- **Regression on RDMs**: Fit multiple model RDMs simultaneously; use weighted least squares or component models (Kriegeskorte & Diedrichsen, 2019)
+
+### Statistical Inference
+
+- **Noise ceiling**: Upper and lower bounds on the best achievable model fit given between-subject variability. Upper bound: average correlation of each subject's RDM with the group mean RDM. Lower bound: same but computed with leave-one-subject-out (Nili et al., 2014)
+- **Stimulus-label randomization**: Permute condition labels to construct a null distribution for RDM correlation (Nili et al., 2014)
+- **Bootstrap confidence intervals**: Resample subjects with replacement to estimate confidence intervals on model-RDM correlations
+
+> **Domain judgment**: If a model falls within the noise ceiling, it explains as much variance as is explainable given the noise in the data. A model below the lower bound leaves systematic variance unexplained. This is NOT the same as a significance test -- a model can be significantly correlated with brain RDMs yet still fall below the noise ceiling (Nili et al., 2014).
+
+See `references/rsa-guide.md` for a complete step-by-step RSA workflow.
+
+## Temporal Generalization (EEG/MEG)
+
+Train a classifier at each time point t, test it at every time point t'. The resulting time x time matrix reveals the dynamics of neural representations (King & Dehaene, 2014).
+
+### Interpreting the Temporal Generalization Matrix
+
+| Pattern | Matrix Shape | Interpretation | Example |
+|---|---|---|---|
+| **Diagonal only** | Thin diagonal stripe | Information is present but the neural code changes over time (chain of transient states) | Sequence of processing stages |
+| **Square block** | Broad off-diagonal generalization | Stable, sustained representation (same code maintained) | Working memory maintenance |
+| **Off-diagonal stripe** | Horizontal or vertical extension | A code trained at one time reactivates later | Memory reactivation |
+| **Below-diagonal spread** | Widening below diagonal | Later representations are decodable by earlier classifiers (persistent code) | Sustained sensory trace |
+
+(King & Dehaene, 2014; Grootswagers et al., 2017)
+
+### Sliding Window Parameters
+
+| Parameter | Recommended Value | Rationale | Source |
+|---|---|---|---|
+| Window width | **50 ms** for EEG/MEG | Balances temporal resolution with SNR | Grootswagers et al., 2017 |
+| Step size | **10 ms** for EEG/MEG | Provides smooth temporal profile without excessive computation | Grootswagers et al., 2017 |
+| Baseline window | **-200 to 0 ms** | Standard pre-stimulus baseline | Grootswagers et al., 2017 |
+| Features | All sensors at time point t | Use all channels; spatial patterns carry information | King & Dehaene, 2014 |
+
+### Statistical Testing for Temporal Generalization
+
+- **Cluster-based permutation test** on the time x time matrix (Maris & Oostenveld, 2007)
+- Cluster-forming threshold: **p < 0.05** (two-tailed) at the individual time-point level; correct at cluster level with **1000+ permutations**
+- **Caution**: Cluster tests control family-wise error rate but do NOT localize the effect to specific time points (Maris & Oostenveld, 2007)
+
+## Encoding Models
+
+Encoding models predict neural responses from stimulus features, complementing decoding (which predicts stimuli from neural responses).
+
+### When to Use Encoding Over Decoding
+
+- When the feature space is continuous or high-dimensional (e.g., image pixels, spectrograms)
+- When you want to characterize *what features* a region encodes, not just *whether* it encodes information
+- Voxelwise encoding models: fit a regularized regression (ridge) from features to each voxel's response (Kriegeskorte & Diedrichsen, 2019)
+
+### Key Considerations
+
+- **Regularization**: Ridge regression (L2) is standard; prevents overfitting when features > samples
+- **Cross-validation**: Leave-one-run-out; evaluate with Pearson correlation between predicted and actual responses
+- **Feature spaces**: Gabor wavelets (V1), DNN layers (ventral stream), semantic embeddings (language regions)
+- **Relationship to RSA**: When using a zero-mean isotropic Gaussian weight prior (ridge regression), encoding models, RSA, and pattern component models test equivalent hypotheses captured by the second moment matrix G (Kriegeskorte & Diedrichsen, 2019)
+
+## Common Pitfalls
+
+### 1. Information Leakage (The Most Common Error)
+
+Feature selection, z-scoring, PCA, or any data-driven preprocessing on the full dataset before cross-validation splitting will leak information from test folds into training, inflating accuracy. ALL such steps must occur WITHIN each fold (Kriegeskorte et al., 2009; Varoquaux et al., 2017).
+
+### 2. Confounds Driving Decoding
+
+Decoding "success" may reflect confounds rather than neural representations:
+- **Eye movements**: Systematic gaze differences between conditions produce decodable EOG artifacts in EEG and BOLD signal differences near the eyes in fMRI
+- **Reaction time differences**: Conditions with different RTs produce different motor preparation signals
+- **Head motion**: Differential motion across conditions (e.g., speech vs. rest) introduces confounded spatial patterns
+- **Run order**: If conditions are blocked or ordered within runs, slow signal drifts can drive classification
+
+### 3. Interpreting Accuracy Magnitude
+
+- Decoding accuracy reflects information *accessible to the classifier*, not the *amount* of neural information (Haynes, 2015)
+- Low accuracy does not mean low information (features may be nonlinearly coded)
+- High accuracy does not imply the region is the *origin* of the representation
+- **Never compare accuracy magnitudes across regions with different voxel counts**, SNR, or dimensionality without controlling for these factors
+
+### 4. Double-Dipping in Searchlight Analysis
+
+Selecting an ROI based on significant searchlight clusters and then performing additional analyses on those clusters is circular (Kriegeskorte et al., 2009; Etzel et al., 2013). Use independent data or pre-registered ROIs for follow-up analyses.
+
+### 5. Classifier Weights Are Not Activation Patterns
+
+Raw SVM or regression weights do NOT indicate which voxels/channels are most *activated* by a condition. They indicate which features are most useful for *discrimination*, which can include suppressing noise. To obtain neurophysiologically interpretable maps, transform weights into activation patterns using the method of Haufe et al. (2014).
+
+### 6. Imbalanced Classes
+
+Unequal trial counts across classes bias accuracy toward the majority class. Solutions:
+- Undersample the majority class
+- Use balanced accuracy (average of per-class recall)
+- Stratify cross-validation folds
+
+## Minimum Reporting Checklist
+
+Based on Haynes (2015), Varoquaux et al. (2017), and Grootswagers et al. (2017):
+
+- [ ] Classifier type and hyperparameters (e.g., linear SVM, C=1)
+- [ ] Cross-validation scheme (e.g., leave-one-run-out, 5-fold stratified)
+- [ ] Feature space (number of voxels/channels, ROI definition or searchlight radius)
+- [ ] Preprocessing steps performed WITHIN vs. OUTSIDE cross-validation folds
+- [ ] Number of trials/samples per class per subject
+- [ ] Statistical test for significance (permutation test with N permutations, or parametric test)
+- [ ] Effect size or confidence intervals on accuracy
+- [ ] For RSA: distance metric, model RDMs tested, noise ceiling
+- [ ] For temporal generalization: window width, step size, cluster-correction parameters
+- [ ] Software and version used
+
+## Key References
+
+- Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. *Journal of Neuroscience Methods*, 250, 126-136.
+- Etzel, J. A., Zacks, J. M., & Braver, T. S. (2013). Searchlight analysis: Promise, pitfalls, and potential. *NeuroImage*, 78, 261-269.
+- Grootswagers, T., Wardle, S. G., & Carlson, T. A. (2017). Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data. *Journal of Cognitive Neuroscience*, 29(4), 677-697.
+- Haufe, S., et al. (2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging. *NeuroImage*, 87, 96-110.
+- Haynes, J.-D. (2015). A primer on pattern-based approaches to fMRI: Principles, pitfalls, and perspectives. *Neuron*, 87(2), 257-270.
+- King, J.-R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: The temporal generalization method. *Trends in Cognitive Sciences*, 18(4), 203-210.
+- Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. *Proceedings of the National Academy of Sciences*, 103(10), 3863-3868.
+- Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis -- Connecting the branches of systems neuroscience. *Frontiers in Systems Neuroscience*, 2, 4.
+- Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in systems neuroscience: The dangers of double dipping. *Nature Neuroscience*, 12(5), 535-540.
+- Kriegeskorte, N., & Diedrichsen, J. (2019). Peeling the onion of brain representations. *Annual Review of Neuroscience*, 42, 407-432.
+- Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. *Journal of Neuroscience Methods*, 164(1), 177-190.
+- Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. *NeuroImage*, 53(1), 103-118.
+- Nili, H., et al. (2014). A toolbox for representational similarity analysis. *PLoS Computational Biology*, 10(4), e1003553.
+- Varoquaux, G., et al. (2017). Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. *NeuroImage*, 145, 166-179.
+- Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. *NeuroImage*, 180, 68-77.
+- Walther, A., et al. (2016). Reliability of dissimilarity measures for multi-voxel pattern analysis. *NeuroImage*, 137, 188-200.
+
+See `references/decoding-methods.md` for detailed classifier comparisons, searchlight parameters, and software tools.
+See `references/rsa-guide.md` for a complete step-by-step RSA analysis workflow.

package/skills/08_Computational_Neuroscience/neural-decoding-analysis/references/decoding-methods.md

@@ -0,0 +1,170 @@
+# Decoding Methods: Detailed Classifier Comparisons, Searchlight Parameters, and Tools
+
+This reference document supplements `SKILL.md` with extended details on classifier selection, searchlight analysis, feature selection, hyperparameter tuning, and software tools for neural decoding.
+
+## Classifier Comparison
+
+### Linear SVM
+
+- **Mechanism**: Finds the maximum-margin hyperplane separating classes in feature space
+- **Regularization**: Controlled by parameter C (inverse regularization strength). Default C=1 is appropriate for most neuroimaging applications (Misaki et al., 2010; Varoquaux et al., 2017)
+- **Strengths**: Robust to high-dimensional data (n_features >> n_samples); well-suited for fMRI where voxel count vastly exceeds trial count; regularization handles collinear features
+- **Weaknesses**: No native probability estimates (use Platt scaling if needed, but this adds a calibration step); sensitive to C parameter when data is noisy
+- **Typical neuroimaging performance**: Among the best linear classifiers for fMRI pattern classification (Misaki et al., 2010)
+
+### Linear Discriminant Analysis (LDA)
+
+- **Mechanism**: Projects data onto the direction that maximizes between-class variance relative to within-class variance, assuming Gaussian class distributions with shared covariance
+- **Regularization**: Shrinkage of covariance matrix toward identity (Ledoit-Wolf shrinkage recommended); critical when n_features > n_samples
+- **Strengths**: Fast to train and test; provides a natural decision boundary; works well after dimensionality reduction
+- **Weaknesses**: Unreliable without dimensionality reduction or regularization when voxel count exceeds sample count; assumes equal covariance across classes (Misaki et al., 2010)
+- **When to use**: After PCA or feature selection has reduced dimensionality; for EEG/MEG where channel count is moderate (~64-306 channels)
+
+### Logistic Regression
+
+- **Mechanism**: Models the log-odds of class membership as a linear function of features
+- **Regularization**: L2 (ridge) penalty is default; L1 (lasso) produces sparse solutions identifying a subset of informative features
+- **Strengths**: Provides calibrated probability estimates; L1 variant enables feature selection; comparable performance to linear SVM (Varoquaux et al., 2017)
+- **Weaknesses**: L1 solutions are unstable when features are highly correlated (as in fMRI); selected voxels may not be neurophysiologically meaningful
+- **When to use**: When probability outputs are needed (e.g., for confidence-weighted analyses); when L1 sparsity is desired for interpretability (with caveats)
+
+### Gaussian Naive Bayes (GNB)
+
+- **Mechanism**: Assumes features are independently Gaussian-distributed within each class
+- **Strengths**: Extremely fast; no hyperparameters to tune
+- **Weaknesses**: The independence assumption is strongly violated in neuroimaging data (neighboring voxels are correlated); generally performs significantly worse than SVM, LDA, and logistic regression for fMRI (Misaki et al., 2010)
+- **When to use**: As a quick sanity check or baseline; avoid for primary analyses
+
+### Nonlinear Classifiers (RBF SVM, Random Forest, Neural Networks)
+
+- **General guidance**: Nonlinear classifiers rarely outperform linear classifiers for typical neuroimaging classification because the feature space is high-dimensional relative to the number of training samples (Misaki et al., 2010)
+- **Exception**: May be useful with very large datasets (e.g., thousands of trials pooled across subjects) or when representations are known to be nonlinearly organized
+- **Risk**: Overfitting is severe with small neuroimaging samples; results are harder to interpret neurally
+
+### Summary Comparison Table
+
+| Classifier | Speed (train) | Probability output | Feature selection | Typical accuracy rank | Recommended for |
+|---|---|---|---|---|---|
+| Linear SVM | Medium | No (without Platt) | Via weight magnitude | 1st-2nd | fMRI ROI, searchlight |
+| LDA (shrinkage) | Fast | Yes | No | 1st-2nd | EEG/MEG time-resolved |
+| Logistic (L2) | Medium | Yes | No | 1st-2nd | General purpose |
+| Logistic (L1) | Slow | Yes | Yes (sparse) | 2nd-3rd | Feature identification |
+| GNB | Very fast | Yes | No | Last | Quick sanity check |
+| RBF SVM | Slow | No | No | Variable | Large datasets only |
+
+(Rankings based on Misaki et al., 2010; Varoquaux et al., 2017)
+
+## Searchlight Analysis
+
+### Overview
+
+Searchlight analysis (Kriegeskorte et al., 2006) maps information content across the brain by applying multivariate classification to a small sphere of voxels centered on each brain voxel, producing a whole-brain accuracy map.
+
+### Parameters
+
+| Parameter | Recommended Value | Rationale | Source |
+|---|---|---|---|
+| **Sphere radius** | **3-4 voxels** (e.g., 4 mm for 2 mm isovoxels) | Balances spatial specificity with sufficient features; 4 mm radius yields ~100-120 voxels per searchlight | Kriegeskorte et al., 2006 |
+| **Classifier** | Linear SVM or LDA | Fast classifiers essential given thousands of searchlight positions | Misaki et al., 2010 |
+| **Cross-validation** | Leave-one-run-out | Respects temporal structure of fMRI data | Varoquaux et al., 2017 |
+| **Correction for multiple comparisons** | Cluster-based permutation or TFCE | Searchlight maps are spatially smooth; voxelwise FWE is overly conservative | Etzel et al., 2013 |
+| **Chance level map** | Subtract 1/n_classes or use permutation-based null | Raw accuracy maps are biased; always subtract chance or z-score | Combrisson & Jerbi, 2015 |
+
+### Searchlight Pitfalls
+
+1. **Spatial blurring**: The accuracy at each voxel reflects information in the entire sphere, not just the center voxel. Clusters in searchlight maps are larger than the true informative region (Etzel et al., 2013)
+2. **Center bias**: Voxels near the center of a large informative region get high accuracy because their spheres overlap maximally with the informative area; edge voxels get lower accuracy even if equally informative
+3. **Double-dipping**: Selecting ROIs based on searchlight maps and then performing further analyses on those ROIs using the same data is circular (Kriegeskorte et al., 2009). Use independent data for follow-up
+4. **Variable SNR**: Searchlights near brain edges contain fewer grey-matter voxels, reducing effective feature dimensionality and potentially accuracy. Use a grey-matter mask
+
+## Feature Selection Methods
+
+Feature selection can improve decoding by removing noisy features, but MUST be performed within each cross-validation fold.
+
+### ANOVA-Based Selection
+
+- Compute a one-way ANOVA F-statistic for each feature (voxel/channel) using training data
+- Select the top k features by F-value (typical: top **500-2000 voxels** for whole-brain fMRI; Haynes, 2015)
+- Fast and effective; widely used in fMRI MVPA
+- **Critical**: Compute ANOVA on training data only; selecting on all data leaks information (Kriegeskorte et al., 2009)
+
+### Recursive Feature Elimination (RFE)
+
+- Iteratively remove the least informative features based on classifier weights
+- Computationally expensive; requires re-training at each elimination step
+- Can improve accuracy but increases risk of overfitting to the cross-validation structure
+- Use only when theoretical motivation exists for sparse feature selection
+
+### Anatomical ROI Selection
+
+- Pre-define ROIs using anatomical atlases or functional localizers from independent data
+- Avoids the circularity of data-driven feature selection entirely
+- **Preferred** when you have strong anatomical hypotheses (Haynes, 2015)
+
+### PCA/ICA-Based Dimensionality Reduction
+
+- PCA: Retain components explaining **95-99%** of variance (community convention; no single citation)
+- ICA: Use independent components as features; more common for EEG/MEG
+- **Critical**: Fit PCA/ICA on training data only, then transform test data with the training-data projection
+
+## Hyperparameter Tuning
+
+### General Principle
+
+**Never optimize hyperparameters on the test set.** Use nested cross-validation: the inner loop tunes hyperparameters, the outer loop evaluates generalization (Varoquaux et al., 2017).
+
+### SVM C Parameter
+
+- **Default C=1** works well for most neuroimaging applications (Varoquaux et al., 2017)
+- If tuning: search over C = {0.001, 0.01, 0.1, 1, 10, 100} in inner cross-validation fold
+- Sane defaults often outperform tuned parameters when sample sizes are small, because nested cross-validation has even smaller training sets (Varoquaux et al., 2017)
+
+### Number of Cross-Validation Folds
+
+- **k=5** provides a good bias-variance tradeoff for accuracy estimation (Varoquaux, 2018)
+- **Leave-one-out** is unbiased but high variance; leads to unstable estimates (Varoquaux et al., 2017)
+- For fMRI, leave-one-run-out is dictated by the data structure (typically 4-12 runs, so k=4-12)
+- **Repeat** random splits 10-100 times to stabilize estimates when using k-fold (Varoquaux et al., 2017)
+
+## Software Tools
+
+### Python Ecosystem
+
+| Tool | Primary Use | URL |
+|---|---|---|
+| **scikit-learn** | General ML: classifiers, cross-validation, feature selection, permutation testing | scikit-learn.org |
+| **Nilearn** | fMRI decoding, searchlight, ROI-based MVPA, brain visualization | nilearn.github.io |
+| **MNE-Python** | EEG/MEG decoding, temporal generalization, sensor-space MVPA | mne.tools |
+| **rsatoolbox** (Python) | RSA: RDM construction, model comparison, noise ceiling, visualization | rsatoolbox.readthedocs.io |
+| **PyMVPA** | Full MVPA pipeline for fMRI; searchlight, cross-validation, feature selection | pymvpa.org |
+| **BrainIAK** | Advanced fMRI analysis: SRM, HTFA, MVPA, connectivity | brainiak.org |
+
+### MATLAB Ecosystem
+
+| Tool | Primary Use | URL |
+|---|---|---|
+| **CoSMoMVPA** | Comprehensive MVPA toolbox for fMRI, EEG, MEG; searchlight, RSA | cosmomvpa.org |
+| **The Decoding Toolbox (TDT)** | fMRI MVPA with SPM integration; decoding, RSA, reconstruction | tdt.sourceforge.net |
+| **RSA Toolbox** (MATLAB) | RSA: RDM construction, model testing, noise ceiling, visualization | github.com/rsagroup/rsatoolbox |
+| **MVPA-Light** | Classification and regression for multi-dimensional data; EEG/MEG focus | github.com/treder/MVPA-Light |
+| **FieldTrip** | EEG/MEG analysis including MVPA and temporal generalization | fieldtriptoolbox.org |
+| **ADAM Toolbox** | EEG MVPA: temporal generalization, group statistics | github.com/fahrenfort/ADAM |
+
+### Tool Selection Guidance
+
+- **fMRI, Python**: Nilearn + scikit-learn for standard MVPA; rsatoolbox for RSA
+- **fMRI, MATLAB**: CoSMoMVPA or TDT (integrates with SPM)
+- **EEG/MEG, Python**: MNE-Python for temporal decoding and RSA
+- **EEG/MEG, MATLAB**: CoSMoMVPA, MVPA-Light, or FieldTrip
+- **RSA specifically**: rsatoolbox (Python) or RSA Toolbox (MATLAB) by Kriegeskorte/Nili
+
+## References
+
+- Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance. *J. Neurosci. Methods*, 250, 126-136.
+- Etzel, J. A., Zacks, J. M., & Braver, T. S. (2013). Searchlight analysis: Promise, pitfalls, and potential. *NeuroImage*, 78, 261-269.
+- Haynes, J.-D. (2015). A primer on pattern-based approaches to fMRI. *Neuron*, 87(2), 257-270.
+- Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. *PNAS*, 103(10), 3863-3868.
+- Kriegeskorte, N., et al. (2009). Circular analysis in systems neuroscience. *Nat. Neurosci.*, 12(5), 535-540.
+- Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. *NeuroImage*, 53(1), 103-118.
+- Varoquaux, G., et al. (2017). Assessing and tuning brain decoders. *NeuroImage*, 145, 166-179.
+- Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. *NeuroImage*, 180, 68-77.