@brainpilot/skills 0.0.6

package/skills/08_Computational_Neuroscience/neural-decoding-analysis/references/rsa-guide.md

@@ -0,0 +1,266 @@
+# RSA Analysis Guide: Step-by-Step Workflow
+
+This reference document supplements `SKILL.md` with a complete step-by-step workflow for representational similarity analysis (RSA), including RDM construction, model specification, noise ceiling computation, and visualization.
+
+## Step-by-Step RSA Workflow
+
+### Step 1: Estimate Condition-Specific Activity Patterns
+
+**For fMRI:**
+1. Run a general linear model (GLM) with one regressor per condition (beta-series or least-squares-separate approach)
+2. Each condition yields a beta-weight map across voxels in an ROI
+3. Use **least-squares-separate (LSS)** for single-trial estimates when computing trial-wise RDMs (Mumford et al., 2012)
+4. Use **least-squares-all (LSA)** when you have a small number of conditions with many trials each
+
+> **Domain judgment**: For single-trial estimation, LSS produces less biased estimates than LSA when trials are closely spaced in time, because including all regressors simultaneously leads to collinearity in the design matrix (Mumford et al., 2012).
+
+**For EEG/MEG:**
+1. Epoch data around stimulus onset; average trials within each condition
+2. Use the spatial pattern at a specific time point, or average over a time window
+3. For time-resolved RSA: compute RDM at each time point separately
+
+### Step 2: Extract Patterns for the ROI or Brain Region
+
+- **ROI approach**: Extract voxel/channel values for each condition from a predefined region
+- **Searchlight RSA**: Compute RDMs in a local sphere around each voxel (same radius considerations as decoding searchlight: **3-4 voxels** for fMRI; Kriegeskorte et al., 2006)
+- **Whole-brain RSA**: Use parcellation (e.g., Glasser atlas with 360 parcels) and compute one RDM per parcel
+
+### Step 3: Construct the Neural RDM
+
+#### Choosing a Distance Metric
+
+| Metric | Formula | Bias | Scale | Best For |
+|---|---|---|---|---|
+| **Correlation distance** | 1 - Pearson(pattern_a, pattern_b) | Positively biased by noise | 0 to 2 | Exploratory RSA; comparing pattern shape |
+| **Euclidean distance** | sqrt(sum((a - b)^2)) | Positively biased by noise | 0 to inf | When amplitude matters |
+| **Crossnobis (cross-validated Mahalanobis)** | Multiply pattern differences across independent data partitions, with multivariate noise normalization | **Unbiased** (expected value = 0 under null) | -inf to inf | **Inferential statistics; model comparison** |
+
+(Walther et al., 2016; Kriegeskorte & Diedrichsen, 2019)
+
+#### Computing the Crossnobis Distance
+
+The crossnobis estimator requires data from multiple independent partitions (e.g., fMRI runs):
+
+1. Estimate condition-specific patterns separately for each run
+2. Compute the residual covariance matrix from the GLM residuals (for noise normalization)
+3. For each pair of conditions (i, j):
+ - Compute the pattern difference (beta_i - beta_j) in each run
+ - Multiply the pattern differences from different runs (cross-validated: never multiply a run's data with itself)
+ - Apply multivariate noise normalization (whiten using the residual covariance)
+4. Average across all cross-validated run pairs
+
+> **Domain judgment**: The cross-validation step ensures the estimator is unbiased. Without it, noise in the patterns inflates all pairwise distances by a positive amount proportional to the noise variance divided by the number of voxels. This bias makes it impossible to test whether two conditions are truly distinguishable (Walther et al., 2016).
+
+#### Noise Normalization (Multivariate Whitening)
+
+- Estimate the noise covariance matrix Sigma from GLM residuals
+- Apply Sigma^(-1/2) to the pattern differences before computing distances
+- This weights informative voxels (low noise correlation) more heavily than noisy or correlated voxels
+- Improves reliability of RDMs substantially, especially for large ROIs (Walther et al., 2016)
+
+### Step 4: Construct Model RDMs
+
+Model RDMs encode your theoretical predictions about representational structure.
+
+#### Common Model RDM Types
+
+| Model Type | Construction | Example |
+|---|---|---|
+| **Categorical** | Binary: 0 for same category, 1 for different category | "Face vs. object" model: 0 within faces, 0 within objects, 1 between |
+| **Feature-based** | Continuous distances from computational features | DNN layer activations; Gabor filter responses; semantic embedding distances |
+| **Behavioral** | Distances derived from human similarity judgments | Multi-arrangement task (Kriegeskorte & Mur, 2012); pairwise similarity ratings |
+| **Hierarchical** | Graded distances reflecting taxonomic depth | 0 within species, 0.5 within class, 1 across kingdoms |
+| **Computational model** | Distances from the internal representations of a model | CNN layer RDMs; RNN hidden state RDMs; reinforcement learning value RDMs |
+
+#### Constructing Model RDMs from Theoretical Predictions
+
+1. **Define your stimulus set**: List all conditions (N conditions)
+2. **For each pair of conditions**: Assign a predicted dissimilarity value
+3. **Organize into an N x N symmetric matrix** with zeros on the diagonal
+4. **Vectorize the lower triangle**: Extract the N*(N-1)/2 unique pairwise distances
+
+> **Domain judgment**: Model RDMs should be constructed BEFORE looking at neural data. Constructing models post-hoc to match observed neural RDMs is circular (Kriegeskorte et al., 2009). Pre-register your model RDMs when possible.
+
+### Step 5: Compare Neural RDMs to Model RDMs
+
+#### Comparison Methods
+
+| Method | Use When | Formula | Source |
+|---|---|---|---|
+| **Spearman rank correlation** | Default; robust to outliers and nonlinear monotonic relationships | rho(vectorized_brain_RDM, vectorized_model_RDM) | Nili et al., 2014 |
+| **Pearson correlation** | When the linear relationship between model and brain distances is theoretically motivated | r(vectorized_brain_RDM, vectorized_model_RDM) | Kriegeskorte et al., 2008 |
+| **Kendall tau-a** | For RDMs with many tied values (e.g., binary categorical models) | tau_a(vectorized_brain_RDM, vectorized_model_RDM) | Nili et al., 2014 |
+| **Partial correlation** | When controlling for a confound model (e.g., low-level visual similarity) | partial_rho(brain, model_A \| model_B) | Kriegeskorte & Diedrichsen, 2019 |
+| **Weighted regression** | Fitting multiple models simultaneously; estimating relative contributions | beta = (X'X)^{-1} X'y on vectorized RDMs | Kriegeskorte & Diedrichsen, 2019 |
+
+#### Choosing Between Spearman and Pearson
+
+- **Spearman** (default): Does not assume linearity between model and brain distances; only assumes a monotonic relationship. More robust to the arbitrary scaling of model predictions (Nili et al., 2014)
+- **Pearson**: Appropriate when the model makes quantitative predictions about the magnitude of distances, not just their rank order
+
+### Step 6: Statistical Inference
+
+#### Permutation Test for Individual Model Significance
+
+1. Permute condition labels **of the neural data** (not the model) to generate null RDMs
+2. Recompute the correlation between each permuted neural RDM and the model RDM
+3. Repeat **10,000 times** (Nili et al., 2014)
+4. p-value = proportion of null correlations >= observed correlation
+
+> **Domain judgment**: You must permute the condition labels (rows and columns of the RDM simultaneously), NOT individual entries of the RDM. Permuting entries destroys the distance-matrix structure and produces an invalid null distribution (Nili et al., 2014).
+
+#### Noise Ceiling Computation
+
+The noise ceiling bounds the best possible model fit given between-subject variability.
+
+**Upper bound:**
+1. For each subject, compute the correlation between that subject's RDM and the group-average RDM (including that subject)
+2. Average across subjects
+
+**Lower bound (preferred -- cross-validated):**
+1. For each subject, compute the correlation between that subject's RDM and the group-average RDM **excluding** that subject
+2. Average across subjects
+
+(Nili et al., 2014)
+
+**Interpretation:**
+- A model whose fit falls within the noise ceiling explains as much representational variance as is possible given the noise in the data
+- A model below the lower bound leaves systematic, explainable variance unaccounted for
+- The gap between upper and lower bound reflects the degree of between-subject variability
+
+#### Comparing Two Models
+
+1. For each subject, compute the correlation of the brain RDM with Model A and with Model B
+2. Compute the difference (corr_A - corr_B) for each subject
+3. Test the group-level difference with a paired t-test or Wilcoxon signed-rank test
+4. Alternatively, use a bootstrap test: resample subjects and test whether the difference in model fit is consistently positive or negative (Nili et al., 2014)
+
+### Step 7: Visualize Results
+
+#### RDM Visualization
+
+- Display as a **color-coded matrix** (conditions x conditions) with an appropriate colormap (diverging for crossnobis, sequential for correlation distance)
+- Order conditions meaningfully: group by category, sort by a relevant feature dimension
+- Always include a **colorbar** with the distance metric labeled
+- For crossnobis RDMs: use a diverging colormap centered at zero (e.g., blue-white-red) to highlight that negative values are meaningful
+
+#### Multidimensional Scaling (MDS)
+
+- Apply classical (metric) MDS to the RDM to project conditions into a 2D or 3D space
+- Useful for visualizing cluster structure and gradients in the representation
+- Report the stress value or proportion of variance explained by the MDS solution
+- **Caution**: MDS is a visualization aid, not a statistical test. Do not draw inferential conclusions from MDS plots alone (Kriegeskorte et al., 2008)
+
+#### Model Comparison Bar Plots
+
+- Plot the correlation between each model RDM and the brain RDM (one bar per model)
+- Include error bars (SEM across subjects or bootstrap confidence intervals)
+- Overlay the noise ceiling as a shaded band
+- Mark significant models and significant model differences
+
+## Category-Level vs. Exemplar-Level RSA
+
+### Category-Level Analysis
+
+- Conditions represent categories (e.g., faces, houses, tools, animals)
+- Typically **4-20 categories** with multiple exemplars per category
+- RDM captures between-category representational distances
+- Appropriate for: testing categorical organization (e.g., animate vs. inanimate distinction)
+- **Limitation**: Cannot distinguish whether a region encodes the category boundary or continuous features that correlate with category membership
+
+### Exemplar-Level Analysis
+
+- Each condition is a unique stimulus (e.g., 96 individual object images)
+- RDM captures fine-grained representational geometry
+- Appropriate for: comparing neural representations to computational model layer representations (e.g., DNN layers)
+- **Requirement**: Many more conditions (typically **48-96+**); each shown multiple times
+- **Challenge**: Single-trial beta estimates are noisier; crossnobis estimator is essential to avoid bias (Walther et al., 2016)
+
+> **Domain judgment**: Category-level RSA with few categories (e.g., 4) has very few unique distances in the RDM (only 6 for 4 categories), making model comparisons statistically weak. Exemplar-level designs with many conditions provide much more statistical power for RSA because the number of unique distances grows as N*(N-1)/2 (Kriegeskorte et al., 2008).
+
+## Multi-Arrangement for Behavioral RDMs
+
+### The Method
+
+The multi-arrangement task (Kriegeskorte & Mur, 2012) efficiently measures perceived similarity among stimuli:
+
+1. Participants view all stimuli on a computer screen and arrange them by perceived similarity (drag-and-drop)
+2. Similar items are placed close together; dissimilar items far apart
+3. The procedure adaptively selects subsets of stimuli for additional arrangements to improve estimation
+4. Pairwise distances between stimulus positions are extracted and combined across arrangements
+
+### Practical Parameters
+
+- **Number of stimuli**: Works well with **12-96** stimuli (fewer than 12: too few distances; more than 96: too many items to arrange simultaneously)
+- **Number of arrangements**: Typically **10-30** per participant; the algorithm adaptively selects informative subsets
+- **Subset size per arrangement**: **6-12 stimuli** per arrangement screen (participant fatigue increases with larger subsets)
+- **Duration**: Approximately **20-60 minutes** depending on stimulus set size
+
+### When to Use Behavioral RDMs
+
+- To test whether neural representations predict perceived similarity
+- To bridge neural and behavioral levels of description (Kriegeskorte et al., 2008)
+- As a model RDM alongside computational models to test which better explains neural data
+
+## Using the RSA Toolbox (Python: rsatoolbox)
+
+### Basic Workflow
+
+```python
+import rsatoolbox
+import numpy as np
+
+# 1. Create dataset objects (one per subject/session)
+# data: n_conditions x n_channels/voxels
+# descriptors: condition labels
+dataset = rsatoolbox.data.Dataset(
+ measurements=beta_patterns, # conditions x voxels
+ obs_descriptors={'stimulus': condition_labels}
+)
+
+# 2. Compute RDM
+rdm = rsatoolbox.rdm.calc_rdm(
+ dataset,
+ method='crossnobis', # preferred for inference
+ descriptor='stimulus',
+ noise=noise_covariance # from GLM residuals
+)
+
+# 3. Define model RDMs
+model_rdm = rsatoolbox.rdm.RDMs(
+ dissimilarities=model_distances, # vectorized lower triangle
+ rdm_descriptors={'name': 'my_model'}
+)
+
+# 4. Compare
+result = rsatoolbox.rdm.compare(
+ rdm, model_rdm,
+ method='spearman' # or 'corr', 'kendall'
+)
+
+# 5. Noise ceiling
+ceiling = rsatoolbox.inference.eval_fixed(
+ model_rdm, subject_rdms,
+ method='spearman'
+)
+```
+
+### Key Function Parameters
+
+| Function | Key Parameter | Recommended Value | Rationale |
+|---|---|---|---|
+| `calc_rdm` | `method` | `'crossnobis'` | Unbiased estimator (Walther et al., 2016) |
+| `calc_rdm` | `noise` | GLM residual covariance | Multivariate noise normalization improves reliability |
+| `compare` | `method` | `'spearman'` | Robust to nonlinear monotonic transforms (Nili et al., 2014) |
+| `eval_fixed` | `method` | `'spearman'` | Consistent with comparison method |
+
+## References
+
+- Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. *PNAS*, 103(10), 3863-3868.
+- Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis. *Front. Syst. Neurosci.*, 2, 4.
+- Kriegeskorte, N., et al. (2009). Circular analysis in systems neuroscience. *Nat. Neurosci.*, 12(5), 535-540.
+- Kriegeskorte, N., & Mur, M. (2012). Inverse MDS: Inferring dissimilarity structure from multiple item arrangements. *Front. Psychol.*, 3, 245.
+- Kriegeskorte, N., & Diedrichsen, J. (2019). Peeling the onion of brain representations. *Annu. Rev. Neurosci.*, 42, 407-432.
+- Mumford, J. A., Turner, B. O., Ashby, F. G., & Poldrack, R. A. (2012). Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses. *NeuroImage*, 59(3), 2636-2643.
+- Nili, H., et al. (2014). A toolbox for representational similarity analysis. *PLoS Comput. Biol.*, 10(4), e1003553.
+- Walther, A., et al. (2016). Reliability of dissimilarity measures for multi-voxel pattern analysis. *NeuroImage*, 137, 188-200.

package/skills/08_Computational_Neuroscience/neural-population-analysis-guide/SKILL.md

@@ -0,0 +1,305 @@
+---
+name: "neural-population-analysis-guide"
+description: "Guides dimensionality reduction and latent-variable analysis of neural populations (PCA, GPFA, dPCA)"
+domain: "computational-neuroscience"
+version: "1.0.0"
+authors:
+ - "Claude Code Agent"
+papers:
+ - "Cunningham & Yu, 2014"
+ - "Yu et al., 2009"
+ - "Kobak et al., 2016"
+ - "Churchland et al., 2012"
+ - "King & Dehaene, 2014"
+ - "Humphries, 2021"
+dependencies:
+ required:
+ - research-literacy
+ recommended:
+ - neural-decoding-analysis
+review_status: "ai-generated"
+---
+
+# Neural Population Analysis Guide
+
+## Purpose
+
+This skill encodes expert methodological knowledge for dimensionality reduction and latent-variable analysis of neural population recordings. A competent programmer without computational neuroscience training will get this wrong because:
+
+- **Not all dimensionality reduction is the same.** PCA, Factor Analysis, GPFA, and dPCA answer fundamentally different questions about neural data. PCA finds axes of maximum variance; dPCA demixes variance by task parameter; GPFA extracts smooth single-trial trajectories. Choosing the wrong method answers the wrong question (Cunningham & Yu, 2014).
+- **Standard data science preprocessing destroys neural signal.** Naive z-scoring or standard scaling of neural firing rates removes important information about rate differences across neurons. Soft normalization is required (Churchland et al., 2012).
+- **Visualization methods are not analysis methods.** t-SNE and UMAP produce visually compelling low-dimensional embeddings but their distances are not interpretable, axes are not meaningful, and results are sensitive to hyperparameters. They must never be used for quantitative inference (Cunningham & Yu, 2014).
+- **Dimensionality is not determined by "percent variance explained."** There is no universal threshold (e.g., 90%) for choosing the number of PCs. Parallel analysis or cross-validation is required to determine true dimensionality (Humphries, 2021).
+
+## When to Use This Skill
+
+- Analyzing simultaneously recorded neural populations (multi-electrode arrays, Neuropixels, calcium imaging)
+- Choosing between PCA, Factor Analysis, GPFA, dPCA, jPCA, or nonlinear methods
+- Extracting low-dimensional neural trajectories from spike trains
+- Demixing neural variance by task parameters (stimulus, decision, time)
+- Setting up population decoding analyses with temporal generalization
+- Determining data requirements (neuron counts, trial counts) for population analyses
+
+Do NOT use this skill for:
+
+- Single-neuron tuning curve analysis (this is a different analytical framework)
+- Spike sorting or preprocessing of raw electrophysiology (upstream of this skill)
+- fMRI multivariate pattern analysis (see `neural-decoding-analysis` skill instead)
+
+## Research Planning Protocol
+
+Before executing the domain-specific steps below, you MUST:
+
+1. **State the research question** -- What specific question is this analysis/paradigm addressing?
+2. **Justify the method choice** -- Why is this approach appropriate? What alternatives were considered?
+3. **Declare expected outcomes** -- What results would support vs. refute the hypothesis?
+4. **Note assumptions and limitations** -- What does this method assume? Where could it mislead?
+5. **Present the plan to the user and WAIT for confirmation** before proceeding.
+
+For detailed methodology guidance, see the `research-literacy` skill.
+
+---
+
+
+## ⚠️ Verification Notice
+
+This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please [open an issue](https://github.com/HaoxuanLiTHUAI/awesome_cognitive_and_neuroscience_skills/issues).
+
+## When to Use Population Analysis vs. Single-Neuron Analysis
+
+Single-neuron analysis characterizes individual tuning curves and response properties. Population analysis characterizes the collective, low-dimensional structure of neural activity. Use population analysis when (Cunningham & Yu, 2014):
+
+- Neural responses are distributed across many neurons with no single neuron carrying the full signal
+- You want to characterize the *geometry* of neural representations (e.g., are stimulus and decision signals orthogonal?)
+- You want to track *trajectories* of population activity over time during behavior
+- Single-neuron tuning is mixed or complex, making classical tuning curve approaches uninformative
+- You have simultaneous recordings from tens to hundreds of neurons
+
+> **Domain judgment**: Population analysis does NOT require that individual neurons be untuned. Even populations of well-tuned neurons have emergent low-dimensional structure that single-neuron analyses miss. The question is whether the research question is about individual neuron properties or collective population geometry (Cunningham & Yu, 2014).
+
+---
+
+## Method Selection Decision Tree
+
+```
+What is your research question?
+|
++-- "What are the dominant patterns of co-variation across the population?"
+| --> PCA (linear, static)
+| Simplest method; start here
+| Output: principal components, variance explained, projections
+|
++-- "What are the smooth latent trajectories on single trials?"
+| --> GPFA (Yu et al., 2009) or Factor Analysis
+| For temporal structure; extracts trial-by-trial dynamics
+| Output: latent trajectories per trial, dimensionality estimate
+|
++-- "How is variance distributed across task parameters?"
+| --> dPCA (Kobak et al., 2016)
+| Demixes stimulus, decision, time, and interaction variance
+| Output: demixed components, marginalization pie chart
+|
++-- "Are there rotational dynamics in the population?"
+| --> jPCA (Churchland et al., 2012)
+| Finds rotational structure in neural trajectories
+| Output: rotational plane projections, R^2 of rotational fit
+|
++-- "I want to visualize population structure in 2-3D"
+ --> t-SNE or UMAP -- VISUALIZATION ONLY
+ NOT for quantitative analysis or distance comparisons
+ Output: 2D/3D embedding (qualitative only)
+```
+
+---
+
+## PCA for Neural Data
+
+### Preprocessing Before PCA
+
+1. **Trial averaging vs. single-trial**: Trial-averaged PCA smooths out single-trial variability and is appropriate when the question is about condition-mean trajectories. Single-trial PCA preserves trial-to-trial variability but requires more data (Cunningham & Yu, 2014).
+
+2. **Soft normalization** (CRITICAL): Do NOT use standard z-scoring (subtract mean, divide by SD). This eliminates firing rate differences that carry information. Instead, use soft normalization (Churchland et al., 2012):
+
+ ```
+ normalized_rate = (rate - mean) / (range + constant)
+ ```
+
+ where `constant` is typically **5 spikes/s** (Churchland et al., 2012). This prevents low-firing neurons from dominating due to noise amplification while preserving rate differences.
+
+3. **Square-root transform**: An alternative normalization for Poisson-like spike counts: `sqrt(mean firing rate + constant)` (Churchland et al., 2012). Stabilizes variance across neurons with different firing rates.
+
+4. **Mean subtraction**: Subtract the condition-averaged PSTH across all conditions from each condition. This removes the shared temporal modulation and focuses PCA on condition-discriminating variance (Kobak et al., 2016).
+
+### Determining Dimensionality
+
+| Method | Description | When to Use | Source |
+|---|---|---|---|
+| **Parallel analysis** | Compare eigenvalues to those from shuffled data | **Recommended default**; principled threshold | Humphries, 2021 |
+| **Cross-validated PCA** | Hold out neurons or time bins; test reconstruction | Avoids overfitting; gold standard | Cunningham & Yu, 2014 |
+| **Scree plot elbow** | Subjective visual inspection | Quick but unreliable; avoid for publication | Expert consensus |
+| **Percent variance threshold** | e.g., "keep PCs explaining 90%" | **NOT recommended**; arbitrary threshold | Humphries, 2021 |
+
+> **Domain judgment**: There is no fixed variance-explained threshold for choosing the number of PCs. The "90% rule" from data science is meaningless for neural data because it confounds signal dimensionality with noise. Use parallel analysis (compare eigenvalues to those expected from random data with matched statistics) or cross-validation (Humphries, 2021).
+
+### Cross-Validated PCA
+
+To avoid overfitting dimensionality estimates:
+
+1. Split neurons into two halves (or use leave-one-neuron-out)
+2. Fit PCA on one half
+3. Project the other half onto the PCs
+4. Measure reconstruction error as a function of the number of PCs
+5. Choose dimensionality where held-out reconstruction stops improving
+
+This is especially critical with small neuron counts (<50 neurons) where noise dimensions can mimic signal dimensions (Cunningham & Yu, 2014).
+
+---
+
+## GPFA -- Gaussian Process Factor Analysis
+
+GPFA extracts smooth, low-dimensional latent trajectories from single-trial spike trains by combining factor analysis with Gaussian process temporal smoothing (Yu et al., 2009).
+
+### When to Use GPFA
+
+- You need **single-trial** latent trajectories (not just condition averages)
+- The research question involves trial-to-trial variability in population dynamics
+- You want an estimate of dimensionality that accounts for temporal smoothness
+
+### Key Parameters
+
+| Parameter | Recommended Value | Rationale | Source |
+|---|---|---|---|
+| Bin size | **20--50 ms** | Smaller bins preserve temporal resolution but increase noise; 20 ms is standard for motor cortex | Yu et al., 2009 |
+| Latent dimensions | Cross-validate | Fit models with 2--15 dimensions; select by leave-one-neuron-out cross-validation log-likelihood | Yu et al., 2009 |
+| GP timescale (tau) | Learned from data | Each latent dimension has its own timescale; inspect for biologically plausible values (10--500 ms) | Yu et al., 2009 |
+
+### Data Requirements
+
+- **Minimum simultaneously recorded neurons**: ~**50--100** for stable latent trajectories; below 30, estimates become unreliable (Yu et al., 2009; Cunningham & Yu, 2014)
+- **Trial count**: **50+ trials per condition** for reliable single-trial extraction (Yu et al., 2009)
+- **Stationarity assumption**: GPFA assumes the mapping from latent variables to neurons is fixed across all trials in the dataset. Non-stationarities (e.g., learning, drift) violate this assumption
+
+> **Domain judgment**: GPFA is NOT simply "smoothed PCA." Factor analysis and PCA make different assumptions about noise structure. PCA treats all variance as signal; factor analysis separates shared (signal) variance from private (noise) variance per neuron. GPFA adds temporal smoothness to the shared component. This distinction matters when neurons have heterogeneous noise levels (Yu et al., 2009).
+
+---
+
+## dPCA -- Demixed Principal Component Analysis
+
+dPCA finds linear projections that capture maximum variance attributable to specific task parameters (e.g., stimulus identity, decision, time, and their interactions), rather than maximum total variance like PCA (Kobak et al., 2016).
+
+### When to Use dPCA
+
+- You have a factorial experimental design with multiple task parameters
+- You want to know how much population variance is explained by each parameter
+- You want to visualize how stimulus, decision, and temporal signals are mixed or separated in the population
+
+### Key Procedures
+
+1. **Marginalization**: Partition the data tensor by task parameter (e.g., average over all decisions to isolate stimulus variance). dPCA finds components that maximize variance within each marginalization (Kobak et al., 2016).
+
+2. **Regularization**: The regularization parameter lambda controls the tradeoff between demixing (fidelity to marginalization) and reconstruction (total variance captured). Select lambda by **cross-validation** (Kobak et al., 2016):
+ - Hold out a subset of trials
+ - Fit dPCA on remaining trials for a range of lambda values
+ - Choose lambda that minimizes reconstruction error on held-out trials
+
+3. **Significance testing**: Use a **shuffle test** to determine whether each component captures significantly more marginalized variance than expected by chance:
+ - Shuffle condition labels for the parameter of interest (1000+ iterations)
+ - Recompute dPCA on shuffled data
+ - Compare observed marginalized variance to the null distribution
+ (Kobak et al., 2016)
+
+4. **Marginalization pie chart**: Report the proportion of total variance explained by each marginalization (stimulus, decision, time, interaction). This provides a concise summary of how the population mixes task variables (Kobak et al., 2016).
+
+### Data Requirements
+
+- **Trial counts**: **20+ trials per condition** minimum; more for stable interaction terms (Kobak et al., 2016)
+- **Neuron counts**: Works with as few as 20--30 neurons but more reliable with 50+ (Kobak et al., 2016)
+- **Factorial design**: dPCA requires a balanced factorial design; missing conditions or severely unbalanced cell sizes degrade performance
+
+---
+
+## Population Decoding
+
+### Classifier Selection for Neural Populations
+
+| Classifier | When to Use | Source |
+|---|---|---|
+| **Linear SVM** | Default; robust for high-dimensional population data | Cunningham & Yu, 2014 |
+| **LDA** | When number of neurons << number of trials | Cunningham & Yu, 2014 |
+| **Logistic Regression** | When probability estimates are needed | Cunningham & Yu, 2014 |
+
+### Cross-Validation for Neural Data
+
+| Strategy | When to Use | Source |
+|---|---|---|
+| **Leave-one-trial-out** | Small trial counts; maximizes training data | Cunningham & Yu, 2014 |
+| **Stratified k-fold** (k=5--10) | Sufficient trials; balances bias/variance | Cunningham & Yu, 2014 |
+
+### Temporal Generalization Matrix
+
+Train a classifier at time t, test at all times t'. The resulting matrix reveals dynamics of neural coding (King & Dehaene, 2014):
+
+- **Diagonal pattern**: Transient coding; neural representation changes over time
+- **Block pattern**: Stable, sustained representation
+- **Off-diagonal**: Reactivation of earlier representations
+
+See the `neural-decoding-analysis` skill for detailed guidance on temporal generalization.
+
+---
+
+## Common Pitfalls
+
+### 1. Using Standard Z-Scoring Instead of Soft Normalization
+
+Standard z-scoring (subtract mean, divide by SD) amplifies noise from low-firing neurons and eliminates informative rate differences. Always use soft normalization or square-root transform for neural population data (Churchland et al., 2012).
+
+### 2. Interpreting t-SNE/UMAP Distances as Meaningful
+
+t-SNE and UMAP distort distances, create spurious clusters, and are sensitive to hyperparameters (perplexity, n_neighbors). They are useful for visualization but must NEVER be used for quantitative claims about distance, clustering, or dimensionality (Cunningham & Yu, 2014).
+
+### 3. Using Percent Variance Explained to Choose Dimensionality
+
+There is no universal threshold. The correct approach is parallel analysis (compare to null eigenvalue distribution) or cross-validated reconstruction (Humphries, 2021).
+
+### 4. Applying GPFA to Condition-Averaged Data
+
+GPFA is designed for single-trial data. Applying it to trial-averaged PSTHs defeats its purpose (separating shared latent signal from private noise) and provides no advantage over PCA (Yu et al., 2009).
+
+### 5. Ignoring Neuron Count Requirements
+
+Population analyses have minimum neuron requirements for reliable estimates. Below ~30 simultaneously recorded neurons, dimensionality estimates from PCA are dominated by noise; GPFA latent trajectories become unstable below ~50 neurons (Cunningham & Yu, 2014; Yu et al., 2009).
+
+### 6. Not Cross-Validating dPCA Regularization
+
+Using default or arbitrary regularization parameters in dPCA can either over-demix (fitting noise) or under-demix (missing true task-related structure). Always cross-validate lambda (Kobak et al., 2016).
+
+---
+
+## Minimum Reporting Checklist
+
+Based on Cunningham & Yu (2014), Kobak et al. (2016), and Yu et al. (2009):
+
+- [ ] Number of simultaneously recorded neurons and recording method
+- [ ] Neuron inclusion criteria (e.g., minimum firing rate, signal quality)
+- [ ] Normalization method (soft normalization constant, or square-root transform)
+- [ ] Method used and justification (PCA, GPFA, dPCA, jPCA)
+- [ ] Dimensionality selection method (parallel analysis, cross-validation) and chosen dimensionality
+- [ ] For GPFA: bin size, number of latent dimensions, learned GP timescales
+- [ ] For dPCA: regularization parameter, cross-validation procedure, shuffle test results
+- [ ] Trial counts per condition
+- [ ] Variance explained (total and per component/marginalization)
+- [ ] Software and version used (e.g., GPFA code from Yu et al., dPCA Python/MATLAB toolbox)
+
+---
+
+## Key References
+
+- Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Foster, J. D., Nuyujukian, P., Ryu, S. I., & Shenoy, K. V. (2012). Neural population dynamics during reaching. *Nature*, 487(7405), 51--56.
+- Cunningham, J. P., & Yu, B. M. (2014). Dimensionality reduction for large-scale neural recordings. *Nature Neuroscience*, 17(11), 1500--1509.
+- Humphries, M. D. (2021). Strong and weak principles of neural dimension reduction. *Neuron*, 109(8), 1230--1234.
+- King, J.-R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: The temporal generalization method. *Trends in Cognitive Sciences*, 18(4), 203--210.
+- Kobak, D., Brendel, W., Constantinidis, C., Feierstein, C. E., Kepecs, A., Mainen, Z. F., Qi, X.-L., Romo, R., Uchida, N., & Machens, C. K. (2016). Demixed principal component analysis of neural population data. *eLife*, 5, e10989.
+- Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S. I., Shenoy, K. V., & Sahani, M. (2009). Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. *Journal of Neurophysiology*, 102(1), 614--635.
+
+See `references/method-comparison.md` for detailed parameter tables and software recommendations.
+See `references/data-requirements.md` for minimum neuron and trial count guidelines by method.

package/skills/08_Computational_Neuroscience/neural-population-analysis-guide/references/data-requirements.md

@@ -0,0 +1,60 @@
+# Data Requirements for Neural Population Analysis Methods
+
+This reference document supplements `SKILL.md` with detailed minimum data requirements for each population analysis method.
+
+## Neuron Count Requirements
+
+| Method | Minimum Neurons | Recommended Neurons | Rationale | Source |
+|---|---|---|---|---|
+| PCA (condition-avg) | 10--20 | 50+ | Fewer PCs than neurons; noise dominates with very few | Cunningham & Yu, 2014 |
+| PCA (single-trial) | 30+ | 100+ | Single-trial noise requires more neurons for stable estimates | Cunningham & Yu, 2014 |
+| Factor Analysis | 20+ | 50+ | Must estimate shared vs. private variance | Cunningham & Yu, 2014 |
+| GPFA | 50+ | 100+ | Single-trial trajectory extraction requires overdetermined system | Yu et al., 2009 |
+| dPCA | 20+ | 50+ | Must have enough neurons to capture variance in each marginalization | Kobak et al., 2016 |
+| jPCA | 30+ | 50+ | Needs stable 6D PCA subspace first | Churchland et al., 2012 |
+| t-SNE/UMAP | Any | N/A | Visualization only; fewer neurons = less meaningful | Expert consensus |
+
+## Trial Count Requirements
+
+| Method | Minimum Trials/Condition | Recommended Trials/Condition | Rationale | Source |
+|---|---|---|---|---|
+| PCA (condition-avg) | 20+ | 50+ | Trial averaging requires sufficient trials per condition to suppress noise | Cunningham & Yu, 2014 |
+| PCA (single-trial) | 50+ | 100+ | Covariance estimation on single trials requires large sample | Cunningham & Yu, 2014 |
+| GPFA | 50+ | 100+ | EM convergence and stable latent trajectory extraction | Yu et al., 2009 |
+| dPCA | 20+ | 40+ | Cross-validation of regularization requires held-out trials | Kobak et al., 2016 |
+| jPCA | 30+ | 50+ | Stable condition-averaged trajectories | Churchland et al., 2012 |
+| Population decoding | 20+ | 50+ | Cross-validated classification requires sufficient trials per fold | Cunningham & Yu, 2014 |
+
+## Recording Duration Guidelines
+
+| Analysis Goal | Minimum Duration | Rationale | Source |
+|---|---|---|---|
+| Peri-event trajectories | Trial duration + 500 ms pre/post | Capture dynamics around events | Expert consensus |
+| Slow dynamics (e.g., working memory) | Full trial including delay period | Must capture sustained activity | Expert consensus |
+| Rotational dynamics (jPCA) | 200+ ms of movement-related activity | Rotations typically occur on 100--300 ms timescale | Churchland et al., 2012 |
+| GPFA latent dynamics | 500+ ms per trial | GP timescale learning needs sufficient temporal extent | Yu et al., 2009 |
+
+## Simultaneous vs. Sequential Recording
+
+| Recording Type | Compatible Methods | Incompatible Methods | Notes |
+|---|---|---|---|
+| Simultaneous (multi-electrode array) | All methods | None | Gold standard; captures noise correlations |
+| Sequential (single electrode, pseudopopulation) | PCA (condition-avg), dPCA | GPFA, noise correlation analysis | Cannot capture trial-by-trial co-variability; single-trial methods invalid |
+
+> **Critical**: GPFA and any method requiring single-trial co-variability estimates MUST use simultaneously recorded neurons. Pseudopopulations (neurons recorded on different sessions stitched together) can be used for condition-averaged analyses like PCA or dPCA, but not for single-trial latent trajectory extraction (Cunningham & Yu, 2014).
+
+## Practical Checklist Before Analysis
+
+- [ ] Confirm recording type (simultaneous vs. sequential)
+- [ ] Count total neurons passing quality criteria
+- [ ] Count trials per condition (verify balance across conditions for dPCA)
+- [ ] Check temporal extent of data relative to analysis goals
+- [ ] Verify that neuron/trial counts meet minimum thresholds for chosen method
+- [ ] If below minimums, consider simpler methods (e.g., PCA instead of GPFA) or pool across sessions (with caveats)
+
+## References
+
+- Churchland, M. M., et al. (2012). Neural population dynamics during reaching. *Nature*, 487(7405), 51--56.
+- Cunningham, J. P., & Yu, B. M. (2014). Dimensionality reduction for large-scale neural recordings. *Nature Neuroscience*, 17(11), 1500--1509.
+- Kobak, D., et al. (2016). Demixed principal component analysis of neural population data. *eLife*, 5, e10989.
+- Yu, B. M., et al. (2009). Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. *Journal of Neurophysiology*, 102(1), 614--635.