@brainpilot/skills 0.0.6

package/skills/07_Computational_Modeling/bayesian-cognitive-model-builder/SKILL.md

@@ -0,0 +1,294 @@
+---
+name: "bayesian-cognitive-model-builder"
+description: "Domain-validated guidance for building hierarchical Bayesian cognitive models with Stan/PyMC: prior specification, model structure, MCMC diagnostics, and posterior predictive checks"
+domain: "computational-cognitive-modeling"
+version: "1.0.0"
+papers:
+ - "Gelman et al., 2013"
+ - "Lee & Wagenmakers, 2014"
+ - "Kruschke, 2015"
+ - "Schad et al., 2021"
+ - "Vehtari et al., 2017"
+ - "Vehtari et al., 2021"
+dependencies:
+ required:
+ - research-literacy
+ recommended:
+ - drift-diffusion-model
+ - signal-detection-analysis
+review_status: "ai-generated"
+---
+
+# Bayesian Cognitive Model Builder
+
+## Purpose
+
+This skill encodes expert knowledge for building hierarchical Bayesian cognitive models using probabilistic programming languages (Stan, PyMC). It addresses the modeling decisions that require domain expertise beyond knowing Stan/PyMC syntax: how to choose priors that respect cognitive constraints, when to use hierarchical structure, how to diagnose MCMC pathologies, and how to evaluate model adequacy through posterior predictive checks.
+
+A competent programmer without cognitive modeling training would get wrong: which prior families are appropriate for cognitive parameters (e.g., RT must be positive, learning rates bounded in [0,1]), when partial pooling outperforms complete pooling or no pooling, how to detect non-identifiability in cognitive models, and what constitutes adequate MCMC convergence for publishable results.
+
+## When to Use This Skill
+
+- Building a generative model of a cognitive process (decision-making, learning, memory, perception) where parameters have psychological interpretations
+- Estimating individual differences in cognitive parameters while borrowing strength across participants (hierarchical/multilevel models)
+- Working with small samples or sparse data per participant where regularization through priors prevents overfitting
+- Parameter uncertainty matters for your scientific conclusions (credible intervals, not just point estimates)
+- Comparing competing cognitive models via information criteria (LOO-CV, WAIC) or Bayes factors
+- Fitting established cognitive models (DDM, signal detection, reinforcement learning, multinomial processing trees) in a Bayesian framework
+
+## When NOT to Use This Skill
+
+- If your model has a closed-form MLE and you have large, balanced samples, frequentist estimation may be simpler and adequate
+- For purely predictive models where parameter interpretability is irrelevant (consider machine learning approaches)
+- If you need a general-purpose Bayesian regression model without cognitive process parameters (see `cogsci-statistics` skill)
+- For EEG/fMRI analysis pipelines without explicit cognitive models (see `erp-analysis` or `fmri-glm-analysis-guide` skills)
+
+## Research Planning Protocol
+
+Before executing the domain-specific steps below, you MUST:
+
+1. **State the research question** -- What cognitive mechanism is this model capturing?
+2. **Justify the method choice** -- Why Bayesian (not MLE, not frequentist)? What alternatives were considered?
+3. **Declare expected outcomes** -- What parameter patterns would support vs. refute the hypothesis?
+4. **Note assumptions and limitations** -- What does this model assume about the cognitive process?
+5. **Present the plan to the user and WAIT for confirmation** before proceeding.
+
+For detailed methodology guidance, see the `research-literacy` skill.
+
+
+## ⚠️ Verification Notice
+
+This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please [open an issue](https://github.com/HaoxuanLiTHUAI/awesome_cognitive_and_neuroscience_skills/issues).
+
+## Model Structure Decision Tree
+
+Choosing the right level of pooling is a fundamental modeling decision that a non-specialist routinely gets wrong.
+
+### Step 1: Do You Have Grouped Data?
+
+If your data has a natural grouping structure (e.g., multiple trials per participant, participants within conditions), you need to decide on a pooling strategy. If not, fit a single model.
+
+### Step 2: Choose the Pooling Level
+
+| Strategy | Structure | When Appropriate | Risk |
+|----------|-----------|------------------|------|
+| **Complete pooling** | One set of parameters for all participants | Large homogeneous groups, nuisance individual differences | Ignores meaningful individual variation; biased group estimates if heterogeneity exists (Gelman et al., 2013, Ch. 5) |
+| **No pooling** | Separate parameters per participant | Many trials per participant (>200), individual-level inference is the goal | Noisy estimates for participants with few trials; no borrowing of strength (Gelman et al., 2013, Ch. 5) |
+| **Partial pooling (hierarchical)** | Individual parameters drawn from group distribution | **Default choice** for cognitive modeling; few-to-moderate trials per participant; individual differences are scientifically meaningful | Requires MCMC; potential convergence issues with centered parameterization (Gelman et al., 2013, Ch. 5) |
+
+> **Critical domain knowledge**: Hierarchical (partial pooling) models should be the default in cognitive science. They automatically regularize extreme individual estimates toward the group mean -- a property called "shrinkage" -- which is especially valuable with typical cognitive science sample sizes of 20-40 participants with 50-200 trials each (Lee & Wagenmakers, 2014, Ch. 8).
+
+### Step 3: Centered vs. Non-Centered Parameterization
+
+For hierarchical models, the parameterization choice affects MCMC efficiency:
+
+- **Centered parameterization**: `theta_j ~ Normal(mu, sigma)`. Use when there are many observations per group (>100 trials per participant) and the data are informative relative to the prior (Betancourt & Girolami, 2015).
+- **Non-centered parameterization**: `theta_j = mu + sigma * eta_j` where `eta_j ~ Normal(0, 1)`. Use when there are few observations per group, the group-level variance is small, or you encounter divergent transitions with centered parameterization (Betancourt & Girolami, 2015; Stan User's Guide, Section 1.13).
+
+> **When in doubt, use non-centered parameterization.** It is more robust across a wider range of data configurations and is the Stan Development Team's default recommendation.
+
+## Prior Selection Principles
+
+### General Philosophy
+
+Use **weakly informative priors** that encode known constraints without dominating the likelihood. The goal is to rule out impossible or implausible parameter values while remaining agnostic about the precise value (Gelman et al., 2008; Gelman et al., 2013, Ch. 2).
+
+> **Domain-critical principle**: Cognitive parameters have natural constraints that generic "flat" or "diffuse" priors violate. Reaction times cannot be negative. Probabilities must lie in [0,1]. Learning rates are bounded. Firing rates are non-negative. Encoding these constraints in the prior is not "being subjective" -- it is encoding physical and psychological reality (Lee & Wagenmakers, 2014, Ch. 4).
+
+### Prior Families for Common Cognitive Parameter Types
+
+| Parameter Type | Recommended Prior | Rationale | Source |
+|---------------|-------------------|-----------|--------|
+| Location (unbounded) | Normal(0, sd) or Student-t(3, 0, sd) | Weakly informative; heavier tails with Student-t for robustness | Gelman et al., 2008 |
+| Scale / variance | Half-Normal(0, sd) or Half-Cauchy(0, sd) | Positive-only; Half-Cauchy allows heavier tails for group-level SDs | Gelman, 2006; Polson & Scott, 2012 |
+| Probability (0 to 1) | Beta(a, b) | Natural conjugate for binomial; Beta(1,1) = Uniform; Beta(2,2) = weakly informative centered at 0.5 | Kruschke, 2015, Ch. 6 |
+| Rate (0 to 1) | Beta(1.1, 1.1) or logit-Normal | Gently regularizes away from boundaries | Gelman et al., 2013, Ch. 2 |
+| Positive continuous | Gamma(shape, rate) or Lognormal(mu, sigma) | For RT, non-decision time, threshold parameters | Lee & Wagenmakers, 2014, Ch. 4 |
+| Correlation matrix | LKJ(eta) | eta=1: uniform over matrices; **eta=2: weakly informative** (Stan default recommendation) | Lewandowski et al., 2009; Stan User's Guide |
+| Simplex (sums to 1) | Dirichlet(alpha) | alpha=1: uniform on simplex; alpha>1: concentrates toward center | Gelman et al., 2013, Ch. 2 |
+
+For detailed cognitive-domain-specific prior tables, see `references/prior-selection-guide.md`.
+
+### Prior Predictive Checking
+
+**Always** run a prior predictive check before fitting to data (Schad et al., 2021; Gabry et al., 2019):
+
+1. Sample parameters from your priors (no data)
+2. Simulate data from the model using those parameters
+3. Check: Does the simulated data look plausible for the domain?
+ - If the prior predicts impossible RTs (e.g., negative, or > 60 seconds), the prior is too diffuse
+ - If the prior predicts accuracy always near 50% or always near 100%, reconsider
+4. Iterate on priors until prior predictive distributions cover plausible data ranges without including absurd values
+
+## Common Cognitive Models in Bayesian Framework
+
+### Drift-Diffusion Model (DDM / HDDM)
+
+- **Key parameters**: drift rate (v), boundary separation (a), non-decision time (t), starting point bias (z)
+- **Bayesian implementation**: HDDM package (Wiecki et al., 2013) uses informative priors from empirical meta-analysis (Matzke & Wagenmakers, 2009)
+- **Typical priors**: See `references/prior-selection-guide.md` for parameter-specific recommendations
+- **Critical note**: Within-trial noise (s) is a scaling parameter fixed by convention at **0.1** (Ratcliff, 1978) or **1.0** (Navarro & Fuss, 2009). All other parameter ranges depend on this choice.
+- Also see the `drift-diffusion-model` skill for detailed DDM guidance
+
+### Signal Detection Theory (SDT)
+
+- **Key parameters**: sensitivity (d'), criterion (c)
+- **Priors for d'**: Normal(0, 2) is weakly informative; typical empirical values range **0 to 4** (Macmillan & Creelman, 2005)
+- **Priors for c**: Normal(0, 1.5) centered at no bias; typical range **-2 to 2** (Macmillan & Creelman, 2005)
+- **Hierarchical structure**: Individual d' and c drawn from group distributions (Rouder & Lu, 2005)
+- Also see the `signal-detection-analysis` skill
+
+### Multinomial Processing Trees (MPT)
+
+- **Key parameters**: Processing probabilities (all in [0,1])
+- **Priors**: Beta(1,1) for non-informative or Beta(a,b) with shape informed by prior studies (Klauer, 2010)
+- **Hierarchical extension**: Latent-trait MPT with probit-transformed parameters drawn from multivariate normal (Klauer, 2010)
+
+### Item Response Theory (IRT)
+
+- **Key parameters**: Ability (theta), difficulty (b), discrimination (a)
+- **Priors**: theta ~ Normal(0,1) by convention; b ~ Normal(0, 2); a ~ Lognormal(0, 0.5) to enforce positivity (de Boeck & Wilson, 2004)
+- **Cognitive application**: Modeling learning, cognitive ability, or item difficulty in memory/attention tasks
+
+### Reinforcement Learning (RL)
+
+- **Key parameters**: Learning rate (alpha in [0,1]), inverse temperature (beta > 0), decay, perseveration
+- **Priors for alpha**: Beta(1.1, 1.1) weakly informative on [0,1] (Daw, 2011; Gershman, 2016)
+- **Priors for beta (inverse temperature)**: Gamma(2, 1) or Lognormal(0, 1) constraining to positive values; typical range **0.5 to 20** (Daw, 2011)
+- **Critical note**: Learning rate and inverse temperature are often poorly identifiable in standard Q-learning; consider reparameterization or strong priors (Daw, 2011; Wilson & Collins, 2019)
+
+## MCMC Diagnostics
+
+Every Bayesian analysis requires thorough convergence diagnostics. **Never report posterior summaries without first verifying convergence.** See `references/diagnostics-checklist.md` for the full step-by-step protocol.
+
+### Minimum Convergence Criteria
+
+| Diagnostic | Threshold | Interpretation | Source |
+|-----------|-----------|----------------|--------|
+| R-hat (split R-hat) | **< 1.01** | Between-chain vs. within-chain variance; values > 1.01 indicate non-convergence | Vehtari et al., 2021 |
+| Bulk-ESS | **> 400** (100 per chain with 4 chains) | Effective independent draws for posterior mean/median estimation | Vehtari et al., 2021 |
+| Tail-ESS | **> 400** | Effective draws for tail quantiles (credible intervals) | Vehtari et al., 2021 |
+| Divergent transitions | **0** | Any divergences indicate the sampler failed to explore the posterior faithfully | Betancourt, 2017 |
+| E-BFMI | **> 0.3** | Energy Bayesian Fraction of Missing Information; low values indicate poor exploration | Betancourt, 2017 |
+| Tree depth saturation | Rare (<1% of transitions) | Hitting maximum tree depth suggests difficult geometry | Stan User's Guide |
+
+> **Critical domain knowledge**: The older threshold of R-hat < 1.1 is **outdated**. Vehtari et al. (2021) demonstrated that the traditional R-hat can miss convergence failures. Use the rank-normalized split R-hat with a threshold of **1.01** and always report both bulk-ESS and tail-ESS.
+
+### When Diagnostics Fail
+
+See `references/diagnostics-checklist.md` for remediation steps. The most common fixes in cognitive modeling:
+
+1. **Divergent transitions** --> Switch to non-centered parameterization; increase `adapt_delta` to 0.95-0.99
+2. **Low ESS** --> Run longer chains; check for multimodality; reparameterize
+3. **High R-hat** --> Run more iterations; check for label switching in mixture models
+4. **E-BFMI warning** --> Reparameterize; consider reducing model complexity
+
+## Model Comparison
+
+### Information Criteria (Preferred for Most Applications)
+
+| Method | When to Use | Implementation | Source |
+|--------|------------|----------------|--------|
+| **PSIS-LOO-CV** | **Default choice** for comparing predictive accuracy; more robust than WAIC with weak priors or influential observations | `loo` package (R), `az.loo` (Python/ArviZ) | Vehtari et al., 2017 |
+| **WAIC** | Asymptotically equivalent to LOO; acceptable when PSIS diagnostics are clean (all Pareto k < 0.7) | `loo` package (R), `az.waic` (Python/ArviZ) | Watanabe, 2010; Vehtari et al., 2017 |
+| **Bayes factors** | When testing a precise null hypothesis (e.g., parameter = 0); sensitive to prior specification | Bridge sampling, Savage-Dickey density ratio | Kass & Raftery, 1995; Lee & Wagenmakers, 2014, Ch. 7 |
+
+> **Critical domain knowledge**: Prefer LOO-CV over WAIC for cognitive models. Vehtari et al. (2017) showed that PSIS-LOO is more robust in the finite-sample case, especially with weak priors or influential observations common in cognitive data. Always check the Pareto k diagnostic: values > **0.7** indicate unreliable LOO estimates for those observations.
+
+### Interpreting Bayes Factors
+
+| Bayes Factor (BF10) | Evidence Category | Source |
+|---------------------|-------------------|--------|
+| 1 - 3 | Anecdotal / not worth more than a bare mention | Jeffreys, 1961; Lee & Wagenmakers, 2014 |
+| 3 - 10 | Moderate evidence | Jeffreys, 1961; Lee & Wagenmakers, 2014 |
+| 10 - 30 | Strong evidence | Jeffreys, 1961; Lee & Wagenmakers, 2014 |
+| 30 - 100 | Very strong evidence | Jeffreys, 1961; Lee & Wagenmakers, 2014 |
+| > 100 | Extreme / decisive evidence | Jeffreys, 1961; Lee & Wagenmakers, 2014 |
+
+> **Caution**: Bayes factors are highly sensitive to prior specification. A diffuse prior on the alternative hypothesis inflates evidence for the null (the Jeffreys-Lindley paradox). Always conduct a prior sensitivity analysis when reporting Bayes factors (Schad et al., 2021).
+
+### Posterior Predictive Checks
+
+After model comparison, the selected model must demonstrate it can reproduce key features of the observed data:
+
+1. **Simulate data** from the posterior predictive distribution (draw parameters from posterior, then generate synthetic data)
+2. **Compare summary statistics**: mean RT, RT quantiles (0.1, 0.3, 0.5, 0.7, 0.9), accuracy, conditional accuracy functions
+3. **Visual checks**: overlay posterior predictive density on observed data; Q-Q plots; residual distributions
+4. **Quantitative checks**: Posterior predictive p-values for test statistics of interest; values near 0 or 1 indicate misfit (Gelman et al., 2013, Ch. 6)
+
+> **Domain-specific checks**: For RT models, always check the fit to the **full RT distribution** (not just the mean). Cognitive models derive their power from fitting distributional shape -- a model that matches mean RT but misses the right tail is inadequate (Ratcliff & McKoon, 2008).
+
+## Common Pitfalls
+
+### 1. Non-Identifiability
+
+**Problem**: Two or more parameters trade off so that many parameter combinations yield equivalent likelihoods. Common in RL models (learning rate vs. inverse temperature) and DDM (boundary vs. drift rate with few conditions).
+
+**Detection**: Pairwise posterior scatter plots show strong correlations or ridges; marginal posteriors are much wider than expected.
+
+**Fix**: Add conditions that differentially constrain parameters; use informative priors; reparameterize (e.g., the ratio v/a in DDM; Wilson & Collins, 2019).
+
+### 2. Prior Sensitivity
+
+**Problem**: Posterior conclusions change substantially when priors are varied within a reasonable range.
+
+**Detection**: Re-fit with 2-3 alternative prior specifications and compare posteriors (Schad et al., 2021).
+
+**Fix**: Collect more data; use more informative priors justified by previous literature; report sensitivity analysis in the paper.
+
+### 3. Label Switching
+
+**Problem**: In mixture models, MCMC chains swap component labels, creating multimodal marginal posteriors even when the model is well-identified.
+
+**Detection**: Trace plots show "switching" between modes; R-hat is high even with long chains.
+
+**Fix**: Impose ordering constraints (e.g., mu_1 < mu_2); use label-invariant summaries; post-hoc relabeling (Stephens, 2000).
+
+### 4. Improper Posterior Geometry
+
+**Problem**: Funnel-shaped posterior in hierarchical models where the group SD approaches zero, creating an increasingly narrow funnel that the sampler cannot traverse.
+
+**Detection**: Divergent transitions concentrated near low group-SD values; non-centered parameterization is the standard fix (Betancourt & Girolami, 2015).
+
+**Fix**: Non-centered parameterization (see Step 3 in Model Structure Decision Tree above).
+
+### 5. Insufficient Trials Per Participant
+
+**Problem**: With too few trials, individual-level parameters are poorly constrained even in hierarchical models.
+
+**Guideline**: For DDM, minimum **40-60 trials per condition per participant** for stable hierarchical estimation (Wiecki et al., 2013; Ratcliff & Childers, 2015). For simpler models (e.g., binomial SDT), **20-30 trials** may suffice with hierarchical priors (Lee & Wagenmakers, 2014).
+
+**Fix**: If data are already collected, rely more heavily on hierarchical shrinkage and report wide credible intervals honestly.
+
+## Reporting Checklist
+
+When reporting Bayesian cognitive models in a manuscript:
+
+1. **Model specification**: Full generative model with likelihood and all priors (consider a graphical model plate diagram)
+2. **Prior justification**: Why each prior was chosen; cite sources for domain-informed priors
+3. **Prior predictive check**: Confirm priors generate plausible data
+4. **Software and sampler settings**: Package, version, number of chains (minimum **4**; Vehtari et al., 2021), warmup iterations, sampling iterations, adapt_delta
+5. **Convergence diagnostics**: R-hat (< 1.01), bulk-ESS (> 400), tail-ESS (> 400), divergent transitions (0)
+6. **Posterior summaries**: Means/medians, credible intervals (89% or 95% HDI; Kruschke, 2015, Ch. 12), and full posterior distributions where space allows
+7. **Posterior predictive checks**: Visual and/or quantitative evidence that the model reproduces key data features
+8. **Model comparison** (if applicable): LOO-CV/WAIC with standard errors; Bayes factors with prior sensitivity
+9. **Sensitivity analysis**: At least one alternative prior specification with comparison of results
+10. **Code and data availability**: Share Stan/PyMC code and (where possible) data for reproducibility
+
+## Key References
+
+- Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. *arXiv:1701.02434*.
+- Betancourt, M., & Girolami, M. (2015). Hamiltonian Monte Carlo for hierarchical models. In *Current Trends in Bayesian Methodology with Applications*. Chapman and Hall/CRC.
+- Daw, N. D. (2011). Trial-by-trial data analysis using computational models. In *Decision Making, Affect, and Learning*. Oxford University Press.
+- Gabry, J., Simpson, D., Vehtari, A., Betancourt, M., & Gelman, A. (2019). Visualization in Bayesian workflow. *Journal of the Royal Statistical Society: Series A*, 182(2), 389-402.
+- Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. *Bayesian Analysis*, 1(3), 515-534.
+- Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y. S. (2008). A weakly informative default prior distribution for logistic and other regression models. *Annals of Applied Statistics*, 2(4), 1360-1383.
+- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). *Bayesian Data Analysis* (3rd ed.). Chapman and Hall/CRC.
+- Gershman, S. J. (2016). Empirical priors for reinforcement learning models. *Journal of Mathematical Psychology*, 71, 1-6.
+- Kruschke, J. K. (2015). *Doing Bayesian Data Analysis* (2nd ed.). Academic Press.
+- Lee, M. D., & Wagenmakers, E. J. (2014). *Bayesian Cognitive Modeling: A Practical Course*. Cambridge University Press.
+- Schad, D. J., Betancourt, M., & Vasishth, S. (2021). Toward a principled Bayesian workflow in cognitive science. *Psychological Methods*, 26(1), 103-126.
+- Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. *Statistics and Computing*, 27, 1413-1432.
+- Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., & Burkner, P. C. (2021). Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC. *Bayesian Analysis*, 16(2), 667-718.
+- Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the drift-diffusion model in Python. *Frontiers in Neuroinformatics*, 7, 14.
+- Wilson, R. C., & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. *eLife*, 8, e49547.

package/skills/07_Computational_Modeling/bayesian-cognitive-model-builder/references/diagnostics-checklist.md

@@ -0,0 +1,351 @@
+# MCMC Diagnostics and Model Checking Checklist
+
+This reference provides a step-by-step diagnostic protocol for Bayesian cognitive models fitted with MCMC (Stan/PyMC). Every threshold is cited. Follow this checklist in order before reporting any posterior results.
+
+---
+
+## Phase 1: Pre-Fitting Checks
+
+### 1.1 Prior Predictive Check
+
+Before fitting the model to data, verify that the priors produce plausible simulated data.
+
+- [ ] Sample 500-1000 parameter sets from the prior distributions
+- [ ] Simulate datasets from the generative model using each parameter set
+- [ ] Verify simulated data falls within the plausible range for the domain
+- [ ] Document the prior predictive check results
+
+See `prior-selection-guide.md` for the full prior predictive checking procedure and domain-specific plausibility criteria.
+
+### 1.2 Computational Setup
+
+- [ ] Run a minimum of **4 chains** (Vehtari et al., 2021)
+- [ ] Set warmup/burn-in to at least **1000 iterations** per chain (Stan default; adjust upward for complex models)
+- [ ] Set sampling iterations to at least **1000 per chain** (total post-warmup draws >= 4000)
+- [ ] For Stan: set `adapt_delta` to at least **0.8** (default); increase to **0.95-0.99** if divergent transitions occur (Stan Reference Manual)
+- [ ] For Stan: set `max_treedepth` to **10** (default); increase to **12-15** if tree depth saturation warnings appear
+- [ ] Record software version, random seed, and all sampler settings for reproducibility
+
+---
+
+## Phase 2: Convergence Diagnostics
+
+Run these checks on every parameter in the model, including derived quantities and hyperparameters. A single failing parameter means the model has not converged.
+
+### 2.1 R-hat (Split R-hat, Rank-Normalized)
+
+**Threshold: R-hat < 1.01 for all parameters** (Vehtari et al., 2021)
+
+| R-hat Value | Interpretation | Action |
+|-------------|---------------|--------|
+| < 1.01 | Chains have converged | Proceed |
+| 1.01 - 1.05 | Possible convergence issue | Investigate; run longer chains; check trace plots |
+| 1.05 - 1.1 | Likely convergence failure | Do not report results; diagnose the problem |
+| > 1.1 | Definite convergence failure | Do not report results; model needs reparameterization or longer run |
+
+**What R-hat measures**: The ratio of between-chain variance to within-chain variance after splitting each chain in half. The improved rank-normalized version (Vehtari et al., 2021) also detects problems in the tails of the distribution that traditional R-hat misses.
+
+**Common causes of high R-hat**:
+- Insufficient warmup/sampling iterations
+- Multimodality in the posterior (multiple modes that chains get stuck in)
+- Label switching in mixture models
+- Near-non-identifiability creating ridges in the posterior
+- Poor initial values causing chains to explore different regions
+
+**Fixes**:
+1. Run longer chains (double iterations)
+2. Check for multimodality via trace plots
+3. For mixture models, impose ordering constraints to prevent label switching (e.g., mu_1 < mu_2) (Stephens, 2000)
+4. Reparameterize: switch centered to non-centered (or vice versa)
+5. Use more informative priors to reduce posterior ambiguity
+
+### 2.2 Effective Sample Size (ESS)
+
+**Threshold: Bulk-ESS > 400 AND Tail-ESS > 400** (Vehtari et al., 2021)
+
+The 400 threshold assumes 4 chains (approximately 100 effective samples per chain).
+
+| ESS Type | What It Measures | Minimum | Source |
+|----------|-----------------|---------|--------|
+| **Bulk-ESS** | Effective samples for estimating the posterior mean/median (center of distribution) | **> 400** | Vehtari et al., 2021 |
+| **Tail-ESS** | Effective samples for estimating tail quantiles (credible interval endpoints) | **> 400** | Vehtari et al., 2021 |
+
+**Why two ESS measures?** A chain can mix well in the bulk of the distribution but poorly in the tails, or vice versa. Reporting only one ESS can mask problems with credible interval estimation. Tail-ESS is the minimum of the ESS for the 5th and 95th percentile quantiles (Vehtari et al., 2021).
+
+**ESS vs. total draws ratio**: If ESS/total_draws < 0.1, there is substantial autocorrelation. This is not necessarily a problem if absolute ESS exceeds 400, but very low ratios (< 0.01) suggest the sampler is struggling.
+
+**Common causes of low ESS**:
+- High autocorrelation from poor parameterization (centered hierarchical when non-centered is needed)
+- Near-non-identifiability creating slow exploration
+- Multimodality causing chains to remain in modes for extended periods
+- Step size too large or too small (diagnosed via tree depth)
+
+**Fixes**:
+1. Reparameterize (non-centered parameterization for hierarchical models)
+2. Run more iterations
+3. Thin chains as a last resort (increases ESS per stored sample but wastes computation)
+4. For Stan: check that `adapt_delta` is not too high (> 0.99 slows down sampling without benefit for well-specified models)
+
+### 2.3 Divergent Transitions
+
+**Threshold: 0 divergent transitions** (Betancourt, 2017)
+
+- [ ] Check for divergent transitions in sampler output
+- [ ] If any divergences occurred, do NOT trust the posterior
+
+**What divergent transitions mean**: The Hamiltonian Monte Carlo sampler encountered regions of the posterior with very high curvature that it could not faithfully traverse. This means the posterior samples may not represent the true posterior distribution (Betancourt, 2017).
+
+| Number of Divergences | Interpretation | Action |
+|-----------------------|---------------|--------|
+| 0 | No detected geometry problems | Proceed |
+| 1-10 | Mild geometry issue; posterior may be biased in problematic region | Investigate; try remediation |
+| 10-100 | Serious geometry problem | Do not trust results; must fix |
+| > 100 | Severe model/parameterization problem | Rethink model specification |
+
+**Common causes**:
+- **Funnel geometry** in centered hierarchical models (the most common cause in cognitive modeling)
+- Highly correlated parameters creating narrow valleys
+- Priors that conflict with the likelihood, creating sharp posterior ridges
+- Boundaries or constraints creating discontinuities
+
+**Fixes** (in order of preference):
+1. **Non-centered parameterization**: Transform `theta_j ~ Normal(mu, sigma)` to `theta_j = mu + sigma * eta_j` where `eta_j ~ Normal(0, 1)` (Betancourt & Girolami, 2015)
+2. **Increase adapt_delta**: Set to 0.95, then 0.99 if needed. This reduces the step size, allowing the sampler to navigate difficult geometry at the cost of slower sampling (Stan Reference Manual)
+3. **Reparameterize correlated parameters**: Use Cholesky decompositions for multivariate priors; consider QR decomposition for regression predictors
+4. **Tighten priors**: More informative priors can smooth posterior geometry
+5. **Simplify the model**: If divergences persist after all fixes, the model may be too complex for the data
+
+**Where divergences occur**: In Stan, you can extract which iterations diverged and plot the parameter values at those iterations. If divergences cluster in a specific region of parameter space (e.g., near sigma = 0 in a hierarchical model), this confirms the funnel geometry diagnosis.
+
+### 2.4 Energy Diagnostics (E-BFMI)
+
+**Threshold: E-BFMI > 0.3** (Betancourt, 2017)
+
+The Energy Bayesian Fraction of Missing Information measures how well the HMC momentum can explore the posterior.
+
+| E-BFMI Value | Interpretation | Action |
+|--------------|---------------|--------|
+| > 0.3 | Adequate exploration | Proceed |
+| 0.2 - 0.3 | Marginal; may miss some posterior mass | Investigate; consider reparameterization |
+| < 0.2 | Poor exploration; posterior may be biased | Reparameterize; increase adapt_delta |
+
+**Common causes of low E-BFMI**:
+- Heavy-tailed posterior that HMC cannot efficiently explore
+- Posterior with very different scales across parameters
+- Funnel geometry (often co-occurs with divergent transitions)
+
+**Fix**: Reparameterize to make the posterior geometry more uniform; standardize parameter scales.
+
+### 2.5 Tree Depth Saturation
+
+**Threshold: < 1% of transitions hitting max_treedepth** (Stan Reference Manual)
+
+When the NUTS sampler hits the maximum tree depth, it terminates the trajectory early, potentially reducing sampling efficiency.
+
+- [ ] Check fraction of transitions that hit max_treedepth
+- [ ] If > 1% saturate, increase max_treedepth by 2-3 (e.g., from 10 to 12 or 13)
+
+**Note**: Tree depth saturation does not bias the posterior (unlike divergences), but it reduces ESS per unit time. The only cost of increasing max_treedepth is computational time (each additional depth level doubles the maximum number of leapfrog steps).
+
+---
+
+## Phase 3: Visual Diagnostics
+
+Even when numerical diagnostics pass, visual inspection can reveal subtle problems.
+
+### 3.1 Trace Plots
+
+- [ ] All chains should be well-mixed (indistinguishable "fuzzy caterpillar" appearance)
+- [ ] No chain should be stuck in a different region from others
+- [ ] No long-term trends (drift) in any chain
+- [ ] No periodic patterns (indicates strong autocorrelation)
+
+**What to look for**:
+- **Good mixing**: Chains overlap completely and look like noise
+- **Poor mixing**: Chains show slow wandering, long excursions, or different levels
+- **Label switching**: Chains periodically swap between two or more values (common in mixture models)
+- **Stationarity violation**: Chain mean or variance changes over time
+
+### 3.2 Rank Plots (Recommended)
+
+- [ ] Generate rank histogram plots (available in ArviZ and bayesplot)
+- [ ] Uniform rank histograms indicate good mixing across chains
+- [ ] Non-uniform patterns indicate convergence problems
+
+Rank plots are more informative than traditional trace plots for detecting subtle mixing problems (Vehtari et al., 2021).
+
+### 3.3 Pairwise Posterior Scatter Plots
+
+- [ ] Check for strong posterior correlations between parameters
+- [ ] Look for banana-shaped or ridge-shaped posteriors (indicate non-identifiability)
+- [ ] Verify that no parameter pair shows a perfect linear trade-off
+
+**Typical problematic correlations in cognitive models**:
+- DDM: drift rate (v) and boundary separation (a) often negatively correlated when few conditions constrain them
+- RL: learning rate (alpha) and inverse temperature (beta) are classically collinear (Daw, 2011)
+- SDT: d' and c can correlate when hit/false alarm rates approach boundaries
+
+---
+
+## Phase 4: Posterior Predictive Checks
+
+After confirming convergence, verify that the model can reproduce the observed data.
+
+### 4.1 Basic Posterior Predictive Protocol
+
+1. **Draw S = 500-1000 samples** from the posterior distribution
+2. **For each posterior sample, simulate a complete dataset** with the same structure as the observed data
+3. **Compute summary statistics** on each simulated dataset (same statistics you would report for the real data)
+4. **Compare simulated vs. observed statistics** using overlaid distributions or difference plots
+
+### 4.2 Domain-Specific Posterior Predictive Checks
+
+#### For Reaction Time Models (DDM, LBA, etc.)
+
+- [ ] **Mean RT by condition**: Simulated mean RT should bracket observed mean RT
+- [ ] **RT quantiles (0.1, 0.3, 0.5, 0.7, 0.9) by condition**: The full distributional shape must be captured, not just the mean (Ratcliff & McKoon, 2008)
+- [ ] **Accuracy by condition**: Simulated accuracy should bracket observed accuracy
+- [ ] **Conditional Accuracy Functions (CAFs)**: Accuracy as a function of RT quantile bin; captures speed-accuracy tradeoff dynamics
+- [ ] **RT distribution shape**: Overlay posterior predictive density on observed RT histogram; check right tail fit
+- [ ] **Error RT distribution**: Often neglected; error RTs should be faster or slower than correct RTs depending on the model (Ratcliff & McKoon, 2008)
+
+#### For Learning Models (RL, etc.)
+
+- [ ] **Learning curves**: Simulated performance over trial blocks should match observed learning trajectory
+- [ ] **Choice proportions by block**: Should bracket observed choice frequencies
+- [ ] **Win-stay / lose-shift rates**: Captures local choice dynamics that aggregate statistics miss (Wilson & Collins, 2019)
+- [ ] **Individual differences**: Simulated between-participant variability should match observed variability
+
+#### For Signal Detection Models
+
+- [ ] **Hit rate and false alarm rate**: Should bracket observed rates
+- [ ] **ROC curve shape** (if multiple criteria/confidence levels available): Simulated ROC should match observed ROC
+- [ ] **z-ROC linearity**: If assuming equal variance, z-ROC should be linear with slope near 1; deviations suggest unequal variance model (Macmillan & Creelman, 2005)
+
+#### For Multinomial Processing Tree Models
+
+- [ ] **Category response frequencies**: Simulated response frequencies should bracket observed frequencies for each response category
+- [ ] **Pearson chi-square or G-squared**: Posterior predictive p-value for model fit (Klauer, 2010)
+
+### 4.3 Quantitative Posterior Predictive Assessment
+
+**Posterior predictive p-value**: For a test statistic T(y), compute:
+
+```
+p_B = Pr(T(y_rep) >= T(y_obs) | y_obs)
+```
+
+where y_rep is data simulated from the posterior predictive distribution (Gelman et al., 2013, Ch. 6).
+
+| p_B Value | Interpretation |
+|-----------|---------------|
+| 0.05 - 0.95 | No evidence of misfit for this statistic |
+| < 0.05 or > 0.95 | Model systematically under- or over-predicts this feature |
+| < 0.01 or > 0.99 | Strong evidence of misfit; model is missing an important data feature |
+
+**Caution**: Posterior predictive p-values are conservative (they tend toward 0.5) because the same data are used for fitting and checking. A p-value near 0.5 does not guarantee good fit; it only means the check did not detect misfit (Gelman et al., 2013, Ch. 6).
+
+**Choose meaningful test statistics**: Select statistics that capture features your model claims to explain. For DDMs, check RT quantiles, not just the mean. For RL models, check block-by-block learning, not just overall accuracy.
+
+---
+
+## Phase 5: Sensitivity Analysis
+
+### 5.1 Prior Sensitivity
+
+- [ ] Re-fit the model with **at least 2 alternative prior specifications**:
+ - One with wider (more diffuse) priors
+ - One with narrower (more informative) priors
+- [ ] Compare posterior means, credible intervals, and qualitative conclusions
+- [ ] If conclusions change substantially, report this and collect more data or justify your preferred prior (Schad et al., 2021)
+
+### 5.2 What to Vary
+
+| What to Vary | How | Concern If Results Change |
+|-------------|-----|--------------------------|
+| Prior SD on group means | Double and halve | Data may not be informative enough for this parameter |
+| Prior on group-level SD | Try Half-Normal(0,0.5) vs Half-Normal(0,2) | Between-participant variance is poorly identified |
+| Prior family | Try Normal vs. Student-t(3) for location parameters | Sensitivity to tail behavior |
+| LKJ eta parameter | Try eta=1 vs eta=2 vs eta=4 | Correlation estimates driven by prior |
+| Boundary-avoiding priors | Try Gamma vs. Lognormal for positive parameters | Choice of family matters for boundary behavior |
+
+### 5.3 Reporting Sensitivity Results
+
+In your manuscript, include a brief sensitivity analysis section:
+
+- State which priors were varied and to what alternatives
+- Report whether the main conclusions (direction and magnitude of key effects) are robust
+- If conclusions are sensitive, acknowledge this as a limitation and specify what additional data or constraints could resolve the ambiguity
+
+---
+
+## Phase 6: Summary Diagnostic Table Template
+
+For your manuscript or supplementary materials, report a table like:
+
+| Parameter | R-hat | Bulk-ESS | Tail-ESS | Prior | Posterior Mean [95% CI] |
+|-----------|-------|----------|----------|-------|------------------------|
+| mu_v | 1.00 | 2847 | 2104 | Normal(0, 2) | 1.52 [0.98, 2.11] |
+| sigma_v | 1.00 | 1523 | 1205 | Half-Normal(0, 1) | 0.61 [0.38, 0.92] |
+| mu_a | 1.00 | 3102 | 2456 | Half-Normal(0, 1) | 1.14 [0.95, 1.35] |
+| ... | ... | ... | ... | ... | ... |
+
+Additional information to report:
+- Number of divergent transitions: 0
+- E-BFMI: all chains > 0.3
+- Software: Stan 2.x / PyMC 5.x (specify version)
+- Chains: 4, Warmup: 1000, Sampling: 2000, adapt_delta: 0.95
+
+---
+
+## Quick Reference: Diagnostic Decision Flowchart
+
+```
+1. Any divergent transitions?
+ YES --> Fix parameterization (non-centered) or increase adapt_delta
+ NO --> Continue
+
+2. R-hat > 1.01 for any parameter?
+ YES --> Run longer; check for multimodality; reparameterize
+ NO --> Continue
+
+3. Bulk-ESS or Tail-ESS < 400 for any parameter?
+ YES --> Run longer; reparameterize; check autocorrelation
+ NO --> Continue
+
+4. E-BFMI < 0.3?
+ YES --> Reparameterize; standardize parameter scales
+ NO --> Continue
+
+5. Tree depth saturation > 1%?
+ YES --> Increase max_treedepth
+ NO --> Continue
+
+6. Visual checks pass (trace plots, rank plots, pair plots)?
+ YES --> Proceed to posterior predictive checks
+ NO --> Investigate identified issues
+
+7. Posterior predictive checks pass?
+ YES --> Proceed to sensitivity analysis and reporting
+ NO --> Revise model; consider missing mechanisms or misspecified priors
+
+8. Sensitivity analysis: conclusions robust to alternative priors?
+ YES --> Report results with confidence
+ NO --> Report sensitivity; acknowledge limitations; consider more data
+```
+
+---
+
+## References
+
+- Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. *arXiv:1701.02434*.
+- Betancourt, M., & Girolami, M. (2015). Hamiltonian Monte Carlo for hierarchical models. In *Current Trends in Bayesian Methodology with Applications*. Chapman and Hall/CRC.
+- Daw, N. D. (2011). Trial-by-trial data analysis using computational models. In *Decision Making, Affect, and Learning*. Oxford University Press.
+- Gelman, A., et al. (2013). *Bayesian Data Analysis* (3rd ed.). Chapman and Hall/CRC.
+- Klauer, K. C. (2010). Hierarchical multinomial processing tree models: A latent-trait approach. *Psychometrika*, 75(1), 70-98.
+- Macmillan, N. A., & Creelman, C. D. (2005). *Detection Theory: A User's Guide* (2nd ed.). Lawrence Erlbaum Associates.
+- Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. *Neural Computation*, 20(4), 873-922.
+- Schad, D. J., Betancourt, M., & Vasishth, S. (2021). Toward a principled Bayesian workflow in cognitive science. *Psychological Methods*, 26(1), 103-126.
+- Stephens, M. (2000). Dealing with label switching in mixture models. *Journal of the Royal Statistical Society: Series B*, 62(4), 795-809.
+- Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., & Burkner, P. C. (2021). Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC. *Bayesian Analysis*, 16(2), 667-718.
+- Wilson, R. C., & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. *eLife*, 8, e49547.