@yibeichan/claude-skills 1.0.2

package/skills/dicom2fmriprep/references/babs-details.md

@@ -0,0 +1,407 @@
+# BABS Detailed Reference
+
+## Table of Contents
+- [Full YAML Configuration Schema](#full-yaml-configuration-schema)
+- [Advanced Workflows](#advanced-workflows)
+- [CLI Command Reference](#cli-command-reference)
+- [Consuming Results](#consuming-results)
+- [Troubleshooting](#troubleshooting)
+
+## Full YAML Configuration Schema
+
+### `input_datasets`
+
+```yaml
+input_datasets:
+    DATASET_NAME:                    # User-chosen key (e.g., "BIDS", "FreeSurfer")
+        required_files:              # Glob patterns — subjects missing these are skipped
+            - "func/*_bold.nii*"
+            - "anat/*_T1w.nii*"
+        is_zipped: false             # true for zipped derivative inputs
+        origin_url: "/path/to/datalad/dataset"
+        path_in_babs: inputs/data/BIDS
+        # For zipped inputs only:
+        unzipped_path_containing_subject_dirs: "freesurfer"
+```
+
+The **first listed dataset** becomes the positional BIDS input argument to the container.
+
+### `cluster_resources`
+
+| Key | SLURM Translation | Example |
+|-----|-------------------|---------|
+| `interpreting_shell` | `#!VALUE` | `/bin/bash` or `/bin/bash -l` |
+| `hard_memory_limit` | `#SBATCH --mem=VALUE` | `32G` |
+| `temporary_disk_space` | `#SBATCH --tmp=VALUE` | `200G` |
+| `number_of_cpus` | `#SBATCH --cpus-per-task=VALUE` | `"6"` |
+| `hard_runtime_limit` | `#SBATCH --time=VALUE` | `"24:00:00"` |
+| `customized_text` | Copied verbatim | See below |
+
+```yaml
+cluster_resources:
+    interpreting_shell: "/bin/bash"
+    hard_memory_limit: 32G
+    temporary_disk_space: 200G
+    number_of_cpus: "6"
+    hard_runtime_limit: "24:00:00"
+    customized_text: |
+        #SBATCH -p all
+        #SBATCH --nodes=1
+        #SBATCH --ntasks=1
+        #SBATCH --propagate=NONE
+```
+
+### `script_preamble`
+
+Bash commands run before the container. Use `|` for multiline. Do NOT quote commands.
+
+```yaml
+script_preamble: |
+    source "${CONDA_PREFIX}"/bin/activate babs
+    module load singularity
+    export TEMPLATEFLOW_HOME=/shared/templateflow
+```
+
+### `job_compute_space`
+
+Ephemeral scratch directory. Should auto-clean after job finishes.
+
+```yaml
+job_compute_space: "${TMPDIR}"
+# Or a specific path:
+job_compute_space: "/scratch/${USER}/babs_tmp"
+```
+
+### `singularity_args`
+
+```yaml
+singularity_args:
+    - --cleanenv
+    - --writable-tmpfs
+    # Add --nv for GPU workloads
+```
+
+### `bids_app_args`
+
+```yaml
+bids_app_args:
+    $SUBJECT_SELECTION_FLAG: "--participant-label"  # BABS special variable
+    -w: "$BABS_TMPDIR"          # BABS placeholder for temp workspace
+    --fs-license-file: "/path/to/license.txt"  # Auto-bound into container
+    --output-spaces: "MNI152NLin2009cAsym:res-2"
+    --force-bbr: ""             # Flag without value (empty string, Null, or NULL)
+    --cifti-output: "91k"
+    --n_cpus: "6"               # Quote numeric values
+    --mem-mb: "30000"
+    --skip-bids-validation: ""
+    --notrack: ""
+    -v: '-v'                    # Double verbose: -v -v
+```
+
+**Rules:**
+- Do NOT include `--participant-label` or `--bids-filter-file` directly — BABS handles these via `$SUBJECT_SELECTION_FLAG`
+- Use `$BABS_TMPDIR` for working directory (`-w`)
+- Use `$SLURM_CPUS_PER_TASK` as env var in script_preamble if needed
+- BABS auto-adds `--bids-filter-file` for fMRIPrep with multi-session data
+- FreeSurfer license path is auto-bound into container at `/SGLR/FREESURFER_HOME/license.txt`
+- BABS handles `$TEMPLATEFLOW_HOME` if the env var is set when running `babs init`
+
+### `zip_foldernames`
+
+Controls how outputs are zipped per subject. Version string becomes part of the zip filename.
+
+**Legacy output layout** (fMRIPrep < 21.0):
+```yaml
+zip_foldernames:
+    fmriprep: "24-1-1"       # → sub-XX_fmriprep-24-1-1.zip
+    freesurfer: "24-1-1"     # → sub-XX_freesurfer-24-1-1.zip
+```
+
+**BIDS output layout** (fMRIPrep >= 21.0, with `all_results_in_one_zip`):
+```yaml
+all_results_in_one_zip: true
+zip_foldernames:
+    fmriprep_anat: "24-1-1"  # Single zip with everything
+```
+
+When `all_results_in_one_zip: true`, only ONE foldername is allowed.
+
+### `imported_files` (optional)
+
+Copy external files into the DataLad dataset (useful for custom config files):
+
+```yaml
+imported_files:
+    - original_path: "/path/to/custom_config.yaml"
+      analysis_path: "code/custom_config.yaml"
+```
+
+### `alert_log_messages` (optional)
+
+Patterns to flag in failed job logs:
+
+```yaml
+alert_log_messages:
+    stdout:
+        - "fMRIPrep failed"
+        - "Cannot allocate memory"
+        - "Excessive topologic defect encountered"
+        - "mris_curvature_stats: Could not open file"
+        - "Numerical result out of range"
+```
+
+## Advanced Workflows
+
+### Anat-Only + Ingressed FreeSurfer (Two-Stage Pipeline)
+
+For large datasets, split fMRIPrep into two stages to optimize resources:
+
+**Stage 1: Anat-only** (long walltime, low CPU)
+
+```yaml
+input_datasets:
+    BIDS:
+        required_files:
+            - "anat/*_T1w.nii*"
+        is_zipped: false
+        origin_url: "/path/to/bids"
+        path_in_babs: inputs/data/BIDS
+
+cluster_resources:
+    interpreting_shell: "/bin/bash"
+    hard_memory_limit: 12G
+    number_of_cpus: "2"
+    hard_runtime_limit: "24:00:00"
+    customized_text: |
+        #SBATCH -p long
+        #SBATCH --nodes=1
+
+bids_app_args:
+    $SUBJECT_SELECTION_FLAG: "--participant-label"
+    -w: "$BABS_TMPDIR"
+    --anat-only: ""
+    --fs-license-file: "/path/to/license.txt"
+    --output-spaces: "MNI152NLin2009cAsym:res-2"
+    --notrack: ""
+
+all_results_in_one_zip: true
+zip_foldernames:
+    fmriprep_anat: "24-1-1"
+```
+
+**Stage 2: Func with ingressed FreeSurfer** (shorter walltime, more CPU)
+
+```yaml
+input_datasets:
+    BIDS:
+        required_files:
+            - "func/*_bold.nii*"
+            - "anat/*_T1w.nii*"
+        is_zipped: false
+        origin_url: "/path/to/bids"
+        path_in_babs: inputs/data/BIDS
+    FreeSurfer:
+        required_files:
+            - "*fmriprep_anat*.zip"
+        is_zipped: true
+        origin_url: "/path/to/stage1_babs_project/output_ria"
+        unzipped_path_containing_subject_dirs: "fmriprep_anat"
+        path_in_babs: inputs/data/freesurfer
+
+cluster_resources:
+    interpreting_shell: "/bin/bash"
+    hard_memory_limit: 30G
+    number_of_cpus: "6"
+    hard_runtime_limit: "8:00:00"
+    customized_text: |
+        #SBATCH -p normal
+        #SBATCH --nodes=1
+
+bids_app_args:
+    $SUBJECT_SELECTION_FLAG: "--participant-label"
+    -w: "$BABS_TMPDIR"
+    --fs-subjects-dir: "inputs/data/freesurfer/freesurfer"
+    --fs-license-file: "/path/to/license.txt"
+    --output-spaces: "MNI152NLin2009cAsym:res-2"
+    --force-bbr: ""
+    --n_cpus: "6"
+    --mem-mb: "28000"
+    --notrack: ""
+
+all_results_in_one_zip: true
+zip_foldernames:
+    fmriprep_func: "24-1-1"
+```
+
+### Multi-Session Processing
+
+For multi-session data, use `--processing_level session` in `babs init`. BABS will:
+- Process each session independently
+- Auto-generate `--bids-filter-file` for fMRIPrep
+- Create per-session output zips
+
+```bash
+babs init \
+    --container_ds /path/to/fmriprep-container \
+    --container_name fmriprep-24-1-1 \
+    --container_config config.yaml \
+    --processing_level session \
+    --queue slurm \
+    /path/to/my_babs_project
+```
+
+### Filtering Subjects
+
+```bash
+# Via CSV file during init
+babs init ... --list_sub_file subjects.csv
+
+# CSV format (subject-level):
+# sub_id
+# sub-01
+# sub-02
+
+# CSV format (session-level):
+# sub_id,ses_id
+# sub-01,ses-01
+# sub-01,ses-02
+
+# Via --select during submit
+babs submit --select sub-01 ses-01 --select sub-02 ses-01
+
+# Via inclusion file during submit
+babs submit --inclusion-file my_subjects.csv
+```
+
+### Throttling Jobs
+
+Limit simultaneous SLURM array tasks to avoid overwhelming the cluster:
+
+```bash
+babs init ... --throttle 10  # Max 10 concurrent jobs
+```
+
+## CLI Command Reference
+
+### `babs init`
+```bash
+babs init \
+    --container_ds PATH \
+    --container_name NAME \
+    --container_config YAML \
+    --processing_level {subject,session} \
+    --queue slurm \
+    [--list_sub_file CSV] \
+    [--keep_if_failed] \
+    [--throttle N] \
+    PROJECT_ROOT
+```
+
+### `babs check-setup`
+```bash
+babs check-setup [PROJECT_ROOT] [--job-test]
+```
+Always run with `--job-test` before bulk submission.
+
+### `babs submit`
+```bash
+babs submit [PROJECT_ROOT] \
+    [--count N] \
+    [--select SUB [SES]] \
+    [--inclusion-file CSV] \
+    [--skip-running-jobs]
+```
+`--count`, `--select`, and `--inclusion-file` are mutually exclusive.
+
+**WARNING**: Never kill `babs submit` while running.
+
+### `babs status`
+```bash
+babs status [PROJECT_ROOT]
+```
+
+### `babs merge`
+```bash
+babs merge [PROJECT_ROOT] [--chunk-size 2000]
+```
+If merge fails, manually remove `merge_ds/` before retrying.
+
+### `babs sync-code`
+```bash
+babs sync-code [PROJECT_ROOT] [-m "message"]
+```
+
+### `babs update-input-data`
+```bash
+babs update-input-data [PROJECT_ROOT] [--dataset-name BIDS]
+```
+
+## Consuming Results
+
+After `babs merge`:
+
+```bash
+# Clone the output dataset
+datalad clone \
+    ria+file:///absolute/path/to/my_babs_project/output_ria#~data \
+    my_outputs
+
+cd my_outputs
+
+# Get specific subject's results
+datalad get sub-01_fmriprep-24-1-1.zip
+unzip sub-01_fmriprep-24-1-1.zip
+
+# Get all results
+datalad get .
+```
+
+### Project Structure After Init
+
+```
+my_babs_project/
+├── analysis/
+│   ├── code/
+│   │   ├── participant_job.sh      # Generated SLURM script
+│   │   └── processing_inclusion.csv
+│   ├── inputs/
+│   │   └── data/
+│   │       └── BIDS/              # Cloned input dataset
+│   └── containers/
+│       └── fmriprep-24-1-1.sif
+└── output_ria/                     # RIA store for results
+```
+
+## Troubleshooting
+
+### `babs init` fails
+- Check all paths are absolute
+- Verify container DataLad dataset was created correctly
+- Use `--keep_if_failed` and run `babs check-setup` to diagnose
+
+### `babs check-setup --job-test` fails
+- Check `script_preamble` — are modules available?
+- Verify Singularity/Apptainer is loadable
+- Check SLURM partition exists (`sinfo -s`)
+- Check FreeSurfer license path
+- Look at the job log in `analysis/logs/`
+
+### Jobs fail immediately
+- Check `cluster_resources` — enough memory/time?
+- Verify `job_compute_space` exists and is writable
+- Check `singularity_args` — `--cleanenv` is usually needed
+
+### `babs merge` fails
+- Remove `merge_ds/` directory and retry
+- Check that successful jobs actually completed (not just submitted)
+- Try with `--chunk-size 500` for very large datasets
+
+### DataLad issues
+```bash
+# If input dataset isn't a DataLad dataset:
+cd /path/to/bids
+datalad create -f -D "BIDS dataset" .
+
+# If container dataset has issues:
+datalad containers-list  # verify container is registered
+```

package/skills/dicom2fmriprep/references/fmriprep-details.md

@@ -0,0 +1,250 @@
+# fMRIPrep Detailed Reference
+
+## Table of Contents
+- [Complete Flag Reference](#complete-flag-reference)
+- [Output Structure](#output-structure)
+- [Confounds Reference](#confounds-reference)
+- [SLURM Script Template](#slurm-script-template)
+- [Troubleshooting](#troubleshooting)
+
+## Complete Flag Reference
+
+### Input/Output
+| Flag | Description |
+|------|-------------|
+| `bids_dir` | Positional: path to BIDS dataset |
+| `output_dir` | Positional: path for derivatives |
+| `analysis_level` | `participant` (subject-level) |
+| `--participant-label` | Subject IDs without `sub-` prefix |
+| `--task-id` | Process only specific task |
+| `--bids-filter-file` | JSON for custom BIDS input filtering |
+| `-w, --work-dir` | Working directory (keep for crash recovery) |
+| `--skip-bids-validation` | Skip BIDS validation (use only after manual validation) |
+
+### Output Spaces
+| Flag | Description |
+|------|-------------|
+| `--output-spaces` | Space-delimited list of output spaces |
+
+Common spaces:
+- `MNI152NLin2009cAsym` — Default volumetric template (most common)
+- `MNI152NLin2009cAsym:res-2` — Same, at 2mm resolution
+- `MNI152NLin6Asym:res-2` — FSL's MNI (required for ICA-AROMA post-processing)
+- `fsaverage` / `fsaverage5` / `fsaverage6` — FreeSurfer surface spaces
+- `fsnative` — Subject's native surface
+- `T1w` — Subject's native anatomical space
+
+### FreeSurfer
+| Flag | Description |
+|------|-------------|
+| `--fs-license-file` | Path to FreeSurfer license.txt |
+| `--fs-no-reconall` | Skip FreeSurfer surface reconstruction (saves hours) |
+| `--fs-subjects-dir` | Reuse existing FreeSurfer recon-all output |
+
+### Distortion Correction
+| Flag | Description |
+|------|-------------|
+| `--ignore fieldmaps` | Skip fieldmap-based SDC |
+| `--use-syn-sdc warn` | Enable fieldmap-less SyN SDC as fallback |
+| `--force-syn-sdc` | Force SyN SDC even with fieldmaps |
+
+### Skull Stripping
+| Flag | Description |
+|------|-------------|
+| `--skull-strip-template` | Template for brain extraction (default: `OASIS30ANTs`) |
+| `--skull-strip-t1w` | `auto`, `skip`, or `force` (default: `force`) |
+| `--skull-strip-fixed-seed` | Reproducible skull stripping |
+
+### Resource Control
+| Flag | Description |
+|------|-------------|
+| `--nthreads` / `--nprocs` / `--n-cpus` | Max total threads |
+| `--omp-nthreads` | Max threads per process |
+| `--mem-mb` / `--mem` | Memory limit in MB |
+| `--low-mem` | Trade disk I/O for lower memory usage |
+
+### Quality & Scans
+| Flag | Description |
+|------|-------------|
+| `--dummy-scans N` | Override auto non-steady-state detection |
+| `--fd-spike-threshold` | FD threshold for motion outliers (default: 0.5mm) |
+| `--dvars-spike-threshold` | DVARS outlier threshold (default: 1.5) |
+| `--bold2anat-dof` | BOLD-to-T1w registration DOF: 6, 9, or 12 (default: 6) |
+
+### Other
+| Flag | Description |
+|------|-------------|
+| `--cifti-output 91k` | Output CIFTI dense timeseries |
+| `--anat-only` | Only anatomical workflows |
+| `--random-seed N` | For reproducibility |
+| `--notrack` | Disable usage tracking |
+| `--force-bbr` | Force boundary-based registration |
+| `--ignore slicetiming` | Skip slice timing correction |
+
+**Note:** `--use-aroma` was removed in fMRIPrep 23.1.0. Use [fmripost-aroma](https://github.com/nipreps/fmripost-aroma) instead.
+
+## Output Structure
+
+```
+<output_dir>/
+├── dataset_description.json
+├── sub-<label>.html                    # Visual QC report (open in browser!)
+├── sub-<label>/
+│   ├── anat/
+│   │   ├── *_desc-preproc_T1w.nii.gz          # Preprocessed T1w
+│   │   ├── *_desc-brain_mask.nii.gz            # Brain mask
+│   │   ├── *_dseg.nii.gz                       # Tissue segmentation
+│   │   ├── *_label-{CSF,GM,WM}_probseg.nii.gz # Probability maps
+│   │   ├── *_from-MNI*_to-T1w_xfm.h5          # MNI→native transform
+│   │   ├── *_from-T1w_to-MNI*_xfm.h5          # Native→MNI transform
+│   │   └── *_hemi-{L,R}_{pial,white,midthickness}.surf.gii  # Surfaces
+│   └── func/
+│       ├── *_space-<space>_desc-preproc_bold.nii.gz  # Preprocessed BOLD
+│       ├── *_space-<space>_desc-brain_mask.nii.gz     # BOLD brain mask
+│       ├── *_desc-confounds_timeseries.tsv            # Confounds
+│       ├── *_desc-confounds_timeseries.json           # Confounds metadata
+│       ├── *_from-boldref_to-T1w_xfm.txt             # BOLD→T1w transform
+│       ├── *_bold.dtseries.nii                        # CIFTI (if --cifti-output)
+│       └── *_space-T1w_desc-aparcaseg_dseg.nii.gz    # Parcellation in BOLD space
+└── sourcedata/
+    └── freesurfer/                     # FreeSurfer recon-all output
+        └── sub-<label>/
+```
+
+## Confounds Reference
+
+The `*_desc-confounds_timeseries.tsv` contains:
+
+### Motion Parameters
+- `trans_x`, `trans_y`, `trans_z` — Translation (mm)
+- `rot_x`, `rot_y`, `rot_z` — Rotation (radians)
+- Each has `_derivative1`, `_power2`, `_derivative1_power2` variants (24-parameter expansion)
+
+### Global Signals
+- `csf`, `white_matter`, `global_signal` (+ derivative/power variants)
+
+### Quality Metrics
+- `framewise_displacement` — Head motion summary (mm)
+- `rmsd` — Root mean squared deviation
+- `dvars`, `std_dvars` — Intensity change metrics
+
+### Noise Components
+- `a_comp_cor_XX` — Anatomical CompCor components
+- `t_comp_cor_XX` — Temporal CompCor components
+- `cosine_XX` — High-pass filter (DCT basis set)
+
+### Outlier Indicators
+- `non_steady_state_outlier_XX` — Initial volume flags
+- `motion_outlier_XX` — Spike regressors for high-motion volumes
+
+### Recommended Confound Strategies
+- **Minimal**: 6 motion parameters + FD
+- **Standard**: 24 motion parameters + aCompCor (top 5) + cosines
+- **Aggressive**: 24 motion + aCompCor + spike regressors + global signal
+
+## SLURM Script Template
+
+```bash
+#!/bin/bash
+#SBATCH --job-name=fmriprep
+#SBATCH --time=24:00:00
+#SBATCH --cpus-per-task=8
+#SBATCH --mem=32G
+#SBATCH --tmp=200G
+#SBATCH --output=log/fmriprep-%A-%a.out
+#SBATCH --error=log/fmriprep-%A-%a.err
+
+# ---- Configuration ----
+BIDS_DIR="/path/to/bids"
+OUTPUT_DIR="/path/to/output"
+WORK_DIR="/tmp/fmriprep_work_${SLURM_ARRAY_TASK_ID}"
+FMRIPREP_SIF="/path/to/fmriprep-24.1.1.sif"
+TEMPLATEFLOW_HOME="$HOME/.cache/templateflow"
+FS_LICENSE="$HOME/.freesurfer.txt"
+
+# ---- Setup ----
+mkdir -p ${WORK_DIR} ${OUTPUT_DIR} log
+
+export SINGULARITYENV_FS_LICENSE=${FS_LICENSE}
+export SINGULARITYENV_TEMPLATEFLOW_HOME="/templateflow"
+
+# Get subject ID from participants.tsv using array index
+SUBJECT=$(sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" \
+    ${BIDS_DIR}/participants.tsv)
+
+echo "Processing sub-${SUBJECT} on $(hostname) at $(date)"
+
+# ---- Run fMRIPrep ----
+singularity run --cleanenv \
+    -B ${BIDS_DIR}:/data:ro \
+    -B ${OUTPUT_DIR}:/out \
+    -B ${WORK_DIR}:/work \
+    -B ${TEMPLATEFLOW_HOME}:/templateflow \
+    ${FMRIPREP_SIF} \
+    /data /out participant \
+    --participant-label ${SUBJECT} \
+    -w /work \
+    --output-spaces MNI152NLin2009cAsym:res-2 \
+    --fs-license-file /opt/freesurfer/license.txt \
+    --nthreads ${SLURM_CPUS_PER_TASK} \
+    --omp-nthreads $(( SLURM_CPUS_PER_TASK - 1 )) \
+    --mem_mb 30000 \
+    --skip-bids-validation \
+    --notrack
+
+EXIT_CODE=$?
+
+# ---- Cleanup ----
+rm -rf ${WORK_DIR}
+
+echo "Finished sub-${SUBJECT} with exit code ${EXIT_CODE} at $(date)"
+exit ${EXIT_CODE}
+```
+
+Submit: `sbatch --array=1-N fmriprep.slurm` where N = number of subjects.
+
+## Troubleshooting
+
+### "recon-all is already running"
+Stale lock files from a crashed run. Remove them:
+```bash
+rm <fs-subjects-dir>/sub-*/scripts/IsRunning.*
+```
+
+### fMRIPrep hangs or freezes
+Usually a memory issue. Try:
+- Increase `--mem-mb`
+- Use `--low-mem`
+- Reduce `--omp-nthreads`
+
+### "insufficient length of BOLD data"
+Too few volumes after discarding non-steady-state frames:
+- Use `--dummy-scans 0` to override
+- Or `--ignore slicetiming`
+- Check if the run is just too short
+
+### TemplateFlow download failures on HPC
+Compute nodes often lack internet. Pre-fetch on login node:
+```bash
+export TEMPLATEFLOW_HOME=/shared/templateflow
+python -c "from templateflow.api import get; get(['MNI152NLin2009cAsym', 'MNI152NLin6Asym', 'OASIS30ANTs', 'fsaverage'])"
+```
+Then bind-mount in Singularity.
+
+### Out of memory
+- Increase SLURM `--mem`
+- Use `--low-mem` flag
+- Use `--fs-no-reconall` if surfaces aren't needed
+- Reduce `--omp-nthreads`
+
+### Race conditions with parallel runs
+Never run multiple fMRIPrep instances writing to the same output directory simultaneously. Use one subject per container instance (SLURM array jobs handle this).
+
+### Crash recovery
+fMRIPrep can resume if the working directory (`-w`) is preserved. Rerun the exact same command.
+
+### QC before fMRIPrep
+Run MRIQC first to catch bad data before spending hours on fMRIPrep:
+```bash
+singularity run mriqc.sif /data /out participant --participant-label ${SUBJECT}
+```