@brainpilot/skills 0.0.6 → 0.0.7

package/skills/06_fMRI_Neuroimaging/nifd-skill/SKILL.md

@@ -0,0 +1,205 @@
+---
+name: nifd-skill
+description: "Use this skill whenever the user wants an end-to-end workflow for the Neuroimaging in Frontotemporal Dementia (NIFD) dataset, including BIDS validation, multimodal processing of sMRI, rs-fMRI, and dMRI, phenotype extraction, and QC integration. Triggers include: 'NIFD', 'frontotemporal dementia', 'FTD', 'bvFTD', 'PPA', 'process NIFD data', or any request to run the NIFD multimodal pipeline."
+license: MIT License (NeuroClaw custom skill - freely modifiable within the project)
+layer: subagent
+skill_type: dataset
+dependencies:
+  - smri-skill
+  - fmri-skill
+  - dwi-skill
+  - bids-organizer
+  - claw-shell
+complementary_skills:
+  - brain-visualization
+  - pet-skill
+---
+# NIFD Skill (Dataset-Orchestration Layer)
+
+## Overview
+
+`nifd-skill` is the NeuroClaw orchestration skill for the **Neuroimaging in Frontotemporal Dementia (NIFD)** dataset, collected at the UCSF Memory and Aging Center.
+
+It strictly follows the NeuroClaw hierarchical design principles:
+- This skill **only describes WHAT needs to be done** and **which tool skill to delegate to**.
+- It contains **no implementation code or concrete commands**.
+- All concrete execution is delegated to existing base/tool skills via `claw-shell`.
+- Companion scripts in `scripts/` provide reference implementations for BIDS validation, phenotype extraction, and QC.
+
+**Core workflow (never bypassed):**
+1. Identify input NIFD data and target modalities.
+2. Generate a **numbered execution plan** clearly stating WHAT needs to be done and which tool skill will handle each step.
+3. Present the full plan, estimated runtime, resource requirements, and risks to the user and wait for explicit confirmation ("YES" / "execute" / "proceed").
+4. On confirmation, delegate every step to the appropriate skill via `claw-shell`.
+5. After execution, save all outputs in a clean directory structure (`nifd_output/`).
+
+**Research use only.**
+
+---
+
+## Quick Reference
+
+| Task | What needs to be done | Delegate to | Expected output |
+|---|---|---|---|
+| BIDS validation | Validate NIFD BIDS structure | `scripts/validate_nifd.py` | Validation report |
+| sMRI processing | Brain extraction, tissue segmentation, cortical thickness | `smri-skill` | `smri_output/` derivatives |
+| rs-fMRI processing | Preprocessing, denoising, connectivity | `fmri-skill` | `fmri_output/` connectivity |
+| dMRI processing | Diffusion preprocessing, tensor metrics, tractography | `dwi-skill` | `dwi_output/` metrics |
+| Phenotype extraction | Diagnosis, cognitive scores, clinical measures | `scripts/extract_nifd_phenotype.py` | Merged phenotype CSV |
+| QC summary | Per-subject quality control | `scripts/nifd_qc_summary.py` | QC summary + exclusion list |
+
+---
+
+## Dataset Characteristics
+
+- **Cohort**: ~120 participants
+  - **bvFTD**: Behavioral variant frontotemporal dementia
+  - **svPPA**: Semantic variant primary progressive aphasia
+  - **nfvPPA**: Nonfluent variant primary progressive aphasia
+  - **Healthy controls**: Age-matched
+- **Scanner**: 3T Siemens TIM Trio
+- **Modalities**: T1w sMRI, rs-fMRI, dMRI/DTI
+- **Clinical**: CDR, MMSE, neuropsychological battery
+- **Access**: OpenNeuro ds004403 (or UCSF MAC portal)
+- **Format**: BIDS-compliant
+
+---
+
+## Supported Modalities
+
+| Modality | Description | Details |
+|---|---|---|
+| T1w | High-resolution structural MRI | 1mm isotropic, cortical thickness/atrophy |
+| rs-fMRI | Resting-state functional MRI | Functional connectivity, network degeneration |
+| dMRI | Diffusion-weighted imaging | DTI, white matter tract integrity |
+
+---
+
+## NIFD Diagnostic Groups
+
+| Group | Description | Typical N |
+|---|---|---|
+| bvFTD | Behavioral variant FTD | ~40 |
+| svPPA | Semantic variant PPA | ~20 |
+| nfvPPA | Nonfluent variant PPA | ~15 |
+| Control | Healthy age-matched controls | ~45 |
+
+---
+
+## BIDS Preparation
+
+### Script: `scripts/validate_nifd.py`
+
+Validates NIFD BIDS structure and generates a compliance report.
+
+```bash
+python skills/nifd-skill/scripts/validate_nifd.py \
+  --input /path/to/NIFD/bids \
+  --output /path/to/nifd_output/qc/bids_validation.csv
+```
+
+Features:
+- BIDS directory structure validation
+- Diagnostic group completeness check
+- Modality completeness (T1w, rs-fMRI, dMRI)
+- Missing data identification
+
+---
+
+## Core Workflow (Never Bypassed)
+
+1. Identify user target: full NIFD processing, imaging subset, phenotype extraction, or BIDS validation only.
+2. Generate a numbered plan with tools, outputs, runtime, storage, and risks.
+3. Wait for explicit confirmation (`YES` / `execute` / `proceed`).
+4. On confirmation, run BIDS validation using `scripts/validate_nifd.py`.
+5. Delegate to `smri-skill` for structural MRI processing.
+6. Delegate to `fmri-skill` for rs-fMRI processing (functional connectivity).
+7. Delegate to `dwi-skill` for dMRI processing (white matter integrity).
+8. If phenotype extraction is requested, run `scripts/extract_nifd_phenotype.py`.
+9. If QC summary is requested, run `scripts/nifd_qc_summary.py`.
+10. Save outputs into `nifd_output/`.
+
+---
+
+## Modality Processing Delegation
+
+| Modality | Delegated skill | Typical tasks | Main outputs |
+|---|---|---|---|
+| sMRI (T1w) | `smri-skill` | brain extraction, tissue segmentation, cortical thickness | `smri_output/` derivatives |
+| rs-fMRI | `fmri-skill` | preprocessing, denoising, connectivity | `fmri_output/` connectivity |
+| dMRI | `dwi-skill` | diffusion preprocessing, tensor metrics, tractography | `dwi_output/` metrics |
+
+---
+
+## Standard Output Layout
+
+```
+nifd_output/
+├── bids/                   # BIDS-staged data (or validation report)
+├── smri/                   # Structural MRI derivatives
+├── fmri/                   # Functional MRI derivatives (rs-fMRI connectivity)
+├── dwi/                    # Diffusion MRI derivatives (DTI metrics)
+├── phenotype/              # Merged phenotype tables (diagnosis, cognitive)
+├── qc/                     # QC summaries and exclusion lists
+└── logs/                   # Processing logs
+```
+
+---
+
+## Benchmark Adapter Guidance
+
+For benchmark-style prompts, do not force the full orchestration when the task only asks for local NIFD data validation.
+
+- If the task starts from NIFD data already present on disk and only asks for BIDS validation:
+  - Skip the download stage
+  - Default to the narrow path `local NIFD discovery -> BIDS validation -> report`
+- In benchmark mode, do not require explicit confirmation before presenting the validation solution.
+
+---
+
+## Safety and Execution Policy
+- No execution before explicit plan confirmation.
+- All execution must be routed via `claw-shell`.
+- Missing dependencies must be resolved by `dependency-planner` before running.
+
+---
+
+## Important Notes and Limitations
+- NIFD is a clinical cohort; patient data requires careful handling.
+- Diagnostic groups (bvFTD, svPPA, nfvPPA) have distinct atrophy patterns; group-level analyses should account for heterogeneity.
+- Cortical thickness and voxel-based morphometry are commonly used structural measures.
+- Network degeneration hypothesis: FTD targets specific large-scale networks.
+- `nifd-skill` is orchestration-only; detailed preprocessing logic remains in modality skills.
+
+---
+
+## When to Call This Skill
+- User asks for end-to-end NIFD workflow.
+- User asks to process NIFD neuroimaging data.
+- User needs BIDS validation for NIFD data.
+- User asks to extract NIFD phenotype data (diagnosis, cognitive scores).
+- User asks for frontotemporal dementia neuroimaging analysis.
+
+---
+
+## Complementary / Related Skills
+- `smri-skill` → structural MRI preprocessing
+- `fmri-skill` → functional MRI preprocessing and analysis
+- `dwi-skill` → diffusion MRI preprocessing
+- `pet-skill` → PET imaging (tau-PET, amyloid-PET if available)
+- `bids-organizer` → BIDS validation and organization
+- `brain-visualization` → visualization of derivatives
+- `dependency-planner` → dependency resolution
+- `conda-env-manager` → environment management
+- `claw-shell` → command execution
+
+---
+
+## Reference
+- NIFD: UCSF Memory and Aging Center
+- Frontotemporal Dementia: FTDC clinical diagnostic criteria
+- OpenNeuro ds004403
+
+Created At: 2026-05-06 13:55 HKT
+Last Updated At: 2026-05-06 13:55 HKT
+Author: chengwang96

package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/extract_nifd_phenotype.py

@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+"""Extract and merge NIFD phenotype data.
+
+Reads NIFD phenotype files (diagnosis, cognitive scores)
+and produces a merged phenotype table aligned with imaging subject list.
+"""
+import argparse
+import csv
+import sys
+from pathlib import Path
+from typing import Dict, List, Optional
+
+COLUMN_MAP = {
+    "subject_id": ["subject", "Subject", "participant_id", "SubID", "PTID"],
+    "age": ["age", "Age"],
+    "sex": ["sex", "Sex", "Gender"],
+    "diagnosis": ["diagnosis", "Diagnosis", "Group", "DX"],
+    "diagnosis_detail": ["diagnosis_detail", "Diagnosis_Detail", "DX_detail"],
+    "cdr": ["CDR", "cdr"],
+    "cdr_sb": ["CDRSB", "cdr_sb"],
+    "mmse": ["MMSE", "mmse"],
+    "gds": ["GDS", "gds"],
+    "fas": ["FAS", "fas"],
+    "category_fluency": ["CategoryFluency", "category_fluency"],
+    "boston_naming": ["BostonNaming", "boston_naming"],
+    "trail_a": ["TrailA", "trail_a"],
+    "trail_b": ["TrailB", "trail_b"],
+    "digit_span": ["DigitSpan", "digit_span"],
+}
+
+
+def load_csv(path: Path) -> List[Dict[str, str]]:
+    delimiter = "\t" if path.suffix == ".tsv" else ","
+    with open(path, "r", encoding="utf-8") as f:
+        reader = csv.DictReader(f, delimiter=delimiter)
+        return list(reader)
+
+
+def find_column(row: Dict[str, str], candidates: List[str]) -> Optional[str]:
+    for col in candidates:
+        if col in row:
+            return col
+    return None
+
+
+def extract_phenotype(
+    phenotype_files: List[Path],
+    imaging_ids: Optional[List[str]] = None,
+    columns: Optional[List[str]] = None,
+) -> List[Dict[str, str]]:
+    all_data = []
+    for fpath in phenotype_files:
+        rows = load_csv(fpath)
+        all_data.extend(rows)
+
+    if not all_data:
+        return []
+
+    target_columns = columns if columns else list(COLUMN_MAP.keys())
+    merged = {}
+
+    for row in all_data:
+        subj_col = find_column(row, COLUMN_MAP["subject_id"])
+        if subj_col is None:
+            continue
+        subj_id = row[subj_col].strip()
+        if not subj_id:
+            continue
+
+        if subj_id not in merged:
+            merged[subj_id] = {"subject_id": subj_id}
+
+        for target_col in target_columns:
+            if target_col == "subject_id":
+                continue
+            if target_col in merged[subj_id] and merged[subj_id][target_col]:
+                continue
+            candidates = COLUMN_MAP.get(target_col, [target_col])
+            src_col = find_column(row, candidates)
+            if src_col and row.get(src_col, "").strip():
+                merged[subj_id][target_col] = row[src_col].strip()
+
+    result = list(merged.values())
+    if imaging_ids:
+        imaging_set = set(imaging_ids)
+        result = [r for r in result if r["subject_id"] in imaging_set]
+    return result
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Extract NIFD phenotype data.")
+    parser.add_argument("--phenotype-files", required=True, nargs="+")
+    parser.add_argument("--output", required=True)
+    parser.add_argument("--imaging-ids")
+    parser.add_argument("--columns")
+    args = parser.parse_args()
+
+    phenotype_files = [Path(f).resolve() for f in args.phenotype_files]
+    for f in phenotype_files:
+        if not f.exists():
+            print(f"File not found: {f}", file=sys.stderr)
+            return 1
+
+    imaging_ids = None
+    if args.imaging_ids:
+        id_file = Path(args.imaging_ids).resolve()
+        if id_file.exists():
+            imaging_ids = [l.strip() for l in id_file.read_text().splitlines() if l.strip()]
+
+    columns = None
+    if args.columns:
+        columns = [c.strip() for c in args.columns.split(",")]
+
+    merged = extract_phenotype(phenotype_files, imaging_ids, columns)
+    if not merged:
+        print("[ERROR] No phenotype data extracted.", file=sys.stderr)
+        return 1
+
+    output_path = Path(args.output).resolve()
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    fieldnames = list(merged[0].keys())
+    with open(output_path, "w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        writer.writerows(merged)
+
+    print(f"Phenotype: {len(merged)} subjects, {len(fieldnames)} columns -> {output_path}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/nifd_qc_summary.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python3
+"""Generate per-subject QC summaries for NIFD processing.
+
+Clinical-specific: tracks diagnostic group and applies appropriate QC criteria.
+"""
+import argparse
+import csv
+import sys
+from pathlib import Path
+from typing import Dict, List
+
+
+def load_confounds(confounds_path: Path) -> Dict[str, float]:
+    metrics = {"fd_mean": float("nan"), "fd_max": float("nan")}
+    try:
+        import pandas as pd
+        df = pd.read_csv(confounds_path, sep="\t")
+        if "framewise_displacement" in df.columns:
+            fd = df["framewise_displacement"].dropna()
+            metrics["fd_mean"] = float(fd.mean())
+            metrics["fd_max"] = float(fd.max())
+    except Exception:
+        pass
+    return metrics
+
+
+def check_exclusion(metrics: Dict[str, float], fd_threshold: float = 0.3) -> List[str]:
+    reasons = []
+    if not (metrics["fd_mean"] != metrics["fd_mean"]):
+        if metrics["fd_mean"] > fd_threshold:
+            reasons.append(f"FD mean {metrics['fd_mean']:.3f} > {fd_threshold}")
+    return reasons
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Generate QC summaries for NIFD processing.")
+    parser.add_argument("--fmriprep-dir", help="Path to fMRIPrep output directory")
+    parser.add_argument("--output", required=True)
+    parser.add_argument("--exclude-output")
+    parser.add_argument("--fd-threshold", type=float, default=0.3)
+    parser.add_argument("--diagnosis-file", help="CSV with subject diagnosis info")
+    args = parser.parse_args()
+
+    output_path = Path(args.output).resolve()
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    diagnosis_map = {}
+    if args.diagnosis_file:
+        diag_path = Path(args.diagnosis_file).resolve()
+        if diag_path.exists():
+            with open(diag_path, "r", encoding="utf-8") as f:
+                reader = csv.DictReader(f)
+                for row in reader:
+                    subj = row.get("subject_id") or row.get("Subject", "")
+                    diag = row.get("diagnosis") or row.get("Diagnosis") or row.get("Group", "")
+                    if subj and diag:
+                        diagnosis_map[subj] = diag
+
+    subjects = set()
+    if args.fmriprep_dir:
+        fmriprep_dir = Path(args.fmriprep_dir).resolve()
+        if fmriprep_dir.exists():
+            for d in fmriprep_dir.glob("sub-*"):
+                if d.is_dir():
+                    subjects.add(d.name)
+
+    if not subjects:
+        print("[WARN] No subjects found.", file=sys.stderr)
+        return 1
+
+    results = []
+    excluded = []
+    for subj in sorted(subjects):
+        metrics = {"subject_id": subj}
+        if diagnosis_map:
+            metrics["diagnosis"] = diagnosis_map.get(subj, "unknown")
+
+        if args.fmriprep_dir:
+            confounds_files = list(Path(args.fmriprep_dir).glob(f"{subj}/func/*_desc-confounds_timeseries.tsv"))
+            if confounds_files:
+                metrics.update(load_confounds(confounds_files[0]))
+
+        exclusion_reasons = check_exclusion(metrics, args.fd_threshold)
+        metrics["excluded"] = len(exclusion_reasons) > 0
+        metrics["exclusion_reasons"] = "; ".join(exclusion_reasons)
+
+        results.append(metrics)
+        if exclusion_reasons:
+            excluded.append({"subject_id": subj, "reasons": "; ".join(exclusion_reasons)})
+
+    if results:
+        fieldnames = list(results[0].keys())
+        with open(output_path, "w", newline="", encoding="utf-8") as f:
+            writer = csv.DictWriter(f, fieldnames=fieldnames)
+            writer.writeheader()
+            writer.writerows(results)
+        print(f"QC: {len(results)} subjects, {len(excluded)} excluded -> {output_path}")
+
+    if args.exclude_output and excluded:
+        exclude_path = Path(args.exclude_output).resolve()
+        exclude_path.parent.mkdir(parents=True, exist_ok=True)
+        with open(exclude_path, "w", newline="", encoding="utf-8") as f:
+            writer = csv.DictWriter(f, fieldnames=["subject_id", "reasons"])
+            writer.writeheader()
+            writer.writerows(excluded)
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/validate_nifd.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python3
+"""Validate NIFD BIDS structure and generate compliance report.
+
+Checks directory structure, diagnostic group completeness, and modality presence.
+"""
+import argparse
+import csv
+import sys
+from pathlib import Path
+from typing import Dict, List
+
+EXPECTED_MODALITIES = {
+    "anat": ["T1w"],
+    "func": ["task-rest_bold"],
+    "dwi": ["dwi"],
+}
+
+
+def validate_subject(subject_dir: Path) -> Dict[str, any]:
+    """Validate a single subject's NIFD BIDS structure."""
+    report = {
+        "subject": subject_dir.name,
+        "anat_complete": False,
+        "rs_fMRI_present": False,
+        "dwi_present": False,
+        "missing_files": [],
+        "warnings": [],
+    }
+
+    # Check anat
+    anat_dir = subject_dir / "anat"
+    if anat_dir.exists():
+        t1w_files = list(anat_dir.glob("*_T1w.nii.gz"))
+        report["anat_complete"] = len(t1w_files) > 0
+        if not t1w_files:
+            report["missing_files"].append("anat/*_T1w.nii.gz")
+    else:
+        report["missing_files"].append("anat/")
+
+    # Check func
+    func_dir = subject_dir / "func"
+    if func_dir.exists():
+        rest_bold = list(func_dir.glob("*_task-rest_bold.nii.gz"))
+        report["rs_fMRI_present"] = len(rest_bold) > 0
+        if not rest_bold:
+            report["missing_files"].append("func/*_task-rest_bold.nii.gz")
+    else:
+        report["warnings"].append("No func directory")
+
+    # Check dwi
+    dwi_dir = subject_dir / "dwi"
+    if dwi_dir.exists():
+        dwi_files = list(dwi_dir.glob("*_dwi.nii.gz"))
+        report["dwi_present"] = len(dwi_files) > 0
+        if not dwi_files:
+            report["missing_files"].append("dwi/*_dwi.nii.gz")
+    else:
+        report["warnings"].append("No dwi directory")
+
+    return report
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Validate NIFD BIDS structure.")
+    parser.add_argument("--input", required=True, help="Path to NIFD BIDS directory")
+    parser.add_argument("--output", required=True, help="Output path for validation CSV")
+    args = parser.parse_args()
+
+    input_dir = Path(args.input).resolve()
+    if not input_dir.exists():
+        print(f"Input directory not found: {input_dir}", file=sys.stderr)
+        return 1
+
+    subjects = sorted([d for d in input_dir.glob("sub-*") if d.is_dir()])
+    print(f"Found {len(subjects)} subjects in {input_dir}")
+
+    if not subjects:
+        print("[ERROR] No subjects found.", file=sys.stderr)
+        return 1
+
+    results = []
+    for sub_dir in subjects:
+        report = validate_subject(sub_dir)
+        results.append(report)
+
+    output_path = Path(args.output).resolve()
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    fieldnames = ["subject", "anat_complete", "rs_fMRI_present", "dwi_present", "missing_files", "warnings"]
+    with open(output_path, "w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        for r in results:
+            r["missing_files"] = "; ".join(r["missing_files"])
+            r["warnings"] = "; ".join(r["warnings"])
+            writer.writerow(r)
+
+    complete = sum(1 for r in results if r["anat_complete"] and r["rs_fMRI_present"])
+    print(f"\nValidation Summary:")
+    print(f"  Total subjects: {len(results)}")
+    print(f"  Complete (anat + rs-fMRI): {complete}")
+    print(f"  With T1w: {sum(1 for r in results if r['anat_complete'])}")
+    print(f"  With rs-fMRI: {sum(1 for r in results if r['rs_fMRI_present'])}")
+    print(f"  With dMRI: {sum(1 for r in results if r['dwi_present'])}")
+    print(f"  Output: {output_path}")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())