@yibeichan/claude-skills 1.0.2

package/skills/bidsapp-nidm-standards/references/nidm_integration.md

@@ -0,0 +1,403 @@
+# NIDM Integration Reference
+
+## NIDM Workflow
+
+The standard NIDM integration follows this sequence:
+
+```
+1. Check --skip-nidm flag
+   ├─ If true: Skip NIDM processing entirely
+   └─ If false: Continue to step 2
+
+2. Locate input NIDM file
+   └─ Path: <nidm-input-dir>/[sub-{id}/[ses-{session}/]]<filename>.ttl
+   
+3. Copy NIDM file to output (if exists)
+   ├─ Destination: <output_dir>/<bidsapp_name>/nidm/sub-{id}/[ses-{session}/]<filename>.ttl
+   └─ If not found: Log warning and continue
+
+4. Run analysis (unless --skip-<analysis>)
+   └─ Output to: <output_dir>/<bidsapp_name>/<analysis>/...
+
+5. Extract metrics from analysis outputs
+   └─ Analysis-specific metric extraction
+
+6. Integrate metrics into NIDM file
+   ├─ If copied file exists: Overwrite with integrated data
+   └─ If no existing file: Create new NIDM file from scratch
+```
+
+## NIDM File Naming
+
+### With Sessions
+```
+sub-{id}/ses-{session}/sub-{id}_ses-{session}.ttl
+```
+Example: `sub-01/ses-baseline/sub-01_ses-baseline.ttl`
+
+### Without Sessions (Session-less)
+```
+sub-{id}/sub-{id}.ttl
+```
+Example: `sub-01/sub-01.ttl`
+
+**Important**: Session-less data uses `sub-01.ttl`, NOT `sub-01_ses-baseline.ttl`
+
+## NIDM File Organization
+
+### Input NIDM Directory Structure
+
+**Option 1: Single file** (for dataset-level NIDM)
+```
+<nidm-input-dir>/
+└── nidm.ttl
+```
+
+**Option 2: Per-subject files with sessions**
+```
+<nidm-input-dir>/
+├── sub-01/
+│   ├── ses-baseline/
+│   │   └── sub-01_ses-baseline.ttl
+│   └── ses-followup/
+│       └── sub-01_ses-followup.ttl
+└── sub-02/
+    └── ses-baseline/
+        └── sub-02_ses-baseline.ttl
+```
+
+**Option 3: Per-subject files without sessions**
+```
+<nidm-input-dir>/
+├── sub-01/
+│   └── sub-01.ttl
+└── sub-02/
+    └── sub-02.ttl
+```
+
+### Output NIDM Directory Structure
+
+**With sessions:**
+```
+<output_dir>/<bidsapp_name>/nidm/
+├── dataset_description.json
+└── sub-{id}/
+    └── ses-{session}/
+        └── sub-{id}_ses-{session}.ttl
+```
+
+**Without sessions:**
+```
+<output_dir>/<bidsapp_name>/nidm/
+├── dataset_description.json
+└── sub-{id}/
+    └── sub-{id}.ttl
+```
+
+## NIDM File Format
+
+NIDM files use Turtle (.ttl) RDF format. Basic structure:
+
+```turtle
+@prefix prov: <http://www.w3.org/ns/prov#> .
+@prefix nidm: <http://purl.org/nidash/nidm#> .
+@prefix freesurfer: <http://purl.org/nidash/freesurfer#> .
+
+# Software Agent
+<software/freesurfer> a prov:Agent, prov:SoftwareAgent ;
+    prov:type nidm:FreeSurfer ;
+    nidm:softwareVersion "8.0.0" .
+
+# Processing Activity
+<activity/recon-all> a prov:Activity ;
+    prov:wasAssociatedWith <software/freesurfer> ;
+    prov:used <input/T1w> ;
+    prov:generated <output/segmentation> .
+
+# Metrics
+<output/segmentation> a prov:Entity ;
+    freesurfer:LeftHippocampalVolume "3847.2"^^xsd:float ;
+    freesurfer:RightHippocampalVolume "3912.5"^^xsd:float .
+```
+
+## NIDM Copy Implementation
+
+### Copy Function Template
+
+```python
+import shutil
+from pathlib import Path
+import logging
+
+logger = logging.getLogger(__name__)
+
+def copy_nidm_file(nidm_input_dir, output_dir, bidsapp_name, subject_id, session_id=None):
+    """
+    Copy existing NIDM file to output directory.
+    
+    If no existing NIDM file is found, logs a warning and returns None.
+    The NIDM converter should handle creating a new file from scratch.
+    
+    Args:
+        nidm_input_dir: Path to input NIDM directory
+        output_dir: Base output directory
+        bidsapp_name: Name of BIDSapp (e.g., 'freesurfer_nidm')
+        subject_id: Subject ID without 'sub-' prefix
+        session_id: Session ID without 'ses-' prefix (optional, None for session-less)
+    
+    Returns:
+        Path to copied NIDM file in output directory, or None if no input file found
+    """
+    # Determine input NIDM file path
+    input_dir = Path(nidm_input_dir)
+    
+    # Try per-subject structure
+    if session_id:
+        # With sessions
+        filename = f"sub-{subject_id}_ses-{session_id}.ttl"
+        input_file = input_dir / f"sub-{subject_id}" / f"ses-{session_id}" / filename
+        if not input_file.exists():
+            # Try generic nidm.ttl name
+            input_file = input_dir / f"sub-{subject_id}" / f"ses-{session_id}" / "nidm.ttl"
+    else:
+        # Without sessions - use sub-{id}.ttl
+        filename = f"sub-{subject_id}.ttl"
+        input_file = input_dir / f"sub-{subject_id}" / filename
+        if not input_file.exists():
+            # Try generic nidm.ttl name
+            input_file = input_dir / f"sub-{subject_id}" / "nidm.ttl"
+    
+    # Fall back to dataset-level file
+    if not input_file.exists():
+        input_file = input_dir / "nidm.ttl"
+    
+    if not input_file.exists():
+        logger.warning(
+            f"No existing NIDM file found in {nidm_input_dir} for sub-{subject_id}"
+            + (f" ses-{session_id}" if session_id else "")
+            + ". NIDM conversion will create a new file from scratch."
+        )
+        return None
+    
+    # Create output directory
+    nidm_output_dir = Path(output_dir) / bidsapp_name / "nidm"
+    if session_id:
+        subject_nidm_dir = nidm_output_dir / f"sub-{subject_id}" / f"ses-{session_id}"
+        output_filename = f"sub-{subject_id}_ses-{session_id}.ttl"
+    else:
+        # Session-less: sub-{id}.ttl
+        subject_nidm_dir = nidm_output_dir / f"sub-{subject_id}"
+        output_filename = f"sub-{subject_id}.ttl"
+    
+    subject_nidm_dir.mkdir(parents=True, exist_ok=True)
+    output_file = subject_nidm_dir / output_filename
+    
+    # Copy file
+    shutil.copy2(input_file, output_file)
+    logger.info(f"Copied NIDM file from {input_file} to {output_file}")
+    
+    return output_file
+```
+
+## NIDM Integration Implementation
+
+### Integration Function Template
+
+```python
+import rdflib
+from pathlib import Path
+from datetime import datetime
+import logging
+
+logger = logging.getLogger(__name__)
+
+def integrate_metrics_to_nidm(nidm_file, metrics, software_name, software_version, 
+                               subject_id, session_id=None, output_dir=None, 
+                               bidsapp_name=None):
+    """
+    Integrate analysis metrics into NIDM file.
+    
+    If nidm_file is None (no existing file), creates a new NIDM file from scratch.
+    
+    Args:
+        nidm_file: Path to NIDM .ttl file (can be None)
+        metrics: Dictionary of metrics to add
+        software_name: Name of analysis software (e.g., 'FreeSurfer')
+        software_version: Version of analysis software
+        subject_id: Subject ID without 'sub-' prefix
+        session_id: Session ID without 'ses-' prefix (optional)
+        output_dir: Base output directory (required if nidm_file is None)
+        bidsapp_name: BIDSapp name (required if nidm_file is None)
+    
+    Returns:
+        Path to the created/updated NIDM file
+    """
+    # Determine output file path
+    if nidm_file is None:
+        if output_dir is None or bidsapp_name is None:
+            raise ValueError("output_dir and bidsapp_name required when creating new NIDM file")
+        
+        # Create new NIDM file
+        nidm_output_dir = Path(output_dir) / bidsapp_name / "nidm"
+        if session_id:
+            subject_nidm_dir = nidm_output_dir / f"sub-{subject_id}" / f"ses-{session_id}"
+            output_filename = f"sub-{subject_id}_ses-{session_id}.ttl"
+        else:
+            subject_nidm_dir = nidm_output_dir / f"sub-{subject_id}"
+            output_filename = f"sub-{subject_id}.ttl"
+        
+        subject_nidm_dir.mkdir(parents=True, exist_ok=True)
+        nidm_file = subject_nidm_dir / output_filename
+        
+        logger.info(f"Creating new NIDM file at {nidm_file}")
+        g = rdflib.Graph()
+    else:
+        # Load existing NIDM graph
+        logger.info(f"Loading existing NIDM file from {nidm_file}")
+        g = rdflib.Graph()
+        g.parse(nidm_file, format='turtle')
+    
+    # Define namespaces
+    PROV = rdflib.Namespace('http://www.w3.org/ns/prov#')
+    NIDM = rdflib.Namespace('http://purl.org/nidash/nidm#')
+    ANALYSIS = rdflib.Namespace(f'http://purl.org/nidash/{software_name.lower()}#')
+    XSD = rdflib.Namespace('http://www.w3.org/2001/XMLSchema#')
+    
+    # Bind namespaces
+    g.bind('prov', PROV)
+    g.bind('nidm', NIDM)
+    g.bind(software_name.lower(), ANALYSIS)
+    g.bind('xsd', XSD)
+    
+    # Create activity node
+    timestamp = datetime.now().isoformat()
+    activity_id = rdflib.URIRef(f'activity/{software_name.lower()}-{timestamp}')
+    g.add((activity_id, rdflib.RDF.type, PROV.Activity))
+    g.add((activity_id, PROV.startedAtTime, rdflib.Literal(datetime.now(), datatype=XSD.dateTime)))
+    
+    # Create software agent
+    software_id = rdflib.URIRef(f'software/{software_name.lower()}')
+    g.add((software_id, rdflib.RDF.type, PROV.SoftwareAgent))
+    g.add((software_id, NIDM.softwareVersion, rdflib.Literal(software_version)))
+    g.add((activity_id, PROV.wasAssociatedWith, software_id))
+    
+    # Create output entity with metrics
+    output_id = rdflib.URIRef(f'output/{software_name.lower()}-result-{timestamp}')
+    g.add((output_id, rdflib.RDF.type, PROV.Entity))
+    g.add((activity_id, PROV.generated, output_id))
+    
+    # Add subject/session info
+    g.add((output_id, NIDM.subjectID, rdflib.Literal(f"sub-{subject_id}")))
+    if session_id:
+        g.add((output_id, NIDM.sessionID, rdflib.Literal(f"ses-{session_id}")))
+    
+    # Add metrics
+    for metric_name, metric_value in metrics.items():
+        predicate = ANALYSIS[metric_name]
+        if isinstance(metric_value, float):
+            obj = rdflib.Literal(metric_value, datatype=XSD.float)
+        elif isinstance(metric_value, int):
+            obj = rdflib.Literal(metric_value, datatype=XSD.integer)
+        else:
+            obj = rdflib.Literal(metric_value)
+        g.add((output_id, predicate, obj))
+    
+    # Save graph (overwrites if existing, creates if new)
+    g.serialize(destination=str(nidm_file), format='turtle')
+    logger.info(f"NIDM file saved to {nidm_file}")
+    
+    return nidm_file
+```
+
+## Complete NIDM Processing Workflow
+
+```python
+def process_nidm(nidm_input_dir, output_dir, bidsapp_name, subject_id, session_id, 
+                 skip_nidm, analysis_outputs, software_name, software_version):
+    """
+    Complete NIDM processing workflow.
+    
+    Handles both cases:
+    1. Existing NIDM file: Copy → Extract metrics → Integrate (overwrite)
+    2. No existing file: Warning → Extract metrics → Create new file
+    """
+    
+    if skip_nidm:
+        logger.info("Skipping NIDM processing (--skip-nidm flag set)")
+        return
+    
+    # Step 1: Try to copy existing NIDM file
+    nidm_file = copy_nidm_file(nidm_input_dir, output_dir, bidsapp_name, 
+                                subject_id, session_id)
+    
+    # Step 2: Extract metrics from analysis outputs
+    logger.info("Extracting metrics from analysis outputs")
+    metrics = extract_metrics(analysis_outputs, subject_id, session_id)
+    
+    # Step 3: Integrate metrics into NIDM file
+    # This works whether nidm_file exists or is None
+    nidm_file = integrate_metrics_to_nidm(
+        nidm_file, 
+        metrics, 
+        software_name, 
+        software_version,
+        subject_id,
+        session_id,
+        output_dir,
+        bidsapp_name
+    )
+    
+    logger.info(f"NIDM processing complete: {nidm_file}")
+```
+
+## Analysis-Specific Metric Extraction
+
+Each analysis tool requires custom metric extraction. Examples:
+
+### FreeSurfer Metrics
+
+```python
+def extract_freesurfer_metrics(freesurfer_dir):
+    """Extract metrics from FreeSurfer outputs."""
+    metrics = {}
+    
+    # Read aseg.stats for volumetric measures
+    aseg_stats = freesurfer_dir / "stats" / "aseg.stats"
+    if aseg_stats.exists():
+        # Parse aseg.stats file
+        # Extract hippocampal volumes, cortical volumes, etc.
+        pass
+    
+    # Read aparc.stats for cortical measures
+    for hemi in ['lh', 'rh']:
+        aparc_stats = freesurfer_dir / "stats" / f"{hemi}.aparc.stats"
+        if aparc_stats.exists():
+            # Parse cortical thickness, surface area, etc.
+            pass
+    
+    return metrics
+```
+
+### MRIQC Metrics
+
+```python
+def extract_mriqc_metrics(mriqc_output_dir):
+    """Extract metrics from MRIQC JSON outputs."""
+    import json
+    
+    metrics = {}
+    
+    # Find JSON file with IQMs
+    json_files = list(mriqc_output_dir.glob("sub-*_T1w.json"))
+    if json_files:
+        with open(json_files[0]) as f:
+            iqms = json.load(f)
+        
+        # Extract image quality metrics
+        metrics['SNR'] = iqms.get('snr_total')
+        metrics['CNR'] = iqms.get('cnr')
+        metrics['EFC'] = iqms.get('efc')
+        # Add more metrics as needed
+    
+    return metrics
+```

package/skills/bidsapp-nidm-standards/references/repo_structure.md

@@ -0,0 +1,121 @@
+# Repository Structure Reference
+
+## Standard Directory Layout
+
+```
+<bidsapp_name>/                    # e.g., freesurfer_bidsapp, mriqc-nidm_bidsapp
+├── src/                           # Source code
+│   ├── <analysis_name>/          # Main analysis package
+│   │   ├── __init__.py
+│   │   ├── run.py                # CLI entry point
+│   │   ├── <analysis>_runner.py  # Core analysis execution
+│   │   ├── utils.py              # Utility functions
+│   │   └── validators.py         # Input validation
+│   └── nidm/                      # NIDM conversion package
+│       ├── __init__.py
+│       ├── nidm_converter.py     # Main NIDM conversion logic
+│       ├── nidm_utils.py         # NIDM utilities
+│       └── data/                  # NIDM reference data
+│           ├── ontology_mappings.json
+│           └── software_metadata.csv
+├── tests/                         # Test suite
+│   ├── __init__.py
+│   ├── test_<analysis>.py        # Analysis tests
+│   ├── test_nidm.py              # NIDM conversion tests
+│   ├── test_integration.py       # End-to-end tests
+│   └── fixtures/                  # Test data
+│       ├── bids_data/
+│       └── nidm_data/
+├── examples/                      # Usage examples
+│   ├── example_usage.sh          # Basic usage script
+│   └── babs_integration.sh       # BABS workflow example
+├── Dockerfile                     # Docker container definition
+├── Singularity                    # Singularity/Apptainer definition
+├── setup.py                       # Package setup and metadata
+├── setup.cfg                      # Setup configuration
+├── requirements.txt               # Python dependencies
+├── VERSION                        # Version number (e.g., 0.1.0)
+├── LICENSE                        # MIT license
+├── README.md                      # Usage documentation
+├── .gitignore                     # Git ignore patterns
+└── .github/                       # GitHub workflows (optional)
+    └── workflows/
+        └── test.yml              # CI/CD tests
+```
+
+## File Responsibilities
+
+### src/<analysis_name>/run.py
+- CLI argument parsing using argparse
+- Input validation
+- Orchestrate analysis and NIDM workflows
+- Error handling and logging
+
+### src/<analysis_name>/<analysis>_runner.py
+- Execute the actual analysis tool
+- Handle tool-specific configurations
+- Manage intermediate outputs
+- Extract metrics for NIDM
+
+### src/nidm/nidm_converter.py
+- Copy input NIDM file to output directory
+- Extract relevant metrics from analysis outputs
+- Integrate metrics into NIDM file (overwrite)
+- Validate NIDM output
+
+### Dockerfile
+- Define container image
+- Install analysis tool dependencies
+- Install Python package
+- Set entry point to run.py
+
+### Singularity
+- Define Apptainer/Singularity container
+- Use existing base images where possible
+- Install analysis tool and Python package
+- Configure for HPC execution
+
+### setup.py
+- Package metadata (name, version, author)
+- Define entry points for CLI commands
+- Specify dependencies
+- Include package data files
+
+## Key Naming Conventions
+
+- **Repository name**: `<tool>_bidsapp` or `<tool>-nidm_bidsapp`
+  - Examples: `freesurfer_bidsapp`, `mriqc-nidm_bidsapp`, `ants_bidsapp`
+
+- **Package name**: `<tool>_nidm` (in setup.py)
+  - Examples: `freesurfer_nidm`, `mriqc_nidm`, `ants_nidm`
+
+- **Output directory**: `<tool>_nidm` or `<tool>-nidm`
+  - Must match package name for consistency
+
+- **Container name**: `<tool>-nidm-bidsapp-<version>`
+  - Example: `mriqc-nidm-bidsapp-0-1-0`
+
+## Modularity Requirements
+
+The NIDM conversion component must be:
+- **Self-contained**: All NIDM logic in `src/nidm/` directory
+- **Reusable**: Can be imported and used independently
+- **Analysis-specific**: Tailored to the specific analysis tool's outputs
+- **Consistent API**: Standard function signatures across all BIDSapps
+
+Example NIDM module API:
+
+```python
+# In src/nidm/nidm_converter.py
+def copy_nidm_input(input_dir, output_dir, subject_id, session_id):
+    """Copy existing NIDM file to output directory."""
+    pass
+
+def extract_metrics(analysis_output_dir, subject_id, session_id):
+    """Extract relevant metrics from analysis outputs."""
+    pass
+
+def integrate_to_nidm(nidm_file, metrics):
+    """Integrate extracted metrics into NIDM file."""
+    pass
+```

package/skills/bidsapp-nidm-standards/references/testing_patterns.md

@@ -0,0 +1,82 @@
+# Testing Patterns
+
+## Test Structure
+
+```
+tests/
+├── __init__.py
+├── test_<analysis>.py       # Analysis-specific tests
+├── test_nidm.py             # NIDM conversion tests
+├── test_integration.py      # End-to-end tests
+└── fixtures/                 # Test data
+    ├── bids_data/           # Minimal BIDS dataset
+    └── nidm_data/           # Sample NIDM files
+```
+
+## Test Coverage Requirements
+
+1. **CLI Argument Parsing**
+   - Valid arguments accepted
+   - Invalid arguments rejected
+   - Default values correct
+
+2. **BIDS Validation**
+   - Valid BIDS datasets accepted
+   - Invalid datasets rejected
+   - Required files detected
+
+3. **Analysis Execution**
+   - Tool runs successfully with valid inputs
+   - Error handling for invalid inputs
+   - Output files created
+
+4. **NIDM Conversion**
+   - Input NIDM file copied correctly
+   - Metrics extracted from analysis outputs
+   - NIDM file updated with new metrics
+   - TTL format valid
+
+5. **Output Structure**
+   - Correct directory structure created
+   - BIDS-compliant derivatives
+   - dataset_description.json files present
+
+## Example Tests
+
+```python
+import pytest
+from pathlib import Path
+
+def test_cli_args():
+    """Test CLI argument parsing."""
+    from <package>.run import parse_args
+    
+    args = parse_args([
+        '/data/bids',
+        '/data/output',
+        'participant',
+        '--participant-label', '01',
+        '--nidm-input-dir', '/data/nidm'
+    ])
+    
+    assert args.bids_dir == '/data/bids'
+    assert args.participant_label == ['01']
+
+def test_nidm_copy(tmp_path):
+    """Test NIDM file copying."""
+    from <package>.nidm.nidm_converter import copy_nidm_file
+    
+    # Setup test data
+    nidm_input = tmp_path / "nidm_input"
+    nidm_input.mkdir()
+    (nidm_input / "nidm.ttl").write_text("test nidm content")
+    
+    # Copy file
+    output = tmp_path / "output"
+    nidm_file = copy_nidm_file(nidm_input, output, "test_app", "01")
+    
+    assert nidm_file.exists()
+    assert nidm_file.read_text() == "test nidm content"
+```
+
+Use pytest fixtures for test data and temporary directories.