@pikaa-ai/pikaa 0.3.23 → 0.3.24

package/skills/neuropixels-analysis/SKILL.md

@@ -1,412 +0,0 @@
----
-name: neuropixels-analysis
-description: Analyze Neuropixels extracellular recordings end-to-end with SpikeInterface. Covers loading SpikeGLX/Open Ephys/NWB data, preprocessing, drift/motion correction, Kilosort4 (and CPU) spike sorting, quality metrics, and unit curation (threshold-based, model-based UnitRefine, and AI-assisted visual review). Use when working with Neuropixels 1.0/2.0 recordings, spike sorting, or extracellular electrophysiology analysis.
-license: MIT license
-metadata:
-  version: "2.3"
-  skill-author: K-Dense Inc.
-  openclaw:
-    primaryEnv: ANTHROPIC_API_KEY
-    envVars:
-    - name: ANTHROPIC_API_KEY
-      required: false
-      description: For optional Claude API calls.
----
-
-# Neuropixels Data Analysis
-
-## Overview
-
-Toolkit for analyzing Neuropixels high-density neural recordings using current best
-practices from [SpikeInterface](https://spikeinterface.readthedocs.io/), the Allen
-Institute, and the International Brain Laboratory (IBL). It covers the full workflow from
-raw data to publication-ready curated units.
-
-All examples use the real SpikeInterface API (`spikeinterface.full as si`) plus the
-companion curation module (`spikeinterface.curation as sc`). The skill ships runnable
-scripts in `scripts/` and a copy-and-edit template in `assets/` that implement this
-workflow directly on top of SpikeInterface — there is no separate package to install
-beyond the dependencies listed under [Installation](#installation).
-
-## When to Use This Skill
-
-This skill should be used when:
-- Working with Neuropixels recordings (`.ap.bin`, `.lf.bin`, `.meta` files)
-- Loading data from SpikeGLX, Open Ephys, or NWB formats
-- Preprocessing neural recordings (filtering, common reference, bad-channel detection)
-- Detecting and correcting motion/drift
-- Running spike sorting (Kilosort4, SpykingCircus2, Mountainsort5, Tridesclous2)
-- Computing quality metrics (SNR, ISI violations, presence ratio, amplitude cutoff)
-- Curating units (threshold-based, model-based, or AI-assisted)
-- Creating visualizations and exporting to Phy or NWB
-
-## Supported Hardware & Formats
-
-| Probe | Electrodes | Channels | Notes |
-|-------|-----------|----------|-------|
-| Neuropixels 1.0 | 960 | 384 | Use `phase_shift` for ADC correction |
-| Neuropixels 2.0 (single) | 1280 | 384 | Denser geometry |
-| Neuropixels 2.0 (4-shank) | 5120 | 384 | Multi-region recording |
-
-| Format | Extension | Reader |
-|--------|-----------|--------|
-| SpikeGLX | `.ap.bin`, `.lf.bin`, `.meta` | `si.read_spikeglx()` |
-| Open Ephys | `.continuous`, `.oebin` | `si.read_openephys()` |
-| NWB | `.nwb` | `si.read_nwb()` |
-
-## Quick Start
-
-### Import and configure parallel processing
-
-```python
-import spikeinterface.full as si
-
-# Global job kwargs are reused by all parallelizable steps
-si.set_global_job_kwargs(n_jobs=-1, chunk_duration="1s", progress_bar=True)
-```
-
-### Loading data
-
-```python
-# Inspect available streams first
-stream_names, stream_ids = si.get_neo_streams("spikeglx", "/path/to/run_g0/")
-print(stream_names)  # e.g. ['imec0.ap', 'imec0.lf', 'nidq']
-
-# SpikeGLX (most common) — select the AP stream by name
-recording = si.read_spikeglx("/path/to/run_g0/", stream_name="imec0.ap", load_sync_channel=False)
-
-# Open Ephys
-recording = si.read_openephys("/path/to/Record_Node_101/")
-
-# For quick iteration, slice the first 60 s
-fs = recording.get_sampling_frequency()
-recording_sub = recording.frame_slice(0, int(60 * fs))
-```
-
-### Full pipeline (bundled script)
-
-The repository ships an end-to-end pipeline built on SpikeInterface:
-
-```bash
-python scripts/neuropixels_pipeline.py /path/to/spikeglx/data output/ --sorter kilosort4 --curation allen
-```
-
-It performs load → preprocess → drift check → optional motion correction → sorting →
-postprocessing → quality metrics → curation → export. Read the steps below to run them
-interactively or customize the pipeline.
-
-## Standard Analysis Workflow
-
-### 1. Preprocessing
-
-Recommended chain, following the SpikeInterface Neuropixels how-to (IBL-style destriping
-with channel removal + common reference):
-
-```python
-rec = si.highpass_filter(recording, freq_min=400.0)
-bad_channel_ids, channel_labels = si.detect_bad_channels(rec)
-rec = rec.remove_channels(bad_channel_ids)
-rec = si.phase_shift(rec)  # ADC phase correction (Neuropixels 1.0)
-rec = si.common_reference(rec, operator="median", reference="global")
-```
-
-Save the preprocessed recording (Kilosort needs a binary file, and it speeds up reuse):
-
-```python
-rec = rec.save(folder="preprocessed/", format="binary")
-```
-
-### 2. Check and correct drift
-
-Always inspect drift before sorting:
-
-```python
-from spikeinterface.sortingcomponents.peak_detection import detect_peaks
-from spikeinterface.sortingcomponents.peak_localization import localize_peaks
-
-noise_levels = si.get_noise_levels(rec, return_in_uV=False)
-peaks = detect_peaks(rec, method="locally_exclusive", noise_levels=noise_levels,
-                     detect_threshold=5, radius_um=50.0)
-peak_locations = localize_peaks(rec, peaks, method="center_of_mass")
-
-# Visualize the drift raster
-si.plot_drift_raster_map(peaks=peaks, peak_locations=peak_locations,
-                         recording=rec, clim=(-50, 50))
-```
-
-Apply correction if needed (presets: `rigid_fast`, `kilosort_like`,
-`nonrigid_accurate`, `nonrigid_fast_and_accurate`, `dredge`, `dredge_fast`):
-
-```python
-rec_corrected = si.correct_motion(rec, preset="nonrigid_fast_and_accurate", folder="motion/")
-```
-
-### 3. Spike sorting
-
-```python
-# Kilosort4 (recommended, requires a CUDA GPU)
-sorting = si.run_sorter("kilosort4", rec_corrected, folder="ks4_output")
-
-# CPU alternatives (internally developed, no external install)
-sorting = si.run_sorter("spykingcircus2", rec_corrected, folder="sc2_output")
-sorting = si.run_sorter("tridesclous2", rec_corrected, folder="tdc2_output")
-sorting = si.run_sorter("mountainsort5", rec_corrected, folder="ms5_output")
-
-# External sorters can run in containers without local install
-sorting = si.run_sorter("kilosort2_5", rec_corrected, folder="ks25_output", docker_image=True)
-
-print(si.installed_sorters())
-```
-
-> Note: `run_sorter` uses the `folder=` argument. The older `output_folder=` is deprecated.
-
-### 4. Postprocessing
-
-```python
-analyzer = si.create_sorting_analyzer(sorting, rec_corrected, sparse=True,
-                                      format="binary_folder", folder="analyzer/")
-
-analyzer.compute("random_spikes", method="uniform", max_spikes_per_unit=500)
-analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0)
-analyzer.compute("templates", operators=["average", "std"])
-analyzer.compute("noise_levels")
-analyzer.compute("spike_amplitudes")
-analyzer.compute("correlograms", window_ms=50.0, bin_ms=1.0)
-analyzer.compute("unit_locations", method="monopolar_triangulation")
-analyzer.compute("template_similarity")
-
-metric_names = ["firing_rate", "presence_ratio", "snr", "isi_violation", "amplitude_cutoff"]
-analyzer.compute("quality_metrics", metric_names=metric_names)
-metrics = analyzer.get_extension("quality_metrics").get_data()
-```
-
-### 5. Curation by metric thresholds
-
-```python
-# Allen-style query (note: column is isi_violations_ratio)
-query = "(amplitude_cutoff < 0.1) & (isi_violations_ratio < 0.5) & (presence_ratio > 0.9)"
-good_unit_ids = metrics.query(query).index.values
-```
-
-For reusable, multi-threshold logic with `allen` / `ibl` / `strict` presets, use the
-bundled `scripts/compute_metrics.py`. See
-[references/AUTOMATED_CURATION.md](references/AUTOMATED_CURATION.md) for details and the
-Bombcell / UnitMatch tools.
-
-### 6. Model-based curation (UnitRefine)
-
-SpikeInterface can apply pretrained machine-learning classifiers from Hugging Face via the
-`spikeinterface.curation` module. The UnitRefine models were trained on real Neuropixels
-data (V1, SC, ALM):
-
-```python
-import spikeinterface.curation as sc
-
-# 1) noise vs neural
-noise_labels = sc.model_based_label_units(
-    sorting_analyzer=analyzer,
-    repo_id="SpikeInterface/UnitRefine_noise_neural_classifier",
-    trust_model=True,
-)
-neural = analyzer.remove_units(noise_labels[noise_labels["prediction"] == "noise"].index)
-
-# 2) single-unit (sua) vs multi-unit (mua) on the surviving units
-sua_mua_labels = sc.model_based_label_units(
-    sorting_analyzer=neural,
-    repo_id="SpikeInterface/UnitRefine_sua_mua_classifier",
-    trust_model=True,
-)
-```
-
-Each call returns a DataFrame with `prediction` and `probability` (confidence) per unit.
-`trust_model=True` (or an explicit `trusted=[...]` list) is required to load the `.skops`
-model — only load models from sources you trust. Models trained on other brain
-areas/datasets may not transfer; validate against a manually labelled subset.
-
-### 7. AI-assisted curation (for uncertain units)
-
-When running inside an agent such as Cursor or Claude Code, the agent can directly inspect
-waveform/correlogram plots and give an expert read — no API setup required. Generate plots
-and ask the agent to assess isolation quality.
-
-For programmatic vision-model access, **read API keys from the environment — never hardcode
-credentials in analysis scripts** (they leak into version control and logs):
-
-```python
-import os
-from anthropic import Anthropic
-
-client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])  # set this in your shell, not in code
-```
-
-See [references/AI_CURATION.md](references/AI_CURATION.md) for the full pattern (rendering a
-unit summary image, building the prompt, and parsing the response).
-
-### 8. Export results
-
-```python
-# Keep only good units, then export
-analyzer_clean = analyzer.select_units(good_unit_ids, folder="analyzer_clean/", format="binary_folder")
-
-# Phy for manual review
-si.export_to_phy(analyzer_clean, output_folder="phy_export/",
-                 compute_pc_features=True, compute_amplitudes=True)
-
-# Figures report
-si.export_report(analyzer_clean, "report/", format="png")
-
-# NWB
-from spikeinterface.exporters import export_to_nwb
-export_to_nwb(analyzer_clean, "output.nwb")
-
-# Metrics table
-metrics.to_csv("quality_metrics.csv")
-```
-
-## Common Pitfalls and Best Practices
-
-1. **Always check drift** before spike sorting — drift > ~10 μm meaningfully degrades quality.
-2. **Use `phase_shift`** for Neuropixels 1.0 to correct ADC sampling offsets.
-3. **Save the preprocessed recording** with `rec.save(folder=...)` to avoid recomputation (Kilosort also needs a binary file).
-4. **Use a GPU** for Kilosort4 — it is far faster than CPU sorters.
-5. **Review uncertain units** — automated/model-based curation is a starting point, not a verdict.
-6. **Combine approaches** — thresholds for clear cases, model/AI for borderline units.
-7. **Document thresholds and model repo IDs** for reproducibility.
-8. **Export to Phy** for critical experiments — human oversight is valuable.
-
-## Key Parameters to Adjust
-
-### Preprocessing
-- `freq_min`: highpass cutoff (300–400 Hz typical)
-- `detect_bad_channels`: returns `(bad_channel_ids, channel_labels)`
-
-### Motion Correction
-- `preset`: `nonrigid_fast_and_accurate` (balanced), `nonrigid_accurate` (severe drift), `dredge` (state of the art)
-
-### Spike Sorting (Kilosort4)
-- `batch_size`: samples per batch (60000 default)
-- `nblocks`: drift blocks (increase for long, drifty recordings)
-- `Th_universal` / `Th_learned`: detection thresholds (lower = more spikes)
-
-### Quality Metrics
-- `snr`: signal-to-noise cutoff (3–5 typical)
-- `isi_violations_ratio`: refractory violations (0.01–0.5)
-- `presence_ratio`: recording coverage (0.5–0.95)
-
-## Bundled Resources
-
-### scripts/explore_recording.py
-Quick inspection of a recording (streams, channels, duration, bad channels):
-```bash
-python scripts/explore_recording.py /path/to/data
-```
-
-### scripts/preprocess_recording.py
-Automated preprocessing:
-```bash
-python scripts/preprocess_recording.py /path/to/data --output preprocessed/
-```
-
-### scripts/run_sorting.py
-Run spike sorting:
-```bash
-python scripts/run_sorting.py preprocessed/ --sorter kilosort4 --output sorting/
-```
-
-### scripts/compute_metrics.py
-Compute quality metrics and apply curation:
-```bash
-python scripts/compute_metrics.py sorting/ preprocessed/ --output metrics/ --curation allen
-```
-
-### scripts/export_to_phy.py
-Export to Phy for manual curation:
-```bash
-python scripts/export_to_phy.py metrics/analyzer --output phy_export/
-```
-
-### scripts/neuropixels_pipeline.py
-Complete end-to-end pipeline (see [Quick Start](#full-pipeline-bundled-script)).
-
-### assets/analysis_template.py
-Complete, editable analysis template. Copy and customize:
-```bash
-cp assets/analysis_template.py my_analysis.py
-# Edit the PARAMETERS section, then run
-python my_analysis.py
-```
-
-## Detailed Reference Guides
-
-| Topic | Reference |
-|-------|-----------|
-| Full workflow | [references/standard_workflow.md](references/standard_workflow.md) |
-| API reference (SpikeInterface) | [references/api_reference.md](references/api_reference.md) |
-| Plotting guide | [references/plotting_guide.md](references/plotting_guide.md) |
-| Preprocessing | [references/PREPROCESSING.md](references/PREPROCESSING.md) |
-| Spike sorting | [references/SPIKE_SORTING.md](references/SPIKE_SORTING.md) |
-| Motion correction | [references/MOTION_CORRECTION.md](references/MOTION_CORRECTION.md) |
-| Quality metrics | [references/QUALITY_METRICS.md](references/QUALITY_METRICS.md) |
-| Automated & model-based curation | [references/AUTOMATED_CURATION.md](references/AUTOMATED_CURATION.md) |
-| AI-assisted curation | [references/AI_CURATION.md](references/AI_CURATION.md) |
-| Waveform analysis | [references/ANALYSIS.md](references/ANALYSIS.md) |
-
-## Installation
-
-Requires Python ≥ 3.10. Using [uv](https://docs.astral.sh/uv/) is recommended.
-
-```bash
-# Core packages (SpikeInterface bundles the curation/model tooling)
-uv pip install "spikeinterface[full]" probeinterface neo
-
-# Spike sorters
-uv pip install kilosort          # Kilosort4 (CUDA GPU required)
-uv pip install spykingcircus     # SpykingCircus (legacy; SpykingCircus2 ships with SpikeInterface)
-uv pip install mountainsort5     # Mountainsort5 (CPU)
-
-# Model-based curation (UnitRefine) downloads from Hugging Face
-uv pip install "huggingface_hub" skops
-
-# Optional: AI-assisted visual curation
-uv pip install anthropic
-
-# Optional: IBL tools and Bombcell
-uv pip install ibl-neuropixel ibllib bombcell
-```
-
-For reproducible environments, pin versions (current as of 2026-06: `spikeinterface==0.104.3`,
-`kilosort==4.1.7`, `probeinterface==0.3.2`, `neo==0.14.4`). Unpinned installs are fine for
-quick experimentation but should be pinned in production pipelines.
-
-## Project Structure
-
-```
-project/
-├── raw_data/
-│   └── recording_g0/
-│       └── recording_g0_imec0/
-│           ├── recording_g0_t0.imec0.ap.bin
-│           └── recording_g0_t0.imec0.ap.meta
-├── preprocessed/           # Saved preprocessed recording
-├── motion/                 # Motion estimation results
-├── sorting_output/         # Spike sorter output
-├── analyzer/               # SortingAnalyzer (waveforms, metrics)
-├── phy_export/             # For manual curation
-├── ai_curation/            # AI analysis reports
-└── results/
-    ├── quality_metrics.csv
-    ├── curation_labels.json
-    └── output.nwb
-```
-
-## Additional Resources
-
-- **SpikeInterface Docs**: https://spikeinterface.readthedocs.io/
-- **Neuropixels Tutorial**: https://spikeinterface.readthedocs.io/en/stable/how_to/analyze_neuropixels.html
-- **Model-based Curation Tutorial**: https://spikeinterface.readthedocs.io/en/stable/tutorials/curation/plot_1_automated_curation.html
-- **UnitRefine Models (Hugging Face)**: https://huggingface.co/SpikeInterface
-- **Kilosort4 GitHub**: https://github.com/MouseLand/Kilosort
-- **IBL Neuropixel Tools**: https://github.com/int-brain-lab/ibl-neuropixel
-- **Allen Institute ecephys**: https://github.com/AllenInstitute/ecephys_spike_sorting
-- **Bombcell (Automated QC)**: https://github.com/Julie-Fabre/bombcell
-- **Awesome Neuropixels**: https://github.com/Julie-Fabre/awesome_neuropixels

package/skills/nextflow/SKILL.md

@@ -1,195 +0,0 @@
----
-name: nextflow
-description: Build, run, and debug Nextflow data pipelines and nf-core workflows end to end. Use whenever the user mentions Nextflow, nf-core, .nf files, nextflow.config, DSL2, processes/channels/operators, samplesheets, or wants to run a community pipeline (e.g. nf-core/rnaseq, nf-core/sarek), write or test a module/subworkflow with nf-test, configure executors/containers (Docker, Singularity/Apptainer, Conda, Wave), scale a workflow to HPC/SLURM or cloud (AWS Batch, Google Batch, Azure, Kubernetes), or debug a failed/-resume run. Make sure to use this skill for any reproducible scientific/bioinformatics workflow work even if the user does not say the word "Nextflow", and for authoring nf-core-compliant pipelines, modules, configs, and linting.
-license: Apache-2.0
-metadata:
-  version: "1.1"
-  skill-author: K-Dense Inc.
----
-
-# Nextflow
-
-## Overview
-
-Nextflow is a workflow language and runtime for building **reproducible, portable, scalable** data pipelines. It is dominant in bioinformatics but works for any data-heavy computation. nf-core is a community curating production-grade Nextflow pipelines, reusable modules, and the `nf-core` tooling on top of Nextflow.
-
-Key ideas:
-- **Dataflow programming**: pipelines are `process` tasks connected by **channels**. Nextflow infers execution order and parallelism from data dependencies — there is no explicit scheduler to write.
-- **Write once, run anywhere**: the same pipeline runs locally, on HPC (SLURM, SGE, LSF, PBS), and on cloud (AWS Batch, Google Batch, Azure Batch, Kubernetes) by changing config/profiles, not code.
-- **Reproducibility**: per-task containers (Docker/Singularity/Apptainer/Conda/Wave) + `-resume` caching + pinned pipeline revisions.
-- **DSL2** is the modern, required syntax: modular `process`/`workflow`/`include` definitions.
-
-This skill covers both **running** existing pipelines and **developing** your own (Nextflow language + nf-core conventions, testing with nf-test, configuration, and deployment).
-
-## When to Use This Skill
-
-Use this skill when the user wants to:
-- Run an nf-core or custom Nextflow pipeline, or debug a failing/resuming run.
-- Write or modify `.nf` scripts, `nextflow.config`, profiles, or `nextflow_schema.json`.
-- Author or test nf-core-style modules/subworkflows (`main.nf`, `meta.yml`, `tests/`, nf-test).
-- Configure executors, containers, or resources; scale to HPC or cloud.
-- Build a reproducible scientific/bioinformatics workflow (even if "Nextflow" is not named).
-- Understand processes, channels, operators, `take`/`emit`, `publishDir`, `ext.args`, meta maps.
-
-## Setup
-
-Nextflow needs **Bash** and **Java 17 or newer** (17–25 supported). Verify with `java -version`.
-
-```bash
-# Install Nextflow (self-contained launcher)
-curl -s https://get.nextflow.io | bash      # creates ./nextflow
-sudo mv nextflow /usr/local/bin/             # put on PATH
-nextflow info                                # verify
-
-# Or via conda/bioconda (also gets a managed Java)
-conda create -n nf -c bioconda -c conda-forge nextflow nf-core
-```
-
-```bash
-# nf-core tools (Python) for creating/linting/running nf-core assets
-uv pip install nf-core            # or: conda install -c bioconda nf-core
-nf-core --version
-```
-
-Pin the engine for reproducibility: `export NXF_VER=24.10.0` (use an [edge] release only if needed). For air-gapped/HPC, see `references/running-pipelines.md` (offline mode) and `references/configuration.md`.
-
-## Two Modes of Work
-
-Decide which path the user is on — it changes everything:
-
-| Goal | Start here |
-|------|-----------|
-| **Run** an existing pipeline (nf-core or a `.nf` you were given) | `references/running-pipelines.md` |
-| **Develop** a new pipeline / module / subworkflow | `references/language.md` + `references/developing.md` |
-| **Configure / scale** (HPC, cloud, containers, resources) | `references/configuration.md` + `references/containers.md` |
-| **Test** modules/pipelines | `references/testing.md` |
-
-## Quick Start
-
-### Run an nf-core pipeline
-
-Always smoke-test with the bundled `test` profile first; it uses tiny data and proves your environment works.
-
-```bash
-# 1. Confirm setup works (downloads pipeline + tiny test data)
-nextflow run nf-core/rnaseq -profile test,docker --outdir results
-
-# 2. Real run: pin a revision (-r), pick a container engine, pass inputs
-nextflow run nf-core/rnaseq -r 3.14.0 \
-  -profile docker \
-  --input samplesheet.csv \
-  --genome GRCh38 \
-  --outdir results \
-  -resume
-```
-
-- `-profile` (single dash) selects bundled config profiles; **combine** them comma-separated, e.g. `test,docker`. Container/infra profiles (`docker`, `singularity`, `conda`) are mutually exclusive — pick one.
-- `--input`, `--genome`, `--outdir` (double dash) are **pipeline** parameters. nf-core pipelines take a **samplesheet CSV**, not loose files.
-- `-resume` reuses cached results from the last run. `-r <version>` pins a release for reproducibility.
-
-Use `nf-core pipelines launch <name>` for an interactive, schema-validated way to build the command and a `-params-file`. See `references/running-pipelines.md`.
-
-### Write a minimal pipeline
-
-```nextflow
-#!/usr/bin/env nextflow
-
-process SAYHELLO {
-    tag "$greeting"
-    publishDir "results", mode: 'copy'
-
-    input:
-    val greeting
-
-    output:
-    path "${greeting}.txt"
-
-    script:
-    """
-    echo '$greeting world' > ${greeting}.txt
-    """
-}
-
-workflow {
-    channel.of('hello', 'bonjour', 'hola') | SAYHELLO
-}
-```
-
-```bash
-nextflow run main.nf            # add -resume on reruns
-```
-
-The full language (processes, channels, operators, DSL2 workflows with `take`/`main`/`emit`, modules) is in `references/language.md`.
-
-## Core Concepts at a Glance
-
-- **Process**: a unit of work that runs a script (Bash by default). Declares `input:`, `output:`, optional `directives` (resources, container, `publishDir`, `tag`, `errorStrategy`), and a `script:`/`shell:`/`exec:` block. Each task runs in its own isolated work directory (`work/xx/yy…`).
-- **Channel**: the async queues that connect processes. **Queue channels** are consumable streams; **value channels** hold a single reusable value. Created with factories like `channel.of`, `channel.fromPath`, `channel.fromFilePairs`, `channel.value`.
-- **Operator**: transforms/combines channels — `map`, `filter`, `collect`, `groupTuple`, `join`, `combine`, `mix`, `flatten`, `branch`, `multiMap`, `splitCsv`, `view`, `set`.
-- **Workflow**: composes processes. DSL2 workflows can declare `take:` (inputs), `main:` (logic), `emit:` (named outputs) and be `include`d as subworkflows. The unnamed `workflow {}` is the entry point.
-- **Module**: a `.nf` file exposing processes/workflows via `include { NAME } from './path'` (supports `as` aliasing).
-- **Configuration**: `nextflow.config` sets `params`, `process` directives, `executor`, container engines, and named `profiles`. Selectors `withName:`/`withLabel:` target specific processes. See `references/configuration.md`.
-- **meta map** (nf-core): the convention of carrying a metadata map (`[ id:'sample1', single_end:false ]`) alongside files in input/output tuples so samples stay labeled through the pipeline. See `references/developing.md`.
-
-## nf-core tools CLI
-
-nf-core tools (v3+) group subcommands under `pipelines`, `modules`, and `subworkflows`. (Bare forms like `nf-core lint` still work but warn — prefer the grouped form.)
-
-| Command | Purpose |
-|---------|---------|
-| `nf-core pipelines list` | List/search nf-core pipelines (`--json`, keywords) |
-| `nf-core pipelines create` | Scaffold a new pipeline from the nf-core template |
-| `nf-core pipelines launch <name>` | Interactive, schema-driven run command + params file |
-| `nf-core pipelines download <name>` | Download pipeline + containers for offline/HPC use |
-| `nf-core pipelines lint` | Lint a pipeline against nf-core standards (run in repo root) |
-| `nf-core pipelines schema build` | Build/edit `nextflow_schema.json` via web GUI |
-| `nf-core pipelines create-params-file <name>` | Generate a documented YAML params file |
-| `nf-core pipelines bump-version` / `sync` | Bump version / sync with template updates |
-| `nf-core modules list/info/install/update/remove` | Manage modules from nf-core/modules |
-| `nf-core modules create` / `lint` / `test` | Author, lint, and nf-test a module |
-| `nf-core modules patch` / `bump-versions` | Patch an installed module / bump tool versions |
-| `nf-core subworkflows install/create/lint/test` | Same lifecycle for subworkflows |
-
-Full command reference, flags, and examples: `references/nf-core-tools.md`.
-
-## Essential `nextflow` CLI
-
-| Command | Purpose |
-|---------|---------|
-| `nextflow run <pipeline> -profile <p> --outdir <dir>` | Run a pipeline (path, `.nf`, or `user/repo`) |
-| `-resume` | Reuse cached results from prior run |
-| `-r <rev>` | Run a specific git revision/tag/branch |
-| `-params-file params.yml` | Supply parameters from YAML/JSON |
-| `-c custom.config` | Layer in an extra config file |
-| `-with-report -with-trace -with-timeline -with-dag flow.html` | Execution report, trace, timeline, DAG |
-| `-stub-run` | Run `stub:` blocks only (dry-run plumbing) |
-| `nextflow log` | Inspect past runs |
-| `nextflow clean -f -before <run>` | Delete old `work/` data |
-| `nextflow pull / drop / list / info <repo>` | Manage cached remote pipelines |
-
-Config, executors, caching internals, and tracing details: `references/configuration.md`.
-
-## Best Practices (high-value habits)
-
-- **Always `test` first**: `-profile test,docker` (or `singularity`/`conda`) before real data — fast and catches environment problems.
-- **Pin everything**: pipeline revision (`-r`), `NXF_VER`, and tool versions (containers). Don't run `latest` for science you'll publish.
-- **Use `-resume`** and understand caching: a task re-runs if its inputs, script, or container change. See cache-debugging in `references/configuration.md`.
-- **Parameterize via config/params-file**, not hardcoded paths. Keep `params` and profiles in `nextflow.config`.
-- **One container/conda env per process**; never rely on tools installed on the host.
-- **For nf-core dev**: reuse existing modules (`nf-core modules install`) before writing new ones; pass tool flags through `ext.args` (not hardcoded in the script); always include a `stub:` block and nf-test tests; run `nf-core pipelines lint` and `prettier` before committing.
-- **Right-size resources** with `process_low/medium/high` labels and `errorStrategy 'retry'` with dynamic `task.attempt` scaling instead of one giant request.
-- **Write forward-compatible syntax**: the strict-syntax parser becomes the default in Nextflow 26.04. Prefer lowercase `channel.of(...)`, explicit closure params (`{ v -> ... }`), `def` for all variables, and `emit:`-named outputs. Check with `nextflow lint`.
-
-## Reference Files
-
-Read the relevant file when you need depth — each is self-contained:
-
-- `references/language.md` — DSL2 language: processes, directives, channels, operators, workflows (`take`/`emit`), modules, dynamic resources, error handling.
-- `references/configuration.md` — `nextflow.config`, scopes, `profiles`, `withName`/`withLabel` selectors, executors (local/SLURM/cloud), caching/`-resume` internals, tracing/reports, the `nextflow` CLI.
-- `references/containers.md` — Docker, Singularity/Apptainer, Podman, Conda, Wave containers; choosing and enabling engines; common gotchas.
-- `references/running-pipelines.md` — finding/running nf-core pipelines, samplesheets, params files, reference genomes (iGenomes), offline runs, institutional configs, Seqera Platform.
-- `references/nf-core-tools.md` — complete `nf-core` CLI reference (pipelines/modules/subworkflows), flags, and workflows.
-- `references/developing.md` — authoring nf-core pipelines & modules: template layout, module `main.nf`/`meta.yml`, meta maps, `ext.args`/`modules.config`, subworkflows, resource labels, linting & Harshil alignment style.
-- `references/testing.md` — nf-test for modules/subworkflows/pipelines: test structure, assertions, snapshots, tags, running tests, CI.
-
-Official docs: Nextflow https://www.nextflow.io/docs/latest/ · nf-core https://nf-co.re/docs/ · Training https://training.nextflow.io/