@pikaa-ai/pikaa 0.2.4 → 0.3.0

package/skills/nextflow/references/developing.md

@@ -0,0 +1,301 @@
+# Developing nf-core Pipelines, Modules & Subworkflows
+
+Conventions for building nf-core-compliant components. Sources: https://nf-co.re/docs/developing/ (guides) and https://nf-co.re/docs/specifications/ (the normative MUST/SHOULD spec).
+
+## Table of Contents
+
+- [Pipeline directory layout](#pipeline-directory-layout)
+- [The meta map convention](#the-meta-map-convention)
+- [Anatomy of a module](#anatomy-of-a-module)
+- [meta.yml](#metayml)
+- [ext.args and modules.config](#extargs-and-modulesconfig)
+- [Subworkflows](#subworkflows)
+- [Resource labels and base.config](#resource-labels-and-baseconfig)
+- [Schema and parameters](#schema-and-parameters)
+- [Linting and the Harshil alignment style](#linting-and-the-harshil-alignment-style)
+
+## Pipeline directory layout
+
+`nf-core pipelines create` scaffolds this structure:
+
+```
+my-pipeline/
+├── main.nf                     # entry: includes the main workflow
+├── nextflow.config             # params defaults, profiles, includes conf/*
+├── nextflow_schema.json        # parameter schema (validation + docs + launch GUI)
+├── workflows/
+│   └── mypipeline.nf           # the primary workflow (orchestrates subworkflows)
+├── subworkflows/
+│   ├── local/                  # pipeline-specific subworkflows
+│   └── nf-core/                # installed shared subworkflows
+├── modules/
+│   ├── local/                  # pipeline-specific modules
+│   └── nf-core/                # installed shared modules
+├── conf/
+│   ├── base.config             # default resources keyed by process_* labels
+│   ├── modules.config          # per-process ext.args, publishDir (withName:)
+│   ├── test.config             # tiny test profile inputs
+│   └── igenomes.config         # reference genome keys
+├── assets/                     # samplesheet schema, email templates, MultiQC config
+├── bin/                        # executable helper scripts (on PATH in tasks)
+├── docs/                       # usage.md, output.md, parameter docs
+├── modules.json                # pins installed nf-core modules/subworkflows by SHA
+└── .nf-core.yml                # tools config (lint rules, template features)
+```
+
+`main.nf` includes the workflow in `workflows/`; that workflow includes subworkflows and modules. Parameters are declared in `nextflow.config` + `nextflow_schema.json`; per-process behavior lives in `conf/modules.config`. Keep logic in workflows/modules, not in `main.nf`.
+
+## The meta map convention
+
+nf-core carries a **metadata map** alongside every sample's files in input/output tuples. This keeps samples labeled and lets `groupTuple`/`join` operate on the key as data flows through the pipeline.
+
+```nextflow
+// channel item shape:
+[ [ id:'sample1', single_end:false ], [ sample1_R1.fastq.gz, sample1_R2.fastq.gz ] ]
+```
+
+- **Only two keys are standard**: `meta.id` (unique sample identifier) and `meta.single_end` (paired vs single reads). No new standard keys are being defined — this is deliberate, to keep modules flexible.
+- Inside a **module**, reference only `meta.id`/`meta.single_end` (for `tag`/`prefix`). A module MUST NOT hardcode custom meta keys; pass per-sample values in via `ext.args` from `conf/modules.config` instead (e.g. `ext.args = { "--strandedness ${meta.strandedness}" }`).
+- The first meta in a tuple is named `meta`, the second `meta2`, etc. — not custom names.
+- Outputs re-emit the **same `meta`** so downstream steps stay aligned: `tuple val(meta), path("*.bam")`.
+- Build it from the samplesheet with `splitCsv` + `map` (see `references/language.md`). **Subworkflows** may create/emit new meta keys (document them in `meta.yml`).
+
+Why it matters: decoupling metadata from module logic lets any pipeline name its metadata however it likes while reusing the same module unchanged.
+
+## Anatomy of a module
+
+A module lives in `modules/nf-core/<tool>/<subtool>/` (all lowercase, one command/subcommand per module) with these files:
+
+```
+modules/nf-core/samtools/sort/
+├── environment.yml     # Conda channels + pinned deps
+├── main.nf             # the process
+├── meta.yml            # documented I/O + tools (schema-validated)
+└── tests/
+    ├── main.nf.test    # nf-test tests (required, incl. a stub test)
+    └── main.nf.test.snap
+```
+
+`environment.yml` (pin the version, not the build):
+
+```yaml
+channels:
+  - conda-forge
+  - bioconda
+dependencies:
+  - bioconda::samtools=1.19.2
+```
+
+Annotated `main.nf`:
+
+```nextflow
+process SAMTOOLS_SORT {
+    tag "$meta.id"                        // per-sample label (only meta.id / meta.single_end allowed here)
+    label 'process_medium'                // exactly ONE bundled resource label (conf/base.config)
+
+    conda "${moduleDir}/environment.yml"  // references the file above (NOT inline package strings)
+    container "${ workflow.containerEngine in ['singularity', 'apptainer'] && !task.ext.singularity_pull_docker_container ?
+        'https://depot.galaxyproject.org/singularity/samtools:1.19.2--h50ea8bc_0' :
+        'quay.io/biocontainers/samtools:1.19.2--h50ea8bc_0' }"
+
+    input:
+    tuple val(meta), path(bam)            // meta map is ALWAYS the first tuple element
+
+    output:
+    tuple val(meta), path("*.bam"), emit: bam
+    path "versions.yml",            emit: versions   // version reporting (see note below)
+
+    when:
+    task.ext.when == null || task.ext.when           // frozen line; gate via ext.when in config
+
+    script:
+    def args   = task.ext.args   ?: ''               // tool flags come from config, never hardcoded
+    def prefix = task.ext.prefix ?: "${meta.id}"
+    """
+    samtools sort $args -@ $task.cpus -o ${prefix}.bam $bam
+
+    cat <<-END_VERSIONS > versions.yml
+    "${task.process}":
+        samtools: \$(samtools --version | sed '1!d; s/samtools //')
+    END_VERSIONS
+    """
+
+    stub:                                            // required: every output channel gets ≥1 file
+    def prefix = task.ext.prefix ?: "${meta.id}"
+    """
+    touch ${prefix}.bam
+    cat <<-END_VERSIONS > versions.yml
+    "${task.process}":
+        samtools: \$(samtools --version | sed '1!d; s/samtools //')
+    END_VERSIONS
+    """
+}
+```
+
+Key module rules:
+- **Both** `conda "${moduleDir}/environment.yml"` and `container` are declared (works under any engine). Containers are Biocontainers (`quay.io/biocontainers/...`) / Galaxy depot (`https://depot.galaxyproject.org/singularity/...`) images pinned by version+build.
+- Tool arguments are **not** hardcoded — they come from `task.ext.args` (and `args2`, `args3`, … for piped tools). The output filename prefix comes from `task.ext.prefix`; output names SHOULD be `${prefix}` + suffix.
+- The `when:` line is boilerplate — never edit it; gate execution via `ext.when` in config.
+- Always include a `stub:` block (touch ≥1 file per output channel; for gzip outputs use `echo '' | gzip > x.gz`, not bare `touch`).
+- One tool/subcommand per module; no pipeline-specific logic; no reading `params.*` inside a module.
+
+### Reporting tool versions (current vs legacy)
+
+Two patterns exist — know both:
+- **`versions.yml`** (shown above): a HEREDOC writes a YAML file emitted as `path "versions.yml", emit: versions`. This is what **most installed modules** use today and is the clearest to read.
+- **Topic channels + `eval()`** (what `nf-core modules create` now generates): the tool version is captured declaratively and routed to a `versions` topic, removing the HEREDOC:
+
+```nextflow
+output:
+tuple val("${task.process}"), val('samtools'),
+      eval('samtools --version | sed "1!d; s/samtools //"'),
+      topic: versions, emit: versions_samtools
+```
+
+Either way, the version string MUST start with a digit (strip a leading `v`). Subworkflows/pipelines aggregate versions (mix the `versions` channels or consume the topic) and feed MultiQC.
+
+## meta.yml
+
+Machine-readable description of the module's interface (generated by `nf-core modules create`, validated by `nf-core modules lint`, used by `nf-core modules info` and docs). Current schema: `input` is a nested list (meta and its file are **separate** entries), `output` is a mapping keyed by `emit` name, each file entry carries an `ontologies` list, and each tool has an `identifier` (bio.tools ID where available):
+
+```yaml
+name: "samtools_sort"
+description: Sort a BAM/CRAM/SAM file
+keywords:
+  - sort
+  - bam
+  - genomics
+tools:
+  - samtools:
+      description: Tools for manipulating SAM/BAM/CRAM
+      homepage: http://www.htslib.org/
+      licence: ["MIT"]
+      identifier: biotools:samtools
+input:
+  - - meta:
+        type: map
+        description: "Groovy Map with sample info, e.g. [ id:'test', single_end:false ]"
+    - bam:
+        type: file
+        description: Input BAM/CRAM/SAM file
+        pattern: "*.{bam,cram,sam}"
+        ontologies: []
+output:
+  bam:
+    - - meta:
+          type: map
+          description: Groovy Map with sample info
+      - "*.bam":
+          type: file
+          description: Sorted BAM file
+          pattern: "*.bam"
+          ontologies: []
+  versions:
+    - "versions.yml":
+        type: file
+        description: File containing software versions
+        pattern: "versions.yml"
+        ontologies: []
+authors:
+  - "@author"
+maintainers:
+  - "@maintainer"
+```
+
+## ext.args and modules.config
+
+Per-process configuration (tool flags, output paths, naming) is injected from `conf/modules.config` using `withName:` selectors — never edit the module to change behavior.
+
+```groovy
+// conf/modules.config
+process {
+    withName: 'SAMTOOLS_SORT' {
+        // use a closure so it is evaluated lazily and can read params/meta; .minus("").join(' ') drops empties
+        ext.args   = { [ '-l 9', params.fast ? '-@ 8' : '' ].minus("").join(' ') }
+        ext.prefix = { "${meta.id}.sorted" }         // closures can read meta
+        publishDir = [
+            path: { "${params.outdir}/samtools" },
+            mode: params.publish_dir_mode,
+            pattern: "*.bam"
+        ]
+    }
+    withName: '.*:ALIGN_BWA:BWA_MEM' { ext.args = '-M' }   // target a fully-qualified path
+}
+```
+
+Permitted `ext` keys: `ext.args`/`args2`/`args3`/`argsN` (numbered by tool order in a piped script), `ext.prefix`/`prefix2`, `ext.when`, `ext.use_gpu`, `ext.singularity_pull_docker_container`. Rule of thumb: optional flags → `ext.args`; but any value whose change could break results MUST be a real `input:` channel (documented in `meta.yml`), not an `ext` key. This separation (logic in the module, config in `modules.config`) is what makes nf-core modules reusable across pipelines.
+
+## Subworkflows
+
+A subworkflow chains modules into a reusable unit, in `subworkflows/nf-core/<name>/main.nf` with `take`/`main`/`emit` and a `meta.yml`. It MUST contain ≥2 modules and MUST aggregate/emit a `versions` channel. Name it `<file-type>_<operation(s)>_<tool(s)>`, e.g. `bam_sort_stats_samtools`.
+
+```nextflow
+include { SAMTOOLS_SORT  } from '../../../modules/nf-core/samtools/sort/main'
+include { SAMTOOLS_INDEX } from '../../../modules/nf-core/samtools/index/main'
+
+workflow BAM_SORT_SAMTOOLS {
+    take:
+    ch_bam            // channel: [ val(meta), path(bam) ]
+
+    main:
+    ch_versions = Channel.empty()
+
+    SAMTOOLS_SORT(ch_bam)
+    ch_versions = ch_versions.mix(SAMTOOLS_SORT.out.versions)
+
+    SAMTOOLS_INDEX(SAMTOOLS_SORT.out.bam)
+    ch_versions = ch_versions.mix(SAMTOOLS_INDEX.out.versions)
+
+    emit:
+    bam      = SAMTOOLS_SORT.out.bam        // [ val(meta), path(bam) ]
+    bai      = SAMTOOLS_INDEX.out.bai
+    versions = ch_versions                  // collect versions from all modules
+}
+```
+
+Convention: collect each module's `versions` into one channel and `emit` it; document channel shapes in comments and `meta.yml`.
+
+## Resource labels and base.config
+
+Modules carry a `process_*` label; `conf/base.config` maps labels → resources (with `task.attempt` scaling for retries):
+
+```groovy
+process {
+    cpus   = { 1    * task.attempt }
+    memory = { 6.GB * task.attempt }
+    time   = { 4.h  * task.attempt }
+    errorStrategy = { task.exitStatus in ((130..145) + 104 + (175..177)) ? 'retry' : 'finish' }
+    maxRetries    = 1
+
+    withLabel: process_single      { cpus = { 1 };             memory = { 6.GB  * task.attempt }; time = { 4.h  * task.attempt } }
+    withLabel: process_low         { cpus = { 2 * task.attempt }; memory = { 12.GB * task.attempt }; time = { 4.h  * task.attempt } }
+    withLabel: process_medium      { cpus = { 6 * task.attempt }; memory = { 36.GB * task.attempt }; time = { 8.h  * task.attempt } }
+    withLabel: process_high        { cpus = { 12 * task.attempt }; memory = { 72.GB * task.attempt }; time = { 16.h * task.attempt } }
+    withLabel: process_long        { time = { 20.h * task.attempt } }
+    withLabel: process_high_memory { memory = { 200.GB * task.attempt } }
+    withLabel: error_ignore        { errorStrategy = 'ignore' }
+    withLabel: error_retry         { errorStrategy = 'retry'; maxRetries = 2 }
+}
+```
+
+Attach exactly **one** bundled label (`process_single/low/medium/high`) per module and optionally stack a modifier (`process_long`, `process_high_memory`). Resources auto-scale with `task.attempt` and retry on out-of-resource exit codes. To cap escalation to what the platform allows, set `process.resourceLimits = [ cpus: 16, memory: 128.GB, time: 24.h ]` (the modern replacement for the old `check_max()`/`--max_cpus`/`--max_memory` pattern) in `nextflow.config` or an institutional config.
+
+## Schema and parameters
+
+`nextflow_schema.json` is a JSON-Schema description of every pipeline parameter. It powers CLI/`-params-file` validation (via the `nf-schema` plugin), the `nf-core pipelines launch` GUI, and auto-generated docs. Keep it in sync with `params` in `nextflow.config`:
+
+```bash
+nf-core pipelines schema build      # interactive web editor to add/edit params
+nf-core pipelines schema lint       # CI checks schema ↔ params consistency
+```
+
+The samplesheet itself is validated against `assets/schema_input.json`.
+
+## Linting and the Harshil alignment style
+
+- Run `nf-core pipelines lint` (pipelines) and `nf-core modules lint <tool>` / `nf-core subworkflows lint <name>` (components) before every PR; CI enforces them. Lint exceptions live in `.nf-core.yml`.
+- Code must be free of Nextflow syntax warnings: `NXF_SYNTAX_PARSER=v2 nextflow lint modules/nf-core/<tool>` (strict syntax becomes the default in Nextflow 26.04 — see `references/language.md`). Common fixes: always `def` your variables, use explicit closure params (`{ meta, file -> ... }`) not `it`, avoid `for` loops.
+- Code is formatted with **Prettier** (`prettier -w .`) and follows the **Harshil alignment** style: align assignment `=`, the commas/`emit:`/`optional:` in I/O declarations, and trailing comments into columns for readability. EditorConfig + pre-commit hooks ship in the template; comment `@nf-core-bot fix linting` on a PR to auto-fix.
+- Other expectations: pinned tool versions, `conda`+`container`, a `stub:` block, nf-test tests for every module/subworkflow, and `CHANGELOG.md`/`CITATIONS.md` updates.
+
+See `references/testing.md` for the testing requirements and `references/nf-core-tools.md` for the CLI. Full normative spec: https://nf-co.re/docs/specifications/components/modules/general .

package/skills/nextflow/references/language.md

@@ -0,0 +1,327 @@
+# Nextflow Language (DSL2)
+
+The complete Nextflow scripting language: processes, channels, operators, workflows, and modules. Nextflow is a Groovy-based DSL; DSL2 is the default and only DSL (DSL1 is removed, so `nextflow.enable.dsl=2` is unnecessary). Source: https://www.nextflow.io/docs/latest/
+
+**Current syntax conventions** (a strict-syntax parser, `NXF_SYNTAX_PARSER=v2`, is opt-in in 25.x and becomes the default in **26.04** — write to it now, it also runs on the legacy parser):
+- **`channel.of(...)`** (lowercase namespace) is canonical; `Channel.of(...)` still works but is discouraged.
+- **Explicit closure parameters** (`{ v -> v * 2 }`) are preferred over the implicit `it`.
+- Name process outputs with **`emit:`**; scale resources with **`task.attempt`**.
+- **`output {}` + `publish:`** (full feature in 25.10) is the new declarative way to publish results; `publishDir` still works and remains dominant in nf-core — both are shown below.
+- Avoid removed/deprecated idioms: `for`/`while` loops (use `each`/`collect`), `import`/custom `class` (use functions or `lib/`), `process shell:` (use `script:`), `include … addParams()`.
+
+## Table of Contents
+
+- [Script structure](#script-structure)
+- [Processes](#processes)
+- [Process directives](#process-directives)
+- [Channels](#channels)
+- [Operators](#operators)
+- [Workflows](#workflows)
+- [Modules](#modules)
+- [Dynamic resources and error handling](#dynamic-resources-and-error-handling)
+- [Groovy essentials and gotchas](#groovy-essentials-and-gotchas)
+
+## Script structure
+
+A Nextflow script (`.nf`) mixes process/workflow definitions with Groovy. Every script enables DSL2 implicitly (it is the default since 22.03). A run begins at the **unnamed `workflow {}`** block (the entry workflow).
+
+```nextflow
+#!/usr/bin/env nextflow
+
+params.input = 'data/*.fastq'        // pipeline parameter with a default
+
+process FASTQC { /* ... */ }          // a process definition
+
+workflow {                            // entry point
+    reads = channel.fromPath(params.input)
+    FASTQC(reads)
+}
+```
+
+Run with `nextflow run main.nf --input 'data/*.fastq'`. Parameters declared as `params.x` are overridable on the CLI (`--x`), in `nextflow.config`, or in a `-params-file`.
+
+## Processes
+
+A `process` defines a task: a (usually Bash) script executed in its own isolated **work directory**. Nextflow stages declared inputs in and declared outputs out, so processes never read/write each other's files directly — they communicate only through channels.
+
+```nextflow
+process ALIGN {
+    tag    "$meta.id"                 // label shown in the log/trace
+    label  'process_high'             // maps to resources in config
+    container 'quay.io/biocontainers/bwa:0.7.17--hed695b0_7'
+    publishDir "${params.outdir}/bam", mode: 'copy'
+
+    input:
+    tuple val(meta), path(reads)      // a sample: metadata map + file(s)
+    path  index                       // a shared reference (value channel)
+
+    output:
+    tuple val(meta), path("*.bam"), emit: bam
+    path  "versions.yml",           emit: versions
+
+    when:
+    meta.run_alignment != false       // skip task if false
+
+    script:
+    def prefix = task.ext.prefix ?: meta.id
+    def args   = task.ext.args  ?: ''   // extra flags injected from config
+    """
+    bwa mem $args -t $task.cpus $index ${reads} | samtools sort -o ${prefix}.bam
+
+    cat <<-END_VERSIONS > versions.yml
+    "${task.process}":
+        bwa: \$(bwa 2>&1 | sed -n 's/Version: //p')
+    END_VERSIONS
+    """
+
+    stub:
+    """
+    touch ${meta.id}.bam
+    touch versions.yml
+    """
+}
+```
+
+### Inputs
+
+Declared one per line under `input:`. Each input consumes one item from a channel.
+
+| Qualifier | Meaning |
+|-----------|---------|
+| `val(x)` | Any value (string, number, map) |
+| `path(f)` | A file/dir; staged into the work dir. Use `path` (not the old `file`) |
+| `tuple val(meta), path(reads)` | A composite item — the nf-core standard: a meta map + files |
+| `env(NAME)` | Value exposed as an environment variable |
+| `stdin` | Feed the channel item to the script's stdin |
+| `each x` | Repeat the process once per value in `x` (combinatorial) |
+
+Inputs are positional and matched to channels in call order: `ALIGN(reads_ch, index_ch)`.
+
+### Outputs
+
+Declared under `output:`; each becomes a channel. Use `emit:` to name outputs so callers can reference `ALIGN.out.bam` instead of positional `ALIGN.out[0]`.
+
+| Form | Meaning |
+|------|---------|
+| `path "*.bam"` | Files matched by glob in the work dir after the script runs |
+| `tuple val(meta), path("*.bam")` | Carry metadata forward with the file |
+| `val x` | Emit a value computed in the process |
+| `stdout` | Capture the script's stdout as the output |
+| `eval('cmd')` | Capture the stdout of a command run in the task env (24.04+) — used for tool versions |
+| `path "out", emit: name` | Named output channel (reference as `PROC.out.name`) |
+| `..., topic: versions` | Also route this output to a named topic channel |
+| `optional true` | Output may be absent without erroring |
+
+### Script, shell, exec
+
+- **`script:`** (default) — a multi-line string run as Bash. Nextflow variables interpolate with `$var`/`${expr}`; escape shell vars you do NOT want Nextflow to touch as `\$var`.
+- **`shell:`** — like script but Nextflow vars use `!{var}`, leaving `$` for the shell. **Deprecated as of 25.04** — use `script:` with `\$` to escape shell vars.
+- **`exec:`** — native Groovy, no external process (for in-line computation).
+- A process can run any interpreter via a shebang (e.g. `#!/usr/bin/env python`).
+
+```nextflow
+process PY {
+    input: val x
+    output: stdout
+    script:
+    """
+    #!/usr/bin/env python
+    print(${x} ** 2)
+    """
+}
+```
+
+### Conditional execution
+
+- `when:` — skip the task when the expression is false (prefer filtering channels upstream when possible).
+- A `script:` can branch with normal Groovy `if/else` returning different command strings.
+- `stub:` — an alternate minimal script run with `-stub-run` to test pipeline wiring without the real tool. nf-core requires stubs.
+
+## Process directives
+
+Set inside a process (or globally via config). Most-used:
+
+| Directive | Purpose |
+|-----------|---------|
+| `cpus`, `memory`, `time`, `disk` | Resource requests (e.g. `memory '8.GB'`, `time '2.h'`) |
+| `container` | Container image for this process |
+| `conda` | Conda packages/env for this process |
+| `publishDir` | Copy/link outputs to a results dir (`mode: 'copy'|'symlink'|'link'`) |
+| `tag` | Human-readable label per task in logs/trace |
+| `label` | Group processes (target with `withLabel:` in config) |
+| `errorStrategy` | `'terminate'` (default), `'ignore'`, `'retry'`, `'finish'` |
+| `maxRetries`, `maxErrors` | Retry limits |
+| `cache` | `true`/`'lenient'`/`'deep'`/`false` — caching behavior |
+| `scratch` | Run in node-local scratch then stage out |
+| `stageInMode`/`stageOutMode` | `'symlink'`/`'copy'`/`'link'` staging |
+| `beforeScript`/`afterScript` | Commands wrapping the task script |
+| `accelerator` | GPU request (e.g. `accelerator 1, type: 'nvidia-tesla-v100'`) |
+| `array` | Submit as a job array (HPC/cloud), e.g. `array 100` |
+| `ext` | Free-form map (`ext.args`, `ext.prefix`) injected from config |
+| `pod` | Kubernetes pod options |
+| `module` | Load an HPC environment module |
+| `maxForks` | Cap parallel tasks for this process |
+
+Access the resolved values at runtime via `task.*` (`task.cpus`, `task.memory`, `task.attempt`, `task.process`, `task.ext.args`).
+
+## Channels
+
+Channels are the asynchronous queues connecting processes. Two kinds:
+
+- **Queue channel**: an ordered, *consumable* stream of items. Produced by most factories/operators and by process outputs. Can be consumed once.
+- **Value channel** (singleton): holds one value that can be read an unlimited number of times. Created by `channel.value()`, by operators like `collect`/`first`, or implicitly from a single value. A process input bound to a value channel is reused for every task.
+
+### Channel factories
+
+```nextflow
+channel.of(1, 2, 3)                                  // emit given values (ranges expand: 1..23)
+channel.fromList([1, 2, 3])                          // emit list items
+channel.value('ref.fa')                              // singleton value channel
+channel.fromPath('data/*.bam')                       // one item per matching file
+channel.fromPath('data/**.fastq', checkIfExists: true)   // also: arity:'1', type:'file', hidden:true
+channel.fromFilePairs('data/*_{1,2}.fastq.gz')       // -> [id, [r1, r2]] for paired reads
+channel.topic('versions')                            // collect values emitted to a named topic (24.04+)
+channel.empty()                                      // emits nothing
+```
+
+> `channel.fromSRA(...)` exists but is deprecated as of 26.04 — prefer a **samplesheet** (`splitCsv`) over fetching reads by accession.
+
+`fromFilePairs` is the idiomatic way to group paired-end reads; it yields `[ sampleId, [read1, read2] ]`, which you typically `map` into the nf-core `[ meta, [reads] ]` shape.
+
+## Operators
+
+Operators transform/combine channels. Chain with `.`; the dataflow graph is built from these connections.
+
+| Operator | Purpose |
+|----------|---------|
+| `map { }` | Transform each item |
+| `filter { }` | Keep items matching a condition/type/regex |
+| `flatten` | Flatten nested emissions into individual items |
+| `collect` | Gather all items into a single list (→ value channel) |
+| `toList` / `toSortedList` | Collect into one (sorted) list |
+| `groupTuple` | Group tuples by key (e.g. by `meta`) — often needs `groupTuple(by: 0)` |
+| `join` | Inner-join two channels by a matching key |
+| `combine` | Cartesian product (optionally `by:` a key) |
+| `cross` | Combine matching keyed items |
+| `mix` | Merge multiple channels into one stream |
+| `concat` | Emit one channel fully, then the next, in order |
+| `branch { }` | Route items into multiple named sub-channels by condition |
+| `multiMap { }` | Emit to several channels from one pass |
+| `splitCsv` / `splitText` / `splitFasta` / `splitFastq` | Split file contents into items |
+| `collectFile` | Write items into one or more files |
+| `unique` / `distinct` | De-duplicate |
+| `first` / `last` / `take` / `until` | Select subsets |
+| `set { ch }` | Name the resulting channel (alternative to `ch =`) |
+| `view { }` | Print items for debugging (returns the channel unchanged) |
+| `ifEmpty` | Provide a default if the channel is empty |
+| `dump(tag:'x')` | Debug-print when run with `-dump-channels x` |
+
+```nextflow
+// Build the nf-core [meta, reads] shape from a samplesheet
+channel
+    .fromPath(params.input)
+    .splitCsv(header: true)
+    .map { row -> tuple([id: row.sample, single_end: row.fastq_2 ? false : true],
+                        row.fastq_2 ? [file(row.fastq_1), file(row.fastq_2)] : [file(row.fastq_1)]) }
+    .set { reads_ch }
+
+// Group per-sample results, then join two channels by meta
+counts.groupTuple()
+       .join(metadata)          // matches on the first (key) element
+       .view()
+```
+
+## Workflows
+
+A `workflow` composes processes and other workflows. The **unnamed** workflow is the entry point. **Named** workflows are reusable (sub)workflows.
+
+```nextflow
+workflow RNASEQ {
+    take:                       // typed inputs (channels)
+    reads
+    index
+
+    main:                       // pipeline logic
+    FASTQC(reads)
+    ALIGN(reads, index)
+    QUANT(ALIGN.out.bam)
+
+    emit:                       // named outputs
+    bam    = ALIGN.out.bam
+    counts = QUANT.out.counts
+    versions = FASTQC.out.versions.mix(ALIGN.out.versions)
+}
+
+workflow {                      // entry: wire inputs and call the named workflow
+    reads = channel.fromFilePairs(params.reads)
+    index = channel.value(file(params.index))
+    RNASEQ(reads, index)
+    RNASEQ.out.counts.view()
+}
+```
+
+- Call a process/workflow like a function: `ALIGN(reads, index)`. Outputs are on `.out` (use `emit:` names: `ALIGN.out.bam`).
+- A process can only be **called once** per workflow; to reuse it, `include` it again under an alias.
+- Pipe syntax works for simple chains: `reads | FASTQC`.
+- **Declarative outputs (25.10+)**: assign channels in the entry workflow's `publish:` section and describe them in a top-level `output {}` block — the recommended replacement for the `publishDir` directive (which still works and dominates nf-core):
+
+```nextflow
+workflow {
+    main:
+    ch = ANALYZE(input)
+    publish:
+    results = ch                       // name the published channel
+}
+output {
+    results { path 'analysis' }        // -> <outputDir>/analysis (default outputDir: results/)
+}
+```
+
+## Modules
+
+Modules are `.nf` files whose processes/workflows are imported with `include`. This is the basis of nf-core's reusable components.
+
+```nextflow
+include { FASTQC }                     from './modules/fastqc/main.nf'
+include { ALIGN as ALIGN_TUMOR;
+          ALIGN as ALIGN_NORMAL }      from './modules/align/main.nf'
+include { RNASEQ }                     from './subworkflows/rnaseq.nf'
+```
+
+- `as` aliases let you include the same component multiple times.
+- Includes are resolved relative to the including file; `.nf` extension optional.
+- Params should be passed explicitly (as inputs), not read globally inside modules — this keeps modules portable (an nf-core requirement).
+
+## Dynamic resources and error handling
+
+Make pipelines robust by retrying failures with more resources instead of over-provisioning everything. `task.attempt` increments on each retry.
+
+```nextflow
+process BIG_JOB {
+    label 'process_high'
+    cpus   { 4 * task.attempt }
+    memory { 8.GB * task.attempt }
+    time   { 4.h * task.attempt }
+
+    errorStrategy { task.exitStatus in [137, 140, 143] ? 'retry' : 'terminate' }
+    maxRetries 3
+    // ...
+}
+```
+
+- Exit codes 137/140/143 typically mean out-of-memory/walltime kills — retry with more resources.
+- `errorStrategy 'ignore'` lets the pipeline continue past a failed task; `'finish'` stops launching new tasks but lets running ones complete.
+- In nf-core, resource scaling lives in `conf/base.config` keyed on `process_*` labels (see `references/developing.md`).
+
+## Groovy essentials and gotchas
+
+- Strings: single-quoted are literal; double-quoted interpolate (`"${x}"`). In `script:` blocks, escape shell variables as `\$VAR`.
+- Define helper values with `def` inside `script:`/closures to avoid leaking globals.
+- Maps use Groovy syntax: `[ id: 'x', single_end: false ]`; access as `meta.id`.
+- **Common gotchas**:
+  - Re-using a consumed **queue** channel yields nothing — use a **value** channel (or `collect`) for things consumed by many tasks (like a reference index).
+  - `groupTuple` may emit before all items arrive unless sizes are known; provide `size:` or use `groupTuple(by:)` carefully.
+  - A process called twice without aliasing is an error; `include ... as`.
+  - Globs in `output:` match the **work directory**, not `publishDir`.
+  - Prefer filtering channels over `when:` for clarity and caching.
+- **Strict syntax / language server**: recent Nextflow ships a VS Code extension + `nextflow lint` and a stricter parser; nf-core is migrating pipelines to it. Keep scripts to documented DSL2 constructs and avoid deprecated DSL1 idioms (`Channel.create()`, `.into{}` overuse, top-level `file()` for inputs).