@galaxyproject/orbit 0.0.0 → 0.9.0

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/SKILL.md

@@ -0,0 +1,74 @@
+---
+name: validate-galaxy-workflow
+description: "Run terminal gxwf validation on an assembled Galaxy workflow and classify workflow-level failures."
+---
+
+# validate-galaxy-workflow
+
+Follow the procedure below and use the artifact/reference sections as the runtime contract.
+
+## When To Use
+
+- Run terminal gxwf validation on an assembled Galaxy workflow and classify workflow-level failures.
+
+## Inputs
+
+- No upstream artifact inputs declared. See the procedure for user-supplied runtime inputs.
+
+## Outputs
+
+- Write artifact `galaxy-workflow-validation-result` as `galaxy-workflow-validation-result.json`. Format: `json`. Terminal gxwf validation handoff: the exact command run, a pass/fail/not-run status, the classified workflow-level diagnostics, and the residual runtime risks static validation cannot settle.
+
+## Required Tools
+
+- **`gxwf`** (gxwf). `npm install -g '@galaxy-tool-util/cli@^1.8.1'`.
+  Ephemeral run: `npx --yes --package @galaxy-tool-util/cli@1.8.1 gxwf`.
+  Check: `gxwf --help | grep -q draft-validate`.
+  Docs: https://github.com/jmchilton/galaxy-tool-util-ts/tree/main/packages/cli
+
+## Load Upfront
+
+- None declared.
+
+## Load On Demand
+
+- `references/cli/validate.json`: CLI command reference packaged as a sidecar. Validate the assembled gxformat2 workflow before runtime testing. Use when: after all Galaxy steps and workflow tests have been assembled.
+- `references/notes/galaxy-workflow-invocation-failure-reference.md`: Research note copied verbatim into the bundle. Keep static workflow validation findings distinct from Galaxy invocation/runtime failure surfaces. Use when: a workflow passes gxwf validation but still has likely runtime risks around invocation scheduling, outputs, conditionals, or collection population.
+- `references/notes/galaxy-workflow-testability-design.md`: Research note copied verbatim into the bundle. Classify validation or pre-test findings that indicate missing labels, omitted workflow outputs, or untestable checkpoint structure. Use when: terminal validation passes but workflow-level outputs, labels, or collection shapes look likely to break future workflow tests.
+
+## Validation
+
+- None declared.
+
+## Procedure
+
+Validate the assembled Galaxy workflow before runtime testing. The skill owns the terminal validation pass: run gxwf validate, classify workflow-level diagnostics, and route failures back to the responsible authoring phase when possible.
+
+This is separate from advance-galaxy-draft-step (which runs `gxwf draft-validate --concrete` inside the per-step loop) because terminal validation no longer has only one fresh step in scope and should reason over cross-step workflow structure.
+
+### Emit the result
+
+Write `galaxy-workflow-validation-result.json` on every run — clean or failing — so a downstream reviewer can cite what was actually checked rather than re-deriving it:
+
+- `command` — the exact gxwf validate invocation, including flags (prefer `--json`);
+- `workflow_path` — the file validated;
+- `status` — `pass`, `fail`, or `not-run`, with a `not_run_reason` whenever validation could not execute;
+- `diagnostics[]` — one entry per finding: severity, message, the step or field it addresses, and the authoring phase it routes back to;
+- `residual_runtime_risks[]` — for a clean run, the risks static validation cannot settle and the runtime artifact that would prove or disprove each.
+
+A `not-run` status is never reported as a pass.
+
+## Feedback Mode
+
+- Feedback mode is off unless the caller explicitly enables `--feedback` or supplies a feedback-ledger path.
+- When enabled, read `_feedback.md` before doing the work and use its registered `foundry-feedback.ledger.yml` protocol.
+- Preserve harness-owned run and phase state. Append only concrete observations about a canonical Foundry source asset or a related project that this run showed to be at fault; do not put ordinary workflow requirements in this ledger.
+- Before reporting completion, make one explicit pass over the work you just did. Do not ask yourself whether anything was unclear — recall what happened: where you guessed at something the instructions should have settled, needed information this bundle does not carry, hit an instruction that contradicted another or contradicted the artifacts in front of you, used a packaged reference that did not cover your case, or did something the procedure never describes.
+- Append an entry for each such event that clears the protocol's bar. If none do, append nothing and report `no feedback` explicitly. Silence and a clean pass are not the same thing, and nothing downstream can tell them apart unless you say which one it was.
+- Pass the same ledger path to any subagent used for this work, and merge updates serially so one writer cannot overwrite another.
+
+## Runtime Notes
+
+- Do not read Foundry source files at runtime; use only files packaged in this skill bundle and user-supplied artifacts.
+- Preserve declared artifact filenames unless the user or harness supplies explicit paths.
+- Carry unresolved assumptions into the output artifact instead of silently inventing missing source evidence.

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/_provenance.json

@@ -0,0 +1,71 @@
+{
+  "provenance_schema_version": 4,
+  "cast_target": "claude",
+  "mold": {
+    "name": "validate-galaxy-workflow",
+    "path": "content/molds/validate-galaxy-workflow/index.md",
+    "revision": 5,
+    "content_hash": "74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd",
+    "commit": "79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359"
+  },
+  "cast_at": "2026-09-12T19:19:47.237Z",
+  "refs": [
+    {
+      "kind": "cli-command",
+      "mode": "sidecar",
+      "ref": "[[gxwf validate]]",
+      "src": "content/cli/gxwf/validate.md",
+      "dst": "references/cli/validate.json",
+      "used_at": "runtime",
+      "load": "on-demand",
+      "evidence": "hypothesis",
+      "purpose": "Validate the assembled gxformat2 workflow before runtime testing.",
+      "trigger": "After all Galaxy steps and workflow tests have been assembled.",
+      "verification": "Run the cast skill on a complete IWC-derived workflow and confirm terminal validation findings are separated from runtime test failures.",
+      "src_hash": "1bf4b4c50e9987f63e65ad36f6deefe6e399fe009280d4bee8533d6c451acf1e",
+      "dst_hash": "ff10c49819b0f7af13f247ea03e4dd24ba2e5144da3beaa6b1ab96698c7e03b7",
+      "source": "deterministic"
+    },
+    {
+      "kind": "research",
+      "mode": "verbatim",
+      "ref": "[[galaxy-workflow-invocation-failure-reference]]",
+      "src": "content/research/galaxy-workflow-invocation-failure-reference/index.md",
+      "dst": "references/notes/galaxy-workflow-invocation-failure-reference.md",
+      "used_at": "runtime",
+      "load": "on-demand",
+      "evidence": "corpus-observed",
+      "purpose": "Keep static workflow validation findings distinct from Galaxy invocation/runtime failure surfaces.",
+      "trigger": "When a workflow passes gxwf validation but still has likely runtime risks around invocation scheduling, outputs, conditionals, or collection population.",
+      "src_hash": "e8115df6ddca447d47d77cb33becda847cdd19b2cbe9924782bfc180b94bb451",
+      "dst_hash": "e8115df6ddca447d47d77cb33becda847cdd19b2cbe9924782bfc180b94bb451",
+      "source": "deterministic"
+    },
+    {
+      "kind": "research",
+      "mode": "verbatim",
+      "ref": "[[galaxy-workflow-testability-design]]",
+      "src": "content/research/galaxy-workflow-testability-design/index.md",
+      "dst": "references/notes/galaxy-workflow-testability-design.md",
+      "used_at": "runtime",
+      "load": "on-demand",
+      "evidence": "corpus-observed",
+      "purpose": "Classify validation or pre-test findings that indicate missing labels, omitted workflow outputs, or untestable checkpoint structure.",
+      "trigger": "When terminal validation passes but workflow-level outputs, labels, or collection shapes look likely to break future workflow tests.",
+      "src_hash": "81b01801ec3d14810d570dd2d7815da2bb11e41a562af9182c29fd2974f4b855",
+      "dst_hash": "81b01801ec3d14810d570dd2d7815da2bb11e41a562af9182c29fd2974f4b855",
+      "source": "deterministic"
+    }
+  ],
+  "artifacts": {
+    "produces": [
+      {
+        "id": "galaxy-workflow-validation-result",
+        "kind": "json",
+        "default_filename": "galaxy-workflow-validation-result.json",
+        "description": "Terminal gxwf validation handoff: the exact command run, a pass/fail/not-run status, the classified workflow-level diagnostics, and the residual runtime risks static validation cannot settle."
+      }
+    ],
+    "consumes": []
+  }
+}

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/_required_tools.json

@@ -0,0 +1,16 @@
+[
+  {
+    "tool": "gxwf",
+    "origin": "npm",
+    "package": "@galaxy-tool-util/cli",
+    "package_version": "^1.8.1",
+    "invoke": "gxwf",
+    "invoke_fallback": "npx --yes --package @galaxy-tool-util/cli@1.8.1 gxwf",
+    "availability_check": "gxwf --help | grep -q draft-validate",
+    "docs_url": "https://github.com/jmchilton/galaxy-tool-util-ts/tree/main/packages/cli",
+    "source": "implied",
+    "implied_by": [
+      "gxwf validate"
+    ]
+  }
+]

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/_verify.json

@@ -0,0 +1,4 @@
+{
+  "verify_schema_version": 1,
+  "entries": []
+}

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/references/cli/validate.json

@@ -0,0 +1,142 @@
+{
+  "type": "cli-command",
+  "tool": "gxwf",
+  "command": "validate",
+  "summary": "Validate Galaxy workflow structure, tool state, and optional connection compatibility before runtime execution.",
+  "source_path": "content/cli/gxwf/validate.md",
+  "source_revision": 3,
+  "package": "@galaxy-tool-util/cli",
+  "description": "Validate Galaxy workflow files (structure + optional tool state)",
+  "synopsis": "gxwf validate [options] <file>",
+  "args": [
+    {
+      "raw": "file",
+      "name": "file",
+      "required": true,
+      "variadic": false,
+      "description": "Workflow file (.ga, .gxwf.yml)"
+    }
+  ],
+  "options": [
+    {
+      "flags": "--format <fmt>",
+      "name": "format",
+      "description": "Force format: native or format2 (auto-detected by default)",
+      "takesArgument": true,
+      "argumentPlaceholder": "<fmt>",
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--no-tool-state",
+      "name": "toolState",
+      "description": "Skip tool state validation",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": true
+    },
+    {
+      "flags": "--cache-dir <dir>",
+      "name": "cacheDir",
+      "description": "Tool cache directory",
+      "takesArgument": true,
+      "argumentPlaceholder": "<dir>",
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--offline",
+      "name": "offline",
+      "description": "Do not fetch uncached tools; read only the local cache (uncached tools are skipped)",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--galaxy-url <url>",
+      "name": "galaxyUrl",
+      "description": "Galaxy instance to try as a tool source after the ToolShed when fetching uncached tools",
+      "takesArgument": true,
+      "argumentPlaceholder": "<url>",
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--mode <mode>",
+      "name": "mode",
+      "description": "Validation backend: effect (default) or json-schema",
+      "takesArgument": true,
+      "argumentPlaceholder": "<mode>",
+      "optionalArgument": false,
+      "negatable": false,
+      "defaultValue": "effect"
+    },
+    {
+      "flags": "--tool-schema-dir <dir>",
+      "name": "toolSchemaDir",
+      "description": "Directory of pre-exported per-tool JSON Schemas (for offline json-schema mode)",
+      "takesArgument": true,
+      "argumentPlaceholder": "<dir>",
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--json",
+      "name": "json",
+      "description": "Output structured JSON report",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--report-html [file]",
+      "name": "reportHtml",
+      "description": "Write HTML report to file (or stdout if omitted)",
+      "takesArgument": true,
+      "argumentPlaceholder": "[file]",
+      "optionalArgument": true,
+      "negatable": false
+    },
+    {
+      "flags": "--connections",
+      "name": "connections",
+      "description": "Validate connection-type compatibility (collection algebra, map-over)",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--strict",
+      "name": "strict",
+      "description": "Shorthand for --strict-structure --strict-encoding --strict-state",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--strict-structure",
+      "name": "strictStructure",
+      "description": "Reject unknown keys at envelope/step level",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--strict-encoding",
+      "name": "strictEncoding",
+      "description": "Reject JSON-string tool_state and format2 field misuse",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    },
+    {
+      "flags": "--strict-state",
+      "name": "strictState",
+      "description": "Require every tool step to validate; no skips allowed",
+      "takesArgument": false,
+      "optionalArgument": false,
+      "negatable": false
+    }
+  ],
+  "body": "# `gxwf validate`\n\nValidate a Galaxy workflow source file before attempting runtime execution. Use this as the design-time guardrail after step implementation and again after final workflow assembly.\n\n## Output\n\nDefault output is human-readable diagnostics. JSON output should be treated as the preferred cast-skill interface; free-text diagnostics are a fallback for humans.\n\n## Examples\n\n```bash\ngxwf validate workflow.ga\ngxwf validate workflow.gxwf.yml --json\ngxwf validate workflow.gxwf.yml --json --connections --strict\ngxwf validate workflow.ga --mode json-schema --tool-schema-dir ./tool-schemas --json\n```\n\n## Gotchas\n\n- Validation is design-time structure checking. It does not prove that a workflow test will pass under Planemo.\n- Run after each generated Galaxy step when the harness can still attribute failures to the fresh step.\n- Run again after assembly to catch cross-step or workflow-level issues before runtime testing.\n- Prefer `--json` whenever a cast skill or harness needs to classify diagnostics.\n- Use `--connections` when tool cache metadata is available and data-shape compatibility matters, especially around collections and map-over.\n- `--no-tool-state` weakens validation. If used, record why tool metadata was unavailable and rerun without it before final runtime testing."
+}

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/references/notes/galaxy-workflow-invocation-failure-reference.md

@@ -0,0 +1,138 @@
+---
+type: research
+title: "Galaxy workflow invocation failure reference"
+tags:
+  - target/galaxy
+status: draft
+created: 2026-05-02
+revised: 2026-05-02
+revision: 1
+related_notes:
+  - "galaxy-tool-job-failure-reference"
+  - "planemo-workflow-test-architecture"
+  - "galaxy-collection-semantics"
+related_molds:
+  - "run-workflow-test"
+  - "debug-galaxy-workflow-output"
+  - "validate-galaxy-workflow"
+sources:
+  - "https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/schema/invocation.py"
+  - "https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/workflow/run.py"
+  - "https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/workflow/modules.py"
+  - "https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/webapps/galaxy/api/workflows.py"
+summary: "Reference for Galaxy workflow invocation states, messages, failure reasons, and invocation API surfaces."
+---
+
+# Galaxy Workflow Invocation Failure Reference
+
+This note describes workflow-level failure surfaces in Galaxy. It is separate from galaxy-tool-job-failure-reference because invocation state answers whether Galaxy could schedule and drive the workflow, while job state answers whether individual tool jobs succeeded.
+
+## Invocation Versus Job Failure
+
+Important distinction:
+
+- Invocation state says whether Galaxy scheduled, cancelled, failed, or completed the workflow invocation.
+- Job state says whether jobs produced by invocation steps succeeded or failed.
+- Invocation messages explain scheduler/evaluation/cancellation problems.
+- Step states usually describe scheduling progress, not actual job success, unless a legacy serialization mode substitutes job state.
+
+A robust workflow test reference should inspect both invocation APIs and job APIs.
+
+## Invocation States
+
+Galaxy invocation states are defined in `https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/schema/invocation.py` and related model code.
+
+| State | Meaning |
+|---|---|
+| `new` | Newly created invocation. |
+| `requires_materialization` | Deferred or undeferred inputs must materialize before scheduling can continue. |
+| `ready` | Ready for another scheduler iteration. |
+| `scheduled` | Workflow has been scheduled. Older clients may treat this as successful terminal state. |
+| `cancelling` | Scheduler will cancel jobs and subworkflow invocations. |
+| `cancelled` | Cancellation terminal state. |
+| `failed` | Scheduler/evaluation/materialization failure terminal state. |
+| `completed` | Newer completion state after all jobs reach terminal states. |
+
+Version caveat: `completed` may not be uniformly available across all Galaxy/Planemo combinations. Planemo accounts for both `scheduled` and `completed` as success-like invocation states.
+
+## Invocation Message Reasons
+
+Structured invocation messages are the main durable surface for workflow-level failures.
+
+| Reason | Typical meaning |
+|---|---|
+| `dataset_failed` | Dataset input or upstream dataset was failed/unusable. |
+| `collection_failed` | Collection was failed, incompletely populated, or unusable. |
+| `job_failed` | Upstream dependent job failed. |
+| `output_not_found` | Expected step/subworkflow output could not be resolved. |
+| `expression_evaluation_failed` | Workflow expression or condition evaluation failed. |
+| `when_not_boolean` | Conditional `when` expression did not return a boolean. |
+| `unexpected_failure` | Scheduler or evaluation failure without a more specific safe reason. |
+| `workflow_parameter_invalid` | Workflow parameter validation failed. |
+| `step_input_deleted` | Step input dataset/collection was deleted. |
+| `history_deleted` | Invocation history was deleted, causing cancellation. |
+| `user_request` | User/API cancellation. |
+| `cancelled_on_review` | Pause/review step rejected continuation. |
+| `workflow_output_not_found` | Warning: declared workflow output was not found. |
+
+Message records can include affected workflow step id, dependent step id, job id, HDA/HDCA ids, output name, and nested workflow step index path. These fields matter more than free-text messages for reference-quality diagnosis.
+
+## Common Workflow-Level Failure Surfaces
+
+| Surface | Reference signal |
+|---|---|
+| Request-time validation | API returns an error before useful invocation state exists. |
+| Input materialization | Invocation enters `requires_materialization`, then may fail with `dataset_failed`. |
+| Conditional evaluation | Invocation messages include `expression_evaluation_failed` or `when_not_boolean`. |
+| Missing output graph edge | Invocation messages include `output_not_found`. |
+| Deleted/purged inputs | Invocation messages include `step_input_deleted` or `dataset_failed`. |
+| Upstream job failure | Invocation messages or summaries point to `job_failed`; job API has details. |
+| Collection population failure | Invocation messages include `collection_failed`; inspect HDCA and mapped jobs. |
+| Pause/review rejection | Cancellation reason `cancelled_on_review`. |
+| User/history cancellation | Cancellation messages `user_request` or `history_deleted`. |
+| Warning-only missing output | `workflow_output_not_found` warning; may later become Planemo assertion failure. |
+
+## API Surfaces
+
+Useful invocation APIs from `https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/webapps/galaxy/api/workflows.py`:
+
+| API | Use |
+|---|---|
+| `GET /api/invocations/{invocation_id}` | Invocation state, messages, inputs, outputs, and steps. |
+| `GET /api/invocations/{invocation_id}?step_details=true` | Richer step detail, jobs, outputs, and output collections. |
+| `GET /api/invocations/steps/{step_id}` | Detail for one invocation step. |
+| `GET /api/invocations/{invocation_id}/steps/{step_id}` | Step detail scoped to invocation. |
+| `GET /api/invocations/{invocation_id}/jobs_summary` | Job state summary across the invocation. |
+| `GET /api/invocations/{invocation_id}/step_jobs_summary` | Job state summary per workflow invocation step. |
+| `GET /api/invocations/{invocation_id}/completion` | Completion record, if available. |
+| `DELETE /api/invocations/{invocation_id}` | Cancel invocation. |
+| `PUT /api/invocations/{invocation_id}/steps/{step_id}` | Continue or cancel pause/review step. |
+| `GET /api/invocations/{invocation_id}/report` | Invocation report surface. |
+
+Workflow aliases under `/api/workflows/{workflow_id}/invocations/...` also exist.
+
+## Planemo Caveats
+
+Planemo polls invocation and job state, but its user-facing output may not expose all structured invocation messages. Treat Planemo terminal output as a summary; use raw invocation and job APIs when the failure class matters.
+
+Likely noisy boundaries:
+
+- Planemo may show failed job details but omit structured invocation messages.
+- `step_jobs_summary` is useful in Galaxy but not necessarily surfaced directly by Planemo output.
+- Subworkflow paths can be represented in Galaxy message fields but may not be displayed by Planemo.
+- A workflow can be `completed` or `scheduled` while one or more jobs failed; job summaries must still be inspected.
+
+## Durable Reference Use
+
+Use this note when a workflow run fails before or around job scheduling, when outputs are missing without an obvious tool stderr cause, or when Planemo only reports a generic invocation failure.
+
+The goal is not to prescribe a single repair loop. The goal is to preserve which Galaxy API surface proves the failure mode so later Molds can decide whether the defect belongs to data-flow, template, concrete step implementation, workflow test assertions, or runtime environment.
+
+## Verification Gaps
+
+Actual runs should verify:
+
+- Whether Planemo surfaces invocation `messages` for each reason.
+- Whether `completed` and `/completion` are populated in the target Galaxy used by tests.
+- How mapped collections and subworkflow failures appear in `step_jobs_summary`.
+- How warning-only `workflow_output_not_found` interacts with Planemo assertions.

package/extensions/orbit/vendor/skills/validate-galaxy-workflow/references/notes/galaxy-workflow-testability-design.md

@@ -0,0 +1,136 @@
+---
+type: research
+tags:
+  - target/galaxy
+status: draft
+created: 2026-05-03
+revised: 2026-05-06
+revision: 2
+related_notes:
+  - "iwc-workflow-testability-survey"
+  - "iwc-test-data-conventions"
+  - "planemo-asserts-idioms"
+  - "iwc-shortcuts-anti-patterns"
+  - "planemo-workflow-test-architecture"
+  - "implement-galaxy-workflow-test"
+  - "gxformat2-schema"
+  - "gxformat2-workflow-inputs"
+  - "galaxy-datatypes-conf"
+summary: "Design guidance for Galaxy workflow inputs, outputs, and checkpoints that make IWC-style workflow tests possible."
+---
+
+# Galaxy workflow testability design
+
+Use this note when authoring or translating a Galaxy workflow **before** the `-tests.yml` file exists. It covers workflow structure choices that make later IWC-style tests meaningful: labels, promoted checkpoints, collection identifiers, and fixture-compatible inputs.
+
+This is not a `content/patterns/` page. It is cross-cutting design guidance for Molds that need testable Galaxy workflows. Assertion syntax lives in planemo-asserts-idioms. Test YAML fixture shapes live in iwc-test-data-conventions. Accepted shortcut vs smell calls live in iwc-shortcuts-anti-patterns. Corpus evidence trail lives in iwc-workflow-testability-survey.
+
+## 1. Treat labels as API
+
+Workflow input and output labels are not cosmetic. Planemo and IWC tests address workflow inputs and outputs by label, and the survey found exact label matches for every asserted output across 114 matched workflow/test pairs. A generated workflow should therefore pick stable, descriptive labels before test authoring starts.
+
+Rules:
+
+- Label every output that may need a test assertion.
+- Treat input/output renames as breaking changes requiring sibling `-tests.yml` updates.
+- Prefer stable domain names over tool-step defaults or positional names.
+- Do not rely on unlabeled or positional outputs for tests.
+
+Evidence:
+
+- Scanpy exposes outputs such as `Initial Anndata General Info`, `UMAP of louvain`, `Ranked genes with Wilcoxon test`, and `Dotplot of top genes on clusters` (`$IWC_FORMAT2/scRNAseq/scanpy-clustering/Preprocessing-and-Clustering-of-single-cell-RNA-seq-data-with-Scanpy.gxwf.yml:105-147`). The sibling test keys assertions by those exact labels (`$IWC/workflows/scRNAseq/scanpy-clustering/Preprocessing-and-Clustering-of-single-cell-RNA-seq-data-with-Scanpy-tests.yml:27-205`).
+- VGP scaffolding uses punctuation-heavy labels such as `Hi-C duplication stats on scaffolds: Raw`, `Hi-C duplication stats on scaffolds: MultiQc`, and `Merged Alignment stats` (`$IWC_FORMAT2/VGP-assembly-v2/Scaffolding-HiC-VGP8/Scaffolding-HiC-VGP8.gxwf.yml:170-196`). The test asserts those exact labels (`$IWC/workflows/VGP-assembly-v2/Scaffolding-HiC-VGP8/Scaffolding-HiC-VGP8-tests.yml:218-245`).
+
+## 2. Promote assertable checkpoints
+
+IWC workflow tests assert workflow-level outputs. Intermediate step results are invisible unless promoted to top-level workflow outputs. When final reports are weakly assertable, expose intermediate checkpoints that carry deterministic content or structure.
+
+Rules:
+
+- Promote intermediate outputs when they are the best deterministic or structural checkpoint.
+- Prefer a checkpoint table/text/HDF5 object that can prove content over a final plot/report that can only prove existence.
+- Accept some output-list clutter when it buys meaningful tests.
+- Do not promote every intermediate by default; expose checkpoints that map to concrete assertion intent.
+
+Evidence:
+
+- Scanpy exposes 21 workflow outputs and the sibling test asserts all 21. These include initial AnnData summaries, intermediate plots, ranked-gene tables, final AnnData, cluster-count tables, and final plots (`$IWC_FORMAT2/scRNAseq/scanpy-clustering/Preprocessing-and-Clustering-of-single-cell-RNA-seq-data-with-Scanpy.gxwf.yml:105-147`; `$IWC/workflows/scRNAseq/scanpy-clustering/Preprocessing-and-Clustering-of-single-cell-RNA-seq-data-with-Scanpy-tests.yml:27-205`).
+- RNA-seq paired-end exposes mapped reads, stranded/unstranded coverage, abundance estimates, expression tables, counts tables, and MultiQC reports (`$IWC_FORMAT2/transcriptomics/rnaseq-pe/rnaseq-pe.gxwf.yml:90-112`). The sibling test asserts sizes for coverage/read outputs and stronger regex/line checks for expression/count outputs (`$IWC/workflows/transcriptomics/rnaseq-pe/rnaseq-pe-tests.yml:48-97`).
+- MGnify complete exposes 83 workflow outputs; 38 are asserted in the sibling test, including MultiQC reports, FASTA collections, taxonomic classifications, OTU tables, and HDF5/JSON outputs (`$IWC_FORMAT2/amplicon/amplicon-mgnify/mgnify-amplicon-pipeline-v5-complete/mgnify-amplicon-pipeline-v5-complete.gxwf.yml:163-329`; `$IWC/workflows/amplicon/amplicon-mgnify/mgnify-amplicon-pipeline-v5-complete/mgnify-amplicon-pipeline-v5-complete-tests.yml:15-120`).
+
+## 3. Stabilize collection output identifiers
+
+Collection tests key assertions by element identifier. If a workflow emits collections with unstable or opaque identifiers, the test cannot target elements cleanly.
+
+Rules:
+
+- Preserve biologically or sample-meaningful identifiers through map-over and collection reshaping.
+- When generating or relabeling collections, make the identifier derivation deterministic and visible in workflow structure.
+- For nested collections, ensure each axis has predictable identifiers.
+- Quote special identifiers in tests when YAML requires it, but do not simplify identifiers merely for YAML convenience.
+
+Evidence:
+
+- 59 of 115 IWC test files use `element_tests:`, with 227 `element_tests:` blocks in the corpus survey.
+- 10x CellPlex tests nested collection outputs by `subsample`, then inner `matrix`, `barcodes`, and `genes` elements (`$IWC/workflows/scRNAseq/fastq-to-matrix-10x/scrna-seq-fastq-to-matrix-10x-cellplex-tests.yml:82-128`). The workflow exposes the corresponding collection outputs (`$IWC_FORMAT2/scRNAseq/fastq-to-matrix-10x/scrna-seq-fastq-to-matrix-10x-cellplex.gxwf.yml:73-91`).
+- HyPhy collection outputs are tested by generated gene identifiers such as `NC_001477.1|capsid_protein_C|95-394_DENV1` (`$IWC/workflows/comparative_genomics/hyphy/hyphy-core-tests.yml:31-71`). The workflow exposes collection outputs for MEME, PRIME, BUSTED, and FEL (`$IWC_FORMAT2/comparative_genomics/hyphy/hyphy-core.gxwf.yml:26-38`).
+
+## 4. Choose checkpoints by assertion strength
+
+Assertion choice is not only a test-file decision. It should feed back into workflow output design. If the only exposed output is a stochastic plot or binary file, the best possible test may be a weak size check. Exposing a sibling table, report, HDF5 structure, or summary line can make the same workflow much more testable.
+
+Rules:
+
+- For image-heavy workflows, expose data or summary outputs behind the plot when possible.
+- For stochastic statistical outputs, expose structural checkpoints and stable summary tokens.
+- For binary outputs, expose a text/table report or stats file when the tool can produce one.
+- Use planemo-asserts-idioms to select the assertion family after choosing the checkpoint.
+
+Evidence:
+
+- Scanpy image outputs are mostly smoke-tested with `has_size`, `has_image_width`, and `has_image_height`, but the same workflow also exposes AnnData HDF5 keys and cluster-count table checks (`$IWC/workflows/scRNAseq/scanpy-clustering/Preprocessing-and-Clustering-of-single-cell-RNA-seq-data-with-Scanpy-tests.yml:33-205`).
+- RNA-seq paired-end pairs coarse size checks for coverage/mapped-read outputs with stronger regex and exact-line checks for expression/count tables (`$IWC/workflows/transcriptomics/rnaseq-pe/rnaseq-pe-tests.yml:48-97`).
+- VGP scaffolding tests combine stable text checks for scaffold/report stats with size checks for map/alignment artifacts (`$IWC/workflows/VGP-assembly-v2/Scaffolding-HiC-VGP8/Scaffolding-HiC-VGP8-tests.yml:173-245`).
+
+## 5. Design inputs with fixtures in mind
+
+Workflow input labels and types constrain the eventual `job:` block. Fixture planning is not only a test-file activity: it should influence whether the workflow exposes a file input, a collection input, a string data-table input, or a typed parameter.
+
+Rules:
+
+- Choose input labels that will be readable as test `job:` keys.
+- Match workflow input collection types to realistic fixture shapes.
+- Decide early whether reference data should be a portable remote file or a CVMFS/data-table string.
+- Keep typed parameters explicit when tests need to set them (`int`, `boolean`, `string`) rather than burying them in step defaults.
+
+Evidence:
+
+- 10x CellPlex job inputs include `fastq PE collection GEX`, `reference genome`, `gtf`, `cellranger_barcodes_3M-february-2018.txt`, `fastq PE collection CMO`, `sample name and CMO sequence collection`, and `Number of expected cells` (`$IWC/workflows/scRNAseq/fastq-to-matrix-10x/scrna-seq-fastq-to-matrix-10x-cellplex-tests.yml:2-75`). The workflow declares matching collection, data, string, boolean, and int inputs (`$IWC_FORMAT2/scRNAseq/fastq-to-matrix-10x/scrna-seq-fastq-to-matrix-10x-cellplex.gxwf.yml:4-72`).
+- HyPhy accepts a `list` collection of unaligned sequences and preserves accession-like fixture identifiers through to output element assertions (`$IWC/workflows/comparative_genomics/hyphy/hyphy-core-tests.yml:7-30`; `$IWC_FORMAT2/comparative_genomics/hyphy/hyphy-core.gxwf.yml:14-25`).
+
+## 6. Know what a gxformat2 output entry contains
+
+Top-level gxformat2 `outputs:` is the public workflow-output surface. It is separate from per-step `out:` declarations and from step post-job actions such as `change_datatype` or `rename`.
+
+Authoring rules:
+
+- Use `label` as the stable public name tests and users will address.
+- Use `outputSource` to point at the producing step output; do not rely on positional output order.
+- Use `doc` for short user-facing context when the label is not self-explanatory.
+- Keep `type` aligned with the exposed value (`data`, `collection`, or scalar vocabulary from gxformat2-workflow-inputs) when the schema needs it.
+- Apply `change_datatype` at the producing step output when Galaxy needs a stronger datatype than the tool reports; choose values from galaxy-datatypes-conf.
+- Use `rename` only for generated dataset names inside Galaxy histories. It is not a substitute for stable workflow-output `label`.
+- Treat `add_tags` and `remove_tags` as metadata helpers, not as the test API. IWC tests key by labels and collection element identifiers, not tags.
+- Avoid `hide` or `delete_intermediate_datasets` on outputs that are promoted as test checkpoints.
+
+Design inference: a workflow-output promotion decision should pick both the public `outputs:` entry and any producer-side post-job action needed to make that output useful. For example, a synthesized BED checkpoint needs a stable output `label` plus a producer-side `change_datatype: bed`; one without the other is incomplete for a testable workflow.
+
+## Cross-references
+
+- iwc-workflow-testability-survey — corpus survey and distribution rationale.
+- iwc-test-data-conventions — job/input YAML shapes, remote fixtures, hashes, collection fixture syntax.
+- planemo-asserts-idioms — assertion-family choice after an output is exposed.
+- iwc-shortcuts-anti-patterns — accepted shortcut vs smell calls for weak assertions and label coupling.
+- planemo-workflow-test-architecture — Planemo execution, output-problem ambiguity, and structured artifacts.
+- gxformat2-schema — structural vocabulary for top-level workflow outputs and step post-job actions.
+- galaxy-datatypes-conf — valid Galaxy datatype extensions for `format` and `change_datatype` choices.