@galaxyproject/orbit 0.0.0 → 0.9.0

package/extensions/orbit/vendor/skills/skills/udt-authoring/references/schema-reference.md

@@ -0,0 +1,155 @@
+# UserToolSource Schema Reference
+
+The canonical schema for a UDT is the `UserToolSource` Pydantic model in
+`galaxy-tool-util` (`galaxy.tool_util_models`). It is `extra="forbid"`: **any unknown field is
+rejected at parse time.** Field names below are exact.
+
+When you submit via `create_user_tool` / `POST /api/unprivileged_tools`, the server validates this
+full `UserToolSource`, so everything here applies. (Galaxy's *in-app* authoring agent targets a
+slimmed view of the same model that drops the `tests` block to shrink its structured-output schema;
+`tests` is optional either way and usually skipped for a first version.)
+
+## Top-level fields
+
+| Field | Required | Type | Notes |
+|-------|----------|------|-------|
+| `class` | **yes** | string | Must be exactly `GalaxyUserTool`. |
+| `name` | **yes** | string | Display name; **min length 5**; not blank/whitespace. |
+| `container` | **yes** | string | Container image, e.g. `quay.io/biocontainers/seqkit:2.8.2--h9ee0642_0`. A plain **string**, never a dict/list. Not blank. |
+| `shell_command` | **yes** | string | Command with `$(...)` expressions. The field is `shell_command`, not `command`. |
+| `id` | no (recommended) | string | Pattern `^[a-z][a-z0-9_-]*$`, length 3-255. Lowercase, starts with a letter; hyphens and underscores allowed. |
+| `version` | **yes** | string | e.g. `"0.1.0"`. Required on 26.1+ -- a versionless tool fails schema validation with `version: Field required`. Not blank/whitespace; quote it so YAML doesn't read it as a number. |
+| `description` | no | string | Short line in the tool menu. |
+| `inputs` | no (default `[]`) | list | Input parameters (see below). |
+| `outputs` | no (default `[]`) | list | Output definitions (see below). |
+| `requirements` | no | list | `javascript` / `resource` / `container` requirements (see below). |
+| `configfiles` | no | list | Files written before execution; content supports `$(...)`. |
+| `citations` | no | list | `{type: doi|bibtex|reference, content: ...}`. DOIs/bibtex are validated. |
+| `license` | no | string | SPDX identifier, e.g. `MIT`. |
+| `help` | no (convention: yes) | object | `{format, content}`; `format` is `markdown`/`restructuredtext`/`plain_text`. Rendered under the tool form. Always include one -- see SKILL.md. |
+| `tests` | no | list | In-tool tests (inputs/outputs). |
+| `profile`, `edam_operations`, `edam_topics`, `xrefs` | no | -- | Metadata; rarely needed for a first version. |
+
+## Input parameter types
+
+Every input has `name` (required) and may have `label`, `help`, `optional` (default `false`).
+Beyond those, each type supports:
+
+| `type` | Extra supported fields |
+|--------|------------------------|
+| `boolean` | `value` (true/false). **No** `truevalue`/`falsevalue` -- turn it into a flag in `shell_command` with a ternary. |
+| `integer` | `value` (default), `min`, `max`, `validators` (`in_range`) |
+| `float` | `value`, `min`, `max`, `validators` (`in_range`) |
+| `text` | `value`, `area` (bool), `validators` (`length`, `regex`, `empty_field`) |
+| `select` | `options` (static, non-empty), `multiple`, `validators` (`no_options`) |
+| `color` | `value` |
+| `data` | `format` (string or list of datatypes), `multiple` |
+| `data_collection` | `collection_type` (e.g. `list`, `paired`), `format` |
+
+`select` options are a list of `{label, value, selected}`:
+
+```yaml
+- name: mode
+  type: select
+  options:
+    - { label: Fast, value: fast, selected: true }
+    - { label: Accurate, value: accurate, selected: false }
+```
+
+**Structural groups** (recurse into the leaf types above): `conditional` (has `test_parameter`
+and `whens`), `repeat` (has `parameters`, `min`, `max`), `section` (has `parameters`).
+
+**Rejected types** (exist in XML, not in UDT -- parse error): `hidden`, `drill_down`,
+`data_column`, `genomebuild`, `group_tag`, `baseurl`, `rules`, `directory`. **Rejected fields on
+any parameter:** `truevalue`, `falsevalue`, `argument`, `is_dynamic`, `hidden`, `parameter_type`.
+
+**`min`/`max` on a `data` input are rejected** (dropped from the authoring schema in Galaxy 26.1).
+They only ever meant the min/max *number of selected datasets* for a `multiple` input, and the
+runtime rejects them on a single one. Authors reached for `min: 1` to mean "required" -- but a
+`data` input is already required (add `optional: true` to make it *not* required). Use `multiple:
+true` to accept a list; never add `min`/`max` to a data input. (`min`/`max` stay valid on
+`integer`/`float`, where they bound the numeric value.)
+
+## Output types
+
+Every output has `name` and `type`. **A dataset output must declare `from_work_dir` OR
+`discover_datasets`; a collection output must declare `discover_datasets`.** Omitting both is the
+`dynamic_tool.output_unclaimed` error.
+
+**Dataset output** (`type: data`):
+
+| Field | Notes |
+|-------|-------|
+| `from_work_dir` | Relative path to the file your command wrote, e.g. `out.vcf`. |
+| `format` | Datatype, e.g. `vcf`, `tabular`, `bam`. |
+| `format_source` | Copy the format from a named input instead of hardcoding. |
+| `metadata_source` | Copy metadata from a named input. |
+| `label` | History display name. |
+| `discover_datasets` | Alternative to `from_work_dir` for pattern-matched files. |
+
+**Collection output** (`type: collection`): `collection_type` (`list`, `paired`, ...) plus
+`discover_datasets` (required). Discovery is either a `pattern` or `tool_provided_metadata`. On
+Galaxy 26.1+ every discovery field except the pattern has a sensible default, so a minimal entry
+validates -- you only supply `pattern`:
+
+```yaml
+outputs:
+  - name: parts
+    type: collection
+    collection_type: list
+    discover_datasets:
+      - pattern: 'part_(?P<designation>.+)\.txt'   # named groups: designation, ext, dbkey
+```
+
+Add a field only to override its default. The ones you'll actually reach for: `directory` (search
+a subdir instead of the job root), `format`, `visible`, `recurse`, `sort_key` (`filename` | `name` |
+`designation` | `dbkey`), `sort_comp` (`lexical` | `numeric`). That's the common subset, not the
+full list -- `discover_via`, `match_relative_path`, `assign_primary_output` and `sort_reverse` are
+also accepted; consult the model for the rest. The descriptor is `extra="forbid"`, so a mistyped key
+(`patern`) or an XML-only one (`sort_by`, `dbkey`) is rejected, not silently ignored.
+
+> **Server-version note.** The minimal one-field form needs Galaxy 26.1+; older servers required
+> every field spelled out. To stay compatible with an older Galaxy, write the fully-specified form
+> -- `discover_via: pattern` plus `directory`, `format`, `visible`, `recurse`,
+> `match_relative_path`, `assign_primary_output`, `sort_key`, `sort_comp` alongside `pattern` --
+> which still validates everywhere.
+
+**Only `data` and `collection` outputs are usable.** They capture files into the history -- write
+your result to a file and claim it. The scalar output types (`text`/`integer`/`float`/`boolean`) do
+exist in the authoring model, and on 26.1 a named one validates without `hidden`, so `validate.py`
+will pass a tool that declares one. It still won't *run*: the YAML tool parser accepts only `data`
+and `collection` and raises `Unknown output_type [integer] encountered` on anything else. A clean
+validation is not a green light here -- use `data` or `collection`.
+
+## Requirements
+
+Prefer the top-level `container:` field for the image -- it is required, and a `requirements:` entry
+of `type: container` does **not** satisfy it. The `requirements` list is for:
+
+```yaml
+requirements:
+  - type: resource          # request cores/ram; exposes $GALAXY_SLOTS at runtime
+    cores_min: 4
+  - type: javascript        # helper functions usable inside $(...)
+    expression_lib:
+      - |
+        function basename(p) { return p.split('/').pop(); }
+```
+
+`resource` supports `cores_min`/`cores_max`, `ram_min`/`ram_max`, `tmpdir_*`, GPU fields, and
+`timelimit`. A `container` requirement also exists but the top-level `container` field is the
+idiomatic choice.
+
+## The four semantic validators (stable error codes)
+
+Beyond type/shape checks, `UserToolSource` runs these. Error `code`s are stable across releases:
+
+| Code | Rule |
+|------|------|
+| `dynamic_tool.blank_string` | `name` and `version` must not be empty/whitespace. |
+| `dynamic_tool.blank_container` | `container` must not be empty/whitespace. |
+| `dynamic_tool.undeclared_input_ref` | Every `inputs.X` referenced in `shell_command` or a configfile must be a declared input. References resolve against the **top-level** name (a conditional's `inputs.cond.test_parameter` resolves via `cond`). |
+| `dynamic_tool.output_unclaimed` | Each dataset output needs `from_work_dir` or `discover_datasets`; each collection output needs `discover_datasets`. |
+
+`id` violations surface as Pydantic's `string_pattern_mismatch`; a too-short `name` as
+`string_too_short`.

package/extensions/orbit/vendor/skills/skills/udt-authoring/references/templating.md

@@ -0,0 +1,196 @@
+# Templating: `$(...)` ECMAScript, not Cheetah
+
+UDT commands and config files interpolate parameters with **sandboxed ECMAScript (JavaScript)
+expressions inside `$(...)`**. This is the single biggest difference from classic XML tools, which
+use Cheetah (`$param`, `#for#`, `#if#`). None of the Cheetah syntax works here.
+
+The sandbox has **no access to the database, filesystem, or network** -- only the `inputs` object
+and any helpers you add via a `javascript` requirement's `expression_lib`.
+
+## Referencing inputs
+
+| You want | Write |
+|----------|-------|
+| A scalar value (integer/float/text/boolean/select) | `$(inputs.count)` |
+| A single data file's path | `$(inputs.reads.path)` |
+| Multiple data files' paths, space-joined | `$(inputs.datasets.map(d => d.path).join(' '))` |
+| A boolean turned into a flag | `$(inputs.trim ? '--trim' : '')` |
+| A select value chosen into an option | `$(inputs.mode === 'fast' ? '-x' : '-y')` |
+
+A `data` input is an **object** (a CWL-style File object, not a bare path string) -- you almost
+always want `.path`. A `multiple: true` data input is an **array** of those objects; map over it.
+
+```yaml
+shell_command: |
+  cat $(inputs.datasets.map((input) => input.path).join(' ')) > output.txt
+```
+
+## Quoting: scalar values are not auto-quoted
+
+Galaxy substitutes `$(...)` expressions into `shell_command` as **raw text** -- unlike `base_command`
+arguments, they are *not* run through `shlex.quote`. So a `text` or `select` value containing a
+space, quote, `$`, or other shell metacharacter will break or alter the command (it is also an
+injection vector if the value is attacker-controlled).
+
+- **Quote interpolations defensively:** `'$(inputs.reads.path)'`, `'$(inputs.label)'`.
+- **For free-form text you don't control, prefer a configfile** -- write the value into a JSON file
+  and read it in the command, rather than inlining it (see Config files below).
+- **`select` inputs with a fixed option list, and numeric `integer`/`float`, are safe to inline** --
+  their values can't contain surprises.
+- Data `.path` values are Galaxy-generated and normally clean, but quoting them costs nothing.
+
+## Job resources: `$GALAXY_SLOTS` (a shell variable, not an expression)
+
+The number of CPU cores allocated to the job is exposed as the **shell environment variable**
+`$GALAXY_SLOTS` (and memory as `$GALAXY_MEMORY_MB`). Because it has no parentheses, it is **not** a
+`$(...)` expression -- it passes through to the shell and expands at runtime. Request cores with a
+`resource` requirement:
+
+```yaml
+requirements:
+  - type: resource
+    cores_min: 4
+shell_command: |
+  seqkit stats --threads $GALAXY_SLOTS '$(inputs.reads.path)' > stats.tsv
+```
+
+Use `$GALAXY_SLOTS` for "however many cores the job got" -- this is usually what a user means by
+"number of threads." Expose an `integer` input only when the user needs explicit manual control.
+
+**Pass it as the flag's value -- recording it is not using it.** The parallelism flag's argument
+must *be* `$GALAXY_SLOTS` (`--threads $GALAXY_SLOTS`, `-@ $GALAXY_SLOTS`, `-p $GALAXY_SLOTS`).
+Echoing it to a log, or storing it in a variable you never pass through, does nothing for
+parallelism -- a command that prints `$GALAXY_SLOTS` "for debugging" but still runs `--threads 8`
+is hardcoded and ignores the real allocation. Wiring it through and logging it are independent: do
+the first; the second is optional.
+
+```yaml
+# WRONG -- "aware" of the allocation, but the flag is still hardcoded
+shell_command: |
+  echo "cores: $GALAXY_SLOTS"             # recorded...
+  samtools sort -@ 8 -o out.bam in.bam    # ...but ignored -- only 8 threads regardless
+
+# RIGHT -- the flag's value IS the allocation
+shell_command: |
+  samtools sort -@ $GALAXY_SLOTS -o out.bam in.bam
+```
+
+## Escaping: literal shell `$(...)` must be `\$(...)`
+
+Galaxy intercepts `$(...)` as a JavaScript expression. If you need a **literal shell command
+substitution** in your command, escape it so Galaxy passes it through:
+
+```yaml
+shell_command: |
+  echo "Run on \$(date)" > log.txt          # \$(date) -> shell runs `date`
+  echo "$(inputs.message)" >> log.txt        # $(inputs.message) -> Galaxy expands the input
+```
+
+A plain shell variable like `$GALAXY_SLOTS`, `$HOME`, or `$PWD` (no parentheses) needs **no**
+escaping -- only the parenthesized `$(...)` form collides with the expression syntax.
+
+## Helper functions via `expression_lib`
+
+For logic too awkward to inline, add ECMAScript 5.1 functions in a `javascript` requirement and
+call them inside `$(...)`:
+
+```yaml
+requirements:
+  - type: javascript
+    expression_lib:
+      - |
+        function pick(cond, a, b) { return cond ? a : b; }
+shell_command: |
+  tool $(pick(inputs.fast, '--fast', '--accurate')) '$(inputs.reads.path)' > out.txt
+```
+
+`expression_lib` and the `javascript` requirement are accepted by the UDT schema and wired up at
+runtime, but they aren't covered by the official UDT docs -- confirm on your target server before
+relying on them.
+
+## Running a script
+
+You have three ways to run script logic. Pick one and complete it fully -- a `shell_command` that
+*names* a script file with nothing that creates it is the most common broken-but-lint-clean tool
+(the file won't exist at runtime, and lint can't catch it).
+
+**1. One-liner: inline it with `python -c` / `Rscript -e`.** Self-contained, nothing else to
+declare; reference inputs directly:
+
+```yaml
+container: quay.io/biocontainers/pandas:2.1.1
+shell_command: >-
+  python -c "import pandas as pd; d = pd.read_csv('$(inputs.table.path)', sep='\t');
+  d.describe().to_csv('summary.tsv', sep='\t')"
+```
+
+**2. A few lines: embed it with a quoted heredoc.** Reference inputs with `$(inputs.x)` inside.
+Galaxy expands the `$(...)` expressions **before** the shell runs, so a quoted heredoc (`<<'PY'`)
+is correct -- it stops the shell from re-expanding things, while Galaxy has already substituted the
+inputs:
+
+```yaml
+container: python:3.11-slim          # an inline Python script needs only a base Python image
+shell_command: |
+  python <<'PY'
+  import csv
+  table = '$(inputs.table.path)'      # Galaxy substitutes the path before the shell sees this
+  column = '$(inputs.column)'
+  with open(table) as fh:
+      rows = list(csv.DictReader(fh, delimiter='\t'))
+  with open('out.txt', 'w') as out:
+      out.write(str(len(rows)))
+  PY
+inputs:
+  - { name: table, type: data }
+  - { name: column, type: text }
+outputs:
+  - { name: out, type: data, format: txt, from_work_dir: out.txt }
+```
+
+Two reminders inside a heredoc: Galaxy still scans the whole `shell_command` for `$(inputs.X)`
+references (so they count toward `dynamic_tool.undeclared_input_ref`), and a **literal** shell
+command substitution like `$(date)` must still be escaped as `\$(date)`.
+
+## Config files
+
+`configfiles` content uses the same `$(...)` rules and is written to disk before the command runs:
+
+```yaml
+configfiles:
+  - name: params
+    filename: params.json
+    content: |
+      {"min_depth": $(inputs.min_dp), "input": "$(inputs.vcf.path)"}
+    eval_engine: ecmascript
+shell_command: |
+  mytool --config params.json > out.txt
+```
+
+Input references inside a configfile count toward `dynamic_tool.undeclared_input_ref`, same as in
+`shell_command`.
+
+**3. Longer script: put it in a `configfiles` entry and run that file by name.** The file is
+materialized in the working directory at `filename`, so `shell_command` runs it by that name:
+
+```yaml
+configfiles:
+  - filename: script.py
+    content: |
+      import pandas as pd
+      df = pd.read_csv("$(inputs.table.path)", sep="\t")
+      df.describe().to_csv("summary.tsv", sep="\t")
+shell_command: python script.py
+```
+
+> **CRITICAL:** if `shell_command` runs a script by name (`python script.py`), you **must** include
+> a `configfiles` entry whose `filename` is exactly that name. `python script.py` with no configfile
+> that creates it is broken -- the file won't exist at runtime, and neither lint nor `create_user_tool`
+> will catch it. If you don't want a configfile, inline the script with `python -c` (option 1) instead.
+
+**Don't interpolate a free-text input straight into script *source*.** Inlining `$(inputs.name)`
+into Python/R code (heredoc or configfile) has the same problem as inlining it into `shell_command`:
+a quote, backslash, or newline in a `text`/`select` value breaks the script or injects code. Data
+`.path` values are Galaxy-generated and safe to inline; user-controlled text is not. For text, write
+it to a JSON configfile with `$(JSON.stringify(inputs.name))` (which safely encodes any string) and
+read it back as data in the script -- see `examples/08-configfile-script.yml`.

package/extensions/orbit/vendor/skills/skills/udt-authoring/scripts/validate.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+"""Validate and lint a Galaxy User-Defined Tool (UDT) offline.
+
+Runs the same checks Galaxy runs on the server -- the ``UserToolSource`` schema
+(structure + the four semantic validators) and ``lint_user_tool_source`` -- without
+needing a live Galaxy. Use it as a fast pre-submit gate before create_user_tool.
+
+Requirements:
+    pip install 'galaxy-tool-util>=26.1' pyyaml   # 26.1+ for --check-container
+
+Usage:
+    python validate.py my-tool.yml
+    python validate.py -                      # read YAML/JSON from stdin
+    python validate.py my-tool.yml --check-container   # also verify the container tag on quay.io
+
+Exit codes:
+    0  valid and lint-clean (and, with --check-container, the image is built or unverifiable)
+    1  schema validation failed
+    2  lint findings, or --check-container found the image is positively absent
+    3  missing dependency or unreadable input
+"""
+
+import sys
+
+try:
+    import yaml
+except ImportError:
+    print("Missing dependency: pip install pyyaml", file=sys.stderr)
+    sys.exit(3)
+
+try:
+    from galaxy.tool_util_models import UserToolSource, format_validation_errors
+    from galaxy.tool_util.lint import lint_user_tool_source
+except ImportError:
+    print("Missing dependency: pip install 'galaxy-tool-util>=26.1'", file=sys.stderr)
+    sys.exit(3)
+
+from pydantic import ValidationError
+
+
+def _check_container(image):
+    """Check whether ``image`` is actually built on quay.io. Returns an exit code.
+
+    Uses the mulled-recommend helper added in Galaxy 26.1; if the installed
+    galaxy-tool-util predates it, say so and don't fail the run.
+    """
+    try:
+        from galaxy.tool_util.deps.mulled.recommend import biocontainer_tag_built
+    except ImportError:
+        print(
+            "--check-container needs a galaxy-tool-util with mulled-recommend (Galaxy 26.1+); "
+            "skipping the image check.",
+            file=sys.stderr,
+        )
+        return 0
+
+    built = biocontainer_tag_built(image)
+    if built is True:
+        print(f"Container OK: {image} is built on quay.io.")
+        return 0
+    if built is False:
+        print(f"Container MISSING: {image} is not built on quay.io -- the job would fail with 'manifest unknown'.")
+        return 2
+    print(f"Container UNVERIFIED: could not check {image} (non-biocontainer reference or transient error).")
+    return 0
+
+
+def main(argv):
+    args = argv[1:]
+    check_container = "--check-container" in args
+    args = [a for a in args if a != "--check-container"]
+    if len(args) != 1:
+        sys.exit(__doc__)
+
+    source = args[0]
+    try:
+        if source == "-":
+            text = sys.stdin.read()
+        else:
+            with open(source) as fh:
+                text = fh.read()
+    except OSError as exc:
+        print(f"Cannot read {source}: {exc}", file=sys.stderr)
+        return 3
+    data = yaml.safe_load(text)
+
+    try:
+        tool = UserToolSource.model_validate(data)
+    except ValidationError as exc:
+        print("Schema validation FAILED:")
+        for bullet in format_validation_errors(exc):
+            print(f"  - {bullet}")
+        return 1
+
+    findings = lint_user_tool_source(tool)
+    if findings:
+        print("Lint findings (server create would reject these):")
+        for bullet in findings:
+            print(f"  - {bullet}")
+        return 2
+
+    print(f"OK: '{tool.name}' is valid and lint-clean.")
+    if check_container:
+        return _check_container(tool.container)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

package/extensions/orbit/vendor/skills/skills/workflow-reports/README.md

@@ -0,0 +1,71 @@
+# Workflow Reports Skill
+
+Claude Code skill for drafting Galaxy workflow report templates for the Workflow Editor's **Report** tab.
+
+## Quick Start
+
+**From a `.ga` file** (e.g. IWC workflows):
+1. Point Claude at the file: *"Write a report for this workflow: `path/to/workflow.ga`"*
+2. Claude drafts the template and writes it into `report.markdown` in the `.ga` file (with confirmation).
+
+**From a live Galaxy instance:**
+1. Get the workflow JSON: `GET https://<instance>/api/workflows/<id>/download?style=editor`
+2. Pass it to Claude: *"Create a report template for this workflow: [URL or paste JSON]"*
+3. Claude drafts the template — paste it into the Workflow Editor's **Report** tab and save.
+
+## Installation
+
+```bash
+# Personal skills (all projects)
+ln -s /path/to/workflow-reports ~/.claude/skills/workflow-reports
+
+# Project skills (shared via git)
+ln -s /path/to/workflow-reports .claude/skills/workflow-reports
+```
+
+## File Organization
+
+| File | Purpose | When to read |
+|------|---------|--------------|
+| `SKILL.md` | Main skill — fetch logic, output selection, report structure, gotchas | **Start here** |
+| `references/directives.md` | Full Galaxy markdown directive reference | Looking up syntax |
+| `examples/histology-staining.md` | Complete worked example with extracted metadata and final template | Understanding expected output |
+
+## Key Concepts
+
+- Reports are **templates**, not post-hoc analyses — written before knowing if the run will succeed
+- All directives use **block syntax only** — `${galaxy ...}` inline syntax does not work
+- When using the API, use `/download?style=editor` — the regular API endpoint is missing step labels and workflow outputs
+- `.ga` files contain a `report.markdown` field — this is what the skill writes; the default is a minimal placeholder that should be replaced
+
+## Decision Tree
+
+```
+Do you have the workflow JSON or URL?
+│
+├─ Yes → Follow SKILL.md steps 1–4
+│
+└─ No → Get it first:
+         GET https://<instance>/api/workflows/<id>/download?style=editor
+```
+
+```
+What outputs does the workflow have?
+│
+├─ Image outputs → history_dataset_as_image(output="<label>")
+│
+├─ Tabular outputs → history_dataset_as_table(...) + history_dataset_link(...)
+│
+├─ No workflow_outputs marked → flag to user, use invocation_outputs() as fallback
+│
+└─ Multiple images at different stages → prefer output closest to final result
+```
+
+## Common Issues
+
+| Problem | Solution |
+|---------|----------|
+| Step labels are null | Skip `job_parameters` for unlabeled steps |
+| No `workflow_outputs` marked | User must star outputs in the Workflow Editor |
+| Key terminal output not marked | Flag it by name — don't silently use an upstream substitute |
+| Report uses `${galaxy ...}` syntax | Replace with block syntax — inline does not work |