@hassangameryt/freeflow 0.8.0 → 0.9.0

package/skills/evaluate-skill/SKILL.md

@@ -1,188 +0,0 @@
----
-name: evaluate-skill
-description: Use when evaluating or comparing agent-skill behavior.
----
-
-# Evaluate Skill
-
-Determine whether an exact skill package changes agent behavior as intended under declared pressure. Evidence outranks confident prose.
-
-A **group** asks one behavioral question through exactly two variants:
-
-- `baseline`: no target skill for a new skill, or one exact immutable previous package for a revision;
-- `candidate`: one exact new or updated package.
-
-The **subject** performs the declared task under one variant. The **evaluator** isolates subjects and preserves canonical evidence. The **deterministic grader** derives fixed mechanical facts after the run is saved. A **suite** runs an ordered set of independent groups.
-
-Evaluate Skill does not author or revise skills, decide semantic quality, claim readiness, or choose publication. [Write Skill](../write-skill/SKILL.md) owns package preservation and revision. The active agent or user judges unresolved meaning; the user decides whether to revise, use, publish, or reject the skill.
-
-The **Rules** below are binding; if one cannot be met, stop and say why. Everything else is **judgment**: follow its stated goal, and adapt when a different choice serves that goal better.
-
-## Rules
-
-- Never mutate canonical evidence or erase an unfavorable result. Preserve partial and infrastructure evidence without converting unavailable behavior into pass or fail.
-- Never fabricate behavioral evidence. Missing evidence stays unavailable.
-- Compare a revision only against an exact immutable baseline. If none exists, return to Write Skill to preserve the current package; never create a baseline by editing the package being evaluated.
-- Keep deterministic criteria and review questions out of subject prompts and declared subject context.
-- Declare every subject resource. Ambient installation or context is not declared composition.
-- Claim only what the evidence class observed; one class never proves another.
-- Never edit the skill from Evaluate Skill. Return one measured revision target to Write Skill.
-- After a behavioral change, rerun the complete fixed group with both variants. Never reuse an earlier run as the fresh counterpart or rerun only the preferred variant.
-- Advisory review and semantic judgment never enter canonical run or grade state, and neither establishes readiness.
-- Never invoke archived evaluators.
-
-## Start From Existing Evidence
-
-Goal: answer the question with the least new evidence that can answer it.
-
-Before proposing a run or revision:
-
-- inspect adequate saved evidence, through views and raw artifact reads when a complete result already answers the question;
-- reuse an existing group unchanged when it still preserves the question, pressure, variants, and fixed criteria;
-- for a draft-only request, use Write Skill instead.
-
-Do not rerun merely to demonstrate process.
-
-## Name One Behavioral Question
-
-Goal: one question, observed at the boundary where the claim lives.
-
-Name the exact claim and observing mechanism before choosing the group shape. Evidence classes include:
-
-- **Description activation:** did a natural prompt cause the exact target to be read, and when?
-- **First-read body behavior:** did explicit body delivery guide the subject using only guaranteed context?
-- **Nearby behavior:** did a close case avoid activation or load without hijacking the task?
-- **Dependency composition:** did exact ordered skills, resources, and context work together?
-- **Retained use:** did guidance remain useful on later declared turns without redelivery?
-- **Artifact outcome:** did files, structured state, events, and responses match fixed criteria?
-- **Cross-host behavior:** did the same behavioral question hold on every named host? The evaluator currently runs only native Pi hosts.
-
-```text
-successful read
-≠ behavioral compliance
-
-explicit body delivery
-≠ natural activation
-
-one turn
-≠ retained use
-
-ambient installation
-≠ declared composition
-
-correct behavior
-≠ proof that the target caused it
-```
-
-Test a skill's Rules and its judgment differently. A Rule needs pressure to break it; it passes only if it holds. Judgment needs a case where following the default literally is the worse choice; it passes on outcome, so a candidate that only obeys cannot pass.
-
-Read [Evaluation Design](references/evaluation-design.md) before choosing or materially changing the question, evidence class, group shape, pressure, or baseline/candidate boundary.
-
-## Design The Smallest Valid Comparison
-
-Goal: the only difference between variants is the target, and the pressure is strong enough to show it.
-
-Before subject output exists:
-
-1. fix one behavioral question;
-2. fix deterministic expectations and review questions;
-3. preserve the earliest natural prompt or strongest pressure capable of distinguishing behavior;
-4. declare exact baseline and candidate identities;
-5. keep prompt or turns, fixture, tools, model, thinking, other skills, context, runtime, and criteria fixed except for the target difference;
-6. run one group or an ordered suite serially;
-7. inspect canonical evidence and deterministic facts before judging meaning;
-8. classify the failed boundary before proposing revision.
-
-For a new skill, the baseline has no target skill. For a revision, the baseline is the exact previous package. For a description-only revision, keep body, references, scripts, and other resources byte-identical.
-
-If both variants pass, the pressure may be weak or the baseline already sufficient. If both fail, distinguish skill, fixture, dependency, environment, host, isolation, and criterion failures before revising instructions.
-
-Inline examples in the skill teach the subject and are part of the candidate intervention; they are not independent evaluation evidence.
-
-## Declare The Complete Environment
-
-Use the exact [Evaluation Definition Schema](references/evaluation-definition-schema.md) when authoring or checking group and suite JSON. Declare:
-
-- prompt or ordered turns;
-- fixture;
-- exact ordered skill packages and target index;
-- working-tree or Git source identity;
-- context resources;
-- tools and any declared commands;
-- model and thinking;
-- runtime host, prompt mode, and extension bundles when needed;
-- literal and inherited environment sources;
-- deterministic expectations and comparison IDs;
-- review questions.
-
-## Run And Preserve Canonical Evidence
-
-Before operating the evaluator, resolving paths, interpreting states, or reasoning about host execution, read [Execution and Evidence](references/execution-and-evidence.md).
-
-Use the canonical [skill evaluator](scripts/skill-eval.mjs) from the definition root:
-
-```text
-node <evaluate-skill-directory>/scripts/skill-eval.mjs run <suite-or-group-path> [--group <id-or-position>] [--variant baseline|candidate] [--trials <n>] [--max-cost <total>]
-node <evaluate-skill-directory>/scripts/skill-eval.mjs view <result-id-or-directory> [--group <id-or-position>] [--variant baseline|candidate]
-node <evaluate-skill-directory>/scripts/skill-eval.mjs review <result-id-or-directory> --model <provider/model> [--group <id-or-position>] [--thinking <level>]
-```
-
-One run per variant cannot separate a behavioral difference from sampling noise. Use `--trials <n>` before claiming a difference; each trial is an independent complete result, and the aggregate only counts their deterministic grades. Use `--max-cost` when trials or suites could spend more than intended.
-
-With no selectors, run or view every suite group and both variants. Selectors narrow execution or viewing scope; they do not modify definitions.
-
-Canonical results live under `<definition-root>/.skill-eval/runs/<result-id>`. A view is a projection of saved evidence, not its canonical source; use ordinary file tools when the exact run, event, transcript, response, stderr, workspace, definition, grade, context-observation, or group artifact matters.
-
-## Derive Facts Without Inventing Meaning
-
-Deterministic grades are appended only after canonical run evidence exists. They can establish exact facts such as:
-
-- skill or resource reads;
-- file and path state;
-- changed paths;
-- file or response text;
-- JSON validity or values;
-- tool-call outcomes and argument predicates;
-- context text at a selected surface, request, and turn;
-- factual baseline-to-candidate transitions.
-
-A failed check remains ordinary behavioral evidence; it does not turn a valid subject run into infrastructure failure. Malformed expectations or grading failures produce separate `grade-error` evidence and do not rewrite run evidence.
-
-A transition such as `fail-to-pass` is a factual comparison. It does not mean bad-to-good, not-ready-to-ready, or rejected-to-promotable.
-
-## Review Meaning After Evidence Exists
-
-After canonical evidence and deterministic grades exist, read [Review and Revision](references/review-and-revision.md) when evidence conflicts, semantic meaning remains unresolved, the failed boundary must be classified, a revision target must be selected, or rerun scope is uncertain.
-
-The evaluator never grades meaning automatically. `review` records an optional advisory answer to each review question from a separate model that sees the two runs unlabeled, in `semantic-grade.json`.
-
-The active agent or user may judge reasoning quality, authority preservation, architectural fitness, recommendation quality, or semantic completeness. Keep that judgment, cited evidence, uncertainty, and limits outside canonical run and grade state.
-
-## Return Revision To The Author
-
-When evidence supports a candidate change:
-
-1. identify the exact failed boundary;
-2. preserve the canonical result path and relevant evidence;
-3. state one measured revision target, keeping unrelated instructions and criteria stable;
-4. return revision to Write Skill;
-5. rerun the complete fixed group with both variants after the change.
-
-If the evaluation definition or criterion is wrong, preserve the old result and state that it no longer answers the revised question. Do not reinterpret it as evidence for the new question.
-
-## Report And Stop
-
-Report:
-
-- what ran;
-- exact baseline and candidate identities;
-- selected groups and variants;
-- deterministic results;
-- infrastructure failures, cancellation, or unavailable evidence;
-- viewed scope;
-- canonical result paths;
-- what the evidence proves;
-- what it does not prove;
-- any supported failed boundary or revision target.
-
-Stop rather than overclaim when required activation, first-read, composition, retained-use, isolation, artifact, or host evidence is unavailable. The user decides whether to revise, use, publish, or reject the skill.

package/skills/evaluate-skill/references/evaluation-definition-schema.md

@@ -1,415 +0,0 @@
-# Evaluation Definition Schema
-
-Read this when authoring or checking schema-version-1 group and suite JSON.
-
-This reference is a contract: the shapes and constraints below are binding as written. Structural violations are rejected before any subject runs; a malformed expectation surfaces afterward as `grade-error` evidence.
-
-This file owns user-authored definition shapes. Read [execution and evidence](execution-and-evidence.md) for commands, path resolution, run states, persistence, views, isolation, and safeguards.
-
-## Definition Root
-
-Run `skill-eval` from the repository or fixture root that owns the evaluation inputs.
-
-| Input | Resolves from |
-| --- | --- |
-| `run <suite-or-group-path>` | current working directory |
-| suite `groups` entries | directory containing the suite file |
-| group `fixture` | definition root/current working directory |
-| environment `skills` and `context` | definition root/current working directory, then the selected working tree or Git commit |
-
-Definition files and declared paths are checked for lexical and canonical containment. Paths are slash-separated relative paths, not absolute paths.
-
-## Common Group Shape
-
-Every group contains exactly these fields, plus optional `model`:
-
-```json
-{
-  "schema_version": 1,
-  "kind": "group",
-  "id": "one-behavioral-question",
-  "type": "description",
-  "input": { "prompt": "Natural task prompt." },
-  "fixture": null,
-  "tools": ["read"],
-  "variants": {
-    "baseline": {
-      "source": { "kind": "working-tree" },
-      "skills": [],
-      "target": null,
-      "context": []
-    },
-    "candidate": {
-      "source": { "kind": "working-tree" },
-      "skills": ["skills/target"],
-      "target": 0,
-      "context": []
-    }
-  },
-  "expectations": [],
-  "review_questions": [],
-  "model": {
-    "model": "openai-codex/gpt-5.6-sol",
-    "thinking": "low"
-  }
-}
-```
-
-Rules:
-
-- `schema_version` is `1`.
-- `kind` is `group`.
-- `id` starts with a lowercase letter, contains lowercase letters, numbers, or single hyphens, and is at most 64 characters.
-- `type` is `description` or `body`.
-- `input` contains exactly one of:
-  - `prompt`: one non-empty string;
-  - `turns`: a non-empty ordered array of non-empty strings. Repeated natural turns are allowed.
-- `fixture` is `null` or one contained relative directory path. The directory's contents become the workspace.
-- `tools` is an ordered array with no duplicates.
-- `variants` contains exactly `baseline` and `candidate`.
-- `expectations` is an ordered array with unique expectation IDs.
-- `review_questions` is an ordered string array. Questions are rendered for the reviewer and never sent to the subject.
-- `model` is optional. `model.model` names the provider/model; optional `model.thinking` selects its thinking level. The CLI has no model override.
-- `runtime` is optional. When omitted, the group runs under a plain `pi` host with the isolated prompt and no declared extensions. When present, it declares the host, prompt mode, ordered extension bundles, and string environment overrides.
-- `commands` is optional and only for body groups that list the `run_command` tool. See [Declared Commands](#declared-commands).
-
-## Description Group Example
-
-A description group uses natural prompts and asks whether the exact target was read and when:
-
-```json
-{
-  "schema_version": 1,
-  "kind": "group",
-  "id": "natural-activation",
-  "type": "description",
-  "input": {
-    "turns": [
-      "Help me inspect this ordinary change.",
-      "Now prepare a deployment rollback."
-    ]
-  },
-  "fixture": null,
-  "tools": ["read"],
-  "variants": {
-    "baseline": {
-      "source": { "kind": "git", "ref": "before-description-change" },
-      "skills": ["skills/target"],
-      "target": 0,
-      "context": []
-    },
-    "candidate": {
-      "source": { "kind": "working-tree" },
-      "skills": ["skills/target"],
-      "target": 0,
-      "context": []
-    }
-  },
-  "expectations": [
-    {
-      "id": "baseline-read-by-two",
-      "kind": "skill-read",
-      "variant": "baseline",
-      "expect": "by-turn",
-      "turn": 2,
-      "comparison": "target-read-by-two"
-    },
-    {
-      "id": "candidate-read-by-two",
-      "kind": "skill-read",
-      "variant": "candidate",
-      "expect": "by-turn",
-      "turn": 2,
-      "comparison": "target-read-by-two"
-    }
-  ],
-  "review_questions": []
-}
-```
-
-For a description-only revision, the two target snapshots must keep body and resources byte-identical while changing only frontmatter description text.
-
-Description groups allow no tools or only `read`. `skill-read` is their target-activation check.
-
-## Body Group Example
-
-A body group explicitly delivers each target on turn one and observes first-read or later-turn behavior:
-
-```json
-{
-  "schema_version": 1,
-  "kind": "group",
-  "id": "retained-artifact-behavior",
-  "type": "body",
-  "input": {
-    "turns": [
-      "Inspect the task and prepare the work.",
-      "Complete it and write result.json."
-    ]
-  },
-  "fixture": "fixtures/task",
-  "tools": ["read", "write", "edit"],
-  "variants": {
-    "baseline": {
-      "source": { "kind": "git", "ref": "before-body-change" },
-      "skills": ["skills/support", "skills/target"],
-      "target": 1,
-      "context": ["context/contract.md"]
-    },
-    "candidate": {
-      "source": { "kind": "working-tree" },
-      "skills": ["skills/support", "skills/target"],
-      "target": 1,
-      "context": ["context/contract.md"]
-    }
-  },
-  "expectations": [
-    {
-      "id": "baseline-result",
-      "kind": "path",
-      "variant": "baseline",
-      "path": "result.json",
-      "turn": 2,
-      "expect": "exists",
-      "comparison": "result-exists"
-    },
-    {
-      "id": "candidate-result",
-      "kind": "path",
-      "variant": "candidate",
-      "path": "result.json",
-      "turn": 2,
-      "expect": "exists",
-      "comparison": "result-exists"
-    },
-    {
-      "id": "candidate-status",
-      "kind": "json",
-      "variant": "candidate",
-      "path": "result.json",
-      "turn": 2,
-      "expect": "field-equals",
-      "pointer": "/status",
-      "value": "complete"
-    },
-    {
-      "id": "candidate-changes",
-      "kind": "changed-paths",
-      "variant": "candidate",
-      "turn": 2,
-      "expect": "equals",
-      "paths": ["result.json"]
-    }
-  ],
-  "review_questions": [
-    "Did the second turn apply the first-turn guidance without redelivery?"
-  ]
-}
-```
-
-Description groups allow no tools or only built-in `read`. Body groups allow built-in `read`, `write`, and `edit`, the evaluator's `run_command` tool for declared commands, plus custom tools declared by a runtime extension bundle. Native `bash`, `powershell`, `grep`, `find`, and `ls` tools are unsupported. A no-target baseline or runtime-only group may use `skills: []` and `target: null`; a candidate that declares skills must identify its target.
-
-The examples are complete definition shapes. Replace fixture, resource, model, and Git-ref values with paths and identities that exist under your definition root.
-
-## Runtime Profile
-
-A runtime profile is shared by both baseline and candidate variants. It changes the host environment, not the skill target:
-
-```json
-{
-  "runtime": {
-    "host": "pi",
-    "session": false,
-    "extensions": [
-      {
-        "entry": "extensions/probe.mjs",
-        "resources": ["extensions"]
-      }
-    ],
-    "environment": {
-      "literal": {
-        "PROBE_MODE": "injection-test"
-      },
-      "inherit": []
-    }
-  }
-}
-```
-
-`host` is `pi`; the evaluator maps it to the installed Pi command and records the selected host. Optional `prompt` is `isolated` (default) or `host`:
-
-- `isolated` replaces the host's system prompt with a minimal evaluator-owned prompt. Use it to isolate the target skill.
-- `host` keeps the host's own system prompt and whatever declared extensions compose onto it, appending declared context. Use it when evidence must reflect the production prompt, for example by declaring the Freeflow Pi extension bundle with a fixture containing `.freeflow/config.json`.
-
-Both modes enforce the same tool and path guards. `session` controls whether the variant receives an isolated persistent session directory; it defaults to `false` in omitted runtime profiles. A runtime may declare zero or more bundles. Each bundle has an entry file and one or more directory resources copied under their original relative paths, so sibling imports and multi-tree extensions remain intact. Bundle bytes are snapshotted once and copied identically to both variants.
-
-`environment.literal` contains non-secret string configuration and is recorded in run evidence. `environment.inherit` names parent-process variables whose values are passed to the child but never persisted; other parent variables are not passed. Missing inherited variables invalidate the variant. The evaluator supplies only a small host-runtime baseline such as `PATH` and `HOME`. Launch-control and loader keys are rejected from inherited values, literal and inherited names must not overlap, and secret-like literal keys are rejected.
-
-Extensions are trusted executable inputs, not OS-sandboxed code. The evaluator proves declared identity and post-run immutability, but extension code retains its host permissions.
-
-When a group declares `context-text` expectations, the evaluator loads its base isolation guard first, then declared bundles, then a non-mutating final observer. The observer records two per-turn surfaces: the chained `before_agent_start` system prompt and each provider-neutral `context` message projection. `context-text` expectations select `surface: "system-prompt"` or `surface: "provider-context"` and may select `request: "first"`, `"last"`, or a positive request number. Groups without context assertions do not create context-observation artifacts.
-
-A custom tool name is allowed in `group.tools` only when the runtime declares an extension bundle. The evaluator remains extension-neutral: it does not interpret feature names or configuration schemas. Put feature configuration in the ordinary group fixture or runtime environment, and test combinations as separate suite groups.
-
-## Variant Environment
-
-Each environment has exactly:
-
-```json
-{
-  "source": { "kind": "git", "ref": "v1.2.3" },
-  "skills": ["skills/target", "skills/support"],
-  "target": 0,
-  "context": ["context/base-guidance.md"]
-}
-```
-
-`source` is either:
-
-```json
-{ "kind": "working-tree" }
-```
-
-or:
-
-```json
-{ "kind": "git", "ref": "<exact-ref>" }
-```
-
-A source applies only to declared skill and context paths in that variant. It never supplies the group fixture: `fixture` is snapshotted once from the definition-root working tree, then copied independently for selected variants. `skills` preserves registration order. `target` is `null` or a zero-based index into `skills`; the candidate target cannot be `null`. `context` preserves declaration order and may name UTF-8 files or directories.
-
-## Expectations
-
-Every expectation has:
-
-- `id`: unique group-local ID using the group-ID syntax;
-- `kind`: one supported deterministic kind;
-- `variant`: `baseline` or `candidate`;
-- optional `comparison`: shared by exactly one baseline and one candidate expectation describing the same expected check;
-- optional body `turn`: one-based declared prompt/turn number.
-
-Without `turn`, response checks use the final response and workspace checks use final state. Turn-scoped workspace checks cause an immutable workspace snapshot after each settled turn.
-
-| `kind` | Kind-specific fields | `expect` |
-| --- | --- | --- |
-| `skill-read` | timing `turn` except for `never` | `never`, `on-turn`, `by-turn`, `not-before-turn` |
-| `resource-read` | `resource`, optional `index`, `path`, optional body `turn` | `read`, `not-read` |
-| `path` | workspace `path`, optional body `turn` | `exists`, `absent` |
-| `changed-paths` | unique workspace `paths`, optional body `turn` | `equals`, `excludes` |
-| `file-text` | workspace `path`, string `value`, optional body `turn` | `contains`, `not-contains`, `equals` |
-| `json` | workspace `path`, optional body `turn`, conditional `pointer` and `value` | `available`, `missing`, `valid`, `malformed`, `field-present`, `field-absent`, `field-equals` |
-| `response-text` | string `value`, optional body `turn` | `contains`, `not-contains`, `equals` |
-| `tool-call` | string `tool`, optional body `turn`, optional `argumentContains`/`argumentNotContains` arrays | `called`, `succeeded`, `failed`, `not-called` |
-| `context-text` | string `value`, optional `surface` (`system-prompt` or `provider-context`), optional `request`, optional body `turn` | `contains`, `not-contains`, `equals` |
-
-For `tool-call` with `expect: "not-called"`, omitted argument filters require zero calls to the named tool. Supplied filters scope the absence check to calls matching those argument predicates.
-
-### Resource paths
-
-- `resource: "workspace"`: omit `index`; `path` is relative to the writable workspace.
-- `resource: "skill"`: `index` selects the zero-based declared skill; `path` is relative to that skill directory.
-- `resource: "context"`: `index` selects the zero-based declared context item; `path` is relative to that materialized item.
-
-### Changed paths
-
-- `equals`: the complete changed-file set must equal `paths`.
-- `excludes`: none of `paths` may appear in the changed-file set.
-
-Directory-only presence does not count as a changed file.
-
-### JSON states
-
-```json
-[
-  {
-    "id": "file-missing",
-    "kind": "json",
-    "variant": "candidate",
-    "path": "missing.json",
-    "expect": "missing"
-  },
-  {
-    "id": "file-malformed",
-    "kind": "json",
-    "variant": "candidate",
-    "path": "broken.json",
-    "expect": "malformed"
-  },
-  {
-    "id": "field-absent",
-    "kind": "json",
-    "variant": "candidate",
-    "path": "result.json",
-    "expect": "field-absent",
-    "pointer": "/optional"
-  },
-  {
-    "id": "field-null",
-    "kind": "json",
-    "variant": "candidate",
-    "path": "result.json",
-    "expect": "field-equals",
-    "pointer": "/value",
-    "value": null
-  }
-]
-```
-
-A JSON Pointer is `""` for the complete value or begins with `/`; `~0` escapes `~` and `~1` escapes `/`.
-
-### Comparison pair
-
-Two expectations may share one comparison ID only when they have opposite variants and the same kind and expected predicate. The report records a factual transition such as `fail-to-pass`, never a quality verdict.
-
-Definition loading validates common structure and unique IDs. Kind-specific expectation validation occurs during deterministic grading after run persistence. A malformed expectation therefore becomes separate `grade-error` evidence rather than changing the subject run. Use the documented shapes to avoid spending a subject run on an invalid check.
-
-Definition-supplied commands run only through the subject's `run_command` tool. Grading never runs commands.
-
-## Declared Commands
-
-A body group that must let the subject run tests or other programs lists the `run_command` tool and declares each allowed command:
-
-```json
-{
-  "tools": ["read", "edit", "run_command"],
-  "commands": [
-    { "id": "unit-tests", "argv": ["node", "--test"], "timeout_ms": 120000 },
-    { "id": "lint", "argv": ["npm", "run", "lint"] }
-  ]
-}
-```
-
-- `run_command` and `commands` require each other and are valid only in body groups.
-- Each command has a unique `id` (group-ID syntax), a non-empty `argv`, and optional integer `timeout_ms` from 1 to 600000 (default 120000).
-- The subject chooses only a declared `id`; it cannot change `argv`. The argv runs without a shell, in the workspace, without evaluator control variables.
-- Output is capped per stream and marked when truncated. A nonzero exit or timeout is an ordinary tool result, not a tool error.
-- Grade calls with `tool-call` expectations, for example `tool: "run_command"` with `argumentContains: ["unit-tests"]`.
-
-Commands are trusted author input with the evaluator's host permissions, like extension bundles; the subject only selects among them.
-
-## Suite
-
-```json
-{
-  "schema_version": 1,
-  "kind": "suite",
-  "id": "workflow-core",
-  "groups": [
-    "groups/natural-activation.json",
-    "groups/body-behavior.json"
-  ]
-}
-```
-
-A suite has only `schema_version`, `kind`, `id`, and ordered unique `groups`. Group references resolve from the suite directory; their resolved definition paths must remain inside the definition root. Referenced group IDs must also be unique.
-
-## Selection
-
-- With no selectors, every suite group and both variants are selected.
-- `--group <id>` selects one stable group ID.
-- `--group <position>` selects one one-based suite position.
-- `--variant baseline|candidate` applies across selected groups.
-- Excluded variants are `not-selected`, never failed.
-- `--group` is invalid for a direct group definition.
-
-Selection changes execution scope only. It does not modify the saved definitions.