@uipath/skills 1.204.0 → 1.204.1

package/skills/uipath-maestro-flow/references/evaluate/CAPABILITY.md

@@ -0,0 +1,134 @@
+# Evaluate — Design and run Flow evaluations
+
+Capability index for `uip maestro flow eval` — evaluator CRUD (7 types), eval set CRUD with entry-point pinning, data point management with file attachments, and Studio Web run start/status/results/list/compare. Local CRUD is offline; runs require `uip login` and a Flow solution that already exists in Studio Web.
+
+> **Where you came from / where to go next.** Evaluate is downstream of Operate (ship the flow → evaluate it on Studio Web) and feeds back into Author (failing eval → fix the `.flow` → re-ship → re-evaluate). Build/edit lives in [author/CAPABILITY.md](../author/CAPABILITY.md); publish/deploy lives in [operate/CAPABILITY.md](../operate/CAPABILITY.md); fault triage on a debug or process run lives in [diagnose/CAPABILITY.md](../diagnose/CAPABILITY.md).
+>
+> **Inherits universal rules from [SKILL.md](../../SKILL.md)** — `--output json` + prefer `--output-filter` for extraction, no `flow debug` without consent, never invoke other skills automatically, dropdown question pattern, **plain-English narration + granular progress list (opt-in — silent by default; engage when the user asks for verbosity)**. The rules below are evaluate-scoped and apply on top.
+
+## When to use this capability
+
+- Add or remove data points (test cases) on a Flow eval set
+- Create evaluators (`exact-match`, `json-similarity`, `contains`, `llm-judge-*` types) for a Flow project
+- Create or remove eval sets, link them to evaluators, pin entry points
+- Add, list, or remove simulations on data points (`uip maestro flow eval simulation`)
+- Add, list, or remove child tool simulations on agent nodes (`--parent` flag)
+- Start an eval run on Studio Web, poll its status, fetch detailed results
+- Compare two eval runs to verify a change improved scores without regressions
+
+For agent (`agent.json`) evaluations read the `uipath-agents` skill. For BPMN evaluations read the `uipath-maestro-bpmn` skill — this capability covers Flow only.
+
+## Critical rules
+
+1. **Check Flow eval CLI availability once.** Run `uip maestro flow eval --help --output json` before using eval commands. If it returns `unknown command 'eval'`, the installed CLI does not expose Flow eval yet. Stop, report that the user needs a CLI/tool version with Flow eval support, and do not spend turns searching npm packages or source bundles.
+2. **Never run `uip solution upload` automatically as part of an eval workflow.** The eval run requires the Flow solution to already exist in Studio Web, but uploading from the local working tree clobbers whatever is on Studio Web. If the project was pulled from Studio Web (`uip solution download`), edited locally in VS Code, or scaffolded on disk and never uploaded, an unprompted upload will overwrite or push unintended state. Ask the user explicitly before any `uip solution upload` — see [upload-safety.md](upload-safety.md).
+3. **`--path` accepts a Flow project directory OR a solution directory containing exactly one Flow project.** If the solution holds multiple Flow projects, point `--path` at the specific project directory.
+4. **Local CRUD does not require login.** `add`, `remove`, `list` (data points / eval sets / evaluators) edit JSON on disk. Only `uip maestro flow eval run *` requires `uip login` and an existing Studio Web solution.
+5. **Pin a model on every LLM-judge evaluator.** Empty/missing `model` produces a cryptic 500 from the LLM gateway after retries. Pass `--model <name>` on `evaluator add` or set `model` in the JSON.
+6. **Declare input variables before adding data points with `--inputs`.** `eval add` validates input keys against the Flow's declared input variables and fails fast on unknown keys. Add missing input variables first (for example, `uip maestro flow variable add My.flow name --direction in --type string --output json`) or change the data point input JSON to match the Flow schema.
+7. **Let the CLI manage evaluator references.** Eval sets store `evaluatorRefs` as the evaluator file refs produced by `evaluator add` (for example, `greeting-match-1234abcd.json`). If the set should use every evaluator that already exists, omit `--evaluators` so the CLI writes those generated file refs. If you must pass `--evaluators`, pass the evaluator id or generated file base/ref from `evaluator add/list` — never the display name alone (`greeting-match`), because Studio Web does not resolve display-name refs.
+8. **Pre-empt timeouts on `run start --wait`.** The CLI blocks until the run reaches a terminal state or `--timeout` elapses. `--timeout` only stops local blocking — the run continues server-side; query progress with `eval run status <run_id>`.
+
+## Quick Start
+
+Standard workflow: scaffold evaluators → create eval set → add data points → confirm project is in Studio Web → run.
+
+```bash
+# 1. Add an evaluator (local; no login required)
+uip maestro flow eval evaluator add greeting-quality \
+  --type llm-judge-output \
+  --model gpt-4.1-2025-04-14 \
+  --path ./MySolution/MyFlow --output json
+
+# 2. Create an eval set, pin the entry point, and let the CLI attach
+#    all current evaluators by generated file ref
+uip maestro flow eval set add "Smoke Tests" \
+  --entry-point /Main.bpmn#start \
+  --path ./MySolution/MyFlow --output json
+
+# 3. Add data points (test cases)
+#    The `message` key must already be declared as a Flow input variable.
+uip maestro flow eval add hello-test \
+  --set "Smoke Tests" \
+  --inputs '{"message":"hello"}' \
+  --expected '{"reply":"Hello! How can I help you?"}' \
+  --path ./MySolution/MyFlow --output json
+
+# 4. Confirm the solution is in Studio Web
+#    DO NOT auto-run `uip solution upload`. Ask the user. See upload-safety.md.
+
+# 5. Start the run and wait
+uip maestro flow eval run start \
+  --set "Smoke Tests" \
+  --path ./MySolution/MyFlow \
+  --wait --timeout 600 --output json
+
+# 6. Inspect failures
+uip maestro flow eval run results <eval_set_run_id> \
+  --set "Smoke Tests" \
+  --only-failed --verbose \
+  --path ./MySolution/MyFlow --output json
+```
+
+## Workflow
+
+| Journey | Read |
+| --- | --- |
+| Look up any `uip maestro flow eval` subcommand syntax, flags, defaults, output codes | [commands-reference.md](commands-reference.md) |
+| Choose among the 7 evaluator types, write custom prompts, hand-write evaluator JSON | [evaluators-guide.md](evaluators-guide.md) |
+| Create eval sets, add data points, map `--inputs`/`--expected`/`--criteria` to evaluator types, attach files | [eval-sets-guide.md](eval-sets-guide.md) |
+| Start a Studio Web run, poll status, read results, export CSV/JSON, compare two runs | [running-guide.md](running-guide.md) |
+| Decide whether to call `uip solution upload` (almost always: don't auto-run; ask first) | [upload-safety.md](upload-safety.md) |
+
+## Common tasks
+
+| I need to... | Read these |
+| --- | --- |
+| **Add an evaluator** | [evaluators-guide.md](evaluators-guide.md) + [commands-reference.md — Evaluators](commands-reference.md#evaluators) |
+| **Pick the right evaluator type** | [evaluators-guide.md — When to Pick Each Type](evaluators-guide.md#when-to-pick-each-type) |
+| **Create an eval set and pin an entry point** | [eval-sets-guide.md — Eval Set Lifecycle](eval-sets-guide.md#eval-set-lifecycle) |
+| **Add a data point with file attachments** | [eval-sets-guide.md — `--input-file`](eval-sets-guide.md#--input-file-keypath) |
+| **Set per-data-point criteria for trajectory evaluators** | [eval-sets-guide.md — `--criteria`](eval-sets-guide.md#--criteria) |
+| **Add a simulation to a data point** | [eval-sets-guide.md — Simulations](eval-sets-guide.md#simulations-on-data-points) + [commands-reference.md — Simulations](commands-reference.md#simulations) |
+| **Add a child tool simulation to an agent node** | [eval-sets-guide.md — Child Simulations](eval-sets-guide.md#child-simulations-agent-tool-simulation) + [commands-reference.md — Simulations](commands-reference.md#simulations) |
+| **Start a Studio Web eval run** | [running-guide.md — Start a Run](running-guide.md#start-a-run) |
+| **Poll run status without `--wait`** | [running-guide.md — Check Status](running-guide.md#check-status) |
+| **Inspect only failed data points** | [running-guide.md — Detailed Results](running-guide.md#detailed-results) (`--only-failed --verbose`) |
+| **Compare two runs side-by-side** | [running-guide.md — Compare Two Runs](running-guide.md#compare-two-runs) |
+| **Decide whether to call `uip solution upload`** | [upload-safety.md](upload-safety.md) |
+| **Look up the `eval` subcommand tree, flags, defaults, output codes** | [commands-reference.md](commands-reference.md) |
+| **My eval run failed** | [running-guide.md — Failure Detection](running-guide.md#failure-detection); for flow-level faults [diagnose/CAPABILITY.md](../diagnose/CAPABILITY.md) |
+
+## Anti-patterns
+
+- **Don't auto-run `uip solution upload`.** Even when an eval run errors with "solution not found in Studio Web", stop and ask the user — see [upload-safety.md](upload-safety.md). The local project may be ahead of, or diverged from, Studio Web.
+- **Don't hand-write `evaluatorRefs` unless you are repairing an eval set.** Prefer the default all-evaluators behavior so the CLI writes generated file refs, or pass generated evaluator ids/file refs explicitly. Do not pass evaluator display names to `--evaluators`.
+- **Don't pass `--type` in PascalCase.** Only kebab-case is accepted: `exact-match`, `json-similarity`, `contains`, `llm-judge-output`, `llm-judge-strict-json`, `llm-judge-trajectory`, `llm-judge-trajectory-simulation`.
+- **Don't depend on a specific `--wait` polling cadence.** Treat `--wait` as a black-box block; if you need precise progress, omit it and call `eval run status` yourself.
+- **Don't compare runs from different eval sets.** `eval run compare` aligns by data-point name within the set; cross-set deltas are meaningless.
+- **Don't omit `--model` on LLM-judge evaluators.** The cloud worker fail-fasts before calling the LLM gateway.
+- **Don't run evals during `flow debug`.** `debug` is a separate Studio Web session; evals run against the deployed/published solution. Mixing them produces confusing run IDs.
+
+## Completion Output
+
+After a run completes, report:
+
+1. **Eval set run ID** and aggregate score (from `run status`)
+2. **Failed data points** (from `run results --only-failed --verbose`)
+3. **Comparison delta** vs the previous run (`run compare`) if one exists
+4. **Suggested next step** — fix the agent/flow, re-run, or accept the result. Do NOT suggest `uip solution upload` unless the user has explicitly asked to publish edits.
+
+## References
+
+### Evaluate-scoped
+
+- [commands-reference.md](commands-reference.md) — every `uip maestro flow eval` subcommand, flags, defaults, output `Code` enum
+- [evaluators-guide.md](evaluators-guide.md) — 7 evaluator types mapped to internal `uipath-*` IDs, JSON shapes, template variables
+- [eval-sets-guide.md](eval-sets-guide.md) — eval set + data point CRUD, `--inputs`/`--expected`/`--criteria`/`--input-file`/`--search-text`, simulations
+- [running-guide.md](running-guide.md) — run start/status/results/list/compare, JMESPath `--output-filter`, failure detection
+- [upload-safety.md](upload-safety.md) — the `solution upload` rule
+
+### Cross-capability (shared)
+
+- [shared/cli-commands.md](../shared/cli-commands.md) — flat CLI lookup including `eval` subcommands
+- [shared/cli-conventions.md](../shared/cli-conventions.md) — login states, `--output json`, JSON output shape

package/skills/uipath-maestro-flow/references/evaluate/commands-reference.md

@@ -0,0 +1,376 @@
+# `uip maestro flow eval` Command Reference
+
+Complete syntax reference for every subcommand under `uip maestro flow eval`. All commands accept the global flags `--output <table|json|yaml|plain>` (default `json`), `--output-filter <jmespath>`, `--log-level <debug|info|warn|error>`, and `--log-file <path>`. Repeated reminders below: always pass `--output json` when an agent will parse the result.
+
+## Command Tree
+
+```
+uip maestro flow eval
+├── add        — Add a data point to an evaluation set
+├── list       — List data points in an evaluation set
+├── remove     — Remove a data point from an evaluation set
+├── set
+│   ├── add    — Create an evaluation set
+│   ├── list   — List evaluation sets
+│   └── remove — Remove an evaluation set
+├── evaluator
+│   ├── add    — Add an evaluator to the flow project
+│   ├── list   — List evaluators in a flow project
+│   └── remove — Remove an evaluator from a flow project
+├── simulation
+│   ├── add    — Add or update a simulation on a data point
+│   ├── list   — List simulations on a data point
+│   └── remove — Remove a simulation from a data point
+└── run
+    ├── start    — Start a Studio Web evaluation run
+    ├── status   — Check the status of a run
+    ├── results  — Get detailed per-data-point results
+    ├── list     — List runs for an eval set
+    └── compare  — Compare two runs side-by-side
+```
+
+`add` / `remove` / `list` at the top level operate on **data points** (test cases) inside an eval set. Data points are stored inline within the eval set JSON, not as separate files.
+
+## Common Options
+
+Every subcommand accepts:
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--path <path>` | No (defaults to `.`) | Flow project directory, or a solution directory containing exactly one Flow project |
+| `--output <fmt>` | No (default `json`) | `table`, `json`, `yaml`, or `plain` |
+| `--output-filter <expr>` | No | JMESPath expression applied to JSON output before printing |
+
+## Data Points (Test Cases)
+
+### `uip maestro flow eval add <name>`
+
+Add a data point to an eval set.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--inputs <json>` | No | Input values as a JSON object; keys must be declared as Flow input variables |
+| `--input-file <key=path>` | No | Attach a file as input `<key>`; **repeatable** |
+| `--expected <json>` | No | Expected output as a JSON object |
+| `--criteria <json>` | No | Per-evaluator criteria JSON object keyed by evaluator id |
+| `--search-text <text>` | No | Search text for `contains` evaluators |
+| `--path <path>` | No | (see Common Options) |
+
+Example:
+
+```bash
+uip maestro flow eval add greeting-test \
+  --set "Smoke Tests" \
+  --inputs '{"name":"Alice"}' \
+  --expected '{"greeting":"Hello, Alice!"}' \
+  --path ./MySolution/MyFlow --output json
+```
+
+### `uip maestro flow eval list`
+
+List data points in an eval set.
+
+```bash
+uip maestro flow eval list --set "Smoke Tests" --path ./MySolution/MyFlow --output json
+```
+
+### `uip maestro flow eval remove <id>`
+
+Remove a data point. `<id>` accepts the data point's UUID or its `name`.
+
+## Evaluation Sets
+
+### `uip maestro flow eval set add <name>`
+
+Create an evaluation set.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--evaluators <refs>` | No (default: all) | Comma-separated evaluator IDs or generated file base names; do not pass display names |
+| `--entry-point <id>` | No | Entry point node id stored as the eval set's `selectedEntrypoint` |
+| `--path <path>` | No | (see Common Options) |
+
+When `--evaluators` is omitted, the new eval set references **all** evaluators present in the project at creation time using their generated evaluator file refs. Prefer this when creating a set immediately after adding the evaluator(s). If passing `--evaluators`, use the generated id/file base returned by `evaluator add/list`, not the evaluator display name.
+
+### `uip maestro flow eval set list`
+
+List eval sets in the project.
+
+### `uip maestro flow eval set remove <id>`
+
+Remove an eval set. `<id>` accepts the eval set's UUID, `name`, or file base name.
+
+## Evaluators
+
+### `uip maestro flow eval evaluator add <name>`
+
+Create an evaluator file in the project's evaluators directory.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--type <type>` | Yes | One of: `exact-match`, `json-similarity`, `contains`, `llm-judge-output`, `llm-judge-strict-json`, `llm-judge-trajectory`, `llm-judge-trajectory-simulation` |
+| `--description <text>` | No | Evaluator description |
+| `--target-key <key>` | No | Output key the evaluator scores against (defaults to `*` — the entire output) |
+| `--model <model>` | No (Yes for llm-judge-*) | LLM model for LLM-judge evaluators (e.g. `gpt-4.1-2025-04-14`) |
+| `--prompt <prompt>` | No | Custom LLM judge prompt; defaults to a built-in template per type |
+| `--path <path>` | No | (see Common Options) |
+
+Only kebab-case `--type` values are accepted; PascalCase fails with an error.
+
+For LLM-judge evaluators, `--model` is effectively required — the cloud worker rejects an empty `model` before sending to the LLM gateway. See [evaluators-guide.md](evaluators-guide.md) for the seven types in detail.
+
+### `uip maestro flow eval evaluator list`
+
+List evaluators in the project.
+
+### `uip maestro flow eval evaluator remove <id>`
+
+Remove an evaluator. `<id>` accepts UUID, name, or file base name. Removing an evaluator does not auto-clean `evaluatorRefs` in eval sets — verify after removing.
+
+## Simulations
+
+Simulations intercept specific nodes (connectors, agents, sub-flows) during an eval run and replace their real execution with a controlled response. Each simulation targets a single component by its `componentId` and applies one of three strategies.
+
+### `uip maestro flow eval simulation add <component-id>`
+
+Add or replace a simulation on a data point. If a simulation for `<component-id>` already exists on the data point it is overwritten.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--data-point <id>` | Yes | Data point name or ID |
+| `--strategy <strategy>` | Yes | `Llm` or `Static` |
+| `--component-type <type>` | No | Component type (e.g. `connector`, `agent`, `subflow`). Required unless `--parent` is used (defaults to `Node`). |
+| `--component-description <text>` | No | Human-readable label for the component |
+| `--simulation-instructions <text>` | No | LLM prompt describing what the component should return (for `Llm` strategy) |
+| `--mock-value <json>` | No | Static JSON output (for `Static` strategy) |
+| `--parent <component-id>` | No | Parent agent node component ID. When set, the simulation is added as a child tool simulation nested inside the parent agent node's simulation. If no parent *simulation* exists yet, one is auto-created (type `agent`, strategy `Llm`). `--component-type` defaults to `Node`. |
+| `--path <path>` | No | (see Common Options) |
+
+**Strategy guide:**
+
+| Strategy | When to use | Key flags |
+|----------|-------------|-----------|
+| `Llm` | Output should be realistic but non-deterministic | `--simulation-instructions` (output schema auto-resolved) |
+| `Static` | Output is fixed and deterministic | `--mock-value` |
+
+**Add the targeted node to `nodes[]` before any `Llm` call** — an eval set and a data point are not enough. Only `Llm` treats a missing node as an error: a top-level `Static` call never reads the `.flow` at all and succeeds against a component that does not exist, and a `--parent` `Static` call reads it but downgrades a miss to a `Mock value key validation was skipped` warning. So a sequence mixing both strategies fails on the `Llm` call alone, and the earlier `Static` success proves nothing about the node. A top-level `Llm` target needs readable outputs: a connector node with a configured `inputs.detail`, an agent node with a non-empty `inputs.agentOutputVariables`, or any node carrying `outputs`. `uip maestro flow node add` never writes `outputs`, so a bare `core.logic.mock` node fails with `Could not auto-resolve output schema … The node has no declared outputs` as a top-level target — it resolves only as a `--parent` child, from its definition's `outputDefinition`.
+
+| Missing | Error |
+|---------|-------|
+| top-level node | `Component "<component-id>" not found in the flow. Verify the component ID.` |
+| `--parent` agent node | `Parent agent "<component-id>" not found in the flow.` |
+
+**Output schema auto-resolution:** The CLI auto-resolves the output schema whenever the simulation does not already carry one — for top-level simulations under `Llm` only, for child (`--parent`) simulations under both strategies:
+
+- **Top-level simulations:** reads the `.flow` file, finds the node by `<component-id>`, and derives the schema from the node's output definition (connector `outputJsonSchema`, agent `agentOutputVariables`, or `node.outputs`).
+- **Child simulations (inline canvas agents):** finds the child tool node via edges in the `.flow` file and extracts its output schema.
+- **Child simulations (same-solution agents):** reads the inline agent's `agent.json` and matches the tool by name in the `resources[]` array.
+- **Child simulations (published agents):** calls the platform API (`simulatableComponents`) using the current login session to fetch the tool's schema. Requires `uip login`.
+
+A `--parent` agent node must also be wired to each child tool node by an edge on its `tool` port. The `--parent` value is the agent node's `id`; each `<component-id>` is a tool node's `id`:
+
+```json
+{ "nodes": [
+    { "id": "agent-1", "type": "uipath.agent.autonomous", "inputs": { "agentInputVariables": {}, "agentOutputVariables": {} } },
+    { "id": "Web_Search", "type": "core.logic.mock" },
+    { "id": "Send_Email", "type": "core.logic.mock" } ],
+  "edges": [
+    { "sourceNodeId": "agent-1", "sourcePort": "tool", "targetNodeId": "Web_Search", "targetPort": "input" },
+    { "sourceNodeId": "agent-1", "sourcePort": "tool", "targetNodeId": "Send_Email", "targetPort": "input" } ] }
+```
+
+`core.logic.mock` tool nodes resolve schemas. See [plugins/inline-agent/impl.md](../author/plugins/inline-agent/impl.md) for a production agent node.
+
+**Static mock value validation:** For `Static` child simulations, the CLI validates that `--mock-value` keys match the auto-resolved schema properties, catching shape mismatches early.
+
+Example — LLM strategy:
+
+```bash
+uip maestro flow eval simulation add connectorSendEmail \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --strategy Llm \
+  --component-type connector \
+  --simulation-instructions "Pretend to send the email and return a success confirmation." \
+  --path ./MySolution/MyFlow --output json
+```
+
+Example — Static strategy:
+
+```bash
+uip maestro flow eval simulation add agentLookup \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --strategy Static \
+  --component-type agent \
+  --mock-value '{"result": "found", "items": []}' \
+  --path ./MySolution/MyFlow --output json
+```
+
+Example — Child simulation (tool inside an agent node):
+
+```bash
+# Add a child tool simulation (Static). No separate parent simulation step
+# needed — --parent auto-creates the parent simulation.
+# Output schema is auto-resolved from the agent's tool definitions.
+uip maestro flow eval simulation add Web_Search \
+  --parent agentLookup \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --strategy Static \
+  --mock-value '{"results": [{"title": "Example", "url": "https://example.com"}]}' \
+  --path ./MySolution/MyFlow --output json
+
+# Add a child tool simulation (Llm). Output schema auto-resolved —
+# works for inline, same-solution, and published agents.
+uip maestro flow eval simulation add Send_Email \
+  --parent agentLookup \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --strategy Llm \
+  --simulation-instructions "Return a success status with a generated messageId." \
+  --path ./MySolution/MyFlow --output json
+```
+
+The child's `<component-id>` is the tool's **runtime name** (e.g. `Web_Search`, `Send_Email`), not a `.flow` node ID. Child simulations are stored in the parent's `childSimulations` array.
+
+### `uip maestro flow eval simulation list`
+
+List all simulations configured on a data point.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--data-point <id>` | Yes | Data point name or ID |
+| `--parent <component-id>` | No | Parent agent component ID. When set, lists child tool simulations on that parent instead of top-level simulations. |
+| `--path <path>` | No | (see Common Options) |
+
+```bash
+# List top-level simulations
+uip maestro flow eval simulation list \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --path ./MySolution/MyFlow --output json
+
+# List child simulations on an agent node
+uip maestro flow eval simulation list \
+  --parent agentLookup \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --path ./MySolution/MyFlow --output json
+```
+
+### `uip maestro flow eval simulation remove <component-id>`
+
+Remove a simulation from a data point. Returns an error if no simulation with the given `<component-id>` exists on the data point.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--data-point <id>` | Yes | Data point name or ID |
+| `--parent <component-id>` | No | Parent agent component ID. When set, removes a child tool simulation from the parent instead of a top-level simulation. |
+| `--path <path>` | No | (see Common Options) |
+
+```bash
+# Remove a top-level simulation
+uip maestro flow eval simulation remove connectorSendEmail \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --path ./MySolution/MyFlow --output json
+
+# Remove a child simulation from an agent node
+uip maestro flow eval simulation remove Web_Search \
+  --parent agentLookup \
+  --set "Smoke Tests" \
+  --data-point "hello-test" \
+  --path ./MySolution/MyFlow --output json
+```
+
+## Run
+
+### `uip maestro flow eval run start`
+
+Start a Studio Web evaluation run. The Flow solution **must already exist in Studio Web** — see [upload-safety.md](upload-safety.md).
+
+| Flag | Required | Description | Default |
+|------|----------|-------------|---------|
+| `--set <name>` | Yes | Eval set name or ID | — |
+| `--solution-id <id>` | No | Solution ID from Studio Web | Auto-resolved from project metadata |
+| `--project-id <id>` | No | Flow project ID from Studio Web | Auto-resolved |
+| `--path <path>` | No | (see Common Options) | `.` |
+| `--entry-point <entry>` | No | Flow entry point path (e.g. `/Main.bpmn#start`) or start node ID | Eval set's `selectedEntrypoint` |
+| `--folder-key <key>` | No | Orchestrator folder key | Personal workspace |
+| `--debug-mode <mode>` | No | Studio Web debug mode override | (server default) |
+| `--wait` | No | Block until terminal state, then print results | `false` |
+| `--timeout <seconds>` | No | Max time to block on `--wait` | `600` (10 min) |
+
+Without `--wait`, returns immediately with `EvalSetRunId`. With `--wait`, the CLI polls until `Completed` or `Failed`, or `--timeout` elapses (the server-side run continues regardless).
+
+### `uip maestro flow eval run status <evalSetRunId>`
+
+Get current status. Terminal states: `Completed`, `Failed`.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--solution-id <id>` | No | Override solution ID |
+| `--project-id <id>` | No | Override project ID |
+| `--path <path>` | No | (see Common Options) |
+
+### `uip maestro flow eval run results <evalSetRunId>`
+
+Per-data-point results.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--set <name>` | Yes | Eval set name or ID |
+| `--only-failed` | No | Show only failed/errored data points |
+| `--verbose` | No | Include evaluator justifications |
+| `--export-format <json\|csv>` | No | Export results to a file |
+| `--solution-id`, `--project-id`, `--path` | No | (see start) |
+
+Per-row output fields: `DataPoint`, `Status`, `EvaluatorScores`, `Duration`, `Error` (plus `Justifications` when `--verbose`).
+
+### `uip maestro flow eval run list`
+
+List runs for an eval set.
+
+```bash
+uip maestro flow eval run list --set "Smoke Tests" --path ./MySolution/MyFlow --output json
+```
+
+### `uip maestro flow eval run compare <evalSetRunId>`
+
+Compare two runs side-by-side.
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--compare-to <id>` | Yes | Second eval set run ID |
+| `--set <name>` | Yes | Eval set name or ID |
+| `--solution-id`, `--project-id`, `--path` | No | (see start) |
+
+`compare` aligns data points by `name` within the eval set. Comparing runs from different eval sets is meaningless.
+
+## Output Codes
+
+The CLI emits a `Code` field on every JSON response. Useful when filtering or scripting:
+
+| Subcommand | `Code` |
+|------------|--------|
+| `eval add` | `FlowEvalAdd` |
+| `eval list` | `FlowEvalList` |
+| `eval remove` | `FlowEvalRemove` |
+| `eval set add` / `list` / `remove` | `FlowEvalSetAdd` / `FlowEvalSetList` / `FlowEvalSetRemove` |
+| `eval evaluator add` / `list` / `remove` | `FlowEvalEvaluatorAdd` / `FlowEvalEvaluatorList` / `FlowEvalEvaluatorRemove` |
+| `eval simulation add` / `list` / `remove` | `FlowEvalSimulationAdd` / `FlowEvalSimulationList` / `FlowEvalSimulationRemove` |
+| `eval simulation add --parent` / `list --parent` / `remove --parent` | `FlowEvalChildSimulationAdd` / `FlowEvalChildSimulationList` / `FlowEvalChildSimulationRemove` |
+| `eval run start` (no `--wait`) | `FlowEvalRunStarted` |
+| `eval run start --wait` (summary) | `FlowEvalRunCompleted` |
+| `eval run status` | `FlowEvalRunStatus` |
+| `eval run results` | `FlowEvalRunResults` |
+| `eval run list` | `FlowEvalRunList` |
+| `eval run compare` | `FlowEvalRunComparison` |
+
+The `eval run *` codes dropped their `Maestro` prefix (e.g. `MaestroFlowEvalRunResults` → `FlowEvalRunResults`) to match the rest of the `eval` family. Older CLI versions emit the `Maestro`-prefixed names. If actual emitted codes diverge from the table above, trust the JSON output — file an issue.