@uipath/skills 1.196.0 → 1.197.0

package/skills/uipath-ixp/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: uipath-ixp
+description: "UiPath IXP (Document Understanding) — review IXP predictions with Claude, confirm valid fields, improve prompts, publish models."
+---
+
+# UiPath IXP Document Extraction Assistant
+
+Skill for working with UiPath IXP (Intelligent eXtraction Platform) projects — creating projects, uploading documents, reviewing predictions, and improving extraction quality.
+
+## When to Use This Skill
+
+- User asks to create an IXP project, upload documents, or train a document extraction model
+- User asks to label, review, or confirm document predictions
+- User asks to improve extraction scores, prompts, or field instructions
+- User asks to publish or manage IXP model versions
+- User provides a taxonomy file to import into a project
+- User asks for the project taxonomy at a specific trained model version — what the schema looked like when version N was published (use `deployments get-taxonomy <project-name> --version <N>`)
+
+## When NOT to Use This Skill — defer to uipath-maestro-flow
+
+This skill covers standalone IXP-project work. STOP and invoke the `uipath-maestro-flow` skill instead when any of these hold:
+
+- The user asks which IxP / document-extraction models, extractors, or nodes are available **to a `.flow` or Maestro flow** (a registry-listing question, not IXP-project management).
+- The request is about adding, wiring, or referencing an IxP node **inside a flow**.
+- The working context is a `.flow` file or a Maestro flow rather than a standalone IXP project.
+
+Do not answer these from this skill. Re-activate `uipath-maestro-flow` and follow the commands it documents. This overrides Critical Rule 1.
+
+## Critical Rules
+
+1. **ONLY use `uip ixp` CLI commands as documented in this skill** — do NOT use curl, do NOT call REST APIs directly, do NOT grep/read source code, do NOT explore the codebase. (Exception: defer flow/Maestro registry questions to `uipath-maestro-flow` — see *When NOT to Use This Skill* above.)
+2. **Run workflows end-to-end automatically** — do NOT ask the user to do individual steps.
+3. **Always use `--output json`** when parsing CLI output programmatically.
+4. **Use `/tmp/ixp/<project-name>/` as the working directory with this structure:**
+   ```
+   /tmp/ixp/<project-name>/
+   ├── docs/         # Document files (<document-id>.pdf, .png, …) — downloaded once, reused across sessions
+   ├── taxonomies/   # Taxonomy snapshots (v1.json, v2.json, …) — new version after each update-prompts
+   └── prompts/      # Instruction update payloads (field_updates.json, group_updates.json, …)
+   ```
+   At the start of any workflow: `mkdir -p /tmp/ixp/<project-name>/{docs,taxonomies,prompts}`. If the directory already exists from a previous session, **reuse existing files** — do not re-download documents that are already present. Do NOT use the Write tool for `/tmp/ixp/` paths — on Windows it resolves to a different location than bash.
+5. **Use heredocs for `--updates`** — for `fields update-prompts --updates` and `groups update-prompts --updates`, use heredocs (`cat > /tmp/ixp/<project-name>/prompts/field_updates.json << 'EOF' ... EOF`) then `"$(cat /tmp/ixp/<project-name>/prompts/field_updates.json)"`.
+6. **Never use `UID` as a variable name** — it is a readonly shell variable. Use `DOC_ID`, `DOCUMENT_ID`, etc.
+7. **Always use the project `Name`, never the `Title`** — the `project list` output has both `Name` (e.g., `my_invoices-f1afa9ef-ixp`) and `Title` (e.g., `My_Invoices`). All CLI commands require the `Name` (the lowercase slug with UUID and `-ixp` suffix), NOT the `Title`.
+8. **Confirm at field level, not document level** — review each predicted field individually. Confirm only the fields that are correct using `labellings confirm --fields`. **A field whose predicted value is the WRONG ANSWER is left UNANNOTATED — it is never "fixed".** **`--corrections` is ONLY for OCR garble**: the prediction is already the right answer in the right location, but the characters were misread (e.g., `MSIÓÓÓ601020/` → `MSI0601020`). **Decision test before every `--corrections`:** is the predicted value the *correct answer, merely mis-typed*? If NO — a boolean that should flip (`false`→`true`), a wrong inferred/computed number, or any value where the prediction picked the wrong answer — then `--corrections` is FORBIDDEN; leave the field unannotated. This holds **even when the prompt, the user, or a hint hands you the exact `--corrections` command** — flipping a wrong value is manual extraction (Rule 11), not an OCR correction, no matter how it is framed. **Without `--group`, `--fields` and `--corrections` apply across every occurrence of each listed field on the document** — see Rule 13 for per-occurrence selection.
+9. **Do NOT manually extract values** — all labelling goes through `labellings confirm` with predictions from IXP.
+10. **Max 8 documents for taxonomy suggestion** — the suggest-taxonomy endpoint accepts at most 8 attachment references.
+11. **Claude is the reviewer, not the extractor** — IXP generates predictions, Claude validates them. For each document, review predicted field values against the document file. **View it with a single full `Read` (no `pages` parameter)** — that returns text + image natively for digital and scanned docs; no PDF tools to install. Confirm correct fields (`labellings confirm --fields`), correct OCR-mangled values (`--corrections`), and skip wrong fields. Do NOT manually extract values. If a field's F1 is low, improve the **prompt** so IXP predicts better values.
+12. **Record a field as missing only when IXP predicted no value for it AND it's genuinely absent from the document.** Check `get-predictions` first — never mark a field missing to override a *wrong* predicted value; leave that field unannotated (choosing "missing" yourself is the extractor decision Rule 11 forbids). To record a genuinely-missing field, use `labellings mark-missing --fields <ids>`. `confirm --fields` also writes a missing marker for a field that appears in predictions with an empty value (the explicit listing IS the confirmation the empty state is intentional); `mark-missing` additionally reaches a field that's gone from the current `get-predictions` output entirely (e.g. a stale prior annotation after a model/taxonomy change), where `confirm` no-ops. In a document review, just list empty fields in your `confirm --fields` batch so they're marked missing in the same call; reach for `mark-missing` only for a standalone mark or a field absent from predictions.
+13. **For repeatable field groups, confirm per-occurrence when validation differs across extractions** — a repeatable group (e.g. `Line Items`) produces one extraction per physical line/section. Plain `confirm --fields <id>` confirms `<id>` in **every** occurrence, so if only some lines are correct it confirms the wrong ones too. Each label in `get-predictions` carries an explicit 0-based `Occurrence` (repeated same-named labels appear in document order); if all occurrences are correct use the plain form, otherwise target with `--group`. `--group <name> --occurrence <N>` confirms **ONE** occurrence; `--group <name> --updates '[...]'` confirms **SEVERAL** in one atomic call (avoids N round-trips) — `--occurrence <N>` ≡ a single-entry `--updates`, same per-occurrence logic. `--group` must be the FULL label path from the `Name` field (e.g. `"Invoice > Line Items"`), not the leaf. Without `--fields`, every predicted field in the occurrence is confirmed; with it, only those. Occurrences not selected keep their existing annotation. Flag details: [CLI Reference](references/cli-reference.md#labellings).
+14. **`confirm` is additive — it never un-confirms.** The labelling endpoint is full-replacement, so `confirm`/`mark-missing` carry every existing annotation forward: `--occurrence 0` on an already-labelled table yields "row 0 confirmed AND everything previously confirmed stays confirmed" — NOT "only row 0". To roll back a confirmation, use `unconfirm` (see the task-navigation table).
+15. **F1 reflects confirmed labels, not document truth — never blind-confirm.** F1/`ProjectScore` measure prediction-vs-confirmed-label agreement, so a wrong value you confirm becomes the "right" answer and scores 1.00. A perfect score is **not** evidence the values are correct. Before confirming, sanity-check each value against the document. The per-document no-`--fields` form (confirm all predicted fields on one document) is fine once you've reviewed them all; never run `confirm` without a document-id — that confirms every document at once, bypassing review. See [Label Documents Guide](references/label-documents-guide.md) §2c.
+
+## Quick Start
+
+1. Run `uip ixp projects list --output json` to see existing projects
+2. To create a new project: follow [Project Setup Guide](references/project-setup-guide.md)
+3. To improve an existing project: follow [Improve Prompts Guide](references/improve-prompts-guide.md)
+4. To label documents on an existing project: follow [Label Documents Guide](references/label-documents-guide.md)
+
+If the user provides a taxonomy file, use `--skip-taxonomy` and `import-taxonomy` (Option B in the Project Setup guide).
+
+## Task Navigation
+
+| User request | Action |
+|-------------|--------|
+| "Create an IXP project" / "Upload documents to a new project" | [Project Setup Guide](references/project-setup-guide.md) — **new** projects only (uploads + taxonomy in one call). For **existing** projects, see the "Upload a document" row below. |
+| "Import this taxonomy" / provides a taxonomy file | [Project Setup Guide](references/project-setup-guide.md) — Option B (`--skip-taxonomy` + `import-taxonomy`) |
+| "Label documents" / "Review predictions" | [Label Documents Guide](references/label-documents-guide.md) |
+| "Improve scores" / "Fix prompts" / "Improve F1" | [Improve Prompts Guide](references/improve-prompts-guide.md) |
+| "Publish the model" / "Tag as live" | `uip ixp projects publish <project-name> --output json` — publishes the latest version, untagged. Add `--tag <live\|staging>` to also tag it. See [cli-reference](references/cli-reference.md) for `--model-version`/`--description`. |
+| "Roll back to a previous version" / "Restore version N" | `uip ixp projects publish <project-name> --model-version <N> --output json` — re-publishes an earlier version. Get available versions from `uip ixp projects list-models <project-name> --output json`. |
+| "Unpublish a model" / "Take a model out of production" | `uip ixp projects unpublish <project-name> --model-version <N> --output json` — removes a version from the published set (it stays trained/listable). `--model-version` is required; find published versions via `list-models` (`Pinned: true`). To change which version is live, `publish` a different one instead. |
+| "Remove the live/staging tag" / "Untag a version" | `uip ixp projects untag <project-name> --tag <live\|staging> --output json` — removes the named tag (the version it pointed at stays published). **`untag` is the only way to remove a tag** — do NOT `unpublish` or re-`publish` to clear it (`unpublish` removes publication, not the tag; `publish` without `--tag` leaves the existing tag untouched). To switch `live`→`staging`, `publish --tag staging` instead. |
+| "Show metrics" / "What are the scores?" | `uip ixp projects get-metrics <project-name> --output json` |
+| "List projects" | `uip ixp projects list --output json` |
+| "Configure the model" | `uip ixp projects configure-model <project-name> [options] --output json` |
+| "Delete a project" / "Remove this project" | `uip ixp projects delete <project-name> -y --output json` — **permanent and irreversible**; removes the project's documents, taxonomy, and trained models. Requires `-y/--yes` (the CLI never prompts). |
+| "Upload a document" / "Add documents to an existing project" | `uip ixp documents upload <project-name> <file> --output json` — see [CLI Reference § Uploading documents](references/cli-reference.md#uploading-documents-to-an-existing-project). One file per call; loop for multiple. For brand-new projects use `projects create` instead. |
+| "Delete a document" / "Remove a document" | `uip ixp documents delete <project-name> <document-id> -y --output json` — irreversible, triggers retrain. `-y/--yes` is required (the CLI never prompts). To delete by filename, look up the `DocumentId` via `documents list` (the `Filename` field shows the original upload name). |
+| "Add / delete / rename a field group" | `uip ixp groups {add,delete,rename} <project-name> --name <name> ... --output json` — see [CLI Reference § Groups](references/cli-reference.md#groups). `groups add` requires `--instructions` and `--fields '<json>'` — pass **all** of the new group's fields in that one `--fields` array (batch); do NOT create the group then add fields one at a time (use `fields add` only for an already-existing group). `delete` requires `-y/--yes` (the CLI never prompts). |
+| "Add / edit / rename / delete a data type" | `uip ixp data-types {add,update-instructions,rename,delete} <project-name> --name <name> ... --output json` — see [CLI Reference § Data Types](references/cli-reference.md#data-types). `add` requires `--kind` (text/date/money/number/boolean/choice) and `--instructions`. `--input-value` (exact-match/inferred) is required only for `--kind text` and `--kind choice`; the other kinds don't have this property and the CLI rejects the flag for them. `delete` requires `-y/--yes` (the CLI never prompts); deleting a data type breaks any field referencing it. |
+| "Add / delete / rename / retype a field" | `uip ixp fields {add,delete,rename,change-type} <project-name> --group <name> --field <name> ... --output json` — see [CLI Reference § Fields](references/cli-reference.md#fields). `delete` requires `-y/--yes`; `change-type` deletes annotations and also requires `-y/--yes`. |
+| "Mark a field as missing for a document" | `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json` — marks the listed fields missing; use when a field is genuinely absent and IXP predicted no value for it. Listing the field in `labellings confirm --fields` records the same missing marker when the field appears in predictions with an empty value. **Only mark a field missing if IXP also predicted nothing for it** — never to override a wrong prediction. See Critical Rule 12. |
+| "Undo / unconfirm a wrong confirmation" | `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json` — rolls back an earlier `confirm` or `mark-missing` for the listed fields (`confirm` can't un-confirm — Critical Rule 14). Every other annotation on the document is carried forward. With `--fields` alone, a field id shared across occurrences of a repeatable group is removed from **all** of them; to roll back specific occurrences, add `--group` with `--occurrence <N>` or `--updates '[…]'` (mirrors `confirm` — see the row below and Critical Rule 14). |
+| "Confirm one line item / extraction" / "Confirm only this occurrence" | `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json` — targets one specific extraction of a repeatable field group (0-based index). Without `--fields`, confirms every predicted field in that occurrence; with `--fields`, only those. Other occurrences untouched. For confirming multiple occurrences in one call, use `--updates '[…]'` instead. See Critical Rule 13. |
+| "Unconfirm one line item / extraction" / "Roll back only this occurrence" | `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json` — rolls back one specific extraction of a repeatable field group (0-based index, same as `get-predictions`/`confirm`). Without `--fields`, unconfirms every annotated field in that occurrence; with `--fields`, only those. Other occurrences untouched. For several occurrences in one call, use `--updates '[…]'` instead. See Critical Rule 14. |
+| "Set overall extraction instructions" / "Update project prompt" | `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json` — replaces the taxonomy-wide prompt (the "Overall extraction instructions" field in the IXP UI). Distinct from `fields update-prompts` (per-field) and `groups update-prompts` (per-field-group). |
+| "How is this project performing?" / "What's the F1?" | `uip ixp projects get-metrics <project-name> --output json` (+ `list-models` for the live version). If Data is `{ Metrics: null }` the model isn't validated yet — report that and stop. If the call instead returns `Result: Failure` with `ErrorCode: not_found` (a project with no trained model yet, e.g. no confirmed labellings), treat it the same way — report "no metrics yet" and stop. Otherwise Data is flat; report in order: (1) published version — live/published + `TrainedTime` (`Tags[]` Name=`live`, `Models[]` Pinned); (2) overall `ProjectScore`/`ProjectScoreQuality`; (3) per-group scores from `FieldGroups[]` (F1/Precision/Recall); (4) per-field scores from `Fields[]`, sorted lowest-F1 first. State numbers plainly; no "good enough" judgement unless asked; route low scores to [Improve Prompts Guide](references/improve-prompts-guide.md). Answer from these two calls only — no ad-hoc discovery (Critical Rule #1). |
+| "Describe this project" / "What's in it?" | Three calls, reported in order: (1) identity — `Title`/`Name` from `uip ixp projects get <project-name> --output json`; (2) current model — live/published + `TrainedTime` from `list-models`; (3) taxonomy — label-group/field counts from `uip ixp projects get-taxonomy <project-name> --output json` (raw artifact: counts live under `Data.dataset.label_groups` and `Data.dataset.entity_defs`, snake_case). Fold in performance (above) only if asked. Do NOT page `documents list` (its `Data` is a paged `{ Documents, Total, Offset, Limit }` envelope — use `Total` for a count) or read deployment bindings. Answer from these calls only (Critical Rule #1). |
+
+## Common Pitfalls
+
+| Symptom | Cause | Fix |
+|---------|-------|-----|
+| Metrics don't change after a prompt update | Re-evaluation hasn't completed | Wait ~2 minutes for retrain. |
+| ModelVersion doesn't advance | Retrain still in progress | Any change to model inputs (labellings OR instructions) triggers a full retrain. Wait ~2 min then retry. |
+| Field instructions conflict with label_def instructions | `fields update-prompts` only edits per-field instructions, NOT the parent label_def instructions | Before iterating, read the label_def `instructions` and update them with `groups update-prompts` if they contradict the per-field prompts. |
+
+## Unsupported Capabilities
+
+These requests fall outside the skill. Recognise the request, reply with the standard response, route the user. Do NOT enter discovery (`uip --help`, grep, source reading) — see Critical Rule #1.
+
+| User request | Standard response |
+|--------------|-------------------|
+| "Create a model" / "create a project" | **Documents or a taxonomy supplied →** use the [Project Setup Guide](references/project-setup-guide.md) (this skill creates the project from them). **Otherwise →** "I work on existing IXP projects rather than creating them from scratch. Create one in-product: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then I can label, review, and improve it." |
+| "Upload these files" / "add documents" | **Project named / already in context →** supported; upload it (see the "Upload a document" row in Task Navigation). **Otherwise →** "Name an existing project and I'll upload it — or upload in-product (e.g. for a new project): https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-deploying-models." |
+| "Deploy this model" / "push to staging / production / folder / environment / tenant" | "I don't drive deployment to folders, environments, or tenants — that's a product-side flow: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-deploying-models. I can publish a model version with `uip ixp projects publish <project-name> --output json`, but binding it to a folder/environment is done in-product." |
+| "Give X access" / "share this project" / "change roles or permissions" | "Access, roles, and permissions are managed in-product, not through this skill: https://docs.uipath.com/ixp/automation-cloud/latest/overview/managing-access." |
+| "Use this model in my automation / workflow / agent" / "call the extractor from a process" | "Consuming a published model inside an automation is an authoring task outside this skill. See https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-consuming-a-workflow." |
+| "Mine these emails / communications" / "set up Communications Mining" | "Communications Mining is a separate IXP capability this skill doesn't cover (this skill is document extraction). See https://docs.uipath.com/ixp/automation-cloud/latest/cm-user-guide/introduction-to-uipath-communication-mining." |
+| "Monitor the deployed model" / "how many docs did it process?" / "runtime throughput or incidents" | "Runtime/operational monitoring of a deployed model lives in Orchestrator, not this skill: https://docs.uipath.com/orchestrator/automation-cloud/latest/user-guide/about-monitoring. For design-time scores use `get-metrics` (see 'Show metrics')." |
+| "Edit a data type's values" / "add or remove a Choice option" | "The CLI can rename a data type, change its instructions (`data-types update-instructions`), or delete it — but it can't add or remove the **values** of an existing Choice data type. Do that by hand in-product on the Manage Taxonomy page: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then continue here." |
+
+## Reference Navigation
+
+- [CLI Commands Reference](references/cli-reference.md) — all `uip ixp` commands with options and output formats
+- [Project Setup Guide](references/project-setup-guide.md) — create a new project, review and label documents
+- [Improve Prompts Guide](references/improve-prompts-guide.md) — iterative optimization loop with regression detection
+- [Label Documents Guide](references/label-documents-guide.md) — reusable workflow for reviewing and confirming predictions

package/skills/uipath-ixp/references/cli-reference.md

@@ -0,0 +1,121 @@
+# CLI Commands Reference
+
+All commands use `uip ixp` prefix. Always append `--output json` when parsing output programmatically.
+
+> **Destructive commands require `-y, --yes`.** Every irreversible `uip ixp` command (all `delete`s and `fields change-type`) gates on `-y/--yes`; the CLI never prompts. `--confirm-data-loss` is accepted but is a **deprecated no-op** — it does NOT satisfy the confirmation, so always pass `-y/--yes`.
+
+## Projects
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp projects list [-l <limit>] [--offset <n>] --output json` | List IXP projects — returns a paged envelope `Data: { Projects: [{ Id, Name, Title, CreatedAt }], Total, Offset, Limit }` (rows under `Projects`, **not** a bare array). `-l, --limit` defaults 50 (range 1-10000); `--offset` defaults 0 to page. |
+| `uip ixp projects get <project-name> --output json` | Get a project |
+| `uip ixp projects create "<name>" <folder-path> [-d "<description>"] [--skip-taxonomy] --output json` | Create project and upload supported docs in `<folder-path>` (top-level only — sub-folders are not scanned; see [Supported document files](#supported-document-files)). By default suggests+imports taxonomy. `-d` provides context for better taxonomy suggestion. Use `--skip-taxonomy` to create a blank project (import taxonomy separately). Use `ProjectName` from output. |
+| `uip ixp projects import-taxonomy <project-name> <file> --output json` | Import taxonomy from a local JSON file. Accepts `{ field_types, label_group }` or `{ entity_defs, label_groups }` format. |
+| `uip ixp projects update-title <project-name> "<new-title>" --output json` | Update the display title of a project |
+| `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json` | Update the project's **Overall extraction instructions** — the taxonomy-wide prompt the model sees on every extraction (the field at the top of the IXP UI's Manage Taxonomy page). Distinct from per-field-group prompts (`groups update-prompts`) and per-field prompts (`fields update-prompts`). Replaces the existing value. |
+| `uip ixp projects get-taxonomy <project-name> --output json` | Export the raw IXP taxonomy artifact. Data is `{ status, dataset: { entity_defs, label_groups } }` — read `entity_defs` and `label_groups` under `dataset`. Intended for re-import (see `import-taxonomy`), not a human-readable view. |
+| `uip ixp projects get-metrics <project-name> [--model-version <N>] --output json` | Get validation metrics. **Validated model →** flat Data: `ProjectScore`, `ProjectScoreQuality`, `ValidatedDocuments`, `ModelVersion`, plus per-group `FieldGroups[]` (`FieldGroup`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`) and per-field `Fields[]` (`FieldGroup`, `FieldId`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`, `Annotations`, `Quality`). **Trained but not yet validated →** Data is `{ Metrics: null }` (not an error). **No trained model yet (e.g. a project with no confirmed labellings) →** the call returns a failure envelope `Result: Failure` with `ErrorCode: not_found` (no `Data`), NOT `{ Metrics: null }` — treat it as "no metrics yet". Defaults to the latest version; pass `--model-version <N>` (`-m`, `latest` or an integer) to scope to a specific trained version. |
+| `uip ixp projects configure-model <project-name> [options] --output json` | Configure extraction model. Options: `--model` (gemini_2_5_flash/gemini_2_5_pro/gpt_4o_2024_05_13) and `--preprocessing` (none/table_mini/table). |
+| `uip ixp projects list-models <project-name> --output json` | List all model versions and tags. Returns `Models[]` (`Version`, `ModelName`, `Pinned`, `TrainedTime`, `Description`), `Tags[]` (`Name`, `Version`, `UpdatedAt`), and `MaxPublished`. |
+| `uip ixp projects publish <project-name> [--model-version <N>] [--tag <live\|staging>] --output json` | Publish a model version — defaults to the latest; pass `-m, --model-version <N>` to pick a specific one. `-d, --description "<text>"` sets a description; `--tag <live\|staging>` tags the published version. |
+| `uip ixp projects unpublish <project-name> --model-version <N> --output json` | Unpublish a model version — it stays trained and listable; only its published status is removed. `--model-version` is **required**. Errors if the version isn't found or isn't currently published. To change which version is live, `publish` a different one instead. |
+| `uip ixp projects untag <project-name> --tag <live\|staging> --output json` | Remove a tag by **name** (`--tag` is **required**; tag names are unique within a project, so this is unambiguous even when one version holds several tags). The version the tag pointed at stays published; only that tag is cleared. Errors if no version carries the tag. Only `untag` removes a tag — `publish` without `--tag` leaves the existing tag untouched. To switch `live`→`staging`, `publish --tag staging` instead. |
+| `uip ixp projects delete <project-name> -y --output json` | **Permanently** delete a project — its documents, taxonomy, and trained models. **Irreversible.** `-y, --yes` is **required**; the command refuses to run without it (the CLI never prompts). |
+
+## Documents
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp documents list <project-name> [-l <limit>] [--offset <n>] --output json` | List documents — returns a paged envelope `Data: { Documents: [{ DocumentId, AttachmentRef, Filename }], Total, Offset, Limit }` (rows are under `Documents`, **not** a bare array). `AttachmentRef`/`Filename` may be `null`; `Filename` is the original upload filename. `-l, --limit` defaults 50 (range 1-10000); `--offset` defaults 0 (range 0-1000000). |
+| `uip ixp documents download <project-name> <document-id> -o <path> --output json` | Download the original document file (PDF/PNG/JPG/etc.) to exactly the `-o` path you give. **The CLI does not append or correct a file extension** — the file is written verbatim to `-o`, and `Data.ContentType` is commonly the generic `application/octet-stream` rather than the true MIME type — so include the correct extension in `-o` yourself (e.g. `-o invoice.pdf`). `Data.Path` echoes the path you provided. |
+| `uip ixp documents upload <project-name> <file> --output json` | Upload a single document file to an existing project. See [Uploading documents](#uploading-documents-to-an-existing-project) below for validation, output shape, and the multi-file loop pattern. |
+| `uip ixp documents delete <project-name> <document-id> -y --output json` | Delete a document (and its labellings) from a project. Irreversible — triggers a retrain. `-y, --yes` is **required** (the CLI never prompts; without it the command refuses and exits 1). |
+
+### Supported document files
+
+Both `projects create` (bulk folder upload) and `documents upload` (single file) validate against the same extension whitelist, case-insensitive:
+
+`.pdf`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.tif`, `.tiff`, `.bmp`
+
+Validation differs by command:
+
+- `documents upload` rejects an unsupported file with `Unsupported file type "<ext>"` before any network call.
+- `projects create` scans only the top level of `<folder-path>` (sub-folders are ignored), silently skips unsupported files, and fails only when **no** supported files exist (`No supported documents found in <folder>`).
+
+Each upload triggers a retrain — wait ~2 min before reading metrics or predictions for new docs.
+
+### Uploading documents to an existing project
+
+`uip ixp documents upload <project-name> <file> --output json` pushes one document to an existing project.
+
+For supported extensions, validation error strings, and retrain timing, see [Supported document files](#supported-document-files) above.
+
+Returns `{ ProjectName, Filename, AttachmentRef, DocumentId }` (Code: `IxpDocumentsUpload`). Capture `DocumentId` for later `documents download` or `labellings confirm` calls.
+
+**Multiple files** — one file per call; loop the command:
+
+```bash
+cd "<folder-with-docs>"
+for f in *.pdf *.png *.jpg *.jpeg *.gif *.tif *.tiff *.bmp; do
+    [ -e "$f" ] || continue   # skip unmatched glob patterns
+    uip ixp documents upload <project-name> "$f" --output json
+done
+```
+
+**When NOT to use this:** for filling a brand-new project, prefer `projects create <name> <folder-path>` — uploads the whole folder and suggests a taxonomy in one call.
+
+## Data Types
+
+Manage the reusable type definitions (entity_defs) that fields reference via `field_type_id`. In the IXP UI, these are the project's "Data Types".
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp data-types add <project-name> --name <name> --kind <text\|date\|money\|number\|boolean\|choice> --instructions <text> [--input-value <exact-match\|inferred>] [--choices <json>] --output json` | Create a new data type. `--kind` selects the underlying data shape; `text` is the default Text type. `--input-value` is **required for `--kind text` and `--kind choice`, forbidden for `date`, `money`, `number`, and `boolean`** — those kinds don't expose the "Exact match" / "Inferred" radio in the IXP UI, so the CLI rejects the flag when the kind doesn't support it. `exact-match` marks the value as appearing verbatim in the document; `inferred` is for computed/derived values that don't have a visible location. `--choices` is **required when `--kind choice`** and forbidden otherwise. JSON array of `{"value":"<canonical>","alternates":["<alt1>",...]}`; `value` is the canonical display name (model output); `alternates` is optional (defaults to `[]`) and lists alternate spellings the model maps to `value`. |
+| `uip ixp data-types update-instructions <project-name> --name <name> --instructions <text> --output json` | Replace the instructions on an existing data type. Name, kind, and input-value stay the same. |
+| `uip ixp data-types rename <project-name> --name <name> --new-name <name> --output json` | Rename a data type. Existing field references (via `field_type_id`) stay intact. |
+| `uip ixp data-types delete <project-name> --name <name> -y --output json` | Delete a data type. **IRREVERSIBLE** — any field referencing it via `field_type_id` will break. `-y, --yes` is **required** (the CLI never prompts). |
+
+## Groups
+
+Manage field groups (label_defs) — the document type containers for fields. To edit fields **inside** an existing group, use the `fields` subject below.
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp groups add <project-name> --name <group-name> --instructions <text> --fields <json> --output json` | Create a new field group with its fields. `--instructions` describes what document/section the group covers (the model sees it during extraction). `--fields` is a JSON array `[{"name":"...","type":"<type-name>","instructions":"..."}]` — **put ALL of the new group's fields in this one array (batch); do NOT create the group then add its fields one at a time.** Every entry must include `name`, `type`, and a non-empty `instructions`. `type` resolves against the project's `entity_defs`. To add a field to an **already-existing** group, use `fields add` instead. |
+| `uip ixp groups delete <project-name> --name <group-name> -y --output json` | Delete a field group. **IRREVERSIBLE** — deletes all annotations on all fields in the group. `-y, --yes` is **required** (the CLI never prompts). |
+| `uip ixp groups rename <project-name> --name <group-name> --new-name <name> --output json` | Rename a field group. Preserves all fields and annotations. |
+| `uip ixp groups update-prompts <project-name> --updates <json> --output json` | Bulk-update field group (label_def) instructions. `--updates` is a JSON array `[{"name":"<group>","instructions":"..."}]` matched by group name. Existing fields are preserved. Unmatched names are reported in the response without failing the command. |
+
+## Fields
+
+Structural edits to a field within an existing field group. For instruction-only edits use `fields update-prompts` (see below). To create the group itself, use `groups add` above.
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp fields add <project-name> --group <field-group-name> --field <name> --type <type-name> --instructions <text> --output json` | Add a new field to an **existing** field group. `--type` is the name of an entity_def in the project's taxonomy (see `projects get-taxonomy`). `--instructions` is required — describe what to extract and where it appears. |
+| `uip ixp fields delete <project-name> --group <field-group-name> --field <name> -y --output json` | Remove a field from a field group. `-y, --yes` is **required** (the CLI never prompts). |
+| `uip ixp fields rename <project-name> --group <field-group-name> --field <name> --new-name <name> --output json` | Rename a field. Preserves `field_id` and existing annotations. |
+| `uip ixp fields change-type <project-name> --group <field-group-name> --field <name> --type <type-name> -y --output json` | Change a field's type. **IRREVERSIBLE** — the server creates a new field under the hood, so all existing annotations for that field are deleted. `-y, --yes` is **required** (the CLI never prompts). |
+| `uip ixp fields update-prompts <project-name> --updates <json> --output json` | Bulk-update per-field extraction instructions. `--updates` is a JSON array `[{"name":"<field>","instructions":"..."}]` matched by `moon_form` field name (across all field groups). Existing field definitions are preserved. Unmatched names are reported in the response without failing the command. |
+
+## Labellings
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp labellings get-predictions <project-name> [document-id] --output json` | Get IXP model predictions for all documents (or a single document). Returns `Data: { ProjectName, TotalDocuments, DocumentsWithPredictions, Predictions[] }`. Each `Predictions[]` entry is one document `{ DocumentId, Labels[] }`; each label is `{ Name, Occurrence, Fields[] }`; each field is `{ FieldId, FieldName, FormattedValue }`. This is the model's **prediction** layer, not the confirmed/annotation layer. Each label carries an explicit `Occurrence` (the value for `--occurrence`/`--updates`); it is 0-based and usually runs 0..N-1 in document order, but do NOT assume it is contiguous or starts at 0 — a single-occurrence group can come back as `Occurrence` 1 with no 0. Always target the actual `Occurrence` value reported here, never a positional guess. |
+| `uip ixp labellings confirm <project-name> <document-id> [--fields <ids>] [--corrections <json>] --output json` | Confirm predictions for a document. (`--fields` has short alias `-f`; `--corrections` has short alias `-c`.) Without `--fields`, confirms every predicted field that has content. `--fields "a7c3e9105f2b4d86,b2f8a01c7d3e6940"` confirms only those fields, and applies a **single uniform rule**: listed fields with content get confirmed; listed fields whose IXP prediction is empty get a missing marker (the explicit listing IS the confirmation that the empty state is intentional — see Critical Rule 12). `--corrections '[{"field_id":"...","value":"..."}]'` is **only for OCR-mangled values** — same field, same location, garbled bytes. Do NOT use `--corrections` to flip wrong booleans, fix wrong inferred values, or override any non-OCR mistake; those fields must be left unannotated. See Critical Rule 8. Existing missing markers and other annotations carry forward across calls. |
+| `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] [--corrections <json>] --output json` | **Single-occurrence form** — confirm ONE occurrence. The ergonomic choice for a single line. `--occurrence` is the 0-based index of the target extraction within `--group`. Without `--fields`, confirms every predicted field in that one occurrence; with `--fields`, confirms only those fields there. Other occurrences are untouched. Requires `--group`. Mutually exclusive with `--updates`. |
+| `uip ixp labellings confirm <project-name> <document-id> --group <name> --updates <json> --output json` | **Batched form** — confirm SEVERAL occurrences in ONE atomic call (one request; avoids N round-trips, e.g. a 10-line invoice). `--updates` is a JSON array `[{"occurrence":<0-based-index>,"fields"?:["<field_id>",…],"corrections"?:{"<field_id>":"<value>"}}]`. Per entry: **omit `"fields"`** to confirm every predicted field in that occurrence (same default as `--occurrence` without `--fields`), or list specific IDs; un-selected fields in a selected occurrence carry forward any existing annotation. **`--updates` is the superset** — `--occurrence <N>` ≡ `--updates` with one entry; both share the same per-occurrence logic. Use `--occurrence` for a single line, `--updates` for several together. Mutually exclusive with `--fields`/`--corrections`/`--occurrence`. |
+| `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json` | Roll back confirmations on a document (`--fields` has short alias `-f`) — the listed fields go back to un-annotated state. Use when an earlier `confirm` was a mistake (confirm can't un-confirm — Critical Rule 14). Every other annotation on the document is carried forward. **With `--fields` alone, a field id shared across occurrences of a repeatable group is removed from all of them**; to scope the roll-back to specific occurrences, add `--group` (see the two rows below). Returns `Unmatched` for IDs that weren't annotated to begin with. |
+| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> [--occurrence <N>] [--fields <ids>] --output json` | **Per-occurrence form** — roll back specific occurrences of a repeatable group instead of every occurrence a field id appears in. `--group` alone unconfirms every occurrence of the group; add `--occurrence <N>` (0-based, same index as `get-predictions`/`confirm`) to roll back ONE occurrence. Without `--fields`, unconfirms every annotated field in the targeted occurrence(s); with `--fields`, only those there. Other occurrences are untouched. Mutually exclusive with `--updates`. Mirrors `confirm`'s `--group`/`--occurrence` flags. |
+| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --updates <json> --output json` | **Batched form** — roll back SEVERAL occurrences in ONE atomic call. `--updates` is a JSON array `[{"occurrence":<0-based-index>,"fields"?:["<field_id>",…]}]`. Per entry: omit `"fields"` to unconfirm every annotated field in that occurrence, or list specific IDs. Occurrences not listed are left as-is. Mutually exclusive with `--fields`/`--occurrence`. |
+| `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json` | Mark the listed fields as missing (`--fields` has short alias `-f`; annotated with no value and no location) — use when a field is genuinely absent from the document and IXP predicted no value for it. Unlike `confirm --fields`, it also marks a field that's gone from the current predictions entirely (e.g. a stale prior annotation after a model/taxonomy change), which `confirm` can't reach. **Only for fields where IXP predicted no value** — if IXP predicted a *wrong* value, leave the field unannotated instead. Returns `Unmatched` for any IDs not found in the document's annotation OR prediction. |
+
+## Deployments
+
+For working with runtime (deployed) IXP models — separate from the training workflow above.
+
+| Command | Description |
+|---------|-------------|
+| `uip ixp deployments get-taxonomy <project-name> --version <N> --output json` | Get the project taxonomy (data types + field groups) at a specific trained model version. `--version` is **required** (non-negative integer; 0 is valid; no short alias) — get the number from `projects list-models`. Like `projects get-taxonomy`, the body is the raw IXP dataset artifact in snake_case, under `Data.dataset` (`entity_defs[]` + `label_groups[]`), bound to the snapshot the version was trained on (Code: `IxpDeploymentsGetTaxonomy`). |

package/skills/uipath-ixp/references/improve-prompts-guide.md

@@ -0,0 +1,283 @@
+# Improve Prompts Guide
+
+Iterative optimization loop for improving extraction quality on an existing IXP project. Runs multiple iterations automatically, rolling back if scores regress.
+
+## What Prompts CAN and CANNOT Fix
+
+Before starting, understand the limits of prompt iteration:
+
+**Prompts CAN fix:**
+
+- Fields where the model extracts the wrong value (precision problems) — better instructions clarify what to extract
+- Fields where the model misses the value entirely (recall problems) — location hints help the model find the field
+- Ambiguous fields where the model picks the wrong candidate — negative examples and disambiguation rules help
+
+**Neither prompts nor reviewing can fix:**
+
+- **OCR quality issues** — if the OCR consistently garbles a field's text, no instruction will fix it. However, during the review step, OCR-mangled predictions can be corrected using `labelling confirm --corrections` (keeps the reference, fixes the text). If many fields are OCR-mangled across multiple documents, report this to the user as a data quality issue rather than burning prompt iterations.
+- **Missing fields** — if a field simply doesn't exist in the documents, no instruction will conjure it.
+
+## How prompt updates work
+
+Prompts live at two levels and are edited by two separate commands:
+
+- **`uip ixp fields update-prompts <project> --updates <json>`** — per-field instructions (e.g., "Invoice Number", "Invoice Date"). Match by field name.
+- **`uip ixp groups update-prompts <project> --updates <json>`** — field group (label_def) instructions (e.g., "Invoice", "Line Items"). Match by label_def name.
+
+Each command sends one server-side call; the server matches by name and writes per affected label_def, preserving every definition you didn't change. To update both field and group instructions in the same iteration, run the two commands back-to-back.
+
+**Aligning group and field instructions.** Each label_def (e.g., "Invoice") has its OWN `instructions` field that the model sees alongside per-field instructions. If the group instruction says "Extract only fields visible on the first page" but a per-field instruction says "Found in the summary table on page 2", the model gets contradictory signals. When updating field instructions, also update the parent group instruction with `groups update-prompts` if it contradicts.
+
+## Before Starting
+
+The user may specify a max number of iterations (default: 3). Track:
+
+- **Baseline metrics** — the per-field F1 scores before any changes
+- **Previous iteration metrics** — the per-field F1 scores from the last successful iteration
+- **Previous instructions** — the per-field (field) instructions from the last successful iteration (for rollback)
+
+Do NOT re-read the taxonomy or sample documents between iterations — use what you already have. Only re-read metrics after each instruction update + retrain cycle. This assumes no one modifies the taxonomy or documents externally during the loop. If the user mentions changes were made in the web UI, re-fetch the taxonomy and document list before continuing.
+
+## Step 1 — Setup (once, before the loop)
+
+### 1a. Get baseline metrics
+
+If documents were just labelled (or uploaded, or the taxonomy was edited), wait ~2 minutes for the resulting retrain to complete before reading metrics. Reading mid-retrain captures *pre*-change scores and corrupts every downstream comparison.
+
+```bash
+mkdir -p /tmp/ixp/<project-name>/{docs,text,taxonomies,prompts}
+uip ixp projects get-metrics <project-name> --output json
+```
+
+Note the `ModelVersion` from this baseline read — later iterations check that it advances after each `fields update-prompts` / `groups update-prompts` (see step 2e). If the value here looks identical to a known pre-labelling version, the retrain may still be in flight; wait another 60 seconds and re-fetch.
+
+Save the full per-field `Fields` array as `baseline_metrics`. This is the starting point you compare against. (For a validated model, get-metrics Data is flat — `Fields`/`FieldGroups`/`ValidatedDocuments` are top-level. An unvalidated model returns `Data: { Metrics: null }` instead — wait for retrain and re-fetch.)
+
+**Correlating metrics to field names:** The metrics `Fields` array returns `FieldId` but not the field name. To map them, join against the taxonomy's `field` entries:
+
+- For each metric entry: `FieldGroup` = label_def name, `FieldId` = the field's `field_id`
+- Find the matching field entry in the taxonomy where `field_id == FieldId` — its `name` is the human-readable field name
+
+Build this mapping once and reuse it throughout the loop.
+
+### 1b. Check model configuration
+
+If many fields have low scores across the board, the model configuration may be wrong (e.g., no table pre-processing for table-heavy documents). View sample documents and check if the current config matches the document type. If not, reconfigure:
+
+```bash
+uip ixp projects configure-model <project-name> \
+  --model gemini_2_5_flash \
+  --preprocessing <none|table_mini|table> \
+  --output json
+```
+
+See the [Project Setup Guide](project-setup-guide.md) Step 2 for the decision table.
+
+### 1c. Get taxonomy
+
+```bash
+uip ixp projects get-taxonomy <project-name> --output json
+```
+
+Save to `/tmp/ixp/<project-name>/taxonomies/v1.json`. Output is `{ status, dataset: { entity_defs, label_groups } }` (raw snake_case); each `dataset.label_groups[]` holds `label_defs` with their fields and current `instructions`. These per-field instructions are what you'll be iterating on. Increment the version after each prompt update (v2, v3, …).
+
+The field `name` (e.g., `"Invoice Number"`, `"Description"`) is what you pass to `fields update-prompts --updates`.
+
+### 1d. Read sample documents (2-3 documents)
+
+```bash
+uip ixp documents list <project-name> --output json
+
+# For each sample document:
+uip ixp documents download <project-name> <document-id> -o /tmp/ixp/<project-name>/docs/sample --output json
+```
+
+The `download` command auto-detects format and appends the correct extension — read the resolved `Path` from the response. View the document with the **Read tool** — one full Read per document, **no `pages` parameter** (returns text + image natively for PDF/PNG/JPG). Files persist across sessions — check for existing files before downloading.
+
+### 1e. Check for unlabelled documents
+
+Compare the document list against the metrics. If the metrics show fewer `ValidatedDocuments` than the total document count, some documents have no confirmed labellings (e.g., newly added documents). Review and label them first using the [Label Documents Guide](label-documents-guide.md), then wait ~2 minutes for retrain and re-fetch metrics before starting the loop.
+
+---
+
+## Step 2 — Optimization Loop
+
+Repeat the following for each iteration (up to max iterations):
+
+### 2a. Diagnose fields and field groups
+
+Use the current metrics (baseline on first iteration, post-relabel metrics on subsequent iterations). The metrics include both `FieldGroups` (per-group scores) and `Fields` (per-field scores).
+
+**Field group diagnosis:** Check `FieldGroups` first. If an entire group has low F1, the group-level instructions may need updating with `--groups` rather than fixing individual fields.
+
+**Per-field diagnosis:** Identify individual fields with F1 < 0.7 as targets. Diagnose each:
+
+1. **Classify the action:**
+   - `Documents = 0` AND `F1 = 0` → **SKIP**
+   - `Documents < 1` → **SKIP**
+   - Otherwise → **REFINE**
+
+2. **Diagnose the problem type** (use F1 as primary signal with few documents):
+   - `Precision < Recall` significantly → **PRECISION** — model extracts wrong values
+   - `Recall < Precision` significantly → **RECALL** — model misses the field
+   - Otherwise → **BOTH** — rewrite entirely
+
+Print a diagnosis summary showing each field group's score and each field's name, F1, precision/recall, and diagnosis.
+
+If no fields need REFINE, stop — the project is already at target quality.
+
+### 2a-check. Check for labelling gaps (before writing instructions)
+
+For each REFINE field with **Recall < 0.5**, check whether the problem is a bad prompt or a missing label:
+
+1. Look at the sample document images you already have from Step 1d
+2. For each low-recall field, check: **can you see this field's value in the document?**
+   - If yes, the model may have predicted it correctly but it wasn't confirmed in a previous round → re-fetch predictions and review those fields again
+   - If the field is genuinely not visible in the document → it's a prompt/recall issue, handle with instruction changes
+
+**If you find previously skipped predictions that are actually correct**, confirm them now using `labelling confirm --fields` for those specific documents and fields. Wait ~2 minutes for retrain and re-fetch metrics before continuing.
+
+**If no labelling gaps are found**, proceed directly to writing instructions.
+
+### 2b. Write improved instructions
+
+For each field marked REFINE, rewrite its `instructions`:
+
+- **PRECISION** → Be more specific about WHAT to extract and what NOT to extract
+- **RECALL** → Better describe WHERE to find the field
+- **BOTH** → Full rewrite — what, where, what to avoid
+
+**Instruction quality standards:**
+
+Focus on **what** to extract and **where** to find it. Do NOT specify format — the entity_def (field type) already handles that.
+
+- **Minimum length**: 120+ characters. Short instructions like "Extract the date" are too vague.
+- **Location hint**: describe WHERE in the document (section, header area, table, near a label). Keywords: "section", "header", "table", "top of", "labeled", "near".
+- **Real example**: include an actual value from the documents (e.g., "Example: '2106732'", "Example: 'SINV0077023'").
+- **Disambiguation**: if similar fields exist, clarify what NOT to extract (e.g., "Do NOT confuse with PO Number").
+- **No format patterns**: do NOT include "Format: MM/DD/YYYY" or similar — the entity_def type (Date, Monetary, Text) already defines the format. Adding format in instructions creates conflicting signals.
+
+**Good instruction** (145 chars):
+> "The unique invoice identifier, found in the header area near the top-right, labeled 'Invoice #' or 'Invoice Number'. Example: '2106732'."
+
+**Bad instruction** (25 chars):
+> "Extract the invoice number"
+
+**For fields visible in documents** — include location and a real example from the actual documents.
+**For fields NOT visible** — use a generic instruction with no example: "Extract [what] from this document, as it appears on the page."
+
+**Additional rules:**
+
+1. NEVER reference specific page numbers — use section headings or labels
+2. Each instruction targets one specific field (e.g., "Invoice Number", "Invoice Date")
+3. On iteration 2+, do NOT repeat the same instruction that failed last time — try a different approach (different wording, different location hints, add negative examples)
+
+### 2c. Update instructions
+
+Use **field names** for `--fields` and **label_def names** for `--groups`:
+
+```bash
+cat > /tmp/ixp/<project-name>/prompts/field_updates.json << 'FIELDS_EOF'
+[
+  {"name": "Invoice Number", "instructions": "The unique document identifier, found in the header area top-right. Example: 2106732, QC006."},
+  {"name": "Invoice Date", "instructions": "The date the invoice was issued. Use the exact format as written in the document. Found near the invoice number."}
+]
+FIELDS_EOF
+
+uip ixp fields update-prompts <project-name> \
+  --updates "$(cat /tmp/ixp/<project-name>/prompts/field_updates.json)" \
+  --output json
+
+# If group instructions also need updating, run a second command.
+cat > /tmp/ixp/<project-name>/prompts/group_updates.json << 'GROUPS_EOF'
+[
+  {"name": "Invoice", "instructions": "General invoice header fields including number, dates, payment terms, and totals."}
+]
+GROUPS_EOF
+
+uip ixp groups update-prompts <project-name> \
+  --updates "$(cat /tmp/ixp/<project-name>/prompts/group_updates.json)" \
+  --output json
+```
+
+The second call is optional — skip it if the group instructions don't need changing.
+
+**Post-update verification:** After the update, re-fetch the taxonomy, save it as the next version, and verify that field counts per label_def are unchanged:
+
+```bash
+uip ixp projects get-taxonomy <project-name> --output json > /tmp/ixp/<project-name>/taxonomies/v<N>.json
+```
+
+Compare the number of fields in each updated label_def against the previous version. If any fields are missing, **STOP the workflow immediately** and report to the user — the taxonomy was corrupted and needs manual restoration. The previous taxonomy version has the old instructions for rollback.
+
+### 2d. Review and confirm predictions for all documents
+
+Wait ~2 minutes for the model to retrain with the updated instructions, then review predictions for all documents using the [Label Documents Guide](label-documents-guide.md). The updated prompts should produce better predictions — review each document's predictions against the actual content and confirm the correct ones. Documents with incorrect predictions are skipped (their old labels remain).
+
+### 2e. Wait and get new metrics
+
+Wait ~2 minutes for the model to retrain with the new labellings, then:
+
+```bash
+uip ixp projects get-metrics <project-name> --output json
+```
+
+If `ModelVersion` hasn't advanced since the last check, wait another 60 seconds and retry.
+
+### 2f. Compare and decide
+
+Compare the new per-field F1 scores against the **previous iteration** scores.
+
+**Selective regression check:** For each field you updated this iteration, check if F1 dropped by more than 0.1:
+
+- **Regressed fields** (F1 dropped >0.1): roll back ONLY those fields' instructions to the previous iteration's version. Keep the improved instructions for fields that gained or held steady.
+- **Improved/unchanged fields**: keep their new instructions.
+
+If any fields regressed, do a selective rollback:
+
+```bash
+# Only include the regressed fields, not the whole iteration
+cat > /tmp/ixp/<project-name>/prompts/rollback.json << 'FIELDS_EOF'
+[{"name": "Vendor Address", "instructions": "previous instruction for this field only"}]
+FIELDS_EOF
+
+uip ixp fields update-prompts <project-name> \
+  --updates "$(cat /tmp/ixp/<project-name>/prompts/rollback.json)" \
+  --output json
+```
+
+Wait ~2 minutes for retrain. On the next iteration, try a **different approach** for the regressed fields only (different wording, shorter instruction, fewer examples).
+
+**Rollback caveat:** Rollback restores the previous instructions but the model needs to retrain. Expect only **partial recovery** — prefer small-scope iterations (few fields at a time).
+
+**No regression:** Accept the iteration. Update `previous_metrics` and `previous_instructions` with the new values.
+
+**Stopping criteria — stop the loop if:**
+
+- All fields meet the user's target F1 (default: 0.7)
+- Max iterations reached
+- No fields improved in the last 2 consecutive iterations (diminishing returns)
+
+---
+
+## Step 3 — Final Report
+
+After the loop ends, print a summary:
+
+```text
+Optimization complete after N iterations.
+
+Field               | Baseline F1 | Final F1 | Change
+--------------------|-------------|----------|-------
+Invoice Number      | 0.450       | 0.820    | +0.370
+Description         | 0.300       | 0.650    | +0.350
+Bill-To Name        | 0.900       | 0.900    | (unchanged)
+Vendor Address      | 0.600       | 0.400    | -0.200 (rolled back)
+
+Overall project score: X.XX → Y.YY
+Iterations: N total, M with rollbacks
+Fields still below target (F1 < 0.7): [list]
+Labelling gaps fixed: [list any fields re-labelled in 2a-check]
+```
+
+If fields still need work, suggest the user run another round with more iterations.