@uipath/skills 1.199.0 → 1.200.0

package/skills/uipath-process-mining/SKILL.md

@@ -0,0 +1,97 @@
+---
+name: uipath-process-mining
+description: "UiPath Process Mining via `uip pm` — build and operate a process app end-to-end from a CSV / event log: templates, data mapping, upload, ingest, the dbt (Snowflake) transformation layer, publish, and query it (metrics, percentiles, RCA). Covers `uipath.custom`, the `Cases.sql` optional-column gotcha, Case-linked data-model tables (add-table + re-ingest), the apply-not-reingest fix loop, fixing a wrong mapping in place via `apps data-mapping get|update` (no app rebuild), and editing the app model via `apps model fields` — a field's data kind / calculated fields, including the numeric→duration mismatch that locks dashboards open (DNA-46960). For Orchestrator/Data Fabric/Integration Service→uipath-platform. For `.flow`/Maestro→uipath-maestro-flow. For IXP→uipath-ixp."
+when_to_use: "User mentions process mining, a process app, an event log, `uip pm`, mining a CSV/log, ingesting data into one, dbt/SQL transformations, steps-to-resolution / throughput / variant / rework analysis, or querying one. Also 'build a process app from this data', 'ingest this log', 'fix my Cases.sql', 'why can't I query my custom table', 'add a table to the data model', 'group by X average Y', 'fix/change/read my data mapping', 'wrong date format in the mapping', 'change a field's data kind', 'set a field to duration', 'add a calculated field/metric', 'my dashboards won't open', 'Must be duration not numeric (DNA-46960)'. For Orchestrator/Data Fabric→uipath-platform; `.flow`→uipath-maestro-flow; IXP→uipath-ixp."
+allowed-tools: Bash, Read, Write, Glob, Grep
+---
+
+# UiPath Process Mining — `uip pm` Assistant
+
+Build and operate a UiPath Process Mining process app end-to-end from the terminal with `uip pm`: from a raw CSV to a queryable process model. The whole loop — templates, data mapping, upload, ingest, the dbt/Snowflake transformation layer, and querying — is scriptable; **use the CLI, don't hand-roll the Process Mining REST API.**
+
+**This works for every app type**, not just `uipath.custom`: the pipeline (mapping → upload → ingest → transform → data model → query) is identical across the `uipath.custom` event-log template and the source-system templates (P2P / O2C / IM / AP / … on SAP, Oracle, NetSuite, ServiceNow, Salesforce, …). Only **what the data mapping / extract must contain** differs. See [`references/app-types.md`](references/app-types.md).
+
+This skill is the **process-mining domain layer** — *what* to build and *why*. The
+low-level mechanics of driving the tool — the command-group map, the `Result`/`Code`/`Data`
+output envelope, the ETag get-modify-put pattern, `--wait`, `--stage`, and field-id
+discovery — are one layer down in [`references/uip-pm-cli.md`](references/uip-pm-cli.md).
+The rules below carry the headline command and link down to it and to the domain
+references for the full detail.
+
+## When to Use This Skill
+
+- **Build a process app from data** — you have a CSV / event log and want a mined process (throughput, variants, rework, steps-to-resolution).
+- **Author the transformation layer** — edit the dbt (Snowflake) SQL models that produce the process model, then re-run.
+- **Query a process app** — pull numbers out: aggregate group-by + metrics, raw detail rows, percentiles, root-cause analysis, process insights.
+- **Expose custom analysis** — surface your own analytical table (a weekly aggregate, an impact study) as a queryable entity.
+- **Edit the app model** — change a field's data kind, add calculated fields, or fix a data-kind mismatch that locks dashboards open (DNA-46960).
+- **Manage the app lifecycle** — stages (dev → published), RBAC, deletion.
+
+## App lifecycle
+
+An app moves through: **create** (from a template + data mapping) → **load** (upload + ingest) → **transform** on the **dev** stage (the ELT/dbt layer) → **publish** to the **published** stage → **query** / build dashboards. Develop against a small subset on `dev`, then publish the full dataset for real analysis ([`references/lifecycle-and-rbac.md`](references/lifecycle-and-rbac.md)). The **ELT editor** is the `transformations` command group over the dbt (Snowflake) model tree that turns loaded source tables into the process model — its command surface and the apply-vs-run distinction are in [`references/transformations.md`](references/transformations.md).
+
+## Critical Rules
+
+1. **To make a custom analytical table queryable, register it as a Case-linked data-model table, then RE-INGEST.** Process Mining is **case-centric**: a queryable table must be the `Cases` root or reach `Cases` via a foreign key — an unlinked table is rejected at query time (`UserError_TableIsDeleted`). First check the built-in Case-child slots: **`Tags`** (multi-valued per-case labels: `Tag`/`Tag_type`) and **`Due_dates`** (per-case SLA/deadline: `Expected_date`/`Actual_date`/`On_time`/`Cost`) — populate their dbt models rather than adding a table when your data fits. Otherwise register a custom table with **`uip pm apps data-model add-table <app> --file <table.json>`**, where the file is a DataModelDto entry `{ name, primaryKey, foreignKeys:[{table:"Cases",column:"Case_ID"}] }` (loose-link a standalone aggregate with a surrogate PK + nullable `Case_ID`). `add-table` edits `/dev/dataModel` (upsert, ETag-safe) then `applyCurrentDatamodel`; the table only becomes queryable after **`ingestions create --wait`** (a data-model edit takes effect only on the next ingestion). Full recipe + Tags/Due_dates decision table in [`references/data-model.md`](references/data-model.md).
+
+2. **Match the template to the data — the rest of the pipeline is identical for all app types.** A single denormalized log (Case, Activity, Timestamp [+ attributes]) ⇒ `uipath.custom` ("Event log"). Otherwise pick the `<process>.<system>` template matching your source system AND process (Purchase-to-Pay on SAP ⇒ `uipath.p2p.sap`; incidents from ServiceNow ⇒ `uipath.im.servicenow`) — but only when you actually have that system's **full multi-table extract**, not a single log you exported from it. Every template shares the same model shape and the same mapping→ingest→transform→query machinery; only the expected input tables differ. Discover with `app-types list`, inspect a template with `app-types get`. See [`references/app-types.md`](references/app-types.md).
+
+3. **Patch the `uipath.custom` `Cases.sql` optional-column gotcha (custom-only).** Source-system templates ship their own correct transformations — this gotcha is specific to the `uipath.custom` event-log template. The template's `models/Cases.sql` references `Event_log."Case"`, `"Case_status"`, `"Case_type"`, `"Case_value"`. A minimal mapping (Case_ID/Activity/timestamp only) doesn't produce those ⇒ dbt `000904 invalid identifier`. Fix: pull the file, replace the missing refs with `cast(null as varchar/float)`, push, and **`transformations apply`**. `Tags.sql`/`Due_dates.sql` are safe `where 1=0` stubs.
+
+4. **After a transform-only failure, `apply` — don't re-ingest.** The data is already loaded. Fix SQL (`transformations get` → edit → `transformations update --etag '<the get's ETag>'`, or `create` for a new file, which needs none) then `transformations apply` (re-transforms loaded data). Re-ingest only when the raw data or the mapping/parse settings change.
+
+5. **A wrong data mapping does NOT mean recreating the app — fix it in place with `apps data-mapping`.** The mapping is not create-only: `uip pm apps data-mapping get <app> --destination ./mapping.json` → edit → `uip pm apps data-mapping update <app> --file ./mapping.json --etag '<etag>'` replaces it on an existing app. **`--etag` is required** — pass the `Data.ETag` that *your* `get` returned, which is what proves the edit was based on the version you read; a lost race is refused `409 UserError_ETagFileConflict` (re-`get` for the new version **and** ETag, re-apply, retry), and a table-less file is refused rather than wiping the stored mapping. Unlike a SQL fix (Rule 4), a **mapping** change is a parse-setting change, so it takes effect only on the **next ingestion** — re-`files upload` if the source columns changed, then `ingestions create`. Only `dev` is writable (`published` is read-only). Facts + failure modes in [`references/pre-flight.md`](references/pre-flight.md).
+
+6. **Use `--wait` on async commands.** `ingestions create --wait` and `transformations apply --wait` block to a terminal state, print the dbt/loader error on failure, and exit non-zero — no hand-rolled `apps list` poll loop.
+
+7. **Query field ids come from `query info`, not column names.** `query run`/`percentile` bodies take the hashed `F__<Table>__<Col>__<hash>` ids. Prefer the sugar: `query run <app> --group-by <col> --metric <col>:<fn>` resolves human names for you (fn ∈ `average|count|sum|min|max`).
+
+8. **Develop on `dev` with a data subset; publish the full dataset.** The `dev` stage is for iterating on the mapping and transformations — keep it fast by loading a **small representative subset** of the data. Once the model is right, **publish** so the **published** stage carries the **full** dataset for the dashboards and sharing. Query/transform against `--stage dev`; consumers read the **published dashboards**. Note CLI `query --stage published` is currently unreachable (no `uip pm` path completes a published-stage ingestion) — do CLI querying on `dev` ([`references/lifecycle-and-rbac.md`](references/lifecycle-and-rbac.md)).
+
+9. **RBAC is folder/role-based at the platform layer, not the process app itself.** A process app lives in a folder; who can view vs. edit vs. publish is governed by Orchestrator/Identity roles and folder assignments — configure it with [`uipath-admin`](/uipath:uipath-admin) (roles, role assignments, effective-access) and [`uipath-platform`](/uipath:uipath-platform) (folders). See [`references/lifecycle-and-rbac.md`](references/lifecycle-and-rbac.md). `uip pm` itself does not grant access.
+
+10. **Edit a field's data kind / calculated fields with `apps model fields` — and a data-kind mismatch can lock the app open.** Change a field's kind (e.g. numeric→duration), rename it, or add a calculated field with `uip pm apps model fields set <app> <field> [--kind|--display-name|--expression]` (the **semantic** model; dev-only, and **no `--etag`** — it merges into the version it just read, so a lost race is fixed by re-running it; a whole-document `apps model update` does require `--etag`). Relational/arithmetic operators require both operands to share a data kind, so flipping a field to `duration` while a metric / calculated field / dashboard filter still compares it to a `numeric` constant persists an invalid model that throws at dashboard open — the *"Must be duration, not numeric, for the 'lt' input"* lockout (DNA-46960), which leaves only the data-upload module reachable. `fields set`/`update` validate and refuse such an edit with a hint; fix an already-broken app by making the comparison consistent (re-type the field or the constant). Full surface + the data-kind rule in [`references/model-editing.md`](references/model-editing.md).
+
+## Quick Start
+
+The end-to-end CSV → queryable-app command sequence (discover template → create →
+upload → ingest → patch transform / fix mapping → query) is in
+[`references/uip-pm-cli.md`](references/uip-pm-cli.md#quick-start--csv--queryable-process-app).
+
+## Extending the model with custom analysis
+
+The killer use case is your own SQL. Add analytical dbt models with `transformations create <path> --file` (use `update` for existing files; inline intermediates as CTEs if you prefer fewer files), then **register each queryable output as a Case-linked data-model table + re-ingest (Rule 1)** so `query` can read it. Full recipe + the DataModelDto entry shape (`type`/`name`/`primaryKey`/`foreignKeys`) and the Tags/Due_dates decision table in [`references/data-model.md`](references/data-model.md); the transformation dev loop and dbt/pm_utils notes in [`references/transformations.md`](references/transformations.md); the query AST and sugar in [`references/querying.md`](references/querying.md).
+
+## Reference Navigation
+
+Two layers: the **`uip pm` CLI** reference (how to drive the tool) and the
+**process-mining domain** references (what to build and why). Start with a domain
+reference for the decision; drop into the CLI reference for the mechanics it uses.
+
+| File | Read when |
+|------|-----------|
+| [`references/uip-pm-cli.md`](references/uip-pm-cli.md) | **CLI mechanics (low-level)** — the command-group map, the `Result`/`Code`/`Data` envelope + exit codes, the ETag get-modify-put pattern, `--wait`, `--stage`, `IngestionNeeded`, field-id discovery, and the CSV→queryable-app Quick Start |
+| [`references/app-types.md`](references/app-types.md) | choosing/targeting a template — custom vs source-system, why the pipeline is the same for all, what the mapping/extract must contain per family |
+| [`references/pre-flight.md`](references/pre-flight.md) | before any upload — encoding/delimiter/date-format/empty-row checks and the minimal `mapping.json` recipe; **also** the post-create mapping fix loop (`apps data-mapping get`/`update`) and its failure modes |
+| [`references/transformations.md`](references/transformations.md) | authoring/fixing dbt models — the `Cases.sql` patch, apply-vs-run, pm_utils macros, Snowflake identifier quoting |
+| [`references/data-model.md`](references/data-model.md) | exposing a custom table to `query`/dashboards — the case-centric add-table pattern (DataModelDto + re-ingest) and the Tags/Due_dates decision table |
+| [`references/model-editing.md`](references/model-editing.md) | editing the app model — a field's **data kind** (e.g. numeric→duration), calculated fields, the two models (semantic `apps model` vs structural `apps data-model`), and the data-kind comparison rule that locks an app open (DNA-46960) |
+| [`references/querying.md`](references/querying.md) | pulling numbers out — the aggregate body AST, the `--group-by/--metric` sugar, the `AggregationFunction` enum, and the event-table restriction |
+| [`references/lifecycle-and-rbac.md`](references/lifecycle-and-rbac.md) | dev vs published stages, publishing, and where process-app RBAC is configured |
+
+## Anti-patterns — what NOT to do
+
+- **Repurposing `Tags.sql`/`Due_dates.sql`** to smuggle an *unrelated* analytics table through a pre-registered entity. Fine — intended, even — to populate them with their real semantics (per-case labels; per-case SLAs); wrong to jam a weekly aggregate into `Due_dates` to dodge add-table. It corrupts those features and fights their primary key. Register a real Case-linked table instead (Rule 1).
+- **Adding a data-model table with no link to `Cases`** — it registers but every query fails `UserError_TableIsDeleted`. Give a standalone table a surrogate PK + nullable `Case_ID` FK to `Cases` (Rule 1).
+- **Forgetting to re-ingest after `add-table`.** The data-model edit is inert until the next `ingestions create` re-materializes the tables (Rule 1).
+- **Re-uploading + re-ingesting after a transform-only failure.** The data is loaded; fix the SQL and `transformations apply`. Re-ingest only when raw data or parse settings change (Rule 4).
+- **Deleting and recreating an app to fix a mapping mistake** (or telling the user that's the only option). The mapping is editable after creation — `apps data-mapping get`/`update` (Rule 5). Recreating also throws away the transformations you already patched.
+- **`transformations apply` after a mapping change.** `apply` only re-runs SQL over *already-parsed* data; a new mapping changes how the raw file is parsed, so it needs `ingestions create` (Rule 5). This is the mirror of Rule 4 — get the direction wrong and the edit silently appears to do nothing.
+- **Re-`get`ting a resource just to harvest a fresh `--etag` for a rejected write.** That defeats the `If-Match` guard — it makes the precondition pass no matter who wrote in between, silently overwriting them. A 409/412 means the resource moved: re-`get` the latest **document**, re-apply your change on top of *that*, then write with the ETag that read returned. Never pair a stale local file with a freshly fetched ETag ([`references/uip-pm-cli.md`](references/uip-pm-cli.md)).
+- **Hand-rolling an `apps list` poll loop.** Use `--wait` on `ingestions create` / `transformations apply` (Rule 6).
+- **Passing column names in a raw `query run` body**, or hand-writing the aggregate AST. Bodies take hashed field ids from `query info`; use the `--group-by/--metric` sugar (Rule 7).
+- **Patching `Cases.sql` on a source-system template.** That gotcha is `uipath.custom`-only; source templates ship correct transformations — feed the expected extract and extend, don't rewrite (Rule 3).
+- **Using a source template for a single flat log** (or `uipath.custom` for a full multi-table extract). Match the template to the data shape (Rule 2).
+- **Iterating on the full dataset.** Develop on `dev` with a small subset; publish the full data (Rule 8).
+- **Changing a field's data kind while a comparison still uses the old kind.** Flipping a field to `duration` (or any kind) while a metric / calculated field / dashboard filter compares it to a constant of the old kind persists an invalid model that locks the app open (Rule 10). Reconcile the comparison first — re-type the field or the constant.

package/skills/uipath-process-mining/references/app-types.md

@@ -0,0 +1,66 @@
+# App types — the pipeline is template-agnostic
+
+Everything in this skill — data mapping, `files upload`, `ingestions create`,
+the `transformations` (ELT) layer, the data model + **add-table**, dev/published
+stages, and `query` — works for **every** app type, not just `uipath.custom`.
+What changes between templates is only **what the data mapping / extract must
+contain**. The machinery around it is identical.
+
+## The two families
+
+| Family | Examples | What you feed it |
+|--------|----------|------------------|
+| **Custom event log** | `uipath.custom` ("Event log") | ONE flat log you build: Case_ID, Activity, Event_end (+ attributes). You construct the event log. |
+| **Source-system templates** | `uipath.p2p.sap`, `uipath.o2c.oraclecloud`, `uipath.im.servicenow`, `uipath.im.salesforce`, `uipath.ap.sap`, `uipath.q2c.netsuite`, … (P2P / O2C / IM / AP / Q2C / … across SAP, Oracle EBS/JDE/Cloud, NetSuite, ServiceNow, Salesforce, Coupa, Ariba) | The source system's **full multi-table extract**, mapped to the template's expected input tables. The template already contains the extraction + event-log construction as dbt models. |
+
+Discover the full list per tenant:
+
+```bash
+uip pm app-types list --output-filter "[].{Key:AppTypeKey,Version:Version,Name:DefaultName}"
+```
+
+## Same model shape everywhere
+
+Every template's `model` has the same top-level shape — `Processes`, `Metrics`,
+**`Tables`**, `Automations`, `DefaultObject` — only the contents differ. The
+semantic entities (`model.Tables[]`) are template-specific: `uipath.custom` ships
+`Cases`/`Event_log`/`Tags`/`Due_dates`; `uipath.im.servicenow` ships
+`Incidents`/`Tags`/`Due_dates`/`__Incident_process_Events`; a P2P template ships
+its purchase-order entities. Inspect a template's model + metrics with:
+
+```bash
+uip pm app-types get <key> <version>
+```
+
+Because the shape is uniform, **the data-model rules generalize**: `query info`
+exposes whatever is in `Tables[]`, and you expose a custom analytical table on
+**any** app type by adding a `Tables[]` entry (see [`data-model.md`](data-model.md)) —
+add-table is not custom-only.
+
+## Choosing and targeting a template
+
+1. **Pick the template.** A single denormalized event log ⇒ `uipath.custom`.
+   Otherwise pick the `<process>.<system>` template that matches your source
+   system AND process (e.g. Purchase-to-Pay on SAP ⇒ `uipath.p2p.sap`). Use a
+   source template only when you actually have that system's expected multi-table
+   extract — not a single log you happened to export from it.
+2. **Learn the expected input.** For a source template, the expected **input
+   tables** (what your extract must provide) are defined by the template's
+   transformation layer, not the `model` object. Create the app, then read the
+   generated sources with `transformations list`/`get` (look at
+   `models/schema/sources.yml`), or follow the template's extractor docs. Build
+   the `--data-mapping` to match those input tables + fields.
+3. **Everything else is identical.** `files upload --input-table <name>` per
+   table, `ingestions create --wait`, fix transforms with `apply` (not re-ingest),
+   extend with custom models + add-table, develop on `dev` (subset) and publish
+   the full dataset, and `query` with the `--group-by/--metric` sugar.
+
+## What is template-specific
+
+- **The `Cases.sql` optional-column gotcha** ([`transformations.md`](transformations.md))
+  is a **`uipath.custom`** issue. Source templates ship their own (already-correct)
+  transformations — you don't patch their event-log construction; you feed the
+  expected extract and, if needed, **extend** with extra models + data-model tables.
+- The set of shipped `Metrics` and semantic entities differs per template — always
+  `query info` (on a built app) or `app-types get` (on the template) to see what's
+  available before writing query bodies.

package/skills/uipath-process-mining/references/data-model.md

@@ -0,0 +1,130 @@
+# Data model — making tables queryable (the add-table pattern)
+
+Process Mining is **case-centric**. A table is queryable only if it is the **Cases**
+root or reaches `Cases` through a foreign key. A standalone dbt model with no link
+to `Cases` is treated as disconnected and rejected at query time
+(`UserError_TableIsDeleted`).
+
+## Two models — do not confuse them
+
+| Model | Endpoint | Shape | Role |
+|-------|----------|-------|------|
+| **Data model** (structural) | `/apps/{id}/{stage}/dataModel` | `tables[]` of `{ type, name, primaryKey, foreignKeys }` | What tables exist + how they link. **`apps data-model add-table` edits this; `apps data-model get` reads it.** |
+| **Semantic model** | `/apps/{id}/{stage}/model` | `Processes` / `Metrics` / `Tables[].Fields[]` | Field-level view `query info` (and `apps model get`) reads; edited by `apps model fields …` ([`model-editing.md`](model-editing.md)). `applyCurrentDatamodel` **reconciles** structural changes into it, preserving your semantic edits (calculated fields, metrics, data-kind overrides) — so add-table won't wipe them; just don't hand-author its per-column fields, edit the structural model. |
+
+Edit the structural data model; `applyCurrentDatamodel` regenerates the semantic
+model (its per-column fields) from it. `add-table` does both.
+
+## The built-in Case-child tables — use these before adding your own
+
+The `uipath.custom` template ships four data-model tables. Two are the process
+backbone; two are ready-made Case-child extension slots. **Check whether your data
+fits Tags or Due_dates before hand-rolling a custom table** — they exist to save you
+the add-table round-trip.
+
+| Table | PK / link | Fields | Use it for |
+|-------|-----------|--------|------------|
+| **Cases** | `Case_ID` (root) | case attributes | One row per process instance (the case). The root everything links to. |
+| **Event_log** | FK→Cases | activity, timestamp, resource… | The event log — one row per activity. The process itself. Not directly group-by-able (it drives the process graph). |
+| **Tags** | `Tag_ID`, FK→Cases | `Tag` (label), `Tag_type` (category) | **Multi-valued categorical labels per case.** A case can carry many tags. Use for flags/segments/attributes that don't fit one Case column — e.g. `Tag_type="Region", Tag="EU"`; `Tag_type="Flag", Tag="Escalated"`. Filter/group cases by tag. |
+| **Due_dates** | `Due_date_ID`, FK→Cases | `Due_date`/`Due_date_type` (which deadline), `Expected_date`, `Actual_date`, `On_time` (bool), `Cost` (currency), `Difference` (duration) | **Per-case SLA / deadline / milestone tracking.** A case can have many due dates (multiple SLAs/milestones). Use for on-time %, SLA-breach counts, cost-of-breach, expected-vs-actual gaps. |
+
+Both `Tags` and `Due_dates` are themselves Case-child tables (FK→`Cases`) — they are
+the template's built-in example of the loose-link pattern below. Populate them by
+authoring their dbt models (`Tags.sql` / `Due_dates.sql`) to emit real rows keyed on
+`Case_ID`. This is the **intended** use — distinct from the anti-pattern of
+repurposing them to smuggle unrelated analytics through (see Anti-patterns).
+
+**Decision:**
+- Per-case label / category, possibly many per case → **Tags**.
+- Per-case deadline / SLA / milestone with target vs actual → **Due_dates**.
+- Per-case fact that fits one column → add a column to `Cases` (via `Event_log`).
+- Anything not one-row-per-case (a weekly aggregate, a cross-case study) → a **custom
+  Case-child table** via `add-table` (loose-linked; below).
+
+## Why a bare dbt model is not queryable
+
+`transformations create models/Workload_weekly.sql` builds a physical Snowflake
+table, but `query info` will not list it and `query run` cannot group by it. Two
+gates:
+
+1. **Not in the data model.** dbt produces tables; the data model decides which
+   become queryable entities. Register it (`add-table`).
+2. **Not linked to Cases.** Even once registered, a table with no path to `Cases` is
+   disconnected → `UserError_TableIsDeleted`. Give it a foreign key to `Cases`.
+
+And one timing gate: `existingTables` (what the query layer treats as "live") is
+derived from the **last successful ingestion's** materialization — so a data-model
+edit only takes effect after a **re-ingest**.
+
+## The data-model table entry (DataModelDto)
+
+```json
+{
+  "type": "Object",
+  "name": "Workload_weekly",
+  "primaryKey": "Workload_ID",
+  "foreignKeys": [{ "table": "Cases", "column": "Case_ID" }]
+}
+```
+
+| Key | Meaning |
+|-----|---------|
+| `type` | `"Object"` (default if omitted). |
+| `name` | MUST equal the dbt model / physical table name. |
+| `primaryKey` | A column that uniquely identifies a row — add a surrogate (`{{ pm_utils.id() }}`) if the table has none. |
+| `foreignKeys` | `[{ table, column }]` links to a parent. For a standalone analytical table, link **loosely** to `Cases` on a nullable `Case_ID` column. |
+
+Per-column display/kind is derived by `applyCurrentDatamodel` — you do **not**
+hand-author a `Fields[]` array.
+
+## Loose-link recipe: expose a custom analytical table
+
+The table isn't one-row-per-case, but must still reach `Cases`. Give it a **surrogate
+PK** and a **nullable `Case_ID`** carrying the FK — a null FK is enough to satisfy the
+case-centric graph; aggregate queries don't need it to resolve to real cases.
+
+**Caveat — a null `Case_ID` makes the table analytically disconnected.** Case-level
+filters/selections (how PM dashboards normally scope data) won't propagate to it, and
+it can't be joined back to real cases. Use the null-FK loose link **only** for a
+genuinely case-independent aggregate (a weekly total, a cross-case study). If its rows
+*do* correspond to real cases, populate `Case_ID` with the real key so case filtering
+flows through.
+
+1. Author the dbt model. First two selected columns:
+
+   ```sql
+   select
+       {{ pm_utils.id() }}   as "Workload_ID",   -- surrogate PK
+       cast(null as varchar) as "Case_ID",       -- loose FK to Cases
+       ...                                        -- your real columns
+   ```
+
+2. Build, register, **re-ingest**, query:
+
+   ```bash
+   uip pm transformations create <app> models/Workload_weekly.sql --file ./Workload_weekly.sql
+   uip pm transformations apply <app> --wait
+   uip pm apps data-model add-table <app> --file ./Workload_weekly.table.json  # edits /dev/dataModel + applyCurrentDatamodel
+   uip pm ingestions create <app> --wait                                       # REQUIRED — materializes the table
+   uip pm query run <app> --group-by Service_Component --metric Closed_Interactions:sum --output table
+   ```
+
+   Where `Workload_weekly.table.json` is the DataModelDto entry above.
+
+`add-table` GETs `/dev/dataModel` (with its ETag), **upserts** the table by name
+(replace if present, else append), PUTs it back `If-Match`-guarded, then POSTs
+`applyCurrentDatamodel`. Because it merges into exactly the document it just read,
+that ETag is a genuine compare-and-swap — so **`add-table` takes no `--etag`**, unlike
+`data-mapping update` / `model update` / `transformations update`, which replace a file
+you edited locally and therefore require it. A concurrent edit surfaces as `412`; **just
+re-run** — `add-table` re-reads and re-applies on top of the other write. (A data model
+returned without an ETag fails the command rather than writing unguarded.) It returns
+`IngestionNeeded: true` — the entity is not queryable until the re-ingest completes.
+
+## Publish vs re-ingest
+
+- **Re-ingest** (`ingestions create`) materializes the table so **dev** `query` sees
+  it. Required for add-table.
+- **Publish** (`apps publish`) pushes dev changes to the **dashboards / published**
+  stage. Separate step; not needed just to query in dev.

package/skills/uipath-process-mining/references/lifecycle-and-rbac.md

@@ -0,0 +1,67 @@
+# Stages (dev vs published) & RBAC
+
+## The two stages
+
+A process app has two stages, selected by `--stage` on data / transformation /
+query commands (default `dev`):
+
+- **`dev`** — the development stage. Iterate on the data mapping and the dbt
+  transformations here. **Keep it fast by loading a small, representative subset**
+  of the data: ingest a sample, get the mapping + `Cases.sql` + your custom models
+  and data-model tables right, verify with `query`, then move on. Short feedback
+  loops matter — a full re-transform on a large dataset is slow.
+- **`published`** — the stage consumers use through the **dashboards / shared UI**,
+  carrying the **full dataset**. `apps publish` promotes the definition here.
+
+Typical loop: develop and validate on `--stage dev` with a subset → publish → let
+consumers read the **dashboards** on the published data.
+
+> **CLI caveat:** `uip pm query --stage published` is currently **not reachable** —
+> `/query/{id}/published` needs a completed ingestion on that stage and no `uip pm`
+> path produces one (`apps publish` answers `IngestionNeeded: true`, but a following
+> `ingestions create --wait` still leaves published querying at
+> `UserError_InvalidOrNoIngestion`; verified against a live tenant). So publishing
+> promotes the app to the **dashboards**, but **CLI-driven `query` stays on `dev`**.
+
+## Publishing
+
+**`uip pm apps publish <app-id>`** promotes the validated dev app (mapping +
+transformations + data model) to the published stage, so dashboards and the query
+layer see it.
+
+The command reads the app's current model version off the data-model ETag and
+sends it as the publish precondition, so a stale caller fails instead of
+clobbering a newer model. The result envelope carries:
+
+- **`Changes`** — what the publish moved.
+- **`IngestionNeeded`** — when `true`, the published stage still needs a
+  re-ingestion before the change reaches the **data**. Dev transformation *or*
+  data-model changes (including `apps data-model add-table`) only become queryable
+  after a re-ingest; publishing alone promotes the definition, not the rows.
+
+So the full promote loop is: validate on `dev` → `uip pm apps publish <app>` →
+`uip pm ingestions create <app> --wait` when `IngestionNeeded` → analyse on
+`--stage published`.
+
+## RBAC — configured at the platform layer, not in `uip pm`
+
+Access to a process app is **not** granted by `uip pm`. A process app lives in a
+**folder**, and who can see / edit / publish it is governed by Orchestrator +
+Identity **roles and folder assignments**:
+
+- **Roles & assignments** — create/inspect roles and assign them to users/groups,
+  and check effective access, with [`uipath-admin`](/uipath:uipath-admin)
+  (Identity Server, Authorization, check-access PDP).
+- **Folders** — organize apps and scope access with folders via
+  [`uipath-platform`](/uipath:uipath-platform).
+
+Quick guidance:
+
+1. Put the process app in a dedicated folder for the audience that should see it.
+2. Assign a **view** role to consumers (they read published dashboards / run
+   `query --stage published`) and an **edit/publish** role to the small team that
+   maintains the mapping and transformations.
+3. Verify with an effective-access / check-access query before sharing.
+
+Keep least privilege: most users need view on published only; editing dev
+transformations is a maintainer capability.

package/skills/uipath-process-mining/references/model-editing.md

@@ -0,0 +1,112 @@
+# Editing the Process Mining app model
+
+There are **two** models behind a process app. Editing the wrong one is the most common source
+of confusion, so be precise about which you mean.
+
+| | `apps model` (semantic) | `apps data-model` (structural) |
+| --- | --- | --- |
+| Endpoint | `/apps/{id}/{stage}/model` | `/apps/{id}/{stage}/dataModel` |
+| Contains | `data` → tables → **fields with their `kind`** (data kind), **calculated fields**, **metrics**; plus `view` → dashboards, charts, `metricFilters` | Tables with `primaryKey`/`foreignKeys` and the process-mining **role columns** (`activityColumn`, `endColumn`, …) |
+| Think of it as | "the app definition" the user sees and edits | "the table plumbing" — which tables exist and how they join to `Cases` |
+| Edited by | `fields set/remove`, `update`; the data manager & dashboard editor | `add-table`; the data-model editor |
+
+`query info` shows the resolved *query* model (field ids, physical `ColumnDataType`, metrics) — useful
+to discover the exact field ids to pass to `fields set` and `query`.
+
+## Field editing surface
+
+```bash
+uip pm apps model fields list <app-id> [--stage dev|published]
+uip pm apps model fields set  <app-id> <field-id> [--kind <k>] [--display-name <t>] [--expression <json|@file>] [--table <table-id>]
+uip pm apps model fields remove <app-id> <field-id>
+```
+
+- **Upsert semantics.** If `<field-id>` exists, its `--kind` / `--display-name` are updated, and
+  passing `--expression` turns it into (or updates) a **calculated field**. If it does not exist, a
+  new **calculated field** is created in `--table` — so `--table` + `--expression` (+ `--kind`) are
+  required to create. Mapped *column* fields can't be created through the model; they come from the
+  ingested data.
+- **Data kinds you can set (`--kind`):** `ordinal, nominal, numeric, datetime, boolean, percentage, currency, duration`
+  — the union of the data manager's field-type options (FE `ColumnDataTypeFieldCompatibilityMap`).
+  `duration`, `currency`, `percentage` are **user choices stored in the model `kind`** (a number column
+  defaults to `numeric` — the user upgrades it). `id` and `ref` are **structural** (system-assigned to
+  key/reference columns) and not settable, though `fields list` may report a field that already has them.
+- **Expressions** are JSON expression-node trees, the same shape the app model stores. A comparison:
+  ```json
+  {"type":"operator","operation":"lt",
+   "left":  {"type":"reference","referenceType":"field","reference":"<field-id>"},
+   "right": {"type":"constant","dataType":"duration","value":86400000}}
+  ```
+  Operators: `lt le gt ge eq ne and or add subtract multiply divide percentage`. Constant
+  `dataType` **must match** the data kind of what it's compared to (see below). Reference a field
+  with `{type:"reference","referenceType":"field","reference":"<field-id>"}`.
+
+Every edit is `If-Match`-guarded and applies on `dev`, returning the new edit `Versions` — but the two
+routes differ in who supplies the ETag:
+
+- **`fields set` / `fields remove` take no `--etag`.** They read the model and merge your change into
+  exactly that version, so the read's own ETag is a real compare-and-swap. On a lost race, **just
+  re-run** — they re-read and re-apply. (They refuse to write at all if the read came back without an
+  ETag, rather than writing unguarded.)
+- **`apps model update` REQUIRES `--etag`** — it replaces a document you edited locally, so it must
+  carry the `Data.ETag` that `apps model get` returned. On 409/412, re-`get` for the latest model
+  **and its new ETag**, re-apply your change on top, then update again with the new `--etag`.
+
+```bash
+uip pm apps model get <app> --destination model.json     # prints Data.ETag
+uip pm apps model update <app> --file model.json --etag 'W/"3"'
+```
+
+Prefer `fields set` for a targeted change: no ETag to thread, and it can't clobber unrelated parts of
+the model. After editing, `publish` to reach the dashboards, and re-ingest if a data kind changed.
+
+## The data-kind rule
+
+Relational/arithmetic operators require their operands to share a data kind (backend
+`OperatorRelationalOrdering` / `CheckFunctionArguments`). So a comparison like `field < constant` is
+only valid when the constant's `dataType` equals the field's `kind`. If they differ the model fails
+validation with:
+
+```
+UserError_UnsupportedOperatorArgumentDataKind
+{ argument:"right", operation:"lt", actual:"numeric", expected:"duration" }
+→ "Must be duration, not numeric, for the 'lt' input."
+```
+
+The `fields set` / `update` commands **run this validation synchronously** and refuse an edit that
+would create the mismatch, surfacing a hint that names the conflicting comparison. So via the CLI you
+cannot flip a field to `duration` while a calculated field / metric compares it to a numeric constant —
+update or remove that comparison first, or make the constant a `duration`. This synchronous check is
+exactly what the **data-manager UI does not do** (it defers the kind change to the next re-ingestion —
+the footgun below), so `fields set` is the *safe* way to change a kind. Caveat: the check covers
+comparisons in the typed model `data` (calculated fields, metrics); a kind change that conflicts only
+with an opaque dashboard **view** filter/chart is not caught, so publish and re-open to confirm.
+
+## The data-kind footgun (DNA-46960)
+
+A customer's app failed to open with exactly the error above. Root cause, from their exported app:
+a metric **`% Tijdigheid`** was `PERCENTAGE( DOORLOOPTIJD[duration] lt 864000000[numeric] )` — a
+throughput-time field (kind **duration**) compared to a **numeric** constant (10 days in ms). That
+`lt(duration, numeric)` is evaluated when the query model is built at open, so it blocks every
+dashboard (the data-upload module stays reachable — hence "I can only reach the data upload module").
+
+How an app reaches this state via the **data-manager UI** (not the CLI, which validates synchronously):
+
+1. Field is **numeric**; a metric/calculated field compares it to a numeric constant → valid.
+2. The field's type is changed to **duration** in the *data manager*. This is applied in a
+   **deferred** way — it is not written to the app model synchronously; it is baked in when the app
+   model is regenerated at the **next re-ingestion**.
+3. On re-ingest the field becomes `duration`, so the pre-existing comparison is now
+   `lt(duration, numeric)` — and the ingestion-time regeneration does **not** re-run the edit
+   validation, so the now-invalid model is persisted → the app won't open.
+
+Takeaways when working with an app in this state:
+- To reproduce/inspect: `apps model get` / `fields list` shows the field `kind` and the offending
+  calculated field/metric; the mismatch is a comparison whose constant `dataType` ≠ the field `kind`.
+- **Range filters do not trigger it** — filters on a field go through a coercing path, so a numeric
+  range filter on a now-duration field still opens. Only real expressions (calculated fields, metrics)
+  hit the operator data-kind check.
+- The fix for a broken app is to make the comparison consistent: either revert the field to the kind
+  the constant expects, or re-type the constant to match the field (e.g. a `duration` constant).
+- Import (`.pmapp`) does not re-run this expression validation (exports are trusted), so importing a
+  broken app reproduces the broken state; that is expected and not the bug.

package/skills/uipath-process-mining/references/pre-flight.md

@@ -0,0 +1,119 @@
+# Pre-flight data checks + the minimal data mapping
+
+Cheap local checks before any upload — each one saves a multi-minute ingest
+round-trip.
+
+## Inspect the file first
+
+1. **Encoding / BOM** — non-UTF-8 (Windows-1252 / ISO-8859-1) must be declared via
+   `ingestions create --encoding` (or the mapping's `SourceSettings.Encoding`), or
+   the load mangles/fails.
+2. **Delimiter + field regularity** — stream the file and assert every line splits
+   into the same field count (catches embedded-delimiter / quoting issues). CSVs
+   here are often `;`-delimited.
+3. **Junk rows** — strip fully-empty trailing rows (`;;;;…`). Combined with a
+   NotNull-error on the key column they cause the whole table to
+   `Failed to load datasources`.
+4. **Date format** — inspect token ranges to tell `dd-mm` from `mm-dd` (token1 max
+   > 12 ⇒ day-first). Feeds `DateTimeFormatString`. Formats vary **per file** in
+   the same dataset — check each.
+5. **Cardinality** — distinct case ids and activities, to sanity-check the mapping.
+
+## Minimal `mapping.json`
+
+```json
+{ "Tables": [ {
+  "SourceName": "Event_log", "TargetName": "Event_log", "Source": "blob",
+  "SourceSettings": { "Encoding": "utf-8", "FieldDelimiter": ";", "QuoteCharacter": "\"" },
+  "IsMandatory": true, "ValidationType": "specificationOnly",
+  "Fields": [
+    { "DataType": "text",     "SourceName": "Incident ID",          "TargetName": "Case_ID",   "IsMandatory": true,  "ValidationType": "specificationOnly" },
+    { "DataType": "text",     "SourceName": "IncidentActivity_Type","TargetName": "Activity",  "IsMandatory": true,  "ValidationType": "specificationOnly" },
+    { "DataType": "datetime", "DataTypeSettings": { "DateTimeFormatString": "dd-mm-yyyy hh:mm:ss" },
+      "SourceName": "DateStamp", "TargetName": "Event_end", "IsMandatory": true, "ValidationType": "specificationOnly" }
+  ] } ] }
+```
+
+Rules:
+
+- Core `uipath.custom` event-log targets: **`Case_ID`**, **`Activity`**,
+  **`Event_end`** (datetime). Optional: `Event_start`, `User`.
+- `DateTimeFormatString` is lowercase, non-strftime: `dd-mm-yyyy hh:mm:ss`
+  (`.nnn` for milliseconds).
+- **`IsNotNull` / `IsUnique` now default** per field (the CLI fills
+  `{ Enabled: false, Severity: "warning" }` when omitted) — you no longer need to
+  hand-write them on every field. `data-mapping update` applies the same defaults,
+  so one mapping file works in both commands. Set them explicitly
+  (`{ Enabled: true, Severity: "error" }`) on `Case_ID`/`Activity`/`Event_end` when
+  you want a null there to fail the load rather than warn.
+- **Map risky columns as `text` and parse in SQL** (dates with odd formats,
+  decimal-comma numbers). Only `Event_end` must be a real `datetime`. Unmapped
+  columns still load under their raw source names and are usable as attributes.
+- **Multi-table apps**: add more `Tables[]` entries (Incidents, Interactions,
+  Changes, …). All load; the template models only reference `Event_log`; your
+  custom models `source('sources', '<Table>')` the rest and join on a shared key.
+
+## Fixing the mapping after the app exists
+
+A mapping mistake is **not** a reason to delete the app and start over. `apps
+data-mapping` reads and replaces the mapping of an existing app:
+
+```bash
+uip pm apps data-mapping get <app> --destination ./mapping.json   # download the mapping + note Data.ETag
+#   ...edit: fix the DateTimeFormatString, move a column to the right TargetName, map one more column...
+uip pm apps data-mapping update <app> --file ./mapping.json --etag 'W/"639…"'   # --etag REQUIRED
+uip pm files upload <app> ./data.csv --input-table Event_log      # ONLY if the source columns changed
+uip pm ingestions create <app> --wait                             # the mapping applies to the NEXT ingestion
+```
+
+`get` without `--destination` inlines the mapping in the envelope as `Data.Mapping`
+(useful with `--output-filter`, e.g.
+`--output-filter "Mapping.Tables[0].Fields[].{Src:SourceName,Tgt:TargetName}"`).
+
+Facts worth not re-learning:
+
+- **A mapping change needs a re-ingest, not `transformations apply`.** `apply`
+  re-runs SQL over already-parsed data; the mapping governs *parsing*. Editing the
+  mapping and then running `apply` looks successful and changes nothing.
+- **`dev` only.** The backend allows `PUT` on the dev stage; `published` is
+  read-only, and the CLI restricts `update --stage` to `dev` up front.
+- **`--etag` is REQUIRED on `update`, and it must be the one *your* `get` returned.**
+  You edited the file locally, so only that ETag proves the edit was based on the
+  version you read; the CLI deliberately does not fetch a fresh one before the `PUT`
+  (which would make the `If-Match` pass no matter who wrote in between). A concurrent
+  edit (someone in the UI's mapping editor) is therefore rejected `409
+  UserError_ETagFileConflict` — recover by re-running `get` for the latest version
+  **and its new ETag**, re-applying your change on top of that, then updating with the
+  new `--etag`. Re-running the same `update` unchanged just fails again.
+- `update` reports `Tables` (the mapped table names) and `IngestionNeeded: true`, not
+  an ETag; to confirm a write landed, `get` again and diff — the `get` ETag is a
+  **content checksum**, so re-pushing an identical mapping leaves it unchanged.
+- **A table-less mapping is refused locally.** `{ "Tables": [] }` (or any file with
+  no usable table) fails `No tables found in …` before any API call, so a bad file
+  cannot overwrite and wipe the stored mapping.
+- **Either key casing works** — PascalCase (`{"Tables":[…]}`, what the recipe above
+  and `apps create --data-mapping` use) and the camelCase the API returns. Note
+  `get --destination` writes the API's response **verbatim**, so the downloaded file
+  is camelCase (`{"tables":[…]}`) while the envelope's `Data.Mapping` is PascalCased
+  like every other envelope — same document, two casings. Either can be fed back to
+  `update --file` or to `apps create --data-mapping` on another app.
+- **A structurally invalid mapping fails safe**: `400
+  UserError_DatapipelineBadRequest` / `INVALID_DATASOURCE_ARGUMENT`, and the stored
+  mapping is left untouched.
+- **An app id you can't see answers `403 UserError_NotAuthorized`, not `404`** —
+  don't read that as a permissions problem on the mapping itself; check the id with
+  `apps list`.
+- Reading the `published` stage of an app that was **never published** still
+  succeeds — you get the *template's* mapping with `ETag: W/"0"` and
+  `UseInLoad: false`. Don't mistake it for the app's real mapping.
+
+## Other app types
+
+The mapping above targets the `uipath.custom` `Event_log`. For a **source-system
+template** (`uipath.p2p.sap`, `uipath.im.servicenow`, …) the same `mapping.json`
+shape applies, but you map your extract to the **template's expected input
+tables** instead — create the app, then read `models/schema/sources.yml`
+(`transformations get <app> models/schema/sources.yml`) to see the exact input
+tables and columns the template's transformations consume, and match your
+`Tables[]`/`Fields[]` to them. The pre-flight checks (encoding, delimiter, dates,
+empty rows) are the same. See [`app-types.md`](app-types.md).