@uipath/skills 1.195.0 → 1.196.0

package/skills/uipath-review/SKILL.md

@@ -7,8 +7,6 @@ user-invocable: true
 
 # UiPath Solution & Artifact Reviewer
 
-> **Preview** — skill is under active development; review surface and report format may change.
-
 Review UiPath solutions and individual artifacts for structural validity, quality, best practices, optimization, and correctness. Produces a structured review report with findings and recommendations.
 
 ## When to Use This Skill
@@ -30,6 +28,10 @@ Review UiPath solutions and individual artifacts for structural validity, qualit
 6. **Use `--output json`** on all CLI validation commands for programmatic parsing.
 7. **Do not duplicate what validation commands catch.** Reference the validation output by rule ID and message — do not manually re-describe the same issue. But DO include every validation result (Error, Warning, Info) in the report.
 8. **Cap the review at 30 minutes of analysis.** For very large solutions (10+ projects), provide a summary review with deep dives on the 3 highest-risk projects. Offer to review remaining projects if the user wants.
+9. **Run the review CLI first, then apply the judgment catalog, for every agent encountered.** First run `uip agent review` (low-code) or `uip codedagent review` (coded) with `--output json` — it returns the deterministic findings (Step 2.5a). Then load the judgment catalog: `references/agents/agents-common-rules.md` plus the format-specific file (`agents-lowcode-rules.md` or `agents-coded-rules.md`); for the agent-builder coded layout (both `agent.json` and `main.py`), load all three and run both CLI commands. Future phases add catalogs for RPA, flows, coded apps.
+10. **Rule findings are authoritative as emitted.** Carry review-CLI findings (`Data.Issues[]`) into the report verbatim — `RuleId`, `Severity`, `Description`, `File`, `SuggestedFix` unchanged. For the judgment catalog, use its `rule_id`, `severity`, `trigger`, and `suggested_fix` verbatim. Map severity to the report's bands: `error` → Critical, `warning` → Warning, `info` → Info. `judgment` severity rows default to Warning; the agent may escalate or de-escalate with reasoning logged in the finding's `description`. Do not re-rank otherwise.
+11. **Report rules that could not be applied** (missing tooling, missing file, review CLI unavailable, `status: deferred`) in a dedicated "Rules Skipped" subsection of the report — never silently skip.
+12. **Never invent `rule_id` values.** Every `rule_id` cited in the report MUST appear verbatim in EITHER a loaded judgment-catalog file (`references/agents/agents-*-rules.md`) OR the `uip agent review` / `uip codedagent review` JSON output. `rule_id` is a stable contract identifier — consumers grep for it, dashboards aggregate by it, audits trace it. An invented identifier looks authoritative but cannot be looked up, doesn't aggregate, and produces a different name for the same observation on the next run. If you observe a real issue covered by neither source, the finding is still valid — report it as a normal Critical / Warning / Info finding **without** a `rule_id` (no `` `RULE_ID` `` backtick token in the line). **Before emitting the report, scan every cited `rule_id` and confirm it appears verbatim in a loaded catalog file or the review-CLI output; demote any that don't to `rule_id`-less findings.**
 
 ## Review Workflow
 
@@ -132,8 +134,9 @@ Record the language per project alongside the type (see solution table below).
 | `project.json` with `expressionLanguage: "VisualBasic"` and no/Legacy `targetFramework` | RPA (Windows-Legacy) | [rpa-review-checklist.md](references/rpa/rpa-review-checklist.md) §10. Also recommend the user invoke `uipath-rpa` (Legacy mode) for Legacy-specific deep validation. Legacy is supported indefinitely in Studio LTS — do NOT flag as Critical. |
 | `project.json` + both `.cs` and `.xaml` | RPA (Hybrid) | [rpa-review-checklist.md](references/rpa/rpa-review-checklist.md) |
 | `project.json` + `.xaml` + DU packages in dependencies (`UiPath.IntelligentOCR.Activities`, `UiPath.DocumentUnderstanding.ML.Activities`) | RPA + Document Understanding | [rpa-review-checklist.md](references/rpa/rpa-review-checklist.md) + [du-review-checklist.md](references/document-understanding/du-review-checklist.md) |
-| `agent.json` (no `main.py`) | Agent (Low-Code) | [agent-review-checklist.md](references/agents/agent-review-checklist.md) |
-| `main.py` + `langgraph.json` / `llama_index.json` / `openai_agents.json` / `uipath.json` | Agent (Coded) | [agent-review-checklist.md](references/agents/agent-review-checklist.md) |
+| `agent.json` (no `main.py`) | Agent (Low-Code) | Checklist: [agent-review-checklist.md](references/agents/agent-review-checklist.md). Rule catalog (Step 2.5): [agents-common-rules.md](references/agents/agents-common-rules.md) + [agents-lowcode-rules.md](references/agents/agents-lowcode-rules.md) |
+| `main.py` + `langgraph.json` / `llama_index.json` / `openai_agents.json` / `google_adk.json` / `pydantic_ai.json` / `agent_framework.json` / `uipath.json` | Agent (Coded) | Checklist: [agent-review-checklist.md](references/agents/agent-review-checklist.md). Rule catalog (Step 2.5): [agents-common-rules.md](references/agents/agents-common-rules.md) + [agents-coded-rules.md](references/agents/agents-coded-rules.md) |
+| `agent.json` + `main.py` + `pyproject.toml` (agent-builder coded layout) | Agent (Low-Code + Coded) | Checklist: [agent-review-checklist.md](references/agents/agent-review-checklist.md). Rule catalog (Step 2.5): all three — [agents-common-rules.md](references/agents/agents-common-rules.md) + [agents-lowcode-rules.md](references/agents/agents-lowcode-rules.md) + [agents-coded-rules.md](references/agents/agents-coded-rules.md) |
 | `*.flow` + `project.uiproj` with `"ProjectType": "Flow"` | Flow | [flow-review-checklist.md](references/flows/flow-review-checklist.md) |
 | `.uipath/` directory or `app.config.json` | Coded App | [coded-app-review-checklist.md](references/coded-apps/coded-app-review-checklist.md) |
 
@@ -163,7 +166,7 @@ Report **all** results — Errors, Warnings, and Info — in the final review re
 2. For **each** entry point file, run validation yourself:
 
 ```bash
-uip rpa validate --file-path "<ENTRY_FILE>" --project-dir "<PROJECT_DIR>" --output json --use-studio
+uip rpa validate --file-path "<ENTRY_FILE>" --project-dir "<PROJECT_DIR>" --output json
 ```
 
 3. **Then run a project-level build** to catch what `validate` misses (unknown member names like `NGetText.Value`, invalid enum values like `Operator="StartsWith"`, member resolution / CacheMetadata failures, attribute-form C# expression JIT failures):
@@ -184,7 +187,7 @@ uip rpa build "<PROJECT_DIR>" --log-level Warn --output json
 The Workflow Analyzer checks code quality rules (ST-NMG naming, ST-DBP design, ST-MRD maintainability, ST-USG usage, ST-SEC security, ST-REL reliability). Run it explicitly:
 
 ```bash
-uip rpa analyze --project-dir "<PROJECT_DIR>" --output json --use-studio
+uip rpa analyze --project-dir "<PROJECT_DIR>" --output json
 ```
 
 If `uip rpa analyze` is not available, `uip rpa validate` includes Workflow Analyzer results. Check the output for all rule violations:
@@ -200,7 +203,6 @@ If `uip rpa analyze` is not available, `uip rpa validate` includes Workflow Anal
 | Project Type | Validation Command | Report All Severities |
 |---|---|---|
 | Agent (Low-Code) | `uip agent validate ./path --output json` | Yes — errors, warnings, info |
-| Agent (Coded) | `uip codedagent eval main evaluations/eval-sets/smoke-test.json --no-report` (if eval sets exist) | Yes — pass/fail per test case |
 | Flow | `uip maestro flow validate <ProjectName>.flow --output json` | Yes — schema errors, reference errors, warnings |
 | Coded App | `uip codedapp pack dist --dry-run` | Yes — build errors, pack warnings |
 | Solution | `uip solution pack <SolutionDir> <OutputDir> --output json` | Yes — per-project pack results |
@@ -230,6 +232,50 @@ For the review report, create a validation summary:
 
 > The validation results section is **required** in every review report. A review without automated validation is incomplete.
 
+### Step 2.5 — Run the Review CLI, then Apply the Judgment Catalog
+
+After Step 2 validation and before manual checklist review, produce rule-ID-level findings in two passes: **first** the `uip agent review` / `uip codedagent review` CLI for the deterministic static checks, **then** the skill's judgment-only catalog for what code cannot decide reliably.
+
+#### 2.5a — Run the review CLI first (deterministic findings)
+
+Run the review command for the agent type, once, capturing JSON:
+
+| Agent type | Command |
+|---|---|
+| Low-code (`agent.json`) | `uip agent review "<PROJECT_DIR>" --output json` |
+| Coded (`main.py` + framework config) | `uip codedagent review "<PROJECT_DIR>" --output json` |
+| Agent-builder coded layout (`agent.json` + `main.py`) | run **both** commands |
+
+The CLI runs every deterministic static check — structural/schema, placeholder cross-refs, eval counts/diversity, secret & import regex, framework symbol existence, eval-run analysis, packaging/git hygiene — and returns them in rule format. Parse `Data.Issues[]`; each issue is `{RuleId, Category, Severity, Description, File, SuggestedFix}`. Carry each into the report **verbatim** — do not re-derive, rename, or re-rank. These rule IDs are authoritative as emitted by the CLI; they are **not** listed in the skill catalog.
+
+#### 2.5b — Apply the judgment catalog (reasoning the CLI cannot do)
+
+1. **Identify which catalog files apply** for the current project type:
+
+| Signals present | Project type | Catalog files |
+|---|---|---|
+| `agent.json` AND no `main.py` AND no `pyproject.toml` | Agent (low-code) | `references/agents/agents-common-rules.md` + `references/agents/agents-lowcode-rules.md` |
+| `pyproject.toml` + `main.py` + any of `langgraph.json` / `llama_index.json` / `openai_agents.json` / `google_adk.json` / `pydantic_ai.json` / `agent_framework.json` | Agent (coded) | `references/agents/agents-common-rules.md` + `references/agents/agents-coded-rules.md` |
+| `pyproject.toml` + `main.py` + `uipath.json[functions]` only (no framework config) | Agent (coded — Simple Function) | same as Agent (coded) |
+| `agent.json` + `pyproject.toml` + `main.py` (agent-builder coded layout) | Agent (low-code + coded) | all three: common + lowcode + coded; tag each finding with its source file |
+| `project.json` + `.xaml` / `.cs` | RPA | *(phase 2)* |
+| `*.flow` | Flow | *(phase 2)* |
+| `.uipath/` or `app.config.json` | Coded App | *(phase 2)* |
+
+2. **Read each catalog file in full.** Every rule is judgment-form.
+3. **Apply each rule's `detection_method`:** read the named source material (system prompt, tool descriptions, eval datapoints, schemas) and reason about it. Emit a finding when the criteria hold; log the reasoning in the finding's `description`.
+4. **Track skipped rules** with their reason (`status: deferred`, missing optional file, review CLI unavailable). Never silently skip.
+5. **Verify rule_id provenance.** Before merging, confirm each cited `rule_id` appears **verbatim** in EITHER a loaded catalog file OR the `uip agent review` / `uip codedagent review` JSON output. Any finding whose `rule_id` matches neither is **demoted** to a `rule_id`-less Critical / Warning / Info finding (the observation stays; the false citation goes). This enforces Critical Rule 12.
+6. **Merge findings into the Step 5 report** under the "Rule Findings" subsection. Use the canonical line format:
+
+   ```
+   [<prefix><n>] `<rule_id>` — <file> — <description>. Fix: <suggested_fix>.
+   ```
+
+   where prefix is `C-D-` (Critical), `W-D-` (Warning), or `I-D-` (Info) per the severity mapping in [`references/rule-format.md`](references/rule-format.md).
+
+See [`references/rule-catalog-workflow.md`](references/rule-catalog-workflow.md) for the full procedure including the CLI contract and determinism rules.
+
 ### Step 3 — Manual Quality Review
 
 For **each** project (one for single-project, all for solution/multi-project), load the relevant checklist from `references/` based on the type classified in Step 1. Read project files, check patterns, evaluate design.
@@ -453,6 +499,28 @@ Output a structured report in chat (do NOT create a file):
 
 > For Legacy projects, note: "Validation CLI (`uip rpa validate`, `uip rpa analyze`) targets Modern projects. Legacy validation runs through `uipath-rpa` Legacy mode (using the `uip rpa-legacy` CLI)."
 
+### Rule Findings
+
+| Project | Source | Errors | Warnings | Info | Skipped |
+|---|---|---|---|---|---|
+| ClassifierAgent | `uip agent review` + judgment catalog | 2 | 5 | 3 | 1 |
+| TriageAgent | `uip codedagent review` + judgment catalog | 1 | 4 | 2 | 1 |
+
+**From the review CLI (deterministic):**
+- [C-D-001] `LOWCODE_MESSAGES_NO_USER` — `ClassifierAgent/agent.json` — messages[] has no role="user" entry. Fix: Add a `{"role": "user", "content": "..."}` message.
+- [C-D-003] `FRAMEWORK_DEP_MISSING` — `TriageAgent/pyproject.toml` — langgraph.json present but uipath-langchain missing from [project] dependencies. Fix: Add `"uipath-langchain"` to [project] dependencies in pyproject.toml.
+
+**From the judgment catalog (reasoning):**
+- [W-D-002] `LC_PROMPT_ROLE_DEFINITION` — `ClassifierAgent/agent.json` — System prompt opens with task instructions before establishing the agent's role. Fix: Add an opening sentence: "You are an X that does Y."
+- [W-D-004] `CODED_ERROR_HANDLING` — `TriageAgent/main.py` — `llm.ainvoke(...)` call has no try/except, fallback, or retry. Fix: Wrap the call in try/except with a fallback path or surface the error in the agent's output state.
+- ...
+
+**Rules Skipped (and why):**
+- `uip codedagent review` — CLI not available in environment (deterministic checks not run)
+- `LC_GUARDRAIL_EVALS_CONSISTENCY` — no eval set present to assess against
+
+> The Rule Findings section is required for every agent project (low-code or coded). It is omitted for project types whose catalog has not yet been authored (RPA, flows, coded apps as of phase 1).
+
 ### Critical Findings (block deployment)
 1. [C-001] <concise title> — `<project/file>` — <what to check + recommended fix>
 
@@ -504,6 +572,11 @@ Route each fix to the appropriate skill:
 
 | I need to... | Read this |
 |---|---|
+| Understand the rule row schema | [rule-format.md](references/rule-format.md) |
+| Run the review CLI + judgment catalog (Step 2.5) | [rule-catalog-workflow.md](references/rule-catalog-workflow.md) |
+| Apply common rules for agents (both formats) | [agents-common-rules.md](references/agents/agents-common-rules.md) |
+| Apply the low-code agent judgment catalog | [agents-lowcode-rules.md](references/agents/agents-lowcode-rules.md) |
+| Apply the coded agent judgment catalog | [agents-coded-rules.md](references/agents/agents-coded-rules.md) |
 | Understand the full review workflow in detail | [review-workflow-guide.md](references/review-workflow-guide.md) |
 | Review a solution structure (.uipx) | [solution-review-guide.md](references/solution-review-guide.md) |
 | Review an RPA project (coded or XAML) | [rpa-review-checklist.md](references/rpa/rpa-review-checklist.md) |
@@ -533,3 +606,4 @@ Route each fix to the appropriate skill:
 8. **Do not flag Windows-Legacy compatibility as Critical.** Legacy is supported **indefinitely** in Studio LTS — 2024.10, 2025.10, 2026.10, and all future LTS releases continue to support creating, opening, editing, running, and deploying Legacy projects. It is NOT a deployment blocker and NOT a mid-term support risk. Deprecation means "no new features added to Legacy," not "Legacy will be removed." Flag as **Warning** (if the project would benefit from capabilities Legacy lacks — see [rpa-review-checklist.md §10](references/rpa/rpa-review-checklist.md) for ranked feature list) or **Info** (if Studio LTS is the organizational standard or SOAP web services are required). When recommending migration, lead with the 2-3 features most relevant to the project's actual pain (typically **Healing Agent**, **Unified Target / Modern UIA**, **Object Repository**, **ScreenPlay**, **coded test cases**, **Autopilot**, **Agents/Maestro**). Route Legacy-specific deep validation to `uipath-rpa` (Legacy mode).
 9. **Do not recommend removing a dependency without grepping for usages.** A package may be the sole supplier of an activity used elsewhere — recommend removal only after confirming no consumers exist.
 10. **Do not flag `-preview` package versions.** Many UiPath packages currently ship preview-by-default during the public preview phase, and resolution defaults to bringing them in with explicit user confirmation. Surface stability concerns through activity-owner channels, not user-facing review reports.
+11. **Do not run scripts or install Python packages from this skill.** Deterministic checks run in the `uip agent review` / `uip codedagent review` CLI (Step 2.5a), not via scripts. The skill itself ships no executable code.

package/skills/uipath-review/references/agents/agent-common-issues.md

@@ -92,6 +92,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Overpermissioned Tools
 
+> *Rule catalog: `LC_TOOL_MINIMALITY_SUFFICIENCY`, `LC_FAILURE_AUTHORITY_ESCALATION`, `LC_TOOL_DANGEROUS_COMBINATION` cover this at Step 2.5.*
+
 **Symptom:** Agent has access to tools that are not needed for its stated purpose.
 
 **Example:** A customer support agent with tools for deleting database records, modifying system configuration, or accessing internal admin APIs.
@@ -104,6 +106,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Missing Escalation Paths
 
+> *Rule catalog: `LC_FAILURE_IRREVERSIBLE_ACTION` covers this at Step 2.5.*
+
 **Symptom:** Agent makes irreversible decisions (approvals, deletions, financial transactions) without any human escalation path.
 
 **Impact:** Agent errors have no safety net. Incorrect decisions cannot be caught before execution.
@@ -114,6 +118,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Missing Guardrails
 
+> *Rule catalog: `LC_GUARDRAIL_PII_MISSING`, `LC_GUARDRAIL_INJECTION_MISSING`, `LOWCODE_TOOL_GUARDRAIL_FIELD_MISSING`, `LOWCODE_GUARDRAIL_TOOL_REF_NONEXISTENT` cover this at Step 2.5.*
+
 **Symptom:** User-facing agent with no input validation, PII detection, or prompt injection defense.
 
 **Impact:** Agent vulnerable to manipulation, data leakage, and unintended behavior.
@@ -124,6 +130,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Poor System Prompt Design
 
+> *Rule catalog: `LC_PROMPT_ROLE_DEFINITION`, `LC_PROMPT_SCOPE_BOUNDARIES`, `LC_PROMPT_WHEN_GUIDANCE`, `LC_PROMPT_STOPPING_CRITERIA`, `LC_PROMPT_OUTPUT_FORMAT`, `LC_PROMPT_INSTRUCTION_CONFLICTS` cover the prompt-quality deficiencies at Step 2.5; missing/empty/over-long system prompts are caught by the review CLI.*
+
 **Symptom:** System prompt is vague, overly broad, or missing key elements.
 
 **Common deficiencies:**
@@ -139,6 +147,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Framework Mismatch
 
+> *Rule catalog: `CODED_FRAMEWORK_FIT` covers this at Step 2.5 (fires only on real mismatch, not as a "looks fine" commentary slot).*
+
 **Symptom:** Simple function agent built with LangGraph, or complex multi-step agent built as a simple function.
 
 **Examples of mismatches:**
@@ -154,6 +164,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### No Evaluation Sets
 
+> *Rule catalog: `NO_EVALS`, `MISSING_EVAL_DIR`, `TOO_FEW_EVALS`, `FEW_EVALS` cover this at Step 2.5.*
+
 **Symptom:** Agent has no evaluation sets at all. No `evaluations/` directory or empty eval sets.
 
 **Impact:** No way to verify agent quality. No regression testing. No confidence in deployment.
@@ -164,6 +176,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Single Evaluator Type
 
+> *Rule catalog: `EVAL_LLM_JUDGE_ONLY`, `EVAL_NO_LLM_JUDGE`, `CODED_EVAL_ARCHETYPE_FIT`, `LC_EVAL_JUDGE_FOR_CLOSED_CLASS` cover this at Step 2.5.*
+
 **Symptom:** All evaluations use only `ExactMatchEvaluator` for a natural-language agent, or only `LLMJudgeOutputEvaluator` for a deterministic agent.
 
 **Impact:** ExactMatch is too strict for natural language (high false-negative rate). LLMJudge is overkill for deterministic outputs (unnecessary cost and non-determinism in tests).
@@ -174,6 +188,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Missing Edge Case Tests
 
+> *Rule catalog: `EVAL_DUPLICATE_EXPECTED_OUTPUTS`, `EVAL_LOW_DIVERSITY`, `EVAL_MODERATE_DIVERSITY`, `EVAL_UNIFORM_EXPECTED_BEHAVIOR` partially cover this at Step 2.5 by flagging low-diversity datasets.*
+
 **Symptom:** Eval sets only cover the happy path. No tests for empty input, malformed input, missing data, or error scenarios.
 
 **Impact:** Agent may fail unpredictably in production when encountering real-world data variability.
@@ -196,6 +212,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Context Loss in Handoffs
 
+> *Rule catalog: `CODED_MULTI_AGENT_CROSS_SCHEMA`, `CODED_MULTI_AGENT_HUMANMESSAGE_NAME`, `CODED_MULTI_AGENT_ROUTING_COHERENCE` cover the structural aspects at Step 2.5.*
+
 **Symptom:** Information is lost when tasks are passed between agents. The receiving agent starts reasoning from a partial snapshot.
 
 **Impact:** Downstream agents make incorrect decisions based on incomplete context. Per UC Berkeley's MAST taxonomy, inter-agent misalignment accounts for 36.9% of all multi-agent failures.
@@ -206,6 +224,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Infinite Delegation Loops
 
+> *Rule catalog: `CODED_SUB_AGENT_PROMPT_DUPLICATION`, `LC_TOOL_DELEGATION_DEPTH` partially cover the routing-confusion root cause. The runtime-loop detection itself remains manual.*
+
 **Symptom:** Agents with slightly conflicting instructions bounce tasks back and forth without resolution (Agent A → Agent B → Agent A → ...).
 
 **Impact:** Consumes tokens indefinitely, stalls workflows. No task completion.
@@ -216,6 +236,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Reasoning-Action Mismatch
 
+> *Rule catalog: `CODED_LLM_OUTPUT_UNVALIDATED` covers the output-validation gap that lets the mismatch slip through. The runtime mismatch detection itself remains manual.*
+
 **Symptom:** Agent's stated reasoning doesn't match its actual actions. The agent says it will do X but does Y. Per MAST taxonomy, this is the second most common failure (13.2% of all failures).
 
 **Impact:** Unpredictable behavior. Debug logs show correct reasoning but wrong tool calls.
@@ -238,6 +260,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### No Observability
 
+> *Rule catalog: `NO_TRACING`, `CODED_HELPER_TRACING`, `EVAL_TRAJECTORY_NEEDS_TRACE_SPANS` cover the tracing-decorator absence at Step 2.5. Dashboard / monitoring setup remains manual.*
+
 **Symptom:** Agent deployed without tracing, logging, or monitoring.
 
 **Impact:** No visibility into production behavior. Cannot diagnose failures or performance issues. Cannot identify drift.
@@ -250,6 +274,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Agent Deployed Without Tool Guardrails
 
+> *Rule catalog: `LOWCODE_TOOL_GUARDRAIL_FIELD_MISSING`, `LC_FAILURE_IRREVERSIBLE_ACTION`, `LC_TOOL_DANGEROUS_COMBINATION` cover this at Step 2.5.*
+
 **Symptom:** Agent deployed to production with write/delete/send tools but no guardrails configured.
 
 **Impact:** Agent can invoke any tool with any arguments — no validation. Misinterpreted prompts trigger unintended destructive actions. UiPath docs recommend Block-type guardrails for sensitive data in production.
@@ -262,6 +288,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Vague Tool Descriptions
 
+> *Rule catalog: `VAGUE_TOOL_DESCRIPTION` (≥20 chars required), `LC_TOOL_DESCRIPTION_MISLEADING` (description contradicts schema), `CODED_TOOL_DOCSTRING_QUALITY` (coded) cover this at Step 2.5.*
+
 **Symptom:** Tool descriptions are generic ("Process invoice", "Update record") without specifying side effects, destructiveness, input/output format, or repeat-call behavior.
 
 **Impact:** Agents select tools based on description matching. Vague = wrong-tool selection or wrong argument format.
@@ -274,6 +302,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Destructive Tools Without Explicit Flags
 
+> *Rule catalog: `LC_FAILURE_IRREVERSIBLE_ACTION`, `LC_TOOL_DANGEROUS_COMBINATION` cover the missing-confirmation aspect at Step 2.5.*
+
 **Symptom:** Tools performing irreversible operations (delete, send email, approve, modify config) without explicit `destructive`/`write`/`irreversible` markers in the description.
 
 **Impact:** Agent has no basis to add confirmation or escalate. May execute destructively on ambiguous input.
@@ -286,6 +316,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Context Grounding Missing for Domain-Specific Agents
 
+> *Rule catalog: `LC_TOOL_RESPONSE_NOT_GROUNDED`, `LOWCODE_CONTEXT_NO_DESCRIPTION`, `LOWCODE_CONTEXT_INDEX_NAME_PLACEHOLDER` cover this at Step 2.5.*
+
 **Symptom:** Agent handles domain-specific tasks (company procedures, product catalogs, policy documents) without Context Grounding / RAG configured.
 
 **Impact:** Agent hallucinates — plausible but incorrect answers about company-specific processes. Context Grounding, when properly configured, returns "An answer could not be found" instead of hallucinating.
@@ -298,6 +330,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Missing Output Interpretation Examples
 
+> *Rule catalog: `LC_PROMPT_FEW_SHOT_MISSING`, `LC_PROMPT_OUTPUT_FORMAT` cover this at Step 2.5.*
+
 **Symptom:** Agent has tools configured but system prompt has no examples showing how to interpret tool output (success/failure states, partial results, edge cases).
 
 **Impact:** Agent misinterprets results. Example: API returns `{"status": "pending"}` → agent treats as success. UiPath docs state examples "significantly improve tool accuracy."
@@ -312,6 +346,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Agent Output Missing Reasoning / Explanation Fields
 
+> *Rule catalog: `LC_PROMPT_OUTPUT_FORMAT`, `LC_OUTPUT_CLASSIFIER_NO_ENUM`, `CODED_OUTPUT_ENUM_MISSING_ON_CLASSIFIER`, `CODED_SCHEMA_COMPLETENESS` partially cover the schema-design root cause.*
+
 **Symptom:** Agent JSON output schema includes only the decision/result (e.g., `{"classification": "X"}`) with no `reasoning` / `explanation` / `why` field.
 
 **Impact:** Production debugging becomes impossible. When the agent classifies wrong, there's no trace of what it considered. Audit reviewers cannot assess whether the agent's logic was sound. Hallucinations are invisible until they cause downstream damage.
@@ -324,6 +360,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### Agent with Both Read and Write Side-Effects (Should Be Separated)
 
+> *Rule catalog: `LC_TOOL_DANGEROUS_COMBINATION` covers this at Step 2.5 (assesses pairwise tool combinations).*
+
 **Symptom:** Single agent has tools spanning both read/query operations and write/delete/send/modify operations. Design coupling: a misclassification can cause both wrong reads AND unintended writes.
 
 **Impact:** Blast radius is the full tool set. Harder to audit, harder to sandbox. Read-only evaluations cannot exercise write paths; write-safe evaluations require mock infrastructure.
@@ -336,6 +374,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### No Trust Score Threshold for Autonomous Actions
 
+> *Rule catalog: `CODED_LLM_OUTPUT_UNVALIDATED`, `LC_OUTPUT_CLASSIFIER_NO_ENUM`, `LC_ESCALATION_OVERBROAD` partially cover the schema / escalation aspect. Threshold-based gating logic in the consuming workflow remains manual.*
+
 **Symptom:** Agent output is consumed directly by downstream actions without a confidence / trust score check.
 
 **Impact:** Low-confidence outputs trigger autonomous actions. Examples: "80% confidence this is fraud" → account automatically suspended. "60% confidence this invoice is approved" → auto-paid.
@@ -348,6 +388,8 @@ from uipath.platform.common import CreateTask, WaitTask  # Correct HITL imports
 
 ### No Escalation Path for UNCLEAR / Low-Confidence Results
 
+> *Rule catalog: `LC_ESCALATION_OVERBROAD`, `LC_OUTPUT_CLASSIFIER_NO_ENUM`, `LC_FAILURE_IRREVERSIBLE_ACTION` partially cover the schema / escalation aspect. The consuming-workflow handling itself remains manual (lives outside the agent project).*
+
 **Symptom:** Agent returns `"status": "UNCLEAR"` or low-confidence output, but the consuming workflow has no branch for this — falls through to error handling or default path.
 
 **Impact:** Ambiguous cases get lost or silently mishandled. No human review for the cases that most need it.

package/skills/uipath-review/references/agents/agent-review-checklist.md

@@ -31,24 +31,24 @@ Before reviewing implementation details, verify the right agent type was chosen:
 | `version` field is set (e.g., `"1.1.0"`) | Critical | Read agent.json |
 | `name` field is set and descriptive | Warning | Read agent.json |
 | Exactly 2 messages (system + user) | Critical | Read agent.json → `messages` array |
-| `inputSchema` and `outputSchema` defined | Warning | Read agent.json |
+| `inputSchema` and `outputSchema` defined | Warning | Read agent.json *(rule: `MISSING_INPUT_SCHEMA`, `MISSING_OUTPUT_SCHEMA`)* |
 | `settings.model` is set | Critical | Read agent.json |
-| `settings.maxIterations` is configured (default 25) | Info | Read agent.json |
-| `settings.temperature` is appropriate for task | Info | Read agent.json |
+| `settings.maxIterations` is configured (default 25) | Info | Read agent.json *(rule: `LC_SETTINGS_MAXITERATIONS_FIT`)* |
+| `settings.temperature` is appropriate for task | Info | Read agent.json *(rule: `LC_SETTINGS_TEMPERATURE_FIT`)* |
 
 ### System Prompt Quality
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| System prompt defines explicit role/persona | Warning | Read first message content |
-| System prompt defines scope and boundaries | Warning | Check for "you should NOT" or boundary statements |
-| System prompt includes tool usage guidelines | Warning | Check for tool-specific instructions |
-| System prompt includes escalation criteria | Warning | Check for "escalate when" conditions |
-| System prompt specifies output format | Info | Check for format instructions |
+| System prompt defines explicit role/persona | Warning | Read first message content *(rule: `LC_PROMPT_ROLE_DEFINITION`)* |
+| System prompt defines scope and boundaries | Warning | Check for "you should NOT" or boundary statements *(rule: `LC_PROMPT_SCOPE_BOUNDARIES`)* |
+| System prompt includes tool usage guidelines | Warning | Check for tool-specific instructions *(rule: `LC_PROMPT_WHEN_GUIDANCE`)* |
+| System prompt includes escalation criteria | Warning | Check for "escalate when" conditions *(rule: `LC_PROMPT_STOPPING_CRITERIA`)* |
+| System prompt specifies output format | Info | Check for format instructions *(rule: `LC_PROMPT_OUTPUT_FORMAT`)* |
 | System prompt is in English (best performance) | Info | Check language |
 | No hardcoded examples that should be in eval sets | Info | Check for inline examples |
-| Prompt length appropriate (not excessively long) | Info | Check token count |
-| Prompts over ~2000 tokens have clear structural headers (Role / Task / Constraints / Output Format / Examples) | Warning | Long prompts without structure degrade LLM attention. Flag >2000-token prompts that read as unstructured prose |
+| Prompt length appropriate (not excessively long) | Info | Check token count *(rule: `SYSTEM_PROMPT_TOO_LONG`)* |
+| Prompts over ~2000 tokens have clear structural headers (Role / Task / Constraints / Output Format / Examples) | Warning | Long prompts without structure degrade LLM attention. Flag >2000-token prompts that read as unstructured prose *(rule: `LC_PROMPT_OVERSPECIFICATION`)* |
 | NO hardcoded customer-specific logic in the system prompt (names, rules, identifiers specific to one tenant/customer) | Warning | Customer-specific content should live in Context Grounding / knowledge base / config — not in the prompt. Makes the agent reusable across customers |
 
 ### Temperature Guidance
@@ -65,20 +65,21 @@ Before reviewing implementation details, verify the right agent type was chosen:
 |---|---|---|
 | All tools have descriptive names (lowercase, alphanumeric, no special chars) | Warning | Read resources array |
 | All tools have clear descriptions (purpose, effects, risks) | Warning | Read tool descriptions |
-| No overpermissioned tools (agent has access to tools it doesn't need) | Warning | Review tool list against agent's purpose |
+| No overpermissioned tools (agent has access to tools it doesn't need) | Warning | Review tool list against agent's purpose *(rule: `LC_TOOL_MINIMALITY_SUFFICIENCY`, `LC_FAILURE_AUTHORITY_ESCALATION`)* |
 | Process/API workflow tools reference correct published resource names | Critical | Verify referenced processes exist |
 | Connector tools have valid connection configurations | Critical | Check connection references |
-| Tool descriptions explain when to use AND when NOT to use each tool | Info | Read descriptions |
+| Tool descriptions explain when to use AND when NOT to use each tool | Info | Read descriptions *(rule: `LC_PROMPT_WHEN_GUIDANCE`, `LC_TOOL_DESCRIPTION_MISLEADING`)* |
 | Tool count is bounded — agents with >3 tools risk tool-selection overload; >7 is a strong smell | Warning | Count tools per agent. If >3, verify each is essential and non-overlapping; if >7, consider splitting into multiple specialized agents |
-| Read-only agents do NOT have write-capable tools | Warning | If agent purpose is classification/triage/lookup (read-only), no write/delete/send/modify tools should be present |
-| Output-consuming workflows validate agent output against expected schema before use | Warning | Check calling workflow: agent output should be validated (schema check, type check) before downstream activities consume it |
+| Read-only agents do NOT have write-capable tools | Warning | If agent purpose is classification/triage/lookup (read-only), no write/delete/send/modify tools should be present *(rule: `LC_TOOL_DANGEROUS_COMBINATION`)* |
+| Output-consuming workflows validate agent output against expected schema before use | Warning | Check calling workflow: agent output should be validated (schema check, type check) before downstream activities consume it *(rule: `CODED_LLM_OUTPUT_UNVALIDATED`, `LC_OUTPUT_CLASSIFIER_NO_ENUM`)* |
 
 ### Context Grounding Configuration
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Retrieval mode appropriate (JIT/DeepRAG for runtime docs, index for stable corpora) | Info | Read context grounding config |
-| Index names are descriptive and versioned | Info | Check naming patterns |
+| Retrieval mode appropriate (JIT/DeepRAG for runtime docs, index for stable corpora) | Info | Read context grounding config *(rule: `LC_TOOL_RESPONSE_NOT_GROUNDED`)* |
+| Index names are descriptive and versioned | Info | Check naming patterns *(rule: `LOWCODE_CONTEXT_INDEX_NAME_PLACEHOLDER`)* |
+| Context resources have descriptions explaining when to retrieve | Info | Check `description` on each context resource *(rule: `LOWCODE_CONTEXT_NO_DESCRIPTION`)* |
 | Chunk count balanced (not too few = missed context, not too many = token waste) | Info | Check retrieval config |
 | Threshold configured appropriately (0 is default — higher risks excluding relevant passages) | Info | Check threshold value |
 
@@ -86,17 +87,21 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Escalation resources defined for high-risk decisions | Warning | Check resources for escalation type |
-| Escalation criteria specified in system prompt | Warning | Read system prompt for escalation guidance |
+| Escalation resources defined for high-risk decisions | Warning | Check resources for escalation type *(rule: `LC_FAILURE_IRREVERSIBLE_ACTION`)* |
+| Escalation criteria specified in system prompt | Warning | Read system prompt for escalation guidance *(rule: `LC_PROMPT_STOPPING_CRITERIA`)* |
 | Escalation assignee configured | Critical | Check escalation resource config |
-| Agent does not make irreversible decisions without escalation path | Warning | Review decision scope |
+| Escalation is not overbroad (universal quantifier without narrowing condition) | Warning | Check escalation description *(rule: `LC_ESCALATION_OVERBROAD`)* |
+| Agent does not make irreversible decisions without escalation path | Warning | Review decision scope *(rule: `LC_FAILURE_IRREVERSIBLE_ACTION`)* |
 
 ### Guardrails
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| PII detection configured (if handling personal data) | Warning | Check guardrails array |
-| Prompt injection guardrails enabled (if user-facing) | Warning | Check guardrails config |
+| PII detection configured (if handling personal data) | Warning | Check guardrails array *(rule: `LC_GUARDRAIL_PII_MISSING`)* |
+| Prompt injection guardrails enabled (if user-facing) | Warning | Check guardrails config *(rule: `LC_GUARDRAIL_INJECTION_MISSING`)* |
+| Each guardrail has `name` and `description` | Warning | Walk guardrails *(rule: `GUARDRAIL_NO_NAME`, `GUARDRAIL_NO_DESCRIPTION`)* |
+| Tool resources have `guardrail.policies` array (Studio Web rejects without it) | Critical | Walk tool resources *(rule: `LOWCODE_TOOL_GUARDRAIL_FIELD_MISSING`)* |
+| Tool-scoped guardrails reference existing tools | Critical | Walk Tool-scope guardrails *(rule: `LOWCODE_GUARDRAIL_TOOL_REF_NONEXISTENT`, `GUARDRAIL_TOOL_SCOPE_NO_MATCHNAMES`)* |
 | Custom tool guardrails for destructive operations | Info | Check tool guardrail rules |
 | Guardrail actions appropriate (Log/Block/Escalate) | Info | Review action types |
 
@@ -114,22 +119,23 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| `main.py` exists with entry point function | Critical | `ls main.py` |
-| Framework config file exists (`langgraph.json`, `llama_index.json`, `openai_agents.json`, or `uipath.json`) | Critical | Check for config file |
-| `pyproject.toml` exists with valid config | Critical | Read pyproject.toml |
-| `pyproject.toml` does NOT have `[build-system]` section | Critical | Read pyproject.toml |
-| `.env` file not committed to source control | Critical | Check .gitignore and git status |
-| `entry-points.json` present | Warning | `ls entry-points.json` |
+| `main.py` exists with entry point function | Critical | `ls main.py` *(rule: `MISSING_AGENT_CONFIG`)* |
+| Framework config file exists (`langgraph.json`, `llama_index.json`, `openai_agents.json`, `google_adk.json`, `pydantic_ai.json`, `agent_framework.json`, or `uipath.json`) | Critical | Check for config file *(rule: `MISSING_AGENT_CONFIG`)* |
+| `pyproject.toml` exists with valid config | Critical | Read pyproject.toml *(rule: `MISSING_PYPROJECT`)* |
+| ~~`pyproject.toml` does NOT have `[build-system]` section~~ | ~~Critical~~ | **Retired** — the corresponding `NO_BUILD_SYSTEM` rule was retired (SDK FAQ shows `pyproject.toml` with `[build-system]` per PEP 518). Keep only as manual check if specific `uip codedagent` packaging errors point at it. |
+| `.env` file not committed to source control | Critical | Check .gitignore and git status *(rule: `ENV_FILE_TRACKED`, `GITIGNORE_INCOMPLETE`)* |
+| `entry-points.json` present | Warning | `ls entry-points.json` *(rule: `ENTRY_POINTS_MISSING`, `LOWCODE_ENTRY_POINTS_MISSING`)* |
 
 ### Dependencies (pyproject.toml)
 
 | Check | Severity | How to Verify |
 |---|---|---|
 | `uipath` package in dependencies | Critical | Read pyproject.toml |
-| `uipath-langchain` present if using LangGraph | Critical | Check for LangGraph usage |
+| Framework integration package present (`uipath-langchain` for LangGraph, `uipath-llamaindex` for LlamaIndex, etc.) | Critical | Check pyproject.toml against framework config *(rule: `FRAMEWORK_DEP_MISSING`)* |
 | No unnecessary dependencies | Info | Review dependency list |
 | Dependencies pinned or version-constrained | Warning | Check version specifiers |
-| No `[build-system]` section | Critical | Read pyproject.toml |
+| `requires-python` is `>=3.11` | Critical | Read pyproject.toml *(rule: `REQUIRES_PYTHON_TOO_LOW`)* |
+| `uipath-dev` in `[dependency-groups] dev`, not `[project] dependencies` | Warning | Read pyproject.toml *(rule: `UIPATH_DEV_IN_RUNTIME_DEPS`)* |
 
 ### Code Quality
 
@@ -142,9 +148,10 @@ Before reviewing implementation details, verify the right agent type was chosen:
 | `@mockable()` decorator on functions calling external services | Warning | Grep for `@mockable` |
 | Correct import: `from uipath.platform import UiPath` (NOT `from uipath import UiPath` — that path does not exist and raises `ImportError`) | Critical | Check import statements |
 | HITL imports correct: `CreateTask`/`WaitTask` (not `CreateAction`/`WaitAction`) | Critical | Check HITL-related imports |
-| Structured output via `with_structured_output(MyModel)` | Info | Check LLM output patterns |
-| No hardcoded API keys or secrets | Critical | Grep for `api_key=`, `secret=` in string literals |
-| Async patterns used correctly (if async) | Warning | Check async/await usage |
+| Structured output via `with_structured_output(MyModel)` | Info | Check LLM output patterns *(rule: `CODED_LLM_OUTPUT_UNVALIDATED` for the inverse)* |
+| No hardcoded API keys or secrets | Critical | Grep for `api_key=`, `secret=` in string literals *(rule: `HARDCODED_CREDENTIALS`)* |
+| No bare `except:` blocks (catch specific exceptions) | Warning | Grep for `except:` without exception type *(rule: `BARE_EXCEPT`)* |
+| Async patterns used correctly (if async) | Warning | Check async/await usage *(rule: `LLAMAINDEX_SYNC_STEP` for LlamaIndex)* |
 
 ### Framework-Specific Checks
 
@@ -152,25 +159,32 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| `graph` variable is a compiled `CompiledStateGraph` | Critical | Check main.py for graph compilation |
-| State class properly defined with all required fields | Warning | Check StateGraph definition |
+| `graph` variable is a compiled `CompiledStateGraph` | Critical | Check main.py for graph compilation *(rule: `LANGGRAPH_NO_GRAPH_VAR`, `LANGGRAPH_GRAPH_NOT_COMPILED`)* |
+| State class properly defined with all required fields | Warning | Check StateGraph definition *(rule: `STATEGRAPH_MISSING_INPUT_OUTPUT`, `CODED_SCHEMA_COMPLETENESS`)* |
 | Conditional edges use proper routing logic | Warning | Review edge conditions |
 | No infinite loops in graph (cycles must have exit conditions) | Critical | Review graph topology |
+| Reducer-accumulated state fields are forwarded in node returns | Warning | Check graph nodes returning reducer-typed fields *(rule: `LIST_ACCUMULATOR_NOT_FORWARDED`)* |
+| No `MemorySaver()` in deployable code (runtime injects checkpointer) | Info | Grep for `MemorySaver` *(rule: `MEMORYSAVER_PRODUCTION`)* |
 | Checkpointing configured for long-running graphs | Info | Check for checkpoint config |
 
 #### LlamaIndex
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Agent workflow properly defined | Warning | Check agent setup |
-| Retry mechanisms present for retrieval failures | Warning | Check error handling |
+| Agent workflow properly defined | Warning | Check agent setup *(rule: `LLAMAINDEX_NO_WORKFLOW_VAR`)* |
+| `@step`-decorated methods are `async def` | Critical | Grep for `@step` *(rule: `LLAMAINDEX_SYNC_STEP`)* |
+| Retry mechanisms present for retrieval failures | Warning | Check error handling *(rule: `CODED_ERROR_HANDLING`)* |
 | Index management follows best practices | Info | Check index usage |
 
 #### OpenAI Agents SDK
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Agent role explicitly defined | Warning | Check agent configuration |
+| Agent role explicitly defined | Warning | Check agent configuration *(rule: `OPENAI_AGENTS_NO_AGENT_VAR`)* |
+| `Agent[ContextModel](...)` subscript used for typed input | Warning | Check `Agent(...)` calls *(rule: `OPENAI_AGENTS_NO_GENERIC_TYPE`)* |
+| `output_type=` keyword passed to `Agent(...)` | Warning | Check `Agent(...)` calls *(rule: `OPENAI_AGENTS_NO_OUTPUT_TYPE`)* |
+| `set_default_openai_client(...)` called (route via Gateway) | Info | Grep for `set_default_openai_client` *(rule: `OPENAI_AGENTS_NO_CLIENT_SETUP`)* |
+| No `interrupt` / `MemorySaver` / `InvokeProcess` (LangGraph-only features) | Critical | Grep for unsupported features *(rule: `OPENAI_AGENTS_UNSUPPORTED_FEATURE`)* |
 | Structured tool use with examples | Info | Check tool definitions |
 | Guardrails configured (input/output validation) | Warning | Check guardrail setup |
 
@@ -178,10 +192,11 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Try-except blocks around external service calls | Warning | Check error handling |
-| Graceful degradation when tools fail | Warning | Check tool error paths |
+| Try-except blocks around external service calls | Warning | Check error handling *(rule: `CODED_ERROR_HANDLING`)* |
+| `InvokeProcess` results branched on `status` (success / failed / faulted) | Warning | Grep for `InvokeProcess` *(rule: `CODED_INVOKEPROCESS_NO_FALLBACK`)* |
+| Graceful degradation when tools fail | Warning | Check tool error paths *(rule: `CODED_ERROR_HANDLING`)* |
 | Timeout configuration for LLM calls | Info | Check timeout settings |
-| No bare `except:` blocks (catch specific exceptions) | Warning | Grep for `except:` without exception type |
+| No bare `except:` blocks (catch specific exceptions) | Warning | Grep for `except:` without exception type *(rule: `BARE_EXCEPT`)* |
 
 ## 4. Evaluation Quality (Both Types)
 
@@ -189,11 +204,14 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| At least one smoke evaluation set exists | Warning | `ls evaluations/eval-sets/smoke-test.json` or similar |
+| At least one smoke evaluation set exists | Warning | `ls evaluations/eval-sets/smoke-test.json` or similar *(rule: `NO_EVALS`, `MISSING_EVAL_DIR`)* |
 | Eval set covers happy path (basic successful scenarios) | Warning | Read eval set |
-| Eval set covers edge cases (missing data, unusual inputs) | Info | Read eval set |
+| Eval set covers edge cases (missing data, unusual inputs) | Info | Read eval set *(rule: `EVAL_DUPLICATE_EXPECTED_OUTPUTS`, `EVAL_LOW_DIVERSITY`)* |
 | Eval set covers error scenarios (invalid inputs, service failures) | Info | Read eval set |
-| Minimum 30 test cases for production readiness | Info | Count evaluations in eval sets |
+| Minimum 30 test cases for production readiness | Info | Count evaluations in eval sets *(rule: `TOO_FEW_EVALS`, `FEW_EVALS`)* |
+| No duplicate / copy-paste eval inputs | Warning | Compare datapoint `inputs` *(rule: `EVAL_COPY_PASTE_INPUTS`)* |
+| Eval `expectedOutput` is not a placeholder literal (`TODO`, `FIXME`, etc.) | Warning | Read eval datapoints *(rule: `EVAL_PLACEHOLDER_OUTPUTS`)* |
+| No eval label leakage (expected output appearing verbatim in inputs) | Warning | Compare expectedOutput against inputs *(rule: `EVAL_LABEL_LEAKAGE`)* |
 | Eval set tiering aligned with FDE Architecture Review guidance: ~10-20 dev evals, ~50 benchmark cases, 200-400 UAT cases for production-critical agents | Warning | Check for tiered eval sets (smoke/dev/benchmark/UAT). Customer-critical agents with <50 benchmark cases are under-tested |
 | Eval sets include **corrected failures** (cases that initially failed, were fixed, and are retained as regression tests) — NOT only happy-path / successful trajectories | Warning | Review eval dataset sourcing. Datasets built only from successful runs suffer selection bias — the agent passes evaluations that don't reflect real-world failure modes |
 | Eval sets include adversarial / malicious inputs (prompt injection attempts, malformed data, boundary conditions) | Warning | Check for negative test cases |
@@ -204,10 +222,13 @@ Before reviewing implementation details, verify the right agent type was chosen:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Multiple evaluator types configured (not just one) | Warning | Check evaluatorRefs |
-| Trajectory evaluator present for multi-step agents | Warning | Check for LLMJudgeTrajectoryEvaluator |
+| Multiple evaluator types configured (not just one) | Warning | Check evaluatorRefs *(rule: `EVAL_LLM_JUDGE_ONLY`, `EVAL_NO_LLM_JUDGE`)* |
+| Trajectory evaluator present for multi-step agents (with specific `expectedAgentBehavior`) | Warning | Check for trajectory evaluators *(rule: `CODED_EVAL_TRAJECTORY_SPECIFICITY`, `EVAL_TRAJECTORY_NEEDS_TRACE_SPANS`)* |
 | Output-based evaluator present | Warning | Check for LLMJudgeOutputEvaluator or ExactMatchEvaluator |
-| Evaluator selection matches agent type (see table below) | Warning | Compare agent type to evaluators |
+| Evaluator selection matches agent archetype (see table below) | Warning | Compare agent type to evaluators *(rule: `CODED_EVAL_ARCHETYPE_FIT`)* |
+| Eval set has no broken evaluator references | Critical | Cross-check `evaluatorRefs` against `evaluations/evaluators/*.json` *(rule: `EVAL_BROKEN_EVALUATOR_REF`)* |
+| Tool-call evaluators present when agent has tools | Warning | Check for `uipath-tool-call-*` evaluators *(rule: `EVAL_NO_TOOL_CALL_EVALUATORS`)* |
+| LLM judges use strong models with reasoning prompts and low temperature | Warning | Walk LLM judge configs *(rule: `EVAL_JUDGE_WEAK_MODEL`, `EVAL_JUDGE_NO_REASONING`, `EVAL_JUDGE_HIGH_TEMPERATURE`)* |
 
 ### Evaluator Selection Guide
 
@@ -275,12 +296,13 @@ Review whether the agent design accounts for known UiPath limitations:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| `entry-points.json` present and correct | Warning | Read entry-points.json |
+| `entry-points.json` present and correct | Warning | Read entry-points.json *(rule: `ENTRY_POINTS_MISSING`, `LOWCODE_ENTRY_POINTS_MISSING`)* |
+| `agent.json` ↔ `entry-points.json` schemas match (agent-builder) | Critical | Compare inputs/outputs *(rule: `LOWCODE_SCHEMA_DRIFT`)* |
 | `bindings.json` present (if using external resources) | Warning | Read bindings.json |
 | Agent validates without errors | Critical | Run validation command |
-| Smoke evaluation passes (70%+ score) | Warning | Run evaluations |
-| No `.env` file or secrets in committed code | Critical | Check git status |
-| No module-level side effects that fail during `uip codedagent init` | Critical | Run init |
+| Smoke evaluation passes (70%+ score) | Warning | Run evaluations *(rule: `EVAL_RUN_LOW_PASS_RATE`, `EVAL_RUN_NEVER_EXECUTED`)* |
+| No `.env` file or secrets in committed code | Critical | Check git status *(rule: `ENV_FILE_TRACKED`, `GITIGNORE_INCOMPLETE`, `HARDCODED_CREDENTIALS`)* |
+| `.venv/` excluded from `uipath.json` `packOptions.directoriesExcluded` | Warning | Read uipath.json *(rule: `DEPLOY_VENV_NOT_EXCLUDED`)* |
 | Version management in place | Info | Check versioning |
 
 ## 7. Agent Security Review
@@ -289,8 +311,8 @@ Review whether the agent design accounts for known UiPath limitations:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| Agent has only necessary tools (principle of least privilege) | Warning | Review tool list |
-| Destructive tools (delete, modify) have guardrails or escalation | Warning | Check tool guardrails |
+| Agent has only necessary tools (principle of least privilege) | Warning | Review tool list *(rule: `LC_TOOL_MINIMALITY_SUFFICIENCY`, `LC_FAILURE_AUTHORITY_ESCALATION`)* |
+| Destructive tools (delete, modify) have guardrails or escalation | Warning | Check tool guardrails *(rule: `LC_TOOL_DANGEROUS_COMBINATION`, `LC_FAILURE_IRREVERSIBLE_ACTION`)* |
 | No tools that expose internal systems unnecessarily | Warning | Review tool descriptions |
 | File-access tools restricted to appropriate paths | Info | Check tool configurations |
 
@@ -298,8 +320,8 @@ Review whether the agent design accounts for known UiPath limitations:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| No PII in agent traces or logs (unless required) | Warning | Review trace content |
-| Sensitive inputs/outputs masked appropriately | Warning | Check data handling |
+| No PII in agent traces or logs (unless required) | Warning | Review trace content *(rule: `CODED_PII_IN_TRACES`)* |
+| Sensitive inputs/outputs masked appropriately | Warning | Check data handling *(rule: `CODED_PII_IN_TRACES`)* |
 | Context grounding indexes don't contain sensitive data in public-facing agents | Critical | Review index content |
 | Memory TTL configured (default 3-month, 80-char key limit) | Info | Check memory config |
 
@@ -307,9 +329,9 @@ Review whether the agent design accounts for known UiPath limitations:
 
 | Check | Severity | How to Verify |
 |---|---|---|
-| System prompt includes boundary instructions | Warning | Review system prompt |
-| External data treated as untrusted (clear delimiters) | Warning | Check data handling patterns |
-| Output validation present (schema-based) | Info | Check output handling |
+| System prompt includes boundary instructions | Warning | Review system prompt *(rule: `LC_PROMPT_SCOPE_BOUNDARIES`)* |
+| External data treated as untrusted (clear delimiters) | Warning | Check data handling patterns *(rule: `CODED_PROMPT_USER_INPUT_UNSANITIZED`, `LC_FAILURE_PROMPT_INJECTION`)* |
+| Output validation present (schema-based) | Info | Check output handling *(rule: `CODED_LLM_OUTPUT_UNVALIDATED`, `LC_OUTPUT_CLASSIFIER_NO_ENUM`)* |
 | Rate limiting configured for user-facing agents | Info | Check rate limit config |
 
 ## 8. AI Trust Layer Audit