@uipath/skills 1.195.0

package/skills/uipath-agents/references/coded/lifecycle/build.md

@@ -0,0 +1,54 @@
+# Build UiPath Agents
+
+Implement agent logic using UiPath SDK and framework-specific patterns.
+
+## Reference Lookup
+
+Read **only** the reference matching the selected framework. Do NOT load other framework references.
+
+| Framework | Reference |
+|-----------|-----------|
+| Coded Function | `../frameworks/coded-function-agents.md` + `../frameworks/agent-patterns.md` |
+| LangGraph | `../frameworks/langgraph-integration.md` |
+| LlamaIndex | `../frameworks/llamaindex-integration.md` |
+| OpenAI Agents | `../frameworks/openai-agents-integration.md` |
+
+Load capability references **only if the task requires them** — do not preload:
+
+| Capability | Reference | Load when... |
+|------------|-----------|-------------|
+| RPA process invocation | `../capabilities/process-invocation.md` | agent invokes UiPath processes/jobs |
+| Human approval / interrupt | `../capabilities/human-in-the-loop.md` | agent needs human-in-the-loop or pause/resume |
+| RAG / context grounding | `../capabilities/context-grounding.md` | agent searches organization documents |
+| Platform API calls | `../capabilities/sdk-services.md` | agent uses UiPath platform services directly |
+| Tracing / monitoring | `../capabilities/tracing.md` | agent needs custom tracing (Coded Function only — LangGraph traces automatically) |
+| File attachments (input or created) | `../capabilities/file-attachments.md` | agent takes a file as input, or creates an attachment |
+| Conversational (chat-style) agents | `../capabilities/conversational-agents.md` | agent receives one message per turn; runtime threads history (LangGraph / LlamaIndex only) |
+
+## Framework Reference
+
+| Framework | Config File | Key Dependency | Entry Point |
+|-----------|------------|----------------|-------------|
+| Coded Function | `uipath.json` | `uipath` | `main.py` function |
+| LangGraph | `langgraph.json` | `uipath-langchain` | `main.py` compiled StateGraph |
+| LlamaIndex | `llama_index.json` | `uipath-llamaindex` | `main.py` Workflow instance |
+| OpenAI Agents | `openai_agents.json` | `uipath-openai-agents` | `main.py` Agent instance |
+
+## Troubleshooting
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `'dict' has no attribute '...'` | `with_structured_output()` returns a dict, not a Pydantic model | Access results with `result['key']` dict syntax, not `result.key` attribute access |
+| `ImportError: Could not import <package>` | External tool package not in `pyproject.toml` | Add all third-party tool packages to dependencies: `uv add <package>` |
+| Agent returns empty output | Entry point not wired correctly | Verify `main.py` exports the correct object (compiled graph, Workflow, Agent) |
+| `TypeError` on Input/Output | Schema mismatch after code change | Re-run `uip codedagent init` to regenerate `entry-points.json` |
+
+## Additional Instructions
+
+- **Select a framework before writing any code.** Infer from the prompt if possible (tools/orchestration → LangGraph, RAG → LlamaIndex, simple LLM → OpenAI Agents, no LLM → Coded Function). If ambiguous, ask the user to choose.
+- **Read ONLY the single framework reference** for the selected framework before writing code. Do NOT read other framework references or capability references unless the task explicitly requires that capability.
+- **Clean generated scaffold code before schema init.** After `uip codedagent new` and before running `uip codedagent init`, inspect `main.py` and remove scaffold hazards: no module-level `UiPathChat`, `UiPathAzureChatOpenAI`, `UiPath`, or other auth-dependent clients; instantiate LLM/SDK clients inside graph nodes/functions only; ensure importing `main.py` works without UiPath auth.
+- **NEVER instantiate LLM clients or SDK clients at module level.** `uip codedagent init` imports your Python file to introspect schemas — module-level `UiPathAzureChatOpenAI()`, `UiPathChat()`, `UiPathChatOpenAI()`, or `UiPath()` will fail because auth may not have happened yet. Always create these instances inside functions or graph nodes, never at the top level of the module.
+- **Correct SDK import: `from uipath.platform import UiPath`** — not `from uipath import UiPath` (that does not exist). Instantiate inside functions only: `sdk = UiPath()`.
+- LangGraph agents get tracing automatically — no `@traced()` needed on graph nodes.
+- Simple function agents require `@traced()` on the `main` function.

package/skills/uipath-agents/references/coded/lifecycle/deployment.md

@@ -0,0 +1,143 @@
+# Deploy UiPath Agents
+
+Build and publish a coded agent to UiPath Cloud with a single command.
+
+## Prerequisites
+
+- Authenticated session (`uip login status` reports `Logged in`). See [authentication](../../authentication.md).
+- `entry-points.json` exists (run `uip codedagent init`).
+- `pyproject.toml` has `name`, `version`, `description`, `authors`.
+- Agent runs cleanly with `uip codedagent run <ENTRYPOINT> '<input>'`.
+
+## Deploy
+
+```bash
+uip codedagent deploy --my-workspace
+```
+
+`deploy` validates the project, locks dependencies (`uv lock`), builds a `.nupkg`, and uploads it in one step. Use this as the standard path. The underlying `pack` and `publish` subcommands are disabled in the wrapper — do not call them directly.
+
+### Options
+
+| Option | Short | Description |
+|--------|-------|-------------|
+| `--my-workspace` | `-w` | Personal workspace |
+| `--tenant` | `-t` | Tenant package feed |
+| `--folder <name>` | `-f` | Specific folder feed (e.g., `"Finance"`) |
+| `root` | (positional) | Project root when deploying from a parent directory |
+
+If no target flag is provided the CLI prompts interactively, which fails in non-interactive shells — always pass one of `--my-workspace`, `--tenant`, or `--folder`.
+
+### Examples
+
+```bash
+# Personal workspace (default for first-time deploys)
+uip codedagent deploy --my-workspace
+
+# Tenant feed
+uip codedagent deploy --tenant
+
+# Specific folder feed
+uip codedagent deploy --folder "Finance"
+
+# Deploy a sibling project without cd'ing
+uip codedagent deploy ./my-agent --my-workspace
+```
+
+## Invoke a Deployed Agent
+
+```bash
+uip codedagent invoke <ENTRYPOINT> '{"query": "test"}'
+```
+
+`<ENTRYPOINT>` is the key from `entry-points.json` (for example `main`), not the project name. Invoke is asynchronous — it returns a monitoring URL immediately; there is no `--wait` flag. Use `uip codedagent run` for local testing.
+
+## What Goes Into the Package
+
+The `.nupkg` produced by `deploy` contains:
+
+```
+content/
+├── operate.json
+├── entry-points.json
+├── bindings_v2.json
+├── package-descriptor.json
+├── main.py                # your source files
+├── pyproject.toml
+└── uv.lock
+```
+
+Control file inclusion via `packOptions` in `uipath.json`:
+
+```json
+{
+  "packOptions": {
+    "fileExtensionsIncluded": [".py", ".json"],
+    "filesIncluded": ["config.yaml"],
+    "filesExcluded": ["test_*.py"],
+    "directoriesExcluded": ["tests", "__pycache__"],
+    "includeUvLock": true
+  }
+}
+```
+
+## Version Bumping
+
+Publishing the same version twice returns `409 Package already exists`. Bump the patch in `pyproject.toml` before each re-deploy:
+
+```toml
+[project]
+version = "0.0.2"  # was 0.0.1
+```
+
+Increment patch for bugfixes; bump minor/major only for feature or breaking changes.
+
+## Idempotent re-deploy: pre-existing tenant state
+
+`deploy` is not always a fresh install. When the goal is "ensure a published version of this agent exists in the tenant" (so a downstream flow / agent can consume it), and a prior publish already satisfies that goal, treat the deploy step as **done** and move on. Do not loop on the failure trying to land a brand-new artifact when the existing one already serves the purpose.
+
+Three conflicts fall in this "already satisfied" bucket:
+
+| Conflict | Server response | Meaning |
+|---|---|---|
+| Version already exists | `409 Package already exists` | A package with this exact `name@version` is already published. The downstream consumer can resolve the agent by name/version. |
+| Package type mismatch | e.g. `now is Function` vs prior `Agent` | A package with the same name was previously published under a different project type. Tenant feeds key on name; the existing entry is what consumers will see. |
+| Missing `--my-workspace` scope | Token lacks `OrchestratorApiUserAccess` | The caller's token cannot publish to the personal workspace. If a prior call already established a published version (in the personal workspace or the tenant feed), the consumer can use that. |
+
+In all three cases, when the downstream goal is consumption rather than upgrade:
+
+- **The deploy command's own JSON output is the authoritative source** for the package key — it prints the Orchestrator-assigned GUID directly. Use it as the consumer's `resourceKey` and move on. Don't run a separate discovery call to re-fetch what `deploy` already returned.
+- Always run `uip maestro flow registry pull --force` after a deploy (success OR conflict) to refresh the local flow registry cache. This is the only registry call worth making post-deploy. Do NOT use `uip maestro flow registry search` for verification — that search only enumerates built-in node types (`uipath.agent.autonomous`, `uipath.agent.resource.escalation`, etc.); user-deployed coded agents do NOT appear there, so an empty result is the *expected* state and is NOT a signal that anything went wrong.
+- If the deploy command's output is unavailable, try `uip or packages list --search "<agent-name>" --output json` (returns 404 on tenants where the caller lacks `Orchestrator.Packages.View` scope) — at most one fallback attempt, not a loop.
+- Do NOT bump the version, switch deploy target, edit `project.uiproj` to flip the project type, or re-`uip login` to acquire a broader scope.
+- Do NOT delete the existing tenant entry — these are shared resources others may depend on.
+
+**Hard rule — at most ONE re-deploy attempt on a conflict.** If the first `deploy` fails with any of the three conflicts above, do NOT immediately bump and retry. Stop, capture the package key from the conflict response or fall back to `uip or packages list --search "<agent-name>"` once, and move on. A re-deploy is only justified when the explicit intent is to publish a *new* version (a real upgrade), in which case bump the patch in `pyproject.toml` once and re-`deploy` exactly once. Looping `deploy → conflict → bump → deploy` more than once burns turns without changing the outcome.
+
+## Configuration Files
+
+| File | Created By | Purpose |
+|------|-----------|---------|
+| `uipath.json` | `uip codedagent init` | Runtime options, pack options |
+| `pyproject.toml` | You | Project name, version, dependencies |
+| `entry-points.json` | `uip codedagent init` | Entry points and input/output schemas |
+| `bindings.json` | `uip codedagent init` | Runtime bindings |
+
+`uip codedagent deploy` and `invoke` read credentials (`UIPATH_URL`, `UIPATH_ACCESS_TOKEN`, org/tenant identifiers) from your active `uip login` session — no manual `.env` wiring is required.
+
+## Typical Flow
+
+1. `uip codedagent run <ENTRYPOINT> '<input>'` — verify locally.
+2. Bump patch version if re-deploying.
+3. `uip codedagent deploy --my-workspace` (or `--tenant` / `--folder`).
+4. `uip codedagent invoke <ENTRYPOINT> '<input>'` — trigger in cloud.
+
+## Troubleshooting
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `Project authors cannot be empty` | Missing `authors` in `pyproject.toml` | Add `authors = [{ name = "Your Name" }]` to `[project]` |
+| `Pack failed: missing fields` | `pyproject.toml` incomplete | Ensure `name`, `version`, `description`, `authors` are all set |
+| `Version already exists` / `409` | Same version already published | Bump the patch version in `pyproject.toml` |
+| `401 Unauthorized` | Session expired | Re-authenticate; see [authentication](../../authentication.md) |
+| `The 'pack'/'publish' command is disabled` | Called the disabled subcommand directly | Use `uip codedagent deploy` instead |

package/skills/uipath-agents/references/coded/lifecycle/evaluate.md

@@ -0,0 +1,101 @@
+# Evaluate UiPath Agents
+
+Design and run tests against coded agents using the UiPath evaluation framework.
+
+## Quick Reference
+
+```bash
+# Local only (no cloud connection, no auth needed)
+uip codedagent eval <ENTRYPOINT> evaluations/eval-sets/smoke-test.json --no-report
+
+# With output file
+uip codedagent eval <ENTRYPOINT> evaluations/eval-sets/smoke-test.json --no-report --output-file results.json
+
+# Run only selected test cases
+uip codedagent eval <ENTRYPOINT> evaluations/eval-sets/smoke-test.json --no-report --eval-ids '["test-1-happy-path"]'
+
+# Report results to Studio Web (requires auth + UIPATH_PROJECT_ID)
+uip codedagent eval <ENTRYPOINT> evaluations/eval-sets/smoke-test.json --report --workers 4
+```
+
+## Prerequisites
+
+- `entry-points.json` exists (run `uip codedagent init`).
+- For `--report`: authenticated session and `UIPATH_PROJECT_ID` in `.env` (obtained by pushing the agent to Studio Web — see [file-sync.md](file-sync.md)). Use `--no-report` to skip both.
+
+## Reference Navigation
+
+- [Evaluators Reference](evaluations/evaluators.md) — every evaluator type, required config, scoring, and `evaluatorTypeId` values
+- [Evaluation Sets](evaluations/evaluation-sets.md) — test-case file format, mocking strategies, examples
+- [Creating Evaluations](evaluations/creating-evaluations.md) — test-case design and organization
+- [Running Evaluations](evaluations/running-evaluations.md) — command options, score interpretation
+- [Best Practices](evaluations/best-practices.md) — patterns by agent type, CI/CD integration
+
+Read Evaluators Reference before choosing an evaluator type, and Evaluation Sets before writing test cases.
+
+## File Structure
+
+```
+evaluations/
+├── eval-sets/
+│   └── smoke-test.json              # Test cases
+└── evaluators/
+    └── llm-judge-trajectory.json    # Evaluator config
+```
+
+Every evaluator referenced in an eval set's `evaluatorRefs` must have a matching config file in `evaluations/evaluators/` — the `id` in the config must match the `evaluatorRefs` value exactly. Evaluators are auto-discovered from this directory.
+
+Example `evaluations/evaluators/llm-judge-trajectory.json`:
+
+```json
+{
+  "version": "1.0",
+  "id": "LLMJudgeTrajectoryEvaluator",
+  "evaluatorTypeId": "uipath-llm-judge-trajectory-similarity",
+  "evaluatorConfig": {
+    "name": "LLMJudgeTrajectoryEvaluator",
+    "model": "gpt-4o-mini-2024-07-18",
+    "defaultEvaluationCriteria": {
+      "expectedAgentBehavior": "Agent should process the input and return a response."
+    }
+  }
+}
+```
+
+## Mocking External Calls
+
+Two mocking paths are available:
+
+**In-code** — Apply `@mockable()` to functions that call external services:
+
+```python
+from uipath.eval.mocks import mockable, ExampleCall
+
+@mockable(example_calls=[
+    ExampleCall(id="sunny-nyc", input="weather in NYC", output='{"temp": 72, "condition": "sunny"}'),
+])
+def fetch_weather(query: str) -> dict:
+    return call_weather_api(query)
+```
+
+During evaluations, calls matching an `ExampleCall.input` return the paired `output`. During normal execution, the real function runs.
+
+`@mockable` only *registers* the function as interceptable — mock values come from the test case's `mockingStrategy`. `example_calls` matter only for LLM-driven mocking:
+
+| Mock values supplied by | `example_calls` needed? |
+|---|---|
+| Declarative `mockingStrategy: mockito` behaviors | No — bare `@mockable()` is correct; mockito ignores `example_calls` |
+| LLM mocking (`mockingStrategy: llm`, or user wants LLM-decided substitution values) | Yes — they ground the LLM mocker; without them outputs are nondeterministic and structured-output evaluators score erratically |
+
+When `example_calls` are needed, provide ≥1 `ExampleCall` per decorated function with `output` matching the real return shape. Do NOT add `example_calls` to mockito-mocked functions. If no mock matches at runtime, the real function runs.
+
+**Declarative** — Set `mockingStrategy` on each test case in the eval set (`type: "mockito"` for function mocks, `type: "llm"` for LLM mocks). See [Evaluation Sets](evaluations/evaluation-sets.md) § Mocking Strategies.
+
+## Troubleshooting
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `typing.Any must be a subclass of BaseEvaluatorConfig` | Invalid `evaluatorTypeId` in evaluator JSON | Check [evaluators.md](evaluations/evaluators.md) for valid type IDs |
+| `target_output_key: Input should be a valid string` | ContainsEvaluator missing required config | Set `target_output_key` to the output field name in the evaluator JSON |
+| `UIPATH_PROJECT_ID not found` | Agent not pushed to Studio Web (only needed for `--report`) | Push with `uip codedagent push` and set `UIPATH_PROJECT_ID` in `.env`, or use `--no-report` |
+| LLM evaluator fails at runtime | Missing or empty `model` in evaluator JSON | Set `"model"` in `evaluatorConfig` to a model available in your tenant |

package/skills/uipath-agents/references/coded/lifecycle/evaluations/best-practices.md

@@ -0,0 +1,222 @@
+# Best Practices & Common Patterns
+
+This guide covers best practices for effective evaluation design and common patterns for different agent types.
+
+## Evaluation Best Practices
+
+### Do
+
+- **Use multiple evaluators** for comprehensive validation
+  - Don't rely on a single evaluator
+  - Combine output-based and trajectory-based evaluators for complex agents
+  - Example: Use both ExactMatchEvaluator and JsonSimilarityEvaluator
+
+- **Create separate eval sets for different scenarios**
+  - Happy path scenarios
+  - Edge cases
+  - Error scenarios
+  - Performance tests
+  - This makes it easier to maintain and debug
+
+- **Mix evaluator types appropriately**
+  - Output-based evaluators for result validation
+  - Trajectory evaluators for multi-step agents
+  - LLM evaluators for natural language outputs
+
+- **Use trajectory evaluators for multi-step agents**
+  - Validates execution flow and tool usage
+  - Ensures agent takes expected decision paths
+  - Useful for orchestration agents
+
+- **Use LLM evaluators for natural language or fuzzy matching**
+  - Better for semantic equivalence
+  - More flexible than exact matching
+  - Handles variations in wording
+
+- **Start with ExactMatch, then add flexibility**
+  - Begin with strict ExactMatchEvaluator during development
+  - Add LLM evaluators for production as needed
+  - Allows refinement as agent matures
+
+- **Mock external dependencies consistently**
+  - Mock all external API calls
+  - Use mocking for deterministic testing
+  - Cache LLM responses in CI/CD
+
+- **Version your evaluation sets**
+  - Use semantic versioning in IDs
+  - Track changes over time
+  - Example: `calculator-v1`, `calculator-v2`
+
+- **Document test purposes clearly**
+  - Use descriptive test names
+  - Explain what each test validates
+  - Make it easy for others to understand
+
+- **Review failed tests carefully**
+  - Examine execution traces
+  - Understand why tests failed
+  - Fix either the agent or test expectations
+
+### Don't
+
+- **Use only ExactMatch for natural language outputs**
+  - Too strict, fails on minor variations
+  - Use LLMJudgeOutputEvaluator instead
+
+- **Forget to test edge cases and error scenarios**
+  - Test boundary values (0, min, max)
+  - Test empty/null values
+  - Test invalid inputs
+
+- **Use trajectory evaluators when output-based is sufficient**
+  - Trajectory evaluation is more expensive
+  - Only use when execution path matters
+  - For simple agents, output validation is enough
+
+- **Set too strict criteria early in development**
+  - Allow flexibility while agent is evolving
+  - Tighten criteria as agent stabilizes
+  - Start with 80%, improve to 95%+
+
+- **Skip schema validation during test creation**
+  - Always validate inputs against schema
+  - Prevents invalid test data
+  - Catches type mismatches early
+
+- **Mix unrelated tests in one eval set**
+  - Keep eval sets focused and organized
+  - Separate happy path from error cases
+  - Makes debugging easier
+
+## Common Evaluation Patterns
+
+### Pattern 1: Calculator/Deterministic Agents
+
+For agents that always produce the same output for the same input:
+
+**Evaluator Selection:**
+- **Primary:** ExactMatchEvaluator
+- **Secondary:** (optional) JsonSimilarityEvaluator for complex outputs
+
+**Test Cases:**
+```
+Happy Path: Basic addition, subtraction, multiplication, division
+Edge Cases: Zero values, negative numbers, very large numbers, decimal results
+Error Scenarios: Non-numeric input, missing parameters, division by zero
+```
+
+**Scoring:** 1.0 (pass) or 0.0 (fail). No partial credit for exact match.
+
+**Example Eval Set:**
+
+```json
+{
+  "version": "1.0",
+  "id": "calculator-comprehensive",
+  "name": "Calculator Comprehensive Tests",
+  "evaluatorRefs": ["ExactMatchEvaluator"],
+  "evaluations": [
+    {
+      "id": "test-1-add",
+      "name": "Basic addition",
+      "inputs": {"a": 5, "b": 3},
+      "evaluationCriterias": {
+        "ExactMatchEvaluator": {
+          "expectedOutput": {"result": "8"}
+        }
+      }
+    },
+    {
+      "id": "test-2-divide-by-zero",
+      "name": "Error handling",
+      "inputs": {"a": 10, "b": 0},
+      "evaluationCriterias": {
+        "ExactMatchEvaluator": {
+          "expectedOutput": {"error": "Division by zero"}
+        }
+      }
+    }
+  ]
+}
+```
+
+### Pattern 2: Natural Language Agents
+
+For agents that generate text, summaries, or natural language output:
+
+**Evaluator Selection:**
+- **Primary:** LLMJudgeOutputEvaluator (semantic matching)
+- **Secondary:** ContainsEvaluator (keyword checks)
+
+**Test Cases:**
+```
+Semantic Equivalence: Different phrasings of same concept, synonymous expressions
+Keyword Validation: Must contain specific terms, key concepts
+Format Validation: Output length constraints, required fields
+```
+
+**Scoring:** 0.0-1.0 range based on semantic similarity. Accept 0.7+ for good match.
+
+### Pattern 3: Multi-Step Orchestration Agents
+
+For agents that coordinate multiple tools or services:
+
+**Evaluator Selection:**
+- **Primary:** LLMJudgeTrajectoryEvaluator (execution path validation)
+- **Secondary:** JsonSimilarityEvaluator (output structure)
+
+**Test Cases:**
+```
+Tool Sequence: Tools called in expected order, correct arguments
+Tool Interaction: Output of one tool becomes input to next
+Error Handling: Fallback paths when tool fails, graceful degradation
+```
+
+### Pattern 4: API Integration Agents
+
+For agents that interact with external APIs:
+
+**Evaluator Selection:**
+- **Primary:** JsonSimilarityEvaluator (response structure)
+- **Secondary:** ExactMatchEvaluator (specific fields)
+
+**Mocking Strategy:** Mock all external API calls using mockito type.
+
+**Test Cases:**
+```
+Success Paths: Valid API responses, different formats, pagination
+Error Handling: API errors (500, 404, 403), timeouts, malformed responses
+Edge Cases: Empty results, large responses, rate limiting
+```
+
+## Test Organization
+
+### By Scenario Type
+
+```
+eval-sets/
+├── {agent}-happy-path.json
+├── {agent}-edge-cases.json
+├── {agent}-error-handling.json
+└── {agent}-performance.json
+```
+
+## CI/CD Integration
+
+```bash
+uip codedagent eval <agent> evaluations/eval-sets/smoke-tests.json \
+  --workers 4 \
+  --mocker-cache \
+  --output-file eval-results.json
+```
+
+## Evaluator Selection Quick Guide
+
+| Agent Type | Primary Evaluator | Secondary | Notes |
+|-----------|------------------|-----------|-------|
+| Calculator | ExactMatch | - | Deterministic |
+| Text Generator | LLMJudge | Contains | Natural language |
+| Orchestrator | LLMJudgeTrajectory | JsonSimilarity | Multi-step flow |
+| API Client | JsonSimilarity | ExactMatch | Structured data |
+| Summarizer | LLMJudge | Contains | Semantic matching |