@salesforce/afv-skills 1.25.0 → 1.27.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/README.md +3 -3
  2. package/package.json +1 -1
  3. package/skills/dx-devops-test-failures-analyze/SKILL.md +89 -0
  4. package/skills/dx-devops-test-failures-analyze/references/code-analyzer-violations.md +26 -0
  5. package/skills/dx-devops-test-failures-analyze/references/failure-categories.md +85 -0
  6. package/skills/{checking-devops-prerequisites/SKILL.md → dx-devops-test-failures-analyze/references/prerequisite-checks.md} +8 -37
  7. package/skills/{creating-fix-work-item/SKILL.md → dx-devops-test-failures-analyze/references/work-item-creation.md} +8 -12
  8. package/skills/dx-devops-test-pipeline-configure/SKILL.md +72 -0
  9. package/skills/dx-devops-test-pipeline-configure/references/configuring-quality-gate.md +133 -0
  10. package/skills/dx-devops-test-pipeline-configure/references/configuring-test-provider.md +80 -0
  11. package/skills/dx-devops-test-pipeline-configure/references/error-handling.md +39 -0
  12. package/skills/dx-devops-test-pipeline-configure/references/gotchas.md +37 -0
  13. package/skills/dx-devops-test-pipeline-configure/references/prerequisite-checks.md +112 -0
  14. package/skills/dx-devops-test-pipeline-configure/references/syncing-test-providers.md +69 -0
  15. package/skills/dx-devops-test-suite-assignments-configure/SKILL.md +74 -0
  16. package/skills/dx-devops-test-suite-assignments-configure/references/api-endpoint.md +30 -0
  17. package/skills/dx-devops-test-suite-assignments-configure/references/error-handling.md +14 -0
  18. package/skills/dx-devops-test-suite-assignments-configure/references/prerequisite-checks.md +112 -0
  19. package/skills/{recommending-devops-tests/SKILL.md → dx-devops-test-suite-assignments-configure/references/recommendation-logic.md} +10 -26
  20. package/skills/dx-devops-test-suite-assignments-configure/references/suite-assignment-modes.md +99 -0
  21. package/skills/dx-devops-test-suite-run/SKILL.md +111 -0
  22. package/skills/dx-devops-test-suite-run/references/error-handling.md +31 -0
  23. package/skills/dx-devops-test-suite-run/references/polling-configuration.md +78 -0
  24. package/skills/dx-devops-test-suite-run/references/prerequisite-checks.md +112 -0
  25. package/skills/dx-devops-test-suite-run/references/retrigger-mode.md +51 -0
  26. package/skills/analyzing-test-failures/SKILL.md +0 -159
  27. package/skills/configuring-quality-gate/SKILL.md +0 -120
  28. package/skills/configuring-test-provider/SKILL.md +0 -113
  29. package/skills/managing-suite-assignments/SKILL.md +0 -161
  30. package/skills/polling-test-results/SKILL.md +0 -72
  31. package/skills/running-devops-test-suite/SKILL.md +0 -144
  32. package/skills/syncing-test-providers/SKILL.md +0 -108
package/README.md CHANGED
@@ -9,9 +9,9 @@ The skills are contributed by Salesforce and the broader community. It’s optim
9
9
  ```
10
10
  sf-skills/
11
11
  ├── skills/ # Directory-based executable workflows
12
- │ ├── generating-apex/
13
- │ ├── generating-custom-object/
14
- │ ├── generating-flow/
12
+ │ ├── platform-apex-generate/
13
+ │ ├── platform-custom-object-generate/
14
+ │ ├── automation-flow-generate/
15
15
  │ └── ...
16
16
  ├── samples/ # Synced sample apps (e.g. from npm)
17
17
  │ └── ui-bundle-template-app-react-sample-b2e/
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@salesforce/afv-skills",
3
- "version": "1.25.0",
3
+ "version": "1.27.0",
4
4
  "description": "Salesforce skills for Agentforce Vibes",
5
5
  "license": "CC-BY-NC-4.0",
6
6
  "files": [
@@ -0,0 +1,89 @@
1
+ ---
2
+ name: dx-devops-test-failures-analyze
3
+ description: "Analyzes DevOps Center test failures and Code Analyzer violations in plain language — failure category, offending file/class/method/line, rule violated, fix direction, and prioritized improvement suggestions (test-code vs production-code) — then optionally creates a tracked fix WorkItem on explicit request. Analysis is pure reasoning; work-item creation is a confirmation-gated write. Use this skill to explain failures or improvement suggestions, translate Code Analyzer violations, or track a fix as a work item. TRIGGER when: a run failed and the user wants root cause; a quality gate failure needs explaining; violations need translating; the user shares a failure payload and asks how to address it; wants to strengthen tests; or wants to create a fix work item, log a remediation, or assign a failure. DO NOT TRIGGER when: the user wants fix code written (use platform-apex-generate) or new test classes authored (use platform-apex-test-generate)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Analyze DevOps Center Test Failures
10
+
11
+ Parses a test failure or Code Analyzer violation payload, explains it in plain language, produces prioritized improvement suggestions, and — only on explicit user request — creates a tracked fix work item. Parts 1–2 are pure reasoning (no writes); Part 3 is an optional, confirmation-gated write.
12
+
13
+ **Never expose raw JSON, stack traces, or internal Salesforce error codes to the user.** Always translate to file name, method, line, and plain description.
14
+
15
+ ---
16
+
17
+ ## Prerequisites
18
+
19
+ - **Parts 1–2 (analysis):** If the failure payload is already in context, no prerequisites are needed — this is pure reasoning. If you must fetch the payload yourself, run prerequisites (`references/prerequisite-checks.md`, Prereqs 1–4) and obtain the execution result via `dx-devops-test-suite-run` (its polling step).
20
+ - **Part 3 (work item):** Run Prerequisites 1–4. You also need a `DevopsProjectId` to file under and an `OwnerId` (assignee). See `references/work-item-creation.md`.
21
+
22
+ ---
23
+
24
+ ## Part 1 — Classify and explain each failure
25
+
26
+ Determine the failure category, then for each failure extract and translate to plain language: offending file/class, method, line number, the rule or assertion violated, and a fix direction (without writing code). Group failures by category if more than one.
27
+
28
+ | Category | Description |
29
+ |---|---|
30
+ | Assertion failure | A test assertion failed (expected vs actual mismatch) |
31
+ | Exception | An unhandled exception was thrown |
32
+ | Code Analyzer violation | A static-analysis rule was violated (e.g. `ApexCRUDViolation`) |
33
+ | Timeout | Test exceeded execution time limit |
34
+ | Compile error | Class failed to compile |
35
+
36
+ **Output format:**
37
+
38
+ ```text
39
+ Test failure summary:
40
+
41
+ <N> failure(s) found:
42
+
43
+ 1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
44
+ What happened: <plain-language description>
45
+ Rule violated: <ruleName or assertion description>
46
+ Fix direction: <plain-language suggestion>
47
+ ```
48
+
49
+ Full category/pattern tables and Code Analyzer rule translations: `references/failure-categories.md` and `references/code-analyzer-violations.md`.
50
+
51
+ **Empty / no-data case:** If the payload contains no failures or violations, report that clearly (e.g. "No failures found in the provided execution results.") and stop. Do NOT fabricate failures or suggestions.
52
+
53
+ ---
54
+
55
+ ## Part 2 — Improvement suggestions
56
+
57
+ Run this **after execution completes with failures**, not on static source. For each failed test, reason over the failure message (the primary signal) to identify what the test is not handling, then produce a specific, actionable suggestion and a **fix location** (Test vs Production code). The full failure-pattern → suggestion mapping is in `references/failure-categories.md`.
58
+
59
+ ```text
60
+ Test improvement suggestions based on execution results:
61
+
62
+ `<testMethodName>()` — [Assertion Failure / Exception / etc.]
63
+ Failure: "<failure message>"
64
+ What this reveals: <plain-language explanation>
65
+ Suggestion: <specific, actionable recommendation>
66
+ Fix location: Test | Production code
67
+
68
+ Overall: <N> improvement(s) across <M> failed test(s).
69
+ ```
70
+
71
+ Do not rewrite the test — only describe what needs to change and why. **Fix location: Production code** indicates a code defect exposed by a sound test (track separately, not a test-quality blocker). **Fix location: Test** indicates the test needs hardening (setup, assertions, edge cases).
72
+
73
+ ---
74
+
75
+ ## Part 3 — Create a fix work item (optional, on request only)
76
+
77
+ Trigger only when the user wants to create a fix work item, log a remediation, or assign a failure to a developer. This is a **write** operation with a mandatory confirmation gate. Follow `references/work-item-creation.md` for inputs, the subject/assignee/project confirmation gate, the `sf data create record --sobject WorkItem` call, and error handling.
78
+
79
+ > Use `WorkItem` (no namespace) — `DevopsWorkItem` is not a supported sObject in this org version.
80
+
81
+ If no `DevopsProject` exists in the org, report that the work item cannot be created until a project is set up — do NOT fabricate a project or proceed.
82
+
83
+ ---
84
+
85
+ ## Related skills
86
+
87
+ - **`dx-devops-test-suite-run`** — produces the failure payload (via its polling step) that feeds this skill.
88
+ - **`dx-devops-test-suite-assignments-configure`** — assign/strengthen the suites whose tests are failing.
89
+ - **`platform-apex-generate` / `platform-apex-test-generate`** — to actually write fix code or new test classes (out of scope here).
@@ -0,0 +1,26 @@
1
+ # Code Analyzer Violations
2
+
3
+ For each Code Analyzer violation, always include:
4
+
5
+ - The rule name translated to plain English
6
+ - The exact line number
7
+ - The fix direction (without writing code)
8
+
9
+ ## Rule-name translations (examples)
10
+
11
+ | Rule name | Plain-language meaning | Fix direction |
12
+ |---|---|---|
13
+ | `ApexCRUDViolation` | A SOQL query or DML was made without checking object-level permissions first | Add a `Schema.sObjectType.<Object>.isAccessible()` (or `isCreateable`/`isUpdateable`) check before the operation |
14
+ | `ApexSharingViolations` | A class that performs data access does not declare sharing enforcement | Add `with sharing` (or an explicit `without sharing` with justification) to the class declaration |
15
+ | `ApexDangerousMethods` | A dangerous or disallowed method was used | Replace with the safe, supported alternative |
16
+ | `EmptyCatchBlock` | An exception is being swallowed silently | Log or handle the exception rather than leaving the catch block empty |
17
+
18
+ When a rule isn't in this table, translate it from its name and message into a plain-language description of what was violated, then give the fix direction.
19
+
20
+ ## Plain-language rule
21
+
22
+ Never paste raw stack traces, JSON payloads, or internal Salesforce error codes into the output. Always translate to file name, method, line, and plain description.
23
+
24
+ ## Fix location for violations
25
+
26
+ Code Analyzer violations almost always indicate a **production-code** fix — the static-analysis rule is flagging the code under test, not the test itself. Flag these as **Fix location: Production code** and track them separately from test-quality issues.
@@ -0,0 +1,85 @@
1
+ # Failure Categories & Improvement Mapping
2
+
3
+ ## Inputs required per failed test
4
+
5
+ - Test method name
6
+ - Failure message (the assertion error or exception text)
7
+ - Failure category (assertion failure, unhandled exception, timeout, compile error)
8
+
9
+ ## Category table
10
+
11
+ | Category | Description |
12
+ |---|---|
13
+ | **Assertion failure** | A test assertion failed (expected vs actual mismatch) |
14
+ | **Exception** | An unhandled exception was thrown |
15
+ | **Code Analyzer violation** | A static analysis rule was violated (e.g. `ApexCRUDViolation`, `ApexSharingViolations`) |
16
+ | **Timeout** | Test exceeded execution time limit |
17
+ | **Compile error** | Class failed to compile |
18
+
19
+ ## Per-failure extraction (Part 1)
20
+
21
+ For each failure, extract and translate to plain language:
22
+ - Offending file and class name
23
+ - Method name
24
+ - Line number
25
+ - What rule or assertion was violated, in plain language
26
+ - Suggested fix direction (without writing code)
27
+
28
+ Group failures by category if more than one.
29
+
30
+ ## Failure-pattern → improvement suggestion (Part 2)
31
+
32
+ Reason over the failure message to identify the root-cause pattern:
33
+
34
+ | Failure pattern | Improvement suggestion |
35
+ |---|---|
36
+ | `NullPointerException` | The test is not handling null input — add a null check or a test setup that ensures the data exists |
37
+ | `Assertion failed: expected X but was Y` | The expected value in the assertion is wrong or the test data setup does not produce the right state |
38
+ | `List has no rows for assignment` | The test is querying for data that doesn't exist — test setup is incomplete |
39
+ | `System.LimitException: Too many SOQL queries` | The test is hitting governor limits — the code under test or test setup is making too many queries |
40
+ | `Insufficient access rights on cross-reference id` | The test user lacks permissions — run as a user with the appropriate profile/permission set |
41
+ | `DML currently not allowed` | The test is performing DML inside a method called from a context that doesn't allow it |
42
+ | Code Analyzer violation message | The production code violates a specific rule — the test exposed it, but the fix is in production code, not the test |
43
+
44
+ ## Producing actionable suggestions
45
+
46
+ For each failure describe, in plain language:
47
+ - What the failure reveals about what the test is not handling
48
+ - What specifically should be added or changed to make the test robust
49
+ - Whether the fix is in the **test** (assertion, setup, permissions) or in **production code** (test is correct, code under test is broken)
50
+
51
+ Do not rewrite the test — only describe what needs to change and why.
52
+
53
+ ## Test fix vs. production-code fix
54
+
55
+ - **Fix location: Production code** — a code defect exposed by a sound test. Should NOT block suite promotion on test-quality grounds; track separately as a production defect.
56
+ - **Fix location: Test** — the test needs hardening: missing setup, wrong assertions, inadequate coverage of edge cases (null inputs, bulk record volumes, mixed permission contexts, governor-limit boundaries).
57
+
58
+ ## Output formats
59
+
60
+ **Part 1 — failure summary:**
61
+
62
+ ```text
63
+ Test failure summary:
64
+
65
+ <N> failure(s) found:
66
+
67
+ 1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
68
+ What happened: <plain-language description>
69
+ Rule violated: <ruleName or assertion description>
70
+ Fix direction: <plain-language suggestion>
71
+ ```
72
+
73
+ **Part 2 — improvement suggestions:**
74
+
75
+ ```text
76
+ Test improvement suggestions based on execution results:
77
+
78
+ `<testMethodName>()` — [Assertion Failure / Exception / etc.]
79
+ Failure: "<failure message>"
80
+ What this reveals: <plain-language explanation>
81
+ Suggestion: <specific, actionable recommendation>
82
+ Fix location: Test | Production code
83
+
84
+ Overall: <N> improvement(s) across <M> failed test(s).
85
+ ```
@@ -1,22 +1,16 @@
1
- ---
2
- name: checking-devops-prerequisites
3
- description: "Validate the environment before any DevOps Center pipeline testing action: confirms an authenticated Salesforce org, the Agentforce DX plugin, an authenticated DevOps Center org, and an identified pipeline (and optionally a pipeline stage). Use this FIRST, internally, from any DevOps testing skill (running a test suite, polling results, syncing tests, configuring or retriggering a quality gate, assigning/mapping suites, creating fix work items, recommending tests, explaining coverage, analyzing failures). TRIGGER when another DevOps testing skill needs to confirm org/pipeline context before a query or system call. DO NOT TRIGGER for non-DevOps-Center work."
4
- metadata:
5
- version: "1.0"
6
- minApiVersion: "67.0"
7
- ---
8
-
9
- # Checking DevOps Prerequisites
1
+ # Prerequisite Checks — Shared DevOps Center Gate
10
2
 
11
- Shared gate for every DevOps Center testing skill. Run these checks **before** any query or system call. On any failure, surface the plain-language message and stop until the user resolves it — never proceed to a write with an unverified environment.
3
+ Shared environment gate for every DevOps Center testing skill. Run these checks **before** any query or system call. On any failure, surface the plain-language message and stop until the user resolves it — never proceed to a write with an unverified environment.
12
4
 
13
5
  > **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
14
6
 
15
7
  **Important:** All DevOps Center data (pipelines, stages, test suites, executions) lives in the Salesforce org — NOT in the local repository. Never search the filesystem for pipeline configuration. Always query the org using `sf data query` or `sf api request rest`.
16
8
 
17
- ## How other skills use this
9
+ **Object model use the STANDARD objects, never the `sf_devops__` managed package:** DevOps Center testing data lives in standard platform objects — `DevopsPipeline`, `DevopsPipelineStage`, `DevopsPipelineStageTrigger`, `DevopsTestSuite`, `DevopsTestSuiteStage`, `DevopsTestSuiteExecution`, `DevopsProject`, `WorkItem`. Do **NOT** query the legacy managed-package objects (`sf_devops__Pipeline__c`, `sf_devops__Pipeline_Stage__c`, etc.) and do **NOT** gate on a `PackageLicense` / namespace check for `sf_devops`. "DevOps Center installed" is determined **solely** by whether `DevopsPipeline` is queryable and returns records (Prerequisite 4) — never by the presence of the `sf_devops__` namespace. If a `sf_devops__*` object is missing, that is expected and is NOT evidence that DevOps Center is uninstalled.
18
10
 
19
- Every DevOps testing skill loads this skill first and runs Prerequisites 1–4 in order. Prerequisite 5 (stage) is run **only** when the calling skill operates on a specific pipeline stage (running/retriggering a suite, syncing, configuring a gate, mapping a suite). The calling skill passes back the resolved `doce-org-alias`, `pipelineId`, and (when applicable) `stageId` to use in its own commands.
11
+ ## How skills use this
12
+
13
+ Run Prerequisites 1–4 in order. Prerequisite 5 (stage) is run **only** when the operation targets a specific pipeline stage (configuring a gate, running/retriggering a suite, mapping a suite). Carry forward the resolved `doce-org-alias`, `pipelineId`, and (when applicable) `stageId`.
20
14
 
21
15
  Resolve the **DevOps Center org alias** without asking the user unless genuinely ambiguous:
22
16
  1. If the user named an org alias in their message, use it.
@@ -94,7 +88,7 @@ sf data query \
94
88
 
95
89
  ## Prerequisite 5 — Pipeline stage identified (conditional)
96
90
 
97
- Run **only** when the calling skill operates on a specific stage.
91
+ Run **only** when the operation targets a specific stage (e.g. configuring a quality gate).
98
92
 
99
93
  If the user's message already names a stage (e.g. "Integration", "Staging", "Production"), use that name directly — do NOT ask again. Look up its Id:
100
94
 
@@ -115,27 +109,4 @@ sf data query \
115
109
  Then ask: "Which pipeline stage are we working with?" — do NOT ask for an org alias; stages are resolved by name from the pipeline.
116
110
 
117
111
  - **Pass:** stage Id and Name confirmed
118
- - **Deferred:** not required until a calling skill needs it
119
-
120
- ---
121
-
122
- ## Error Handling
123
-
124
- | Error condition | Response |
125
- |---|---|
126
- | `sf org list` fails entirely | "Could not reach the Salesforce CLI. Make sure `sf` is installed and on your PATH." |
127
- | `sf org display` returns auth error | Surface plain-language re-auth instructions. Do not expose the raw error. |
128
- | `DevopsPipeline` query fails with 5xx | "The DevOps Center org is returning a server error. Try again in a few minutes." |
129
- | Any check throws unexpectedly | "Something went wrong checking prerequisites. Error: [plain summary]. Let's try again — or resolve it manually and let me know when ready." |
130
-
131
- Never expose raw API errors, stack traces, or JSON error payloads to the user.
132
-
133
- ## Gotchas
134
-
135
- | Issue | Resolution |
136
- |---|---|
137
- | `DevopsPipeline` query returns empty | DevOps Center not installed on the target org, or wrong org alias — ask the user to verify |
138
- | `DevopsWorkItem` sObject not supported | Use `WorkItem` (no namespace) — the correct API name for this org version |
139
- | Review trigger is pipeline-level, not stage-level | Query `DevopsPipelineStageTrigger` where `TriggerType = 'Review'` and `RelatedRecordId = <pipelineId>` |
140
- | Connect API `testSuites` returns empty with `?stageId=` | Use `?triggerId=<reviewTriggerId>` — `stageId` only works for stage-level triggers |
141
- | `sf plugins` doesn't match `agentforce` | The installed plugin is `@salesforce/plugin-agent` — match on `plugin-agent` too |
112
+ - **Deferred:** not required until the operation needs it
@@ -1,14 +1,10 @@
1
- ---
2
- name: creating-fix-work-item
3
- description: "Creates a DevOps Center WorkItem to track a fix for a test failure or Code Analyzer violation. Before creating anything, it shows a subject/assignee/project preview and requires explicit confirmation. Use this skill when you want to create a fix work item, track a remediation task, or assign a test or analysis failure to a developer. TRIGGER when: the user wants to create a fix work item, log a remediation, or assign a failure to a developer for resolution. DO NOT TRIGGER when: writing the fix code itself (use platform-apex-generate)."
4
- metadata:
5
- version: "1.0"
6
- minApiVersion: "67.0"
7
- ---
1
+ # Part 3 — Create a Fix Work Item
2
+
3
+ Creates a DevOps Center `WorkItem` to track a fix for a test failure or Code Analyzer violation. This is an **optional write**, triggered only when the user asks to create a fix work item, log a remediation, or assign a failure to a developer.
8
4
 
9
5
  ## Prerequisites
10
6
 
11
- Load `checking-devops-prerequisites` first — Prerequisites 1–4. You need `doce-org-alias`, the `DevopsProjectId` to file under, and an `OwnerId` (assignee). If no DevopsProject exists, surface that the work item cannot be created until a project exists.
7
+ Run Prerequisites 1–4 (`references/prerequisite-checks.md`). You need `doce-org-alias`, a `DevopsProjectId` to file under, and an `OwnerId` (assignee). If no `DevopsProject` exists, surface that the work item cannot be created until a project exists — do NOT fabricate a project or work item.
12
8
 
13
9
  ## Inputs required before creating
14
10
 
@@ -16,7 +12,7 @@ Load `checking-devops-prerequisites` first — Prerequisites 1–4. You need `do
16
12
  |---|---|
17
13
  | `DevopsProjectId` | From the pipeline's associated project — query `DevopsProject WHERE Name = '<projectName>'` on the doce org if not already known |
18
14
  | `Subject` | Derived from failure analysis — e.g. "Fix: Missing code-analyzer-v5.yml workflow in blitz-10-06 repository" |
19
- | `OwnerId` | User ID of the developer to assign to — query `SELECT Id, Name FROM User WHERE Username = '<username>'` on the doce org if not known. Ask the user if the username is unknown. |
15
+ | `OwnerId` | User ID of the developer to assign to — query `SELECT Id, Name FROM User WHERE Username = '<username>'` on the doce org if not known. Default to the requesting user when no assignee is specified; ask only if the username is unknown and no default applies. |
20
16
  | `doce-org-alias` | Established in Prerequisites |
21
17
 
22
18
  ## Confirmation gate
@@ -46,7 +42,7 @@ sf data create record \
46
42
 
47
43
  ## On success
48
44
 
49
- Parse the returned `id` from the result and confirm:
45
+ Parse the returned `id` and confirm:
50
46
 
51
47
  > "Fix work item created (`<id>`): `<subject>`. Assigned to `<assigneeName>` in the `<projectName>` project."
52
48
 
@@ -61,6 +57,6 @@ Never expose raw API error messages. Map errors to plain-language responses:
61
57
  | `INSUFFICIENT_ACCESS` | "Your user doesn't have permission to create work items in this project." |
62
58
  | Any other error | "The work item could not be created. Error: `<plain summary>`. Try again or create it manually in DevOps Center." |
63
59
 
64
- ## Related skills
60
+ ## No-project case
65
61
 
66
- - `analyzing-test-failures` provides the failure analysis and improvement suggestions that motivate creating a fix work item
62
+ If the `DevopsProject` query returns 0 records, report clearly that no DevOps Center project exists and the work item cannot be created until one is set up. Do NOT fabricate a project name/ID, do NOT proceed to the confirmation gate or the create command.
@@ -0,0 +1,72 @@
1
+ ---
2
+ name: dx-devops-test-pipeline-configure
3
+ description: "Configures DevOps Center pipeline testing infrastructure: enables a test provider so its suites become available, re-syncs a configured provider to pull in new suites, or creates a quality gate with rules on a stage. Routes by intent across three modes after running shared prerequisite checks and an explicit confirmation gate. Use this skill when a user wants to set up, configure, enable, sync, or refresh a test provider, or set/configure a quality gate or coverage threshold on a DevOps Center pipeline stage. TRIGGER when: the user wants to configure/enable/add/set up a test provider, re-sync or refresh a provider's suite list, pull in new suites, or set/configure a quality gate, coverage threshold, or testing benchmark on a stage. DO NOT TRIGGER when: assigning existing suites to a stage (use dx-devops-test-suite-assignments-configure), running or retriggering a suite (use dx-devops-test-suite-run), or non-DevOps-Center work."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Configure DevOps Center Pipeline Testing Infrastructure
10
+
11
+ Sets up and configures a DevOps Center pipeline's testing infrastructure. This skill handles three closely related "configure your pipeline" operations that share the same org context, prerequisites, and entity scope (the pipeline level). Pick the mode that matches the user's intent.
12
+
13
+ > **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
14
+
15
+ **Important:** All DevOps Center data (pipelines, stages, providers, suites, gates) lives in the Salesforce org — NOT the local repo. Never search the filesystem for pipeline configuration. Always query the org with `sf data query` or `sf api request rest`.
16
+
17
+ ---
18
+
19
+ ## Step 1 — Run prerequisites first (always)
20
+
21
+ Before any query or system call, run the prerequisite checks in `references/prerequisite-checks.md`. On any failure, surface the plain-language message and stop — never write to an unverified environment.
22
+
23
+ - **Modes A & B (provider configure/sync):** run Prerequisites 1–4 (org login, Agentforce DX plugin, DevOps Center org auth, pipeline identified). Prerequisite 5 (stage) is **not** required — providers are configured at the pipeline level.
24
+ - **Mode C (quality gate):** run Prerequisites 1–4 **and** Prerequisite 5 (stage). Prereq 5 gives the `DevopsPipelineStage` only — the target `DevopsTestSuiteStage` record Id is resolved separately in Mode C's Step 0 (trigger → suite-stage row).
25
+
26
+ Carry forward the resolved `doce-org-alias`, `pipelineId`, and (Mode C) `stageId` / `testSuiteStageId`.
27
+
28
+ ---
29
+
30
+ ## Step 2 — Select the mode
31
+
32
+ | If the user wants to… | Mode | Follow |
33
+ |---|---|---|
34
+ | Enable / set up / add a provider that is **not yet configured** | **A — Configure a test provider** | `references/configuring-test-provider.md` |
35
+ | Re-sync / refresh an **already-configured** provider to pull in new suites | **B — Sync a configured provider** | `references/syncing-test-providers.md` |
36
+ | Set / configure a quality gate, coverage threshold, or testing benchmark on a stage | **C — Configure a quality gate** | `references/configuring-quality-gate.md` |
37
+
38
+ **Disambiguating A vs B (the critical decision):** First fetch the pipeline's providers (`GET .../testProviders?status=all`) — both modes start there. Then:
39
+
40
+ - Provider is **Available** (not configured) → **Mode A** (configure).
41
+ - Provider is **Configured** but suites are stale/missing → **Mode B** (sync).
42
+ - Provider is **Configured** and the user can't see suites *when assigning to a stage* → this is a **stage-assignment gap, not a configuration gap**. Redirect to `dx-devops-test-suite-assignments-configure`.
43
+
44
+ ⚠ **Never POST to the configure endpoint for an already-configured provider** — it creates duplicate `DevopsPipelineTestProvider` records. See `references/gotchas.md`.
45
+
46
+ ---
47
+
48
+ ## Step 3 — Confirmation gate (required in every mode)
49
+
50
+ Every mode mutates org state and **must** show a confirmation gate before any write. Each mode's reference file contains its exact gate wording (Mode C additionally requires a mandatory impact preview before the gate). Do not call any write API until the user gives an affirmative response. If the user declines, stop without writing.
51
+
52
+ ---
53
+
54
+ ## Step 4 — Execute and report
55
+
56
+ Follow the chosen reference file for the exact API calls, success messages, and error handling:
57
+
58
+ - `references/configuring-test-provider.md` — Mode A
59
+ - `references/syncing-test-providers.md` — Mode B
60
+ - `references/configuring-quality-gate.md` — Mode C
61
+ - `references/error-handling.md` — consolidated status-code → plain-language tables for all modes
62
+ - `references/gotchas.md` — duplicate-provider trap, API-name differences, trigger-type rules
63
+
64
+ Never expose raw API errors, stack traces, or JSON payloads to the user — always translate to plain language.
65
+
66
+ ---
67
+
68
+ ## Related skills
69
+
70
+ - **`dx-devops-test-suite-assignments-configure`** — after configuring/syncing a provider, assign or map its suites to a stage; also recommends which suites to run for a commit.
71
+ - **`dx-devops-test-suite-run`** — run a suite, or retrigger a quality gate after fixes meet the threshold.
72
+ - **`dx-devops-test-failures-analyze`** — explain failures from a run and optionally create a fix work item.
@@ -0,0 +1,133 @@
1
+ # Mode C — Configure a Quality Gate
2
+
3
+ Creates a DevOps Center quality gate with associated rules (`PASS_PERCENTAGE`, `SEVERITY`, `ESSENTIAL`) and links it to a pipeline-stage suite assignment, after a mandatory impact preview and explicit confirmation.
4
+
5
+ ## Prerequisites
6
+
7
+ Run Prerequisites 1–4 **and** Prerequisite 5 (stage). You need `doce-org-alias`, `pipelineId`, and `stageId`. Prereq 5 resolves only the **`DevopsPipelineStage`** — it does **not** give you the `DevopsTestSuiteStage` record. Resolve that separately in Step 0 below.
8
+
9
+ ## Inputs required
10
+
11
+ | Input | Source |
12
+ |---|---|
13
+ | `name` | User-provided name for the quality gate |
14
+ | `rules` | List of `{type, threshold?}` — see rule types below |
15
+ | `doce-org-alias` | Prerequisite 1 |
16
+ | `event` | `Pre-Promote`, `Post-Promote`, or `Review` — which trigger the gate applies to |
17
+ | `testSuiteName` | The suite the gate will gate (the gate links to a specific suite-on-stage) |
18
+ | `testSuiteStageId` | Target `DevopsTestSuiteStage` record Id — **resolved in Step 0**, not from Prereq 5 |
19
+
20
+ ---
21
+
22
+ ## Step 0 — Resolve the target `DevopsTestSuiteStage` record
23
+
24
+ A quality gate links to a specific suite assigned to a stage's trigger, represented by a `DevopsTestSuiteStage` record. That object is keyed by `DevopsPipelineStageTriggerId` + `TestSuiteId` (it is **not** linked directly to `DevopsPipelineStage`), so resolve it in two queries.
25
+
26
+ **0a — Find the stage's trigger for the chosen event:**
27
+
28
+ ```bash
29
+ sf data query \
30
+ --query "SELECT Id, TriggerType FROM DevopsPipelineStageTrigger WHERE RelatedRecordId = '<stageId>' AND TriggerType = '<event>'" \
31
+ --target-org <doce-org-alias> --json
32
+ ```
33
+
34
+ > For a `Review` gate the trigger is pipeline-level — use `RelatedRecordId = '<pipelineId>'` with `TriggerType = 'Review'` instead. Record the returned Id as `<triggerId>`.
35
+
36
+ **0b — Find the suite-stage row for the target suite on that trigger:**
37
+
38
+ ```bash
39
+ sf data query \
40
+ --query "SELECT Id, TestSuiteId, TestSuite.Name, IsQualityGateEnabled, DevopsQualityGateId FROM DevopsTestSuiteStage WHERE DevopsPipelineStageTriggerId = '<triggerId>'" \
41
+ --target-org <doce-org-alias> --json
42
+ ```
43
+
44
+ Match the row whose `TestSuite.Name` is the suite the user named; its `Id` is `<testSuiteStageId>`.
45
+
46
+ - If **no** `DevopsTestSuiteStage` row matches (the suite isn't assigned to that trigger), stop and report it — the suite must be assigned first via `dx-devops-test-suite-assignments-configure`. Do **not** fabricate a `testSuiteStageId` or proceed.
47
+ - If the matched row already has `IsQualityGateEnabled = true`, tell the user a gate is already linked and confirm whether to replace it before continuing.
48
+
49
+ ## Rule types
50
+
51
+ | Type | Description | Threshold |
52
+ |---|---|---|
53
+ | `PASS_PERCENTAGE` | Minimum % of tests that must pass | Required (0–100) |
54
+ | `SEVERITY` | Maximum allowed severity level of failures | Required (numeric, e.g. 1–5) |
55
+ | `ESSENTIAL` | All essential tests must pass | Not required |
56
+
57
+ ---
58
+
59
+ ## MANDATORY IMPACT PREVIEW
60
+
61
+ **Before executing any commands**, show the user this preview and wait for explicit confirmation:
62
+
63
+ > "Here's what this quality gate will enforce on `<stageName>`:
64
+ > - Rule: `<type>` — `<description>`
65
+ > - Threshold: `<value>`
66
+ > - Affected pipelines: `<list>`
67
+ >
68
+ > Confirm to apply?"
69
+
70
+ **Never proceed past this point without showing the impact preview first and receiving explicit confirmation.**
71
+
72
+ ## Confirmation gate
73
+
74
+ Only proceed after the user has explicitly confirmed (e.g., "yes", "confirm", "go ahead"). If the user declines or does not confirm, stop and do not execute any commands.
75
+
76
+ ---
77
+
78
+ ## Step 1 — Create the gate record
79
+
80
+ ```bash
81
+ sf api request rest \
82
+ "/services/data/v67.0/connect/devopstesting/qualityGate" \
83
+ --method POST \
84
+ --body '{"name": "<gateName>"}' \
85
+ --target-org <doce-org-alias>
86
+ ```
87
+
88
+ Extract `qualityGateId` from the response.
89
+
90
+ ## Step 2 — Create each rule as an sObject record
91
+
92
+ Rules are not accepted in the Connect API payload — create them as `DevopsQualityGateRule` records directly. Only create rules the user has requested. `ESSENTIAL` has no threshold.
93
+
94
+ ```bash
95
+ sf data create record \
96
+ --sobject DevopsQualityGateRule \
97
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='PASS_PERCENTAGE' Threshold=<value>" \
98
+ --target-org <doce-org-alias> --json
99
+
100
+ sf data create record \
101
+ --sobject DevopsQualityGateRule \
102
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='ESSENTIAL'" \
103
+ --target-org <doce-org-alias> --json
104
+
105
+ sf data create record \
106
+ --sobject DevopsQualityGateRule \
107
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='SEVERITY' Threshold=<value>" \
108
+ --target-org <doce-org-alias> --json
109
+ ```
110
+
111
+ ## Step 3 — Link the gate to the suite stage
112
+
113
+ Update the `DevopsTestSuiteStage` record to link the new gate:
114
+
115
+ ```bash
116
+ sf data update record \
117
+ --sobject DevopsTestSuiteStage \
118
+ --record-id <testSuiteStageId> \
119
+ --values "IsQualityGateEnabled=true DevopsQualityGateId='<qualityGateId>'" \
120
+ --target-org <doce-org-alias> --json
121
+ ```
122
+
123
+ ---
124
+
125
+ ## On success
126
+
127
+ > "Quality gate `<gateName>` created with `<N>` rule(s) and assigned to `<suiteName>` on `<stageName>`."
128
+
129
+ ## Note
130
+
131
+ To *re-run* an existing gate after meeting its threshold, that is **not** this mode — use `dx-devops-test-suite-run` (retrigger mode). To assign or map suites to stages first, use `dx-devops-test-suite-assignments-configure`.
132
+
133
+ See `references/error-handling.md` for status-code responses.
@@ -0,0 +1,80 @@
1
+ # Mode A — Configure a Test Provider
2
+
3
+ Configures an *Available* (not-yet-configured) test provider on a DevOps Center pipeline, making its test suites available for assignment to pipeline stages.
4
+
5
+ **Confirmation required:** Yes — explicit confirmation before the provider is configured.
6
+
7
+ ## Inputs
8
+
9
+ | Variable | Source |
10
+ |---|---|
11
+ | `doce-org-alias` | Prerequisite 1 |
12
+ | `pipelineId` | Prerequisite 4 (pipeline selection) |
13
+ | `testProviderId` | Resolved by fetching the pipeline's test providers (Step 1 below) |
14
+
15
+ Prerequisite 5 (stage) is **not** required — providers are configured at the pipeline level.
16
+
17
+ ---
18
+
19
+ ## Step 1 — Fetch test providers to resolve the provider ID
20
+
21
+ ```bash
22
+ sf api request rest \
23
+ "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testProviders?status=all" \
24
+ --target-org <doce-org-alias>
25
+ ```
26
+
27
+ Each provider entry includes `testProviderId`, `testProviderName`, and a status (Configured vs. Available). Present a short summary grouped by status:
28
+
29
+ ```text
30
+ Test providers for <pipelineName>:
31
+
32
+ ✓ Configured:
33
+ - Code Analyzer (63 suites)
34
+ - Apex Unit Tests (5 suites)
35
+
36
+ Available (not yet configured):
37
+ - Flow Tests
38
+ ```
39
+
40
+ - **Only an Available provider can be configured.** If the pipeline has no available providers, report that and stop — do NOT fabricate a provider or ID.
41
+
42
+ ### If the named provider is already Configured
43
+
44
+ Do **not** present the confirmation gate and do **not** POST to the configure endpoint (Steps 2–3) — that would create a duplicate `DevopsPipelineTestProvider`. Instead:
45
+
46
+ 1. State plainly that the provider is already configured, including its synced suite count and last-sync time if returned (e.g. *"Flow Tests is already configured on `<pipelineName>` with 3 suites synced (last sync 2026-06-23)."*).
47
+ 2. Diagnose the user's actual goal and redirect **by name**:
48
+ - If the user says the provider's **suites don't appear when assigning tests to a stage**, this is a **stage-assignment gap, not a provider-configuration gap** — the suites already exist at the pipeline level; they just need to be linked to the stage. Redirect to **`dx-devops-test-suite-assignments-configure`**.
49
+ - If the user expects **newly created suites** that aren't yet synced, use **Mode B — Sync a configured provider** (`references/syncing-test-providers.md`) to pull them in.
50
+ 3. Do not loop back to configuring — finish cleanly after the explanation and redirect.
51
+
52
+ ## Step 2 — Confirmation gate
53
+
54
+ **Required — do not call the API before the user confirms.**
55
+
56
+ > "I'll configure `<testProviderName>` on the `<pipelineName>` pipeline. This will make its suites available for assignment to stages. Confirm?"
57
+
58
+ Do not proceed until the user gives an affirmative response.
59
+
60
+ ## Step 3 — Configure the provider
61
+
62
+ On confirmation, call the configure endpoint with the provider ID:
63
+
64
+ ```bash
65
+ sf api request rest \
66
+ "/services/data/v67.0/connect/devops/pipeline/<pipelineId>/testProvider" \
67
+ --method POST \
68
+ --body '{"testProviderId": "<testProviderId>"}' \
69
+ --target-org <doce-org-alias>
70
+ ```
71
+
72
+ ## On success
73
+
74
+ > "Provider `<testProviderName>` is now configured on the `<pipelineName>` pipeline. Its suites are available for assignment to stages."
75
+
76
+ Newly configured suites can then be assigned to stages with `dx-devops-test-suite-assignments-configure`.
77
+
78
+ ---
79
+
80
+ See `references/gotchas.md` for the duplicate-provider trap and `references/error-handling.md` for status-code responses.