@salesforce/afv-skills 1.18.0 → 1.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@salesforce/afv-skills",
3
- "version": "1.18.0",
3
+ "version": "1.19.0",
4
4
  "description": "Salesforce skills for Agentforce Vibes",
5
5
  "license": "CC-BY-NC-4.0",
6
6
  "files": [
@@ -0,0 +1,159 @@
1
+ ---
2
+ name: analyzing-test-failures
3
+ description: "Parses a DevOps Center test-failure or Code Analyzer violation payload and explains it in plain language — failure category, offending file/class/method/line, rule violated, and fix direction — then gives prioritized test-improvement suggestions (distinguishing test-code from production-code fixes). Pure reasoning: no system calls or code authoring. TRIGGER when: a test run failed and the user wants root cause; a quality gate failure needs explaining; Code Analyzer violations need translating to plain language; the user shares a failure payload and asks how to address it; tests keep failing and the user wants suggestions; or the user wants to close coverage gaps, strengthen assertions, or fix flaky/weak tests. DO NOT TRIGGER when: the user wants fix code written (use generating-apex) or new test classes authored (use generating-apex-test)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # analyzing-test-failures
10
+
11
+ **Type:** Pure reasoning — no system calls, no code authoring.
12
+
13
+ **What it does:** Parses a test failure or Code Analyzer violation payload and produces a plain-language explanation, then follows up with concrete, prioritized improvement suggestions per failed test — including whether each fix belongs in the test or in production code. Never exposes raw JSON, stack traces, or API error bodies to the user.
14
+
15
+ ---
16
+
17
+ ## Prerequisites
18
+
19
+ If you need to fetch the failure payload yourself rather than receiving it, load `checking-devops-prerequisites` first, then use `polling-test-results` to obtain the execution result. If the payload is already in context, no prerequisites are needed — this is pure reasoning.
20
+
21
+ ---
22
+
23
+ ## Inputs
24
+
25
+ - JSON failure payload provided by the `polling-test-results` skill, or pasted directly into context from a test run or Code Analyzer execution. Specifically required per failed test:
26
+ - Test method name
27
+ - Failure message (the assertion error or exception text)
28
+ - Failure category (assertion failure, unhandled exception, timeout, compile error)
29
+
30
+ ---
31
+
32
+ ## Empty-org / no-data case
33
+
34
+ If the payload contains no failures or violations, report that clearly (e.g. "No failures found in the provided execution results.") and stop. Do NOT fabricate failures, violations, or improvement suggestions when none are present.
35
+
36
+ ---
37
+
38
+ ## Reasoning steps
39
+
40
+ 1. Determine the failure category:
41
+
42
+ | Category | Description |
43
+ |---|---|
44
+ | **Assertion failure** | A test assertion failed (expected vs actual mismatch) |
45
+ | **Exception** | An unhandled exception was thrown |
46
+ | **Code Analyzer violation** | A static analysis rule was violated (e.g. `ApexCRUDViolation`, `ApexSharingViolations`) |
47
+ | **Timeout** | Test exceeded execution time limit |
48
+ | **Compile error** | Class failed to compile |
49
+
50
+ 2. For each failure, extract and translate to plain language:
51
+ - Offending file and class name
52
+ - Method name
53
+ - Line number
54
+ - What rule or assertion was violated, in plain language
55
+ - Suggested fix direction (without writing code)
56
+
57
+ 3. Group failures by category if more than one.
58
+
59
+ ---
60
+
61
+ ## Output format
62
+
63
+ ```text
64
+ Test failure summary:
65
+
66
+ <N> failure(s) found:
67
+
68
+ 1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
69
+ What happened: <plain-language description>
70
+ Rule violated: <ruleName or assertion description>
71
+ Fix direction: <plain-language suggestion>
72
+
73
+ 2. [<Category>] `<ClassName2>.cls` — `<methodName2>()` at line <N>
74
+ ...
75
+ ```
76
+
77
+ ---
78
+
79
+ ## Code Analyzer violations
80
+
81
+ For violations, always include:
82
+ - The rule name translated to plain English (e.g. "ApexCRUDViolation" → "A SOQL query was made without checking object-level permissions first")
83
+ - The exact line number
84
+ - The fix direction (e.g. "Add a `Schema.sObjectType.Account.isAccessible()` check before the query")
85
+
86
+ ---
87
+
88
+ ## Plain-language rule
89
+
90
+ Never paste raw stack traces, JSON payloads, or internal Salesforce error codes into the output. Always translate to file name, method, line, and plain description.
91
+
92
+ ---
93
+
94
+ ## Suggesting improvements
95
+
96
+ Invoke this section **after test execution completes with failures**, not on static source code. The failure message is the primary signal — it describes what the test expected vs. what actually happened, which directly informs what the test needs to handle better.
97
+
98
+ ### 1 — Read each failure message
99
+
100
+ For each failed test in the execution result, extract:
101
+ - Test method name
102
+ - Failure message (the assertion error or exception message)
103
+ - Failure category (assertion failure, unhandled exception, timeout, compile error)
104
+
105
+ ### 2 — Infer what the test is not handling
106
+
107
+ Reason over the failure message to identify the root cause pattern:
108
+
109
+ | Failure pattern | Improvement suggestion |
110
+ |---|---|
111
+ | `NullPointerException` | The test is not handling null input — add a null check or a test setup that ensures the data exists |
112
+ | `Assertion failed: expected X but was Y` | The expected value in the assertion is wrong or the test data setup does not produce the right state |
113
+ | `List has no rows for assignment` | The test is querying for data that doesn't exist — test setup is incomplete |
114
+ | `System.LimitException: Too many SOQL queries` | The test is hitting governor limits — the code under test or test setup is making too many queries |
115
+ | `Insufficient access rights on cross-reference id` | The test user lacks the required permissions — the test needs to run as a user with appropriate profile/permission set |
116
+ | `DML currently not allowed` | The test is performing DML inside a method called from a context that doesn't allow it |
117
+ | Code Analyzer violation message | The production code violates a specific rule — the test exposed it but the fix is in the production code, not the test |
118
+
119
+ ### 3 — Produce actionable suggestions
120
+
121
+ For each failure, describe in plain language:
122
+ - What the failure reveals about what the test is not handling
123
+ - What specifically should be added or changed to make the test robust
124
+ - Whether the fix is in the **test** (assertion, setup, permissions) or in the **production code** (the test is correct but the code under test is broken)
125
+
126
+ Do not rewrite the test. Only describe what needs to change and why.
127
+
128
+ ### Output format for improvements
129
+
130
+ ```text
131
+ Test improvement suggestions based on execution results:
132
+
133
+ `<testMethodName>()` — [Assertion Failure / Exception / etc.]
134
+ Failure: "<failure message>"
135
+ What this reveals: <plain-language explanation>
136
+ Suggestion: <specific, actionable recommendation>
137
+ Fix location: Test | Production code
138
+
139
+ `<testMethodName2>()` — [NullPointerException]
140
+ Failure: "Attempt to de-reference a null object"
141
+ What this reveals: The test is calling the method without setting up the required input data
142
+ Suggestion: Add test data setup for <object/field> before calling the method
143
+ Fix location: Test
144
+
145
+ Overall: <N> improvement(s) across <M> failed test(s).
146
+ ```
147
+
148
+ ### Test fix vs. production-code fix
149
+
150
+ Suggestions flagged as **Fix location: Production code** indicate a code defect exposed by the test — the test logic itself is sound. These should not block suite promotion on the grounds of test quality; they should be tracked separately as production defects.
151
+
152
+ Suggestions flagged as **Fix location: Test** indicate the test needs to be hardened — missing setup, wrong assertions, inadequate coverage of edge cases (null inputs, bulk record volumes, mixed permission contexts, governor-limit boundaries).
153
+
154
+ ---
155
+
156
+ ## Related skills
157
+
158
+ - **`polling-test-results`** — obtains the failure payload that feeds into this skill.
159
+ - **`creating-fix-work-item`** — use to create a tracked fix item from the analysis output or to track an approved improvement suggestion as a work item once the user decides to act on it.
@@ -0,0 +1,141 @@
1
+ ---
2
+ name: checking-devops-prerequisites
3
+ description: "Validate the environment before any DevOps Center pipeline testing action: confirms an authenticated Salesforce org, the Agentforce DX plugin, an authenticated DevOps Center org, and an identified pipeline (and optionally a pipeline stage). Use this FIRST, internally, from any DevOps testing skill (running a test suite, polling results, syncing tests, configuring or retriggering a quality gate, assigning/mapping suites, creating fix work items, recommending tests, explaining coverage, analyzing failures). TRIGGER when another DevOps testing skill needs to confirm org/pipeline context before a query or system call. DO NOT TRIGGER for non-DevOps-Center work."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Checking DevOps Prerequisites
10
+
11
+ Shared gate for every DevOps Center testing skill. Run these checks **before** any query or system call. On any failure, surface the plain-language message and stop until the user resolves it — never proceed to a write with an unverified environment.
12
+
13
+ > **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
14
+
15
+ **Important:** All DevOps Center data (pipelines, stages, test suites, executions) lives in the Salesforce org — NOT in the local repository. Never search the filesystem for pipeline configuration. Always query the org using `sf data query` or `sf api request rest`.
16
+
17
+ ## How other skills use this
18
+
19
+ Every DevOps testing skill loads this skill first and runs Prerequisites 1–4 in order. Prerequisite 5 (stage) is run **only** when the calling skill operates on a specific pipeline stage (running/retriggering a suite, syncing, configuring a gate, mapping a suite). The calling skill passes back the resolved `doce-org-alias`, `pipelineId`, and (when applicable) `stageId` to use in its own commands.
20
+
21
+ Resolve the **DevOps Center org alias** without asking the user unless genuinely ambiguous:
22
+ 1. If the user named an org alias in their message, use it.
23
+ 2. Otherwise, use the default org (`sf org display --json`, no `--target-org`).
24
+ 3. Only if the default org has no `DevopsPipeline` records (Prereq 4 fails), ask: "Which org alias is your DevOps Center org?"
25
+
26
+ ---
27
+
28
+ ## Prerequisite 1 — Salesforce org: active login
29
+
30
+ ```bash
31
+ sf org list --json
32
+ ```
33
+
34
+ Look for at least one entry in `result.nonScratchOrgs`, `result.scratchOrgs`, or `result.sandboxes` with `"connectedStatus": "Connected"`.
35
+
36
+ - **Pass:** at least one org is Connected
37
+ - **Fail:** no orgs listed, or all show a non-connected status
38
+
39
+ **On fail:** "No authenticated Salesforce org found. Run `sf org login web --alias <your-alias>` in your terminal, then come back."
40
+
41
+ ---
42
+
43
+ ## Prerequisite 2 — Agentforce DX plugin installed
44
+
45
+ ```bash
46
+ sf plugins --json
47
+ ```
48
+
49
+ Look for a plugin entry whose `name` contains `plugin-agent`, `agentforce`, or `einstein` (case-insensitive).
50
+
51
+ - **Pass:** plugin found
52
+ - **Fail:** no matching entry
53
+
54
+ **On fail:** "The Agentforce DX Plugin is not installed. Run `sf plugins install @salesforce/plugin-agent`, then restart the IDE and try again."
55
+
56
+ ---
57
+
58
+ ## Prerequisite 3 — DevOps Center org authenticated
59
+
60
+ ```bash
61
+ sf org display --target-org <doce-org-alias> --json
62
+ ```
63
+
64
+ Check that `"connectedStatus"` is `"Connected"`.
65
+
66
+ - **Pass:** Connected
67
+ - **Fail:** expired session or error
68
+
69
+ **On fail:** "Your DevOps Center org session has expired. Run `sf org login web --alias <doce-org-alias>` to re-authenticate."
70
+
71
+ ---
72
+
73
+ ## Prerequisite 4 — Pipeline identified
74
+
75
+ ```bash
76
+ sf data query \
77
+ --query "SELECT Id, Name, CreatedDate FROM DevopsPipeline ORDER BY Name ASC" \
78
+ --target-org <doce-org-alias> \
79
+ --json
80
+ ```
81
+
82
+ - **Pass (exactly one):** use it automatically — do NOT ask.
83
+ - **Pass (multiple):** if the user already named a pipeline, match by name. Otherwise display a numbered list and ask:
84
+ ```text
85
+ Found <N> pipelines:
86
+ 1. <Name>
87
+ 2. <Name>
88
+ Which pipeline would you like to work with?
89
+ ```
90
+ - **Fail (no records):** "No DevOps Center pipeline found. Create a project and pipeline in DevOps Center before using the DevOps Testing Skills."
91
+ - **Fail (unsupported object):** "DevOps Center does not appear to be installed on `<doce-org-alias>`. Check that you're pointing at the correct org."
92
+
93
+ ---
94
+
95
+ ## Prerequisite 5 — Pipeline stage identified (conditional)
96
+
97
+ Run **only** when the calling skill operates on a specific stage.
98
+
99
+ If the user's message already names a stage (e.g. "Integration", "Staging", "Production"), use that name directly — do NOT ask again. Look up its Id:
100
+
101
+ ```bash
102
+ sf data query \
103
+ --query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' AND Name = '<stageName>'" \
104
+ --target-org <doce-org-alias> --json
105
+ ```
106
+
107
+ Only if no stage is mentioned, fetch the full list and ask which one:
108
+
109
+ ```bash
110
+ sf data query \
111
+ --query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' ORDER BY Name ASC" \
112
+ --target-org <doce-org-alias> --json
113
+ ```
114
+
115
+ Then ask: "Which pipeline stage are we working with?" — do NOT ask for an org alias; stages are resolved by name from the pipeline.
116
+
117
+ - **Pass:** stage Id and Name confirmed
118
+ - **Deferred:** not required until a calling skill needs it
119
+
120
+ ---
121
+
122
+ ## Error Handling
123
+
124
+ | Error condition | Response |
125
+ |---|---|
126
+ | `sf org list` fails entirely | "Could not reach the Salesforce CLI. Make sure `sf` is installed and on your PATH." |
127
+ | `sf org display` returns auth error | Surface plain-language re-auth instructions. Do not expose the raw error. |
128
+ | `DevopsPipeline` query fails with 5xx | "The DevOps Center org is returning a server error. Try again in a few minutes." |
129
+ | Any check throws unexpectedly | "Something went wrong checking prerequisites. Error: [plain summary]. Let's try again — or resolve it manually and let me know when ready." |
130
+
131
+ Never expose raw API errors, stack traces, or JSON error payloads to the user.
132
+
133
+ ## Gotchas
134
+
135
+ | Issue | Resolution |
136
+ |---|---|
137
+ | `DevopsPipeline` query returns empty | DevOps Center not installed on the target org, or wrong org alias — ask the user to verify |
138
+ | `DevopsWorkItem` sObject not supported | Use `WorkItem` (no namespace) — the correct API name for this org version |
139
+ | Review trigger is pipeline-level, not stage-level | Query `DevopsPipelineStageTrigger` where `TriggerType = 'Review'` and `RelatedRecordId = <pipelineId>` |
140
+ | Connect API `testSuites` returns empty with `?stageId=` | Use `?triggerId=<reviewTriggerId>` — `stageId` only works for stage-level triggers |
141
+ | `sf plugins` doesn't match `agentforce` | The installed plugin is `@salesforce/plugin-agent` — match on `plugin-agent` too |
@@ -0,0 +1,120 @@
1
+ ---
2
+ name: configuring-quality-gate
3
+ description: "Creates a DevOps Center quality gate with associated rules (PASS_PERCENTAGE, SEVERITY, ESSENTIAL) and links it to a pipeline-stage suite assignment, after showing a mandatory impact preview and obtaining explicit confirmation. Use this skill when a user wants to set or configure a quality gate, change a coverage threshold, or establish testing benchmarks on a pipeline stage. TRIGGER when: the user wants to set/configure a quality gate, change a coverage threshold, or set testing benchmarks on a stage. DO NOT TRIGGER when: only re-running an existing gate (use running-devops-test-suite in retrigger mode)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ ## Prerequisites
10
+
11
+ Load `checking-devops-prerequisites` first — Prerequisites 1–4 AND Prerequisite 5 (stage). You need `doce-org-alias`, `pipelineId`, `stageId`, and the target `DevopsTestSuiteStage` record Id.
12
+
13
+ ## Inputs required
14
+
15
+ | Input | Source |
16
+ |---|---|
17
+ | `name` | User-provided name for the quality gate |
18
+ | `rules` | List of `{type, threshold?}` — see rule types below |
19
+ | `doce-org-alias` | Established in Prereq 1 |
20
+ | `testSuiteStageId` | Target `DevopsTestSuiteStage` record Id (Prereq 5) |
21
+
22
+ ## Rule types
23
+
24
+ | Type | Description | Threshold |
25
+ |---|---|---|
26
+ | `PASS_PERCENTAGE` | Minimum % of tests that must pass | Required (0–100) |
27
+ | `SEVERITY` | Maximum allowed severity level of failures | Required (numeric, e.g. 1–5) |
28
+ | `ESSENTIAL` | All essential tests must pass | Not required |
29
+
30
+ ---
31
+
32
+ ## MANDATORY IMPACT PREVIEW
33
+
34
+ **Before executing any commands**, show the user this preview and wait for explicit confirmation:
35
+
36
+ > "Here's what this quality gate will enforce on `<stageName>`:
37
+ > - Rule: `<type>` — `<description>`
38
+ > - Threshold: `<value>`
39
+ > - Affected pipelines: `<list>`
40
+ >
41
+ > Confirm to apply?"
42
+
43
+ **Never proceed past this point without showing the impact preview first and receiving explicit confirmation.**
44
+
45
+ ---
46
+
47
+ ## Confirmation gate
48
+
49
+ Only proceed with the steps below after the user has explicitly confirmed (e.g., "yes", "confirm", "go ahead"). If the user declines or does not confirm, stop and do not execute any commands.
50
+
51
+ ---
52
+
53
+ ## Step 1 — Create the gate record
54
+
55
+ ```bash
56
+ sf api request rest \
57
+ "/services/data/v67.0/connect/devopstesting/qualityGate" \
58
+ --method POST \
59
+ --body '{"name": "<gateName>"}' \
60
+ --target-org <doce-org-alias>
61
+ ```
62
+
63
+ Extract `qualityGateId` from the response.
64
+
65
+ ## Step 2 — Create each rule as a sObject record
66
+
67
+ Rules are not accepted in the Connect API payload — create them as `DevopsQualityGateRule` records directly. Only create rules the user has requested. `ESSENTIAL` has no threshold.
68
+
69
+ ```bash
70
+ sf data create record \
71
+ --sobject DevopsQualityGateRule \
72
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='PASS_PERCENTAGE' Threshold=<value>" \
73
+ --target-org <doce-org-alias> --json
74
+
75
+ sf data create record \
76
+ --sobject DevopsQualityGateRule \
77
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='ESSENTIAL'" \
78
+ --target-org <doce-org-alias> --json
79
+
80
+ sf data create record \
81
+ --sobject DevopsQualityGateRule \
82
+ --values "DevopsQualityGateId='<qualityGateId>' Rule='SEVERITY' Threshold=<value>" \
83
+ --target-org <doce-org-alias> --json
84
+ ```
85
+
86
+ ## Step 3 — Link the gate to the suite stage
87
+
88
+ Update the `DevopsTestSuiteStage` record to link the new gate:
89
+
90
+ ```bash
91
+ sf data update record \
92
+ --sobject DevopsTestSuiteStage \
93
+ --record-id <testSuiteStageId> \
94
+ --values "IsQualityGateEnabled=true DevopsQualityGateId='<qualityGateId>'" \
95
+ --target-org <doce-org-alias> --json
96
+ ```
97
+
98
+ ---
99
+
100
+ ## On success
101
+
102
+ Confirm to the user:
103
+ > "Quality gate `<gateName>` created with `<N>` rule(s) and assigned to `<suiteName>` on `<stageName>`."
104
+
105
+ ## Error handling
106
+
107
+ Never expose raw API errors. Use the following responses:
108
+
109
+ | Status | Response |
110
+ |---|---|
111
+ | 400 | "The quality gate configuration is invalid. Check that all rule types and thresholds are correct." |
112
+ | 403 | "You don't have permission to configure quality gates on this org." |
113
+ | 500 | "A server error occurred. Try again in a few minutes." |
114
+
115
+ ---
116
+
117
+ ## Related skills
118
+
119
+ - To re-run a gate after meeting threshold, use `running-devops-test-suite` (retrigger mode).
120
+ - To assign or map suites to stages, use `managing-suite-assignments`.
@@ -0,0 +1,113 @@
1
+ ---
2
+ name: configuring-test-provider
3
+ description: "Configures an available test provider (e.g. Apex Unit Tests, Code Analyzer, Flow Tests, Provar) on a DevOps Center pipeline via the Connect API, after explicit user confirmation, so the provider's test suites become available for assignment to pipeline stages. Takes a pipelineId and a testProviderId. Use this skill when a provider is available on the pipeline but not yet configured and the user wants to enable it. TRIGGER when: the user wants to configure, enable, set up, or add a test provider on a pipeline, or wants a not-yet-configured provider's suites to become available. DO NOT TRIGGER when: re-syncing an already-configured provider to pick up new suites (use syncing-test-providers), or assigning existing suites to a stage (use managing-suite-assignments)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Configuring a Test Provider
10
+
11
+ Configures an available test provider on a DevOps Center pipeline, making its test suites available for assignment to pipeline stages.
12
+
13
+ **Confirmation required:** Yes — explicit confirmation before the provider is configured.
14
+
15
+ ## Prerequisites
16
+
17
+ Load `checking-devops-prerequisites` first — Prerequisites 1–4 (org login, Agentforce DX plugin, DevOps Center org auth, pipeline identified). Prerequisite 5 (stage) is **not** required: providers are configured at the pipeline level, not the stage level.
18
+
19
+ | Variable | Source |
20
+ |---|---|
21
+ | `doce-org-alias` | Established in Prerequisite 1 |
22
+ | `pipelineId` | Identified in Prerequisite 4 (pipeline selection) |
23
+ | `testProviderId` | Resolved by fetching the pipeline's test providers (below) |
24
+
25
+ ---
26
+
27
+ ## Step 1 — Fetch test providers to resolve the provider ID
28
+
29
+ Get all test providers for the pipeline so you can resolve the `testProviderId` and confirm which providers are still available to configure:
30
+
31
+ ```bash
32
+ sf api request rest \
33
+ "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testProviders?status=all" \
34
+ --target-org <doce-org-alias>
35
+ ```
36
+
37
+ Each provider entry includes `testProviderId`, `testProviderName`, and a status (Configured vs. Available). Present a short summary grouped by status:
38
+
39
+ ```text
40
+ Test providers for <pipelineName>:
41
+
42
+ ✓ Configured:
43
+ - Code Analyzer (63 suites)
44
+ - Apex Unit Tests (5 suites)
45
+
46
+ Available (not yet configured):
47
+ - Flow Tests
48
+ ```
49
+
50
+ - **Only an Available provider can be configured.** If the pipeline has no available providers, report that and stop — do NOT fabricate a provider or ID.
51
+
52
+ ### If the named provider is already Configured
53
+
54
+ Do **not** present the confirmation gate and do **not** POST to the configure endpoint (Steps 2–3) — that would create a duplicate `DevopsPipelineTestProvider`. Instead:
55
+
56
+ 1. State plainly that the provider is already configured, including its synced suite count and last-sync time if returned (e.g. *"Flow Tests is already configured on `<pipelineName>` with 3 suites synced (last sync 2026-06-23)."*).
57
+ 2. Diagnose the user's actual goal and redirect **by name**:
58
+ - If the user says the provider's **suites don't appear when assigning tests to a stage**, this is a **stage-assignment gap, not a provider-configuration gap** — the suites already exist at the pipeline level; they just need to be linked to the stage. Redirect to **`managing-suite-assignments`**.
59
+ - If the user expects **newly created suites** that aren't yet synced, redirect to **`syncing-test-providers`** (re-sync via `POST /connect/devops/sync`) to pull them in.
60
+ 3. Do not loop back to configuring — finish cleanly after the explanation and redirect.
61
+
62
+ ## Step 2 — Confirmation gate
63
+
64
+ **Required — do not call the API before the user confirms.**
65
+
66
+ > "I'll configure `<testProviderName>` on the `<pipelineName>` pipeline. This will make its suites available for assignment to stages. Confirm?"
67
+
68
+ Do not proceed until the user gives an affirmative response.
69
+
70
+ ## Step 3 — Configure the provider
71
+
72
+ On confirmation, call the configure endpoint with the provider ID:
73
+
74
+ ```bash
75
+ sf api request rest \
76
+ "/services/data/v67.0/connect/devops/pipeline/<pipelineId>/testProvider" \
77
+ --method POST \
78
+ --body '{"testProviderId": "<testProviderId>"}' \
79
+ --target-org <doce-org-alias>
80
+ ```
81
+
82
+ ## On success
83
+
84
+ > "Provider `<testProviderName>` is now configured on the `<pipelineName>` pipeline. Its suites are available for assignment to stages."
85
+
86
+ Newly configured suites can then be assigned to stages with `managing-suite-assignments`.
87
+
88
+ ---
89
+
90
+ ## Critical gotcha
91
+
92
+ This `POST .../pipeline/<pipelineId>/testProvider` endpoint **creates a new provider configuration record** (`DevopsPipelineTestProvider`). Use it ONLY to configure a provider for the first time. To re-sync an already-configured provider for new suites, use `syncing-test-providers` (`POST /connect/devops/sync`) — calling this configure endpoint on an already-configured provider produces **duplicate** `DevopsPipelineTestProvider` records.
93
+
94
+ ## Error Handling
95
+
96
+ Never expose raw API error messages, stack traces, or JSON payloads to the user. Map response status codes to plain-language messages:
97
+
98
+ | Status | User-facing message |
99
+ |---|---|
100
+ | 400 | "The request was invalid. Check that the provider ID and pipeline ID are correct." |
101
+ | 403 | "You don't have permission to configure test providers on this pipeline." |
102
+ | 404 | "The pipeline or test provider was not found." |
103
+ | 409 | "That provider appears to already be configured on this pipeline. To pick up new suites, re-sync it instead." |
104
+ | 500 | "A server error occurred. Try again in a few minutes." |
105
+
106
+ ---
107
+
108
+ ## Related skills
109
+
110
+ - **`checking-devops-prerequisites`** — loaded first to establish org and pipeline context.
111
+ - **`syncing-test-providers`** — once a provider is configured, use this to re-sync it later and pull in newly added suites.
112
+ - **`managing-suite-assignments`** — after configuring a provider, use this to assign or map its suites to a pipeline stage.
113
+ - **`recommending-devops-tests`** — to recommend which of the newly available suites to run.
@@ -0,0 +1,66 @@
1
+ ---
2
+ name: creating-fix-work-item
3
+ description: "Creates a DevOps Center WorkItem to track a fix for a test failure or Code Analyzer violation. Before creating anything, it shows a subject/assignee/project preview and requires explicit confirmation. Use this skill when you want to create a fix work item, track a remediation task, or assign a test or analysis failure to a developer. TRIGGER when: the user wants to create a fix work item, log a remediation, or assign a failure to a developer for resolution. DO NOT TRIGGER when: writing the fix code itself (use generating-apex)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ ## Prerequisites
10
+
11
+ Load `checking-devops-prerequisites` first — Prerequisites 1–4. You need `doce-org-alias`, the `DevopsProjectId` to file under, and an `OwnerId` (assignee). If no DevopsProject exists, surface that the work item cannot be created until a project exists.
12
+
13
+ ## Inputs required before creating
14
+
15
+ | Input | How to obtain |
16
+ |---|---|
17
+ | `DevopsProjectId` | From the pipeline's associated project — query `DevopsProject WHERE Name = '<projectName>'` on the doce org if not already known |
18
+ | `Subject` | Derived from failure analysis — e.g. "Fix: Missing code-analyzer-v5.yml workflow in blitz-10-06 repository" |
19
+ | `OwnerId` | User ID of the developer to assign to — query `SELECT Id, Name FROM User WHERE Username = '<username>'` on the doce org if not known. Ask the user if the username is unknown. |
20
+ | `doce-org-alias` | Established in Prerequisites |
21
+
22
+ ## Confirmation gate
23
+
24
+ Before creating the work item, show a summary and wait for explicit confirmation:
25
+
26
+ > "I'll create a fix work item with the following details:
27
+ > - **Subject:** `<subject>`
28
+ > - **Assigned to:** `<assigneeName>`
29
+ > - **Project:** `<projectName>`
30
+ >
31
+ > Shall I create it?"
32
+
33
+ Do not proceed until the user confirms.
34
+
35
+ ## Creating the work item
36
+
37
+ ```bash
38
+ sf data create record \
39
+ --sobject WorkItem \
40
+ --values "Subject='<subject>' DevopsProjectId='<DevopsProjectId>' OwnerId='<OwnerId>'" \
41
+ --target-org <doce-org-alias> \
42
+ --json
43
+ ```
44
+
45
+ > **Important:** Use `WorkItem` (no namespace) — `DevopsWorkItem` is not a supported sObject in this org version.
46
+
47
+ ## On success
48
+
49
+ Parse the returned `id` from the result and confirm:
50
+
51
+ > "Fix work item created (`<id>`): `<subject>`. Assigned to `<assigneeName>` in the `<projectName>` project."
52
+
53
+ ## Error handling
54
+
55
+ Never expose raw API error messages. Map errors to plain-language responses:
56
+
57
+ | Error | Response |
58
+ |---|---|
59
+ | `FIELD_INTEGRITY_EXCEPTION` | "The assignee ID is invalid. Let me look up the correct user ID — what's the developer's username?" |
60
+ | `REQUIRED_FIELD_MISSING` | "A required field is missing. Check that `Subject` and `DevopsProjectId` are both provided." |
61
+ | `INSUFFICIENT_ACCESS` | "Your user doesn't have permission to create work items in this project." |
62
+ | Any other error | "The work item could not be created. Error: `<plain summary>`. Try again or create it manually in DevOps Center." |
63
+
64
+ ## Related skills
65
+
66
+ - `analyzing-test-failures` — provides the failure analysis and improvement suggestions that motivate creating a fix work item
@@ -0,0 +1,161 @@
1
+ ---
2
+ name: managing-suite-assignments
3
+ description: "Manages all forms of test suite assignment for DevOps Center pipeline stages via the Connect API testSuiteStages endpoint. Covers three modes: (A) attaching a single suite to a stage as a one-off add, (B) bulk-mapping multiple suites to a stage as a testing strategy with a mandatory impact-preview table before any changes, and (C) adding or removing individual test classes within a suite assignment with governance rules that exclude rejected tests and re-present the final list before committing. TRIGGER when: a suite is found unlinked from a pipeline stage and the user wants to assign it; the user wants to configure suite-to-stage mappings across a pipeline or assign multiple suites to stages as part of a testing strategy; the user wants to add or remove tests in a suite, sync reviewed tests into a suite, or promote tests to a suite. DO NOT TRIGGER when: authoring new test classes (use generating-apex-test) or running tests directly (use running-apex-tests)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Managing Suite Assignments
10
+
11
+ ## Prerequisites
12
+
13
+ Load `checking-devops-prerequisites` first — Prerequisites 1–4 AND Prerequisite 5 (stage). You need `doce-org-alias`, `pipelineId`, and `stageId` before proceeding in any mode.
14
+
15
+ | Variable | Description |
16
+ |---|---|
17
+ | `doce-org-alias` | Established in Prerequisite 1 |
18
+ | `pipelineId` | Identified in Prerequisite 4 (pipeline selection) |
19
+ | `stageId` | Identified in Prerequisite 5 (stage selection) |
20
+ | `event` | `Pre-Promote`, `Post-Promote`, or `Review` |
21
+
22
+ ---
23
+
24
+ ## Mode A — Assign a single suite to a stage
25
+
26
+ Use this mode when a relevant test suite already exists but is not yet linked to a pipeline stage and the user wants to add it as a single one-off assignment.
27
+
28
+ **Additional inputs required:**
29
+
30
+ | Variable | Description |
31
+ |---|---|
32
+ | `testSuiteId` | ID of the suite to assign |
33
+ | `testSuiteName` | Name of the suite (for display in confirmation) |
34
+
35
+ **Confirmation gate**
36
+
37
+ **Confirmation required before any API call is made.**
38
+
39
+ Present the following prompt to the user and wait for an affirmative response:
40
+
41
+ > "The suite `<testSuiteName>` is not currently assigned to the `<stageName>` stage (`<event>`). Would you like me to assign it now?"
42
+
43
+ Do not proceed until the user confirms.
44
+
45
+ **On success**
46
+
47
+ Report to the user:
48
+
49
+ > "Suite `<testSuiteName>` has been assigned to the `<stageName>` stage (`<event>`)."
50
+
51
+ ---
52
+
53
+ ## Mode B — Map multiple suites to a stage
54
+
55
+ Use this mode when configuring suite-to-stage mappings across a pipeline or assigning multiple suites to stages as part of a testing strategy.
56
+
57
+ **Additional inputs required:**
58
+
59
+ | Input | Description |
60
+ |---|---|
61
+ | `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
62
+
63
+ **MANDATORY IMPACT PREVIEW — Required Before Any Changes**
64
+
65
+ **Do not call the API until the user has confirmed the preview below.**
66
+
67
+ Present the full mapping summary and wait for explicit confirmation:
68
+
69
+ > "Here's the suite mapping I'll apply:
70
+ >
71
+ > | Suite | Stage | Event | Action |
72
+ > |---|---|---|---|
73
+ > | `<suiteName>` | `<stageName>` | `<event>` | Add |
74
+ > | `<suiteName2>` | `<stageName>` | `<event>` | Remove |
75
+ >
76
+ > Confirm to apply all changes?"
77
+
78
+ Only proceed after the user explicitly confirms.
79
+
80
+ **On success**
81
+
82
+ > "Suite mapping applied. `<N>` suite(s) updated for the `<stageName>` stage."
83
+
84
+ ---
85
+
86
+ ## Mode C — Add/remove test classes in a suite
87
+
88
+ Use this mode when the user wants to add or remove individual test classes within an existing suite assignment, sync reviewed tests into a suite, or promote tests to a suite.
89
+
90
+ **Additional inputs required:**
91
+
92
+ | Input | Source |
93
+ |---|---|
94
+ | `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
95
+
96
+ **Governance rules**
97
+
98
+ - **Never call this without explicit approval.** AI-reviewed or modified tests must be re-presented to the user before this call is made.
99
+ - Rejected tests must be EXCLUDED from the payload — do not include them even if they were previously in the suite.
100
+ - If tests were modified during review, re-present the final list of tests before requesting confirmation.
101
+
102
+ **Confirmation gate**
103
+
104
+ **REQUIRED — do not skip.** Show the user the full list of changes before calling:
105
+
106
+ > "I'm about to sync the following changes to the test suite:
107
+ > - **Add:** `<testSuiteName1>`, `<testSuiteName2>`
108
+ > - **Remove:** `<testSuiteName3>`
109
+ > - Stage: `<stageName>` | Event: `<event>`
110
+ >
111
+ > Confirm?"
112
+
113
+ Only proceed after receiving explicit confirmation.
114
+
115
+ **On success**
116
+
117
+ Confirm to the user:
118
+ > "Test suite updated successfully for the `<stageName>` stage."
119
+
120
+ ---
121
+
122
+ ## The system call
123
+
124
+ All three modes use the same endpoint. Substitute all `<placeholder>` values before executing.
125
+
126
+ ```bash
127
+ sf api request rest "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testSuiteStages" --method POST --body '{"pipelineStageId":"<stageId>","event":"<event>","assignments":[{"testSuiteId":"<id>","action":"add|remove"}]}' --target-org <doce-org-alias>
128
+ ```
129
+
130
+ Full payload schema:
131
+
132
+ ```json
133
+ {
134
+ "pipelineStageId": "<stageId>",
135
+ "event": "<event>",
136
+ "assignments": [
137
+ {"testSuiteId": "<suiteId1>", "action": "add"},
138
+ {"testSuiteId": "<suiteId2>", "action": "remove"}
139
+ ]
140
+ }
141
+ ```
142
+
143
+ **Error handling**
144
+
145
+ Never expose raw API error messages to the user. Map response status codes to the following user-facing messages:
146
+
147
+ | Status | User-facing message |
148
+ |---|---|
149
+ | 400 | "The request was invalid. Check that all suite and stage IDs are correct and the event type is valid." |
150
+ | 403 | "You don't have permission to modify suite assignments on this pipeline." |
151
+ | 404 | "The pipeline or stage was not found." |
152
+ | 500 | "A server error occurred. Try again in a few minutes." |
153
+
154
+ ---
155
+
156
+ ## Related skills
157
+
158
+ - **`syncing-test-providers`** — use when the suite you want to assign doesn't appear yet; re-syncing the provider pulls in newly added suites.
159
+ - **`recommending-devops-tests`** — use when a suite has not yet been identified and you need a recommendation for which suite to assign.
160
+ - **`configuring-quality-gate`** — use to configure a quality gate on the stage after mapping suites.
161
+ - **`analyzing-test-failures`** — use for failure analysis and suggested improvements to the tests within a suite.
@@ -0,0 +1,72 @@
1
+ ---
2
+ name: polling-test-results
3
+ description: "Polls a DevOps Center async test execution by runId until it completes, fails, or times out — using read-only SOQL on the execution record at provider-specific intervals (Apex 15s/5m, Code Analyzer 10s/3m, Provar UI 60s/20m, Flow 20s/8m) — then surfaces results. Use this skill when a test suite run is in progress and the user is waiting on results, or immediately after running a suite. TRIGGER when: a test suite run is in progress and the user is waiting on results, or immediately after running a suite and a runId is available. DO NOT TRIGGER when: there is no active runId."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # polling-test-results
10
+
11
+ **Confirmation required:** No — polling is automatic and read-only. No user confirmation gate is needed.
12
+
13
+ **What it does:** Polls the DevOps Center org for the status of an async test execution until it completes, times out, or fails.
14
+
15
+ ---
16
+
17
+ ## Prerequisites
18
+
19
+ You need an active `runId` (returned by the `running-devops-test-suite` skill) and the confirmed `doce-org-alias`. If org context is not yet established, load `checking-devops-prerequisites` first (Prerequisites 1–3).
20
+
21
+ ---
22
+
23
+ ## Inputs required
24
+
25
+ | Input | Source |
26
+ |---|---|
27
+ | `runId` | Returned by the `running-devops-test-suite` skill |
28
+ | `testType` | Derived from the suite's test provider (Apex, Code Analyzer, UI/Provar, Flow) |
29
+ | `doce-org-alias` | Established via `checking-devops-prerequisites` |
30
+
31
+ ## Polling configuration
32
+
33
+ | Test type | Poll interval | Max wait | Timeout action |
34
+ |---|---|---|---|
35
+ | Apex unit tests | 15 seconds | 5 minutes | Surface runId, offer retry |
36
+ | Code Analyzer | 10 seconds | 3 minutes | Surface runId, offer retry |
37
+ | UI tests (Provar) | 60 seconds | 20 minutes | Surface runId, mark as pending |
38
+ | Flow tests | 20 seconds | 8 minutes | Surface runId, offer retry |
39
+
40
+ ## Poll query
41
+
42
+ Query the execution record by `runId` on each interval:
43
+
44
+ ```bash
45
+ sf data query \
46
+ --query "SELECT Id, Status, TestsRan, TestsPassed, TestsFailed, CoveragePercentage FROM DevopsTestExecution WHERE Id = '<runId>' LIMIT 1" \
47
+ --target-org <doce-org-alias> \
48
+ --json
49
+ ```
50
+
51
+ Check the `Status` field on each poll:
52
+ - `Queued` / `Running` → wait and poll again
53
+ - `Completed` → proceed to result analysis
54
+ - `Failed` → surface error and offer retry or skip
55
+
56
+ ## On timeout
57
+
58
+ Surface the `runId` to the user:
59
+ > "The test run is taking longer than expected. Your run ID is `<runId>`. You can check the status manually in DevOps Center, or I can keep waiting — what would you prefer?"
60
+
61
+ Do not automatically retry after timeout. Wait for user instruction.
62
+
63
+ ## On completion
64
+
65
+ Pass the full result payload to the `analyzing-test-failures` skill for reasoning. Surface `Coverage`, `SuccessCount`, `FailureCount`, and `QualityGateStatus` inline. Do not surface raw JSON to the user.
66
+
67
+ ---
68
+
69
+ ## Related skills
70
+
71
+ - Runs are started by `running-devops-test-suite` (including its retrigger mode)
72
+ - Failure analysis is handled by `analyzing-test-failures`
@@ -0,0 +1,137 @@
1
+ ---
2
+ name: recommending-devops-tests
3
+ description: "Analyzes a commit diff and available DevOps Center test suite metadata to recommend the most relevant existing test suites, then flags coverage gaps where no suite covers a new method — via pure reasoning, no system calls beyond the prerequisite queries. Use this skill when a developer wants to know which test suites to run for a specific commit or diff, what tests cover their changes, or wants suite recommendations at commit time. TRIGGER when: the user asks which test suites to run for a commit/diff, asks what tests cover their changes, or asks for suite recommendations before promoting. DO NOT TRIGGER when: authoring new tests (use generating-apex-test) or running sf apex run test directly (use running-apex-tests)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Recommending DevOps Tests
10
+
11
+ **Type:** Pure reasoning — no system calls beyond the prerequisite data-fetch queries documented below.
12
+
13
+ **What it does:** Analyzes the commit diff and available suite metadata to recommend the most relevant existing test suites assigned to the Review pipeline stage. Flags coverage gaps where no suite covers a new method.
14
+
15
+ ---
16
+
17
+ ## Prerequisites
18
+
19
+ Load and follow the `checking-devops-prerequisites` skill first (Prerequisites 1–4). This skill needs a confirmed DevOps Center org alias and pipeline Id before it can proceed.
20
+
21
+ ---
22
+
23
+ ## Step 1 — Fetch suite metadata
24
+
25
+ Before reasoning can begin, fetch the test suite metadata assigned to the Review pipeline stage. This requires two queries.
26
+
27
+ **1a — Find the Review pipeline stage trigger:**
28
+
29
+ ```bash
30
+ sf data query \
31
+ --query "SELECT Id FROM DevopsPipelineStageTrigger WHERE TriggerType = 'Review' AND RelatedRecordId = '<pipelineId>'" \
32
+ --target-org <doce-org-alias> \
33
+ --json
34
+ ```
35
+
36
+ Record the `Id` returned as `<reviewTriggerId>`.
37
+
38
+ **1b — Fetch suites assigned to that trigger:**
39
+
40
+ ```bash
41
+ sf data query \
42
+ --query "SELECT Id, TestSuiteId, TestSuite.Name, IsQualityGateEnabled, DevopsQualityGateId FROM DevopsTestSuiteStage WHERE DevopsPipelineStageTriggerId = '<reviewTriggerId>'" \
43
+ --target-org <doce-org-alias> \
44
+ --json
45
+ ```
46
+
47
+ Each row provides: `TestSuiteId`, `TestSuite.Name`, `IsQualityGateEnabled`, and `DevopsQualityGateId` (when a gate is configured).
48
+
49
+ > Note: Do NOT use the beta `/connect/devops/.../testSuites` endpoint — it returns empty results. Query `DevopsTestSuiteStage` directly as shown above.
50
+
51
+ Never expose raw API errors or raw JSON to the user. If a query fails, report the problem in plain language and stop.
52
+
53
+ The commit diff comes from the user or surrounding context.
54
+
55
+ ---
56
+
57
+ ## Reasoning steps
58
+
59
+ ### 1 — Classify the diff by change type
60
+
61
+ Parse the diff file extensions and paths to determine what types of changes were made:
62
+
63
+ | File pattern | Change type |
64
+ |---|---|
65
+ | `*.cls`, `*.trigger` | Apex |
66
+ | `*.flow-meta.xml`, `*.flow` | Flow |
67
+ | `*.java` | Java |
68
+ | `*.js`, `*.html` (LWC paths) | LWC / JavaScript |
69
+
70
+ A single commit may contain multiple change types. Identify all of them.
71
+
72
+ ### 2 — Map change types to test providers
73
+
74
+ For each change type identified, match against the `testProviderName` of the fetched suites:
75
+
76
+ | Change type | Match suites whose `testProviderName` contains |
77
+ |---|---|
78
+ | Apex | `Apex` |
79
+ | Flow | `Flow` |
80
+ | Java | `Java` |
81
+ | LWC / JavaScript | `LWC` or `JavaScript` |
82
+ | Code Analyzer (any) | `Code Analyzer` — then sub-filter by suite name convention: `recommended` → Apex/general rules (suggest for Apex and Java changes), `html` → suggest for HTML/LWC template changes, `css` → suggest for CSS changes. If no suite name matches the convention, suggest all Code Analyzer suites and note the ambiguity. |
83
+
84
+ Only recommend suites whose provider matches at least one change type in the diff. Suites from non-matching providers are excluded from the recommendation.
85
+
86
+ ### 3 — Rank matched suites
87
+
88
+ Within each matched provider group, rank suites by:
89
+ 1. `triggerType` alignment — `Pre-Promote` suites first for commit-time checks
90
+ 2. Quality gate presence — suites with a `qualityGateRuleName` are higher priority (they gate promotion)
91
+ 3. Suite name relevance — if the suite name contains a term from the modified file names, boost it
92
+
93
+ ### 4 — Flag new methods with no suite coverage
94
+
95
+ For any new method added in the diff, flag it as a gap regardless of provider matching:
96
+ > "⚠ `processRefund()` in `OrderService.cls` is a new method. No existing suite can confirm it's covered until tests are authored and linked."
97
+
98
+ ### 5 — Return the recommendation list
99
+
100
+ Group recommendations by change type so it's clear why each suite was suggested.
101
+
102
+ ---
103
+
104
+ ## Output format
105
+
106
+ ```text
107
+ Recommended suite(s) for this commit:
108
+
109
+ Apex changes detected (OrderService.cls, InvoiceService.cls):
110
+ 1. <SuiteName> — Apex Test Provider | Trigger: Pre-Promote | Gate: <gateName or none>
111
+ 2. <SuiteName2> — Apex Test Provider | Trigger: Post-Promote | Gate: <gateName or none>
112
+
113
+ Flow changes detected (OrderApproval.flow):
114
+ 1. <SuiteName3> — Flow Test Provider | Trigger: Pre-Promote | Gate: <gateName or none>
115
+
116
+ Code Analyzer — Apex rules:
117
+ 1. <SuiteName4> — Code Analyzer | Trigger: Pre-Promote | Gate: <gateName or none>
118
+
119
+ No matching suite found for:
120
+ - LWC changes — no LWC test suite is assigned to this stage
121
+
122
+ Coverage gaps (new methods — manual authoring required):
123
+ - `processRefund()` in `OrderService.cls` — new method, not yet covered by any suite
124
+ ```
125
+
126
+ ---
127
+
128
+ ## v1 constraint
129
+
130
+ Only recommend existing suites. Never suggest generating new tests. If gaps exist, direct the developer to author tests manually and explain exactly which methods need coverage.
131
+
132
+ ---
133
+
134
+ ## Related skills
135
+
136
+ - To analyze failures from a run, use `analyzing-test-failures`.
137
+ - To actually execute a recommended suite, use `running-devops-test-suite`.
@@ -0,0 +1,144 @@
1
+ ---
2
+ name: running-devops-test-suite
3
+ description: "Triggers async execution of one or more DevOps Center test suites on a pipeline stage (Pre-Promote, Post-Promote, or Review event) via the Connect API, after an explicit user confirmation gate, then hands off to result polling via the `polling-test-results` skill. Also re-runs (retriggers) a quality gate after fixes — but only once validation confirms the coverage threshold is now met. TRIGGER when: the user wants to run, kick off, or launch test suites on a pipeline stage; execute tests before or after a promotion; trigger a Pre-Promote, Post-Promote, or Review-event run; re-run a quality gate after fixing failures; retry a failed gate once coverage is met; or unblock a blocked promotion after adding tests. DO NOT TRIGGER when: running sf apex run test directly (use running-apex-tests); or configuring a NEW gate or threshold (use configuring-quality-gate)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Running a DevOps Center Test Suite
10
+
11
+ ## Prerequisites
12
+
13
+ Load and follow `checking-devops-prerequisites` first — run Prerequisites 1–4 AND Prerequisite 5 (pipeline stage), since this skill operates on a specific stage. You need the confirmed `doce-org-alias`, `pipelineId`, and `stageId` before proceeding.
14
+
15
+ ## Inputs required before calling this skill
16
+
17
+ | Input | How to obtain |
18
+ |---|---|
19
+ | `pipelineId` | From Prerequisite 4 (pipeline selection) |
20
+ | `stageId` | From Prerequisite 5 (pipeline stage confirmation) |
21
+ | `event` | Confirm with user: `Pre-Promote` or `Post-Promote` (or `Review` if the context is a review environment) |
22
+ | `testSuiteIds` | Confirmed suite IDs from the suite selection or recommendation step |
23
+ | `doce-org-alias` | Established in Prerequisite 1 |
24
+
25
+ ## Confirmation gate
26
+
27
+ **This call mutates org state — do not proceed without explicit user confirmation.**
28
+
29
+ Before calling the API, show the user:
30
+
31
+ > "I'm about to run tests with the following configuration:
32
+ > - Pipeline: `<pipelineName>`
33
+ > - Stage: `<stageName>`
34
+ > - Event: `<event>`
35
+ > - Suite(s): `<suiteName(s)>`
36
+ > - Org: `<doce-org-alias>`
37
+ >
38
+ > Shall I proceed?"
39
+
40
+ Do not make the API call until the user confirms.
41
+
42
+ ## API call
43
+
44
+ ```bash
45
+ sf api request rest \
46
+ "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" \
47
+ --method POST \
48
+ --body '{
49
+ "stageId": "<stageId>",
50
+ "event": "<event>",
51
+ "testSuiteIds": ["<suiteId1>", "<suiteId2>"]
52
+ }' \
53
+ --target-org <doce-org-alias>
54
+ ```
55
+
56
+ ### Body schema
57
+
58
+ | Field | Type | Description |
59
+ |---|---|---|
60
+ | `stageId` | string | The ID of the pipeline stage to execute tests on |
61
+ | `event` | string | `Pre-Promote`, `Post-Promote`, or `Review` |
62
+ | `testSuiteIds` | string[] | One or more test suite IDs to execute |
63
+
64
+ ## On success
65
+
66
+ Extract the `runId` (or execution ID) from the response. Inform the user:
67
+
68
+ > "Tests are running in `<doce-org-alias>`. I'll update you when results are ready."
69
+
70
+ Immediately hand off the `runId` to the `polling-test-results` skill to begin the polling loop.
71
+
72
+ ## On error
73
+
74
+ | Status | Message to user |
75
+ |---|---|
76
+ | 400 | "The test execution request was invalid. Check that the stage and suite IDs are correct." |
77
+ | 403 | "You don't have permission to run tests on this pipeline. Check your DevOps Testing API access." |
78
+ | 404 | "The pipeline or stage was not found. It may have been deleted." |
79
+ | 500 | "The DevOps Center org returned a server error. Try again in a few minutes." |
80
+
81
+ Never expose raw API errors to the user.
82
+
83
+ ---
84
+
85
+ ## Retrigger mode (re-running a quality gate)
86
+
87
+ Use this mode when a promotion was blocked by a quality gate failure and the coverage gap has since been addressed.
88
+
89
+ ### Extra preconditions — all must be true before proceeding
90
+
91
+ 1. The `Coverage` field on the latest `DevopsTestSuiteExecution` meets or exceeds the threshold defined in the `DevopsQualityGateRule`
92
+ 2. The user has explicitly asked to retrigger the gate
93
+ 3. The same `pipelineId`, `stageId`, and `event` from the blocked promotion are known
94
+
95
+ If coverage is still below threshold, do **not** retrigger. Instead, respond:
96
+
97
+ > "Coverage is still at `<X>%`, below the `<threshold>%` gate. The gate cannot be retriggered until the threshold is met. Here are the remaining uncovered methods: `<list>`."
98
+
99
+ Do not retry. Explain what must be resolved first and stop.
100
+
101
+ ### Inputs required for retrigger
102
+
103
+ | Input | Source |
104
+ |---|---|
105
+ | `pipelineId` | From the blocked promotion context |
106
+ | `stageId` | From the blocked promotion context |
107
+ | `event` | Same event type that originally blocked (`Pre-Promote` or `Post-Promote`) |
108
+ | `suiteIds` | Same suites that were originally run |
109
+ | `doce-org-alias` | Established in Prerequisite 1 |
110
+
111
+ ### Confirmation gate (retrigger)
112
+
113
+ Before executing the API call, present this confirmation prompt and wait for explicit user approval:
114
+
115
+ > "Coverage is confirmed at `<X>%`, which meets the `<threshold>%` gate. I'll retrigger the quality gate check for the `<stageName>` stage (`<event>`). Confirm?"
116
+
117
+ Only proceed after the user confirms. If the user declines, stop without making any API call.
118
+
119
+ ### API call (retrigger)
120
+
121
+ Uses the same Connect API stage/execute endpoint:
122
+
123
+ ```bash
124
+ sf api request rest "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" --method POST --body '{"stageId":"<stageId>","event":"<event>","testSuiteIds":["<suiteId1>"]}' --target-org <doce-org-alias>
125
+ ```
126
+
127
+ After the call returns a `runId`, hand off to the `polling-test-results` skill with the new `runId` to monitor the execution result.
128
+
129
+ ### Error handling (retrigger)
130
+
131
+ If the API returns an error indicating the gate cannot be retriggered, respond with:
132
+
133
+ > "The quality gate cannot be retriggered right now. Reason: `<plain-language summary>`. Here's what needs to be resolved first: `<list>`."
134
+
135
+ Never expose raw API error details to the user.
136
+
137
+ ---
138
+
139
+ ## Related skills
140
+
141
+ - **`polling-test-results`** — poll for async run results after receiving the `runId`
142
+ - **`recommending-devops-tests`** — recommend which suites to run first before triggering execution
143
+ - **`managing-suite-assignments`** — assign or map a suite to a pipeline stage if it isn't linked yet
144
+ - **`configuring-quality-gate`** — configure a new gate or threshold
@@ -0,0 +1,108 @@
1
+ ---
2
+ name: syncing-test-providers
3
+ description: "Re-syncs a configured DevOps Center test provider on a pipeline to pick up new test suites, via the Connect API sync endpoint, after explicit user confirmation. Takes a pipelineId and one or more testProviderIds and triggers an asynchronous re-sync so newly added suites become available for assignment. Use this skill when a test provider (e.g. Apex Unit Tests, Code Analyzer, Flow Tests, Provar) has had suites added since it was last configured and the user wants those suites to show up in DevOps Center. TRIGGER when: the user wants to re-sync a test provider, pull in new suites from a provider, refresh a provider's suite list, or says assigned suites are missing/out of date for a configured provider. DO NOT TRIGGER when: configuring a provider for the first time (that creates a new DevopsPipelineTestProvider configuration), or assigning/mapping existing suites to a stage (use managing-suite-assignments)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Syncing Test Providers
10
+
11
+ Re-syncs a configured test provider on a DevOps Center pipeline so that suites added to the provider since it was last configured become available for assignment to stages.
12
+
13
+ **Confirmation required:** Yes — explicit confirmation before the sync is triggered.
14
+
15
+ ## Prerequisites
16
+
17
+ Load `checking-devops-prerequisites` first — Prerequisites 1–4 (org login, Agentforce DX plugin, DevOps Center org auth, pipeline identified). Prerequisite 5 (stage) is **not** required: providers are synced at the pipeline level, not the stage level.
18
+
19
+ | Variable | Source |
20
+ |---|---|
21
+ | `doce-org-alias` | Established in Prerequisite 1 |
22
+ | `pipelineId` | Identified in Prerequisite 4 (pipeline selection) |
23
+ | `testProviderId` | Resolved by fetching the pipeline's test providers (below) |
24
+
25
+ ---
26
+
27
+ ## Step 1 — Fetch test providers to resolve the provider ID
28
+
29
+ Get all test providers configured on the pipeline so you can resolve the `testProviderId` and confirm the provider name with the user:
30
+
31
+ ```bash
32
+ sf api request rest \
33
+ "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testProviders?status=all" \
34
+ --target-org <doce-org-alias>
35
+ ```
36
+
37
+ Each provider entry includes `testProviderId`, `testProviderName`, and a status (Configured vs. Available). Present a short summary grouped by status:
38
+
39
+ ```text
40
+ Test providers for <pipelineName>:
41
+
42
+ ✓ Configured:
43
+ - Code Analyzer (63 suites)
44
+ - Apex Unit Tests (5 suites)
45
+
46
+ Available (not yet configured):
47
+ - Flow Tests
48
+ ```
49
+
50
+ - **Only a Configured provider can be synced.** If the user names an *Available* (not-yet-configured) provider, explain it must be configured first — this skill does not configure providers.
51
+ - If the pipeline has no configured providers, report that and stop — do NOT fabricate a provider or ID.
52
+
53
+ ## Step 2 — Confirmation gate
54
+
55
+ **Required — do not call the API before the user confirms.**
56
+
57
+ > "I'll re-sync `<testProviderName>` on the `<pipelineName>` pipeline to pick up any new suites. Confirm?"
58
+
59
+ Do not proceed until the user gives an affirmative response.
60
+
61
+ ## Step 3 — Trigger the sync
62
+
63
+ On confirmation, call the sync endpoint with the provider ID(s) and pipeline ID:
64
+
65
+ ```bash
66
+ sf api request rest \
67
+ "/services/data/v67.0/connect/devops/sync" \
68
+ --method POST \
69
+ --body '{
70
+ "testProviderIds": ["<testProviderId>"],
71
+ "pipelineId": "<pipelineId>"
72
+ }' \
73
+ --target-org <doce-org-alias>
74
+ ```
75
+
76
+ `testProviderIds` is a list — multiple configured providers can be synced in one call.
77
+
78
+ ## On success
79
+
80
+ > "Provider `<testProviderName>` sync started. The operation is running asynchronously — new suites will be available shortly."
81
+
82
+ The sync runs asynchronously; newly synced suites can then be assigned to stages with `managing-suite-assignments`.
83
+
84
+ ---
85
+
86
+ ## Critical gotcha
87
+
88
+ **Do NOT use** `POST /connect/devops/pipeline/<pipelineId>/testProvider` to sync — that endpoint **creates a new provider configuration** and will result in duplicate `DevopsPipelineTestProvider` records. Sync only via `POST /connect/devops/sync`.
89
+
90
+ ## Error Handling
91
+
92
+ Never expose raw API error messages, stack traces, or JSON payloads to the user. Map response status codes to plain-language messages:
93
+
94
+ | Status | User-facing message |
95
+ |---|---|
96
+ | 400 | "The sync request was invalid. Check that the provider ID and pipeline ID are correct." |
97
+ | 403 | "You don't have permission to sync test providers on this pipeline." |
98
+ | 404 | "The pipeline or test provider was not found." |
99
+ | 500 | "A server error occurred. Try again in a few minutes." |
100
+
101
+ ---
102
+
103
+ ## Related skills
104
+
105
+ - **`checking-devops-prerequisites`** — loaded first to establish org and pipeline context.
106
+ - **`configuring-test-provider`** — use to configure an available provider for the first time; this skill only re-syncs already-configured providers.
107
+ - **`managing-suite-assignments`** — after a sync surfaces new suites, use this to assign or map them to a pipeline stage.
108
+ - **`recommending-devops-tests`** — to recommend which of the newly synced suites to run.