@salesforce/afv-skills 1.25.0 → 1.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/package.json +1 -1
- package/skills/dx-devops-test-failures-analyze/SKILL.md +89 -0
- package/skills/dx-devops-test-failures-analyze/references/code-analyzer-violations.md +26 -0
- package/skills/dx-devops-test-failures-analyze/references/failure-categories.md +85 -0
- package/skills/{checking-devops-prerequisites/SKILL.md → dx-devops-test-failures-analyze/references/prerequisite-checks.md} +8 -37
- package/skills/{creating-fix-work-item/SKILL.md → dx-devops-test-failures-analyze/references/work-item-creation.md} +8 -12
- package/skills/dx-devops-test-pipeline-configure/SKILL.md +72 -0
- package/skills/dx-devops-test-pipeline-configure/references/configuring-quality-gate.md +133 -0
- package/skills/dx-devops-test-pipeline-configure/references/configuring-test-provider.md +80 -0
- package/skills/dx-devops-test-pipeline-configure/references/error-handling.md +39 -0
- package/skills/dx-devops-test-pipeline-configure/references/gotchas.md +37 -0
- package/skills/dx-devops-test-pipeline-configure/references/prerequisite-checks.md +112 -0
- package/skills/dx-devops-test-pipeline-configure/references/syncing-test-providers.md +69 -0
- package/skills/dx-devops-test-suite-assignments-configure/SKILL.md +74 -0
- package/skills/dx-devops-test-suite-assignments-configure/references/api-endpoint.md +30 -0
- package/skills/dx-devops-test-suite-assignments-configure/references/error-handling.md +14 -0
- package/skills/dx-devops-test-suite-assignments-configure/references/prerequisite-checks.md +112 -0
- package/skills/{recommending-devops-tests/SKILL.md → dx-devops-test-suite-assignments-configure/references/recommendation-logic.md} +10 -26
- package/skills/dx-devops-test-suite-assignments-configure/references/suite-assignment-modes.md +99 -0
- package/skills/dx-devops-test-suite-run/SKILL.md +111 -0
- package/skills/dx-devops-test-suite-run/references/error-handling.md +31 -0
- package/skills/dx-devops-test-suite-run/references/polling-configuration.md +78 -0
- package/skills/dx-devops-test-suite-run/references/prerequisite-checks.md +112 -0
- package/skills/dx-devops-test-suite-run/references/retrigger-mode.md +51 -0
- package/skills/analyzing-test-failures/SKILL.md +0 -159
- package/skills/configuring-quality-gate/SKILL.md +0 -120
- package/skills/configuring-test-provider/SKILL.md +0 -113
- package/skills/managing-suite-assignments/SKILL.md +0 -161
- package/skills/polling-test-results/SKILL.md +0 -72
- package/skills/running-devops-test-suite/SKILL.md +0 -144
- package/skills/syncing-test-providers/SKILL.md +0 -108
|
@@ -1,120 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: configuring-quality-gate
|
|
3
|
-
description: "Creates a DevOps Center quality gate with associated rules (PASS_PERCENTAGE, SEVERITY, ESSENTIAL) and links it to a pipeline-stage suite assignment, after showing a mandatory impact preview and obtaining explicit confirmation. Use this skill when a user wants to set or configure a quality gate, change a coverage threshold, or establish testing benchmarks on a pipeline stage. TRIGGER when: the user wants to set/configure a quality gate, change a coverage threshold, or set testing benchmarks on a stage. DO NOT TRIGGER when: only re-running an existing gate (use running-devops-test-suite in retrigger mode)."
|
|
4
|
-
metadata:
|
|
5
|
-
version: "1.0"
|
|
6
|
-
minApiVersion: "67.0"
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## Prerequisites
|
|
10
|
-
|
|
11
|
-
Load `checking-devops-prerequisites` first — Prerequisites 1–4 AND Prerequisite 5 (stage). You need `doce-org-alias`, `pipelineId`, `stageId`, and the target `DevopsTestSuiteStage` record Id.
|
|
12
|
-
|
|
13
|
-
## Inputs required
|
|
14
|
-
|
|
15
|
-
| Input | Source |
|
|
16
|
-
|---|---|
|
|
17
|
-
| `name` | User-provided name for the quality gate |
|
|
18
|
-
| `rules` | List of `{type, threshold?}` — see rule types below |
|
|
19
|
-
| `doce-org-alias` | Established in Prereq 1 |
|
|
20
|
-
| `testSuiteStageId` | Target `DevopsTestSuiteStage` record Id (Prereq 5) |
|
|
21
|
-
|
|
22
|
-
## Rule types
|
|
23
|
-
|
|
24
|
-
| Type | Description | Threshold |
|
|
25
|
-
|---|---|---|
|
|
26
|
-
| `PASS_PERCENTAGE` | Minimum % of tests that must pass | Required (0–100) |
|
|
27
|
-
| `SEVERITY` | Maximum allowed severity level of failures | Required (numeric, e.g. 1–5) |
|
|
28
|
-
| `ESSENTIAL` | All essential tests must pass | Not required |
|
|
29
|
-
|
|
30
|
-
---
|
|
31
|
-
|
|
32
|
-
## MANDATORY IMPACT PREVIEW
|
|
33
|
-
|
|
34
|
-
**Before executing any commands**, show the user this preview and wait for explicit confirmation:
|
|
35
|
-
|
|
36
|
-
> "Here's what this quality gate will enforce on `<stageName>`:
|
|
37
|
-
> - Rule: `<type>` — `<description>`
|
|
38
|
-
> - Threshold: `<value>`
|
|
39
|
-
> - Affected pipelines: `<list>`
|
|
40
|
-
>
|
|
41
|
-
> Confirm to apply?"
|
|
42
|
-
|
|
43
|
-
**Never proceed past this point without showing the impact preview first and receiving explicit confirmation.**
|
|
44
|
-
|
|
45
|
-
---
|
|
46
|
-
|
|
47
|
-
## Confirmation gate
|
|
48
|
-
|
|
49
|
-
Only proceed with the steps below after the user has explicitly confirmed (e.g., "yes", "confirm", "go ahead"). If the user declines or does not confirm, stop and do not execute any commands.
|
|
50
|
-
|
|
51
|
-
---
|
|
52
|
-
|
|
53
|
-
## Step 1 — Create the gate record
|
|
54
|
-
|
|
55
|
-
```bash
|
|
56
|
-
sf api request rest \
|
|
57
|
-
"/services/data/v67.0/connect/devopstesting/qualityGate" \
|
|
58
|
-
--method POST \
|
|
59
|
-
--body '{"name": "<gateName>"}' \
|
|
60
|
-
--target-org <doce-org-alias>
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
Extract `qualityGateId` from the response.
|
|
64
|
-
|
|
65
|
-
## Step 2 — Create each rule as a sObject record
|
|
66
|
-
|
|
67
|
-
Rules are not accepted in the Connect API payload — create them as `DevopsQualityGateRule` records directly. Only create rules the user has requested. `ESSENTIAL` has no threshold.
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
sf data create record \
|
|
71
|
-
--sobject DevopsQualityGateRule \
|
|
72
|
-
--values "DevopsQualityGateId='<qualityGateId>' Rule='PASS_PERCENTAGE' Threshold=<value>" \
|
|
73
|
-
--target-org <doce-org-alias> --json
|
|
74
|
-
|
|
75
|
-
sf data create record \
|
|
76
|
-
--sobject DevopsQualityGateRule \
|
|
77
|
-
--values "DevopsQualityGateId='<qualityGateId>' Rule='ESSENTIAL'" \
|
|
78
|
-
--target-org <doce-org-alias> --json
|
|
79
|
-
|
|
80
|
-
sf data create record \
|
|
81
|
-
--sobject DevopsQualityGateRule \
|
|
82
|
-
--values "DevopsQualityGateId='<qualityGateId>' Rule='SEVERITY' Threshold=<value>" \
|
|
83
|
-
--target-org <doce-org-alias> --json
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
## Step 3 — Link the gate to the suite stage
|
|
87
|
-
|
|
88
|
-
Update the `DevopsTestSuiteStage` record to link the new gate:
|
|
89
|
-
|
|
90
|
-
```bash
|
|
91
|
-
sf data update record \
|
|
92
|
-
--sobject DevopsTestSuiteStage \
|
|
93
|
-
--record-id <testSuiteStageId> \
|
|
94
|
-
--values "IsQualityGateEnabled=true DevopsQualityGateId='<qualityGateId>'" \
|
|
95
|
-
--target-org <doce-org-alias> --json
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
---
|
|
99
|
-
|
|
100
|
-
## On success
|
|
101
|
-
|
|
102
|
-
Confirm to the user:
|
|
103
|
-
> "Quality gate `<gateName>` created with `<N>` rule(s) and assigned to `<suiteName>` on `<stageName>`."
|
|
104
|
-
|
|
105
|
-
## Error handling
|
|
106
|
-
|
|
107
|
-
Never expose raw API errors. Use the following responses:
|
|
108
|
-
|
|
109
|
-
| Status | Response |
|
|
110
|
-
|---|---|
|
|
111
|
-
| 400 | "The quality gate configuration is invalid. Check that all rule types and thresholds are correct." |
|
|
112
|
-
| 403 | "You don't have permission to configure quality gates on this org." |
|
|
113
|
-
| 500 | "A server error occurred. Try again in a few minutes." |
|
|
114
|
-
|
|
115
|
-
---
|
|
116
|
-
|
|
117
|
-
## Related skills
|
|
118
|
-
|
|
119
|
-
- To re-run a gate after meeting threshold, use `running-devops-test-suite` (retrigger mode).
|
|
120
|
-
- To assign or map suites to stages, use `managing-suite-assignments`.
|
|
@@ -1,113 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: configuring-test-provider
|
|
3
|
-
description: "Configures an available test provider (e.g. Apex Unit Tests, Code Analyzer, Flow Tests, Provar) on a DevOps Center pipeline via the Connect API, after explicit user confirmation, so the provider's test suites become available for assignment to pipeline stages. Takes a pipelineId and a testProviderId. Use this skill when a provider is available on the pipeline but not yet configured and the user wants to enable it. TRIGGER when: the user wants to configure, enable, set up, or add a test provider on a pipeline, or wants a not-yet-configured provider's suites to become available. DO NOT TRIGGER when: re-syncing an already-configured provider to pick up new suites (use syncing-test-providers), or assigning existing suites to a stage (use managing-suite-assignments)."
|
|
4
|
-
metadata:
|
|
5
|
-
version: "1.0"
|
|
6
|
-
minApiVersion: "67.0"
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# Configuring a Test Provider
|
|
10
|
-
|
|
11
|
-
Configures an available test provider on a DevOps Center pipeline, making its test suites available for assignment to pipeline stages.
|
|
12
|
-
|
|
13
|
-
**Confirmation required:** Yes — explicit confirmation before the provider is configured.
|
|
14
|
-
|
|
15
|
-
## Prerequisites
|
|
16
|
-
|
|
17
|
-
Load `checking-devops-prerequisites` first — Prerequisites 1–4 (org login, Agentforce DX plugin, DevOps Center org auth, pipeline identified). Prerequisite 5 (stage) is **not** required: providers are configured at the pipeline level, not the stage level.
|
|
18
|
-
|
|
19
|
-
| Variable | Source |
|
|
20
|
-
|---|---|
|
|
21
|
-
| `doce-org-alias` | Established in Prerequisite 1 |
|
|
22
|
-
| `pipelineId` | Identified in Prerequisite 4 (pipeline selection) |
|
|
23
|
-
| `testProviderId` | Resolved by fetching the pipeline's test providers (below) |
|
|
24
|
-
|
|
25
|
-
---
|
|
26
|
-
|
|
27
|
-
## Step 1 — Fetch test providers to resolve the provider ID
|
|
28
|
-
|
|
29
|
-
Get all test providers for the pipeline so you can resolve the `testProviderId` and confirm which providers are still available to configure:
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
sf api request rest \
|
|
33
|
-
"/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testProviders?status=all" \
|
|
34
|
-
--target-org <doce-org-alias>
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
Each provider entry includes `testProviderId`, `testProviderName`, and a status (Configured vs. Available). Present a short summary grouped by status:
|
|
38
|
-
|
|
39
|
-
```text
|
|
40
|
-
Test providers for <pipelineName>:
|
|
41
|
-
|
|
42
|
-
✓ Configured:
|
|
43
|
-
- Code Analyzer (63 suites)
|
|
44
|
-
- Apex Unit Tests (5 suites)
|
|
45
|
-
|
|
46
|
-
Available (not yet configured):
|
|
47
|
-
- Flow Tests
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
- **Only an Available provider can be configured.** If the pipeline has no available providers, report that and stop — do NOT fabricate a provider or ID.
|
|
51
|
-
|
|
52
|
-
### If the named provider is already Configured
|
|
53
|
-
|
|
54
|
-
Do **not** present the confirmation gate and do **not** POST to the configure endpoint (Steps 2–3) — that would create a duplicate `DevopsPipelineTestProvider`. Instead:
|
|
55
|
-
|
|
56
|
-
1. State plainly that the provider is already configured, including its synced suite count and last-sync time if returned (e.g. *"Flow Tests is already configured on `<pipelineName>` with 3 suites synced (last sync 2026-06-23)."*).
|
|
57
|
-
2. Diagnose the user's actual goal and redirect **by name**:
|
|
58
|
-
- If the user says the provider's **suites don't appear when assigning tests to a stage**, this is a **stage-assignment gap, not a provider-configuration gap** — the suites already exist at the pipeline level; they just need to be linked to the stage. Redirect to **`managing-suite-assignments`**.
|
|
59
|
-
- If the user expects **newly created suites** that aren't yet synced, redirect to **`syncing-test-providers`** (re-sync via `POST /connect/devops/sync`) to pull them in.
|
|
60
|
-
3. Do not loop back to configuring — finish cleanly after the explanation and redirect.
|
|
61
|
-
|
|
62
|
-
## Step 2 — Confirmation gate
|
|
63
|
-
|
|
64
|
-
**Required — do not call the API before the user confirms.**
|
|
65
|
-
|
|
66
|
-
> "I'll configure `<testProviderName>` on the `<pipelineName>` pipeline. This will make its suites available for assignment to stages. Confirm?"
|
|
67
|
-
|
|
68
|
-
Do not proceed until the user gives an affirmative response.
|
|
69
|
-
|
|
70
|
-
## Step 3 — Configure the provider
|
|
71
|
-
|
|
72
|
-
On confirmation, call the configure endpoint with the provider ID:
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
sf api request rest \
|
|
76
|
-
"/services/data/v67.0/connect/devops/pipeline/<pipelineId>/testProvider" \
|
|
77
|
-
--method POST \
|
|
78
|
-
--body '{"testProviderId": "<testProviderId>"}' \
|
|
79
|
-
--target-org <doce-org-alias>
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
## On success
|
|
83
|
-
|
|
84
|
-
> "Provider `<testProviderName>` is now configured on the `<pipelineName>` pipeline. Its suites are available for assignment to stages."
|
|
85
|
-
|
|
86
|
-
Newly configured suites can then be assigned to stages with `managing-suite-assignments`.
|
|
87
|
-
|
|
88
|
-
---
|
|
89
|
-
|
|
90
|
-
## Critical gotcha
|
|
91
|
-
|
|
92
|
-
This `POST .../pipeline/<pipelineId>/testProvider` endpoint **creates a new provider configuration record** (`DevopsPipelineTestProvider`). Use it ONLY to configure a provider for the first time. To re-sync an already-configured provider for new suites, use `syncing-test-providers` (`POST /connect/devops/sync`) — calling this configure endpoint on an already-configured provider produces **duplicate** `DevopsPipelineTestProvider` records.
|
|
93
|
-
|
|
94
|
-
## Error Handling
|
|
95
|
-
|
|
96
|
-
Never expose raw API error messages, stack traces, or JSON payloads to the user. Map response status codes to plain-language messages:
|
|
97
|
-
|
|
98
|
-
| Status | User-facing message |
|
|
99
|
-
|---|---|
|
|
100
|
-
| 400 | "The request was invalid. Check that the provider ID and pipeline ID are correct." |
|
|
101
|
-
| 403 | "You don't have permission to configure test providers on this pipeline." |
|
|
102
|
-
| 404 | "The pipeline or test provider was not found." |
|
|
103
|
-
| 409 | "That provider appears to already be configured on this pipeline. To pick up new suites, re-sync it instead." |
|
|
104
|
-
| 500 | "A server error occurred. Try again in a few minutes." |
|
|
105
|
-
|
|
106
|
-
---
|
|
107
|
-
|
|
108
|
-
## Related skills
|
|
109
|
-
|
|
110
|
-
- **`checking-devops-prerequisites`** — loaded first to establish org and pipeline context.
|
|
111
|
-
- **`syncing-test-providers`** — once a provider is configured, use this to re-sync it later and pull in newly added suites.
|
|
112
|
-
- **`managing-suite-assignments`** — after configuring a provider, use this to assign or map its suites to a pipeline stage.
|
|
113
|
-
- **`recommending-devops-tests`** — to recommend which of the newly available suites to run.
|
|
@@ -1,161 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: managing-suite-assignments
|
|
3
|
-
description: "Manages all forms of test suite assignment for DevOps Center pipeline stages via the Connect API testSuiteStages endpoint. Covers three modes: (A) attaching a single suite to a stage as a one-off add, (B) bulk-mapping multiple suites to a stage as a testing strategy with a mandatory impact-preview table before any changes, and (C) adding or removing individual test classes within a suite assignment with governance rules that exclude rejected tests and re-present the final list before committing. TRIGGER when: a suite is found unlinked from a pipeline stage and the user wants to assign it; the user wants to configure suite-to-stage mappings across a pipeline or assign multiple suites to stages as part of a testing strategy; the user wants to add or remove tests in a suite, sync reviewed tests into a suite, or promote tests to a suite. DO NOT TRIGGER when: authoring new test classes (use platform-apex-test-generate) or running tests directly (use platform-apex-test-run)."
|
|
4
|
-
metadata:
|
|
5
|
-
version: "1.0"
|
|
6
|
-
minApiVersion: "67.0"
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# Managing Suite Assignments
|
|
10
|
-
|
|
11
|
-
## Prerequisites
|
|
12
|
-
|
|
13
|
-
Load `checking-devops-prerequisites` first — Prerequisites 1–4 AND Prerequisite 5 (stage). You need `doce-org-alias`, `pipelineId`, and `stageId` before proceeding in any mode.
|
|
14
|
-
|
|
15
|
-
| Variable | Description |
|
|
16
|
-
|---|---|
|
|
17
|
-
| `doce-org-alias` | Established in Prerequisite 1 |
|
|
18
|
-
| `pipelineId` | Identified in Prerequisite 4 (pipeline selection) |
|
|
19
|
-
| `stageId` | Identified in Prerequisite 5 (stage selection) |
|
|
20
|
-
| `event` | `Pre-Promote`, `Post-Promote`, or `Review` |
|
|
21
|
-
|
|
22
|
-
---
|
|
23
|
-
|
|
24
|
-
## Mode A — Assign a single suite to a stage
|
|
25
|
-
|
|
26
|
-
Use this mode when a relevant test suite already exists but is not yet linked to a pipeline stage and the user wants to add it as a single one-off assignment.
|
|
27
|
-
|
|
28
|
-
**Additional inputs required:**
|
|
29
|
-
|
|
30
|
-
| Variable | Description |
|
|
31
|
-
|---|---|
|
|
32
|
-
| `testSuiteId` | ID of the suite to assign |
|
|
33
|
-
| `testSuiteName` | Name of the suite (for display in confirmation) |
|
|
34
|
-
|
|
35
|
-
**Confirmation gate**
|
|
36
|
-
|
|
37
|
-
**Confirmation required before any API call is made.**
|
|
38
|
-
|
|
39
|
-
Present the following prompt to the user and wait for an affirmative response:
|
|
40
|
-
|
|
41
|
-
> "The suite `<testSuiteName>` is not currently assigned to the `<stageName>` stage (`<event>`). Would you like me to assign it now?"
|
|
42
|
-
|
|
43
|
-
Do not proceed until the user confirms.
|
|
44
|
-
|
|
45
|
-
**On success**
|
|
46
|
-
|
|
47
|
-
Report to the user:
|
|
48
|
-
|
|
49
|
-
> "Suite `<testSuiteName>` has been assigned to the `<stageName>` stage (`<event>`)."
|
|
50
|
-
|
|
51
|
-
---
|
|
52
|
-
|
|
53
|
-
## Mode B — Map multiple suites to a stage
|
|
54
|
-
|
|
55
|
-
Use this mode when configuring suite-to-stage mappings across a pipeline or assigning multiple suites to stages as part of a testing strategy.
|
|
56
|
-
|
|
57
|
-
**Additional inputs required:**
|
|
58
|
-
|
|
59
|
-
| Input | Description |
|
|
60
|
-
|---|---|
|
|
61
|
-
| `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
|
|
62
|
-
|
|
63
|
-
**MANDATORY IMPACT PREVIEW — Required Before Any Changes**
|
|
64
|
-
|
|
65
|
-
**Do not call the API until the user has confirmed the preview below.**
|
|
66
|
-
|
|
67
|
-
Present the full mapping summary and wait for explicit confirmation:
|
|
68
|
-
|
|
69
|
-
> "Here's the suite mapping I'll apply:
|
|
70
|
-
>
|
|
71
|
-
> | Suite | Stage | Event | Action |
|
|
72
|
-
> |---|---|---|---|
|
|
73
|
-
> | `<suiteName>` | `<stageName>` | `<event>` | Add |
|
|
74
|
-
> | `<suiteName2>` | `<stageName>` | `<event>` | Remove |
|
|
75
|
-
>
|
|
76
|
-
> Confirm to apply all changes?"
|
|
77
|
-
|
|
78
|
-
Only proceed after the user explicitly confirms.
|
|
79
|
-
|
|
80
|
-
**On success**
|
|
81
|
-
|
|
82
|
-
> "Suite mapping applied. `<N>` suite(s) updated for the `<stageName>` stage."
|
|
83
|
-
|
|
84
|
-
---
|
|
85
|
-
|
|
86
|
-
## Mode C — Add/remove test classes in a suite
|
|
87
|
-
|
|
88
|
-
Use this mode when the user wants to add or remove individual test classes within an existing suite assignment, sync reviewed tests into a suite, or promote tests to a suite.
|
|
89
|
-
|
|
90
|
-
**Additional inputs required:**
|
|
91
|
-
|
|
92
|
-
| Input | Source |
|
|
93
|
-
|---|---|
|
|
94
|
-
| `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
|
|
95
|
-
|
|
96
|
-
**Governance rules**
|
|
97
|
-
|
|
98
|
-
- **Never call this without explicit approval.** AI-reviewed or modified tests must be re-presented to the user before this call is made.
|
|
99
|
-
- Rejected tests must be EXCLUDED from the payload — do not include them even if they were previously in the suite.
|
|
100
|
-
- If tests were modified during review, re-present the final list of tests before requesting confirmation.
|
|
101
|
-
|
|
102
|
-
**Confirmation gate**
|
|
103
|
-
|
|
104
|
-
**REQUIRED — do not skip.** Show the user the full list of changes before calling:
|
|
105
|
-
|
|
106
|
-
> "I'm about to sync the following changes to the test suite:
|
|
107
|
-
> - **Add:** `<testSuiteName1>`, `<testSuiteName2>`
|
|
108
|
-
> - **Remove:** `<testSuiteName3>`
|
|
109
|
-
> - Stage: `<stageName>` | Event: `<event>`
|
|
110
|
-
>
|
|
111
|
-
> Confirm?"
|
|
112
|
-
|
|
113
|
-
Only proceed after receiving explicit confirmation.
|
|
114
|
-
|
|
115
|
-
**On success**
|
|
116
|
-
|
|
117
|
-
Confirm to the user:
|
|
118
|
-
> "Test suite updated successfully for the `<stageName>` stage."
|
|
119
|
-
|
|
120
|
-
---
|
|
121
|
-
|
|
122
|
-
## The system call
|
|
123
|
-
|
|
124
|
-
All three modes use the same endpoint. Substitute all `<placeholder>` values before executing.
|
|
125
|
-
|
|
126
|
-
```bash
|
|
127
|
-
sf api request rest "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/testSuiteStages" --method POST --body '{"pipelineStageId":"<stageId>","event":"<event>","assignments":[{"testSuiteId":"<id>","action":"add|remove"}]}' --target-org <doce-org-alias>
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
Full payload schema:
|
|
131
|
-
|
|
132
|
-
```json
|
|
133
|
-
{
|
|
134
|
-
"pipelineStageId": "<stageId>",
|
|
135
|
-
"event": "<event>",
|
|
136
|
-
"assignments": [
|
|
137
|
-
{"testSuiteId": "<suiteId1>", "action": "add"},
|
|
138
|
-
{"testSuiteId": "<suiteId2>", "action": "remove"}
|
|
139
|
-
]
|
|
140
|
-
}
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
**Error handling**
|
|
144
|
-
|
|
145
|
-
Never expose raw API error messages to the user. Map response status codes to the following user-facing messages:
|
|
146
|
-
|
|
147
|
-
| Status | User-facing message |
|
|
148
|
-
|---|---|
|
|
149
|
-
| 400 | "The request was invalid. Check that all suite and stage IDs are correct and the event type is valid." |
|
|
150
|
-
| 403 | "You don't have permission to modify suite assignments on this pipeline." |
|
|
151
|
-
| 404 | "The pipeline or stage was not found." |
|
|
152
|
-
| 500 | "A server error occurred. Try again in a few minutes." |
|
|
153
|
-
|
|
154
|
-
---
|
|
155
|
-
|
|
156
|
-
## Related skills
|
|
157
|
-
|
|
158
|
-
- **`syncing-test-providers`** — use when the suite you want to assign doesn't appear yet; re-syncing the provider pulls in newly added suites.
|
|
159
|
-
- **`recommending-devops-tests`** — use when a suite has not yet been identified and you need a recommendation for which suite to assign.
|
|
160
|
-
- **`configuring-quality-gate`** — use to configure a quality gate on the stage after mapping suites.
|
|
161
|
-
- **`analyzing-test-failures`** — use for failure analysis and suggested improvements to the tests within a suite.
|
|
@@ -1,72 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: polling-test-results
|
|
3
|
-
description: "Polls a DevOps Center async test execution by runId until it completes, fails, or times out — using read-only SOQL on the execution record at provider-specific intervals (Apex 15s/5m, Code Analyzer 10s/3m, Provar UI 60s/20m, Flow 20s/8m) — then surfaces results. Use this skill when a test suite run is in progress and the user is waiting on results, or immediately after running a suite. TRIGGER when: a test suite run is in progress and the user is waiting on results, or immediately after running a suite and a runId is available. DO NOT TRIGGER when: there is no active runId."
|
|
4
|
-
metadata:
|
|
5
|
-
version: "1.0"
|
|
6
|
-
minApiVersion: "67.0"
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# polling-test-results
|
|
10
|
-
|
|
11
|
-
**Confirmation required:** No — polling is automatic and read-only. No user confirmation gate is needed.
|
|
12
|
-
|
|
13
|
-
**What it does:** Polls the DevOps Center org for the status of an async test execution until it completes, times out, or fails.
|
|
14
|
-
|
|
15
|
-
---
|
|
16
|
-
|
|
17
|
-
## Prerequisites
|
|
18
|
-
|
|
19
|
-
You need an active `runId` (returned by the `running-devops-test-suite` skill) and the confirmed `doce-org-alias`. If org context is not yet established, load `checking-devops-prerequisites` first (Prerequisites 1–3).
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## Inputs required
|
|
24
|
-
|
|
25
|
-
| Input | Source |
|
|
26
|
-
|---|---|
|
|
27
|
-
| `runId` | Returned by the `running-devops-test-suite` skill |
|
|
28
|
-
| `testType` | Derived from the suite's test provider (Apex, Code Analyzer, UI/Provar, Flow) |
|
|
29
|
-
| `doce-org-alias` | Established via `checking-devops-prerequisites` |
|
|
30
|
-
|
|
31
|
-
## Polling configuration
|
|
32
|
-
|
|
33
|
-
| Test type | Poll interval | Max wait | Timeout action |
|
|
34
|
-
|---|---|---|---|
|
|
35
|
-
| Apex unit tests | 15 seconds | 5 minutes | Surface runId, offer retry |
|
|
36
|
-
| Code Analyzer | 10 seconds | 3 minutes | Surface runId, offer retry |
|
|
37
|
-
| UI tests (Provar) | 60 seconds | 20 minutes | Surface runId, mark as pending |
|
|
38
|
-
| Flow tests | 20 seconds | 8 minutes | Surface runId, offer retry |
|
|
39
|
-
|
|
40
|
-
## Poll query
|
|
41
|
-
|
|
42
|
-
Query the execution record by `runId` on each interval:
|
|
43
|
-
|
|
44
|
-
```bash
|
|
45
|
-
sf data query \
|
|
46
|
-
--query "SELECT Id, Status, TestsRan, TestsPassed, TestsFailed, CoveragePercentage FROM DevopsTestExecution WHERE Id = '<runId>' LIMIT 1" \
|
|
47
|
-
--target-org <doce-org-alias> \
|
|
48
|
-
--json
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
Check the `Status` field on each poll:
|
|
52
|
-
- `Queued` / `Running` → wait and poll again
|
|
53
|
-
- `Completed` → proceed to result analysis
|
|
54
|
-
- `Failed` → surface error and offer retry or skip
|
|
55
|
-
|
|
56
|
-
## On timeout
|
|
57
|
-
|
|
58
|
-
Surface the `runId` to the user:
|
|
59
|
-
> "The test run is taking longer than expected. Your run ID is `<runId>`. You can check the status manually in DevOps Center, or I can keep waiting — what would you prefer?"
|
|
60
|
-
|
|
61
|
-
Do not automatically retry after timeout. Wait for user instruction.
|
|
62
|
-
|
|
63
|
-
## On completion
|
|
64
|
-
|
|
65
|
-
Pass the full result payload to the `analyzing-test-failures` skill for reasoning. Surface `Coverage`, `SuccessCount`, `FailureCount`, and `QualityGateStatus` inline. Do not surface raw JSON to the user.
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## Related skills
|
|
70
|
-
|
|
71
|
-
- Runs are started by `running-devops-test-suite` (including its retrigger mode)
|
|
72
|
-
- Failure analysis is handled by `analyzing-test-failures`
|
|
@@ -1,144 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: running-devops-test-suite
|
|
3
|
-
description: "Triggers async execution of one or more DevOps Center test suites on a pipeline stage (Pre-Promote, Post-Promote, or Review event) via the Connect API, after an explicit user confirmation gate, then hands off to result polling via the `polling-test-results` skill. Also re-runs (retriggers) a quality gate after fixes — but only once validation confirms the coverage threshold is now met. TRIGGER when: the user wants to run, kick off, or launch test suites on a pipeline stage; execute tests before or after a promotion; trigger a Pre-Promote, Post-Promote, or Review-event run; re-run a quality gate after fixing failures; retry a failed gate once coverage is met; or unblock a blocked promotion after adding tests. DO NOT TRIGGER when: running sf apex run test directly (use platform-apex-test-run); or configuring a NEW gate or threshold (use configuring-quality-gate)."
|
|
4
|
-
metadata:
|
|
5
|
-
version: "1.0"
|
|
6
|
-
minApiVersion: "67.0"
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# Running a DevOps Center Test Suite
|
|
10
|
-
|
|
11
|
-
## Prerequisites
|
|
12
|
-
|
|
13
|
-
Load and follow `checking-devops-prerequisites` first — run Prerequisites 1–4 AND Prerequisite 5 (pipeline stage), since this skill operates on a specific stage. You need the confirmed `doce-org-alias`, `pipelineId`, and `stageId` before proceeding.
|
|
14
|
-
|
|
15
|
-
## Inputs required before calling this skill
|
|
16
|
-
|
|
17
|
-
| Input | How to obtain |
|
|
18
|
-
|---|---|
|
|
19
|
-
| `pipelineId` | From Prerequisite 4 (pipeline selection) |
|
|
20
|
-
| `stageId` | From Prerequisite 5 (pipeline stage confirmation) |
|
|
21
|
-
| `event` | Confirm with user: `Pre-Promote` or `Post-Promote` (or `Review` if the context is a review environment) |
|
|
22
|
-
| `testSuiteIds` | Confirmed suite IDs from the suite selection or recommendation step |
|
|
23
|
-
| `doce-org-alias` | Established in Prerequisite 1 |
|
|
24
|
-
|
|
25
|
-
## Confirmation gate
|
|
26
|
-
|
|
27
|
-
**This call mutates org state — do not proceed without explicit user confirmation.**
|
|
28
|
-
|
|
29
|
-
Before calling the API, show the user:
|
|
30
|
-
|
|
31
|
-
> "I'm about to run tests with the following configuration:
|
|
32
|
-
> - Pipeline: `<pipelineName>`
|
|
33
|
-
> - Stage: `<stageName>`
|
|
34
|
-
> - Event: `<event>`
|
|
35
|
-
> - Suite(s): `<suiteName(s)>`
|
|
36
|
-
> - Org: `<doce-org-alias>`
|
|
37
|
-
>
|
|
38
|
-
> Shall I proceed?"
|
|
39
|
-
|
|
40
|
-
Do not make the API call until the user confirms.
|
|
41
|
-
|
|
42
|
-
## API call
|
|
43
|
-
|
|
44
|
-
```bash
|
|
45
|
-
sf api request rest \
|
|
46
|
-
"/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" \
|
|
47
|
-
--method POST \
|
|
48
|
-
--body '{
|
|
49
|
-
"stageId": "<stageId>",
|
|
50
|
-
"event": "<event>",
|
|
51
|
-
"testSuiteIds": ["<suiteId1>", "<suiteId2>"]
|
|
52
|
-
}' \
|
|
53
|
-
--target-org <doce-org-alias>
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
### Body schema
|
|
57
|
-
|
|
58
|
-
| Field | Type | Description |
|
|
59
|
-
|---|---|---|
|
|
60
|
-
| `stageId` | string | The ID of the pipeline stage to execute tests on |
|
|
61
|
-
| `event` | string | `Pre-Promote`, `Post-Promote`, or `Review` |
|
|
62
|
-
| `testSuiteIds` | string[] | One or more test suite IDs to execute |
|
|
63
|
-
|
|
64
|
-
## On success
|
|
65
|
-
|
|
66
|
-
Extract the `runId` (or execution ID) from the response. Inform the user:
|
|
67
|
-
|
|
68
|
-
> "Tests are running in `<doce-org-alias>`. I'll update you when results are ready."
|
|
69
|
-
|
|
70
|
-
Immediately hand off the `runId` to the `polling-test-results` skill to begin the polling loop.
|
|
71
|
-
|
|
72
|
-
## On error
|
|
73
|
-
|
|
74
|
-
| Status | Message to user |
|
|
75
|
-
|---|---|
|
|
76
|
-
| 400 | "The test execution request was invalid. Check that the stage and suite IDs are correct." |
|
|
77
|
-
| 403 | "You don't have permission to run tests on this pipeline. Check your DevOps Testing API access." |
|
|
78
|
-
| 404 | "The pipeline or stage was not found. It may have been deleted." |
|
|
79
|
-
| 500 | "The DevOps Center org returned a server error. Try again in a few minutes." |
|
|
80
|
-
|
|
81
|
-
Never expose raw API errors to the user.
|
|
82
|
-
|
|
83
|
-
---
|
|
84
|
-
|
|
85
|
-
## Retrigger mode (re-running a quality gate)
|
|
86
|
-
|
|
87
|
-
Use this mode when a promotion was blocked by a quality gate failure and the coverage gap has since been addressed.
|
|
88
|
-
|
|
89
|
-
### Extra preconditions — all must be true before proceeding
|
|
90
|
-
|
|
91
|
-
1. The `Coverage` field on the latest `DevopsTestSuiteExecution` meets or exceeds the threshold defined in the `DevopsQualityGateRule`
|
|
92
|
-
2. The user has explicitly asked to retrigger the gate
|
|
93
|
-
3. The same `pipelineId`, `stageId`, and `event` from the blocked promotion are known
|
|
94
|
-
|
|
95
|
-
If coverage is still below threshold, do **not** retrigger. Instead, respond:
|
|
96
|
-
|
|
97
|
-
> "Coverage is still at `<X>%`, below the `<threshold>%` gate. The gate cannot be retriggered until the threshold is met. Here are the remaining uncovered methods: `<list>`."
|
|
98
|
-
|
|
99
|
-
Do not retry. Explain what must be resolved first and stop.
|
|
100
|
-
|
|
101
|
-
### Inputs required for retrigger
|
|
102
|
-
|
|
103
|
-
| Input | Source |
|
|
104
|
-
|---|---|
|
|
105
|
-
| `pipelineId` | From the blocked promotion context |
|
|
106
|
-
| `stageId` | From the blocked promotion context |
|
|
107
|
-
| `event` | Same event type that originally blocked (`Pre-Promote` or `Post-Promote`) |
|
|
108
|
-
| `suiteIds` | Same suites that were originally run |
|
|
109
|
-
| `doce-org-alias` | Established in Prerequisite 1 |
|
|
110
|
-
|
|
111
|
-
### Confirmation gate (retrigger)
|
|
112
|
-
|
|
113
|
-
Before executing the API call, present this confirmation prompt and wait for explicit user approval:
|
|
114
|
-
|
|
115
|
-
> "Coverage is confirmed at `<X>%`, which meets the `<threshold>%` gate. I'll retrigger the quality gate check for the `<stageName>` stage (`<event>`). Confirm?"
|
|
116
|
-
|
|
117
|
-
Only proceed after the user confirms. If the user declines, stop without making any API call.
|
|
118
|
-
|
|
119
|
-
### API call (retrigger)
|
|
120
|
-
|
|
121
|
-
Uses the same Connect API stage/execute endpoint:
|
|
122
|
-
|
|
123
|
-
```bash
|
|
124
|
-
sf api request rest "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" --method POST --body '{"stageId":"<stageId>","event":"<event>","testSuiteIds":["<suiteId1>"]}' --target-org <doce-org-alias>
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
After the call returns a `runId`, hand off to the `polling-test-results` skill with the new `runId` to monitor the execution result.
|
|
128
|
-
|
|
129
|
-
### Error handling (retrigger)
|
|
130
|
-
|
|
131
|
-
If the API returns an error indicating the gate cannot be retriggered, respond with:
|
|
132
|
-
|
|
133
|
-
> "The quality gate cannot be retriggered right now. Reason: `<plain-language summary>`. Here's what needs to be resolved first: `<list>`."
|
|
134
|
-
|
|
135
|
-
Never expose raw API error details to the user.
|
|
136
|
-
|
|
137
|
-
---
|
|
138
|
-
|
|
139
|
-
## Related skills
|
|
140
|
-
|
|
141
|
-
- **`polling-test-results`** — poll for async run results after receiving the `runId`
|
|
142
|
-
- **`recommending-devops-tests`** — recommend which suites to run first before triggering execution
|
|
143
|
-
- **`managing-suite-assignments`** — assign or map a suite to a pipeline stage if it isn't linked yet
|
|
144
|
-
- **`configuring-quality-gate`** — configure a new gate or threshold
|