@salesforce/afv-skills 1.25.0 → 1.27.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/README.md +3 -3
  2. package/package.json +1 -1
  3. package/skills/dx-devops-test-failures-analyze/SKILL.md +89 -0
  4. package/skills/dx-devops-test-failures-analyze/references/code-analyzer-violations.md +26 -0
  5. package/skills/dx-devops-test-failures-analyze/references/failure-categories.md +85 -0
  6. package/skills/{checking-devops-prerequisites/SKILL.md → dx-devops-test-failures-analyze/references/prerequisite-checks.md} +8 -37
  7. package/skills/{creating-fix-work-item/SKILL.md → dx-devops-test-failures-analyze/references/work-item-creation.md} +8 -12
  8. package/skills/dx-devops-test-pipeline-configure/SKILL.md +72 -0
  9. package/skills/dx-devops-test-pipeline-configure/references/configuring-quality-gate.md +133 -0
  10. package/skills/dx-devops-test-pipeline-configure/references/configuring-test-provider.md +80 -0
  11. package/skills/dx-devops-test-pipeline-configure/references/error-handling.md +39 -0
  12. package/skills/dx-devops-test-pipeline-configure/references/gotchas.md +37 -0
  13. package/skills/dx-devops-test-pipeline-configure/references/prerequisite-checks.md +112 -0
  14. package/skills/dx-devops-test-pipeline-configure/references/syncing-test-providers.md +69 -0
  15. package/skills/dx-devops-test-suite-assignments-configure/SKILL.md +74 -0
  16. package/skills/dx-devops-test-suite-assignments-configure/references/api-endpoint.md +30 -0
  17. package/skills/dx-devops-test-suite-assignments-configure/references/error-handling.md +14 -0
  18. package/skills/dx-devops-test-suite-assignments-configure/references/prerequisite-checks.md +112 -0
  19. package/skills/{recommending-devops-tests/SKILL.md → dx-devops-test-suite-assignments-configure/references/recommendation-logic.md} +10 -26
  20. package/skills/dx-devops-test-suite-assignments-configure/references/suite-assignment-modes.md +99 -0
  21. package/skills/dx-devops-test-suite-run/SKILL.md +111 -0
  22. package/skills/dx-devops-test-suite-run/references/error-handling.md +31 -0
  23. package/skills/dx-devops-test-suite-run/references/polling-configuration.md +78 -0
  24. package/skills/dx-devops-test-suite-run/references/prerequisite-checks.md +112 -0
  25. package/skills/dx-devops-test-suite-run/references/retrigger-mode.md +51 -0
  26. package/skills/analyzing-test-failures/SKILL.md +0 -159
  27. package/skills/configuring-quality-gate/SKILL.md +0 -120
  28. package/skills/configuring-test-provider/SKILL.md +0 -113
  29. package/skills/managing-suite-assignments/SKILL.md +0 -161
  30. package/skills/polling-test-results/SKILL.md +0 -72
  31. package/skills/running-devops-test-suite/SKILL.md +0 -144
  32. package/skills/syncing-test-providers/SKILL.md +0 -108
@@ -0,0 +1,99 @@
1
+ # Modes B–D — Assign / Map / Manage Suite Classes
2
+
3
+ All three modes use the same `testSuiteStages` endpoint (`references/api-endpoint.md`). They differ in scope and the confirmation pattern required.
4
+
5
+ ## Prerequisites & shared inputs
6
+
7
+ Run Prerequisites 1–4 **and** Prerequisite 5 (stage). You need:
8
+
9
+ | Variable | Description |
10
+ |---|---|
11
+ | `doce-org-alias` | Prerequisite 1 |
12
+ | `pipelineId` | Prerequisite 4 (pipeline selection) |
13
+ | `stageId` | Prerequisite 5 (stage selection) |
14
+ | `event` | `Pre-Promote`, `Post-Promote`, or `Review` |
15
+
16
+ ---
17
+
18
+ ## Mode B — Assign a single suite to a stage
19
+
20
+ Use when a relevant test suite already exists but is not yet linked to a stage and the user wants to add it as a single one-off assignment.
21
+
22
+ **Additional inputs:**
23
+
24
+ | Variable | Description |
25
+ |---|---|
26
+ | `testSuiteId` | ID of the suite to assign |
27
+ | `testSuiteName` | Name of the suite (for display in confirmation) |
28
+
29
+ **Confirmation gate (required before any API call):**
30
+
31
+ > "The suite `<testSuiteName>` is not currently assigned to the `<stageName>` stage (`<event>`). Would you like me to assign it now?"
32
+
33
+ Do not proceed until the user confirms.
34
+
35
+ **On success:**
36
+
37
+ > "Suite `<testSuiteName>` has been assigned to the `<stageName>` stage (`<event>`)."
38
+
39
+ ---
40
+
41
+ ## Mode C — Map multiple suites to a stage
42
+
43
+ Use when configuring suite-to-stage mappings across a pipeline or assigning multiple suites as part of a testing strategy.
44
+
45
+ **Additional inputs:**
46
+
47
+ | Input | Description |
48
+ |---|---|
49
+ | `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
50
+
51
+ **MANDATORY IMPACT PREVIEW — required before any changes.** Do not call the API until the user has confirmed the preview:
52
+
53
+ > "Here's the suite mapping I'll apply:
54
+ >
55
+ > | Suite | Stage | Event | Action |
56
+ > |---|---|---|---|
57
+ > | `<suiteName>` | `<stageName>` | `<event>` | Add |
58
+ > | `<suiteName2>` | `<stageName>` | `<event>` | Remove |
59
+ >
60
+ > Confirm to apply all changes?"
61
+
62
+ Only proceed after the user explicitly confirms.
63
+
64
+ **On success:**
65
+
66
+ > "Suite mapping applied. `<N>` suite(s) updated for the `<stageName>` stage."
67
+
68
+ ---
69
+
70
+ ## Mode D — Add/remove test classes in a suite
71
+
72
+ Use when the user wants to add or remove individual test classes within an existing suite assignment, sync reviewed tests into a suite, or promote tests to a suite.
73
+
74
+ **Additional inputs:**
75
+
76
+ | Input | Source |
77
+ |---|---|
78
+ | `testSuiteOperations` | List of `{testSuiteId, action: "add"\|"remove"}` |
79
+
80
+ **Governance rules:**
81
+
82
+ - **Never call this without explicit approval.** AI-reviewed or modified tests must be re-presented to the user before this call is made.
83
+ - Rejected tests must be EXCLUDED from the payload — do not include them even if they were previously in the suite.
84
+ - If tests were modified during review, re-present the final list before requesting confirmation.
85
+
86
+ **Confirmation gate (required — do not skip):**
87
+
88
+ > "I'm about to sync the following changes to the test suite:
89
+ > - **Add:** `<testSuiteName1>`, `<testSuiteName2>`
90
+ > - **Remove:** `<testSuiteName3>`
91
+ > - Stage: `<stageName>` | Event: `<event>`
92
+ >
93
+ > Confirm?"
94
+
95
+ Only proceed after explicit confirmation.
96
+
97
+ **On success:**
98
+
99
+ > "Test suite updated successfully for the `<stageName>` stage."
@@ -0,0 +1,111 @@
1
+ ---
2
+ name: dx-devops-test-suite-run
3
+ description: "Runs DevOps Center test suites on a pipeline stage (Pre-Promote, Post-Promote, or Review event) end to end: triggers async execution via the Connect API after an explicit confirmation gate, then polls by runId at provider-specific intervals until it completes, fails, or times out, and hands results to failure analysis. Also retriggers a quality gate after fixes, but only once coverage meets the threshold. Use this skill when a user wants to run, kick off, or launch test suites on a stage, re-run a quality gate, or watch an in-progress run to completion. TRIGGER when: the user wants to run/launch suites on a stage, execute tests before or after promotion, re-run a quality gate after fixing failures, unblock a blocked promotion after adding tests, or poll/watch an in-progress run. DO NOT TRIGGER when: running sf apex run test directly (use platform-apex-test-run), or configuring a NEW gate or threshold (use dx-devops-test-pipeline-configure)."
4
+ metadata:
5
+ version: "1.0"
6
+ minApiVersion: "67.0"
7
+ ---
8
+
9
+ # Run a DevOps Center Test Suite
10
+
11
+ Triggers a DevOps Center test suite execution and watches it to completion. Running and polling are two halves of one operation — never poll without first having (or being handed) a `runId`.
12
+
13
+ > **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
14
+
15
+ **Important:** All DevOps Center data lives in the Salesforce org — NOT the local repo. Always query the org with `sf data query` or `sf api request rest`.
16
+
17
+ ---
18
+
19
+ ## Prerequisites
20
+
21
+ Run the prerequisite checks in `references/prerequisite-checks.md` — Prerequisites 1–4 **and** Prerequisite 5 (stage), since this skill operates on a specific stage. You need the confirmed `doce-org-alias`, `pipelineId`, and `stageId`.
22
+
23
+ ## Inputs required
24
+
25
+ | Input | How to obtain |
26
+ |---|---|
27
+ | `pipelineId` | Prerequisite 4 (pipeline selection) |
28
+ | `stageId` | Prerequisite 5 (pipeline stage confirmation) |
29
+ | `event` | Confirm with user: `Pre-Promote`, `Post-Promote`, or `Review` |
30
+ | `testSuiteIds` | Confirmed suite IDs from selection or recommendation |
31
+ | `doce-org-alias` | Prerequisite 1 |
32
+
33
+ ---
34
+
35
+ ## Step 1 — Trigger execution
36
+
37
+ ### Confirmation gate
38
+
39
+ **This call mutates org state — do not proceed without explicit user confirmation.** Before calling the API, show:
40
+
41
+ > "I'm about to run tests with the following configuration:
42
+ > - Pipeline: `<pipelineName>`
43
+ > - Stage: `<stageName>`
44
+ > - Event: `<event>`
45
+ > - Suite(s): `<suiteName(s)>`
46
+ > - Org: `<doce-org-alias>`
47
+ >
48
+ > Shall I proceed?"
49
+
50
+ Do not make the API call until the user confirms.
51
+
52
+ ### API call
53
+
54
+ ```bash
55
+ sf api request rest \
56
+ "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" \
57
+ --method POST \
58
+ --body '{
59
+ "stageId": "<stageId>",
60
+ "event": "<event>",
61
+ "testSuiteIds": ["<suiteId1>", "<suiteId2>"]
62
+ }' \
63
+ --target-org <doce-org-alias>
64
+ ```
65
+
66
+ | Field | Type | Description |
67
+ |---|---|---|
68
+ | `stageId` | string | The ID of the pipeline stage to execute tests on |
69
+ | `event` | string | `Pre-Promote`, `Post-Promote`, or `Review` |
70
+ | `testSuiteIds` | string[] | One or more test suite IDs to execute |
71
+
72
+ ### On success
73
+
74
+ Extract the `runId` (execution ID) from the response. Inform the user:
75
+
76
+ > "Tests are running in `<doce-org-alias>`. I'll update you when results are ready."
77
+
78
+ Then proceed **immediately** to Step 2 (polling) with the `runId`.
79
+
80
+ ### On error
81
+
82
+ See `references/error-handling.md`. If the org rejects execution (e.g. `environmentId: null`, or `classIdList is null or empty — no tests to execute`), read the actual error, explain the root cause and required fix in plain language, and finish cleanly. **Do not retry in a loop and do not fabricate a `runId` or results.**
83
+
84
+ ---
85
+
86
+ ## Step 2 — Poll until completion
87
+
88
+ **Confirmation required:** No — polling is automatic and read-only.
89
+
90
+ Poll the execution record by `runId` at the provider-appropriate interval. Full intervals, timeout behavior, and the poll query are in `references/polling-configuration.md`.
91
+
92
+ Summary of the loop (the `runId` is a `DevopsTestSuiteExecution` Id — poll that object, not `DevopsTestExecution`):
93
+ - Query `DevopsTestSuiteExecution` by `runId` each interval for `Status, Coverage, SuccessCount, FailureCount, QualityGateStatus`.
94
+ - `InProgress` → wait and poll again.
95
+ - `Passed` / `Failed` → surface `Coverage`, `SuccessCount`, `FailureCount`, and `QualityGateStatus` inline (no raw JSON). If `FailureCount > 0`, fetch the child `DevopsTestExecution` failure rows and hand off to **`dx-devops-test-failures-analyze`**.
96
+ - `Error` → the run itself errored (not test failures); surface `ResultDetails`/`Message` in plain language and offer retry or skip.
97
+ - **Timeout** → surface the `runId`, do NOT auto-retry, wait for user instruction.
98
+
99
+ ---
100
+
101
+ ## Retrigger mode (re-running a quality gate)
102
+
103
+ Use when a promotion was blocked by a gate failure and the coverage gap has since been addressed. **All preconditions, gate, and the retrigger API call are in `references/retrigger-mode.md`.** Key rule: do **not** retrigger unless the latest `Coverage` meets or exceeds the `DevopsQualityGateRule` threshold. After the retrigger returns a new `runId`, hand it to Step 2 (polling).
104
+
105
+ ---
106
+
107
+ ## Related skills
108
+
109
+ - **`dx-devops-test-failures-analyze`** — receives the failure payload on completion; can also create a fix work item.
110
+ - **`dx-devops-test-suite-assignments-configure`** — recommend which suites to run, or assign a suite to the stage if it isn't linked yet.
111
+ - **`dx-devops-test-pipeline-configure`** — configure a new quality gate or threshold (this skill only re-runs existing gates).
@@ -0,0 +1,31 @@
1
+ # Error Handling — Execution & Polling
2
+
3
+ Never expose raw API errors, stack traces, or JSON payloads to the user.
4
+
5
+ ## Step 1 — Execute call
6
+
7
+ | Status | Message to user |
8
+ |---|---|
9
+ | 400 | "The test execution request was invalid. Check that the stage and suite IDs are correct." |
10
+ | 403 | "You don't have permission to run tests on this pipeline. Check your DevOps Testing API access." |
11
+ | 404 | "The pipeline or stage was not found. It may have been deleted." |
12
+ | 500 | "The DevOps Center org returned a server error. Try again in a few minutes." |
13
+
14
+ ## Org-side execution rejections (read the real error, explain, finish — do NOT loop)
15
+
16
+ | Org error | Explanation to user |
17
+ |---|---|
18
+ | `environmentId: null` | "The environment connection isn't resolving for this stage. Re-authenticate or reconnect the stage's environment in DevOps Center, then try again." |
19
+ | `classIdList is null or empty — no tests to execute` | "The assigned suite(s) contain no test classes, so there's nothing to run. Add test classes to the suite (via dx-devops-test-suite-assignments-configure) and try again." |
20
+
21
+ In both cases: attempt the call once, explain the root cause and required fix, finish cleanly. Do NOT retry in a loop and do NOT fabricate a `runId` or results.
22
+
23
+ **Empty-org / no-data case:** If the stage has no assigned suites, report that clearly and do NOT call the execute endpoint or fabricate a suite/run.
24
+
25
+ ## Step 2 — Polling
26
+
27
+ If no `DevopsTestExecution` record matches the runId, report it gracefully and do not fabricate result data. On timeout, surface the runId and wait for user instruction — never auto-retry.
28
+
29
+ ## Do NOT use `sf apex run test`
30
+
31
+ That command is for the `platform-apex-test-run` skill, not DevOps Center test suites.
@@ -0,0 +1,78 @@
1
+ # Polling Configuration (Step 2)
2
+
3
+ **Confirmation required:** No — polling is automatic and read-only. No user confirmation gate is needed.
4
+
5
+ **What it does:** Polls the DevOps Center org for the status of an async test execution until it completes, times out, or fails.
6
+
7
+ ## Prerequisites
8
+
9
+ You need an active `runId` (from Step 1, the execute call) and the confirmed `doce-org-alias`. If org context is not yet established, run Prerequisites 1–3 (`references/prerequisite-checks.md`).
10
+
11
+ ## Inputs
12
+
13
+ | Input | Source |
14
+ |---|---|
15
+ | `runId` | Returned by the Step 1 execute call (or supplied by the user for an in-progress run) |
16
+ | `testType` | Derived from the suite's test provider (Apex, Code Analyzer, UI/Provar, Flow) |
17
+ | `doce-org-alias` | Established via prerequisites |
18
+
19
+ If the user is watching an already-running execution, resolve the `runId` by querying the most recent `DevopsTestExecution` for the relevant trigger, or use the runId the user provides.
20
+
21
+ ## Polling intervals
22
+
23
+ | Test type | Poll interval | Max wait | Timeout action |
24
+ |---|---|---|---|
25
+ | Apex unit tests | 15 seconds | 5 minutes | Surface runId, offer retry |
26
+ | Code Analyzer | 10 seconds | 3 minutes | Surface runId, offer retry |
27
+ | UI tests (Provar) | 60 seconds | 20 minutes | Surface runId, mark as pending |
28
+ | Flow tests | 20 seconds | 8 minutes | Surface runId, offer retry |
29
+
30
+ ## Object model — read the right object
31
+
32
+ The `runId` returned by the execute call is a **`DevopsTestSuiteExecution`** record Id (the suite-level run). Poll *that* object — it holds the status and the aggregate results. Per-test detail lives on child `DevopsTestExecution` records linked by `DevopsTestSuiteExecutionId`.
33
+
34
+ | Object | Role | Key fields |
35
+ |---|---|---|
36
+ | `DevopsTestSuiteExecution` (poll this) | Suite-level run = the `runId` | `Status`, `Coverage`, `SuccessCount`, `FailureCount`, `SuccessRate`, `FailureRate`, `QualityGateStatus`, `ExecutionEndTime`, `ResultDetails`, `ReportUrl` |
37
+ | `DevopsTestExecution` (per-test detail) | One row per test, linked via `DevopsTestSuiteExecutionId` | `Status`, `Message`, `Severity`, `ResultDetails`, `DevopsTestId` |
38
+
39
+ > Do NOT query `TestsRan`, `TestsPassed`, `TestsFailed`, or `CoveragePercentage` — those fields do not exist on either object and the query will fail with `INVALID_FIELD`. Use the field names in the table above.
40
+
41
+ ## Poll query
42
+
43
+ Query the suite execution record by `runId` on each interval:
44
+
45
+ ```bash
46
+ sf data query \
47
+ --query "SELECT Id, Status, Coverage, SuccessCount, FailureCount, QualityGateStatus FROM DevopsTestSuiteExecution WHERE Id = '<runId>' LIMIT 1" \
48
+ --target-org <doce-org-alias> \
49
+ --json
50
+ ```
51
+
52
+ Check the `Status` picklist on each poll (valid values: `Passed`, `Failed`, `InProgress`, `Error`):
53
+ - `InProgress` → wait and poll again
54
+ - `Passed` → run completed successfully; surface results
55
+ - `Failed` → run completed with test failures; surface results, then fetch per-test detail (below) and hand off to analysis
56
+ - `Error` → the run itself errored (not a test failure); surface the `ResultDetails`/`Message` in plain language and offer retry or skip
57
+
58
+ ## On timeout
59
+
60
+ Surface the `runId` to the user:
61
+ > "The test run is taking longer than expected. Your run ID is `<runId>`. You can check the status manually in DevOps Center, or I can keep waiting — what would you prefer?"
62
+
63
+ Do not automatically retry after timeout. Wait for user instruction.
64
+
65
+ ## On completion
66
+
67
+ When `Status` is `Passed` or `Failed`, surface `Coverage`, `SuccessCount`, `FailureCount`, and `QualityGateStatus` from the `DevopsTestSuiteExecution` record inline (no raw JSON).
68
+
69
+ If `FailureCount > 0` (or `Status = Failed`), fetch the per-test failure detail from the child `DevopsTestExecution` records, then pass that payload to the **`dx-devops-test-failures-analyze`** skill:
70
+
71
+ ```bash
72
+ sf data query \
73
+ --query "SELECT Id, Status, Message, Severity, ResultDetails, DevopsTestId FROM DevopsTestExecution WHERE DevopsTestSuiteExecutionId = '<runId>' AND Status = 'Failed'" \
74
+ --target-org <doce-org-alias> \
75
+ --json
76
+ ```
77
+
78
+ **Empty / no-data case:** If no `DevopsTestSuiteExecution` record matches the runId, report that clearly and do NOT fabricate a runId, status, or result values.
@@ -0,0 +1,112 @@
1
+ # Prerequisite Checks — Shared DevOps Center Gate
2
+
3
+ Shared environment gate for every DevOps Center testing skill. Run these checks **before** any query or system call. On any failure, surface the plain-language message and stop until the user resolves it — never proceed to a write with an unverified environment.
4
+
5
+ > **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
6
+
7
+ **Important:** All DevOps Center data (pipelines, stages, test suites, executions) lives in the Salesforce org — NOT in the local repository. Never search the filesystem for pipeline configuration. Always query the org using `sf data query` or `sf api request rest`.
8
+
9
+ **Object model — use the STANDARD objects, never the `sf_devops__` managed package:** DevOps Center testing data lives in standard platform objects — `DevopsPipeline`, `DevopsPipelineStage`, `DevopsPipelineStageTrigger`, `DevopsTestSuite`, `DevopsTestSuiteStage`, `DevopsTestSuiteExecution`, `DevopsProject`, `WorkItem`. Do **NOT** query the legacy managed-package objects (`sf_devops__Pipeline__c`, `sf_devops__Pipeline_Stage__c`, etc.) and do **NOT** gate on a `PackageLicense` / namespace check for `sf_devops`. "DevOps Center installed" is determined **solely** by whether `DevopsPipeline` is queryable and returns records (Prerequisite 4) — never by the presence of the `sf_devops__` namespace. If a `sf_devops__*` object is missing, that is expected and is NOT evidence that DevOps Center is uninstalled.
10
+
11
+ ## How skills use this
12
+
13
+ Run Prerequisites 1–4 in order. Prerequisite 5 (stage) is run **only** when the operation targets a specific pipeline stage (configuring a gate, running/retriggering a suite, mapping a suite). Carry forward the resolved `doce-org-alias`, `pipelineId`, and (when applicable) `stageId`.
14
+
15
+ Resolve the **DevOps Center org alias** without asking the user unless genuinely ambiguous:
16
+ 1. If the user named an org alias in their message, use it.
17
+ 2. Otherwise, use the default org (`sf org display --json`, no `--target-org`).
18
+ 3. Only if the default org has no `DevopsPipeline` records (Prereq 4 fails), ask: "Which org alias is your DevOps Center org?"
19
+
20
+ ---
21
+
22
+ ## Prerequisite 1 — Salesforce org: active login
23
+
24
+ ```bash
25
+ sf org list --json
26
+ ```
27
+
28
+ Look for at least one entry in `result.nonScratchOrgs`, `result.scratchOrgs`, or `result.sandboxes` with `"connectedStatus": "Connected"`.
29
+
30
+ - **Pass:** at least one org is Connected
31
+ - **Fail:** no orgs listed, or all show a non-connected status
32
+
33
+ **On fail:** "No authenticated Salesforce org found. Run `sf org login web --alias <your-alias>` in your terminal, then come back."
34
+
35
+ ---
36
+
37
+ ## Prerequisite 2 — Agentforce DX plugin installed
38
+
39
+ ```bash
40
+ sf plugins --json
41
+ ```
42
+
43
+ Look for a plugin entry whose `name` contains `plugin-agent`, `agentforce`, or `einstein` (case-insensitive).
44
+
45
+ - **Pass:** plugin found
46
+ - **Fail:** no matching entry
47
+
48
+ **On fail:** "The Agentforce DX Plugin is not installed. Run `sf plugins install @salesforce/plugin-agent`, then restart the IDE and try again."
49
+
50
+ ---
51
+
52
+ ## Prerequisite 3 — DevOps Center org authenticated
53
+
54
+ ```bash
55
+ sf org display --target-org <doce-org-alias> --json
56
+ ```
57
+
58
+ Check that `"connectedStatus"` is `"Connected"`.
59
+
60
+ - **Pass:** Connected
61
+ - **Fail:** expired session or error
62
+
63
+ **On fail:** "Your DevOps Center org session has expired. Run `sf org login web --alias <doce-org-alias>` to re-authenticate."
64
+
65
+ ---
66
+
67
+ ## Prerequisite 4 — Pipeline identified
68
+
69
+ ```bash
70
+ sf data query \
71
+ --query "SELECT Id, Name, CreatedDate FROM DevopsPipeline ORDER BY Name ASC" \
72
+ --target-org <doce-org-alias> \
73
+ --json
74
+ ```
75
+
76
+ - **Pass (exactly one):** use it automatically — do NOT ask.
77
+ - **Pass (multiple):** if the user already named a pipeline, match by name. Otherwise display a numbered list and ask:
78
+ ```text
79
+ Found <N> pipelines:
80
+ 1. <Name>
81
+ 2. <Name>
82
+ Which pipeline would you like to work with?
83
+ ```
84
+ - **Fail (no records):** "No DevOps Center pipeline found. Create a project and pipeline in DevOps Center before using the DevOps Testing Skills."
85
+ - **Fail (unsupported object):** "DevOps Center does not appear to be installed on `<doce-org-alias>`. Check that you're pointing at the correct org."
86
+
87
+ ---
88
+
89
+ ## Prerequisite 5 — Pipeline stage identified (conditional)
90
+
91
+ Run **only** when the operation targets a specific stage (e.g. configuring a quality gate).
92
+
93
+ If the user's message already names a stage (e.g. "Integration", "Staging", "Production"), use that name directly — do NOT ask again. Look up its Id:
94
+
95
+ ```bash
96
+ sf data query \
97
+ --query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' AND Name = '<stageName>'" \
98
+ --target-org <doce-org-alias> --json
99
+ ```
100
+
101
+ Only if no stage is mentioned, fetch the full list and ask which one:
102
+
103
+ ```bash
104
+ sf data query \
105
+ --query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' ORDER BY Name ASC" \
106
+ --target-org <doce-org-alias> --json
107
+ ```
108
+
109
+ Then ask: "Which pipeline stage are we working with?" — do NOT ask for an org alias; stages are resolved by name from the pipeline.
110
+
111
+ - **Pass:** stage Id and Name confirmed
112
+ - **Deferred:** not required until the operation needs it
@@ -0,0 +1,51 @@
1
+ # Retrigger Mode — Re-running a Quality Gate
2
+
3
+ Use this mode when a promotion was blocked by a quality gate failure and the coverage gap has since been addressed.
4
+
5
+ ## Extra preconditions — all must be true before proceeding
6
+
7
+ 1. The `Coverage` field on the latest `DevopsTestSuiteExecution` meets or exceeds the threshold defined in the `DevopsQualityGateRule`.
8
+ 2. The user has explicitly asked to retrigger the gate.
9
+ 3. The same `pipelineId`, `stageId`, and `event` from the blocked promotion are known.
10
+
11
+ If coverage is still below threshold, do **not** retrigger. Instead respond:
12
+
13
+ > "Coverage is still at `<X>%`, below the `<threshold>%` gate. The gate cannot be retriggered until the threshold is met. Here are the remaining uncovered methods: `<list>`."
14
+
15
+ Do not retry. Explain what must be resolved first and stop.
16
+
17
+ ## Inputs required for retrigger
18
+
19
+ | Input | Source |
20
+ |---|---|
21
+ | `pipelineId` | From the blocked promotion context |
22
+ | `stageId` | From the blocked promotion context |
23
+ | `event` | Same event type that originally blocked (`Pre-Promote` or `Post-Promote`) |
24
+ | `suiteIds` | Same suites that were originally run |
25
+ | `doce-org-alias` | Prerequisite 1 |
26
+
27
+ ## Confirmation gate (retrigger)
28
+
29
+ Before executing the API call, present this confirmation prompt and wait for explicit user approval:
30
+
31
+ > "Coverage is confirmed at `<X>%`, which meets the `<threshold>%` gate. I'll retrigger the quality gate check for the `<stageName>` stage (`<event>`). Confirm?"
32
+
33
+ Only proceed after the user confirms. If the user declines, stop without making any API call.
34
+
35
+ ## API call (retrigger)
36
+
37
+ Uses the same Connect API stage/execute endpoint:
38
+
39
+ ```bash
40
+ sf api request rest "/services/data/v67.0/connect/devopstesting/pipeline/<pipelineId>/stage/execute" --method POST --body '{"stageId":"<stageId>","event":"<event>","testSuiteIds":["<suiteId1>"]}' --target-org <doce-org-alias>
41
+ ```
42
+
43
+ After the call returns a `runId`, hand off to Step 2 (polling) with the new `runId` to monitor the execution result.
44
+
45
+ ## Error handling (retrigger)
46
+
47
+ If the API returns an error indicating the gate cannot be retriggered, respond with:
48
+
49
+ > "The quality gate cannot be retriggered right now. Reason: `<plain-language summary>`. Here's what needs to be resolved first: `<list>`."
50
+
51
+ Never expose raw API error details to the user.
@@ -1,159 +0,0 @@
1
- ---
2
- name: analyzing-test-failures
3
- description: "Parses a DevOps Center test-failure or Code Analyzer violation payload and explains it in plain language — failure category, offending file/class/method/line, rule violated, and fix direction — then gives prioritized test-improvement suggestions (distinguishing test-code from production-code fixes). Pure reasoning: no system calls or code authoring. TRIGGER when: a test run failed and the user wants root cause; a quality gate failure needs explaining; Code Analyzer violations need translating to plain language; the user shares a failure payload and asks how to address it; tests keep failing and the user wants suggestions; or the user wants to close coverage gaps, strengthen assertions, or fix flaky/weak tests. DO NOT TRIGGER when: the user wants fix code written (use platform-apex-generate) or new test classes authored (use platform-apex-test-generate)."
4
- metadata:
5
- version: "1.0"
6
- minApiVersion: "67.0"
7
- ---
8
-
9
- # analyzing-test-failures
10
-
11
- **Type:** Pure reasoning — no system calls, no code authoring.
12
-
13
- **What it does:** Parses a test failure or Code Analyzer violation payload and produces a plain-language explanation, then follows up with concrete, prioritized improvement suggestions per failed test — including whether each fix belongs in the test or in production code. Never exposes raw JSON, stack traces, or API error bodies to the user.
14
-
15
- ---
16
-
17
- ## Prerequisites
18
-
19
- If you need to fetch the failure payload yourself rather than receiving it, load `checking-devops-prerequisites` first, then use `polling-test-results` to obtain the execution result. If the payload is already in context, no prerequisites are needed — this is pure reasoning.
20
-
21
- ---
22
-
23
- ## Inputs
24
-
25
- - JSON failure payload provided by the `polling-test-results` skill, or pasted directly into context from a test run or Code Analyzer execution. Specifically required per failed test:
26
- - Test method name
27
- - Failure message (the assertion error or exception text)
28
- - Failure category (assertion failure, unhandled exception, timeout, compile error)
29
-
30
- ---
31
-
32
- ## Empty-org / no-data case
33
-
34
- If the payload contains no failures or violations, report that clearly (e.g. "No failures found in the provided execution results.") and stop. Do NOT fabricate failures, violations, or improvement suggestions when none are present.
35
-
36
- ---
37
-
38
- ## Reasoning steps
39
-
40
- 1. Determine the failure category:
41
-
42
- | Category | Description |
43
- |---|---|
44
- | **Assertion failure** | A test assertion failed (expected vs actual mismatch) |
45
- | **Exception** | An unhandled exception was thrown |
46
- | **Code Analyzer violation** | A static analysis rule was violated (e.g. `ApexCRUDViolation`, `ApexSharingViolations`) |
47
- | **Timeout** | Test exceeded execution time limit |
48
- | **Compile error** | Class failed to compile |
49
-
50
- 2. For each failure, extract and translate to plain language:
51
- - Offending file and class name
52
- - Method name
53
- - Line number
54
- - What rule or assertion was violated, in plain language
55
- - Suggested fix direction (without writing code)
56
-
57
- 3. Group failures by category if more than one.
58
-
59
- ---
60
-
61
- ## Output format
62
-
63
- ```text
64
- Test failure summary:
65
-
66
- <N> failure(s) found:
67
-
68
- 1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
69
- What happened: <plain-language description>
70
- Rule violated: <ruleName or assertion description>
71
- Fix direction: <plain-language suggestion>
72
-
73
- 2. [<Category>] `<ClassName2>.cls` — `<methodName2>()` at line <N>
74
- ...
75
- ```
76
-
77
- ---
78
-
79
- ## Code Analyzer violations
80
-
81
- For violations, always include:
82
- - The rule name translated to plain English (e.g. "ApexCRUDViolation" → "A SOQL query was made without checking object-level permissions first")
83
- - The exact line number
84
- - The fix direction (e.g. "Add a `Schema.sObjectType.Account.isAccessible()` check before the query")
85
-
86
- ---
87
-
88
- ## Plain-language rule
89
-
90
- Never paste raw stack traces, JSON payloads, or internal Salesforce error codes into the output. Always translate to file name, method, line, and plain description.
91
-
92
- ---
93
-
94
- ## Suggesting improvements
95
-
96
- Invoke this section **after test execution completes with failures**, not on static source code. The failure message is the primary signal — it describes what the test expected vs. what actually happened, which directly informs what the test needs to handle better.
97
-
98
- ### 1 — Read each failure message
99
-
100
- For each failed test in the execution result, extract:
101
- - Test method name
102
- - Failure message (the assertion error or exception message)
103
- - Failure category (assertion failure, unhandled exception, timeout, compile error)
104
-
105
- ### 2 — Infer what the test is not handling
106
-
107
- Reason over the failure message to identify the root cause pattern:
108
-
109
- | Failure pattern | Improvement suggestion |
110
- |---|---|
111
- | `NullPointerException` | The test is not handling null input — add a null check or a test setup that ensures the data exists |
112
- | `Assertion failed: expected X but was Y` | The expected value in the assertion is wrong or the test data setup does not produce the right state |
113
- | `List has no rows for assignment` | The test is querying for data that doesn't exist — test setup is incomplete |
114
- | `System.LimitException: Too many SOQL queries` | The test is hitting governor limits — the code under test or test setup is making too many queries |
115
- | `Insufficient access rights on cross-reference id` | The test user lacks the required permissions — the test needs to run as a user with appropriate profile/permission set |
116
- | `DML currently not allowed` | The test is performing DML inside a method called from a context that doesn't allow it |
117
- | Code Analyzer violation message | The production code violates a specific rule — the test exposed it but the fix is in the production code, not the test |
118
-
119
- ### 3 — Produce actionable suggestions
120
-
121
- For each failure, describe in plain language:
122
- - What the failure reveals about what the test is not handling
123
- - What specifically should be added or changed to make the test robust
124
- - Whether the fix is in the **test** (assertion, setup, permissions) or in the **production code** (the test is correct but the code under test is broken)
125
-
126
- Do not rewrite the test. Only describe what needs to change and why.
127
-
128
- ### Output format for improvements
129
-
130
- ```text
131
- Test improvement suggestions based on execution results:
132
-
133
- `<testMethodName>()` — [Assertion Failure / Exception / etc.]
134
- Failure: "<failure message>"
135
- What this reveals: <plain-language explanation>
136
- Suggestion: <specific, actionable recommendation>
137
- Fix location: Test | Production code
138
-
139
- `<testMethodName2>()` — [NullPointerException]
140
- Failure: "Attempt to de-reference a null object"
141
- What this reveals: The test is calling the method without setting up the required input data
142
- Suggestion: Add test data setup for <object/field> before calling the method
143
- Fix location: Test
144
-
145
- Overall: <N> improvement(s) across <M> failed test(s).
146
- ```
147
-
148
- ### Test fix vs. production-code fix
149
-
150
- Suggestions flagged as **Fix location: Production code** indicate a code defect exposed by the test — the test logic itself is sound. These should not block suite promotion on the grounds of test quality; they should be tracked separately as production defects.
151
-
152
- Suggestions flagged as **Fix location: Test** indicate the test needs to be hardened — missing setup, wrong assertions, inadequate coverage of edge cases (null inputs, bulk record volumes, mixed permission contexts, governor-limit boundaries).
153
-
154
- ---
155
-
156
- ## Related skills
157
-
158
- - **`polling-test-results`** — obtains the failure payload that feeds into this skill.
159
- - **`creating-fix-work-item`** — use to create a tracked fix item from the analysis output or to track an approved improvement suggestion as a work item once the user decides to act on it.