@keber/qa-framework 1.11.3 → 1.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/docs/architecture.md +30 -26
- package/docs/decisions/lane-heartbeat-caller.md +86 -0
- package/docs/final-report.md +23 -23
- package/docs/folder-structure-guide.md +100 -80
- package/docs/installation.md +35 -7
- package/docs/skills-architecture.md +23 -23
- package/docs/spec-driven-philosophy.md +9 -9
- package/docs/usage-with-agent.md +83 -9
- package/docs/visuals/diag-01-ejecutivo-1.png +0 -0
- package/docs/visuals/diag-02-tactico-1.png +0 -0
- package/docs/visuals/diag-03-tecnico-1.png +0 -0
- package/docs/visuals/info-01-ejecutivo-1.png +0 -0
- package/docs/visuals/info-02-tactico-1.png +0 -0
- package/docs/visuals/info-03-tecnico-1.png +0 -0
- package/docs/visuals/pres-01-ejecutivo-1.png +0 -0
- package/docs/visuals/pres-02-tactico-1.png +0 -0
- package/docs/visuals/pres-03-tecnico-1.png +0 -0
- package/docs/visuals/qa-01-problema-1.png +0 -0
- package/docs/visuals/qa-02-que-es-1.png +0 -0
- package/docs/visuals/qa-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-04-6-archivos-1.png +0 -0
- package/docs/visuals/qa-05-humano-1.png +0 -0
- package/docs/visuals/qa-06-agente-1.png +0 -0
- package/docs/visuals/qa-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-08-traceability-1.png +0 -0
- package/docs/visuals/qa-en-01-problem-1.png +0 -0
- package/docs/visuals/qa-en-02-what-is-1.png +0 -0
- package/docs/visuals/qa-en-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-en-04-6files-1.png +0 -0
- package/docs/visuals/qa-en-05-human-1.png +0 -0
- package/docs/visuals/qa-en-06-agent-1.png +0 -0
- package/docs/visuals/qa-en-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-en-08-traceability-1.png +0 -0
- package/examples/module-example/suppliers/00-inventory.md +1 -1
- package/examples/module-example/suppliers/suppliers-create.spec.ts +8 -8
- package/integrations/ado-powershell/README.md +1 -1
- package/integrations/ado-powershell/scripts/sync-ado-titles.ps1 +2 -2
- package/integrations/playwright/README.md +4 -4
- package/package.json +7 -3
- package/qa-framework.config.json +23 -2
- package/scripts/check-forbidden-chars.js +187 -0
- package/scripts/check-spanish-accents.js +159 -0
- package/scripts/cli.js +1 -1
- package/scripts/generate.js +2 -2
- package/scripts/init.js +55 -7
- package/scripts/lib/claude-agents.js +78 -0
- package/scripts/lib/claude-commands.js +56 -0
- package/scripts/sync-version.js +36 -0
- package/scripts/upgrade.js +356 -20
- package/scripts/validate.js +17 -17
- package/skills/qa-ado-integration/SKILL.md +19 -19
- package/skills/qa-ado-integration/references/scripts-and-config.md +6 -6
- package/skills/qa-automation/SKILL.md +49 -27
- package/skills/qa-automation/references/config-checklist.md +4 -4
- package/skills/qa-automation/references/dom-inspection-template.js +7 -7
- package/skills/qa-automation/references/patterns.md +20 -20
- package/skills/qa-automation/references/pom-template.md +161 -0
- package/skills/qa-maintenance/SKILL.md +11 -11
- package/skills/qa-maintenance/references/update-rules.md +8 -8
- package/skills/qa-module-analysis/SKILL.md +20 -19
- package/skills/qa-module-analysis/references/exploration-checklist.md +4 -4
- package/skills/qa-module-analysis/references/output-verification.md +69 -0
- package/skills/qa-module-analysis/references/spec-file-formats.md +14 -14
- package/skills/qa-spec-generation/SKILL.md +13 -7
- package/skills/qa-test-cases/SKILL.md +11 -81
- package/skills/qa-test-cases/references/test-case-template.md +5 -5
- package/skills/qa-test-plan/SKILL.md +22 -86
- package/skills/qa-test-plan/references/plan-de-pruebas-template.md +24 -24
- package/skills/qa-test-stabilization/SKILL.md +59 -30
- package/skills/qa-test-stabilization/references/classification-protocol.md +35 -35
- package/templates/agent-next-steps.md +10 -10
- package/templates/agents/qa-analisis.md +142 -0
- package/templates/agents/qa-asesoria.md +48 -0
- package/templates/agents/qa-informe-resultados.md +234 -0
- package/templates/agents/qa-plan.md +131 -0
- package/templates/automation-scaffold/.env.example +36 -6
- package/templates/automation-scaffold/fixtures/auth.ts +27 -5
- package/templates/automation-scaffold/fixtures/base.ts +23 -1
- package/templates/automation-scaffold/fixtures/test-helpers.ts +1 -1
- package/templates/automation-scaffold/global-setup.ts +117 -48
- package/templates/automation-scaffold/playwright.config.ts +72 -32
- package/templates/automation-scaffold/scripts/global-setup-guards.js +184 -0
- package/templates/automation-scaffold/scripts/lane-config.js +269 -0
- package/templates/automation-scaffold/scripts/lane-lock.js +589 -0
- package/templates/defect-report.md +17 -3
- package/templates/execution-report.md +9 -9
- package/templates/integration-scaffold/.env.example +1 -1
- package/templates/integration-scaffold/global-setup.ts +21 -1
- package/templates/integration-scaffold/playwright.config.ts +15 -1
- package/templates/qa-framework.instructions.md +8 -7
- package/templates/qa-framework.rules.md +53 -0
- package/templates/qa-readme.md +8 -8
- package/templates/session-summary.md +2 -2
- package/templates/specification/00-inventory.md +3 -3
- package/templates/specification/01-business-rules.md +6 -6
- package/templates/specification/02-workflows.md +1 -1
- package/templates/specification/03-roles-permissions.md +1 -1
- package/templates/specification/04-test-data.md +8 -8
- package/templates/specification/05-test-scenarios.md +5 -5
- package/templates/test-case.md +3 -3
- package/templates/test-data-guidelines.md +7 -7
- package/templates/test-plan-sprint.md +13 -13
- package/templates/test-plan.md +3 -3
- package/skills/qa-test-plan/references/priority-and-feasibility.md +0 -159
|
@@ -87,7 +87,27 @@ async function setupWithBrowserLogin(): Promise<void> {
|
|
|
87
87
|
|
|
88
88
|
await page.locator(emailSelector).waitFor({ state: 'visible', timeout: 30_000 });
|
|
89
89
|
await page.locator(emailSelector).fill(email);
|
|
90
|
-
|
|
90
|
+
// Trace safety: Playwright records fill() argument values in traces, so a project
|
|
91
|
+
// that enables trace/video capture would persist the plaintext password in its
|
|
92
|
+
// artifacts. Setting the value through evaluate() keeps it out of the trace.
|
|
93
|
+
// Do not "simplify" this back to fill().
|
|
94
|
+
// The input/change events are required because assigning .value directly does
|
|
95
|
+
// not notify SPA frameworks (Blazor/Radzen bind on those events).
|
|
96
|
+
// evaluate() has no auto-wait, so the explicit waitFor replaces the one that
|
|
97
|
+
// locator().fill() performed implicitly.
|
|
98
|
+
await page.locator(passwordSelector).waitFor({ state: 'visible', timeout: 30_000 });
|
|
99
|
+
await page.evaluate(
|
|
100
|
+
([selector, pwd]) => {
|
|
101
|
+
const input = document.querySelector(selector) as HTMLInputElement | null;
|
|
102
|
+
if (!input) {
|
|
103
|
+
throw new Error(`[qa-framework] Password input not found for selector: ${selector}`);
|
|
104
|
+
}
|
|
105
|
+
input.value = pwd;
|
|
106
|
+
input.dispatchEvent(new Event('input', { bubbles: true }));
|
|
107
|
+
input.dispatchEvent(new Event('change', { bubbles: true }));
|
|
108
|
+
},
|
|
109
|
+
[passwordSelector, password] as const
|
|
110
|
+
);
|
|
91
111
|
await page.locator(submitSelector).click();
|
|
92
112
|
await page.waitForSelector(successSelector, { timeout: 60_000 });
|
|
93
113
|
|
|
@@ -14,8 +14,22 @@
|
|
|
14
14
|
|
|
15
15
|
import { defineConfig } from '@playwright/test';
|
|
16
16
|
import * as dotenv from 'dotenv';
|
|
17
|
+
import * as fs from 'fs';
|
|
17
18
|
|
|
18
|
-
dotenv.
|
|
19
|
+
// Least privilege: read only the keys this config actually uses. dotenv.parse() returns
|
|
20
|
+
// a plain object without touching process.env, so the other secrets in .env - the other
|
|
21
|
+
// QA accounts, ADO tokens - are never loaded into a process that has no use for them.
|
|
22
|
+
// QA_USER_PASSWORD is included because the credentials check below reads it; its value
|
|
23
|
+
// is only tested for presence, never logged. Do not "simplify" this back to config().
|
|
24
|
+
const ENV_KEYS = ['QA_BASE_URL', 'QA_API_TOKEN', 'QA_USER_EMAIL', 'QA_USER_PASSWORD'] as const;
|
|
25
|
+
if (fs.existsSync('.env')) {
|
|
26
|
+
const parsed = dotenv.parse(fs.readFileSync('.env'));
|
|
27
|
+
for (const key of ENV_KEYS) {
|
|
28
|
+
if (process.env[key] === undefined && parsed[key] !== undefined) {
|
|
29
|
+
process.env[key] = parsed[key];
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
}
|
|
19
33
|
|
|
20
34
|
const required = ['QA_BASE_URL'];
|
|
21
35
|
for (const key of required) {
|
|
@@ -8,7 +8,7 @@ This project uses `@keber/qa-framework` v{{VERSION}} for spec-driven automated t
|
|
|
8
8
|
## Agent behavior rules
|
|
9
9
|
|
|
10
10
|
0. **On every conversation start:** check if `qa/AGENT-NEXT-STEPS.md` exists. If it does, read it and complete its steps before anything else.
|
|
11
|
-
1. Before performing any QA task, load the relevant skill from `.github/skills/`. For project context, read `qa/memory/INDEX.md` if it exists, then load only the memory files relevant to the current task
|
|
11
|
+
1. Before performing any QA task, load the relevant skill from `.github/skills/`. For project context, read `qa/memory/INDEX.md` if it exists, then load only the memory files relevant to the current task - do not load all memory files unconditionally. Never execute a pipeline stage unless its prerequisite is satisfied.
|
|
12
12
|
2. Always save artifacts in the correct `qa/` subfolder - refer to `qa/QA-STRUCTURE-GUIDE.md`
|
|
13
13
|
3. Never hardcode credentials - always use env vars and `<PLACEHOLDER>` in documentation
|
|
14
14
|
4. Follow the naming conventions in `qa/00-standards/naming-conventions.md`
|
|
@@ -16,9 +16,10 @@ This project uses `@keber/qa-framework` v{{VERSION}} for spec-driven automated t
|
|
|
16
16
|
6. Project QA config is at `qa/qa-framework.config.json`
|
|
17
17
|
7. Save learnings in `qa/memory/` with proper naming and metadata. Always update `qa/memory/INDEX.md` after adding or updating any memory file.
|
|
18
18
|
8. **Sprint/module completion criteria:** A sprint or module is complete when every TC in scope has one of these three valid states - and no other: (a) `test()` passing: the app meets the spec; (b) `test.fail()`: the app violates the spec **and** a file `qa/06-defects/open/DEF-SIS-NNN_*.md` exists documenting which business rule is violated and what the app actually does; (c) `test.skip()`: the TC is not yet executable, annotated with `PENDING-CODE` and a linked issue. A sprint is **not** complete if any TC is failing without `test.fail()`, or has `test.fail()` without a defect file, or has `test.skip()` without a `PENDING-CODE` reason.
|
|
19
|
-
9. **On sprint/module completion:** (1) update the module status row in `qa/README.md`; (2) move the completed sprint checklist from `AGENT-NEXT-STEPS.md` to the `## Sprint History` section of `qa/README.md`; (3) trim `AGENT-NEXT-STEPS.md` so it contains only the next sprint
|
|
20
|
-
10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use `test.fail()` + correct assertion to document the defect
|
|
19
|
+
9. **On sprint/module completion:** (1) update the module status row in `qa/README.md`; (2) move the completed sprint checklist from `AGENT-NEXT-STEPS.md` to the `## Sprint History` section of `qa/README.md`; (3) trim `AGENT-NEXT-STEPS.md` so it contains only the next sprint - never let it accumulate more than one active sprint.
|
|
20
|
+
10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use `test.fail()` + correct assertion to document the defect - **never invert or weaken an assertion to make a test go green.** A failing test that exposes a real defect is always more valuable than a passing test that hides one.
|
|
21
21
|
11. **Character encoding safety (BLOCKING):** In ALL generated content (markdown, scripts, test titles, ADO fields, config files), never use characters at these Unicode code points: em-dash (U+2014), en-dash (U+2013), horizontal ellipsis (U+2026), smart/curly quotes (U+201C, U+201D, U+2018, U+2019), or directional arrows (U+2190-U+21FF). Use ASCII equivalents instead: ` - ` (hyphen with surrounding spaces) for dashes, `...` for ellipsis, `"` and `'` for quotes, `->` and `<-` for arrows. Exception: Latin Extended characters (U+00C0-U+024F) are always permitted - this covers Spanish vowels with accents (U+00E0-U+00FA), n with tilde (U+00F1/U+00D1), and u/o with umlaut (U+00FC/U+00DC, U+00F6/U+00D6).
|
|
22
|
+
12. **Lane lock for parallel QA execution (BLOCKING):** Parallel QA lanes exist because the applications under test invalidate a session when the same account logs in again - two concurrent workers on one account silently destroy each other's session, and the damage surfaces later as an unexplained logout or as evidence recorded under the wrong account. Before dispatching any worker (sub-agent, test runner project, or CI job) against one of the lanes configured in `parallelLanes.lanes` in `qa/qa-framework.config.json`, **acquire a lane through the project's lane-lock script and release it when the work is done.** Never assign a lane by prompt or config alone without going through the lock. Use `acquire` when the worker does not care which lane it gets, and `reserve <laneId>` when it needs a specific predictable lane (a CI job or a local CLI run) - these are two verbs because there are two use cases, and collapsing them is how a CI job lands on a random lane. If no lane is free, queue (capped at the configured lane count) or reject - **never force a lane assignment past that cap.** Check that a lane actually has the capability a worker needs before handing it over: capabilities are per-lane, so a lane that lacks (for example) an MCP server must not be given to a worker that needs one. A held lane is swept as stale only after the configured threshold and only when no recent heartbeat proves the holder alive, so **the dispatcher must keep the lane alive with `heartbeat` while the work is in flight** - do not rely on the dispatched worker to do it, since a worker that was never told the mechanism exists cannot be expected to use it. A lock held far longer than the app's session TTL is a signal to verify, not a defect to ignore; each project's agent-specific instructions define how it verifies a lock is still legitimately held and how it clears a stale one.
|
|
22
23
|
|
|
23
24
|
## Formatting rules for generated content
|
|
24
25
|
1. Generate the output in `UTF-8` encoding without BOM (`UTF-8`, no signature).
|
|
@@ -27,7 +28,7 @@ This project uses `@keber/qa-framework` v{{VERSION}} for spec-driven automated t
|
|
|
27
28
|
## Azure DevOps integration
|
|
28
29
|
|
|
29
30
|
Before any ADO operation, check if `.github/skills/ado-qa/` exists or if
|
|
30
|
-
`integrations.
|
|
31
|
+
`integrations.azureDevOps.enabled` is `true` in `qa/qa-framework.config.json`. If either
|
|
31
32
|
condition is met, load `.github/skills/qa-ado-integration/SKILL.md` and use it for
|
|
32
33
|
all ADO interactions (work items, test plans, test cases, bugs, etc.).
|
|
33
34
|
|
|
@@ -46,8 +47,8 @@ Stages must run in order. Never start a stage unless its prerequisite is met.
|
|
|
46
47
|
| 5 | Generate automation | `.github/skills/qa-automation/SKILL.md` | Specs approved, no PENDING-CODE |
|
|
47
48
|
| 5b | Stabilize failing tests | `.github/skills/qa-test-stabilization/SKILL.md` | Failing tests exist |
|
|
48
49
|
| 6 | Maintenance | `.github/skills/qa-maintenance/SKILL.md` | Application change delivered |
|
|
49
|
-
|
|
|
50
|
+
| - | ADO integration | `.github/skills/qa-ado-integration/SKILL.md` | ADO enabled in config |
|
|
50
51
|
|
|
51
52
|
## Project KB
|
|
52
|
-
1. `qa/README.md`
|
|
53
|
-
2. `qa/memory/INDEX.md`
|
|
53
|
+
1. `qa/README.md` - The Living Index: module status, sprint history, blockers, quick-start commands.
|
|
54
|
+
2. `qa/memory/INDEX.md` - Index of all memory files; load this before selecting which files to read.
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# QA Framework Instructions
|
|
2
|
+
|
|
3
|
+
This project uses `@keber/qa-framework` v{{VERSION}} for spec-driven automated testing.
|
|
4
|
+
|
|
5
|
+
## Agent behavior rules
|
|
6
|
+
|
|
7
|
+
0. **On every conversation start:** check if `qa/AGENT-NEXT-STEPS.md` exists. If it does, read it and complete its steps before anything else.
|
|
8
|
+
1. Before performing any QA task, run the corresponding `/qa-{name}` command (see `.claude/commands/`), which loads the relevant skill via `.github/skills/qa-{name}/SKILL.md`. For project context, read `qa/memory/INDEX.md` if it exists, then load only the memory files relevant to the current task - do not load all memory files unconditionally. Never execute a pipeline stage unless its prerequisite is satisfied.
|
|
9
|
+
2. Always save artifacts in the correct `qa/` subfolder - refer to `qa/QA-STRUCTURE-GUIDE.md`
|
|
10
|
+
3. Never hardcode credentials - always use env vars and `<PLACEHOLDER>` in documentation
|
|
11
|
+
4. Follow the naming conventions in `qa/00-standards/naming-conventions.md`
|
|
12
|
+
5. Read previous test suites code in `qa/07-automation/e2e/tests/` if available before writing new automation to maintain consistency in style and approach
|
|
13
|
+
6. Project QA config is at `qa/qa-framework.config.json`
|
|
14
|
+
7. Save learnings in `qa/memory/` with proper naming and metadata. Always update `qa/memory/INDEX.md` after adding or updating any memory file.
|
|
15
|
+
8. **Sprint/module completion criteria:** A sprint or module is complete when every TC in scope has one of these three valid states - and no other: (a) `test()` passing: the app meets the spec; (b) `test.fail()`: the app violates the spec **and** a file `qa/06-defects/open/DEF-SIS-NNN_*.md` exists documenting which business rule is violated and what the app actually does; (c) `test.skip()`: the TC is not yet executable, annotated with `PENDING-CODE` and a linked issue. A sprint is **not** complete if any TC is failing without `test.fail()`, or has `test.fail()` without a defect file, or has `test.skip()` without a `PENDING-CODE` reason.
|
|
16
|
+
9. **On sprint/module completion:** (1) update the module status row in `qa/README.md`; (2) move the completed sprint checklist from `AGENT-NEXT-STEPS.md` to the `## Sprint History` section of `qa/README.md`; (3) trim `AGENT-NEXT-STEPS.md` so it contains only the next sprint - never let it accumulate more than one active sprint.
|
|
17
|
+
10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use `test.fail()` + correct assertion to document the defect - **never invert or weaken an assertion to make a test go green.** A failing test that exposes a real defect is always more valuable than a passing test that hides one.
|
|
18
|
+
11. **Character encoding safety (BLOCKING):** In ALL generated content (markdown, scripts, test titles, ADO fields, config files), never use characters at these Unicode code points: em-dash (U+2014), en-dash (U+2013), horizontal ellipsis (U+2026), smart/curly quotes (U+201C, U+201D, U+2018, U+2019), or directional arrows (U+2190-U+21FF). Use ASCII equivalents instead: ` - ` (hyphen with surrounding spaces) for dashes, `...` for ellipsis, `"` and `'` for quotes, `->` and `<-` for arrows. Exception: Latin Extended characters (U+00C0-U+024F) are always permitted - this covers Spanish vowels with accents (U+00E0-U+00FA), n with tilde (U+00F1/U+00D1), and u/o with umlaut (U+00FC/U+00DC, U+00F6/U+00D6).
|
|
19
|
+
|
|
20
|
+
## Formatting rules for generated content
|
|
21
|
+
1. Generate the output in `UTF-8` encoding without BOM (`UTF-8`, no signature).
|
|
22
|
+
2. Ensure the raw output is `UTF-8` encoded with no `EF BB BF` bytes at the beginning.
|
|
23
|
+
|
|
24
|
+
## Azure DevOps integration
|
|
25
|
+
|
|
26
|
+
Before any ADO operation, check if `.github/skills/ado-qa/` exists or if
|
|
27
|
+
`integrations.azureDevOps.enabled` is `true` in `qa/qa-framework.config.json`. If either
|
|
28
|
+
condition is met, run `/qa-ado-integration` (see `.claude/commands/`), which loads
|
|
29
|
+
`.github/skills/qa-ado-integration/SKILL.md`, and use it for all ADO interactions
|
|
30
|
+
(work items, test plans, test cases, bugs, etc.).
|
|
31
|
+
|
|
32
|
+
> To enable Azure DevOps integration run: `npm install github:keber/ado-qa`
|
|
33
|
+
|
|
34
|
+
## QA Pipeline
|
|
35
|
+
|
|
36
|
+
Stages must run in order. Never start a stage unless its prerequisite is met.
|
|
37
|
+
|
|
38
|
+
| Stage | Task | Command | Prerequisite |
|
|
39
|
+
|---|---|---|---|
|
|
40
|
+
| 1 | Analyze module | `/qa-module-analysis` | None |
|
|
41
|
+
| 2 | Generate specifications | `/qa-spec-generation` | 00-inventory.md exists |
|
|
42
|
+
| 3 | Generate test plan | `/qa-test-plan` | 05-test-scenarios.md exists |
|
|
43
|
+
| 4 | Generate test cases | `/qa-test-cases` | Test plan exists |
|
|
44
|
+
| 5 | Generate automation | `/qa-automation` | Specs approved, no PENDING-CODE |
|
|
45
|
+
| 5b | Stabilize failing tests | `/qa-test-stabilization` | Failing tests exist |
|
|
46
|
+
| 6 | Maintenance | `/qa-maintenance` | Application change delivered |
|
|
47
|
+
| - | ADO integration | `/qa-ado-integration` | ADO enabled in config |
|
|
48
|
+
|
|
49
|
+
Each command reads its full skill from `.github/skills/qa-{name}/SKILL.md` before acting - see `.claude/commands/` for the generated wrappers.
|
|
50
|
+
|
|
51
|
+
## Project KB
|
|
52
|
+
1. `qa/README.md` - The Living Index: module status, sprint history, blockers, quick-start commands.
|
|
53
|
+
2. `qa/memory/INDEX.md` - Index of all memory files; load this before selecting which files to read.
|
package/templates/qa-readme.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# QA
|
|
1
|
+
# QA - {PROJECT_DISPLAY_NAME}
|
|
2
2
|
|
|
3
3
|
> Framework: `@keber/qa-framework` v{VERSION}
|
|
4
4
|
> Language: {LANGUAGE} | Base URL: `{QA_BASE_URL}`
|
|
5
|
-
> ADO Project: `{ADO_ORG} / {ADO_PROJECT}` _(optional
|
|
5
|
+
> ADO Project: `{ADO_ORG} / {ADO_PROJECT}` _(optional - remove line if not using Azure DevOps)_
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
| Module | Submodule | TCs Total | TCs Automated | Plan | ADO Suite | Status | Last Run |
|
|
12
12
|
|---|---|---|---|---|---|---|---|
|
|
13
|
-
| {MODULE_NAME} | {SUBMODULE_NAME} |
|
|
13
|
+
| {MODULE_NAME} | {SUBMODULE_NAME} | - | - | ⬜ | - | 🔲 Not started | - |
|
|
14
14
|
|
|
15
15
|
**Legend**: ✅ Done · ⚠️ Partial · 🔲 Not started · ⛔ Blocked
|
|
16
16
|
|
|
@@ -44,7 +44,7 @@ npx playwright test --last-failed
|
|
|
44
44
|
|
|
45
45
|
| ID | Description | Affects | Opened | Status |
|
|
46
46
|
|---|---|---|---|---|
|
|
47
|
-
|
|
|
47
|
+
| - | No active blockers | - | - | - |
|
|
48
48
|
|
|
49
49
|
---
|
|
50
50
|
|
|
@@ -52,8 +52,8 @@ npx playwright test --last-failed
|
|
|
52
52
|
|
|
53
53
|
| Suite | Date | Pass | Fail | Skip | CI Link |
|
|
54
54
|
|---|---|---|---|---|---|
|
|
55
|
-
| E2E |
|
|
56
|
-
| API |
|
|
55
|
+
| E2E | - | - | - | - | - |
|
|
56
|
+
| API | - | - | - | - | - |
|
|
57
57
|
|
|
58
58
|
> Update this table after each significant run. The CI Link can be an ADO TestRun URL, a
|
|
59
59
|
> GitHub Actions run URL, or a path to a local report file.
|
|
@@ -62,7 +62,7 @@ npx playwright test --last-failed
|
|
|
62
62
|
|
|
63
63
|
| Date | Suite | Pass | Skip | Fail | Duration | Report |
|
|
64
64
|
|---|---|---|---|---|---|---|
|
|
65
|
-
|
|
|
65
|
+
| - | - | - | - | - | - | - |
|
|
66
66
|
|
|
67
67
|
## Flaky Tests
|
|
68
68
|
|
|
@@ -81,7 +81,7 @@ npx playwright test --last-failed
|
|
|
81
81
|
<!-- Completed sprint checklists are moved here from AGENT-NEXT-STEPS.md -->
|
|
82
82
|
<!-- Format per sprint: -->
|
|
83
83
|
<!--
|
|
84
|
-
### Sprint N
|
|
84
|
+
### Sprint N - {SPRINT_NAME} ({YYYY-MM-DD})
|
|
85
85
|
|
|
86
86
|
**Submodules**: {list}
|
|
87
87
|
**TCs automated**: {N} P0 + {M} P1
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Session Summary
|
|
1
|
+
# Session Summary - {{SESSION_TITLE}}
|
|
2
2
|
|
|
3
3
|
**Date**: YYYY-MM-DD
|
|
4
4
|
**Session type**: Module Analysis / Spec Generation / Test Plan / Automation / ADO Sync / Maintenance
|
|
@@ -44,7 +44,7 @@
|
|
|
44
44
|
|
|
45
45
|
| DEF-ID | Title | Severity | File opened? |
|
|
46
46
|
|--------|-------|----------|--------------|
|
|
47
|
-
| DEF-{{NNN}} | {{title}} | {{severity}} | Yes
|
|
47
|
+
| DEF-{{NNN}} | {{title}} | {{severity}} | Yes - `qa/06-defects/open/DEF-{{NNN}}.md` |
|
|
48
48
|
|
|
49
49
|
---
|
|
50
50
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# MODULE: {{MODULE_NAME}}
|
|
1
|
+
# MODULE: {{MODULE_NAME}} - Submodule: {{SUBMODULE_NAME}}
|
|
2
2
|
|
|
3
3
|
> **Template version**: 1.0 | Replace all `{{...}}` with actual values before committing.
|
|
4
4
|
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
|
|
19
19
|
| Level | Label | URL |
|
|
20
20
|
|-------|-------|-----|
|
|
21
|
-
| 1 | {{TOP_MENU}} |
|
|
21
|
+
| 1 | {{TOP_MENU}} | - |
|
|
22
22
|
| 2 | {{SUBMENU}} | `{{URL}}` |
|
|
23
23
|
|
|
24
24
|
---
|
|
@@ -71,7 +71,7 @@
|
|
|
71
71
|
| Submodule | Relationship | Notes |
|
|
72
72
|
|-----------|-------------|-------|
|
|
73
73
|
| {{submodule}} | Provides data to this module | Dropdown populated from {{API}} |
|
|
74
|
-
| {{submodule}} | Depends on data from this module |
|
|
74
|
+
| {{submodule}} | Depends on data from this module | - |
|
|
75
75
|
|
|
76
76
|
---
|
|
77
77
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Business Rules
|
|
1
|
+
# Business Rules - {{MODULE_NAME}}: {{SUBMODULE_NAME}}
|
|
2
2
|
|
|
3
3
|
> **Module code**: {{MODULE_CODE}} | **Submodule code**: {{SUBMODULE_CODE}}
|
|
4
4
|
> **Last updated**: YYYY-MM-DD
|
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
- **Type**: Access Control
|
|
33
33
|
- **Trigger**: When user with role {{role}} navigates to {{URL}}
|
|
34
34
|
- **Behavior**: {{redirect / error / empty page}}
|
|
35
|
-
- **Notes**:
|
|
35
|
+
- **Notes**: -
|
|
36
36
|
|
|
37
37
|
---
|
|
38
38
|
|
|
@@ -42,7 +42,7 @@
|
|
|
42
42
|
- **Trigger**: When {{action}} is performed on a record with status {{STATUS_A}}
|
|
43
43
|
- **Behavior**: Record transitions from {{STATUS_A}} to {{STATUS_B}}
|
|
44
44
|
- **Invalid state**: Attempting this action on status {{STATUS_C}} results in {{error}}
|
|
45
|
-
- **Notes**:
|
|
45
|
+
- **Notes**: -
|
|
46
46
|
|
|
47
47
|
---
|
|
48
48
|
|
|
@@ -51,7 +51,7 @@
|
|
|
51
51
|
- **Type**: Calculation
|
|
52
52
|
- **Trigger**: When {{field}} or {{field}} changes
|
|
53
53
|
- **Behavior**: {{calculated_field}} = {{formula}}
|
|
54
|
-
- **Notes**:
|
|
54
|
+
- **Notes**: -
|
|
55
55
|
|
|
56
56
|
---
|
|
57
57
|
|
|
@@ -59,9 +59,9 @@
|
|
|
59
59
|
|
|
60
60
|
- **Type**: Integration
|
|
61
61
|
- **Trigger**: When {{action}} completes successfully
|
|
62
|
-
- **Behavior**: {{external system}} is notified/called
|
|
62
|
+
- **Behavior**: {{external system}} is notified/called - {{what happens}}
|
|
63
63
|
- **Fallback**: {{what happens if external system fails}}
|
|
64
|
-
- **Notes**:
|
|
64
|
+
- **Notes**: -
|
|
65
65
|
|
|
66
66
|
---
|
|
67
67
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Test Data
|
|
1
|
+
# Test Data - {{MODULE_NAME}}: {{SUBMODULE_NAME}}
|
|
2
2
|
|
|
3
3
|
> **Module code**: {{MODULE_CODE}} | **Submodule code**: {{SUBMODULE_CODE}}
|
|
4
4
|
> **Last updated**: YYYY-MM-DD
|
|
@@ -23,13 +23,13 @@ The following data must exist in the QA environment before tests run:
|
|
|
23
23
|
|
|
24
24
|
## Data Shapes for Key Scenarios
|
|
25
25
|
|
|
26
|
-
### Scenario: Happy path
|
|
26
|
+
### Scenario: Happy path - create record
|
|
27
27
|
|
|
28
28
|
| Field | Value / Rule |
|
|
29
29
|
|-------|-------------|
|
|
30
|
-
| {{name-field}} | `Test-{EXEC_IDX}`
|
|
30
|
+
| {{name-field}} | `Test-{EXEC_IDX}` - unique per run via EXEC_IDX pattern |
|
|
31
31
|
| {{select-field}} | Any valid option (first option is acceptable) |
|
|
32
|
-
| {{date-field}} | `${2120 + (EXEC_IDX % 10)}-01-01`
|
|
32
|
+
| {{date-field}} | `${2120 + (EXEC_IDX % 10)}-01-01` - future date, collision-resistant |
|
|
33
33
|
| User | `{{Role 1}}`, credentials from `QA_USER_{{ROLE1_UPPER}}_EMAIL` |
|
|
34
34
|
|
|
35
35
|
### Scenario: Required field validation
|
|
@@ -49,7 +49,7 @@ The following data must exist in the QA environment before tests run:
|
|
|
49
49
|
|
|
50
50
|
**Expected**: System error message: "{{exact duplicate error text}}"
|
|
51
51
|
|
|
52
|
-
### Scenario: State transition
|
|
52
|
+
### Scenario: State transition - {{action}}
|
|
53
53
|
|
|
54
54
|
| Field | Value |
|
|
55
55
|
|-------|-------|
|
|
@@ -82,8 +82,8 @@ const testDate = `${year}-${month}-${day}`;
|
|
|
82
82
|
|
|
83
83
|
## Data Isolation Rules
|
|
84
84
|
|
|
85
|
-
1. Each test creates its own data
|
|
86
|
-
2. `beforeAll` creates records for the entire suite
|
|
85
|
+
1. Each test creates its own data - no shared mutable state between tests
|
|
86
|
+
2. `beforeAll` creates records for the entire suite - provision ≥ N records for N consuming tests
|
|
87
87
|
3. No test teardown required when EXEC_IDX-based names are used (future-dated records do not collide across runs)
|
|
88
88
|
4. If a test modifies shared data (approve/reject), it must create its own record in `beforeAll`
|
|
89
89
|
|
|
@@ -93,7 +93,7 @@ const testDate = `${year}-${month}-${day}`;
|
|
|
93
93
|
|
|
94
94
|
| Item | Reason for retirement | Replaced by |
|
|
95
95
|
|------|-----------------------|-------------|
|
|
96
|
-
| {{item}} | Real employee data
|
|
96
|
+
| {{item}} | Real employee data - PII risk | EXEC_IDX-generated test record |
|
|
97
97
|
|
|
98
98
|
---
|
|
99
99
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Test Scenarios
|
|
1
|
+
# Test Scenarios - {{MODULE_NAME}}: {{SUBMODULE_NAME}}
|
|
2
2
|
|
|
3
3
|
> **Module code**: {{MODULE_CODE}} | **Submodule code**: {{SUBMODULE_CODE}}
|
|
4
4
|
> **Last updated**: YYYY-MM-DD
|
|
@@ -75,7 +75,7 @@
|
|
|
75
75
|
|
|
76
76
|
---
|
|
77
77
|
|
|
78
|
-
### TC-{{M}}-{{S}}-004: Create new record
|
|
78
|
+
### TC-{{M}}-{{S}}-004: Create new record - happy path
|
|
79
79
|
|
|
80
80
|
| Field | Value |
|
|
81
81
|
|-------|-------|
|
|
@@ -97,7 +97,7 @@
|
|
|
97
97
|
|
|
98
98
|
---
|
|
99
99
|
|
|
100
|
-
### TC-{{M}}-{{S}}-005: Required field validation
|
|
100
|
+
### TC-{{M}}-{{S}}-005: Required field validation - name empty
|
|
101
101
|
|
|
102
102
|
| Field | Value |
|
|
103
103
|
|-------|-------|
|
|
@@ -213,9 +213,9 @@
|
|
|
213
213
|
|
|
214
214
|
| TC ID | Business Rules | Workflow |
|
|
215
215
|
|-------|---------------|---------|
|
|
216
|
-
| TC-{{M}}-{{S}}-001 |
|
|
216
|
+
| TC-{{M}}-{{S}}-001 | - | - |
|
|
217
217
|
| TC-{{M}}-{{S}}-004 | RN-{{M}}-001 | FL-{{M}}-001 |
|
|
218
|
-
| TC-{{M}}-{{S}}-005 | RN-{{M}}-001 |
|
|
218
|
+
| TC-{{M}}-{{S}}-005 | RN-{{M}}-001 | - |
|
|
219
219
|
|
|
220
220
|
---
|
|
221
221
|
|
package/templates/test-case.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Test Case
|
|
1
|
+
# Test Case - TC-{{MODULE}}-{{SUB}}-{{NNN}}: {{Title}}
|
|
2
2
|
|
|
3
3
|
| Field | Value |
|
|
4
4
|
|-------|-------|
|
|
@@ -11,8 +11,8 @@
|
|
|
11
11
|
| Type | Functional / Integration / Regression / Smoke |
|
|
12
12
|
| Execution | Manual / Automated / Both |
|
|
13
13
|
| Automation file | `qa/07-automation/e2e/{{module}}/{{submodule}}.spec.ts` (if automated) |
|
|
14
|
-
| Preconditions | {{List required setup
|
|
15
|
-
| Author | (agent output
|
|
14
|
+
| Preconditions | {{List required setup - see 04-test-data.md}} |
|
|
15
|
+
| Author | (agent output - reviewed by human) |
|
|
16
16
|
| Last updated | YYYY-MM-DD |
|
|
17
17
|
|
|
18
18
|
---
|
|
@@ -16,7 +16,7 @@ All user emails, passwords, API keys, and tokens must come from environment vari
|
|
|
16
16
|
const email = process.env.QA_USER_EMAIL!;
|
|
17
17
|
const password = process.env.QA_USER_PASSWORD!;
|
|
18
18
|
|
|
19
|
-
// ❌ WRONG
|
|
19
|
+
// ❌ WRONG - never commit credentials
|
|
20
20
|
const email = 'admin@example.com';
|
|
21
21
|
const password = 'secret123';
|
|
22
22
|
```
|
|
@@ -25,7 +25,7 @@ Document env var names in `qa/04-test-data/users.md`. Use `<PLACEHOLDER>` for va
|
|
|
25
25
|
|
|
26
26
|
### 2. Use EXEC_IDX for unique values per run
|
|
27
27
|
|
|
28
|
-
Never create records with static names
|
|
28
|
+
Never create records with static names - they collide between runs and can leave leftover data.
|
|
29
29
|
|
|
30
30
|
```typescript
|
|
31
31
|
const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
|
|
@@ -42,7 +42,7 @@ const date = `${year}-${month}-${day}`;
|
|
|
42
42
|
|
|
43
43
|
### 3. Provision in `beforeAll`, not `beforeEach`
|
|
44
44
|
|
|
45
|
-
Create required records once per suite, not once per test. Each test that **consumes** a record needs its own provisioned record
|
|
45
|
+
Create required records once per suite, not once per test. Each test that **consumes** a record needs its own provisioned record - never share a single record among multiple tests.
|
|
46
46
|
|
|
47
47
|
```typescript
|
|
48
48
|
test.beforeAll(async ({ browser }) => {
|
|
@@ -61,10 +61,10 @@ Complex setup that cannot be done via the UI goes in a seeder script, not inside
|
|
|
61
61
|
|
|
62
62
|
```
|
|
63
63
|
qa/04-test-data/
|
|
64
|
-
├── users.md
|
|
65
|
-
├── fixtures/
|
|
66
|
-
├── factories/
|
|
67
|
-
└── seeders/
|
|
64
|
+
├── users.md <- Role catalog + env var names (no real values)
|
|
65
|
+
├── fixtures/ <- Static JSON/Markdown for reference data
|
|
66
|
+
├── factories/ <- Dynamic data generation patterns
|
|
67
|
+
└── seeders/ <- DB/API seed scripts for complex prerequisites
|
|
68
68
|
```
|
|
69
69
|
|
|
70
70
|
### 5. Add `.env` and `.auth/` to `.gitignore`
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Plan de Pruebas
|
|
1
|
+
# Plan de Pruebas - {{PROYECTO}} - Sprint {{NNN}} - {{MODULO_NOMBRE}}
|
|
2
2
|
|
|
3
3
|
**Proyecto:** {{PROYECTO}}
|
|
4
4
|
**Sprint:** {{NNN}}
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
**Código módulo:** {{MODULO_CODE}}
|
|
7
7
|
**Fecha planificación:** {{YYYY-MM-DD}}
|
|
8
8
|
**Ventana QA:** {{YYYY-MM-DD}} y {{YYYY-MM-DD}}
|
|
9
|
-
**ADO Test Plan:** [Plan {{ADO_PLAN_ID}}
|
|
9
|
+
**ADO Test Plan:** [Plan {{ADO_PLAN_ID}} - {{PROYECTO}} Sprint {{NNN}}]({{ADO_TEST_PLAN_URL}}) *(omitir si ADO no está habilitado)*
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
@@ -37,27 +37,27 @@
|
|
|
37
37
|
> Si no hay ADO, listar los requerimientos o módulos abordados.
|
|
38
38
|
|
|
39
39
|
### {{AREA_FUNCIONAL_1}}
|
|
40
|
-
- Task {{ID}}
|
|
41
|
-
- Task {{ID}}
|
|
40
|
+
- Task {{ID}} - {{DESCRIPCION}}
|
|
41
|
+
- Task {{ID}} - {{DESCRIPCION}}
|
|
42
42
|
|
|
43
43
|
### {{AREA_FUNCIONAL_2}}
|
|
44
|
-
- Task {{ID}}
|
|
44
|
+
- Task {{ID}} - {{DESCRIPCION}}
|
|
45
45
|
|
|
46
46
|
---
|
|
47
47
|
|
|
48
48
|
## 4. Priorización
|
|
49
49
|
|
|
50
|
-
**P0
|
|
50
|
+
**P0 - Confirmación (obligatorio)**
|
|
51
51
|
- {{ESCENARIO_CRITICO_1}}
|
|
52
52
|
- {{ESCENARIO_CRITICO_2}}
|
|
53
53
|
|
|
54
|
-
**P1
|
|
54
|
+
**P1 - Smoke (camino feliz)**
|
|
55
55
|
- {{FLUJO_PRINCIPAL}}
|
|
56
56
|
|
|
57
|
-
**P2
|
|
57
|
+
**P2 - Regresión mínima adyacente**
|
|
58
58
|
- {{REGRESION_1}}
|
|
59
59
|
|
|
60
|
-
**P3
|
|
60
|
+
**P3 - Exploratoria (timeboxed)**
|
|
61
61
|
- Charter 1 ({{N}} min): {{DESCRIPCION_CHARTER}}
|
|
62
62
|
|
|
63
63
|
---
|
|
@@ -77,8 +77,8 @@
|
|
|
77
77
|
|
|
78
78
|
- Ambiente QA disponible en `{{QA_BASE_URL}}`
|
|
79
79
|
- Roles disponibles:
|
|
80
|
-
- Usuario **{{ROL_1}}**
|
|
81
|
-
- Usuario **{{ROL_2}}**
|
|
80
|
+
- Usuario **{{ROL_1}}** - para {{ACCION}}
|
|
81
|
+
- Usuario **{{ROL_2}}** - para {{ACCION}}
|
|
82
82
|
- Datos base:
|
|
83
83
|
- {{DATO_BASE_1}}
|
|
84
84
|
- {{DATO_BASE_2}}
|
|
@@ -109,7 +109,7 @@
|
|
|
109
109
|
|
|
110
110
|
> Convenciones:
|
|
111
111
|
> - **TC-ID:** `TC-{{MODULO_CODE}}-{{SUBMODULO_CODE}}-NNN` (o `N/A` si no hay ID asignado aún)
|
|
112
|
-
> - **Steps:** numerados, separados por `<br>`
|
|
112
|
+
> - **Steps:** numerados, separados por `<br>` - verbos concretos de UI (`Navegar a`, `Ingresar`, `Hacer click`, `Verificar`)
|
|
113
113
|
> - **Confirma:** `Task NNNNN` si hay tarea ADO relacionada; `N/A` si no aplica
|
|
114
114
|
> - **Tipo:** `Manual` | `Automatizado` | `Ambos`
|
|
115
115
|
> - **Prioridad:** `P0` | `P1` | `P2` | `P3`
|
|
@@ -149,4 +149,4 @@
|
|
|
149
149
|
> `qa/02-test-plans/sprints/Sprint-{{NNN}}/Plan-de-Pruebas-{{PROYECTO}}-Sprint-{{NNN}}.md`
|
|
150
150
|
>
|
|
151
151
|
> El consolidado se genera con el script `@keber/ado-qa consolidate-sprint --sprint {{NNN}}`.
|
|
152
|
-
> **No editar manualmente**
|
|
152
|
+
> **No editar manualmente** - es derivado de los archivos por módulo.
|
package/templates/test-plan.md
CHANGED
|
@@ -4,12 +4,12 @@
|
|
|
4
4
|
>
|
|
5
5
|
> This file is retained for reference only.
|
|
6
6
|
|
|
7
|
-
# Test Plan
|
|
7
|
+
# Test Plan - {{MODULE_NAME}}
|
|
8
8
|
|
|
9
9
|
**Plan ID**: TP-{{MODULE}}-{{VERSION}}
|
|
10
10
|
**Module**: {{MODULE_NAME}}
|
|
11
11
|
**Sprint / Release**: Sprint {{NNN}}
|
|
12
|
-
**Prepared by**: (agent output
|
|
12
|
+
**Prepared by**: (agent output - reviewed by human)
|
|
13
13
|
**Date**: YYYY-MM-DD
|
|
14
14
|
**ADO Test Plan**: #{ADO_PLAN_ID} (if ADO enabled)
|
|
15
15
|
|
|
@@ -74,7 +74,7 @@
|
|
|
74
74
|
|
|
75
75
|
| Scenario | Feasible? | Reason if not |
|
|
76
76
|
|----------|-----------|---------------|
|
|
77
|
-
| Happy path create | Yes |
|
|
77
|
+
| Happy path create | Yes | - |
|
|
78
78
|
| Export to PDF | No | Requires visual validation |
|
|
79
79
|
| Third-party OAuth | No | External system |
|
|
80
80
|
|