@keber/qa-framework 1.12.0 → 1.13.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +359 -0
- package/README.md +41 -1
- package/docs/installation.md +11 -4
- package/docs/usage-with-agent.md +14 -9
- package/package.json +3 -4
- package/qa-framework.config.json +3 -2
- package/scripts/init.js +51 -10
- package/scripts/lib/claude-agents.js +278 -9
- package/scripts/lib/module-keys.js +54 -0
- package/scripts/sync-version.js +27 -0
- package/scripts/upgrade.js +16 -3
- package/scripts/validate.js +31 -6
- package/templates/agents/qa-informe-resultados.md +20 -18
- package/templates/agents/qa-investigador.md +143 -0
- package/templates/agents/qa-reporte-ejecucion.md +138 -0
- package/templates/qa-framework.instructions.md +1 -1
- package/templates/qa-framework.rules.md +1 -1
- package/docs/visuals/diag-01-ejecutivo-1.png +0 -0
- package/docs/visuals/diag-02-tactico-1.png +0 -0
- package/docs/visuals/diag-03-tecnico-1.png +0 -0
- package/docs/visuals/info-01-ejecutivo-1.png +0 -0
- package/docs/visuals/info-02-tactico-1.png +0 -0
- package/docs/visuals/info-03-tecnico-1.png +0 -0
- package/docs/visuals/pres-01-ejecutivo-1.png +0 -0
- package/docs/visuals/pres-02-tactico-1.png +0 -0
- package/docs/visuals/pres-03-tecnico-1.png +0 -0
- package/docs/visuals/qa-01-problema-1.png +0 -0
- package/docs/visuals/qa-02-que-es-1.png +0 -0
- package/docs/visuals/qa-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-04-6-archivos-1.png +0 -0
- package/docs/visuals/qa-05-humano-1.png +0 -0
- package/docs/visuals/qa-06-agente-1.png +0 -0
- package/docs/visuals/qa-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-08-traceability-1.png +0 -0
- package/docs/visuals/qa-en-01-problem-1.png +0 -0
- package/docs/visuals/qa-en-02-what-is-1.png +0 -0
- package/docs/visuals/qa-en-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-en-04-6files-1.png +0 -0
- package/docs/visuals/qa-en-05-human-1.png +0 -0
- package/docs/visuals/qa-en-06-agent-1.png +0 -0
- package/docs/visuals/qa-en-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-en-08-traceability-1.png +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,365 @@ Versioning follows [Semantic Versioning](https://semver.org/).
|
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
+
## [1.13.1] - 2026-11-06
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **`package.json`** - `docs/visuals/` no longer ships to npm. The `files` allow list
|
|
15
|
+
carried a blanket `docs/` entry, so the 25 presentation images added in `f1064e1`
|
|
16
|
+
started being published without anyone deciding to: the tarball went from 410 KB
|
|
17
|
+
unpacked in 1.11.3 to 51.8 MB in 1.12.0, and every `npm install` has paid for it
|
|
18
|
+
since. The entry is now `docs/**/*.md`, which keeps all eight documents - including
|
|
19
|
+
`docs/folder-structure-guide.md` and `docs/spec-driven-philosophy.md`, both named in
|
|
20
|
+
text that `scripts/` writes into consuming projects - and drops the images. The
|
|
21
|
+
images stay in the repository; they are reference material, not runtime content.
|
|
22
|
+
Unpacked size: 51.8 MB -> 630 KB. Exactly the 25 images leave the package and
|
|
23
|
+
nothing else moves.
|
|
24
|
+
- **`package.json`** - removed `agent-instructions/` from `files`. The directory does
|
|
25
|
+
not exist, so the entry had been matching nothing.
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## [1.13.0] - 2026-11-04
|
|
30
|
+
|
|
31
|
+
Dated by its release, not by its bump. `package.json` was moved to `1.13.0` on 2026-10-16,
|
|
32
|
+
but the version was never tagged or published, so everything below ships together under
|
|
33
|
+
this number. The infrastructure fixes listed in Added and Fixed were written later, on the
|
|
34
|
+
branch that unblocked this release.
|
|
35
|
+
|
|
36
|
+
### Added
|
|
37
|
+
|
|
38
|
+
- `templates/agents/qa-reporte-ejecucion.md` - new Claude Code agent that exports the ADO
|
|
39
|
+
test plan results, versions the raw export under `ado-export-<YYYY-MM-DD>/` and writes the
|
|
40
|
+
sprint `Reporte-de-Ejecucion-de-Pruebas-<project>-Sprint-<N>.md` (summary with executed,
|
|
41
|
+
success and coverage metrics, plus promoted findings).
|
|
42
|
+
- `templates/agents/qa-investigador.md` - new Claude Code agent that investigates one root
|
|
43
|
+
cause against the real environment. Its tools are generated from `parallelLanes`: each lane
|
|
44
|
+
with an `mcpNamespace` adds 15 Playwright MCP browser tools, a lane table and the mandatory
|
|
45
|
+
heartbeat protocol; with no MCP lane it gets a no-browser section instead.
|
|
46
|
+
- Agent template engine: `{{#if FLAG}}` / `{{#unless FLAG}}` blocks, single-pass placeholder
|
|
47
|
+
replacement and a guard that fails on any unresolved `{{...}}` token.
|
|
48
|
+
- New placeholders: `INVESTIGATOR_TOOLS`, `LANE_MCP_TABLE`, `APPROVAL_THRESHOLD_PERCENT`,
|
|
49
|
+
`APPROVAL_THRESHOLD_RATIO`, `DECIMAL_SEPARATOR`, `TC_ID_PATTERN`, `QA_BASE_URL`,
|
|
50
|
+
`LOGIN_PATH`, `TECH_STACK`.
|
|
51
|
+
- Config keys: `integrations.azureDevOps.sprintCycle.approvalThresholdPercent` (default 85),
|
|
52
|
+
optional `conventions.decimalSeparator` and `conventions.defectIdPrefix`. Documented in
|
|
53
|
+
`qa-framework.config.schema.json`, together with the `mcpNamespace` pattern.
|
|
54
|
+
- `qa-framework validate` checks present-but-invalid values of those keys and of
|
|
55
|
+
`parallelLanes`, and now finds the config under `qa/` as well as at the project root.
|
|
56
|
+
- **`scripts/lib/module-keys.js`** - single source of truth for how a module key maps
|
|
57
|
+
to a spec folder (prefixed) and to an automation folder (bare). Shared by `init.js`
|
|
58
|
+
and `validate.js` so the two cannot drift apart again.
|
|
59
|
+
- **`scripts/sync-version.js`** - now also rewrites `frameworkVersion` in
|
|
60
|
+
`qa-framework.config.json`, by field rather than by the loose `vX.Y.Z` prose pattern.
|
|
61
|
+
- **`test/module-keys.test.js`, `test/sync-version.test.js`** - unit coverage for the
|
|
62
|
+
key helpers plus a drift guard that fails CI whenever a published version surface
|
|
63
|
+
disagrees with `package.json`.
|
|
64
|
+
- **`.github/workflows/publish.yml`** - the suite now runs before `npm publish`.
|
|
65
|
+
`ci.yml` triggers on pushes to `main` and on pull requests, so it never sees a tag,
|
|
66
|
+
and publishing is triggered by a tag push: nothing was verified on the path that
|
|
67
|
+
actually reaches npm. The drift guard above only protects a release once it runs
|
|
68
|
+
there.
|
|
69
|
+
|
|
70
|
+
### Changed
|
|
71
|
+
|
|
72
|
+
- `init` and `upgrade` generate six agents (was four) when `sprintCycle.enabled` is true.
|
|
73
|
+
Agents are rendered independently: an invalid lane config fails `qa-investigador` only,
|
|
74
|
+
the other five are still written and the command exits 1.
|
|
75
|
+
- As npm postinstall (`init --skip-if-exists`), an agent that cannot be rendered is reported
|
|
76
|
+
as a WARNING naming the agent and asking to fix the config and run
|
|
77
|
+
`npx qa-framework upgrade`; postinstall exits 0 so `npm install` does not fail. Explicit
|
|
78
|
+
`init` and `upgrade` keep exit code 1.
|
|
79
|
+
- `qa-informe-resultados` reads its approval threshold from config instead of a hardcoded
|
|
80
|
+
85% and states the decimal separator rule.
|
|
81
|
+
|
|
82
|
+
### Fixed
|
|
83
|
+
|
|
84
|
+
- `qa-informe-resultados` looked for `Reporte-de-Ejecucion-de-Pruebas-...` and pointed to the
|
|
85
|
+
`qa-ado-integration` skill, which does not produce that file. It now names the
|
|
86
|
+
`qa-reporte-ejecucion` agent, the canonical filename (active and historical sprint folders)
|
|
87
|
+
and accepts the older `Reporte-ejecución-de-pruebas-*` files.
|
|
88
|
+
- `templates/qa-framework.instructions.md` and `templates/qa-framework.rules.md` carried a
|
|
89
|
+
project-specific `DEF-SIS-NNN` example; it is now `DEF-{PREFIX}-NNN`.
|
|
90
|
+
- **`scripts/init.js`** - module spec folders are now scaffolded as
|
|
91
|
+
`qa/01-specifications/module-{name}/submodule-{name}/`, the layout declared by every
|
|
92
|
+
skill, by `docs/folder-structure-guide.md` and by the `specPath` field of
|
|
93
|
+
`qa-framework.config.json`. init.js wrote the bare key, so a freshly scaffolded
|
|
94
|
+
project violated the spec-path convention its own agent instructions impose.
|
|
95
|
+
Prefixing is idempotent: a config whose key already reads `module-suppliers` is not
|
|
96
|
+
turned into `module-module-suppliers`.
|
|
97
|
+
- **`scripts/init.js`** - automation stubs and test case ids keep using the **bare**
|
|
98
|
+
key (`e2e/tests/suppliers/create.spec.ts`, `TC-SUPPLIERS-CREATE-001`), matching the
|
|
99
|
+
`automationPath` field and the live consuming projects. Only the spec tree is
|
|
100
|
+
prefixed.
|
|
101
|
+
- **`scripts/validate.js`** - `--strict` strips the `module-` / `submodule-` prefix
|
|
102
|
+
before deriving the automation path. It was pointing the warning at
|
|
103
|
+
`e2e/tests/module-{name}/`, a folder no project has ever created.
|
|
104
|
+
- **`qa-framework.config.json`** - `frameworkVersion` was stuck at `1.10.0` while the
|
|
105
|
+
package shipped `1.12.0`. The file is published and is the template copied into every
|
|
106
|
+
consuming project, where the value is injected into `{{VERSION}}` of the generated
|
|
107
|
+
agent instructions, so every fresh install since 1.11.0 reported the wrong version.
|
|
108
|
+
- **`package.json`** - the npm `version` lifecycle script now also stages
|
|
109
|
+
`qa-framework.config.json`. sync-version.js rewrites it, but the staging list did not
|
|
110
|
+
include it, so the release commit and its tag would have carried the OLD
|
|
111
|
+
frameworkVersion - and the publish workflow builds from a clean checkout of that tag,
|
|
112
|
+
reintroducing the exact drift this change set out to fix.
|
|
113
|
+
- **`test/sync-version.test.js`** - two tests compared files against their starting bytes,
|
|
114
|
+
which assumed the tree was already synced. On a drifted tree one of them ran
|
|
115
|
+
`sync-version` and left `qa-framework.config.json` rewritten: the first run failed, the
|
|
116
|
+
retry passed, and the drift vanished from the working tree unnoticed. A suite that edits
|
|
117
|
+
tracked files turns a red run green on the retry and hides what it exists to catch.
|
|
118
|
+
- **`test/init.test.js`** - the agent summary assertion matched only the forward-slash form
|
|
119
|
+
of `.claude/agents/` and took the first matching line. `init.js` prints those lines
|
|
120
|
+
through `path.relative`, so on Windows the per-file `[created]` lines never matched and it
|
|
121
|
+
landed on the summary line, while on Linux it landed on the first `[created]` line. The
|
|
122
|
+
test passed locally and failed the first time it ran on CI.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
## [1.12.0] - 2026-09-08
|
|
127
|
+
|
|
128
|
+
Reconstructed from `4fe32fb..f52257a` (39 commits). Minor rather than major: a project
|
|
129
|
+
with no `parallelLanes` block behaves exactly as before, `PENDING-BROWSER` adds an enum
|
|
130
|
+
value without removing any, and the spec-path change affects what `init` generates from
|
|
131
|
+
then on, not what already exists. The one user-visible break is `engines` moving from
|
|
132
|
+
`>=18` to `>=20`.
|
|
133
|
+
|
|
134
|
+
### Added
|
|
135
|
+
|
|
136
|
+
- **`parallelLanes` lane table** - `qa-framework.config.json` gains a lane block and
|
|
137
|
+
`qa-framework.config.schema.json` is published alongside it. A lane is four coupled
|
|
138
|
+
values (account env prefix, `storageState`, MCP namespace, runner project), none of
|
|
139
|
+
which can be inferred from the others, plus per-lane capabilities: a project can have
|
|
140
|
+
six accounts but only three MCP servers. A project with no block, or `enabled: false`,
|
|
141
|
+
resolves a single implicit default lane.
|
|
142
|
+
- **`templates/automation-scaffold/scripts/lane-lock.js`** - config-driven lane lock
|
|
143
|
+
ported from the reference implementation. `acquire` (random free lane, queues) and
|
|
144
|
+
`reserve` (exact lane, no queue, throws when busy) stay two separate verbs; staleness
|
|
145
|
+
is a composite predicate (past the threshold AND no recent heartbeat); the lock
|
|
146
|
+
directory resolves by shell > `.env` > per-checkout precedence and the resolved path
|
|
147
|
+
and its source are reported on stderr. Reads `.env` with `dotenv.parse()`, never
|
|
148
|
+
`config()`, so a script whose only job is choosing a directory never gains ambient
|
|
149
|
+
access to the QA account passwords in the same file.
|
|
150
|
+
- **`templates/automation-scaffold/scripts/global-setup-guards.js`** - `global-setup`
|
|
151
|
+
performs the only login, so it is the last point at which a run about to drive an
|
|
152
|
+
account it does not hold can be stopped. `assertColdSetupIsIntentional` throws before
|
|
153
|
+
any login when neither `QA_LANE_ONLY` nor `QA_COLD_SETUP=1` is set, since a cold setup
|
|
154
|
+
re-logs in every lane and invalidates other running sessions; `assertLaneIsLocked`
|
|
155
|
+
verifies outside CI that the lane is held; `assertProjectsMatchLane` catches a
|
|
156
|
+
`--project` whose `storageState` points at a different lane than the one reserved.
|
|
157
|
+
Plain CommonJS so it is testable with `node --test` without adding `ts-node`.
|
|
158
|
+
- **`templates/automation-scaffold/scripts/lane-config.js`** - lane table resolver shared
|
|
159
|
+
by the config and the guards.
|
|
160
|
+
- **`templates/qa-framework.rules.md`** plus **`templates/agents/`** - four agent
|
|
161
|
+
templates (`qa-analisis.md`, `qa-asesoria.md`, `qa-plan.md`,
|
|
162
|
+
`qa-informe-resultados.md`) for analysis, sprint advisory, test planning and ADO result
|
|
163
|
+
reporting, deployed through `scripts/lib/claude-agents.js` and
|
|
164
|
+
`scripts/lib/claude-commands.js`.
|
|
165
|
+
- **`scripts/check-forbidden-chars.js`** (`npm run check-chars`) - enforces the BLOCKING
|
|
166
|
+
character-safety rule the package stated but never checked. Flags em-dash, en-dash,
|
|
167
|
+
ellipsis, smart quotes and arrows; leaves Latin Extended and emoji alone. Reports
|
|
168
|
+
`file:line:column` with the code point and its ASCII replacement.
|
|
169
|
+
- **`scripts/check-spanish-accents.js`** (`npm run check-accents`) - language-aware accent
|
|
170
|
+
gate for Markdown. Classification is the hard part, not detection: it measures the
|
|
171
|
+
density of Spanish function words in prose only, after stripping fenced code, inline
|
|
172
|
+
code, link targets, tags, placeholders and paths. Against this package's corpus English
|
|
173
|
+
lands at 0.000-0.001 and Spanish templates at 0.125-0.165, with the threshold at 0.07;
|
|
174
|
+
a test asserts the gap stays wide.
|
|
175
|
+
- **`scripts/sync-version.js`** - wired into the `npm version` lifecycle to keep
|
|
176
|
+
`README.md` and `.github/copilot-instructions.md` aligned with `package.json`.
|
|
177
|
+
- **Test suite and CI** - black-box tests for `cli.js`, `init.js`, `validate.js`,
|
|
178
|
+
`upgrade.js`, the lane scripts and both character gates, using Node's built-in runner
|
|
179
|
+
with no new devDependency, plus `.github/workflows/ci.yml`.
|
|
180
|
+
- **`scripts/upgrade.js` section 9** - carries the v1.12.0 scaffold into existing
|
|
181
|
+
projects. `upgrade` force-copied only `.github/skills/`, `copilot-instructions.md` and
|
|
182
|
+
`QA-STRUCTURE-GUIDE.md`, so everything this release changed under `templates/` reached
|
|
183
|
+
new projects through `init.js` alone and existing installations kept the old scaffold,
|
|
184
|
+
including the login that types the password with `fill()`. The rule is per file:
|
|
185
|
+
overwrite what is framework-owned, warn and continue where overwriting would destroy
|
|
186
|
+
working user code, and treat a dependency group as all-or-nothing, since
|
|
187
|
+
`playwright.config.ts` and `global-setup.ts` both `require()` the lane scripts and a
|
|
188
|
+
partial install leaves a project that cannot start Playwright. Baseline detection is
|
|
189
|
+
content-hash based, not `frameworkVersion` based: the recorded values in live projects
|
|
190
|
+
were `1.10.0`, `1.1.3`, `1.1.3` and `1.0.0`, two of them malformed. Files left
|
|
191
|
+
user-owned are scanned for `.fill(password)` and reported with file and line, without a
|
|
192
|
+
textual patch.
|
|
193
|
+
- **`PENDING-BROWSER`** - fourth value in the TC Origin vocabulary, for an element
|
|
194
|
+
observed and confirmed to exist whose behavior could not be exercised end to end.
|
|
195
|
+
Without it agents either overclaim with `UI-OBSERVED` or misuse `PENDING-CODE`, which
|
|
196
|
+
pollutes gap tracking downstream.
|
|
197
|
+
- **Known-issues lookup before defect creation** - nothing required checking whether a
|
|
198
|
+
defect had already been filed. The gate distinguishes the two outcomes that matter: an
|
|
199
|
+
open match is referenced instead of refiled, a closed match is filed as a regression.
|
|
200
|
+
`templates/defect-report.md` gains the section that records the search.
|
|
201
|
+
- **Stage 5 gates** - `tsc --noEmit` plus a single smoke run of a newly written spec
|
|
202
|
+
before the submodule is marked complete. Both run before the completion checklist; a
|
|
203
|
+
full green suite remains Stage 5b's job.
|
|
204
|
+
- **Closing measurement for doc-generating stages** - the four skills that write Markdown
|
|
205
|
+
now end on evidence (accent count, forbidden characters, BOM) rather than a compliance
|
|
206
|
+
claim, with the language boundary stated so English artifacts are not asked for accents
|
|
207
|
+
they should not have.
|
|
208
|
+
- **`skills/qa-automation/references/pom-template.md`** - page-object template taken from
|
|
209
|
+
real project use: an abstract per-module CRUD base plus thin per-submodule classes,
|
|
210
|
+
with the accent-insensitive label matching that pattern needs against Spanish UIs.
|
|
211
|
+
- **`MIGRATION-NOTES.md`**, **`docs/decisions/lane-heartbeat-caller.md`** and
|
|
212
|
+
**`docs/visuals/`** - embedded-QA migration guidance, the record that the dispatcher
|
|
213
|
+
(not the executor) calls heartbeat, and the diagrams referenced from the architecture
|
|
214
|
+
docs.
|
|
215
|
+
|
|
216
|
+
### Changed
|
|
217
|
+
|
|
218
|
+
- **`engines`** - `>=18` becomes `>=20`, and CI tests Node 20, 22 and 24. The matrix had
|
|
219
|
+
been testing 18 and 20, both out of maintenance, while `publish.yml` released on 24,
|
|
220
|
+
which no job tested. Users on 18 now get an npm warning, which is the point.
|
|
221
|
+
- **`templates/*/playwright.config.ts`** - both configs called `dotenv.config()`, which
|
|
222
|
+
injects the entire `.env` into `process.env`. In a real project that file holds every QA
|
|
223
|
+
account password plus ADO tokens (82 lines in one case) while these configs need a
|
|
224
|
+
handful of non-secret values. They now read an explicit key list with `dotenv.parse()`,
|
|
225
|
+
which returns a plain object and never touches `process.env`, and only fill keys the
|
|
226
|
+
environment has not already set, so shell and CI variables still win.
|
|
227
|
+
- **`templates/automation-scaffold/playwright.config.ts`** - runner projects and auth are
|
|
228
|
+
generated from the lane table, replacing the commented-out
|
|
229
|
+
`chromium-user1` / `chromium-user2` block that sat outside the `projects` array and
|
|
230
|
+
could not have worked as written. `storageState` is derived through lane-lock rather
|
|
231
|
+
than hardcoded: a parallel hardcoded mapping caused real evidence corruption once, a run
|
|
232
|
+
recorded under the wrong account because the project was pinned to one lane while
|
|
233
|
+
`QA_LANE_ONLY` named another.
|
|
234
|
+
- **`global-setup.ts`** - `resolveAuthUsers` resolved lanes by substring
|
|
235
|
+
(`storageState.includes('user-2')`, which also matches `user-20`) and had no branch for
|
|
236
|
+
a third lane. It now matches the lane table exactly and reads credentials from each
|
|
237
|
+
lane's declared account prefix.
|
|
238
|
+
- **Agent behavior rules** - a BLOCKING lane-lock rule states the requirement only
|
|
239
|
+
(acquire before dispatching, release when done, dispatcher keeps the lane alive, queue
|
|
240
|
+
or reject at the cap, never force past it), agent-agnostic so each agent's adaptation
|
|
241
|
+
prescribes its own mechanism.
|
|
242
|
+
- **`qa-test-plan` / `qa-test-cases`** - skill content consolidated into the templates;
|
|
243
|
+
`priority-and-feasibility.md` merged into `plan-de-pruebas-template.md`.
|
|
244
|
+
- **Typographic characters** - em-dashes, en-dashes, arrows, smart quotes and ellipses
|
|
245
|
+
replaced with ASCII across `scripts/`, `skills/`, `templates/`, `docs/`, `examples/`,
|
|
246
|
+
`integrations/` and the PowerShell scripts. Substitution is context-sensitive: padded
|
|
247
|
+
prose dashes become ` - `, numeric and label ranges keep a tight hyphen, standalone
|
|
248
|
+
table cells become a bare `-`, and ASCII rules keep their original column width.
|
|
249
|
+
`CHANGELOG.md` keeps its historical occurrences deliberately - rewriting past entries to
|
|
250
|
+
satisfy a rule adopted later would falsify the log.
|
|
251
|
+
|
|
252
|
+
### Fixed
|
|
253
|
+
|
|
254
|
+
- **`scripts/init.js`** - module specs are scaffolded under
|
|
255
|
+
`qa/01-specifications/{module}/{submodule}/`. init wrote them to `qa/{module}/`, leaving
|
|
256
|
+
`01-specifications/` permanently empty while scattering specs as siblings of the
|
|
257
|
+
numbered folders. The existing tests ran with no config, so the module loop iterated
|
|
258
|
+
zero times and no spec path was ever asserted.
|
|
259
|
+
- **`scripts/validate.js`** - check 2 walked all of `qa/`, treating every top-level
|
|
260
|
+
directory as a module, so it read `01-specifications/` as the module and demanded the 6
|
|
261
|
+
spec files one level above where they exist. The scan is now rooted at
|
|
262
|
+
`01-specifications/`, which also stops sibling folders such as `02-test-plans/` and
|
|
263
|
+
`memory/` from being mistaken for submodules; the `SKIP_DIRS` denylist is removed rather
|
|
264
|
+
than extended.
|
|
265
|
+
- **`scripts/validate.js`** - `--strict` looked for automation specs at
|
|
266
|
+
`e2e/{module}/{submodule}.spec.ts`, a path nothing creates. `init.js` writes to
|
|
267
|
+
`e2e/tests/{module}/` and `upgrade.js` documents the v1.6.0 migration that moved them
|
|
268
|
+
there, so `--strict` emitted a permanently false warning for every submodule in all
|
|
269
|
+
three live projects.
|
|
270
|
+
- **`scripts/validate.js`** - stopped requiring `05-test-plans`, a stale name from an
|
|
271
|
+
older convention; the current structure uses `05-test-execution` with test plans under
|
|
272
|
+
`02-test-plans`.
|
|
273
|
+
- **`templates/*/fixtures` and both scaffolds** - the four remaining `fill(password)` call
|
|
274
|
+
sites are listed under Security below.
|
|
275
|
+
- **`templates/agents/`** - the four Spanish-language agent templates shipped entirely
|
|
276
|
+
accent-stripped (144, 190, 88 and 52 Spanish markers, 0 to 2 accented characters between
|
|
277
|
+
them). This is functional, not cosmetic: page objects locate fields by visible label, so
|
|
278
|
+
a spec written as `Codigo` never matches a UI rendering the accented form. In one
|
|
279
|
+
project that cost 24 of 39 failing smoke tests and two wrong mitigations before the
|
|
280
|
+
cause was found.
|
|
281
|
+
- **`templates/defect-report.md`** - the defect ID prefix is a placeholder. The template
|
|
282
|
+
hardcoded `DEF-{{NNN}}` while `defectIdPattern` carried `DEF-{NUM}`, and real projects
|
|
283
|
+
use a project prefix such as `DEF-SIS-014`.
|
|
284
|
+
- **`templates/qa-framework.instructions.md` and `qa-framework.rules.md`** - referenced
|
|
285
|
+
`integrations.ado.enabled`, a config path that never existed. The real schema is
|
|
286
|
+
`integrations.azureDevOps.enabled`.
|
|
287
|
+
- **`skills/qa-spec-generation`** - pointed at `references/spec-file-formats.md`, which
|
|
288
|
+
only exists under `qa-module-analysis/references/`, so Stage 2 could not find the
|
|
289
|
+
template it is told to use. `skills/qa-automation` referenced a `pom-template.md` that
|
|
290
|
+
did not exist.
|
|
291
|
+
- **`qa-test-stabilization`** - the official suite is the only source of truth. A polling
|
|
292
|
+
diagnostic script saw a toast 4 of 4 times while the official suite saw none, 6 of 6
|
|
293
|
+
across two independent runs, and the defect had already been closed on that standalone
|
|
294
|
+
evidence. Standalone output is now diagnostic input, never a verdict.
|
|
295
|
+
- **`templates/*/.env.example`** - both files carried a UTF-8 BOM and three em-dashes. A
|
|
296
|
+
BOM in a `.env` is not cosmetic: some parsers read the first key as `QA_BASE_URL`
|
|
297
|
+
and silently fail to find it, and these files ship to every consuming project as the
|
|
298
|
+
template developers copy to `.env`. The character gate's extension list gains
|
|
299
|
+
`.example` and `.ps1`, both previous blind spots.
|
|
300
|
+
- **CI** - the test script used `test/**/*.test.js`, which needs globstar the runner's
|
|
301
|
+
shell does not enable, and the follow-up fix kept the pattern quoted so node received a
|
|
302
|
+
literal path. The validate step ran against this repository's own gitignored `qa/`
|
|
303
|
+
directory and failed with four missing-folder errors; it now scaffolds a throwaway
|
|
304
|
+
project and validates that, which exercises the contract that actually matters - what
|
|
305
|
+
`init.js` generates must pass `validate.js`.
|
|
306
|
+
|
|
307
|
+
### Security
|
|
308
|
+
|
|
309
|
+
- **Password no longer reaches Playwright traces.** Both scaffolds typed the QA password
|
|
310
|
+
with `locator().fill(password)`, and the automation scaffold ships `trace` and `video`
|
|
311
|
+
set to `retain-on-failure`, so any failed CI run persisted an artifact containing the
|
|
312
|
+
plaintext password. All four call sites - `global-setup.ts` in both scaffolds,
|
|
313
|
+
`fixtures/base.ts` (TTL-aware reauth) and `fixtures/auth.ts` (`loginAs`) - now set the
|
|
314
|
+
value through `page.evaluate()`, the mechanism this package already documented as
|
|
315
|
+
"Pattern 7: Password Injection (Trace Safety)". The fixture sites carried more exposure
|
|
316
|
+
than `global-setup`, which runs once per run, since reauth runs for every test past the
|
|
317
|
+
TTL and `loginAs` for every test that logs in as a role. Bubbling `input` and `change`
|
|
318
|
+
events are dispatched after the assignment because a bare `.value` write does not notify
|
|
319
|
+
the Blazor/Radzen-style frameworks these scaffolds target, and an explicit visible
|
|
320
|
+
`waitFor` replaces the auto-wait `fill()` provided. Email `fill()` calls are left in
|
|
321
|
+
place: emails are not secret and aid debugging. There are no remaining
|
|
322
|
+
`fill(password)` call sites in `templates/`.
|
|
323
|
+
- **`.env` is no longer loaded wholesale** by the Playwright configs or the lane lock. See
|
|
324
|
+
Changed above.
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
## [1.11.3] - 2026-06-24
|
|
329
|
+
|
|
330
|
+
Reconstructed from `5507f1a..4fe32fb`. Release plumbing only.
|
|
331
|
+
|
|
332
|
+
### Changed
|
|
333
|
+
|
|
334
|
+
- **`.github/workflows/publish.yml`** - `actions/checkout` and `actions/setup-node` move
|
|
335
|
+
from v4 to v6, and the publish job's Node version from 20 to 24.
|
|
336
|
+
|
|
337
|
+
### Removed
|
|
338
|
+
|
|
339
|
+
- **`NODE_AUTH_TOKEN` / `secrets.NPM_TOKEN`** from the publish step, completing the move
|
|
340
|
+
to npm trusted publishing begun in 1.11.1: the job now authenticates with the
|
|
341
|
+
short-lived OIDC token the `id-token: write` permission grants, so no long-lived npm
|
|
342
|
+
token is stored in the repository.
|
|
343
|
+
|
|
344
|
+
No package content changed. Nothing reached consuming projects.
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
## [1.11.2] - 2026-06-24
|
|
349
|
+
|
|
350
|
+
Version bump only - a republish of 1.11.1 after a failed publish run. No source,
|
|
351
|
+
template, skill or script change; the diff is `package.json` and `package-lock.json`.
|
|
352
|
+
|
|
353
|
+
---
|
|
354
|
+
|
|
355
|
+
## [1.11.1] - 2026-06-24
|
|
356
|
+
|
|
357
|
+
Reconstructed from `3a29617..59f6fe0`. Release plumbing only.
|
|
358
|
+
|
|
359
|
+
### Added
|
|
360
|
+
|
|
361
|
+
- **`.github/workflows/publish.yml`** - `id-token: write` permission on the publish job,
|
|
362
|
+
the prerequisite for npm trusted publishing. The long-lived token it replaces is dropped in
|
|
363
|
+
1.11.3.
|
|
364
|
+
|
|
365
|
+
No package content changed. Nothing reached consuming projects.
|
|
366
|
+
|
|
367
|
+
---
|
|
368
|
+
|
|
10
369
|
## [1.11.0] - 2026-06-24
|
|
11
370
|
|
|
12
371
|
### Added
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# qa-framework
|
|
2
2
|
|
|
3
3
|
> Reusable, installable, agent-oriented QA framework for spec-driven automated testing.
|
|
4
|
-
> Current version: **v1.
|
|
4
|
+
> Current version: **v1.13.1**
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -245,6 +245,46 @@ See [docs/skills-architecture.md](docs/skills-architecture.md) for the full desi
|
|
|
245
245
|
|
|
246
246
|
---
|
|
247
247
|
|
|
248
|
+
## Claude Code Sprint Agents (optional)
|
|
249
|
+
|
|
250
|
+
When `integrations.azureDevOps.sprintCycle.enabled` is `true`, `init` (only if missing) and
|
|
251
|
+
`upgrade` (always, framework-owned) generate six Claude Code subagents under `.claude/agents/`:
|
|
252
|
+
|
|
253
|
+
| Agent | What it does |
|
|
254
|
+
|---|---|
|
|
255
|
+
| `qa-analisis` | Sprint analysis document: test universe, automation feasibility, P0-P3 prioritization, traceability matrix |
|
|
256
|
+
| `qa-plan` | Sprint test plan: executive summary plus a trace-linked test case table that fits the manual-testing timebox |
|
|
257
|
+
| `qa-asesoria` | Chat-only QA advisory (no `Write`/`Bash`) |
|
|
258
|
+
| `qa-reporte-ejecucion` | Exports the ADO test plan results (via the `ado-qa` `getADO.ps1` script), versions the raw export by date and writes `Reporte-de-Ejecucion-de-Pruebas-<project>-Sprint-<N>.md` |
|
|
259
|
+
| `qa-informe-resultados` | Narrative sprint closing report (metrics, failures, risks, QA verdict) built from the Reporte above |
|
|
260
|
+
| `qa-investigador` | Investigates one concrete root cause against the real environment and returns a verdict with evidence; drives a browser over Playwright MCP when lanes provide one |
|
|
261
|
+
|
|
262
|
+
Config keys read by the agents (all optional; defaults shown):
|
|
263
|
+
|
|
264
|
+
| Key | Default | Used for |
|
|
265
|
+
|---|---|---|
|
|
266
|
+
| `integrations.azureDevOps.sprintCycle.approvalThresholdPercent` | `85` | Success rate over executed cases required for "Aprobado con observaciones" |
|
|
267
|
+
| `integrations.azureDevOps.sprintCycle.sprintDurationDays` / `manualTestingTimeboxDays` | `8` / `2` | Sprint planning agents |
|
|
268
|
+
| `integrations.azureDevOps.sprintCycle.dateFormat` / `timezone` | `dd-mm-aaaa` / `America/Santiago` | Dates in generated documents |
|
|
269
|
+
| `conventions.decimalSeparator` | derived: `,` for `es*` locales or no locale, `.` otherwise | Percentages in reports |
|
|
270
|
+
| `conventions.defectIdPrefix` | derived from `project.name` (uppercased) | Local defect candidate ids `DEF-<prefix>-NNN` |
|
|
271
|
+
| `conventions.testCaseIdPattern` | `TC-{MODULE}-{SUBMODULE}-{NUM}` | Local ids kept out of ADO-facing documents |
|
|
272
|
+
| `project.qaBaseUrl` / `loginPath` / `techStack` | "(no configurado: ...)" | Environment context for `qa-investigador` |
|
|
273
|
+
|
|
274
|
+
`qa-investigador` and `parallelLanes`:
|
|
275
|
+
|
|
276
|
+
- **0 MCP lanes** (no `parallelLanes`, `enabled: false`, or no lane with `mcpNamespace`): tools are
|
|
277
|
+
`Read, Write, Edit, Grep, Glob, Bash` and the agent investigates through code, config, logs and the
|
|
278
|
+
CLI runner, reporting browser-only hypotheses as blocked.
|
|
279
|
+
- **1 or N MCP lanes**: every lane with an `mcpNamespace` (e.g. `mcp__playwright-qauser__`) adds its
|
|
280
|
+
15 Playwright MCP browser tools, in config order, plus a lane table and the mandatory
|
|
281
|
+
`lane-lock.js heartbeat` protocol. CLI-only lanes are skipped.
|
|
282
|
+
- An invalid lane block or an `mcpNamespace` not matching `^mcp__[A-Za-z0-9_-]+__$` fails
|
|
283
|
+
`qa-investigador` only: the other five agents are still written and the command exits 1.
|
|
284
|
+
`qa-framework validate` reports the same problems.
|
|
285
|
+
|
|
286
|
+
---
|
|
287
|
+
|
|
248
288
|
## CLI Reference
|
|
249
289
|
|
|
250
290
|
```
|
package/docs/installation.md
CHANGED
|
@@ -161,8 +161,9 @@ Use environment variables:
|
|
|
161
161
|
#### Optional: ANALISIS/PLAN agent mode (Claude Code only)
|
|
162
162
|
|
|
163
163
|
Gated by `integrations.azureDevOps.sprintCycle.enabled`. When `true`, `init`/`upgrade`
|
|
164
|
-
generate
|
|
165
|
-
`qa-asesoria.md`, `qa-informe-resultados.md
|
|
164
|
+
generate six Claude Code subagents (`.claude/agents/qa-analisis.md`, `qa-plan.md`,
|
|
165
|
+
`qa-asesoria.md`, `qa-reporte-ejecucion.md`, `qa-informe-resultados.md`,
|
|
166
|
+
`qa-investigador.md`) for a sprint-centric manual testing
|
|
166
167
|
workflow parallel to the 6-stage pipeline. See
|
|
167
168
|
[docs/usage-with-agent.md](usage-with-agent.md) for details.
|
|
168
169
|
|
|
@@ -176,7 +177,8 @@ workflow parallel to the 6-stage pipeline. See
|
|
|
176
177
|
"sprintDurationDays": 8,
|
|
177
178
|
"manualTestingTimeboxDays": 2,
|
|
178
179
|
"dateFormat": "dd-mm-aaaa",
|
|
179
|
-
"timezone": "America/Santiago"
|
|
180
|
+
"timezone": "America/Santiago",
|
|
181
|
+
"approvalThresholdPercent": 85
|
|
180
182
|
}
|
|
181
183
|
}
|
|
182
184
|
}
|
|
@@ -184,7 +186,12 @@ workflow parallel to the 6-stage pipeline. See
|
|
|
184
186
|
```
|
|
185
187
|
|
|
186
188
|
All `sprintCycle` fields are optional; omitted fields fall back to neutral framework
|
|
187
|
-
defaults (8-day sprint, 2-day manual testing timebox, `dd-mm-aaaa`, `America/Santiago
|
|
189
|
+
defaults (8-day sprint, 2-day manual testing timebox, `dd-mm-aaaa`, `America/Santiago`,
|
|
190
|
+
85% approval threshold over executed cases). Optional overrides under `conventions`:
|
|
191
|
+
`decimalSeparator` (`","` or `"."`, otherwise derived from the locale) and
|
|
192
|
+
`defectIdPrefix` (e.g. `"ACME"`, otherwise derived from `project.name`).
|
|
193
|
+
`qa-investigador` gets Playwright MCP browser tools only for `parallelLanes.lanes[]`
|
|
194
|
+
entries with an `mcpNamespace` (pattern `^mcp__[A-Za-z0-9_-]+__$`).
|
|
188
195
|
- `ADO_PLAN_ID` - Test Plan ID (can also go in `module-registry.json`)
|
|
189
196
|
- `ADO_SUITE_ID` - Suite ID
|
|
190
197
|
|
package/docs/usage-with-agent.md
CHANGED
|
@@ -125,26 +125,31 @@ ad-hoc QA advisory chat, and a sprint closing results report. This mode is:
|
|
|
125
125
|
|
|
126
126
|
- **Optional** - off by default.
|
|
127
127
|
- **Gated** by `integrations.azureDevOps.sprintCycle.enabled` in
|
|
128
|
-
`qa/qa-framework.config.json` (it requires Azure DevOps enabled, since
|
|
129
|
-
|
|
128
|
+
`qa/qa-framework.config.json` (it requires Azure DevOps enabled, since most of the
|
|
129
|
+
agents depend on ADO work items or an ADO-generated execution report).
|
|
130
130
|
- **Claude Code only, by design, not by omission.** Each agent is a Claude Code subagent
|
|
131
131
|
with its own `tools`/`model` frontmatter (e.g. `qa-asesoria` intentionally has no
|
|
132
132
|
`Write`/`Bash`). GitHub Copilot has no equivalent primitive, so no `.github/` artifact
|
|
133
133
|
is generated for this mode.
|
|
134
134
|
|
|
135
|
-
When enabled, `init`/`upgrade` generate
|
|
135
|
+
When enabled, `init`/`upgrade` generate six subagents under `.claude/agents/`:
|
|
136
136
|
|
|
137
137
|
| Agent | Purpose |
|
|
138
138
|
|---|---|
|
|
139
139
|
| `qa-analisis.md` | Analysis document: test universe, automation feasibility, P0-P3 prioritization, excluded universe, traceability matrix |
|
|
140
140
|
| `qa-plan.md` | Default mode: executive summary + a trace-linked test case table, executable within the configured manual-testing timebox |
|
|
141
141
|
| `qa-asesoria.md` | Chat-only QA advisory (no `Write`/`Bash`) - answers point questions, redirects to `qa-plan`/`qa-analisis` when a full document is actually needed |
|
|
142
|
-
| `qa-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
142
|
+
| `qa-reporte-ejecucion.md` | Exports the ADO test plan results, versions the raw export under `ado-export-<YYYY-MM-DD>/` and writes the sprint `Reporte-de-Ejecucion-de-Pruebas-<project>-Sprint-<N>.md` (summary + promoted findings) |
|
|
143
|
+
| `qa-informe-resultados.md` | Narrative sprint closing report built from the Reporte produced by `qa-reporte-ejecucion`; updates incrementally, never overwrites prior sections |
|
|
144
|
+
| `qa-investigador.md` | Investigates one concrete root cause against the real environment and returns a verdict with evidence. Gets Playwright MCP browser tools for every `parallelLanes` lane with an `mcpNamespace`; with no such lane it works without a browser |
|
|
145
|
+
|
|
146
|
+
Sprint duration, manual-testing timebox, date format, timezone and the approval threshold
|
|
147
|
+
(`approvalThresholdPercent`, default 85) are parameterized from
|
|
148
|
+
`integrations.azureDevOps.sprintCycle` (with neutral defaults - 8-day sprint, 2-day
|
|
149
|
+
timebox, `dd-mm-aaaa`, `America/Santiago` - when a field is omitted). The optional
|
|
150
|
+
`conventions.decimalSeparator` and `conventions.defectIdPrefix` override values otherwise
|
|
151
|
+
derived from the locale and the project name. See [docs/installation.md](installation.md)
|
|
152
|
+
for the config shape.
|
|
148
153
|
|
|
149
154
|
---
|
|
150
155
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@keber/qa-framework",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.13.1",
|
|
4
4
|
"description": "Reusable spec-driven QA framework for IDE-agent-assisted automated testing. Installable as an npm package. Provides structure, templates, agent instructions, and optional integrations for Playwright and Azure DevOps.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"qa",
|
|
@@ -25,13 +25,12 @@
|
|
|
25
25
|
"check-accents": "node scripts/check-spanish-accents.js",
|
|
26
26
|
"check-chars": "node scripts/check-forbidden-chars.js",
|
|
27
27
|
"test": "node --test test/*.test.js",
|
|
28
|
-
"version": "node scripts/sync-version.js && git add README.md .github/copilot-instructions.md"
|
|
28
|
+
"version": "node scripts/sync-version.js && git add README.md .github/copilot-instructions.md qa-framework.config.json"
|
|
29
29
|
},
|
|
30
30
|
"files": [
|
|
31
|
-
"agent-instructions/",
|
|
32
31
|
"skills/",
|
|
33
32
|
"templates/",
|
|
34
|
-
"docs
|
|
33
|
+
"docs/**/*.md",
|
|
35
34
|
"examples/",
|
|
36
35
|
"integrations/",
|
|
37
36
|
"scripts/",
|
package/qa-framework.config.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "./qa-framework.config.schema.json",
|
|
3
|
-
"frameworkVersion": "1.
|
|
3
|
+
"frameworkVersion": "1.13.1",
|
|
4
4
|
|
|
5
5
|
"project": {
|
|
6
6
|
"name": "{{PROJECT_NAME}}",
|
|
@@ -89,7 +89,8 @@
|
|
|
89
89
|
"sprintDurationDays": 8,
|
|
90
90
|
"manualTestingTimeboxDays": 2,
|
|
91
91
|
"dateFormat": "dd-mm-aaaa",
|
|
92
|
-
"timezone": "America/Santiago"
|
|
92
|
+
"timezone": "America/Santiago",
|
|
93
|
+
"approvalThresholdPercent": 85
|
|
93
94
|
}
|
|
94
95
|
}
|
|
95
96
|
},
|