@keber/qa-framework 1.11.3 → 1.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +340 -0
- package/README.md +42 -2
- package/docs/architecture.md +30 -26
- package/docs/decisions/lane-heartbeat-caller.md +86 -0
- package/docs/final-report.md +23 -23
- package/docs/folder-structure-guide.md +100 -80
- package/docs/installation.md +42 -7
- package/docs/skills-architecture.md +23 -23
- package/docs/spec-driven-philosophy.md +9 -9
- package/docs/usage-with-agent.md +88 -9
- package/docs/visuals/diag-01-ejecutivo-1.png +0 -0
- package/docs/visuals/diag-02-tactico-1.png +0 -0
- package/docs/visuals/diag-03-tecnico-1.png +0 -0
- package/docs/visuals/info-01-ejecutivo-1.png +0 -0
- package/docs/visuals/info-02-tactico-1.png +0 -0
- package/docs/visuals/info-03-tecnico-1.png +0 -0
- package/docs/visuals/pres-01-ejecutivo-1.png +0 -0
- package/docs/visuals/pres-02-tactico-1.png +0 -0
- package/docs/visuals/pres-03-tecnico-1.png +0 -0
- package/docs/visuals/qa-01-problema-1.png +0 -0
- package/docs/visuals/qa-02-que-es-1.png +0 -0
- package/docs/visuals/qa-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-04-6-archivos-1.png +0 -0
- package/docs/visuals/qa-05-humano-1.png +0 -0
- package/docs/visuals/qa-06-agente-1.png +0 -0
- package/docs/visuals/qa-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-08-traceability-1.png +0 -0
- package/docs/visuals/qa-en-01-problem-1.png +0 -0
- package/docs/visuals/qa-en-02-what-is-1.png +0 -0
- package/docs/visuals/qa-en-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-en-04-6files-1.png +0 -0
- package/docs/visuals/qa-en-05-human-1.png +0 -0
- package/docs/visuals/qa-en-06-agent-1.png +0 -0
- package/docs/visuals/qa-en-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-en-08-traceability-1.png +0 -0
- package/examples/module-example/suppliers/00-inventory.md +1 -1
- package/examples/module-example/suppliers/suppliers-create.spec.ts +8 -8
- package/integrations/ado-powershell/README.md +1 -1
- package/integrations/ado-powershell/scripts/sync-ado-titles.ps1 +2 -2
- package/integrations/playwright/README.md +4 -4
- package/package.json +7 -3
- package/qa-framework.config.json +25 -3
- package/scripts/check-forbidden-chars.js +187 -0
- package/scripts/check-spanish-accents.js +159 -0
- package/scripts/cli.js +1 -1
- package/scripts/generate.js +2 -2
- package/scripts/init.js +101 -12
- package/scripts/lib/claude-agents.js +347 -0
- package/scripts/lib/claude-commands.js +56 -0
- package/scripts/lib/module-keys.js +54 -0
- package/scripts/sync-version.js +63 -0
- package/scripts/upgrade.js +369 -20
- package/scripts/validate.js +46 -21
- package/skills/qa-ado-integration/SKILL.md +19 -19
- package/skills/qa-ado-integration/references/scripts-and-config.md +6 -6
- package/skills/qa-automation/SKILL.md +49 -27
- package/skills/qa-automation/references/config-checklist.md +4 -4
- package/skills/qa-automation/references/dom-inspection-template.js +7 -7
- package/skills/qa-automation/references/patterns.md +20 -20
- package/skills/qa-automation/references/pom-template.md +161 -0
- package/skills/qa-maintenance/SKILL.md +11 -11
- package/skills/qa-maintenance/references/update-rules.md +8 -8
- package/skills/qa-module-analysis/SKILL.md +20 -19
- package/skills/qa-module-analysis/references/exploration-checklist.md +4 -4
- package/skills/qa-module-analysis/references/output-verification.md +69 -0
- package/skills/qa-module-analysis/references/spec-file-formats.md +14 -14
- package/skills/qa-spec-generation/SKILL.md +13 -7
- package/skills/qa-test-cases/SKILL.md +11 -81
- package/skills/qa-test-cases/references/test-case-template.md +5 -5
- package/skills/qa-test-plan/SKILL.md +22 -86
- package/skills/qa-test-plan/references/plan-de-pruebas-template.md +24 -24
- package/skills/qa-test-stabilization/SKILL.md +59 -30
- package/skills/qa-test-stabilization/references/classification-protocol.md +35 -35
- package/templates/agent-next-steps.md +10 -10
- package/templates/agents/qa-analisis.md +142 -0
- package/templates/agents/qa-asesoria.md +48 -0
- package/templates/agents/qa-informe-resultados.md +236 -0
- package/templates/agents/qa-investigador.md +143 -0
- package/templates/agents/qa-plan.md +131 -0
- package/templates/agents/qa-reporte-ejecucion.md +138 -0
- package/templates/automation-scaffold/.env.example +36 -6
- package/templates/automation-scaffold/fixtures/auth.ts +27 -5
- package/templates/automation-scaffold/fixtures/base.ts +23 -1
- package/templates/automation-scaffold/fixtures/test-helpers.ts +1 -1
- package/templates/automation-scaffold/global-setup.ts +117 -48
- package/templates/automation-scaffold/playwright.config.ts +72 -32
- package/templates/automation-scaffold/scripts/global-setup-guards.js +184 -0
- package/templates/automation-scaffold/scripts/lane-config.js +269 -0
- package/templates/automation-scaffold/scripts/lane-lock.js +589 -0
- package/templates/defect-report.md +17 -3
- package/templates/execution-report.md +9 -9
- package/templates/integration-scaffold/.env.example +1 -1
- package/templates/integration-scaffold/global-setup.ts +21 -1
- package/templates/integration-scaffold/playwright.config.ts +15 -1
- package/templates/qa-framework.instructions.md +9 -8
- package/templates/qa-framework.rules.md +53 -0
- package/templates/qa-readme.md +8 -8
- package/templates/session-summary.md +2 -2
- package/templates/specification/00-inventory.md +3 -3
- package/templates/specification/01-business-rules.md +6 -6
- package/templates/specification/02-workflows.md +1 -1
- package/templates/specification/03-roles-permissions.md +1 -1
- package/templates/specification/04-test-data.md +8 -8
- package/templates/specification/05-test-scenarios.md +5 -5
- package/templates/test-case.md +3 -3
- package/templates/test-data-guidelines.md +7 -7
- package/templates/test-plan-sprint.md +13 -13
- package/templates/test-plan.md +3 -3
- package/skills/qa-test-plan/references/priority-and-feasibility.md +0 -159
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,346 @@ Versioning follows [Semantic Versioning](https://semver.org/).
|
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
+
## [1.13.0] - 2026-11-04
|
|
11
|
+
|
|
12
|
+
Dated by its release, not by its bump. `package.json` was moved to `1.13.0` on 2026-10-16,
|
|
13
|
+
but the version was never tagged or published, so everything below ships together under
|
|
14
|
+
this number. The infrastructure fixes listed in Added and Fixed were written later, on the
|
|
15
|
+
branch that unblocked this release.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- `templates/agents/qa-reporte-ejecucion.md` - new Claude Code agent that exports the ADO
|
|
20
|
+
test plan results, versions the raw export under `ado-export-<YYYY-MM-DD>/` and writes the
|
|
21
|
+
sprint `Reporte-de-Ejecucion-de-Pruebas-<project>-Sprint-<N>.md` (summary with executed,
|
|
22
|
+
success and coverage metrics, plus promoted findings).
|
|
23
|
+
- `templates/agents/qa-investigador.md` - new Claude Code agent that investigates one root
|
|
24
|
+
cause against the real environment. Its tools are generated from `parallelLanes`: each lane
|
|
25
|
+
with an `mcpNamespace` adds 15 Playwright MCP browser tools, a lane table and the mandatory
|
|
26
|
+
heartbeat protocol; with no MCP lane it gets a no-browser section instead.
|
|
27
|
+
- Agent template engine: `{{#if FLAG}}` / `{{#unless FLAG}}` blocks, single-pass placeholder
|
|
28
|
+
replacement and a guard that fails on any unresolved `{{...}}` token.
|
|
29
|
+
- New placeholders: `INVESTIGATOR_TOOLS`, `LANE_MCP_TABLE`, `APPROVAL_THRESHOLD_PERCENT`,
|
|
30
|
+
`APPROVAL_THRESHOLD_RATIO`, `DECIMAL_SEPARATOR`, `TC_ID_PATTERN`, `QA_BASE_URL`,
|
|
31
|
+
`LOGIN_PATH`, `TECH_STACK`.
|
|
32
|
+
- Config keys: `integrations.azureDevOps.sprintCycle.approvalThresholdPercent` (default 85),
|
|
33
|
+
optional `conventions.decimalSeparator` and `conventions.defectIdPrefix`. Documented in
|
|
34
|
+
`qa-framework.config.schema.json`, together with the `mcpNamespace` pattern.
|
|
35
|
+
- `qa-framework validate` checks present-but-invalid values of those keys and of
|
|
36
|
+
`parallelLanes`, and now finds the config under `qa/` as well as at the project root.
|
|
37
|
+
- **`scripts/lib/module-keys.js`** - single source of truth for how a module key maps
|
|
38
|
+
to a spec folder (prefixed) and to an automation folder (bare). Shared by `init.js`
|
|
39
|
+
and `validate.js` so the two cannot drift apart again.
|
|
40
|
+
- **`scripts/sync-version.js`** - now also rewrites `frameworkVersion` in
|
|
41
|
+
`qa-framework.config.json`, by field rather than by the loose `vX.Y.Z` prose pattern.
|
|
42
|
+
- **`test/module-keys.test.js`, `test/sync-version.test.js`** - unit coverage for the
|
|
43
|
+
key helpers plus a drift guard that fails CI whenever a published version surface
|
|
44
|
+
disagrees with `package.json`.
|
|
45
|
+
- **`.github/workflows/publish.yml`** - the suite now runs before `npm publish`.
|
|
46
|
+
`ci.yml` triggers on pushes to `main` and on pull requests, so it never sees a tag,
|
|
47
|
+
and publishing is triggered by a tag push: nothing was verified on the path that
|
|
48
|
+
actually reaches npm. The drift guard above only protects a release once it runs
|
|
49
|
+
there.
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- `init` and `upgrade` generate six agents (was four) when `sprintCycle.enabled` is true.
|
|
54
|
+
Agents are rendered independently: an invalid lane config fails `qa-investigador` only,
|
|
55
|
+
the other five are still written and the command exits 1.
|
|
56
|
+
- As npm postinstall (`init --skip-if-exists`), an agent that cannot be rendered is reported
|
|
57
|
+
as a WARNING naming the agent and asking to fix the config and run
|
|
58
|
+
`npx qa-framework upgrade`; postinstall exits 0 so `npm install` does not fail. Explicit
|
|
59
|
+
`init` and `upgrade` keep exit code 1.
|
|
60
|
+
- `qa-informe-resultados` reads its approval threshold from config instead of a hardcoded
|
|
61
|
+
85% and states the decimal separator rule.
|
|
62
|
+
|
|
63
|
+
### Fixed
|
|
64
|
+
|
|
65
|
+
- `qa-informe-resultados` looked for `Reporte-de-Ejecucion-de-Pruebas-...` and pointed to the
|
|
66
|
+
`qa-ado-integration` skill, which does not produce that file. It now names the
|
|
67
|
+
`qa-reporte-ejecucion` agent, the canonical filename (active and historical sprint folders)
|
|
68
|
+
and accepts the older `Reporte-ejecución-de-pruebas-*` files.
|
|
69
|
+
- `templates/qa-framework.instructions.md` and `templates/qa-framework.rules.md` carried a
|
|
70
|
+
project-specific `DEF-SIS-NNN` example; it is now `DEF-{PREFIX}-NNN`.
|
|
71
|
+
- **`scripts/init.js`** - module spec folders are now scaffolded as
|
|
72
|
+
`qa/01-specifications/module-{name}/submodule-{name}/`, the layout declared by every
|
|
73
|
+
skill, by `docs/folder-structure-guide.md` and by the `specPath` field of
|
|
74
|
+
`qa-framework.config.json`. init.js wrote the bare key, so a freshly scaffolded
|
|
75
|
+
project violated the spec-path convention its own agent instructions impose.
|
|
76
|
+
Prefixing is idempotent: a config whose key already reads `module-suppliers` is not
|
|
77
|
+
turned into `module-module-suppliers`.
|
|
78
|
+
- **`scripts/init.js`** - automation stubs and test case ids keep using the **bare**
|
|
79
|
+
key (`e2e/tests/suppliers/create.spec.ts`, `TC-SUPPLIERS-CREATE-001`), matching the
|
|
80
|
+
`automationPath` field and the live consuming projects. Only the spec tree is
|
|
81
|
+
prefixed.
|
|
82
|
+
- **`scripts/validate.js`** - `--strict` strips the `module-` / `submodule-` prefix
|
|
83
|
+
before deriving the automation path. It was pointing the warning at
|
|
84
|
+
`e2e/tests/module-{name}/`, a folder no project has ever created.
|
|
85
|
+
- **`qa-framework.config.json`** - `frameworkVersion` was stuck at `1.10.0` while the
|
|
86
|
+
package shipped `1.12.0`. The file is published and is the template copied into every
|
|
87
|
+
consuming project, where the value is injected into `{{VERSION}}` of the generated
|
|
88
|
+
agent instructions, so every fresh install since 1.11.0 reported the wrong version.
|
|
89
|
+
- **`package.json`** - the npm `version` lifecycle script now also stages
|
|
90
|
+
`qa-framework.config.json`. sync-version.js rewrites it, but the staging list did not
|
|
91
|
+
include it, so the release commit and its tag would have carried the OLD
|
|
92
|
+
frameworkVersion - and the publish workflow builds from a clean checkout of that tag,
|
|
93
|
+
reintroducing the exact drift this change set out to fix.
|
|
94
|
+
- **`test/sync-version.test.js`** - two tests compared files against their starting bytes,
|
|
95
|
+
which assumed the tree was already synced. On a drifted tree one of them ran
|
|
96
|
+
`sync-version` and left `qa-framework.config.json` rewritten: the first run failed, the
|
|
97
|
+
retry passed, and the drift vanished from the working tree unnoticed. A suite that edits
|
|
98
|
+
tracked files turns a red run green on the retry and hides what it exists to catch.
|
|
99
|
+
- **`test/init.test.js`** - the agent summary assertion matched only the forward-slash form
|
|
100
|
+
of `.claude/agents/` and took the first matching line. `init.js` prints those lines
|
|
101
|
+
through `path.relative`, so on Windows the per-file `[created]` lines never matched and it
|
|
102
|
+
landed on the summary line, while on Linux it landed on the first `[created]` line. The
|
|
103
|
+
test passed locally and failed the first time it ran on CI.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## [1.12.0] - 2026-09-08
|
|
108
|
+
|
|
109
|
+
Reconstructed from `4fe32fb..f52257a` (39 commits). Minor rather than major: a project
|
|
110
|
+
with no `parallelLanes` block behaves exactly as before, `PENDING-BROWSER` adds an enum
|
|
111
|
+
value without removing any, and the spec-path change affects what `init` generates from
|
|
112
|
+
then on, not what already exists. The one user-visible break is `engines` moving from
|
|
113
|
+
`>=18` to `>=20`.
|
|
114
|
+
|
|
115
|
+
### Added
|
|
116
|
+
|
|
117
|
+
- **`parallelLanes` lane table** - `qa-framework.config.json` gains a lane block and
|
|
118
|
+
`qa-framework.config.schema.json` is published alongside it. A lane is four coupled
|
|
119
|
+
values (account env prefix, `storageState`, MCP namespace, runner project), none of
|
|
120
|
+
which can be inferred from the others, plus per-lane capabilities: a project can have
|
|
121
|
+
six accounts but only three MCP servers. A project with no block, or `enabled: false`,
|
|
122
|
+
resolves a single implicit default lane.
|
|
123
|
+
- **`templates/automation-scaffold/scripts/lane-lock.js`** - config-driven lane lock
|
|
124
|
+
ported from the reference implementation. `acquire` (random free lane, queues) and
|
|
125
|
+
`reserve` (exact lane, no queue, throws when busy) stay two separate verbs; staleness
|
|
126
|
+
is a composite predicate (past the threshold AND no recent heartbeat); the lock
|
|
127
|
+
directory resolves by shell > `.env` > per-checkout precedence and the resolved path
|
|
128
|
+
and its source are reported on stderr. Reads `.env` with `dotenv.parse()`, never
|
|
129
|
+
`config()`, so a script whose only job is choosing a directory never gains ambient
|
|
130
|
+
access to the QA account passwords in the same file.
|
|
131
|
+
- **`templates/automation-scaffold/scripts/global-setup-guards.js`** - `global-setup`
|
|
132
|
+
performs the only login, so it is the last point at which a run about to drive an
|
|
133
|
+
account it does not hold can be stopped. `assertColdSetupIsIntentional` throws before
|
|
134
|
+
any login when neither `QA_LANE_ONLY` nor `QA_COLD_SETUP=1` is set, since a cold setup
|
|
135
|
+
re-logs in every lane and invalidates other running sessions; `assertLaneIsLocked`
|
|
136
|
+
verifies outside CI that the lane is held; `assertProjectsMatchLane` catches a
|
|
137
|
+
`--project` whose `storageState` points at a different lane than the one reserved.
|
|
138
|
+
Plain CommonJS so it is testable with `node --test` without adding `ts-node`.
|
|
139
|
+
- **`templates/automation-scaffold/scripts/lane-config.js`** - lane table resolver shared
|
|
140
|
+
by the config and the guards.
|
|
141
|
+
- **`templates/qa-framework.rules.md`** plus **`templates/agents/`** - four agent
|
|
142
|
+
templates (`qa-analisis.md`, `qa-asesoria.md`, `qa-plan.md`,
|
|
143
|
+
`qa-informe-resultados.md`) for analysis, sprint advisory, test planning and ADO result
|
|
144
|
+
reporting, deployed through `scripts/lib/claude-agents.js` and
|
|
145
|
+
`scripts/lib/claude-commands.js`.
|
|
146
|
+
- **`scripts/check-forbidden-chars.js`** (`npm run check-chars`) - enforces the BLOCKING
|
|
147
|
+
character-safety rule the package stated but never checked. Flags em-dash, en-dash,
|
|
148
|
+
ellipsis, smart quotes and arrows; leaves Latin Extended and emoji alone. Reports
|
|
149
|
+
`file:line:column` with the code point and its ASCII replacement.
|
|
150
|
+
- **`scripts/check-spanish-accents.js`** (`npm run check-accents`) - language-aware accent
|
|
151
|
+
gate for Markdown. Classification is the hard part, not detection: it measures the
|
|
152
|
+
density of Spanish function words in prose only, after stripping fenced code, inline
|
|
153
|
+
code, link targets, tags, placeholders and paths. Against this package's corpus English
|
|
154
|
+
lands at 0.000-0.001 and Spanish templates at 0.125-0.165, with the threshold at 0.07;
|
|
155
|
+
a test asserts the gap stays wide.
|
|
156
|
+
- **`scripts/sync-version.js`** - wired into the `npm version` lifecycle to keep
|
|
157
|
+
`README.md` and `.github/copilot-instructions.md` aligned with `package.json`.
|
|
158
|
+
- **Test suite and CI** - black-box tests for `cli.js`, `init.js`, `validate.js`,
|
|
159
|
+
`upgrade.js`, the lane scripts and both character gates, using Node's built-in runner
|
|
160
|
+
with no new devDependency, plus `.github/workflows/ci.yml`.
|
|
161
|
+
- **`scripts/upgrade.js` section 9** - carries the v1.12.0 scaffold into existing
|
|
162
|
+
projects. `upgrade` force-copied only `.github/skills/`, `copilot-instructions.md` and
|
|
163
|
+
`QA-STRUCTURE-GUIDE.md`, so everything this release changed under `templates/` reached
|
|
164
|
+
new projects through `init.js` alone and existing installations kept the old scaffold,
|
|
165
|
+
including the login that types the password with `fill()`. The rule is per file:
|
|
166
|
+
overwrite what is framework-owned, warn and continue where overwriting would destroy
|
|
167
|
+
working user code, and treat a dependency group as all-or-nothing, since
|
|
168
|
+
`playwright.config.ts` and `global-setup.ts` both `require()` the lane scripts and a
|
|
169
|
+
partial install leaves a project that cannot start Playwright. Baseline detection is
|
|
170
|
+
content-hash based, not `frameworkVersion` based: the recorded values in live projects
|
|
171
|
+
were `1.10.0`, `1.1.3`, `1.1.3` and `1.0.0`, two of them malformed. Files left
|
|
172
|
+
user-owned are scanned for `.fill(password)` and reported with file and line, without a
|
|
173
|
+
textual patch.
|
|
174
|
+
- **`PENDING-BROWSER`** - fourth value in the TC Origin vocabulary, for an element
|
|
175
|
+
observed and confirmed to exist whose behavior could not be exercised end to end.
|
|
176
|
+
Without it agents either overclaim with `UI-OBSERVED` or misuse `PENDING-CODE`, which
|
|
177
|
+
pollutes gap tracking downstream.
|
|
178
|
+
- **Known-issues lookup before defect creation** - nothing required checking whether a
|
|
179
|
+
defect had already been filed. The gate distinguishes the two outcomes that matter: an
|
|
180
|
+
open match is referenced instead of refiled, a closed match is filed as a regression.
|
|
181
|
+
`templates/defect-report.md` gains the section that records the search.
|
|
182
|
+
- **Stage 5 gates** - `tsc --noEmit` plus a single smoke run of a newly written spec
|
|
183
|
+
before the submodule is marked complete. Both run before the completion checklist; a
|
|
184
|
+
full green suite remains Stage 5b's job.
|
|
185
|
+
- **Closing measurement for doc-generating stages** - the four skills that write Markdown
|
|
186
|
+
now end on evidence (accent count, forbidden characters, BOM) rather than a compliance
|
|
187
|
+
claim, with the language boundary stated so English artifacts are not asked for accents
|
|
188
|
+
they should not have.
|
|
189
|
+
- **`skills/qa-automation/references/pom-template.md`** - page-object template taken from
|
|
190
|
+
real project use: an abstract per-module CRUD base plus thin per-submodule classes,
|
|
191
|
+
with the accent-insensitive label matching that pattern needs against Spanish UIs.
|
|
192
|
+
- **`MIGRATION-NOTES.md`**, **`docs/decisions/lane-heartbeat-caller.md`** and
|
|
193
|
+
**`docs/visuals/`** - embedded-QA migration guidance, the record that the dispatcher
|
|
194
|
+
(not the executor) calls heartbeat, and the diagrams referenced from the architecture
|
|
195
|
+
docs.
|
|
196
|
+
|
|
197
|
+
### Changed
|
|
198
|
+
|
|
199
|
+
- **`engines`** - `>=18` becomes `>=20`, and CI tests Node 20, 22 and 24. The matrix had
|
|
200
|
+
been testing 18 and 20, both out of maintenance, while `publish.yml` released on 24,
|
|
201
|
+
which no job tested. Users on 18 now get an npm warning, which is the point.
|
|
202
|
+
- **`templates/*/playwright.config.ts`** - both configs called `dotenv.config()`, which
|
|
203
|
+
injects the entire `.env` into `process.env`. In a real project that file holds every QA
|
|
204
|
+
account password plus ADO tokens (82 lines in one case) while these configs need a
|
|
205
|
+
handful of non-secret values. They now read an explicit key list with `dotenv.parse()`,
|
|
206
|
+
which returns a plain object and never touches `process.env`, and only fill keys the
|
|
207
|
+
environment has not already set, so shell and CI variables still win.
|
|
208
|
+
- **`templates/automation-scaffold/playwright.config.ts`** - runner projects and auth are
|
|
209
|
+
generated from the lane table, replacing the commented-out
|
|
210
|
+
`chromium-user1` / `chromium-user2` block that sat outside the `projects` array and
|
|
211
|
+
could not have worked as written. `storageState` is derived through lane-lock rather
|
|
212
|
+
than hardcoded: a parallel hardcoded mapping caused real evidence corruption once, a run
|
|
213
|
+
recorded under the wrong account because the project was pinned to one lane while
|
|
214
|
+
`QA_LANE_ONLY` named another.
|
|
215
|
+
- **`global-setup.ts`** - `resolveAuthUsers` resolved lanes by substring
|
|
216
|
+
(`storageState.includes('user-2')`, which also matches `user-20`) and had no branch for
|
|
217
|
+
a third lane. It now matches the lane table exactly and reads credentials from each
|
|
218
|
+
lane's declared account prefix.
|
|
219
|
+
- **Agent behavior rules** - a BLOCKING lane-lock rule states the requirement only
|
|
220
|
+
(acquire before dispatching, release when done, dispatcher keeps the lane alive, queue
|
|
221
|
+
or reject at the cap, never force past it), agent-agnostic so each agent's adaptation
|
|
222
|
+
prescribes its own mechanism.
|
|
223
|
+
- **`qa-test-plan` / `qa-test-cases`** - skill content consolidated into the templates;
|
|
224
|
+
`priority-and-feasibility.md` merged into `plan-de-pruebas-template.md`.
|
|
225
|
+
- **Typographic characters** - em-dashes, en-dashes, arrows, smart quotes and ellipses
|
|
226
|
+
replaced with ASCII across `scripts/`, `skills/`, `templates/`, `docs/`, `examples/`,
|
|
227
|
+
`integrations/` and the PowerShell scripts. Substitution is context-sensitive: padded
|
|
228
|
+
prose dashes become ` - `, numeric and label ranges keep a tight hyphen, standalone
|
|
229
|
+
table cells become a bare `-`, and ASCII rules keep their original column width.
|
|
230
|
+
`CHANGELOG.md` keeps its historical occurrences deliberately - rewriting past entries to
|
|
231
|
+
satisfy a rule adopted later would falsify the log.
|
|
232
|
+
|
|
233
|
+
### Fixed
|
|
234
|
+
|
|
235
|
+
- **`scripts/init.js`** - module specs are scaffolded under
|
|
236
|
+
`qa/01-specifications/{module}/{submodule}/`. init wrote them to `qa/{module}/`, leaving
|
|
237
|
+
`01-specifications/` permanently empty while scattering specs as siblings of the
|
|
238
|
+
numbered folders. The existing tests ran with no config, so the module loop iterated
|
|
239
|
+
zero times and no spec path was ever asserted.
|
|
240
|
+
- **`scripts/validate.js`** - check 2 walked all of `qa/`, treating every top-level
|
|
241
|
+
directory as a module, so it read `01-specifications/` as the module and demanded the 6
|
|
242
|
+
spec files one level above where they exist. The scan is now rooted at
|
|
243
|
+
`01-specifications/`, which also stops sibling folders such as `02-test-plans/` and
|
|
244
|
+
`memory/` from being mistaken for submodules; the `SKIP_DIRS` denylist is removed rather
|
|
245
|
+
than extended.
|
|
246
|
+
- **`scripts/validate.js`** - `--strict` looked for automation specs at
|
|
247
|
+
`e2e/{module}/{submodule}.spec.ts`, a path nothing creates. `init.js` writes to
|
|
248
|
+
`e2e/tests/{module}/` and `upgrade.js` documents the v1.6.0 migration that moved them
|
|
249
|
+
there, so `--strict` emitted a permanently false warning for every submodule in all
|
|
250
|
+
three live projects.
|
|
251
|
+
- **`scripts/validate.js`** - stopped requiring `05-test-plans`, a stale name from an
|
|
252
|
+
older convention; the current structure uses `05-test-execution` with test plans under
|
|
253
|
+
`02-test-plans`.
|
|
254
|
+
- **`templates/*/fixtures` and both scaffolds** - the four remaining `fill(password)` call
|
|
255
|
+
sites are listed under Security below.
|
|
256
|
+
- **`templates/agents/`** - the four Spanish-language agent templates shipped entirely
|
|
257
|
+
accent-stripped (144, 190, 88 and 52 Spanish markers, 0 to 2 accented characters between
|
|
258
|
+
them). This is functional, not cosmetic: page objects locate fields by visible label, so
|
|
259
|
+
a spec written as `Codigo` never matches a UI rendering the accented form. In one
|
|
260
|
+
project that cost 24 of 39 failing smoke tests and two wrong mitigations before the
|
|
261
|
+
cause was found.
|
|
262
|
+
- **`templates/defect-report.md`** - the defect ID prefix is a placeholder. The template
|
|
263
|
+
hardcoded `DEF-{{NNN}}` while `defectIdPattern` carried `DEF-{NUM}`, and real projects
|
|
264
|
+
use a project prefix such as `DEF-SIS-014`.
|
|
265
|
+
- **`templates/qa-framework.instructions.md` and `qa-framework.rules.md`** - referenced
|
|
266
|
+
`integrations.ado.enabled`, a config path that never existed. The real schema is
|
|
267
|
+
`integrations.azureDevOps.enabled`.
|
|
268
|
+
- **`skills/qa-spec-generation`** - pointed at `references/spec-file-formats.md`, which
|
|
269
|
+
only exists under `qa-module-analysis/references/`, so Stage 2 could not find the
|
|
270
|
+
template it is told to use. `skills/qa-automation` referenced a `pom-template.md` that
|
|
271
|
+
did not exist.
|
|
272
|
+
- **`qa-test-stabilization`** - the official suite is the only source of truth. A polling
|
|
273
|
+
diagnostic script saw a toast 4 of 4 times while the official suite saw none, 6 of 6
|
|
274
|
+
across two independent runs, and the defect had already been closed on that standalone
|
|
275
|
+
evidence. Standalone output is now diagnostic input, never a verdict.
|
|
276
|
+
- **`templates/*/.env.example`** - both files carried a UTF-8 BOM and three em-dashes. A
|
|
277
|
+
BOM in a `.env` is not cosmetic: some parsers read the first key as `QA_BASE_URL`
|
|
278
|
+
and silently fail to find it, and these files ship to every consuming project as the
|
|
279
|
+
template developers copy to `.env`. The character gate's extension list gains
|
|
280
|
+
`.example` and `.ps1`, both previous blind spots.
|
|
281
|
+
- **CI** - the test script used `test/**/*.test.js`, which needs globstar the runner's
|
|
282
|
+
shell does not enable, and the follow-up fix kept the pattern quoted so node received a
|
|
283
|
+
literal path. The validate step ran against this repository's own gitignored `qa/`
|
|
284
|
+
directory and failed with four missing-folder errors; it now scaffolds a throwaway
|
|
285
|
+
project and validates that, which exercises the contract that actually matters - what
|
|
286
|
+
`init.js` generates must pass `validate.js`.
|
|
287
|
+
|
|
288
|
+
### Security
|
|
289
|
+
|
|
290
|
+
- **Password no longer reaches Playwright traces.** Both scaffolds typed the QA password
|
|
291
|
+
with `locator().fill(password)`, and the automation scaffold ships `trace` and `video`
|
|
292
|
+
set to `retain-on-failure`, so any failed CI run persisted an artifact containing the
|
|
293
|
+
plaintext password. All four call sites - `global-setup.ts` in both scaffolds,
|
|
294
|
+
`fixtures/base.ts` (TTL-aware reauth) and `fixtures/auth.ts` (`loginAs`) - now set the
|
|
295
|
+
value through `page.evaluate()`, the mechanism this package already documented as
|
|
296
|
+
"Pattern 7: Password Injection (Trace Safety)". The fixture sites carried more exposure
|
|
297
|
+
than `global-setup`, which runs once per run, since reauth runs for every test past the
|
|
298
|
+
TTL and `loginAs` for every test that logs in as a role. Bubbling `input` and `change`
|
|
299
|
+
events are dispatched after the assignment because a bare `.value` write does not notify
|
|
300
|
+
the Blazor/Radzen-style frameworks these scaffolds target, and an explicit visible
|
|
301
|
+
`waitFor` replaces the auto-wait `fill()` provided. Email `fill()` calls are left in
|
|
302
|
+
place: emails are not secret and aid debugging. There are no remaining
|
|
303
|
+
`fill(password)` call sites in `templates/`.
|
|
304
|
+
- **`.env` is no longer loaded wholesale** by the Playwright configs or the lane lock. See
|
|
305
|
+
Changed above.
|
|
306
|
+
|
|
307
|
+
---
|
|
308
|
+
|
|
309
|
+
## [1.11.3] - 2026-06-24
|
|
310
|
+
|
|
311
|
+
Reconstructed from `5507f1a..4fe32fb`. Release plumbing only.
|
|
312
|
+
|
|
313
|
+
### Changed
|
|
314
|
+
|
|
315
|
+
- **`.github/workflows/publish.yml`** - `actions/checkout` and `actions/setup-node` move
|
|
316
|
+
from v4 to v6, and the publish job's Node version from 20 to 24.
|
|
317
|
+
|
|
318
|
+
### Removed
|
|
319
|
+
|
|
320
|
+
- **`NODE_AUTH_TOKEN` / `secrets.NPM_TOKEN`** from the publish step, completing the move
|
|
321
|
+
to npm trusted publishing begun in 1.11.1: the job now authenticates with the
|
|
322
|
+
short-lived OIDC token the `id-token: write` permission grants, so no long-lived npm
|
|
323
|
+
token is stored in the repository.
|
|
324
|
+
|
|
325
|
+
No package content changed. Nothing reached consuming projects.
|
|
326
|
+
|
|
327
|
+
---
|
|
328
|
+
|
|
329
|
+
## [1.11.2] - 2026-06-24
|
|
330
|
+
|
|
331
|
+
Version bump only - a republish of 1.11.1 after a failed publish run. No source,
|
|
332
|
+
template, skill or script change; the diff is `package.json` and `package-lock.json`.
|
|
333
|
+
|
|
334
|
+
---
|
|
335
|
+
|
|
336
|
+
## [1.11.1] - 2026-06-24
|
|
337
|
+
|
|
338
|
+
Reconstructed from `3a29617..59f6fe0`. Release plumbing only.
|
|
339
|
+
|
|
340
|
+
### Added
|
|
341
|
+
|
|
342
|
+
- **`.github/workflows/publish.yml`** - `id-token: write` permission on the publish job,
|
|
343
|
+
the prerequisite for npm trusted publishing. The long-lived token it replaces is dropped in
|
|
344
|
+
1.11.3.
|
|
345
|
+
|
|
346
|
+
No package content changed. Nothing reached consuming projects.
|
|
347
|
+
|
|
348
|
+
---
|
|
349
|
+
|
|
10
350
|
## [1.11.0] - 2026-06-24
|
|
11
351
|
|
|
12
352
|
### Added
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# qa-framework
|
|
2
2
|
|
|
3
3
|
> Reusable, installable, agent-oriented QA framework for spec-driven automated testing.
|
|
4
|
-
> Current version: **v1.
|
|
4
|
+
> Current version: **v1.13.0**
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -176,7 +176,7 @@ qa/
|
|
|
176
176
|
├── {module}/{submodule}/ <- 6-file spec sets created per module/submodule in config
|
|
177
177
|
├── 02-test-plans/ <- Test plans (automated + manual)
|
|
178
178
|
├── 03-test-cases/ <- TC-*.md step-by-step docs (manual track; optional if fully automated)
|
|
179
|
-
├── 04-test-data/ <- Shared test data factories/seeders (optional
|
|
179
|
+
├── 04-test-data/ <- Shared test data factories/seeders (optional - per-module data lives in 01-specs)
|
|
180
180
|
├── 05-test-execution/ <- Execution reports and results
|
|
181
181
|
├── 06-defects/open|resolved/ <- Defect tracking
|
|
182
182
|
├── 07-automation/ <- Playwright automation code and config
|
|
@@ -245,6 +245,46 @@ See [docs/skills-architecture.md](docs/skills-architecture.md) for the full desi
|
|
|
245
245
|
|
|
246
246
|
---
|
|
247
247
|
|
|
248
|
+
## Claude Code Sprint Agents (optional)
|
|
249
|
+
|
|
250
|
+
When `integrations.azureDevOps.sprintCycle.enabled` is `true`, `init` (only if missing) and
|
|
251
|
+
`upgrade` (always, framework-owned) generate six Claude Code subagents under `.claude/agents/`:
|
|
252
|
+
|
|
253
|
+
| Agent | What it does |
|
|
254
|
+
|---|---|
|
|
255
|
+
| `qa-analisis` | Sprint analysis document: test universe, automation feasibility, P0-P3 prioritization, traceability matrix |
|
|
256
|
+
| `qa-plan` | Sprint test plan: executive summary plus a trace-linked test case table that fits the manual-testing timebox |
|
|
257
|
+
| `qa-asesoria` | Chat-only QA advisory (no `Write`/`Bash`) |
|
|
258
|
+
| `qa-reporte-ejecucion` | Exports the ADO test plan results (via the `ado-qa` `getADO.ps1` script), versions the raw export by date and writes `Reporte-de-Ejecucion-de-Pruebas-<project>-Sprint-<N>.md` |
|
|
259
|
+
| `qa-informe-resultados` | Narrative sprint closing report (metrics, failures, risks, QA verdict) built from the Reporte above |
|
|
260
|
+
| `qa-investigador` | Investigates one concrete root cause against the real environment and returns a verdict with evidence; drives a browser over Playwright MCP when lanes provide one |
|
|
261
|
+
|
|
262
|
+
Config keys read by the agents (all optional; defaults shown):
|
|
263
|
+
|
|
264
|
+
| Key | Default | Used for |
|
|
265
|
+
|---|---|---|
|
|
266
|
+
| `integrations.azureDevOps.sprintCycle.approvalThresholdPercent` | `85` | Success rate over executed cases required for "Aprobado con observaciones" |
|
|
267
|
+
| `integrations.azureDevOps.sprintCycle.sprintDurationDays` / `manualTestingTimeboxDays` | `8` / `2` | Sprint planning agents |
|
|
268
|
+
| `integrations.azureDevOps.sprintCycle.dateFormat` / `timezone` | `dd-mm-aaaa` / `America/Santiago` | Dates in generated documents |
|
|
269
|
+
| `conventions.decimalSeparator` | derived: `,` for `es*` locales or no locale, `.` otherwise | Percentages in reports |
|
|
270
|
+
| `conventions.defectIdPrefix` | derived from `project.name` (uppercased) | Local defect candidate ids `DEF-<prefix>-NNN` |
|
|
271
|
+
| `conventions.testCaseIdPattern` | `TC-{MODULE}-{SUBMODULE}-{NUM}` | Local ids kept out of ADO-facing documents |
|
|
272
|
+
| `project.qaBaseUrl` / `loginPath` / `techStack` | "(no configurado: ...)" | Environment context for `qa-investigador` |
|
|
273
|
+
|
|
274
|
+
`qa-investigador` and `parallelLanes`:
|
|
275
|
+
|
|
276
|
+
- **0 MCP lanes** (no `parallelLanes`, `enabled: false`, or no lane with `mcpNamespace`): tools are
|
|
277
|
+
`Read, Write, Edit, Grep, Glob, Bash` and the agent investigates through code, config, logs and the
|
|
278
|
+
CLI runner, reporting browser-only hypotheses as blocked.
|
|
279
|
+
- **1 or N MCP lanes**: every lane with an `mcpNamespace` (e.g. `mcp__playwright-qauser__`) adds its
|
|
280
|
+
15 Playwright MCP browser tools, in config order, plus a lane table and the mandatory
|
|
281
|
+
`lane-lock.js heartbeat` protocol. CLI-only lanes are skipped.
|
|
282
|
+
- An invalid lane block or an `mcpNamespace` not matching `^mcp__[A-Za-z0-9_-]+__$` fails
|
|
283
|
+
`qa-investigador` only: the other five agents are still written and the command exits 1.
|
|
284
|
+
`qa-framework validate` reports the same problems.
|
|
285
|
+
|
|
286
|
+
---
|
|
287
|
+
|
|
248
288
|
## CLI Reference
|
|
249
289
|
|
|
250
290
|
```
|
package/docs/architecture.md
CHANGED
|
@@ -2,18 +2,18 @@
|
|
|
2
2
|
|
|
3
3
|
## Framework Architecture
|
|
4
4
|
|
|
5
|
-
**Version**: 1.
|
|
6
|
-
**Date**: 2026-
|
|
5
|
+
**Version**: 1.11.4
|
|
6
|
+
**Date**: 2026-07-08
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
## Design Goals
|
|
11
11
|
|
|
12
|
-
1. **Decoupled core**
|
|
13
|
-
2. **Layered optionality**
|
|
14
|
-
3. **Agent-first design**
|
|
15
|
-
4. **Spec-before-automation**
|
|
16
|
-
5. **Parameterization over hardcoding**
|
|
12
|
+
1. **Decoupled core** - the framework works without Playwright, without Azure DevOps, without any specific CI/CD system
|
|
13
|
+
2. **Layered optionality** - features are added as explicit opt-in integrations, not baked into the core
|
|
14
|
+
3. **Agent-first design** - every convention exists so that an IDE agent can navigate and produce artifacts predictably
|
|
15
|
+
4. **Spec-before-automation** - the specification layer is always the source of truth; automation references specs, never the reverse
|
|
16
|
+
5. **Parameterization over hardcoding** - project-specific values live in `qa-framework.config.json`, not in framework files
|
|
17
17
|
|
|
18
18
|
---
|
|
19
19
|
|
|
@@ -84,8 +84,10 @@
|
|
|
84
84
|
│ ├── 08-azure-integration/ <- optional ADO integration │
|
|
85
85
|
│ └── memory/ <- agent learnings (project-maintained) │
|
|
86
86
|
│ │
|
|
87
|
-
│ .github/
|
|
87
|
+
│ .github/instructions/qa-framework.instructions.md <- generated (framework) │
|
|
88
88
|
│ .github/skills/ <- copied from package (framework-owned) │
|
|
89
|
+
│ .claude/rules/qa-framework.md <- generated (framework, Claude Code) │
|
|
90
|
+
│ .claude/commands/qa-*.md <- generated (framework, Claude Code) │
|
|
89
91
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
90
92
|
```
|
|
91
93
|
|
|
@@ -98,12 +100,12 @@ Module analysis (agent)
|
|
|
98
100
|
│
|
|
99
101
|
▼
|
|
100
102
|
01-specifications/module-X/submodule-Y/
|
|
101
|
-
├── 00-inventory.md
|
|
102
|
-
├── 01-business-rules.md
|
|
103
|
-
├── 02-workflows.md
|
|
103
|
+
├── 00-inventory.md <- what exists in the UI
|
|
104
|
+
├── 01-business-rules.md <- RN-* identifiers
|
|
105
|
+
├── 02-workflows.md <- FL-* flowcharts
|
|
104
106
|
├── 03-roles-permissions.md
|
|
105
107
|
├── 04-test-data.md
|
|
106
|
-
└── 05-test-scenarios.md
|
|
108
|
+
└── 05-test-scenarios.md <- TC-* identifiers ──────────────────────┐
|
|
107
109
|
│
|
|
108
110
|
│ │
|
|
109
111
|
▼ ▼
|
|
@@ -120,8 +122,8 @@ Module analysis (agent)
|
|
|
120
122
|
│
|
|
121
123
|
▼
|
|
122
124
|
[ADO enabled?]
|
|
123
|
-
YES
|
|
124
|
-
NO
|
|
125
|
+
YES -> playwright-azure-reporter syncs results to ADO Test Plan
|
|
126
|
+
NO -> 05-test-execution/automated/{date}.md (local report)
|
|
125
127
|
│
|
|
126
128
|
▼
|
|
127
129
|
06-defects/ (if test.skip for known bug)
|
|
@@ -132,18 +134,20 @@ Module analysis (agent)
|
|
|
132
134
|
|
|
133
135
|
## Layer Definitions
|
|
134
136
|
|
|
135
|
-
### Layer 1
|
|
137
|
+
### Layer 1 - Framework Core (mandatory)
|
|
136
138
|
|
|
137
139
|
Installed always. Contains:
|
|
138
140
|
|
|
139
141
|
- `qa/` directory skeleton (10 folders)
|
|
140
|
-
- `.github/skills/`
|
|
141
|
-
- `00-standards/`
|
|
142
|
+
- `.github/skills/` - 8 agent skill sets (3-layer model: SKILL.md + references/)
|
|
143
|
+
- `00-standards/` - naming conventions, templates
|
|
142
144
|
- `QA-STRUCTURE-GUIDE.md`
|
|
143
|
-
- `.github/
|
|
145
|
+
- `.github/instructions/qa-framework.instructions.md` - generated pipeline sequencer (Copilot)
|
|
146
|
+
- `.claude/rules/qa-framework.md` - generated pipeline sequencer (Claude Code, unconditional load)
|
|
147
|
+
- `.claude/commands/qa-*.md` - generated thin wrappers, one per skill (Claude Code)
|
|
144
148
|
- `qa-framework.config.json` schema
|
|
145
149
|
|
|
146
|
-
### Layer 2
|
|
150
|
+
### Layer 2 - Playwright Integration (opt-in)
|
|
147
151
|
|
|
148
152
|
Installed always (scaffold is always created by `init`):
|
|
149
153
|
|
|
@@ -156,7 +160,7 @@ Installed always (scaffold is always created by `init`):
|
|
|
156
160
|
- `qa/07-automation/integration/README.md` (placeholder)
|
|
157
161
|
- `qa/07-automation/load/README.md` (placeholder)
|
|
158
162
|
|
|
159
|
-
### Layer 3
|
|
163
|
+
### Layer 3 - Azure DevOps Integration (opt-in)
|
|
160
164
|
|
|
161
165
|
Installed when `integrations.azureDevOps.enabled = true`:
|
|
162
166
|
|
|
@@ -173,11 +177,11 @@ Installed when `integrations.azureDevOps.enabled = true`:
|
|
|
173
177
|
```
|
|
174
178
|
qa-framework.config.json (project-level, committed to repo)
|
|
175
179
|
│
|
|
176
|
-
├── project.*
|
|
177
|
-
├── modules[]
|
|
178
|
-
├── conventions.*
|
|
179
|
-
├── testUsers[]
|
|
180
|
-
└── integrations.*
|
|
180
|
+
├── project.* -> Display values, URLs (non-secret)
|
|
181
|
+
├── modules[] -> Module codes, paths, ADO IDs
|
|
182
|
+
├── conventions.* -> Naming patterns, TC ID format
|
|
183
|
+
├── testUsers[] -> Role->envVar mapping (NOT credentials)
|
|
184
|
+
└── integrations.* -> Feature flags + integration config
|
|
181
185
|
│
|
|
182
186
|
└── credentials come from:
|
|
183
187
|
.env (local, gitignored)
|
|
@@ -227,6 +231,6 @@ Full rules: [docs/folder-structure-guide.md](folder-structure-guide.md)
|
|
|
227
231
|
|
|
228
232
|
1. The primary test runner is Playwright. Other runners (Jest, Cypress) are not excluded but are not provided adapters in v1.0.
|
|
229
233
|
2. The target application runs in a browser. Back-end API-only testing is not the primary use case of this framework (though API testing can be added to `07-automation/` as needed).
|
|
230
|
-
3. The IDE agent is GitHub Copilot or equivalent. The instructions are written in Markdown and are IDE-agnostic.
|
|
234
|
+
3. The IDE agent is GitHub Copilot, Claude Code, or equivalent. The instructions are written in Markdown and are IDE-agnostic; `init`/`upgrade` generate native artifacts for both Copilot (`.github/instructions/`) and Claude Code (`.claude/rules/`, `.claude/commands/`) in parallel, without detecting which agent the consumer uses.
|
|
231
235
|
4. The project uses Git. The `qa/` directory lives inside the same repository as the application code (monorepo-friendly).
|
|
232
236
|
5. Credentials are always managed via environment variables. There is no fallback to hardcoded credentials in any framework file.
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# Decision: the dispatcher calls the lane heartbeat, not the executor
|
|
2
|
+
|
|
3
|
+
**Status**: accepted, not implemented
|
|
4
|
+
**Date**: 2026-09-08
|
|
5
|
+
**Scope**: the N-lane work still pending upstream (FRAMEWORK-FIXES v3 items 1-4, v5 items 1, 2, 4)
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context
|
|
10
|
+
|
|
11
|
+
Parallel QA lanes exist to work around a constraint of the applications under test:
|
|
12
|
+
one account cannot hold two sessions at once, so a second login silently invalidates
|
|
13
|
+
the first. A lane lock assigns each concurrent worker its own account.
|
|
14
|
+
|
|
15
|
+
A lock is only useful if a held lane stays held. The reference implementation
|
|
16
|
+
(`lane-lock.js` in the Sispro Exportadora project) frees a lane when it looks stale,
|
|
17
|
+
using a composite predicate: the acquisition is older than 20 minutes AND either no
|
|
18
|
+
heartbeat was ever sent OR the last heartbeat is itself older than 20 minutes. A
|
|
19
|
+
recent heartbeat proves the holder is alive, so the lane is never swept.
|
|
20
|
+
|
|
21
|
+
That predicate is only as good as whoever calls `heartbeat`.
|
|
22
|
+
|
|
23
|
+
## The incident that forced the decision
|
|
24
|
+
|
|
25
|
+
On 2026-08-22 a delegated sub-agent spent over 20 minutes on a legitimate live
|
|
26
|
+
investigation - genuine environment congestion, 80 to 90 concurrent browser and node
|
|
27
|
+
processes - and never called `heartbeat`. The sweep did exactly what it was written
|
|
28
|
+
to do and freed the lane while the sub-agent was still driving that account: the
|
|
29
|
+
precise scenario the heartbeat exists to prevent.
|
|
30
|
+
|
|
31
|
+
No cross-session collision followed, but only because a person noticed the lane was
|
|
32
|
+
free with no matching completion report, paused the sub-agent, confirmed by explicit
|
|
33
|
+
reply that nothing had run against the browser after the sweep, and re-reserved the
|
|
34
|
+
lane. The protection failed by design and was caught by attentiveness.
|
|
35
|
+
|
|
36
|
+
## Decision
|
|
37
|
+
|
|
38
|
+
**The dispatcher calls `heartbeat` from its own wait loop.** The executor is not
|
|
39
|
+
required to call it and must not be relied upon to.
|
|
40
|
+
|
|
41
|
+
## Why
|
|
42
|
+
|
|
43
|
+
The executor option fails for a structural reason, not a discipline one. A delegated
|
|
44
|
+
agent has no reason to discover `heartbeat` unless the dispatching prompt tells it,
|
|
45
|
+
every single time. That is the same failure this project has now recorded four times
|
|
46
|
+
over: a known-issues record that existed and was not consulted, an orthography rule
|
|
47
|
+
that was asserted rather than measured, and this heartbeat. A mechanism whose
|
|
48
|
+
protection depends on correctly instructing an uninstructed party is not a control.
|
|
49
|
+
|
|
50
|
+
The dispatcher, by contrast, already knows whether the task is still running - it can
|
|
51
|
+
query task state directly. The executor cannot see how long it has been between its
|
|
52
|
+
own tool calls. Putting the call where the knowledge already is removes the
|
|
53
|
+
instruction step entirely, which is where option 1 actually broke.
|
|
54
|
+
|
|
55
|
+
The cost is real and accepted: the dispatcher's wait loop becomes more complex, and
|
|
56
|
+
the concrete implementation is agent-specific rather than framework-generic.
|
|
57
|
+
|
|
58
|
+
## Layering
|
|
59
|
+
|
|
60
|
+
This follows the split established in FRAMEWORK-FIXES v3 section 5.
|
|
61
|
+
|
|
62
|
+
The agent-agnostic framework states the requirement: a lane must be acquired through
|
|
63
|
+
the lock before any worker is dispatched against it, released when the work ends, and
|
|
64
|
+
kept alive by the dispatcher while the work is in flight. Each agent's adaptation
|
|
65
|
+
prescribes the mechanism. For Claude Code that means a `/loop` wrapper with a
|
|
66
|
+
`ScheduleWakeup` fallback and a non-blocking task-state check - primitives that have
|
|
67
|
+
no equivalent in other agents, and so must not be written into
|
|
68
|
+
`templates/qa-framework.instructions.md`.
|
|
69
|
+
|
|
70
|
+
## Consequences
|
|
71
|
+
|
|
72
|
+
- The N-lane scaffold ships `heartbeat` together with the rule naming its caller.
|
|
73
|
+
Shipping the sweep without that answer reproduces the 2026-08-22 incident in every
|
|
74
|
+
project that adopts it.
|
|
75
|
+
- A lane held far longer than the application's session TTL is a signal to verify,
|
|
76
|
+
not a defect to ignore. Each project's agent-specific instructions must define how
|
|
77
|
+
it verifies a lock is still legitimately held and how it clears a stale one.
|
|
78
|
+
- Neither option was implemented in the originating project, so there is no verified
|
|
79
|
+
reference implementation of the dispatcher-side loop to lift. It has to be written.
|
|
80
|
+
|
|
81
|
+
## Sources
|
|
82
|
+
|
|
83
|
+
- `temp/FRAMEWORK-FIXES-qa-framework-v5.md` item 5, which framed the fork and
|
|
84
|
+
recommended this side.
|
|
85
|
+
- `temp/FRAMEWORK-FIXES-qa-framework-v3.md` section 5, for the agent-agnostic versus
|
|
86
|
+
agent-specific split.
|