@sun-asterisk/sungen 3.2.33 → 3.2.34-beta.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/commands/audit.d.ts +4 -0
- package/dist/cli/commands/audit.d.ts.map +1 -1
- package/dist/cli/commands/audit.js +54 -10
- package/dist/cli/commands/audit.js.map +1 -1
- package/dist/cli/commands/capability.d.ts.map +1 -1
- package/dist/cli/commands/capability.js +5 -1
- package/dist/cli/commands/capability.js.map +1 -1
- package/dist/cli/commands/changed.d.ts.map +1 -1
- package/dist/cli/commands/changed.js +2 -3
- package/dist/cli/commands/changed.js.map +1 -1
- package/dist/cli/commands/dashboard.d.ts.map +1 -1
- package/dist/cli/commands/dashboard.js +2 -28
- package/dist/cli/commands/dashboard.js.map +1 -1
- package/dist/cli/commands/defect.d.ts +7 -0
- package/dist/cli/commands/defect.d.ts.map +1 -0
- package/dist/cli/commands/defect.js +68 -0
- package/dist/cli/commands/defect.js.map +1 -0
- package/dist/cli/commands/delivery.d.ts.map +1 -1
- package/dist/cli/commands/delivery.js +75 -42
- package/dist/cli/commands/delivery.js.map +1 -1
- package/dist/cli/commands/results-merge.d.ts +9 -0
- package/dist/cli/commands/results-merge.d.ts.map +1 -1
- package/dist/cli/commands/results-merge.js +7 -0
- package/dist/cli/commands/results-merge.js.map +1 -1
- package/dist/cli/commands/select.d.ts +12 -0
- package/dist/cli/commands/select.d.ts.map +1 -0
- package/dist/cli/commands/select.js +122 -0
- package/dist/cli/commands/select.js.map +1 -0
- package/dist/cli/commands/triage.d.ts +7 -0
- package/dist/cli/commands/triage.d.ts.map +1 -0
- package/dist/cli/commands/triage.js +98 -0
- package/dist/cli/commands/triage.js.map +1 -0
- package/dist/cli/commands/update.d.ts.map +1 -1
- package/dist/cli/commands/update.js +3 -0
- package/dist/cli/commands/update.js.map +1 -1
- package/dist/cli/commands/viewpoint.d.ts +7 -0
- package/dist/cli/commands/viewpoint.d.ts.map +1 -0
- package/dist/cli/commands/viewpoint.js +70 -0
- package/dist/cli/commands/viewpoint.js.map +1 -0
- package/dist/cli/index.js +13 -2
- package/dist/cli/index.js.map +1 -1
- package/dist/dashboard/templates/index.html +1 -1
- package/dist/dashboard/types.d.ts +1 -0
- package/dist/dashboard/types.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.d.ts +4 -0
- package/dist/exporters/csv-exporter.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.js +12 -11
- package/dist/exporters/csv-exporter.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +8 -0
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +12 -0
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/json-exporter.d.ts.map +1 -1
- package/dist/exporters/json-exporter.js +7 -0
- package/dist/exporters/json-exporter.js.map +1 -1
- package/dist/exporters/manual-hybrid-oracle.d.ts +54 -0
- package/dist/exporters/manual-hybrid-oracle.d.ts.map +1 -0
- package/dist/exporters/manual-hybrid-oracle.js +114 -0
- package/dist/exporters/manual-hybrid-oracle.js.map +1 -0
- package/dist/exporters/matrix/build.d.ts.map +1 -1
- package/dist/exporters/matrix/build.js +55 -12
- package/dist/exporters/matrix/build.js.map +1 -1
- package/dist/exporters/matrix/carry-manual-results.d.ts +19 -0
- package/dist/exporters/matrix/carry-manual-results.d.ts.map +1 -1
- package/dist/exporters/matrix/carry-manual-results.js +76 -6
- package/dist/exporters/matrix/carry-manual-results.js.map +1 -1
- package/dist/exporters/matrix/export.d.ts +8 -2
- package/dist/exporters/matrix/export.d.ts.map +1 -1
- package/dist/exporters/matrix/export.js +15 -3
- package/dist/exporters/matrix/export.js.map +1 -1
- package/dist/exporters/matrix/gates.js +6 -0
- package/dist/exporters/matrix/gates.js.map +1 -1
- package/dist/exporters/matrix/render-csv.js +1 -1
- package/dist/exporters/matrix/render-csv.js.map +1 -1
- package/dist/exporters/matrix/render-testcase-sheet.d.ts.map +1 -1
- package/dist/exporters/matrix/render-testcase-sheet.js +10 -4
- package/dist/exporters/matrix/render-testcase-sheet.js.map +1 -1
- package/dist/exporters/matrix/render-xlsx.d.ts.map +1 -1
- package/dist/exporters/matrix/render-xlsx.js +3 -0
- package/dist/exporters/matrix/render-xlsx.js.map +1 -1
- package/dist/exporters/matrix/types.d.ts +13 -1
- package/dist/exporters/matrix/types.d.ts.map +1 -1
- package/dist/exporters/matrix/types.js.map +1 -1
- package/dist/exporters/matrix/wording.d.ts +1 -1
- package/dist/exporters/matrix/wording.d.ts.map +1 -1
- package/dist/exporters/matrix/wording.js +10 -1
- package/dist/exporters/matrix/wording.js.map +1 -1
- package/dist/exporters/playwright-report-parser.d.ts.map +1 -1
- package/dist/exporters/playwright-report-parser.js +1 -0
- package/dist/exporters/playwright-report-parser.js.map +1 -1
- package/dist/exporters/run-environment.d.ts +32 -0
- package/dist/exporters/run-environment.d.ts.map +1 -0
- package/dist/exporters/run-environment.js +125 -0
- package/dist/exporters/run-environment.js.map +1 -0
- package/dist/exporters/scenario-merger.d.ts +18 -0
- package/dist/exporters/scenario-merger.d.ts.map +1 -1
- package/dist/exporters/scenario-merger.js +49 -3
- package/dist/exporters/scenario-merger.js.map +1 -1
- package/dist/exporters/test-data-resolver.d.ts +0 -4
- package/dist/exporters/test-data-resolver.d.ts.map +1 -1
- package/dist/exporters/test-data-resolver.js +0 -7
- package/dist/exporters/test-data-resolver.js.map +1 -1
- package/dist/exporters/types.d.ts +5 -0
- package/dist/exporters/types.d.ts.map +1 -1
- package/dist/generators/gherkin-parser/index.d.ts +11 -4
- package/dist/generators/gherkin-parser/index.d.ts.map +1 -1
- package/dist/generators/gherkin-parser/index.js +49 -19
- package/dist/generators/gherkin-parser/index.js.map +1 -1
- package/dist/generators/test-generator/adapters/adapter-interface.d.ts +4 -0
- package/dist/generators/test-generator/adapters/adapter-interface.d.ts.map +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/click-select-action.hbs +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/select-action.hbs +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/partials/pick-option.hbs +13 -0
- package/dist/generators/test-generator/adapters/playwright/templates/test-file.hbs +16 -0
- package/dist/generators/test-generator/code-generator.d.ts.map +1 -1
- package/dist/generators/test-generator/code-generator.js +23 -2
- package/dist/generators/test-generator/code-generator.js.map +1 -1
- package/dist/generators/test-generator/diagnostics.d.ts +3 -1
- package/dist/generators/test-generator/diagnostics.d.ts.map +1 -1
- package/dist/generators/test-generator/diagnostics.js +3 -1
- package/dist/generators/test-generator/diagnostics.js.map +1 -1
- package/dist/harness/audit.d.ts +3 -0
- package/dist/harness/audit.d.ts.map +1 -1
- package/dist/harness/audit.js +48 -15
- package/dist/harness/audit.js.map +1 -1
- package/dist/harness/counted-assertions.d.ts +2 -0
- package/dist/harness/counted-assertions.d.ts.map +1 -0
- package/dist/harness/counted-assertions.js +38 -0
- package/dist/harness/counted-assertions.js.map +1 -0
- package/dist/harness/data-driven-lint.d.ts.map +1 -1
- package/dist/harness/data-driven-lint.js +69 -27
- package/dist/harness/data-driven-lint.js.map +1 -1
- package/dist/harness/defect-draft.d.ts +14 -0
- package/dist/harness/defect-draft.d.ts.map +1 -0
- package/dist/harness/defect-draft.js +143 -0
- package/dist/harness/defect-draft.js.map +1 -0
- package/dist/harness/defects.d.ts +24 -0
- package/dist/harness/defects.d.ts.map +1 -0
- package/dist/harness/defects.js +118 -0
- package/dist/harness/defects.js.map +1 -0
- package/dist/harness/manifest.d.ts.map +1 -1
- package/dist/harness/manifest.js +2 -2
- package/dist/harness/manifest.js.map +1 -1
- package/dist/harness/next-step.d.ts +2 -4
- package/dist/harness/next-step.d.ts.map +1 -1
- package/dist/harness/next-step.js +50 -5
- package/dist/harness/next-step.js.map +1 -1
- package/dist/harness/parse.d.ts +45 -0
- package/dist/harness/parse.d.ts.map +1 -1
- package/dist/harness/parse.js +142 -24
- package/dist/harness/parse.js.map +1 -1
- package/dist/harness/quality-gates.d.ts.map +1 -1
- package/dist/harness/quality-gates.js +6 -4
- package/dist/harness/quality-gates.js.map +1 -1
- package/dist/harness/read-text.d.ts +15 -0
- package/dist/harness/read-text.d.ts.map +1 -1
- package/dist/harness/read-text.js +25 -0
- package/dist/harness/read-text.js.map +1 -1
- package/dist/harness/script-check.d.ts.map +1 -1
- package/dist/harness/script-check.js +7 -1
- package/dist/harness/script-check.js.map +1 -1
- package/dist/harness/select-by-ids.d.ts +31 -0
- package/dist/harness/select-by-ids.d.ts.map +1 -0
- package/dist/harness/select-by-ids.js +106 -0
- package/dist/harness/select-by-ids.js.map +1 -0
- package/dist/harness/sensors.d.ts +8 -1
- package/dist/harness/sensors.d.ts.map +1 -1
- package/dist/harness/sensors.js +92 -21
- package/dist/harness/sensors.js.map +1 -1
- package/dist/harness/spec-branches.js +1 -1
- package/dist/harness/spec-branches.js.map +1 -1
- package/dist/harness/triage-signals.d.ts +20 -0
- package/dist/harness/triage-signals.d.ts.map +1 -0
- package/dist/harness/triage-signals.js +30 -0
- package/dist/harness/triage-signals.js.map +1 -0
- package/dist/harness/triage-unit.d.ts +23 -0
- package/dist/harness/triage-unit.d.ts.map +1 -0
- package/dist/harness/triage-unit.js +142 -0
- package/dist/harness/triage-unit.js.map +1 -0
- package/dist/harness/triage.d.ts +28 -0
- package/dist/harness/triage.d.ts.map +1 -0
- package/dist/harness/triage.js +99 -0
- package/dist/harness/triage.js.map +1 -0
- package/dist/harness/ui-inventory.d.ts +40 -0
- package/dist/harness/ui-inventory.d.ts.map +1 -0
- package/dist/harness/ui-inventory.js +191 -0
- package/dist/harness/ui-inventory.js.map +1 -0
- package/dist/harness/viewpoint-baseline.d.ts.map +1 -1
- package/dist/harness/viewpoint-baseline.js +7 -1
- package/dist/harness/viewpoint-baseline.js.map +1 -1
- package/dist/harness/viewpoint-draft-header.d.ts +7 -0
- package/dist/harness/viewpoint-draft-header.d.ts.map +1 -0
- package/dist/harness/viewpoint-draft-header.js +100 -0
- package/dist/harness/viewpoint-draft-header.js.map +1 -0
- package/dist/harness/viewpoint-ledger.d.ts +13 -3
- package/dist/harness/viewpoint-ledger.d.ts.map +1 -1
- package/dist/harness/viewpoint-ledger.js +104 -18
- package/dist/harness/viewpoint-ledger.js.map +1 -1
- package/dist/harness/viewpoint-mode.d.ts +9 -0
- package/dist/harness/viewpoint-mode.d.ts.map +1 -0
- package/dist/harness/viewpoint-mode.js +52 -0
- package/dist/harness/viewpoint-mode.js.map +1 -0
- package/dist/harness/viewpoint-priority.d.ts +19 -0
- package/dist/harness/viewpoint-priority.d.ts.map +1 -0
- package/dist/harness/viewpoint-priority.js +36 -0
- package/dist/harness/viewpoint-priority.js.map +1 -0
- package/dist/harness/viewpoint-review-open.d.ts +6 -0
- package/dist/harness/viewpoint-review-open.d.ts.map +1 -0
- package/dist/harness/viewpoint-review-open.js +90 -0
- package/dist/harness/viewpoint-review-open.js.map +1 -0
- package/dist/harness/viewpoint-scan-refs.d.ts +50 -0
- package/dist/harness/viewpoint-scan-refs.d.ts.map +1 -0
- package/dist/harness/viewpoint-scan-refs.js +200 -0
- package/dist/harness/viewpoint-scan-refs.js.map +1 -0
- package/dist/harness/viewpoint-scan.d.ts +41 -0
- package/dist/harness/viewpoint-scan.d.ts.map +1 -0
- package/dist/harness/viewpoint-scan.js +118 -0
- package/dist/harness/viewpoint-scan.js.map +1 -0
- package/dist/orchestrator/flow-manager.d.ts +0 -1
- package/dist/orchestrator/flow-manager.d.ts.map +1 -1
- package/dist/orchestrator/flow-manager.js +3 -52
- package/dist/orchestrator/flow-manager.js.map +1 -1
- package/dist/orchestrator/screen-manager.d.ts.map +1 -1
- package/dist/orchestrator/screen-manager.js +8 -62
- package/dist/orchestrator/screen-manager.js.map +1 -1
- package/dist/orchestrator/templates/ai-src/agents/discovery.md +11 -3
- package/dist/orchestrator/templates/ai-src/agents/reviewer.md +8 -0
- package/dist/orchestrator/templates/ai-src/commands/create-test.md +32 -24
- package/dist/orchestrator/templates/ai-src/commands/review.md +2 -2
- package/dist/orchestrator/templates/ai-src/commands/run-test.md +78 -15
- package/dist/orchestrator/templates/ai-src/skills/sungen-capture/mode-local.md +2 -2
- package/dist/orchestrator/templates/ai-src/skills/sungen-error-mapping/SKILL.md +22 -0
- package/dist/orchestrator/templates/ai-src/skills/sungen-gherkin-syntax/SKILL.md +18 -9
- package/dist/orchestrator/templates/ai-src/skills/sungen-harness-audit/SKILL.md +5 -1
- package/dist/orchestrator/templates/ai-src/skills/sungen-selector-fix/SKILL.md +58 -14
- package/dist/orchestrator/templates/ai-src/skills/sungen-tc-generation/SKILL.md +42 -2
- package/dist/orchestrator/templates/ai-src/skills/sungen-tc-review/SKILL.md +7 -0
- package/dist/orchestrator/templates/ai-src/skills/sungen-viewpoint/SKILL.md +18 -0
- package/dist/orchestrator/templates/ai-src/skills/sungen-viewpoint/review-checklist.md +63 -0
- package/dist/orchestrator/templates/ai-src/skills/sungen-viewpoint/viewpoint-template.md +158 -0
- package/dist/orchestrator/templates/playwright.config.ts +4 -2
- package/dist/orchestrator/templates/specs-url-assert.d.ts.map +1 -1
- package/dist/orchestrator/templates/specs-url-assert.js +24 -2
- package/dist/orchestrator/templates/specs-url-assert.js.map +1 -1
- package/dist/orchestrator/templates/specs-url-assert.ts +23 -2
- package/dist/orchestrator/trace-mode-install.d.ts +14 -0
- package/dist/orchestrator/trace-mode-install.d.ts.map +1 -0
- package/dist/orchestrator/trace-mode-install.js +132 -0
- package/dist/orchestrator/trace-mode-install.js.map +1 -0
- package/dist/orchestrator/viewpoint-pattern-catalog.d.ts +23 -0
- package/dist/orchestrator/viewpoint-pattern-catalog.d.ts.map +1 -0
- package/dist/orchestrator/viewpoint-pattern-catalog.js +48 -0
- package/dist/orchestrator/viewpoint-pattern-catalog.js.map +1 -0
- package/dist/orchestrator/viewpoint-scaffold.d.ts +3 -0
- package/dist/orchestrator/viewpoint-scaffold.d.ts.map +1 -0
- package/dist/orchestrator/viewpoint-scaffold.js +70 -0
- package/dist/orchestrator/viewpoint-scaffold.js.map +1 -0
- package/package.json +3 -3
- package/src/cli/commands/audit.ts +49 -10
- package/src/cli/commands/capability.ts +3 -1
- package/src/cli/commands/changed.ts +2 -4
- package/src/cli/commands/dashboard.ts +1 -26
- package/src/cli/commands/defect.ts +49 -0
- package/src/cli/commands/delivery.ts +70 -42
- package/src/cli/commands/results-merge.ts +7 -1
- package/src/cli/commands/select.ts +88 -0
- package/src/cli/commands/triage.ts +57 -0
- package/src/cli/commands/update.ts +3 -0
- package/src/cli/commands/viewpoint.ts +58 -0
- package/src/cli/index.ts +13 -2
- package/src/dashboard/templates/index.html +1 -1
- package/src/dashboard/types.ts +1 -0
- package/src/exporters/csv-exporter.ts +12 -11
- package/src/exporters/feature-parser.ts +12 -0
- package/src/exporters/json-exporter.ts +6 -1
- package/src/exporters/manual-hybrid-oracle.ts +112 -0
- package/src/exporters/matrix/build.ts +57 -11
- package/src/exporters/matrix/carry-manual-results.ts +70 -7
- package/src/exporters/matrix/export.ts +16 -2
- package/src/exporters/matrix/gates.ts +6 -0
- package/src/exporters/matrix/render-csv.ts +1 -1
- package/src/exporters/matrix/render-testcase-sheet.ts +10 -4
- package/src/exporters/matrix/render-xlsx.ts +3 -0
- package/src/exporters/matrix/types.ts +13 -1
- package/src/exporters/matrix/wording.ts +10 -1
- package/src/exporters/playwright-report-parser.ts +1 -0
- package/src/exporters/run-environment.ts +89 -0
- package/src/exporters/scenario-merger.ts +51 -5
- package/src/exporters/test-data-resolver.ts +0 -6
- package/src/exporters/types.ts +5 -0
- package/src/generators/gherkin-parser/index.ts +50 -18
- package/src/generators/test-generator/adapters/adapter-interface.ts +4 -0
- package/src/generators/test-generator/adapters/playwright/templates/steps/actions/click-select-action.hbs +1 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/actions/select-action.hbs +1 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/partials/pick-option.hbs +13 -0
- package/src/generators/test-generator/adapters/playwright/templates/test-file.hbs +16 -0
- package/src/generators/test-generator/code-generator.ts +23 -2
- package/src/generators/test-generator/diagnostics.ts +3 -1
- package/src/harness/audit.ts +52 -17
- package/src/harness/counted-assertions.ts +33 -0
- package/src/harness/data-driven-lint.ts +82 -24
- package/src/harness/defect-draft.ts +115 -0
- package/src/harness/defects.ts +94 -0
- package/src/harness/manifest.ts +2 -2
- package/src/harness/next-step.ts +47 -5
- package/src/harness/parse.ts +151 -27
- package/src/harness/quality-gates.ts +6 -4
- package/src/harness/read-text.ts +25 -0
- package/src/harness/script-check.ts +7 -1
- package/src/harness/select-by-ids.ts +83 -0
- package/src/harness/sensors.ts +91 -21
- package/src/harness/spec-branches.ts +1 -1
- package/src/harness/triage-signals.ts +46 -0
- package/src/harness/triage-unit.ts +113 -0
- package/src/harness/triage.ts +127 -0
- package/src/harness/ui-inventory.ts +164 -0
- package/src/harness/viewpoint-baseline.ts +7 -1
- package/src/harness/viewpoint-draft-header.ts +55 -0
- package/src/harness/viewpoint-ledger.ts +111 -24
- package/src/harness/viewpoint-mode.ts +49 -0
- package/src/harness/viewpoint-priority.ts +38 -0
- package/src/harness/viewpoint-review-open.ts +51 -0
- package/src/harness/viewpoint-scan-refs.ts +154 -0
- package/src/harness/viewpoint-scan.ts +111 -0
- package/src/orchestrator/flow-manager.ts +3 -53
- package/src/orchestrator/screen-manager.ts +7 -61
- package/src/orchestrator/templates/ai-src/agents/discovery.md +11 -3
- package/src/orchestrator/templates/ai-src/agents/reviewer.md +8 -0
- package/src/orchestrator/templates/ai-src/commands/create-test.md +32 -24
- package/src/orchestrator/templates/ai-src/commands/review.md +2 -2
- package/src/orchestrator/templates/ai-src/commands/run-test.md +78 -15
- package/src/orchestrator/templates/ai-src/skills/sungen-capture/mode-local.md +2 -2
- package/src/orchestrator/templates/ai-src/skills/sungen-error-mapping/SKILL.md +22 -0
- package/src/orchestrator/templates/ai-src/skills/sungen-gherkin-syntax/SKILL.md +18 -9
- package/src/orchestrator/templates/ai-src/skills/sungen-harness-audit/SKILL.md +5 -1
- package/src/orchestrator/templates/ai-src/skills/sungen-selector-fix/SKILL.md +58 -14
- package/src/orchestrator/templates/ai-src/skills/sungen-tc-generation/SKILL.md +42 -2
- package/src/orchestrator/templates/ai-src/skills/sungen-tc-review/SKILL.md +7 -0
- package/src/orchestrator/templates/ai-src/skills/sungen-viewpoint/SKILL.md +18 -0
- package/src/orchestrator/templates/ai-src/skills/sungen-viewpoint/review-checklist.md +63 -0
- package/src/orchestrator/templates/ai-src/skills/sungen-viewpoint/viewpoint-template.md +158 -0
- package/src/orchestrator/templates/playwright.config.ts +4 -2
- package/src/orchestrator/templates/specs-url-assert.ts +23 -2
- package/src/orchestrator/trace-mode-install.ts +94 -0
- package/src/orchestrator/viewpoint-pattern-catalog.ts +61 -0
- package/src/orchestrator/viewpoint-scaffold.ts +36 -0
|
@@ -64,7 +64,8 @@ restated from screen specs the project does not hold, say so: the audit reports
|
|
|
64
64
|
`SPEC-RESTATED-UNVERIFIED` because `specFR 100%` over a hand-copied list certifies the copy, not
|
|
65
65
|
the source.
|
|
66
66
|
|
|
67
|
-
**
|
|
67
|
+
**A viewpoint you draft (mode 2, step 3) carries BOTH classification axes** — on a QA-written one
|
|
68
|
+
(mode 1) a missing axis or guarantee line is a row in `viewpoint-review.md`, not an edit. `VP-LOGIC / VP-VAL / VP-SEC / VP-NAV` classify
|
|
68
69
|
by TECHNICAL RISK; `BF / AF / EF` classify by BUSINESS BRANCH. They answer different questions and
|
|
69
70
|
neither replaces the other — a reviewer needs to see both, so use the compound id
|
|
70
71
|
(`VP-SEC-EF02-001`) and give the viewpoint a section per flow as well as its priority table by
|
|
@@ -81,8 +82,8 @@ questions and only one of them is a yardstick:
|
|
|
81
82
|
| `flow-contract.yaml` | the USE CASE — actor · trigger · goal · precondition · both guarantees · the flow inventory | `flowCoverage` |
|
|
82
83
|
| `test-viewpoint.md` | the CLAIMS the suite must prove — cross-screen concerns, edge cases, design decisions, open questions, priorities | `atomicLedger` · `traceability` |
|
|
83
84
|
|
|
84
|
-
If `requirements/test-viewpoint.md` is missing, **
|
|
85
|
-
|
|
85
|
+
If `requirements/test-viewpoint.md` is missing or still the scaffold, **draft it before generating**
|
|
86
|
+
(mode 2, step 3 — `add-flow` scaffolds the template; a run that deletes or replaces it has removed the yardstick). Without it two scored
|
|
86
87
|
axes are excluded and four sensors cannot run at all — the atomic ledger, the
|
|
87
88
|
declaration-integrity checks, the browser-gesture check and the two-way continuity check — so the
|
|
88
89
|
audit reports `VIEWPOINT-MISSING` and the result is *unmeasured*, not passing. And the reason the
|
|
@@ -95,8 +96,8 @@ ordering choice: a yardstick derived from the scenarios reads 100% by constructi
|
|
|
95
96
|
dropped along the way stops being missing from anything. `sungen audit` now reports
|
|
96
97
|
`VIEWPOINT-ADOPTED-POST-HOC` on the first sighting of a viewpoint alongside an existing suite and
|
|
97
98
|
holds `atomicLedger` + `traceability` as unverified until a human accepts it — so the honest move
|
|
98
|
-
is to write the claims from `spec.md` and the contract, mark
|
|
99
|
-
|
|
99
|
+
is to write the claims from `spec.md` and the contract, mark the file `drafted-by: ai` (its first
|
|
100
|
+
line), and let the QA revise before `--accept-viewpoint`. **You never run `--accept-viewpoint` yourself** — `sungen next` lists it as a `⚖ decide` step for exactly this reason: an agent accepting the yardstick it wrote in the same session makes `atomicLedger` and `traceability` read 100% against its own checklist, and the Exception Flow it forgot stops being missing from anything. Present the acceptance to the QA in the hand-back (`AskUserQuestion`) with what the file declares — never as a question mid-run.
|
|
100
101
|
|
|
101
102
|
**When you give a claim an id, a scenario must carry that id.** The id is the contract: a claim
|
|
102
103
|
whose id no scenario carries is reported MISSING, and word overlap will not rescue it. Several
|
|
@@ -110,8 +111,9 @@ narrow the claim to what is actually proven.
|
|
|
110
111
|
|
|
111
112
|
**Priority comes from the declaration.** A scenario tagged `@high` under a viewpoint row that says
|
|
112
113
|
`Low` is `PRIORITY-CONTRADICTED` — a smoke run selects on the tag, so it picks up work the QA
|
|
113
|
-
deliberately deprioritised. Inherit the declared priority; if the tag is right,
|
|
114
|
-
|
|
114
|
+
deliberately deprioritised. Inherit the declared priority; if the tag is right, the declaration is
|
|
115
|
+
what is wrong — fix it in your own draft (mode 2), or report it in `viewpoint-review.md` for the QA
|
|
116
|
+
(mode 1). Never leave both on record silently.
|
|
115
117
|
|
|
116
118
|
---
|
|
117
119
|
|
|
@@ -205,26 +207,15 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
205
207
|
**Screen requirements** — check `<base>/${input:name}/requirements/`:
|
|
206
208
|
{{/cap}}
|
|
207
209
|
- If `spec.md` exists → read it as PRIMARY source (sections, fields, validation rules, business rules, states).
|
|
210
|
+
- `test-viewpoint.md` → **Viewpoint — review or draft** below, once every source is read. No question here.
|
|
208
211
|
{{#cap parallel-subagents}}
|
|
209
|
-
- If `test-viewpoint.md` exists → read it. If it only contains HTML comments (scaffold template), use `AskUserQuestion` to ask:
|
|
210
|
-
- **Fill test-viewpoint.md first** — I'll help you identify edge cases, known issues, and design decisions for this screen before generating tests
|
|
211
|
-
- **Continue without it** — generate tests from spec and other sources only
|
|
212
|
-
|
|
213
|
-
**A filled `test-viewpoint.md` is an INPUT — never an output.** Do NOT rewrite, re-order, or replace its viewpoint declarations as part of generating tests. It is the yardstick two scored axes measure the suite against (`traceability`, `atomicLedger`): rewrite it to match what you just generated and both read 100% by construction, while any viewpoint you quietly dropped stops being missing from anything. If the declared taxonomy genuinely does not fit the screen, **report the mismatch and ask** — propose the diff (ids added / removed and why) and let the QA decide. `sungen audit` reports `VIEWPOINT-BASELINE-CHANGED` and stops scoring both axes when this file moves, so an unannounced rewrite lowers the score rather than raising it.
|
|
214
212
|
|
|
215
213
|
**Context discovery (prefer an isolated agent).** Reading all sources here can flood this context. **Claude Code:** spawn the **`sungen-discovery`** sub-agent (Task tool, `subagent_type: sungen-discovery`) to read spec/figma/ui/live in isolation and return a **compact discovery report** (sources, completeness, conflicts, recommended route, key facts); use that report instead of pasting raw sources. **Copilot / no sub-agents:** do the reading inline as below.
|
|
216
214
|
{{/cap}}
|
|
217
|
-
{{^cap parallel-subagents}}
|
|
218
|
-
- If `test-viewpoint.md` exists → read it. If it only contains HTML comments (scaffold template), ask:
|
|
219
|
-
- **1) Fill test-viewpoint.md first** — identify edge cases, known issues, and design decisions before generating tests
|
|
220
|
-
- **2) Continue without it** — generate tests from spec and other sources only
|
|
221
|
-
|
|
222
|
-
**A filled `test-viewpoint.md` is an INPUT — never an output.** Do NOT rewrite, re-order, or replace its viewpoint declarations while generating tests. It is the yardstick two scored axes measure the suite against (`traceability`, `atomicLedger`): rewrite it to match what you just generated and both read 100% by construction, while any viewpoint you quietly dropped stops being missing from anything. If the declared taxonomy genuinely does not fit the screen, **report the mismatch and ask** — propose the diff (ids added / removed and why) and let the QA decide. `sungen audit` reports `VIEWPOINT-BASELINE-CHANGED` and stops scoring both axes when this file moves.
|
|
223
|
-
{{/cap}}
|
|
224
215
|
|
|
225
216
|
**Auto-detect visual source** — do NOT ask the user to pick a source. Instead, check what already exists and use it:
|
|
226
217
|
1. If `spec_figma.md` exists → read it as Figma supplement (PAT flow already completed during `add-screen`). Do NOT call any `mcp__figma__*` tool.
|
|
227
|
-
2. If `ui/` has images (`.png`, `.jpg`, etc.) → read them
|
|
218
|
+
2. If `ui/` has images (`.png`, `.jpg`, etc.) or `ui/figma-meta.md` → read **all** of them; they feed the Design Inventory (below), not just visual context.
|
|
228
219
|
{{#cap parallel-subagents}}
|
|
229
220
|
3. If neither exists → use `AskUserQuestion` to ask: *"No visual source found. Pick one:"* — then invoke the **`sungen-capture`** skill with the matching **mode** (read only that mode's file):
|
|
230
221
|
- **Figma PAT** — ask for URL, run `sungen add --screen <screen> --figma '<url>'` via Bash, then `sungen-capture` **mode figma-pat**
|
|
@@ -242,7 +233,19 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
242
233
|
|
|
243
234
|
(When `spec_figma.md` exists, that is also `sungen-capture` **mode figma-pat**; when `ui/` images exist, that is **mode local**.)
|
|
244
235
|
|
|
245
|
-
**Cross-check**:
|
|
236
|
+
**Cross-check + Design Inventory**: when the unit holds any design source (`ui/` images, `ui/figma-meta.md`, `spec_figma.md`), follow `sungen-tc-generation` § **Design Inventory** (reading order, one `UI-NNN` row per item, `spec+design` / `design-only` / `conflict`) and **write `requirements/ui-inventory.md`** before generating — from the discovery report's UI INVENTORY block when a sub-agent read the sources. Every `design-only` / `conflict` item must then be covered (`@ui:UI-NNN`) or deferred with a reason (`# UI-DEFERRED: UI-NNN — <reason>`) in the `.feature`; `sungen audit` reports `UI-ITEM-UNCOVERED` otherwise. Show the user the conflicts and design-only items — they are decisions, not details.
|
|
237
|
+
|
|
238
|
+
**Viewpoint — review or draft (no question asked).** Run `sungen viewpoint scan --screen <name> --json` (a flow: `--flow <name>`). It reads ids and item numbers only. Its `mode` (1 or 2) and `modeReason` name the row below — follow them; the table explains them:
|
|
239
|
+
|
|
240
|
+
| `requirements/test-viewpoint.md` | Mode | You |
|
|
241
|
+
|---|---|---|
|
|
242
|
+
| missing; nothing but the scaffold (headings, `<…>` placeholders, empty tables); or ONE OR MORE VP rows and every one of them lacks a `Kỳ vọng` | **2 — draft** | Write it from `sungen-viewpoint/viewpoint-template.md`, first line `drafted-by: ai`, keeping any row already there. Each row answers "test CÁI GÌ" and "kỳ vọng ra sao"; where the spec states the expected result, write it; only what the spec leaves open becomes a closed question in mục 5, never an invented expectation; every accepted-input class the spec names (whitelist, value list, conversion) gets an "accepted, no error" row, citing the list by name; a VP that needs a real device, account or time is `# MANUAL/ENV`. Then scan again. |
|
|
243
|
+
| first line `drafted-by: ai` | **2 — re-review** | Your draft, not yet accepted: you may revise it, from the spec and the review. |
|
|
244
|
+
| anything else — QA-written, with or without VP ids (a scaffold from an older sungen reads as this row too: delete the file to have it drafted) | **1 — review only** | Never edit it. Generate from it as written; a gap is reported, not filled. |
|
|
245
|
+
|
|
246
|
+
Then review it with `sungen-viewpoint/review-checklist.md` and the scan JSON (mode 2: fix your draft until only QA decisions remain) and write `requirements/viewpoint-review.md`, overwritten each run. Mode 2 also runs `sungen audit --screen <name> --no-baseline` before generating. Keep the review's row count for the hand-back.
|
|
247
|
+
|
|
248
|
+
**A filled `test-viewpoint.md` is an INPUT — never an output.** Do NOT rewrite, re-order, or replace its viewpoint declarations as part of generating tests — your own draft included: a draft is revised from the spec, never from the scenarios. It is the yardstick two scored axes measure the suite against (`traceability`, `atomicLedger`): rewrite it to match what you just generated and both read 100% by construction, while any viewpoint you quietly dropped stops being missing from anything. If the declared taxonomy genuinely does not fit the screen, **report the mismatch in `viewpoint-review.md`** — the diff (ids added / removed and why) — and let the QA decide. `sungen audit` reports `VIEWPOINT-BASELINE-CHANGED` and stops scoring both axes when this file moves, so an unannounced rewrite lowers the score rather than raising it.
|
|
246
249
|
|
|
247
250
|
Summarize what you found in requirements and present to the user.
|
|
248
251
|
|
|
@@ -284,6 +287,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
284
287
|
- **BALANCE** → stop expanding secondary viewpoints; add business-core scenarios first.
|
|
285
288
|
- **TRACE** → align `VP-` ids with the viewpoint-overview.
|
|
286
289
|
- Stop when the gate PASSes and findings clear, **or** the budget is exhausted → report residual gaps honestly (never fake a pass).
|
|
290
|
+
- **Repair never edits a QA-written viewpoint (mode 1).** A finding whose fix is an edit to `test-viewpoint.md` becomes a row in `viewpoint-review.md` for the QA when it cites one of the file's VP ids (the checklist's audit rule; anything else goes in the hand-back); in a mode-2 draft, fix the draft.
|
|
287
291
|
|
|
288
292
|
5.6. **Record (reuse + observability).** Build the manifest and report usage:
|
|
289
293
|
- `sungen manifest --screen <name>` — fingerprints for next-run change detection. On a **re-run**, start the whole command by `sungen manifest --screen <name> --diff` and only regenerate scenarios whose spec section changed (keep/regenerate/retire).
|
|
@@ -297,7 +301,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
297
301
|
> **No parallel fan-out here.** Copilot has no sub-agents, so generation is sequential (the Claude Code variant fans out one `sungen-generator` per viewpoint group and merges). Same output, no speedup.
|
|
298
302
|
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + real `constraints` + `errorMap`), then run `sungen data lint --screen ${input:name}` and `sungen data gen --screen ${input:name}` per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves the rest), so it is safe here. Weave the standardized boundary/invalid sets into `@cases` with `CHK-*` trace. Then `… data validate` (values correct) + `… data crosscheck` (testcase↔data agree); for a stateful precondition, `… data state --scaffold <name>` to auto-draft → fill entities + confirm with QA → `… data state`, weave the `@manual:data-setup` `Background` (works spec+viewpoint only). **Same standard as `/sungen:create-data-test`, applied inline — do NOT run that command separately afterwards.** No input fields (navigation/list) → skip.
|
|
299
303
|
5.4. **Depth self-check (deterministic — BEFORE the audit).** Run `sungen depth-lint --screen ${input:name}`. It splits every shallow business-critical scenario into **DEEPEN IN PLACE** (add a real value assertion — the printed `template` is a theme-keyed hint, apply judgment to the actual claim; never fake one onto a visibility/behavior scenario) and **CROSS-SCREEN** (route to a flow / tag `@manual:Mx` + reason — removes it from the depth denominator honestly). Act on both, re-run until `deepen` is empty (or only honest over-counts remain), THEN gate. Lifts first-pass `businessDepth` mechanically instead of via 2–3 repair rounds.
|
|
300
|
-
5.5. **Quality gate & repair (harness — always run).** Per `sungen-harness-audit`: run `sungen audit --screen ${input:name}` (structural), THEN do an **independent semantic review inline** using the `sungen-reviewer` criteria (does each scenario's steps PROVE its title/viewpoint? observable Thens? business-critical assertion depth?). Merge both sets of issues; if gate FAILs / findings exist, repair (budget 3) and re-audit — GATE missing theme → generate it (cross-screen → **automate it in the flow** via `/sungen:add-flow`, NOT a full `@manual` screen duplicate — `sungen audit` flags an automatable `@manual` as `MANUAL-AUTOMATABLE`; reserve `@manual:Mx` for true judgment/missing-capability); DEPTH → add data assertions; BALANCE → add business-core first; TRACE → align VP ids. Never fake a pass.
|
|
304
|
+
5.5. **Quality gate & repair (harness — always run).** Per `sungen-harness-audit`: run `sungen audit --screen ${input:name}` (structural), THEN do an **independent semantic review inline** using the `sungen-reviewer` criteria (does each scenario's steps PROVE its title/viewpoint? observable Thens? business-critical assertion depth?). Merge both sets of issues; if gate FAILs / findings exist, repair (budget 3) and re-audit — GATE missing theme → generate it (cross-screen → **automate it in the flow** via `/sungen:add-flow`, NOT a full `@manual` screen duplicate — `sungen audit` flags an automatable `@manual` as `MANUAL-AUTOMATABLE`; reserve `@manual:Mx` for true judgment/missing-capability); DEPTH → add data assertions; BALANCE → add business-core first; TRACE → align VP ids. Never fake a pass. Repair never edits a QA-written viewpoint (mode 1): a finding whose fix is an edit to `test-viewpoint.md` and cites one of its VP ids becomes a row in `viewpoint-review.md`; in a mode-2 draft, fix the draft.
|
|
301
305
|
5.6. **Record.** `sungen manifest --screen ${input:name}`. Ledger **each phase** (not just repair) — pick one `runId` at the start and pass it so `trace`/`ledger report` show THIS run, not a mix: `sungen ledger record --screen ${input:name} --run <runId> --step <discovery|viewpoint|gherkin|audit|repair:N> --ms <elapsed>`. On re-run, start with `sungen manifest --screen ${input:name} --diff` and only regenerate changed sections.
|
|
302
306
|
6. **Converge — show the trace.** Run `sungen trace --screen ${input:name}` and present: process map (phases + repair rounds), bottlenecks, **HUMAN-LOOP FOCUS** (@manual to verify), audit score + gate + residual gaps. Then offer next steps based on which tier was just generated:
|
|
303
307
|
{{/cap}}
|
|
@@ -363,9 +367,13 @@ Do not stop after writing the files. Close every run with, in this order:
|
|
|
363
367
|
1. **The traceability check below** — an untagged requirement becomes an uncovered gap at delivery.
|
|
364
368
|
2. **The harness result in one line** — score, gate verdict, and the axis the `SCORE-CAPPED` finding
|
|
365
369
|
names (that finding is the single answer to "how do I raise the score").
|
|
366
|
-
3.
|
|
370
|
+
3. **The viewpoint review in one line** — `Viewpoint review: N điểm cần QA xem` (N = the `[ ]` rows
|
|
371
|
+
of `requirements/viewpoint-review.md`).
|
|
372
|
+
4. **`AskUserQuestion` with the next actions** — never end with prose alone. Offer
|
|
367
373
|
`/sungen:run-test <name>` (recommended once the tiers are written), re-running `create-test` to
|
|
368
|
-
extend, and "Done for now".
|
|
374
|
+
extend, "Mở viewpoint-review.md" when N > 0, and "Done for now". After a mode-2 draft also list
|
|
375
|
+
the decision `sungen audit --screen <name> --accept-viewpoint` — for the QA to run once the draft
|
|
376
|
+
is reviewed; you never run it yourself. A run that ends without this leaves the operator guessing.
|
|
369
377
|
|
|
370
378
|
### Requirement traceability
|
|
371
379
|
|
|
@@ -51,7 +51,7 @@ You are an **independent QA Reviewer** — you did not author these tests. You d
|
|
|
51
51
|
1. **Enumerate feature files** — glob `<base>/<name>/features/*.feature` (main + any sub-features). If none → `/sungen:create-test` first.
|
|
52
52
|
2. **Run the harness (source of truth) — do NOT re-score with a separate rubric:**
|
|
53
53
|
- `sungen audit --screen <name>` (Bash) → gate status, business-weighted score, findings, gaps (coverage / assertion-depth / balance / duplicate / traceability).
|
|
54
|
-
- **`sungen-reviewer`** sub-agent (Task tool, `subagent_type: sungen-reviewer`) → semantic verdict the gate can't see (do steps prove the title, observable Then, @manual justified). Copilot/no-sub-agents: apply the `sungen-reviewer` criteria inline.
|
|
54
|
+
- **`sungen-reviewer`** sub-agent (Task tool, `subagent_type: sungen-reviewer`) → semantic verdict the gate can't see (do steps prove the title, observable Then, @manual justified) and, when the unit holds design sources (`requirements/ui/*`, `ui/figma-meta.md`, `spec_figma.md`), the **Design ↔ Feature diff** against `ui-inventory.md`. Copilot/no-sub-agents: apply the `sungen-reviewer` criteria inline.
|
|
55
55
|
- `sungen script-check --screen <name>` (Bash) → is the generated spec a 1:1 of the Gherkin (only if a spec exists; flags hand-edit / stale drift).
|
|
56
56
|
3. **Unified scorecard** — present ONE report per feature, anchored on the harness signals (the `sungen-tc-review` 7 dimensions are a *presentation layer* over these, not a competing score):
|
|
57
57
|
{{/cap}}
|
|
@@ -63,7 +63,7 @@ You are an **independent QA Reviewer** — you did not author these tests. You d
|
|
|
63
63
|
1. **Enumerate** `<base>/${input:name}/features/*.feature`. If none → `/sungen-create-test` first.
|
|
64
64
|
2. **Run the harness (source of truth) — no separate rubric:**
|
|
65
65
|
- `sungen audit --screen ${input:name}` → gate, business-weighted score, findings, gaps.
|
|
66
|
-
- Apply the **`sungen-reviewer` criteria inline** → semantic verdict (do steps prove the title? observable Then? business-critical depth? @manual justified?).
|
|
66
|
+
- Apply the **`sungen-reviewer` criteria inline** → semantic verdict (do steps prove the title? observable Then? business-critical depth? @manual justified?) and, when `requirements/ui/*` / `ui/figma-meta.md` / `spec_figma.md` exist, read them and diff the design against `ui-inventory.md` and the `.feature` (criterion 7).
|
|
67
67
|
- `sungen script-check --screen ${input:name}` → spec is 1:1 with the Gherkin (flags hand-edit / stale drift; only if a spec exists).
|
|
68
68
|
3. **Unified scorecard** per feature, anchored on harness signals (the `sungen-tc-review` 7 dimensions are a presentation layer, not a competing score):
|
|
69
69
|
{{/cap}}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: run-test
|
|
3
3
|
description: "Generate selectors + auth state (Playwright MCP for web / Appium MCP for mobile), compile, and run tests — auto-fixes selectors on failure"
|
|
4
|
-
argument-hint: "[screen-name] [--env <locale>] [--ios|--android|--both]"
|
|
4
|
+
argument-hint: "[screen-name] [--ids <ID,…>] [--env <locale>] [--ios|--android|--both]"
|
|
5
5
|
order: 40
|
|
6
6
|
claude-tools: "Read, Grep, Bash, Glob, Edit, Write, AskUserQuestion, mcp__playwright__browser_navigate, mcp__playwright__browser_snapshot, mcp__playwright__browser_take_screenshot, mcp__playwright__browser_wait_for, mcp__playwright__browser_evaluate, mcp__playwright__browser_run_code, mcp__playwright__browser_storage_state, mcp__playwright__browser_set_storage_state, mcp__appium-mcp__select_device, mcp__appium-mcp__appium_session_management, mcp__appium-mcp__generate_locators, mcp__appium-mcp__appium_get_page_source, mcp__appium-mcp__appium_find_element, mcp__appium-mcp__appium_gesture, mcp__appium-mcp__appium_screenshot, mcp__appium-mcp__appium_set_value"
|
|
7
7
|
copilot-tools: "[read, execute, edit, vscode/askQuestions, playwright/*, appium/*]"
|
|
@@ -79,12 +79,21 @@ Stop here for mobile — do not continue to the web path.
|
|
|
79
79
|
> ⛔ **The `.feature` Gherkin is QA-approved and FROZEN.** During auto-fix you may write
|
|
80
80
|
> **only** `selectors.yaml`, `test-data.yaml` (and other `*.yaml` data), plus the **one narrow
|
|
81
81
|
> `.feature` edit** allowed: the **element-type token** on a `[Reference]` (e.g. `field`→`textbox`,
|
|
82
|
-
> `button`→`link` — selector resolution, not test logic).
|
|
82
|
+
> `button`→`link` — selector resolution, not test logic). A second, equally narrow exception:
|
|
83
|
+
> adding **`@known-defect:DEF-NNN`** to a scenario's tag line once its defect record exists (step 8)
|
|
84
|
+
> — that tag only, the tag line only. You must **NOT** change the Gherkin's
|
|
83
85
|
> content or logic: no rewriting/adding/removing steps, no `wait for` steps, no changed expected
|
|
84
86
|
> values, no weakened assertions. A failure that would need any of those is a **CANDIDATE BUG /
|
|
85
87
|
> QA finding → report it, let it FAIL** (it may be a real bug). The live page is NOT the oracle;
|
|
86
88
|
> never hand-edit the generated `.spec.ts` either. See `sungen-error-mapping` § "HARD RULE".
|
|
87
89
|
> (A `password > 8` test that fails on 6 chars is a bug to report, not a `>= 6` edit.)
|
|
90
|
+
>
|
|
91
|
+
> **Blocked by missing seed data / capability** (the run cannot create the state the scenario
|
|
92
|
+
> needs)? Keep every Given/When/Then; add only a tag — `@requires:<cap>` when a driver exists,
|
|
93
|
+
> else `@manual:M1` with a `# MANUAL (<date>, run-test): <reason>` comment inside the scenario —
|
|
94
|
+
> and list each downgrade in the run summary for QA. Never replace the assertion with a presence
|
|
95
|
+
> placeholder and park the oracle in a comment (`SG-W008`). See `sungen-error-mapping` § "Fails
|
|
96
|
+
> because the environment lacks the DATA or CAPABILITY".
|
|
88
97
|
|
|
89
98
|
## Parameters
|
|
90
99
|
|
|
@@ -95,7 +104,9 @@ Parse from `$ARGUMENTS`:
|
|
|
95
104
|
the result over the previous run's file instead of replacing it. Passing it also **skips the scope gate**:
|
|
96
105
|
the flag IS the answer, so asking again would be noise. Without the flag the gate asks — but only when the
|
|
97
106
|
selection is narrower than the whole unit (see the table). Either way a subset run must follow the merge
|
|
98
|
-
|
|
107
|
+
flow below — web writes the subset to a temp report, mobile backs the fixed-name file up BEFORE the run.
|
|
108
|
+
- **`--ids <ID,…>`** — run exactly these test-case IDs (`VP-UI-001,VP-UI-002`) and merge the result
|
|
109
|
+
over the previous run's file — see the `--ids` section below. Skips the scope gate, like `--change-only`.
|
|
99
110
|
- **`--ios`**, **`--android`**, **`--both`** (mobile only) — optional. Sets `MOBILE_PLATFORM` for the test run. Default: `android` if not specified. Parsed into `MOBILE_PLATFORM=<value>` and used in WebdriverIO test runs.
|
|
100
111
|
|
|
101
112
|
If `--env` is passed but no value follows, ask the user which locale to use.
|
|
@@ -172,6 +183,7 @@ Read `summary.selected`, `summary.total`, `all`, and `reasons`, then follow this
|
|
|
172
183
|
| Situation | What to do |
|
|
173
184
|
|---|---|
|
|
174
185
|
| `--change-only` was passed | **Do NOT ask.** Run the changed set, print the list + reasons for the record, follow the merge flow below. |
|
|
186
|
+
| `--ids` was passed | **Do NOT ask, and do not run `sungen changed` at all.** The user named the cases — follow the `--ids` section below. |
|
|
175
187
|
| `all: true` | **Do NOT ask** — there is nothing to choose. Print the reason (`no previous run recorded`, `the Background changed`, `project config changed`, `git reports …`) and run the FULL suite. |
|
|
176
188
|
| `selected == 0` | **Do NOT ask.** Say plainly that nothing changed since the last recorded run, and use `AskUserQuestion` to offer: re-run the full suite anyway / stop here. |
|
|
177
189
|
| `0 < selected < total` | **ASK** — this is the only case with two honest answers. |
|
|
@@ -187,6 +199,10 @@ recorded` and runs everything again — the narrowing never starts. Record it af
|
|
|
187
199
|
results you accept (a full run, or a subset once merged), and NOT after a run you are about to
|
|
188
200
|
redo: leaving the baseline where it is keeps the same scenarios selected on the next attempt.
|
|
189
201
|
|
|
202
|
+
**Never record after an `--ids` run.** The user's IDs need not cover what an edit changed;
|
|
203
|
+
recording would mark every changed scenario as re-run, and the next `--change-only` would skip the
|
|
204
|
+
ones this run never touched.
|
|
205
|
+
|
|
190
206
|
When you ask, first PRINT the selection so the choice is informed — one line per scenario with its
|
|
191
207
|
reason, exactly as `sungen changed` renders it — then `AskUserQuestion` with two options:
|
|
192
208
|
|
|
@@ -256,8 +272,8 @@ question already.
|
|
|
256
272
|
5. **Phase 1 — Smoke Check**: Run first 5 `@high` scenarios only. If failures → diagnose, fix fundamentals (page selector, auth, base @steps), re-run. Max 2 attempts. If still broken → ask user.
|
|
257
273
|
6. **Phase 2 — Priority Wave**: Run all `@high` scenarios. Fix only failures from this wave. Max 2 attempts. Shared selectors fixed here cascade to later phases.
|
|
258
274
|
7. **Phase 3 — Full Run**: Run all tests. Fix only **new** failures (elements unique to `@normal`/`@low`). Max 1 attempt. Don't loop on low-priority failures.
|
|
259
|
-
8. **Phase 4 — Regression**: One final full run.
|
|
260
|
-
9. **Phase 8 — Serial-cascade risk check** (#414): If any scenario has many steps, a skipped test may cause downstream steps to be incorrectly skipped due to the serial-cascade sensor detecting false "not reached" conditions. When you see "test skipped" messages during Phase 3–4 runs, check if the skipped scenario is a **blockers-exist** dependency in the test ledger (`sungen ledger query --screen <name>`). If the scenario had high step count and cascade fired, offer: *"Scenarios with deep steps can trigger serial-cascade skipping. Consider parallelizing this spec with `@concurrent` tags or breaking into smaller scenarios.
|
|
275
|
+
8. **Phase 4 — Regression**: One final full run. No more fix loops. Then run `sungen triage --screen <name>` and render its table as the results summary: every red or flaky scenario already sits in one class (`product-candidate`, `unclassified`, `test-defect`, `environment`, `blocked-capability`, `known-defect`, `flaky`) with the rule and one evidence line. You may move an `unclassified` row into a class only with a written evidence line of your own. You may never turn a `product-candidate` into a `test-defect` by editing the `.feature` or the expected value — the Gherkin is frozen; report it as a candidate bug. For each `product-candidate` you judge to be a product bug, run `sungen defect draft --screen <name> --scenario "<title>"`, fill the record's Summary, check its Expected against `spec.md`, then add `@known-defect:DEF-NNN` to that scenario's tag line (the tag-only exception above). The severity in the draft is `proposed` — confirm it with QA (`AskUserQuestion`) before the hand-back and set `severity_source: qa`. A row you judge is not a product bug stays in the summary with your reason, and gets no record. A `bare-timeout` row (`unclassified`) is never re-run inside this command — Phase 4 was the final run; offer the re-run in Next steps.
|
|
276
|
+
9. **Phase 8 — Serial-cascade risk check** (#414): If any scenario has many steps, a skipped test may cause downstream steps to be incorrectly skipped due to the serial-cascade sensor detecting false "not reached" conditions. When you see "test skipped" messages during Phase 3–4 runs, check if the skipped scenario is a **blockers-exist** dependency in the test ledger (`sungen ledger query --screen <name>`). If the scenario had high step count and cascade fired, offer: *"Scenarios with deep steps can trigger serial-cascade skipping. Consider parallelizing this spec with `@concurrent` tags or breaking into smaller scenarios."* Let the user decide whether to refactor (do not auto-apply parallelization).
|
|
261
277
|
{{#cap parallel-subagents}}
|
|
262
278
|
9. **Integrity check & trace (always run after the final run).**
|
|
263
279
|
- `sungen script-check --screen <name>` — verify the generated spec is a **1:1** of the Gherkin (every non-@manual scenario ↔ one `test()`, no drift). If it reports **DRIFT** (spec hand-edited or stale), re-run `sungen generate --screen <name>` so the spec matches the feature, then re-run — **never hand-edit the generated spec** (auto-fix must edit `selectors.yaml`, not the `.spec.ts`).
|
|
@@ -325,7 +341,7 @@ sungen results merge --screen <name> specs/generated/<name>/subset-tmp.json
|
|
|
325
341
|
rm specs/generated/<name>/subset-tmp.json
|
|
326
342
|
```
|
|
327
343
|
|
|
328
|
-
`--screen` takes the FEATURE basename — for a multi-feature screen (`home.spec.ts` + `home-modal.spec.ts`), each feature has its own result file: merge `home-modal`'s subset with `--screen home-modal`. If the previous run had zero failures, `--last-failed`
|
|
344
|
+
`--screen` takes the FEATURE basename — for a multi-feature screen (`home.spec.ts` + `home-modal.spec.ts`), each feature has its own result file: merge `home-modal`'s subset with `--screen home-modal`. If the previous run had zero failures, `--last-failed` finds no tests: Playwright prints "No tests found", exits 1, and still writes a zero-test report — don't merge it; use `-g "<scenario title>"` to pick the cases instead. `--last-failed` reads the failures of the *previous run only*, so any run in between replaces the list.
|
|
329
345
|
|
|
330
346
|
For a locale run (`SUNGEN_ENV=vi`), the config appends `.vi` before `.json` to WHATEVER `PLAYWRIGHT_JSON_OUTPUT_NAME` says — the temp report lands at `subset-tmp.vi.json`; merge that file into the matching `<name>-test-result.vi.json` explicitly: `sungen results merge specs/generated/<name>/<name>-test-result.vi.json specs/generated/<name>/subset-tmp.vi.json`. A scenario RENAMED by a spec edit leaves its old-title result in the file — the merge warns and keeps it (delivery builds rows from the feature, so it never renders).
|
|
331
347
|
|
|
@@ -360,29 +376,33 @@ If you want to filter scenarios, use `-g "<pattern>"` instead of a reporter over
|
|
|
360
376
|
|
|
361
377
|
## `--change-only` — re-run just what an edit obliges, without losing the rest
|
|
362
378
|
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
Two commands close that, and the order matters — the backup must exist BEFORE the run, because by
|
|
368
|
-
the time the run finishes the old file is already gone:
|
|
379
|
+
A per-target result file holds the whole unit, so a subset run must never replace it: re-running 3
|
|
380
|
+
cases of a 100-case screen used to throw away the 97 that still passed. The subset is MERGED in
|
|
381
|
+
instead — on web via a temp report, on mobile via a backup taken BEFORE the run (the WDIO reporter
|
|
382
|
+
always writes the fixed-name file, so by the time the run finishes the old one is gone):
|
|
369
383
|
|
|
370
384
|
```bash
|
|
371
385
|
# 1. what does the edit oblige? (empty output = nothing to run, do NOT fall through to a full run)
|
|
372
386
|
npx sungen changed --screen <name> # human-readable, with the reason per scenario
|
|
373
387
|
GREP=$(npx sungen changed --screen <name> --grep)
|
|
374
388
|
|
|
389
|
+
# MOBILE — the WDIO reporter always writes the canonical per-OS file, so back it up first:
|
|
375
390
|
# 2. keep the previous results BEFORE the runner overwrites them
|
|
376
391
|
cp specs/generated/<name>/<name>-test-result.<os>.json /tmp/<name>.prev.json
|
|
377
|
-
|
|
378
392
|
# 3. run only the selected scenarios
|
|
379
393
|
SUNGEN_SPECS='./specs/generated/<name>/<name>.spec.ts' npm run test:mobile -- --mochaOpts.grep "$GREP"
|
|
380
|
-
# web: npx playwright test specs/generated/<name>/<name>.spec.ts -g "$GREP"
|
|
381
|
-
|
|
382
394
|
# 4. merge the subset back over the kept results (same title → new wins, others kept)
|
|
383
395
|
npx sungen results merge /tmp/<name>.prev.json specs/generated/<name>/<name>-test-result.<os>.json \
|
|
384
396
|
-o specs/generated/<name>/<name>-test-result.<os>.json
|
|
385
397
|
|
|
398
|
+
# WEB — Playwright writes its report to PLAYWRIGHT_JSON_OUTPUT_NAME (else test-results/results.json),
|
|
399
|
+
# never to the per-target file by itself: write the subset to a temp report and merge THAT in (#837).
|
|
400
|
+
# With --env <x>: prefix BOTH commands with SUNGEN_ENV=<x>; the config then writes subset-tmp.<x>.json.
|
|
401
|
+
PLAYWRIGHT_JSON_OUTPUT_NAME=specs/generated/<name>/subset-tmp.json \
|
|
402
|
+
npx playwright test specs/generated/<name>/<name>.spec.ts -g "$GREP"
|
|
403
|
+
npx sungen results merge --screen <name> specs/generated/<name>/subset-tmp.json # --env x → subset-tmp.x.json
|
|
404
|
+
rm specs/generated/<name>/subset-tmp*.json
|
|
405
|
+
|
|
386
406
|
# 5. record the new baseline — ONLY after a run you accept
|
|
387
407
|
npx sungen changed --screen <name> --record
|
|
388
408
|
```
|
|
@@ -398,6 +418,45 @@ change that broke it.
|
|
|
398
418
|
**Without `--change-only`** the full run still writes the file wholesale, which is correct — a full
|
|
399
419
|
run IS the complete picture. Merging only matters for subsets.
|
|
400
420
|
|
|
421
|
+
## `--ids` — run a chosen list of test-case IDs
|
|
422
|
+
|
|
423
|
+
The user names the cases (`--ids VP-UI-001,VP-UI-002`). `sungen select` resolves them —
|
|
424
|
+
deterministic, exact ID match, grouped per feature — and refuses anything it cannot run:
|
|
425
|
+
|
|
426
|
+
```bash
|
|
427
|
+
npx sungen select --screen <name> --ids <ID,…> # human-readable
|
|
428
|
+
npx sungen select --screen <name> --ids <ID,…> --json # what you act on
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
**Exit 1 = do not run.** Report every entry of `unknown` (with its `didYouMean`), `notRunnable`
|
|
432
|
+
(manual / owned by a flow — with the reason) and `notCompiled` (→ `sungen generate` first), then
|
|
433
|
+
use `AskUserQuestion` to offer: run the valid IDs only / correct the list. Never fall back to a
|
|
434
|
+
full run, and never run with an empty grep.
|
|
435
|
+
|
|
436
|
+
Then, **for each entry of `features`** (a multi-feature screen has one spec and one result file per
|
|
437
|
+
feature), with `<spec>` = its `spec`, `<f>` = its `feature`, `<dir>` = the spec's directory and
|
|
438
|
+
`GREP` = its `grep`:
|
|
439
|
+
|
|
440
|
+
```bash
|
|
441
|
+
# WEB — write the subset to a temp report, then merge it into <f>'s per-target file.
|
|
442
|
+
# With --env <x>: prefix BOTH commands with SUNGEN_ENV=<x>; the config then writes subset-tmp.<x>.json.
|
|
443
|
+
PLAYWRIGHT_JSON_OUTPUT_NAME=<dir>/subset-tmp.json npx playwright test <spec> -g "$GREP"
|
|
444
|
+
npx sungen results merge --screen <f> <dir>/subset-tmp.json # --env x → <dir>/subset-tmp.x.json
|
|
445
|
+
rm <dir>/subset-tmp*.json
|
|
446
|
+
|
|
447
|
+
# MOBILE — the reporter always writes <dir>/<f>-test-result.<android|ios>[.<env>].json: back it up first
|
|
448
|
+
# (with --env <x> every <f>-test-result.<os>.json below is <f>-test-result.<os>.<x>.json)
|
|
449
|
+
cp <dir>/<f>-test-result.<os>.json /tmp/<f>.prev.json
|
|
450
|
+
SUNGEN_SPECS='./<spec>' npm run test:mobile -- --mochaOpts.grep "$GREP"
|
|
451
|
+
npx sungen results merge /tmp/<f>.prev.json <dir>/<f>-test-result.<os>.json -o <dir>/<f>-test-result.<os>.json
|
|
452
|
+
```
|
|
453
|
+
|
|
454
|
+
With no previous result file (first run of the unit, or of this `--env`) there is nothing to merge
|
|
455
|
+
into — `results merge --screen` says so and exits 1: on web run with
|
|
456
|
+
`PLAYWRIGHT_JSON_OUTPUT_NAME=<dir>/<f>-test-result.json`, on mobile skip the backup and the merge.
|
|
457
|
+
With `--both`, back up and merge each OS's file. **Do NOT run `sungen changed --record`** (see the
|
|
458
|
+
scope gate). Report the per-ID outcome, plus the IDs that were refused.
|
|
459
|
+
|
|
401
460
|
## Next steps
|
|
402
461
|
|
|
403
462
|
After showing results, use `AskUserQuestion` to offer next steps:
|
|
@@ -421,3 +480,7 @@ If tests **failed** (after retries):
|
|
|
421
480
|
- **`/sungen-create-test <name>`** — Revise test cases
|
|
422
481
|
{{/cap}}
|
|
423
482
|
- **Done for now** — I'll fix manually later
|
|
483
|
+
|
|
484
|
+
If the triage table has `bare-timeout` rows, put this option FIRST and mark it Recommended: **Re-run the N timed-out scenario(s) once** — it tells a slow environment from a hanging page (name them). Ask; never re-run them on your own: a scenario may create data or spend an OTP/rate limit on a shared environment. On a yes, re-run only those scenarios with the subset-merge flow (§ Playwright command guidelines — never overwrite the result file), then `sungen triage` again:
|
|
485
|
+
- green now → move the row to `flaky` (or `environment`) with that as the evidence line;
|
|
486
|
+
- the same timeout again → read the trace's last action: stuck after a submit or a navigation → candidate bug, handled as in Phase 4 step 8 (`sungen defect draft`, QA confirms the severity); stuck in a wait the test itself defines → `test-defect`.
|
|
@@ -6,7 +6,7 @@ Use **pre-existing images** in `requirements/ui/` as visual context. No network,
|
|
|
6
6
|
|
|
7
7
|
### 1. List available images
|
|
8
8
|
|
|
9
|
-
Glob `requirements/ui/*.{png,jpg,jpeg,webp,gif}` and report count + filenames. Filter out metadata files (e.g. `figma-meta.md` written by mode figma-mcp) — those are read by `tc-generation` separately, not treated as images here.
|
|
9
|
+
Glob `requirements/ui/*.{png,jpg,jpeg,webp,gif}` and report count + filenames. Filter out metadata files (e.g. `figma-meta.md` written by mode figma-mcp) — those are read by `tc-generation` (§ Design Inventory) separately, not treated as images here.
|
|
10
10
|
|
|
11
11
|
### 2. Handle empty folder
|
|
12
12
|
|
|
@@ -21,7 +21,7 @@ If no images found:
|
|
|
21
21
|
|
|
22
22
|
### 3. Read images for context
|
|
23
23
|
|
|
24
|
-
Use the `Read` tool on each image — Claude Code reads PNG/JPG/WebP directly
|
|
24
|
+
Use the `Read` tool on **each** image — Claude Code reads PNG/JPG/WebP directly. For large sets (>10 images), ask which are primary and which are states/variants so they can be grouped — but still read them all: every image feeds the Design Inventory (`sungen-tc-generation` § Design Inventory), and the inventory records `images read: N/N`.
|
|
25
25
|
|
|
26
26
|
### 4. Summarize
|
|
27
27
|
|
|
@@ -39,6 +39,26 @@ weakening assertions, "fixing" an unknown step pattern, adding table-scope steps
|
|
|
39
39
|
needs any of these, it is a **finding for QA** — surface it in the run summary and let the test
|
|
40
40
|
**FAIL**. A failing test may be a **real bug**; do not edit the QA-approved Gherkin to make it pass.
|
|
41
41
|
|
|
42
|
+
### Fails because the environment lacks the DATA or CAPABILITY (not the app, not the locator)
|
|
43
|
+
|
|
44
|
+
Sometimes the scenario is right and the app may be right, but the run cannot set up the state —
|
|
45
|
+
the dev seed has no tie-break pair, no locked account, no second tenant. That is **neither** a
|
|
46
|
+
selector fix **nor** a reason to rewrite. The only allowed `.feature` edit is a **tag** (plus a
|
|
47
|
+
reason comment); the Given/When/Then stay **exactly** as QA approved them:
|
|
48
|
+
|
|
49
|
+
1. The capability exists as a driver (`db`, `api`, data-factory) → tag `@requires:<cap>` (e.g.
|
|
50
|
+
`@requires:db`) — it compiles to a skip stub (`AUTOMATION-READY-PENDING`) and runs, unchanged,
|
|
51
|
+
once `sungen capability add <cap>` is done.
|
|
52
|
+
2. Otherwise → tag `@manual:M1` and put the reason as a comment **inside** the scenario (below the
|
|
53
|
+
`Scenario:` line — a comment above the tags is attached to the previous scenario):
|
|
54
|
+
`# MANUAL (<date>, run-test): no seeded pair with equal plan + distinct last-updated`.
|
|
55
|
+
3. **Never** swap the real assertion (`table match data:`, `with {{v}}`, `count is …`) for a
|
|
56
|
+
presence placeholder (`see [X] table`) and move the oracle into a `# Tester verifies:` comment —
|
|
57
|
+
that hybrid exports an Expected cell that says nothing about what is checked, and `sungen
|
|
58
|
+
generate` flags it `SG-W008`.
|
|
59
|
+
4. List **every** downgrade of the run (scenario, tag added, reason) in the run summary so QA
|
|
60
|
+
reviews it — a downgrade is a coverage decision, not an auto-fix.
|
|
61
|
+
|
|
42
62
|
## ⛔ Source of truth — classify EVERY failure before you "fix" it
|
|
43
63
|
|
|
44
64
|
`.feature` + `test-data.yaml` + `spec.md` are the **oracle**. The **live page is NOT** — it may be the thing that's broken. A failing test is not automatically a test to "make pass". Classify first:
|
|
@@ -53,6 +73,8 @@ needs any of these, it is a **finding for QA** — surface it in the run summary
|
|
|
53
73
|
|
|
54
74
|
> **Cardinal sin (do NOT do this):** a `password > 8 chars` rule fails on a 6-char input → "fix" it to `>= 6` so the test passes. The logic is now meaningless. A failing assertion is a **finding**, not a chore.
|
|
55
75
|
|
|
76
|
+
**`sungen triage` makes the first pass for you.** Its classes are this section's split: `test-defect` = selector-resolution / test-data failure (auto-fix scope), `product-candidate` = assertion-value failure with an Expected and a Received (apply the spec question above), `environment` = the run itself (fix the setup, not the test), `unclassified` = yours to judge — an assertion on an element that never appeared lands here, because a wrong selector and a missing message look the same.
|
|
77
|
+
|
|
56
78
|
**Auto-fix loop scope:** the run-test auto-fix loop engages ONLY on **selector-resolution** failures. On an assertion-value failure where the app contradicts the spec → **HALT and report**, do not loop it into passing.
|
|
57
79
|
|
|
58
80
|
**Never hand-edit the generated `.spec.ts`** (e.g. inserting `page.evaluate`/`fetch` to bypass a broken control). `sungen script-check` regenerates the spec from `.feature` and flags any edit as DRIFT — regenerate, don't patch.
|
|
@@ -629,8 +629,9 @@ on mobile a dialog-scoped ref still resolves against the whole screen (`SG-W021`
|
|
|
629
629
|
| Tag | Effect |
|
|
630
630
|
|---|---|
|
|
631
631
|
| `@spec:<id>` | **Traceability**: this scenario proves requirement `<id>` from `requirements/spec.md` (`@spec:FR-003`). Repeatable — a scenario may prove several. Read by `sungen audit` (requirement coverage) and by delivery (the Coverage sheet's requirement table). Without it the link is only inferred from wording and cannot survive an edit. |
|
|
632
|
+
| `@ui:<UI-NNN>` | **Design traceability**: this scenario proves design item `UI-NNN` from `requirements/ui-inventory.md` (`@ui:UI-014`). Repeatable. Read by `sungen audit` (`UI-ITEM-UNCOVERED`). An item deliberately not tested is recorded as a comment instead: `# UI-DEFERRED: UI-002..013 — <reason>`. Rules: `sungen-tc-generation` § Design Inventory. |
|
|
632
633
|
| `@manual` | Skip in generation |
|
|
633
|
-
| `@known-defect` | The scenario is EXPECTED to be red — it asserts a live defect. It still runs and still reports as failed (never skipped, never "fixed"), and it stays a pass-through tag so `--grep @known-defect` selects it. **`[mobile]`** Its failure does NOT escalate the next scenario's `@reset:` tier: a red that was designed in is not evidence of leftover state |
|
|
634
|
+
| `@known-defect` | The scenario is EXPECTED to be red — it asserts a live defect. It still runs and still reports as failed (never skipped, never "fixed"), and it stays a pass-through tag so `--grep @known-defect` selects it. `@known-defect:DEF-NNN` names the record in `qa/defects/DEF-NNN-*.md` (`sungen defect draft` writes it); `sungen audit` reports `DEFECT-REF-MISSING` when that file does not exist. **`[mobile]`** Its failure does NOT escalate the next scenario's `@reset:` tier: a red that was designed in is not evidence of leftover state |
|
|
634
635
|
| `@auth:role` | Use auth storage state for role |
|
|
635
636
|
| `@no-auth` | Disable inherited auth |
|
|
636
637
|
| `@steps:name` | Define reusable step block (base scenario) |
|
|
@@ -640,7 +641,8 @@ on mobile a dialog-scoped ref still resolves against the whole screen (`SG-W021`
|
|
|
640
641
|
| `@cleanup:scroll` | Auto-cleanup: scroll to top after each test (cleanupPage) |
|
|
641
642
|
| `@cleanup:storage` | Auto-cleanup: clear sessionStorage after each test (cleanupPage) |
|
|
642
643
|
| `@screenshot:on-failure` | Auto-capture screenshot when test fails (base.ts fixture) |
|
|
643
|
-
| `@
|
|
644
|
+
| `@serial` | Opt-in: one shared page for an ordered, genuinely stateful sequence → `test.describe.serial()`. A failure skips the rest |
|
|
645
|
+
| `@parallel` | Names the default (fresh page per test) — changes nothing. Compiles on mobile too, but a single device/emulator gets no session-isolation benefit from it |
|
|
644
646
|
| `@beforeAll` | Hook: runs once before all tests → `test.beforeAll()` |
|
|
645
647
|
| `@afterEach` | Hook: runs after each test → `test.afterEach()` (custom cleanup) |
|
|
646
648
|
| `@afterAll` | Hook: runs once after all tests → `test.afterAll()` |
|
|
@@ -774,14 +776,14 @@ npx playwright test --grep "@smoke|@high" # smoke OR high
|
|
|
774
776
|
|
|
775
777
|
### Serial vs Parallel (test execution mode)
|
|
776
778
|
|
|
777
|
-
**Default:
|
|
779
|
+
**Default: isolated** — `test.describe()` with a fresh page per test. Background runs as `beforeEach`; a failing scenario does not skip the others. Write every scenario so it stands alone — no scenario may rely on state another one left behind. `@parallel` only names this default.
|
|
778
780
|
|
|
779
|
-
**`@
|
|
781
|
+
**`@serial` opt-in** — `test.describe.serial()` with one shared page. Background runs once in `beforeAll`; a failure skips the rest. Use it only for a genuinely stateful, ordered sequence.
|
|
780
782
|
|
|
781
783
|
| Mode | Generated | Page | Background | On fail |
|
|
782
784
|
|---|---|---|---|---|
|
|
783
|
-
|
|
|
784
|
-
| `@
|
|
785
|
+
| Isolated (default, `@parallel`) | `test.describe()` | Fresh per test | `beforeEach` (N goto) | Continue |
|
|
786
|
+
| `@serial` | `test.describe.serial()` | Shared | `beforeAll` (1 goto) | Skip remaining |
|
|
785
787
|
|
|
786
788
|
**Multiple auth groups need the default isolation.** A feature mixing `@auth:user` with `@no-auth` compiles as-is — features are parallel by default, and `@parallel` only names that default. What breaks it is `@serial`: one shared browser context cannot carry two different sessions, so the compiler errors and asks you to drop `@serial` or split the feature.
|
|
787
789
|
|
|
@@ -942,7 +944,13 @@ Feature: User Management
|
|
|
942
944
|
|
|
943
945
|
### Layer 2: `@afterEach` scenario (custom cleanup)
|
|
944
946
|
|
|
945
|
-
Only when `@cleanup:*` tags aren't enough — feature-specific logic.
|
|
947
|
+
Only when `@cleanup:*` tags aren't enough — feature-specific logic. `@afterEach` gets the per-test
|
|
948
|
+
`page`, so UI cleanup steps work in it.
|
|
949
|
+
|
|
950
|
+
It runs after every test it applies to — and it does NOT apply to the `@no-auth` group when the
|
|
951
|
+
feature mixes `@no-auth` with other scenarios: an anonymous test has an empty session, so a cleanup
|
|
952
|
+
that needs the signed-in page would fail it although its own steps passed. The hook is emitted inside
|
|
953
|
+
each other group instead. A feature whose scenarios are ALL `@no-auth` keeps the hook for them.
|
|
946
954
|
|
|
947
955
|
### Layer 3: `@beforeAll` / `@afterAll` (optional)
|
|
948
956
|
|
|
@@ -967,8 +975,9 @@ Scenario: drop the seeded data after the last scenario
|
|
|
967
975
|
recorded and the next step still runs; a failing action step (`Given`/`When`) stops the hook there.
|
|
968
976
|
The hook fails ONCE with the full list (`Hook preconditions not met`), not with the first failure.
|
|
969
977
|
- `@cases` and `@extend` are not supported on a hook — they are ignored with a compile warning.
|
|
970
|
-
-
|
|
971
|
-
`@query:` preconditions and `expect` assertions there.
|
|
978
|
+
- `@beforeAll` / `@afterAll` have no `page` fixture on an isolated (non-`@serial`) feature — keep
|
|
979
|
+
their steps to `@api:`/`@query:` preconditions and `expect` assertions there. `@afterEach` does get
|
|
980
|
+
the per-test `page`.
|
|
972
981
|
|
|
973
982
|
**Rendering order in `.spec.ts`:**
|
|
974
983
|
`test.describe` → `test.use(storageState)` → `test.use(autoCleanup)` → `test.beforeAll` → `test.beforeEach` → `test.afterEach` → `test.afterAll` → `test()` blocks.
|
|
@@ -35,7 +35,11 @@ user-invocable: false
|
|
|
35
35
|
| **BALANCE** | secondary viewpoints (UI/validation/security) outweigh business-core | **Stop expanding** secondary viewpoints; generate the missing business-core scenarios first. Do not add more subscription/UI variants while core is thin. |
|
|
36
36
|
| **TRACE** | scenarios use ad-hoc `VP-<CAT>-NNN` codes not linked to the viewpoint-overview | Make each scenario map to a viewpoint-overview id (align category codes, or add a mapping comment). |
|
|
37
37
|
| **ORACLE-URL-ONLY** | every `Then` in the scenario is a page/URL assertion — it proves a path was pushed, not that the page rendered; on a permission/guard scenario a late 403 redirect turns it falsely green | Add `anchor:` to the page selector (one line per screen; every `see [X] page` on it becomes a content oracle) or a content assertion after the page step. Never delete the page step to silence it. |
|
|
38
|
+
| **DEFECT-REF-MISSING** | a `@known-defect:DEF-NNN` tag names no file in `qa/defects/` | Draft the record (`sungen defect draft --screen <name> --scenario "<title>"`) or correct the id. Never drop the tag to silence it — the scenario still asserts a live defect. |
|
|
39
|
+
| **DEFECT-ORPHAN** (info) | an open defect record names a scenario this unit no longer has | Update the record's `scenarios:` to the renamed title, or close the record if the scenario was removed on purpose. Nothing in the suite to repair. |
|
|
38
40
|
| **UNIVERSAL** | a universal theme (error/empty-state, accessibility) is absent | Low priority — add if in scope; otherwise note as out-of-scope with reason. |
|
|
41
|
+
| **VIEWPOINT-DRAFT-UNREVIEWED** | `test-viewpoint.md` carries `drafted-by: ai`: no baseline, `atomicLedger` + `traceability` held | A QA action, not a repair. Keep generating; list the decision `sungen audit --screen <name> --accept-viewpoint` in the hand-back for the QA to run after reviewing the draft. Never run it, never delete the line yourself. |
|
|
42
|
+
| **VIEWPOINT-REVIEW-OPEN** (info) | `requirements/viewpoint-review.md` still has `[ ]` rows | A QA action. Report the count in the hand-back with the option to open the file; never tick rows or edit a QA-written viewpoint to clear it. Not scored. |
|
|
39
43
|
|
|
40
44
|
## P5 steps for deep cross-screen / list coverage
|
|
41
45
|
|
|
@@ -64,7 +68,7 @@ Use these when repairing GATE/DEPTH findings for the hard viewpoints (cart/detai
|
|
|
64
68
|
## Repair loop rules
|
|
65
69
|
|
|
66
70
|
1. **Budget = 3 rounds.** Re-run `sungen audit` after each repair; track score delta.
|
|
67
|
-
2. **Stop when** `gateStatus == PASS` AND `findings` empty — or budget exhausted.
|
|
71
|
+
2. **Stop when** `gateStatus == PASS` AND `findings` empty — or budget exhausted. QA-action findings (`VIEWPOINT-DRAFT-UNREVIEWED`, `VIEWPOINT-REVIEW-OPEN`) do not count: no repair clears them.
|
|
68
72
|
3. **Never fake a pass.** A shallow `see [Cart] page` does not satisfy `cart-correctness`. If a gap is genuinely cross-screen or needs capabilities the DSL lacks (e.g. capture an element value to compare elsewhere), **report it as a residual gap / flow item** instead of forcing a green gate.
|
|
69
73
|
4. **EP/data families are OK.** A `duplicates` cluster with `sameDataLikely=false` is an intentional equivalence-partition family (e.g. many invalid-email cases) — keep it; only collapse `sameDataLikely=true` exact duplicates.
|
|
70
74
|
5. **Advisory findings — surface, don't gate.** `MANUAL-REASON-MISMATCH` → fix the scenario's `@manual:Mx` code (so the planner recommends the right driver) during repair. `CAPABILITY-SUGGESTION` → **present it to the user as a next-step option** (e.g. "N @manual could be automated — `sungen capability add api db`?"), **recommend-only — never auto-install**. `VERIFICATION-OUT-OF-SCOPE (api|db)` → the unit uses `@api`/`@query`/`@requires:api|db` verification the project's recorded **test type** never opted into (E2E/UI-only per `capabilities.yaml`, and the viewpoint doesn't ask for it). During repair: **rewrite the oracle to be UI-observable** (or downgrade the deep check to an `(optional deep check: …)` note); only keep the API/DB verification if the project genuinely tests that layer — then record it (`verification: [ui, api]` / `sungen capability add`). Keep API/DB-in-E2E to the ≤20% band. Neither of these fails the gate.
|