@sun-asterisk/sungen 2.7.0-beta.1 → 3.0.0-beta.72
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/dist/cli/commands/add.js +3 -3
- package/dist/cli/commands/add.js.map +1 -1
- package/dist/cli/commands/audit.d.ts +3 -0
- package/dist/cli/commands/audit.d.ts.map +1 -0
- package/dist/cli/commands/audit.js +134 -0
- package/dist/cli/commands/audit.js.map +1 -0
- package/dist/cli/commands/blindspot.d.ts +3 -0
- package/dist/cli/commands/blindspot.d.ts.map +1 -0
- package/dist/cli/commands/blindspot.js +58 -0
- package/dist/cli/commands/blindspot.js.map +1 -0
- package/dist/cli/commands/capability.d.ts +3 -0
- package/dist/cli/commands/capability.d.ts.map +1 -0
- package/dist/cli/commands/capability.js +196 -0
- package/dist/cli/commands/capability.js.map +1 -0
- package/dist/cli/commands/challenge.d.ts +3 -0
- package/dist/cli/commands/challenge.d.ts.map +1 -0
- package/dist/cli/commands/challenge.js +102 -0
- package/dist/cli/commands/challenge.js.map +1 -0
- package/dist/cli/commands/feedback.d.ts +3 -0
- package/dist/cli/commands/feedback.d.ts.map +1 -0
- package/dist/cli/commands/feedback.js +72 -0
- package/dist/cli/commands/feedback.js.map +1 -0
- package/dist/cli/commands/flow-check.d.ts +3 -0
- package/dist/cli/commands/flow-check.d.ts.map +1 -0
- package/dist/cli/commands/flow-check.js +136 -0
- package/dist/cli/commands/flow-check.js.map +1 -0
- package/dist/cli/commands/generate.d.ts.map +1 -1
- package/dist/cli/commands/generate.js +50 -2
- package/dist/cli/commands/generate.js.map +1 -1
- package/dist/cli/commands/ledger.d.ts +3 -0
- package/dist/cli/commands/ledger.d.ts.map +1 -0
- package/dist/cli/commands/ledger.js +71 -0
- package/dist/cli/commands/ledger.js.map +1 -0
- package/dist/cli/commands/manifest.d.ts +3 -0
- package/dist/cli/commands/manifest.d.ts.map +1 -0
- package/dist/cli/commands/manifest.js +101 -0
- package/dist/cli/commands/manifest.js.map +1 -0
- package/dist/cli/commands/script-check.d.ts +3 -0
- package/dist/cli/commands/script-check.d.ts.map +1 -0
- package/dist/cli/commands/script-check.js +97 -0
- package/dist/cli/commands/script-check.js.map +1 -0
- package/dist/cli/commands/trace.d.ts +3 -0
- package/dist/cli/commands/trace.d.ts.map +1 -0
- package/dist/cli/commands/trace.js +110 -0
- package/dist/cli/commands/trace.js.map +1 -0
- package/dist/cli/commands/update.d.ts.map +1 -1
- package/dist/cli/commands/update.js +22 -9
- package/dist/cli/commands/update.js.map +1 -1
- package/dist/cli/index.js +20 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/generators/test-generator/adapters/adapter-interface.d.ts +1 -0
- package/dist/generators/test-generator/adapters/adapter-interface.d.ts.map +1 -1
- package/dist/generators/test-generator/adapters/adapter-registry.d.ts +13 -0
- package/dist/generators/test-generator/adapters/adapter-registry.d.ts.map +1 -1
- package/dist/generators/test-generator/adapters/adapter-registry.js +73 -1
- package/dist/generators/test-generator/adapters/adapter-registry.js.map +1 -1
- package/dist/generators/test-generator/adapters/index.d.ts +1 -1
- package/dist/generators/test-generator/adapters/index.d.ts.map +1 -1
- package/dist/generators/test-generator/adapters/index.js +5 -1
- package/dist/generators/test-generator/adapters/index.js.map +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/capture-variable.hbs +1 -0
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/all-contain-assertion.hbs +7 -0
- package/dist/generators/test-generator/adapters/playwright/templates/test-file.hbs +6 -0
- package/dist/generators/test-generator/code-generator.d.ts.map +1 -1
- package/dist/generators/test-generator/code-generator.js +6 -2
- package/dist/generators/test-generator/code-generator.js.map +1 -1
- package/dist/generators/test-generator/patterns/capture-patterns.d.ts +16 -0
- package/dist/generators/test-generator/patterns/capture-patterns.d.ts.map +1 -0
- package/dist/generators/test-generator/patterns/capture-patterns.js +54 -0
- package/dist/generators/test-generator/patterns/capture-patterns.js.map +1 -0
- package/dist/generators/test-generator/patterns/form-patterns.d.ts.map +1 -1
- package/dist/generators/test-generator/patterns/form-patterns.js +3 -1
- package/dist/generators/test-generator/patterns/form-patterns.js.map +1 -1
- package/dist/generators/test-generator/patterns/index.d.ts.map +1 -1
- package/dist/generators/test-generator/patterns/index.js +2 -0
- package/dist/generators/test-generator/patterns/index.js.map +1 -1
- package/dist/generators/test-generator/step-mapper.d.ts.map +1 -1
- package/dist/generators/test-generator/step-mapper.js +1 -0
- package/dist/generators/test-generator/step-mapper.js.map +1 -1
- package/dist/generators/test-generator/utils/data-resolver.d.ts +5 -0
- package/dist/generators/test-generator/utils/data-resolver.d.ts.map +1 -1
- package/dist/generators/test-generator/utils/data-resolver.js +17 -0
- package/dist/generators/test-generator/utils/data-resolver.js.map +1 -1
- package/dist/generators/test-generator/utils/runtime-data-transformer.d.ts.map +1 -1
- package/dist/generators/test-generator/utils/runtime-data-transformer.js +4 -0
- package/dist/generators/test-generator/utils/runtime-data-transformer.js.map +1 -1
- package/dist/generators/test-generator/utils/selector-resolver.d.ts.map +1 -1
- package/dist/generators/test-generator/utils/selector-resolver.js +12 -6
- package/dist/generators/test-generator/utils/selector-resolver.js.map +1 -1
- package/dist/harness/audit.d.ts +24 -0
- package/dist/harness/audit.d.ts.map +1 -0
- package/dist/harness/audit.js +115 -0
- package/dist/harness/audit.js.map +1 -0
- package/dist/harness/blindspot.d.ts +15 -0
- package/dist/harness/blindspot.d.ts.map +1 -0
- package/dist/harness/blindspot.js +85 -0
- package/dist/harness/blindspot.js.map +1 -0
- package/dist/harness/capability-plan.d.ts +49 -0
- package/dist/harness/capability-plan.d.ts.map +1 -0
- package/dist/harness/capability-plan.js +215 -0
- package/dist/harness/capability-plan.js.map +1 -0
- package/dist/harness/capability.d.ts +23 -0
- package/dist/harness/capability.d.ts.map +1 -0
- package/dist/harness/capability.js +98 -0
- package/dist/harness/capability.js.map +1 -0
- package/dist/harness/catalog/drivers.yaml +57 -0
- package/dist/harness/catalog/universal-viewpoints.yaml +114 -0
- package/dist/harness/challenge.d.ts +21 -0
- package/dist/harness/challenge.d.ts.map +1 -0
- package/dist/harness/challenge.js +151 -0
- package/dist/harness/challenge.js.map +1 -0
- package/dist/harness/feedback.d.ts +29 -0
- package/dist/harness/feedback.d.ts.map +1 -0
- package/dist/harness/feedback.js +106 -0
- package/dist/harness/feedback.js.map +1 -0
- package/dist/harness/flow-check.d.ts +23 -0
- package/dist/harness/flow-check.d.ts.map +1 -0
- package/dist/harness/flow-check.js +132 -0
- package/dist/harness/flow-check.js.map +1 -0
- package/dist/harness/flow-plan.d.ts +23 -0
- package/dist/harness/flow-plan.d.ts.map +1 -0
- package/dist/harness/flow-plan.js +166 -0
- package/dist/harness/flow-plan.js.map +1 -0
- package/dist/harness/intent.d.ts +11 -0
- package/dist/harness/intent.d.ts.map +1 -0
- package/dist/harness/intent.js +86 -0
- package/dist/harness/intent.js.map +1 -0
- package/dist/harness/ledger.d.ts +42 -0
- package/dist/harness/ledger.d.ts.map +1 -0
- package/dist/harness/ledger.js +171 -0
- package/dist/harness/ledger.js.map +1 -0
- package/dist/harness/manifest.d.ts +42 -0
- package/dist/harness/manifest.d.ts.map +1 -0
- package/dist/harness/manifest.js +209 -0
- package/dist/harness/manifest.js.map +1 -0
- package/dist/harness/parse.d.ts +22 -0
- package/dist/harness/parse.d.ts.map +1 -0
- package/dist/harness/parse.js +163 -0
- package/dist/harness/parse.js.map +1 -0
- package/dist/harness/script-check.d.ts +39 -0
- package/dist/harness/script-check.d.ts.map +1 -0
- package/dist/harness/script-check.js +251 -0
- package/dist/harness/script-check.js.map +1 -0
- package/dist/harness/secret-scan.d.ts +8 -0
- package/dist/harness/secret-scan.d.ts.map +1 -0
- package/dist/harness/secret-scan.js +88 -0
- package/dist/harness/secret-scan.js.map +1 -0
- package/dist/harness/sensors.d.ts +88 -0
- package/dist/harness/sensors.d.ts.map +1 -0
- package/dist/harness/sensors.js +232 -0
- package/dist/harness/sensors.js.map +1 -0
- package/dist/harness/trace.d.ts +31 -0
- package/dist/harness/trace.d.ts.map +1 -0
- package/dist/harness/trace.js +173 -0
- package/dist/harness/trace.js.map +1 -0
- package/dist/orchestrator/ai-rules-updater.d.ts +1 -0
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +55 -11
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/figma/spec-figma-renderer.d.ts +2 -2
- package/dist/orchestrator/figma/spec-figma-renderer.js +2 -2
- package/dist/orchestrator/figma/spec-figma-section-renderers.d.ts +1 -1
- package/dist/orchestrator/figma/spec-figma-section-renderers.js +1 -1
- package/dist/orchestrator/project-initializer.d.ts +5 -0
- package/dist/orchestrator/project-initializer.d.ts.map +1 -1
- package/dist/orchestrator/project-initializer.js +30 -6
- package/dist/orchestrator/project-initializer.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-agent-challenge.md +46 -0
- package/dist/orchestrator/templates/ai-instructions/claude-agent-discovery.md +32 -0
- package/dist/orchestrator/templates/ai-instructions/claude-agent-reviewer.md +37 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-flow.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +36 -12
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-design.md +12 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-feedback.md +36 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +27 -30
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -1
- package/dist/orchestrator/templates/ai-instructions/claude-config.md +1 -4
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-mode-figma-mcp.md +82 -0
- package/dist/orchestrator/templates/ai-instructions/{github-skill-sungen-figma-source.md → claude-skill-capture-mode-figma-pat.md} +14 -48
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-mode-live.md +60 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-mode-local.md +38 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture.md +35 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-harness-audit.md +84 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-flow.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +18 -10
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-design.md +13 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-feedback.md +24 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +20 -30
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-config.md +1 -4
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-figma-mcp.md +82 -0
- package/{src/orchestrator/templates/ai-instructions/claude-skill-figma-source.md → dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-figma-pat.md} +14 -48
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-live.md +60 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-local.md +38 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture.md +35 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-harness-audit.md +84 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -1
- package/dist/orchestrator/templates/specs-test-data.ts +20 -6
- package/dist/tools/figma/figma-auth.d.ts +5 -2
- package/dist/tools/figma/figma-auth.d.ts.map +1 -1
- package/dist/tools/figma/figma-auth.js +19 -9
- package/dist/tools/figma/figma-auth.js.map +1 -1
- package/docs/orchestration-spec.md +267 -0
- package/package.json +12 -7
- package/src/cli/commands/add.ts +3 -3
- package/src/cli/commands/audit.ts +92 -0
- package/src/cli/commands/blindspot.ts +48 -0
- package/src/cli/commands/capability.ts +160 -0
- package/src/cli/commands/challenge.ts +55 -0
- package/src/cli/commands/feedback.ts +65 -0
- package/src/cli/commands/flow-check.ts +97 -0
- package/src/cli/commands/generate.ts +47 -2
- package/src/cli/commands/ledger.ts +61 -0
- package/src/cli/commands/manifest.ts +55 -0
- package/src/cli/commands/script-check.ts +50 -0
- package/src/cli/commands/trace.ts +60 -0
- package/src/cli/commands/update.ts +30 -10
- package/src/cli/index.ts +20 -0
- package/src/generators/test-generator/adapters/adapter-interface.ts +1 -0
- package/src/generators/test-generator/adapters/adapter-registry.ts +37 -0
- package/src/generators/test-generator/adapters/index.ts +4 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/actions/capture-variable.hbs +1 -0
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/all-contain-assertion.hbs +7 -0
- package/src/generators/test-generator/adapters/playwright/templates/test-file.hbs +6 -0
- package/src/generators/test-generator/code-generator.ts +6 -2
- package/src/generators/test-generator/patterns/capture-patterns.ts +59 -0
- package/src/generators/test-generator/patterns/form-patterns.ts +3 -1
- package/src/generators/test-generator/patterns/index.ts +2 -0
- package/src/generators/test-generator/step-mapper.ts +1 -0
- package/src/generators/test-generator/utils/data-resolver.ts +20 -0
- package/src/generators/test-generator/utils/runtime-data-transformer.ts +8 -0
- package/src/generators/test-generator/utils/selector-resolver.ts +13 -6
- package/src/harness/audit.ts +112 -0
- package/src/harness/blindspot.ts +51 -0
- package/src/harness/capability-plan.ts +180 -0
- package/src/harness/capability.ts +75 -0
- package/src/harness/catalog/drivers.yaml +57 -0
- package/src/harness/catalog/universal-viewpoints.yaml +114 -0
- package/src/harness/challenge.ts +131 -0
- package/src/harness/feedback.ts +84 -0
- package/src/harness/flow-check.ts +99 -0
- package/src/harness/flow-plan.ts +135 -0
- package/src/harness/intent.ts +58 -0
- package/src/harness/ledger.ts +155 -0
- package/src/harness/manifest.ts +173 -0
- package/src/harness/parse.ts +145 -0
- package/src/harness/script-check.ts +222 -0
- package/src/harness/secret-scan.ts +51 -0
- package/src/harness/sensors.ts +279 -0
- package/src/harness/trace.ts +138 -0
- package/src/orchestrator/ai-rules-updater.ts +57 -10
- package/src/orchestrator/figma/spec-figma-renderer.ts +2 -2
- package/src/orchestrator/figma/spec-figma-section-renderers.ts +1 -1
- package/src/orchestrator/project-initializer.ts +33 -7
- package/src/orchestrator/templates/ai-instructions/claude-agent-challenge.md +46 -0
- package/src/orchestrator/templates/ai-instructions/claude-agent-discovery.md +32 -0
- package/src/orchestrator/templates/ai-instructions/claude-agent-reviewer.md +37 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-add-flow.md +3 -3
- package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +36 -12
- package/src/orchestrator/templates/ai-instructions/claude-cmd-design.md +12 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-feedback.md +36 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +27 -30
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -1
- package/src/orchestrator/templates/ai-instructions/claude-config.md +1 -4
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-mode-figma-mcp.md +82 -0
- package/{dist/orchestrator/templates/ai-instructions/copilot-skill-figma-source.md → src/orchestrator/templates/ai-instructions/claude-skill-capture-mode-figma-pat.md} +14 -48
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-mode-live.md +60 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-mode-local.md +38 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture.md +35 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-harness-audit.md +84 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-flow.md +3 -3
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +18 -10
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-design.md +13 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-feedback.md +24 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +20 -30
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -1
- package/src/orchestrator/templates/ai-instructions/copilot-config.md +1 -4
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-figma-mcp.md +82 -0
- package/{dist/orchestrator/templates/ai-instructions/claude-skill-figma-source.md → src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-figma-pat.md} +14 -48
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-live.md +60 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-mode-local.md +38 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture.md +35 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-harness-audit.md +84 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -1
- package/src/orchestrator/templates/specs-test-data.ts +20 -6
- package/src/tools/figma/figma-auth.ts +20 -9
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-figma.md +0 -142
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-live.md +0 -112
- package/dist/orchestrator/templates/ai-instructions/claude-skill-capture-local.md +0 -73
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-figma.md +0 -142
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-live.md +0 -112
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-capture-local.md +0 -73
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-figma.md +0 -142
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-live.md +0 -112
- package/src/orchestrator/templates/ai-instructions/claude-skill-capture-local.md +0 -73
- package/src/orchestrator/templates/ai-instructions/copilot-skill-figma-source.md +0 -151
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-figma.md +0 -142
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-live.md +0 -112
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-capture-local.md +0 -73
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-figma-source.md +0 -151
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sungen-harness-audit
|
|
3
|
+
description: 'How to read `sungen audit` output and repair test-design findings. Auto-loaded by the design orchestrator.'
|
|
4
|
+
user-invocable: false
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## What `sungen audit` measures
|
|
8
|
+
|
|
9
|
+
`sungen audit --screen <name>` runs deterministic sensors over `features/<name>.feature` + `requirements/test-viewpoint.md` and writes `.sungen/reports/<name>-audit.json`. It is the **gate** the orchestrator repairs against. Run with `--json` to parse it.
|
|
10
|
+
|
|
11
|
+
### Report shape (key fields)
|
|
12
|
+
```jsonc
|
|
13
|
+
{
|
|
14
|
+
"score": { "overall": 3.9, "coverage": 0.4, "businessDepth": 0.18, "balance": 0.5, "traceability": 0.7 },
|
|
15
|
+
"gateStatus": "PASS" | "FAIL",
|
|
16
|
+
"gate": { "pageType": "ecommerce-list", "themesCovered": 2, "themesTotal": 5,
|
|
17
|
+
"gaps": [ { "theme": "cart-correctness", "keywords": [...] } ] },
|
|
18
|
+
"depth": { "businessCriticalShallow": 9, "businessCriticalTotal": 11,
|
|
19
|
+
"shallowBusinessCritical": [ { "name": "...", "category": "PRODUCT" } ] },
|
|
20
|
+
"balance": { "imbalanced": true, "coreCount": 11, "secondaryCount": 22, "byBucket": {...} },
|
|
21
|
+
"duplicates": { "clusters": [ { "sameDataLikely": false, "scenarios": [...] } ] },
|
|
22
|
+
"trace": { "mappedRatio": 0.4, "note": "..." },
|
|
23
|
+
"findings": [ "GATE: ...", "DEPTH: ...", "BALANCE: ...", "TRACE: ..." ]
|
|
24
|
+
}
|
|
25
|
+
```
|
|
26
|
+
- **`overall` score is business-weighted** (coverage 0.4 + businessDepth 0.3 + balance 0.15 + traceability 0.15). It is intentionally strict on business value — a high count with shallow business coverage scores low. Don't optimize the count; optimize coverage + depth.
|
|
27
|
+
- Exit code **2** when `gateStatus == FAIL` (usable in CI / loop).
|
|
28
|
+
|
|
29
|
+
## Finding → repair mapping
|
|
30
|
+
|
|
31
|
+
| Finding prefix | Meaning | Repair action |
|
|
32
|
+
|---|---|---|
|
|
33
|
+
| **GATE** | a critical theme for the page-type has no covering scenario | Generate scenarios for that theme. **If cross-screen** (cart-correctness, product-detail-consistency, filter-result-correctness, multi-item cart) → do NOT fake it on a single screen; plan a **flow** (`/sungen:add-flow`) and record the deferral. |
|
|
34
|
+
| **DEPTH** | business-critical scenarios assert only visibility/navigation | Replace `Then User see [X] page/section` with **observable data assertions**: `Then User see [X] with {{value}}`, `Then User see [T] table match data:`. Capture real expected values into `test-data.yaml`. |
|
|
35
|
+
| **BALANCE** | secondary viewpoints (UI/validation/security) outweigh business-core | **Stop expanding** secondary viewpoints; generate the missing business-core scenarios first. Do not add more subscription/UI variants while core is thin. |
|
|
36
|
+
| **TRACE** | scenarios use ad-hoc `VP-<CAT>-NNN` codes not linked to the viewpoint-overview | Make each scenario map to a viewpoint-overview id (align category codes, or add a mapping comment). |
|
|
37
|
+
| **UNIVERSAL** | a universal theme (error/empty-state, accessibility) is absent | Low priority — add if in scope; otherwise note as out-of-scope with reason. |
|
|
38
|
+
|
|
39
|
+
## P5 steps for deep cross-screen / list coverage
|
|
40
|
+
|
|
41
|
+
Use these when repairing GATE/DEPTH findings for the hard viewpoints (cart/detail/filter correctness). They need **runtime data mode** (the default).
|
|
42
|
+
|
|
43
|
+
- **Capture a value to compare across screens** (product-detail-consistency, cart-correctness):
|
|
44
|
+
```gherkin
|
|
45
|
+
When User remember [Product Name] text as {{selected_product_name}}
|
|
46
|
+
And User remember [Product Price] text as {{selected_product_price}}
|
|
47
|
+
And User click [View Product] link
|
|
48
|
+
Then User see [Detail Product Name] header with {{selected_product_name}}
|
|
49
|
+
And User see [Detail Product Price] text with {{selected_product_price}}
|
|
50
|
+
```
|
|
51
|
+
`remember` stores the element's text/value at runtime; later `{{var}}` resolves to it. This proves the detail/cart shows the SAME product, not a random one.
|
|
52
|
+
|
|
53
|
+
- **Assert every item in a result matches** (category/brand-filter-correctness):
|
|
54
|
+
```gherkin
|
|
55
|
+
When User click [Women] link
|
|
56
|
+
And User click [Dress] link
|
|
57
|
+
Then User see all [Result Product Name] contain {{selected_category}}
|
|
58
|
+
```
|
|
59
|
+
`see all [X] contain {{v}}` asserts EVERY matching element contains the value → "all displayed products belong to the selected category/brand", not just one.
|
|
60
|
+
|
|
61
|
+
> Cross-screen flows (home → detail/cart): if the target screen is a separate screen, prefer a **flow** (`/sungen:add-flow`) so the journey is one test. On a single screen, keep the cross-screen assertion but tag `@manual` with a `# Deferred to a flow` comment.
|
|
62
|
+
|
|
63
|
+
## Repair loop rules
|
|
64
|
+
|
|
65
|
+
1. **Budget = 3 rounds.** Re-run `sungen audit` after each repair; track score delta.
|
|
66
|
+
2. **Stop when** `gateStatus == PASS` AND `findings` empty — or budget exhausted.
|
|
67
|
+
3. **Never fake a pass.** A shallow `see [Cart] page` does not satisfy `cart-correctness`. If a gap is genuinely cross-screen or needs capabilities the DSL lacks (e.g. capture an element value to compare elsewhere), **report it as a residual gap / flow item** instead of forcing a green gate.
|
|
68
|
+
4. **EP/data families are OK.** A `duplicates` cluster with `sameDataLikely=false` is an intentional equivalence-partition family (e.g. many invalid-email cases) — keep it; only collapse `sameDataLikely=true` exact duplicates.
|
|
69
|
+
|
|
70
|
+
## Discovery / fallback tree (when input is limited)
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
spec.md đủ tốt? → YES: Spec-first
|
|
74
|
+
│ NO
|
|
75
|
+
source code có? → YES: Source-first (mine behavior từ code)
|
|
76
|
+
│ NO
|
|
77
|
+
testcase cũ tương tự?→ YES: History-first
|
|
78
|
+
│ NO
|
|
79
|
+
domain rủi ro+defect?→ YES: Defect-first
|
|
80
|
+
│ NO
|
|
81
|
+
→ hỏi QA; QA chưa phản hồi → OUTPUT kèm ASSUMPTION LIST rõ ràng (không stall)
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
See `docs/orchestration-spec.md` for the full flow and `reports/sungen_refactor_spec.md` for the design rationale.
|
|
@@ -117,7 +117,7 @@ When running Phase 0 for a **flow** (`qa/flows/<name>/`), check existing screen
|
|
|
117
117
|
| 5 | `text` | Plain text node (not inside interactive component) |
|
|
118
118
|
| 6 | `locator` (CSS) | Last resort — `#id` or `[attr=value]` only, never classes or XPath |
|
|
119
119
|
|
|
120
|
-
See `sungen-
|
|
120
|
+
See `sungen-capture` skill (mode figma-pat) for the authoritative heuristics table mapping Figma signals to YAML entry types.
|
|
121
121
|
|
|
122
122
|
4. **Write entries**: add each entry to `selectors.yaml`, prefixed with the required comment:
|
|
123
123
|
```yaml
|
|
@@ -7,7 +7,7 @@ user-invocable: false
|
|
|
7
7
|
## ⚠️ Gotchas — read before generating
|
|
8
8
|
|
|
9
9
|
- `spec_figma.md` exists → read file only, **NEVER** call `mcp__figma__*`
|
|
10
|
-
→ PAT auth flow already done by `sungen-figma-
|
|
10
|
+
→ PAT auth flow already done by `sungen-capture` (mode figma-pat); re-calling fails or duplicates work.
|
|
11
11
|
|
|
12
12
|
- `selectors.yaml` → do **NOT** generate — handled by `run-test`
|
|
13
13
|
→ Selectors need live DOM inspection via Playwright MCP, only `run-test` triggers it.
|
|
@@ -208,6 +208,45 @@ Security: [S1 – admin only]
|
|
|
208
208
|
|
|
209
209
|
**✅ Good** — see admin notice example above: `Display surfaces` lists every URL spec mentions as output, `Cross-surface rules` maps each admin action to its user-facing outcome, `Inclusive bounds` flags every `<=`/`>=` for BVA. Every item maps to a VP-ID in `Tier 1 output`.
|
|
210
210
|
|
|
211
|
+
#### Critical business-viewpoint pre-gate — pass `sungen audit` on the FIRST pass
|
|
212
|
+
|
|
213
|
+
> The harness gate FAILS (and forces repair rounds → wasted tokens) when a page-type's critical **business** viewpoints are missing or **shallow**. Generate them correctly the first time. A business-critical `Then` must assert **DATA**, never just `see [X] page/section/modal`.
|
|
214
|
+
|
|
215
|
+
**By page-type, generate a DEEP scenario for each (before expanding UI/validation/subscription):**
|
|
216
|
+
|
|
217
|
+
| Page-type | Must-cover viewpoints (each with a data assertion) |
|
|
218
|
+
|---|---|
|
|
219
|
+
| **e-commerce list / home** | list-data (card has image+name+price+add) · product-detail-consistency · cart-correctness · category-filter-correctness · **brand-filter-correctness (separate from category)** · add-to-cart success · nav-core |
|
|
220
|
+
| **form** | required-validation · format/boundary · submit-success |
|
|
221
|
+
| **auth** | valid-login · invalid-credential · access-control |
|
|
222
|
+
|
|
223
|
+
**Required assertion shapes (use these, not bare visibility):**
|
|
224
|
+
- Card info: assert at **card level** (image+name+price together), e.g. `User see all [Product Card] contain {{...}}` — not `see [Section]`.
|
|
225
|
+
- Cross-screen consistency (detail/cart): **capture then compare** —
|
|
226
|
+
```gherkin
|
|
227
|
+
When User remember [Product Name] text as {{selected_product_name}}
|
|
228
|
+
And User remember [Product Price] text as {{selected_product_price}}
|
|
229
|
+
And User click [View Product] link
|
|
230
|
+
Then User see [Detail Product Name] header with {{selected_product_name}}
|
|
231
|
+
And User see [Detail Product Price] text contains {{selected_product_price}}
|
|
232
|
+
```
|
|
233
|
+
Cross-screen target → tag `@manual` + `# Deferred to a flow (home -> detail)`.
|
|
234
|
+
- Filter result (category AND brand, separately): `Then User see all [Result Product Name] contain {{selected_category}}` — proves EVERY item belongs, not one.
|
|
235
|
+
|
|
236
|
+
**Depth is a GATE dimension (harness-roadmap P1) — self-raise, never silently go shallow:**
|
|
237
|
+
- For every data-correctness theme the catalog marks `depth.requires: data-assertion`, emit its `depth.template` shape by **default** — don't wait for the repair loop. `sungen audit` measures `businessDepth` (ratio of these scenarios that assert data) against an intent threshold (functional ≥ 0.70); below it the **gate FAILs**.
|
|
238
|
+
- `depth.cross_screen: true` (cart / detail / filter / brand correctness) → write the deep capture/compare shape but tag `@manual` + `# Deferred to a flow (...)`. These are excluded from the ratio (they're correctly deferred), so they don't hurt depth.
|
|
239
|
+
- **If the spec lacks the concrete value** a deep assertion needs (exact message, price, count): still write the deep shape with a `{{var}}` placeholder and leave a `# SPEC-GAP: <field> value not in spec` comment — do **not** downgrade to `see [X] section`. A visible gap is better than a silent shallow pass.
|
|
240
|
+
- **Blind-Spot Memory:** before finishing, run `sungen blindspot list --prompt` (Bash) and make sure the suite satisfies each recorded pattern (e.g. "for any Add/Create action: check success + resulting data state + duplicate/double-submit"). These are gaps QA hit before — don't repeat them.
|
|
241
|
+
|
|
242
|
+
**First-pass anti-patterns (exactly what the gate/reviewer reject — avoid them):**
|
|
243
|
+
- Title↔steps mismatch (e.g. a "no-result" scenario that clicks a query which returns products).
|
|
244
|
+
- Tautology `Then`: `click [Next Slide]` → `see [Carousel] section` (proves nothing).
|
|
245
|
+
- Business-critical scenario ending at `see [Added] modal` / `see [Cart] page` with no data assertion.
|
|
246
|
+
- Brand filter covered only as navigation (must assert products belong to the brand).
|
|
247
|
+
|
|
248
|
+
**Balance:** cover all the above (deep) BEFORE expanding subscription / UI-presence / extra validation edge cases.
|
|
249
|
+
|
|
211
250
|
#### Tier 1 guard — minimum before writing scenarios
|
|
212
251
|
|
|
213
252
|
| Spec section | Minimum requirement | Tag |
|
|
@@ -41,19 +41,33 @@ export class TestDataLoader {
|
|
|
41
41
|
}
|
|
42
42
|
|
|
43
43
|
get(key: string): string {
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
44
|
+
// Captured/runtime vars (set() below) are stored under their literal — possibly
|
|
45
|
+
// dotted — key (e.g. "cart.product_name"), so check the flat key first.
|
|
46
|
+
let current: any = this.data[key];
|
|
47
|
+
if (current === undefined || current === null) {
|
|
48
|
+
// Fall back to nested navigation for YAML-structured keys (e.g. "cart.qty_two").
|
|
49
|
+
current = this.data;
|
|
50
|
+
for (const part of key.split('.')) {
|
|
51
|
+
if (current == null || typeof current !== 'object') {
|
|
52
|
+
throw new Error(`Test data key not found: ${key} (failed at '${part}')`);
|
|
53
|
+
}
|
|
54
|
+
current = current[part];
|
|
49
55
|
}
|
|
50
|
-
current = current[part];
|
|
51
56
|
}
|
|
52
57
|
if (current === undefined || current === null) {
|
|
53
58
|
throw new Error(`Test data key not found: ${key}`);
|
|
54
59
|
}
|
|
55
60
|
return String(current);
|
|
56
61
|
}
|
|
62
|
+
|
|
63
|
+
/**
|
|
64
|
+
* Set a value at runtime (used by captured-variable steps:
|
|
65
|
+
* `User remember [X] text as {{var}}`). Stored at top level so a later
|
|
66
|
+
* `{{var}}` reference resolves via get().
|
|
67
|
+
*/
|
|
68
|
+
set(key: string, value: string): void {
|
|
69
|
+
this.data[key] = value;
|
|
70
|
+
}
|
|
57
71
|
}
|
|
58
72
|
|
|
59
73
|
function loadYamlSync(filePath: string): Record<string, any> | null {
|
|
@@ -6,7 +6,9 @@
|
|
|
6
6
|
*
|
|
7
7
|
* Safeguards enforced on every save AND every assertSafeToUse call:
|
|
8
8
|
* 1. .env must appear in .gitignore (abort if not).
|
|
9
|
-
* 2. No tracked file may contain
|
|
9
|
+
* 2. No tracked file may contain an assigned token value (FIGMA_PAT=<token>).
|
|
10
|
+
* The scan matches a token-shaped value, not bare mentions of the key in
|
|
11
|
+
* code or docs (those are not secrets).
|
|
10
12
|
*/
|
|
11
13
|
/**
|
|
12
14
|
* Load the Figma PAT.
|
|
@@ -28,7 +30,8 @@ export declare function savePat(cwd: string, token: string): Promise<void>;
|
|
|
28
30
|
export declare function clearPat(cwd: string): void;
|
|
29
31
|
/**
|
|
30
32
|
* Run tracked-file scan on every Figma CLI / skill invocation.
|
|
31
|
-
* Aborts with actionable message if
|
|
33
|
+
* Aborts with an actionable message if an assigned token value is found in any
|
|
34
|
+
* tracked file (bare mentions of the key name are ignored).
|
|
32
35
|
*
|
|
33
36
|
* @throws Error with remediation instructions on safeguard failure.
|
|
34
37
|
*/
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"figma-auth.d.ts","sourceRoot":"","sources":["../../../src/tools/figma/figma-auth.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"figma-auth.d.ts","sourceRoot":"","sources":["../../../src/tools/figma/figma-auth.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAyEH;;;;GAIG;AACH,wBAAgB,OAAO,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAWvD;AAED;;;;;GAKG;AACH,wBAAsB,OAAO,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAiCvE;AAED;;;GAGG;AACH,wBAAgB,QAAQ,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAM1C;AAED;;;;;;GAMG;AACH,wBAAsB,eAAe,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAShE"}
|
|
@@ -7,7 +7,9 @@
|
|
|
7
7
|
*
|
|
8
8
|
* Safeguards enforced on every save AND every assertSafeToUse call:
|
|
9
9
|
* 1. .env must appear in .gitignore (abort if not).
|
|
10
|
-
* 2. No tracked file may contain
|
|
10
|
+
* 2. No tracked file may contain an assigned token value (FIGMA_PAT=<token>).
|
|
11
|
+
* The scan matches a token-shaped value, not bare mentions of the key in
|
|
12
|
+
* code or docs (those are not secrets).
|
|
11
13
|
*/
|
|
12
14
|
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
13
15
|
if (k2 === undefined) k2 = k;
|
|
@@ -53,6 +55,13 @@ const exec_file_no_throw_1 = require("../../utils/exec-file-no-throw");
|
|
|
53
55
|
const PAT_KEY = 'FIGMA_PAT';
|
|
54
56
|
const ENV_FILE = '.env';
|
|
55
57
|
const GITIGNORE_FILE = '.gitignore';
|
|
58
|
+
// A real leak is an assigned token value (`FIGMA_PAT=<token>`), not a bare
|
|
59
|
+
// mention of the key name in code or documentation. Figma PATs are long
|
|
60
|
+
// (~40+ chars), so we only flag an assignment whose value looks like an actual
|
|
61
|
+
// token: at least 16 non-space characters after `=` (an optional quote allowed).
|
|
62
|
+
// This is an ERE pattern for `git grep -E`; it deliberately does NOT match this
|
|
63
|
+
// file's own comments or template literals (`${PAT_KEY}=...`).
|
|
64
|
+
const PAT_VALUE_PATTERN = `${PAT_KEY}=['"]?[^[:space:]]{16,}`;
|
|
56
65
|
// ---------------------------------------------------------------------------
|
|
57
66
|
// Internal helpers
|
|
58
67
|
// ---------------------------------------------------------------------------
|
|
@@ -89,11 +98,11 @@ function isEnvIgnored(cwd) {
|
|
|
89
98
|
return /^\s*(\.env[\w.*]*|\*\.env)\s*$/m.test(content);
|
|
90
99
|
}
|
|
91
100
|
/**
|
|
92
|
-
*
|
|
101
|
+
* Scan tracked files for an assigned Figma token value (not bare key mentions).
|
|
93
102
|
* Returns list of matching tracked file paths (empty = safe).
|
|
94
103
|
*/
|
|
95
104
|
async function findTrackedFilesWithPat(cwd) {
|
|
96
|
-
const result = await (0, exec_file_no_throw_1.execFileNoThrow)('git', ['grep', '-
|
|
105
|
+
const result = await (0, exec_file_no_throw_1.execFileNoThrow)('git', ['grep', '-lE', PAT_VALUE_PATTERN], cwd);
|
|
97
106
|
if (result.status === 0 && result.stdout.trim()) {
|
|
98
107
|
return result.stdout.trim().split('\n').filter(Boolean);
|
|
99
108
|
}
|
|
@@ -131,12 +140,12 @@ async function savePat(cwd, token) {
|
|
|
131
140
|
throw new Error(`Refusing to save FIGMA_PAT: ".env" is not listed in ${GITIGNORE_FILE}.\n` +
|
|
132
141
|
`Add the following line to ${path.join(cwd, GITIGNORE_FILE)} and re-run:\n\n .env\n`);
|
|
133
142
|
}
|
|
134
|
-
// Safeguard 2: no tracked file may contain
|
|
143
|
+
// Safeguard 2: no tracked file may contain an assigned token value
|
|
135
144
|
const leakedFiles = await findTrackedFilesWithPat(cwd);
|
|
136
145
|
if (leakedFiles.length > 0) {
|
|
137
|
-
throw new Error(`Refusing to save FIGMA_PAT: token
|
|
146
|
+
throw new Error(`Refusing to save FIGMA_PAT: a token value appears in tracked file(s):\n` +
|
|
138
147
|
leakedFiles.map((f) => ` ${f}`).join('\n') +
|
|
139
|
-
`\n\nRemove
|
|
148
|
+
`\n\nRemove the token from those files, commit the removal, then re-run.\n`);
|
|
140
149
|
}
|
|
141
150
|
// Upsert: replace existing line or append
|
|
142
151
|
const lines = readEnvLines(cwd);
|
|
@@ -167,16 +176,17 @@ function clearPat(cwd) {
|
|
|
167
176
|
}
|
|
168
177
|
/**
|
|
169
178
|
* Run tracked-file scan on every Figma CLI / skill invocation.
|
|
170
|
-
* Aborts with actionable message if
|
|
179
|
+
* Aborts with an actionable message if an assigned token value is found in any
|
|
180
|
+
* tracked file (bare mentions of the key name are ignored).
|
|
171
181
|
*
|
|
172
182
|
* @throws Error with remediation instructions on safeguard failure.
|
|
173
183
|
*/
|
|
174
184
|
async function assertSafeToUse(cwd) {
|
|
175
185
|
const leakedFiles = await findTrackedFilesWithPat(cwd);
|
|
176
186
|
if (leakedFiles.length > 0) {
|
|
177
|
-
throw new Error(`Security check failed:
|
|
187
|
+
throw new Error(`Security check failed: a Figma token value appears in tracked file(s):\n` +
|
|
178
188
|
leakedFiles.map((f) => ` ${f}`).join('\n') +
|
|
179
|
-
`\n\nRemove
|
|
189
|
+
`\n\nRemove the token from those files and commit the removal before using Figma commands.\n`);
|
|
180
190
|
}
|
|
181
191
|
}
|
|
182
192
|
//# sourceMappingURL=figma-auth.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"figma-auth.js","sourceRoot":"","sources":["../../../src/tools/figma/figma-auth.ts"],"names":[],"mappings":";AAAA
|
|
1
|
+
{"version":3,"file":"figma-auth.js","sourceRoot":"","sources":["../../../src/tools/figma/figma-auth.ts"],"names":[],"mappings":";AAAA;;;;;;;;;;;GAWG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AA8EH,0BAWC;AAQD,0BAiCC;AAMD,4BAMC;AASD,0CASC;AA9JD,uCAAyB;AACzB,2CAA6B;AAC7B,uEAAiE;AAEjE,MAAM,OAAO,GAAG,WAAW,CAAC;AAC5B,MAAM,QAAQ,GAAG,MAAM,CAAC;AACxB,MAAM,cAAc,GAAG,YAAY,CAAC;AAEpC,2EAA2E;AAC3E,wEAAwE;AACxE,+EAA+E;AAC/E,iFAAiF;AACjF,gFAAgF;AAChF,+DAA+D;AAC/D,MAAM,iBAAiB,GAAG,GAAG,OAAO,yBAAyB,CAAC;AAE9D,8EAA8E;AAC9E,mBAAmB;AACnB,8EAA8E;AAE9E,SAAS,WAAW,CAAC,GAAW;IAC9B,OAAO,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;AAClC,CAAC;AAED,SAAS,aAAa,CAAC,GAAW;IAChC,OAAO,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,cAAc,CAAC,CAAC;AACxC,CAAC;AAED;;GAEG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,EAAE,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC;IAC5B,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,EAAE,CAAC;QAAE,OAAO,EAAE,CAAC;IAClC,OAAO,EAAE,CAAC,YAAY,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;AACjD,CAAC;AAED;;GAEG;AACH,SAAS,aAAa,CAAC,GAAW,EAAE,KAAe;IACjD,EAAE,CAAC,aAAa,CAAC,WAAW,CAAC,GAAG,CAAC,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,CAAC;AAC/D,CAAC;AAED;;GAEG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,EAAE,GAAG,aAAa,CAAC,GAAG,CAAC,CAAC;IAC9B,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,EAAE,CAAC;QAAE,OAAO,KAAK,CAAC;IACrC,MAAM,OAAO,GAAG,EAAE,CAAC,YAAY,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;IAC5C,6DAA6D;IAC7D,OAAO,iCAAiC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;AACzD,CAAC;AAED;;;GAGG;AACH,KAAK,UAAU,uBAAuB,CAAC,GAAW;IAChD,MAAM,MAAM,GAAG,MAAM,IAAA,oCAAe,EAAC,KAAK,EAAE,CAAC,MAAM,EAAE,KAAK,EAAE,iBAAiB,CAAC,EAAE,GAAG,CAAC,CAAC;IACrF,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QAChD,OAAO,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAC1D,CAAC;IACD,OAAO,EAAE,CAAC;AACZ,CAAC;AAED,8EAA8E;AAC9E,aAAa;AACb,8EAA8E;AAE9E;;;;GAIG;AACH,SAAgB,OAAO,CAAC,GAAW;IACjC,IAAI,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC;QAAE,OAAO,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;IAEtD,MAAM,KAAK,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC;IAChC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC;QAC5B,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,OAAO,GAAG,CAAC,EAAE,CAAC;YACtC,OAAO,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAClD,CAAC;IACH,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;;;GAKG;AACI,KAAK,UAAU,OAAO,CAAC,GAAW,EAAE,KAAa;IACtD,uCAAuC;IACvC,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACvB,MAAM,IAAI,KAAK,CACb,uDAAuD,cAAc,KAAK;YACxE,6BAA6B,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,cAAc,CAAC,0BAA0B,CACxF,CAAC;IACJ,CAAC;IAED,mEAAmE;IACnE,MAAM,WAAW,GAAG,MAAM,uBAAuB,CAAC,GAAG,CAAC,CAAC;IACvD,IAAI,WAAW,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CACb,yEAAyE;YACvE,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;YAC3C,2EAA2E,CAC9E,CAAC;IACJ,CAAC;IAED,0CAA0C;IAC1C,MAAM,KAAK,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC;IAChC,MAAM,GAAG,GAAG,KAAK,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,GAAG,OAAO,GAAG,CAAC,CAAC,CAAC;IACvE,MAAM,OAAO,GAAG,GAAG,OAAO,IAAI,KAAK,EAAE,CAAC;IACtC,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC;QACb,KAAK,CAAC,GAAG,CAAC,GAAG,OAAO,CAAC;IACvB,CAAC;SAAM,CAAC;QACN,iDAAiD;QACjD,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC;YACvD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACjB,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;IACtB,CAAC;IACD,aAAa,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;AAC5B,CAAC;AAED;;;GAGG;AACH,SAAgB,QAAQ,CAAC,GAAW;IAClC,MAAM,KAAK,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC;IAChC,MAAM,QAAQ,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,GAAG,OAAO,GAAG,CAAC,CAAC,CAAC;IAC1E,IAAI,QAAQ,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM,EAAE,CAAC;QACrC,aAAa,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;IAC/B,CAAC;AACH,CAAC;AAED;;;;;;GAMG;AACI,KAAK,UAAU,eAAe,CAAC,GAAW;IAC/C,MAAM,WAAW,GAAG,MAAM,uBAAuB,CAAC,GAAG,CAAC,CAAC;IACvD,IAAI,WAAW,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CACb,0EAA0E;YACxE,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;YAC3C,6FAA6F,CAChG,CAAC;IACJ,CAAC;AACH,CAAC"}
|
|
@@ -0,0 +1,267 @@
|
|
|
1
|
+
# Sungen Orchestration & Harness — Detailed Spec
|
|
2
|
+
|
|
3
|
+
> Spec triển khai cho lần refactor lớn: chuyển từ **"Context Engine + Skill" (workflow tuyến tính)** sang **Orchestration + Harness Engine**.
|
|
4
|
+
> Mục tiêu chính: **nâng chất lượng testcase** (đo & gate thay vì nhờ AI). Bất biến: `generate`, auto-fix selector, `delivery`, reports.
|
|
5
|
+
> Tài liệu nền: `reports/sungen_direction_solution.md`, `reports/sungen_refactor_spec.md`, `reports/sungen_home_gherkin_viewpoint_coverage_review.md`.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 0. TL;DR cho người triển khai
|
|
10
|
+
|
|
11
|
+
- **Orchestration** = bộ não điều phối: *khám phá input → chọn route → gọi AI sinh artifact → đo bằng Harness → repair → hội tụ → ghi ledger*.
|
|
12
|
+
- **Harness** = lớp kiểm soát chất lượng: **Guide** (skill markdown, đã có) + **Sensor** (code tất định, MỚI) + **Repair loop** + **QA checkpoint**.
|
|
13
|
+
- **Nguyên tắc vàng:** chất lượng đến từ **đo & gate** (Sensor), không từ "nhờ AI làm đúng" (Guide). Bằng chứng: case `home` 6.5/10 — viewpoint AI tốt nhưng không gì enforce.
|
|
14
|
+
- **Refactor Phase 1 (làm trước, tất định, verify được):** `sungen audit` = 4 Sensor + Viewpoint Gate + score + report. Đây là phần code lõi của lần refactor này.
|
|
15
|
+
- **Orchestrator AI** (`/sungen:design`) gọi `sungen audit` trong vòng repair. Skill cũ giữ làm Guide.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 1. Trách nhiệm của Orchestration (Orchestrator làm gì)
|
|
20
|
+
|
|
21
|
+
Orchestrator KHÔNG sinh testcase trực tiếp. Nó trả lời và điều phối theo các câu hỏi:
|
|
22
|
+
|
|
23
|
+
| # | Câu hỏi | Thành phần phụ trách | Output |
|
|
24
|
+
|---|---|---|---|
|
|
25
|
+
| 1 | Goal là gì? Loại test gì (web-ui/api/mobile/perf)? | Goal Resolver | `goal` |
|
|
26
|
+
| 2 | Có những input nào? Đầy đủ tới đâu? Mâu thuẫn gì? | **Context Discovery** | `discovery-report` |
|
|
27
|
+
| 3 | Nên đi route nào (và song song không)? | **Route Planner** | `route-plan` |
|
|
28
|
+
| 4 | Sinh viewpoint-overview + Gherkin | AI (qua Guide skill) | `viewpoint-overview`, `.feature` |
|
|
29
|
+
| 5 | Output đã đủ tốt chưa? Thiếu/trùng/nông ở đâu? | **Harness Sensors** | `audit-report` |
|
|
30
|
+
| 6 | Chưa đạt → sửa ở đâu, sinh bù gì? | **Repair Loop** | artifact mới |
|
|
31
|
+
| 7 | Spec đã đổi chưa? Phần nào tái dùng/tái tạo? | **Change Planner** | `change-plan` |
|
|
32
|
+
| 8 | Tốn bao nhiêu tài nguyên, ở bước nào? | **Usage Ledger** | `usage-report` |
|
|
33
|
+
| 9 | QA quyết định gì? Học được gì? | **QA Checkpoint + Feedback** | `feedback.jsonl` |
|
|
34
|
+
|
|
35
|
+
**Bất biến:** Orchestrator dừng ở ranh giới artifact (`.feature`/`test-data`/`selectors`). Từ đó trở đi là `generate → run-test (auto-fix) → delivery` — **không đổi**.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## 2. FLOW CHÍNH (Main Flow) — Spec-first, web-ui
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
┌─────────────────────────────┐
|
|
43
|
+
QA cung cấp: │ /sungen:design <screen> │
|
|
44
|
+
spec.md, figma, │ (Orchestrator entrypoint) │
|
|
45
|
+
screenshot, url, └──────────────┬──────────────┘
|
|
46
|
+
prompt │
|
|
47
|
+
▼
|
|
48
|
+
┌─────────────────────────────┐
|
|
49
|
+
(1) │ CONTEXT DISCOVERY │ đọc MỌI nguồn, không bắt chọn
|
|
50
|
+
│ - liệt kê nguồn + độ đầy đủ │
|
|
51
|
+
│ - phát hiện mâu thuẫn │
|
|
52
|
+
│ - RECOMMEND route + nguồn ưu │──► discovery-report.json
|
|
53
|
+
└──────────────┬──────────────┘
|
|
54
|
+
▼
|
|
55
|
+
┌─────────────────────────────┐
|
|
56
|
+
(2) │ ROUTE PLANNER │ mặc định: Spec-first
|
|
57
|
+
│ chọn 1..n route (song song) │──► route-plan.json
|
|
58
|
+
└──────────────┬──────────────┘
|
|
59
|
+
▼
|
|
60
|
+
┌─────────────────────────────┐
|
|
61
|
+
(3) │ VIEWPOINT PLANNER (AI+Guide) │ LUÔN sinh overview trước
|
|
62
|
+
│ → viewpoint-overview │──► test-viewpoint.md
|
|
63
|
+
│ (id, priority, evidence) │
|
|
64
|
+
└──────────────┬──────────────┘
|
|
65
|
+
▼
|
|
66
|
+
┌─────────────────────────────┐
|
|
67
|
+
(4) │ VIEWPOINT GATE (Sensor) │◄── seed Universal Catalog (reference)
|
|
68
|
+
│ Required/High đã đủ chưa? │
|
|
69
|
+
└───────┬──────────────┬───────┘
|
|
70
|
+
FAIL │ │ PASS
|
|
71
|
+
(sinh bù viewpoint) ▼
|
|
72
|
+
┌─────────────────────────────┐
|
|
73
|
+
(5) │ GHERKIN GENERATOR (AI+Guide)│ Tier critical+high trước
|
|
74
|
+
│ → .feature + test-data │
|
|
75
|
+
└──────────────┬──────────────┘
|
|
76
|
+
▼
|
|
77
|
+
┌─────────────────────────────┐
|
|
78
|
+
(6) │ HARNESS SENSORS (sungen audit) │
|
|
79
|
+
│ ① traceability ② coverage-balance │──► audit-report.json
|
|
80
|
+
│ ③ assertion-depth ④ duplicate │ + score
|
|
81
|
+
└───────┬──────────────────────┬─────────┘
|
|
82
|
+
FAIL │ │ PASS
|
|
83
|
+
▼ │
|
|
84
|
+
┌───────────────┐ │
|
|
85
|
+
(7) │ REPAIR LOOP │ ngân sách N vòng
|
|
86
|
+
│ sửa đúng chỗ │ (vd: sinh bù VP-DATA-CONSISTENCY,
|
|
87
|
+
│ sensor chỉ ra │ nâng assertion nông → có {{data}})
|
|
88
|
+
└──────┬────────┘
|
|
89
|
+
└──────────► quay lại (5)/(6)
|
|
90
|
+
│ (đạt gate hoặc hết ngân sách → báo gap)
|
|
91
|
+
▼
|
|
92
|
+
┌─────────────────────────────┐
|
|
93
|
+
(8) │ CONVERGE + LEDGER + CHECKPOINT │
|
|
94
|
+
│ artifact contract đầy đủ + usage-report │
|
|
95
|
+
│ + manifest (scenario↔vp↔spec-hash) │
|
|
96
|
+
│ → QA review/feedback │
|
|
97
|
+
└──────────────┬──────────────┘
|
|
98
|
+
▼
|
|
99
|
+
[BẤT BIẾN] generate → run-test(auto-fix) → delivery → dashboard
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Điểm khác biệt cốt lõi so với hiện tại: **(1) Discovery + recommend** thay cho hỏi-chọn; **(4) Gate** + **(6) Sensors** thay cho "nhờ AI tự đủ"; **(7) Repair** có ngân sách; **(8) Ledger + manifest** để đo & tái dùng.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 3. FLOW BỔ SUNG (Supplementary Routes)
|
|
107
|
+
|
|
108
|
+
Tất cả route hội tụ về cùng **artifact contract** (Mục 6). Route chỉ khác ở *nguồn tri thức đầu vào*.
|
|
109
|
+
|
|
110
|
+
| Route | Khi nào kích hoạt | Khác biệt so với main |
|
|
111
|
+
|---|---|---|
|
|
112
|
+
| **A. Spec-first** (mặc định) | spec.md đủ tốt | Như Mục 2 |
|
|
113
|
+
| **B. Source-first** | spec mỏng/thiếu, có source code | Bước (3) thay bằng *Source Behavior Miner*: trích endpoint/screen/state/validation/permission từ code → viewpoint |
|
|
114
|
+
| **C. History-first** | có testcase cũ tương tự (cần Lớp 2/feedback) | Bước (3): search cluster testcase cũ → trích viewpoint chung → recommend |
|
|
115
|
+
| **D. Defect-first** | feature vùng rủi ro cao (cần defect history) | Ưu tiên viewpoint từng bắt defect; risk-based |
|
|
116
|
+
| **E. QA-guided** | QA bổ sung viewpoint / AI confidence thấp | QA thêm viewpoint → phân loại → check trùng → sinh variant |
|
|
117
|
+
|
|
118
|
+
**Parallel execution:** khi nhiều nguồn cùng có (vd spec + source + history), Route Planner chạy song song (Agent đọc spec / Agent đọc source / Agent search history) → **merge viewpoint candidates → khử trùng → rank theo risk/coverage/evidence**. Đây cũng là cách trả lời yêu cầu "recommend thay vì bắt chọn": tất cả nguồn được *gộp*, không chọn 1.
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## 4. CÂY QUYẾT ĐỊNH KHI GẶP HẠN CHẾ (Discovery / Fallback)
|
|
123
|
+
|
|
124
|
+
> "Khi gặp hạn chế thì khám phá hoặc phát triển theo flow nào" — đây là logic fallback của Orchestrator. Nguyên tắc: **không đứt giữa chừng**; luôn có bước kế tiếp; nếu bí thì *output assumption rõ ràng* thay vì im lặng.
|
|
125
|
+
|
|
126
|
+
```
|
|
127
|
+
Bắt đầu: cần sinh viewpoint/Gherkin cho <screen>
|
|
128
|
+
│
|
|
129
|
+
├─ spec.md đủ tốt? ───────────── YES ─► Route A (Spec-first)
|
|
130
|
+
│ │ NO
|
|
131
|
+
│ ▼
|
|
132
|
+
├─ có source code? ───────────── YES ─► Route B (Source-first): mine behavior từ code
|
|
133
|
+
│ │ NO
|
|
134
|
+
│ ▼
|
|
135
|
+
├─ có testcase/lịch sử tương tự? ─ YES ─► Route C (History-first): trích viewpoint từ cluster
|
|
136
|
+
│ │ NO
|
|
137
|
+
│ ▼
|
|
138
|
+
├─ thuộc domain rủi ro + có defect history? ─ YES ─► Route D (Defect-first)
|
|
139
|
+
│ │ NO
|
|
140
|
+
│ ▼
|
|
141
|
+
└─ HỎI QA (Route E) ─► QA chưa phản hồi? ─► OUTPUT với ASSUMPTION LIST rõ ràng (đánh dấu cần xác minh)
|
|
142
|
+
|
|
143
|
+
Trong quá trình chạy, nếu Sensor (bước 6) FAIL:
|
|
144
|
+
│
|
|
145
|
+
├─ Viewpoint Gate FAIL (thiếu Required/High) ─► Repair: sinh bù viewpoint + scenario cho VP thiếu
|
|
146
|
+
├─ Assertion-depth FAIL (quá nhiều Then nông) ─► Repair: nâng assertion (thêm {{data}}/observable), hoặc
|
|
147
|
+
│ nếu thiếu data cụ thể → cần spec/flow ► gợi ý add-flow (cross-screen)
|
|
148
|
+
├─ Coverage-balance FAIL (VP phụ expand sâu, VP High mỏng) ─► Repair: dừng expand VP phụ, sinh VP High
|
|
149
|
+
├─ Duplicate FAIL (cluster trùng nghĩa) ─► Repair: merge/loại, giữ 1 đại diện + ghi rõ EP variants
|
|
150
|
+
└─ Traceability FAIL (scenario không map viewpoint) ─► Repair: gắn vp-id vào scenario
|
|
151
|
+
|
|
152
|
+
Hết ngân sách repair (N vòng) mà vẫn FAIL:
|
|
153
|
+
└─► KHÔNG lặp vô hạn. Ghi gap vào audit-report + đưa QA checkpoint (human-in-the-loop).
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
**Quy tắc "khám phá thêm khi nông":** nếu assertion-depth sensor báo một viewpoint critical chỉ assert "see page/section" (vụ home: Cart/Detail/Filter), Orchestrator phải nhận diện đây là **cross-screen** → khuyến nghị chuyển sang **flow** (`add-flow`) và/hoặc cần **năng lực DSL capture biến** (xem `reports/sungen_refactor_spec.md` §5.4) — thay vì giả vờ pass bằng assertion nông.
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## 5. HARNESS — Guide / Sensor / Repair / Checkpoint
|
|
161
|
+
|
|
162
|
+
### 5.1 Guide (AI, giữ nguyên skill markdown)
|
|
163
|
+
Skill hiện có (`sungen-tc-generation`, `sungen-viewpoint`, `sungen-test-design-techniques`) là **nửa Guide** — steer AI làm đúng từ đầu. **Giữ**, nhưng đồng bộ threshold với Sensor.
|
|
164
|
+
|
|
165
|
+
### 5.2 Sensor (tất định — code MỚI, phần lõi refactor)
|
|
166
|
+
Bốn sensor + một gate. Input: `.feature` (qua GherkinParser) + `test-viewpoint.md` + (optional) `spec.md`.
|
|
167
|
+
|
|
168
|
+
| Sensor | Đo | Output | Tất định? |
|
|
169
|
+
|---|---|---|---|
|
|
170
|
+
| **Viewpoint Gate** | mỗi viewpoint Required/High có ≥1 scenario phủ | pass/fail + danh sách thiếu | heuristic keyword (hybrid) |
|
|
171
|
+
| **Traceability** | scenario map được tới viewpoint id; có evidence | % traceable | tất định |
|
|
172
|
+
| **Coverage-balance** | phân bố scenario theo VP-group & priority; phát hiện lệch | cảnh báo imbalance | tất định |
|
|
173
|
+
| **Assertion-depth** | tỉ lệ `Then` nông (chỉ see page/section) vs có data assert | % shallow | tất định |
|
|
174
|
+
| **Duplicate** | cluster scenario cùng shape/nghĩa | cluster + đề xuất | heuristic |
|
|
175
|
+
|
|
176
|
+
> Trung thực: Gate & Duplicate (ngữ nghĩa) là **hybrid** — heuristic rẻ chặn phần rõ, AI-judge cho vùng xám. Không hứa thuần tất định cho ngữ nghĩa.
|
|
177
|
+
|
|
178
|
+
### 5.3 Repair Loop
|
|
179
|
+
- Sensor trả **phản hồi cụ thể** (vd: "VP-DATA-CONSISTENCY Required nhưng 0 scenario; gợi ý: add-flow checkout").
|
|
180
|
+
- Orchestrator đưa phản hồi này lại bước sinh. **Ngân sách N vòng** (mặc định 2–3, như vòng auto-fix selector). Hết ngân sách → báo gap, không lặp vô hạn.
|
|
181
|
+
|
|
182
|
+
### 5.4 QA Checkpoint
|
|
183
|
+
- QA accept/reject/edit/add viewpoint. Mọi quyết định ghi `feedback.jsonl` (xem `reports/sungen_refactor_spec.md` §8).
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## 5b. Agent layer (tầng thứ 4 — isolation / independence / parallel)
|
|
188
|
+
|
|
189
|
+
Prompt (command) + Skill + Harness (deterministic) là đủ cho work tuần tự. **Agent** (context LLM riêng) chỉ thêm khi cần **isolation**, **góc nhìn độc lập**, hoặc **parallel**. Nguyên tắc chọn:
|
|
190
|
+
|
|
191
|
+
| Loại | Khi nào | Sungen |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| Deterministic (code) | đo/gate/fingerprint | `sungen audit/manifest/ledger` — KHÔNG agent-hóa |
|
|
194
|
+
| Skill (tri thức) | quy tắc context chính theo | gherkin-syntax, tc-generation, viewpoint, harness-audit |
|
|
195
|
+
| **Agent (context riêng)** | khám phá tốn token cần tóm gọn; cần phán xử độc lập; chạy song song | **discovery, reviewer** |
|
|
196
|
+
|
|
197
|
+
> "Senior QA" đã là role-prompt trong command — KHÔNG cần agent riêng cho nó.
|
|
198
|
+
|
|
199
|
+
**Hai agent (Claude Code sub-agent; Copilot chạy inline, fallback tuần tự):**
|
|
200
|
+
|
|
201
|
+
1. **`sungen-reviewer` (Judge — đòn bẩy cao nhất).** Reviewer độc lập, KHÔNG viết test, chỉ phán xử **ngữ nghĩa** mà gate tất định bỏ sót: steps có *chứng minh* title/viewpoint không; `Then` có observable không; business-critical có assertion sâu không; `@manual` có chính đáng không. Trả `VERDICT + ISSUES(+fix)`. Nhúng vào **repair loop** (step 5.5): `audit (structural) → reviewer (semantic) → merge → repair`. Generator và reviewer **tách context** → không rubber-stamp. Giải đúng vấn đề kẹt ~6.7 ("title đúng nhưng steps chưa prove").
|
|
202
|
+
|
|
203
|
+
2. **`sungen-discovery` (Explorer).** Đọc mọi nguồn (spec/figma/ui/live) trong **context riêng**, trả **discovery-report cô đọng** (sources/completeness/conflicts/route/key-facts) → orchestrator chính giữ context sạch để generate. Dùng ở bước discovery.
|
|
204
|
+
|
|
205
|
+
**Lưu ý:** mỗi agent tốn token → **ledger phải đo per-agent**; output agent phải có **schema rõ** (verdict / discovery-report) tránh chat lan man; **parallel route agents** (spec/source/history/defect) hoãn tới khi multi-source thật sự cần.
|
|
206
|
+
|
|
207
|
+
## 6. Convergence Contract (artifact bắt buộc — mọi route hội tụ về đây)
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
qa/screens/<screen>/requirements/test-viewpoint.md # viewpoint-overview (id, priority, evidence)
|
|
211
|
+
qa/screens/<screen>/features/<screen>.feature # Gherkin (format hiện tại — downstream không đổi)
|
|
212
|
+
qa/screens/<screen>/test-data/<screen>.yaml
|
|
213
|
+
.sungen/manifest/<screen>.json # scenario ↔ viewpoint ↔ spec-section fingerprint (cho reuse)
|
|
214
|
+
.sungen/reports/<screen>-audit.json # sensor + gate + score
|
|
215
|
+
.sungen/reports/<screen>-usage.json # token/cost/time ledger theo bước
|
|
216
|
+
.sungen/feedback/*.jsonl # QA feedback (local-first)
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
`.feature`/`test-data`/`selectors` giữ schema cũ → `generate`/`delivery` không đổi.
|
|
220
|
+
|
|
221
|
+
---
|
|
222
|
+
|
|
223
|
+
## 7. Orchestrator command (AI) — `/sungen:design` (sketch)
|
|
224
|
+
|
|
225
|
+
Lệnh AI mới điều phối flow chính; thay thế dần phần "hỏi-chọn" trong `create-test`. (`create-test` cũ vẫn chạy được — `design` là superset có harness.)
|
|
226
|
+
|
|
227
|
+
```
|
|
228
|
+
Role: Test Design Orchestrator.
|
|
229
|
+
Steps:
|
|
230
|
+
1. CONTEXT DISCOVERY: đọc spec.md, spec_figma.md, ui/*, (live url nếu có). KHÔNG hỏi "chọn nguồn".
|
|
231
|
+
→ Tóm tắt nguồn + độ đầy đủ + mâu thuẫn + RECOMMEND route. Trình bày recommendation (QA override được).
|
|
232
|
+
2. VIEWPOINT PLANNER: sinh test-viewpoint.md (id/priority/evidence) — LUÔN làm, kể cả có sẵn (diff & bổ sung).
|
|
233
|
+
3. Chạy `sungen audit --screen <s> --gate viewpoint` → nếu thiếu Required/High: sinh bù.
|
|
234
|
+
4. GHERKIN: sinh .feature + test-data (critical+high trước), gắn vp-id (traceability).
|
|
235
|
+
5. Chạy `sungen audit --screen <s>` (full). Đọc audit-report.
|
|
236
|
+
6. REPAIR (≤ N vòng): sửa đúng chỗ sensor chỉ ra. Hết ngân sách → báo gap.
|
|
237
|
+
7. Ghi usage-ledger + manifest. Trình bày score + gap cho QA (checkpoint).
|
|
238
|
+
8. AskUserQuestion next: review / run-test / feedback.
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
Skill được auto-load: `sungen-viewpoint`, `sungen-tc-generation`, `sungen-test-design-techniques`, `sungen-gherkin-syntax` (Guide); `sungen-harness-audit` (mới — mô tả cách đọc audit-report & repair).
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## 8. Mở rộng (test-type packs + executor providers)
|
|
246
|
+
|
|
247
|
+
Orchestrator/Harness/Discovery là **chung**. Chỉ thay theo loại test:
|
|
248
|
+
- **Test-type pack** = (viewpoint catalog + sensor set + contract): `web-ui`, `mobile-ui`, `api`, `performance`, `db-verify`.
|
|
249
|
+
- **Executor provider** = sinh code: Playwright / Appium / k6 / SQL runner (xem `reports/mobile-proposal.md`).
|
|
250
|
+
|
|
251
|
+
Ví dụ: pack `performance` thay Assertion-depth sensor bằng "Threshold sensor" (có p95/error-rate?), Viewpoint Gate dùng catalog load-profile. Cùng một orchestration loop.
|
|
252
|
+
|
|
253
|
+
---
|
|
254
|
+
|
|
255
|
+
## 9. Phase triển khai (không phá luồng cũ)
|
|
256
|
+
|
|
257
|
+
```
|
|
258
|
+
P1 (refactor lõi, TẤT ĐỊNH — làm trong lần này):
|
|
259
|
+
sungen audit = 4 sensor + viewpoint gate + score + report JSON + seed catalog
|
|
260
|
+
→ verify trên examples/automationexercise home: phải tái hiện gap 6.5/10
|
|
261
|
+
P2: /sungen:design (orchestrator AI) gọi audit trong repair loop; discovery+recommend
|
|
262
|
+
P3: manifest + spec-fingerprint (reuse); usage ledger
|
|
263
|
+
P4: feedback local (Nấc 1)
|
|
264
|
+
P5: test-type packs (api/perf/db) + executor providers (Appium…)
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
P1 là phần code của lần refactor này (Mục 5.2). Các phase sau bám thin-first: 1 workflow thật → học → mở rộng.
|