@sun-asterisk/sungen 3.2.12-beta.7 → 3.2.12-beta.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/orchestrator/templates/ai-src/commands/create-data-test.md +1 -1
- package/dist/orchestrator/templates/ai-src/commands/create-test.md +2 -2
- package/dist/orchestrator/templates/ai-src/skills/sungen-data-factory/SKILL.md +27 -0
- package/package.json +3 -3
- package/src/orchestrator/templates/ai-src/commands/create-data-test.md +1 -1
- package/src/orchestrator/templates/ai-src/commands/create-test.md +2 -2
- package/src/orchestrator/templates/ai-src/skills/sungen-data-factory/SKILL.md +27 -0
|
@@ -96,7 +96,7 @@ Run (local-first): `[ -x ./bin/sungen.js ] && ./bin/sungen.js data validate --sc
|
|
|
96
96
|
|
|
97
97
|
### 5.6 Domain / stateful preconditions (spec + viewpoint only — no DB/API needed)
|
|
98
98
|
|
|
99
|
-
If the spec describes a **stateful precondition** — an entity with relationships or a business state the test needs *before* it runs (e.g. "a paid order of user A", "a valid unused reset token ≤24h", "the session-storage buffer holds the entered values") —
|
|
99
|
+
If the spec describes a **stateful precondition** — an entity with relationships or a business state the test needs *before* it runs (e.g. "a paid order of user A", "a valid unused reset token ≤24h", "the session-storage buffer holds the entered values") — **auto-draft it: run `… data state --scaffold <name>`** to generate a recipe skeleton from the spec's precondition sentences (one resource per signal, provenance guessed, entity as `TODO`). Then **fill each `entity`/`desiredState`/`bindings` and confirm the domain specifics with the QA** (`AskUserQuestion`) — never invent domain values. (You can also hand-author the `resources:` graph in `qa/data-factory/recipes.yaml`.) Provenance is **not only api/db** — pick `prior-flow` / `fixture` / `client-storage` / `server-context` / `manual` (use `api`/`db` only when a datasource exists). Then run `data state --recipe <name>`: it validates the graph and renders a standardized **manual precondition** block (`@manual:data-setup` when it can't be auto-provisioned) to weave into the scenario's `Background`. This is the domain/stateful improvement that works with **only spec + viewpoint** — the testcase states the required state precisely instead of a vague note; it auto-provisions later if an api/db datasource is added (no rewrite). For a **journey where the same data flows across screens** (setup → confirmation → complete), declare a `shared: { entity, screens: [...] }` so one dataset is used across all of them (the Cross-artifact Gate then asserts they match).
|
|
100
100
|
|
|
101
101
|
### 5.7 Cross-artifact check (testcase ↔ data agree)
|
|
102
102
|
|
|
@@ -167,7 +167,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
167
167
|
|
|
168
168
|
**5d. Sequential fallback.** Use the single-context incremental path (Step 2: tier-by-tier `Write`/`Edit` batches) when: only **one** shard applies, **Copilot / no sub-agents**, or a constrained setup. Same output, just no speedup. **For flows**: `[Screen:Element]` namespace refs, test-data namespaced by phase, `@flow` tag.
|
|
169
169
|
|
|
170
|
-
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** Don't hand-invent per-field values — make them **standard**. For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + the spec's real `constraints` + `errorMap` placeholder→real code), then run (Bash, local-first) `[ -x ./bin/sungen.js ] && ./bin/sungen.js data lint --screen <name> || npx sungen data lint --screen <name>` and `… data gen --screen <name>`, per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves your scenario/namespaced data), so it is safe to run here. Weave the standardized **boundary/invalid** sets into `@cases` (`Scenario Outline` + `Examples`) with `CHK-*` trace; keep the map's `errorMap`/`constraints` aligned to the spec. Then run `… data validate --screen <name>` (values are semantically correct) and — once the `.feature` exists — `… data crosscheck --screen <name>` (testcase ↔ data agree: no dangling `{{var}}`, shared datasets identical); fix any `error`. If the spec has a **stateful precondition** (an entity/state the test needs first),
|
|
170
|
+
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** Don't hand-invent per-field values — make them **standard**. For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + the spec's real `constraints` + `errorMap` placeholder→real code), then run (Bash, local-first) `[ -x ./bin/sungen.js ] && ./bin/sungen.js data lint --screen <name> || npx sungen data lint --screen <name>` and `… data gen --screen <name>`, per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves your scenario/namespaced data), so it is safe to run here. Weave the standardized **boundary/invalid** sets into `@cases` (`Scenario Outline` + `Examples`) with `CHK-*` trace; keep the map's `errorMap`/`constraints` aligned to the spec. Then run `… data validate --screen <name>` (values are semantically correct) and — once the `.feature` exists — `… data crosscheck --screen <name>` (testcase ↔ data agree: no dangling `{{var}}`, shared datasets identical); fix any `error`. If the spec has a **stateful precondition** (an entity/state the test needs first), run `… data state --scaffold <name>` to auto-draft a recipe skeleton from the spec, **fill each `entity`/`desiredState` + confirm domain values with the QA**, then `… data state` and weave its `@manual:data-setup` `Background` block (see the `sungen-data-factory` skill — works with spec+viewpoint only, no DB/API). **This is the same standard as `/sungen:create-data-test`, applied inline — so you do NOT run that command separately afterwards.** A unit with **no input fields** (navigation / list / capture-compare) → **skip this step** (nothing to standardize).
|
|
171
171
|
|
|
172
172
|
5.4. **Depth self-check (deterministic — run BEFORE the audit).** Run `sungen depth-lint --screen <name>` (Bash). It reuses the audit's businessDepth classifier and splits every shallow business-critical scenario into two actionable buckets — act on them now so the audit/repair loop doesn't burn rounds on depth:
|
|
173
173
|
- **DEEPEN IN PLACE** — add a real value assertion to each (`User see all [X] contain {{v}}`, `User remember [X] as {{v}}` + `… with {{v}}`). The printed `template` is a **hint** keyed off the theme — apply judgment to the scenario's actual claim; do NOT paste a value assertion that doesn't fit (e.g. a carousel-visibility scenario should assert the product SET, not a price). If a flagged scenario is genuinely visibility/behavior (not data-correctness), that's an over-count — leave it and note it, never fake an assertion.
|
|
@@ -196,7 +196,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
196
196
|
4. Follow the `sungen-tc-generation` skill for section identification, viewpoint generation, and output format. **For flows**, use the "Flow Test Generation" section in the skill. When requirements exist, use the "Requirements-Driven Generation" strategy. **For Tier 1**, apply the **Lightweight Guard** — verify required fields, validation rules, business rules, security checks, and key state transitions all have TCs after generation. **For Tier 2+**, **MUST** apply the full **Mapping Contract** — walk every `spec.md` section top-to-bottom and produce the indicated TCs per Table 1; handle `test-viewpoint.md` per Table 2. Do not silently skip sections. Present sections as a numbered list and let user pick.
|
|
197
197
|
5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills. Generate **group-by-group** (one viewpoint group at a time, tier-by-tier `Write`/`Edit` batches) to stay under the output-token cap. **For flows**: use `[Screen:Element]` namespace format, namespace test-data by phase, add `@flow` tag. **If a scenario needs `@query` DB verification**, check the datasource `engine` first — unsupported engine → follow `sungen-gherkin-syntax` § "Unsupported DB engine — fallback" instead of authoring `@query` steps.
|
|
198
198
|
> **No parallel fan-out here.** Copilot has no sub-agents, so generation is sequential (the Claude Code variant fans out one `sungen-generator` per viewpoint group and merges). Same output, no speedup.
|
|
199
|
-
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + real `constraints` + `errorMap`), then run `sungen data lint --screen ${input:name}` and `sungen data gen --screen ${input:name}` per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves the rest), so it is safe here. Weave the standardized boundary/invalid sets into `@cases` with `CHK-*` trace. Then `… data validate` (values correct) + `… data crosscheck` (testcase↔data agree); for a stateful precondition,
|
|
199
|
+
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + real `constraints` + `errorMap`), then run `sungen data lint --screen ${input:name}` and `sungen data gen --screen ${input:name}` per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves the rest), so it is safe here. Weave the standardized boundary/invalid sets into `@cases` with `CHK-*` trace. Then `… data validate` (values correct) + `… data crosscheck` (testcase↔data agree); for a stateful precondition, `… data state --scaffold <name>` to auto-draft → fill entities + confirm with QA → `… data state`, weave the `@manual:data-setup` `Background` (works spec+viewpoint only). **Same standard as `/sungen:create-data-test`, applied inline — do NOT run that command separately afterwards.** No input fields (navigation/list) → skip.
|
|
200
200
|
5.4. **Depth self-check (deterministic — BEFORE the audit).** Run `sungen depth-lint --screen ${input:name}`. It splits every shallow business-critical scenario into **DEEPEN IN PLACE** (add a real value assertion — the printed `template` is a theme-keyed hint, apply judgment to the actual claim; never fake one onto a visibility/behavior scenario) and **CROSS-SCREEN** (route to a flow / tag `@manual:Mx` + reason — removes it from the depth denominator honestly). Act on both, re-run until `deepen` is empty (or only honest over-counts remain), THEN gate. Lifts first-pass `businessDepth` mechanically instead of via 2–3 repair rounds.
|
|
201
201
|
5.5. **Quality gate & repair (harness — always run).** Per `sungen-harness-audit`: run `sungen audit --screen ${input:name}` (structural), THEN do an **independent semantic review inline** using the `sungen-reviewer` criteria (does each scenario's steps PROVE its title/viewpoint? observable Thens? business-critical assertion depth?). Merge both sets of issues; if gate FAILs / findings exist, repair (budget 3) and re-audit — GATE missing theme → generate it (cross-screen → **automate it in the flow** via `/sungen:add-flow`, NOT a full `@manual` screen duplicate — `sungen audit` flags an automatable `@manual` as `MANUAL-AUTOMATABLE`; reserve `@manual:Mx` for true judgment/missing-capability); DEPTH → add data assertions; BALANCE → add business-core first; TRACE → align VP ids. Never fake a pass.
|
|
202
202
|
5.6. **Record.** `sungen manifest --screen ${input:name}`. Ledger **each phase** (not just repair) — pick one `runId` at the start and pass it so `trace`/`ledger report` show THIS run, not a mix: `sungen ledger record --screen ${input:name} --run <runId> --step <discovery|viewpoint|gherkin|audit|repair:N> --ms <elapsed>`. On re-run, start with `sungen manifest --screen ${input:name} --diff` and only regenerate changed sections.
|
|
@@ -15,6 +15,17 @@ Turn a screen/flow's fields into **standardized test data** — not guessed valu
|
|
|
15
15
|
Data Factory is **bundled with core** (runtime-free) — no `capability add` needed; the `sungen data …`
|
|
16
16
|
CLI is available out of the box.
|
|
17
17
|
|
|
18
|
+
**Auto-detection (act on these, don't wait to be asked).** `sungen generate`/`sungen audit` now NOTICE
|
|
19
|
+
the work a unit needs, even before any field-map exists:
|
|
20
|
+
- **`DATA-FACTORY-UNSTANDARDIZED`** — the unit has input fields but no `qa/data-factory/<name>.fields.yaml`.
|
|
21
|
+
→ author the field-map + `data gen` (this skill's main job).
|
|
22
|
+
- **`DATA-FACTORY-PRECONDITION-UNDECLARED`** — the spec describes a stateful precondition (a required
|
|
23
|
+
entity/state) but the test declares none. → **auto-draft a state-recipe** from the spec sentence, run
|
|
24
|
+
`sungen data state`, and weave its `@manual:data-setup` `Background`; confirm the drafted values with
|
|
25
|
+
the QA (`AskUserQuestion`) — never invent domain values.
|
|
26
|
+
When you see either finding, do the work rather than leaving it — that is what makes Sungen self-driving
|
|
27
|
+
instead of a toolbox the QA must operate.
|
|
28
|
+
|
|
18
29
|
## The 4-source method (phuong-phap-tao-test-data.md)
|
|
19
30
|
|
|
20
31
|
Never invent values from memory. Cross-reference four sources — each covers the previous one's gap:
|
|
@@ -139,6 +150,12 @@ Field values are not enough when a test needs a **business state first** — "a
|
|
|
139
150
|
a **`provenance`** (`prior-flow` | `fixture` | `client-storage` | `server-context` | `manual` |
|
|
140
151
|
`api` | `db`), optional `dependsOn`, `desiredState`, `lifecycle` (`reusable`/`consumable`/`expiring`/
|
|
141
152
|
`leased`/`mutable-state`/`unique-per-run`), and `with:` values — plus `bindings:` (`orderId: ${order.id}`).
|
|
153
|
+
**Auto-draft it — don't make the QA hand-write it.** When the spec describes a precondition (or you saw
|
|
154
|
+
a `PRECONDITION-UNDECLARED` finding), run **`sungen data state --scaffold <unit>`**: it drafts a recipe
|
|
155
|
+
skeleton from the spec's precondition sentences (one resource per signal, provenance guessed, entity
|
|
156
|
+
left as `TODO`). Then **fill each `entity` + `desiredState`/`bindings` from the spec and confirm the
|
|
157
|
+
domain specifics with the QA (`AskUserQuestion`)** — never invent domain values.
|
|
158
|
+
|
|
142
159
|
Run **`sungen data state`**: it validates the graph (cycles, dangling deps, bindings) and renders a
|
|
143
160
|
standardized **manual precondition** block — tagged **`@manual:data-setup`** when it can't be
|
|
144
161
|
auto-provisioned — to weave into the scenario's `Background`. **This works with only spec + viewpoint:
|
|
@@ -147,6 +164,16 @@ a resource uses `api`/`db` provenance (+ a `provider:` endpoint), the *same* rec
|
|
|
147
164
|
rewrite. For a **journey where the same data flows across screens** (setup → confirmation → complete),
|
|
148
165
|
add `shared: { entity, screens: [...] }` so one dataset is used across all of them.
|
|
149
166
|
|
|
167
|
+
## One QA-facing door: `/sungen:create-test`
|
|
168
|
+
|
|
169
|
+
QA never needs to remember which `sungen data …` sub-command to run. **`/sungen:create-test` is the
|
|
170
|
+
single entry** — it standardizes fields, validates, cross-checks, and drafts any stateful precondition
|
|
171
|
+
inline. The `sungen data gen|lint|validate|crosscheck|state|provenance|fixture` commands are the
|
|
172
|
+
**deterministic plumbing** the create-test loop (and CI) drives; run them directly only for a targeted
|
|
173
|
+
re-standardize. `/sungen:create-data-test` is the same pipeline on demand (re-standardize after editing
|
|
174
|
+
a field-map/`.overwrite`, or a unit authored before Data Factory) — not a step you must invoke after
|
|
175
|
+
create-test.
|
|
176
|
+
|
|
150
177
|
## Cross-artifact check — testcase ↔ data agree
|
|
151
178
|
|
|
152
179
|
`data validate` checks the data alone; **`sungen data crosscheck`** checks the testcase and its data
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sun-asterisk/sungen",
|
|
3
|
-
"version": "3.2.12-beta.
|
|
3
|
+
"version": "3.2.12-beta.9",
|
|
4
4
|
"description": "Deterministic E2E Test Compiler - Gherkin + Selectors → Playwright tests",
|
|
5
5
|
"main": "src/index.ts",
|
|
6
6
|
"types": "src/index.ts",
|
|
@@ -39,8 +39,8 @@
|
|
|
39
39
|
"@babel/types": "^7.28.5",
|
|
40
40
|
"@cucumber/gherkin": "^37.0.0",
|
|
41
41
|
"@cucumber/messages": "^31.0.0",
|
|
42
|
-
"@sungen/driver-data-factory": "3.2.12-beta.
|
|
43
|
-
"@sungen/driver-ui": "3.2.12-beta.
|
|
42
|
+
"@sungen/driver-data-factory": "3.2.12-beta.9",
|
|
43
|
+
"@sungen/driver-ui": "3.2.12-beta.9",
|
|
44
44
|
"chalk": "^5.6.2",
|
|
45
45
|
"commander": "^14.0.2",
|
|
46
46
|
"dotenv": "^17.2.3",
|
|
@@ -96,7 +96,7 @@ Run (local-first): `[ -x ./bin/sungen.js ] && ./bin/sungen.js data validate --sc
|
|
|
96
96
|
|
|
97
97
|
### 5.6 Domain / stateful preconditions (spec + viewpoint only — no DB/API needed)
|
|
98
98
|
|
|
99
|
-
If the spec describes a **stateful precondition** — an entity with relationships or a business state the test needs *before* it runs (e.g. "a paid order of user A", "a valid unused reset token ≤24h", "the session-storage buffer holds the entered values") —
|
|
99
|
+
If the spec describes a **stateful precondition** — an entity with relationships or a business state the test needs *before* it runs (e.g. "a paid order of user A", "a valid unused reset token ≤24h", "the session-storage buffer holds the entered values") — **auto-draft it: run `… data state --scaffold <name>`** to generate a recipe skeleton from the spec's precondition sentences (one resource per signal, provenance guessed, entity as `TODO`). Then **fill each `entity`/`desiredState`/`bindings` and confirm the domain specifics with the QA** (`AskUserQuestion`) — never invent domain values. (You can also hand-author the `resources:` graph in `qa/data-factory/recipes.yaml`.) Provenance is **not only api/db** — pick `prior-flow` / `fixture` / `client-storage` / `server-context` / `manual` (use `api`/`db` only when a datasource exists). Then run `data state --recipe <name>`: it validates the graph and renders a standardized **manual precondition** block (`@manual:data-setup` when it can't be auto-provisioned) to weave into the scenario's `Background`. This is the domain/stateful improvement that works with **only spec + viewpoint** — the testcase states the required state precisely instead of a vague note; it auto-provisions later if an api/db datasource is added (no rewrite). For a **journey where the same data flows across screens** (setup → confirmation → complete), declare a `shared: { entity, screens: [...] }` so one dataset is used across all of them (the Cross-artifact Gate then asserts they match).
|
|
100
100
|
|
|
101
101
|
### 5.7 Cross-artifact check (testcase ↔ data agree)
|
|
102
102
|
|
|
@@ -167,7 +167,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
167
167
|
|
|
168
168
|
**5d. Sequential fallback.** Use the single-context incremental path (Step 2: tier-by-tier `Write`/`Edit` batches) when: only **one** shard applies, **Copilot / no sub-agents**, or a constrained setup. Same output, just no speedup. **For flows**: `[Screen:Element]` namespace refs, test-data namespaced by phase, `@flow` tag.
|
|
169
169
|
|
|
170
|
-
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** Don't hand-invent per-field values — make them **standard**. For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + the spec's real `constraints` + `errorMap` placeholder→real code), then run (Bash, local-first) `[ -x ./bin/sungen.js ] && ./bin/sungen.js data lint --screen <name> || npx sungen data lint --screen <name>` and `… data gen --screen <name>`, per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves your scenario/namespaced data), so it is safe to run here. Weave the standardized **boundary/invalid** sets into `@cases` (`Scenario Outline` + `Examples`) with `CHK-*` trace; keep the map's `errorMap`/`constraints` aligned to the spec. Then run `… data validate --screen <name>` (values are semantically correct) and — once the `.feature` exists — `… data crosscheck --screen <name>` (testcase ↔ data agree: no dangling `{{var}}`, shared datasets identical); fix any `error`. If the spec has a **stateful precondition** (an entity/state the test needs first),
|
|
170
|
+
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** Don't hand-invent per-field values — make them **standard**. For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + the spec's real `constraints` + `errorMap` placeholder→real code), then run (Bash, local-first) `[ -x ./bin/sungen.js ] && ./bin/sungen.js data lint --screen <name> || npx sungen data lint --screen <name>` and `… data gen --screen <name>`, per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves your scenario/namespaced data), so it is safe to run here. Weave the standardized **boundary/invalid** sets into `@cases` (`Scenario Outline` + `Examples`) with `CHK-*` trace; keep the map's `errorMap`/`constraints` aligned to the spec. Then run `… data validate --screen <name>` (values are semantically correct) and — once the `.feature` exists — `… data crosscheck --screen <name>` (testcase ↔ data agree: no dangling `{{var}}`, shared datasets identical); fix any `error`. If the spec has a **stateful precondition** (an entity/state the test needs first), run `… data state --scaffold <name>` to auto-draft a recipe skeleton from the spec, **fill each `entity`/`desiredState` + confirm domain values with the QA**, then `… data state` and weave its `@manual:data-setup` `Background` block (see the `sungen-data-factory` skill — works with spec+viewpoint only, no DB/API). **This is the same standard as `/sungen:create-data-test`, applied inline — so you do NOT run that command separately afterwards.** A unit with **no input fields** (navigation / list / capture-compare) → **skip this step** (nothing to standardize).
|
|
171
171
|
|
|
172
172
|
5.4. **Depth self-check (deterministic — run BEFORE the audit).** Run `sungen depth-lint --screen <name>` (Bash). It reuses the audit's businessDepth classifier and splits every shallow business-critical scenario into two actionable buckets — act on them now so the audit/repair loop doesn't burn rounds on depth:
|
|
173
173
|
- **DEEPEN IN PLACE** — add a real value assertion to each (`User see all [X] contain {{v}}`, `User remember [X] as {{v}}` + `… with {{v}}`). The printed `template` is a **hint** keyed off the theme — apply judgment to the scenario's actual claim; do NOT paste a value assertion that doesn't fit (e.g. a carousel-visibility scenario should assert the product SET, not a price). If a flagged scenario is genuinely visibility/behavior (not data-correctness), that's an over-count — leave it and note it, never fake an assertion.
|
|
@@ -196,7 +196,7 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
196
196
|
4. Follow the `sungen-tc-generation` skill for section identification, viewpoint generation, and output format. **For flows**, use the "Flow Test Generation" section in the skill. When requirements exist, use the "Requirements-Driven Generation" strategy. **For Tier 1**, apply the **Lightweight Guard** — verify required fields, validation rules, business rules, security checks, and key state transitions all have TCs after generation. **For Tier 2+**, **MUST** apply the full **Mapping Contract** — walk every `spec.md` section top-to-bottom and produce the indicated TCs per Table 1; handle `test-viewpoint.md` per Table 2. Do not silently skip sections. Present sections as a numbered list and let user pick.
|
|
197
197
|
5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills. Generate **group-by-group** (one viewpoint group at a time, tier-by-tier `Write`/`Edit` batches) to stay under the output-token cap. **For flows**: use `[Screen:Element]` namespace format, namespace test-data by phase, add `@flow` tag. **If a scenario needs `@query` DB verification**, check the datasource `engine` first — unsupported engine → follow `sungen-gherkin-syntax` § "Unsupported DB engine — fallback" instead of authoring `@query` steps.
|
|
198
198
|
> **No parallel fan-out here.** Copilot has no sub-agents, so generation is sequential (the Claude Code variant fans out one `sungen-generator` per viewpoint group and merges). Same output, no speedup.
|
|
199
|
-
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + real `constraints` + `errorMap`), then run `sungen data lint --screen ${input:name}` and `sungen data gen --screen ${input:name}` per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves the rest), so it is safe here. Weave the standardized boundary/invalid sets into `@cases` with `CHK-*` trace. Then `… data validate` (values correct) + `… data crosscheck` (testcase↔data agree); for a stateful precondition,
|
|
199
|
+
5.3. **Standardize field test-data — Data Factory (when the unit has input fields).** For each input field, author the field-map `qa/data-factory/<name>.fields.yaml` (field → catalog `type` + real `constraints` + `errorMap`), then run `sungen data lint --screen ${input:name}` and `sungen data gen --screen ${input:name}` per the **`sungen-data-factory`** skill. `data gen` **MERGES** into `test-data.yaml` (replaces only its field keys, preserves the rest), so it is safe here. Weave the standardized boundary/invalid sets into `@cases` with `CHK-*` trace. Then `… data validate` (values correct) + `… data crosscheck` (testcase↔data agree); for a stateful precondition, `… data state --scaffold <name>` to auto-draft → fill entities + confirm with QA → `… data state`, weave the `@manual:data-setup` `Background` (works spec+viewpoint only). **Same standard as `/sungen:create-data-test`, applied inline — do NOT run that command separately afterwards.** No input fields (navigation/list) → skip.
|
|
200
200
|
5.4. **Depth self-check (deterministic — BEFORE the audit).** Run `sungen depth-lint --screen ${input:name}`. It splits every shallow business-critical scenario into **DEEPEN IN PLACE** (add a real value assertion — the printed `template` is a theme-keyed hint, apply judgment to the actual claim; never fake one onto a visibility/behavior scenario) and **CROSS-SCREEN** (route to a flow / tag `@manual:Mx` + reason — removes it from the depth denominator honestly). Act on both, re-run until `deepen` is empty (or only honest over-counts remain), THEN gate. Lifts first-pass `businessDepth` mechanically instead of via 2–3 repair rounds.
|
|
201
201
|
5.5. **Quality gate & repair (harness — always run).** Per `sungen-harness-audit`: run `sungen audit --screen ${input:name}` (structural), THEN do an **independent semantic review inline** using the `sungen-reviewer` criteria (does each scenario's steps PROVE its title/viewpoint? observable Thens? business-critical assertion depth?). Merge both sets of issues; if gate FAILs / findings exist, repair (budget 3) and re-audit — GATE missing theme → generate it (cross-screen → **automate it in the flow** via `/sungen:add-flow`, NOT a full `@manual` screen duplicate — `sungen audit` flags an automatable `@manual` as `MANUAL-AUTOMATABLE`; reserve `@manual:Mx` for true judgment/missing-capability); DEPTH → add data assertions; BALANCE → add business-core first; TRACE → align VP ids. Never fake a pass.
|
|
202
202
|
5.6. **Record.** `sungen manifest --screen ${input:name}`. Ledger **each phase** (not just repair) — pick one `runId` at the start and pass it so `trace`/`ledger report` show THIS run, not a mix: `sungen ledger record --screen ${input:name} --run <runId> --step <discovery|viewpoint|gherkin|audit|repair:N> --ms <elapsed>`. On re-run, start with `sungen manifest --screen ${input:name} --diff` and only regenerate changed sections.
|
|
@@ -15,6 +15,17 @@ Turn a screen/flow's fields into **standardized test data** — not guessed valu
|
|
|
15
15
|
Data Factory is **bundled with core** (runtime-free) — no `capability add` needed; the `sungen data …`
|
|
16
16
|
CLI is available out of the box.
|
|
17
17
|
|
|
18
|
+
**Auto-detection (act on these, don't wait to be asked).** `sungen generate`/`sungen audit` now NOTICE
|
|
19
|
+
the work a unit needs, even before any field-map exists:
|
|
20
|
+
- **`DATA-FACTORY-UNSTANDARDIZED`** — the unit has input fields but no `qa/data-factory/<name>.fields.yaml`.
|
|
21
|
+
→ author the field-map + `data gen` (this skill's main job).
|
|
22
|
+
- **`DATA-FACTORY-PRECONDITION-UNDECLARED`** — the spec describes a stateful precondition (a required
|
|
23
|
+
entity/state) but the test declares none. → **auto-draft a state-recipe** from the spec sentence, run
|
|
24
|
+
`sungen data state`, and weave its `@manual:data-setup` `Background`; confirm the drafted values with
|
|
25
|
+
the QA (`AskUserQuestion`) — never invent domain values.
|
|
26
|
+
When you see either finding, do the work rather than leaving it — that is what makes Sungen self-driving
|
|
27
|
+
instead of a toolbox the QA must operate.
|
|
28
|
+
|
|
18
29
|
## The 4-source method (phuong-phap-tao-test-data.md)
|
|
19
30
|
|
|
20
31
|
Never invent values from memory. Cross-reference four sources — each covers the previous one's gap:
|
|
@@ -139,6 +150,12 @@ Field values are not enough when a test needs a **business state first** — "a
|
|
|
139
150
|
a **`provenance`** (`prior-flow` | `fixture` | `client-storage` | `server-context` | `manual` |
|
|
140
151
|
`api` | `db`), optional `dependsOn`, `desiredState`, `lifecycle` (`reusable`/`consumable`/`expiring`/
|
|
141
152
|
`leased`/`mutable-state`/`unique-per-run`), and `with:` values — plus `bindings:` (`orderId: ${order.id}`).
|
|
153
|
+
**Auto-draft it — don't make the QA hand-write it.** When the spec describes a precondition (or you saw
|
|
154
|
+
a `PRECONDITION-UNDECLARED` finding), run **`sungen data state --scaffold <unit>`**: it drafts a recipe
|
|
155
|
+
skeleton from the spec's precondition sentences (one resource per signal, provenance guessed, entity
|
|
156
|
+
left as `TODO`). Then **fill each `entity` + `desiredState`/`bindings` from the spec and confirm the
|
|
157
|
+
domain specifics with the QA (`AskUserQuestion`)** — never invent domain values.
|
|
158
|
+
|
|
142
159
|
Run **`sungen data state`**: it validates the graph (cycles, dangling deps, bindings) and renders a
|
|
143
160
|
standardized **manual precondition** block — tagged **`@manual:data-setup`** when it can't be
|
|
144
161
|
auto-provisioned — to weave into the scenario's `Background`. **This works with only spec + viewpoint:
|
|
@@ -147,6 +164,16 @@ a resource uses `api`/`db` provenance (+ a `provider:` endpoint), the *same* rec
|
|
|
147
164
|
rewrite. For a **journey where the same data flows across screens** (setup → confirmation → complete),
|
|
148
165
|
add `shared: { entity, screens: [...] }` so one dataset is used across all of them.
|
|
149
166
|
|
|
167
|
+
## One QA-facing door: `/sungen:create-test`
|
|
168
|
+
|
|
169
|
+
QA never needs to remember which `sungen data …` sub-command to run. **`/sungen:create-test` is the
|
|
170
|
+
single entry** — it standardizes fields, validates, cross-checks, and drafts any stateful precondition
|
|
171
|
+
inline. The `sungen data gen|lint|validate|crosscheck|state|provenance|fixture` commands are the
|
|
172
|
+
**deterministic plumbing** the create-test loop (and CI) drives; run them directly only for a targeted
|
|
173
|
+
re-standardize. `/sungen:create-data-test` is the same pipeline on demand (re-standardize after editing
|
|
174
|
+
a field-map/`.overwrite`, or a unit authored before Data Factory) — not a step you must invoke after
|
|
175
|
+
create-test.
|
|
176
|
+
|
|
150
177
|
## Cross-artifact check — testcase ↔ data agree
|
|
151
178
|
|
|
152
179
|
`data validate` checks the data alone; **`sungen data crosscheck`** checks the testcase and its data
|