@sun-asterisk/sungen 3.2.12-beta.1 → 3.2.12-beta.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/dashboard/templates/index.html +1 -1
- package/dist/exporters/csv-exporter.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.js +7 -2
- package/dist/exporters/csv-exporter.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +15 -2
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/json-exporter.d.ts.map +1 -1
- package/dist/exporters/json-exporter.js +10 -5
- package/dist/exporters/json-exporter.js.map +1 -1
- package/dist/exporters/result-variants.d.ts +23 -5
- package/dist/exporters/result-variants.d.ts.map +1 -1
- package/dist/exporters/result-variants.js +65 -9
- package/dist/exporters/result-variants.js.map +1 -1
- package/dist/exporters/test-data-resolver.d.ts.map +1 -1
- package/dist/exporters/test-data-resolver.js.map +1 -1
- package/dist/orchestrator/templates/ai-src/commands/create-data-test.md +13 -1
- package/dist/orchestrator/templates/ai-src/commands/create-test.md +2 -2
- package/dist/orchestrator/templates/ai-src/skills/sungen-data-factory/SKILL.md +105 -8
- package/package.json +3 -3
- package/src/dashboard/templates/index.html +1 -1
- package/src/exporters/csv-exporter.ts +7 -3
- package/src/exporters/feature-parser.ts +15 -2
- package/src/exporters/json-exporter.ts +10 -6
- package/src/exporters/result-variants.ts +68 -9
- package/src/exporters/test-data-resolver.ts +1 -0
- package/src/orchestrator/templates/ai-src/commands/create-data-test.md +13 -1
- package/src/orchestrator/templates/ai-src/commands/create-test.md +2 -2
- package/src/orchestrator/templates/ai-src/skills/sungen-data-factory/SKILL.md +105 -8
|
@@ -15,6 +15,17 @@ Turn a screen/flow's fields into **standardized test data** — not guessed valu
|
|
|
15
15
|
Data Factory is **bundled with core** (runtime-free) — no `capability add` needed; the `sungen data …`
|
|
16
16
|
CLI is available out of the box.
|
|
17
17
|
|
|
18
|
+
**Auto-detection (act on these, don't wait to be asked).** `sungen generate`/`sungen audit` now NOTICE
|
|
19
|
+
the work a unit needs, even before any field-map exists:
|
|
20
|
+
- **`DATA-FACTORY-UNSTANDARDIZED`** — the unit has input fields but no `qa/data-factory/<name>.fields.yaml`.
|
|
21
|
+
→ author the field-map + `data gen` (this skill's main job).
|
|
22
|
+
- **`DATA-FACTORY-PRECONDITION-UNDECLARED`** — the spec describes a stateful precondition (a required
|
|
23
|
+
entity/state) but the test declares none. → **auto-draft a state-recipe** from the spec sentence, run
|
|
24
|
+
`sungen data state`, and weave its `@manual:data-setup` `Background`; confirm the drafted values with
|
|
25
|
+
the QA (`AskUserQuestion`) — never invent domain values.
|
|
26
|
+
When you see either finding, do the work rather than leaving it — that is what makes Sungen self-driving
|
|
27
|
+
instead of a toolbox the QA must operate.
|
|
28
|
+
|
|
18
29
|
## The 4-source method (phuong-phap-tao-test-data.md)
|
|
19
30
|
|
|
20
31
|
Never invent values from memory. Cross-reference four sources — each covers the previous one's gap:
|
|
@@ -85,20 +96,26 @@ This is the core discipline: *common is never enough; when blind, think and ask
|
|
|
85
96
|
## Customizing the catalog — `.overwrite` (survives `sungen update`)
|
|
86
97
|
|
|
87
98
|
The shipped `common/` is a floor. Project rules go in `qa/data-factory/common.overwrite.yaml`,
|
|
88
|
-
deep-merged over the common (**later-wins**, keyed by `type` + data-point `id`)
|
|
89
|
-
new `id` appends,
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
`
|
|
93
|
-
|
|
99
|
+
deep-merged over the common (**later-wins**, keyed by `type` + data-point `id`) via three explicit
|
|
100
|
+
operations: **EXTEND** (a new `id` appends), **REPLACE** (same `id` wins; `constraints` merge
|
|
101
|
+
key-by-key), **DISABLE** (list ids under `disable:` to drop shipped cases that don't apply — a stale
|
|
102
|
+
target id warns). You can also override `security:` banks and `techniques:` the same way. `sungen
|
|
103
|
+
update` refreshes `common/` but never touches your overlay; a stale entry **warns you to prune it** —
|
|
104
|
+
it never edits your file. Nothing is silently overwritten: **`sungen data provenance`** (and a
|
|
105
|
+
`⟲ .overwrite →` line at `data gen`) shows exactly what your overlay changed, and each overlaid value
|
|
106
|
+
is tagged `via: .overwrite` in the test-data. Prefer a targeted `errorMap`/`constraints` in the
|
|
107
|
+
field-map for per-screen tweaks; use `.overwrite` only for rules that apply project-wide.
|
|
94
108
|
|
|
95
109
|
## Bulk-import fixtures (CSV)
|
|
96
110
|
|
|
97
111
|
For CSV/Excel import or bulk-upload testing, `sungen data fixture --screen <name> [--rows N] [--invalid]`
|
|
98
112
|
turns the field-map into a fixture FILE under `qa/fixtures/`: `<name>.csv` (N valid rows; nested groups
|
|
99
113
|
→ dotted headers; `unique` fields row-indexed) that should import cleanly, and — with `--invalid` —
|
|
100
|
-
`<name>.invalid.csv` (one seeded-bad row per field
|
|
101
|
-
|
|
114
|
+
`<name>.invalid.csv` (one seeded-bad row per field, **same columns as the valid file so a strict
|
|
115
|
+
importer fails on the data, not an extra column**) plus a **sidecar** `<name>.invalid.expected.yaml`
|
|
116
|
+
that says which 0-based row must be rejected and why (`field`, `chk`, `reason`, `expected_error`).
|
|
117
|
+
Reference the CSV from an `@api` upload (`files:`) or a UI file input, and assert row rejection against
|
|
118
|
+
the sidecar.
|
|
102
119
|
|
|
103
120
|
## Verify before done — the phuong-phap §6 checklist
|
|
104
121
|
|
|
@@ -109,6 +126,86 @@ Then `sungen data gen` writes the standardized `test-data/<name>.yaml`. Do not h
|
|
|
109
126
|
generated values — change the field-map or `.overwrite` and re-run (determinism: same input → same
|
|
110
127
|
data).
|
|
111
128
|
|
|
129
|
+
Then run **`sungen data validate`** — the semantic check beyond coverage. It verifies each generated
|
|
130
|
+
value is *correct*, not just *present*: every `valid`/`boundary` value actually satisfies the
|
|
131
|
+
constraints and every synthesized `invalid` value actually violates them. A boundary is expanded to
|
|
132
|
+
**one case per point** (`min / min+1` → two cases; `< min` → min-1, below the minimum), each stamped
|
|
133
|
+
`expected: valid|invalid` in the output. Any `VALUE_NOT_VALID` error means a value labeled valid breaks
|
|
134
|
+
a constraint — fix the field-map/constraints and re-run. This is what stops the "structurally right but
|
|
135
|
+
semantically wrong" data (e.g. a "below minimum length" case that is actually long enough to pass).
|
|
136
|
+
|
|
137
|
+
**Applicability & profiles.** The generator only emits cases that apply, and reports the rest (never
|
|
138
|
+
silent). Set **`required: true|false`** correctly — an optional field drops the "empty (required)" case;
|
|
139
|
+
a numeric field with `min ≥ 0` drops the negative example. Security/injection payloads land in their own
|
|
140
|
+
**`adversarial:`** bucket, separate from `invalid:`. Dropped cases show up as `_not_applicable` in the
|
|
141
|
+
output + a `NOT_APPLICABLE` lint info. A `--profile` (`regression` default · `functional` · `smoke` ·
|
|
142
|
+
`security-min`) and `--locale`/`--channel`/`--sink` on `data gen|validate|lint` select a reduced set;
|
|
143
|
+
selection is deterministic. Prefer setting `required` in the field-map over disabling the empty case.
|
|
144
|
+
|
|
145
|
+
## Domain / stateful preconditions (spec + viewpoint only — no DB/API)
|
|
146
|
+
|
|
147
|
+
Field values are not enough when a test needs a **business state first** — "a paid order of user A",
|
|
148
|
+
"a valid unused reset token ≤24h", "the session-storage buffer holds the entered values". Declare a
|
|
149
|
+
**state-recipe** in `qa/data-factory/recipes.yaml` under `resources:` — each resource has an `entity`,
|
|
150
|
+
a **`provenance`** (`prior-flow` | `fixture` | `client-storage` | `server-context` | `manual` |
|
|
151
|
+
`api` | `db`), optional `dependsOn`, `desiredState`, `lifecycle` (`reusable`/`consumable`/`expiring`/
|
|
152
|
+
`leased`/`mutable-state`/`unique-per-run`), and `with:` values — plus `bindings:` (`orderId: ${order.id}`).
|
|
153
|
+
**Auto-draft it — don't make the QA hand-write it.** When the spec describes a precondition (or you saw
|
|
154
|
+
a `PRECONDITION-UNDECLARED` finding), run **`sungen data state --scaffold <unit>`**: it drafts a recipe
|
|
155
|
+
skeleton from the spec's precondition sentences (one resource per signal, provenance guessed, entity
|
|
156
|
+
left as `TODO`). Then **fill each `entity` + `desiredState`/`bindings` from the spec and confirm the
|
|
157
|
+
domain specifics with the QA (`AskUserQuestion`)** — never invent domain values.
|
|
158
|
+
|
|
159
|
+
Run **`sungen data state`**: it validates the graph (cycles, dangling deps, bindings) and renders a
|
|
160
|
+
standardized **manual precondition** block — tagged **`@manual:data-setup`** when it can't be
|
|
161
|
+
auto-provisioned — to weave into the scenario's `Background`. **This works with only spec + viewpoint:
|
|
162
|
+
provenance is usually `prior-flow`/`fixture`/`manual`, NOT DB/API.** If a datasource is later added and
|
|
163
|
+
a resource uses `api`/`db` provenance (+ a `provider:` endpoint), the *same* recipe auto-provisions — no
|
|
164
|
+
rewrite. For a **journey where the same data flows across screens** (setup → confirmation → complete),
|
|
165
|
+
add `shared: { entity, screens: [...] }` so one dataset is used across all of them.
|
|
166
|
+
|
|
167
|
+
## One QA-facing door: `/sungen:create-test`
|
|
168
|
+
|
|
169
|
+
QA never needs to remember which `sungen data …` sub-command to run. **`/sungen:create-test` is the
|
|
170
|
+
single entry** — it standardizes fields, validates, cross-checks, and drafts any stateful precondition
|
|
171
|
+
inline. The `sungen data gen|lint|validate|crosscheck|state|provenance|fixture` commands are the
|
|
172
|
+
**deterministic plumbing** the create-test loop (and CI) drives; run them directly only for a targeted
|
|
173
|
+
re-standardize. `/sungen:create-data-test` is the same pipeline on demand (re-standardize after editing
|
|
174
|
+
a field-map/`.overwrite`, or a unit authored before Data Factory) — not a step you must invoke after
|
|
175
|
+
create-test.
|
|
176
|
+
|
|
177
|
+
## Cross-artifact check — testcase ↔ data agree
|
|
178
|
+
|
|
179
|
+
`data validate` checks the data alone; **`sungen data crosscheck`** checks the testcase and its data
|
|
180
|
+
AGREE: no dangling `{{var}}` (every referenced var has a value), a cross-screen `shared` dataset is
|
|
181
|
+
identical across the journey (confirmation shows what setup entered), and a declared state-precondition
|
|
182
|
+
is surfaced in the feature. Run it after the `.feature` exists; fix any `error`. Together
|
|
183
|
+
`data validate` + `data crosscheck` are the **Data Gate + Cross-artifact Gate** (the `data-only`
|
|
184
|
+
harness profile). Every `data gen` also stamps a `_fingerprint` (catalog+generator+context hash) so a
|
|
185
|
+
reader/cache can tell whether the inputs that produced the data changed.
|
|
186
|
+
|
|
187
|
+
## Boundary completeness + trim
|
|
188
|
+
|
|
189
|
+
Boundary Value Analysis is **complete by construction**: for every bounded dimension in the field-map's
|
|
190
|
+
`constraints` — including ones the field-type's catalog didn't model (e.g. an **email `minLength`**) —
|
|
191
|
+
the generator auto-emits the full set **min-1 (invalid) · min · min+1 · max-1 · max · max+1 (invalid)**.
|
|
192
|
+
So a spec "email min 5" yields length 4 (invalid), 5, 6 — don't hand-add them. Text fields also carry a
|
|
193
|
+
**leading/trailing-whitespace** case (trim behaviour: the server must trim then accept, or reject per
|
|
194
|
+
spec — confirm). If `data validate`/the QA still finds a missing boundary, it means the constraint was
|
|
195
|
+
absent from the field-map — add it there.
|
|
196
|
+
|
|
197
|
+
## Login / authentication screens — credential-state cases (experience)
|
|
198
|
+
|
|
199
|
+
A login screen is not just field-format validation. The catalog carries an **experience checklist**
|
|
200
|
+
(`auth-account-states`) of credential/account STATES a login must cover — each a scenario, most needing
|
|
201
|
+
a domain-state precondition (a seeded account in that state; declare it with a state-recipe →
|
|
202
|
+
`@manual:data-setup` Background):
|
|
203
|
+
- **active** (happy path) · **wrong-password** · **non-existent** (same generic error — anti-enumeration)
|
|
204
|
+
- **soft-deleted** (must NOT authenticate — real-world bug precedent) · **locked/suspended** ·
|
|
205
|
+
**unverified-email** · **password-expired** · **valid-with-surrounding-whitespace** (trim → accept).
|
|
206
|
+
When the unit is a login/auth screen, generate these as scenarios (confirm the exact outcome/message
|
|
207
|
+
code with the spec). Do not assume the outcome — ask the QA for states the spec doesn't pin.
|
|
208
|
+
|
|
112
209
|
## Weaving into scenarios
|
|
113
210
|
|
|
114
211
|
When a `.feature` exists, express invalid/boundary sets as data-driven `Scenario Outline` + `Examples`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sun-asterisk/sungen",
|
|
3
|
-
"version": "3.2.12-beta.
|
|
3
|
+
"version": "3.2.12-beta.10",
|
|
4
4
|
"description": "Deterministic E2E Test Compiler - Gherkin + Selectors → Playwright tests",
|
|
5
5
|
"main": "src/index.ts",
|
|
6
6
|
"types": "src/index.ts",
|
|
@@ -39,8 +39,8 @@
|
|
|
39
39
|
"@babel/types": "^7.28.5",
|
|
40
40
|
"@cucumber/gherkin": "^37.0.0",
|
|
41
41
|
"@cucumber/messages": "^31.0.0",
|
|
42
|
-
"@sungen/driver-data-factory": "3.2.12-beta.
|
|
43
|
-
"@sungen/driver-ui": "3.2.12-beta.
|
|
42
|
+
"@sungen/driver-data-factory": "3.2.12-beta.10",
|
|
43
|
+
"@sungen/driver-ui": "3.2.12-beta.10",
|
|
44
44
|
"chalk": "^5.6.2",
|
|
45
45
|
"commander": "^14.0.2",
|
|
46
46
|
"dotenv": "^17.2.3",
|