@sun-asterisk/sungen 3.1.2-beta.118 → 3.1.2-beta.120
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/commands/delivery.d.ts.map +1 -1
- package/dist/cli/commands/delivery.js +33 -33
- package/dist/cli/commands/delivery.js.map +1 -1
- package/dist/cli/commands/script-check.d.ts.map +1 -1
- package/dist/cli/commands/script-check.js +10 -8
- package/dist/cli/commands/script-check.js.map +1 -1
- package/dist/cli/commands/trace.d.ts.map +1 -1
- package/dist/cli/commands/trace.js +6 -4
- package/dist/cli/commands/trace.js.map +1 -1
- package/dist/harness/script-check.d.ts +3 -1
- package/dist/harness/script-check.d.ts.map +1 -1
- package/dist/harness/script-check.js +16 -7
- package/dist/harness/script-check.js.map +1 -1
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +2 -0
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-agent-reviewer.md +6 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +5 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +16 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-api-design.md +49 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +5 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +10 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-api-design.md +49 -0
- package/package.json +2 -2
- package/src/cli/commands/delivery.ts +32 -33
- package/src/cli/commands/script-check.ts +9 -7
- package/src/cli/commands/trace.ts +6 -4
- package/src/harness/script-check.ts +20 -7
- package/src/orchestrator/ai-rules-updater.ts +2 -0
- package/src/orchestrator/templates/ai-instructions/claude-agent-reviewer.md +6 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +5 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +16 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-api-design.md +49 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +5 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +10 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-api-design.md +49 -0
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sungen-api-design
|
|
3
|
+
description: The API-first design loop for an api unit (qa/api/<area> or qa/api/flows/<flow>) — discover the catalog, lay out the API viewpoints, generate @api/@cases/flow/@concurrent scenarios, then drive the sungen audit --api gate + reviewer + repair to a high businessDepth (≥0.7). Use when create-test/run-test detects an api unit (no selectors, no visual capture).
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# API design loop (driver-api · Orchestration + Harness)
|
|
7
|
+
|
|
8
|
+
Use this when the unit is **api-first** — `qa/api/<area>/` or `qa/api/flows/<flow>/`. There are **no selectors and no visual capture**: the contract is the **named-endpoint catalog** (`api/apis.yaml`), referenced by `@api:<name>`. QA writes **no HTTP code**. Full annotation reference: the **API Steps** guide (`@api` / `@cases` / flows / `@concurrent` / `@hybrid`).
|
|
9
|
+
|
|
10
|
+
## The loop (mirror of /sungen:design, API-native)
|
|
11
|
+
|
|
12
|
+
### 1. Discover (no capture)
|
|
13
|
+
Run `sungen context --api <name>` — it reads the catalog and prints the **endpoints** + the **generation units** (one `matrix` unit per endpoint, an `async` unit per mutating endpoint, a `flow` unit for an api flow). Read `qa/api/<name>/requirements/spec.md` if present. No `apis.yaml` yet? → `sungen api import <openapi|csv>` or `sungen api add --area <name>` first.
|
|
14
|
+
|
|
15
|
+
### 2. API viewpoint overview (by method-profile)
|
|
16
|
+
For each endpoint, cover its viewpoints — severity-weighted by method:
|
|
17
|
+
|
|
18
|
+
| Profile | Endpoints | Must cover | Then |
|
|
19
|
+
|---|---|---|---|
|
|
20
|
+
| read | GET, HEAD | `contract` (status + body shape) | `pagination`/`filter` (list), `not-found` (by-id) |
|
|
21
|
+
| mutating | POST/PUT/PATCH/DELETE | `contract`, `error` (validation/4xx/auth) | `idempotency` (`@concurrent`), `side-effect` (`@query`) |
|
|
22
|
+
|
|
23
|
+
Bands: **~70%** success+failure matrix · **~20%** flows (auth/CRUD chains) · **~10%** async/idempotency.
|
|
24
|
+
|
|
25
|
+
### 3. Generate (incremental — never the whole suite in one Write)
|
|
26
|
+
- **Contract**: `@api:<name>` + `expect {{name.status}} is …` **and a body assertion** (`{{name.body.<path>}}`).
|
|
27
|
+
- **Error matrix**: `@api:<name>(p={{p}}) @cases:<dataset>` — one scenario, a dataset of `input → expected status`.
|
|
28
|
+
- **Flow**: ordered `@api` tags threading a prior response (`token={{login.body.token}}` → the catalog `Bearer :token` header; `id={{create.body.id}}` → a path param). Self-clean (delete what you create).
|
|
29
|
+
- **Idempotency**: `@api:<name> @concurrent:N` + `expect {{name.ok_count}} is 1`, cross-checked with `@query` (the DB is the oracle).
|
|
30
|
+
|
|
31
|
+
### 4. Gate + repair (always — businessDepth ≥ 0.7 is the bar)
|
|
32
|
+
Run `sungen audit --api <name>`; read `gateStatus` + `findings`. Then the **semantic reviewer** (sungen-reviewer sub-agent, API criteria). Repair **both** (budget 3 rounds), re-audit until PASS:
|
|
33
|
+
|
|
34
|
+
| Finding | Repair |
|
|
35
|
+
|---|---|
|
|
36
|
+
| `VIEWPOINT-API-CONTRACT` | the endpoint is invoked but its response is never asserted → add `expect {{name.status}}` + a `{{name.body.…}}` check |
|
|
37
|
+
| `VIEWPOINT-API-ERROR` | a mutating endpoint has no failure scenario → add a `@cases` error matrix (or an explicit 4xx) |
|
|
38
|
+
| `VIEWPOINT-API-IDEMPOTENCY` | a mutating endpoint has no race check → add `@concurrent:N` + a `@query` DB cross-check |
|
|
39
|
+
| **`DEPTH-FAIL`** (businessDepth < 0.7) | a **mutating success** scenario asserts only `status` → make it **prove the effect**: assert a response **body** field, a **`@query`** side-effect, or a **`@concurrent` `ok_count`** invariant. (An error/`@cases` scenario proving the status is correct — it is *not* depth-required.) |
|
|
40
|
+
|
|
41
|
+
Stop when the gate PASSes + businessDepth ≥ 0.7, or the budget is exhausted → report residual gaps honestly (mark genuinely-unautomatable cases `@manual` with an oracle). Never fake a pass.
|
|
42
|
+
|
|
43
|
+
### 5. Record + converge
|
|
44
|
+
`sungen manifest --api <name>` (reuse) and ledger each phase; show the trace + the HUMAN-LOOP FOCUS. (Integrity `script-check`/`trace` for api: see run-test.)
|
|
45
|
+
|
|
46
|
+
## Rules
|
|
47
|
+
- **No HTTP, no selectors** — only `.feature` + the reviewed `apis.yaml` + `test-data`.
|
|
48
|
+
- **Non-prod default** — a `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
49
|
+
- **The DB is the oracle** for idempotency/side-effects — HTTP status alone can lie; pair `@api` with `@query`.
|
|
@@ -18,7 +18,11 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
|
|
|
18
18
|
|
|
19
19
|
- **name** — ${input:name:screen or flow name (e.g., login, award-submission)}
|
|
20
20
|
|
|
21
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists → flow mode (base path: `qa/flows/<name>/`). Else
|
|
21
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode (base path: `qa/flows/<name>/`). Else `qa/screens/<name>/` → screen mode (base path: `qa/screens/<name>/`).
|
|
22
|
+
|
|
23
|
+
## API unit mode (driver-api)
|
|
24
|
+
|
|
25
|
+
If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the design loop differs — **no visual capture, no selectors**; the contract is the named-endpoint catalog. **Follow the `sungen-api-design` skill end-to-end** instead of the screen/flow steps: `sungen context --api <name>` (discover) → API viewpoint overview → generate `@api`/`@cases`/flow/`@concurrent`/`@query` scenarios → **`sungen audit --api <name>` gate + reviewer + repair loop to businessDepth ≥ 0.7** → record + trace. Then recommend `/sungen-run-test <name>`. The capture / viewpoint-group / selector steps do **not** apply.
|
|
22
26
|
|
|
23
27
|
## Steps
|
|
24
28
|
|
|
@@ -30,7 +30,16 @@ Count 0 → offer the user:
|
|
|
30
30
|
|
|
31
31
|
Skip when `--env` matches the base locale.
|
|
32
32
|
|
|
33
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists → flow mode (base path: `qa/flows/<name>/`). Else
|
|
33
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode (base path: `qa/flows/<name>/`). Else `qa/screens/<name>/` → screen mode (base path: `qa/screens/<name>/`).
|
|
34
|
+
|
|
35
|
+
## API unit mode (driver-api) — no selectors
|
|
36
|
+
|
|
37
|
+
If the unit is **api-first**, skip every selector/capture phase (an API test has no DOM):
|
|
38
|
+
1. **Resolve the datasource** — `base_url` + auth wired in `qa/datasources.yaml` + `.env.qa` (`${X_URL}` from `sungen api init`); a `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
39
|
+
2. **Compile**: `npx sungen generate --api <name>` → `specs/generated/api/<name>/`.
|
|
40
|
+
3. **Run**: `npx playwright test specs/generated/api/<name>/<name>.spec.ts`.
|
|
41
|
+
4. **Auto-fix** (use `sungen-error-mapping`): 401/403 → `@hybrid`+`@auth` or `Bearer :token` header (`sungen makeauth`); base_url unresolved → set `${X_URL}`; missing param → trace `{{var}}` to test-data/a prior `@api` response; `expect.status` mismatch → reconcile against `apis.yaml` (re-`generate --api`, never hand-edit the spec); flaky → self-clean + `@concurrent` caps.
|
|
42
|
+
5. **Integrity + trace** — `sungen script-check --api <name>` (1:1; on DRIFT re-`generate --api`, never hand-edit the spec) + `sungen trace --api <name>` (process map + HUMAN-LOOP FOCUS). Report + offer next steps.
|
|
34
43
|
|
|
35
44
|
## Pre-run (phased — per `sungen-selector-fix` skill)
|
|
36
45
|
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sungen-api-design
|
|
3
|
+
description: The API-first design loop for an api unit (qa/api/<area> or qa/api/flows/<flow>) — discover the catalog, lay out the API viewpoints, generate @api/@cases/flow/@concurrent scenarios, then drive the sungen audit --api gate + reviewer + repair to a high businessDepth (≥0.7). Use when create-test/run-test detects an api unit (no selectors, no visual capture).
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# API design loop (driver-api · Orchestration + Harness)
|
|
7
|
+
|
|
8
|
+
Use this when the unit is **api-first** — `qa/api/<area>/` or `qa/api/flows/<flow>/`. There are **no selectors and no visual capture**: the contract is the **named-endpoint catalog** (`api/apis.yaml`), referenced by `@api:<name>`. QA writes **no HTTP code**. Full annotation reference: the **API Steps** guide (`@api` / `@cases` / flows / `@concurrent` / `@hybrid`).
|
|
9
|
+
|
|
10
|
+
## The loop (mirror of /sungen:design, API-native)
|
|
11
|
+
|
|
12
|
+
### 1. Discover (no capture)
|
|
13
|
+
Run `sungen context --api <name>` — it reads the catalog and prints the **endpoints** + the **generation units** (one `matrix` unit per endpoint, an `async` unit per mutating endpoint, a `flow` unit for an api flow). Read `qa/api/<name>/requirements/spec.md` if present. No `apis.yaml` yet? → `sungen api import <openapi|csv>` or `sungen api add --area <name>` first.
|
|
14
|
+
|
|
15
|
+
### 2. API viewpoint overview (by method-profile)
|
|
16
|
+
For each endpoint, cover its viewpoints — severity-weighted by method:
|
|
17
|
+
|
|
18
|
+
| Profile | Endpoints | Must cover | Then |
|
|
19
|
+
|---|---|---|---|
|
|
20
|
+
| read | GET, HEAD | `contract` (status + body shape) | `pagination`/`filter` (list), `not-found` (by-id) |
|
|
21
|
+
| mutating | POST/PUT/PATCH/DELETE | `contract`, `error` (validation/4xx/auth) | `idempotency` (`@concurrent`), `side-effect` (`@query`) |
|
|
22
|
+
|
|
23
|
+
Bands: **~70%** success+failure matrix · **~20%** flows (auth/CRUD chains) · **~10%** async/idempotency.
|
|
24
|
+
|
|
25
|
+
### 3. Generate (incremental — never the whole suite in one Write)
|
|
26
|
+
- **Contract**: `@api:<name>` + `expect {{name.status}} is …` **and a body assertion** (`{{name.body.<path>}}`).
|
|
27
|
+
- **Error matrix**: `@api:<name>(p={{p}}) @cases:<dataset>` — one scenario, a dataset of `input → expected status`.
|
|
28
|
+
- **Flow**: ordered `@api` tags threading a prior response (`token={{login.body.token}}` → the catalog `Bearer :token` header; `id={{create.body.id}}` → a path param). Self-clean (delete what you create).
|
|
29
|
+
- **Idempotency**: `@api:<name> @concurrent:N` + `expect {{name.ok_count}} is 1`, cross-checked with `@query` (the DB is the oracle).
|
|
30
|
+
|
|
31
|
+
### 4. Gate + repair (always — businessDepth ≥ 0.7 is the bar)
|
|
32
|
+
Run `sungen audit --api <name>`; read `gateStatus` + `findings`. Then the **semantic reviewer** (sungen-reviewer sub-agent, API criteria). Repair **both** (budget 3 rounds), re-audit until PASS:
|
|
33
|
+
|
|
34
|
+
| Finding | Repair |
|
|
35
|
+
|---|---|
|
|
36
|
+
| `VIEWPOINT-API-CONTRACT` | the endpoint is invoked but its response is never asserted → add `expect {{name.status}}` + a `{{name.body.…}}` check |
|
|
37
|
+
| `VIEWPOINT-API-ERROR` | a mutating endpoint has no failure scenario → add a `@cases` error matrix (or an explicit 4xx) |
|
|
38
|
+
| `VIEWPOINT-API-IDEMPOTENCY` | a mutating endpoint has no race check → add `@concurrent:N` + a `@query` DB cross-check |
|
|
39
|
+
| **`DEPTH-FAIL`** (businessDepth < 0.7) | a **mutating success** scenario asserts only `status` → make it **prove the effect**: assert a response **body** field, a **`@query`** side-effect, or a **`@concurrent` `ok_count`** invariant. (An error/`@cases` scenario proving the status is correct — it is *not* depth-required.) |
|
|
40
|
+
|
|
41
|
+
Stop when the gate PASSes + businessDepth ≥ 0.7, or the budget is exhausted → report residual gaps honestly (mark genuinely-unautomatable cases `@manual` with an oracle). Never fake a pass.
|
|
42
|
+
|
|
43
|
+
### 5. Record + converge
|
|
44
|
+
`sungen manifest --api <name>` (reuse) and ledger each phase; show the trace + the HUMAN-LOOP FOCUS. (Integrity `script-check`/`trace` for api: see run-test.)
|
|
45
|
+
|
|
46
|
+
## Rules
|
|
47
|
+
- **No HTTP, no selectors** — only `.feature` + the reviewed `apis.yaml` + `test-data`.
|
|
48
|
+
- **Non-prod default** — a `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
49
|
+
- **The DB is the oracle** for idempotency/side-effects — HTTP status alone can lie; pair `@api` with `@query`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sun-asterisk/sungen",
|
|
3
|
-
"version": "3.1.2-beta.
|
|
3
|
+
"version": "3.1.2-beta.120",
|
|
4
4
|
"description": "Deterministic E2E Test Compiler - Gherkin + Selectors → Playwright tests",
|
|
5
5
|
"main": "src/index.ts",
|
|
6
6
|
"types": "src/index.ts",
|
|
@@ -33,7 +33,7 @@
|
|
|
33
33
|
"node": ">=18.0.0"
|
|
34
34
|
},
|
|
35
35
|
"dependencies": {
|
|
36
|
-
"@sungen/driver-ui": "3.1.2-beta.
|
|
36
|
+
"@sungen/driver-ui": "3.1.2-beta.120",
|
|
37
37
|
"@anthropic-ai/sdk": "^0.71.0",
|
|
38
38
|
"@babel/parser": "^7.28.5",
|
|
39
39
|
"@babel/traverse": "^7.28.5",
|
|
@@ -52,47 +52,49 @@ function log(msg: string): void {
|
|
|
52
52
|
* `name` is kept as an alias of `featureBaseName` so existing callers/labels
|
|
53
53
|
* (preflight table, summary) read naturally — every visible row is per-feature.
|
|
54
54
|
*/
|
|
55
|
+
type UnitKind = 'screen' | 'flow' | 'api';
|
|
56
|
+
/** qa/ subfolder for a unit kind. */
|
|
57
|
+
const qaParent = (kind: UnitKind): string => (kind === 'flow' ? 'flows' : kind === 'api' ? 'api' : 'screens');
|
|
58
|
+
|
|
55
59
|
interface DeliveryTarget {
|
|
56
60
|
screen: string;
|
|
57
61
|
featureBaseName: string;
|
|
58
62
|
/** Alias of `featureBaseName` — preserves the old `target.name` call sites. */
|
|
59
63
|
name: string;
|
|
64
|
+
kind: UnitKind;
|
|
65
|
+
/** Back-compat: flows kept distinct labels/paths before api was added. */
|
|
60
66
|
isFlow: boolean;
|
|
61
67
|
}
|
|
62
68
|
|
|
63
|
-
function makeTarget(screen: string, featureBaseName: string,
|
|
64
|
-
return { screen, featureBaseName, name: featureBaseName, isFlow };
|
|
69
|
+
function makeTarget(screen: string, featureBaseName: string, kind: UnitKind): DeliveryTarget {
|
|
70
|
+
return { screen, featureBaseName, name: featureBaseName, kind, isFlow: kind === 'flow' };
|
|
65
71
|
}
|
|
66
72
|
|
|
67
73
|
/**
|
|
68
|
-
* List all `.feature` files inside a screen/flow as separate targets.
|
|
74
|
+
* List all `.feature` files inside a screen/flow/api unit as separate targets.
|
|
69
75
|
* Returns empty array when the directory has no features yet.
|
|
70
76
|
*/
|
|
71
|
-
function listFeatureTargets(cwd: string, screen: string,
|
|
72
|
-
const featuresDir = path.join(cwd, 'qa',
|
|
77
|
+
function listFeatureTargets(cwd: string, screen: string, kind: UnitKind): DeliveryTarget[] {
|
|
78
|
+
const featuresDir = path.join(cwd, 'qa', qaParent(kind), screen, 'features');
|
|
73
79
|
if (!fs.existsSync(featuresDir)) return [];
|
|
74
80
|
return fs.readdirSync(featuresDir)
|
|
75
81
|
.filter((f) => f.endsWith('.feature'))
|
|
76
|
-
.map((f) => makeTarget(screen, f.slice(0, -'.feature'.length),
|
|
82
|
+
.map((f) => makeTarget(screen, f.slice(0, -'.feature'.length), kind))
|
|
77
83
|
.sort((a, b) => a.featureBaseName.localeCompare(b.featureBaseName));
|
|
78
84
|
}
|
|
79
85
|
|
|
80
86
|
function listAllTargets(cwd: string): DeliveryTarget[] {
|
|
81
87
|
const targets: DeliveryTarget[] = [];
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
for (const d of fs.readdirSync(
|
|
86
|
-
if (d.isDirectory()) targets.push(...listFeatureTargets(cwd, d.name,
|
|
87
|
-
}
|
|
88
|
-
}
|
|
89
|
-
|
|
90
|
-
const flowsDir = path.join(cwd, 'qa', 'flows');
|
|
91
|
-
if (fs.existsSync(flowsDir)) {
|
|
92
|
-
for (const d of fs.readdirSync(flowsDir, { withFileTypes: true })) {
|
|
93
|
-
if (d.isDirectory()) targets.push(...listFeatureTargets(cwd, d.name, true));
|
|
88
|
+
const scan = (kind: UnitKind, skip: (n: string) => boolean = () => false) => {
|
|
89
|
+
const root = path.join(cwd, 'qa', qaParent(kind));
|
|
90
|
+
if (!fs.existsSync(root)) return;
|
|
91
|
+
for (const d of fs.readdirSync(root, { withFileTypes: true })) {
|
|
92
|
+
if (d.isDirectory() && !skip(d.name)) targets.push(...listFeatureTargets(cwd, d.name, kind));
|
|
94
93
|
}
|
|
95
|
-
}
|
|
94
|
+
};
|
|
95
|
+
scan('screen');
|
|
96
|
+
scan('flow');
|
|
97
|
+
scan('api', (n) => n === 'flows'); // qa/api/<area> (api flows live under qa/api/flows — a follow-up)
|
|
96
98
|
|
|
97
99
|
return targets.sort((a, b) => a.featureBaseName.localeCompare(b.featureBaseName));
|
|
98
100
|
}
|
|
@@ -108,28 +110,24 @@ function listAllTargets(cwd: string): DeliveryTarget[] {
|
|
|
108
110
|
* feature file with the basename across all screens & flows.
|
|
109
111
|
*/
|
|
110
112
|
function resolveTargetsFromArg(cwd: string, name: string): DeliveryTarget[] {
|
|
111
|
-
if (fs.existsSync(path.join(cwd, 'qa', 'flows', name)))
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
return listFeatureTargets(cwd, name, false);
|
|
116
|
-
}
|
|
117
|
-
// Treat as feature basename — find the parent screen/flow that hosts it.
|
|
113
|
+
if (fs.existsSync(path.join(cwd, 'qa', 'flows', name))) return listFeatureTargets(cwd, name, 'flow');
|
|
114
|
+
if (fs.existsSync(path.join(cwd, 'qa', 'screens', name))) return listFeatureTargets(cwd, name, 'screen');
|
|
115
|
+
if (fs.existsSync(path.join(cwd, 'qa', 'api', name))) return listFeatureTargets(cwd, name, 'api');
|
|
116
|
+
// Treat as feature basename — find the parent unit that hosts it.
|
|
118
117
|
const candidates = listAllTargets(cwd).filter((t) => t.featureBaseName === name);
|
|
119
118
|
if (candidates.length > 0) return candidates;
|
|
120
119
|
// Fallback: treat as screen name even if directory missing (lets preflight
|
|
121
120
|
// surface the "feature file missing" error with the right path).
|
|
122
|
-
return [makeTarget(name, name,
|
|
121
|
+
return [makeTarget(name, name, 'screen')];
|
|
123
122
|
}
|
|
124
123
|
|
|
125
124
|
function qaDir(cwd: string, target: DeliveryTarget): string {
|
|
126
|
-
return path.join(cwd, 'qa', target.
|
|
125
|
+
return path.join(cwd, 'qa', qaParent(target.kind), target.screen);
|
|
127
126
|
}
|
|
128
127
|
|
|
129
128
|
function generatedDir(cwd: string, target: DeliveryTarget): string {
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
: path.join(cwd, 'specs', 'generated', target.screen);
|
|
129
|
+
const sub = target.kind === 'flow' ? path.join('flows', target.screen) : target.kind === 'api' ? path.join('api', target.screen) : target.screen;
|
|
130
|
+
return path.join(cwd, 'specs', 'generated', sub);
|
|
133
131
|
}
|
|
134
132
|
|
|
135
133
|
// ----------------------------------------------------------------------------
|
|
@@ -262,7 +260,8 @@ function runPreflight(cwd: string, target: DeliveryTarget): PreflightCheck {
|
|
|
262
260
|
|
|
263
261
|
const featureOk = checkFeatureReal(featureFile);
|
|
264
262
|
const testDataOk = checkTestDataHasVars(testDataFile);
|
|
265
|
-
|
|
263
|
+
// API units have no DOM → no selectors; the catalog (apis.yaml) is the contract. N/A, not missing.
|
|
264
|
+
const selectorsOk = target.kind === 'api' ? true : checkSelectorsHasEntries(selectorsFile, target.featureBaseName);
|
|
266
265
|
const specOk = fs.existsSync(specFile);
|
|
267
266
|
const resultsOk = resultsFile !== null;
|
|
268
267
|
|
|
@@ -284,7 +283,7 @@ function runPreflight(cwd: string, target: DeliveryTarget): PreflightCheck {
|
|
|
284
283
|
}
|
|
285
284
|
if (!specOk) {
|
|
286
285
|
missing.push(`compiled .spec.ts missing: ${path.relative(cwd, specFile)}`);
|
|
287
|
-
suggestions.push(target.
|
|
286
|
+
suggestions.push(`sungen generate --${target.kind === 'flow' ? 'flow' : target.kind === 'api' ? 'api' : 'screen'} ${target.screen}`);
|
|
288
287
|
}
|
|
289
288
|
if (!resultsOk) {
|
|
290
289
|
const env = process.env.SUNGEN_ENV;
|
|
@@ -8,18 +8,20 @@ export function registerScriptCheckCommand(program: Command): void {
|
|
|
8
8
|
.command('script-check')
|
|
9
9
|
.description('Verify the generated Playwright spec is a faithful 1:1 of the Gherkin feature (no hand-edit / stale drift)')
|
|
10
10
|
.option('-s, --screen <name>', 'Screen or flow name')
|
|
11
|
+
.option('--api <name>', 'API-first area or api flow (qa/api/<name>)')
|
|
11
12
|
.option('--json', 'Output raw JSON')
|
|
12
13
|
.action(async (options) => {
|
|
13
14
|
try {
|
|
14
|
-
const name = options.screen;
|
|
15
|
-
if (!name) throw new Error('Provide --screen <name>');
|
|
15
|
+
const name = options.screen || options.api;
|
|
16
|
+
if (!name) throw new Error('Provide --screen <name> (or --api <area>)');
|
|
16
17
|
const screen = path.join(process.cwd(), 'qa', 'screens', name);
|
|
17
18
|
const flow = path.join(process.cwd(), 'qa', 'flows', name);
|
|
18
|
-
const
|
|
19
|
-
const
|
|
20
|
-
|
|
19
|
+
const api = path.join(process.cwd(), 'qa', 'api', name);
|
|
20
|
+
const kind = fs.existsSync(screen) ? 'screen' : fs.existsSync(flow) ? 'flow' : fs.existsSync(api) ? 'api' : null;
|
|
21
|
+
const dir = kind === 'screen' ? screen : kind === 'flow' ? flow : kind === 'api' ? api : null;
|
|
22
|
+
if (!dir || !kind) throw new Error(`Not found: qa/screens|flows|api/${name}`);
|
|
21
23
|
|
|
22
|
-
const r = await runScriptCheck(dir, name,
|
|
24
|
+
const r = await runScriptCheck(dir, name, kind);
|
|
23
25
|
|
|
24
26
|
const outDir = path.join(process.cwd(), '.sungen', 'reports');
|
|
25
27
|
fs.mkdirSync(outDir, { recursive: true });
|
|
@@ -39,7 +41,7 @@ export function registerScriptCheckCommand(program: Command): void {
|
|
|
39
41
|
if (r.findings.length) { L(' findings:'); for (const f of r.findings) L(` • ${f}`); }
|
|
40
42
|
else L(' ✓ The test code faithfully reflects the Gherkin (1:1).');
|
|
41
43
|
L('');
|
|
42
|
-
if (r.drift === 'drift') L(
|
|
44
|
+
if (r.drift === 'drift') L(` → Fix: re-run \`sungen generate --${kind === 'api' ? 'api' : kind === 'flow' ? 'flow' : 'screen'} ${name}\` (or /sungen:run-test) so the spec matches the feature. Never hand-edit generated specs.`);
|
|
43
45
|
L('');
|
|
44
46
|
process.exit(r.status === 'OK' ? 0 : 2);
|
|
45
47
|
} catch (error) {
|
|
@@ -8,16 +8,18 @@ export function registerTraceCommand(program: Command): void {
|
|
|
8
8
|
.command('trace')
|
|
9
9
|
.description('Visualise the executed test-design process (workflow/skill steps, repair loops), find bottlenecks, and show where to focus human review')
|
|
10
10
|
.option('-s, --screen <name>', 'Screen or flow name')
|
|
11
|
+
.option('--api <name>', 'API-first area or api flow (qa/api/<name>)')
|
|
11
12
|
.option('--json', 'Output raw JSON')
|
|
12
13
|
.option('--mermaid', 'Print only the Mermaid flowchart')
|
|
13
14
|
.action((options) => {
|
|
14
15
|
try {
|
|
15
|
-
const name = options.screen;
|
|
16
|
-
if (!name) throw new Error('Provide --screen <name>');
|
|
16
|
+
const name = options.screen || options.api;
|
|
17
|
+
if (!name) throw new Error('Provide --screen <name> (or --api <area>)');
|
|
17
18
|
const screen = path.join(process.cwd(), 'qa', 'screens', name);
|
|
18
19
|
const flow = path.join(process.cwd(), 'qa', 'flows', name);
|
|
19
|
-
const
|
|
20
|
-
|
|
20
|
+
const api = path.join(process.cwd(), 'qa', 'api', name);
|
|
21
|
+
const dir = fs.existsSync(screen) ? screen : fs.existsSync(flow) ? flow : fs.existsSync(api) ? api : null;
|
|
22
|
+
if (!dir) throw new Error(`Not found: qa/screens|flows|api/${name}`);
|
|
21
23
|
|
|
22
24
|
const r = buildTrace(dir, name);
|
|
23
25
|
if (options.json) { console.log(JSON.stringify(r, null, 2)); return; }
|
|
@@ -106,9 +106,18 @@ function normalize(src: string): string {
|
|
|
106
106
|
.trim();
|
|
107
107
|
}
|
|
108
108
|
|
|
109
|
-
|
|
109
|
+
/** The unit kind — drives the generated-spec subdir + the qa source dir. */
|
|
110
|
+
export type UnitKind = 'screen' | 'flow' | 'api';
|
|
111
|
+
|
|
112
|
+
/** Generated-spec subdir for a unit: screen → <name>, flow → flows/<name>, api → api/<name>. */
|
|
113
|
+
function specSubdir(dir: string, name: string, kind: UnitKind): string {
|
|
114
|
+
return kind === 'flow' ? path.join(dir, 'flows', name) : kind === 'api' ? path.join(dir, 'api', name) : path.join(dir, name);
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
function findSpec(dir: string, name: string, kind: UnitKind): string | null {
|
|
110
118
|
// Screens compile to <dir>/<name>/<feature>.spec.ts
|
|
111
119
|
// Flows compile to <dir>/flows/<name>/<feature>.spec.ts
|
|
120
|
+
// Api compile to <dir>/api/<name>/<feature>.spec.ts
|
|
112
121
|
// Scope the search to THIS target's own subdir — otherwise the first spec of
|
|
113
122
|
// ANY other screen/flow is returned, which (for an uncompiled flow) falsely
|
|
114
123
|
// reports the wrong screen's tests as drift.
|
|
@@ -121,19 +130,19 @@ function findSpec(dir: string, name: string, flowMode: boolean): string | null {
|
|
|
121
130
|
else if (e.name.endsWith('.spec.ts')) hits.push(p);
|
|
122
131
|
}
|
|
123
132
|
};
|
|
124
|
-
const scoped =
|
|
133
|
+
const scoped = specSubdir(dir, name, kind);
|
|
125
134
|
if (!fs.existsSync(scoped)) return null; // no spec for this target (e.g. not compiled yet)
|
|
126
135
|
walk(scoped);
|
|
127
136
|
return hits[0] ?? null;
|
|
128
137
|
}
|
|
129
138
|
|
|
130
|
-
export async function runScriptCheck(screenDir: string, screenName: string,
|
|
139
|
+
export async function runScriptCheck(screenDir: string, screenName: string, kind: UnitKind): Promise<ScriptCheckResult> {
|
|
131
140
|
const featurePath = path.join(screenDir, 'features', `${screenName}.feature`);
|
|
132
141
|
const scenarios = loadScenarios(featurePath);
|
|
133
142
|
const automated = scenarios.filter((s) => !s.manual);
|
|
134
143
|
const manual = scenarios.filter((s) => s.manual);
|
|
135
144
|
|
|
136
|
-
const committedSpec = findSpec(path.join(process.cwd(), 'specs', 'generated'), screenName,
|
|
145
|
+
const committedSpec = findSpec(path.join(process.cwd(), 'specs', 'generated'), screenName, kind);
|
|
137
146
|
|
|
138
147
|
const findings: string[] = [];
|
|
139
148
|
let specTitles: string[] = [];
|
|
@@ -167,10 +176,14 @@ export async function runScriptCheck(screenDir: string, screenName: string, flow
|
|
|
167
176
|
try {
|
|
168
177
|
const { CodeGenerator } = require('../generators/test-generator/code-generator');
|
|
169
178
|
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'sungen-scriptcheck-'));
|
|
170
|
-
const qaSourceDir = path.join(process.cwd(), 'qa',
|
|
171
|
-
|
|
179
|
+
const qaSourceDir = path.join(process.cwd(), 'qa', kind === 'flow' ? 'flows' : kind === 'api' ? 'api' : 'screens');
|
|
180
|
+
// api units derive their unit id (api/<area>) from the feature path — like `generate --api`;
|
|
181
|
+
// screen/flow pass screenName + flowMode explicitly (unchanged → byte-identical regenerate).
|
|
182
|
+
const gen = kind === 'api'
|
|
183
|
+
? new CodeGenerator({ framework: 'playwright', runtimeData: true })
|
|
184
|
+
: new CodeGenerator({ framework: 'playwright', screenName, runtimeData: true, flowMode: kind === 'flow' });
|
|
172
185
|
await gen.generateAllTests(qaSourceDir, tmp, [featurePath]);
|
|
173
|
-
const fresh = findSpec(tmp, screenName,
|
|
186
|
+
const fresh = findSpec(tmp, screenName, kind);
|
|
174
187
|
if (fresh) {
|
|
175
188
|
const a = normalize(specSrc);
|
|
176
189
|
const b = normalize(fs.readFileSync(fresh, 'utf-8'));
|
|
@@ -47,6 +47,7 @@ export const AI_RULES_FILE_MAPPING: [string, string][] = [
|
|
|
47
47
|
['claude-skill-selector-fix.md', '.claude/skills/sungen-selector-fix/SKILL.md'],
|
|
48
48
|
['claude-skill-tc-review.md', '.claude/skills/sungen-tc-review/SKILL.md'],
|
|
49
49
|
['claude-skill-harness-audit.md', '.claude/skills/sungen-harness-audit/SKILL.md'],
|
|
50
|
+
['claude-skill-api-design.md', '.claude/skills/sungen-api-design/SKILL.md'],
|
|
50
51
|
['claude-skill-ingest-legacy.md', '.claude/skills/sungen-ingest-legacy/SKILL.md'],
|
|
51
52
|
['claude-skill-viewpoint.md', '.claude/skills/sungen-viewpoint/SKILL.md'],
|
|
52
53
|
['claude-skill-viewpoint-group-a-data-entry.md', '.claude/skills/sungen-viewpoint/group-a-data-entry.md'],
|
|
@@ -79,6 +80,7 @@ export const AI_RULES_FILE_MAPPING: [string, string][] = [
|
|
|
79
80
|
['github-skill-sungen-selector-fix.md', '.github/skills/sungen-selector-fix/SKILL.md'],
|
|
80
81
|
['github-skill-sungen-tc-review.md', '.github/skills/sungen-tc-review/SKILL.md'],
|
|
81
82
|
['github-skill-sungen-harness-audit.md', '.github/skills/sungen-harness-audit/SKILL.md'],
|
|
83
|
+
['github-skill-sungen-api-design.md', '.github/skills/sungen-api-design/SKILL.md'],
|
|
82
84
|
['github-skill-sungen-ingest-legacy.md', '.github/skills/sungen-ingest-legacy/SKILL.md'],
|
|
83
85
|
['github-skill-sungen-viewpoint.md', '.github/skills/sungen-viewpoint/SKILL.md'],
|
|
84
86
|
['github-skill-sungen-viewpoint-group-a-data-entry.md', '.github/skills/sungen-viewpoint/group-a-data-entry.md'],
|
|
@@ -19,6 +19,12 @@ You are an **independent Senior QA Reviewer**. You did **not** write these tests
|
|
|
19
19
|
3. **Business-critical depth.** For cart / product-detail / filter / list viewpoints, do steps assert **DATA** (name, price, quantity, all-items-belong) — not just page/modal visibility? Recommend the concrete deep step: `User remember [X] text as {{v}}` + `... with {{v}}`, or `User see all [X] contain {{v}}`.
|
|
20
20
|
4. **@manual justification.** Is each `@manual` genuinely unautomatable (cross-screen/external/visual) — or a cop-out to dodge the gate? Cross-screen → should be a flow.
|
|
21
21
|
5. **Meaning-level duplicates & missing criticals** the keyword gate can't see.
|
|
22
|
+
6. **API units** (`qa/api/<area>/` — `@api` scenarios, no UI). Judge what the api gate can't:
|
|
23
|
+
- **Prove the effect, not the status.** A mutating endpoint's success path asserting only `{{r.status}} is 201` proves nothing about WHAT changed — demand a **body** assertion (`{{r.body.id}}` / `{{r.body.<field>}}`), a **`@query`** DB side-effect, or (idempotency) a `{{r.ok_count}}` invariant. This is the API businessDepth bar.
|
|
24
|
+
- **Error matrix coherent.** `@cases` rows are a real failure family (validation/auth/conflict) with realistic inputs → declared statuses, not padding.
|
|
25
|
+
- **Flows self-clean.** A CRUD/auth chain deletes what it created (final `@api:delete_*`) or is `@cleanup`-tagged.
|
|
26
|
+
- **Idempotency uses the DB oracle.** A "no double-charge / exactly once" claim is proven by `@concurrent` + a `@query` count, not HTTP status alone (status can lie under a race).
|
|
27
|
+
- **Auth negatives** exist for protected mutations (401/403), not just the happy path.
|
|
22
28
|
|
|
23
29
|
## Output (do NOT edit any file)
|
|
24
30
|
Return a concise verdict:
|
|
@@ -23,7 +23,11 @@ You are a **Senior QA Engineer** specialized in test case design. You structure
|
|
|
23
23
|
|
|
24
24
|
Parse **name** from `$ARGUMENTS`. If missing, ask the user.
|
|
25
25
|
|
|
26
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists →
|
|
26
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode. Else `qa/screens/<name>/` → screen mode. This determines paths, generation strategy, and CLI commands.
|
|
27
|
+
|
|
28
|
+
## API unit mode (driver-api)
|
|
29
|
+
|
|
30
|
+
If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the design loop differs — **no visual capture, no selectors**; the contract is the named-endpoint catalog. **Follow the `sungen-api-design` skill end-to-end** instead of the screen/flow steps below: `sungen context --api <name>` (discover) → API viewpoint overview → generate `@api`/`@cases`/flow/`@concurrent`/`@query` scenarios → **`sungen audit --api <name>` gate + the `sungen-reviewer` sub-agent + repair loop to businessDepth ≥ 0.7** → record + trace. Then jump to the "Converge" next-step options (recommend `/sungen:run-test <name>`). The capture / viewpoint-group / selector steps do **not** apply.
|
|
27
31
|
|
|
28
32
|
## Steps
|
|
29
33
|
|
|
@@ -30,7 +30,22 @@ If the count is 0 → use `AskUserQuestion` to offer:
|
|
|
30
30
|
|
|
31
31
|
Skip this pre-flight when `--env` matches the base locale (no overlay needed in that case).
|
|
32
32
|
|
|
33
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists → flow mode (base path: `qa/flows/<name>/`). Else
|
|
33
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode (base path: `qa/flows/<name>/`). Else `qa/screens/<name>/` → screen mode (base path: `qa/screens/<name>/`).
|
|
34
|
+
|
|
35
|
+
## API unit mode (driver-api) — no selectors
|
|
36
|
+
|
|
37
|
+
If the unit is **api-first**, skip every selector/capture phase (an API test has no DOM). Instead:
|
|
38
|
+
|
|
39
|
+
1. **Resolve the datasource** — ensure the `kind: api` datasource's `base_url` + auth are wired in `qa/datasources.yaml` + `.env.qa` (the `${X_URL}` key from `sungen api init`). A `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
40
|
+
2. **Compile**: `[ -x ./bin/sungen.js ] && ./bin/sungen.js generate --api <name> || npx sungen generate --api <name>` → `specs/generated/api/<name>/`.
|
|
41
|
+
3. **Run**: `npx playwright test specs/generated/api/<name>/<name>.spec.ts` (per-spec JSON results, as below).
|
|
42
|
+
4. **Auto-fix** (no selectors — the failure classes differ): use `sungen-error-mapping`.
|
|
43
|
+
- **401/403** → wire `@hybrid` + `@auth:<role>` (reuse the UI session) or the catalog `Bearer :token` header; suggest `sungen makeauth <role>`.
|
|
44
|
+
- **datasource/base_url unresolved** → set the `${X_URL}` key in `.env.qa`.
|
|
45
|
+
- **missing/empty bound param** → trace `{{var}}` to test-data or a prior `@api` response; fill it.
|
|
46
|
+
- **`expect.status` mismatch** → reconcile against `apis.yaml`/spec (the catalog is the oracle); **never hand-edit the generated spec** (re-`generate --api` instead).
|
|
47
|
+
- **flaky** → enforce self-cleaning flows, per-row isolation (`@cases`), `@concurrent` caps.
|
|
48
|
+
5. **Integrity + trace** — `sungen script-check --api <name>` (verify the spec is a 1:1 of the Gherkin; on DRIFT re-`generate --api`, never hand-edit) and `sungen trace --api <name>` (process map + HUMAN-LOOP FOCUS). Then report + offer next steps.
|
|
34
49
|
|
|
35
50
|
## Pre-run (phased — per `sungen-selector-fix` skill)
|
|
36
51
|
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sungen-api-design
|
|
3
|
+
description: The API-first design loop for an api unit (qa/api/<area> or qa/api/flows/<flow>) — discover the catalog, lay out the API viewpoints, generate @api/@cases/flow/@concurrent scenarios, then drive the sungen audit --api gate + reviewer + repair to a high businessDepth (≥0.7). Use when create-test/run-test detects an api unit (no selectors, no visual capture).
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# API design loop (driver-api · Orchestration + Harness)
|
|
7
|
+
|
|
8
|
+
Use this when the unit is **api-first** — `qa/api/<area>/` or `qa/api/flows/<flow>/`. There are **no selectors and no visual capture**: the contract is the **named-endpoint catalog** (`api/apis.yaml`), referenced by `@api:<name>`. QA writes **no HTTP code**. Full annotation reference: the **API Steps** guide (`@api` / `@cases` / flows / `@concurrent` / `@hybrid`).
|
|
9
|
+
|
|
10
|
+
## The loop (mirror of /sungen:design, API-native)
|
|
11
|
+
|
|
12
|
+
### 1. Discover (no capture)
|
|
13
|
+
Run `sungen context --api <name>` — it reads the catalog and prints the **endpoints** + the **generation units** (one `matrix` unit per endpoint, an `async` unit per mutating endpoint, a `flow` unit for an api flow). Read `qa/api/<name>/requirements/spec.md` if present. No `apis.yaml` yet? → `sungen api import <openapi|csv>` or `sungen api add --area <name>` first.
|
|
14
|
+
|
|
15
|
+
### 2. API viewpoint overview (by method-profile)
|
|
16
|
+
For each endpoint, cover its viewpoints — severity-weighted by method:
|
|
17
|
+
|
|
18
|
+
| Profile | Endpoints | Must cover | Then |
|
|
19
|
+
|---|---|---|---|
|
|
20
|
+
| read | GET, HEAD | `contract` (status + body shape) | `pagination`/`filter` (list), `not-found` (by-id) |
|
|
21
|
+
| mutating | POST/PUT/PATCH/DELETE | `contract`, `error` (validation/4xx/auth) | `idempotency` (`@concurrent`), `side-effect` (`@query`) |
|
|
22
|
+
|
|
23
|
+
Bands: **~70%** success+failure matrix · **~20%** flows (auth/CRUD chains) · **~10%** async/idempotency.
|
|
24
|
+
|
|
25
|
+
### 3. Generate (incremental — never the whole suite in one Write)
|
|
26
|
+
- **Contract**: `@api:<name>` + `expect {{name.status}} is …` **and a body assertion** (`{{name.body.<path>}}`).
|
|
27
|
+
- **Error matrix**: `@api:<name>(p={{p}}) @cases:<dataset>` — one scenario, a dataset of `input → expected status`.
|
|
28
|
+
- **Flow**: ordered `@api` tags threading a prior response (`token={{login.body.token}}` → the catalog `Bearer :token` header; `id={{create.body.id}}` → a path param). Self-clean (delete what you create).
|
|
29
|
+
- **Idempotency**: `@api:<name> @concurrent:N` + `expect {{name.ok_count}} is 1`, cross-checked with `@query` (the DB is the oracle).
|
|
30
|
+
|
|
31
|
+
### 4. Gate + repair (always — businessDepth ≥ 0.7 is the bar)
|
|
32
|
+
Run `sungen audit --api <name>`; read `gateStatus` + `findings`. Then the **semantic reviewer** (sungen-reviewer sub-agent, API criteria). Repair **both** (budget 3 rounds), re-audit until PASS:
|
|
33
|
+
|
|
34
|
+
| Finding | Repair |
|
|
35
|
+
|---|---|
|
|
36
|
+
| `VIEWPOINT-API-CONTRACT` | the endpoint is invoked but its response is never asserted → add `expect {{name.status}}` + a `{{name.body.…}}` check |
|
|
37
|
+
| `VIEWPOINT-API-ERROR` | a mutating endpoint has no failure scenario → add a `@cases` error matrix (or an explicit 4xx) |
|
|
38
|
+
| `VIEWPOINT-API-IDEMPOTENCY` | a mutating endpoint has no race check → add `@concurrent:N` + a `@query` DB cross-check |
|
|
39
|
+
| **`DEPTH-FAIL`** (businessDepth < 0.7) | a **mutating success** scenario asserts only `status` → make it **prove the effect**: assert a response **body** field, a **`@query`** side-effect, or a **`@concurrent` `ok_count`** invariant. (An error/`@cases` scenario proving the status is correct — it is *not* depth-required.) |
|
|
40
|
+
|
|
41
|
+
Stop when the gate PASSes + businessDepth ≥ 0.7, or the budget is exhausted → report residual gaps honestly (mark genuinely-unautomatable cases `@manual` with an oracle). Never fake a pass.
|
|
42
|
+
|
|
43
|
+
### 5. Record + converge
|
|
44
|
+
`sungen manifest --api <name>` (reuse) and ledger each phase; show the trace + the HUMAN-LOOP FOCUS. (Integrity `script-check`/`trace` for api: see run-test.)
|
|
45
|
+
|
|
46
|
+
## Rules
|
|
47
|
+
- **No HTTP, no selectors** — only `.feature` + the reviewed `apis.yaml` + `test-data`.
|
|
48
|
+
- **Non-prod default** — a `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
49
|
+
- **The DB is the oracle** for idempotency/side-effects — HTTP status alone can lie; pair `@api` with `@query`.
|
|
@@ -18,7 +18,11 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
|
|
|
18
18
|
|
|
19
19
|
- **name** — ${input:name:screen or flow name (e.g., login, award-submission)}
|
|
20
20
|
|
|
21
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists → flow mode (base path: `qa/flows/<name>/`). Else
|
|
21
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode (base path: `qa/flows/<name>/`). Else `qa/screens/<name>/` → screen mode (base path: `qa/screens/<name>/`).
|
|
22
|
+
|
|
23
|
+
## API unit mode (driver-api)
|
|
24
|
+
|
|
25
|
+
If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the design loop differs — **no visual capture, no selectors**; the contract is the named-endpoint catalog. **Follow the `sungen-api-design` skill end-to-end** instead of the screen/flow steps: `sungen context --api <name>` (discover) → API viewpoint overview → generate `@api`/`@cases`/flow/`@concurrent`/`@query` scenarios → **`sungen audit --api <name>` gate + reviewer + repair loop to businessDepth ≥ 0.7** → record + trace. Then recommend `/sungen-run-test <name>`. The capture / viewpoint-group / selector steps do **not** apply.
|
|
22
26
|
|
|
23
27
|
## Steps
|
|
24
28
|
|
|
@@ -30,7 +30,16 @@ Count 0 → offer the user:
|
|
|
30
30
|
|
|
31
31
|
Skip when `--env` matches the base locale.
|
|
32
32
|
|
|
33
|
-
**Auto-detect context**: check if `qa/flows/<name>/` exists → flow mode (base path: `qa/flows/<name>/`). Else
|
|
33
|
+
**Auto-detect context**: check if `qa/api/<name>/` or `qa/api/flows/<name>/` exists → **API unit mode** (below). Else if `qa/flows/<name>/` → flow mode (base path: `qa/flows/<name>/`). Else `qa/screens/<name>/` → screen mode (base path: `qa/screens/<name>/`).
|
|
34
|
+
|
|
35
|
+
## API unit mode (driver-api) — no selectors
|
|
36
|
+
|
|
37
|
+
If the unit is **api-first**, skip every selector/capture phase (an API test has no DOM):
|
|
38
|
+
1. **Resolve the datasource** — `base_url` + auth wired in `qa/datasources.yaml` + `.env.qa` (`${X_URL}` from `sungen api init`); a `production` datasource is refused unless `SUNGEN_ALLOW_PROD=1`.
|
|
39
|
+
2. **Compile**: `npx sungen generate --api <name>` → `specs/generated/api/<name>/`.
|
|
40
|
+
3. **Run**: `npx playwright test specs/generated/api/<name>/<name>.spec.ts`.
|
|
41
|
+
4. **Auto-fix** (use `sungen-error-mapping`): 401/403 → `@hybrid`+`@auth` or `Bearer :token` header (`sungen makeauth`); base_url unresolved → set `${X_URL}`; missing param → trace `{{var}}` to test-data/a prior `@api` response; `expect.status` mismatch → reconcile against `apis.yaml` (re-`generate --api`, never hand-edit the spec); flaky → self-clean + `@concurrent` caps.
|
|
42
|
+
5. **Integrity + trace** — `sungen script-check --api <name>` (1:1; on DRIFT re-`generate --api`, never hand-edit the spec) + `sungen trace --api <name>` (process map + HUMAN-LOOP FOCUS). Report + offer next steps.
|
|
34
43
|
|
|
35
44
|
## Pre-run (phased — per `sungen-selector-fix` skill)
|
|
36
45
|
|