@sun-asterisk/sungen 3.2.24-beta.5 → 3.2.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/commands/audit.d.ts.map +1 -1
- package/dist/cli/commands/audit.js +9 -1
- package/dist/cli/commands/audit.js.map +1 -1
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/biometric-action.hbs +9 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/check-action.hbs +11 -6
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/deep-link-action.hbs +8 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/location-services-action.hbs +11 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/reinstall-app-action.hbs +21 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/restart-app-action.hbs +16 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/set-connectivity-action.hbs +11 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/actions/uncheck-action.hbs +9 -5
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/checked-assertion.hbs +10 -4
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/device-log-assertion.hbs +16 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/disappears-within-assertion.hbs +5 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/not-checked-assertion.hbs +6 -4
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/state-with-filter-assertion.hbs +4 -3
- package/dist/generators/test-generator/adapters/appium/templates/steps/assertions/still-visible-after-assertion.hbs +4 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/gestures/pull-to-refresh-action.hbs +3 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/gestures/swipe-action.hbs +3 -0
- package/dist/generators/test-generator/adapters/appium/templates/steps/partials/appium-selector.hbs +7 -1
- package/dist/generators/test-generator/adapters/appium/templates/test-file.hbs +129 -15
- package/dist/generators/test-generator/template-engine.d.ts.map +1 -1
- package/dist/generators/test-generator/template-engine.js +3 -0
- package/dist/generators/test-generator/template-engine.js.map +1 -1
- package/dist/generators/test-generator/utils/selector-resolver.d.ts.map +1 -1
- package/dist/generators/test-generator/utils/selector-resolver.js +4 -0
- package/dist/generators/test-generator/utils/selector-resolver.js.map +1 -1
- package/dist/harness/audit.d.ts.map +1 -1
- package/dist/harness/audit.js +38 -2
- package/dist/harness/audit.js.map +1 -1
- package/dist/harness/capability-plan.d.ts.map +1 -1
- package/dist/harness/capability-plan.js +6 -1
- package/dist/harness/capability-plan.js.map +1 -1
- package/dist/harness/parse.d.ts +1 -0
- package/dist/harness/parse.d.ts.map +1 -1
- package/dist/harness/parse.js +1 -0
- package/dist/harness/parse.js.map +1 -1
- package/dist/harness/script-check.d.ts.map +1 -1
- package/dist/harness/script-check.js +6 -2
- package/dist/harness/script-check.js.map +1 -1
- package/dist/harness/sensors.d.ts +31 -0
- package/dist/harness/sensors.d.ts.map +1 -1
- package/dist/harness/sensors.js +79 -1
- package/dist/harness/sensors.js.map +1 -1
- package/dist/harness/viewpoint-ledger.d.ts +21 -0
- package/dist/harness/viewpoint-ledger.d.ts.map +1 -1
- package/dist/harness/viewpoint-ledger.js +57 -5
- package/dist/harness/viewpoint-ledger.js.map +1 -1
- package/dist/orchestrator/mobile-runtime-scaffolder.d.ts.map +1 -1
- package/dist/orchestrator/mobile-runtime-scaffolder.js +5 -1
- package/dist/orchestrator/mobile-runtime-scaffolder.js.map +1 -1
- package/dist/orchestrator/templates/ai-src/commands/create-test.md +23 -0
- package/dist/orchestrator/templates/ai-src/skills/sungen-gherkin-syntax/SKILL.md +10 -4
- package/dist/orchestrator/templates/ai-src/skills/sungen-mobile-gestures/SKILL.md +23 -0
- package/dist/orchestrator/templates/env.appium.example +11 -0
- package/dist/orchestrator/templates/specs-db.d.ts.map +1 -1
- package/dist/orchestrator/templates/specs-db.js +79 -6
- package/dist/orchestrator/templates/specs-db.js.map +1 -1
- package/dist/orchestrator/templates/specs-db.ts +70 -7
- package/dist/orchestrator/templates/wdio.conf.ts +100 -3
- package/dist/utils/selector-types.d.ts +1 -1
- package/dist/utils/selector-types.d.ts.map +1 -1
- package/dist/utils/selector-types.js +1 -0
- package/dist/utils/selector-types.js.map +1 -1
- package/package.json +3 -3
- package/src/cli/commands/audit.ts +9 -1
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/biometric-action.hbs +9 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/check-action.hbs +11 -6
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/deep-link-action.hbs +8 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/location-services-action.hbs +11 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/reinstall-app-action.hbs +21 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/restart-app-action.hbs +16 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/set-connectivity-action.hbs +11 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/actions/uncheck-action.hbs +9 -5
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/checked-assertion.hbs +10 -4
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/device-log-assertion.hbs +16 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/disappears-within-assertion.hbs +5 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/not-checked-assertion.hbs +6 -4
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/state-with-filter-assertion.hbs +4 -3
- package/src/generators/test-generator/adapters/appium/templates/steps/assertions/still-visible-after-assertion.hbs +4 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/gestures/pull-to-refresh-action.hbs +3 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/gestures/swipe-action.hbs +3 -0
- package/src/generators/test-generator/adapters/appium/templates/steps/partials/appium-selector.hbs +7 -1
- package/src/generators/test-generator/adapters/appium/templates/test-file.hbs +129 -15
- package/src/generators/test-generator/template-engine.ts +4 -0
- package/src/generators/test-generator/utils/selector-resolver.ts +4 -0
- package/src/harness/audit.ts +39 -2
- package/src/harness/capability-plan.ts +3 -1
- package/src/harness/parse.ts +2 -0
- package/src/harness/script-check.ts +6 -2
- package/src/harness/sensors.ts +73 -1
- package/src/harness/viewpoint-ledger.ts +68 -5
- package/src/orchestrator/mobile-runtime-scaffolder.ts +5 -1
- package/src/orchestrator/templates/ai-src/commands/create-test.md +23 -0
- package/src/orchestrator/templates/ai-src/skills/sungen-gherkin-syntax/SKILL.md +10 -4
- package/src/orchestrator/templates/ai-src/skills/sungen-mobile-gestures/SKILL.md +23 -0
- package/src/orchestrator/templates/env.appium.example +11 -0
- package/src/orchestrator/templates/specs-db.ts +70 -7
- package/src/orchestrator/templates/wdio.conf.ts +100 -3
- package/src/utils/selector-types.ts +1 -0
|
@@ -72,20 +72,38 @@ async function __ensureVisible(selector: string): Promise<WebdriverIO.Element> {
|
|
|
72
72
|
}
|
|
73
73
|
const safeBottom = Math.round(winH * 0.82); // above the floating bottom-nav band
|
|
74
74
|
let lastY: number | null = null;
|
|
75
|
+
let lastEl: WebdriverIO.Element | null = null;
|
|
75
76
|
for (let __i = 0; __i < 12; __i++) {
|
|
76
77
|
let y: number | null = null;
|
|
78
|
+
let existsUndisplayed = false;
|
|
77
79
|
try {
|
|
78
80
|
const el = await $(selector);
|
|
79
81
|
if (await el.isDisplayed()) {
|
|
80
82
|
const r = await driver.getElementRect(el.elementId);
|
|
81
|
-
|
|
83
|
+
// Done when the element sits fully above the floating bottom-nav band — OR is simply too
|
|
84
|
+
// tall to ever fit there (a scroll container / full-screen list): demanding "fully above"
|
|
85
|
+
// from such an element makes the loop scroll a page that was already correct, dragging
|
|
86
|
+
// list-hosted siblings (e.g. the top bar as first list item) out of the viewport.
|
|
87
|
+
if (r.y + r.height <= safeBottom || r.height >= safeBottom) return el;
|
|
82
88
|
y = r.y;
|
|
89
|
+
lastEl = el;
|
|
90
|
+
} else if (el.elementId) {
|
|
91
|
+
existsUndisplayed = true; // in the tree, just not visible right now
|
|
83
92
|
}
|
|
84
93
|
} catch { /* off-screen / not in tree yet → keep scrolling */ }
|
|
94
|
+
// Overlay grace (Android mirror of the iOS render grace above): the element is IN the tree but
|
|
95
|
+
// reports not-displayed — typically a launch popup/overlay covering it that auto-dismisses
|
|
96
|
+
// within seconds. Scrolling now would carry a list-hosted element (a Compose top bar is often
|
|
97
|
+
// the first list item) clean out of the viewport, where a down-only loop can never recover it —
|
|
98
|
+
// so wait the overlay out before the first scroll.
|
|
99
|
+
if (existsUndisplayed && __i < 4) { await driver.pause(1200); continue; }
|
|
85
100
|
// No-progress break: the element is on screen but a prior scroll didn't move it — it's a FIXED
|
|
86
101
|
// element (e.g. a bottom-nav tab, intentionally in the nav band) or we've hit the end of the
|
|
87
102
|
// scrollable. Stop rather than burn the whole budget; the caller acts on it where it is.
|
|
88
|
-
|
|
103
|
+
// Return the RESOLVED element, never a fresh $() stub: on an animation-churned tree (a looping
|
|
104
|
+
// banner is enough) a from-scratch re-find takes seconds and outlasts the caller's assertion
|
|
105
|
+
// budget — the element we just measured is the one the caller should act on.
|
|
106
|
+
if (y !== null && lastY !== null && Math.abs(y - lastY) < 2) return lastEl ?? $(selector);
|
|
89
107
|
lastY = y;
|
|
90
108
|
await driver.execute('mobile: scrollGesture', { left: 60, top: 420, width: 960, height: 1320, direction: 'down', percent: 0.5 });
|
|
91
109
|
await driver.pause(300);
|
|
@@ -116,6 +134,74 @@ async function __assertVisible(selector: string): Promise<void> {
|
|
|
116
134
|
}
|
|
117
135
|
throw new Error(`Expect ${JSON.stringify(selector)} to be displayed — iOS geometry oracle: rect=${JSON.stringify(r)} is not on screen (0x0 = not rendered / scrolled out)`);
|
|
118
136
|
}
|
|
137
|
+
|
|
138
|
+
// Checkbox / toggle / radio state oracle. There is NO single authoritative source for it on
|
|
139
|
+
// mobile, and the standard one lies in a way that cannot be detected from the read itself:
|
|
140
|
+
// UiAutomator2 puts `checked` on EVERY Android node and defaults it to "false" when the widget
|
|
141
|
+
// publishes no checked semantics. So a custom-rendered control — a Flutter widget without
|
|
142
|
+
// `Semantics(checked: …)`, an RN `Pressable` without `accessibilityState={ checked: … }` — is not
|
|
143
|
+
// ABSENT from the tree, it reports a confident false while sitting visibly ON, and a strict
|
|
144
|
+
// `toHaveAttribute('checked','true')` fails with no way to see why.
|
|
145
|
+
// Hence: probe every signal both runtimes expose, and let the DIRECTION of the assertion decide
|
|
146
|
+
// how to combine them (__assertCheckedState below).
|
|
147
|
+
// Android: `checked`, `selected`, isSelected() iOS: `value` ("1"/"0"), `selected`, isSelected()
|
|
148
|
+
// Descendants are probed too, bounded: on a composite row (label + control merged into one a11y
|
|
149
|
+
// node) the state routinely sits on an inner node while the queried outer node reports the default.
|
|
150
|
+
// A child xpath must start with `.//` — a leading `//` is ABSOLUTE in Appium even when chained.
|
|
151
|
+
async function __probeCheckedSignals(el: WebdriverIO.Element): Promise<{ source: string; value: boolean }[]> {
|
|
152
|
+
const out: { source: string; value: boolean }[] = [];
|
|
153
|
+
const read = async (node: WebdriverIO.Element, prefix: string): Promise<void> => {
|
|
154
|
+
for (const attr of driver.isIOS ? ['value', 'selected'] : ['checked', 'selected']) {
|
|
155
|
+
let raw: string | null = null;
|
|
156
|
+
try { raw = await node.getAttribute(attr); } catch { continue; } // attribute unsupported on this node
|
|
157
|
+
const v = String(raw ?? '').trim().toLowerCase();
|
|
158
|
+
if (v === 'true' || v === '1') out.push({ source: `${prefix}${attr}="${raw}"`, value: true });
|
|
159
|
+
else if (v === 'false' || v === '0') out.push({ source: `${prefix}${attr}="${raw}"`, value: false });
|
|
160
|
+
// anything else (empty, a real iOS `value` string like "Off") carries no parseable state → skip
|
|
161
|
+
}
|
|
162
|
+
try { out.push({ source: `${prefix}isSelected()`, value: await node.isSelected() }); } catch { /* not supported here */ }
|
|
163
|
+
};
|
|
164
|
+
await read(el, '');
|
|
165
|
+
// Only descend when the node itself yielded nothing positive — otherwise we already have the
|
|
166
|
+
// answer and a subtree walk would turn one assertion into dozens of round-trips for nothing.
|
|
167
|
+
if (!out.some(s => s.value)) {
|
|
168
|
+
let kids: WebdriverIO.Element[] = [];
|
|
169
|
+
try { kids = await el.$$('.//*'); } catch { /* no child query on this node */ }
|
|
170
|
+
for (const k of kids.slice(0, 8)) await read(k, 'descendant.');
|
|
171
|
+
}
|
|
172
|
+
return out;
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
// Assert checked/unchecked from the probed signals. The two directions are deliberately NOT
|
|
176
|
+
// symmetric, because the evidence isn't:
|
|
177
|
+
// checked → ANY credible positive signal passes. One runtime exposing the state is enough;
|
|
178
|
+
// demanding the standard attribute is what breaks custom-rendered controls.
|
|
179
|
+
// unchecked → NO signal anywhere may be positive. Strictly stronger than reading one attribute,
|
|
180
|
+
// so a visibly-ON custom toggle can't slip through as a false PASS.
|
|
181
|
+
// Polls like WDIO's own matchers do — a tap-then-assert reads the tree mid-animation.
|
|
182
|
+
// KNOWN LIMIT, stated rather than hidden: on Android a UiAutomator2-defaulted `checked="false"` is
|
|
183
|
+
// indistinguishable from a genuine false — that information is not in the tree. So `unchecked`
|
|
184
|
+
// passes on "no positive signal anywhere", which is the strongest available evidence, not proof.
|
|
185
|
+
async function __assertCheckedState(el: WebdriverIO.Element, expected: boolean, label: string): Promise<void> {
|
|
186
|
+
// Fail loud with the REAL cause when the element never resolved (mirrors check-action, #430).
|
|
187
|
+
if (!el.elementId) throw new Error((el as any).error?.message || `Element not found: ${el.selector}`);
|
|
188
|
+
let signals: { source: string; value: boolean }[] = [];
|
|
189
|
+
for (let __i = 0; __i < 10; __i++) {
|
|
190
|
+
signals = await __probeCheckedSignals(el);
|
|
191
|
+
if (signals.some(s => s.value) === expected) return;
|
|
192
|
+
await driver.pause(500);
|
|
193
|
+
}
|
|
194
|
+
const want = expected ? 'checked' : 'unchecked';
|
|
195
|
+
if (signals.length === 0) {
|
|
196
|
+
throw new Error(
|
|
197
|
+
`Expect ${label} to be ${want} — the control publishes NO checked/selected state, on itself or any child. ` +
|
|
198
|
+
`A custom-rendered control has to publish it for any tool to read it: Flutter → wrap it in Semantics(checked: …); ` +
|
|
199
|
+
`React Native → accessibilityState={ checked: … }. Until then assert a visible consequence instead ` +
|
|
200
|
+
`(a label/text change), or tag the scenario @manual.`,
|
|
201
|
+
);
|
|
202
|
+
}
|
|
203
|
+
throw new Error(`Expect ${label} to be ${want} — read [${signals.map(s => `${s.source} → ${s.value}`).join(', ')}]`);
|
|
204
|
+
}
|
|
119
205
|
{{#if singleAuthRole}}
|
|
120
206
|
import * as fs from 'node:fs';
|
|
121
207
|
import * as path from 'node:path';
|
|
@@ -133,7 +219,9 @@ function __authSelector(s: { type: string; value: string }): string {
|
|
|
133
219
|
case 'accessibility-id': case 'testid': case 'label': case 'placeholder': case 'text': return '~' + s.value;
|
|
134
220
|
case 'android-uiautomator': return 'android=' + s.value;
|
|
135
221
|
case 'ios-predicate': return '-ios predicate string:' + s.value;
|
|
136
|
-
|
|
222
|
+
// Appium's id strategy prepends `<appPackage>:id/` to colon-less values, which breaks bare
|
|
223
|
+
// Compose testTagsAsResourceId ids — match those exactly via UiSelector on Android instead.
|
|
224
|
+
case 'id': return (!driver.isIOS && !s.value.includes(':')) ? 'android=new UiSelector().resourceId("' + s.value + '")' : 'id=' + s.value;
|
|
137
225
|
case 'xpath': case 'locator': default: return s.value;
|
|
138
226
|
}
|
|
139
227
|
}
|
|
@@ -205,10 +293,29 @@ async function __ensureAuth(role: string): Promise<void> {
|
|
|
205
293
|
}
|
|
206
294
|
|
|
207
295
|
// ── Mode B: legacy authedMarker (present on the landing screen ⟺ logged in) ──
|
|
208
|
-
|
|
209
|
-
|
|
296
|
+
// markerTimeoutMs: contract override for slow-booting apps — a cold start that needs 10-15s+ to
|
|
297
|
+
// render the landing screen false-negatives the 6s default into "logged out" and then blocks the
|
|
298
|
+
// spec when the login form never appears.
|
|
299
|
+
const __budget = cfg.markerTimeoutMs || 6000;
|
|
300
|
+
// When the contract also names a loggedOutMarker (an element unique to the login screen), the
|
|
301
|
+
// state is decided by WHICHEVER marker renders first (alternating short probes): a slow-booting
|
|
302
|
+
// landing screen costs only its real boot time instead of gambling on one fixed timeout, and a
|
|
303
|
+
// genuine logged-out state is detected without waiting out the full authed budget first.
|
|
304
|
+
if (cfg.loggedOutMarker) {
|
|
305
|
+
let loggedOut = false;
|
|
306
|
+
const __t0 = Date.now();
|
|
307
|
+
while (Date.now() - __t0 < __budget) {
|
|
308
|
+
if (await present(cfg.authedMarker, 3000)) return; // logged in — nothing to do
|
|
309
|
+
if (await present(cfg.loggedOutMarker, 3000)) { loggedOut = true; break; }
|
|
310
|
+
}
|
|
311
|
+
if (!loggedOut) {
|
|
312
|
+
throw new Error(`@auth:${role} could not decide the login state within ${__budget}ms — neither authedMarker nor loggedOutMarker rendered. Check the markers in ${rel}.`);
|
|
313
|
+
}
|
|
314
|
+
} else if (await present(cfg.authedMarker, __budget)) {
|
|
315
|
+
return; // idempotent — already authenticated (session kept by noReset)
|
|
316
|
+
}
|
|
210
317
|
await runSteps(cfg.loginSteps);
|
|
211
|
-
if (!(await
|
|
318
|
+
if (!(await present(cfg.authedMarker, __budget))) {
|
|
212
319
|
throw new Error(`@auth:${role} login failed — verify credentials / loginSteps / authedMarker in ${rel}.`);
|
|
213
320
|
}
|
|
214
321
|
}
|
|
@@ -267,25 +374,32 @@ describe('{{featureName}}', () => {
|
|
|
267
374
|
// of cold-starting, which strands top-of-screen elements above the viewport.
|
|
268
375
|
await driver.pause(500);
|
|
269
376
|
await driver.activateApp(__APP_PKG__);
|
|
377
|
+
// Restore the NATIVE context — a prior scenario that entered a WebView (`switch to [X] frame`)
|
|
378
|
+
// without switching back would otherwise poison every following native find.
|
|
379
|
+
try { await driver.switchContext('NATIVE_APP'); } catch { /* single-context session */ }
|
|
270
380
|
// Restore portrait so a prior scenario's `rotate to landscape` can't leak into this one — the
|
|
271
381
|
// app relaunch above resets in-app state, but device orientation is OS-level and persists.
|
|
272
382
|
try { await driver.setOrientation('PORTRAIT'); } catch { /* orientation control not supported */ }
|
|
273
383
|
// Settle after a cold (re)launch: a Flutter app first renders a BLANK accessibility skeleton
|
|
274
384
|
// (nested FrameLayout/View, no content-desc) for several seconds before the real tree appears.
|
|
275
|
-
//
|
|
276
|
-
//
|
|
277
|
-
//
|
|
278
|
-
//
|
|
279
|
-
//
|
|
280
|
-
|
|
385
|
+
// So require rendered content — at least one non-empty content-desc/text — on two consecutive
|
|
386
|
+
// reads before proceeding. A generous timeout covers slow cold renders on a stressed emulator.
|
|
387
|
+
// Best-effort: a timeout never fails the scenario, it just falls through to the step's own
|
|
388
|
+
// auto-wait.
|
|
389
|
+
// "Settled" = rendered content present on TWO consecutive reads. Deliberately NOT full source
|
|
390
|
+
// equality: a UI with any looping animation (a rotating promo carousel is enough) never yields
|
|
391
|
+
// two identical snapshots, so an equality oracle starves and burns the whole timeout on every
|
|
392
|
+
// scenario. Two content-bearing reads still reject the blank first-frame skeleton (no
|
|
393
|
+
// content-desc/text at all) that a single read could false-positive on.
|
|
394
|
+
let __prevHadContent = false;
|
|
281
395
|
await driver
|
|
282
396
|
.waitUntil(
|
|
283
397
|
async () => {
|
|
284
398
|
const src = await driver.getPageSource();
|
|
285
399
|
const hasContent = /content-desc="[^"]+"|\btext="[^"]+"/.test(src);
|
|
286
|
-
const
|
|
287
|
-
|
|
288
|
-
return
|
|
400
|
+
const settled = hasContent && __prevHadContent;
|
|
401
|
+
__prevHadContent = hasContent;
|
|
402
|
+
return settled;
|
|
289
403
|
},
|
|
290
404
|
{ timeout: 30000, interval: 400, timeoutMsg: 'UI did not settle (no rendered content)' },
|
|
291
405
|
)
|
|
@@ -43,6 +43,10 @@ export class TemplateEngine {
|
|
|
43
43
|
return String(text).replace(/[.*+?^${}()|[\]\\/]/g, '\\$&');
|
|
44
44
|
});
|
|
45
45
|
|
|
46
|
+
Handlebars.registerHelper('includes', function(text: string, search: string) {
|
|
47
|
+
return typeof text === 'string' && text.includes(search);
|
|
48
|
+
});
|
|
49
|
+
|
|
46
50
|
// Comparison helpers for conditional logic
|
|
47
51
|
Handlebars.registerHelper('eq', function(a: any, b: any) {
|
|
48
52
|
return a === b;
|
|
@@ -677,6 +677,10 @@ export class SelectorResolver {
|
|
|
677
677
|
case 'android-uiautomator':
|
|
678
678
|
case 'ios-predicate':
|
|
679
679
|
case 'id':
|
|
680
|
+
// css: only meaningful INSIDE a WebView context (hybrid screens, after `switch to [X]
|
|
681
|
+
// frame`) — the appium adapter's partial already renders it raw; without this pass-through
|
|
682
|
+
// the type fell to the placeholder strategy and compiled to an undefined selector.
|
|
683
|
+
case 'css':
|
|
680
684
|
return withExtras({
|
|
681
685
|
strategy: type,
|
|
682
686
|
value,
|
package/src/harness/audit.ts
CHANGED
|
@@ -15,6 +15,7 @@ import { featureFilesFor } from './unit-paths';
|
|
|
15
15
|
import {
|
|
16
16
|
loadCatalog, viewpointGate, assertionDepth, dataThemesFor, depthThresholdFor, coverageBalance, duplicateClusters, traceability, claimProof, taxonomyLint,
|
|
17
17
|
automatableManual, flowCoveredThemes, flowRegressionDepth, oracleStrength, declaredPageType,
|
|
18
|
+
numericClaimGaps, priorityMismatches,
|
|
18
19
|
GateResult, DepthResult, BalanceResult, DuplicateResult, TraceResult, ClaimProofResult, TaxonomyResult, Catalog, AutomatableManualResult, FlowDepthResult, OracleStrengthResult,
|
|
19
20
|
} from './sensors';
|
|
20
21
|
import { loadFlowScenarios } from './flow-check';
|
|
@@ -235,7 +236,9 @@ export function runAudit(screenDir: string, screenName: string): AuditReport {
|
|
|
235
236
|
const ledger = viewpointLedger(viewpointPath, scenarios, featureText);
|
|
236
237
|
const negSideEffect = negativeSideEffect(scenarios);
|
|
237
238
|
const ownership = crossArtifactOwnership(screenDir, scenarios);
|
|
238
|
-
|
|
239
|
+
// @exploration scenarios are probes outside the official suite — the finding's own fix
|
|
240
|
+
// ("tag it @exploration") must actually exempt them from source tracing.
|
|
241
|
+
const unsourced = sourceBacked(scenarios.filter((s) => !s.exploration), parseSpecClauses(specPath).frs.map((f) => f.id), parseViewpointItems(viewpointPath).map((i) => i.text), viewpoints.map((v) => v.id), featureText);
|
|
239
242
|
|
|
240
243
|
// H3 — stateful-flow regression depth. For a UI flow whose scenarios mutate a cart/checkout
|
|
241
244
|
// collection, the regression dimensions (count/quantity proof · teardown · multi-source) cap the
|
|
@@ -303,7 +306,13 @@ export function runAudit(screenDir: string, screenName: string): AuditReport {
|
|
|
303
306
|
// that rewrites that file scores them 100% by construction — which is exactly
|
|
304
307
|
// what a create-test run did, silently dropping the performance viewpoint on
|
|
305
308
|
// the way. While the declaration is unconfirmed, neither axis is evidence.
|
|
306
|
-
const
|
|
309
|
+
const adoptedPostHoc = viewpointBaseline.status === 'new' && scenarios.length > 0;
|
|
310
|
+
// `changed` was the only untrusted state, but `new` alongside an EXISTING suite is the same
|
|
311
|
+
// problem seen earlier: the yardstick is being introduced against scenarios that already exist,
|
|
312
|
+
// so it can only measure what their author already thought of. One field run wrote the file
|
|
313
|
+
// after generating the suite and said so in its own header — and both axes still read 100%,
|
|
314
|
+
// because a first audit records the baseline silently (#622). Untrusted until a human accepts it.
|
|
315
|
+
const viewpointMoved = viewpointBaseline.status === 'changed' || adoptedPostHoc;
|
|
307
316
|
// #618 — is there a YARDSTICK at all? `atomicLedger` and `traceability` are both measured
|
|
308
317
|
// against test-viewpoint.md, and four more sensors read it (the ledger, the declaration-integrity
|
|
309
318
|
// checks, the browser-gesture check, the continuity check). When the file is absent every one of
|
|
@@ -574,6 +583,9 @@ export function runAudit(screenDir: string, screenName: string): AuditReport {
|
|
|
574
583
|
} else if (balance.imbalanced && !flowScored) {
|
|
575
584
|
findings.push(`BALANCE: ${balance.note} Stop expanding secondary viewpoints until business-core gaps are filled.`);
|
|
576
585
|
}
|
|
586
|
+
if (adoptedPostHoc) {
|
|
587
|
+
findings.push(`VIEWPOINT-ADOPTED-POST-HOC: this is the first audit of \`requirements/test-viewpoint.md\` and the suite ALREADY has ${scenarios.length} scenarios, so the yardstick is being introduced against tests that already exist. A declaration written after the suite can only measure what its author already thought of — and if it was derived FROM the suite, \`atomicLedger\` and \`traceability\` read 100% by construction while anything dropped along the way stops being missing from anything. Both axes are held as unverified until you review the claims (add what you care about, delete what you do not) and confirm: \`sungen audit --screen ${screenName} --accept-viewpoint\`.`);
|
|
588
|
+
}
|
|
577
589
|
if (!hasYardstick) {
|
|
578
590
|
const lost = [
|
|
579
591
|
'atomicLedger', 'traceability',
|
|
@@ -658,6 +670,31 @@ export function runAudit(screenDir: string, screenName: string): AuditReport {
|
|
|
658
670
|
if (depth.deferredBusinessCritical > 0 && depth.deferredBusinessCritical >= depth.businessCriticalTotal) {
|
|
659
671
|
findings.push(`DEPTH-DEFERRED: businessDepth ${businessDepth.toFixed(2)} is computed over only ${depth.businessCriticalTotal} on-screen scenario(s); ${depth.deferredBusinessCritical} business-critical scenario(s) are deferred to @manual (excluded from the ratio). Automate them in a flow and verify with \`sungen flow-check\` — this ratio is NOT "all business depth covered".`);
|
|
660
672
|
}
|
|
673
|
+
// #622 — a claim that names a count, proved by fewer assertions than it names. The count sits
|
|
674
|
+
// in the viewpoint CLAIM, so both the title-level and the claim-level shape are checked.
|
|
675
|
+
for (const g of ledger.partial.slice(0, 5)) {
|
|
676
|
+
findings.push(`CLAIM-PARTIAL: claim ${g.id ?? ''} names ${g.claimed} but its scenario asserts ${g.asserted} value(s) — "${g.text.slice(0, 70)}". Mapping a claim to a scenario id proves it is ADDRESSED, not that the oracle demonstrates it: assert all ${g.claimed}, use a \`see all …\` set assertion, or narrow the claim to what is actually proven.`);
|
|
677
|
+
}
|
|
678
|
+
for (const g of numericClaimGaps(scenarios).slice(0, 3)) {
|
|
679
|
+
findings.push(`CLAIM-PARTIAL: "${g.name}" claims ${g.claimed} ${g.noun} but asserts ${g.asserted} value(s) — assert all ${g.claimed}, or say in the title what the scenario actually proves.`);
|
|
680
|
+
}
|
|
681
|
+
// #622 — the priority table is where release selection comes from; a tag that contradicts it
|
|
682
|
+
// means nobody can tell which of the two was the decision.
|
|
683
|
+
{
|
|
684
|
+
const pm = priorityMismatches(hasYardstick ? (readText(viewpointPath) ?? '') : '', scenarios);
|
|
685
|
+
const byId = new Map<string, { declared: string; tagged: string; n: number }>();
|
|
686
|
+
for (const m of pm) {
|
|
687
|
+
const cur = byId.get(m.id) ?? { declared: m.declared, tagged: m.tagged, n: 0 };
|
|
688
|
+
cur.n++;
|
|
689
|
+
byId.set(m.id, cur);
|
|
690
|
+
}
|
|
691
|
+
for (const [id, v] of [...byId.entries()].slice(0, 5)) {
|
|
692
|
+
findings.push(`PRIORITY-CONTRADICTED: the viewpoint declares ${id} as "${v.declared}" but ${v.n} of its scenario(s) are tagged @${v.tagged}. A smoke run selects on the tag, so it picks up work the declaration deprioritised — reconcile them, and if the tag is right, fix the declaration rather than leaving both on record.`);
|
|
693
|
+
}
|
|
694
|
+
}
|
|
695
|
+
if (ledger.manualOnly.length > 0) {
|
|
696
|
+
findings.push(`VIEWPOINT-ITEM-MANUAL-ONLY: ${ledger.manualOnly.length} viewpoint claim(s) are carried only by @manual scenarios — ${ledger.manualOnly.slice(0, 4).map((m) => m.id).join(', ')}${ledger.manualOnly.length > 4 ? ', …' : ''}. The claim is ADDRESSED (a procedure exists) but nothing runs for it, so a regression would not notice it breaking. \`atomicLedger\` counts them, because the design work is real — the delivery must not report them as automated coverage.`);
|
|
697
|
+
}
|
|
661
698
|
if (ledger.hasViewpoint && ledger.missing.length) {
|
|
662
699
|
const sample = ledger.missing.slice(0, 6).map((m) => m.id || `"${m.text}"`).join(', ');
|
|
663
700
|
findings.push(`VIEWPOINT-ITEM-MISSING: ${ledger.missing.length}/${ledger.total} atomic viewpoint items have no covering scenario (${(ledger.ratio * 100).toFixed(0)}% covered) — e.g. ${sample}. Cover each item or mark it deferred/spec-gap.`);
|
|
@@ -153,7 +153,9 @@ export function manualReasonMismatches(featurePath: string): ReasonMismatch[] {
|
|
|
153
153
|
for (let k = i + 1; k < lines.length && k <= i + 16; k++) {
|
|
154
154
|
const l = lines[k].trim();
|
|
155
155
|
if (/^#/.test(l)) parts.push(l.replace(/^#+\s*/, ''));
|
|
156
|
-
|
|
156
|
+
// A blank line after collected comments ends the body block — scanning past it would
|
|
157
|
+
// swallow the NEXT section's banner comment and mis-infer the reason code from it.
|
|
158
|
+
else if (l === '') { if (parts.length) break; continue; }
|
|
157
159
|
else break;
|
|
158
160
|
}
|
|
159
161
|
// The block scanned here includes the tester PROCEDURE, so several codes can
|
package/src/harness/parse.ts
CHANGED
|
@@ -36,6 +36,7 @@ export interface ScenarioInfo {
|
|
|
36
36
|
queryRefs?: string[]; // named queries referenced by this scenario (inline `query [name]` + @query: tags)
|
|
37
37
|
apiRefs?: string[]; // named API endpoints referenced by this scenario (@api: tags)
|
|
38
38
|
requiresCaps?: string[]; // @requires:<cap> — automation-ready but needs an opt-in driver (TQ-11)
|
|
39
|
+
exploration?: boolean; // @exploration — a probe outside the official suite, exempt from source tracing
|
|
39
40
|
deferredToFlow?: boolean; // @deferred:flow — owned by a flow, not automated on this screen (H6)
|
|
40
41
|
ownedByFlow?: string; // @owned-by:<flow> — the flow that owns this deferred scenario (H6)
|
|
41
42
|
/** Ordered steps with their resolved bucket (And/But inherit) — flow handoff analysis (#569). */
|
|
@@ -253,6 +254,7 @@ function classifyScenario(sc: ParsedScenario): ScenarioInfo {
|
|
|
253
254
|
queryRefs: queryRefs.size ? [...queryRefs] : undefined,
|
|
254
255
|
apiRefs: apiRefs.size ? [...apiRefs] : undefined,
|
|
255
256
|
requiresCaps: requiresCaps.length ? requiresCaps : undefined,
|
|
257
|
+
exploration: tags.some((t) => /^@exploration\b/i.test(t)) || undefined,
|
|
256
258
|
deferredToFlow: deferredToFlow || undefined,
|
|
257
259
|
ownedByFlow,
|
|
258
260
|
};
|
|
@@ -197,8 +197,12 @@ export async function runScriptCheck(screenDir: string, screenName: string, kind
|
|
|
197
197
|
// ONLY the template form, so on the current codegen every @cases scenario was
|
|
198
198
|
// reported MISSING (+ its test EXTRA) despite compiling and running fine.
|
|
199
199
|
// Canonicalize the extracted title to the bare scenario name so both shapes
|
|
200
|
-
// (and plain titles) compare on the same footing.
|
|
201
|
-
|
|
200
|
+
// (and plain titles) compare on the same footing. Mobile/mocha specs append the
|
|
201
|
+
// tags to the title (`it('<name> [@high @spec:FR-01]')` — mocha has no tag option),
|
|
202
|
+
// so strip that suffix too or every mobile scenario reads MISSING + EXTRA (#429).
|
|
203
|
+
const canonSpecTitle = (t: string) => t
|
|
204
|
+
.replace(/\s*\[@[^\]]*\]$/, '')
|
|
205
|
+
.replace(/\s*—(\s*\$\{__row\.__label\})?$/, '');
|
|
202
206
|
const specTitleSet = new Set(specTitles.map(canonSpecTitle));
|
|
203
207
|
const expectedSet = new Set(automated.map((s) => s.name));
|
|
204
208
|
const missingInSpec = automated.filter((s) => !specTitleSet.has(s.name)).map((s) => s.name);
|
package/src/harness/sensors.ts
CHANGED
|
@@ -17,6 +17,8 @@ import { readTextFile } from './read-text';
|
|
|
17
17
|
const BUSINESS_CRITICAL_CATS = [
|
|
18
18
|
// UI commerce cores
|
|
19
19
|
'LIST', 'CART', 'PRODUCT', 'FILTER', 'CHECKOUT', 'ORDER', 'DETAIL', 'DISCOVERY', 'CATEGORY', 'BRAND', 'DUPLICATE', 'CONSISTENCY',
|
|
20
|
+
// Payment surfaces are business-core in any commerce/fintech app (JALPAY, PAYMENT, WALLET …)
|
|
21
|
+
'PAY', 'WALLET',
|
|
20
22
|
// API / DB capability cores — for an api/db suite the operation IS the business core
|
|
21
23
|
'API', 'ENDPOINT', 'CRUD', 'QUERY', 'CONTRACT', 'RESOURCE',
|
|
22
24
|
];
|
|
@@ -27,7 +29,9 @@ const BUSINESS_CRITICAL_CATS = [
|
|
|
27
29
|
const BUCKET_ORDER: Array<[string, string[]]> = [
|
|
28
30
|
['business-core', BUSINESS_CRITICAL_CATS],
|
|
29
31
|
['behavior', ['LOGIC', 'TRANSITION', 'WORKFLOW']],
|
|
30
|
-
|
|
32
|
+
// ERROR/EMPTY: error handling and empty-state guards are the same defensive family as
|
|
33
|
+
// validation — every non-trivial suite declares them, so `other` would swallow them.
|
|
34
|
+
['validation-security', ['VAL', 'SEC', 'SUB', 'AUTH', 'LOGIN', 'ERROR', 'EMPTY']],
|
|
31
35
|
['navigation', ['NAV']],
|
|
32
36
|
['presentation', ['UI', 'LAYOUT', 'RESPONSIVE', 'DISPLAY', 'SEO', 'ACCESSIBILITY', 'USABILITY', 'VISUAL']],
|
|
33
37
|
];
|
|
@@ -843,3 +847,71 @@ export function claimProof(scenarios: ScenarioInfo[], focus = 'functional'): Cla
|
|
|
843
847
|
|
|
844
848
|
return { total: measured.length, withClaims, proven, unproven, ratio, focus, threshold, verdict };
|
|
845
849
|
}
|
|
850
|
+
|
|
851
|
+
// ---------- Numeric claims, and priority that disagrees with the declaration ----------
|
|
852
|
+
|
|
853
|
+
const NUMBER_WORDS: Record<string, number> = {
|
|
854
|
+
two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10,
|
|
855
|
+
};
|
|
856
|
+
|
|
857
|
+
/**
|
|
858
|
+
* A title that names a COUNT, proved by fewer assertions than it claims.
|
|
859
|
+
*
|
|
860
|
+
* "restores all **six** buffered values" asserted four fields; "all **seven** confirmation rows
|
|
861
|
+
* are read-only" checked two. Exact id mapping made both read as covered, and `claimProof` had no
|
|
862
|
+
* rule for it — a claim can be mapped to a scenario and still be only partly proven, which is the
|
|
863
|
+
* gap between "declared coverage" and "coverage an oracle demonstrates" (#622).
|
|
864
|
+
*
|
|
865
|
+
* Counting is the whole point here: the number is in the title, so the check is arithmetic rather
|
|
866
|
+
* than semantic. A `see all …` assertion is exempt — it proves a set in one step, which is the
|
|
867
|
+
* stronger shape, not a weaker one.
|
|
868
|
+
*/
|
|
869
|
+
export function numericClaimGaps(scenarios: ScenarioInfo[]): Array<{ name: string; claimed: number; asserted: number; noun: string }> {
|
|
870
|
+
const out: Array<{ name: string; claimed: number; asserted: number; noun: string }> = [];
|
|
871
|
+
for (const s of scenarios) {
|
|
872
|
+
if (s.manual) continue; // a procedure counts by hand
|
|
873
|
+
if (/\bsee all\b/.test(s.stepsText)) continue; // a set proved in one assertion
|
|
874
|
+
const m = s.name.toLowerCase().match(/\b(?:all\s+)?(two|three|four|five|six|seven|eight|nine|ten|[2-9]|10)\s+([a-z][a-z-]{2,})/);
|
|
875
|
+
if (!m) continue;
|
|
876
|
+
const claimed = NUMBER_WORDS[m[1]] ?? Number(m[1]);
|
|
877
|
+
if (!Number.isFinite(claimed) || claimed < 2) continue;
|
|
878
|
+
// Only value-bearing assertions count: `see [X] … with/contains {{v}}`. A visibility check
|
|
879
|
+
// does not demonstrate one of N values.
|
|
880
|
+
const asserted = (s.stepsText.match(/\bsee \[[^\]]+\][^|]*?\b(?:with|contains) \{\{/g) ?? []).length;
|
|
881
|
+
if (asserted > 0 && asserted < claimed) out.push({ name: s.name, claimed, asserted, noun: m[2] });
|
|
882
|
+
}
|
|
883
|
+
return out;
|
|
884
|
+
}
|
|
885
|
+
|
|
886
|
+
/**
|
|
887
|
+
* A scenario whose priority tag contradicts the priority its viewpoint declares.
|
|
888
|
+
*
|
|
889
|
+
* The Priority-Viewpoints table is where release selection is supposed to come from. When the
|
|
890
|
+
* table says a flow is `Low` and its scenario is tagged `@high`, a smoke run picks up work the QA
|
|
891
|
+
* deliberately deprioritised — and nobody can tell which of the two was the decision (#622).
|
|
892
|
+
*/
|
|
893
|
+
export function priorityMismatches(
|
|
894
|
+
viewpointText: string, scenarios: ScenarioInfo[],
|
|
895
|
+
): Array<{ id: string; declared: string; tagged: string; scenario: string }> {
|
|
896
|
+
const declared = new Map<string, string>();
|
|
897
|
+
for (const m of viewpointText.matchAll(/^\|\s*((?:VP|FL)[A-Z0-9._-]*)\s*\|\s*(critical|high|medium|normal|low|deferred)\s*\|/gim)) {
|
|
898
|
+
declared.set(m[1].toUpperCase(), m[2].toLowerCase());
|
|
899
|
+
}
|
|
900
|
+
if (declared.size === 0) return [];
|
|
901
|
+
// `medium` and `normal` are the same rank under different names in real documents.
|
|
902
|
+
const rank = (p: string): string => (p === 'medium' ? 'normal' : p);
|
|
903
|
+
const out: Array<{ id: string; declared: string; tagged: string; scenario: string }> = [];
|
|
904
|
+
for (const s of scenarios) {
|
|
905
|
+
const id = (s.vpId ?? '').toUpperCase();
|
|
906
|
+
if (!id) continue;
|
|
907
|
+
// The most specific declared row that prefixes this id.
|
|
908
|
+
const key = [...declared.keys()].filter((k) => id.startsWith(k)).sort((a, b) => b.length - a.length)[0];
|
|
909
|
+
if (!key) continue;
|
|
910
|
+
const want = rank(declared.get(key)!);
|
|
911
|
+
const got = rank(String(s.priority).toLowerCase());
|
|
912
|
+
// `critical` and `deferred` have no tag equivalent — nothing to contradict.
|
|
913
|
+
if (want === 'critical' || want === 'deferred') continue;
|
|
914
|
+
if (want !== got) out.push({ id: key, declared: declared.get(key)!, tagged: got, scenario: s.name });
|
|
915
|
+
}
|
|
916
|
+
return out;
|
|
917
|
+
}
|
|
@@ -20,6 +20,39 @@ export interface LedgerResult {
|
|
|
20
20
|
covered: number;
|
|
21
21
|
ratio: number;
|
|
22
22
|
missing: { id?: string; text: string }[];
|
|
23
|
+
/**
|
|
24
|
+
* Items whose id is carried ONLY by `@manual` scenarios. Accounted for — a documented
|
|
25
|
+
* procedure exists — but nothing runs, so a regression would not notice. Counted as covered
|
|
26
|
+
* (the claim IS addressed) and reported separately, the same split `FLOW-MANUAL-ONLY` makes
|
|
27
|
+
* for declared flows (#622).
|
|
28
|
+
*/
|
|
29
|
+
manualOnly: { id?: string; text: string }[];
|
|
30
|
+
/**
|
|
31
|
+
* Items whose CLAIM names a count that the scenario carrying its id does not fully assert —
|
|
32
|
+
* "restores all six buffered values" proved on four fields. Exact id mapping shows a claim is
|
|
33
|
+
* ADDRESSED; it says nothing about whether the oracle demonstrates all of it (#622).
|
|
34
|
+
*/
|
|
35
|
+
partial: { id?: string; text: string; claimed: number; asserted: number }[];
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
const NUMBER_WORDS: Record<string, number> = {
|
|
39
|
+
two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10,
|
|
40
|
+
};
|
|
41
|
+
|
|
42
|
+
/** The count a claim names, when it names one: "all six values", "seven rows", "3 items". */
|
|
43
|
+
function claimedCount(text: string): number | null {
|
|
44
|
+
// A digit after `step`/`screen`/`phase` is an ORDINAL, not a quantity — "the buffer exists at
|
|
45
|
+
// step 4" claims nothing about four of anything. And the counted thing has to be a noun: a
|
|
46
|
+
// function word after the number means the number was not counting.
|
|
47
|
+
const FUNCTION_WORD = /^(and|the|is|are|was|were|of|or|in|on|at|to|for|with|that|which|but|not|has|have|its)$/;
|
|
48
|
+
for (const m of text.toLowerCase().matchAll(/(\w+\s+)?\b(?:all\s+)?(two|three|four|five|six|seven|eight|nine|ten|[2-9]|10)\s+([a-z][a-z-]{2,})/g)) {
|
|
49
|
+
const before = (m[1] ?? '').trim();
|
|
50
|
+
if (/^(step|steps|screen|screens|phase|phases|page|pages|tier|round|part|section|version|mail_0?)$/.test(before)) continue;
|
|
51
|
+
if (FUNCTION_WORD.test(m[3])) continue;
|
|
52
|
+
const n = NUMBER_WORDS[m[2]] ?? Number(m[2]);
|
|
53
|
+
if (Number.isFinite(n) && n >= 2) return n;
|
|
54
|
+
}
|
|
55
|
+
return null;
|
|
23
56
|
}
|
|
24
57
|
|
|
25
58
|
const ID_RE = /\b([A-Z]{1,5}\d{0,2}(?:[.\-][A-Za-z0-9]+)*-?\d{0,3})\b/; // VP0.Title, VP7-002, MS-HP-001, TV-01
|
|
@@ -74,16 +107,43 @@ export function parseViewpointItems(viewpointPath: string): { id?: string; text:
|
|
|
74
107
|
export function viewpointLedger(viewpointPath: string, scenarios: ScenarioInfo[], featureText: string): LedgerResult {
|
|
75
108
|
const items = parseViewpointItems(viewpointPath);
|
|
76
109
|
if (!fs.existsSync(viewpointPath) || items.length === 0) {
|
|
77
|
-
return { hasViewpoint: fs.existsSync(viewpointPath), total: 0, covered: 0, ratio: 1, missing: [] };
|
|
110
|
+
return { hasViewpoint: fs.existsSync(viewpointPath), total: 0, covered: 0, ratio: 1, missing: [], manualOnly: [], partial: [] };
|
|
78
111
|
}
|
|
79
112
|
const featLower = featureText.toLowerCase();
|
|
80
113
|
const missing: { id?: string; text: string }[] = [];
|
|
114
|
+
const manualOnly: { id?: string; text: string }[] = [];
|
|
115
|
+
const partial: { id?: string; text: string; claimed: number; asserted: number }[] = [];
|
|
81
116
|
let covered = 0;
|
|
82
117
|
|
|
83
118
|
for (const item of items) {
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
119
|
+
// WHEN AN ITEM DECLARES AN ID, THAT ID IS THE CONTRACT. It used to fall through to word
|
|
120
|
+
// matching when the id was absent from the suite, so an item whose id nobody carried was
|
|
121
|
+
// still counted covered on two shared non-generic words with ANY scenario. Two claims written
|
|
122
|
+
// deliberately WITHOUT a scenario were both scored covered — one on
|
|
123
|
+
// "confirmation"/"buffered", the other on "name"/"blocked". `atomicLedger 100%` then meant
|
|
124
|
+
// "no claim is more than two words away from some scenario", which is not a coverage
|
|
125
|
+
// statement at all, and the ledger could never report the gap it exists to report (#622).
|
|
126
|
+
//
|
|
127
|
+
// Word matching survives only where there is nothing better: a prose claim with no id.
|
|
128
|
+
let isCovered: boolean;
|
|
129
|
+
if (item.id) {
|
|
130
|
+
isCovered = featLower.includes(item.id.toLowerCase());
|
|
131
|
+
if (isCovered) {
|
|
132
|
+
// An id carried only by @manual scenarios is ADDRESSED but not RUNNING.
|
|
133
|
+
const carriers = scenarios.filter((s) => (s.vpId ?? '').toLowerCase() === item.id!.toLowerCase()
|
|
134
|
+
|| s.haystack.includes(item.id!.toLowerCase()));
|
|
135
|
+
if (carriers.length > 0 && carriers.every((s) => s.manual)) manualOnly.push({ id: item.id, text: item.text });
|
|
136
|
+
// The count is in the CLAIM, not in the scenario title — so this is the only place both
|
|
137
|
+
// halves are in hand. `see all …` proves a set in one step and is exempt.
|
|
138
|
+
const n = claimedCount(item.text);
|
|
139
|
+
const auto = carriers.filter((s) => !s.manual);
|
|
140
|
+
if (n !== null && auto.length > 0 && !auto.some((s) => /\bsee all\b/.test(s.stepsText))) {
|
|
141
|
+
const asserted = Math.max(...auto.map((s) =>
|
|
142
|
+
(s.stepsText.match(/\bsee \[[^\]]+\][^|]*?\b(?:with|contains) \{\{/g) ?? []).length));
|
|
143
|
+
if (asserted > 0 && asserted < n) partial.push({ id: item.id, text: item.text, claimed: n, asserted });
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
} else {
|
|
87
147
|
const words = [...new Set((item.text.toLowerCase().match(/[a-z][a-z-]{3,}/g) || []).filter((w) => !GENERIC.has(w)))];
|
|
88
148
|
const need = Math.min(2, words.length);
|
|
89
149
|
isCovered = words.length > 0 && scenarios.some((s) => words.filter((w) => s.haystack.includes(w)).length >= need);
|
|
@@ -92,7 +152,10 @@ export function viewpointLedger(viewpointPath: string, scenarios: ScenarioInfo[]
|
|
|
92
152
|
else missing.push({ id: item.id, text: item.text });
|
|
93
153
|
}
|
|
94
154
|
|
|
95
|
-
return {
|
|
155
|
+
return {
|
|
156
|
+
hasViewpoint: true, total: items.length, covered,
|
|
157
|
+
ratio: items.length ? covered / items.length : 1, missing, manualOnly, partial,
|
|
158
|
+
};
|
|
96
159
|
}
|
|
97
160
|
|
|
98
161
|
/**
|
|
@@ -61,7 +61,11 @@ export function emitMobileRuntime(cwd: string, opts: MobileInitOptions): Scaffol
|
|
|
61
61
|
// appActivity is optional — leave empty when not provided so UiAutomator2
|
|
62
62
|
// auto-resolves the launcher activity from appPackage. Only pin when explicitly passed.
|
|
63
63
|
.replace('__APP_ACTIVITY__', opts.appActivity || '')
|
|
64
|
-
|
|
64
|
+
// GLOBAL replace: the template holds this placeholder twice (the connectivity self-heal
|
|
65
|
+
// helper + the 'appium:udid' capability, #615). String-form .replace would fill only the
|
|
66
|
+
// first and ship a literal '__ANDROID_UDID__' as the session udid. Do NOT globalize the
|
|
67
|
+
// others: __APP_ACTIVITY__/__IOS_VERSION__ keep a second occurrence as sentinel comparisons.
|
|
68
|
+
.replace(/__ANDROID_UDID__/g, opts.androidUdid || 'emulator-5554')
|
|
65
69
|
.replace('__IOS_BUNDLE_ID__', opts.iosBundleId || 'com.example.app')
|
|
66
70
|
.replace('__IOS_DEVICE__', opts.iosDevice || 'iPhone 16')
|
|
67
71
|
.replace('__IOS_VERSION__', opts.iosVersion || '18.0');
|
|
@@ -81,6 +81,29 @@ contract cannot stand in: the viewpoint is the one artifact that must stay **ind
|
|
|
81
81
|
you generated**. That is why a filled one is an INPUT you never rewrite, and why authoring it
|
|
82
82
|
FIRST — from the spec, before the scenarios exist — is the only order that keeps it honest.
|
|
83
83
|
|
|
84
|
+
**If you find yourself writing it AFTER the suite, stop and say so.** That is not a neutral
|
|
85
|
+
ordering choice: a yardstick derived from the scenarios reads 100% by construction, and anything
|
|
86
|
+
dropped along the way stops being missing from anything. `sungen audit` now reports
|
|
87
|
+
`VIEWPOINT-ADOPTED-POST-HOC` on the first sighting of a viewpoint alongside an existing suite and
|
|
88
|
+
holds `atomicLedger` + `traceability` as unverified until a human accepts it — so the honest move
|
|
89
|
+
is to write the claims from `spec.md` and the contract, mark plainly that they are a draft for QA
|
|
90
|
+
review, and let the QA revise before `--accept-viewpoint`.
|
|
91
|
+
|
|
92
|
+
**When you give a claim an id, a scenario must carry that id.** The id is the contract: a claim
|
|
93
|
+
whose id no scenario carries is reported MISSING, and word overlap will not rescue it. Several
|
|
94
|
+
scenarios may prove one claim, and a claim carried only by `@manual` scenarios is `ADDRESSED` but
|
|
95
|
+
not running (`VIEWPOINT-ITEM-MANUAL-ONLY`) — do not let the delivery read it as automated coverage.
|
|
96
|
+
|
|
97
|
+
**A claim that names a count must be proved that many times.** "restores all six buffered values"
|
|
98
|
+
asserted on four fields is `CLAIM-PARTIAL`: mapping a claim to a scenario shows it is addressed,
|
|
99
|
+
not that the oracle demonstrates it. Assert all of them, use one `see all …` set assertion, or
|
|
100
|
+
narrow the claim to what is actually proven.
|
|
101
|
+
|
|
102
|
+
**Priority comes from the declaration.** A scenario tagged `@high` under a viewpoint row that says
|
|
103
|
+
`Low` is `PRIORITY-CONTRADICTED` — a smoke run selects on the tag, so it picks up work the QA
|
|
104
|
+
deliberately deprioritised. Inherit the declared priority; if the tag is right, fix the
|
|
105
|
+
declaration instead of leaving both on record.
|
|
106
|
+
|
|
84
107
|
---
|
|
85
108
|
|
|
86
109
|
## Platform detection (do this FIRST)
|