@aarwitz/tapp 0.16.5 → 0.17.0-rc.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +35 -21
- package/Harness/OCQAHarnessUITests/ExplorerTests.swift +100 -31
- package/README.md +70 -67
- package/bin/tapp.js +265 -74
- package/browser/app.js +12 -6
- package/docs/BROWSER-PRODUCT.md +75 -0
- package/docs/PRODUCT-ENGINE.md +107 -0
- package/docs/application-model.md +278 -0
- package/docs/scenarios.md +95 -0
- package/mcp-server/src/android-driver.js +88 -3
- package/mcp-server/src/android-explorer.js +83 -14
- package/mcp-server/src/application-model.js +13 -9
- package/mcp-server/src/browser-product.js +1 -1
- package/mcp-server/src/ci-report.js +84 -62
- package/mcp-server/src/ci-setup.js +35 -5
- package/mcp-server/src/enrich.js +1 -1
- package/mcp-server/src/html-report.js +41 -7
- package/mcp-server/src/index.js +198 -72
- package/mcp-server/src/pr-selection.js +4 -3
- package/mcp-server/src/product-execution.js +1 -1
- package/mcp-server/src/product-operations.js +96 -7
- package/mcp-server/src/project-config.js +1 -2
- package/mcp-server/src/project-paths.js +5 -17
- package/mcp-server/src/release-contract.js +3 -3
- package/mcp-server/src/report.js +185 -51
- package/mcp-server/src/task-runtime.js +1 -1
- package/mcp-server/src/web-explorer.js +1 -1
- package/package.json +2 -2
- package/scripts/ci-gate.sh +11 -9
- package/scripts/flow_ai_judge.py +1 -1
- package/scripts/platform-gate.js +11 -5
- package/scripts/quick-capture.sh +72 -38
- package/scripts/run-flow.sh +1 -1
package/AGENTS.md
CHANGED
|
@@ -8,19 +8,28 @@ Android emulators/devices, plus (beta) web apps in a real browser.
|
|
|
8
8
|
Every core capability works as a plain command — no server, no config. `[target]` is
|
|
9
9
|
optional: with nothing, tapp finds + builds the Xcode project in the cwd (or falls back to
|
|
10
10
|
the app already on the simulator); it also accepts a repo dir, a `path/to/App.app`, a
|
|
11
|
-
bundle id, or (
|
|
11
|
+
bundle id, or (web) an http(s) URL. You never need to know a bundle id up front.
|
|
12
12
|
|
|
13
13
|
```bash
|
|
14
|
-
npx -y @aarwitz/tapp
|
|
14
|
+
npx -y @aarwitz/tapp explore [target] # autonomous exploration → findings + evidence (observation, not a gate; ≈ tapp_explore)
|
|
15
15
|
npx -y @aarwitz/tapp open [target] # launch + screen summary + screenshot saved to a file (≈ tapp_open_app)
|
|
16
16
|
npx -y @aarwitz/tapp tree [target] # accessibility tree, --json for every element (≈ tapp_ui_tree)
|
|
17
17
|
npx -y @aarwitz/tapp shot # screenshot the booted sim → file path (≈ tapp_screenshot)
|
|
18
18
|
npx -y @aarwitz/tapp apps # what's installed on the simulator, with bundle ids
|
|
19
19
|
npx -y @aarwitz/tapp build [dir] # build the app in an Xcode repo + install it (≈ tapp_build)
|
|
20
|
-
npx -y @aarwitz/tapp
|
|
20
|
+
npx -y @aarwitz/tapp explore app.apk --platform android --app-id com.acme.app
|
|
21
21
|
npx -y @aarwitz/tapp flow run .tapp/flows/smoke.yml # committed, keyless E2E replay
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
+
If repository onboarding detects multiple application targets, target detection is deterministic but
|
|
25
|
+
the choice is the user's. In a human TTY, Tapp displays a numbered selector and continues in the same
|
|
26
|
+
command. A non-interactive CLI prints the exact choices and exits before building. MCP returns
|
|
27
|
+
`reason: "target-selection-required"` with structured `choices[]` (`platform`, `name`, `sourcePath`,
|
|
28
|
+
`selector`, and exact `command`). **Do not pick one yourself.** Present those choices to the user
|
|
29
|
+
with the client's native multiple-choice question UI when available, then rerun using the selected
|
|
30
|
+
`--platform` and `--target`. Plain chat can list the same choices when the client has no question
|
|
31
|
+
widget.
|
|
32
|
+
|
|
24
33
|
For focused web evidence, `open` and `tree` accept one semantic interaction plus an async content
|
|
25
34
|
wait: `tapp open https://example.com --tap "Not now" --wait-for "Dashboard"`. Tapp waits for the
|
|
26
35
|
page to stabilize before capturing it and warns honestly if the bounded wait ends while it is still
|
|
@@ -30,21 +39,22 @@ loading or changing.
|
|
|
30
39
|
Code's Read tool, Codex's view-image), open the saved screenshot path the CLI prints —
|
|
31
40
|
that IS the screen. If you cannot (Cursor, VS Code Copilot), connect the MCP server
|
|
32
41
|
instead: its tool results carry the screenshot inline. Screen *recordings* are for the
|
|
33
|
-
human: `tapp
|
|
34
|
-
page — tell the user the report path so they can
|
|
42
|
+
human: on **iOS**, `tapp explore` records the full exploration and embeds it in the report.html
|
|
43
|
+
evidence page (Android does not currently record video) — tell the user the report path so they can
|
|
44
|
+
watch it.
|
|
35
45
|
|
|
36
46
|
The interactive session/record loop is MCP-only (it needs a long-lived process). Flow replay is
|
|
37
47
|
also available in the CLI. The rest of this playbook assumes the `tapp_*` MCP tools are connected. With
|
|
38
48
|
MCP, the no-bundle-id path is: `tapp_build {projectDir}` (auto-detects + builds +
|
|
39
|
-
installs, returns the bundle id) → `
|
|
49
|
+
installs, returns the bundle id) → `tapp_explore {appBundleId}`.
|
|
40
50
|
|
|
41
51
|
## Pick the right tool for the job
|
|
42
52
|
|
|
43
53
|
| The user wants… | Use | NOT |
|
|
44
54
|
|---|---|---|
|
|
45
|
-
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `
|
|
55
|
+
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `tapp_explore` (a full multi-minute QA exploration) |
|
|
46
56
|
| "Tap through / drive / fill a form / log in" | `tapp_session_start` → `session_act` loop | repeated `open_app` calls (cold relaunch each time) |
|
|
47
|
-
| "Is my app broken?
|
|
57
|
+
| "Is my app broken? Find bugs" | `tapp_explore` — `appBundleId` for iOS, `androidAppId` for Android, `url` for owned web apps; returns an observation (findings + evidence), not a ship verdict — gate a merge with the CI gate (`tapp ci` CLI / the GitHub Action) + a contract | a manual session (exploration is autonomous) |
|
|
48
58
|
| "Make this flow a repeatable test" | drive it in a session, then `tapp_flow_save`; replay with `tapp_flow_run` | re-driving it by hand every time |
|
|
49
59
|
| "What's on screen right now?" | `tapp_screenshot` / `tapp_ui_tree` | relaunching the app |
|
|
50
60
|
|
|
@@ -80,15 +90,19 @@ Rules that prevent 90% of failures:
|
|
|
80
90
|
auto-dismisses keyboards and retries), `not_found` (nothing matches — re-read the tree).
|
|
81
91
|
6. One session at a time. `session_start` always begins from a fresh app launch.
|
|
82
92
|
|
|
83
|
-
## Autonomous
|
|
93
|
+
## Autonomous exploration (`tapp_explore`)
|
|
94
|
+
|
|
95
|
+
Exploration **observes** — it returns an observation, NOT a ship verdict or score. To get a release
|
|
96
|
+
decision, run the deterministic gate (`tapp ci` / the GitHub Action). It applies versioned policy to
|
|
97
|
+
the findings + coverage plus any selected Flows/Scenarios/contracts and an optional target-scoped
|
|
98
|
+
baseline (both optional — a clean bootstrap can pass without them) and returns `pass | fail |
|
|
99
|
+
inconclusive`.
|
|
84
100
|
|
|
85
|
-
Returns `{
|
|
86
|
-
|
|
87
|
-
|
|
101
|
+
Returns `{kind:"tapp-exploration-run", headline, inconclusive, screensExplored, actionsPerformed,
|
|
102
|
+
findingCounts, findings[]}` — **no** `verdict`, `confidence`, or `releaseScore`. Each finding carries
|
|
103
|
+
`authority` (`deterministic` — marker-derived; model/vision findings would be `model-observed` and
|
|
104
|
+
advisory). Report the finding counts and coverage; do not invent a scalar or a ship verdict.
|
|
88
105
|
|
|
89
|
-
- `verdict`: `ready` | `caution` | `blocked`. Report it as-is; never soften a `blocked` or inflate a
|
|
90
|
-
`caution`. Judgment is deterministic for a given evidence trace, while adaptive exploration and
|
|
91
|
-
live target state can still change which evidence a run observes.
|
|
92
106
|
- `inconclusive: true` means the run couldn't see enough (crash on launch, login wall). That is
|
|
93
107
|
**not a pass** — tell the user what blocked exploration and what would unblock it.
|
|
94
108
|
- Login walls: pass `testEmail`/`testPassword` (auto-typed into login forms), `appLaunchArgs`
|
|
@@ -96,12 +110,12 @@ finding counts, advisory sampled-probe counts, and coverage instead of inventing
|
|
|
96
110
|
backend URL). If the result shows `inputFieldsEncountered` and you have no credentials, **ask
|
|
97
111
|
the user** for them rather than re-running blind.
|
|
98
112
|
- Diff two runs: pass the previous run's `findings` as `baselineFindings` → you get a
|
|
99
|
-
`regression`
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
not validate marketing claims against APIs, API field privacy, brand consistency, or
|
|
104
|
-
marketplace credibility unless an explicit reviewed test/contract or verifier covers them.
|
|
113
|
+
`regression` **comparison** (`new` / `persisting` / `resolved`) — an observation, not a gate. To gate
|
|
114
|
+
a merge on regressions, run the CI gate (`tapp ci` / the GitHub Action).
|
|
115
|
+
- On web, preserve scope: Tapp deterministically checks technical behavior such as failed requests,
|
|
116
|
+
missing assets, and placeholder links. Dead-control probes are budget-capped advisory findings.
|
|
117
|
+
Tapp does not validate marketing claims against APIs, API field privacy, brand consistency, or
|
|
118
|
+
subjective marketplace credibility unless an explicit reviewed test/contract or verifier covers them.
|
|
105
119
|
|
|
106
120
|
## Flows (deterministic E2E tests)
|
|
107
121
|
|
|
@@ -946,6 +946,7 @@ class ExplorerTests: XCTestCase {
|
|
|
946
946
|
var elementScreenPresence: [String: Set<String>] = [:]
|
|
947
947
|
var totalDistinctStates = 0
|
|
948
948
|
var recentStateHashes: [String] = []
|
|
949
|
+
var recentScreenTitles: [String] = []
|
|
949
950
|
var actionsSinceNewState = 0
|
|
950
951
|
var sameScreenStreak = 0
|
|
951
952
|
var screenTextEntryCount: [String: Int] = [:]
|
|
@@ -1238,6 +1239,15 @@ class ExplorerTests: XCTestCase {
|
|
|
1238
1239
|
let initialControlsJson = mapControlsJSON(initialElements)
|
|
1239
1240
|
print("OCQA_STATE:{\"screen\":\"\(escapeJSON(initialTitle))\",\"hash\":\"\(computeHash(initialElements))\",\"elements\":\(initialElements.count),\"action\":0,\"role\":\"\(escapeJSON(initialRole))\",\"summary\":\"\(escapeJSON(initialSummary))\",\"settled\":\(isScreenSettled() ? "true" : "false"),\"atext\":[\(initialAtext)],\"inputs\":[\(initialInputJson)],\"controls\":[\(initialControlsJson)]}")
|
|
1240
1241
|
|
|
1242
|
+
// The launch surface is real evidence even when root normalization immediately dismisses
|
|
1243
|
+
// it. Attach it before the first action so the HTML report's "every screen explored" claim
|
|
1244
|
+
// includes onboarding/login sheets rather than beginning at the post-dismiss destination.
|
|
1245
|
+
let initialScreenshot = app.screenshot()
|
|
1246
|
+
let initialAttachment = XCTAttachment(screenshot: initialScreenshot)
|
|
1247
|
+
initialAttachment.name = "state_0_\(initialTitle.replacingOccurrences(of: " ", with: "_"))"
|
|
1248
|
+
initialAttachment.lifetime = .keepAlways
|
|
1249
|
+
add(initialAttachment)
|
|
1250
|
+
|
|
1241
1251
|
navigateToRootScreen(actionCount: &actionCount)
|
|
1242
1252
|
|
|
1243
1253
|
// The first state is the true customer launch surface. Directed replay begins after
|
|
@@ -1386,8 +1396,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1386
1396
|
}
|
|
1387
1397
|
|
|
1388
1398
|
recentStateHashes.append(stateHash)
|
|
1399
|
+
recentScreenTitles.append(titleStr)
|
|
1389
1400
|
if recentStateHashes.count > 12 {
|
|
1390
1401
|
recentStateHashes.removeFirst(recentStateHashes.count - 12)
|
|
1402
|
+
recentScreenTitles.removeFirst(recentScreenTitles.count - 12)
|
|
1391
1403
|
}
|
|
1392
1404
|
|
|
1393
1405
|
if previousStateHash == stateHash {
|
|
@@ -1474,10 +1486,19 @@ class ExplorerTests: XCTestCase {
|
|
|
1474
1486
|
// is the fuller a11y text inventory that grounds the vision reviewer (see visionTextInventory).
|
|
1475
1487
|
let escapedTitle = escapeJSON(titleStr)
|
|
1476
1488
|
let settled = isScreenSettled()
|
|
1477
|
-
let
|
|
1489
|
+
let visibleTextInventory = visionTextInventory(elements)
|
|
1490
|
+
let atextJson = visibleTextInventory.map { "\"\(escapeJSON($0))\"" }.joined(separator: ",")
|
|
1478
1491
|
let controlsJson = mapControlsJSON(elements)
|
|
1479
1492
|
print("OCQA_STATE:{\"screen\":\"\(escapedTitle)\",\"hash\":\"\(stateHash)\",\"elements\":\(elements.count),\"action\":\(actionCount),\"role\":\"\(escapeJSON(screenRole))\",\"summary\":\"\(escapeJSON(screenSummary))\",\"settled\":\(settled ? "true" : "false"),\"atext\":[\(atextJson)],\"inputs\":[\(inputJsonArray)],\"controls\":[\(controlsJson)]}")
|
|
1480
1493
|
|
|
1494
|
+
// Signing out after a successful login is a completed auth cycle, not a navigation
|
|
1495
|
+
// trap and not lost-form-state. Stop cleanly instead of probing the root login screen
|
|
1496
|
+
// for an impossible back path or expecting credentials to persist after logout.
|
|
1497
|
+
if authSucceeded && detectedInputs.contains(where: { $0.secure }) {
|
|
1498
|
+
print("OCQA_STATE:auth_cycle_complete screen=\(escapedTitle) step=\(actionCount)")
|
|
1499
|
+
break
|
|
1500
|
+
}
|
|
1501
|
+
|
|
1481
1502
|
// ---- Persistence probe: on a fresh RE-ARRIVAL at a screen, fields we previously
|
|
1482
1503
|
// typed into (and verified visible in the a11y value) should still hold their value.
|
|
1483
1504
|
// An empty field here means the entered state was silently lost on navigation —
|
|
@@ -1497,6 +1518,8 @@ class ExplorerTests: XCTestCase {
|
|
|
1497
1518
|
if current.isEmpty || current == ph {
|
|
1498
1519
|
reportedPersistenceKeys.insert(memKey)
|
|
1499
1520
|
let t = "Entered value did not persist: '\(fieldKey)' on \(titleStr)"
|
|
1521
|
+
issues.append((type: "state_persistence", severity: "medium", title: t,
|
|
1522
|
+
desc: "Typed '\(typed)' into this field earlier in the run; after navigating away and returning, the field is empty — entered state was silently lost."))
|
|
1500
1523
|
print("OCQA_ISSUE:{\"type\":\"state_persistence\",\"severity\":\"medium\",\"title\":\"\(escapeJSON(t))\",\"screen\":\"\(escapedTitle)\",\"control\":\"\(escapeJSON(fieldKey))\",\"step\":\(actionCount),\"desc\":\"Typed '\(escapeJSON(typed))' into this field earlier in the run; after navigating away and returning, the field is empty — entered state was silently lost.\"}")
|
|
1501
1524
|
}
|
|
1502
1525
|
}
|
|
@@ -1591,14 +1614,14 @@ class ExplorerTests: XCTestCase {
|
|
|
1591
1614
|
// content-feed app: post detail + replies-loading spinner flagged app_hang HIGH).
|
|
1592
1615
|
let visibleTextCount = elements.filter { isStaticTextType($0.type) && normalizeVisibleText($0.label).count >= 3 }.count
|
|
1593
1616
|
if screenVisitCount[titleStr] ?? 0 <= 1, visibleTextCount <= 4,
|
|
1594
|
-
|
|
1617
|
+
hasIndeterminateLoadingIndicator() {
|
|
1595
1618
|
let loadingKey = "loading:\(titleStr)"
|
|
1596
1619
|
if !reportedIssueKeys.contains(loadingKey) {
|
|
1597
1620
|
var resolved = false
|
|
1598
1621
|
let deadline = Date().addingTimeInterval(8.0)
|
|
1599
1622
|
while Date() < deadline {
|
|
1600
1623
|
Thread.sleep(forTimeInterval: 1.0)
|
|
1601
|
-
if !(
|
|
1624
|
+
if !hasIndeterminateLoadingIndicator() {
|
|
1602
1625
|
resolved = true
|
|
1603
1626
|
break
|
|
1604
1627
|
}
|
|
@@ -1658,7 +1681,13 @@ class ExplorerTests: XCTestCase {
|
|
|
1658
1681
|
|
|
1659
1682
|
// ---- Blank-screen detection ----
|
|
1660
1683
|
// Distinguish between "no a11y labels / custom UI" vs genuinely empty.
|
|
1661
|
-
|
|
1684
|
+
// A system Back/Close affordance is navigation chrome, not screen content. Treat a
|
|
1685
|
+
// destination whose only usable control is that chrome as blank too; otherwise an
|
|
1686
|
+
// EmptyView pushed by NavigationStack looks like a healthy one-control screen.
|
|
1687
|
+
let contentInteractables = interactable.filter {
|
|
1688
|
+
!isNavBackButton($0) && !isLikelyGlobalNavigation($0, screenBounds: screenBounds)
|
|
1689
|
+
}
|
|
1690
|
+
if visibleTextInventory.isEmpty && contentInteractables.isEmpty {
|
|
1662
1691
|
let blankKey = "blank:\(titleStr)"
|
|
1663
1692
|
let blankCount = (actionCounts[blankKey] ?? 0) + 1
|
|
1664
1693
|
actionCounts[blankKey] = blankCount
|
|
@@ -1690,16 +1719,26 @@ class ExplorerTests: XCTestCase {
|
|
|
1690
1719
|
}
|
|
1691
1720
|
|
|
1692
1721
|
// ---- Navigation-loop detection ----
|
|
1693
|
-
//
|
|
1722
|
+
// A cycle must actually move across distinct states. Four identical reads satisfy the
|
|
1723
|
+
// arithmetic shape A,A,A,A of the old period-2 check, which mislabeled ordinary
|
|
1724
|
+
// scroll/probe recovery on a stable screen as a navigation loop.
|
|
1694
1725
|
if recentStateHashes.count >= 6 {
|
|
1695
1726
|
let recent = recentStateHashes
|
|
1727
|
+
// Distinct structural hashes are not enough: a list and its detail rows can share
|
|
1728
|
+
// one navigation title and alternate A/B while the explorer intentionally samples
|
|
1729
|
+
// different rows. Calling that a navigation loop is a false positive. Require the
|
|
1730
|
+
// cycle to cross distinct user-visible screen titles as well.
|
|
1696
1731
|
let hasLoop2 = recent.count >= 4 &&
|
|
1697
1732
|
recent[recent.count - 1] == recent[recent.count - 3] &&
|
|
1698
|
-
recent[recent.count - 2] == recent[recent.count - 4]
|
|
1733
|
+
recent[recent.count - 2] == recent[recent.count - 4] &&
|
|
1734
|
+
Set(recent.suffix(2)).count == 2 &&
|
|
1735
|
+
Set(recentScreenTitles.suffix(2)).count == 2
|
|
1699
1736
|
let hasLoop3 = recent.count >= 6 &&
|
|
1700
1737
|
recent[recent.count - 1] == recent[recent.count - 4] &&
|
|
1701
1738
|
recent[recent.count - 2] == recent[recent.count - 5] &&
|
|
1702
|
-
recent[recent.count - 3] == recent[recent.count - 6]
|
|
1739
|
+
recent[recent.count - 3] == recent[recent.count - 6] &&
|
|
1740
|
+
Set(recent.suffix(3)).count == 3 &&
|
|
1741
|
+
Set(recentScreenTitles.suffix(3)).count == 3
|
|
1703
1742
|
if (hasLoop2 || hasLoop3) && !(authSucceeded && detectedInputs.contains { $0.secure }) {
|
|
1704
1743
|
let loopKey = "nav_loop:\(titleStr)"
|
|
1705
1744
|
if actionCounts[loopKey] == nil {
|
|
@@ -1711,17 +1750,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1711
1750
|
}
|
|
1712
1751
|
}
|
|
1713
1752
|
|
|
1714
|
-
//
|
|
1715
|
-
//
|
|
1716
|
-
//
|
|
1717
|
-
|
|
1718
|
-
let unrespKey = "unresponsive:\(titleStr)"
|
|
1719
|
-
if actionCounts[unrespKey] == nil {
|
|
1720
|
-
issues.append((type: "unresponsive_element", severity: "medium", title: "Unresponsive UI on \(titleStr)", desc: "Actions are not changing app state — possible frozen or broken screen"))
|
|
1721
|
-
print("OCQA_ISSUE:{\"type\":\"unresponsive_element\",\"severity\":\"medium\",\"title\":\"Unresponsive UI\",\"screen\":\"\(escapedTitle)\",\"repeated_state_count\":\(repeatedStateCount),\"step\":\(actionCount)}")
|
|
1722
|
-
actionCounts[unrespKey] = 1
|
|
1723
|
-
}
|
|
1724
|
-
}
|
|
1753
|
+
// Do not infer an unresponsive app merely from an unchanged state streak: recovery
|
|
1754
|
+
// gestures (scroll, carousel probe, center probe) are expected to be no-ops on many
|
|
1755
|
+
// healthy screens. Labeled controls have a stronger detector below: a direct tap plus
|
|
1756
|
+
// two delayed, content-signature reads. Hangs have their own time-based detector.
|
|
1725
1757
|
|
|
1726
1758
|
if interactable.count < 3 {
|
|
1727
1759
|
print("OCQA_STATE:low_interactable screen=\(escapedTitle) total=\(elements.count) interactable=\(interactable.count) global=\(globalNavElements.count) nonGlobal=\(nonGlobalCandidates.count)")
|
|
@@ -1757,10 +1789,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1757
1789
|
break
|
|
1758
1790
|
}
|
|
1759
1791
|
|
|
1760
|
-
|
|
1761
|
-
|
|
1762
|
-
|
|
1763
|
-
|
|
1792
|
+
// Exhausting Tapp's untried candidate pool is not itself a user-visible dead end:
|
|
1793
|
+
// leaf screens commonly have only a working Back control that was already mapped.
|
|
1794
|
+
// Recover first; only the stronger navigation-trap path below emits a finding when
|
|
1795
|
+
// every real back/dismiss route fails.
|
|
1764
1796
|
// tryGoBack does swipe-down as its last resort (sheet dismiss)
|
|
1765
1797
|
let preBackTitle = titleStr
|
|
1766
1798
|
let backWorked = tryGoBack()
|
|
@@ -1772,6 +1804,7 @@ class ExplorerTests: XCTestCase {
|
|
|
1772
1804
|
print("OCQA_ACTION:{\"type\":\"back\",\"reason\":\"dead_end_escape\",\"from\":\"\(escapedTitle)\",\"to\":\"\(escapeJSON(postTitle))\",\"step\":\(actionCount),\"screen\":\"\(escapedTitle)\",\"narrative\":\"\(escapeJSON(recoveryNarrative("back_dead_end", screen: titleStr, to: postTitle)))\"}")
|
|
1773
1805
|
continue
|
|
1774
1806
|
}
|
|
1807
|
+
if actionCount >= maxActions { break }
|
|
1775
1808
|
// Swipe right (back gesture) as another option
|
|
1776
1809
|
let swipeStart = app.coordinate(withNormalizedOffset: CGVector(dx: 0.02, dy: 0.5))
|
|
1777
1810
|
let swipeEnd = app.coordinate(withNormalizedOffset: CGVector(dx: 0.8, dy: 0.5))
|
|
@@ -2047,6 +2080,7 @@ class ExplorerTests: XCTestCase {
|
|
|
2047
2080
|
}
|
|
2048
2081
|
// Back didn't change screens — fall through to global nav
|
|
2049
2082
|
print("OCQA_ACTION:{\"type\":\"back\",\"reason\":\"screen_exhausted_failed\",\"screen\":\"\(escapedTitle)\",\"step\":\(actionCount),\"narrative\":\"\(escapeJSON(recoveryNarrative("back_failed", screen: titleStr)))\"}")
|
|
2083
|
+
if actionCount >= maxActions { break }
|
|
2050
2084
|
// Stuck on this screen — use global navigation (tab bar) to reach unexplored areas
|
|
2051
2085
|
let globalNav = interactable
|
|
2052
2086
|
.filter { isLikelyGlobalNavigation($0, screenBounds: screenBounds) }
|
|
@@ -2247,12 +2281,21 @@ class ExplorerTests: XCTestCase {
|
|
|
2247
2281
|
// the crash itself goes unreported. app.state is non-throwing even when the app is dead.
|
|
2248
2282
|
// (Found on a real app: a login submit terminated the app; the run limped on but emitted
|
|
2249
2283
|
// no crash finding.) Try one relaunch to distinguish a hard crash from a transient exit.
|
|
2250
|
-
|
|
2251
|
-
|
|
2284
|
+
let stateAfterAction = app.state
|
|
2285
|
+
if stateAfterAction != .runningForeground {
|
|
2286
|
+
print("OCQA_STATE:app_left_foreground step=\(actionCount) state=\(stateAfterAction.rawValue)")
|
|
2287
|
+
let crashKey = "crash:\(titleStr)|\(key)"
|
|
2288
|
+
// A terminated process is already proof of an in-run crash. Relaunching it may
|
|
2289
|
+
// succeed, but that must not erase the user-visible failure that just happened.
|
|
2290
|
+
if stateAfterAction == .notRunning && !reportedIssueKeys.contains(crashKey) {
|
|
2291
|
+
reportedIssueKeys.insert(crashKey)
|
|
2292
|
+
issues.append((type: "crash", severity: "critical", title: "App crashed after \(actionType) on \(titleStr)",
|
|
2293
|
+
desc: "The app process terminated after \(actionType) '\(targetName)' on '\(titleStr)'."))
|
|
2294
|
+
print("OCQA_ISSUE:{\"type\":\"crash\",\"severity\":\"critical\",\"title\":\"\(escapeJSON("App crashed after \(actionType) on \(titleStr)"))\",\"screen\":\"\(escapedTitle)\",\"control\":\"\(escapedTarget)\",\"step\":\(actionCount)}")
|
|
2295
|
+
}
|
|
2252
2296
|
app.activate()
|
|
2253
2297
|
Thread.sleep(forTimeInterval: 3.0)
|
|
2254
2298
|
if app.state != .runningForeground {
|
|
2255
|
-
let crashKey = "crash:\(titleStr)|\(key)"
|
|
2256
2299
|
if !reportedIssueKeys.contains(crashKey) {
|
|
2257
2300
|
reportedIssueKeys.insert(crashKey)
|
|
2258
2301
|
issues.append((type: "crash", severity: "critical", title: "App crashed after \(actionType) on \(titleStr)",
|
|
@@ -2356,16 +2399,28 @@ class ExplorerTests: XCTestCase {
|
|
|
2356
2399
|
|
|
2357
2400
|
// ---- App left foreground / crash detection ----
|
|
2358
2401
|
// Check both .exists and .state — external links may cause either to fail
|
|
2359
|
-
let
|
|
2402
|
+
let stateAfterDelayedChecks = app.state
|
|
2403
|
+
let appInForeground = stateAfterDelayedChecks == .runningForeground
|
|
2360
2404
|
if !appInForeground || !app.exists {
|
|
2361
|
-
print("OCQA_STATE:app_left_foreground step=\(actionCount) state=\(
|
|
2405
|
+
print("OCQA_STATE:app_left_foreground step=\(actionCount) state=\(stateAfterDelayedChecks.rawValue)")
|
|
2406
|
+
let crashKey = "crash:\(titleStr)|\(key)"
|
|
2407
|
+
if stateAfterDelayedChecks == .notRunning && !reportedIssueKeys.contains(crashKey) {
|
|
2408
|
+
reportedIssueKeys.insert(crashKey)
|
|
2409
|
+
issues.append((type: "crash", severity: "critical",
|
|
2410
|
+
title: "App crashed after \(actionType) on \(titleStr)",
|
|
2411
|
+
desc: "The app process terminated after: \(actionDesc)"))
|
|
2412
|
+
print("OCQA_ISSUE:{\"type\":\"crash\",\"severity\":\"critical\",\"title\":\"\(escapeJSON("App crashed after \(actionType) on \(titleStr)"))\",\"screen\":\"\(escapedTitle)\",\"control\":\"\(escapedTarget)\",\"step\":\(actionCount)}")
|
|
2413
|
+
}
|
|
2362
2414
|
app.activate()
|
|
2363
2415
|
Thread.sleep(forTimeInterval: 3.0)
|
|
2364
2416
|
if app.state != .runningForeground {
|
|
2365
|
-
|
|
2366
|
-
|
|
2367
|
-
|
|
2368
|
-
|
|
2417
|
+
if !reportedIssueKeys.contains(crashKey) {
|
|
2418
|
+
reportedIssueKeys.insert(crashKey)
|
|
2419
|
+
issues.append((type: "crash", severity: "critical",
|
|
2420
|
+
title: "App not recoverable",
|
|
2421
|
+
desc: "App left foreground after: \(actionDesc)"))
|
|
2422
|
+
print("OCQA_ISSUE:{\"type\":\"crash\",\"severity\":\"critical\",\"title\":\"App not recoverable\",\"action\":\"\(escapedTarget)\",\"step\":\(actionCount)}")
|
|
2423
|
+
}
|
|
2369
2424
|
break
|
|
2370
2425
|
}
|
|
2371
2426
|
print("OCQA_STATE:app_reactivated step=\(actionCount)")
|
|
@@ -3679,6 +3734,20 @@ class ExplorerTests: XCTestCase {
|
|
|
3679
3734
|
return kb.exists ? kb.frame : .zero
|
|
3680
3735
|
}
|
|
3681
3736
|
|
|
3737
|
+
/// Activity indicators are inherently indeterminate. `ProgressIndicator`, however, is also the
|
|
3738
|
+
/// XCTest type for legitimate determinate progress bars (loyalty points, upload percentage,
|
|
3739
|
+
/// onboarding completion). Only value-less/loading-valued progress indicators are hang signals.
|
|
3740
|
+
private func hasIndeterminateLoadingIndicator() -> Bool {
|
|
3741
|
+
if app.activityIndicators.allElementsBoundByIndex.contains(where: { $0.exists && $0.frame.width > 0 && $0.frame.height > 0 }) {
|
|
3742
|
+
return true
|
|
3743
|
+
}
|
|
3744
|
+
return app.progressIndicators.allElementsBoundByIndex.contains { indicator in
|
|
3745
|
+
guard indicator.exists, indicator.frame.width > 0, indicator.frame.height > 0 else { return false }
|
|
3746
|
+
let value = (indicator.value as? String ?? "").trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
|
|
3747
|
+
return value.isEmpty || value == "in progress" || value == "loading"
|
|
3748
|
+
}
|
|
3749
|
+
}
|
|
3750
|
+
|
|
3682
3751
|
/// True when the screen is in a "settled" resting state — no on-screen keyboard and no open
|
|
3683
3752
|
/// transient overlay (menu / dropdown / popover / sheet / picker wheel). A screenshot taken while
|
|
3684
3753
|
/// one of these is up is inherently ambiguous to a visual reviewer (the keyboard "covers" the
|