@aarwitz/tapp 0.16.5 → 0.17.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +26 -21
- package/Harness/OCQAHarnessUITests/ExplorerTests.swift +62 -22
- package/README.md +64 -67
- package/bin/tapp.js +168 -45
- package/browser/app.js +12 -6
- package/docs/BROWSER-PRODUCT.md +75 -0
- package/docs/PRODUCT-ENGINE.md +107 -0
- package/docs/application-model.md +271 -0
- package/docs/scenarios.md +95 -0
- package/mcp-server/src/android-driver.js +15 -2
- package/mcp-server/src/android-explorer.js +54 -12
- package/mcp-server/src/application-model.js +9 -8
- package/mcp-server/src/browser-product.js +1 -1
- package/mcp-server/src/ci-report.js +84 -62
- package/mcp-server/src/ci-setup.js +35 -5
- package/mcp-server/src/enrich.js +1 -1
- package/mcp-server/src/html-report.js +20 -6
- package/mcp-server/src/index.js +182 -67
- package/mcp-server/src/pr-selection.js +4 -3
- package/mcp-server/src/product-execution.js +1 -1
- package/mcp-server/src/product-operations.js +2 -2
- package/mcp-server/src/project-config.js +1 -2
- package/mcp-server/src/project-paths.js +5 -17
- package/mcp-server/src/release-contract.js +3 -3
- package/mcp-server/src/report.js +185 -51
- package/mcp-server/src/task-runtime.js +1 -1
- package/mcp-server/src/web-explorer.js +1 -1
- package/package.json +2 -2
- package/scripts/ci-gate.sh +9 -8
- package/scripts/platform-gate.js +11 -5
- package/scripts/quick-capture.sh +72 -38
- package/scripts/run-flow.sh +1 -1
package/AGENTS.md
CHANGED
|
@@ -8,16 +8,16 @@ Android emulators/devices, plus (beta) web apps in a real browser.
|
|
|
8
8
|
Every core capability works as a plain command — no server, no config. `[target]` is
|
|
9
9
|
optional: with nothing, tapp finds + builds the Xcode project in the cwd (or falls back to
|
|
10
10
|
the app already on the simulator); it also accepts a repo dir, a `path/to/App.app`, a
|
|
11
|
-
bundle id, or (
|
|
11
|
+
bundle id, or (web) an http(s) URL. You never need to know a bundle id up front.
|
|
12
12
|
|
|
13
13
|
```bash
|
|
14
|
-
npx -y @aarwitz/tapp
|
|
14
|
+
npx -y @aarwitz/tapp explore [target] # autonomous exploration → findings + evidence (observation, not a gate; ≈ tapp_explore)
|
|
15
15
|
npx -y @aarwitz/tapp open [target] # launch + screen summary + screenshot saved to a file (≈ tapp_open_app)
|
|
16
16
|
npx -y @aarwitz/tapp tree [target] # accessibility tree, --json for every element (≈ tapp_ui_tree)
|
|
17
17
|
npx -y @aarwitz/tapp shot # screenshot the booted sim → file path (≈ tapp_screenshot)
|
|
18
18
|
npx -y @aarwitz/tapp apps # what's installed on the simulator, with bundle ids
|
|
19
19
|
npx -y @aarwitz/tapp build [dir] # build the app in an Xcode repo + install it (≈ tapp_build)
|
|
20
|
-
npx -y @aarwitz/tapp
|
|
20
|
+
npx -y @aarwitz/tapp explore app.apk --platform android --app-id com.acme.app
|
|
21
21
|
npx -y @aarwitz/tapp flow run .tapp/flows/smoke.yml # committed, keyless E2E replay
|
|
22
22
|
```
|
|
23
23
|
|
|
@@ -30,21 +30,22 @@ loading or changing.
|
|
|
30
30
|
Code's Read tool, Codex's view-image), open the saved screenshot path the CLI prints —
|
|
31
31
|
that IS the screen. If you cannot (Cursor, VS Code Copilot), connect the MCP server
|
|
32
32
|
instead: its tool results carry the screenshot inline. Screen *recordings* are for the
|
|
33
|
-
human: `tapp
|
|
34
|
-
page — tell the user the report path so they can
|
|
33
|
+
human: on **iOS**, `tapp explore` records the full exploration and embeds it in the report.html
|
|
34
|
+
evidence page (Android does not currently record video) — tell the user the report path so they can
|
|
35
|
+
watch it.
|
|
35
36
|
|
|
36
37
|
The interactive session/record loop is MCP-only (it needs a long-lived process). Flow replay is
|
|
37
38
|
also available in the CLI. The rest of this playbook assumes the `tapp_*` MCP tools are connected. With
|
|
38
39
|
MCP, the no-bundle-id path is: `tapp_build {projectDir}` (auto-detects + builds +
|
|
39
|
-
installs, returns the bundle id) → `
|
|
40
|
+
installs, returns the bundle id) → `tapp_explore {appBundleId}`.
|
|
40
41
|
|
|
41
42
|
## Pick the right tool for the job
|
|
42
43
|
|
|
43
44
|
| The user wants… | Use | NOT |
|
|
44
45
|
|---|---|---|
|
|
45
|
-
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `
|
|
46
|
+
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `tapp_explore` (a full multi-minute QA exploration) |
|
|
46
47
|
| "Tap through / drive / fill a form / log in" | `tapp_session_start` → `session_act` loop | repeated `open_app` calls (cold relaunch each time) |
|
|
47
|
-
| "Is my app broken?
|
|
48
|
+
| "Is my app broken? Find bugs" | `tapp_explore` — `appBundleId` for iOS, `androidAppId` for Android, `url` for owned web apps; returns an observation (findings + evidence), not a ship verdict — gate a merge with the CI gate (`tapp ci` CLI / the GitHub Action) + a contract | a manual session (exploration is autonomous) |
|
|
48
49
|
| "Make this flow a repeatable test" | drive it in a session, then `tapp_flow_save`; replay with `tapp_flow_run` | re-driving it by hand every time |
|
|
49
50
|
| "What's on screen right now?" | `tapp_screenshot` / `tapp_ui_tree` | relaunching the app |
|
|
50
51
|
|
|
@@ -80,15 +81,19 @@ Rules that prevent 90% of failures:
|
|
|
80
81
|
auto-dismisses keyboards and retries), `not_found` (nothing matches — re-read the tree).
|
|
81
82
|
6. One session at a time. `session_start` always begins from a fresh app launch.
|
|
82
83
|
|
|
83
|
-
## Autonomous
|
|
84
|
+
## Autonomous exploration (`tapp_explore`)
|
|
84
85
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
86
|
+
Exploration **observes** — it returns an observation, NOT a ship verdict or score. To get a release
|
|
87
|
+
decision, run the deterministic gate (`tapp ci` / the GitHub Action). It applies versioned policy to
|
|
88
|
+
the findings + coverage plus any selected Flows/Scenarios/contracts and an optional target-scoped
|
|
89
|
+
baseline (both optional — a clean bootstrap can pass without them) and returns `pass | fail |
|
|
90
|
+
inconclusive`.
|
|
91
|
+
|
|
92
|
+
Returns `{kind:"tapp-exploration-run", headline, inconclusive, screensExplored, actionsPerformed,
|
|
93
|
+
findingCounts, findings[]}` — **no** `verdict`, `confidence`, or `releaseScore`. Each finding carries
|
|
94
|
+
`authority` (`deterministic` — marker-derived; model/vision findings would be `model-observed` and
|
|
95
|
+
advisory). Report the finding counts and coverage; do not invent a scalar or a ship verdict.
|
|
88
96
|
|
|
89
|
-
- `verdict`: `ready` | `caution` | `blocked`. Report it as-is; never soften a `blocked` or inflate a
|
|
90
|
-
`caution`. Judgment is deterministic for a given evidence trace, while adaptive exploration and
|
|
91
|
-
live target state can still change which evidence a run observes.
|
|
92
97
|
- `inconclusive: true` means the run couldn't see enough (crash on launch, login wall). That is
|
|
93
98
|
**not a pass** — tell the user what blocked exploration and what would unblock it.
|
|
94
99
|
- Login walls: pass `testEmail`/`testPassword` (auto-typed into login forms), `appLaunchArgs`
|
|
@@ -96,12 +101,12 @@ finding counts, advisory sampled-probe counts, and coverage instead of inventing
|
|
|
96
101
|
backend URL). If the result shows `inputFieldsEncountered` and you have no credentials, **ask
|
|
97
102
|
the user** for them rather than re-running blind.
|
|
98
103
|
- Diff two runs: pass the previous run's `findings` as `baselineFindings` → you get a
|
|
99
|
-
`regression`
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
not validate marketing claims against APIs, API field privacy, brand consistency, or
|
|
104
|
-
marketplace credibility unless an explicit reviewed test/contract or verifier covers them.
|
|
104
|
+
`regression` **comparison** (`new` / `persisting` / `resolved`) — an observation, not a gate. To gate
|
|
105
|
+
a merge on regressions, run the CI gate (`tapp ci` / the GitHub Action).
|
|
106
|
+
- On web, preserve scope: Tapp deterministically checks technical behavior such as failed requests,
|
|
107
|
+
missing assets, and placeholder links. Dead-control probes are budget-capped advisory findings.
|
|
108
|
+
Tapp does not validate marketing claims against APIs, API field privacy, brand consistency, or
|
|
109
|
+
subjective marketplace credibility unless an explicit reviewed test/contract or verifier covers them.
|
|
105
110
|
|
|
106
111
|
## Flows (deterministic E2E tests)
|
|
107
112
|
|
|
@@ -946,6 +946,7 @@ class ExplorerTests: XCTestCase {
|
|
|
946
946
|
var elementScreenPresence: [String: Set<String>] = [:]
|
|
947
947
|
var totalDistinctStates = 0
|
|
948
948
|
var recentStateHashes: [String] = []
|
|
949
|
+
var recentScreenTitles: [String] = []
|
|
949
950
|
var actionsSinceNewState = 0
|
|
950
951
|
var sameScreenStreak = 0
|
|
951
952
|
var screenTextEntryCount: [String: Int] = [:]
|
|
@@ -1238,6 +1239,15 @@ class ExplorerTests: XCTestCase {
|
|
|
1238
1239
|
let initialControlsJson = mapControlsJSON(initialElements)
|
|
1239
1240
|
print("OCQA_STATE:{\"screen\":\"\(escapeJSON(initialTitle))\",\"hash\":\"\(computeHash(initialElements))\",\"elements\":\(initialElements.count),\"action\":0,\"role\":\"\(escapeJSON(initialRole))\",\"summary\":\"\(escapeJSON(initialSummary))\",\"settled\":\(isScreenSettled() ? "true" : "false"),\"atext\":[\(initialAtext)],\"inputs\":[\(initialInputJson)],\"controls\":[\(initialControlsJson)]}")
|
|
1240
1241
|
|
|
1242
|
+
// The launch surface is real evidence even when root normalization immediately dismisses
|
|
1243
|
+
// it. Attach it before the first action so the HTML report's "every screen explored" claim
|
|
1244
|
+
// includes onboarding/login sheets rather than beginning at the post-dismiss destination.
|
|
1245
|
+
let initialScreenshot = app.screenshot()
|
|
1246
|
+
let initialAttachment = XCTAttachment(screenshot: initialScreenshot)
|
|
1247
|
+
initialAttachment.name = "state_0_\(initialTitle.replacingOccurrences(of: " ", with: "_"))"
|
|
1248
|
+
initialAttachment.lifetime = .keepAlways
|
|
1249
|
+
add(initialAttachment)
|
|
1250
|
+
|
|
1241
1251
|
navigateToRootScreen(actionCount: &actionCount)
|
|
1242
1252
|
|
|
1243
1253
|
// The first state is the true customer launch surface. Directed replay begins after
|
|
@@ -1386,8 +1396,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1386
1396
|
}
|
|
1387
1397
|
|
|
1388
1398
|
recentStateHashes.append(stateHash)
|
|
1399
|
+
recentScreenTitles.append(titleStr)
|
|
1389
1400
|
if recentStateHashes.count > 12 {
|
|
1390
1401
|
recentStateHashes.removeFirst(recentStateHashes.count - 12)
|
|
1402
|
+
recentScreenTitles.removeFirst(recentScreenTitles.count - 12)
|
|
1391
1403
|
}
|
|
1392
1404
|
|
|
1393
1405
|
if previousStateHash == stateHash {
|
|
@@ -1474,10 +1486,19 @@ class ExplorerTests: XCTestCase {
|
|
|
1474
1486
|
// is the fuller a11y text inventory that grounds the vision reviewer (see visionTextInventory).
|
|
1475
1487
|
let escapedTitle = escapeJSON(titleStr)
|
|
1476
1488
|
let settled = isScreenSettled()
|
|
1477
|
-
let
|
|
1489
|
+
let visibleTextInventory = visionTextInventory(elements)
|
|
1490
|
+
let atextJson = visibleTextInventory.map { "\"\(escapeJSON($0))\"" }.joined(separator: ",")
|
|
1478
1491
|
let controlsJson = mapControlsJSON(elements)
|
|
1479
1492
|
print("OCQA_STATE:{\"screen\":\"\(escapedTitle)\",\"hash\":\"\(stateHash)\",\"elements\":\(elements.count),\"action\":\(actionCount),\"role\":\"\(escapeJSON(screenRole))\",\"summary\":\"\(escapeJSON(screenSummary))\",\"settled\":\(settled ? "true" : "false"),\"atext\":[\(atextJson)],\"inputs\":[\(inputJsonArray)],\"controls\":[\(controlsJson)]}")
|
|
1480
1493
|
|
|
1494
|
+
// Signing out after a successful login is a completed auth cycle, not a navigation
|
|
1495
|
+
// trap and not lost-form-state. Stop cleanly instead of probing the root login screen
|
|
1496
|
+
// for an impossible back path or expecting credentials to persist after logout.
|
|
1497
|
+
if authSucceeded && detectedInputs.contains(where: { $0.secure }) {
|
|
1498
|
+
print("OCQA_STATE:auth_cycle_complete screen=\(escapedTitle) step=\(actionCount)")
|
|
1499
|
+
break
|
|
1500
|
+
}
|
|
1501
|
+
|
|
1481
1502
|
// ---- Persistence probe: on a fresh RE-ARRIVAL at a screen, fields we previously
|
|
1482
1503
|
// typed into (and verified visible in the a11y value) should still hold their value.
|
|
1483
1504
|
// An empty field here means the entered state was silently lost on navigation —
|
|
@@ -1591,14 +1612,14 @@ class ExplorerTests: XCTestCase {
|
|
|
1591
1612
|
// content-feed app: post detail + replies-loading spinner flagged app_hang HIGH).
|
|
1592
1613
|
let visibleTextCount = elements.filter { isStaticTextType($0.type) && normalizeVisibleText($0.label).count >= 3 }.count
|
|
1593
1614
|
if screenVisitCount[titleStr] ?? 0 <= 1, visibleTextCount <= 4,
|
|
1594
|
-
|
|
1615
|
+
hasIndeterminateLoadingIndicator() {
|
|
1595
1616
|
let loadingKey = "loading:\(titleStr)"
|
|
1596
1617
|
if !reportedIssueKeys.contains(loadingKey) {
|
|
1597
1618
|
var resolved = false
|
|
1598
1619
|
let deadline = Date().addingTimeInterval(8.0)
|
|
1599
1620
|
while Date() < deadline {
|
|
1600
1621
|
Thread.sleep(forTimeInterval: 1.0)
|
|
1601
|
-
if !(
|
|
1622
|
+
if !hasIndeterminateLoadingIndicator() {
|
|
1602
1623
|
resolved = true
|
|
1603
1624
|
break
|
|
1604
1625
|
}
|
|
@@ -1658,7 +1679,7 @@ class ExplorerTests: XCTestCase {
|
|
|
1658
1679
|
|
|
1659
1680
|
// ---- Blank-screen detection ----
|
|
1660
1681
|
// Distinguish between "no a11y labels / custom UI" vs genuinely empty.
|
|
1661
|
-
if
|
|
1682
|
+
if visibleTextInventory.isEmpty && interactable.count == 0 {
|
|
1662
1683
|
let blankKey = "blank:\(titleStr)"
|
|
1663
1684
|
let blankCount = (actionCounts[blankKey] ?? 0) + 1
|
|
1664
1685
|
actionCounts[blankKey] = blankCount
|
|
@@ -1690,16 +1711,26 @@ class ExplorerTests: XCTestCase {
|
|
|
1690
1711
|
}
|
|
1691
1712
|
|
|
1692
1713
|
// ---- Navigation-loop detection ----
|
|
1693
|
-
//
|
|
1714
|
+
// A cycle must actually move across distinct states. Four identical reads satisfy the
|
|
1715
|
+
// arithmetic shape A,A,A,A of the old period-2 check, which mislabeled ordinary
|
|
1716
|
+
// scroll/probe recovery on a stable screen as a navigation loop.
|
|
1694
1717
|
if recentStateHashes.count >= 6 {
|
|
1695
1718
|
let recent = recentStateHashes
|
|
1719
|
+
// Distinct structural hashes are not enough: a list and its detail rows can share
|
|
1720
|
+
// one navigation title and alternate A/B while the explorer intentionally samples
|
|
1721
|
+
// different rows. Calling that a navigation loop is a false positive. Require the
|
|
1722
|
+
// cycle to cross distinct user-visible screen titles as well.
|
|
1696
1723
|
let hasLoop2 = recent.count >= 4 &&
|
|
1697
1724
|
recent[recent.count - 1] == recent[recent.count - 3] &&
|
|
1698
|
-
recent[recent.count - 2] == recent[recent.count - 4]
|
|
1725
|
+
recent[recent.count - 2] == recent[recent.count - 4] &&
|
|
1726
|
+
Set(recent.suffix(2)).count == 2 &&
|
|
1727
|
+
Set(recentScreenTitles.suffix(2)).count == 2
|
|
1699
1728
|
let hasLoop3 = recent.count >= 6 &&
|
|
1700
1729
|
recent[recent.count - 1] == recent[recent.count - 4] &&
|
|
1701
1730
|
recent[recent.count - 2] == recent[recent.count - 5] &&
|
|
1702
|
-
recent[recent.count - 3] == recent[recent.count - 6]
|
|
1731
|
+
recent[recent.count - 3] == recent[recent.count - 6] &&
|
|
1732
|
+
Set(recent.suffix(3)).count == 3 &&
|
|
1733
|
+
Set(recentScreenTitles.suffix(3)).count == 3
|
|
1703
1734
|
if (hasLoop2 || hasLoop3) && !(authSucceeded && detectedInputs.contains { $0.secure }) {
|
|
1704
1735
|
let loopKey = "nav_loop:\(titleStr)"
|
|
1705
1736
|
if actionCounts[loopKey] == nil {
|
|
@@ -1711,17 +1742,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1711
1742
|
}
|
|
1712
1743
|
}
|
|
1713
1744
|
|
|
1714
|
-
//
|
|
1715
|
-
//
|
|
1716
|
-
//
|
|
1717
|
-
|
|
1718
|
-
let unrespKey = "unresponsive:\(titleStr)"
|
|
1719
|
-
if actionCounts[unrespKey] == nil {
|
|
1720
|
-
issues.append((type: "unresponsive_element", severity: "medium", title: "Unresponsive UI on \(titleStr)", desc: "Actions are not changing app state — possible frozen or broken screen"))
|
|
1721
|
-
print("OCQA_ISSUE:{\"type\":\"unresponsive_element\",\"severity\":\"medium\",\"title\":\"Unresponsive UI\",\"screen\":\"\(escapedTitle)\",\"repeated_state_count\":\(repeatedStateCount),\"step\":\(actionCount)}")
|
|
1722
|
-
actionCounts[unrespKey] = 1
|
|
1723
|
-
}
|
|
1724
|
-
}
|
|
1745
|
+
// Do not infer an unresponsive app merely from an unchanged state streak: recovery
|
|
1746
|
+
// gestures (scroll, carousel probe, center probe) are expected to be no-ops on many
|
|
1747
|
+
// healthy screens. Labeled controls have a stronger detector below: a direct tap plus
|
|
1748
|
+
// two delayed, content-signature reads. Hangs have their own time-based detector.
|
|
1725
1749
|
|
|
1726
1750
|
if interactable.count < 3 {
|
|
1727
1751
|
print("OCQA_STATE:low_interactable screen=\(escapedTitle) total=\(elements.count) interactable=\(interactable.count) global=\(globalNavElements.count) nonGlobal=\(nonGlobalCandidates.count)")
|
|
@@ -1757,10 +1781,10 @@ class ExplorerTests: XCTestCase {
|
|
|
1757
1781
|
break
|
|
1758
1782
|
}
|
|
1759
1783
|
|
|
1760
|
-
|
|
1761
|
-
|
|
1762
|
-
|
|
1763
|
-
|
|
1784
|
+
// Exhausting Tapp's untried candidate pool is not itself a user-visible dead end:
|
|
1785
|
+
// leaf screens commonly have only a working Back control that was already mapped.
|
|
1786
|
+
// Recover first; only the stronger navigation-trap path below emits a finding when
|
|
1787
|
+
// every real back/dismiss route fails.
|
|
1764
1788
|
// tryGoBack does swipe-down as its last resort (sheet dismiss)
|
|
1765
1789
|
let preBackTitle = titleStr
|
|
1766
1790
|
let backWorked = tryGoBack()
|
|
@@ -1772,6 +1796,7 @@ class ExplorerTests: XCTestCase {
|
|
|
1772
1796
|
print("OCQA_ACTION:{\"type\":\"back\",\"reason\":\"dead_end_escape\",\"from\":\"\(escapedTitle)\",\"to\":\"\(escapeJSON(postTitle))\",\"step\":\(actionCount),\"screen\":\"\(escapedTitle)\",\"narrative\":\"\(escapeJSON(recoveryNarrative("back_dead_end", screen: titleStr, to: postTitle)))\"}")
|
|
1773
1797
|
continue
|
|
1774
1798
|
}
|
|
1799
|
+
if actionCount >= maxActions { break }
|
|
1775
1800
|
// Swipe right (back gesture) as another option
|
|
1776
1801
|
let swipeStart = app.coordinate(withNormalizedOffset: CGVector(dx: 0.02, dy: 0.5))
|
|
1777
1802
|
let swipeEnd = app.coordinate(withNormalizedOffset: CGVector(dx: 0.8, dy: 0.5))
|
|
@@ -2047,6 +2072,7 @@ class ExplorerTests: XCTestCase {
|
|
|
2047
2072
|
}
|
|
2048
2073
|
// Back didn't change screens — fall through to global nav
|
|
2049
2074
|
print("OCQA_ACTION:{\"type\":\"back\",\"reason\":\"screen_exhausted_failed\",\"screen\":\"\(escapedTitle)\",\"step\":\(actionCount),\"narrative\":\"\(escapeJSON(recoveryNarrative("back_failed", screen: titleStr)))\"}")
|
|
2075
|
+
if actionCount >= maxActions { break }
|
|
2050
2076
|
// Stuck on this screen — use global navigation (tab bar) to reach unexplored areas
|
|
2051
2077
|
let globalNav = interactable
|
|
2052
2078
|
.filter { isLikelyGlobalNavigation($0, screenBounds: screenBounds) }
|
|
@@ -3679,6 +3705,20 @@ class ExplorerTests: XCTestCase {
|
|
|
3679
3705
|
return kb.exists ? kb.frame : .zero
|
|
3680
3706
|
}
|
|
3681
3707
|
|
|
3708
|
+
/// Activity indicators are inherently indeterminate. `ProgressIndicator`, however, is also the
|
|
3709
|
+
/// XCTest type for legitimate determinate progress bars (loyalty points, upload percentage,
|
|
3710
|
+
/// onboarding completion). Only value-less/loading-valued progress indicators are hang signals.
|
|
3711
|
+
private func hasIndeterminateLoadingIndicator() -> Bool {
|
|
3712
|
+
if app.activityIndicators.allElementsBoundByIndex.contains(where: { $0.exists && $0.frame.width > 0 && $0.frame.height > 0 }) {
|
|
3713
|
+
return true
|
|
3714
|
+
}
|
|
3715
|
+
return app.progressIndicators.allElementsBoundByIndex.contains { indicator in
|
|
3716
|
+
guard indicator.exists, indicator.frame.width > 0, indicator.frame.height > 0 else { return false }
|
|
3717
|
+
let value = (indicator.value as? String ?? "").trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
|
|
3718
|
+
return value.isEmpty || value == "in progress" || value == "loading"
|
|
3719
|
+
}
|
|
3720
|
+
}
|
|
3721
|
+
|
|
3682
3722
|
/// True when the screen is in a "settled" resting state — no on-screen keyboard and no open
|
|
3683
3723
|
/// transient overlay (menu / dropdown / popover / sheet / picker wheel). A screenshot taken while
|
|
3684
3724
|
/// one of these is up is inherently ambiguous to a visual reviewer (the keyboard "covers" the
|
package/README.md
CHANGED
|
@@ -12,21 +12,24 @@ It turns a repository and real product into an observed UI Map, a compact review
|
|
|
12
12
|
suite, and an inspectable merge decision.
|
|
13
13
|
|
|
14
14
|
Coding agents can write the code, and (with Playwright & friends) they can even drive the app.
|
|
15
|
-
What nobody gives them is **
|
|
16
|
-
no
|
|
17
|
-
|
|
15
|
+
What nobody gives them is **proof it works**. tapp explores your app like a user — no test code,
|
|
16
|
+
no app changes — and surfaces what's broken, with evidence. Then a deterministic gate applies
|
|
17
|
+
versioned policy to that evidence — the findings and coverage, any selected deterministic suites
|
|
18
|
+
(Flows/Scenarios/contracts), and, when available, a target-scoped baseline — and returns a merge
|
|
19
|
+
decision your queue can trust: `pass`, `fail`, or `inconclusive`. Exploration **observes**; the gate
|
|
20
|
+
**judges** — never a soft "ship-ready" guess.
|
|
18
21
|
|
|
19
|
-
Three platforms, one
|
|
22
|
+
Three platforms, one observe-and-gate engine:
|
|
20
23
|
|
|
21
|
-
- **iOS** — the missing Playwright for iOS. tapp is hands *and*
|
|
24
|
+
- **iOS** — the missing Playwright for iOS. tapp is hands *and* eyes: a generic XCUITest
|
|
22
25
|
harness drives any app on the simulator via the accessibility surface. Native — no Appium,
|
|
23
26
|
no WebDriverAgent.
|
|
24
27
|
- **Android** — black-box native driving through ADB + UIAutomator. Install an APK, target its
|
|
25
|
-
application id, and run the same
|
|
26
|
-
does not link a Tapp SDK.
|
|
28
|
+
application id, and run the same exploration, committed Flows, evidence, and regression gate. The
|
|
29
|
+
app does not link a Tapp SDK.
|
|
27
30
|
- **Web (beta)** — built *on* Playwright. Your agent already has browser hands; tapp adds the
|
|
28
31
|
autonomous exploration, the deterministic detectors (uncaught exceptions, failed requests,
|
|
29
|
-
dead buttons, broken links, placeholder `href="#"` links, error pages), and the same
|
|
32
|
+
dead buttons, broken links, placeholder `href="#"` links, error pages), and the same gate.
|
|
30
33
|
|
|
31
34
|
```
|
|
32
35
|
you: "Add a logout button to the settings screen"
|
|
@@ -40,17 +43,17 @@ agent: "Done — and here it is working on the simulator: [screenshot]"
|
|
|
40
43
|
Requirements: **Node ≥ 18**. iOS needs **macOS + Xcode**; Android needs `adb` plus a connected
|
|
41
44
|
emulator/device; web needs Playwright + Chromium.
|
|
42
45
|
|
|
43
|
-
From the app repository, let the agent see the current screen and then
|
|
46
|
+
From the app repository, let the agent see the current screen and then explore it:
|
|
44
47
|
|
|
45
48
|
```bash
|
|
46
49
|
npx -y @aarwitz/tapp open # builds/launches as needed; prints a screenshot path + screen summary
|
|
47
|
-
npx -y @aarwitz/tapp
|
|
50
|
+
npx -y @aarwitz/tapp explore # explores the real app; prints findings + evidence (an observation, not a gate)
|
|
48
51
|
```
|
|
49
52
|
|
|
50
53
|
Claude Code can read the saved image with its file-reading tool; Codex can open it with
|
|
51
|
-
`view_image`. The agent should report what the screenshot proves,
|
|
52
|
-
`
|
|
53
|
-
file, test code, API key, or bundle id is required for this loop.
|
|
54
|
+
`view_image`. The agent should report what the screenshot proves, relay the exploration findings
|
|
55
|
+
as-is (an observation, not a merge decision — `tapp ci` gates that), and link the HTML evidence
|
|
56
|
+
report. No server, account, config file, test code, API key, or bundle id is required for this loop.
|
|
54
57
|
|
|
55
58
|
The product, executable, and package leaf are all Tapp: npm distributes it as
|
|
56
59
|
`@aarwitz/tapp`, while the installed command remains `tapp`.
|
|
@@ -77,7 +80,7 @@ npx -y @aarwitz/tapp baseline create . --platform web
|
|
|
77
80
|
npx -y @aarwitz/tapp ci install .
|
|
78
81
|
```
|
|
79
82
|
|
|
80
|
-
The baseline command writes only after
|
|
83
|
+
The baseline command writes only after exploration and every selected deterministic suite pass
|
|
81
84
|
conclusively. It stores `.tapp/baselines/<platform>/<target-id>.json`; the generated workflow
|
|
82
85
|
uses that exact target identity so two apps on the same platform never share a baseline. `ci
|
|
83
86
|
install` writes `.github/workflows/tapp.yml` plus `.tapp/ci.json`, refuses unresolved build
|
|
@@ -96,7 +99,7 @@ npx -y @aarwitz/tapp apps # what's installed on the simulator (names
|
|
|
96
99
|
npx -y @aarwitz/tapp build [dir] # just build + install (scheme auto-detected)
|
|
97
100
|
```
|
|
98
101
|
|
|
99
|
-
Web (beta): `npx -y @aarwitz/tapp
|
|
102
|
+
Web (beta): `npx -y @aarwitz/tapp explore http://localhost:3000` *(one-time setup:
|
|
100
103
|
`npm i -g playwright && npx playwright install chromium`)*
|
|
101
104
|
|
|
102
105
|
Focused web inspection waits briefly for loading states to settle. If a consent or location modal
|
|
@@ -110,7 +113,7 @@ npx -y @aarwitz/tapp tree https://example.com --tap "Not now" --wait-for "Dashbo
|
|
|
110
113
|
Android:
|
|
111
114
|
|
|
112
115
|
```bash
|
|
113
|
-
npx -y @aarwitz/tapp
|
|
116
|
+
npx -y @aarwitz/tapp explore path/to/app-debug.apk --platform android --app-id com.acme.app
|
|
114
117
|
npx -y @aarwitz/tapp open com.acme.app --platform android
|
|
115
118
|
```
|
|
116
119
|
|
|
@@ -124,7 +127,7 @@ npx -y @aarwitz/tapp doctor # verify Xcode / simulators / toolchain
|
|
|
124
127
|
|
|
125
128
|
`npx -y @aarwitz/tapp app .` opens a local Release Studio for people who want visual repository
|
|
126
129
|
onboarding, release-plan review, and CI preparation. It is not required for the coding-agent
|
|
127
|
-
`open`/`
|
|
130
|
+
`open`/`explore` workflow.
|
|
128
131
|
|
|
129
132
|
### MCP hookup (optional)
|
|
130
133
|
|
|
@@ -156,7 +159,7 @@ args = ["-y", "@aarwitz/tapp", "mcp"]
|
|
|
156
159
|
**Any other MCP client:** stdio command `npx -y @aarwitz/tapp mcp`.
|
|
157
160
|
|
|
158
161
|
Then ask your agent:
|
|
159
|
-
> "
|
|
162
|
+
> "Explore my app and show me what breaks."
|
|
160
163
|
> "Open com.mycompany.app on the simulator and screenshot the home screen."
|
|
161
164
|
> "Log in with test@example.com, drive to checkout, and record it as a replayable test."
|
|
162
165
|
|
|
@@ -168,7 +171,7 @@ Then ask your agent:
|
|
|
168
171
|
| 📸 | `tapp_screenshot` | Whatever's on the sim right now, as an inline image. |
|
|
169
172
|
| 🌳 | `tapp_ui_tree` | The accessibility tree of the current screen (ids, labels, hittability). |
|
|
170
173
|
| 🕹 | `tapp_session_start/act/end` | **Interactive driving** — the Playwright loop. App launches once; each act (tap/type/swipe/back/wait) returns the fresh tree. |
|
|
171
|
-
|
|
|
174
|
+
| 🔭 | `tapp_explore` | **Autonomous exploration** — explores with no authored test, returns an observation `{kind, headline, inconclusive, findingCounts, findings[]}` (no verdict/score — the deterministic gate judges). Takes `appBundleId` (iOS), `androidAppId` (Android), or `url` (web). |
|
|
172
175
|
| 🧭 | `tapp_init` | **Repository import** — detect targets; optionally explore a real surface; persist the shared UI Map; construct the evidence-classified model and grounded release plan. |
|
|
173
176
|
| 👤 | `tapp_actor_config` | **Actor/session setup** — store roles, isolation/provisioning, and environment-variable names without accepting or persisting credential values. |
|
|
174
177
|
| ✅ | `tapp_release_plan` | **Release-plan lifecycle** — inspect, approve/reject/defer, generate, real-target validate, and explicitly promote proposed guarantees without silent test edits. |
|
|
@@ -189,49 +192,43 @@ The desktop Coverage view reads the same `.tapp/application-model.json`,
|
|
|
189
192
|
does not maintain a separate product model. Map nodes identify both the real launch entry and the
|
|
190
193
|
deterministic per-platform navigation root used for bounded changed-surface replay.
|
|
191
194
|
|
|
192
|
-
##
|
|
193
|
-
|
|
194
|
-
**
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
in the decision loop. PR gating keys on the
|
|
198
|
-
(stable finding signatures vs. a baseline), so it reacts to what *changed*, not to
|
|
199
|
-
run-to-run path variance. For critical user journeys, committed **Tasks and Flows** provide the stable
|
|
200
|
-
suite: reusable semantic actions, exact assertions, condition-based waits, fresh launch state,
|
|
201
|
-
and evidence on failure. We call this *flake-resistant*, not magically flake-free—
|
|
202
|
-
unstable test data, and poorly identified controls can still make any E2E test fail.
|
|
203
|
-
|
|
204
|
-
**
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
Web beta presents a `ready` result as **AUTOMATED CHECKS COMPLETE**, not “ship-ready,” and displays
|
|
224
|
-
no scalar score. Exhaustive checks on each exercised page drive the verdict; sampled control probes
|
|
225
|
-
remain visible findings but are advisory. The report
|
|
226
|
-
explicitly excludes content/claim accuracy, privacy and API data minimization, brand/SEO
|
|
227
|
-
consistency, and subjective visual credibility. Those require reviewed contracts, privacy review,
|
|
228
|
-
or human/vision judgment; an exploratory crawl must not imply they were validated.
|
|
195
|
+
## Observation you can read, judgment you can trust
|
|
196
|
+
|
|
197
|
+
**Exploration observes; the gate judges.** Exploration is adaptive — two runs may traverse different
|
|
198
|
+
paths — and it produces an **observation**, never a ship verdict: findings + coverage + evidence. The
|
|
199
|
+
**gate** is deterministic: the same evidence trace, contracts, and baseline always produce the same
|
|
200
|
+
outcome — `pass`, `fail`, or `inconclusive` — with no LLM in the decision loop. PR gating keys on the
|
|
201
|
+
**regression diff** (stable finding signatures vs. a baseline), so it reacts to what *changed*, not to
|
|
202
|
+
run-to-run path variance. For critical user journeys, committed **Tasks and Flows** provide the stable
|
|
203
|
+
CI suite: reusable semantic actions, exact assertions, condition-based waits, fresh launch state,
|
|
204
|
+
bounded timeouts, and evidence on failure. We call this *flake-resistant*, not magically flake-free —
|
|
205
|
+
backend outages, unstable test data, and poorly identified controls can still make any E2E test fail.
|
|
206
|
+
|
|
207
|
+
**No scores, no soft verdicts.** Exploration reports deterministic findings, advisory budget-capped
|
|
208
|
+
control probes, and concrete coverage — never a 0–100 number or a "ready/ship" label a shallow crawl
|
|
209
|
+
can't honestly earn. The merge decision comes only from the gate: reviewed contracts, committed
|
|
210
|
+
Flows/Scenarios, and baseline regressions.
|
|
211
|
+
|
|
212
|
+
`tapp_explore` explores like a user — accessibility surfaces on iOS/Android and a real browser on web
|
|
213
|
+
— and detects crashes, failed sign-ins, dead buttons, stuck loading screens, error surfaces,
|
|
214
|
+
navigation loops, and dead ends (plus, on web: uncaught JS exceptions, failed/5xx requests, broken
|
|
215
|
+
links and assets, and visible placeholder links with no destination). The observation is honest about
|
|
216
|
+
its limits:
|
|
217
|
+
|
|
218
|
+
- It **surfaces findings**; it does not pronounce "ready." **A shallow run is never a pass** — if the
|
|
219
|
+
app crashed on launch or a login wall blocked exploration, you get `inconclusive: true`, not a false
|
|
220
|
+
green. Absence of findings is not a pass.
|
|
221
|
+
- The **gate** turns findings + coverage + your reviewed contracts + a baseline into `pass` / `fail` /
|
|
222
|
+
`inconclusive` (`inconclusive` fails closed). Deterministic — no LLM in the run loop.
|
|
223
|
+
- An exploratory crawl explicitly **excludes** content/claim accuracy, privacy and API data
|
|
224
|
+
minimization, brand/SEO consistency, and subjective visual credibility. Those require reviewed
|
|
225
|
+
contracts, privacy review, or human/vision judgment; a crawl must not imply they were validated.
|
|
229
226
|
|
|
230
227
|
For a business guarantee such as “every coach is insured,” use a deterministic app-owned verifier
|
|
231
228
|
endpoint that returns success only when the invariant holds, then require that status and the
|
|
232
229
|
customer-visible claim in a release contract. The current DSL does not yet read arbitrary JSON
|
|
233
230
|
response bodies or compare a cross-origin API payload directly with page copy; use a verifier or an
|
|
234
|
-
explicit CI preflight rather than assuming
|
|
231
|
+
explicit CI preflight rather than assuming exploration inferred the guarantee.
|
|
235
232
|
|
|
236
233
|
Apps behind a login? Pass `testEmail`/`testPassword` (typed into the login form automatically),
|
|
237
234
|
`appLaunchArgs` (e.g. `["--uitesting"]` if your app supports a bypass), or explicit `loginSteps`
|
|
@@ -350,30 +347,30 @@ your agent proves its UI work instead of claiming it:
|
|
|
350
347
|
This repo uses tapp (https://github.com/aarwitz/tapp) to verify UI work on a real app surface
|
|
351
348
|
(iOS simulator, Android emulator/device, or a browser for web). After any UI change, run `npx -y @aarwitz/tapp open` from the
|
|
352
349
|
repo root (it finds and builds the Xcode project itself) and look at the screenshot it saves as
|
|
353
|
-
proof. Before declaring a feature done, run `npx -y @aarwitz/tapp
|
|
354
|
-
report the
|
|
350
|
+
proof. Before declaring a feature done, run `npx -y @aarwitz/tapp explore` (or `explore <url>` for web)
|
|
351
|
+
and report the findings + evidence. A change is not "done" until it has been seen working.
|
|
355
352
|
(If the tapp MCP server is connected, the tapp_* tools do the same with inline screenshots —
|
|
356
|
-
tapp_build builds + installs the app and returns the bundle id for
|
|
353
|
+
tapp_build builds + installs the app and returns the bundle id for tapp_explore.)
|
|
357
354
|
```
|
|
358
355
|
|
|
359
356
|
## How it works
|
|
360
357
|
|
|
361
358
|
Every driver speaks one protocol: structured `OCQA_*` markers (state, actions, issues,
|
|
362
|
-
transitions) that the
|
|
359
|
+
transitions) that the engine parses into trees, screenshots, findings, coverage, and the gate outcome.
|
|
363
360
|
On **iOS**, a generic **XCUITest harness** attaches to any app by bundle id — no SDK or app code
|
|
364
361
|
changes — and acts through the accessibility tree. On **Android**, ADB + UIAutomator provide the
|
|
365
362
|
same black-box driver contract. On **web** (beta), a deterministic **Playwright crawler** does the
|
|
366
363
|
same in a real browser. Same detectors' spirit,
|
|
367
|
-
same dedup, same regression gate, same honest
|
|
368
|
-
|
|
364
|
+
same dedup, same regression gate, same honest `pass`/`fail`/`inconclusive` outcome. Core exploration,
|
|
365
|
+
evidence collection, and gate evaluation run entirely locally — no telemetry, nothing phones home. Optional AI
|
|
369
366
|
features are explicit: finding enrichment requires `TAPP_ENABLE_REMOTE_AI=1` (an ambient
|
|
370
367
|
API key alone never changes data handling), and AI flow generation / `assert_ai` only run
|
|
371
368
|
when you invoke them; these send selected metadata (screen names, finding titles) to your
|
|
372
369
|
configured model provider. Runtime configuration uses `TAPP_*` environment variables. Repository
|
|
373
|
-
artifacts live under `.tapp/`, and desktop run configuration uses `.tapp.yml`.
|
|
374
|
-
`.autotap/`, `.autotap.yml`, and `AUTOTAP_*` inputs
|
|
375
|
-
|
|
376
|
-
reviewed
|
|
370
|
+
artifacts live under `.tapp/`, and desktop run configuration uses `.tapp.yml`. These are the only
|
|
371
|
+
names Tapp reads: the pre-rename `.autotap/`, `.autotap.yml`, and `AUTOTAP_*` inputs are no longer
|
|
372
|
+
supported. If you still have an old directory or config, rename it to the `.tapp` names in a
|
|
373
|
+
reviewed change—no secrets or evidence formats need to be rewritten.
|
|
377
374
|
|
|
378
375
|
Committed Flow replay, recording a driven session, autonomous exploration, exact assertions,
|
|
379
376
|
regression comparison, and CI gating require **no API key and no coding agent at runtime**. AI is
|