@aarwitz/tapp 0.16.4 → 0.17.0-rc.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +26 -17
- package/Harness/OCQAHarnessUITests/ExplorerTests.swift +1 -1
- package/README.md +69 -63
- package/bin/tapp.js +102 -37
- package/browser/app.js +12 -6
- package/docs/BROWSER-PRODUCT.md +75 -0
- package/docs/PRODUCT-ENGINE.md +107 -0
- package/docs/application-model.md +271 -0
- package/docs/scenarios.md +95 -0
- package/mcp-server/src/application-model.js +9 -8
- package/mcp-server/src/browser-product.js +1 -1
- package/mcp-server/src/ci-report.js +84 -53
- package/mcp-server/src/ci-setup.js +35 -5
- package/mcp-server/src/enrich.js +1 -1
- package/mcp-server/src/html-report.js +4 -3
- package/mcp-server/src/index.js +142 -43
- package/mcp-server/src/product-execution.js +1 -1
- package/mcp-server/src/product-operations.js +2 -2
- package/mcp-server/src/project-config.js +1 -2
- package/mcp-server/src/project-paths.js +5 -17
- package/mcp-server/src/release-contract.js +3 -3
- package/mcp-server/src/report.js +157 -31
- package/mcp-server/src/task-runtime.js +1 -1
- package/package.json +2 -2
- package/scripts/ci-gate.sh +3 -3
- package/scripts/quick-capture.sh +1 -1
- package/scripts/run-flow.sh +1 -1
package/AGENTS.md
CHANGED
|
@@ -8,16 +8,16 @@ Android emulators/devices, plus (beta) web apps in a real browser.
|
|
|
8
8
|
Every core capability works as a plain command — no server, no config. `[target]` is
|
|
9
9
|
optional: with nothing, tapp finds + builds the Xcode project in the cwd (or falls back to
|
|
10
10
|
the app already on the simulator); it also accepts a repo dir, a `path/to/App.app`, a
|
|
11
|
-
bundle id, or (
|
|
11
|
+
bundle id, or (web) an http(s) URL. You never need to know a bundle id up front.
|
|
12
12
|
|
|
13
13
|
```bash
|
|
14
|
-
npx -y @aarwitz/tapp
|
|
14
|
+
npx -y @aarwitz/tapp explore [target] # autonomous exploration → findings + evidence (observation, not a gate; ≈ tapp_explore)
|
|
15
15
|
npx -y @aarwitz/tapp open [target] # launch + screen summary + screenshot saved to a file (≈ tapp_open_app)
|
|
16
16
|
npx -y @aarwitz/tapp tree [target] # accessibility tree, --json for every element (≈ tapp_ui_tree)
|
|
17
17
|
npx -y @aarwitz/tapp shot # screenshot the booted sim → file path (≈ tapp_screenshot)
|
|
18
18
|
npx -y @aarwitz/tapp apps # what's installed on the simulator, with bundle ids
|
|
19
19
|
npx -y @aarwitz/tapp build [dir] # build the app in an Xcode repo + install it (≈ tapp_build)
|
|
20
|
-
npx -y @aarwitz/tapp
|
|
20
|
+
npx -y @aarwitz/tapp explore app.apk --platform android --app-id com.acme.app
|
|
21
21
|
npx -y @aarwitz/tapp flow run .tapp/flows/smoke.yml # committed, keyless E2E replay
|
|
22
22
|
```
|
|
23
23
|
|
|
@@ -30,21 +30,22 @@ loading or changing.
|
|
|
30
30
|
Code's Read tool, Codex's view-image), open the saved screenshot path the CLI prints —
|
|
31
31
|
that IS the screen. If you cannot (Cursor, VS Code Copilot), connect the MCP server
|
|
32
32
|
instead: its tool results carry the screenshot inline. Screen *recordings* are for the
|
|
33
|
-
human: `tapp
|
|
34
|
-
page — tell the user the report path so they can
|
|
33
|
+
human: on **iOS**, `tapp explore` records the full exploration and embeds it in the report.html
|
|
34
|
+
evidence page (Android does not currently record video) — tell the user the report path so they can
|
|
35
|
+
watch it.
|
|
35
36
|
|
|
36
37
|
The interactive session/record loop is MCP-only (it needs a long-lived process). Flow replay is
|
|
37
38
|
also available in the CLI. The rest of this playbook assumes the `tapp_*` MCP tools are connected. With
|
|
38
39
|
MCP, the no-bundle-id path is: `tapp_build {projectDir}` (auto-detects + builds +
|
|
39
|
-
installs, returns the bundle id) → `
|
|
40
|
+
installs, returns the bundle id) → `tapp_explore {appBundleId}`.
|
|
40
41
|
|
|
41
42
|
## Pick the right tool for the job
|
|
42
43
|
|
|
43
44
|
| The user wants… | Use | NOT |
|
|
44
45
|
|---|---|---|
|
|
45
|
-
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `
|
|
46
|
+
| "Show me / screenshot a screen" | `tapp_open_app` (launch + screenshot + tree, ~15s) | `tapp_explore` (a full multi-minute QA exploration) |
|
|
46
47
|
| "Tap through / drive / fill a form / log in" | `tapp_session_start` → `session_act` loop | repeated `open_app` calls (cold relaunch each time) |
|
|
47
|
-
| "Is my app broken?
|
|
48
|
+
| "Is my app broken? Find bugs" | `tapp_explore` — `appBundleId` for iOS, `androidAppId` for Android, `url` for owned web apps; returns an observation (findings + evidence), not a ship verdict — gate a merge with the CI gate (`tapp ci` CLI / the GitHub Action) + a contract | a manual session (exploration is autonomous) |
|
|
48
49
|
| "Make this flow a repeatable test" | drive it in a session, then `tapp_flow_save`; replay with `tapp_flow_run` | re-driving it by hand every time |
|
|
49
50
|
| "What's on screen right now?" | `tapp_screenshot` / `tapp_ui_tree` | relaunching the app |
|
|
50
51
|
|
|
@@ -80,12 +81,19 @@ Rules that prevent 90% of failures:
|
|
|
80
81
|
auto-dismisses keyboards and retries), `not_found` (nothing matches — re-read the tree).
|
|
81
82
|
6. One session at a time. `session_start` always begins from a fresh app launch.
|
|
82
83
|
|
|
83
|
-
## Autonomous
|
|
84
|
+
## Autonomous exploration (`tapp_explore`)
|
|
84
85
|
|
|
85
|
-
|
|
86
|
+
Exploration **observes** — it returns an observation, NOT a ship verdict or score. To get a release
|
|
87
|
+
decision, run the deterministic gate (`tapp ci` / the GitHub Action). It applies versioned policy to
|
|
88
|
+
the findings + coverage plus any selected Flows/Scenarios/contracts and an optional target-scoped
|
|
89
|
+
baseline (both optional — a clean bootstrap can pass without them) and returns `pass | fail |
|
|
90
|
+
inconclusive`.
|
|
91
|
+
|
|
92
|
+
Returns `{kind:"tapp-exploration-run", headline, inconclusive, screensExplored, actionsPerformed,
|
|
93
|
+
findingCounts, findings[]}` — **no** `verdict`, `confidence`, or `releaseScore`. Each finding carries
|
|
94
|
+
`authority` (`deterministic` — marker-derived; model/vision findings would be `model-observed` and
|
|
95
|
+
advisory). Report the finding counts and coverage; do not invent a scalar or a ship verdict.
|
|
86
96
|
|
|
87
|
-
- `verdict`: `ready` | `caution` | `blocked`. **Trust it — it's deterministic.** Report it to the
|
|
88
|
-
user as-is; never soften a `blocked` or inflate a `caution`.
|
|
89
97
|
- `inconclusive: true` means the run couldn't see enough (crash on launch, login wall). That is
|
|
90
98
|
**not a pass** — tell the user what blocked exploration and what would unblock it.
|
|
91
99
|
- Login walls: pass `testEmail`/`testPassword` (auto-typed into login forms), `appLaunchArgs`
|
|
@@ -93,11 +101,12 @@ Returns `{verdict, confidence, headline, screensExplored, actionsPerformed, find
|
|
|
93
101
|
backend URL). If the result shows `inputFieldsEncountered` and you have no credentials, **ask
|
|
94
102
|
the user** for them rather than re-running blind.
|
|
95
103
|
- Diff two runs: pass the previous run's `findings` as `baselineFindings` → you get a
|
|
96
|
-
`regression`
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
104
|
+
`regression` **comparison** (`new` / `persisting` / `resolved`) — an observation, not a gate. To gate
|
|
105
|
+
a merge on regressions, run the CI gate (`tapp ci` / the GitHub Action).
|
|
106
|
+
- On web, preserve scope: Tapp deterministically checks technical behavior such as failed requests,
|
|
107
|
+
missing assets, and placeholder links. Dead-control probes are budget-capped advisory findings.
|
|
108
|
+
Tapp does not validate marketing claims against APIs, API field privacy, brand consistency, or
|
|
109
|
+
subjective marketplace credibility unless an explicit reviewed test/contract or verifier covers them.
|
|
101
110
|
|
|
102
111
|
## Flows (deterministic E2E tests)
|
|
103
112
|
|
|
@@ -1759,7 +1759,7 @@ class ExplorerTests: XCTestCase {
|
|
|
1759
1759
|
|
|
1760
1760
|
let issueTitle = "Dead end: \(titleStr)"
|
|
1761
1761
|
issues.append((type: "dead_end", severity: "medium", title: issueTitle, desc: "No interactable elements found"))
|
|
1762
|
-
print("OCQA_ISSUE:{\"type\":\"dead_end\",\"severity\":\"medium\",\"title\":\"\(
|
|
1762
|
+
print("OCQA_ISSUE:{\"type\":\"dead_end\",\"severity\":\"medium\",\"title\":\"\(escapeJSON(issueTitle))\",\"screen\":\"\(escapedTitle)\",\"step\":\(actionCount)}")
|
|
1763
1763
|
|
|
1764
1764
|
// tryGoBack does swipe-down as its last resort (sheet dismiss)
|
|
1765
1765
|
let preBackTitle = titleStr
|
package/README.md
CHANGED
|
@@ -12,21 +12,24 @@ It turns a repository and real product into an observed UI Map, a compact review
|
|
|
12
12
|
suite, and an inspectable merge decision.
|
|
13
13
|
|
|
14
14
|
Coding agents can write the code, and (with Playwright & friends) they can even drive the app.
|
|
15
|
-
What nobody gives them is **
|
|
16
|
-
no
|
|
17
|
-
|
|
15
|
+
What nobody gives them is **proof it works**. tapp explores your app like a user — no test code,
|
|
16
|
+
no app changes — and surfaces what's broken, with evidence. Then a deterministic gate applies
|
|
17
|
+
versioned policy to that evidence — the findings and coverage, any selected deterministic suites
|
|
18
|
+
(Flows/Scenarios/contracts), and, when available, a target-scoped baseline — and returns a merge
|
|
19
|
+
decision your queue can trust: `pass`, `fail`, or `inconclusive`. Exploration **observes**; the gate
|
|
20
|
+
**judges** — never a soft "ship-ready" guess.
|
|
18
21
|
|
|
19
|
-
Three platforms, one
|
|
22
|
+
Three platforms, one observe-and-gate engine:
|
|
20
23
|
|
|
21
|
-
- **iOS** — the missing Playwright for iOS. tapp is hands *and*
|
|
24
|
+
- **iOS** — the missing Playwright for iOS. tapp is hands *and* eyes: a generic XCUITest
|
|
22
25
|
harness drives any app on the simulator via the accessibility surface. Native — no Appium,
|
|
23
26
|
no WebDriverAgent.
|
|
24
27
|
- **Android** — black-box native driving through ADB + UIAutomator. Install an APK, target its
|
|
25
|
-
application id, and run the same
|
|
26
|
-
does not link a Tapp SDK.
|
|
28
|
+
application id, and run the same exploration, committed Flows, evidence, and regression gate. The
|
|
29
|
+
app does not link a Tapp SDK.
|
|
27
30
|
- **Web (beta)** — built *on* Playwright. Your agent already has browser hands; tapp adds the
|
|
28
31
|
autonomous exploration, the deterministic detectors (uncaught exceptions, failed requests,
|
|
29
|
-
dead buttons, broken links, placeholder `href="#"` links, error pages), and the same
|
|
32
|
+
dead buttons, broken links, placeholder `href="#"` links, error pages), and the same gate.
|
|
30
33
|
|
|
31
34
|
```
|
|
32
35
|
you: "Add a logout button to the settings screen"
|
|
@@ -40,17 +43,17 @@ agent: "Done — and here it is working on the simulator: [screenshot]"
|
|
|
40
43
|
Requirements: **Node ≥ 18**. iOS needs **macOS + Xcode**; Android needs `adb` plus a connected
|
|
41
44
|
emulator/device; web needs Playwright + Chromium.
|
|
42
45
|
|
|
43
|
-
From the app repository, let the agent see the current screen and then
|
|
46
|
+
From the app repository, let the agent see the current screen and then explore it:
|
|
44
47
|
|
|
45
48
|
```bash
|
|
46
49
|
npx -y @aarwitz/tapp open # builds/launches as needed; prints a screenshot path + screen summary
|
|
47
|
-
npx -y @aarwitz/tapp
|
|
50
|
+
npx -y @aarwitz/tapp explore # explores the real app; prints findings + evidence (an observation, not a gate)
|
|
48
51
|
```
|
|
49
52
|
|
|
50
53
|
Claude Code can read the saved image with its file-reading tool; Codex can open it with
|
|
51
|
-
`view_image`. The agent should report what the screenshot proves,
|
|
52
|
-
`
|
|
53
|
-
file, test code, API key, or bundle id is required for this loop.
|
|
54
|
+
`view_image`. The agent should report what the screenshot proves, relay the exploration findings
|
|
55
|
+
as-is (an observation, not a merge decision — `tapp ci` gates that), and link the HTML evidence
|
|
56
|
+
report. No server, account, config file, test code, API key, or bundle id is required for this loop.
|
|
54
57
|
|
|
55
58
|
The product, executable, and package leaf are all Tapp: npm distributes it as
|
|
56
59
|
`@aarwitz/tapp`, while the installed command remains `tapp`.
|
|
@@ -77,7 +80,7 @@ npx -y @aarwitz/tapp baseline create . --platform web
|
|
|
77
80
|
npx -y @aarwitz/tapp ci install .
|
|
78
81
|
```
|
|
79
82
|
|
|
80
|
-
The baseline command writes only after
|
|
83
|
+
The baseline command writes only after exploration and every selected deterministic suite pass
|
|
81
84
|
conclusively. It stores `.tapp/baselines/<platform>/<target-id>.json`; the generated workflow
|
|
82
85
|
uses that exact target identity so two apps on the same platform never share a baseline. `ci
|
|
83
86
|
install` writes `.github/workflows/tapp.yml` plus `.tapp/ci.json`, refuses unresolved build
|
|
@@ -96,7 +99,7 @@ npx -y @aarwitz/tapp apps # what's installed on the simulator (names
|
|
|
96
99
|
npx -y @aarwitz/tapp build [dir] # just build + install (scheme auto-detected)
|
|
97
100
|
```
|
|
98
101
|
|
|
99
|
-
Web (beta): `npx -y @aarwitz/tapp
|
|
102
|
+
Web (beta): `npx -y @aarwitz/tapp explore http://localhost:3000` *(one-time setup:
|
|
100
103
|
`npm i -g playwright && npx playwright install chromium`)*
|
|
101
104
|
|
|
102
105
|
Focused web inspection waits briefly for loading states to settle. If a consent or location modal
|
|
@@ -110,7 +113,7 @@ npx -y @aarwitz/tapp tree https://example.com --tap "Not now" --wait-for "Dashbo
|
|
|
110
113
|
Android:
|
|
111
114
|
|
|
112
115
|
```bash
|
|
113
|
-
npx -y @aarwitz/tapp
|
|
116
|
+
npx -y @aarwitz/tapp explore path/to/app-debug.apk --platform android --app-id com.acme.app
|
|
114
117
|
npx -y @aarwitz/tapp open com.acme.app --platform android
|
|
115
118
|
```
|
|
116
119
|
|
|
@@ -124,7 +127,7 @@ npx -y @aarwitz/tapp doctor # verify Xcode / simulators / toolchain
|
|
|
124
127
|
|
|
125
128
|
`npx -y @aarwitz/tapp app .` opens a local Release Studio for people who want visual repository
|
|
126
129
|
onboarding, release-plan review, and CI preparation. It is not required for the coding-agent
|
|
127
|
-
`open`/`
|
|
130
|
+
`open`/`explore` workflow.
|
|
128
131
|
|
|
129
132
|
### MCP hookup (optional)
|
|
130
133
|
|
|
@@ -156,7 +159,7 @@ args = ["-y", "@aarwitz/tapp", "mcp"]
|
|
|
156
159
|
**Any other MCP client:** stdio command `npx -y @aarwitz/tapp mcp`.
|
|
157
160
|
|
|
158
161
|
Then ask your agent:
|
|
159
|
-
> "
|
|
162
|
+
> "Explore my app and show me what breaks."
|
|
160
163
|
> "Open com.mycompany.app on the simulator and screenshot the home screen."
|
|
161
164
|
> "Log in with test@example.com, drive to checkout, and record it as a replayable test."
|
|
162
165
|
|
|
@@ -168,7 +171,7 @@ Then ask your agent:
|
|
|
168
171
|
| 📸 | `tapp_screenshot` | Whatever's on the sim right now, as an inline image. |
|
|
169
172
|
| 🌳 | `tapp_ui_tree` | The accessibility tree of the current screen (ids, labels, hittability). |
|
|
170
173
|
| 🕹 | `tapp_session_start/act/end` | **Interactive driving** — the Playwright loop. App launches once; each act (tap/type/swipe/back/wait) returns the fresh tree. |
|
|
171
|
-
|
|
|
174
|
+
| 🔭 | `tapp_explore` | **Autonomous exploration** — explores with no authored test, returns an observation `{kind, headline, inconclusive, findingCounts, findings[]}` (no verdict/score — the deterministic gate judges). Takes `appBundleId` (iOS), `androidAppId` (Android), or `url` (web). |
|
|
172
175
|
| 🧭 | `tapp_init` | **Repository import** — detect targets; optionally explore a real surface; persist the shared UI Map; construct the evidence-classified model and grounded release plan. |
|
|
173
176
|
| 👤 | `tapp_actor_config` | **Actor/session setup** — store roles, isolation/provisioning, and environment-variable names without accepting or persisting credential values. |
|
|
174
177
|
| ✅ | `tapp_release_plan` | **Release-plan lifecycle** — inspect, approve/reject/defer, generate, real-target validate, and explicitly promote proposed guarantees without silent test edits. |
|
|
@@ -189,40 +192,43 @@ The desktop Coverage view reads the same `.tapp/application-model.json`,
|
|
|
189
192
|
does not maintain a separate product model. Map nodes identify both the real launch entry and the
|
|
190
193
|
deterministic per-platform navigation root used for bounded changed-surface replay.
|
|
191
194
|
|
|
192
|
-
##
|
|
193
|
-
|
|
194
|
-
**
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
in the decision loop. PR gating keys on the
|
|
198
|
-
(stable finding signatures vs. a baseline), so it reacts to what *changed*, not to
|
|
199
|
-
run-to-run path variance. For critical user journeys, committed **Tasks and Flows** provide the stable
|
|
200
|
-
suite: reusable semantic actions, exact assertions, condition-based waits, fresh launch state,
|
|
201
|
-
and evidence on failure. We call this *flake-resistant*, not magically flake-free—
|
|
202
|
-
unstable test data, and poorly identified controls can still make any E2E test fail.
|
|
203
|
-
|
|
204
|
-
**
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
`
|
|
210
|
-
and detects crashes, failed sign-ins, dead buttons, stuck loading screens, error surfaces,
|
|
211
|
-
navigation loops, and dead ends (plus, on web: uncaught JS exceptions, failed/5xx requests,
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
195
|
+
## Observation you can read, judgment you can trust
|
|
196
|
+
|
|
197
|
+
**Exploration observes; the gate judges.** Exploration is adaptive — two runs may traverse different
|
|
198
|
+
paths — and it produces an **observation**, never a ship verdict: findings + coverage + evidence. The
|
|
199
|
+
**gate** is deterministic: the same evidence trace, contracts, and baseline always produce the same
|
|
200
|
+
outcome — `pass`, `fail`, or `inconclusive` — with no LLM in the decision loop. PR gating keys on the
|
|
201
|
+
**regression diff** (stable finding signatures vs. a baseline), so it reacts to what *changed*, not to
|
|
202
|
+
run-to-run path variance. For critical user journeys, committed **Tasks and Flows** provide the stable
|
|
203
|
+
CI suite: reusable semantic actions, exact assertions, condition-based waits, fresh launch state,
|
|
204
|
+
bounded timeouts, and evidence on failure. We call this *flake-resistant*, not magically flake-free —
|
|
205
|
+
backend outages, unstable test data, and poorly identified controls can still make any E2E test fail.
|
|
206
|
+
|
|
207
|
+
**No scores, no soft verdicts.** Exploration reports deterministic findings, advisory budget-capped
|
|
208
|
+
control probes, and concrete coverage — never a 0–100 number or a "ready/ship" label a shallow crawl
|
|
209
|
+
can't honestly earn. The merge decision comes only from the gate: reviewed contracts, committed
|
|
210
|
+
Flows/Scenarios, and baseline regressions.
|
|
211
|
+
|
|
212
|
+
`tapp_explore` explores like a user — accessibility surfaces on iOS/Android and a real browser on web
|
|
213
|
+
— and detects crashes, failed sign-ins, dead buttons, stuck loading screens, error surfaces,
|
|
214
|
+
navigation loops, and dead ends (plus, on web: uncaught JS exceptions, failed/5xx requests, broken
|
|
215
|
+
links and assets, and visible placeholder links with no destination). The observation is honest about
|
|
216
|
+
its limits:
|
|
217
|
+
|
|
218
|
+
- It **surfaces findings**; it does not pronounce "ready." **A shallow run is never a pass** — if the
|
|
219
|
+
app crashed on launch or a login wall blocked exploration, you get `inconclusive: true`, not a false
|
|
220
|
+
green. Absence of findings is not a pass.
|
|
221
|
+
- The **gate** turns findings + coverage + your reviewed contracts + a baseline into `pass` / `fail` /
|
|
222
|
+
`inconclusive` (`inconclusive` fails closed). Deterministic — no LLM in the run loop.
|
|
223
|
+
- An exploratory crawl explicitly **excludes** content/claim accuracy, privacy and API data
|
|
224
|
+
minimization, brand/SEO consistency, and subjective visual credibility. Those require reviewed
|
|
225
|
+
contracts, privacy review, or human/vision judgment; a crawl must not imply they were validated.
|
|
226
|
+
|
|
227
|
+
For a business guarantee such as “every coach is insured,” use a deterministic app-owned verifier
|
|
228
|
+
endpoint that returns success only when the invariant holds, then require that status and the
|
|
229
|
+
customer-visible claim in a release contract. The current DSL does not yet read arbitrary JSON
|
|
230
|
+
response bodies or compare a cross-origin API payload directly with page copy; use a verifier or an
|
|
231
|
+
explicit CI preflight rather than assuming exploration inferred the guarantee.
|
|
226
232
|
|
|
227
233
|
Apps behind a login? Pass `testEmail`/`testPassword` (typed into the login form automatically),
|
|
228
234
|
`appLaunchArgs` (e.g. `["--uitesting"]` if your app supports a bypass), or explicit `loginSteps`
|
|
@@ -341,30 +347,30 @@ your agent proves its UI work instead of claiming it:
|
|
|
341
347
|
This repo uses tapp (https://github.com/aarwitz/tapp) to verify UI work on a real app surface
|
|
342
348
|
(iOS simulator, Android emulator/device, or a browser for web). After any UI change, run `npx -y @aarwitz/tapp open` from the
|
|
343
349
|
repo root (it finds and builds the Xcode project itself) and look at the screenshot it saves as
|
|
344
|
-
proof. Before declaring a feature done, run `npx -y @aarwitz/tapp
|
|
345
|
-
report the
|
|
350
|
+
proof. Before declaring a feature done, run `npx -y @aarwitz/tapp explore` (or `explore <url>` for web)
|
|
351
|
+
and report the findings + evidence. A change is not "done" until it has been seen working.
|
|
346
352
|
(If the tapp MCP server is connected, the tapp_* tools do the same with inline screenshots —
|
|
347
|
-
tapp_build builds + installs the app and returns the bundle id for
|
|
353
|
+
tapp_build builds + installs the app and returns the bundle id for tapp_explore.)
|
|
348
354
|
```
|
|
349
355
|
|
|
350
356
|
## How it works
|
|
351
357
|
|
|
352
358
|
Every driver speaks one protocol: structured `OCQA_*` markers (state, actions, issues,
|
|
353
|
-
transitions) that the
|
|
359
|
+
transitions) that the engine parses into trees, screenshots, findings, coverage, and the gate outcome.
|
|
354
360
|
On **iOS**, a generic **XCUITest harness** attaches to any app by bundle id — no SDK or app code
|
|
355
361
|
changes — and acts through the accessibility tree. On **Android**, ADB + UIAutomator provide the
|
|
356
362
|
same black-box driver contract. On **web** (beta), a deterministic **Playwright crawler** does the
|
|
357
363
|
same in a real browser. Same detectors' spirit,
|
|
358
|
-
same dedup, same regression gate, same honest
|
|
359
|
-
|
|
364
|
+
same dedup, same regression gate, same honest `pass`/`fail`/`inconclusive` outcome. Core exploration,
|
|
365
|
+
evidence collection, and gate evaluation run entirely locally — no telemetry, nothing phones home. Optional AI
|
|
360
366
|
features are explicit: finding enrichment requires `TAPP_ENABLE_REMOTE_AI=1` (an ambient
|
|
361
367
|
API key alone never changes data handling), and AI flow generation / `assert_ai` only run
|
|
362
368
|
when you invoke them; these send selected metadata (screen names, finding titles) to your
|
|
363
369
|
configured model provider. Runtime configuration uses `TAPP_*` environment variables. Repository
|
|
364
|
-
artifacts live under `.tapp/`, and desktop run configuration uses `.tapp.yml`.
|
|
365
|
-
`.autotap/`, `.autotap.yml`, and `AUTOTAP_*` inputs
|
|
366
|
-
|
|
367
|
-
reviewed
|
|
370
|
+
artifacts live under `.tapp/`, and desktop run configuration uses `.tapp.yml`. These are the only
|
|
371
|
+
names Tapp reads: the pre-rename `.autotap/`, `.autotap.yml`, and `AUTOTAP_*` inputs are no longer
|
|
372
|
+
supported. If you still have an old directory or config, rename it to the `.tapp` names in a
|
|
373
|
+
reviewed change—no secrets or evidence formats need to be rewritten.
|
|
368
374
|
|
|
369
375
|
Committed Flow replay, recording a driven session, autonomous exploration, exact assertions,
|
|
370
376
|
regression comparison, and CI gating require **no API key and no coding agent at runtime**. AI is
|
package/bin/tapp.js
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
// tapp CLI — ship with proof.
|
|
3
3
|
//
|
|
4
4
|
// Zero-config verbs (the same engine the MCP tools use, exported by mcp-server/src/index.js):
|
|
5
|
-
// tapp
|
|
5
|
+
// tapp explore <bundleId|appId|url> Autonomous exploration → findings + evidence (observation)
|
|
6
6
|
// tapp open <bundleId> Launch app → screen summary + screenshot file
|
|
7
7
|
// tapp tree <bundleId> Accessibility tree of the current screen
|
|
8
8
|
// tapp shot Screenshot the booted simulator
|
|
@@ -30,11 +30,12 @@ const pkg = JSON.parse(fs.readFileSync(path.join(packageRoot, "package.json"), "
|
|
|
30
30
|
|
|
31
31
|
// Redirect all writable output away from the (possibly read-only) package dir.
|
|
32
32
|
// The old environment alias remains a read-only fallback for older integrations.
|
|
33
|
-
const tappHome = (process.env.TAPP_HOME ||
|
|
33
|
+
const tappHome = (process.env.TAPP_HOME || path.join(os.homedir(), ".tapp")).trim();
|
|
34
34
|
process.env.TAPP_HOME = tappHome;
|
|
35
|
-
|
|
35
|
+
// TAPP_HOME is created lazily (just before the switch) so `--help`, `help`, and `version` never
|
|
36
|
+
// write anything — not even the home directory.
|
|
36
37
|
|
|
37
|
-
|
|
38
|
+
let [, , command = "help", ...rest] = process.argv;
|
|
38
39
|
|
|
39
40
|
function run(cmd, args, opts = {}) {
|
|
40
41
|
const result = spawnSync(cmd, args, { encoding: "utf8", ...opts });
|
|
@@ -133,7 +134,7 @@ const engineImport = () => import(path.join(packageRoot, "mcp-server", "src", "i
|
|
|
133
134
|
|
|
134
135
|
function requireMacFor(what) {
|
|
135
136
|
if (process.platform === "darwin") return;
|
|
136
|
-
console.error(`❌ ${what} requires macOS (Xcode + iOS simulator). The web beta runs anywhere: tapp
|
|
137
|
+
console.error(`❌ ${what} requires macOS (Xcode + iOS simulator). The web beta runs anywhere: tapp explore https://localhost:3000`);
|
|
137
138
|
process.exit(1);
|
|
138
139
|
}
|
|
139
140
|
|
|
@@ -196,6 +197,20 @@ async function resolveTargetOrExit(engine, input) {
|
|
|
196
197
|
return resolved.bundleId;
|
|
197
198
|
}
|
|
198
199
|
|
|
200
|
+
// Safe help: `--help`/`-h` on ANY verb prints the command reference and does NOTHING else — never
|
|
201
|
+
// builds, launches, writes, or opens (ADR-0005 manual-testing requirement). `ci` keeps its own
|
|
202
|
+
// richer `--help` (a safe usage print in ci-gate.sh); help/version don't need interception.
|
|
203
|
+
if ((rest.includes("--help") || rest.includes("-h")) && !["help", "version", "--version", "-v", "ci"].includes(command)) {
|
|
204
|
+
console.log(`ℹ️ '${command} --help' — showing the command reference (--help never builds, launches, writes, or opens):\n`);
|
|
205
|
+
command = "help";
|
|
206
|
+
rest = [];
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
// Create TAPP_HOME only for commands that actually use it — never for help/version/--help.
|
|
210
|
+
if (!["help", "version", "--version", "-v"].includes(command)) {
|
|
211
|
+
fs.mkdirSync(tappHome, { recursive: true });
|
|
212
|
+
}
|
|
213
|
+
|
|
199
214
|
switch (command) {
|
|
200
215
|
case "mcp": {
|
|
201
216
|
// Agents spawn `tapp mcp`; the engine module is import-safe, so start explicitly.
|
|
@@ -280,7 +295,7 @@ switch (command) {
|
|
|
280
295
|
console.log(`🧭 Tapp init — ${built.model.application.name}`);
|
|
281
296
|
console.log(` targets: ${built.model.targets.length ? built.model.targets.map((target) => `${target.platform}:${target.name}`).join(", ") : "none"}`);
|
|
282
297
|
console.log(` UI Map: ${built.model.uiMap.status} · ${built.model.uiMap.nodeCount} states · ${built.model.uiMap.edgeCount} transitions`);
|
|
283
|
-
if (exploration) console.log(` Exploration: ${exploration.
|
|
298
|
+
if (exploration) console.log(` Exploration: ${(exploration.findings || []).length} finding(s)${exploration.inconclusive ? " (inconclusive)" : ""} · ${exploration.uiMap.nodeCount} states · evidence: ${exploration.reportHtml || exploration.capture?.path || "capture recorded"}`);
|
|
284
299
|
if (exploration?.managedRuntime) console.log(` Managed web runtime: built/started ${exploration.target} for exploration and stopped it afterward · log: ${exploration.runtime.logPath}`);
|
|
285
300
|
console.log(` release plan: ${(written?.plan || built.plan).items.length} item(s) · ${pending.length} pending review · ${blocking.length} blocking requirement(s)`);
|
|
286
301
|
for (const requirement of built.model.requirements) console.log(` ${requirement.severity === "blocking" ? "❌" : "⚠️"} ${requirement.message} Next: ${requirement.remediation}`);
|
|
@@ -387,9 +402,13 @@ switch (command) {
|
|
|
387
402
|
// ---- Zero-config verbs: the same engine the MCP tools use (exported by index.js),
|
|
388
403
|
// invokable by any agent or human with no server setup at all.
|
|
389
404
|
|
|
405
|
+
case "explore":
|
|
390
406
|
case "qa": {
|
|
407
|
+
// `explore` is the canonical verb (ADR-0005: exploration observes; the gate judges). `qa` is a
|
|
408
|
+
// hidden deprecated alias.
|
|
409
|
+
if (command === "qa") console.error("note: 'qa' is now 'explore' — 'qa' still works for now.\n");
|
|
391
410
|
const { flags, positionals } = parseVerbArgs(rest);
|
|
392
|
-
|
|
411
|
+
let target = positionals[0] || "";
|
|
393
412
|
let baselineFindings;
|
|
394
413
|
if (flags.baseline) {
|
|
395
414
|
try {
|
|
@@ -401,6 +420,41 @@ switch (command) {
|
|
|
401
420
|
}
|
|
402
421
|
}
|
|
403
422
|
const engine = await engineImport();
|
|
423
|
+
// Source-preparing bare explore (ADR-0005 §5): no explicit target + a repo application model →
|
|
424
|
+
// drive the model's default target end to end. Managed web is built/started/waited-for and
|
|
425
|
+
// always stopped; iOS is built + installed on the simulator; Android is built to an APK +
|
|
426
|
+
// installed. `--platform`/`--target` narrow which model target is chosen. With no model we fall
|
|
427
|
+
// through to the ordinary target resolution below, so nothing regresses.
|
|
428
|
+
if (!target && !flags["app-id"] && !flags.apk) {
|
|
429
|
+
const modelPath = existingProjectArtifactPath(process.cwd(), "application-model.json");
|
|
430
|
+
if (modelPath && fs.existsSync(modelPath)) {
|
|
431
|
+
const modelPlatform = typeof flags.platform === "string" ? flags.platform.toLowerCase() : "";
|
|
432
|
+
if (modelPlatform === "ios") requireMacFor("iOS testing");
|
|
433
|
+
const onProgress = (p) =>
|
|
434
|
+
process.stderr.write(`\r🔍 Exploring… ${p.action}/${p.max || flags.actions || 60} actions · ${p.states} reached `);
|
|
435
|
+
const r = await engine.runExploreTarget({
|
|
436
|
+
projectDir: process.cwd(),
|
|
437
|
+
platform: modelPlatform,
|
|
438
|
+
target: typeof flags.target === "string" ? flags.target : "",
|
|
439
|
+
maxActions: flags.actions,
|
|
440
|
+
timeout: flags.timeout,
|
|
441
|
+
testEmail: flags.email,
|
|
442
|
+
testPassword: flags.password,
|
|
443
|
+
baselineFindings,
|
|
444
|
+
surface: "cli",
|
|
445
|
+
onProgress,
|
|
446
|
+
onStatus: (t) => console.error(`ℹ️ ${t}`),
|
|
447
|
+
});
|
|
448
|
+
process.stderr.write("\n");
|
|
449
|
+
if (r.error) { printEngineError(r); process.exit(1); }
|
|
450
|
+
console.log(r.text);
|
|
451
|
+
if (flags.json && typeof flags.json === "string") {
|
|
452
|
+
fs.writeFileSync(flags.json, JSON.stringify(r.structured, null, 2));
|
|
453
|
+
console.log(`\n📄 Full report JSON: ${flags.json} (pass as --baseline next run to diff regressions)`);
|
|
454
|
+
}
|
|
455
|
+
break;
|
|
456
|
+
}
|
|
457
|
+
}
|
|
404
458
|
const platform = requestedPlatform(flags, target);
|
|
405
459
|
if (!["ios", "android", "web"].includes(platform)) {
|
|
406
460
|
console.error("❌ --platform must be ios|android|web");
|
|
@@ -625,7 +679,7 @@ switch (command) {
|
|
|
625
679
|
}
|
|
626
680
|
console.log("📱 Installed on the booted simulator:\n");
|
|
627
681
|
for (const a of la.apps) console.log(` ${a.bundleId} (${a.name})`);
|
|
628
|
-
console.log(`\nTest one: tapp
|
|
682
|
+
console.log(`\nTest one: tapp explore <bundleId>`);
|
|
629
683
|
break;
|
|
630
684
|
}
|
|
631
685
|
|
|
@@ -655,7 +709,7 @@ switch (command) {
|
|
|
655
709
|
process.exit(1);
|
|
656
710
|
}
|
|
657
711
|
console.log(`🔨 Built ${path.basename(built.appPath)} (scheme ${built.scheme}) — installed as ${inst.bundleId}`);
|
|
658
|
-
console.log(`\nNext: tapp
|
|
712
|
+
console.log(`\nNext: tapp explore ${inst.bundleId}`);
|
|
659
713
|
break;
|
|
660
714
|
}
|
|
661
715
|
|
|
@@ -1065,7 +1119,7 @@ switch (command) {
|
|
|
1065
1119
|
|
|
1066
1120
|
console.log(`\n Home: ${tappHome}`);
|
|
1067
1121
|
console.log(healthy
|
|
1068
|
-
? "\nReady. Start with:\n npx -y @aarwitz/tapp open [target]\n npx -y @aarwitz/tapp
|
|
1122
|
+
? "\nReady. Start with:\n npx -y @aarwitz/tapp open [target]\n npx -y @aarwitz/tapp explore [target]"
|
|
1069
1123
|
: "\nFix the ❌ items above, then re-run: tapp doctor");
|
|
1070
1124
|
process.exit(healthy ? 0 : 1);
|
|
1071
1125
|
}
|
|
@@ -1249,7 +1303,7 @@ switch (command) {
|
|
|
1249
1303
|
replace: flags.replace === true,
|
|
1250
1304
|
});
|
|
1251
1305
|
console.log(`✅ Conclusive baseline established — ${selectedTarget.platform}:${selectedTarget.name}`);
|
|
1252
|
-
console.log(` ${written.validation.screensExplored} states · ${written.validation.actionsPerformed} actions · ${written.validation.suite.contracts} contracts ·
|
|
1306
|
+
console.log(` ${written.validation.screensExplored} states · ${written.validation.actionsPerformed} actions · ${written.validation.suite.contracts} contracts · outcome ${written.validation.outcome}`);
|
|
1253
1307
|
console.log(` baseline: ${written.path}\n source gate report: ${reportPath}`);
|
|
1254
1308
|
} catch (error) { console.error(`❌ Baseline not written: ${error.message}`); process.exit(2); }
|
|
1255
1309
|
break;
|
|
@@ -1372,15 +1426,32 @@ switch (command) {
|
|
|
1372
1426
|
}
|
|
1373
1427
|
|
|
1374
1428
|
default: {
|
|
1375
|
-
console.log(`tapp v${pkg.version} — ship with proof. Autonomous
|
|
1429
|
+
console.log(`tapp v${pkg.version} — ship with proof. Autonomous exploration and deterministic release gates for iOS, Android, and web.
|
|
1430
|
+
|
|
1431
|
+
Core — explore, prove, gate (agents and humans can just run these — no server, no setup):
|
|
1432
|
+
tapp explore [target] Autonomous exploration → findings + evidence (an observation, NOT a
|
|
1433
|
+
release decision — run 'tapp ci' to gate a merge)
|
|
1434
|
+
(--platform ios|android|web · --app-id ID · --apk FILE · --actions N)
|
|
1435
|
+
tapp contract run FILE Replay a business-level release contract — the guarantees that must hold
|
|
1436
|
+
tapp ci ... Merge-blocking release gate — explore + suites + baseline → pass/fail/inconclusive
|
|
1437
|
+
(see: tapp ci --help)
|
|
1376
1438
|
|
|
1377
|
-
|
|
1439
|
+
Primitives — an agent's eyes and hands (no setup):
|
|
1378
1440
|
tapp open [target] Launch the app → screen summary + screenshot saved to a file
|
|
1379
1441
|
(web: --tap TEXT · --wait-for TEXT · --out FILE)
|
|
1380
|
-
tapp qa [target] Autonomous QA → verdict + findings + evidence
|
|
1381
|
-
(--platform ios|android|web · --app-id ID · --apk FILE · --actions N)
|
|
1382
1442
|
tapp tree [target] Accessibility tree of the current screen (--json for every element)
|
|
1383
1443
|
(web: --tap TEXT · --wait-for TEXT)
|
|
1444
|
+
|
|
1445
|
+
Repository & release:
|
|
1446
|
+
tapp init [repo] Detect targets and write the application model + reviewable release plan
|
|
1447
|
+
(--explore grounds the UI Map · --url URL · --platform · --dry-run · --refresh)
|
|
1448
|
+
tapp baseline create [repo] Run/import a conclusive full gate and save a target-scoped baseline
|
|
1449
|
+
tapp report [captureId] Open the HTML evidence page for a capture (default: latest)
|
|
1450
|
+
tapp ci install [repo] Generate a reviewable target-aware GitHub workflow + CI manifest
|
|
1451
|
+
tapp actor set NAME Configure an actor using environment-variable names only (never values)
|
|
1452
|
+
tapp actor list [repo] Inspect named actors, sessions, provisioning, and secret env bindings
|
|
1453
|
+
|
|
1454
|
+
Advanced — deterministic suites, lifecycle & compilers:
|
|
1384
1455
|
tapp flow run FILE Replay a committed deterministic Flow (no AI/API key)
|
|
1385
1456
|
tapp flow validate FILE Validate a Flow without launching a target
|
|
1386
1457
|
tapp task validate FILE Validate a reusable deterministic Task (+ optional UI Map grounding)
|
|
@@ -1388,38 +1459,32 @@ Zero-config verbs (agents and humans can just run these — no server, no setup)
|
|
|
1388
1459
|
tapp task run FILE Replay a Task directly on iOS, Android, or web
|
|
1389
1460
|
tapp contract validate FILE Validate a business-level TypeScript release contract
|
|
1390
1461
|
tapp contract compile FILE Compile a contract to the shared deterministic executor
|
|
1391
|
-
tapp contract run FILE Replay a release contract without AI or a coding agent
|
|
1392
|
-
tapp pr plan --base REF Select critical + diff-relevant contracts and report uncovered changes
|
|
1393
|
-
tapp pr adopt PLAN --item ID Explicitly add an observed PR coverage proposal to the release plan
|
|
1394
1462
|
tapp scenario run FILE Replay an isolated multi-actor system Scenario (web)
|
|
1395
1463
|
tapp scenario validate FILE Validate actors, lifecycle, and deterministic steps
|
|
1464
|
+
tapp pr plan --base REF Select critical + diff-relevant contracts and report uncovered changes
|
|
1465
|
+
tapp pr adopt PLAN --item ID Explicitly add an observed PR coverage proposal to the release plan
|
|
1466
|
+
tapp plan show [FILE] Inspect the proposed/accepted release-contract plan
|
|
1467
|
+
tapp plan review [FILE] Explicitly approve, reject, or defer proposed plan items
|
|
1468
|
+
tapp plan generate [FILE] Generate compile-checked, untrusted contract drafts from approved Tasks
|
|
1469
|
+
tapp plan validate [FILE] Replay drafts on a real target; trust only after all platforms pass
|
|
1470
|
+
tapp plan promote [FILE] Move fully validated drafts into reviewed Tasks/contracts + map coverage
|
|
1396
1471
|
tapp map build MARKERS Build/merge the persistent platform-neutral UI Map
|
|
1397
1472
|
tapp map inspect [FILE] Inspect states, controls, platforms, and map validity
|
|
1398
1473
|
tapp map diff A B Diff observed UI structure without false reachability claims
|
|
1399
|
-
|
|
1474
|
+
|
|
1475
|
+
Simulator & workspace:
|
|
1400
1476
|
tapp shot Screenshot the booted simulator → file path (--out file.jpg)
|
|
1401
1477
|
tapp build [dir] Build the iOS app in a repo for the simulator + install it (--scheme S)
|
|
1402
1478
|
tapp apps List apps installed on the booted simulator (with bundle ids)
|
|
1403
|
-
tapp report [captureId] Open the HTML evidence page for a capture (default: latest)
|
|
1404
1479
|
tapp app [repo] Optional local browser workspace for repository onboarding and review
|
|
1405
1480
|
(loopback-only; --no-open · --port PORT)
|
|
1406
|
-
tapp init [repo] Detect targets and write the application model + reviewable release plan
|
|
1407
|
-
(--explore builds/starts or connects, grounds the UI Map, then tears down)
|
|
1408
|
-
(--url URL · --platform PLATFORM · --dry-run · --refresh)
|
|
1409
|
-
tapp actor list [repo] Inspect named actors, sessions, provisioning, and secret env bindings
|
|
1410
|
-
tapp actor set NAME Configure an actor using environment-variable names only (never values)
|
|
1411
|
-
tapp plan show [FILE] Inspect the proposed/accepted release-contract plan
|
|
1412
|
-
tapp plan review [FILE] Explicitly approve, reject, or defer proposed plan items
|
|
1413
|
-
tapp plan generate [FILE] Generate compile-checked, untrusted contract drafts from approved Tasks
|
|
1414
|
-
tapp plan validate [FILE] Replay drafts on a real target; trust only after all platforms pass
|
|
1415
|
-
tapp plan promote [FILE] Move fully validated drafts into reviewed Tasks/contracts + map coverage
|
|
1416
|
-
tapp ci ... Merge-blocking release gate — explore + flows + baseline diff (see: tapp ci --help)
|
|
1417
|
-
tapp ci install [repo] Generate a reviewable target-aware GitHub workflow + CI manifest
|
|
1418
1481
|
|
|
1419
|
-
[target] is whatever you have — nothing (
|
|
1420
|
-
|
|
1421
|
-
|
|
1422
|
-
|
|
1482
|
+
[target] is whatever you have — nothing (in an initialized repo, bare 'tapp explore' drives the
|
|
1483
|
+
application model's default target from source: managed web is built/started/stopped, iOS is
|
|
1484
|
+
built + installed, Android is built to an APK + installed; otherwise it finds + builds the Xcode
|
|
1485
|
+
project in the current dir, or falls back to the app on the simulator), a repo dir, a
|
|
1486
|
+
path/to/App.app, a bundle id, an Android app id/APK (--platform android --app-id ...), or an
|
|
1487
|
+
http(s) URL. For iOS you never need to know a bundle id up front.
|
|
1423
1488
|
|
|
1424
1489
|
Setup:
|
|
1425
1490
|
tapp install Prebuild the exploration harness (~2 min; otherwise builds on first use)
|
|
@@ -1432,7 +1497,7 @@ MCP hookup (optional — for inline screenshots and the tap/type/inspect session
|
|
|
1432
1497
|
{ "servers": { "tapp": { "type": "stdio", "command": "npx", "args": ["-y", "@aarwitz/tapp", "mcp"] } } }
|
|
1433
1498
|
|
|
1434
1499
|
Then ask your agent things like:
|
|
1435
|
-
"
|
|
1500
|
+
"Explore com.mycompany.app and show me what breaks"
|
|
1436
1501
|
"Open the settings screen and show me the screenshot"
|
|
1437
1502
|
"Drive the login flow and record it as a replayable test"
|
|
1438
1503
|
|