opencode-agent-harness 0.0.0-stage → 0.3.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +217 -0
- package/LICENSE +21 -0
- package/README.md +636 -2
- package/bin/agent-harness.js +3182 -0
- package/contrib/global-plugins/README.md +49 -0
- package/contrib/global-plugins/harness-freshness.ts +355 -0
- package/package.json +86 -4
- package/templates/.agents/AGENTS.md +148 -0
- package/templates/.agents/features/INDEX.md +10 -0
- package/templates/.agents/handoffs/README.md +87 -0
- package/templates/.agents/memory/dev-loop.md +59 -0
- package/templates/.agents/memory/domain-map.md +29 -0
- package/templates/.agents/memory/environments.md +35 -0
- package/templates/.agents/memory/flow-map.md +23 -0
- package/templates/.agents/memory/history.md +7 -0
- package/templates/.agents/memory/locations.md +14 -0
- package/templates/.agents/memory/model-routing.md +261 -0
- package/templates/.agents/memory/orchestration.md +183 -0
- package/templates/.agents/memory/owned-files.md +19 -0
- package/templates/.agents/memory/product-principles.md +37 -0
- package/templates/.agents/memory/quality-parameters.md +37 -0
- package/templates/.agents/memory/stack-versions.md +35 -0
- package/templates/.agents/memory/test-design-parameters.md +36 -0
- package/templates/.agents/nightly.json +17 -0
- package/templates/.agents/qa-plans/README.md +70 -0
- package/templates/.agents/qa-runs/README.md +21 -0
- package/templates/.agents/reviews/README.md +47 -0
- package/templates/.agents/rules/00-operating.md +199 -0
- package/templates/.agents/skills/data-engineer/SKILL.md +31 -0
- package/templates/.agents/skills/devops-engineer/SKILL.md +50 -0
- package/templates/.agents/skills/diagnostics-expert/SKILL.md +44 -0
- package/templates/.agents/skills/docs/SKILL.md +140 -0
- package/templates/.agents/skills/features/SKILL.md +187 -0
- package/templates/.agents/skills/frontend-engineer/SKILL.md +81 -0
- package/templates/.agents/skills/handoff/SKILL.md +85 -0
- package/templates/.agents/skills/mobile-engineer/SKILL.md +101 -0
- package/templates/.agents/skills/planner/SKILL.md +112 -0
- package/templates/.agents/skills/product-manager/SKILL.md +115 -0
- package/templates/.agents/skills/qa-architect/SKILL.md +224 -0
- package/templates/.agents/skills/qa-runner/SKILL.md +64 -0
- package/templates/.agents/skills/security-engineer/SKILL.md +53 -0
- package/templates/.agents/skills/system-architect/SKILL.md +42 -0
- package/templates/.agents/skills/ui-designer/SKILL.md +71 -0
- package/templates/.github/workflows/ci.yml +148 -0
- package/templates/.github/workflows/deploy-uat.yml +95 -0
- package/templates/.github/workflows/nightly.yml +83 -0
- package/templates/.github/workflows/uat-health.yml +41 -0
- package/templates/.husky/pre-push +23 -0
- package/templates/.ignore +28 -0
- package/templates/.opencode/agents/data-engineer.md +37 -0
- package/templates/.opencode/agents/devops-engineer.md +37 -0
- package/templates/.opencode/agents/diagnostics-expert.md +37 -0
- package/templates/.opencode/agents/docs.md +37 -0
- package/templates/.opencode/agents/features.md +21 -0
- package/templates/.opencode/agents/frontend-engineer.md +37 -0
- package/templates/.opencode/agents/handoff.md +21 -0
- package/templates/.opencode/agents/mobile-engineer.md +37 -0
- package/templates/.opencode/agents/planner.md +34 -0
- package/templates/.opencode/agents/product-manager.md +29 -0
- package/templates/.opencode/agents/qa-architect.md +27 -0
- package/templates/.opencode/agents/qa-runner.md +17 -0
- package/templates/.opencode/agents/security-engineer.md +37 -0
- package/templates/.opencode/agents/system-architect.md +37 -0
- package/templates/.opencode/agents/ui-designer.md +37 -0
- package/templates/.opencode/commands/board.md +15 -0
- package/templates/.opencode/commands/checkpoint.md +25 -0
- package/templates/.opencode/commands/feature.md +56 -0
- package/templates/.opencode/commands/finish.md +36 -0
- package/templates/.opencode/commands/next.md +49 -0
- package/templates/.opencode/commands/report.md +55 -0
- package/templates/.opencode/commands/resume.md +39 -0
- package/templates/.opencode/commands/switch-model.md +34 -0
- package/templates/.opencode/commands/tidy.md +49 -0
- package/templates/.opencode/plugins/backlog-notify.ts +57 -0
- package/templates/.opencode/plugins/backlog.ts +272 -0
- package/templates/.opencode/plugins/checkpoint.ts +160 -0
- package/templates/.opencode/plugins/context-guard.ts +1164 -0
- package/templates/.opencode/plugins/diff.ts +85 -0
- package/templates/.opencode/plugins/gates.ts +71 -0
- package/templates/.opencode/plugins/job-notify.ts +241 -0
- package/templates/.opencode/plugins/orient.ts +67 -0
- package/templates/.opencode/plugins/session.ts +89 -0
- package/templates/.opencode/plugins/wait.ts +101 -0
- package/templates/.opencode/plugins/where.ts +70 -0
- package/templates/.opencode/skills/bounded-explore/SKILL.md +38 -0
- package/templates/.opencode/skills/dev-loop/SKILL.md +87 -0
- package/templates/.vercelignore +16 -0
- package/templates/AGENTS.md +27 -0
- package/templates/BACKLOG.md +95 -0
- package/templates/docs/ci-cd-pipeline.md +81 -0
- package/templates/docs/gates.md +128 -0
- package/templates/docs/mcp-integrations.md +97 -0
- package/templates/docs/mobile-testing.md +555 -0
- package/templates/docs/nightly-testing.md +106 -0
- package/templates/docs/wsl-optimizations.md +290 -0
- package/templates/opencode.json +188 -0
- package/templates/scripts/agent-report.js +541 -0
- package/templates/scripts/agent-search.sh +117 -0
- package/templates/scripts/backlog.js +1924 -0
- package/templates/scripts/board-service.sh +214 -0
- package/templates/scripts/checkpoint.js +290 -0
- package/templates/scripts/finalize.js +416 -0
- package/templates/scripts/nightly.js +913 -0
- package/templates/scripts/orient.js +514 -0
- package/templates/scripts/qa/board-client.js +323 -0
- package/templates/scripts/qa/board-lib.js +96 -0
- package/templates/scripts/qa/board-sync.js +499 -0
- package/templates/scripts/qa/capacity-lib.js +916 -0
- package/templates/scripts/qa/check-backlog.js +434 -0
- package/templates/scripts/qa/check-capacity.js +85 -0
- package/templates/scripts/qa/check-ci-policy.js +164 -0
- package/templates/scripts/qa/check-concurrency.js +344 -0
- package/templates/scripts/qa/check-dispatch-config.js +296 -0
- package/templates/scripts/qa/check-docs.js +288 -0
- package/templates/scripts/qa/check-integrate.js +98 -0
- package/templates/scripts/qa/check-linger.js +160 -0
- package/templates/scripts/qa/check-prefix-stability.js +187 -0
- package/templates/scripts/qa/check-product.js +177 -0
- package/templates/scripts/qa/check-qa-plan.js +378 -0
- package/templates/scripts/qa/check-qa-scripts.js +85 -0
- package/templates/scripts/qa/check-quality.js +249 -0
- package/templates/scripts/qa/check-test-wiring.js +253 -0
- package/templates/scripts/qa/check-worktrees.js +93 -0
- package/templates/scripts/qa/ci-triage.js +346 -0
- package/templates/scripts/qa/ci-usage-lib.js +185 -0
- package/templates/scripts/qa/ci-wait-lib.js +294 -0
- package/templates/scripts/qa/command-classifier.js +552 -0
- package/templates/scripts/qa/context-guard-core.js +1618 -0
- package/templates/scripts/qa/diff.js +174 -0
- package/templates/scripts/qa/feature-scaffold.js +224 -0
- package/templates/scripts/qa/fixtures/backlog.md +104 -0
- package/templates/scripts/qa/governance.js +174 -0
- package/templates/scripts/qa/hash-file.js +32 -0
- package/templates/scripts/qa/lease-lib.js +263 -0
- package/templates/scripts/qa/merge-backlog.js +324 -0
- package/templates/scripts/qa/mobile-e2e/analyze-timings.js +300 -0
- package/templates/scripts/qa/mobile-e2e/apk-cache.js +420 -0
- package/templates/scripts/qa/mobile-e2e/backend-build-cache.js +203 -0
- package/templates/scripts/qa/mobile-e2e/boot-emulator.sh +432 -0
- package/templates/scripts/qa/mobile-e2e/build-apk.sh +210 -0
- package/templates/scripts/qa/mobile-e2e/check-flows.js +568 -0
- package/templates/scripts/qa/mobile-e2e/config.yaml +18 -0
- package/templates/scripts/qa/mobile-e2e/device-down.sh +88 -0
- package/templates/scripts/qa/mobile-e2e/device-up.sh +258 -0
- package/templates/scripts/qa/mobile-e2e/ios-simulator-down.sh +72 -0
- package/templates/scripts/qa/mobile-e2e/ios-simulator-up.sh +153 -0
- package/templates/scripts/qa/mobile-e2e/job-events.js +181 -0
- package/templates/scripts/qa/mobile-e2e/prewarm.sh +56 -0
- package/templates/scripts/qa/mobile-e2e/run-detached.sh +222 -0
- package/templates/scripts/qa/mobile-e2e/run-maestro.js +1543 -0
- package/templates/scripts/qa/mobile-e2e/ui-dump.js +327 -0
- package/templates/scripts/qa/models.allowlist.txt +32 -0
- package/templates/scripts/qa/provider-stats.js +878 -0
- package/templates/scripts/qa/run-clean.js +83 -0
- package/templates/scripts/qa/run-gates.js +213 -0
- package/templates/scripts/qa/run-loop.js +220 -0
- package/templates/scripts/qa/session-recon.js +512 -0
- package/templates/scripts/qa/store-lib.js +324 -0
- package/templates/scripts/qa/tool-lib.js +134 -0
- package/templates/scripts/qa/where.js +214 -0
- package/templates/scripts/qa/worktree-hygiene.js +217 -0
- package/templates/scripts/qa/wslconfig-merge.js +157 -0
- package/templates/scripts/serve.js +1328 -0
- package/templates/scripts/setup-nightly.sh +185 -0
- package/templates/scripts/setup-tmpfs.sh +374 -0
- package/templates/scripts/ship.js +586 -0
- package/templates/scripts/switch-reasoning-model.ps1 +101 -0
- package/templates/scripts/switch-reasoning-model.sh +104 -0
- package/templates/scripts/wait-for.js +342 -0
- package/templates/scripts/wsl-gpu.sh +180 -0
- package/templates/scripts/wsl-tune.sh +319 -0
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# MCP integrations (project-side, per-agent)
|
|
2
|
+
|
|
3
|
+
Model Context Protocol (MCP) servers give an agent live external state (the dev
|
|
4
|
+
database, a browser, a tracker) through tools. They are powerful and expensive:
|
|
5
|
+
**every MCP tool schema is billed on every request that can see it**, and opencode
|
|
6
|
+
spawns the server itself — so it bypasses the `serve.js` machine-capacity envelope
|
|
7
|
+
and consumes RAM per session. This project wires MCP **project-side, per-agent,
|
|
8
|
+
dev-only**. Read this before adding or changing an MCP server.
|
|
9
|
+
|
|
10
|
+
## The two rules (non-negotiable)
|
|
11
|
+
|
|
12
|
+
1. **Per agent, never global.** Disable every MCP tool at the top level, then
|
|
13
|
+
re-enable the server's tools only for the persona that genuinely needs them.
|
|
14
|
+
An MCP tool the orchestrator never uses is pure context tax.
|
|
15
|
+
2. **Development data only.** An MCP server that reaches a database must point at
|
|
16
|
+
the `development` database (`DATABASE_URL`). Never uat/prod — the forward
|
|
17
|
+
boundary in `.agents/memory/environments.md` still applies. Prefer a read-only
|
|
18
|
+
or restricted mode where the server offers one.
|
|
19
|
+
|
|
20
|
+
## Config shape
|
|
21
|
+
|
|
22
|
+
`opencode.json` (top level):
|
|
23
|
+
|
|
24
|
+
```jsonc
|
|
25
|
+
{
|
|
26
|
+
"mcp": {
|
|
27
|
+
"servers": {
|
|
28
|
+
"pg": {
|
|
29
|
+
"type": "local",
|
|
30
|
+
"command": ["npx", "-y", "prisma", "mcp"],
|
|
31
|
+
"disabled": false
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
},
|
|
35
|
+
// Deny ALL of the server's tools globally (the action prefix is the server
|
|
36
|
+
// name + "_"; V2 permissions replace the V1 top-level `tools` disable).
|
|
37
|
+
"permissions": [{ "action": "pg_*", "resource": "*", "effect": "deny" }],
|
|
38
|
+
"agents": {
|
|
39
|
+
// Re-enable only for the personas that need live DB state (agent rules are
|
|
40
|
+
// appended after global rules, so a later allow wins).
|
|
41
|
+
"data-engineer": { "permissions": [{ "action": "pg_*", "resource": "*", "effect": "allow" }] },
|
|
42
|
+
"diagnostics-expert": { "permissions": [{ "action": "pg_*", "resource": "*", "effect": "allow" }] }
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
- MCP tools are registered **prefixed by the server name** (`pg_*`), so in V2 the
|
|
48
|
+
deny/allow rule is an `action` of `"pg_*"` (the form used above). If action
|
|
49
|
+
wildcards do not match in your V2 build, list per-tool rules instead.
|
|
50
|
+
- `--yes`/`-y` on `npx` avoids an interactive install prompt at startup.
|
|
51
|
+
- Keep a secret URL out of the tracked config: use `{env:...}` or a local
|
|
52
|
+
untracked override. `npx` servers inherit the project `.env` (dev) as usual.
|
|
53
|
+
|
|
54
|
+
## Included servers in this project
|
|
55
|
+
|
|
56
|
+
| Server | Package | Enabled for | Notes |
|
|
57
|
+
|---|---|---|---|
|
|
58
|
+
| `pg` (Prisma/Postgres) | `npx -y prisma mcp` | `data-engineer`, `diagnostics-expert` | schema- and migration-aware; **dev DB only** |
|
|
59
|
+
| `playwright` | `npx -y @playwright/mcp@latest --headless` | `qa-runner`, `diagnostics-expert` | interactive web-E2E debugging; run against the `serve.js` dev server |
|
|
60
|
+
|
|
61
|
+
**GitHub MCP is deliberately not enabled.** It is the most token-heavy server and
|
|
62
|
+
is mostly obviated by the direct-land flow (`ship` + `gh` + `wait --ci`). Add it
|
|
63
|
+
only on a concrete demonstrated need, per-agent (`devops-engineer`) only.
|
|
64
|
+
|
|
65
|
+
## Capacity
|
|
66
|
+
|
|
67
|
+
A local MCP server is a process opencode spawns — it is **not** counted by
|
|
68
|
+
`serve.js`. On a small WSL2 VM a heavy server (a browser, an indexer) per parallel
|
|
69
|
+
session can exhaust RAM. Prefer remote servers (no local RSS) or one local server
|
|
70
|
+
enabled for a single agent. Watch RSS with `node scripts/serve.js list` /
|
|
71
|
+
`npm run test:capacity`.
|
|
72
|
+
|
|
73
|
+
## Verify
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
opencode mcp list # server status + auth
|
|
77
|
+
opencode mcp debug <name> # connection/auth diagnostics
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Deploy safety
|
|
81
|
+
|
|
82
|
+
`agent-harness update` deep-merges `opencode.json` by key: it preserves this
|
|
83
|
+
project's `mcp` servers and its `permissions`. If the harness ever ships a default
|
|
84
|
+
`mcp` block it merges **by server name** (project wins), so a project's server is
|
|
85
|
+
never clobbered. Sources: `mergeOpencodeJson` / `mergeMcp` in
|
|
86
|
+
`bin/agent-harness.js`.
|
|
87
|
+
|
|
88
|
+
**V1 `tools` gating is migrated, not preserved.** V2 has no top-level or
|
|
89
|
+
per-agent `tools` field (only `permissions`), so an older project that gates MCP
|
|
90
|
+
the V1 way (`"tools": { "pg*": false }` plus `agents.<name>.tools: { "pg*": true }`)
|
|
91
|
+
would otherwise be silently broken: the server reports *connected* while no agent
|
|
92
|
+
can call its tools. `update` translates both maps into ordered rules — the
|
|
93
|
+
top-level map into global `{ "action": "<name>", "resource": "*", "effect":
|
|
94
|
+
"deny" }` rules and each `agents.<name>.tools` map into that agent's own rules
|
|
95
|
+
(an `allow` wins the global deny, since agent rules are appended last) — and drops
|
|
96
|
+
the V1 `tools` keys. `agent-harness doctor` warns while a project still declares
|
|
97
|
+
`tools`; re-run `update` to convert it.
|
|
@@ -0,0 +1,555 @@
|
|
|
1
|
+
# Mobile / device testing runbook
|
|
2
|
+
|
|
3
|
+
How an agent boots the Android emulator, installs a build, and runs Maestro
|
|
4
|
+
flows **without freezing its own tool call**. The mechanics here are the reason
|
|
5
|
+
this doc exists: the naïve way to background the emulator hangs the agent's
|
|
6
|
+
`shell` tool forever (no error, no timeout), which looks like "the tests are
|
|
7
|
+
taking ages" when in fact nothing is running at all.
|
|
8
|
+
|
|
9
|
+
## The rule
|
|
10
|
+
|
|
11
|
+
Any long-lived process — emulator, Metro/dev server, adb server, Maestro driver —
|
|
12
|
+
must be launched **detached with its own stdio**, via `scripts/serve.js`:
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
node scripts/serve.js start --cmd "<command>" [--url <health url> | --port <n>] [--name <label>] [--timeout <ms>]
|
|
16
|
+
node scripts/serve.js list
|
|
17
|
+
node scripts/serve.js stop --name <label>
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
`serve.js` spawns with `stdio: "ignore"` in a detached session (POSIX) / its own
|
|
21
|
+
hidden console (Windows), records the PID under `.serve/`, and polls the
|
|
22
|
+
readiness target with live heartbeats. The tool call returns as soon as the
|
|
23
|
+
server is ready.
|
|
24
|
+
|
|
25
|
+
A **one-shot** heavy command (the APK/Gradle build, a heavy test tier) must run
|
|
26
|
+
**capped** too — synchronously, via `run`:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
node scripts/serve.js run --cmd "<one-shot command>" [--name <label>] [--mem-max 3G] [--cpu-max 600%]
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
`run` executes the command in a transient systemd scope with a `MemoryMax`,
|
|
33
|
+
streams its output, and returns its exit code; a build that balloons is
|
|
34
|
+
OOM-killed inside its own cgroup instead of thrashing the VM. It leaves no
|
|
35
|
+
`.serve/` record (there is nothing to stop). An **inline** `./gradlew
|
|
36
|
+
assembleRelease` with no cgroup is exactly the 2026-09-30 freeze (Gradle + the
|
|
37
|
+
emulator + the dev server filled the 10 GB VM).
|
|
38
|
+
|
|
39
|
+
## Booting the device tier
|
|
40
|
+
|
|
41
|
+
**One command brings the whole tier up** (adb server + emulator + app build/install
|
|
42
|
+
+ dev server, verified against the health URL), idempotently — reuse it instead of
|
|
43
|
+
hand-rolling the ~15 setup calls a device session otherwise needs:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
bash scripts/qa/mobile-e2e/device-up.sh # full bring-up (reuses what is already up)
|
|
47
|
+
bash scripts/qa/mobile-e2e/device-up.sh --no-dev # emulator + app only
|
|
48
|
+
bash scripts/qa/mobile-e2e/device-down.sh # stop dev server + prune
|
|
49
|
+
bash scripts/qa/mobile-e2e/device-down.sh --emulator # also stop the emulator (session end)
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
`device-up.sh` sequences the lower-level primitives, which you can still call
|
|
53
|
+
directly. The **adb-server + emulator** primitive brings the adb server up first
|
|
54
|
+
(detached via `serve.js`, because a bare `adb devices`/`adb start-server` hangs
|
|
55
|
+
forever when the server is down — the forked daemon is reaped with the tool's
|
|
56
|
+
process group), then the emulator, then runs a **bounded** readiness gate. It is
|
|
57
|
+
safe to re-run — an already-booted emulator and a live adb server are reused — and
|
|
58
|
+
it reads the AVD from `scripts/qa/mobile-e2e/local.json` / `MOBILE_E2E_AVD`:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
node scripts/qa/mobile-e2e/boot-emulator.sh # auto-sizes CPU/RAM/GPU to the box
|
|
62
|
+
node scripts/qa/mobile-e2e/boot-emulator.sh --print-profile # preview the resolved profile as JSON
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
It expands to (adb server, then emulator, then a bounded gate):
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
node scripts/serve.js prune
|
|
69
|
+
node scripts/serve.js start --cmd "~/android-sdk/platform-tools/adb -a nodaemon server" \
|
|
70
|
+
--name adb-server --port 5037 --timeout 30000
|
|
71
|
+
node scripts/serve.js start --group mobile \
|
|
72
|
+
--cmd "~/android-sdk/emulator/emulator -avd <avd> -no-window -no-audio -no-boot-anim -no-snapshot -cores <n> -memory <mb> -gpu <backend>" \
|
|
73
|
+
--name emulator --port 5554 --timeout 180000
|
|
74
|
+
timeout 90 adb wait-for-device
|
|
75
|
+
timeout 10 adb shell getprop sys.boot_completed # must print 1
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
**Right-sizing** (the device tier is the slowest, heaviest part of a run):
|
|
79
|
+
`boot-emulator.sh` derives the profile instead of hardcoding it —
|
|
80
|
+
`cores = min(AVD hw.cpu.ncore, nproc − reserve)`, guest RAM from the envelope's
|
|
81
|
+
`emulator_ram_mb` (the single owner of that number), and the GPU backend is
|
|
82
|
+
auto-detected (`host` only when a real renderer answers, else
|
|
83
|
+
`swiftshader_indirect`; `-gpu host` on a box whose only renderer is `llvmpipe` is
|
|
84
|
+
software GL anyway). Override any of them with `MOBILE_E2E_CORES`,
|
|
85
|
+
`MOBILE_E2E_MEMORY`, `MOBILE_E2E_GPU`.
|
|
86
|
+
|
|
87
|
+
The **display** is a separate latency lever: `MOBILE_E2E_RESOLUTION=WxH` and
|
|
88
|
+
`MOBILE_E2E_DENSITY=dpi` are applied after boot with `wm size`/`wm density`
|
|
89
|
+
(validated, never interpolated raw into an adb shell). Scale size and density
|
|
90
|
+
together — e.g. `720x1600` @ `280` keeps the same dp geometry as the AVD's
|
|
91
|
+
`1080x2400` @ `420`, so selectors are unchanged while the emulator moves ~2.3×
|
|
92
|
+
fewer pixels per command.
|
|
93
|
+
|
|
94
|
+
### iOS Simulator (`platform.ios`)
|
|
95
|
+
|
|
96
|
+
The driver is platform-aware. Select iOS with `MOBILE_E2E_PLATFORM=ios` or
|
|
97
|
+
`"platform": "ios"` in `scripts/qa/mobile-e2e/local.json` (env wins; default is
|
|
98
|
+
`android`). The iOS route needs **macOS with Xcode command-line tools** — it
|
|
99
|
+
cannot boot a Simulator on Linux — and brings up the tier with `xcrun simctl`
|
|
100
|
+
instead of adb/emulator:
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
node scripts/qa/mobile-e2e/ios-simulator-up.sh # boot/reuse sim + install the unsigned .app
|
|
104
|
+
node scripts/qa/mobile-e2e/device-up.sh # platform=ios: skips Gradle/adb, boots sim + dev server
|
|
105
|
+
node scripts/qa/mobile-e2e/run-maestro.js --flows all --platform ios
|
|
106
|
+
node scripts/qa/mobile-e2e/ios-simulator-down.sh # shutdown (add --delete to remove the device)
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`ios-simulator-up.sh` reads the `ios` block from `local.json` — `bundleId`,
|
|
110
|
+
`deviceName`, `deviceType`, `appPath` (the unsigned simulator `.app`) — and emits
|
|
111
|
+
`UDID=…` / `BUNDLE_ID=…`; in CI it appends `MOBILE_E2E_IOS_UDID` /
|
|
112
|
+
`MOBILE_E2E_IOS_BUNDLE_ID` to `$GITHUB_ENV`. It fails loudly when `xcrun` is
|
|
113
|
+
missing (not macOS), the `.app` is absent (build it first with an EAS local
|
|
114
|
+
unsigned profile), or no iOS runtime is installed.
|
|
115
|
+
|
|
116
|
+
`boot-emulator.sh` owns the Android bring-up only: on `platform=ios` it delegates
|
|
117
|
+
to `ios-simulator-up.sh`. `device-up.sh` skips the APK/Gradle build, the adb
|
|
118
|
+
server, and the adb install/readiness gate on iOS (the `.app` install happens in
|
|
119
|
+
the up script), and still starts the dev server in the `mobile` group and gates
|
|
120
|
+
on its health URL. `run-maestro.js` resolves the Simulator UDID (explicit
|
|
121
|
+
`--udid`, else `MOBILE_E2E_IOS_UDID`, else a booted sim matched to
|
|
122
|
+
`ios.deviceName` via `xcrun simctl list devices booted`), skips the adb preflight,
|
|
123
|
+
and passes `--device <udid>` to every `maestro test`. `config.yaml` disables
|
|
124
|
+
animations on both platforms.
|
|
125
|
+
|
|
126
|
+
A Simulator has no camera: a camera-based journey (e.g. scan-add) is **not**
|
|
127
|
+
covered on iOS — that stays a physical-device check.
|
|
128
|
+
|
|
129
|
+
```jsonc
|
|
130
|
+
// scripts/qa/mobile-e2e/local.json
|
|
131
|
+
{
|
|
132
|
+
"platform": "ios",
|
|
133
|
+
"ios": {
|
|
134
|
+
"bundleId": "com.example.app",
|
|
135
|
+
"deviceName": "iOS Simulator",
|
|
136
|
+
"deviceType": "com.apple.CoreSimulator.SimDeviceType.iPhone-15",
|
|
137
|
+
"appPath": "mobile/build/ios-simulator-app.app"
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### GPU acceleration on WSL2 (the biggest wall-clock lever)
|
|
143
|
+
|
|
144
|
+
On a software renderer the emulator is CPU-bound — ~2.5 s/command, ~3.6 s/tap
|
|
145
|
+
(F-291/F-296), pinned near 3 cores just compositing. WSL2 can pass the Windows
|
|
146
|
+
GPU through, but Mesa does **not** use it by default, and the DRM node is often
|
|
147
|
+
missing too, so `boot-emulator.sh` silently resolves to `swiftshader_indirect`
|
|
148
|
+
even on a capable box. Two things are needed, both handled by
|
|
149
|
+
`scripts/wsl-gpu.sh`:
|
|
150
|
+
|
|
151
|
+
1. **Load `vgem`.** Modern WSL kernels (≥ 2.5) ship the DRM module but do not
|
|
152
|
+
auto-load it, so `/dev/dri` is absent and every GL app falls back to
|
|
153
|
+
`llvmpipe` (WSL issues [#12702](https://github.com/microsoft/WSL/issues/12702),
|
|
154
|
+
[#11837](https://github.com/microsoft/WSL/issues/11837)).
|
|
155
|
+
2. **Select the `d3d12` driver.** Even with `/dev/dri` present, Mesa still picks
|
|
156
|
+
`llvmpipe` unless `GALLIUM_DRIVER=d3d12` (with `/usr/lib/wsl/lib` on the
|
|
157
|
+
loader path) is set.
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
bash scripts/wsl-gpu.sh status # diagnose: /dev/dxg, vgem, d3d12 renderer
|
|
161
|
+
bash scripts/wsl-gpu.sh apply # load + persist vgem, verify d3d12 (auto-sudo)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Once active, `glxinfo -B` reports `D3D12 (<Windows GPU>)` and `Accelerated: yes`,
|
|
165
|
+
and `boot-emulator.sh --print-profile` resolves `gpu: "host"` with a `gpuEnv`
|
|
166
|
+
that exports the two variables to the emulator (serve.js forwards the caller's
|
|
167
|
+
env), plus a `gpuExtra` of `-feature -Vulkan`. **Vulkan must be disabled on this
|
|
168
|
+
path**: the emulator's gfxstream Vulkan backend segfaults qemu mid-boot under
|
|
169
|
+
Mesa `d3d12` (the `Segmentation fault (core dumped)` that made every F-296 run
|
|
170
|
+
fall back to the software rasterizer), while GLES-over-`host` on d3d12 is stable
|
|
171
|
+
and hardware-accelerated. Verified on this box: `-gpu host -feature -Vulkan`
|
|
172
|
+
boots to `D3D12 (Intel(R) Graphics)` in ~20 s; `-gpu host` without it crashes. Verification on this box: before — `llvmpipe`, `Accelerated: no`; after —
|
|
173
|
+
`D3D12 (Intel(R) Graphics)`, `Accelerated: yes`. The script is a no-op outside
|
|
174
|
+
WSL (no `/dev/dxg`). The Huawei/Xiaomi/etc. GPU must have a WSL-capable driver
|
|
175
|
+
(the Intel/AMD/NVIDIA "WSL" driver); if `apply` loads `vgem` but d3d12 still
|
|
176
|
+
reports software, `wsl --shutdown` from Windows and retry.
|
|
177
|
+
|
|
178
|
+
The emulator joins the machine-wide **`mobile` tier group** (`--group mobile`):
|
|
179
|
+
the emulator and the dev server are two heavy processes that legitimately run
|
|
180
|
+
together for device E2E, so they share ONE budgeted tier instead of tripping the
|
|
181
|
+
single-heavy-tier lock. Start the dev server in the same group (see below). A
|
|
182
|
+
third, unrelated heavy tier is still refused.
|
|
183
|
+
|
|
184
|
+
The group shares ONE **combined** memory budget (`group_mem_max_mb`, default 7 GB):
|
|
185
|
+
each member's per-process cap is generous (a 2048 MB AVD peaks ~4.4 GB RSS), so
|
|
186
|
+
`serve.js` tightens a new member to the group's remaining budget instead of letting
|
|
187
|
+
emulator + dev server each take 5 GB and fill the VM. An explicit `--mem-max` wins;
|
|
188
|
+
a near-exhausted budget refuses (`--force` overrides).
|
|
189
|
+
|
|
190
|
+
**Both members set an explicit `--mem-max`** (the group budget sums *caps*, not
|
|
191
|
+
usage, so leaving them implicit lets the first-started member claim the full
|
|
192
|
+
`heavy_mem_max_mb` and tightens the second to the tiny remainder — which pinned
|
|
193
|
+
the emulator at exactly its guest RAM, ~95 k reclaim events):
|
|
194
|
+
|
|
195
|
+
| Member | Cap | Override |
|
|
196
|
+
| --- | --- | --- |
|
|
197
|
+
| dev server (`next dev`) | `2560 MB` | `MOBILE_E2E_DEV_MEM_MAX` |
|
|
198
|
+
| emulator (`qemu`) | `guest RAM + 2048 MB` | `MOBILE_E2E_MEM_MAX` |
|
|
199
|
+
|
|
200
|
+
Keep `devCap + emuCap ≤ group_mem_max_mb` (2560 + 4096 = 6656 ≤ 7168). If you
|
|
201
|
+
raise `emulator_ram_mb`, the emulator cap scales with it and you may need to raise
|
|
202
|
+
`group_mem_max_mb` too; on a software renderer (`swiftshader_indirect`) set
|
|
203
|
+
`MOBILE_E2E_MEM_MAX` higher, since QEMU's RSS there is well above guest RAM.
|
|
204
|
+
|
|
205
|
+
#### Optional: run the backend as a production server
|
|
206
|
+
|
|
207
|
+
`next dev` compiles on demand and swings up to ~1.9 GB, competing with the
|
|
208
|
+
emulator for the shared group budget. A production backend (`next build` once,
|
|
209
|
+
then `next start`) is far leaner and steadier across the long matrix run. Opt in
|
|
210
|
+
per project in `local.json`:
|
|
211
|
+
|
|
212
|
+
```json
|
|
213
|
+
{
|
|
214
|
+
"devCmd": "npm run start",
|
|
215
|
+
"devBuildCmd": "npm run build",
|
|
216
|
+
"devBuildCache": ["app/**", "lib/**", "next.config.*", "package.json"],
|
|
217
|
+
"devBuildOutput": ".next/BUILD_ID"
|
|
218
|
+
}
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
When `devBuildCmd` is set, `device-up.sh` runs it as a **capped one-shot**
|
|
222
|
+
(`serve.js run`, group `mobile`) after the APK build and before the emulator, so
|
|
223
|
+
two heavy one-shots never overlap; a failed build aborts the bring-up. The
|
|
224
|
+
`devBuildCache` globs are resolved through `git ls-files` (so `node_modules`/
|
|
225
|
+
`.next` never enter the hash) and `backend-build-cache.js` skips the build when
|
|
226
|
+
nothing under them changed **and** `devBuildOutput` (when declared) still exists
|
|
227
|
+
— a deleted build is a miss, never a skip that leaves `next start` with nothing.
|
|
228
|
+
Force a rebuild with `MOBILE_E2E_DEV_BUILD_FORCE=1`;
|
|
229
|
+
override the command with `MOBILE_E2E_DEV_BUILD_CMD`. With no `devBuildCmd` the
|
|
230
|
+
behavior is unchanged (`npm run dev`, no build).
|
|
231
|
+
|
|
232
|
+
### Building the APK (always capped)
|
|
233
|
+
|
|
234
|
+
`scripts/qa/mobile-e2e/build-apk.sh` builds the release APK: it runs `expo prebuild`
|
|
235
|
+
only when `mobile/android/` is absent, patches `usesCleartextTraffic` into the
|
|
236
|
+
generated manifest, right-sizes Gradle (no daemon, one worker, one ABI, 1 GB heap),
|
|
237
|
+
and runs `assembleRelease` through `serve.js run` (default `mem≤3G`). Run the
|
|
238
|
+
build BEFORE the dev server is up, and never call `./gradlew assembleRelease`
|
|
239
|
+
inline — that is the 2026-09-30 freeze. `device-up.sh` calls it automatically:
|
|
240
|
+
|
|
241
|
+
Every adb call is `timeout`-wrapped: a wedged adb fails fast instead of hanging
|
|
242
|
+
the tool. `run-maestro.js` also preflights adb (10 s probe) and exits with the
|
|
243
|
+
exact recovery step rather than blocking. `serve.js list`/`start` auto-prune dead
|
|
244
|
+
`.serve/` records, so a stale adb-server/emulator entry from a killed session
|
|
245
|
+
cannot misreport the machine.
|
|
246
|
+
|
|
247
|
+
On Linux with a systemd user bus, `--mem-max`/`--cpu-max` put the emulator in
|
|
248
|
+
its own cgroup: if it runs away it is throttled or OOM-killed *inside the unit*
|
|
249
|
+
rather than dragging the whole WSL VM into swap-thrash. They are ignored with a
|
|
250
|
+
warning elsewhere.
|
|
251
|
+
|
|
252
|
+
- **Size it to the box.** The emulator is a heavy tier: software rendering
|
|
253
|
+
(`-gpu swiftshader_indirect`) pins a core and *still* used ~3 GB in the
|
|
254
|
+
2026-09-29 freeze (QEMU + software-GPU overhead far above `hw.ramSize`) — the
|
|
255
|
+
observed QEMU RSS at a 2048 MB guest is ~4.4 GB. `boot-emulator.sh` right-sizes
|
|
256
|
+
cores to the host (a fixed 2 cores made QEMU the bottleneck on an 8-vCPU box),
|
|
257
|
+
reads guest RAM from the envelope, and picks the GPU backend automatically.
|
|
258
|
+
The emulator's own cgroup cap defaults to `emulator_ram_mb` + ~2 GB
|
|
259
|
+
(`MOBILE_E2E_MEM_MAX`), leaving room for QEMU's host-side overhead and page
|
|
260
|
+
cache instead of pinning it at guest RAM. `serve.js` refuses a heavy launch that would
|
|
261
|
+
exceed the capacity envelope (`.agents/memory/orchestration.md`) — stop another
|
|
262
|
+
tier first, or `--force` to override deliberately.
|
|
263
|
+
- The SDK path and AVD name are per-project. On WSL2 the active AVD is
|
|
264
|
+
`libraryos`. The emulator exposes the adb port at **5554**, which is what
|
|
265
|
+
`--port` probes.
|
|
266
|
+
- A full cold boot takes ~30 s; `--timeout 120000` gives headroom. A start
|
|
267
|
+
prints a "launching" line up front, so an in-progress wait is never silent.
|
|
268
|
+
- On a systemd-less WSL there is no `/run/user/<uid>`; the emulator aborts
|
|
269
|
+
(`Aborted (core dumped)`) without a writable `XDG_RUNTIME_DIR`. `boot-emulator.sh`
|
|
270
|
+
resolves a private temp runtime dir automatically, so this is not a manual step.
|
|
271
|
+
- The dev server is a **separate** detached process, but part of the same device
|
|
272
|
+
tier — start it in the **same group** so the pair is budgeted as one:
|
|
273
|
+
`node scripts/serve.js start --cmd "npm run dev" --url http://localhost:3000 --name app-dev --group mobile`.
|
|
274
|
+
|
|
275
|
+
### Startup speed: content-addressed APK cache + quickboot
|
|
276
|
+
|
|
277
|
+
The cold `assembleRelease` dominates the tier (7-11 min on a fresh worktree; the
|
|
278
|
+
reanimated/worklets C++ compile, new-arch codegen, and dex merge are the cost).
|
|
279
|
+
Almost all of it is invariant across worktrees for a fixed dependency set — only
|
|
280
|
+
the JS bundle changes. `scripts/qa/mobile-e2e/apk-cache.js` content-addresses each
|
|
281
|
+
stage so the invariant work is done once:
|
|
282
|
+
|
|
283
|
+
```
|
|
284
|
+
~/.cache/agent-harness/mobile-e2e/
|
|
285
|
+
├── native/<nativeKey>/base.apk native shell (deps/toolchain; ignores JS)
|
|
286
|
+
├── bundle/<bundleKey>/ Hermes bundle + exported assets (JS sources)
|
|
287
|
+
└── apk/<fullKey>/app-release.apk ready-to-install APK (native + JS)
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
`build-apk.sh` (and therefore `device-up.sh`) tries, in order: **full-key hit →
|
|
291
|
+
overlay → full build**. The overlay re-bundles only JS, drops it into the cached
|
|
292
|
+
native base, and re-signs with the app's debug keystore (`zipalign` +
|
|
293
|
+
`apksigner`); it never touches `resources.arsc`, and it declines (falling back to
|
|
294
|
+
Gradle) if the bundle references an asset the base lacks or if any verify step
|
|
295
|
+
fails. It is a pure accelerator — correctness never depends on it. Measured on
|
|
296
|
+
LibraryOS: **~7 s warm / ~20-30 s cold** for the overlay versus minutes for a
|
|
297
|
+
cold build.
|
|
298
|
+
|
|
299
|
+
| env | default | effect |
|
|
300
|
+
| --- | --- | --- |
|
|
301
|
+
| `MOBILE_E2E_SKIP_CACHE` | `0` | `1` bypasses the cache entirely |
|
|
302
|
+
| `MOBILE_E2E_APK_OVERLAY` | `1` | `0` disables the overlay (full build on JS change) |
|
|
303
|
+
| `MOBILE_E2E_SNAPSHOT` | `1` | `0` disables the emulator quickboot snapshot |
|
|
304
|
+
| `MOBILE_E2E_SNAPSHOT_NAME` | `agent-e2e` | snapshot name |
|
|
305
|
+
| `MOBILE_E2E_OVERLAP` | `1` | `0` starts the dev server after the boot, not with it |
|
|
306
|
+
|
|
307
|
+
- **Gradle reuse.** The managed `gradle.properties` block now enables
|
|
308
|
+
`org.gradle.caching` (shared local cache in `~/.gradle`) and the configuration
|
|
309
|
+
cache, and the assemble runs with `--build-cache --configuration-cache`, so a
|
|
310
|
+
fresh worktree reuses invariant task outputs instead of recompiling.
|
|
311
|
+
- **Prewarm off the critical path.** `scripts/qa/mobile-e2e/prewarm.sh` warms the
|
|
312
|
+
cache (hit/overlay/build, capped) and is idempotent; `prewarm.sh --detach`
|
|
313
|
+
runs it in the background. Set `AGENT_HARNESS_PREWARM=1` to have
|
|
314
|
+
`agent-harness update` launch it automatically. It obeys the capacity envelope
|
|
315
|
+
(`prewarm.sh` classifies as an "apk build" one-shot), so it refuses when a
|
|
316
|
+
heavy tier is live rather than thrashing the VM.
|
|
317
|
+
- **Quickboot snapshot.** `boot-emulator.sh` boots with `-snapshot agent-e2e`
|
|
318
|
+
when enabled, so a warm snapshot turns the ~24-31 s cold boot into seconds.
|
|
319
|
+
This VHD/AVD keeps its data in a qcow2 overlay and the emulator ships with
|
|
320
|
+
`Feature QuickbootFileBacked` **off** ("disabled due to stability issues"), so
|
|
321
|
+
a snapshot save/load silently fails (`Device 'cache' does not have the
|
|
322
|
+
requested snapshot`) and every bring-up cold-boots. `boot-emulator.sh` now
|
|
323
|
+
forces `ANDROID_EMU_FORCE_QUICKBOOT_FILE_BACKED=1`, which is what makes it
|
|
324
|
+
actually work — measured **9 s wall-clock / ~4 s to restore** vs 24-31 s cold
|
|
325
|
+
on this box. An incomplete save (a stray `snapshot.pb` with no `ram.bin`) is
|
|
326
|
+
discarded up front. Because a snapshot is stateful, a failed boot retries once
|
|
327
|
+
**cold** before the GPU fallback; `device-down.sh --emulator` saves the
|
|
328
|
+
snapshot before stopping and reports a save failure to
|
|
329
|
+
`.serve/snapshot-save.log` instead of swallowing it. Disable it entirely with
|
|
330
|
+
`MOBILE_E2E_SNAPSHOT=0` if quickboot ever proves unstable.
|
|
331
|
+
- **Overlap.** The dev-server start (independent of the device) runs
|
|
332
|
+
concurrently with the emulator boot; the install joins both. `device-up.sh`
|
|
333
|
+
prints `device-up:timings {…}` per phase and appends it to
|
|
334
|
+
`.serve/device-up-timings.jsonl`.
|
|
335
|
+
|
|
336
|
+
### Readiness gate (never `sleep`-and-pray)
|
|
337
|
+
|
|
338
|
+
```bash
|
|
339
|
+
adb wait-for-device
|
|
340
|
+
adb shell getprop sys.boot_completed # must print 1
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
Only then `adb install -r <apk>` and run flows. Both are synchronous one-shots —
|
|
344
|
+
run them inline and let output stream; never pipe a run through a buffering
|
|
345
|
+
filter (`| tail`, redirect-to-file-then-read), or it looks frozen and gets
|
|
346
|
+
aborted mid-flight.
|
|
347
|
+
|
|
348
|
+
## Making a full run fast and observable
|
|
349
|
+
|
|
350
|
+
A full multi-journey Maestro matrix on a software-rendered emulator can take tens
|
|
351
|
+
of minutes. The runner (`scripts/qa/mobile-e2e/run-maestro.js`, harness-owned)
|
|
352
|
+
carries a performance layer; coverage and assertions are unchanged:
|
|
353
|
+
|
|
354
|
+
1. **Launch detached, then get notified — do NOT sleep/poll.** Never block the
|
|
355
|
+
tool for the whole run, and never guess a finish time with `sleep N; grep`.
|
|
356
|
+
The helper runs the matrix with its own session/stdio and writes a
|
|
357
|
+
machine-readable job:
|
|
358
|
+
|
|
359
|
+
```bash
|
|
360
|
+
scripts/qa/mobile-e2e/run-detached.sh --flows all --reuse-session
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
It writes `<jobdir>/<id>.events.jsonl` (one JSON line per scenario: start,
|
|
364
|
+
pass/fail, fail-fast, done, with `durationMs`) plus `<jobdir>/<id>.json`
|
|
365
|
+
(pid + log + events paths). The `job-notify` opencode plugin tails the stream
|
|
366
|
+
and turns it into a **push**: a console-log line per scenario (V2 server
|
|
367
|
+
plugins have no TUI toast; live progress with no LLM turn), and a message
|
|
368
|
+
injected back into the session (`session.prompt`) on the **first failure** and
|
|
369
|
+
at **completion** — so the agent resumes on real results instead of polling.
|
|
370
|
+
The report lands at `.serve/mobile-e2e-last-run.json` and now carries the
|
|
371
|
+
**invocation count, matrix/invocation wall time, and each
|
|
372
|
+
journey's `durationMs`** (parsed from Maestro's per-flow summary in a batch,
|
|
373
|
+
or the invocation wall clock for a single flow), plus a `Slowest journeys:`
|
|
374
|
+
line — the timing map H-34 asked for. A batched failure is matched by its
|
|
375
|
+
extension-free flow key, so it can never be silently reported as a pass (the
|
|
376
|
+
2026-10-01 run lost 2 of its 3 failures that way).
|
|
377
|
+
|
|
378
|
+
**`analyze-timings.js`** is the per-command ruler for the optimization loop.
|
|
379
|
+
It parses each invocation's `~/.maestro/tests/<run>/maestro.log` and reports
|
|
380
|
+
per-kind latency plus **dead waits** — an `(Optional)` command that ends
|
|
381
|
+
WARNED, i.e. burned its whole timeout doing nothing (the 2026-10-02 matrix
|
|
382
|
+
lost 7.4 min to a single re-expiring `dashboard-profile-button` wait):
|
|
383
|
+
|
|
384
|
+
```bash
|
|
385
|
+
node scripts/qa/mobile-e2e/analyze-timings.js --prefix 2026-10-02_
|
|
386
|
+
node scripts/qa/mobile-e2e/analyze-timings.js --dir ~/.maestro/tests/2026-10-02_080659 --json
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
**To block for the run's exit code, prefer the async `wait` TOOL, not a
|
|
390
|
+
`--wait` shell call.** The general `wait` tool (`scripts/wait-for.js`, e.g.
|
|
391
|
+
`wait --job <record.json>`) spawns its child asynchronously, so it does
|
|
392
|
+
**not** freeze opencode's event loop — the plugin's log lines and
|
|
393
|
+
first-failure/completion wakes keep flowing while the agent waits. (Before
|
|
394
|
+
0.1.115 the `wait` tool used `execFileSync` and starved the notifier: every
|
|
395
|
+
event flushed in one burst when the wait returned, which is why the F-296 run
|
|
396
|
+
looked frozen.)
|
|
397
|
+
|
|
398
|
+
`--wait` on the helper is the **headless fallback** (no plugin watching, e.g.
|
|
399
|
+
a bare CI shell). It blocks the calling shell for the entire run:
|
|
400
|
+
|
|
401
|
+
```bash
|
|
402
|
+
# headless only — from an agent `shell` call this holds the tool for the whole
|
|
403
|
+
# matrix and leaves the transcript blind (the F-296 resume did exactly this).
|
|
404
|
+
scripts/qa/mobile-e2e/run-detached.sh --flows all --reuse-session --wait
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
Because that foot-gun recurs, `run-detached.sh` prints a loud warning before
|
|
408
|
+
blocking. From an agent session: launch detached, then either let the plugin
|
|
409
|
+
push, or call the `wait` tool — never `--wait`.
|
|
410
|
+
|
|
411
|
+
2. **`--reuse-session`** reuses the device session and signs in only when there
|
|
412
|
+
is none, batching consecutive same-actor, setup-free journeys into one
|
|
413
|
+
`maestro test` invocation (fewer JVMs). The shared
|
|
414
|
+
`flows/_shared/session.yaml` subflow **never `clearState`s** — it launches
|
|
415
|
+
without clearing, signs the current actor out when the journey's `actor:`
|
|
416
|
+
(in `flows/matrix.yaml`) changes (`MOBILE_E2E_SWITCH=1`), and signs in as the
|
|
417
|
+
target actor. This is the single biggest wall-clock win. Omit it for a strict
|
|
418
|
+
per-journey run (the default still switches session per flow, but no longer
|
|
419
|
+
clears app data). The runner also **regroups the selected flows by actor**
|
|
420
|
+
(firstLogin first, then each actor's standalone setup/oracle/`endsAs` flows
|
|
421
|
+
before its setup-free run) so a batch is never interrupted by an unrelated
|
|
422
|
+
flow. Alphabetical order otherwise scattered an actor's journeys among
|
|
423
|
+
standalone flows — the F-296 run batched just 2 of 26 journeys (24
|
|
424
|
+
invocations).
|
|
425
|
+
|
|
426
|
+
**First login is one dedicated scenario.** The first-login walkthrough is
|
|
427
|
+
gated on a per-member `AsyncStorage` flag, and `clearState` (`pm clear`)
|
|
428
|
+
wipes it for *every* member. So exactly one journey declares
|
|
429
|
+
`firstLogin: true` in `flows/matrix.yaml`: it is run **first** and alone, it
|
|
430
|
+
is the only place that `clearState`s, and it asserts the fresh
|
|
431
|
+
onboarding/walkthrough. Every other journey reuses a mixed set of sessions
|
|
432
|
+
and never sees the walkthrough. Ordering the `clearState` first stops its
|
|
433
|
+
app-data wipe from resetting the sessions the rest of the run depends on.
|
|
434
|
+
3. **Animations are disabled** for the run by `config.yaml`
|
|
435
|
+
(`platform.android.disableAnimations: true` and
|
|
436
|
+
`platform.ios.disableAnimations: true`, passed via `maestro test --config`),
|
|
437
|
+
removing the per-command animation-settle wait on either platform. Override
|
|
438
|
+
with `--config <file>`.
|
|
439
|
+
4. **`--no-seed`** runs `reset.js --no-seed` (mint actor auth, keep the seeded
|
|
440
|
+
data) so a re-verification run skips the full DB seed.
|
|
441
|
+
5. **`--failed`** re-runs only the journeys that failed last time, from
|
|
442
|
+
`.serve/mobile-e2e-last-run.json` (written after every run). Iterate on the
|
|
443
|
+
failing subset, then do one full pass at the end.
|
|
444
|
+
6. **`--max-same-reason`** (default 3) is the fail-fast stop. Leave the default
|
|
445
|
+
for a verification run so a systemic break stops early; pass `0` only for a
|
|
446
|
+
full attribution sweep that must touch every journey.
|
|
447
|
+
7. The Maestro driver APKs are **not reinstalled** per invocation
|
|
448
|
+
(`--no-reinstall-driver`); pass `--reinstall-driver` only if the driver is
|
|
449
|
+
suspect.
|
|
450
|
+
8. **Setup batching (opt-in).** A journey that declares an additive `setup:` may
|
|
451
|
+
add `batchSetup: true` in `matrix.yaml` so the runner runs its setup before a
|
|
452
|
+
shared warm invocation instead of forcing a standalone one. It is refused
|
|
453
|
+
(and the journey stays standalone) if its setup clears app data (`pm clear`/
|
|
454
|
+
`clearState`), declares an `oracle:`, is `firstLogin:`, or changes actor
|
|
455
|
+
(`endsAs`). Off by default: only opt in after a device run confirms the
|
|
456
|
+
setups are order-independent.
|
|
457
|
+
9. **Scroll visibility.** `scrollUntilVisible` defaults to 100 % visibility,
|
|
458
|
+
which flakes on a bottom-edge element (the `book-detail`
|
|
459
|
+
`book-remove-arm` false negative: the node *was* on screen). The flows set
|
|
460
|
+
`visibilityPercentage: 50` so a partially-scrolled target is accepted.
|
|
461
|
+
When a target's position varies with content height (the same
|
|
462
|
+
`book-remove-arm` is sometimes already on-screen and sometimes below the
|
|
463
|
+
fold), guard the scroll with `runFlow: when: notVisible:` so the swipe runs
|
|
464
|
+
only when the target is genuinely off-screen — a no-op swipe on an
|
|
465
|
+
already-visible bottom-edge row can graze the tab bar and flake the step
|
|
466
|
+
(F-313).
|
|
467
|
+
|
|
468
|
+
Project-specific defaults (app id, default actor/AVD, the dev-server and
|
|
469
|
+
optional backend-build commands) live in `scripts/qa/mobile-e2e/local.json` —
|
|
470
|
+
project-owned, not overwritten by the harness. The mechanism (`run-maestro.js`,
|
|
471
|
+
`check-flows.js`, `config.yaml`, `boot-emulator.sh`, `device-up.sh`,
|
|
472
|
+
`ios-simulator-up.sh`, `ios-simulator-down.sh`, `backend-build-cache.js`,
|
|
473
|
+
`run-detached.sh`, `job-events.js`) plus the
|
|
474
|
+
`job-notify` plugin are harness-owned under `templates/`, so fix them upstream
|
|
475
|
+
and `agent-harness update` the projects.
|
|
476
|
+
|
|
477
|
+
## Reuse vs. stop
|
|
478
|
+
|
|
479
|
+
- **Reuse** the emulator across runs *within a session* — leave it booted
|
|
480
|
+
between tiers so a cold boot is not paid twice. Never kill an emulator you
|
|
481
|
+
didn't start.
|
|
482
|
+
- **Stop the emulator at session end**, and a dev server as soon as its tier is
|
|
483
|
+
done: `node scripts/serve.js stop --name <label>` (or `stop-all`). An idle
|
|
484
|
+
emulator left up for hours held ~3 GB and was a direct contributor to the
|
|
485
|
+
2026-09-29 freeze. You do not have to remember: `serve.js` records the owning
|
|
486
|
+
session and **auto-closes** the serve — `reap` stops one whose owner's lease is
|
|
487
|
+
gone/stale (or an untagged one past `idle_reap_minutes` with no recent log
|
|
488
|
+
activity) and reconciles an untracked `agent-serve-*` unit from a killed
|
|
489
|
+
session. `reap` runs at every `start` and when the context-guard plugin sees
|
|
490
|
+
`session.deleted`, so an abandoned emulator/dev server is not kept alive.
|
|
491
|
+
- `node scripts/serve.js list` is the machine view: capacity (free RAM / swap /
|
|
492
|
+
load / live serves) plus each process's age and whether it is heavy. Treat an
|
|
493
|
+
old process as an idle-reap candidate; `npm run test:capacity` fails when the
|
|
494
|
+
envelope is already exceeded.
|
|
495
|
+
|
|
496
|
+
## Diagnosing a failed step (UI hierarchy dump)
|
|
497
|
+
|
|
498
|
+
When a Maestro step fails (`Element not found`, `assertVisible …`), the fix starts
|
|
499
|
+
from the hierarchy the app actually rendered — which testIDs, text, and bounds
|
|
500
|
+
were on screen. Read it with the sanctioned probe, **not** a hand-rolled
|
|
501
|
+
`adb … uiautomator dump` / `python3` over `~/.maestro` (those trip the capacity
|
|
502
|
+
and tool guards, and were the F-311 dead end):
|
|
503
|
+
|
|
504
|
+
```bash
|
|
505
|
+
# Live: dump the current on-device hierarchy (lightweight; safe while the tier is up)
|
|
506
|
+
node scripts/qa/mobile-e2e/ui-dump.js --live
|
|
507
|
+
node scripts/qa/mobile-e2e/ui-dump.js --live --filter 'confirm-add|search-result' # narrow the tree
|
|
508
|
+
|
|
509
|
+
# Offline: read Maestro's saved failure hierarchy (newest run by default)
|
|
510
|
+
node scripts/qa/mobile-e2e/ui-dump.js --latest
|
|
511
|
+
node scripts/qa/mobile-e2e/ui-dump.js --latest --flow scan-add --step step-063
|
|
512
|
+
node scripts/qa/mobile-e2e/ui-dump.js --file ~/.maestro/tests/<run>/<flow>/screen-hierarchy/<step>.json
|
|
513
|
+
|
|
514
|
+
# Never `ls ~/.maestro` by hand — list the artifacts (hierarchy + screenshots):
|
|
515
|
+
node scripts/qa/mobile-e2e/ui-dump.js --list-artifacts --flow scan-add
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
`--json` emits the normalized tree for programmatic diffing. The probe launches
|
|
519
|
+
nothing heavy and is never gated by the capacity guard; it normalizes both a live
|
|
520
|
+
UIAutomator XML dump and Maestro's `screen-hierarchy/*.json` to the same shape
|
|
521
|
+
(`id | text | desc | class | bounds`).
|
|
522
|
+
|
|
523
|
+
## Anti-pattern that hangs the agent
|
|
524
|
+
|
|
525
|
+
```bash
|
|
526
|
+
# DO NOT DO THIS
|
|
527
|
+
setsid bash -c '~/android-sdk/emulator/emulator -avd <avd> … > /tmp/emulator.log 2>&1' < /dev/null &
|
|
528
|
+
```
|
|
529
|
+
|
|
530
|
+
The redirect lands on the **inner** command, not on the detached `bash -c`
|
|
531
|
+
wrapper. The wrapper therefore keeps the agent tool's stdout/stderr socket open
|
|
532
|
+
and stays alive as the emulator's parent — so the launch call never sees EOF and
|
|
533
|
+
subtly "runs" forever. No timeout fires, because the launched shell itself
|
|
534
|
+
already exited; the agent is simply blocked. The same failure hits a plain
|
|
535
|
+
trailing `&` (the process tree stays attached to the tool's console).
|
|
536
|
+
|
|
537
|
+
If you must launch by hand, redirect the wrapper's own fds:
|
|
538
|
+
|
|
539
|
+
```bash
|
|
540
|
+
setsid sh -c 'exec ~/android-sdk/emulator/emulator -avd <avd> …' </dev/null >/tmp/emulator.log 2>&1 &
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
## Recovering an already-wedged run
|
|
544
|
+
|
|
545
|
+
1. Interrupt the hung tool call in the agent (or restart the session).
|
|
546
|
+
2. **Keep the emulator** — it booted fine; verify with `adb devices` and
|
|
547
|
+
`adb shell getprop sys.boot_completed`. Do not reboot it.
|
|
548
|
+
3. If the detached wrapper still lingers, confirm ownership before killing it:
|
|
549
|
+
`node scripts/serve.js list` and check the PID's session/parent. Never kill an
|
|
550
|
+
emulator you didn't start.
|
|
551
|
+
4. Relaunch through `serve.js` per this runbook before continuing.
|
|
552
|
+
|
|
553
|
+
See also: [wsl-optimizations.md](wsl-optimizations.md) (why WSL2 is the primary
|
|
554
|
+
environment) and `.agents/skills/mobile-engineer/SKILL.md` (the persona's device
|
|
555
|
+
rules).
|